Code Roll Back Question

Paul Perry's picture
Paul Perry asked on December 18, 2015 - 8:40am | Replies (1).

Hello Everyone,

I'd like to understand how companies using SVN, TFS, Git, GitHub perform backing out production deployments from two perspectives, your re-deployment method but more importantly, how are you indicating in your respective repo that if code was rolled back, what are you doing internally with your SCM tool to force a message to developers that code was backed out.


Paul Perry

1 Answer

Drew Benson's picture

(Without being too flippant) - in my experience the real answer is "They don't!"............  

Real life experience of all but the most catastrophic deployments is "FIX FORWARD"
and to be frank fixes to even the most catastrophic deployments tend to be along the lines of either:

Restore the backups taken (just) prior to implementation and sort it out later (via fix forward)
Hack it enough to get it standing and sort it out later 

This is because in very many cases (despite CM efforts and process dictat) Backout Plans are not really devised in an end-to-end manner. (Individual "steps" of a complex implementation plan may well have simple backouts - they may even be documented - but in reality they have never had the time and analysis applied to garuantee credibility)
Implementation plans get rejected at CCB/approval as there is no docoumented "Back Out".... then approved 20minutes later once "Restore to previous backup" has been cut&pasted into relevant sections of the Imp Plan.   

To be fair often business drivers (system availability/SLA etc) do tend to mean that there is no realistic window of opportunity to step back to the star cleranly.
eg When in order to carry out a major implementation:

a) Take "the system" offline
b) Delay overnight/batch/bau processes
c) Carry out complex implementation
d) Squeeze in overnight/batch/bau processes
e) Give the system "back" so users can catch up

There is very little chance to "Back out cleanly step by step" during the available window before Steps d & e become imperitive.
Compounded because whenever implementations don't go 100% smoothly time is eaten attempting to fix it on the fly...  and when this "works" it means successful implementation is only "delayed" (but when it fails...... :-( )


CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.