The topic for today is how to perform containment on a 'mision critical' service or system that your organization depends on, and cannot be shut down? There are many examples of systems that are in production, and under normal circumstances an operational team can't pull the plug. Web servers that are the primary sales vehicle, legacy systems that no one understands, the backend database that has EVERYTHING on it, the email servers, and the list goes on. In the event that an incident occurs on a normal system normally we can pull the plug to stop the bleeding and move on. However, if downing that box will be career limiting, and allowing the incident to continue is almost as bad, what to do?
I'll summarise suggestions here.
Francois write in "This doesn't speak directly to security incidents, but ... This situation showcases a vulnerability on the business side, and highlights a flaw in business thinking. The vulnerability is neglecting to plan. Incidents like this are the reason we have disaster recovery and business continuity plans. Downtime is inevitable, the only controllable factor is how we respond.
From my own perspective and experience, containment has always been a business decision. So long as management are aware of the risks of remainin in operation rather than taking the system/network/application down then operational staff must abide by that choice.
Adrien de Beaupre
Oct 14th 2008
1 decade ago