The cost of cleaning up

Published: 2013-05-01
Last Updated: 2013-05-01 13:43:22 UTC
by Daniel Wesemann (Version: 1)
13 comment(s)

As Johannes mentions in yesterday's ISC StormCast, the city of Schwerin in Germany apparently decided to throw 170 PCs into the trash, because cleaning them from a Conficker worm infestation was estimated at around 130'000 Euros, whereas the replacement of the old PCs had already been budgeted for at 150'000 Euros. Our recent discussion aside on whether a modern malware infection can actually be "cleaned" or if wiping and reinstallation from scratch is always called for, "the cost of cleaning up" is actually  relevant in either case. Schwerin's 130kEuro estimate amounts to about 1000$ per PC. The report doesn't say if this calculation includes lost productivity of the employee who has to wait for his/her computer to be returned from scrubbing, or if this is just for the cleaning/reinstall itself.

Some Google searches gave me a going rate between 79$ and 299$ for a malware clean-up on a single home user PC, and several of the providers mention explicitly that they offer a "fresh install" for a lower price than the cleanup, which is one more indication that "re-install" seems to become the norm.

My search didn't result in any decent figures for virus cleanup costs in a mid-to-large corporate environment though. Companies of a certain size are likely set up to automatically provision and install new computers, so a replacement/re-stage should be a standard process for them, and relatively quick and cost effective. If you have any figures on the actual cost of cleanup/restage in a larger organization, or know any recent studies that have analyzed this in some depth, please let us know.

13 comment(s)

Comments

OK, OK... scrub the drive. So when does all the effort we put into backups come into play?
.
> the city of Schwerin in Germany apparently decided to throw 170 PCs into the trash,

Well, just kick the environment in the nuts-and-bolts! :-(

The British Columbia government has an "Asset Investment Recovery" branch, where "used" PCs (without their hard-drives) are sold/auctioned to the public. Much better for the environment, and some "cost-recovery" for the province. Win-Win.

Of course, the original Microsoft "decal" is still affixed to the computer's case, so that a person can reinstall Windows onto a self-supplied hard-drive, and get a not-too-old working system. (As a matter of policy, the government "upgrades" 1/3 of their computers every year.)

P.S. Google for "EDDIE asset investment recovery" and watch what EDDIE does to the virus-infected hard-drives. :-)
Being a linux shop (with a few exceptions), we just baddblocks -w the infected drives, then put them in a pool of re-usable drives if they pass, of junk them if they fail. If we had a major epidemic, we probably would have to purchase some new drives. If we had a standard setup (which we don't), we could image the drives ahead of time so they would only need user files restored from backup, and possibly a system update to get them ready to go into a sick machine as a brain transplant.
Those figures seemed a little fishy to me to begin with, and the $1,000/PC cleanup estimate just confirms that. I get the feeling the decision was less about economics and more about "we're replacing them anyway, why bother with a cleanup effort". They fudged the costs to make it look like a wash and hoped nobody would question the math.
I want 1000 dollars for running a conficker removal tool.
Using an effective AV package on all 170 PCs would not have cost nearly that much and would have prevented an old worm like that from establishing itself there.
Since re-imaging is fast becoming the norm, this becomes a compelling argument in favor of VDI plateforms. These can minimize the costs and downtime tremendously. Yes, its not a perfect system since the cost of VDI will wipe-out these savings. But for each dollar spent you get more value in terms of availability, agility, etc. In the end, making the remediation of infected system less Sisyphic. (wink to this TED presentation: http://www.ted.com/talks/dan_ariely_what_makes_us_feel_good_about_our_work.html)
I think they key driver of the decision to buy new systems is the fact that replacement systems had already been budgeted: "...whereas the replacement of the old PCs had already been budgeted for at 150'000 Euros".

They simply decided to move their new system rollout plan/replacement cycle forward instead of spending the money on cleaning old systems. I'm sure their decision would have been different if Conficker had infected their new batch of systems.

For medium to large companies, the cost of this kind of security event is the downtime and lost productivity which can be millions depending on the size of the company. For small companies, the cost is not only the downtime and lost productivity, but also the cost to clean/repair the systems (or buy new ones) since they will most likely have to hire an outside security/IT firm to perform the work. A medium/large company will most likely have an IT or security staff to perform the work and the cost is already built into their IT/security budgets/salaries.

The cost really depends on the size and how the organization is structured.
I work at a company with over 1000 locations, many of which are connected with 2 Mbit connections. Respooling a bunch of PCs over such a line is impossible, as it would disrupt operations for the rest of the location.

The only way to do it would either have a set of people driving around to 1-2 locations a day, reinstalling from DVD, or have the machines shipped to HQ for reinstall.
It would be expensive in handling, would easily cost 2-3 hours in manpower for packaging, shipping, unpack, respool, repack, ship + some shipping costs, and then downtime for the PC for 3-4 days. But $1000 sounds excessive.

If the machines are homogenous, you could ship new drives faster than new PCs, and they would be pretty fast replicate. Of course assuming you can get the supported type of drives. Maybe it was old ATA drives. Of course that would require some on-site guy who can install it. And maybe they are afraid of cross-infection.

But again, replacing the machine was in budget, guess they were planned to be retired in the summer holidays anyway, and pushing up the replacement for 2-3 months is no big deal.
As I wrote here:
https://isc.sans.edu/diary/What+is+%22up+to+date+anti-virus+software%22%3F/15692

...it is even worse.


The expected costs were 130.000Eur (Cleaning) + 35.000Eur (Rebuild).

The bottom line is ~1300USD.

Diary Archives