Their execution was poor. Start by depleting SF and then look elsewhere? Short-sighted at best.
They should have moved quickly before Best Buy/Costco implemented constraints that should have been anticipated. During one weekend, via train or one-way flights, employees could have been mobilized to cities across the US. Utilizing USPS flat rate boxes or low-cost equivalent, hard drives could have been shipped back to HQ en masse. Three days, disaster is over.
That's predicated on them being able to spare many employees just for drive sourcing, which as a relatively small company they probably couldn't. They went with the lower cost stop gap at first, if the shortage hadn't lasted as long they would have been better off with how they did it.
Yeah sure. In a weekend they could have purchased 1500 hard drives when there is a shortage and stores are limiting them to 2? I would challenge you to go to retail stores in a single day under those conditions and buy 25.
If it only takes 1 mistake, would having a hot backup or failover be a best practice, so that if something does happen, you can immediately channel traffic to a live site?
Thanks EwanToo - that makes perfect sense. A combination of some security monitoring system that notifies you of the vulnerabilities along with someone to update your system is needed. But what if the updates have dependencies, for instance, incompatible Ruby gems or so. At that point, do you have to make the tradeoff of security risk vs time to update all gems/resolve incompatibility issues/deal with bugs in latest release?
Exactly, if you've got a complex environment, you might not even know that one of your suppliers deployed an insecure ruby gem (or any other package), but you'll want to do full testing before upgrading.
All this leaves big windows of opportunity for attacks.
