Storm of the Century: How Did Your Cloud Providers Perform?

Boy oh boy. Hurricane Sandy made a real mess of the East Coast, and the effects will linger on for some time. For some companies, it was a great scapegoat. When my home land line and internet service went down on Tuesday, Verizon told me it was due to storms in New York. I know it was a terrible storm, but was it really responsible for knocking out DSL service for 24 hours in Los Gatos, CA, 3,000 miles away? I doubt it.

Unfortunately, I wasn’t the only one experiencing an unplanned outage. I received this email from a TSIA member at 5:30am PT on Tuesday:

“My cloud PSA solution is completely down today. They made no provision for a backup data center, despite having a week to prepare for the storm. They have not communicated anything about the situation to their customers. It is our quarter end and my core business system is completely unavailable with no word on when it will be restored. If I could move off of this product tomorrow, it would be at least a day too late.”

That stings! Especially for companies that take disaster recovery seriously, finding out your cloud application vendors didn’t prepare a fallback plan is unacceptable. At least for this PSA vendor, I suspect considerable customer churn is right around the corner. I am obligated to pass along this outage information to any members evaluating solutions in the future, so there are impacts to future prospects as well.

I was working for Giga Information Group, an early analyst firm, when September 11th happened. Reeling from shock, we tried to do something useful in the days that followed, and the research organization published immediate reports about disaster recovery planning for every area of technology Giga covered. Hopefully, Hurricane Sandy will similarly force tech firms to create a realistic plan to keep systems running regardless of weather.  In light of the enormous shift to the cloud, it appears that many companies have launched cloud initiatives without much thought to emergency situations.  I have to agree with the above email about the PSA solution–this wasn’t a freak earthquake that no one expected–companies had nearly a full week’s notice that this “storm of a lifetime” was brewing, and any cloud vendor in that vicinity that didn’t immediately plan for a backup server farm on the other side of the country seems downright negligent.

Apparently, someone needs to state the obvious: if you are going to make money by assuming the complexities of your customers’ deployments via cloud hardware and software, you darn well better make sure you have recovery plans for every conceivable emergency…and even some inconceivable emergencies. Your customers have placed their trust in you. When your systems go down…and you don’t even have the common decency to reach out to customers with updates…you probably won’t last long in this marketplace. There are too many competitors who pay attention to the fundamentals.

So if you are being pushed to launch a cloud alternative or a managed service offering this year, I hope you take a lesson from Hurricane Sandy. A single data center isn’t enough. Multiple data centers in a single region isn’t enough. And you better have a plan in place when the unthinkable happens on how to contact customers and keep them updated. Radio silence is not acceptable.

Just my 2 cents.

 

Advertisement

Explore posts in the same categories: Technology

Tags: cloud, disaster recovery, Hurriance Sandy, OnDemand, outages, PSA, server farms

You can comment below, or link to this permanent URL from your own site.