What are you supposed to do when your backup dies. Most people decide the cloud is a better idea. This way someone else is responsible for your backups. But what happens if they have a problem? Today one of the largest data centers in France has been destroyed by fire. We have no idea how it started. But we do know that it is owned by the largest hosting provider in Europe: OVH.
One would assume it can’t be that bad because a hosting company would have backups. Unless the backups are in the same building. Or series of buildings. See OVH has a number of buildings right next to each other. This means when the fire department shut down the area all four of their major data centers where shut down.
Little is being told to the public but the company did have a message to give out:
We are currently facing a major incident in our DataCenter of Strasbourg with a fire declared in the building SBG2.
Firefighters were immediately on the scene but could not control the fire in SBG2.
The whole site has been isolated, which impacts all our services on SBG1, SBG2, SBG3 and SBG4.
If your production is in Strasbourg, we recommend to activate your Disaster Recovery Plan.
All our teams are fully mobilized along with the firefighters.
We will keep you updated as more information becomes available.
A few hours later the OVH Status website completely removed two of the data centers,SBG2 and ABG3 off of their list.
Now the question remains… what do you do when your cloud is dead. Did they even offer or suggest to their clients to have a disaster recovery plan that did not include them. I personally have always suggested three difference backup options for local, cloud, manual, and automatic backup of your data.
For example one of their clients, the video game maker RUST, the situation has lead to a complete and total data loss leaving no way to recover from it. “We’ve confirmed a total loss of the affected EU servers during the OVH data centre fire. We’re now exploring replacing the affected servers. Data will be unable to be restored,” the game developer tweeted.
Now if it was only video games, I sure not many would care but this is alot of websites, shopping carts, a cyber threat intelligence company, a cryptocurrency exchange, a telecom company, and a number of news outlets and lots of other services. It will be some time before we know if any government services were stored their and it is possible that we will never know exactly all the resources that was lost to the fire.
“Fire is over. Firefighters continue to cool the buildings with the water.””We don’t have the access to the site. That is why SBG1, SBG3, SBG4 won’t be restarted today,” said Klaba, representing OVH in a tweet. “Fortunately, there have been no reports of any casualties arising from the fire so far.Customers should immediately bring into effect their disaster recovery plans as OVH is working on restoring its services.”
The plan for restarting servers will begin Monday March 15th. SBG1 and SBG4 will be restarted first and see how it goes. They will be added ten thousand servers by the end of the month at no extra cost to their customers. SBG3 should be restarted by Friday March 19.
It will take some time to see how much of the world this one incident has affected, but it does prove one thing for sure. Always be responsible for your backups. If you hire someone to do your backups ensure you have it done a number of ways and ask how to do it.
Personally I run a computer company in Orlando and have always suggested a number of solutions. My company offers a cloud option that is mirrored on four different sites with YCSBackup. This does a daily backup of the files that you direct. Then I suggest a sync service like dropbox. This is a low cost solution that allows a copy of your documents to be on the cloud. Always be sure to have at least one computer setup with local settings so a copy of all your data resides on ONE or more of your computers. One Drive does not do this. Please do not rely on one drive to keep it on your computer and the cloud. And finally I suggest a local USB or network drive backup. This is the fastest way to recover for small issues. Either way you would have three recovery options that I am sure the makers of RUST is wishing they had right now.
Will there be lawsuits? Will providers that use the services of OVH get sued for their servers being down and websites as well as other security services? Only time will tell. But if anything good can come of it, please ensure you backup YOUR stuff. Because you cannot replace that photo of your child being born.
