Chip Somodevilla/Getty
This week, an Amazon Web Service (AWS) failure caused a massive outage all over the internet. Now we know why: a typo.
The company released a detailed report explaining what happened. An employee entered what they thought was a routine command to remove servers from an S3 subsystem. By mistake, they entered a larger number than intended. These servers supported two other S3 subsystems, both of which manage the storage and metadata for the entire region. Down went the dominoes.
AWS assures everyone that it’s prepared for the occasional failure. Fixing the employee’s error should have been as simple as rebooting the subsystems. However, AWS admitted that it hasn’t actually restarted those subsystems in years, and S3 has grown considerably in the meantime.
If you’ve ever rebooted an older computer and notice it chugging on start-up, you’ll understand the feeling AWS must have had while waiting for the system to come back.
In response, AWS says it’s putting in safeguards to prevent this kind of error from happening again. Apparently, creating these safeguards had been in the works for a while, and AWS is making it a priority after this outage. Considering it even brought down AWS’s own conference mid-speech, this is probably for the best.
https://twitter.com/mims/statuses/836645989972918272
The moment the presenter realizes S3 is down. #AWS #S3 #Amazon pic.twitter.com/S8gY1J9lrH
NOW WATCH: 7 little-known benefits of Amazon Prime
