Folks,
As you are probably aware, we had an outage today that lasted almost 5 hours, due to our primary hosting provider (Amazon Web Services) experiencing significant network connectivity problems with their EBS (Elastic Block Store) which we use to store our master database server data.
Once the problems crept up at 11:30 AM CST, we brought the site offline to prevent having other major problems on our back end infrastructure. We finally made the decision to promote on of our back slave servers to the new master at about 4:30 PM CST. This means that the site is currently running in an emergency recovery configuration - therefore expect to see maintenance during the day tomorrow so we can get back to a more robust database implementation.
In the mean time, you can see Amazon Web Services status here:
AWS Service Health Dashboard - Oct 22, 2012
Also note that this issue was not just unique to RadioReference.com. Many of some of the largest Web properties were impacted.
Update: Amazon Web Services Down In North Virginia
Thanks all your patience during this outage. It was one of the largest we've had in the history of the site.
As you are probably aware, we had an outage today that lasted almost 5 hours, due to our primary hosting provider (Amazon Web Services) experiencing significant network connectivity problems with their EBS (Elastic Block Store) which we use to store our master database server data.
Once the problems crept up at 11:30 AM CST, we brought the site offline to prevent having other major problems on our back end infrastructure. We finally made the decision to promote on of our back slave servers to the new master at about 4:30 PM CST. This means that the site is currently running in an emergency recovery configuration - therefore expect to see maintenance during the day tomorrow so we can get back to a more robust database implementation.
In the mean time, you can see Amazon Web Services status here:
AWS Service Health Dashboard - Oct 22, 2012
Also note that this issue was not just unique to RadioReference.com. Many of some of the largest Web properties were impacted.
Update: Amazon Web Services Down In North Virginia
Thanks all your patience during this outage. It was one of the largest we've had in the history of the site.