- Joined
- Dec 18, 2002
- Messages
- 581
- Reaction score
- 65
Hurray – new meatier more powerful servers coming!
We're going to be moving our entire set of broadcast servers over to a new provider to save on bandwidth costs and increasing processing power. We'll have 8 audio servers total (not including our archiving servers) which will serve the icecast infrastructure on RadioReference.com. Each server has 8GB of RAM and a quad-core Xeon processor. 3 servers will be master servers, 4 will be relay servers, and 1 will be a management server. This new design can efficiently support 3500 feeds and 80,000 listeners. If we need to scale past that we have the ability to instantly provision new servers on our existing Amazon EC2 cloud infrastructure.
Everything is staged and ready to go… The only requirements will be about 5 minutes of downtime for the audio system while we make the necessary configuration and DNS changes.
Here is how we'll make the transition (each step)
1) The recording processes on each of the 3 archive servers will be stopped
2) The new DNS entries for the master servers will be updated
3) The 3 master servers will be stopped (the ones that receive audio from volunteers)
4) The 3 relay servers will be stopped (the ones people connect to to listen)
5) Our internal configuration database will be updated with the new server configuration details (masters and relays)
6) The archiving servers will be configured to archive from the new servers
7) The new masters will be brought online
8) The new relays will be brought online
9) The archive server recording processes will be started back up
That's it!
We expect to do this at 6:15 AM CST one morning this week – we're still trying to figure out when would be best. An e-mail will be sent out to all feed providers one night in advance letting them know. Most feed broadcasters should reconnect without any intervention, however some broadcasters might need to be rebooted before their broadcast client picks up the DNS changes.
Note that any feeds which have not been certified as being TOS compliant by the time the above changes are made will not be able to reconnect.
-- Gordon
We're going to be moving our entire set of broadcast servers over to a new provider to save on bandwidth costs and increasing processing power. We'll have 8 audio servers total (not including our archiving servers) which will serve the icecast infrastructure on RadioReference.com. Each server has 8GB of RAM and a quad-core Xeon processor. 3 servers will be master servers, 4 will be relay servers, and 1 will be a management server. This new design can efficiently support 3500 feeds and 80,000 listeners. If we need to scale past that we have the ability to instantly provision new servers on our existing Amazon EC2 cloud infrastructure.
Everything is staged and ready to go… The only requirements will be about 5 minutes of downtime for the audio system while we make the necessary configuration and DNS changes.
Here is how we'll make the transition (each step)
1) The recording processes on each of the 3 archive servers will be stopped
2) The new DNS entries for the master servers will be updated
3) The 3 master servers will be stopped (the ones that receive audio from volunteers)
4) The 3 relay servers will be stopped (the ones people connect to to listen)
5) Our internal configuration database will be updated with the new server configuration details (masters and relays)
6) The archiving servers will be configured to archive from the new servers
7) The new masters will be brought online
8) The new relays will be brought online
9) The archive server recording processes will be started back up
That's it!
We expect to do this at 6:15 AM CST one morning this week – we're still trying to figure out when would be best. An e-mail will be sent out to all feed providers one night in advance letting them know. Most feed broadcasters should reconnect without any intervention, however some broadcasters might need to be rebooted before their broadcast client picks up the DNS changes.
Note that any feeds which have not been certified as being TOS compliant by the time the above changes are made will not be able to reconnect.
-- Gordon
