The RadioReference.com Forums

The RadioReference.com Forums (https://forums.radioreference.com/)
-   System-wide Administration (https://forums.radioreference.com/system-wide-administration/)
-   -   Outage 1/11/2018 (https://forums.radioreference.com/system-wide-administration/364315-outage-1-11-2018-a.html)

blantonl 01-11-2018 2:02 PM

Outage 1/11/2018
 
Folks, I know you've got a lot of questions and concerns about today's multi-hour outage today. Rest assured we were scrambling to get everything back up and running.

We needed to migrate to a new database infrastructure and unfortunately some other events required us to push up our plans with little notice and a lot more downtime than we wanted.

I'll have a full post-mortem here soon after things settle down. Thanks for your patience.

majoco 01-11-2018 4:45 PM

No worries, Lyndsay, you gotta do what you gotta do. I'll just have another mug of joe and wait until the withdrawal symptoms die down!

Thanks for an excellent forum, we all appreciate you hard work.

n9mxq 01-11-2018 5:49 PM

You and the crew do a great job keeping the system up.. Despite what all the whiners on Facebook and Twitter say..

KK4JUG 01-11-2018 5:58 PM

Quote:

Originally Posted by n9mxq (Post 2866689)
You and the crew do a great job keeping the system up.. Despite what all the whiners on Facebook and Twitter say..

Same here. Stuff happens. As radio and/or scanner enthusiasts, everyone should know that. My advice to them: Get Over It.

JeffDS3 01-11-2018 7:42 PM

Quote:

Originally Posted by n9mxq (Post 2866689)
You and the crew do a great job keeping the system up.. Despite what all the whiners on Facebook and Twitter say..

Most of us just want info on whatís going on, versus just sitting there wondering.

wtp 01-11-2018 7:45 PM

i will use the time
 
to look through some police call books...

kma371 01-11-2018 8:28 PM

Quote:

Originally Posted by n9mxq (Post 2866689)
Despite what all the whiners on Facebook and Twitter say..

Agreed. It was a bit ridiculous. A few hours of not being able to access the site isn't the end of the world. At least for me.

K9DAK 01-11-2018 9:30 PM

Yup! Here's what I posted on the FB comments criticizing Broadcastify...

"As a feed provider, I can count on one hand the number of times my feed has been down due to a Broadcastify issue in the five years my feedís been running. Kudos! https://static.xx.fbcdn.net/images/e...1/16/1f37b.png"

Great job guys (and/or gals)... keep it up!

CQ 01-11-2018 10:53 PM

Image Uploading Broken?
 
Tried to post images and nothing works.
Formats: .jpg, gif and .png
Less than maximum resolution
Less than maximum size



This is not a valid image file.

jasonhouk 01-12-2018 6:47 AM

Quote:

Originally Posted by blantonl (Post 2866548)
Folks, I know you've got a lot of questions and concerns about today's multi-hour outage today. Rest assured we were scrambling to get everything back up and running.

We needed to migrate to a new database infrastructure and unfortunately some other events required us to push up our plans with little notice and a lot more downtime than we wanted.

I'll have a full post-mortem here soon after things settle down. Thanks for your patience.

Looking forward to the "post-mortem".

Thanks

J

blantonl 01-12-2018 8:13 AM

Quote:

Originally Posted by CQ (Post 2866825)
Tried to post images and nothing works.
Formats: .jpg, gif and .png
Less than maximum resolution
Less than maximum size



This is not a valid image file.

Where are you trying to post images? In the forums?

blantonl 01-12-2018 8:33 AM

I think you might have meant the forums or the wiki. i've submitted a fix.

N9JIG 01-12-2018 8:58 AM

I had the same issue uploading a pic to the Forums. I get an error saying it is not a valid image file and it appends the image name with a colon. I would attach an image with the screenshot but alas, I can't...

blantonl 01-12-2018 2:15 PM

Are you still seeing the error? I rolled fixes and tested it on the forums this morning.

blantonl 01-12-2018 2:16 PM

Quote:

Originally Posted by jasonhouk (Post 2866910)
Looking forward to the "post-mortem".

Thanks

J

Give us a while, per you we're "amateurs" over here when it comes to running infrastructure. :roll:

KC4RAF 01-12-2018 2:48 PM

All I can say Mr. Blanton III, you guys do a great job! Wish the cry babies would get a life and go some where else.
While the site was down, I was up and did some honey-dos around the house. My XYL said to tell you all thanks...lol

N9JIG 01-12-2018 2:54 PM

Quote:

Originally Posted by blantonl (Post 2867104)
Are you still seeing the error? I rolled fixes and tested it on the forums this morning.


It seems to have been fixed. I posted a pic a few minutes ago and it worked fine.

Thanks Lindsay!

bharvey2 01-12-2018 3:49 PM

No worries. I've been on both sides of a misbehaving website. It's no fun but hey, it happens and life goes on.

delaware74b 01-12-2018 4:54 PM

Was this outage a result of the problems that also occurred on Jan 4? I had a banner at the top of my Scanner Radio App (android free version) "Broadcastify.com is currently experiencing server or network problems. Scanner feeds relayed by them (which is the majority of the ones in the directory) are currently unavailable."

Kudos to blantonl and everybody who works to keeping broadcastify up and running!

blantonl 01-12-2018 5:45 PM

Quote:

Originally Posted by delaware74b (Post 2867198)
Was this outage a result of the problems that also occurred on Jan 4? I had a banner at the top of my Scanner Radio App (android free version) "Broadcastify.com is currently experiencing server or network problems. Scanner feeds relayed by them (which is the majority of the ones in the directory) are currently unavailable."

Kudos to blantonl and everybody who works to keeping broadcastify up and running!

The problems on Jan 4th and in late Dec 2017, were all due to the same issues.

Yesterday's issues required us to move up a migrate timeline from a few weeks to *now*. That necessitated us taking down everything vs an orderly migration. That also resulted in more downtime yesterday because we had to prepare a backup plan if things didn't go right.

Most important thing for me is to protect data first, and all things come second.

More to follow. It's a long and sordid technical story :)


All times are GMT -5. The time now is 9:41 AM.

Powered by vBulletin® Version 3.8.2
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
All information here is Copyright 2012 by RadioReference.com LLC and Lindsay C. Blanton III.Ad Management by RedTyger