Outage 1/11/2018

blantonl

Founder and CEO
Staff member
Joined
Dec 9, 2000
Messages
9,033
Location
Shavano Park, TX
#1
Folks, I know you've got a lot of questions and concerns about today's multi-hour outage today. Rest assured we were scrambling to get everything back up and running.

We needed to migrate to a new database infrastructure and unfortunately some other events required us to push up our plans with little notice and a lot more downtime than we wanted.

I'll have a full post-mortem here soon after things settle down. Thanks for your patience.
 
Joined
Dec 25, 2008
Messages
3,039
Location
New Zealand
#2
No worries, Lyndsay, you gotta do what you gotta do. I'll just have another mug of joe and wait until the withdrawal symptoms die down!

Thanks for an excellent forum, we all appreciate you hard work.
 

n9mxq

Member
Premium Subscriber
Joined
Dec 15, 2005
Messages
1,596
Location
Belvidere IL
#3
You and the crew do a great job keeping the system up.. Despite what all the whiners on Facebook and Twitter say..
 

K9DAK

Member
Premium Subscriber
Joined
Dec 16, 2010
Messages
381
Location
Wauconda, IL
#8
Yup! Here's what I posted on the FB comments criticizing Broadcastify...

"As a feed provider, I can count on one hand the number of times my feed has been down due to a Broadcastify issue in the five years my feed’s been running. Kudos!
"

Great job guys (and/or gals)... keep it up!
 

CQ

Member
Joined
Jan 2, 2015
Messages
511
Location
Exosphere
#9
Image Uploading Broken?

Tried to post images and nothing works.
Formats: .jpg, gif and .png
Less than maximum resolution
Less than maximum size



This is not a valid image file.
 

jasonhouk

Member
Premium Subscriber
Joined
Mar 29, 2013
Messages
855
Location
Marion, Ohio
#10
Folks, I know you've got a lot of questions and concerns about today's multi-hour outage today. Rest assured we were scrambling to get everything back up and running.

We needed to migrate to a new database infrastructure and unfortunately some other events required us to push up our plans with little notice and a lot more downtime than we wanted.

I'll have a full post-mortem here soon after things settle down. Thanks for your patience.
Looking forward to the "post-mortem".

Thanks

J
 

N9JIG

Sheriff
Moderator
Joined
Dec 14, 2001
Messages
3,751
Location
Far NW Valley
#13
I had the same issue uploading a pic to the Forums. I get an error saying it is not a valid image file and it appends the image name with a colon. I would attach an image with the screenshot but alas, I can't...
 

delaware74b

Member
Premium Subscriber
Joined
May 8, 2010
Messages
31
#19
Was this outage a result of the problems that also occurred on Jan 4? I had a banner at the top of my Scanner Radio App (android free version) "Broadcastify.com is currently experiencing server or network problems. Scanner feeds relayed by them (which is the majority of the ones in the directory) are currently unavailable."

Kudos to blantonl and everybody who works to keeping broadcastify up and running!
 

blantonl

Founder and CEO
Staff member
Joined
Dec 9, 2000
Messages
9,033
Location
Shavano Park, TX
#20
Was this outage a result of the problems that also occurred on Jan 4? I had a banner at the top of my Scanner Radio App (android free version) "Broadcastify.com is currently experiencing server or network problems. Scanner feeds relayed by them (which is the majority of the ones in the directory) are currently unavailable."

Kudos to blantonl and everybody who works to keeping broadcastify up and running!
The problems on Jan 4th and in late Dec 2017, were all due to the same issues.

Yesterday's issues required us to move up a migrate timeline from a few weeks to *now*. That necessitated us taking down everything vs an orderly migration. That also resulted in more downtime yesterday because we had to prepare a backup plan if things didn't go right.

Most important thing for me is to protect data first, and all things come second.

More to follow. It's a long and sordid technical story :)
 
Top