laptop radio console and IP path failures

[speedway_navigator]

We tested a Zetron max laptop on a wifi connection and it sounded OK. When we switched to a cat 5 cable the audio was noticeably better. Have you guys seen this?

When we went to the backup site for dispatching it took over 20 minutes for the IT guy and engineer at the 2 way shop to work through firewall issues to get the laptop working.
We also found out the fiber backup system has a similar issue, the county uses microwave from the ECC to the 2 sites. It had some rain fades and we found out the system failed over to fiber but did not realize the fiber path wasn't there.
Our goal is to make fiber primary and fail to microwave.

How do you guys monitor when your primary link fails and do you stay on the backup for a specific time period or do you switch back to primary after you determine the path is solid again?

 

[mmckenna]

I'm not a network guy.
But I do leave this sort of stuff up to our network engineers. OSFP on the Cisco side takes care of all that for us/me. The router will learn what it can connect to and build tables. When something fails, it jumps to the next available route. Sometimes it can take a second or so, but we see it happen on some of our long fiber routes when a tree/accident/squirrel gets involved.

 

[TampaTyron]

Strongly recommend you have those failovers dumped to a email distribution group of you, your team, your chain of command, etc.

I am not a networking schmuck, but our guys have tons of monitoring and alerting.

My guess is you are pulling that info from SNMP or a vendor monitoring software.

I would also use fiber as the primary and microwave as the backup. But, please practice the backup process, even if it is in the middle of the night.

TT

 

[mmckenna]

My guess is you are pulling that info from SNMP or a vendor monitoring software.

I'm on the mailing list for our network monitoring system.
Large network, lots of OSPF flaps, every time a switch bounces, every time one of the engineers changed something, I was getting hundreds of e-mails a day and finally had to block it all. I can log in to their system to see what's going on, but I usually need one of them to translate for me.

Mixed blessing, for sure, but I just pay attention to my own system alarms and use those to tell me when I need to start looking into it more.

 

[bunangst]

We tested a Zetron max laptop on a wifi connection and it sounded OK. When we switched to a cat 5 cable the audio was noticeably better. Have you guys seen this?

When we went to the backup site for dispatching it took over 20 minutes for the IT guy and engineer at the 2 way shop to work through firewall issues to get the laptop working.
We also found out the fiber backup system has a similar issue, the county uses microwave from the ECC to the 2 sites. It had some rain fades and we found out the system failed over to fiber but did not realize the fiber path wasn't there.
Our goal is to make fiber primary and fail to microwave.

How do you guys monitor when your primary link fails and do you stay on the backup for a specific time period or do you switch back to primary after you determine the path is solid again?

Audio packet Latency and Jitter will always be better on wired ethernet (cat5, cat6) then wifi. I would only use wifi as a last resort.

Rain fade on microwave does occasionally happen. Any idea if these are fully engineered and maintained links?

Public safety MW is usually engineered for 99.999% uptime through rain and atmospheric events in your region. That is only 5 minutes and 15 seconds of downtime.

Battery backup, generators, etc all maintained?

As others have stated, networks usually use OSPF at a minimum for path changes.
More complex networks use MPLS (multi-protocol label switching) which are generally configured for more complex link monitoring and switching timers.

 

[speedway_navigator]

The county IT guy and the ISP are working on this, at first our fiber was just a generic setup, the IT guy ran trace routes and found the path going out of state and back which caused latency issues. Now they say the path is close to dark fiber.
I think they have a Cisco 9300. I don't know Cisco IOS, I use Mikrotik.

 

[speedway_navigator]

I was surprised at the difference in audio between wifi and wired since the audio bandwidth is so small. I know a speed test will never show theoretical limits. They only have a few APs going so I would not think RFI from adjacent channels would affect audio.
I think the microwave is 19 GHz at 18 miles, the license limit is 61 dBm EIRP.

I don't know the RX sens level to calculate fade margin, my microwave experience is limited to frozen dinners and popcorn from my bachelor days.

 

[Firebuff880]

I'm not a network guy.
But I do leave this sort of stuff up to our network engineers. OSFP on the Cisco side takes care of all that for us/me. The router will learn what it can connect to and build tables. When something fails, it jumps to the next available route. Sometimes it can take a second or so, but we see it happen on some of our long fiber routes when a tree/accident/squirrel gets involved.

Be careful with Open Shortest Path First (OSPF) routing; it works great, until five years down the road it finds a route that no one remembers was there, or worse, should not be there. It may be a valid route, but it may not be capable of handling all the traffic; when a failure suddenly makes it the "only" route.

I'm on the mailing list for our network monitoring system.
Large network, lots of OSPF flaps, every time a switch bounces, every time one of the engineers changed something, I was getting hundreds of e-mails a day and finally had to block it all. I can log in to their system to see what's going on, but I usually need one of them to translate for me.

There are two critical fixes you can implement for this scenario that often results in "pager" burnout and someone not getting the message when a critical alert is sent.

First, add some delay to the process so that the alert waits some number of minutes, appropriate for the interface, to clear before you send it out. This may be custom code, or it may be using a feature in software like Nagios.

Second, be sure the software doing the alerting has both on-shift rotation schedules and the ability to bypass the schedule should the worst happen.

 

[mmckenna]

Be careful with Open Shortest Path First (OSPF) routing; it works great, until five years down the road it finds a route that no one remembers was there, or worse, should not be there. It may be a valid route, but it may not be capable of handling all the traffic; when a failure suddenly makes it the "only" route.

That's why we have some of the best network engineers in the country.

First, add some delay to the process so that the alert waits some number of minutes, appropriate for the interface, to clear before you send it out. This may be custom code, or it may be using a feature in software like Nagios.

That's what I've done on my equipment. My DC power plants have a hysteresis setting for this. PBX waits a few minutes before sending a message, and then only repeats every 20 minutes.

Second, be sure the software doing the alerting has both on-shift rotation schedules and the ability to bypass the schedule should the worst happen.

"On call" has been kind of a permanent thing for me due to retirements, budget cuts, and the dangers of putting two way radio under the IT heading.

 

[mmckenna]

I was surprised at the difference in audio between wifi and wired since the audio bandwidth is so small. I know a speed test will never show theoretical limits. They only have a few APs going so I would not think RFI from adjacent channels would affect audio.
I think the microwave is 19 GHz at 18 miles, the license limit is 61 dBm EIRP.

I don't know the RX sens level to calculate fade margin, my microwave experience is limited to frozen dinners and popcorn from my bachelor days.

Make sure you talk to the network guys about QOS settings.

When we migrated our PBX to VoIP site linking, the network guys were not really excited about having to give our traffic priority. It took some work, but eventually we got there. You have to do the same thing with your carriers, most ISP's don't want to do QOS for customers, and they'll probably want to charge you extra for it.

Dark fiber is a better approach if you can get it.

 

[dickie757]

"On call" has been kind of a permanent thing for me due to retirements, budget cuts, and the dangers of putting two way radio under the IT heading.

I interpreted FireBuff's comment as shift change and what happens when the console or other monitored gear is powered down or logged out due to their shift change.