Virginia FD Report: Digital Radios Extremely Vulnerable

[kenisned]

Virginia FD Report: Digital Radios Extremely Vulnerable

............

LYNNETTE LUNA
Mobile Radio Technology



On April 16, 2007, firefighter Kyle Wilson was part of a crew dispatched to fight a residential fire in Woodbridge, Va. He died in the line of duty.

A detailed report on the incident recently released by Prince William County Department of Fire and Rescue concluded that problems associated with the use of the county's Motorola digital trunked radio system contributed to the tragedy. Issues reported by other firefighters during that incident, which was further complicated by strong winds, ranged from signal distortion and transmission failure to radios displaying "out of range" signals.

Fire safety advocates now are encouraging fire departments across the country to study the incident in hopes that future tragedies could be avoided. Prince William County's fire department, through further tests, concluded that digital portable radios are "extremely vulnerable to poor environmental conditions and interference of digital noise from ambient sources, which negatively impact the ability of emergency personnel to effectively communicate."

A handful of fire and police departments, fearing the loss of lives, have opted to continue using analog systems even when the rest of their county's emergency personnel are using digital trunking systems.

The common complaint, which most affects fire departments, concerns the digital vocoder's inability to differentiate between a voice transmission and background noise - whether a chain saw, sprayed water or personal alarm. Background noise renders the voice transmission distorted and often unintelligible. Another critical problem is that digital radios lose contact inside buildings. "In most cases, it is a very political and sensitive position to abandon expensive technology and go back to something that is old," said Daryl Jones, owner and president of Telecommunications Engineering Associates, which manages public safety systems throughout the San Mateo area in California. "But many agencies are finding that complaints from line personnel, both in fire and police, are so significant."

The Boise (Idaho) Fire Department spent about $1 million two years ago on mobile and portable radio equipment to join a cutting-edge countywide 700 MHz digital trunking system. While training users on the system, the fire department discovered problems with voice intelligibility when a firefighter's low-air alarm went off. That led the department to investigate the issue further, and it found more instances where alarms interfered with the quality of voice transmissions. Today, the Boise Fire Department and other fire departments in the county remain on analog VHF radios while the rest of the county operates on the 700 MHz digital trunking system.

"Right now our dispatch center wants to dump VHF," said Paul Roberts, a captain with the Boise Fire Department, "[and] we are trying to look at alternatives to at least get on a system that will lessen the load on dispatchers having to patch all of this together. ... But until there is a solution to the digital processing of speech when you have competing noises, we have to stay on analog."

The problems associated with digital systems became known in 2006. Since then, the International Association of Fire Chiefs (IAFC) established a Digital Problem Working Group and appointed Chief Charles Werner of Charlottesville, Va., to serve as its chair. So far, the working group has explored the creation of a best practices solution to work around the problem until a long-term solution can be found. Prince William County's findings have been forwarded to that group for inclusion in the process.

The National Institute of Standards and Technology (NIST) also is analyzing the problem, as are radio manufacturers such as M/A-COM and Motorola. They are expected to jointly release a formal analysis - in conjunction with the IAFC - that encompasses best practices to help departments to minimize the problems.

"We're running through this scientifically and hope to distribute a wrap-up summary shortly," Werner said.

Roberts, who chairs the IAFC testing group, says the testing - conducted by radio engineers - involves taking words that sound alike and requiring the listener to distinguish which word is being said over the background noise of chainsaws and hose sprays.

Motorola declined to comment, saying it was cooperating with the testing and awaiting the conclusions from NIST and the IAFC.

But Chris Lougee, vice president with LMR vendor Icom America, believes better vocoder technology would help solve these background noise problems. "Everyone knew from the beginning that the P25 vocoder was a half-rate vocoder. As you speak into the microphone, you are converting human voice into a data stream that is reassembled at the end," Lougee said. "TIA ... is encouraging a move to a full-rate vocoder, which we are doing. It vastly improves the amount of audio and quality."

Lougee added that, scientifically speaking, digital signals penetrate buildings better than analog signals. "I'm puzzled by that problem," he said. "All of our testing shows that a digital signal produces a higher-quality signal in noise conditions than an analog signal."

Nevertheless, perception has a nasty way of becoming reality, and first responders' perceptions are based on what happens in the field and exacerbated by complaints from the front line.

"The perception of quality of communications in my opinion is much lower on trunked radio systems because it's always based on a comparison of what an agency had before," Jones said. "If they are coming off an analog system that provided 100% coverage and go to a digital trunked system that has different characteristics and less coverage, it's going to be worse."

However, others say the problems have to do with training, as digital systems operate differently than analog. For instance, digital systems require key-up time, forcing first responders to hold down the transmission key longer before they can begin talking. "It's a long and arduous process to educate police and firemen to change the way they have always communicated," Jones said.

But Roberts says best practices aren't the total solution. "They can be easily implemented on a normal day-to-day basis," he said, "but take the same firefighting crew and put them in a panic state. Then I would argue that a lot of best practices are not always utilized."

The Phoenix Fire Department perhaps has had the most experience with the issue. In 2004, the department gathered radio experts and conducted a study of a 700 MHz/800 MHz digital trunked system that covered a 2000-square-mile area. The study concluded that radios lost contact inside buildings and that users often encountered delays and background interference. The department today still operates on a VHF analog system, despite the fact that the rest of the city operates on a digital system.

"Back when we first started working on this project and told our working group that we needed simplex channels, we got a lot of guff about it," said Leif Anderson, deputy chief with the Phoenix Fire Department. "It took us a month or two to finally give them enough information about our problems. ... The SWAT team is now using our simplex channels because they can't have a delay for critical communications."

Today, the department is close to finding a way to bridge its analog system with the rest of the city so that dispatchers don't become overwhelmed with patching the communications systems together, Anderson said. For nearly two years, the department has been studying the use of digital vehicle repeaters (DVRs) mounted on fire trucks that surround the incident area.

"Think of DVRs as being two mobile radios linked together, one working in simplex and the other in duplex or trunked mode," Anderson said. "We'd use the simplex mode when talking to each other on the fireground, but messages to the dispatch center go through the duplex mode."

The Phoenix Fire Department has done much testing of the DVRs around dense buildings that typically pose significant transmission problems. The department is now at a nearly 100% success rate. Now it is moving its existing drills throughout 19 different cities in the department's jurisdiction of about 2500 square miles.

Additionally, the fire department has developed standard operating procedures in case they lose communications. The procedures mostly involve a lot of people repeating messages. For instance, to ensure a fireground message reaches the dispatch center, an incident commander on the outside - who also would have heard the message on the simplex radio - relays the message to the dispatch center. "We've always done this back forth, and it's always been successful," Anderson said.

The operating procedures also involve some changes to the DVRs. "There aren't many customers who use DVRs in simplex-to-trunked mode," said Anderson, "so our vendor had to do some algorithm work. We want every apparatus to have a master/subordinate relationship. If one failed, the next one would take over." Futurecom Systems is the department's DVR vendor.

In the future, Anderson expects firefighters to carry radios, such as the Motorola XTS 5000 radios, with multiple decks and talk groups that will enable first responders to do their jobs much the same way they do today. A call would be dispatched, and the firefighter would be told which tactical radio channel to switch to - simplex or trunked.

For Anderson, it has been a long and arduous process. "We have a saying, 'The sooner we resolve this problem, the sooner we get to go back to the station,'" he said. "That applies here."

Copyright 2008 LexisNexis, a division of Reed Elsevier Inc. All rights reserved.

[richardc63]

Hi,

This report has been sent around the world and quite a number of us have read it and studied the awful events. However the significance of the role of radio in the tragedy, I think, is being grossly exaggerated by people who seem to have an "agenda" when it comes to the use of digital modulation.

Lastly, anyone who actually writes such a silly comment as " coming off an analog system that provided 100% coverage" loses all credability. There is no such system other than in someone's imagination. I've worked on systems from 70 to 420MHz, and far larger than most US counties would have experience with- not one had/has 100% coverage, they weren't designed to achieve that, and it wouldn't be possible even someone asked for it. I wish the quack techs and "experts" would stop raising the expectations of emergency personnel by saying otherwise. Maybe if they actually had to design radio systems for a living they wouldn't be so irresponsible.

Cheers,


Richard

Originally Posted by kenisned

Virginia FD Report: Digital Radios Extremely Vulnerable

............

LYNNETTE LUNA
Mobile Radio Technology



On April 16, 2007, firefighter Kyle Wilson was part of a crew dispatched to fight a residential fire in Woodbridge, Va. He died in the line of duty.

A detailed report on the incident recently released by Prince William County Department of Fire and Rescue concluded that problems associated with the use of the county's Motorola digital trunked radio system contributed to the tragedy. Issues reported by other firefighters during that incident, which was further complicated by strong winds, ranged from signal distortion and transmission failure to radios displaying "out of range" signals.

Fire safety advocates now are encouraging fire departments across the country to study the incident in hopes that future tragedies could be avoided. Prince William County's fire department, through further tests, concluded that digital portable radios are "extremely vulnerable to poor environmental conditions and interference of digital noise from ambient sources, which negatively impact the ability of emergency personnel to effectively communicate."

A handful of fire and police departments, fearing the loss of lives, have opted to continue using analog systems even when the rest of their county's emergency personnel are using digital trunking systems.

The common complaint, which most affects fire departments, concerns the digital vocoder's inability to differentiate between a voice transmission and background noise - whether a chain saw, sprayed water or personal alarm. Background noise renders the voice transmission distorted and often unintelligible. Another critical problem is that digital radios lose contact inside buildings. "In most cases, it is a very political and sensitive position to abandon expensive technology and go back to something that is old," said Daryl Jones, owner and president of Telecommunications Engineering Associates, which manages public safety systems throughout the San Mateo area in California. "But many agencies are finding that complaints from line personnel, both in fire and police, are so significant."

The Boise (Idaho) Fire Department spent about $1 million two years ago on mobile and portable radio equipment to join a cutting-edge countywide 700 MHz digital trunking system. While training users on the system, the fire department discovered problems with voice intelligibility when a firefighter's low-air alarm went off. That led the department to investigate the issue further, and it found more instances where alarms interfered with the quality of voice transmissions. Today, the Boise Fire Department and other fire departments in the county remain on analog VHF radios while the rest of the county operates on the 700 MHz digital trunking system.

"Right now our dispatch center wants to dump VHF," said Paul Roberts, a captain with the Boise Fire Department, "[and] we are trying to look at alternatives to at least get on a system that will lessen the load on dispatchers having to patch all of this together. ... But until there is a solution to the digital processing of speech when you have competing noises, we have to stay on analog."

The problems associated with digital systems became known in 2006. Since then, the International Association of Fire Chiefs (IAFC) established a Digital Problem Working Group and appointed Chief Charles Werner of Charlottesville, Va., to serve as its chair. So far, the working group has explored the creation of a best practices solution to work around the problem until a long-term solution can be found. Prince William County's findings have been forwarded to that group for inclusion in the process.

The National Institute of Standards and Technology (NIST) also is analyzing the problem, as are radio manufacturers such as M/A-COM and Motorola. They are expected to jointly release a formal analysis - in conjunction with the IAFC - that encompasses best practices to help departments to minimize the problems.

"We're running through this scientifically and hope to distribute a wrap-up summary shortly," Werner said.

Roberts, who chairs the IAFC testing group, says the testing - conducted by radio engineers - involves taking words that sound alike and requiring the listener to distinguish which word is being said over the background noise of chainsaws and hose sprays.

Motorola declined to comment, saying it was cooperating with the testing and awaiting the conclusions from NIST and the IAFC.

But Chris Lougee, vice president with LMR vendor Icom America, believes better vocoder technology would help solve these background noise problems. "Everyone knew from the beginning that the P25 vocoder was a half-rate vocoder. As you speak into the microphone, you are converting human voice into a data stream that is reassembled at the end," Lougee said. "TIA ... is encouraging a move to a full-rate vocoder, which we are doing. It vastly improves the amount of audio and quality."

Lougee added that, scientifically speaking, digital signals penetrate buildings better than analog signals. "I'm puzzled by that problem," he said. "All of our testing shows that a digital signal produces a higher-quality signal in noise conditions than an analog signal."

Nevertheless, perception has a nasty way of becoming reality, and first responders' perceptions are based on what happens in the field and exacerbated by complaints from the front line.

"The perception of quality of communications in my opinion is much lower on trunked radio systems because it's always based on a comparison of what an agency had before," Jones said. "If they are coming off an analog system that provided 100% coverage and go to a digital trunked system that has different characteristics and less coverage, it's going to be worse."

However, others say the problems have to do with training, as digital systems operate differently than analog. For instance, digital systems require key-up time, forcing first responders to hold down the transmission key longer before they can begin talking. "It's a long and arduous process to educate police and firemen to change the way they have always communicated," Jones said.

But Roberts says best practices aren't the total solution. "They can be easily implemented on a normal day-to-day basis," he said, "but take the same firefighting crew and put them in a panic state. Then I would argue that a lot of best practices are not always utilized."

The Phoenix Fire Department perhaps has had the most experience with the issue. In 2004, the department gathered radio experts and conducted a study of a 700 MHz/800 MHz digital trunked system that covered a 2000-square-mile area. The study concluded that radios lost contact inside buildings and that users often encountered delays and background interference. The department today still operates on a VHF analog system, despite the fact that the rest of the city operates on a digital system.

"Back when we first started working on this project and told our working group that we needed simplex channels, we got a lot of guff about it," said Leif Anderson, deputy chief with the Phoenix Fire Department. "It took us a month or two to finally give them enough information about our problems. ... The SWAT team is now using our simplex channels because they can't have a delay for critical communications."

Today, the department is close to finding a way to bridge its analog system with the rest of the city so that dispatchers don't become overwhelmed with patching the communications systems together, Anderson said. For nearly two years, the department has been studying the use of digital vehicle repeaters (DVRs) mounted on fire trucks that surround the incident area.

"Think of DVRs as being two mobile radios linked together, one working in simplex and the other in duplex or trunked mode," Anderson said. "We'd use the simplex mode when talking to each other on the fireground, but messages to the dispatch center go through the duplex mode."

The Phoenix Fire Department has done much testing of the DVRs around dense buildings that typically pose significant transmission problems. The department is now at a nearly 100% success rate. Now it is moving its existing drills throughout 19 different cities in the department's jurisdiction of about 2500 square miles.

Additionally, the fire department has developed standard operating procedures in case they lose communications. The procedures mostly involve a lot of people repeating messages. For instance, to ensure a fireground message reaches the dispatch center, an incident commander on the outside - who also would have heard the message on the simplex radio - relays the message to the dispatch center. "We've always done this back forth, and it's always been successful," Anderson said.

The operating procedures also involve some changes to the DVRs. "There aren't many customers who use DVRs in simplex-to-trunked mode," said Anderson, "so our vendor had to do some algorithm work. We want every apparatus to have a master/subordinate relationship. If one failed, the next one would take over." Futurecom Systems is the department's DVR vendor.

In the future, Anderson expects firefighters to carry radios, such as the Motorola XTS 5000 radios, with multiple decks and talk groups that will enable first responders to do their jobs much the same way they do today. A call would be dispatched, and the firefighter would be told which tactical radio channel to switch to - simplex or trunked.

For Anderson, it has been a long and arduous process. "We have a saying, 'The sooner we resolve this problem, the sooner we get to go back to the station,'" he said. "That applies here."

Copyright 2008 LexisNexis, a division of Reed Elsevier Inc. All rights reserved.

[kenisned]

Originally Posted by richardc63

Lastly, anyone who actually writes such a silly comment as " coming off an analog system that provided 100% coverage" loses all credability. There is no such system other than in someone's imagination.

That's true. Most "great" analog systems have ~90% coverage. There are almost always dead spots.

I do not believe that we should be using digital for fireground operations. There just isn't a compelling reason to do so.

[richardc63]

Originally Posted by kenisned

That's true. Most "great" analog systems have ~90% coverage. There are almost always dead spots.

I do not believe that we should be using digital for fireground operations. There just isn't a compelling reason to do so.

Hi,

From the "feedback" I've received (pardon the pun) I believe there is. Our firefighters have complained about accoustic feedback when operating closely in confined spaces- this problem has existed with analogue FM since the dawn of time. As the noise level around them rises what do they do? Turn the volume up- exaccerbating the problem. I couldn't count the number of times they've complained that "we can't hear each other because of the noise". Go to C4FM and the problem is solved (combined with the wearing of good quality sound conduction/noise suppression headsets).

I cannot find any rational reason for not using C4FM simplex in such applications. If you can give me technical weaknesses of C4FM over FM in such applications please tell me as I really want to know- but please don't blame the modulation type for the inherent problems associated with working in high ambient noise- I've heard that argument so many times but in every case it is related to the radio user not wearing proper voice conduction equipment.

Regards,


Richard

[JungleJim]

Digital is better, they said so on TV

[ResQguy]

Originally Posted by JungleJim

Digital is better, they said so on TV

Digital TV or Analog TV?

[zerg901]

Richard - FDNY ran into a problem with their digital portables. When doubles occurred, the receivers went silent. Apparently when the Bit Error Rate goes too high, the radios just mute. I am not sure if this issue has been solved or not. If you look in the Announcements Forum, in the 'Firefighters Lose Faith in Digital' thread, you will see a NYC Council report concerning the XTS3500R digital portables used by FDNY (Fire Dept of NY City). Peter Sz

[Mr_Observer]

A poorly designed system will function poorly regardless of the modulation type. The problem as I see it is that politicians and not RF communications experts are designing more and more of these new systems.

I can also think of a situation where the users of the system changed the antennas on their portables from a full sized portable antenna (as they were instructed to use) in favor of a much smaller and much less capable "stubby" antenna. Those radios also went "out of range" when the users entered some buildings. The users were using improperly self-modified portable setups but that did not stop them from complaining about the "poor" system coverage.

All I am saying is that there is more to this than just the flat statement that analog is better than digital.

MR_O

[DaveNF2G]

It's not the system. It's the Motorola radios. As has been noted elsewhere, Motorola's vocoder only operates at half the rate of other brands, which are less vulnerable to background noise. These departments with such problems should switch brands, not systems.

[JungleJim]

I'm sorry but is a comparison between analog and digital. Digital works well within good to moderatly poor conditions. But when the received signal gets to the point where the error correction can no longer keep up, you hit what is known as the wall. You lose a frame of data and the message gets garbled. Not just like dropping a syllable either. Todays coding schemes base each frame on the last one. If you missed the last frame the DSP has to effectively guess where this one should be. Tone and pitch are off.

Miss too many frames and the radio will lose synch and then you need several more frames for it to synch back up. If it can't, it will try and jump back to the control channel and then it will have to synch up and catch its channel assignment again.

With an analog receiver, you get a noisy signal. The human brain is the most powerful computer in the world (it's analog too, by the way) and it can pick out the message and make sense of it with syllables and even whole words missing. Have you ever listened on your scanner and heard the most scratchy, noisy awful burst of crap and think "what the hell was that?" and the dispatcher comes back with a quick 10-4. Her experience allowed her to pick the message out from the noise.

TV is a good representation of this also. I can easily watch a program on tv with some snow and not even notice, but when a digital tv signal gets noised up it gets blocky and then you lose whole frames of video. Sometimes you have to wait a minute or two for it to synch back up.

Don't give me any crap about system design, either. We live in a dynamic world and you will never get perfect coverage in the precise spot you need it at the exact moment you want it. Further, the FCC's position is if you have 100% coverage, you have to much signal and you're causing interference to your neighbors.

The solution to the original problem is actually quite simple. Just like "interoperability" it's not a technical issue but policy. When you get on scene of a major incident, switch to a conventional, simplex, analog channel for fireground. The command vehicle will have enough radios to communicate on the fireground channel and keep in contact with dispatch.

Life is analog

[pogbobo]

off topic, but i've always wondered why some places use digital+encrypted talkgroups for fireground communications ... just seems dumb to me ... and don't get me started on interoperability! to me, nothing says interop like plain old analog ... maybe i missed the point in the P25 interop frenzy thats going on now ...

back on topic!! -->

[richardc63]

Originally Posted by zerg901

Richard - FDNY ran into a problem with their digital portables. When doubles occurred, the receivers went silent. Apparently when the Bit Error Rate goes too high, the radios just mute. I am not sure if this issue has been solved or not. If you look in the Announcements Forum, in the 'Firefighters Lose Faith in Digital' thread, you will see a NYC Council report concerning the XTS3500R digital portables used by FDNY (Fire Dept of NY City). Peter Sz

Thanks Peter- that is interesting. "Doubling" has been a problem since the dawn of radio so isn't the problem more to do with radio etiquette than anything else?

Like take this example I heard the other day (and this ANALOGUE FM)... A senior officer was trying to transmit a lengthy "stop" message but everytime he got started one particular firefighter deliberately transmitted over the top of him... until Comms became so frustrated that they asked him to use his mobile phone to send the stop message. Mind you- they passed on the name & radio ID of the offender. In this case it was relatively "harmless" but blocking & bad manners is nothing new.

Cheers,


Richard

[DaveNF2G]

Originally Posted by JungleJim

Have you ever listened on your scanner and heard the most scratchy, noisy awful burst of crap and think "what the hell was that?" and the dispatcher comes back with a quick 10-4. Her experience allowed her to pick the message out from the noise.

It used to be like that.

What I hear more often is a slightly scratchy but perfectly understandable signal, then a reply like, "You're totally unreadable. Please repeat."

[zerg901]

Richard - I agree, doubling has been a problem for a long time. It can be a huge problem. I am always hoping that someone will manufacture a two way radio that reduces the problem of doubling to nill. However, the conversion from analog to digital has apparently worsened the problems caused by doubling.

Many people say that doubling is caused by operator error. In the high ambient noise situations found at emergency scenes, it doesnt surprise me at all when it happens.

Peter Sz

[richardc63]

Originally Posted by zerg901

Richard - I agree, doubling has been a problem for a long time. It can be a huge problem. I am always hoping that someone will manufacture a two way radio that reduces the problem of doubling to nill. However, the conversion from analog to digital has apparently worsened the problems caused by doubling.

Many people say that doubling is caused by operator error. In the high ambient noise situations found at emergency scenes, it doesnt surprise me at all when it happens.

Peter Sz

Hi Peter,

We have the option now on many radios to inhibit transmit on busy- but that isn't an option my people would accept.

I'm not going to criticise radio users that are in a high stress situation that are too focused on survival to notice that someone else is transmitting. But at the same time technology should not be expected to defy the laws of physics either! We all use some form of Frequency Modulation for the benefits it provides over the alternatives (imagine going back to AM...) so we have to live with its few weaknesses. I can't understand how C4FM & FM "blocking" would be any different- maybe some are so determined to find fault with new technology that they will make out an existing problem to be something new?

Cheers,


Richard

[zerg901]

Richard - a double on a analog system causes noise and a garbled message. A double on a digital system can cause silence. I dont think the digital system is better in this situation. Peter Sz

[richardc63]

Originally Posted by zerg901

Richard - a double on a analog system causes noise and a garbled message. A double on a digital system can cause silence. I dont think the digital system is better in this situation. Peter Sz

Peter,

That depends on the signal levels of the competing signals. If one is sufficiently strong to block the other, how would the radio user be any better or worse off than in the digital scenario? Either way the call can get missed entirely. I certainly wouldn't suggest that digital modulation (C4FM) is any better than analogue FM, just that it isn't any worse. FM is FM... Could I also suggest that the impact of accoustic feedback in this situation shouldn't be underestimated- the sheer fact that firefighters need to turn their radio volume DOWN in a high noise environment due to feedback is, I think, a more likely cause of calls being missed than the situation you describe. The fact that they can turn the volume up when operating C4FM simplex without fear of feedback is a very big plus (according to the firefighters I have spoken to).

Cheers,


Richard

[zerg901]

Richard - I agree - digital is apparently good at preventing audio feedback.

With a double on a analog system, users will hear noise, and be aware that they missed a message. With a double on a digital system - there will just be silence - users may not be aware that they missed a message.

Peter Sz

[kyparamedic]

Originally Posted by zerg901

Richard - I agree - digital is apparently good at preventing audio feedback.

With a double on a analog system, users will hear noise, and be aware that they missed a message. With a double on a digital system - there will just be silence - users may not be aware that they missed a message.

Peter Sz

Can this not be remedied by using a busy channel lockout or something similar? I would take the elimination of feedback anyday over potentially missing a message due to doubling.

[rescue161]

Digital is fine for fire ground ops.

The problem comes into play when they are using a trunking system. Why in the World would you want to run your FG ops through a repeater/trunking system when you're talking to fire team members that are a few feet away or to the Command Post that is right outside???

Back before digital, we NEVER ran our FG ops through a repeater - ever. It was always on a single frequency for transmit and receive.

Switch to a conventional (non-repeater) channel for FG ops. The Command Post can relay info on the TRS if the need arises.