op25 boatbod “control channel timeout”

[telvana]

Greetings,

I’ve been using the same install/setup for the last two years and haven’t had a singe issue out of the install — now I’m getting this after about 3 or 4 hours of running and I can’t figure out what in the world changed other than updating to a newer release a few days ago.

Using this for my startup (it’s also running as a service);

./rx.py --nocrypt --args "rtl=0" --gains 'lna:36' -S 57600 -q 0 -d 0 -v 1 -2 -T lake_trunk.tsv -V -w -M my_meta.json -l http:192.168.0.15:12345

tsv file;

"Sysname" "Control Channel List" "Offset" "NAC" "Modulation" "TGID Tags File" "Whitelist" "Blacklist" "Center Frequency"
"Lake County" 855.8625 0 0 "c4fm" "lake.tsv" "white.tsv"

(this is of course properly tab delimited in the actual file, but RR just mangled it)

May 20 23:42:11 raspberrypi bash[23988]: 05/20/22 23:42:11.168480 control channel timeout
May 20 23:42:12 raspberrypi bash[23988]: 05/20/22 23:42:12.192170 control channel timeout
May 20 23:42:13 raspberrypi bash[23988]: 05/20/22 23:42:13.216538 control channel timeout
May 20 23:42:14 raspberrypi bash[23988]: 05/20/22 23:42:14.238941 control channel timeout
May 20 23:42:15 raspberrypi bash[23988]: 05/20/22 23:42:15.263471 control channel timeout
May 20 23:42:16 raspberrypi bash[23988]: 05/20/22 23:42:16.287285 control channel timeout
May 20 23:42:17 raspberrypi bash[23988]: 05/20/22 23:42:17.439046 control channel timeout

I assumed the dongle all of a sudden started to drift, but the plots all look great until this happens then this just randomly happens. If I restart the service it’s just fine again for a few hours and then it just drops the control channel again. Only thing I can think of is something changed with the antenna. I changed the dongle to a brand new one, same issue.

 

[boatbod]

What do plots #2 (constellation) and #5 (mixer) look like when the control channel disappears?

 

[telvana]

I’m not sure, I didn’t get a chance to check it out — but I did figure the issue out, and it’s strange. It turned out to be the antenna, I swapped to an indoor antenna and all is well now, at least it seems to be. I’m curious if it’s multipath that wasn’t there before they cut down the tree in the backyard the day this all started? lol.

 

[boatbod]

I’m not sure, I didn’t get a chance to check it out — but I did figure the issue out, and it’s strange. It turned out to be the antenna, I swapped to an indoor antenna and all is well now, at least it seems to be. I’m curious if it’s multipath that wasn’t there before they cut down the tree in the backyard the day this all started? lol.

You can usually see multipath in the constellation plot; it's appear as pulsing/instability in the amplitude. Ideally the plot should show 4 tight clusters at the corners of a box, all located on the unit circle. Under multipath conditions there will be brief appearances of reduced amplitude and dispersed points more in an "X" pattern than a cluster.

 

[telvana]

Yep, that’s what I was seeing — but it was working just fine until the tree came down. I switched to the interior antenna and the plot looks great now. I’ll grab pictures;

Interior antenna:

Exterior antenna:

I used to get around this by using a Yagi, but I now use that mast for other purposes. I get good reception on the interior antenna, so I’ll just stick with that for now.

Thanks for the info, it at least gives me an idea of what was going wrong.

 

[boatbod]

Those are some great plots! Wish mine were always that clean, but usually there's a good dose of multipath going on around here. Moving the antenna just 6" can change things significantly for the better or worse!

 

[telvana]

Really? I am about 400 feet from the transmitting tower, I have at times been outside and pulled the antenna off of my scanner and still received it without issue, if anything it stopped the multipath issues on the BCD436HP I was using at the time. lol

 

[boatbod]

Really? I am about 400 feet from the transmitting tower, I have at times been outside and pulled the antenna off of my scanner and still received it without issue, if anything it stopped the multipath issues on the BCD436HP I was using at the time. lol

Lowering receive signal strength is definitely one way of reducing the impact of multipath. The point being to receive just the closest transmitter rather than have the more distant lower-strength signals start mixing in and causing fading and phase distortion.

 

[telvana]

Well, it looks like it’s dumping the control channel again. I’ve changed my rx.py command around a bit, but I still can’t quite figure out why it’s dumping the channel, the difference is now it’s actually recovering — but still doing it all the same. I haven’t had any complaints yet, and so far the only way I know it is losing the channel is due to the TSBKS resetting.

(op25.sh)
./rx.py --args "rtl" --gains 'lna:46' -S 960000 -T lake_trunk.tsv -v 2 -2 -V -w -M my_meta.json -l http:192.168.0.4:12345 2> stderr.2

I feel like I’m missing something in this command, it’s evolved from roughly 2017-2018 to what it is now, so it’s likely a mix of the old way of doing things and the new way of doing things.

Any idea what I should be doing to diagnose this?

The first image is the constellation of the control channel, and the second is during RX of a channel.

 

[boatbod]

Televana, are you running my recently released update (posted 6/10/22) or previous earlier code? That update was pretty significant in that it contained major revisions to the demodulator. The constellations look tight and clean. How often are the timeouts occurring and do you see any pulsing/noise bursts that cause transient degradation of the signal?

Is there any chance you have an equivalent multi_rx.py config that you could try? I personally don't run the old rx.py app much any more, so it's possible I've introduced a bug somewhere along the way and not noticed it during testing in my dev environment.

 

[telvana]

It’s possible it’s a code regression issue, I’d imagine — the issues started after a git pull; but, I don’t use multi-rx even though I should look into it. I’m only running a single feed which is why I’ve never really bothered to do it yet.

I haven’t seen any pulsing issues, etc, if anything the overall signal quality has become much better with the new release — it’s just the loss of the control channel that is the only issue. As for the timeouts they don’t really follow a pattern, they are completely random, and I can’t even find them in the stderr quite easily without awking because they are nowhere near as frequent as they were before, but I know they are still happening.

For instance, it’s running great right now. I believe this is the take your car to the mechanic issue; I find an issue, ask for help, then it quits doing it. 😀 I just did another git pull and it pulled in some new code, I believe the last update I did was on the 8th according to my logs. I’ll try the 6/10 revision and get back to you.

 

[boatbod]

There are some circumstances where a timeout is predictable:
- any time you toggle a plot on/off (which briefly pauses the flowgraph)
- if the signal shifts enough in phase or frequency to cause a rotation of the costas phase-locked-loop into the next quadrant
- if the incoming signal is significantly disturbed by multipath fading

If the occurrences are infrequent I really won't spend too much time worrying about it.

 

[telvana]

I’ll keep an eye on it and see if I can further pin it down. I thought maybe there might be a clean-cut answer for it, but so far I’m not any closer to an answer than I was a few weeks ago. The good news is so far it is much less frequent, I’d say once every couple of days it’ll lose it for a few seconds and come back, the issue I was originally having is that it never came back, so at least it’s getting better.

 

[boatbod]

I’ll keep an eye on it and see if I can further pin it down. I thought maybe there might be a clean-cut answer for it, but so far I’m not any closer to an answer than I was a few weeks ago. The good news is so far it is much less frequent, I’d say once every couple of days it’ll lose it for a few seconds and come back, the issue I was originally having is that it never came back, so at least it’s getting better.

Have you been running with "-X" option? The current release has a whole different automatic tracking method that disables & obsoletes that setting. Previously it did have cases where it would occasionally "walk away" from the frequency of interest and never make it back. The new FLL does not seem to do that.

 

[telvana]

No, I’m using a TXCO SDR dongle (Nooelec) and didn’t really see it drifting much so I just left it be. I’m not even using -q or -d, it just works out of the box, and honestly adding anything +/- just introduces other issues, so I just left it alone.

 

[boatbod]

No, I’m using a TXCO SDR dongle (Nooelec) and didn’t really see it drifting much so I just left it be. I’m not even using -q or -d, it just works out of the box, and honestly adding anything +/- just introduces other issues, so I just left it alone.

Ok good. Out of curiosity, what sort of tuning-error readings are you seeing now? If you're running the curses terminal it's the (xxHz) reading on that appears after the tuned frequency down near the bottom of the screen. On the web terminal it's labeled "Frequency error:"

 

[telvana]

Frequency error bounces between -50 and +160 with an average of +65ish

 

[boatbod]

Frequency error bounces between -50 and +160 with an average of +65ish

Pretty darned good! No adjustment needed there.

 

[telvana]

I have decided to go w/ multi_rx.py though, from the looks of it, it would streamline a lot of the mess I’ve got going on with my setup in regards to streaming to broadcastify — is there any resources I can look at that says what each of the various options do in cfg.json? I’ve been messing with it lately and I’m pretty much flying blind due to I simply don’t know what all of the various settings do.

 

[telvana]

nvm. that gave me more questions than answers — i’ll just make a new post.