op25 frequency tracking

[boatbod]

I have ported Max's recently announced "-X" tracking code to the boatbod op25 repo. After experimenting with it a bit I changed the way the frequency corrections are handled so that it utilizes the source device's ppm correction then makes residual fine tuning adjustments much as if you had manually entered them either via the terminal or command line parameters.

Operation:
- Specify "-X" on the rx.py command line

Logging Example:

p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721175.743043 control channel timeout
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721175.867949 Adjusting tuning: ppm(-2), fine_tune(309) ["-q -2 -d 309"]
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721181.809586 Adjusting tuning: ppm(-2), fine_tune(230) ["-q -2 -d 230"]
1583721187.820191 Adjusting tuning: ppm(-2), fine_tune(151) ["-q -2 -d 151"]

Notes:
- Only works for rx.py (not multi_rx.py) with the the default cqpsk demodulator (i.e. disabled if you specify "-D fsk4" on the command line)
- Happily works in conjunction with "-q <ppm>" command line options to manually get the tuning close enough to start tracking. Likely to be most useful with old dongles that require significant ppm correction
- For speedier startup you can use the logged "-q" and "-d" parameters directly in your rx.py startup script. e.g. rx.py .... -q -2 -d 151 ....
- The new functionality requires a *full rebuild* of op25 due to changes in the library code interface functions. Suggest running the following script:

cd ~/op25
./rebuild.sh

 

[Night_Watchman]

I pulled your latest commit to test it on my backup RPi. While streaming over my local network I noticed that alpha tags are no longer working. I did not change anything other than pulling and rebuilding.

 

[Night_Watchman]

Disregard the last post. I nuked the installation and redid it from scratch and now the alpha tags are working.

 

[DRL-XM43]

I have ported Max's recently announced "-X" tracking code to the boatbod op25 repo. After experimenting with it a bit I changed the way the frequency corrections are handled so that it utilizes the source device's ppm correction then makes residual fine tuning adjustments much as if you had manually entered them either via the terminal or command line parameters.

Operation:
- Specify "-X" on the rx.py command line

Logging Example:

p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721175.743043 control channel timeout
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721175.867949 Adjusting tuning: ppm(-2), fine_tune(309) ["-q -2 -d 309"]
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
p25_framer::rx_sym() tuning error +1200
1583721181.809586 Adjusting tuning: ppm(-2), fine_tune(230) ["-q -2 -d 230"]
1583721187.820191 Adjusting tuning: ppm(-2), fine_tune(151) ["-q -2 -d 151"]

Notes:
- Only works for rx.py (not multi_rx.py) with the the default cqpsk demodulator (i.e. disabled if you specify "-D fsk4" on the command line)
- Happily works in conjunction with "-q <ppm>" command line options to manually get the tuning close enough to start tracking. Likely to be most useful with old dongles that require significant ppm correction
- For speedier startup you can use the logged "-q" and "-d" parameters directly in your rx.py startup script. e.g. rx.py .... -q -2 -d 151 ....
- The new functionality requires a *full rebuild* of op25 due to changes in the library code interface functions. Suggest running the following script:

cd ~/op25
./rebuild.sh

My set up does not have a rebuild.sh

I did do a git pull and ./build

Is there something else I need to do?

 

[waynedc]

Hi Boatbod,

Thank you for the updates. This is my command line with the -X added:

./rx.py --args 'rtl' -N 'LNA:40' -S 57600 -T bs.tsv -X -l http:0.0.0.0:8080 -V -2 -v 5 -U 2> stderr.2

This is what I use to get in the terminal window after starting OP25 prior to pulling down this last update:

n7zqs@ubuntu:~/op25/op25/gr-op25_repeater/apps$ ./bs.sh
linux; GNU C++ version 7.3.0; Boost_106501; UHD_003.010.003.000-0-unknown

gain: name: LNA range: start 0 stop 0 step 0
setting gain LNA to 40
supported sample rates 250000-2560000 step 24000

This is what I get in the terminal window with the latest Boatbod git pull whether the -X is there or not:

n7zqs@ubuntu:~/op25/op25/gr-op25_repeater/apps$ ./bs.sh
linux; GNU C++ version 7.3.0; Boost_106501; UHD_003.010.003.000-0-unknown

OP25 still works great with the latest update, I just lost the visual on the set LNA gain and step rate. Just a note in case that was an issue...

 

[waynedc]

My set up does not have a rebuild.sh

I did do a git pull and ./build

Is there something else I need to do?

I see the rebuild.sh in the "first" OP25 directory after doing the latest pull. It is there along with the install.sh.

 

[DRL-XM43]

I see the rebuild.sh in the "first" OP25 directory after doing the latest pull. It is there along with the install.sh.

OK thanks - I did over today and it all worked as advertised

 

[boatbod]

This is what I get in the terminal window with the latest Boatbod git pull whether the -X is there or not:

n7zqs@ubuntu:~/op25/op25/gr-op25_repeater/apps$ ./bs.sh
linux; GNU C++ version 7.3.0; Boost_106501; UHD_003.010.003.000-0-unknown

OP25 still works great with the latest update, I just lost the visual on the set LNA gain and step rate. Just a note in case that was an issue...

That is correct, the "missing" information now appears in the log file. It's a seemingly innocuous update but it encompasses a lot of small tweaks to make the python2 code compatible with python3 so that it'll eventually be able to run under gnuradio-3.8. The latter is somewhat of a work-in progress and not high on my priority list (yet).

 

[KA1RBI]

Hi Graham

Thank you for your work on improving OP25. I took a look at your implementation and had a few comments...

1) The way that -d (fine-tune) works is problematic. When multiple systems on multiple different VHF/UHF bands are being scanned the required Hz offset will be different (for a given PPM). For example an error of 1 PPM would be ~450 Hz at 450 MHz, and ~850 Hz in the 800 MHz band; however there's no way to do that using -d. Currently we define the "-q" parameter as type float (even though the actual hardware register is just an int). It seems tempting to change -q so that if the value entered by the user includes a fractional part (e.g., 1.2) we would use it to derive a starting value for the frequency offset. I'm not sure how much trouble it's even worth since the tuning can change by up to, say, 100 Hz (of offset error) each time the hardware frequency is changed, even within the same band, due to imprecision in the SDR tuning process. Ideally I would very much like to not even have to have a -d option...

2) Each time src.set_freq_corr() is used to change the correction (up or down by X) there are several bad things that can happen - there is a lag time before it takes effect; the resulting change in observed error may not jump up or down by exactly X PPM; the Costas loop may even fall one or more zones out of phase (requiring a resync). For these and other reasons I suggest the call to set_freq_corr be scheduled (1) at a much slower rate than the potential 3 sec time quantum currently used, and (2) waiting until after any changes in error_band have been well and truly digested and sorted out first.

3) in the comment that
# avoid hunting by only compensating errors over 200hz
were you seeing hunting? oscillations? some other related problem? did you try reducing the 0.15 error update multiplier?

73

Max

 

[boatbod]

1) The way that -d (fine-tune) works is problematic. When multiple systems on multiple different VHF/UHF bands are being scanned the required Hz offset will be different (for a given PPM). For example an error of 1 PPM would be ~450 Hz at 450 MHz, and ~850 Hz in the 800 MHz band; however there's no way to do that using -d. Currently we define the "-q" parameter as type float (even though the actual hardware register is just an int). It seems tempting to change -q so that if the value entered by the user includes a fractional part (e.g., 1.2) we would use it to derive a starting value for the frequency offset. I'm not sure how much trouble it's even worth since the tuning can change by up to, say, 100 Hz (of offset error) each time the hardware frequency is changed, even within the same band, due to imprecision in the SDR tuning process. Ideally I would very much like to not even have to have a -d option...

I concur that any fixed offset can be problematic, especially during larger tuning jumps when a ppm-derived correction would be more accurate. Unfortunately I don't think the average casual op25 user would wrap their head around setting up an accurate floating point ppm value for the -q parameter. Certainly I can implement it in case anyone feels like trying it, but I think the real longer-term solution is to figure out the auto-tune so that people just don't have to figure out ppm values and tuning offsets etc.

Perhaps it's just me only using op25 on the 700Mhz band, but do people really use the rx.py variant tuning to both 700/800Mhz and VHF/UHF in the same active config? While it is possible, I think it's pretty unlikely given the different antenna needs etc.

2) Each time src.set_freq_corr() is used to change the correction (up or down by X) there are several bad things that can happen - there is a lag time before it takes effect; the resulting change in observed error may not jump up or down by exactly X PPM; the Costas loop may even fall one or more zones out of phase (requiring a resync). For these and other reasons I suggest the call to set_freq_corr be scheduled (1) at a much slower rate than the potential 3 sec time quantum currently used, and (2) waiting until after any changes in error_band have been well and truly digested and sorted out first.

Good point relative to updating ppm via set_freq_corr(). In practice once the ppm has been determined early in the tuning cycle it is unlikely to change much (if at all) during subsequent tracking. I imagine it would be a good idea to save the last adjusted ppm value and simply not call the driver for an update if the value hasn't changed.

As an observation, tuning adjustment is generally very slow to get going; approx 3 seconds or so from application startup, then it takes several additional cycles to home-in to the final value. I wouldn't say this is necessarily bad, but I have considered whether it should be triggered differently, possibly from it's own timer-based thread.

3) in the comment that
# avoid hunting by only compensating errors over 200hz
were you seeing hunting? oscillations? some other related problem? did you try reducing the 0.15 error update multiplier?

When left to continuously adjust itself, I find the tuning offset value walks incrementally up and down with little change in calculated error frequency until it suddenly takes a big jump in one direction or another. It's as if you need to cross an unseen threshold for anything to happen, then all of a sudden the error frequency will change sign (positive to negative or vice versa) and jump by about 300-400hz. At this point the tracking loop starts compensating the other way until the same thing happens in reverse. Putting a 200hz limit on the adjustments took away most of the "chasing it's tail".

I did experiment with different increment multipliers and even tried using freq_corr=sqrt(err_hz) to make super-fine adjustments when the error was small, but the overall tracking behavior was the same. I'm not good with SDR math, but I'd guess the error freq calculation in gardner_costas might need some tweaking.

As an aside, watching the datascope plot during tracking shows correlation between large reported error hz at the same time as the eye diagram toggles between it's two stable vertical positions. With manual tuning I usually associate this step change in the datascope eye with the exact tuning midpoint. In other words, stepping one side or other of the mid-point likely corresponds to a sign change of the error freq.

 

[boatbod]

Max, here is an example of what happens when tracking is allowed to run continuously.

audio device: default
1583934112.641239 frequency_tracking    -122    0    -18    0    18
1583934118.648055 frequency_tracking    339    0    32    0    -32
1583934124.655085 frequency_tracking    -120    0    14    0    -14
1583934130.662442 frequency_tracking    335    0    64    0    -64
1583934136.669468 frequency_tracking    -123    0    46    0    -46
1583934142.679467 frequency_tracking    -123    0    27    0    -27
1583934148.687146 frequency_tracking    -122    0    9    0    -9
1583934154.693924 frequency_tracking    337    0    60    0    -60
1583934160.701810 frequency_tracking    -121    0    41    0    -41
1583934166.707892 frequency_tracking    -122    0    23    0    -23
1583934172.715326 frequency_tracking    -123    0    5    0    -5
1583934178.723387 frequency_tracking    335    0    55    0    -55
1583934184.729590 frequency_tracking    -122    0    37    0    -37

First column is the freq_error, second is error_band, third column is the tuning_error and the last two columns are the applied ppm and fine_tune offset.

As you can see, applied correction went from -41 to -23 to -5 and next cycle measured error was -122, -123, 335 respectively.

 

[NCAmateurRadioGuy]

Perhaps it's just me only using op25 on the 700Mhz band, but do people really use the rx.py variant tuning to both 700/800Mhz and VHF/UHF in the same active config? While it is possible, I think it's pretty unlikely given the different antenna needs etc.

Funnily enough, I use OP25 with one active config to monitor a VHF high P25 P2 system and a 7/800 MHz P25 P1 system. Works very well, I use a discone and the 7/800 suffers from somewhat sloppy signal (largeish dots on constellation) from being aroudn 15-20 miles away while the VHF is super strong being only a couple miles out.

 

[daverdfw]

Looks like the -X option doesn't work for airspy devices, op25 never starts when issuing that argument.

 

[wgbecks]

I can't speak for Max, KA1RBI who wrote the code, but I have reason to believe that it was targeted for the RTL SDR type dongles that seem to
always need TLC in order to get them properly compensated. On the other hand, I run several Airspy R2's and a couple of Mini's and have never had any problems getting them to decode trunk signal blocks (TSBK's) right out of the box with nothing more that (-q 0 ) on my rx.py command lines.

 

[KA1RBI]

Looks like the -X option doesn't work for airspy devices, op25 never starts when issuing that argument.

can you be more specific? Assuming you're running rx.py. Does rx.py quit immediately? Or does it start up and run but fail to acquire the control channel? What happens when you run without including the -X option?

As far as the type of SDR device, OP25 device support is provided by the osmosdr package, and each device has its own specific "args" string. The airspy is listed as supported.

There is nothing special in -X that applies to the RTL without applying equally to the airspy.

For details on the driver and the rx.py --args string, refer to GrOsmoSDR - gr-osmosdr - Open Source Mobile Communications

Max

 

[boatbod]

Looks like the -X option doesn't work for airspy devices, op25 never starts when issuing that argument.

I'm running an airspy mini with my version of op25 using -X right now and it is working just fine. Can you run your op25 with and airspy but without the -X or does it not work at all?

ETA: I just built and installed Max's osmocom version and am able to run that with my airspy as well, both with and without -X on the command line.

 

[wgbecks]

Just tested (-X) with boatbod's fork using an Airspy R2 and it works. See output from my logfile below....

gr-osmosdr 0.1.4 (0.1.4) gnuradio 3.7.13.4
built-in source types: file osmosdr fcd rtl rtl_tcp uhd miri hackrf bladerf rfspace airspy airspyhf soapy redpitaya freesrp
Using AirSpy NOS v1.0.0-rc10-3-g7120e77 2018-04-28, samplerates: 2.5M 10M
gain: name: LNA range: start 0 stop 15 step 1
gain: name: MIX range: start 0 stop 15 step 1
gain: name: IF range: start 0 stop 15 step 1
setting gain LNA to 15
setting gain MIX to 15
setting gain IF to 15
supported sample rates 2500000-10000000 step 7500000
RTL Gain of 15 set to: 15.0
Airspy decim:1 kernel size:47
Using two-stage decimator for speed=2500000, decim=25/4 if1=100000 if2=25000
Project 25 IMBE Encoder/Decoder Fixed-Point implementation
Developed by Pavel Yazev E-mail: pyazev@gmail.com
Version 1.0 (c) Copyright 2009
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see the file ``LICENSE'' for details.
op25_audio:pen_socket(): enabled udp host(127.0.0.1), wireshark(23456), audio(23456)
p25_frame_assembler_impl: do_imbe[1], do_output[0], do_audio_output[1], do_phase2_tdma[0], do_nocrypt[0]
03/24/20 18:13:18.913899 Reading whitelist file
03/24/20 18:13:18.914013 Reading blacklist file
03/24/20 18:13:18.969820 set control channel=853.925000
streaming server url="http://localhost:8000/op25.m3u"
03/24/20 18:13:19.360267 do_metadata state=1: [None] None
03/24/20 18:13:22.429520 Adjusting tuning: ppm(0), fine_tune(47) ["-q 0 -d 47"]
03/24/20 18:13:25.442090 Adjusting tuning: ppm(0), fine_tune(83) ["-q 0 -d 83"]
03/24/20 18:13:28.511189 Adjusting tuning: ppm(0), fine_tune(119) ["-q 0 -d 119"]