Heartbeat monitor for OP-25

[DC31]

I am running OP-25 on a RPi4 monitoring a P25 phase 1/2 system and using the audio output as the input for TwoToneDetect. It all works quite well except that OP-25 occasionally apparently freezes, maybe monthly or a little more. When I notice that pages are being missed, I open the OP-25 webpage on port 8088. Normally the frequency error number is varying a little displaying a different number every second or two. If this number is not changing, then I know that it is locked up. A reboot quickly fixes it.

is there any heartbeat feature built in to the program that I could monitor to trigger a notification or an auto reboot of the program? Would it be possible for me to monitor the frequency error somehow for changes? If so, where can I find that number generated?

Thanks for the help.

 

[wgbecks]

A common reason for OP25 to freeze is because of a disruption of the USB communications with the SDR. This condition is usually attributable
to mechanical movement of the USB connections due to poor fitting USB plugs, connectors, or extension cables. You might be able to switch
to a different USB connector on your RPI-4 that provides for a tighter electro-mechanical connection.

The author of boatbod op25 has created a udev rule to detect this condition that will attempt to restore normal operation while assuming the
interruption of communications with the SDR was momentary.

Quoiting @boatbod, (bolded text) perform the following steps;

Copy the file "5-sdr_hw.rules" from op25 apps directory to /etc/udev/rules.d and edit it to make the script home location correct if it is anything other than /home/pi. Reboot or run the command "udevadm control -R" to make the new rule active.

The way this works is that the udev trigger causes any instances of rx.py (or multi_rx.py) to be killed, then systemctl detects this and restarts the process approximately 5 seconds later. Assuming the rtl driver issue corrected itself the app should start working again.

 

[n5pwp]

I'm running the Osmocom version of OP25 and I notice mine will stop receiving periodically. I notice that the CC frequency will be toggling back and forth between the main and backup frequencies. I have the log file being tailed and there are a bunch of frequency errors. I find that just ctrl-C-ing it and restarting my shell script is sufficient to get it back to trunking properly. Its as if either the software is slipping on the frequency lock or the SDR I have drifts periodically. I'm using a RTL Blog V3 SDR.

 

[wgbecks]

I'm running the Osmocom version of OP25 and I notice mine will stop receiving periodically. ....

What Linux OS are you using and are you running it nativity or as a Virtual Machine? VM's often require specific attention to configuration
options as may be necessary to maintain constant communications with the SDR. Otherwise, the general information in Post #2 applies.

I run both Osmocom and Boatbod versions flawlessly on a number of Linux platforms including VM's that perform solidly so long as the
USB data transfer to/from the SDR remains constant.

 

[n5pwp]

What Linux OS are you using and are you running

I'm running Rasbian Buster.

options as may be necessary to maintain constant communications with the SDR. Otherwise, the general information in Post #2 applies.

I tried the USB troubleshooting. It uncovered no issues. I have run other apps using the same SDR and the same USB port and had no issues. This only happens with OP25. I sent logs to Max when I first saw it and he said that others had reported the same thing. He had no solutions at the time. I figured an update might resolve the issue but I have gone back to see if there is an update as recently as yesterday and it seems the project has no activity. I have my work around so I'm kind of in the same boat as the OP. But I don't find it a major hardship to notice that I have heard nothing for a while from the scanner and check it and then stop the app and re-start it. It doesn't seem to need a re-start of the device.

The lack of direct responses to the OP seems to indicate there is no solution short of manual intervention.
Mike

 

[Outerdog]

One good workaround might be to use a combination of shell scripting and cron to automate a periodic restart process, say once every 8 hours. Kill the existing process and start it again.

Something like this as an example (untested):

#!/bin/bash

# Find the process ID of the Python3 process running multi_rx.py
PID=$(pgrep -f "python3.*multi_rx.py")

# Check if the PID exists
if [ -z "$PID" ]; then
  echo "No process found for multi_rx.py running under python3."
else
  # Kill the process
  kill $PID
  echo "Process $PID (multi_rx.py) stopped."
fi

# Execute the start-op25.sh script
./start-op25.sh

 

[wgbecks]

Normally the frequency error number is varying a little displaying a different number every second or two. If this number is not changing, then I know that it is locked up. A reboot quickly fixes it.

I have Osmocom OP25 along with OPLOG running under Ubuntu 22.04.3 that runs 24x7x365 setup to log all activity on a local P25 TRS that
has never frozen or crashed. It occurred to me in rereading your post that you may have the (-X) Error Tracking option enabled that could
have a bug that eventually leads to the reported problem.

Upon my initial installations, I do enable (-X) to observe Error Tracking, but only use this to guide in trimming the SDR to operate with a minimum of frequency error, and then I disable it for normal operation. You may want to verify this parameter is set, disable it, and retest to determine if this feature is at fault with your installation.

 

[DC31]

Thanks for the input @wgbecks. I was also starting my OP-25 with the -X option set. I have now installed a usb extension cord between the port on my pi and the sdr and have also removed the -X

so far, so good

even so, I was kinda hoping that someone could point me toward where I could grab the frequency error number from in the program. Then a simple watchdog script monitoring changes to this number could detect a freeze and send an alert.

 

[boatbod]

Thanks for the input @wgbecks. I was also starting my OP-25 with the -X option set. I have now installed a usb extension cord between the port on my pi and the sdr and have also removed the -X

so far, so good

even so, I was kinda hoping that someone could point me toward where I could grab the frequency error number from in the program. Then a simple watchdog script monitoring changes to this number could detect a freeze and send an alert.

Freq error is passed over the http interface between the app and the browser. That said, the only way to access it would be to emulate the web terminal. I think it would be easier to hack the underlying python code.

 

[DC31]

Update. Since installing the usb extension cable and removing the -X It hasn’t dropped out.
An extension on another unit running SDRTrunk seems to have fixed drop-outs there, too.

thanks for the help.