OP25 Boatbod OP25 locks up

[ewh01]

Recently I have been having OP25 lock up. I finally pulled the stderr.2, listed below from the point of error. Any help on why it is doing this and how to correct it?
Please let me know if I need to post the entire stderr file text.





freq(771618750), slot(1), prio(3) 01/26/23 18:33:02.916864 PCM underrun Exception in

thread Thread-1: Traceback (most recent call last):

File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner

self.run()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 987, in run

self.callback(msg)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 1604, in process_qmsg

self.update_state('tdma_duid%d' % m_type, curr_time)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 2060, in update_state

self.set_frequency({

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 1343, in set_frequency

self.frequency_set(params)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 525, in change_freq

self.freq_update()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 533, in freq_update

error = self.demod.get_freq_error()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/p25_demodulator.py", line 441, in get_freq_>

return int((self.fll.get_frequency() / TWO_PI) * self.if_rate)

OverflowError: cannot convert float infinity to integer

Exception in thread Thread-5:

Traceback (most recent call last):

File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner

self.run()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 987, in run

self.callback(msg)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 998, in process_qmsg

if self.tb.process_qmsg(msg):

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 934, in process_qmsg

self.freq_update()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 533, in freq_update

error = self.demod.get_freq_error()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/p25_demodulator.py", line 441, in get_freq_>

return int((self.fll.get_frequency() / TWO_PI) * self.if_rate)

OverflowError: cannot convert float infinity to integer

OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO>

Thanks!!

 

[boatbod]

Recently I have been having OP25 lock up. I finally pulled the stderr.2, listed below from the point of error. Any help on why it is doing this and how to correct it?
Please let me know if I need to post the entire stderr file text.





freq(771618750), slot(1), prio(3) 01/26/23 18:33:02.916864 PCM underrun Exception in

thread Thread-1: Traceback (most recent call last):

File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner

self.run()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 987, in run

self.callback(msg)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 1604, in process_qmsg

self.update_state('tdma_duid%d' % m_type, curr_time)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 2060, in update_state

self.set_frequency({

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/trunking.py", line 1343, in set_frequency

self.frequency_set(params)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 525, in change_freq

self.freq_update()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 533, in freq_update

error = self.demod.get_freq_error()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/p25_demodulator.py", line 441, in get_freq_>

return int((self.fll.get_frequency() / TWO_PI) * self.if_rate)

OverflowError: cannot convert float infinity to integer

Exception in thread Thread-5:

Traceback (most recent call last):

File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner

self.run()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 987, in run

self.callback(msg)

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 998, in process_qmsg

if self.tb.process_qmsg(msg):

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 934, in process_qmsg

self.freq_update()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/./rx.py", line 533, in freq_update

error = self.demod.get_freq_error()

File "/home/radiopi/op25/op25/gr-op25_repeater/apps/p25_demodulator.py", line 441, in get_freq_>

return int((self.fll.get_frequency() / TWO_PI) * self.if_rate)

OverflowError: cannot convert float infinity to integer

OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO>

Thanks!!

Is this gnuradio-3.10 or gnuradio-3.8?
I've seen this error before, but only in gr-3.10 when I was having issues with some not thread-safe behavior. A workaround is easy enough to implement, but on it's own that doesn't explain why digital.fll_band_edge_cc::get_freq_error() is returning infinity.

 

[ewh01]

I did a fresh install on a clean card about 1-2 months ago, so whatever came with the package. How would I check to be sure?

and- of course it's been running fine since I posted the previous message...

 

[boatbod]

If you've recently run a "git pull" you should have the python change necessary to prevent the lockup.

 

[ewh01]

ok, thanks. FYI - it's been going good for a while now, thanks for the info anyway.

 

[ewh01]

I spoke too soon. Locked up again today. Here is what is in the stderr.2 after the last "voice update":

02/01/23 11:56:24.756571 UI Timeout
02/01/23 12:27:24.941039 UI Timeout
02/01/23 12:55:24.911815 UI Timeout
02/01/23 13:26:25.098590 UI Timeout
02/01/23 13:57:25.288754 UI Timeout
02/01/23 14:28:25.477449 UI Timeout
02/01/23 14:59:25.653066 UI Timeout
02/01/23 15:31:24.898535 UI Timeout
02/01/23 16:04:25.228250 UI Timeout
02/01/23 16:39:26.692951 UI Timeout

I am also seeing some PCM Underrun messages. There was an incident going on at the time so I did a quick copy and restart before I could copy the stderr.2 file.

 

[boatbod]

I spoke too soon. Locked up again today. Here is what is in the stderr.2 after the last "voice update":

02/01/23 11:56:24.756571 UI Timeout
02/01/23 12:27:24.941039 UI Timeout
02/01/23 12:55:24.911815 UI Timeout
02/01/23 13:26:25.098590 UI Timeout
02/01/23 13:57:25.288754 UI Timeout
02/01/23 14:28:25.477449 UI Timeout
02/01/23 14:59:25.653066 UI Timeout
02/01/23 15:31:24.898535 UI Timeout
02/01/23 16:04:25.228250 UI Timeout
02/01/23 16:39:26.692951 UI Timeout

I am also seeing some PCM Underrun messages. There was an incident going on at the time so I did a quick copy and restart before I could copy the stderr.2 file.

A "UI Timeout" generally occurs after you close the web browser, as a result of rx.py/multi_rx.py noticing the browser is no longer responding to the heartbeat messages. On the one hand it means that the central message dispatcher was still working, but on the other it gets us no closer to knowing which thread became deadlocked.

Next time it happens you might try running top for the rx.py pid.
e.g.

ps -ef | grep rx.py
gnorbury  499403  499402 99 Jan30 pts/0    3-18:12:05 /usr/bin/python3 ./multi_rx.py -v 1 -c cfg-test.json
gnorbury  571884    4030  0 18:12 pts/2    00:00:00 grep --color=auto rx.py
gnorbury@yoga2 ~/op25/op25/gr-op25_repeater/apps $ top -H -p 499403

PCM underruns can generally be ignored. They are the result of ALSA audio not filling the buffer quickly enough.