OP25 OP25 Update for Christmas, 2020

[KA1RBI]

posted on Osmocom op25-dev
------------------------------------------------------------------------------------------------------------

OP25 Update December 25, 2020
=============================

1. Experimental HTTP Live Streaming
These hacks ("OP25-hls hacks") add a new option for audio reception
and playback in OP25; namely, via an HTTP live stream to any remote
client using a standard Web browser. The web server software used
(nginx) is industrial-strength and immediately scalable to dozens or
hundreds of simultaneous remote users with zero added effort. More
than one upstream source (in parallel) can be served simultaneously.

See the file apps/README-hls in the repo for more information.

2. NXDN48/NXDN96 RX and TX
The multi_rx and multi_tx apps have been updated accordingly. The
"filter_type" JSON keyword for NXDN channels should be set to
"nxdn", and the "symbol_rate" keyword should be set to 2400 or 4800
for NXDN48 and NXDN96, respectively. A fair number of trunking and
control packets are also currently decoded, although trunked call
following for NXDN systems is not currently implemented.

I would like to take this opportunity to thank Steve, KB9MWR, whose
contribution greatly accelerated the NXDN effort.

3. Graphical (Plotting) Updates
The constellation plot appearance has been enhanced and a new plot
type "correlation" added. The correlation plot is actually five
separate plots (nxdn 48 and 96, p25, dmr, and ysf) and can be used
to show at a glance which of the five modulation types is in use.
The correlation plot was present in OP25 long ago in scope.py and
is now available in both rx.py and multi_rx.py.

4. UDP Linkage Between OP25 TX and RX
Prior to these additions, a hardware RF transmitter/radio was always
required to utilize the OP25 software TX. The OP25 TX application
supports P25, DMR, YSF, NXDN, and DSTAR, and supports both halfrate
and fullrate xMBE encoding, in real time, via its built-in software
voice encoders. An arbitrary number of channels and modes may be
transmitted simultaneously, within the usual spectrum spacing rules.

The UDP additions allow the TX and RX to be linked. The example
configuration contains two P25 channels, one trunk control channel
and one voice channel. This permits rapid testing with no need to
wait for a chance to receive a transmission. There is a python
utility used to generate the fake P25 control channel data to be
broadcast.

See the file apps/README-example-udp in the repo for more
information.

5. Ubuntu 20.04, Python 3 and GNU Radio 3.8
These are all supported in OP25 but until such time as these
versions become more mainstream the changes are applied via a
patch located in the TLD of the repo.

6. Experimental Online Web Configuration Interface
This is a balky and complex addition and is very experimental. To
invoke, run http.py instead of rx.py or multi_rx.py. These latter
two are invoked automatically in the backend by http.py. Starting
from a "blank" configuration, the details are filled in to a web
form and then saved (to a server-side json config). In theory
with these additions there is no further need for TSV files,
although legacy TSVs can be loaded and then saved as json configs.

 

[tctx79]

Would this be an appropriate thread to ask a question about the experimental html streaming?

 

[KA1RBI]

No problem (at least as far as I am concerned); fire away

 

[tctx79]

Thank you.

I followed the instructions in the README_HLS file and was able to successfully get everything running as far as I can tell. I am running it on Raspberry PI OS on RASPI 4/8GB. I am able to open the native browser on the RPI and navigate to both 127.0.0.1:/live.html and 192.168.1.x:8081/live.html and get streaming audio through the browser.

What I can't seem to figure out is how to access it from another device. I have tried on a Windows 10 machine on same LAN using Chrome. I have also tried iPhone Chrome and Safari on wifi. Each of the three will load the page, showing the player and a blank grey background. On the RPI I see the PNG image/video with the scanner data as soon as it loads. This never loads on remote devices nor does any audio play. I'm sure I have missed something somewhere but having gone through the config files I don't see anywhere that would change this.

Any thoughts?

 

[KA1RBI]

ok could you please post the full rx.py command line you're executing.

Also, took a look at the live.html and live.m3u8 files, it appears there are (total) three references to localhost between them, I wonder if changing them from "localhost" to your "192.168.1.x" address would help. It appears that things were set up exclusively for localhost........

Failing that, separately I'd like to see the nginx access and error logs. Since you have it working locally the messages in the logs should reflect what things would look like in a proper working scenario.

What needs to be investigated is what the messages look like in the non-working case. Hopefully the two sets of messages will reveal distinct differences between the two.

hth

Max

 

[tctx79]

The command line I used is:

sudo ./rx.py --args 'rtl' -N 'LNA:47' -S 2000000 -q -1 -T trunk.tsv -V -2 -w

I found I was getting an error if I didn't run it as sudo: "IOError: [Errno 13] Permission denied: '../www/images/tmp-status.png'"

I didn't think to look in the live.html/m3u8 files. I went in and edited those and saved as alternate files and moved them over to the correct directory. When I navigated to the "live2.html" file I created, having replaced localhost with the IP of the RPI, I get the same page loading but now have a revolving circle like it's processing something. Still not video or audio.

Could you point me in the direction of getting the nginx access and error logs? This is the first time I've dealt with this program, my apologies.

 

[KA1RBI]

nginx logs location is somewhat system dependent, but in mine they are in
/var/log/nginx/access.log
and
/var/log/nginx/error.log

One other place to look for error messages is in the JS console on the client.

In firefox this is Tools -> Web Developer -> Web console (tools popup from Alt-T). Another related useful tool in here is the Network monitor.

In looking at the nginx logs there might be an error message, or, alternatively, the *lack* of an access message (though present in the working case) may need to be noted and acted on...

Max

 

[KA1RBI]

I ran some tests and found that the page does load properly from another machine on the LAN.

By default firefox in linux does not support mp4 out of the box.

In the setup for OP25-hls, one of the steps is to install ffmpeg shared libs which is needed for the liquidsoap streaming process, but also has the side effect of enabling firefox mp4 support (firefox must be reloaded, of course).

The reason I mention this is that, at first, the page failed to load for me because the hls.js checks for firefox mp4 support were failing, and it was bombing out (until I installed the ffmpeg shared libs on that machine).

I've pushed a new version of live.html which adds several runtime messages that display the status of HLS initialization when live.html is loaded. Could you please pull this (cd to the op25 ..../www directory and run "git pull", then re-copy the live.html file to /var/www/html. You'll need to re-apply the changes for "localhost" to "192.168.1.x").

It should print messages at the bottom of the page - let me know......

Max

 

[WX4JCW]

can this be installed on the same machine as the Boatbod mod, and can they coexist?

 

[tctx79]

In messing around with everything I have borked my system. I was also running Pihole and now its unresponsive. I'm probably going to re-image and start over, so I will download your updated files and try that and report back.

Below are what i got from the errors log and from the console on chrome. I installed Firefox and checked that console and was getting a missing decoder/codec error. Mind i am trying to access the raspi both from a windows machine and iphone/ipad. i have not tried from another linux machine. i tried installing ffmpeg in my c:/directory and updating my windows system path to point to it, but that didn't change anything.

I was also running NUT and nut-cgi to serve a webpage that i could see UPS stats with...thats still working. When i re-image I'll just start with the op25 install. Linux gets a little difficult for me to track down issues when i have multiple programs installed. I'm very much a beginner and do not understand the structure of the OS or troubleshooting so i have to google everything. Been a windows user for too long :/


error.log:
2021/02/13 11:12:42 [error] 719#719: *3 open() "/var/www/html/hls.js.map" failed (2: No such file or directory), client: 192.168.1.6, server: , request: "GET /hls.js.map $
2021/02/13 11:14:47 [error] 719#719: *4 open() "/var/www/html/hls.js.map" failed (2: No such file or directory), client: 192.168.1.6, server: , request: "GET /hls.js.map $




Chrome console (via F12):
DevTools failed to load SourceMap: Could not load content for http://192.168.1.8:8081/hls.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
live2.html:20 Uncaught (in promise) DOMException: play() failed because the user didn't interact with the document first. Autoplay Policy Changes | Web | Google Developers
(anonymous) @ live2.html:20
emit @ hls.js:311
trigger @ hls.js:9194
onManifestLoaded @ hls.js:12250
onEventGeneric @ hls.js:6772
onEvent @ hls.js:6757
emit @ hls.js:311
trigger @ hls.js:9194
_handleMasterPlaylist @ hls.js:8070
loadsuccess @ hls.js:7995
readystatechange @ hls.js:14600
XMLHttpRequest.send (async)
(anonymous) @ VM22:1
loadInternal @ hls.js:14557
load @ hls.js:14512
load @ hls.js:7958
onManifestLoading @ hls.js:7856
onEventGeneric @ hls.js:6772
onEvent @ hls.js:6757
emit @ hls.js:311
trigger @ hls.js:9194
loadSource @ hls.js:20947
attach_hls @ live2.html:17
onload1 @ live2.html:11
onload @ live2.html:45
VM22:1 GET http://localhost:8081/hls/stream.m3u8 net::ERR_CONNECTION_REFUSED

 

[tctx79]

can this be installed on the same machine as the Boatbod mod, and can they coexist?

I had both. I didnt think theyd both be able to run at the same time as my understanding is that the rtl device gets locked into the first running instance. Maybe with two rtl"s?

 

[WX4JCW]

im working on it now

 

[KA1RBI]

tctx - a couple things

1) I've also seen the failures to serve hls.js.map - my impression is this is harmless unless you want to debug the internals of hls.js. The version of hls.js that I distribute is the "debug" version (as far as I can tell) and if it's bothersome you could obtain an "official" copy of the hls.js distro and populate the /var/www/html directory with a copy of both hls.js and of hls.js.map (make sure to use a matched pair).

2) the DOM exception in "play" is likely due to a handy firefox restriction that prevents pages with embedded ads from playing sound without the user explicitly asking to play the file. There is some sort of way to add an exception but I haven't played with that.

WX4JCW - haven't attempted to run both versions concurrently but my suspicion is that whichever version is installed last would overwrite the installed libraries of any previous version. To be on the safe side I'd recommend that you change to the 'build' directory of the currently-installed copy of OP25 and run the "sudo make uninstall" command from that dir, prior to installing any other version. It's possible Graham may have a different take on this.........

Max

 

[boatbod]

can this be installed on the same machine as the Boatbod mod, and can they coexist?

Not necessarily impossible, but certainly not straight-forward; the main issue being two different versions of the same shared libraries which would not happily coexist.

If you really wanted to do it, you'd need to use LXC/LXD, Docker, or a full virtual solution. For development purposes I run a series of VirtualBox machines with different distros. e.g. Ubuntu 16.04/Mint19/Ubuntu 20.04/Mint20/Debian Buster/Debian Bullseye. Works fine, but requires disk space to dedicate to each machine.

I've played with LXC/LXD as a way of running four instances of PiHole on two IP subnets with just two real Pi4 boards. It's lighter weight and works ok, but has it's PITA factor getting things set up in a sane manner. YMMV

 

[WX4JCW]

Thanks Boat

 

[tctx79]

I ran some tests and found that the page does load properly from another machine on the LAN.

By default firefox in linux does not support mp4 out of the box.

In the setup for OP25-hls, one of the steps is to install ffmpeg shared libs which is needed for the liquidsoap streaming process, but also has the side effect of enabling firefox mp4 support (firefox must be reloaded, of course).

The reason I mention this is that, at first, the page failed to load for me because the hls.js checks for firefox mp4 support were failing, and it was bombing out (until I installed the ffmpeg shared libs on that machine).

I've pushed a new version of live.html which adds several runtime messages that display the status of HLS initialization when live.html is loaded. Could you please pull this (cd to the op25 ..../www directory and run "git pull", then re-copy the live.html file to /var/www/html. You'll need to re-apply the changes for "localhost" to "192.168.1.x").

It should print messages at the bottom of the page - let me know......

Max

Sorry for the late response. I had to reimage two or three times as I did some stupid stuff and didn't install everything by the book.

Finally got a solid install and edited the live.html/m3u8 files to my RPI IP and it is working. I'm streaming on my Windows 10 PC through Chrome and Firefox. It is working on my iPhone as well.

Thank you very much for the help and guidance!

 

[KA1RBI]

ok thanks. I've updated the README-hls file to add info about editing the html/m3u8 files for the non-localhost situation.

Also, separately, there was a crash reported in rx.py a couple of weeks ago that seems to occur on busier trunk control channels. I've pushed a possible fix for that crash to the repo. A simple 'git pull' (with the current directory somewhere inside the repo) will pull down the updates. No need to rebuild as there are no changes to the libraries...

I'd be curious to hear what happens if you connect multiple concurrent clients, say, win10 + chrome + iphone all at once. My expectation would be a) little to no effect on server load, and b) a few seconds of time skew between the various feeds, unless the client(s) connect at precisely the same instant...

Max