Trunk Recorder + Trunk Player (op25 + recording + web interface)

Status
Not open for further replies.

northernsummit

Member
Feed Provider
Joined
Jul 10, 2009
Messages
289
Location
Macedonia, Ohio
I was surprised to find out these two open source applications were not already being discussed here so I'm going to out them for those of you looking for something like this.

A few months ago friends of mine were playing with DSD+ and the windows based Trunking Recorder using nooelec neSDRs and we were struggling to get it to decode cleanly, remain stable, etc. I've found that this method using op25 seems to produce far more consistent and nicer quality decode.

One of those friends of mine stumbled upon trunk-recorder written by Luke Berndt which uses op25 and gnuradio, but has matured. What was missing was a Trunking Recorder like web interface - but then we somehow found this trunk-player (written by Dylan Reinhold) which is already written to accept data from Trunk Recorder. When we showed an interest in his work he released it on GitHub and has been tweaking it based on some feedback we've given him.

https://github.com/robotastic/trunk-recorder
https://github.com/ScanOC/trunk-player

Screen shot of the web based player: https://github.com/ScanOC/trunk-player/raw/master/docs/images/trunk_player_main.png?raw=true

A few of us have trunk-recorder running on (admittedly beefier CPU) Ubuntu linux machines using 3 SDR dongles to capture an entire P25 or Smartnet system. The nice thing about trunk-recorder is it is entirely self contained - there is no need to run a CC channel app, nothing to direct decoders, nothing to interact via virtual cables. Trunk recorder basically listens to the CC, directs the SDRs, and outputs a .wav and a corresponding .json file with metadata. (ie: every unit ID in that transmission - see the screenshot above)

What you need is enough SDRs to effectively cover the system you want to capture and a modest amount of CPU. For me that was just under 6mhz of dongles and a new-to-me computer off eBay - your mileage may vary. I am able to run 3 at 2mhz sample rates. I am using a dual quad core L5420 @ 2.5Ghz - the Ubuntu box looks like this during off peak time for the system:

top - 00:45:58 up 5 days, 4:38, 2 users, load average: 3.65, 4.11, 4.21
Tasks: 178 total, 1 running, 177 sleeping, 0 stopped, 0 zombie
%Cpu0 : 15.0 us, 4.0 sy, 0.0 ni, 80.7 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
%Cpu1 : 8.1 us, 2.0 sy, 0.0 ni, 89.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu2 : 21.0 us, 3.1 sy, 0.0 ni, 74.2 id, 0.7 wa, 0.0 hi, 1.0 si, 0.0 st
%Cpu3 : 10.9 us, 4.2 sy, 0.0 ni, 84.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu4 : 14.5 us, 2.6 sy, 0.0 ni, 82.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu5 : 60.9 us, 1.0 sy, 0.0 ni, 38.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu6 : 30.4 us, 1.7 sy, 0.0 ni, 67.2 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu7 : 8.9 us, 3.8 sy, 0.0 ni, 87.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

Every time trunk-recorder writes a .wav and .json file it will (optionally) run a shell script. There are pre-written shell scripts included between the two applications that act on those two files to make them easily playable. This is where it gets really spiffy.

Every time a .wav and .json file is generated on my SDR machine it will:

1. Push the two files to a second machine (where I'm running the django / web based trunk player).
(You could do this at your house or put the web based server out on the internet)
2. Add a entry to a sql database for the webserver. This makes the web interface know everything.
3. optionally convert the .wav file to a .mp3 for size. I am doing this.
4. push the .mp3 file up to Amazon S3 storage so it can be referenced - this isn't required, but its already written into the scripts. If you don't do this you have to find a place to put the .wav files where they can be accessed via a web browser.

In the web interface you can simply go to the "default" page where every single .mp3 file is played as they arrive - or you can build scanlists, which basically creates a web-based scanner.

The really cool scaling feature about the way trunk-player works is you could theoretically have 2-3+ friends each setup a recorder for a given P25 / Smartnet system and all of you feed the same web based front end. Then all of you can listen to each of those systems, mixing talk-groups into scan lists of interest and being able to access it from anywhere on the Internet - which is great if one of you is in a better position to overcome simulcast distortion than your friends?

The way trunk-recorder works is if radio traffic occurs back-to-back on the same voice frequency it will record that all in a single .mp3 file until a timeout is reached or some other talk-group appears there. This takes some getting used to and admittedly adds a little bit of delay to when you hear a transmission in real-time, but by default trunk-recorder doesn't record the actual silence between the transmissions, so you hear everything back-to-back quickly. That is now adjustable, but some people may like that to "catch up" to the delay of listening to multiple talk-groups.

The benefits of this are pretty huge since you get 100% of every transmission for every talk-group, so while you may setup a scan list that gets "behind" by 2-3 minutes you know you're hearing every transmission of every talk-group that interests you.

This isn't for the faint at heart. While installation has clearly improved and become easier you're going to need to modify some files, tweak some settings, etc. to make it do what you want. If you're comfortable doing that these two apps are worth your time.

As more of us start providing feedback and learning the quirks the installation should become more and more easy for the novice end user.
 

deboe

Member
Joined
Dec 19, 2002
Messages
142
Location
0x045C
Thanks for the write up northernsummit...

Agreed, both apps are rough around the edges but once you get them running it's awesome.

As more of us start providing feedback and learning the quirks the installation should become more and more easy for the novice end user.

As the author of https://github.com/ScanOC/trunk-player could not agree more with that.

I'm spending most of my time making it work, not making it work easy, so when people like northernsummit ask question or get stuck, I end up re writing that part to make it easier for others.

The author of trunk-recorder also is awesome about taking user feedback and updating the program.

So both programs are open to suggestions (and even code updated via github).

Dylan
- If you want to see trunk-player in action this is the primary site I use it on https://scanoc.com/scan/default
It is the Orange County, CA CCCS Smartnet trunked system, it's capturing the county wide cell and sometimes the South Cell (I don't have a great signal from my place, working a new location to capture from).
 

northernsummit

Member
Feed Provider
Joined
Jul 10, 2009
Messages
289
Location
Macedonia, Ohio
I've evolved my system to use it as a source for a Broadcastify feed. I know of at least 3 different people using this now.

The benefit of streaming off Trunk Recorder is you get 100% of the transmissions and simply place them into a queue. So if your stream has 3-4 talk-groups and occasionally the FD talks while the PD is talking both transmissions make it into your stream in their entirety (just one of them slightly delayed in a queue)

Details here:

Streaming with Trunk Recorder and Liquidsoap - The RadioReference Wiki
 

boatbod

Member
Joined
Mar 3, 2007
Messages
3,316
Location
Talbot Co, MD
Thanks, I'll take a look. Presently I am running a lightly-modified copy of 'max' branch op25 as a broadcastify source with great success, but am interested in being able to save the transmissions for later review rather than having to go to the broadcastify archives and post-process through audacity to remove the (extensive) silences.
 

northernsummit

Member
Feed Provider
Joined
Jul 10, 2009
Messages
289
Location
Macedonia, Ohio
Thanks, I'll take a look. Presently I am running a lightly-modified copy of 'max' branch op25 as a broadcastify source with great success, but am interested in being able to save the transmissions for later review rather than having to go to the broadcastify archives and post-process through audacity to remove the (extensive) silences.

You have no idea how often I use this functionality either because I want to do a DVR like immediate replay of something i just heard or because a local Facebook community will ask "what the heck happened up at XYZ 20 minutes ago?" and you can play back instantly, jump around, etc.

With the archive I always find myself downloading multiple mp3's and using Audacity to jump around.

What were you using for the stream piece? how were you getting the audio from op25 to broadcastify?
 

boatbod

Member
Joined
Mar 3, 2007
Messages
3,316
Location
Talbot Co, MD
You have no idea how often I use this functionality either because I want to do a DVR like immediate replay of something i just heard or because a local Facebook community will ask "what the heck happened up at XYZ 20 minutes ago?" and you can play back instantly, jump around, etc.

With the archive I always find myself downloading multiple mp3's and using Audacity to jump around.

What were you using for the stream piece? how were you getting the audio from op25 to broadcastify?

My broadcastify setup is op25->ALSA->DarkIce. My alsa config has both a loopback device and tee defined so that I can route the audio to darkice at the same time as sending it to the speakers for local monitoring.
Code:
/etc/asound.conf

# output device
pcm.loopout {
  type dmix
  ipc_key 328211
  slave.pcm "hw:Loopback,0,0"
}

# input device
pcm.loopin {
  type dsnoop
  ipc_key 686592
  slaver.pcm "hw:Loopback,1,0"
}

# duplex plug device
pcm.loop {
  type plug
  slave {
    pcm {
      type asym
      playback.pcm "loopout"
      capture.pcm "loopin"
    }
  }
}

pcm.mout {
  type plug
  slave.pcm mdev
  route_policy "duplicate"
}

pcm.mdev {
        type multi

        slaves.a.pcm "hw:Loopback,0,0"
        slaves.a.channels 2
        slaves.b.pcm "hw:0,0"
        slaves.b.channels 2

        bindings.0.slave a
        bindings.0.channel 0
        bindings.1.slave a
        bindings.1.channel 1
        bindings.2.slave b
        bindings.2.channel 0
        bindings.3.slave b
        bindings.3.channel 1
}
The radio app (op25) plays to alsa device "mout" and Darkice (streaming client) received it's input from alsa device "loop".
 
Last edited:

northernsummit

Member
Feed Provider
Joined
Jul 10, 2009
Messages
289
Location
Macedonia, Ohio
My broadcastify setup is op25->ALSA->DarkIce. My alsa config has both a loopback device and tee defined so that I can route the audio to darkice at the same time as sending it to the speakers for local monitoring.

I need to read up on this when I have a clearer mind. I kept struggling with ALSA and Pulse and was trying to do something like this. Stumbled upon liquidsoap and ran with it. I've yet to find a way to efficiently update the alpha tags with darkice, and with liquidsoap it appears they'r slightly out of sync (occur too early and end early)
 

boatbod

Member
Joined
Mar 3, 2007
Messages
3,316
Location
Talbot Co, MD
I need to read up on this when I have a clearer mind. I kept struggling with ALSA and Pulse and was trying to do something like this. Stumbled upon liquidsoap and ran with it. I've yet to find a way to efficiently update the alpha tags with darkice, and with liquidsoap it appears they'r slightly out of sync (occur too early and end early)

Lack of tag update is the one main limitation with darkice. I was trying to figure out a way to make it work, but it would require enough changes to the op25 udp audio interface to make it incompatible with the nc | aplay method of decoding and pulling it out separately would lead to exactly the synchronization problems you described.
 
Last edited:

vabiro

Member
Premium Subscriber
Joined
Jan 6, 2004
Messages
271
Location
Toronto, Ontario
Has anyone modified the liquidsoap script that would allow multiple mount points?

As it is currently configured, all the streams are presented over the same mount point consecutively, so on a busy system you could have considerable delay between the transmission, clearing of the cue, and the broadcast of the recording. To get around this, I set-up separate mount points for each.

To do this I am using my own icecast server with a mount point for each TGID. I am wondering if the silence injected in the liquidsoap script used here would be a solution to keeping the stream up, rather than dieing, or looping through the file.

Since the silence needs to be fed continuously to all the mountpoints in the absence of a new wav file I can't feed the appropriate TGID as part of the Streamthis to liquidsoap request.

I have hunted through several tutorials and can't seem to find anything that doesn't involve sending the same stream to multiple servers. I want to send different streams to the same localhost server with different mount points.

Any suggestions?

Thanks in advance.
 

northernsummit

Member
Feed Provider
Joined
Jul 10, 2009
Messages
289
Location
Macedonia, Ohio
I have hunted through several tutorials and can't seem to find anything that doesn't involve sending the same stream to multiple servers. I want to send different streams to the same localhost server with different mount points.

Any suggestions?

Thanks in advance.

Multiple instances of liquidsoap and the inject script. If you look at Cuyahoga County, Ohio there are 5 Cleveland Police Districts (in addition to the "everything" feed that predated those). The idea of the isolated districts is you can listen to JUST that district and not miss 10-15 minutes of transmissions that took place because the citywide feed got stuck on a 3 minute discussion about a license plate.

Anyway - each of those individual districts is a separate instance of streamthis and the liquidsoap script, run in a unique directory. Arguably the 'streamthis' script could be made smart enough to be one script and just run its inject commands to different file sockets.
 

10-75

Member
Feed Provider
Joined
Jan 20, 2004
Messages
97
Location
New York
Thanks for posting this. I, for the most part, have the trunk recorder running but I ran into trouble setting up the Trunk Player and I have given up for now. Cool setup tho...
 

mattnelson26

Member
Feed Provider
Joined
Mar 20, 2007
Messages
388
Location
Hopkins, Minnesota
i have trunk recorder running great. im a newbie to the Linux side and am having trouble getting trunk player working. was looking to see if anyone could help me out.
 

mattnelson26

Member
Feed Provider
Joined
Mar 20, 2007
Messages
388
Location
Hopkins, Minnesota
here is a screen shot of the folder and the ./manage.py runworker saying files not found when clearly its in the folder. i need some help figuring out what the problem is
 

Attachments

  • image.jpg
    image.jpg
    84.1 KB · Views: 469

Dygear

Member
Joined
Nov 18, 2010
Messages
74
Location
Levittown, NY
I'm currently working on the streaming front end for Trunk-Recorder to improve the over all interface that comes stock with it. audioplayer.php. I've actually got a newer version of this running on my website, but I'm trying to make everything work well together. I'm looking at a much more wholistic solution from the config file, to the script it uses to upload information, to the web socket server that's now built into Trunk-Recorder for getting new information to the web front end. Ideally you'd only need to run `php -S 0.0.0.0:8080` on your computer, and get a web working player immediately.
 

mattnelson26

Member
Feed Provider
Joined
Mar 20, 2007
Messages
388
Location
Hopkins, Minnesota
I'm currently working on the streaming front end for Trunk-Recorder to improve the over all interface that comes stock with it. audioplayer.php. I've actually got a newer version of this running on my website, but I'm trying to make everything work well together. I'm looking at a much more wholistic solution from the config file, to the script it uses to upload information, to the web socket server that's now built into Trunk-Recorder for getting new information to the web front end. Ideally you'd only need to run `php -S 0.0.0.0:8080` on your computer, and get a web working player immediately.

wow sounds cool. when its ready to go let me know ill test it out..

thanks for the hard work
 

scan18

Member
Premium Subscriber
Joined
Dec 23, 2004
Messages
289
Location
Honoka'a, HI



I currently have Trunk Recorder and Trunk Player working together and it’s been tremendous. However, I’m interested in exploring the audio player/web socket setup directly within Trunk Recorder.

Sorry for probably a dumb question, is there an example or something someone can point me to that details how to configure the web socket config and the audioplayer.php file?
 

Dygear

Member
Joined
Nov 18, 2010
Messages
74
Location
Levittown, NY
I don't actually have a real answer for you right now. The main design goal for the audioplayer.php bundled within Trunk-Recorder for me, was to have a stand alone page that was written in PHP that allows you to listen to your audio archive without needing anything else. Simply running
Code:
php -S 0.0.0.0:8080
will start a PHP server that you can then use to listen to your audio archive within a nice interface.

The audio player -v2 that's also there, wasn't written by me. My plan for that page is to make it the web socket version of what you see before you now. That will require a little more configuration to get ready for prime time. I'm actually going to implement a config.json parser that mirrors the one found in the C++ code that they can maintain lock step with each other. From there it's a matter of using the WebSocket interface that robotistic has made, but currently isn't very well documented. I'll start poking him for more details and glean what I can from the C++ code.

Hopefully by the end of the month, I'll have something that's ready to be merged into the main branch.
 
Status
Not open for further replies.
Top