OP25 VirtualBox Project - Run OP25 on Windows 7

[KA1RBI]

trace exactly what happens when Phase 1 receives a DUID3 or DUID15, both of which represent "end of transmission".

afaik, 3's and 15's are simply bounced to python-land, via a call to process_duid(), where they are handled by update_state()

Max

 

[krbvroc1]

So yes it seems I did miss something - there are actually two places that the Phase 1 code will output to UDP - one is in p25p1_fdma::rx_sym() and the other in p25p1_voice_decode::rxframe(). More importantly, the latter sends 320 byte frames which handily explains what krbvroc1 was seeing.

Let me step back for the bigger picture here. Can you explain what is the purpose of the 'drain' call? If I understand correctly this is to notify the sound driver that there are no more samples at this time and the 'drain' call blocks until the sample period is completed. The sound driver then goes into some sort of suspended mode.

If the drain is not called the driver expects to be continuously fed samples. Since that doesn't happen during idle scanner time, the start of the next audio transmission causes the underrun (EPIPE) from the alsa driver.

Does that sound about right?

So that is why we want some sort of end of transmission detection to trigger this drain call?

 

[boatbod]

Let me step back for the bigger picture here. Can you explain what is the purpose of the 'drain' call? If I understand correctly this is to notify the sound driver that there are no more samples at this time and the 'drain' call blocks until the sample period is completed. The sound driver then goes into some sort of suspended mode.

If the drain is not called the driver expects to be continuously fed samples. Since that doesn't happen during idle scanner time, the start of the next audio transmission causes the underrun (EPIPE) from the alsa driver.

Does that sound about right?

So that is why we want some sort of end of transmission detection to trigger this drain call?

Yes, you have a good understanding of the situation. Once an application begins feeding frames to an ALSA pcm stream, the kernel expects to keep receiving them in a timely manner until told to stop, at which point it finishes playing and returns to the "prepare" state waiting for more data. If the application cannot keep up, or simply ceases sending data (such as when a transmission ends) the kernel will raise an XRUN error and the pcm stream will return EPIPE for every write() operation and be unusable until recovered or prepared.

The challenge to determining end of transmission in a way that can invoke snd_pcm_drain(), is that it must be done in a transparent manner because not everyone uses sockaudio.py as the audio player. I chose to write some null (silent) samples which would be unlikely to occur in normal usage and detect them during the playback handling. The key question is whether they are are being sent at the appropriate times for Phase 1 audio.

Aside from detecting end of transmission and cleanly handling pcm data, some underruns are inevitable. I see them very rarely on my i5 & i7 laptops, but my Pi3 spits 'em out every 2-4 seconds because I'm demanding so much of it's small cpu. Running op25 with local sockaudio and darkice all on one processor is a handful. Offloading sockaudio and darkice to another machine definitely makes things much better and the underruns occur much less frequently.

Note: Max and I are working on refining the drain triggering. I've got some updated code that I may ask you to try out later this evening if you're willing.

 

[krbvroc1]

Note: Max and I are working on refining the drain triggering. I've got some updated code that I may ask you to try out later this evening if you're willing.

Sure I can try it out. Just so you know, I put some prints in the code (for P1)

Yes, there are 0x3 and 0xf at the end of transmissions. However your write of 288 zeros is never executed.

When d_do_imbe==1 and d_do_audio_output==1, d_do_output will always be 0, so your write is never executed. If you look at the code, this makes sense because it is the write of 320 bytes that occurs in p1voice_decode.rxframe() triggered by d_do_audio_output==1

I think the fix is as follows. I tested this and it worked for me.

https://gist.github.com/krbvroc1/ef069afa387e070ccb75aee883796c32

 

[boatbod]

Sure I can try it out. Just so you know, I put some prints in the code (for P1)

Yes, there are 0x3 and 0xf at the end of transmissions. However your write of 288 zeros is never executed.

When d_do_imbe==1 and d_do_audio_output==1, d_do_output will always be 0, so your write is never executed. If you look at the code, this makes sense because it is the write of 320 bytes that occurs in p1voice_decode.rxframe() triggered by d_do_audio_output==1

I think the fix is as follows. I tested this and it worked for me.

Looks like you found the bug, thank you!

Since d_do_output is never set when sending audio over udp and there would be no point sending the zero bytes for non-audio applications I have completely dropped it from the test condition.

Please try the following versions;
- modified drain flagging methodology (2 bytes vs 288/320) per Max's suggestion
- incorporated your bugfix
- streamlined data shuffling within sockaudio.py

p25p1_fdma.cc
p25p2_tdma.cc
sockaudio.py

 

[KA1RBI]

Please try the following versions;
- modified drain flagging methodology (2 bytes vs 288/320) per Max's suggestion
- incorporated your bugfix
- streamlined data shuffling within sockaudio.py

nice!!

can't wait to try these out, I think we're close to ironing out all issues. I'm looking fwd to applying the patch here and running in on a few P1 systems! I like that you got rid of the entire "while (i < dlen)" loop. Not to nitpick or anything, but I think as long as what you're looking at is the actual size of the received UDP packet, a test for len==2 is sufficient to trigger a drain, i.e., no need to test the values of the two bytes; given that we know in advance that only two sizes would ever be expected (2 or 320)...

73

Max

 

[krbvroc1]

Looks like you found the bug, thank you!

You are very welcome.

Please try the following versions;
- modified drain flagging methodology (2 bytes vs 288/320) per Max's suggestion
- incorporated your bugfix
- streamlined data shuffling within sockaudio.py

I am running them now. So far no PCM underruns on either a P1 (tested for a few hours) or P2 system (just a few minutes of testing).

I am trying to understand the code path, but it is quite confusing. I started down the path of muting encrypted channels. The problem so far (for P1 at least) is that things really should be 'state' based. HDU->LDU1->LDU2 ... TDU
The decode routine simply checks for certain duid id's with no maintained state. For muting I need state to keep track of beginning to end of transmission. Maybe it is in there somewhere, but haven't figured out the flow yet. Whether the transmissions is encrypted is embedded in both the HDU and LDU2. But you want to mute the alternating LDU1 LDU2 which is why state is needed. LDU1 does not contain the info but it is implied from the HDU/LDU2 of the same transmission stream.

Another idea I have would be to output a stereo audio channel. One channel would be the current audio. The second would actually be binary metadata such as current NAC/SYSID/tsig, etc. That way you use the one channel for showing 'tags' on a data feed like shoutcast yet still maintain compatibility with 'aplay' if you isolate one channel. Not sure if there are other ways to do this or if the feature is already in there somehow.

 

[boatbod]

nice!!

can't wait to try these out, I think we're close to ironing out all issues. I'm looking fwd to applying the patch here and running in on a few P1 systems! I like that you got rid of the entire "while (i < dlen)" loop. Not to nitpick or anything, but I think as long as what you're looking at is the actual size of the received UDP packet, a test for len==2 is sufficient to trigger a drain, i.e., no need to test the values of the two bytes; given that we know in advance that only two sizes would ever be expected (2 or 320)...

73

Max

Let me know if you need me to shoot the relevant files over via email now that we know they work.

The loop within sockaudio was originally used to turn the 1 channel (mono) samples into 2 channel because my earlier ALSA buffering scheme let me set a much bigger buffer on 2 channel than mono. Once I figured out how to buffer more simple and effectively I was able to get rid of that unnecessary duplication and drop back to single channel.

I considered a simple 2 byte length test but wanted to leave the option for sending different numeric flags at some point in the future. In an ideal world I'd like to figure a way to embed the tgid/tag name so that it can be output to broadcastify. Haven't figure out quite how to do that yet, so it's a back-burner issue for now.

I am running them now. So far no PCM underruns on either a P1 (tested for a few hours) or P2 system (just a few minutes of testing).

Great news! Thanks for checking it out.

I am trying to understand the code path, but it is quite confusing. I started down the path of muting encrypted channels. The problem so far (for P1 at least) is that things really should be 'state' based. HDU->LDU1->LDU2 ... TDU
The decode routine simply checks for certain duid id's with no maintained state. For muting I need state to keep track of beginning to end of transmission. Maybe it is in there somewhere, but haven't figured out the flow yet. Whether the transmissions is encrypted is embedded in both the HDU and LDU2. But you want to mute the alternating LDU1 LDU2 which is why state is needed. LDU1 does not contain the info but it is implied from the HDU/LDU2 of the same transmission stream.

Another idea I have would be to output a stereo audio channel. One channel would be the current audio. The second would actually be binary metadata such as current NAC/SYSID/tsig, etc. That way you use the one channel for showing 'tags' on a data feed like shoutcast yet still maintain compatibility with 'aplay' if you isolate one channel. Not sure if there are other ways to do this or if the feature is already in there somehow.

I had a notion that it might be a little complicated. One possibility might be to store an "encrypted flag" as a class member, but the difficulty is making sure it always gets properly reset when a non-encrypted HDU/LDU comes along.

The one thing I do have a lot of here is encrypted traffic. MD DNR encrypts everything and they are quite active on the radio. When my new rtl dongle arrives I'll give it some thought if you haven't cracked the nut already.

 

[krbvroc1]

I considered a simple 2 byte length test but wanted to leave the option for sending different numeric flags at some point in the future. In an ideal world I'd like to figure a way to embed the tgid/tag name so that it can be output to broadcastify. Haven't figure out quite how to do that yet, so it's a back-burner issue for now.

That is why I was thinking about using 1 channel of stereo to encode this info so it is present during the entire transmission. Remember your 2 byte 'sentinal' marks the end of the transmission so I am thinking it might not be too useful to indicate the tsig/tag at that point since the transmission is already completed (unless it was for archive purposes rather than real-time display).

 

[PiccoIntegra]

This is how I have dealt with encrypted talk groups. This is in no way an elegant solution, but it helps avoid the awful noise.

krbvroc1 has the right idea, but the code isn't there to provide LDU data to the trunking logic. There needs to be a lcn/frequency state machine to keep track of talk group status. This presents a new problem when a voice grant is issued and it switches frequency. You then lose the control channel and your state machine is now out of sync. So this is no better than the solution I provided above.

The only way to avoid encrypted TGs altogether is to have a dedicated control channel. Doing so, you can eliminate late entry voice grants (TSBK 0x02) completely.

Two ways you can do this;

1. The system fits within the usable bandwidth of your SDR device
2. Use more than one SDR device

 

[boatbod]

Conceivably one might also dynamically blacklist upon determination of an encrypted tg. Obviously it would take down the whole tgid but how much mixed traffic really exists on a single tg?

 

[krbvroc1]

This is how I have dealt with encrypted talk groups. This is in no way an elegant solution, but it helps avoid the awful noise.

krbvroc1 has the right idea, but the code isn't there to provide LDU data to the trunking logic.

I am still finding trying to find my bearings in the code. What is this TSBK value and where does it come from? I don't think that is a P25 term so it must be some sort of op25 term. I see it is derived from some message arriving on a queue. Is that sent from the 'c' code?

There is also some data path between the 'c' code and the curses terminal, but I haven't figured out that path yet.

 

[boatbod]

I am still finding trying to find my bearings in the code. What is this TSBK value and where does it come from? I don't think that is a P25 term so it must be some sort of op25 term. I see it is derived from some message arriving on a queue. Is that sent from the 'c' code?

There is also some data path between the 'c' code and the curses terminal, but I haven't figured out that path yet.

p25p1_fdma sends messages back up the chain. Ultimately they funnel through the message handler in rx.py and get sent over to trunking.py or terminal.py etc.

Frankly I've never bothered much with tsbks - it's more the message content as it arrives in trunking or elsewhere that is usually important to me.

 

[krbvroc1]

p25p1_fdma sends messages back up the chain. Ultimately they funnel through the message handler in rx.py and get sent over to trunking.py or terminal.py etc.

Frankly I've never bothered much with tsbks - it's more the message content as it arrives in trunking or elsewhere that is usually important to me.

Ahh okay. I see. It actually is a P25 term. TSBK are the trunk signaling blocks which are transmitted in packets with DUID of 0x7 (TSDU) or DUID (PDU) 0xc.

Based on PiccoIntegra comment, I was also confused as to how the code would even know about a late entry voice grant if the TSBK has to do with signaling. If you aren't tuned to the control channel how would you know? It appears the code cheats and when it encounters other DUID cases, a 'fake' TSBK message is generated to signal that condition and sent to the trunking code. So that is how it knows - it gets a notitication for DUID 3,5,10,15 also, even though those are not true trunking related messages.

I merged PiccoIntegra changes into my local trunking.py (Thanks!) and it really does a good job of skipping encrypted channels. And that code actually does add it to a sort of dynamic blacklist. I would like to modify the curses display to add some sort of indicator like an 'X' next to the encrypted tsigs. And it would be nice to add a keyboard command that dumps the current list of encrypted tsigs encounterd to a file so you can manually blacklist them in your trunk.tsv file.

 

[boatbod]

deleted

 

[MattSR]

Since this is the place for all info these days, can we please make this a sticky?

 

[boatbod]

Since this is the place for all info these days, can we please make this a sticky?

Its an awfully long thread and a lot has changed since the beginning. Probably be better if someone wrote a concise "howto" and made that sticky.

 

[kc0nzk]

Hello all,

Bear with me, I'm brand new to both Linux and SDR. I'm attempting to monitor an APCO 25 Ph II network, and am having two issues when running rx.py:

----------------------------------------------------------------------------------------------------------

Issue #1. I can trunk with the "-t" argument, but not with the "-T trunk.tsv" argument.
Here is the command where trunking appears to work (i.e.-information populates and tgids with times appear):
./rx.py --args 'rtl' --gains 'lna:39' -f 772.28125e6 -S 2500000 -t -w -2 2> stderr$

And here is the command referencing the trunk file that doesn't initiate trunking (i.e. - no information populates):
./rx.py --args 'rtl' --gains 'lna:39' -f 772.28125e6 -S 2500000 -T trunkhsv.tsv -w -2 2> stderr$

See image "T.png" below for output
Also, see image "trunkhsv.jpg" below for a screenshot of my trunkhsv.tsv file.

--------------------------------------------------------------------------------------------------------------

Issue #2. I am unable to receive audio output.
Here is the command I use in a new Terminal Window:
nc -kluvw 1 127.0.0.1 23456 | aplay -c1 -f S16_LE -r 8000

And the output reads: "Listening on [127.0.0.1] (family 0, port 23456)"

I have used the Ubuntu OS "Test Sound" feature to verify that speakers are not muted, etc.
I'm still trying to study up on ALSA vs. pulse audio but the discussion has been over my head.

---------------------------------------------------------------------------------------------------------------
Background Info:

SDR Device: RTL-SDR dongle

OS: Ubuntu 14.04 dual boot beside Windows 7

Computer: Core2Duo P8600 @ 2.4GHz w/ 2GB RAM. When trunking with -t CPU use is only ~50%

Installation Method: I followed Matthew Lindstrom's YouTube video for the most part, except I tried to create the trunk.tsv and talkgroup tsv files myself using the SignalScope OP25 page instead of downloading from RadioReference. Since scope.py is now rx.py, I followed the README instead of the YouTube video to create the ./rx.py arguments.
https://www.youtube.com/watch?v=s2iJMtPJu-I&t=793s

Updates: I have downloaded boatbod's three new files from post #1065 and overwritten the old files:
p25_frame_assembler_impl.cc
p25p2_tdma.h
p25p2_tdma.cc

Radio System: Alabama First Responder Network, APCO 25 Ph2, Huntsville Simulcast.
It has a large number of frequencies and ~88MHz difference between min and max frequencies.
https://www.radioreference.com/apps/db/?siteId=19859

Any help would be greatly appreciated!! Thanks in advance!

73

Drew

 

[biloxicoast]

Alert tone

Tucson police broadcast a beeping tone in the background of transmission during emergencies. Regular scanners play this but op25 won't. Any
Ideas

 

[slicerwizard]

The system is likely sending tone frames. OP25 would have to decode those frames and synthesize the tones. Don't know if that functionality is in OP25.