OP25 VirtualBox Project - Run OP25 on Windows 7

[boatbod]

Using nc | aplay setup at moment, but will definitely try your changes later.
Anyway, verbosity level 10, stderr output and some source code reading helped me to pin down the problem.
From what I see, phase1 and phase2 decoders each using a separate UDP connection, so first decoded audio grabs the nc UDP port, making it unable to receive another phase audio packets. So you either hear every p1 or every p2 conversation, on a first-come first-serve basis.. Perhaps sockaudio.py is handling connections differently, will take a look.

You are correct that p25p1_fdma and p25p2_tdma separately open their own UDP sockets, but the protocol is by it's nature connectionless so there shouldn't be any downsides of doing that since the receiving app will neither know nor care where the data originated.

boatbod, I've downloaded and built your changes, had to manually merge your rx.py code with some latest git changes though.

Both phase1 and phase2 audio works now, no more UDP problems, so sockaudio.py its definitely more robust and comfortable way to listen. Thank you for your work, good job! Hope your code get pushed to official git repository soon.

The primary difference in logical handling between nc | aplay and sockaudio is that the latter knows when to flush the pcm stream at the end of a transmission. In itself this shouldn't really change the handling of intermingled Ph1 and Ph2 transmissions, but the new code you are running also has subtle changes to the trunking logic to keep the radio tuned to a given phase 2 audio frequency during MAC_HANGTIME until the repeater notifies MAC_END_PTT. This allows more stable listening in cases where back to back chatter occurs on a given TGID on the same voice channel. Previously you'd see numerous re-tunes to the control channel and back again every time a subscriber released the PTT, even if momentarily prior to reacquiring. Practically it seems to add about 1-2 sec silence at the end of a conversation (depending on repeater configuration) but the overall listening experience is better. Unfortunately there is no equivalent Ph1 capability, so that re-tunes immediately on receipt of DUID3 or DUID15.

 

[whatradio]

You are correct that p25p1_fdma and p25p2_tdma separately open their own UDP sockets, but the protocol is by it's nature connectionless so there shouldn't be any downsides of doing that since the receiving app will neither know nor care where the data originated.

Agree, in theory UDP is supposed to be stateless fire-and-forget thing..

Launched Wireshark to check whats going on loopback network interface during "nc | aplay"

Phase1 packet, gets send by op25, received by nc and played by aplay just fine.
User Datagram Protocol, Src Port: 56366, Dst Port: 23456
Source Port: 56366
Destination Port: 23456
Length: 328
Checksum: 0xff5b [unverified]
[Checksum Status: Unverified]
[Stream index: 1]

Phase2 packet, sent but instantly followed by "Port unreachable" ICMP reply.
User Datagram Protocol, Src Port: 51252, Dst Port: 23456
Source Port: 51252
Destination Port: 23456
Length: 328
Checksum: 0xff5b [unverified]
[Checksum Status: Unverified]
[Stream index: 0]
Internet Control Message Protocol
Type: 3 (Destination unreachable)
Code: 3 (Port unreachable)
Checksum: 0xd883 [correct]
[Checksum Status: Good]
Unused: 00000000
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
User Datagram Protocol, Src Port: 51252, Dst Port: 23456

Obviosly it can be other way around with P1/P2, depending on what comes first during OP25 session.

 

[boatbod]

But you say the same is not true with sockaudio? Curious. My code (sockaudio) only opens the inbound udp socket once and keeps the handle going for the whole session. The nc | aplay appears to be opening/closing/reopening possibly based on aplay receiving an xrun.

whatradio: which options are you passing to nc? When I used to run it, this was my start script:

nc -kluv 127.0.0.1 23456 | aplay --buffer-size=1024 -c 1 -f S16_LE -r 8000

 

[krbvroc1]

For the 'max' branch (rx.py) I have written a pure python module (sockaudio.py) that integrates into op25 to capture udp audio frames and plays them through an ALSA device so you no longer need to run the separate "nc" and "aplay" tools.

Probably the easiest way to give it a try is to download the changed versions of the following files from my google drive. https://drive.google.com/open?id=0B-9lC78gYedOMDFrTGx4Vkh3em8

I just downloaded this from the above google drive and appreciate the better audio. (Not running on a virtual machine however - using a dedicated GB-BXBT-2807 (Intel Celeron 2807) that I had laying around.

In the longer run I really want some sort of sockaudio like yours that doesn't glitch, but can send the audio to another machine on the network so it can be listened to on other computers, etc (some sort of audio server).

 

[boatbod]

I just downloaded this from the above google drive and appreciate the better audio. (Not running on a virtual machine however - using a dedicated GB-BXBT-2807 (Intel Celeron 2807) that I had laying around.

In the longer run I really want some sort of sockaudio like yours that doesn't glitch, but can send the audio to another machine on the network so it can be listened to on other computers, etc (some sort of audio server).

To get the least glitchy audio you need to also pull in the changes to the p25p1_fdma and p25p2_tdma library code and the associated python updates to rx.py and trunking.py if you haven't already. Depending on the horsepower of the machine being used, you might also need to tweak the buffer size in sockaudio.py. Some minimal underruns are ok, if you get lots of them then the buffers are too small.

 

[krbvroc1]

To get the least glitchy audio you need to also pull in the changes to the p25p1_fdma and p25p2_tdma library code and the associated python updates to rx.py and trunking.py if you haven't already. Depending on the horsepower of the machine being used, you might also need to tweak the buffer size in sockaudio.py. Some minimal underruns are ok, if you get lots of them then the buffers are too small.

It appears to be working well. Yes, I am using your changes to the above libraries. I didn't get any glitches in the library. I found your other thread that provided an audio.py receiver to send the audio to another computer. I just ran that and that is working also. With that I seem to get a PCM underrun between each channel change. I probably want to modify that socketaudio to send the audio to an icecast server or something like that instead.

We are Phase 1 system here. There is the Maryland FIRST, but not much activity on that. But there is a Phase 2 across the border in Virginia.

Since it sounds like you dove into some of the internals. Does the channel 'algorithm' get flagged or made available at all? I ask because I would like to modify something to mute encrypted channels. Right now I am blacklisting them, but it would nice if it just skipped them automatically. Occasionally some channels popup that I haven't seen before.

Finally, I saw you mention 'mixed' phase 1/phase 2 systems. Does that mean if you pass op25 the '-2' it still handle phase 1? I was thinking it was mutually exclusive but did not know.

 

[boatbod]

It appears to be working well. Yes, I am using your changes to the above libraries. I didn't get any glitches in the library. I found your other thread that provided an audio.py receiver to send the audio to another computer. I just ran that and that is working also. With that I seem to get a PCM underrun between each channel change. I probably want to modify that socketaudio to send the audio to an icecast server or something like that instead.

We are Phase 1 system here. There is the Maryland FIRST, but not much activity on that. But there is a Phase 2 across the border in Virginia.

Since it sounds like you dove into some of the internals. Does the channel 'algorithm' get flagged or made available at all? I ask because I would like to modify something to mute encrypted channels. Right now I am blacklisting them, but it would nice if it just skipped them automatically. Occasionally some channels popup that I haven't seen before.

Finally, I saw you mention 'mixed' phase 1/phase 2 systems. Does that mean if you pass op25 the '-2' it still handle phase 1? I was thinking it was mutually exclusive but did not know.

I monitor FiRST almost exclusively - it's the only P25 system we have where I live on the Eastern Shore. Over here everything is Phase 2, except on rare days when a Ph1 radio manages to affiliate. Anyway, to answer your question, there is no harm to enabling "-2" as it automatically figures out if something is phase 1 or phase 2 and behaves accordingly.

I've not tried filtering out encryption. Theoretically that would be possible in Phase 2 because there are flags that could be observed. Unfortunately I don't have a full copy of the TIA102 spec so I'm not sure how Phase 1 handles it.

Do you receive underruns at the end of every tgid change regardless of Ph1 / Ph2 or is it specific to one variant? I have code in p25p1_fdma and p25p2_tdma to flag and request a buffer flush (a full frame of 0's), but I can't test the phase 1 variant.

I'd be extremely interested to see if you can feed directly to icecast and bypass alsa. The way I've approached this task so far is easy enough though... I configured the alsa aloop kernel module and set up some virtual loopback devices. If you like I can send you the asound.conf that gets the job done.

 

[krbvroc1]

I've not tried filtering out encryption. Theoretically that would be possible in Phase 2 because there are flags that could be observed. Unfortunately I don't have a full copy of the TIA102 spec so I'm not sure how Phase 1 handles it.

Here is what I found so far...Not sure if that relates to Phase 1 / Phase 2 or both.
http://forums.radioreference.com/in...etecting-encryption-protocol.html#post1690125

Do you receive underruns at the end of every tgid change regardless of Ph1 / Ph2 or is it specific to one variant? I have code in p25p1_fdma and p25p2_tdma to flag and request a buffer flush (a full frame of 0's), but I can't test the phase 1 variant.

It seems to be the same - a single PCM underrun regardless of Phase 1 or 2 each time a tranmissions stops. It doesn't seem to cause any issues with the audio quality.

I'd be extremely interested to see if you can feed directly to icecast and bypass alsa.

I will let you know if I come up with something. From a quick search, I believe there are some python icecast clients which I might try - it might need conversion from WAV to something compressed.

 

[boatbod]

It seems to be the same - a single PCM underrun regardless of Phase 1 or 2 each time a tranmissions stops. It doesn't seem to cause any issues with the audio quality.

That is the behavior I would expect if p25p1_fdma.cc and p25p2_tdma.cc have not been patched to send a full frame of zero bytes at the end of a transmission.

What does line 184 of p25p2_tdma.cc and line 345 of p25p1_fdma.cc say? They should have exactly the same comment if you have the correctly modified code. ("//write a block of...")

Relative to detecting & filtering encryption, yes it should be possible, but no the current code does not appear to do it at the moment. From a cursory inspection I don't think it's going to be particularly easy as the phase 1 handler does not do any decoding of LDU2 except when higher levels of logging are selected. In phase 2 you basically have to look in the MAC_PTT pdu, so again the info is there, but not presently pulled out.

I can take a more in depth look, but I am lacking a proper development environment at present. Like an idiot I used my spare RTL stick to host a second Broadcastify feed, so until the new one arrives in my mailbox next week, I don't have anything I can play with. Doh!

 

[krbvroc1]

That is the behavior I would expect if p25p1_fdma.cc and p25p2_tdma.cc have not been patched to send a full frame of zero bytes at the end of a transmission.

What does line 184 of p25p2_tdma.cc and line 345 of p25p1_fdma.cc say? They should have exactly the same comment if you have the correctly modified code. ("//write a block of...")

Line 184 is a comment.
// write a block of zero audio samples at end of voice to trigger pcm drain
Line 345 is the same comment.

I am certain I even did a make clean before a make install.I am running the rx.py out of my local
op25/op25/gr-op25_repeater/apps/ dir. That is correct right?

 

[krbvroc1]

I took a look at the code and am confused about something. You have a comment that data received for P1 is 288 bytes and P2 is 320 bytes. I see some comments in p25p1_fdma.cc about 288 bytes. However in p25p1_voice_decode.cc (imbe) the FRAME size is 320. I do not understand the data flow yet.

To verify, a wireshark capture on my P1 system UDP packet is 320, not 288. So if during silence you are inserting 288 rather than 320 I can see an underrun.I just tried a P2 system and didn't see underruns, but I know I did yesterday. Doesn't a P2 system allow a P1 device for interoperability. I am not sure of the code flow in that scenario but if I saw P1 on a P2 yesterday maybe that is why I saw underruns.

 

[boatbod]

I took a look at the code and am confused about something. You have a comment that data received for P1 is 288 bytes and P2 is 320 bytes. I see some comments in p25p1_fdma.cc about 288 bytes. However in p25p1_voice_decode.cc (imbe) the FRAME size is 320. I do not understand the data flow yet.

To verify, a wireshark capture on my P1 system UDP packet is 320, not 288. So if during silence you are inserting 288 rather than 320 I can see an underrun.I just tried a P2 system and didn't see underruns, but I know I did yesterday. Doesn't a P2 system allow a P1 device for interoperability. I am not sure of the code flow in that scenario but if I saw P1 on a P2 yesterday maybe that is why I saw underruns.

Phase 1 and Phase 2 send different sized audio chunks out over UDP. You can see this on line 337 of p25p1_fdma.cc as compared to line 187 of p25p2_tdma.cc. Both protocols batch up a full superframe worth of audio data before sending, so I suspect the difference is due to the different coding schemes being used by IBME/AMBE codecs.

To see if the drain is working, add an additional line

sys.stderr.write("draining\n")

somewhere around line 223 or 276 of sockaudio.py

 

[krbvroc1]

Phase 1 and Phase 2 send different sized audio chunks out over UDP. You can see this on line 337 of p25p1_fdma.cc as compared to line 187 of p25p2_tdma.cc. Both protocols batch up a full superframe worth of audio data before sending, so I suspect the difference is due to the different coding schemes being used by IBME/AMBE codecs.

But how can that be if for both P1 and P2 I am getting the same size UDP packets via wireshark? Where does the p25p1_voice_decode.cc come into play? Look at line 105 of that file, it too does a UDP send of size 'FRAME' which is 320, not 288.

Also looking at p25p1_fdma.cc the write of 288 bytes you refer to is surrounded by an if (d_do_output).
There seems to be a d_do_audio_output which is different than d_do_output. The former outputs 320, the later 288. Your send of 288 is done for d_do_audio_output - that should be 320.

To see if the drain is working, add an additional line

sys.stderr.write("draining\n")

somewhere around line 223 or 276 of sockaudio.py

Yes the 'draining' prints to stderr, but see my description above -- looks like a bug. I bet if I change your write of zeros to 320 the pcm underruns go away.

 

[krbvroc1]

I bet if I change your write of zeros to 320 the pcm underruns go away.

It didn't fix it. I am still confused why I am getting UDP packets with a payload size of 320 bytes though.

Also, it does not always prints 'draining'. When it does print 'draining' I don't see the PCM underrun, but for the majority of tsig changes when there is a PCM underrun printed, there is no 'draining' printed.

 

[boatbod]

It didn't fix it. I am still confused why I am getting UDP packets with a payload size of 320 bytes though.

Also, it does not always prints 'draining'. When it does print 'draining' I don't see the PCM underrun, but for the majority of tsig changes when there is a PCM underrun printed, there is no 'draining' printed.

In Phase 2, a drain is triggered based on the results of process_mac_pdu(). At present it looks like I'm only calling for a drain when rc=3, which maps primarily to MAC_HANGTIME. If your Phase 2 system (and it has to be phase 2 since you are getting 320 byte chunks) isn't sending MAC_HANGTIME prior to sending MAC_END_PTT then there will be no drain.

Try changing p25p2_tdma.cc line 185 as follows:

from:
	if ((rc == 3) && (write_sock > 0)) {
to:
	if (((rc == 3) || (rc == 15)) && (write_sock > 0)) {

Don't forget to perform a "sudo make install" from the build directory after changing the file.

If that still doesn't fix it then I would conclude your op25 is changing tgid's without waiting for the end of a transmission.

 

[krbvroc1]

I'll give that a shot. This is the system I am connected to. Not sure how/why that would be Phase 2.
https://www.radioreference.com/apps/db/?sid=7321

I don't use the '-2' option and while I am new to SDR, I was decoding/listening to this using DSDPlus under Windows which I do not believe supports P2. So my gut tells me there is another explanation here.

 

[boatbod]

I'll give that a shot. This is the system I am connected to. Not sure how/why that would be Phase 2.
https://www.radioreference.com/apps/db/?sid=7321

I don't use the '-2' option and while I am new to SDR, I was decoding/listening to this using DSDPlus under Windows which I do not believe supports P2. So my gut tells me there is another explanation here.

I don't have an explanation, but there are only two places in the code that output UDP and one of them (in p25p1_fdma.cc) is hard coded at 288 bytes and the other (in p25p2_tdma.cc) is hard coded at 320 bytes. If you are seeing 320 byte chunks, it *has* to be coming from the phase 2 code. In the rx.py terminal screen do you see any reference to "TMDA Slot 0" or "TDMA Slot 1"?

I've updated both of these files with additional drain logic to account for corner cases that might cause premature end of transmission.
- phase 2 MAC_END_PTT prior to MAC_HANGTIME
- rx_sym() timeout (loss of framing)
You'll need to rebuild and install as before.

You might also want to try this version of trunking.py as it logs every voice update with a timestamp, tgid, freq and slot. It may help to show if there is phase 2 activity taking place.

One last question: do you have multiple control channels or system defined in trunk.tsv?

 

[KA1RBI]

there are only two places in the code that output UDP and one of them (in p25p1_fdma.cc) is hard coded at 288 bytes and the other (in p25p2_tdma.cc)

These are the only two? Are you sure? ;-)

Max

 

[boatbod]

These are the only two? Are you sure? ;-)

Max

Yup, unless I've somehow missed some. The main "sendto" calls for sending voice data are in p25p1_fdma::rx_sym() and p25p2_tdma::handle_voice_frame(), with a few additional sendto()'s in places that never get called when voice decoding is enabled (wireshark legacy code perhaps?).

Elsewhere, there are a couple of sendto()'s in p25p1_voice_decode() and p25p1_voice_encode() but I don't think they are part of the active codebase for regular op25.

 

[boatbod]

Yup, unless I've somehow missed some. The main "sendto" calls for sending voice data are in p25p1_fdma::rx_sym() and p25p2_tdma::handle_voice_frame(), with a few additional sendto()'s in places that never get called when voice decoding is enabled (wireshark legacy code perhaps?).

Elsewhere, there are a couple of sendto()'s in p25p1_voice_decode() and p25p1_voice_encode() but I don't think they are part of the active codebase for regular op25.

So yes it seems I did miss something - there are actually two places that the Phase 1 code will output to UDP - one is in p25p1_fdma::rx_sym() and the other in p25p1_voice_decode::rxframe(). More importantly, the latter sends 320 byte frames which handily explains what krbvroc1 was seeing.

In absence of having a phase 1 system to test against, I am trying to figure out the logical codepath. To say it's a web of intrigue is something of an understatement! On the face of it, even though I had misunderstood where exactly the phase 1 udp was sourced, I don't think it affects the pcm drain logic.... but I can't be sure. Perhaps someone can sprinkle in a few temporary logging messages fprintf(stderr, "hello world\n"); to trace exactly what happens when Phase 1 receives a DUID3 or DUID15, both of which represent "end of transmission".