OP25 Alpha Tags for encrypted TGs

[jschmall]

Current setup is as follows, OP25 multi_rx (boatbod fork) feeding a liquidsoap instance in turn feeding Broadcastify and my personal Icecast server. Everything is working perfectly with the exception of Alpha Tags.

I have multi_rx set to ignore encrypted TGs and have whitelisted only the TG that I am interested in. However it still sends those encrypted Alpha Tags to liquidsoap briefely and I see that tag info come across in Icecast. I'm not sure about the Broadcastify instance since I can't quite get those tags to work (but that's a problem for another day.)

Is there any way to prevent those encrypted TG tags from coming across? Is it something to do with my whitelist? I was told before that whitelisting a singel (or multiple) TGs will, by default, blacklist all others but I'm not sure if thats correct.

 

[wgbecks]

It sounds like perhaps your whitelist may be associated with the Voice Channel section instead of the Trunking section.
Can you post your JSON file being sure to redact the BCFY mount points and passwords?

 

[jschmall]

It sounds like perhaps your whitelist may be associated with the Voice Channel section instead of the Trunking section.
Can you post your JSON file being sure to redact the BCFY mount points and passwords?

And that's what I thought so I added it to the Trunking section as well and it didn't make a difference.

{
    "channels": [
        {
            "name": "Redding P25 System",
            "device": "sdr0",
            "trunking_sysname": "Redding P25 System",
            "meta_stream_name": "stream_0",
            "demod_type": "cqpsk",
            "cqpsk_tracking": true,
            "tracking_threshold": 30,
            "tracking_feedback": 0.85,
            "destination": "udp://127.0.0.1:23450",
            "excess_bw": 0.2,
            "filter_type": "rc",
            "if_rate": 24000,
            "plot": "",
            "symbol_rate": 4800,
            "enable_analog": "off",
            "blacklist": "",
            "whitelist": "rpd_bcfy_wlist.tsv"
        }
    ],
    "devices": [
        {
            "args": "rtl=01",
            "gains": "LNA:30",
            "gain_mode": false,
            "name": "sdr0",
            "offset": 0,
            "ppm": 0.0,
            "rate": 1000000,
            "usable_bw_pct": 0.85,
            "tunable": true
        }
    ],
    "trunking": {
        "module": "tk_p25.py",
        "chans": [
            {
                "nac": "0xb71",
                "sysname": "Redding P25 System",
                "control_channel_list": "856.4375,857.4375,858.4375,859.4375",
                "whitelist": "rpd_bcfy_wlist.tsv",
                "blacklist": "",
                "tgid_tags_file": "rpd_tg.tsv",
                "rid_tags_file": "rpd_rid.tsv",
                "tdma_cc": false,
                "crypt_behavior": 2
            }
        ]
    },
    "metadata": {
        "module": "icemeta.py",
        "streams": [
            {
                "stream_name": "stream_0",
                "meta_format_idle": "[idle]",
                "meta_format_tgid": "[%TGID%]",
                "meta_format_tag":  "[%TGID%] %TAG%",
                "icecastServerAddress": "[REDACTED]",
                "icecastMountpoint": "[REDACTED]",
                "icecastMountExt": "",
                "icecastPass": "[REDACTED]",
                "delay": 0.0
            }
        ]
    },
    "audio": {
        "module": "sockaudio.py",
        "instances": [
            {
                "instance_name": "",
                "device_name": "default",
                "udp_port": 23456,
                "audio_gain": 1.0,
                "number_channels": 1
            }
        ]
    },
    "terminal": {
        "module": "terminal.py",
        "terminal_type": "",
        "terminal_type": "http:0.0.0.0:8585",
        "curses_plot_interval": 0.1,
        "http_plot_interval": 1.0,
        "http_plot_directory": "../www/images",
        "tuning_step_large": 1200,
        "tuning_step_small": 100
    }
}

 

[jschmall]

Bit of an update, I removed the "stream_0" from the channels config "meta_stream_name" and now I have no metadata at all, which I expected. I still have my icecast stream as that's being fed from the liquidsoap instance but I appear to have no tags at all.

 

[wgbecks]

Bit of an update, I removed the "stream_0" from the channels config "meta_stream_name" and now I have no metadata at all, which I expected. I still have my icecast stream as that's being fed from the liquidsoap instance but I appear to have no tags at all.

The value of the meta_stream_name serves as a logical link that ties a given voice channel with its associated icemeta section
of the JSON file. When you remove that linkage then there will be no tags transmitted.

I suspect the reason you're seeing the Alpha Tags appear from encrypted TG's is that you likely have one or more groups in your
whitelist that are encrypted fulltime, or that selectively use encryption whereby some transmissions are in the clear while others
are encrypted. The latter is the reason why the crypt-behavior 2 option exists. That is to say when a TG is encrypted that op25
upon detection of it being encrypted will drop or skip it and will place that group into a temporary blacklist for the next four
seconds.

Unfortunately, there's in the OSW on the control channel that signals encrypted transmissions. Therefore, if a group has active
encryption then it will not be known util op25 processes the channel grant and begins to decode the audio frames. This can result
in the transmission of the groups tag even though op25 drops the channel as described above.

What I would suggest is to review the groups in your whitelist to determine if any of them are fulltime encrypted, and if they are then
to just remove them altogether. Another solution is to try using the delay feature in the icemeta section. I don't know for sure
but setting a value of one or two seconds may act to suppress those sporadic tags associated with encrypted transmissions that
are skipped (dropped) by the crypt-behavior 2 functionality. Perhaps @boatbod can confirm if the delay option will address
this condition?

Bill

 

[boatbod]

Couple thoughts...

Blacklist/Whitelist at Trunking level will be overridden by Blacklist/Whitelist specified at Channel level. If you want one set of blacklists/whitelists across the whole P25 system set them in the trunking parameters, otherwise set them in the channel which they apply to.

If you have a whitelist active, only calls for those tgids will be decoded.

If you have encrypted calls set to skip, they will only be skipped once op25 positively identified that encryption is present. If op25 receives the HDU or MAC_PTT the call can be screened from the outset, but more often than not due to tuning latency, op25 will have late entry onto the tg so from there it can take up to 360ms to receive the ESS parameters which identify the algorithm and keyid. In the interim, garbled audio and the associated metadata will be sent to the stream.

In conclusion, if you have just one whitelisted tgid with mixed encrypted/clear traffic you'll unavoidably have some bursts of garbled audio and associated meta tags because of the late entry caused by tuning latency. If OTOH you have just one whitelisted tgid which is always transmitted in the clear, you should never have encrypted traffic leaking through.

 

[wgbecks]

@boatbod,

Would use of the icemeta delay feature possibly help to suppress tags from these encrypted group bursts or does this just delay tags
that are already queued for transmission?

 

[jschmall]

The value of the meta_stream_name serves as a logical link that ties a given voice channel with its associated icemeta section
of the JSON file. When you remove that linkage then there will be no tags transmitted.

I suspect the reason you're seeing the Alpha Tags appear from encrypted TG's is that you likely have one or more groups in your
whitelist that are encrypted fulltime, or that selectively use encryption whereby some transmissions are in the clear while others
are encrypted. The latter is the reason why the crypt-behavior 2 option exists. That is to say when a TG is encrypted that op25
upon detection of it being encrypted will drop or skip it and will place that group into a temporary blacklist for the next four
seconds.

Unfortunately, there's in the OSW on the control channel that signals encrypted transmissions. Therefore, if a group has active
encryption then it will not be known util op25 processes the channel grant and begins to decode the audio frames. This can result
in the transmission of the groups tag even though op25 drops the channel as described above.

What I would suggest is to review the groups in your whitelist to determine if any of them are fulltime encrypted, and if they are then
to just remove them altogether. Another solution is to try using the delay feature in the icemeta section. I don't know for sure
but setting a value of one or two seconds may act to suppress those sporadic tags associated with encrypted transmissions that
are skipped (dropped) by the crypt-behavior 2 functionality. Perhaps @boatbod can confirm if the delay option will address
this condition?

Bill

So as of right now, at least as it pertains to the police department, only 1 TG is unencrypted. The local fire departments will be swinging over to this new system over the next several months but they aren't using it right now. My whitelist only contains that one unencrypted TG so I don't really have anything I could remove there.

Couple thoughts...

Blacklist/Whitelist at Trunking level will be overridden by Blacklist/Whitelist specified at Channel level. If you want one set of blacklists/whitelists across the whole P25 system set them in the trunking parameters, otherwise set them in the channel which they apply to.

If you have a whitelist active, only calls for those tgids will be decoded.

If you have encrypted calls set to skip, they will only be skipped once op25 positively identified that encryption is present. If op25 receives the HDU or MAC_PTT the call can be screened from the outset, but more often than not due to tuning latency, op25 will have late entry onto the tg so from there it can take up to 360ms to receive the ESS parameters which identify the algorithm and keyid. In the interim, garbled audio and the associated metadata will be sent to the stream.

In conclusion, if you have just one whitelisted tgid with mixed encrypted/clear traffic you'll unavoidably have some bursts of garbled audio and associated meta tags because of the late entry caused by tuning latency. If OTOH you have just one whitelisted tgid which is always transmitted in the clear, you should never have encrypted traffic leaking through.

Oringinally I had my whitelist set on the channel level but still had tags coming through for TGs that weren't on that list. In my example above, I added the same whitelist to the trunking section as a test, it's not normally there.

Could I simply remove those unwanted TGs from my TG.tsv file? Or would that not really do anything? I could add a delay as @wgbecks suggested to see if that helps. I do get short audio pops when an encrypted TG is tuned to so I doubt that will go away as well. (Not really the end of the world)

 

[wgbecks]

You shouldn's be seeing any Tags associated with an encrypted group in in fact your white list only has one TG and that you're confident
it dosn't carry any encrypted traffic. In this applicaiton you should remove the whitelist from the Channel section and only apply the list
in the Trunking Section.

 

[jschmall]

You shouldn's be seeing any Tags associated with an encrypted group in in fact your white list only has one TG and that you're confident
it dosn't carry any encrypted traffic. In this applicaiton you should remove the whitelist from the Channel section and only apply the list
in the Trunking Section.

Unfortunately I do still see those references to the encrypted TGs. I will remove the whitelist from the channels section, leaving it on the trunking section and keep an eye on it.

 

[wgbecks]

If it turns out you're still transmitting tags for and encrypted TG then I would as you to setup your logging verbose to -v 11 then
restart op25, allow it to run long enough until you to see the unwanted tags. Stop the application and save the logfile (stderr.2)
that will have captured some valuable information to aid in troubleshooting.

 

[jschmall]

If it turns out you're still transmitting tags for and encrypted TG then I would as you to setup your logging verbose to -v 11 then
restart op25, allow it to run long enough until you to see the unwanted tags. Stop the application and save the logfile (stderr.2)
that will have captured some valuable information to aid in troubleshooting.

I have attached a zip file that contains my .json file (sanitized), TG list, whitelist, stderr (verbosity at 11), and just for kicks and giggles, my liquidsoap config. (You actually wrote this liq file for me in July of last year. I've since modified it as this P25 system came into play.)

 

[wgbecks]

jschmall,

Your rpd_bcfy_wlist.tsv (whitelist) file is incorrectly formed in that you've appended "Alpha Tag" information that seems to have
caused multi_rx.py to ignore the entry and seems to be acting as if no whitelist exists. Therefore, op25 has no TGID filters and
is following other traffic on the system causing sporadic tags and burst of scrambled audio briefely in accordance with the
functionality of the --crypt-behavior 2 parameter.

rpd_bcfy_wlist.tsv
201<TAB> Patrol 1

Whitelists and Blacklists only contain TGID's in decimal format that are usually listed one per line with the exception that
you can populate a range of TGID's separated with a <TAB> on a single line. See the following example whitelist.

white.list
201
205
210<TAB>214
300
325
379<TAB>394

Edit: rpd_bcfy_wlist.tsv to remove the "Patrol 1", save the file and restart op25 and you should be good to go.

Bill

 

[jschmall]

jschmall,

Your rpd_bcfy_wlist.tsv (whitelist) file is incorrectly formed in that you've appended "Alpha Tag" information that seems to have
caused multi_rx.py to ignore the entry and seems to be acting as if no whitelist exists. Therefore, op25 has no TGID filters and
is following other traffic on the system causing sporadic tags and burst of scrambled audio briefely in accordance with the
functionality of the --crypt-behavior 2 parameter.

rpd_bcfy_wlist.tsv
201<TAB> Patrol 1

Whitelists and Blacklists only contain TGID's in decimal format that are usually listed one per line with the exception that
you can populate a range of TGID's separated with a <TAB> on a single line. See the following example whitelist.

white.list
201
205
210<TAB>214
300
325
379<TAB>394

Edit: rpd_bcfy_wlist.tsv to remove the "Patrol 1", save the file and restart op25 and you should be good to go.

Bill

That was it! Thank you so much for catching that for me.

Another question regarding black/whitelists; eventually this system will grow to incorporate the local fire department as well. When that happens, I'll create another channel for them and use a secondary SDR. I'd like to blacklist the encrypted TGs for both systems, and whitelist each respective TGs (201 for the police and 401 for fire). I assume I can have a blacklist in the trunking section and a respective whitelist under each channel, yes?

 

[wgbecks]

Another question regarding black/whitelists; eventually this system will grow to incorporate the local fire department as well. When that happens, I'll create another channel for them and use a secondary SDR. I'd like to blacklist the encrypted TGs for both systems, and whitelist each respective TGs (201 for the police and 401 for fire). I assume I can have a blacklist in the trunking section and a respective whitelist under each channel, yes?

Actually, you don't need to create another channel for the Fire Department unless you will be setting up a separate BCFY feed for it.
You can scan and broadcast both on the existing channel by simply adding 401 to the current whitelist. Regardless, it's not necessary
to blacklist any other TGID's, encrypted or not in that whitelist(s) will only allow trunk tracking on the specified TG's contained in the list.

An example use of a blacklist would be that you're not using a whitelist to receive all possible groups on your system but desire to
exclude specific TG's from being tracked (played). If your'e very careful in crafting your white/black lists, you can use both at the
same time. That is, you'd might have a wide range of TG's in your whitelist but desire to drop one or more TG's out of that wide range
and would only need to place just those TG's in the blacklist.

Most generally, you'd normally only utilize one or ther other as outlined above.

 

[jschmall]

Actually, you don't need to create another channel for the Fire Department unless you will be setting up a separate BCFY feed for it.
You can scan and broadcast both on the existing channel by simply adding 401 to the current whitelist. Regardless, it's not necessary
to blacklist any other TGID's, encrypted or not in that whitelist(s) will only allow trunk tracking on the specified TG's contained in the list.

An example use of a blacklist would be that you're not using a whitelist to receive all possible groups on your system but desire to
exclude specific TG's from being tracked (played). If your'e very careful in crafting your white/black lists, you can use both at the
same time. That is, you'd might have a wide range of TG's in your whitelist but desire to drop one or more TG's out of that wide range
and would only need to place just those TG's in the blacklist.

Most generally, you'd normally only utilize one or ther other as outlined above.

I do plan on setting up a separate BCFY feed for fire yes. That is probably a couple months out but it is in the pipeline. I'll review my current black/whitelists to make sure I won't have any clashes as per your recommendations.

Thank you again!

 

[boatbod]

@boatbod,

Would use of the icemeta delay feature possibly help to suppress tags from these encrypted group bursts or does this just delay tags
that are already queued for transmission?

No, the delay feature just adds a fixed delay to whatever is in the queue. It'll all still get sent eventually.

To the OP:
When op25 starts up it logs the tg's that have been whitelisted or blacklisted. If there are file format errors you won't see the tg you want listed and this is the only clue that there might be issues.