Broadcastify Calls Duplicate Calls Handing Updates

mtindor

OH/WV DB Admin
Database Admin
Joined
Dec 5, 2006
Messages
11,222
Location
Carroll Co OH / EN90LN
There really needs to be some way for the Calls provider (at least one using SDRTrunk) to check into why things copied / decoded / recorded do not make it up to the Calls platform.

Reference attached screenshot (from Ohio MARCS node 87 SDRTrunk). In this example, the 3 Jefferson 41SHERIFF 1 transmissions were copied by my SDRTrunk and were uploaded to Broadcastify.

Then the next exchange (i was listening to it at the time in SDRTrunk) was POST41 / STCLAIR). NONE of the 7:32 AM traffic made it up to BCFY Calls. It's not available under either talkgroup 51541 or 51752. Then, the 7:36 transmissions (there were two), only one made it up (and it was uploaded from Ohio MARCS node 2540.

So part of the 7:36 transmissions didn't make it to BCFY and all of the 7:32 transmissions (which was over 20 seconds of voice) made it up.

I have no idea if MY SDRTrunk uploaded it. And presumably node 2540 was the preferred node for that talkgroup 4 minutes later -- and it never uploaded it.

So was this 7:32 AM traffic ever uploaded by any node? I have no way telling in SDRTrunk if MY SDRTrunk uploaded it. But I know it copied / decoded and played it. And if node 2540 was the "desginated" node for those transmissions at that time, why didn't it upload it.

In the absense of any way to get upload information from SDRTRunk (at least not that I can find), I can only defer to you, Lindsay, as to what happened with that traffic. Did anybody upload it? Who was responsible for uploading it? was mine uploaded and rejected because I wasn't the preferred node for that talkgroup at the time?

In my mind, a lot of traffic appears to be not making it to the platform, at least for Ohio MARCS, from SDRTrunk node ops. Looks like something that should be investigated.
 

Attachments

  • example_lindsay.png
    example_lindsay.png
    186.8 KB · Views: 11

blantonl

Founder and CEO
Staff member
Super Moderator
Joined
Dec 9, 2000
Messages
11,393
Location
San Antonio, Whitefish, New Orleans
Might have been an issue or a race condition where nodes were being switched. Not sure. I'll keep an eye out on it.

However, keep in mind one of the things that we're seeing with Broadcastify calls is when when a patch occurs, SDRTrunk sends to a single talkgroup. It might be a patch ID, it might be one of the talkgroups patched, it might be the expected talkgroup, but I don't quite know what the behavior is.

I've already worked a few support requests where folks SWEAR UP AND DOWN that we're somehow responsible for calls not making it up to the platform, where low and behold they ended up on some random talkgroup that was part of a patch. One of the ways to figure out what might have happened is to click on a unit ID and see what other talkgroups that unitID has been participating on.

Note that this behavior might be different between different system types (Motorola, Harris) and setups / versions on the SDRTrunk side. Trunk Recorder sends all the talkgroups as part of a call upload (comma delimited) and we pull out the first entry and use that as the talkgroup to record to. We also have a rudimentary solutions for "priority patch talkgroups" where we can specify that if a talkgroup is part of a patch, that talkgroup is the one that "takes the call." I might need to get with the SDRTrunk team to see if we can get the patch talkgroups added to the API call that sends the call up.
 

mtindor

OH/WV DB Admin
Database Admin
Joined
Dec 5, 2006
Messages
11,222
Location
Carroll Co OH / EN90LN
Might have been an issue or a race condition where nodes were being switched. Not sure. I'll keep an eye out on it.

However, keep in mind one of the things that we're seeing with Broadcastify calls is when when a patch occurs, SDRTrunk sends to a single talkgroup. It might be a patch ID, it might be one of the talkgroups patched, it might be the expected talkgroup, but I don't quite know what the behavior is.

I've already worked a few support requests where folks SWEAR UP AND DOWN that we're somehow responsible for calls not making it up to the platform, where low and behold they ended up on some random talkgroup that was part of a patch. One of the ways to figure out what might have happened is to click on a unit ID and see what other talkgroups that unitID has been participating on.

Note that this behavior might be different between different system types (Motorola, Harris) and setups / versions on the SDRTrunk side. Trunk Recorder sends all the talkgroups as part of a call upload (comma delimited) and we pull out the first entry and use that as the talkgroup to record to. We also have a rudimentary solutions for "priority patch talkgroups" where we can specify that if a talkgroup is part of a patch, that talkgroup is the one that "takes the call." I might need to get with the SDRTrunk team to see if we can get the patch talkgroups added to the API call that sends the call up.

I appreciate you being open-minded about something possibly needing done that may be outside the purvue of the Calls node provider. And I'm not saying I can guarantee my setup is uploading. I did put in a feature request for SDRTrunk for BCFY Calls logging. I think that is crucial.

I don't typically listen/record on my SDRTrunk locally. But I did enable both for the sole purpose of making sure that calls indeed were being decoded / were playing / and were being recorded to files on my system. And that part is working superbly. And when that patch was up, when the calls did make it to BCFY they were under TG 51541 (POST41). So it has to be something between SDRTrunk and BCFY.

Of note is that although I do understand what you are saying about patched talkgroups, and work may need to be done there, I see similar issues with non-patched talkgroups. I might locally hear/recorder an hourly radio check for the SO and hear/record every mobile and dispatch comm during that event, but some of those may be missing on BCFY calls.

I only reached out to you because of the lack of any facility for me to troubleshoot on my end. And like I said, I did submit a feature request to SDRTrunk for some logging.

Thanks man. I really do like the playlists.

Mike
 

blantonl

Founder and CEO
Staff member
Super Moderator
Joined
Dec 9, 2000
Messages
11,393
Location
San Antonio, Whitefish, New Orleans
Understood.

This system is complicated. There are so many variables that all feed into this system that we have to normalize into our "vision"
  • node quality
  • node software
  • the system itself
  • people getting "cute" with their setups
  • certainly bugs on our end
Also, I'm now posting in the talkgroup details dialog the nodes that are participating in sending calls for a group and which node is assigned.

[A] = assigned
[NC] - not considered.

Before a node provider sees "not considered" and get's completely offended that we're not considering their node, please note that this means that the node is "newer on the scene" and we're not YET considering it. It's gotta get some better history.
 

Outerdog

T¹ ÆS Ø
Premium Subscriber
Joined
Jul 1, 2016
Messages
669
There are so many variables that all feed into this system that we have to normalize into our "vision"
  • node quality
  • node software

Can you talk a little bit about how node quality is determined? Also, is one software platform preferred over another? If so, for what reason?
 

GTR8000

NY/NJ Database Guy
Database Admin
Joined
Oct 4, 2007
Messages
16,166
Location
BEE00
Also, I'm now posting in the talkgroup details dialog the nodes that are participating in sending calls for a group and which node is assigned.

[A] = assigned
[NC] - not considered.
I'm seeing statuses of Current [A], Current, Current [NC], and Removing.
 

blantonl

Founder and CEO
Staff member
Super Moderator
Joined
Dec 9, 2000
Messages
11,393
Location
San Antonio, Whitefish, New Orleans
Current with no “NC” means that the node is “good” and would be considered if the currently active node goes stale.

removing means that node has gone stale and it’s about to be removed from the current node list
 

JasonCox

Member
Feed Provider
Joined
Apr 26, 2021
Messages
10
Location
Little Elm, Texas
We're already lining up developers on the iOS/Android side with an API we should have released soon, which will greatly get things moving on better client interaction.
Do you guys plan on opening up the API to new applications again once this is done? From what I remember back when I last looked around the App Store, there were only a handful of active apps and a ton of might-as-well-be abandoned ones.

Not that I need another side project...
 

blantonl

Founder and CEO
Staff member
Super Moderator
Joined
Dec 9, 2000
Messages
11,393
Location
San Antonio, Whitefish, New Orleans
Can you talk a little bit about how node quality is determined? Also, is one software platform preferred over another? If so, for what reason?
It’s very, um, complicated. But it has to do with the skew of the node and how long it has been providing the talkgroup. One software platform is not considered over another
 

blantonl

Founder and CEO
Staff member
Super Moderator
Joined
Dec 9, 2000
Messages
11,393
Location
San Antonio, Whitefish, New Orleans
Do you guys plan on opening up the API to new applications again once this is done? From what I remember back when I last looked around the App Store, there were only a handful of active apps and a ton of might-as-well-be abandoned ones.

Not that I need another side project...
We’ll see how the current app developers adopt the new API. But for now the answer is no.
 

mam1081

Member
Feed Provider
Joined
Dec 19, 2002
Messages
1,103
Location
Next to a scanner...
It’s very, um, complicated. But it has to do with the skew of the node and how long it has been providing the talkgroup. One software platform is not considered over another
Does the streaming software matter for calculating skew value? I know you said the software isn't taken into account for rating...
For the two feeds on one of the systems I'm providing, one is trunk-recorder, and one is SDRTrunk. The SDRTrunk shows considerably less skew, but has terrible audio quality. The trunk-recorder has around 4x the skew number, but audio quality/decode rate is much better. The trunk-recorder is running on a dedicated linux computer and has decent fiber internet...not sure why the skew looks that bad!
 

mtindor

OH/WV DB Admin
Database Admin
Joined
Dec 5, 2006
Messages
11,222
Location
Carroll Co OH / EN90LN
Does the streaming software matter for calculating skew value? I know you said the software isn't taken into account for rating...
For the two feeds on one of the systems I'm providing, one is trunk-recorder, and one is SDRTrunk. The SDRTrunk shows considerably less skew, but has terrible audio quality. The trunk-recorder has around 4x the skew number, but audio quality/decode rate is much better. The trunk-recorder is running on a dedicated linux computer and has decent fiber internet...not sure why the skew looks that bad!

I just checked a handful of SDRTRunk Calls nodes, and they are all under 2 seconds (usually much less). I checked a handful of Trunk Recorder nodes, all but one had skew above 5 seconds.

Given that most people are likely running SDRTrunk on Windows and using a reliable time source, that might have something to do with it. I bet a lot of people running Trunk Recorder on Linux do not have anything set up to sync time with a reliable time source.
 

mam1081

Member
Feed Provider
Joined
Dec 19, 2002
Messages
1,103
Location
Next to a scanner...
I just checked a handful of SDRTRunk Calls nodes, and they are all under 2 seconds (usually much less). I checked a handful of Trunk Recorder nodes, all but one had skew above 5 seconds.

Given that most people are likely running SDRTrunk on Windows and using a reliable time source, that might have something to do with it. I bet a lot of people running Trunk Recorder on Linux do not have anything set up to sync time with a reliable time source.

That could be. Back when Calls was first implemented on Broadcastify, a NPTsync was required - not sure if that's still the issue. Mine consistently shows skew around 4, and I know my time accuracy is checked at least hourly and is within 0.15 sec of NTP sources.
 

mtindor

OH/WV DB Admin
Database Admin
Joined
Dec 5, 2006
Messages
11,222
Location
Carroll Co OH / EN90LN
That could be. Back when Calls was first implemented on Broadcastify, a NPTsync was required - not sure if that's still the issue. Mine consistently shows skew around 4, and I know my time accuracy is checked at least hourly and is within 0.15 sec of NTP sources.

Interesting, I do have to wonder then if Trunk Recorder really just requires more resources -- and it makes me wonder if there is just inherent additional delay between the time raw audio comes in, and decoded audio is delivered to BCFY when running Trunk Recorder.
 

mam1081

Member
Feed Provider
Joined
Dec 19, 2002
Messages
1,103
Location
Next to a scanner...
I'm hoping it's not correct, but what if SDRTrunk skew if calculated from the call end and Trunk-Recorder was from the call start? I could see the average call duration being around that 4 second mark...
 

Napalm

Active Member
Premium Subscriber
Joined
Mar 2, 2006
Messages
722
Location
Lake Co, Ind
My skews are usually under a second. 2+ seconds is unusual. I'm running a Windows PC with SDRTrunk. I'm streaming five systems one is constantly active. (Node 2800).

I sync time every hour.
 
Top