Broadcastify Calls Node Skew Problems

[Twister_2]

I, too, have been experiencing lost calls on my node 272 over the past few months. SDRTrunk successfully decodes all the calls but thousands never make it to my node. I checked my skew time (it's fine) and I checked for adjacent uploaders. Turns out I do have someone else uploading the system under node 1927. When I check the system wide calls, the missing calls are still missing. Most often this happens when there are two quick back to back transmissions - either a back and forth conversation or a voice dispatch that follows an alert tone. This is super annoying because you'll hear the fire dispatch tone but no voice whatsoever.

I have since configured rdio-scanner and so far it is working flawlessly with no missing calls. I really hope there's a way to fix this on Broadcastify Calls because it's a great service with an even better interface. I just wish it would work better for me.

 

[scan18]

Similar to above post, I’ve been seeing a similar situation with Calls rejecting calls as false duplicates. I’m using SDR Trunk and I also have my TG’s set to record for ingest into Trunking Recorder (the windows program, not to be confused with Trunk Recorder). All transmissions are captured there without issue. So as a test, I disabled the Calls uploading directly in SDR Trunk and started uploading to my node from Trunking Recorder instead. Some quick back to back transmissions are still being rejected as duplicates according to the Trunking Recorder logs. There is one other node in my county, but it’s a different site and I know for a fact this particular TG I’m seeing the issue on is not covered on that node.

 

[techlogix]

I, too, have been experiencing lost calls on my node 272 over the past few months. SDRTrunk successfully decodes all the calls but thousands never make it to my node. I checked my skew time (it's fine) and I checked for adjacent uploaders. Turns out I do have someone else uploading the system under node 1927. When I check the system wide calls, the missing calls are still missing. Most often this happens when there are two quick back to back transmissions - either a back and forth conversation or a voice dispatch that follows an alert tone. This is super annoying because you'll hear the fire dispatch tone but no voice whatsoever.

I have since configured rdio-scanner and so far it is working flawlessly with no missing calls. I really hope there's a way to fix this on Broadcastify Calls because it's a great service with an even better interface. I just wish it would work better for me.

Same issue here. I have 2 nodes with SDRTrunk (2 separate computers) and find that back-and-forth conversations will have a few missed transmissions. I also regularly have the issue of no voice after tones. This has been going on since I setup the nodes months ago. I also have audio streams running on the same SDRTrunk instances and they catch everything with no problems.

I should add that these 2 nodes have no overlapping channels with each other or with any other nodes.

 

[blantonl]

I'm aware of these issues. We're trying to work out some best approaches to what is a very complex problem. Multiple different clients, trying to synch times between consumer level equipment, etc.

It's a hard problem that we're slowly working towards solutions for.

 

[bamx2]

Would something like this be powerful enough to support 3-4 SDRs with a powered USB?

Mini PC Windows 11 Pro, KAMRUI AK1 PRO 12GB RAM 256GB SSD Mini Desktop Computer Intel Celeron N5105 Processor,Quad Core Micro Computer 4K, Support 2.5-inch SSD, 2.4G/ 5.0G WiFi, Gigabit Ethernet, HTPC​

 

[Twister_2]

I'm aware of these issues. We're trying to work out some best approaches to what is a very complex problem. Multiple different clients, trying to synch times between consumer level equipment, etc.

It's a hard problem that we're slowly working towards solutions for.

I think a possible stopgap would be to allow users to monitor a specific node that is not affected by other users' nodes. If you want to use dupe protection on systemwide monitoring points, great. But what about allowing users to listen to just one feeder without worrying about dupes? I guess a single user can unknowingly upload dupes but what fraction of the total traffic would this account for?

 

[lynchy135]

I think a possible stopgap would be to allow users to monitor a specific node that is not affected by other users' nodes.

It's already possible to get calls that a node provides https://www.broadcastify.com/calls/node/{nodeid}. This will show you live calls. You can't go back in history for that particular node though.

I am not sure if thats exactly what you're looking for or not.

 

[Twister_2]

It's already possible to get calls that a node provides https://www.broadcastify.com/calls/node/{nodeid}. This will show you live calls. You can't go back in history for that particular node though.

I am not sure if thats exactly what you're looking for or not.

From my observations and what others have said, my node will delete dupes if someone else is uploading the same call to their node. With that in mind, re-read my suggestion and see if it makes more sense.

 

[scan18]

From my observations and what others have said, my node will delete dupes if someone else is uploading the same call to their node. With that in mind, re-read my suggestion and see if it makes more sense.

Agreed, if I monitor just my own node, valid dupes from another node are not included. Which I guess makes sense considering my client never uploaded the dupe call to begin with because the broadcastify server is essentially “rejecting” it.
I agree though, it kind of makes it pointless to listen to a single node if you know you are “missing” calls.

 

[ThreatLevelMidnight]

It's already possible to get calls that a node provides https://www.broadcastify.com/calls/node/{nodeid}. This will show you live calls. You can't go back in history for that particular node though.

I am not sure if thats exactly what you're looking for or not.

Would love the ability to specify which node(s) ingest calls to a custom playlist

 

[rjdj2000]

The solution is to make sure your PC is regularly syncing the time, not to wait until it's way off to get notified. Lindsay posted links explaining how to make the PC sync more frequently, so start with that.

There is also a lightweight software for Windows called Dimension 4 that will keep the time synced up. It's old software and is no longer maintained, but it works fine even with Windows 11. I suggest finding and installing that for the easiest solution.

I've went in and did a task schedule to repeat time sync every hour. This morning, checked things and was a little over 1.8 seconds off. So went and manually synced the time, next call came in was over 2 seconds. Have a very good internet connection and the system isn't busy at the moment so not sure what is going on. Will look into the program now, not that it is going to help much probably. May just start sending with my mac as well as the pc. Only thing is the last time I did that, I started hearing duplicate transmissions when I was listening so I turned off the mac from streaming the calls.

Edit: Found program mentioned, installed, synced and now it is like .77 seconds. Must be something with the stock windows time server but will see how it goes. Now if I can get the dang duplicates gone, I'd be good. Must be when someone keys back up on the same channel instantly after the first one and SDRTrunk sends it and it thinks it's a duplicate. I'm the only person sending stuff in, that I know of, for the site so it is weird.

 

[jjbllitz]

Must be something with the stock windows time server

Windows time sucks. You need a tool to track the drift and adjust sync based on that.

 

[rjdj2000]

Windows time sucks. You need a tool to track the drift and adjust sync based on that.

Well I did find and install the program that GTR mentioned and set it up. Sometimes have ones over 1 second others are .112 or so. Still getting the duplicate calls but that will have to be for another thread.

 

[belvdr]

Whatever NTP client you use, always specify at least 3 pool addresses (not servers). I use:

0.pool.ntp.org
1.pool.ntp.org
2.pool.ntp.org
3.pool.ntp.org

You'll always be connecting to NTP peers that are functional and it gives your NTP client enough info to make intelligent choices on what the correct time is.

 

[egftechman]

I think a possible stopgap would be to allow users to monitor a specific node that is not affected by other users' nodes. If you want to use dupe protection on systemwide monitoring points, great. But what about allowing users to listen to just one feeder without worrying about dupes? I guess a single user can unknowingly upload dupes but what fraction of the total traffic would this account for?

I'm guessing Broadcastify is trying to minimize their AWS S3 costs, including storage, bandwidth, and transaction by doing dupe detection from metadata in API request.
Eliminating 1,000s of S3 Put or Post requests every day can create major savings at billing time (I'm guessing transaction costs are the bulk of the fees each month) ....and although each call is only a couple of KB, the storage and bandwidth savings at the scale of Broadcastify by eliminating dupes is likely significant.
I don't see that @blantonl has published what the dupe algorithm is (nor are they obligated to publish it), but I read a mention somewhere that they were working on improving the hashing used for dupe identification, and with all the regional/statewide systems now, it needs to be very fast as a wide area talkgroup could generate 100s of nearly simultaneous uploads.

 

[belvdr]

I'm guessing Broadcastify is trying to minimize their AWS S3 costs, including storage, bandwidth, and transaction by doing dupe detection from metadata in API request.
Eliminating 1,000s of S3 Put or Post requests every day can create major savings at billing time (I'm guessing transaction costs are the bulk of the fees each month) ....and although each call is only a couple of KB, the storage and bandwidth savings at the scale of Broadcastify by eliminating dupes is likely significant.
I don't see that @blantonl has published what the dupe algorithm is (nor are they obligated to publish it), but I read a mention somewhere that they were working on improving the hashing used for dupe identification, and with all the regional/statewide systems now, it needs to be very fast as a wide area talkgroup could generate 100s of nearly simultaneous uploads.

The actual data is likely the major cost. There's no value in storing the same data twice.

S3 PUTs and POSTs are inexpensive, comparatively. To go from 5 million to 50 million PUT/POST/LIST is $250/month. For anyone interested, AWS has a calculator, based on retail (no discounts):

AWS Pricing Calculator

AWS Pricing Calculator lets you explore AWS services, and create an estimate for the cost of your use cases on AWS.

calculator.aws