DSD FME

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
I've been tinkering away, a little here and there, adding some more decoding for things like CSBK info, and LC opcodes like aliases, etc. If anybody has any DMR samples that have embedded GPS information in it and could provide a sample, that would be splendid.

Currently on the lookout for:
  • DMR with GPS
  • dPMR Unencrypted
 
Last edited by a moderator:

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Hello all, once again, going to update the status of my project, DSD-FME. A majority of the updates are bug fixes, tweaks, and performance issue improvements (especially in the Cygwin builds). I did however, sneak in a few new features.

DMR 1/2 Rate Data LRRP decoding

I have things like the IP header, Source, Destination, Latitude, Longitude, Velocity, and Direction decoding. If anybody could chime in, and let me know the appropriate term for the 'direction' in the context of what is being measured here, that would be great. I wasn't honestly sure if I should have used 'Heading', 'Bearing', 'Course', 'Track', or similar. The calculation for it is the value given times 2.

Also, if anybody can confirm, I am using the calculation (vi+vf)/128 for velocity in m/s, using vi as byte one and vf as byte two. Comparing this data to Fast Lane, I notice that sometimes my calculation matches Fast Lane, and other times, if the first byte (or is it the second, its been a little bit) is set to a value and not 0, then the calculation will differ from Fast Lane.

Screenshot_281.png

Sadly :cry: DMR 3/4 Rate Data isn't working until I can find or write a viable Trellis/Viterbi decoder to reconstitute the original 3/4 Rate Data.

CSBK Data Decoding

I have added some limited decoding of things in the CSBK frames, mostly just things gleamed from OP codes defined in other projects. Its not fully fleshed out yet, and may be pretty hit or miss. It does have some Connect Plus decoding, but I can't say its working very well, its just really there as extra information, but may or may not be very accurate or add much.

dPMR Smooth Decoding

I believe I have finally worked out issues I was having with dPMR, in particular frame sync issues. With the limited sample size I have for dPMR, I have achieved smooth and coherent playback finally. Further testing is certainly welcome from anybody wanting to try it out and let me know how it works for them.

Installer Script for Ubuntu/Debian/Mint/Pi

This has been on the Github page for quite a while, but I worked out an installer script that will handle building and compiling DSD-FME in certain Ubuntu/Debian/Mint/Pi environments, requiring the user to only input three commands into the console. I haven't tried the script on newer Raspberry Pi environments yet, would love to hear if it handles everything correctly there as well.

Code:
wget https://raw.githubusercontent.com/lwvmobile/dsd-fme/pulseaudio/install.sh
chmod +x install.sh
./install.sh

Set up quick Virtual Sink to use with DSD-FME in pulse audio with:
Code:
pacmd load-module module-null-sink sink_name=virtual_sink  sink_properties=device.description=Virtual_Sink

pacmd load-module module-null-sink sink_name=virtual_sink2  sink_properties=device.description=Virtual_Sink2

Of course, manual install instructions are on the branch readme page as well.

The Github page for the Pulse Audio branch for Linux Desktop Users and Pi users can be found here:


Cygwin Build Improvements

The Cygwin build has had some ups and downs, performance breaking bugs, audio breaking bugs, some code rewrite, and other things. I believe I finally have the Cygwin build up to speed and mostly on par with the Linux build, the decoding performing the same, but still suffering a performance penalty due to the nature of Cygwin. I must admit though, the Cygwin build does have some weird quirks that I like to blame on Cygwin, and not at all on my terrible coding :ROFLMAO:

The Ncurses terminal in Cygwin ONLY works if built and used in the Cygwin environment directly. Sadly, at current, I don't know of any simple solution to make this feature work in a distributed precompiled release. That is to say, the command prompt nor Windows Powershell are compatible with the cygwin implementation of ncurses.

All that being said, I do have a new precompiled x86-64 version of the Cygwin build available on the cygwin branch available under the releases link under assets.


Also, general build and use instructions can be found on the Cygwin build branch main page at:


Again, I believe I have a lot of the performance related bugs worked out, but would love to have a few people try it out and see. If you are interested, again, you can check it out in the links above.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Would it be possible to display keyid in dmr simplex?

Just for those playing along at home, I did some testing with Simplex (I haven't had the chance to until the last day or so) and for the time being it doesn't look like DSD-FME is going to be able to handle Simplex (MS) signals properly for the time being. Looks like I'm going to have to dig into the ETSI manual and see what I can work out, but it may be quite a while before I have anything figured out and working.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Thought I'd take some time to update the status of this project. I've been working here and there in my free time on a few new features and/or improvements, and I'll share a few of the points here. I'm going to take my time in this phase, and try not to rush the code to Github right away, try to offer a bug free experience going forward with the releases...hopefully.

Pretty Pretty Colors

Seriously...Any sort of actual maybe important data will be highlighted with colors from now on, make it easier to spot in otherwise dull and monotonous console/log output. If anybody has any input as to whether or not certain colors work well needs to let me know, I know some color combinations don't work well together, and also colorblind individuals may not be able to read certain text on certain background. Anybody feel free to chime in and let me know. I'm also kind of bound to using certain colors, terminal output doesn't exactly have a wide range of selection readily available.

Screenshot_300.png

Seriously...I really even call it that in the source code...

Code:
 //pretty pretty colors
 #define KNRM  "\x1B[0m"
 #define KRED  "\x1B[31m"
 #define KGRN  "\x1B[32m"
 #define KYEL  "\x1B[33m"
 #define KBLU  "\x1B[34m"
 #define KMAG  "\x1B[35m"
 #define KCYN  "\x1B[36m"
 #define KWHT  "\x1B[37m"

Alias Headers and Blocks

I've mentioned I was working on this before, but don't recall mentioning it was in working order. I have the Alias Headers and Blocks decoding while running the NCurses terminal and outputting to the log.

Screenshot_295.png

Screenshot_297.png

3/4 Rate Data Decoding

Its still a work in progress, but finally, I've been able to get some expected output from the Trellis decoder I've borrowed from the MMDVMHost with a little bit of reworking for DSD-FME. I should also take this time to mention I've seperated the data decoding for 1/2 rate and 3/4 rate loading into seperate superframes according to which slot they reside in, so they don't collide and mangle, here is an example of 3/4 rate data decoding coming in slot 1 and slot 2 but decoded. Just ignore that extra velocity measurement...little quirks to work out still..

Screenshot_292.png


I've also got ASCII dumps, I know, there is a few bits that specify whether its UTC-8 or ASCII or otherwise, in the future I'll weed those out as well, but virtually, it'll be about the same output. I had a good friend in Israel submit a sample to me of 3/4 rate data showing the coordinates in the ASCII interpretation of the 3/4 rate data.

Code:
Full 3/4 Rate Trellis Payload
  [01][62][01][88][08][B0][03][00][2D][0A][00][00][3D][02][41][31][38][32]
  Hex to Ascii - .b......-...=.A182
23:23:48 Sync: +DMR   slot1  [slot2] | Color Code=04 | (CRC OK ) | Slot idle   
23:23:48 Sync: +DMR  [slot1]  slot2  |               | (CRC OK ) | RATE 3/4 DATA
Full 3/4 Rate Trellis Payload
  [03][22][35][33][37][32][34][30][32][32][32][4E][33][31][34][37][28][35]
  Hex to Ascii - ."537240222N3147(5
23:23:48 Sync: +DMR   slot1  [slot2] | Color Code=04 | (CRC OK ) | Slot idle   
23:23:48 Sync: +DMR  [slot1]  slot2  |               | (CRC OK ) | RATE 3/4 DATA
Full 3/4 Rate Trellis Payload
  [04][1A][35][32][3A][00][23][0A][20][E0][00][18][AE][51][82][E3][08][2E]
  Hex to Ascii - ..52:.#. ....Q....
23:23:48 Sync: +DMR   slot1  [slot2] | Color Code=04 | (CRC OK ) | Slot idle   
23:23:48 Sync: +DMR  [slot1]  slot2  |               | (CRC OK ) | RATE 3/4 DATA
Full 3/4 Rate Trellis Payload
  [06][92][30][31][31][30][D4][03][80][00][40][00][00][00][C7][00][5C][85]
  Hex to Ascii - ..0110....@.....\.
23:23:48 Sync: +DMR   slot1  [slot2] | Color Code=04 | (CRC OK ) | Slot idle   
23:23:48 Sync: +DMR  [slot1]  slot2  |               | (CRC OK ) | DATA Header   
  LRRP Control ACK -  Source: 61 Destination: 100

I'm told the string "537240222N3147.5 should be conveying coordinates. Admittedly, I'm still working out the error correction for 3/4 rate data, I'm currently cheating the path metrics and not going through ALL OF THEM. So, once the bits start to slip on 3/4 rate data, chances are the data for that frame will be bad, but its better than it was before and kind of works now. Certainly something I plan of getting back to and getting to work fully.

Well, with that being said, and those with a keen eye may have noticed already in a previous screenshot, I've been working on the big thing. I know this seems like a duh thing, considering that that should be in there, but to be honest, I haven't seen a version of the original DSD actually ever implement this, nor the super popular version of DSD on Github that shall go unnamed...from France...but yeah, I got it working...

DMR Stereo TDMA

Yeah, I know, surprising right? Not many people realize this, but the original DSD project never had anything but single channel 8Khz audio, and also couldn't properly handle voices in both slots simultaneously. Older projects would just run them all together and just a bunch of unintelligible audio would result, and later projects just locked onto the voice frame sync and then skipped the next TDMA frame until all six VC frames played out, missing out on the other slot entirely. I've got a working prototype of Stereophonic TDMA working inside of the DSD code. Considering every other project already does this besides the open source DSD and its variants, its finally good to finally catch up on a seemingly expected feature for DMR decoding.

Since screenshots won't work, I'll just link yet another half-*** YouTube video so you can see for yourself. Skip to 4:20 *ba dum tiss*, missed it by just a couple days, if you want to skip directly to the dual voice stereo audio portion. Headphones might help.


Again, everything in this update is still a work in progress, and hasn't make it to the Github page just yet, but when I feel good about the current code and have it tweaked and fixed up to my liking I'll publish it. There may even be a few Easter Eggs involved if you check it out close enough..who knows. :cool:. eBay Prisca Vicky.

So, that leaves just one last point.

Why?

Why bother, right? Sometimes I even ask myself that. I really have no idea the user base I've aquired yet, but I expect it to be in the single digits at best. To be honest, I'd rather learn about a hobby, and share what I've learned. In this case, its the raw source code, experience, and software I've worked on and compiled that maybe somebody maybe might will use. That's an easier answer for me than simply answering with 'Pay $25 Dollars and ask a forum for customer support" and then become persona incognito. If you need help with my project, I don't expect you to ask a forum, I hope you'll ask me directly, I'm always more than willing to go out of my way to help people as best as I can. I learned a long time ago, from my very first job, we don't sell computers, we don't sell contracts, we sell 'service'. I still believe in that today, and I'm not even trying to sell this thing. I also see that exact same example on this forum all the time with three individuals who inspire me to keep going through their projects and actions. Boatbod, KA1RBI (Maxx) and DSheirer. Not only do they open source their projects, they even directly answer and help people on this forum all the time.

Aside from that, I just think its a cool project to share, and I feel passionate about projects like this that I work on. I like to make things better (hopefully) than it was before. Even if somebody just pops in and learn something new, or learns from mistakes I've made, that's good enough for me. I know the project isn't perfect, its never going to be, but I at least want it to be a viable simple solution for Linux Desktop Users.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Hi, all, just wanted to drop in real quick and let everybody know the previous update with the new improvements is now live on the Github page under the pulseaudio branch at:


Just for a quick recap, the following improvement/features were added to the latest release:
Pretty Pretty Colors
All important information being color coded in the terminal output.

Alias Headers and Blocks
Any caller aliases or callsigns are decoded, if present. This now also includes Embedded GPS (not LRRP) on LC opcode 0x08 if present as well.

3/4 Rate Data Decoding
LRRP data decoding on 3/4 Rate Data avaliable and Hex to Ascii dump as well. Same as 1/2 Data.

DMR Stereo TDMA
The big feature of this release, allowing both TDMA channel/slots to play voice simultaneously and the ability to route it to the left or right speaker.

NCURSES DMR STEREO Layout
Ncurses terminal has been updated and the code organized (finally) for DMR to show and differentiate the activity in both slots simultaneously, and will also show any extra data pertinent to what is happening in that slot, such as aformentioned LRRP data, Alias blocks, PI information, etc.

Screenshot_326.png


MS Voice/Simplex Improvements

In addition, thanks to the DMR Stereo method I wrote, I was able to get MS/Simplex Voice decoding to work a lot better by using the same method for DMR Stereo. The data decoding is improved, but is still hit or miss (mostly miss) due to the very short bursts the data usually comes in if not accompanying a voice transmission. From my observations, usually the Voice LC Header, and PI header may be lost, and any data only bursts may be decoded improperly. Any data at the end of a transmission will usually decode fine. This is usually caused by the demodulator needing a few repetitions of frame sync to get proper parameters to decode by, so initial frames may be lost or decoded improperly. In the future, I will attempt to remedy this by making a more robust buffer.
---

This leaves me with one other thing to mention. This is not a decision I make lightly, but due to the segmentation and extra work that is involved with porting any changes I make to the Cygwin Windows builds, I am going to officially discontinue the Cygwin Windows builds. I have several reasons to do so.

Firstly, I am not happy with the performance of Cygwin, nor the builds that result from using them. The Cygwin builds run okay sometimes, but compared to the Linux builds, skip a step and chug in comparison. Also, its hard for me to accurately document and relay a method to building these versions for others or offering help with Cygwin in general.

Secondly, as I have already mentioned, it takes time to port all the changes I made to work in the Cygwin build, making sure not to undo things I have specifically set just to make the Cygwin version run adequately and not have bugs. This last undertaking was a month or so in the making, and it would most likely take me an equal amount of time to port these features and properly test them in Cygwin builds, as opposed to working on other improvements and features.

Thirdly, I'm just not a Windows person. I'm tired of fighting with my VM wanting to do updates and security scans every time I do boot it again, taking away from productivity time. The truth is, if you are using Windows, you will be much better served by other software compared to the Cygwin version of this. Either that, or to use a VM, or old computer with a Linux Distro installed if you wish to try this version out.

Edit: Use the correct screenshot.
 
Last edited:

KA1RBI

Member
Joined
Aug 15, 2008
Messages
799
Location
Portage Escarpment
dPMR Smooth Decoding

I believe I have finally worked out issues I was having with dPMR, in particular frame sync issues. With the limited sample size I have for dPMR, I have achieved smooth and coherent playback finally. Further testing is certainly welcome from anybody wanting to try it out and let me know how it works for them.

Good to see this activity - looks like you've been having a good time making these additions --- and not to mention one of my favorite old low-tech technologies - CURSES!!!

Reason for writing today is to see if I could get a copy of any dPMR samples. Are the samples that you have IF (I/Q), disc tap, or symbol captures?

Thx

Max
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Is this program able to run on 32 bit system? Also, compared to the LEH version, how easy is this to install? I have xubunto, cygwin and of course windows.

Unfortunately, I don't have a 32-bit Windows version compiled, and cygwin is too much of a headache to maintain, but the 32-bit ubuntu/debian version should be feasible. Honestly can't say I've tried it before, but I'm hoping all the dependencies are the same (or rather, named the same thing in the repo) You can try to install it with the installer script by running:

Code:
wget https://raw.githubusercontent.com/lwvmobile/dsd-fme/pulseaudio/install.sh
chmod +x install.sh
./install.sh

Its been tested to work as far back as Ubuntu 18.04 and as recent as 22.04.

Also, make sure to see the example usage on the github page, its right there on the readme, its going to vary a bit from LEH version, or you can always message me directly if you have any questions on how to use it.


Good to see this activity - looks like you've been having a good time making these additions --- and not to mention one of my favorite old low-tech technologies - CURSES!!!

Reason for writing today is to see if I could get a copy of any dPMR samples. Are the samples that you have IF (I/Q), disc tap, or symbol captures?

Oh, yes, curses, my old friend. I'll take it any day over endless scrolling console output. I'll gladly take more dPMR samples. For now, I'll have to take them as wav files if you can do that. Thanks.
 

iscottybotty

Member
Joined
Dec 7, 2016
Messages
51
Location
Birmingham, UK
Morning all!
I have installed the program with Ubuntu, it’s on and seems to be working…. Well.
So I’m trying to play a wav, two terminal windows are open.
One window has the command:
dsd -fr -Z -pu -d ./MBE/ -N 2>> dmrframes.log
The second window uses the command: tail -n 100 -f dmrframes.log
Once I play the wav, the data trail appears, however I keep getting the message, “egmentation fault (core dumped), then I keep having to renter the command, but it does the same over and over?
Could anyone help or advise?
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
You have to make a folder in that directory your working in named MBE to dump those mbe files into, otherwise it will sefgault when you try to run it with the -d ./MBE/ switch.

Code:
mkdir MBE

But it does raise a good point, I should write some code to check for the existence of a directory there, and attempt to create it if it doesn't exist so that people don't segfault when using the -d option and trying to place amb/imb files inside of a directory that doesn't exist.
 
Last edited:

iscottybotty

Member
Joined
Dec 7, 2016
Messages
51
Location
Birmingham, UK
You know members, I hardly ever post cheesey comments but, crédit where credits due.
@Iwvmobile - thank you for the help with my constant questions! It’s very much appreciated and I just wanted to say, thank you.
Dove 🕊 Award 🥇 to Iwvmobile.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
I hardly ever post cheesey comments
I've laid out a few cringe worthy ones lately, so no worries. In my defense, I may have had a few to drink when doing so.

Dove 🕊 Award 🥇 to Iwvmobile.

Well, I've never won an award before, so I'll take it! Now I just got to find a way to work this prestigious award into the source code of the software, maybe as an easter egg or something.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
I should also take this opportunity to mention that I've pushed a few minor bug fixes related to handling inverted DMR signal and frame sync passing it to the DMR Stereo method as well as the bugfix for the segfault that occurs when attempting to write MBE files to a directory that doesn't exist. Now it will look for that directory, and create it if not present. These aren't major bugs, unless its something you are affected by, there won't be any reason to pull the latest. These bugfixes are on the pulseaudio branch.

This also gives me an opportunity to mention, in addition to the install.sh script for installing on debian/ubuntu systems, in the dsd-fme folder, you will find a rebuild.sh script that will do all the leg work of pulling the latest from the pulse audio branch, building it, and installing it for you. You may need to give it execute permisison with a quick chmod +x rebuild.sh and you can run it with ./rebuild.sh. Again, this will do all the leg work for you to make a quick and seemless upgrade of the software anytime it gets updated.

A word of caution though, if you make modifications to the source code, and try to run rebuild.sh, git will usually nag you that you have made changes, so in that case, it may be more beneficial to make a backup before doing anything like stashing the changes, or just easier to backup and install the new version from scratch and add your changes back in. Using Git commands has some strange behavior when it comes to git stash and so on, its a level of voodoo I don't fully understand.
 

lwvmobile

DSD-FME
Joined
Apr 26, 2020
Messages
1,297
Location
Lafayette County, FL
Just a quick update, but over the past two weeks I've addressed a few bugs, notably, a bug when compiling and running on Raspberry Pi (64-bit Bullseye) and potentially other versions/distros, that I didn't have a few variables initialized, so the Pulse Audio output stream was not opening by default in those builds. I've also found out this issue also happened if one were to compile the Pulse Audio branch inside of Cygwin, so if anybody had attempted to compile and use the Pulse Audio branch on RPi or inside of Cygwin (it does actually work inside the Cygwin environment now!) they would have issues with either crashing upon opening or just no output stream to listen to.

I invite anybody who is interested to give the Pulse Audio branch a testing on Raspberry Pi if by chance you tried it and it didn't work previously, or if you are really courageous/crazy enough to attempt to build it for yourself inside of Cygwin. I do have loose documentation for building inside of Cygwin, but installing the dependencies from the setup utility is the major hurdle I've found for the few who have attempted it and reported back to me.

Again, the self installer on Ubuntu/Debian/RPi (now working correctly on RPi and working correctly!) can be run in the terminal with just three steps. Best practice is to run this in a new empty folder in your home directory.

Code:
wget https://raw.githubusercontent.com/lwvmobile/dsd-fme/pulseaudio/install.sh
chmod +x install.sh
./install.sh

And if you are really brave, you can try to manually build the Pulse Audio branch inside of Cygwin by cloning the Pulse Audio branch and then looking at the docs in the Cygwin branch, mainly the screenshots of the dependencies in the cygwin cheatsheet folder.



Also, don't forget this step at the end for Cygwin builds

After make has successfully completed, go into the mbelib/build folder and copy cygmbe-1.dll and paste it into the dsd-fme/build folder.
 
Top