DSD improvements I am working on

Status
Not open for further replies.

EricCottrell

Member
Premium Subscriber
Joined
Nov 8, 2002
Messages
2,413
Location
Boston, Ma
Hello,

I fixed a number of issues I had with dsd and improved the program.

I added support for ALSA because it plays better with the PulseAudio used in recent linux distributions than OSS does. I added table-based error correction to DMR and can decode most CSBK and TLC frames. I use an USB3000 instead of mbelib.

I am currently working on improving the display of information to get away from the "dump everything to the screen" mode where data just flies past. This uses the ncurses library. So far I just display the dump messages to different areas of the screen and do not display some duplicate information.

For DMR it displays basic sync and slot information on the first line. The Cach and Color Code displays on the second line. The rest of the screen is split into three areas, with the first area displaying control channel information (Connect Plus Neighbor and goto information), the second area displaying slot 0 information and the third area displaying slot 1 information. I need to extend and refine it, but this already works better for me. The graphic I attached shows an example of current decoding and display of a Connect Plus control channel.

A potential problem is ncurses may not work for cygwin under windows or additional packages need to be added. I am not familar with using cygwin since I run under Linux. I am thinking of having the option of both the old display format and a new curses-based format.

I am more familiar with Mercurial and Bitbucket than Git and Github. I do not know the best way to contribute code yet.

73 Eric
 

Attachments

  • ncurses.png
    ncurses.png
    130.6 KB · Views: 4,901

groovy

Member
Joined
Jul 10, 2013
Messages
29
It is nice to know that there is still work progressing on the open-source project!

There are a few people watching here who submit pull requests to the github branch; I'm sure they could comment on how to best do it with git.

One thing that comes up is the lack of ECC support for the p25 (HDU and LDU2) frames. That causes the key and encryption algorithm bits to be corrupted, and results in dropoffs. The DSD+ author fixed that, but isn't releasing code. Sadly, the mbelib ecc.c has most of the required routines (not coded efficiently ... I digress), but dsd doesn't use them, rather it makes liberal use of skipDibit to skip the ECC data- see https://github.com/szechyjs/dsd/blob/master/p25p1_hdu.c line 48 and https://github.com/szechyjs/dsd/blob/master/p25p1_ldu2.c line 134 for example. This results in inferior decode of p25. If you're comfortable with ECC, would you be interested in tackling these in the open source branch? (The former is a shortened golay, the latter a shortened hamming- spec is here http://www.qsl.net/kb9mwr/projects/...aa-a-project_25-fdma-common_air_interface.pdf)

I know some of the developers test on cygwin, so the ncurses stuff can be tested by them.

Meanwhile I'm going to see about getting the optimizations ported from the half-rate codec to the full-rate one, and maybe get table-based cosf/sqrt code in there, both to improve CPU usage in mbelib. I think some of the bad experience people are having is simply a result of poorly optimized code.

Hello,

I fixed a number of issues I had with dsd and improved the program.
[...]
A potential problem is ncurses may not work for cygwin under windows or additional packages need to be added. I am not familar with using cygwin since I run under Linux. I am thinking of having the option of both the old display format and a new curses-based format.
[...]
I am more familiar with Mercurial and Bitbucket than Git and Github. I do not know the best way to contribute code yet.

73 Eric
 

EricCottrell

Member
Premium Subscriber
Joined
Nov 8, 2002
Messages
2,413
Location
Boston, Ma
Hello,

I implemented table-based Golay and Hamming error correction for DMR, so I can look at that for P25. There are so many areas for improvement. I want to also start decoding Nxdn as well, but that uses a more complex punched convolutional code and needs a Viterbi Decoder.

73 Eric
 

mtindor

OH/WV DB Admin
Database Admin
Joined
Dec 5, 2006
Messages
10,366
Location
Carroll Co OH / EN90LN
Eric,

I for one want to say thank you for providing all of the very useful information that you have thus far regarding DSD. I've incorporated all of your proposed 'fixes' that you've posted on the forums to the older pristine DSD code, and it had made a real difference on decoding in DSD.

Of course, right now I'm using DSDPlus, but I'm always interested in seeing DSD improve. Every one of your suggested fixes / improvements [complete with code] have improved DSD performance significantly.

I look forward to anything new you have to add to it in the future.

Mike
 

deboe

Member
Joined
Dec 19, 2002
Messages
142
Location
0x045C
Eric,
I started working on a native win32 build from szechyjs' code in github.
I get a clean complie, no audio support yet. I started working on using portaudio to have audio support on all platforms.

I just pushed my win32 branch to github ( https://github.com/dreinhold/dsd/tree/win32 ) **THIS is pre alpha, and cannot even open the sound drivers. but it complies **

I have been using mingwin ( mxe ) to cross complie the win32 exe's from linux. There is ncurses support for the native win32 apps in mxe so that's good.

As for using git, if you keep it simple, you should be fine.
github has some documentaion on uing git, I would just jump right to collaborating section
https://help.github.com/categories/63/articles
 

Boatanchor

Member
Joined
Jul 17, 2011
Messages
991
To those working on DSD, I'm wondering if you could address the poor C4FM performance and the audio frequency response?

These comments relate only to P25 Phase 1 only.

Issues/problems:

C4FM implementation:

C4FM decoding is awful! Many people (myself included) get terrible/non existent decode on C4FM, but near perfect decode when forced into GFSK mode. Regardless of cpu power.

Recovered audio:

While I actually prefer the (0-3Khz) audio response of DSD, compared to the somewhat tinny and compressed audio of DSDPlus, there are some things that could be tweaked to improve this further.

* It would be nice if the top end LPF/roll off could be extended and made adjustable of between 3-4Khz with an additional switch.

* When the recovered audio output of DSD is viewed on a Spectrum analyser program, DSD has got some nasty low level harmonics and noises extending from 3Khz way up to 10Khz. This is unlike DSDPlus, that has a much cleaner output.

The harmonics may not be heard on laptop speakers, but the high frequency noise/harmonics could be an issue when listening with headphones or on decent speakers.

Any, just some suggestions :)

It's great to see ongoing development into these fantastic programs.
 

inigo88

California DB Admin
Database Admin
Joined
Oct 31, 2004
Messages
1,990
Location
San Diego, CA
Hey Eric, this looks really really good, and I just wanted to thank you for your significant contributions over the years. I was a big fan of your LTRDump program back in the day for PassPort decoding. I'm curious if something with a similar UI could ever be made for mototrbo and NXDN trunking. With your recent discoveries on how Capacity Plus and Connect Plus announce activity on other voice channels, I think it's possible to progress from line by line decoding to a more trunker-like format. It looks like you are already headed this direction with your latest modification to DSD. :)
 

EricCottrell

Member
Premium Subscriber
Joined
Nov 8, 2002
Messages
2,413
Location
Boston, Ma
Hello,

The display stuff is not finished yet. It is mainly an experiment to see if I could do screen formatting required for a trunking program. I wil finish it because I find the display is better.

I could work on improving P25, but doing it properly requires a redesign of dsd. I could just put in the proper filtering, but that will degrade other modes. It appears the author of DSDPlus put in the proper filtering for P25, and users have found DMR suffers. Just implementing error correction for P25 may help somewhat.

I would also like to work on decoding Nxdn. This will require a Viterbi decoder to do the error correction and extract the data bits.

I am interested in doing a program to decode Mototrbo and NXDN trunking. The old trunker framework I used in the past is too limited. The dsd program is designed to decode voice, so it is not suitable. It will require a new program from the ground up, so I am doing some experiments, and preliminary design work. This will take months of work.

The program should decode both the control channel and the desired traffic channel. This way you can use one stereo line input (left = control channel, right = traffic channel). I am also thinking of supporting conventional digital voice as it could be considered a single frequency system. There are still groups and radio ids used. The program will likely end up being a DSDPlusPlus program, but I will not use the DSD name.

There are limited ways to find the traffic channel frequencies on most analog trunked systems. The typical function is finding the LCN order, which depends on knowing the frequencies associated with the system. There could be a program that decodes the subaudible data, and detects if a frequency was part of a trunked system, but there is no information linking it to a particular system. Some digital systems, like Capacity Plus, have a similar problem, although it might be possible for a program to figure it out over time by comparing data between the rest channel and other found channels.

Connect Plus is an interesting case, because the network and site are identified on the traffic channels. You could have a feature where if you tune into a new Connect Plus control channel, the program could search the band looking for the unknown traffic channels. Of course it will not work if the system is always idle.

I use a Icom R2500 with DSD and other trunking programs. I like it because it is two separate receivers in one package. I modified the packet outputs for better decoding. I have an adapter that converts the two packet outputs into a single stereo connector, so I have the hardware to use for testing.

I recently bought an Intel Galileo development board. This is basically a tiny 400 MHz Pentium system that can run Linux. I am planning on trying dsd to see if it has enough processing power for dsd to work.

I got a lot of experiments and projects to keep me busy. I also need to clean up some of the code and submit it to the dsd repository.

73 Eric
 

groovy

Member
Joined
Jul 10, 2013
Messages
29
Since it came up on P25- In addition to the unimplemented golay, Reed Solomon and hamming protection for the kid/algid fields that impede the encryption muting, I just realized that the DUID/NID ECC (using BCH) is unimplemented as well. This means that a single flipped bit in the DUID field causes the whole block to be lost. I think we could get reasonable p25 performance by fixing the ECC and perhaps filtering. I'm not convinced that a complete rewrite is needed.

Whoever wrote the P25 spec was an ECC mad scientist. Extended golay, Shortened golay, hamming, shortened hamming, BCH, Reed Solomon, Trellis codes, and CRCs are all required to fully decode P25. Wow.
 

morfis

Member
Joined
Jan 24, 2004
Messages
1,631
I use a Icom R2500 with DSD and other trunking programs. I like it because it is two separate receivers in one package. I modified the packet outputs for better decoding. I have an adapter that converts the two packet outputs into a single stereo connector, so I have the hardware to use for testing.

Eric,
Appologies for going off topic a little but in what way have you modified the R2500 'packet' output? I've recently started looking at P25 traffic but haven't tried any of my PCRs as the signal source as yet.
 

Ed_F

Member
Joined
Jan 6, 2014
Messages
5
ECC support for the NID

One thing that comes up is the lack of ECC support for the p25 (HDU and LDU2) frames. That causes the key and encryption algorithm bits to be corrupted, and results in dropoffs.

Hi there. I got a working modification of DSD that does ECC on the NID field. For the BCH implementation I'm using ITPP (Welcome to IT++!). If anyone is interested I can submit the patch or advise on the BCH configuration, it took me a while to figure it out.

Cheers,

Ed
 

groovy

Member
Joined
Jul 10, 2013
Messages
29
I (for one) would be interested in seeing the code- Would you be able to post the diff somewhere or attach it here?

Thanks,
--groovy

Hi there. I got a working modification of DSD that does ECC on the NID field. For the BCH implementation I'm using ITPP (Welcome to IT++!). If anyone is interested I can submit the patch or advise on the BCH configuration, it took me a while to figure it out.

Cheers,

Ed
 

EricCottrell

Member
Premium Subscriber
Joined
Nov 8, 2002
Messages
2,413
Location
Boston, Ma
Hello,

The IT++ code looks interesting. NXDN makes use of Punctured Convolutional Code and the library may make it easier to decode it.

73 Eric
 

Ed_F

Member
Joined
Jan 6, 2014
Messages
5
The error correction on the header fields using Golay and Reed Solomon is now implemented and seems to work.

However there is still way to go with the quality of the demodulated audio. I agree that the C4FM stage should be improved. I wonder if anyone has tried the GNU Radio compatible fsk4_demod_ff.cc (op25.osmocom.org) ??

Kind regards,

Ed
 

groovy

Member
Joined
Jul 10, 2013
Messages
29
Ed- Did you try the Golay implementation from mbelib ecc.c? It might negate the need for a second implementation if it produces the same results. The only difference between golay 23 and golay 24 is the parity bit- And mbelib treats everything as 23 and simply ignores the parity bit.
 
Status
Not open for further replies.
Top