DSD 1.6? Closed fork or open source

[gariac]

The title says it all. Is there source code for DSD 1.6 for the linux users? If the program has been forked and made closed source, fine, but call it something else.

 

[mtindor]

The title says it all. Is there source code for DSD 1.6 for the linux users? If the program has been forked and made closed source, fine, but call it something else.

Source code is only [known to be] available for DSD 1.4.1.

m

 

[racingfan360]

I think woodpecker put the work into creating DSD 1.6 but he doesn't seem to respond to any posts or PMs, let alone those related to sharing the source for DSD 1.6....a real shame as I bet there is a huge performance advantage to be gained over 1.4.1 :-(

 

[mtindor]

I think woodpecker put the work into creating DSD 1.6 but he doesn't seem to respond to any posts or PMs, let alone those related to sharing the source for DSD 1.6....a real shame as I bet there is a huge performance advantage to be gained over 1.4.1 :-(

There is a very significant performance increase on weak signals [and even not so weak] when using 1.6. And, it no longer crashes after a period of time while monitoring.

It is what it is though. I'm grateful to Woodpecker for making those improvements and certainly don't hold it against him for not providing the source. I'll take what I can get.

Mike

 

[gariac]

Well if the person is a total black hole, I think I'll pass on the download. Who knows what virus it contains? Also, I've never been a fan of cygwin.

I only used it for trbo. I played with Apco-25 demod, but I have a scanner for that. ;-) Trbo is getting to be annoying. I hope a scanner comes out for it. It wouldn't surprise me to find a police team of some sort using a trbo commercial system to hide under the radar.

I spent a bit of time tracking down some TRBO around the Nevada Test Site, thinking I found a contractor. It turned out to be the bus service for the facility. Well yeah, a contractor, but not exactly an exciting one!

At this point, we really don't know if it has been tweaked since we can't see the source code. It may only be perceived to be better, unless someone has done carefully controlled A/B testing.

 

[n5ims]

There was talk that there were some patent threats over some of the code, which is why the newer versions were released only in binary (they were reportedly compiled in areas where the patent threat wasn't reachable) as a way to remove the threat. The RR WIKI makes a passing remark about this issue.

Patent Issues with mbelib

While DSD was intended to be patent-free, mbelib describes functions that may be covered by one or more U.S. patents owned by DVSI Inc. The source code itself should not be infringing as it merely describes possible methods of implementation. Compiling or using mbelib may infringe on patents rights and/or require licensing. It is unknown if DVSI will sell licenses for software that uses mbelib. If you do not have a license and are in a jurisdiction protected by the DVSI patents you should not compile or use this source code.

 

[n4yek]

It may only be perceived to be better, unless someone has done carefully controlled A/B testing.

Personal experience is it works much better for me, before the program would crash after about 5 minutes of monitoring trbo, now it no longer does that.

 

[mtindor]

\
At this point, we really don't know if it has been tweaked since we can't see the source code. It may only be perceived to be better, unless someone has done carefully controlled A/B testing.

I've got no figures to give you for an A/B comparison, nor am I inclined to gather any.

1.6 definitely has two notable differences, both of them advertised. As far as I'm concerned, there is no discounting the two facts below:

1. It not longer crashes after a short period of operation. [and it is documented in the forum as to what the root cause of this was, as well as what was fixed].

2. Much better performance on weak signal DMR. I don't use it to monitor P25, but the filter is claimed to be active for P25 as well. This filtering addition was also discussed in back-and-forth conversations on the forum. I have put 1.6 through its paces on DMR systems, and there is a huge difference between quality of decodes on 1.6 vs 1.4.

Sure, anything else could have been added, and sure there could be a virus even if virustotals.com claims it's totally clean. It's simple. Don't use it if you don't trust it . I don't think trust is your issue though.

The following post outlines what was fixed / added in 1.6:

http://forums.radioreference.com/di...-new-dsd-windows-build-1-6-a.html#post1799475

Mike

 

[mtindor]

Personal experience is it works much better for me, before the program would crash after about 5 minutes of monitoring trbo, now it no longer does that.

Same results here with 1.6. I've had DSD running for a couple hours at a time and haven't had a segfault.

M

 

[mtindor]

There was talk that there were some patent threats over some of the code, which is why the newer versions were released only in binary (they were reportedly compiled in areas where the patent threat wasn't reachable) as a way to remove the threat. The RR WIKI makes a passing remark about this issue.

That's interesting. I was mildly curious as to why source wasn't released, but I'm not a programmer and couldn't do anything useful with the code [other than compile it] if I had it. So the absence of source didn't bother me.

Mike

 

[gariac]

There was talk that there were some patent threats over some of the code, which is why the newer versions were released only in binary (they were reportedly compiled in areas where the patent threat wasn't reachable) as a way to remove the threat. The RR WIKI makes a passing remark about this issue.

Patents are enforced where the device is used, not where it is built. I could build the device in some lawless country (Somalia) but couldn't sell it in a country where it is patented. So I don't buy that claim.

If you read the Red Hat licenes agreement on Cygwin, it is not entirely clear that you can ship your program inside the Cygwin environment. I haven't installed rev 1.6, so I don't know if it first installs cygwin, then DSD, which is the proper way to do it.

The linux version never crashed. I'm sure if we had the code for the 1.6 version, it would work well under linux too. It is just horrible that this went closed source, but that is the programmers right. However, it should be called something else since we have established it is a fork.

 

[slicerwizard]

The linux version never crashed.

Any unmodified version will crash if you park it on a DMR control channel.

It is just horrible that this went closed source, but that is the programmers (sic) right. However, it should be called something else since we have established it is a fork.

No need to get your panties in a bunch. Any programmer can add woodpecker's changes to the source code. Let's take a look at his list...

Fixes:-
1. Golay 23,12 syndrome table corrected with missing entries 1889 to 2048.
2. Changed the crazy dibit buffer which filled for ever until it ran out of allocated memory then caused a segmentation error. The buffer is now allowed to fill to 90% before being returned to the initial pointer value where it over writes the old samples, it seems to work ok and no segmentation errors now.
3. Input level is now calculated differently, before when it reported 50% the soundcard would be fully overloaded. With the new method aim for 30% inlvl

New features:-
1. Auto mutes P25 encrypted signals by default, run with option -pu to unmute if you want the r2d2 sounds.
2. Raised cosine filters can be disabled by running with option -l that's a lowecase L. Only useful if you want to see the difference the filters make.


None of that is rocket surgery, so let's go through it:

1. Golay 23,12 syndrome table corrected with missing entries 1889 to 2048.

The original table actually contains 1839 entries, not 1888. The missing values are:

257, 512, 192, 2048, 33, 512, 512, 512, 14, 2048,
264, 2048, 2048, 512, 1024, 2048, 80, 24, 258,
2624, 4, 258, 258, 32, 258, 1025, 4, 4, 4, 64,
258, 136, 4, 24, 24, 24, 33, 24, 258, 1028, 2176,
24, 640, 256, 4, 2080, 1024, 515, 80, 1025, 192,
32, 8, 32, 258, 32, 32, 1025, 1025, 1025, 4, 1025,
2568, 32, 80, 24, 2052, 130, 1792, 512, 1, 32, 80,
1025, 34, 2048, 80, 388, 80, 80, 80, 1026, 2064,
1026, 1026, 512, 40, 1026, 68, 2064, 2064, 1026,
2064, 64, 2064, 136, 257, 512, 260, 1026, 33, 512,
512, 512, 2176, 129, 2064, 256, 584, 512, 1024,
52, 2, 512, 192, 1026, 8, 512, 512, 512, 257, 12,
2064, 96, 257, 512, 257, 257, 257, 512, 512, 512,
16, 512, 512, 512, 512, 512, 34, 2048, 1156, 512,
512, 512, 257, 164, 513, 1026, 8, 64, 258, 17,
2176, 64, 2064, 256, 4, 64, 64, 64, 1568, 24,
1088, 256, 2176, 512, 2176, 2176, 2176, 256, 34,
256, 256, 64, 13, 256, 2176, 2304, 8, 8, 8, 512,
1044, 32, 8, 1025, 34, 656, 8, 64, 128, 2054, 257,
512, 34, 69, 8, 512, 512, 512, 2176, 34, 34, 256,
34, 512, 34, 1032, 80


2. Changed the crazy dibit buffer which filled for ever until it ran out of allocated memory then caused a segmentation error. The buffer is now allowed to fill to 90% before being returned to the initial pointer value where it over writes the old samples, it seems to work ok and no segmentation errors now.

Ok, so in dsd.h, there it is - dibit_buf_p; hm, 34 references in 7 .c files; the ones that cause it to overflow are the increment (++) operations in dsd_dibit.c and dsd_frame_sync.c.

In getDibit(), all of the increments are in the same chunk of code, so before // determine dibit state, just add if (state->dibit_buf_p > state->dibit_buf + 900000) state->dibit_buf_p = state->dibit_buf + 200;

In getFrameSync(), ditto.


3. Input level is now calculated differently, before when it reported 50% the soundcard would be fully overloaded. With the new method aim for 30% inlvl

Well, this one is too easy - wherever the input level is reported, just scale it up by 2x...

inlvl% is reported in six locations (sigh...) in dsd_frame.c and it's always the same code:

level = (int) (((float) state->max / (float) 32768) * (float) 100);

So change them all to level = state->max / 164;


1. Auto mutes P25 encrypted signals by default, run with option -pu to unmute if you want the r2d2 sounds.

Also fairly easy. Encrypted audio will have non-zero values in the HDU KID field, so just save that KID value and only decode P25 audio if KID==0. So add int P25KID to the state structure and in p25p1_hdu.c, after skipDibit (opts, state, 160);, add state->P25KID = strtol(kid, NULL, 2); and in the LDU functions, preface the processMbeFrame calls with if (state->P25KID == 0)


2. Raised cosine filters can be disabled by running with option -l that's a lowecase L. Only useful if you want to see the difference the filters make.

In the getSymbol() function, you'll find:

result = read (opts->audio_in_fd, &sample, 2);

There's your 16 bit signed audio samples being read one at a time, so just pass each sample value through a low pass filter.

IIRC, woodpecker posted the filter parameters he used, so plug them into a filter generator and you're done. Raised Cosine Filters should do the trick.

After plugging some plausible numbers into that website, I get output that leads to this:

result = read (opts->audio_in_fd, &sample, 2);

{
#define NZEROS 60
#define GAIN 7.423339364e+00

static float xv[NZEROS+1];

static float xcoeffs[] =
{ -0.0083649323, -0.0265444850, -0.0428141462, -0.0537571943,
-0.0564141052, -0.0489161045, -0.0310068662, -0.0043393881,
+0.0275375106, +0.0595423283, +0.0857543325, +0.1003565948,
+0.0986944931, +0.0782804830, +0.0395670487, -0.0136691535,
-0.0744390415, -0.1331834575, -0.1788967208, -0.2005995448,
-0.1889627181, -0.1378439993, -0.0454976231, +0.0847488694,
+0.2444859269, +0.4209222342, +0.5982295474, +0.7593684540,
+0.8881539892, +0.9712773915, +0.9999999166, +0.9712773915,
+0.8881539892, +0.7593684540, +0.5982295474, +0.4209222342,
+0.2444859269, +0.0847488694, -0.0454976231, -0.1378439993,
-0.1889627181, -0.2005995448, -0.1788967208, -0.1331834575,
-0.0744390415, -0.0136691535, +0.0395670487, +0.0782804830,
+0.0986944931, +0.1003565948, +0.0857543325, +0.0595423283,
+0.0275375106, -0.0043393881, -0.0310068662, -0.0489161045,
-0.0564141052, -0.0537571943, -0.0428141462, -0.0265444850,
-0.0083649323,
};

float sum; int i;

for (i = 0; i < NZEROS; i++)
xv = xv[i+1];

xv[NZEROS] = sample; // unfiltered sample in
sum = 0.0;

for (i = 0; i <= NZEROS; i++)
sum += (xcoeffs * xv);

sample = sum / GAIN; // filtered sample out
}


And now you have your source code...

 

[gariac]

--------------------
1. Auto mutes P25 encrypted signals by default, run with option -pu to unmute if you want the r2d2 sounds.

Also fairly easy. Encrypted audio will have non-zero values in the HDU KID field, so just save that KID value and only decode P25 audio if KID==0. So add int P25KID to the state structure and in p25p1_hdu.c, after skipDibit (opts, state, 160);, add state->P25KID = strtol(kid, NULL, 2); and in the LDU functions, preface the processMbeFrame calls with if (state->P25KID == 0)
--------------------------------
This is clear:
"after skipDibit (opts, state, 160);, add state->P25KID = strtol(kid, NULL, 2);"
but I don't follow this:
"and in the LDU functions, preface the processMbeFrame calls with if (state->P25KID == 0)"

---------------

Regarding the Golay table, I can't find it. I grepped all sorts of likely expressions and then vimed every file looking for a very large table. Where is it located?

 

[mtindor]

ecc_const.h (from mbelib) ?

M

Regarding the Golay table, I can't find it. I grepped all sorts of likely expressions and then vimed every file looking for a very large table. Where is it located?

 

[SCPD]

I don't follow this:
"and in the LDU functions, preface the processMbeFrame calls with if (state->P25KID == 0)"

Look in p25p1_ldu1.c and p25p1_ldu2.c ...

 

[mtindor]

1. Golay 23,12 syndrome table corrected with missing entries 1889 to 2048.

The original table actually contains 1839 entries, not 1888. The missing values are:

257, 512, 192, 2048, 33, 512, 512, 512, 14, 2048,
264, 2048, 2048, 512, 1024, 2048, 80, 24, 258,
2624, 4, 258, 258, 32, 258, 1025, 4, 4, 4, 64,
258, 136, 4, 24, 24, 24, 33, 24, 258, 1028, 2176,
24, 640, 256, 4, 2080, 1024, 515, 80, 1025, 192,
32, 8, 32, 258, 32, 32, 1025, 1025, 1025, 4, 1025,
2568, 32, 80, 24, 2052, 130, 1792, 512, 1, 32, 80,
1025, 34, 2048, 80, 388, 80, 80, 80, 1026, 2064,
1026, 1026, 512, 40, 1026, 68, 2064, 2064, 1026,
2064, 64, 2064, 136, 257, 512, 260, 1026, 33, 512,
512, 512, 2176, 129, 2064, 256, 584, 512, 1024,
52, 2, 512, 192, 1026, 8, 512, 512, 512, 257, 12,
2064, 96, 257, 512, 257, 257, 257, 512, 512, 512,
16, 512, 512, 512, 512, 512, 34, 2048, 1156, 512,
512, 512, 257, 164, 513, 1026, 8, 64, 258, 17,
2176, 64, 2064, 256, 4, 64, 64, 64, 1568, 24,
1088, 256, 2176, 512, 2176, 2176, 2176, 256, 34,
256, 256, 64, 13, 256, 2176, 2304, 8, 8, 8, 512,
1044, 32, 8, 1025, 34, 656, 8, 64, 128, 2054, 257,
512, 34, 69, 8, 512, 512, 512, 2176, 34, 34, 256,
34, 512, 34, 1032, 80

For those people like myself who really have no idea how those numbers are generated or how they are supposed to fit in, do you mean that the original table only contained 1839 entries but should have contained 2048 entries, and that the other 209 entries [that you provided above] should be added on to the end of the lookup table?

The "1888" and "1889" references above confused me til I checked the count of the original table [1839] and added the count from your addition [209] and came up with 2048.

I just tacked your table entries directly onto the end of the lookup table, and I'm assuming that's how it is supposed to be done.

Mike

 

[mtindor]

And now you have your source code...

Thanks, Andrew. I made all the changes to the DSD 1.4 source and it compiled just fine and works great. Of course I'm running on Windows and already have Woodpeckers 1.6.1. But somehow, someway it seems to me that what I just compiled works even better. Maybe the values you put in for the filter were a little different than what Woodpecker used. Or maybe it's just my imagination. But it works. So Gariac shouldn't have a problem.

Mike

 

[racingfan360]

So I really appreciate the work on this and info from slicerwizard....like Mike, I'm not an expert when it comes to (re)writing c code, so I'd sure appreciate if someone could verify i have the right steps here, to describe in a simple form of what modifications are required:

1. Edit the ecc_const.h file in the mbelib-1.2.3 directory, and add the following text (and a comma after the last 136 item) to the end of the large lookup table that ends 514, 18, 36, 136

257, 512, 192, 2048, 33, 512, 512, 512, 14, 2048,
264, 2048, 2048, 512, 1024, 2048, 80, 24, 258,
2624, 4, 258, 258, 32, 258, 1025, 4, 4, 4, 64,
258, 136, 4, 24, 24, 24, 33, 24, 258, 1028, 2176,
24, 640, 256, 4, 2080, 1024, 515, 80, 1025, 192,
32, 8, 32, 258, 32, 32, 1025, 1025, 1025, 4, 1025,
2568, 32, 80, 24, 2052, 130, 1792, 512, 1, 32, 80,
1025, 34, 2048, 80, 388, 80, 80, 80, 1026, 2064,
1026, 1026, 512, 40, 1026, 68, 2064, 2064, 1026,
2064, 64, 2064, 136, 257, 512, 260, 1026, 33, 512,
512, 512, 2176, 129, 2064, 256, 584, 512, 1024,
52, 2, 512, 192, 1026, 8, 512, 512, 512, 257, 12,
2064, 96, 257, 512, 257, 257, 257, 512, 512, 512,
16, 512, 512, 512, 512, 512, 34, 2048, 1156, 512,
512, 512, 257, 164, 513, 1026, 8, 64, 258, 17,
2176, 64, 2064, 256, 4, 64, 64, 64, 1568, 24,
1088, 256, 2176, 512, 2176, 2176, 2176, 256, 34,
256, 256, 64, 13, 256, 2176, 2304, 8, 8, 8, 512,
1044, 32, 8, 1025, 34, 656, 8, 64, 128, 2054, 257,
512, 34, 69, 8, 512, 512, 512, 2176, 34, 34, 256,
34, 512, 34, 1032, 80

2. In the dsd-1.4.1 directory, edit the dsd_dibit.c file.
find the line:
// determine dibit state
replace it with:
if (state->dibit_buf_p > state->dibit_buf + 900000) state->dibit_buf_p = state->dibit_buf + 200;
// determine dibit state

3. In the dsd-1.4.1 directory, edit the dsd_frame_sync.c file.
find the line:
// determine dibit state
replace it with:
if (state->dibit_buf_p > state->dibit_buf + 900000) state->dibit_buf_p = state->dibit_buf + 200;
// determine dibit state

4. In the dsd-1.4.1 directory, edit the dsd_frame.c file.
Locate the six instances where level = (int) (((float) state->max / (float) 32768) * (float) 100); occurs and replace them with level = state->max / 164;

[Note - I have deliberately skipped the Auto mutes P25 encrypted signals feature as I'm not interested in this - please spell the specific steps to take if you want here]

5. In the dsd-1.4.1 directory, edit the dsd_symbol.c file.
Locate:
result = read (opts->audio_in_fd, &sample, 2);
Replace it with:
result = read (opts->audio_in_fd, &sample, 2);

{
#define NZEROS 60
#define GAIN 7.423339364e+00

static float xv[NZEROS+1];

static float xcoeffs[] =
{ -0.0083649323, -0.0265444850, -0.0428141462, -0.0537571943,
-0.0564141052, -0.0489161045, -0.0310068662, -0.0043393881,
+0.0275375106, +0.0595423283, +0.0857543325, +0.1003565948,
+0.0986944931, +0.0782804830, +0.0395670487, -0.0136691535,
-0.0744390415, -0.1331834575, -0.1788967208, -0.2005995448,
-0.1889627181, -0.1378439993, -0.0454976231, +0.0847488694,
+0.2444859269, +0.4209222342, +0.5982295474, +0.7593684540,
+0.8881539892, +0.9712773915, +0.9999999166, +0.9712773915,
+0.8881539892, +0.7593684540, +0.5982295474, +0.4209222342,
+0.2444859269, +0.0847488694, -0.0454976231, -0.1378439993,
-0.1889627181, -0.2005995448, -0.1788967208, -0.1331834575,
-0.0744390415, -0.0136691535, +0.0395670487, +0.0782804830,
+0.0986944931, +0.1003565948, +0.0857543325, +0.0595423283,
+0.0275375106, -0.0043393881, -0.0310068662, -0.0489161045,
-0.0564141052, -0.0537571943, -0.0428141462, -0.0265444850,
-0.0083649323,
};

float sum; int i;

for (i = 0; i < NZEROS; i++)
xv = xv[i+1];

xv[NZEROS] = sample; // unfiltered sample in
sum = 0.0;

for (i = 0; i <= NZEROS; i++)
sum += (xcoeffs * xv);

sample = sum / GAIN; // filtered sample out
}

6. I assume I then cd to the mbelib-1.2.3 directory, sudo make and then sudo make install


7. I assume I then cd to the dsd-1.4.1 directory, sudo make and then sudo make install


8. I assuming I should then be able to run DSD just as before, but with these new features.

Any corrections/updated would be much appreciated - before I screw up my DSD install !!

Thanks,

Jim

 

[SCPD]

Audi is only needed for the install.

 

[mtindor]

For anyone adding these features based upon what Slicer said, you might also want to include a 'fix' from ericcottrell in another post:

http://forums.radioreference.com/1879375-post298.html

Mike