Unattended Logging Of Whistler Scanners

[sibbley]

Thanks Don, I'll give that a try this weekend.

 

[AggieCon]

Is that reading the files based on the ARINDEX.DAT file listing?

 

[DonS]

Is that reading the files based on the ARINDEX.DAT file listing?

Yes, just like the scanner and PC app do. It reads 4-byte long ints from that file, generates the directory and filename from that number, and opens the resulting AU file.

Of course, I could've searched for matching directory and filenames and opened every AU file present (without regard for whether it was listed in ARINDEX.DAT), but I was going for speed. Mostly to prove to myself that the Whistler PC app really isn't any slower than a lean console app would be (and it's not: after first inserting a card into a USB card reader, my little utility takes 230 seconds to parse 23498 .AU files; the PC app takes 240 seconds).

EDIT: just realized I forgot to include the TSYS type in the output. Fixed.

 

[AggieCon]

I achieved 2837 files per second, also from a card reader. Pretty much the same speed as EZ Scan.

However, I have a lot of duplicates due to a bad ARINDEX.DAT, I assume. I'll try to motivate myself to have the scanner or EZ Scan rebuild the list. About 25% of the files on my CSV were duplicated. The number matched EZ Scan, so I just realized that all along I had duplicate recordings in EZ Scan.

 

[AggieCon]

Moral of the story: It's the scanner that is slow to communicate not EZ Scan that slows things down.

 

[DonS]

I achieved 2837 files per second, also from a card reader. Pretty much the same speed as EZ Scan.

However, I have a lot of duplicates due to a bad ARINDEX.DAT, I assume. I'll try to motivate myself to have the scanner or EZ Scan rebuild the list. About 25% of the files on my CSV were duplicated. The number matched EZ Scan, so I just realized that all along I had duplicate recordings in EZ Scan.

Or you could rebuild the list yourself.

#include <stdio.h>

int main( int argc, char *argv[] )
  {
  unsigned long filenum, dir, num;
  unsigned char fnbuf[20];
  FILE *fp, *afp;

  fp = fopen( "X:\\REC\\ARINDEX.DAT", "wb" );
  fwrite( "ARINDEX", 1, 8, fp );
  for ( dir=0; dir<100000; dir++ )
    {
    for ( num=0; num<200; num++ )
      {
      sprintf( fnbuf, "X:\\REC\\%05ld\\%03ld.AU", dir, num );
      if ( NULL != (afp = fopen( fnbuf, "rb" )) )
        {
        filenum = dir * 200 + num;
        fwrite( &filenum, 4, 1, fp );
        fclose( afp );
        }
      }
    }
  fclose( fp );

  return 0;
  }

(For speed, replace the 100000 limit of 'dir' with the highest directory number you have in REC, plus 1).

 

[SOFA_KING]

OK...Sorry Don. My frustration has been building over a few years. No acknowledgement on the issues reported, nor on suggestions made, so it's like no one is listening, cares, or anyone who could do something about them ever even heard them. That's the frustration. Thanks for that app. Nice of you to share that.

I have now over 114,000 data recordings (tracking long term trends) and will have to give that a try. Speed on a large amount of data like that is understandable, I guess. Audio recordings are somewhat large. But having all information in a CSV sortable file...and now having it out of EZscan is much better.

Phil

 

[Swipesy]

I am able to see the detail of the AUParse in the command window but when it output to a log file it creates an Excel.CSV but there is no data. My SD Card is in Drive E. and the AUParse.exe is in C:\. So I used C:\AUParse E: > Log.csv and get a blank spreadsheet. Am I missing something here?

Thanks

PS I am using PSR-800

 

[DonS]

I am able to see the detail of the AUParse in the command window but when it output to a log file it creates an Excel.CSV but there is no data. My SD Card is in Drive E. and the AUParse.exe is in C:\. So I used C:\AUParse E: > Log.csv and get a blank spreadsheet. Am I missing something here?

Thanks

PS I am using PSR-800

The PSR-800 doesn't have in its .AU header data a couple of fields that were added with the Whistler scanners (though the header size is unchanged - I reserved 320 bytes back in PSR-800 days). That might be messing up the CSV output, but if you can see it in the console window it should appear in the file when you redirect stdout.

Can you post a small sample of what appears in the console window? Maybe the first 10 lines, or so?

 

[DonS]

OK...Sorry Don. My frustration has been building over a few years. No acknowledgement on the issues reported, nor on suggestions made, so it's like no one is listening, cares, or anyone who could do something about them ever even heard them. That's the frustration. Thanks for that app. Nice of you to share that.

I have now over 114,000 data recordings (tracking long term trends) and will have to give that a try. Speed on a large amount of data like that is understandable, I guess. Audio recordings are somewhat large. But having all information in a CSV sortable file...and now having it out of EZscan is much better.

Phil

No worries.

A couple of things to keep in mind, though: I neither specified the PSR-800 hardware or UI nor wrote the associated PC app. (Heck, even GRE didn't specify that UI, at least not directly). Also, I'm a contractor (have been since I started doing work for GRE 11 years ago - PSR-500) and not a Whistler employee.

I don't know if, in your previous post, you were using the terms "you" and "Whistler" interchangeably or if "you" referred to me, specifically. If the latter, see previous paragraph

 

[Swipesy]

you can see it in the console window it should appear in the file when you redirect stdout.

Can you post a small sample of what appears in the console window? Maybe the first 10 lines, or so?

Screen Shot

 

[DonS]

Screen Shot

That looks OK. The unused values (i.e. fields the utility is trying to parse but which were not defined in the PSR-800) are filled with zeroes, which generates acceptable output.

When you redirect output to a file, does the file:
a) not exist
b) have zero length
c) not import into Excel
?

 

[Swipesy]

Don, when I output an Excel.CSV file is created but all the cells are blank. I outputted it several times to different directories using different names but still the Excel file is completely blank. No error messages are seen.

 

[DonS]

Don, when I output an Excel.CSV file is created but all the cells are blank. I outputted it several times to different directories using different names but still the Excel file is completely blank.

Can you post the resulting CSV or email it to me? don@starrsoft.com

 

[Swipesy]

Can you post the resulting CSV or email it to me? don@starrsoft.com

Will do.

John

 

[SOFA_KING]

No worries.

A couple of things to keep in mind, though: I neither specified the PSR-800 hardware or UI nor wrote the associated PC app. (Heck, even GRE didn't specify that UI, at least not directly). Also, I'm a contractor (have been since I started doing work for GRE 11 years ago - PSR-500) and not a Whistler employee.

I don't know if, in your previous post, you were using the terms "you" and "Whistler" interchangeably or if "you" referred to me, specifically. If the latter, see previous paragraph

Thanks, Don. I really didn't know your relationship or level of involvement with these companies. And you do know I really like WIN500, right? I would surely purchase TRX-2 software from you if you made it.

I just ran a "parse", now about to look at it.

Thanks again!

Phil

 

[Swipesy]

Issue resoved with a reboot of my PC. AUParse works great with the PSR-800.

Thanks Don for doing this.

John

 

[DonS]

Some people were asking for the AUParse source. I've updated the ZIP file above.

 

[AggieCon]

Are there any other source codes relating to Whistler scanners or EZ Scan that can be posted? It would be nice if they would be shared.

I spent a lot of time essentially creating the same thing, especially when it comes to making the arrays for non-standardized squelch value lists. Now that I said something about one of my projects, it is handed out.

At any rate, what is the byte at 164 for? In the documentation released by Whistler, it said it was reserved, but it looks to me to be in use.

 

[DonS]

Are there any other source codes relating to Whistler scanners or EZ Scan that can be posted? It would be nice if they would be shared.

I spent a lot of time essentially creating the same thing, especially when it comes to making the arrays for non-standardized squelch value lists. Now that I said something about one of my projects, it is handed out.

At any rate, what is the byte at 164 for? In the documentation released by Whistler, it said it was reserved, but it looks to me to be in use.

This wasn't existing Whistler code that was handed out. It was something I knocked off in an hour in order to test parsing speeds outside the EZ Scan app.

Byte offset 164 in the .AU header? If my quick count is correct, that should be:
unsigned char TsysType

If the recording type is "TGRP", it's the trunked system type ("see _eTrunkingSystemType enum" above).

EDIT... (Well, yeah, I did copy/paste the DCS and CTCSS string data.)