Unattended Logging Of Whistler Scanners

Status
Not open for further replies.

AggieCon

Member
Premium Subscriber
Joined
Nov 25, 2015
Messages
1,448
Location
Texas
Is that reading the files based on the ARINDEX.DAT file listing?
 

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
Is that reading the files based on the ARINDEX.DAT file listing?

Yes, just like the scanner and PC app do. It reads 4-byte long ints from that file, generates the directory and filename from that number, and opens the resulting AU file.

Of course, I could've searched for matching directory and filenames and opened every AU file present (without regard for whether it was listed in ARINDEX.DAT), but I was going for speed. Mostly to prove to myself that the Whistler PC app really isn't any slower than a lean console app would be (and it's not: after first inserting a card into a USB card reader, my little utility takes 230 seconds to parse 23498 .AU files; the PC app takes 240 seconds).

EDIT: just realized I forgot to include the TSYS type in the output. Fixed.
 
Last edited:

AggieCon

Member
Premium Subscriber
Joined
Nov 25, 2015
Messages
1,448
Location
Texas
I achieved 2837 files per second, also from a card reader. Pretty much the same speed as EZ Scan.

However, I have a lot of duplicates due to a bad ARINDEX.DAT, I assume. I'll try to motivate myself to have the scanner or EZ Scan rebuild the list. About 25% of the files on my CSV were duplicated. The number matched EZ Scan, so I just realized that all along I had duplicate recordings in EZ Scan.
 

AggieCon

Member
Premium Subscriber
Joined
Nov 25, 2015
Messages
1,448
Location
Texas
Moral of the story: It's the scanner that is slow to communicate not EZ Scan that slows things down.
 

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
I achieved 2837 files per second, also from a card reader. Pretty much the same speed as EZ Scan.

However, I have a lot of duplicates due to a bad ARINDEX.DAT, I assume. I'll try to motivate myself to have the scanner or EZ Scan rebuild the list. About 25% of the files on my CSV were duplicated. The number matched EZ Scan, so I just realized that all along I had duplicate recordings in EZ Scan.

Or you could rebuild the list yourself.
Code:
#include <stdio.h>

int main( int argc, char *argv[] )
  {
  unsigned long filenum, dir, num;
  unsigned char fnbuf[20];
  FILE *fp, *afp;

  fp = fopen( "X:\\REC\\ARINDEX.DAT", "wb" );
  fwrite( "ARINDEX", 1, 8, fp );
  for ( dir=0; dir<100000; dir++ )
    {
    for ( num=0; num<200; num++ )
      {
      sprintf( fnbuf, "X:\\REC\\%05ld\\%03ld.AU", dir, num );
      if ( NULL != (afp = fopen( fnbuf, "rb" )) )
        {
        filenum = dir * 200 + num;
        fwrite( &filenum, 4, 1, fp );
        fclose( afp );
        }
      }
    }
  fclose( fp );

  return 0;
  }

(For speed, replace the 100000 limit of 'dir' with the highest directory number you have in REC, plus 1).
 

SOFA_KING

Member
Joined
Apr 25, 2004
Messages
1,581
Location
SE Florida
OK...Sorry Don. My frustration has been building over a few years. No acknowledgement on the issues reported, nor on suggestions made, so it's like no one is listening, cares, or anyone who could do something about them ever even heard them. That's the frustration. Thanks for that app. Nice of you to share that.

I have now over 114,000 data recordings (tracking long term trends) and will have to give that a try. Speed on a large amount of data like that is understandable, I guess. Audio recordings are somewhat large. But having all information in a CSV sortable file...and now having it out of EZscan is much better.

Phil
 
Last edited:

Swipesy

Member
Joined
Jan 27, 2003
Messages
2,055
Location
Northern Ohio
I am able to see the detail of the AUParse in the command window but when it output to a log file it creates an Excel.CSV but there is no data. My SD Card is in Drive E. and the AUParse.exe is in C:\. So I used C:\AUParse E: > Log.csv and get a blank spreadsheet. Am I missing something here?

Thanks

PS I am using PSR-800
 

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
I am able to see the detail of the AUParse in the command window but when it output to a log file it creates an Excel.CSV but there is no data. My SD Card is in Drive E. and the AUParse.exe is in C:\. So I used C:\AUParse E: > Log.csv and get a blank spreadsheet. Am I missing something here?

Thanks

PS I am using PSR-800

The PSR-800 doesn't have in its .AU header data a couple of fields that were added with the Whistler scanners (though the header size is unchanged - I reserved 320 bytes back in PSR-800 days). That might be messing up the CSV output, but if you can see it in the console window it should appear in the file when you redirect stdout.

Can you post a small sample of what appears in the console window? Maybe the first 10 lines, or so?
 

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
OK...Sorry Don. My frustration has been building over a few years. No acknowledgement on the issues reported, nor on suggestions made, so it's like no one is listening, cares, or anyone who could do something about them ever even heard them. That's the frustration. Thanks for that app. Nice of you to share that.

I have now over 114,000 data recordings (tracking long term trends) and will have to give that a try. Speed on a large amount of data like that is understandable, I guess. Audio recordings are somewhat large. But having all information in a CSV sortable file...and now having it out of EZscan is much better.

Phil

No worries.

A couple of things to keep in mind, though: I neither specified the PSR-800 hardware or UI nor wrote the associated PC app. (Heck, even GRE didn't specify that UI, at least not directly). Also, I'm a contractor (have been since I started doing work for GRE 11 years ago - PSR-500) and not a Whistler employee.

I don't know if, in your previous post, you were using the terms "you" and "Whistler" interchangeably or if "you" referred to me, specifically. If the latter, see previous paragraph ;)
 

Swipesy

Member
Joined
Jan 27, 2003
Messages
2,055
Location
Northern Ohio
you can see it in the console window it should appear in the file when you redirect stdout.

Can you post a small sample of what appears in the console window? Maybe the first 10 lines, or so?

Screen Shot
 

Attachments

  • AU Parse.jpg
    AU Parse.jpg
    91.6 KB · Views: 329

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
Screen Shot

That looks OK. The unused values (i.e. fields the utility is trying to parse but which were not defined in the PSR-800) are filled with zeroes, which generates acceptable output.

When you redirect output to a file, does the file:
a) not exist
b) have zero length
c) not import into Excel
?
 

Swipesy

Member
Joined
Jan 27, 2003
Messages
2,055
Location
Northern Ohio
Don, when I output an Excel.CSV file is created but all the cells are blank. I outputted it several times to different directories using different names but still the Excel file is completely blank. No error messages are seen.
 

SOFA_KING

Member
Joined
Apr 25, 2004
Messages
1,581
Location
SE Florida
No worries.

A couple of things to keep in mind, though: I neither specified the PSR-800 hardware or UI nor wrote the associated PC app. (Heck, even GRE didn't specify that UI, at least not directly). Also, I'm a contractor (have been since I started doing work for GRE 11 years ago - PSR-500) and not a Whistler employee.

I don't know if, in your previous post, you were using the terms "you" and "Whistler" interchangeably or if "you" referred to me, specifically. If the latter, see previous paragraph ;)

Thanks, Don. I really didn't know your relationship or level of involvement with these companies. And you do know I really like WIN500, right? I would surely purchase TRX-2 software from you if you made it.

I just ran a "parse", now about to look at it.

Thanks again!

Phil
 

AggieCon

Member
Premium Subscriber
Joined
Nov 25, 2015
Messages
1,448
Location
Texas
Are there any other source codes relating to Whistler scanners or EZ Scan that can be posted? It would be nice if they would be shared.

I spent a lot of time essentially creating the same thing, especially when it comes to making the arrays for non-standardized squelch value lists. Now that I said something about one of my projects, it is handed out.

At any rate, what is the byte at 164 for? In the documentation released by Whistler, it said it was reserved, but it looks to me to be in use.
 

DonS

Member
Joined
Jun 17, 2003
Messages
4,102
Location
Franktown, CO
Are there any other source codes relating to Whistler scanners or EZ Scan that can be posted? It would be nice if they would be shared.

I spent a lot of time essentially creating the same thing, especially when it comes to making the arrays for non-standardized squelch value lists. Now that I said something about one of my projects, it is handed out.

At any rate, what is the byte at 164 for? In the documentation released by Whistler, it said it was reserved, but it looks to me to be in use.

This wasn't existing Whistler code that was handed out. It was something I knocked off in an hour in order to test parsing speeds outside the EZ Scan app.

Byte offset 164 in the .AU header? If my quick count is correct, that should be:
unsigned char TsysType

If the recording type is "TGRP", it's the trunked system type ("see _eTrunkingSystemType enum" above).

EDIT... (Well, yeah, I did copy/paste the DCS and CTCSS string data.)
 
Status
Not open for further replies.
Top