Detective work (analysis) is the best way.
I've heard new and unknown talk groups in encrypted mode and wanted to know what it was. By using Pro96Com and/or UniTrunker to collect the control channel data, you can find out which radio IDs are using that unknown TG. Then find the other channels they use and it gives you an idea of what it is used for. To be sure, listen to the other related TGs, then when you hear them say, "Switch to Bla bla TG" and you hear (or see on your scanner or other equipment) there is traffic on that encrypted channel, then you know what the TG is called and used for. To be absolutely sure, when you hear them say to switch to another TG and then you see on your Pro96Com or UniTruncker data that they did switch to the encrypted TG then you have it.
Archiving the audio for the known un-encrypted TGs (using ProScan) and control channel data (using Pro96Com or Unitruncker) for the unknown encrypted TGs, you will find the info.
I've done this on a lot of unknown TGs. I've even heard them say to switch to specific TGs and they mentioned "so we can have encryption."