Scannerowner
Your opening sentence to paragraph 2 sums it up quite well - I would tend to agree with what you have said, that improving receive gain overall (by way of a better or higher antenna) is just going to raise the level of all the other tower communications (in this case "noise") around you by the same degree i.e. it ain't going to help, there will be no improvement to the channel specific signal SNR.
The only real way to resolve the problem is to use a directional antenna, or, if you use an omni-directional antenna, establish what the exact Tx frequency is of this distant tower you want to listen in to, and try find a band-pass filter that is bang on that frequency & which has the required band-pass band width, which you can then switch in and out of the coax circuit as and when you wish to listen to that tower. That should the job.
Thats the problem in lay-man terms, to take the subject to the next tech level requires acknowledging that SNR is not really an accurate way to define noise or interference in digital comm's. For digital modulated and encoded com’s one needs to know the C/N ratio (carrier to noise ratio), which, in the case of P25 should be no more than around 5% BER (bit error rate) i.e. overall noise shouldn’t cause your receiver to “loose” any more than around 5% of the bits that make up the data stream.
To put this into context, we revert back to analogue comm's for a moment, which folk are more familiar with: in an analogue receiver whose spec’s are based around the commonly used trade figure of 12dB SINAD (the majority of scanners/receivers tend to use 12dB SINAD), this would equate to a signal level of around 12db – 13dB above SINAD. In other words, to demodulate and recover pretty good audio from an analogue transmission the received signal should be around 12dB above overall noise (and this includes receiver generated noise).
With respect to a P25 type transmission, to keep within the 5% BER figure requires a signal level which should be around 5dB -6dB above overall noise (again, receiver generated noise included). However, in reality as it has turned out, P25 users have found that a lower BER tends to be required - not exceeding around 2.5% turns out to be ideal when fading is brought into the equation. And the reason for this difference lies in the fact that when the P25 standards were defined no one remembered to keep in mind the impact voice compression (which all digital audio comm's implement) would impart on overall performance.
Just as each MPEG2 audio/video hardware manufacturer is free to adopt whatever video or voice encoder they wish to use, so long as it complies with the overall MPEG2 standard, so to can P25 equipment manufacturers adopt whatever voice encoder they want, so long as it complies with the P25 standard.
Like different MPEG2 encoders, different P25 encoders adopt different sampling rates and different bit rates and as we all know, the higher the sampling rate and the higher the bit rate, the more important "noise" becomes ... see where this is going?
One of the most common P25 voice encoders turns out to be the IMBE vocoder - just so happens that it's well adapted for P25 use, but it requires a +/- 2.5% BER!!
Quite why the issue of variability between different encoders was not given the consideration that it required in the P25 design process is an ongoing dispute and one of the ongoing teething problems in P25 comm's. In time it will all get sorted out.
Anyhow, that is a basic tech level insight to the problem you are having to overcome to recover audio from this tower: between signal strength and overall noise your receiver is struggling to recover the required +/- 97.5% of the bits required to demodulate & decode the received rf signal and reconstruct the analogue audio content. Its that simple!
Hope it all helps.