This will tell you how much isolation you can get, but not how much isolation you need.
What is the transmitter noise level 300 KHz from carrier frequency, and on the receiver frequency? If you don't know that value, then calculating the isolation is pointless. You don't know what the target value needs to be.
I suspect I'm preaching to the choir, but I'll put it out there for everyone's read - adjacent channel power ratio (ACPR) is determined by two things: transmitter spectral purity and receiver filtering. Not knowing the specifics of Tim's setup in terms of measured response, it's not possible to derive a threshold figure. There are a couple of tactics he can explore. That being sharp bandpass filtering on the transmitter output (which could potentially attenuate half or more of the output power), and sharp bandpass filtering on the linked receiver, which stands to reduce the signal level and impact the systems' link budget. Additionally, receiver IF filtering should be as sharp as possible, as well. Digital capable equipment tend to have much better filtering that is aided by DSP. But not everyone goes out and buys a Quantar or MASTR-III. My experience in helping people build and maintain ham repeaters has been that many will put two mobiles together with a talking controller and a modest duplexer and take great pride in doing so (mostly rightfully so, but that should be the beginning of the learning process). Those devices may be excessively broad and noisy because they are designed to work in as wide a bandspread as possible (450 - 470 or greater), while a repeater is designed to be on one frequency only, until it's re-frequed and realigned. I also have to mention the antenna. Just about every ham grade antenna is not built to commercial/public safety specifications. Things like construction techniques could affect performance (i.e., creating non-linear junctions and an environment for passive intermod [PIM]), especially in RF dense environments.
So, there is no short answer to that without much more information and testing. If I were doing that, I would shoot for as much isolation as I could get, use dissimilar CTCSS tones, and then experiment. But I wouldn't put any stake into reliability until I had some solid experience with the two systems in operation.
