The model definition file (GTX.MDF) needs to be modified for the RSS (not CPS) to accept out of band frequencies. No .exe mods required, no hardware mods required. All of the information you need (and more) is available on the
Repeater Builder site's GTX section.
Keep in mind that GTX radios are ancient and I would expect them to have some issues just due to their age. The programming software is DOS based and if you are not already set up to program Motorola radios that require DOS RSS it is IMO not worth your time whatsoever. If your goal is to play on ham 900 there are cheap radios that are much easier to work with (for example, the Motorola XPR6580, a wonderful little portable for ham use and available for well under $100/ea).