Try the following:
Hook up the handheld to the outside antenna.
Does it's reception drop off too ?? Does the base "improve" with it's back of set antenna, or when using the HT's duck on it??
If yes, there is a problem, most likley in the cable.
If the handheld is fine on the outdoor antenna, it is probably either a problem in the base unit, or just the difference in sensitivity/selectivity between the 2 units.
Generally speaking, a base antenna, high up in the air, fed with even 150ft of the thin coax, should still at least be equal or better at most freqs, to a handheld inside the house with the stock rubber duck.
If the problem is in the coax, it is probably a connector, open or short, rather than the coax itself, for such an extreme difference,( ASSuming you aren't running something like 600 feet
)
Another possibility if it affects both is that you are very near a high power xmitter, that is overloading or "De-sensing" the radios when hooked up to the outside antenna.