RG-11 would be okay for a receive-only application. It is 75 ohm coax, so there's an immediate mismatch with a 50 ohm antenna and radio system. I used to think that mismatch would be significant, but ran the numbers once and it's really not that much.
However, if the system was intended for transmitting, I'd probably be more concerned about it. Even if everything else was *perfect*, use of 75 ohm cable means the lowest SWR you can get is 1.5:1. While that's not too terrible, things are seldom perfect, so the transmitter could see a SWR of 2:1 on an antenna that would otherwise be "okay". Many transmitters now fold back their output to protect the finals when they encounter high SWR, so not only is there inefficiency loss, now the radio simply doesn't put out as much power in the first place.
At least, many HAM transmitters now do that. I don't know if commercial systems do, or they may be built more rugged and just go for broke...
The one exception, where I've seen many people go ahead and use 75 ohm coax for transmitting antennas, is in the case of CATV hardline. It is (was?) often possible to get "ends and pieces" from the cable company that they would otherwise toss, that are up to 75 or even 100 feet long. Many people have gotten that to run out to and up their towers, as the expense for 50 ohm heliax or hardline is otherwise quite high.