As an aside, what causes a cqpsk constellation diagram to tend towards an "X" rather than four tight clusters?
I've seen this "X" pattern in constellations for a long time, but never 100% understood. It's limited to simulcast and as far as I can tell it's most pronounced in /\/\ LSM. The constellation space can be thought of as a circle divided into four quadrants as a Pizza is cut into four slices. If the symbol appears anywhere inside one of the quadrants it's assigned to that quadrant. The dividing lines between quadrants are the X and Y axes. The distance (radius) from the center of the circle to each data point is the magnitude of the point. The angular offset (0-360 degrees) is what solely determines in which of the four quadrants the point resides - the magnitude is ignored. In standard PSK the radius is always constant (modulo any fading) and is normalized to 1.0 via AGC.
It's known that the OP25 Gardner/Costas differential demod loop is less than ideal so it's possible something could be falling into an edge case or partially losing lock, etc.
I think more likely there are two interrelated causes of the "X" (i.e., points whose magnitude or radius from center is < 1.0)... The first one is improved by increasing the signal/noise ratio, i.e., maximizing the height of the FFT spectral pip relative to the noise floor. The second problem may be due to cancellation effects caused by the usual "simulcast distortion" - reception of an ensemble of signals from multiple transmitters with symbols arriving somewhat out of phase.
In many cases if using a directional antenna the two issues may be addressed at the same time by re-aiming the antenna.
When many points in the "X" are very close to the center (indicating magnitudes approaching zero) those symbols are the most likely to be received as errors. Sometime the symbols skate close to the edge and approach zero magnitude but still have a perimeter about the origin where the symbols mostly stay out of. In theory reception would be close to 100% error free in this case. It's speculated that the "X" phenomenon isn't just an artifact but may actually be something exploited by the LSM designers to squeeze out a few dB of additional ISI margin, or something...
I'd also suspect trouble if you were to take a particular receiving location that has a good strong "X" pattern and then (without moving the antenna) try to receive the station on a conventional FSK4-style scanner after reconnecting the antenna. I.e., this may be a good method to find locations where FSK4 scanners have the most trouble...
Max