I became concerned that my KX3 was misbehaving when I saw some fairly horrible signals in the waterfall, for example, the CW transmission below.
Horrible isn't it!
I noticed similar nastiness for SSB transmissions too. Then, by a stroke of luck I retuned gqrx's demodulator away from the signal and much to my surprise, the nastiness went away.
I then resorted to using a gnuradio-companion flowgraph instead and the KX3's transmissions looked clean
I'm not sure what is going on in gqrx that such a quirk can exist and I haven't had a look at the code yet. I wonder if there might be a recursive filter operating on the same data that is being used in the waterfall and PSD plot, or similar.
I've added another screenshot below. When this was taken, I'd been scrolling the modem back and forth across the CW signal. This time I used a modem with a wider bandwidth.
It seems to be related to the squelch somehow. In the following screenshot, I alternated the squelch between fully open (previous setting) and fully closed. The spurii seemed to go away when the squelch was closed.
Turning AGC off gets rid of it.