While the encoder on your computer likely provides a small delay - probably not more than a second - the more likely points of delay are buffers on the server and client.
The buffers are designed to pre-store some of the feed, so that if a short connection issue develops, it might hopefully be resolved before the buffer has been exhausted, resulting in a (nearly) seamless stream, or at least no disconnection to the listener.
Server buffers are usually fixed and consistent for all streams that are being fed to the server. I would imagine that RadioReference keeps their buffer sizes to a minimum, because they consume memory on the server, in addition to adding delay.
Client buffers are likely where the largest part of any delay is. Some players define buffers based on time (Windows Media Player does this), while others set buffers based on an amount of data (Winamp does this). Time-based buffers are obviously preferred, because then it doesn't matter the feed bitrate; the player will always keep x number of seconds regardless. Data-based buffers will vary the amount of time buffered based on the bitrate of the stream. The low-bitrates of scanner feeds may mean that a 100KByte or larger buffer (not uncommon for regular music streaming) could easily result in a good amount of delay to a scanner feed.
The most you can do is provide information to your listeners to check their settings and reduce the buffer as low as possible while still being able to hear a seamless audio stream (yes, if it is set too low, you might end up getting broken-up audio). And also keep in mind that these settings are specific to the player, so if someone uses Winamp for their daily internet radio fix, as well as your scanner stream, they will likely need to keep settings that allow their internet radio stream to work properly, which might add delay to the scanner feed.