Before you get too involved with this, I would suggest that you first elimate the possibility that your listening client might be the problem.
I've seen a few cases where the feed was flawless, but the software used to listen to the feed was at fault.
As a test, instead of restarting your feed when you hear a problem, restart your listening software then reconnect to the feed to see if the problem persists.
You can also download an archive file of your feed for the time period you were experiencing problems, then listen again using a different program.
The second opinion given by the alternate software may save you hours of chasing a problem that may not exist on the streaming side.
As I write this, your stream sounds good.