Though they tend to introduce noise even on the FM bands a little, the only exception (at least from what I've tried) is if you get a connector adapter and plug a coaxial cable connected to an outdoor antenna into the 3.5mm antenna jack, than it doesn't pick up the noise.
Otherwise rechargeable batteries are a good way to go and the Eneloops (especially the Pro Version) last a nice while.