I have a streaming setup similar to yours. See the attached picture below for how the audio on my desktop Remote Desktop Connection/Local Resources/Remote Audio Settings is configured. On the General tab, I saved the RDP connection settings to disk so they'd be retained.
My streaming PC, a modified HP thin client running Windows 7 Embedded, uses an external USB audio device (a Behringer UCA-202). The UCA-202 is set as the default device for both recording (as a "line-in") and as playback. The speakers and the system sounds are both muted on the thin client.
In RadioFeed, the UCA-202 is set as the input sound device on the 'Source Client' tab.
I'm no expert on RDP, but with these settings, I can log into my streaming PC from my desktop PC without issues.