Wait a second...you went from RG-8 to RG-6QS? Are you sure you had RG-8 and not RG-8X or RG-58? Big difference! RG-8 is good coax (provided it's quality cable in good condition with good quality, properly installed connectors), and has LESS loss than RG-6QS.
RG-6QS will work just fine for RECEIVE ONLY applications (don't transmit on 75 ohm cable), but not all coax is created equal. Do not buy coax at Radio Shack, their quality is less than desirable. If you're dead set on using RG-6QS, and don't want to order online, then go to WalMart and pick up the Philips brand they sell that has the compression F connectors already installed.
http://www.radioreference.com/forums/showthread.php?t=94388
That alone will reduce your loss quite a bit. Looking at the numbers on the loss charts and calculators is a good starting point, but again, not all coax is created equal. If you're using a quality brand like Belden or Times Microwave, then you can be assured the coax is meeting the spec they publish. Anything else is a crapshoot, especially Radio Shack's stuff.
Please check the coax that was already installed and verify if it was RG-8 or not, and what brand and condition it was in and if it had properly installed connectors.