The signal "loss" for the SMA to BNC adapter is a total non-factor. Don't pay any attention to claims that removing it will make any noticeable difference in performance, because it will not.
If you are frequently switching between the rubber ducky antenna and an external antenna, you'll absolutely want to stick with the BNC adapter. The SMA is not designed for repeated and frequent coupling and decoupling, the BNC is.
Also, using the BNC adapter saves all that wear and tear on the scanner's SMA connector. If you ever wear out the BNC side of the adapter, you simply replace the adapter for a few bucks, and the scanner's SMA connector stays in great shape.
Stick with the BNC adapter, bottom line.