"Scanning" a trunked radio system, be it digital or analog is much different than scanning a conventional system. With a conventional system, the scanner checks each frequency for activity. If there is activity, then the scanner remains on the frequency.
With a trunked system, the scanner monitors the control channel for grants. The scanner gets information from the control channel whenever a grant occurs, including TGID information. Next the scanner looks at programmed talkgroups to see if the "granted" TGID is present and not locked-out. If there is a match, the scanner does "its thing" and the communication is monitored.
In other words, a trunked radio system does not listen to each talkgroup to see if it is active.
Thus there is nothing to be improved scan speed wise by reducing the number of talkgroups. A trunked system with 1 talkgroup will scan just as fast as one with 1,000 talkgroups.
(That being said, a trunked system with multiple sites is another story. The scanner dwells on each site for about 1.5 to 2 seconds, monitoring the control channel for grants. I don't think the OP is describing this type of system, but thought I'd mention it.)