During non-peak demand time periods, the Service can scale back the number of on-air dispatchers on duty. When this happens, boroughs will be consolidated. In addition to the combination you've observed above, Manhattan Central's operations have been known to be shifted to the Manhattan South channel.
Bottom line...when not hearing activity on your normal borough's channel, check an adjoining one.