It appears that you have a system with 4 sites and each site is simulcast over multiple towers. You can simplify things by ignoring the simulcast part (since each tower on the site broadcasts the same signals over the same frequencies). In most situations, you can simply hold on an active talkgroup and hear what you want. It is possible that you may miss something if they move to a different site, but only if every radio on that talkgroup has left the site you're holding on.
For example, Radios 1, 2, 3, 4, & 5 are on talkgroup 32 and you hear them on site 1 and place your scanner on hold there. In reality, radios 1 & 2 are on site 1, 3 & 4 are on site 3, and radio 5 is on site 4. You would hear talkgroup 32 on sites 1, 3, & 4 (but not site 2) with the system controlling what frequency each is using. Radio 5 moves from site 4 to site 3. Site 4 will no longer host talkgroup 32, but sites 1 & 3 will continue to do so. Since your hold is on site 1, you're still fine. Radios 1 & 2 are moved into the area handled by site 3 (you probably heard them reporting that they were in route and likely even that they had arrived on location). They switch to site 3 for better coverage. At that point, your hold from site 1 will no longer be valid since talkgroup 32 no longer has any radios using that talkgroup on site 1. You won't hear anything at that point until you switch to site 3.
Hopefully that will help you answer your question. Multi-site systems are great for wide area coverage, but you can get unexpected traffic if things happen that are out-of-the-norm. For example, I once monitored a state-wide system where, as expected, most of the traffic on our local site was from the local area. At random times, we had traffic pop up from way out of range for a brief period of time and we wondered why that was. I asked a local officer who was also a ham friend who explained why this happened.
He stated that every so often a PD from the south-east part of the state had to pick up a prisoner from the city in the north-west part of the state. Often, they had multiple cars so one officer could always be watching the prisoners while the other was doing paperwork or to isolate the prisoners if multiple ones were being transported. They often would remain on their local talkgroup and roam from site to site as they crossed the state. While they were in our area, we could hear calls from that talkgroup and once they moved on those calls would stop.