In most cases, yes you would only hear bits and pieces of conversations. If you let the scanner sit on a single TRS voice frequency for a minute or two, you might hear 10 or more different conversations from non-related talkgroups within that short time frame.
Using the New Jersey State Police system as an example, there are over 200 talkgroups throughout the entire state. If you let the scanner sit on one of the assigned voice frequencies, such as 852.2375, you could easily hear an SP unit in Sussex County run a plate, followed by hearing a DOT worker in Union County reporting a pothole, followed by NorthSTAR medevac asking for coordinates, and so on...all within a matter of seconds. The trouble is that the SP dispatcher for Sussex County would acknowledge the trooper on a different frequency, the DOT conversation would also move to a different frequency, as would the rest of the NorthSTAR conversation. After about 5 minutes of trying to make sense of it, you'd be reaching for the Advil!
Now, on a much smaller system that doesn't have much activity, maybe only 2-3 frequencies and a handful of talkgroups...it's very possible to scan those 3 frequencies and be able to follow a normal conversation most of the time. The trick in that case would be to setup each TRS frequency with no delay so the scanner moves on to the next frequency right away, in case the conversation changes frequencies.
But otherwise yes, the analog voice frequencies appear as "normal" 2-way traffic on a non-trunked scanner.