zz0468's analysis and comments are dead on and absolutely should be kept in mind throughout all future tests and result analysis!
Considering the limitations of the testing method and very small sample size, the relative types of tests (using the same method to test two different items and comparing them - in this case the 436 vs. the 396) have the most "relevance", pun not actually intended. So the difference between the 396 and the 436 results probably have more merit than the absolute value of the single unit measurement of the sensitivity. For example, if we see that the 396 is relatively "flat" across the measured bandwidth versus the comparatively "uneven" spread for the same bandwidth with the same setup then that is a data point of interest.
Having said that, though, zz0468 is quite right in pointing out how little a few dB of difference makes to a moderate signal level as heard by a human given all other variables are equal.
Sensitivity, as I've mentioned many times before, is a largely overrated (and vastly oversimplified) consumer scanner specification. And it is VERY misunderstood by the average hobbyist. What you really want is usable sensitivity given more real world conditions. So how "sensitive" is the radio in the presence of local noise both internally and externally generated?
Tests like P1dB and Third Order Intercept require more than just a service monitor to perform but are very important and telling tests!
The easiest test to perform is for basic SINAD sensitivity. But that doesn't tell you everything nor does it tell you all that much outside of making sure the radio meets its published spec. within measurement error and acceptable product manufacturing variation.
Considering the poor phase noise of the synthesizers in a typical consumer scanner as well as other internally generated broadband noise sources (poor shielding allowing nearly all noise generators to be problematic such as all digital electronics including clock generators, all gates, microprocessor, timers, etc.) having the squelch behave oddly on different bands is not all that "strange" of a result. Consider that a typical FM squelch uses a noise gate method whereby a high frequency filter is used to "isolate" the FM noise present on a signal. The more noise the more it mutes depending on your setting of the threshold ("squelch" setting). So a signal with "full quieting" means that there is little to no noise coming through that filter and being detected so the signal is "allowed" through easily. A moderate signal with some noise present will also pass assuming the threshold is not set too high. And on down the line until we come to very noisy signals that have more high frequency noise present than the squelch noise detector's threshold is set for and we get muting.
So when you think about that, you can see that, while in a nice clean RF environment, say inside a screenroom, and with a purely resistive or nearly so 50 ohm source driving it with a very clean signal and assuming the local oscillator driving the receiver's mixers is also clean than the squelch action is largely dependent on signal level and nothing else - higher level means less noise and vice-versa.
But then you add noise to the mix both internally and externally and the noise gate squelch will act on what it is designed for - noise, regardless of level.
When strong signals are present within the passband of an RF amplifier its noise floor will rise - more broadband noise and reduced "sensitivity" (not to mention other non-linear effects). When you are talking about the final RF amp right after the final IF conversion, as zz0468 stated quite well, you have now down converted all of the combined noise sources along with the desired signal signal and the final discriminator and noise gate ("squelch") must act on that amalgam of a signal. If the signal is otherwise a reasonably moderate or even strong signal but is noisy it will still likely get muted by the squelch (assuming the content of the noise is strong within the noise gate's filter output). This is why you can see a strong S meter reading and yet it won't break the squelch.
I'm guessing that the squelch tables in the 436/536 attempt to compensate for variations in internally generated broadband noise on different bands as well as the IF filter bandwidths selected. They are used to attempt to yield a more "smooth" and "linear" squelch action across the entire RF bandwidth and IF filter selection of the scanner. I am guessing that without those tables, you would see a very large variation in how the squelch behaves on different frequencies and using different IF modes. The tables probably adjust the noise gate threshold up or down relative to mode and frequency so that if, for example, a user sets the squelch at position "4" it attempts to compensate for noise floor variations between what is present at 155MHz versus what is present at 40MHz and 860MHz so the user's "4" setting seems relatively the same across all bands and modes. Without the table, that setting might have to change between 2 and 8 or so across the different bands and modes which wouldn't be practical for normal use. Of course it isn't perfect, you only have a limited set of tables and compensation points; so some variation still occurs and those tables really only account for internally generated noise and not outside noise sources of course.
The P25 signals may not use the squelch in the same way as they don't "quiet" the same way an analog FM signal would. They may, in fact, use a completely different squelch action solely dependent on detected signal level or BER (Bit Error Rate) so setting the squelch just right for an analog FM signal at 45MHz might not work well for a P25 signal at 856MHz.
My guess is that the firmware changes that seem to affect sensitivity do so partly by adjusting these squelch table settings. They may also do things like slightly altering certain digital timing settings (including even, maybe adjusting the microprocessor clock or the number of allowed clock cycles needed for various functions) which, in turn, can change the internal noise characteristics thereby changing the noise floor of the radio receiver.
So, as simply put as I can think of, I think the firmware changes likely affect what the user perceives as "sensitivity" through two means - altering the noise floor of the receiver by "adjusting" internal necessary but noisy "noise sources" (either indirectly in order to improve or modify some function or directly in order to reduce RF noise) and by adjusting the squelch threshold tables to compensate for those changes.
That's just for analog FM - there are a slew of other settings likely available which can affect apparent P25 "sensitivity".
-Mike