With a proper counterpoise (also called a 'ground plane') a 1/4 wave vertical is a 1/2 wave dipole (one element being the antenna and the other element being the counterpoise). This is why they show to be unity (or 0 dBd) gain antennas. Without a proper counterpoise the vertical will not perform properly and with no counterpoise at all would indeed be half the gain as you indicate (but this generally can't happen outside of straight theory since the feed line will attempt to act as the counterpoise).
Your 5/8 wave numbers also assume the counterpoise doesn't exist. With a proper counterpoise they will provide "about 3 dB of gain" as indicated.
If you simply run the numbers based on straight theory you'll get results that translate to real world use like the isotropic dipole translates to a real world dipole, the numbers just don't line up. The problem with the theory is that many real-world variables are simply left out (basically because they vary with the environment). This is how real-world factors like the counterpoise often turns up to be missing when running the formulas.
In several aspects I have no real argument with what your are saying, however there is a need to level the field, and that was what I was doing by reverting to the theoretical.
Any apparent gain of a ¼ wave in reference to either an isotropic or a dipole is dependent on the specifics of the counterpoise. Indeed, a ¼ wave monopole can be said to have 5.16 dBi (Kraus, 1950) under certain specific conditions, far greater than unity (you might also look at Wolf, 1966, defining the ¼ wave monopole as having twice the affective area of a dipole, although I have always taken exception to that paper). It is also possible to define conditions where both ½ and 5/8 wave antennas have greater gain than their theoretical configurations. And we have not touched on the ½ wave monopole issue, an antenna that certainly has more gain than the ¼ wave monopole.
The “real-world” application can be extremely variable.
For example, a 39 foot long 18 element wide spaced Yagi on 2 meters could be said to have as much as 27 dBd of gain…but unless you specify the exact height above ground, the ground conditions, and the departure angle, it would not exhibit such a number. So better manufacturers go with the “free space” (a condition impossible in most applications) gain value instead of over ground, and in such a case the value is more like 15 dBd. And this is a bit of an oddity, because many Yagi manufacturers keep to the theoretical while many mobile manufacturers do not. It is not uncommon to see ¼ wave mobile advertised as “0 dB” without specifying either the efficiency or the mechanical configuration of the ground plane or counterpoise.
As you said, my 5/8 numbers are correct without counterpoise, the gain is even greater in many real World applications.
To the OP, and related to his question. As you can see, there are many variables. As a general rule of thumb, for a specific application, the physically larger an antenna is for a given frequency the better its potential to perform is. A full sized ¼ wave antenna will sometimes outperform, in the real world, a reduced size 5/8 wave antenna, despite the fact that the 5/8 wave has a theoretical advantage.
Mobile antennas typically are ¼ wave or 5/8 wave, or some multiple or combination thereof (multiples and combinations used in collinear arrays). ½ wave antennas traditionally have not been used in mobile application, but certainly can be, either as monopoles or as dipoles...and there has been a large trend recently (last 10 years or a bit more) to use ½ waves in mobile application. ½ wave antennas in the form of dipoles are frequently used in fixed location applications (call it base station application, if you want).
For practical applications in a mobile world the ¼ wave vertical is often all that is needed. Unless you are in a fringe area or are looking for just a little extra range the added gain of a 5/8 might not matter. A high gain collinear mobile antenna can add up to 6 or even 9 dB to your signal, but again, unless you are in a fringe area you might never notice the difference. If you want the best possible performance decide what is the tallest antenna you can get away with, keeping in mind your parking garage/overhang limitations, and find the largest antenna (in actual size, height in inches) designed for your frequency range that you can fit in that space. As a general starting point this is hard to beat, real capture area on the antenna is an important factor.
With either of my off-road SUVs on HF (160 to 10 meters) I use a ¼ wave, depending on the band it might be a loaded ¼ or not. On 6M I use a 5/8 wave. On 2M it depends on the vehicle, either a 5/8 wave or a collinear antenna. On UHF and 1.2 GHz they are all collinears of some kind. However, these are all fairly tall antennas, and I must keep them in mind at times when driving under things. My wifes SUV has a simple dual band antenna (Diamond NR770B), ½ wave on 2M and collinear 5/8 wave on 70cm, and still mounted on a motorized mount so she can lower it in parking garages. I use the same antenna on my convertible to keep everything a bit more low key.
There is a near infinite selection of antenna options out there, at some point you just pull the trigger and jump in.
T!