if you look at this picture you will see the screw threads or barrel is MUCH SHORTER on the LMR400 than any other Coax cable I have
Don't get hung up on that part. Just different connector designs. The longer center pin the RG-8 is just the crimp/solder on those types of connectors.
The LMR-400 connector center pin length looks fine also.
Don't go cutting connectors off and replacing them.
You'll quickly get yourself into trouble if you start using the scatter gun approach to fixing this. You'll start changing so many different things that you'll end up spending a lot of money you don't need to and often never figure out exactly what the issue is. One of my early mentors taught me that. Don't fall into that trap.
You need to sectionalize/isolate the faults. Work ONE at a time and rule out individual components until you find the issue.
One thing you need to be able to trust is your test equipment. If you have any questions about the test equipment you are using, you need to stop and sort that out first. There's no point in swapping out connectors/parts/components if you don't know if you can trust your test gear.
First thing I'd do is get rid of the 90º connector on your tester. I've had more of those things go bad than I'd care to admit.
Start by connecting the 50Ω load directly to the RigExpert (do not use the 90º adapter) and see what it shows. Directly connecting the 50Ω load to the tester should show low SWR. If it doesn't, then you need to suspect the tester or the 50Ω load. You can check the load with a multimeter and confirm that it shows something close to 50Ω as an easy test for that.
You already said earlier that you connected the LMR-400 to the tester and connected the 50Ω load to the end of the coax, and it tested OK. You also said you checked for continuity/shorts and it tested OK. I think that tells you that the LMR-400 and its connectors are OK. Do the same test with the RG-8. It should look similar.
I'd also run the RigExpert to your mobile antenna since you said that was working OK and showing low SWR. Do that, and then compare it to the analog SWR meter and see if they show similar. They should. If not, you've got another issue to sort out.
Not saying I know what the issue is, just trying to point out the importance of testing each individual part on its own and sectionalizing everything.
While it's good to have a lot of us looking at this, it also can lead to confusion with too many people tossing out ideas. I think you have some mostly excellent advice above. Just try to focus on one thing at a time, proof out each part individually.
I think if one of us was there to help, we'd be able to help you find the fault rather quickly. Maybe taking prcguy up on his offer to visit would be worth the wait, worth the dinner, and worth the 6 pack of beer. He'll have the tools, test equipment and knowledge to solve this quickly on site without all the noise of internet posts in the way.