Well looks like the MAX232 doesn't do much more than voltage buffering of the RS-232 signal, down to 5V TTL levels (since RS-232 voltages CAN be much higher), but with a USB/serial adapter running 5V USB, voltage level shouldn't be a problem. The +8V in that circuit is just used to power the MAX232 (via a regulator), since RS-232 doesn't provide power.
I did check for the +8V, to make sure I had the RJ wired properly so I wouldn't fry anything - it's there on pin 1, and the hanger function works properly as well, so I don't think there's an issue there. I'll probably wire the thing back the way I had it at first, and fiddle with settings from there.
I remember there was some kind of issue getting my Baofeng USB cable working properly on my old laptop, although I think that was a specific driver issue for the USB/serial chip they used; this is a tried-and-true ATEN adapter... granted, this is also the first time I'm using it with 64-bit drivers, so maybe I'll find another serial device to test it on. I have some managed switches around that have serial console ports
Anyway, time to get to work... I'll keep this thread updated if I figure it out. I'm tempted to just order the proper cable, but keep futzing with this anyway because I don't want to let it beat me
I may pick up the MAX232 and regulator to make that circuit too.
Thanks for all the help!