Because of the various different radio systems in use in Central Ohio, what you will need to buy depends on who you want to listen to and what radio system they're on. You might start by listing the individual agencies that you want to listen to, perhaps in order of importance, and using that to help you decide on a scanner. Simply speaking, it comes down to whether or not to buy a digital scanner.
Columbus Police, Fire, and most (but not all) of the suburbs are on Columbus's analog 800MHz trunking system; you don't need digital scanner for that. Columbus doesn't seem to be planning to go digital any time soon, so if you want to save money and are mostly interested in listening to them, you would probably be safe with an analog trunking scanner. I can't think of any current models off the top of my head; someone else can chime in with recommendations there.
On the other hand, OSP, all of Delaware county public safety, all of Union county public safety, and a number of suburbs in Franklin county are on digital systems (MARCS or the delaware county COIRS system). If you want to listen to any of them, you'll need a digital scanner. The handheld that I would recommend is the Radio Shack PRO-106. I have the base/mobile version of it (PRO-197) and a friend has the 106. It is pretty easy to use (considering the inherent complexity of it), and it works well. If you look for gently used ones on here (in the classified section) or on eBay you can pay considerably less than new price for one. I also own a Uniden BC-396T and am happy with it as well. Properly programmed, it can be fairly easy to use.
If you have any further questions, feel free to ask.