That is a user preference matter and they vary.
For me, the default is 2 seconds and for "special channels" which are the more important ones that I don't want to miss a reply, I'll bump up to 4 seconds.
Another school of thought is the delay time should approximately the time it takes you to find the Hold button. That way if there is an interesting conversation, you'll have time to stop the scanning and hold on that interesting channel.