I've never ridden with ear plugs, but only a couple of bikes I've had have been what I would actually term properly loud. The Fazer's can is louder than standard, but not annoyingly so, even with no baffle. Never to the point where it has bothered me anyway.
As has been said by others, wind noise is likely to be the worse problem, unless you have a lid that keeps the noise out well. But a large part of that is surely down to a good fit? Also, since discovering the HJC RPHA, I get a lot less buffeting than I've ever had with any other helmet, and this reduces the noise considerably, through smooth airflow.
I know ideally I ought to use earplugs anyway, but I just don't like having them in when riding - I have tried them once or twice, and it's not for reasons of discomfort that I don't like them. I'd just prefer to be able to hear better what's going on around me. I gave up on having music on whilst riding for the same reason. Particularly, I like to be able to hear clearly what the bike itself is up to. I know I don't need to hear everything that well, but I just prefer it. I don't think my hearing has been damaged that badly by riding without; more likely to have been damaged by how loud I have music on headphones at home!