I think youll be alright sir. I (stupidly) grew up shootng without ear plugs. Figured it would help down the road with being used to it and not flinching, go figure right? I remember two distinct times when I was in my late teens. First was when I shot my dad`s .45 ACP Glock with no earplugs. It was bad. I took about 4 or 5 shots before my hearing was nearly overwhelmed with ringing in both ears. Another time I shot one of our friend`s .300 WM w/ brake, no earplugs. And that was bad. Only took one shot that time. I could almost say it was like in the movies when a arty shell goes off next to the guy and everything is muffled with dominant ringing. Almost. That one made me a believer.
16" Colt 6920 never used to bother me. But after that .300, I dont shoot without plugs unless its a .22 LR.
You should be fine though. Both times, when I shot the Glock and the .300 without plugs, it affected me for quite a while. My hearing did come back to normal, I think. I do have a slight ringing sometimes when its real quiet out in the woods. Needless to say, I doubt I can hear like a Whitetail buck, but regular conversation dont give me any trouble. People with a New York city accent however, is another story altogether.