However, ball ammo runs a lot faster than 3100-3200 fps (his handloads).
Hence my suggestion: Have the OP physically verify the twist rate. This sounds like a 55gr bullet in a slightly too slow twist barrel...like a 16 or 18 twist. With the number of rimfire uppers around these days for AR's, I wouldn't be surprised if the OP bought a "gunshow special" that was really special...
M193 Ball is spec'd at 3,165 fps, so if the OP is claiming 3100 to 3200fps out of this Nosler load, I don't see the difference in velocity between them.
Still no report from OP on what the actual twist of the barrel is though...