You didn't say if the rifle was bought new or used; if used, did you get an accurate round count through the barrel? What barrel profile? Chassis or traditional stock? If the latter, is it bedded? Have you asked a more experienced shooter to try your rifle?
I have about two years experience with 6.5CM, first in a Tikka T3X TAC A1 and then a custom build on a Stiller TAC-30 action in an MPA chassis. Both were/are consistent 0.5-0.6MOA rifles at 100 yards with every factory round I've tried (Hornady 140/147 ELD-M, Hornady American Gunner 140 HPBT, Federal 140 something, Defender 142 SMK).
If you're new to precision rifle, there is a very good chance that your trigger pull and follow-through are responsible for inconsistency. I've been shooting almost six decades and I still have days where I think I could encase my rifle in a ton of concrete and still jerk at least one round in five off whatever point of aim it set up on.
Here's one simple mechanism for checking trigger pull and follow through: Have a range buddy load your mag or rifle out of your sight and include at least one dummy round. If you're pulling the rifle off point of aim as the shot breaks, you'll see the crosshair jerk when you trigger on the dummy round. The crosshair shouldn't move at all with the dummy round, and it should come back from recoil on or very very close to POA with a live round. You need to learn to see exactly what that crosshair is doing at the precise instant the shot breaks.
Finally - are you avoiding an overheated barrel, especially if you have a lighter profile? I was recently approached on the range by a new shooter with similar questions to yours. Turned out he had a Ruger RPR in 6.5CM and he was basically dumping one mag after another, shots maybe 5-10 seconds apart for a full magazine, reload & repeat. That barrel was smokin' hot. He was just banging away at the relatively huge steel plates; he wasn't learning anything from his shots beyond whether he hit the plate or not. Try to learn something with each round fired. The online training offered here on the Hide is a bargain for new shooters.
Good luck.