The latest Hodgdon shows 41.0 - 45.0 so you're definitely not running hot. That is assuming you have a .308 which it sounds like based on OAL.
I have a 700 in .30-06 that likes Nosler Ballistic Tips .010" off the lands and am not constrained by mag length, they go into and feed out of the mag just fine.
Also, can you define what you mean by, "could not seem to group to well?" My 700 runs that load around 5/8 - 3/4 MOA and I'm perfectly happy, but that might not be so if I were trying to place in a match.
Have you tried other powders, maybe something faster or slower? I was using IMR 4895 in my 700. Just as a particular rifle likes some bullets but not others, the same is true of powders. Have you had excellent results with Varget using other bullets in a similar weight class? Do you know what the twist rate is?
I'll follow this with great interest because I just picked up 1,000 178gr ELD-X for the long range BC that will double as an effective hunting bullet, even though I have yet to load or fire one round. I'd like to use those instead of the BTs in the 700, an M1A, an RPR and an AR-10.