If groups open up when the barrel gets to a heat level that is common in match shooting, then maybe the load needs to be redeveloped to achieve best accuracy at that barrel temp.
Some barrels have internal stresses that cause a 'walking POI', so a more simple tactic than a barrel replacement to overcome a walking POI when the barrel gets hot is to consider a barrel contact/tensioner pad at the end of the barrel channel. Some Remington factory varmint stocks are(were?) equipped with such a pad built into the stock, most likely to cope with such issues. My experience is that they should be left undisturbed for best accuracy with the original barrel.
For best results, enough upward force needs to be applied that ensures that barrel vibration does not break the contact with the pad. Often a degree of flex/compressibility in the pad (as with firm neoprene foam) can be advantageous. Since these pads will almost certainly modify barrel harmonics, load redevelopment is usually necessary.
This also usually results in a higher POI; and may coincidentally provide the same effect as additional scope base slope.
Greg