Dhom,
I am speaking on my shooting experience but when I had groups that seemed to shift (with a properly torqued, scope in good working order), then return to zero again, it was usually the result of one or more of the following factors.
1. Poorly set parallax - head bobble test to verify correct parallax with each target. when I’d lift my head slightly off the comb and move my eye position by imagining my cheek as the pivot point, the rifle would move less which allowed me to get a getter sense of whether or not the parallax was set correctly.
2. Muscling the rifle - even with a shift of 2-3 inches to the next Shoot-n-see target, the natural point of aim needs to be reset. reset the NPOA with each new target, no matter how small of a shift it may be.
3. Loose thread protector - most of my 22LR rifles are threaded. From the factory, I had issues with the thread protector starting to come loose during my initial break ins. My new procedure for New rifles with thread protectors is to clean the thread with brake cleaner and use blue locktire as my pre-range Inspection and cleaning of a new rifle.
These are my limited experiences With wandering zero shift with #1 and #2 having to do with marksmanship fundamentals and #3 with equipment set up. I hope that you find some value in it.
YMMV, happy shooting.