Caliber and grain orientation.
If it was a "quarter sawn" set up (which would actually be flat sawn due to configuration) and really straight grain, I think it'd hold together. But you are right, that would be a weak point... talk about scope "eyebrow"... damn. That one might be permanent!
if the grain is straight and running top to bottom it has the best chance of survival. Grain running across, side to side, and it will be more prone to failure.
Downside is The wood will move left to right if it does at all, whereas flat sawn will move up and down if it does. Depending on how much barrel relief you have, contact could happen.
You could also also do a laminate with a sugar maple core and black walnut on either side. I'd still orient the grain top to bottom. Maybe some blood wood veneer for color in between.
I did a few solid quarter sawn (bolt on) necks and all my neck-thru body ones were laminated quarter sawn sugar maple ironically with black walnut veneers at the glue lines (mainly for appearance) and those are very stable. But I also air dried my boards after they were kiln dried, and the glued up blanks were also allowed to age at least a half year before moving on to more work.