Much like Steve, I've have/had an XRS and an S&B 3-20 as well.
The XRS wasn't bad at all but side by side with the 3-20 the S&B was noticeably brighter, resolved finer detail, and was much better at picking up details in lower light situations. Better clarity off-center as well.
Didn't care much for the XRS from about 24-30x, the brightness noticeably decreased and the resolution just wasn't there to go along with the magnification. Really noticed the lack of fine detail resolution in the XRS when playing the "ground squirrel or dirt clod?" game between 400-600Y; I do a lot of varminting and love to blast the little guys at longer ranges. They tend to blend in with their surroundings really well and at longer ranges it can be tough to distinguish between the terrain and a squirrel if they're just sitting there. With the S&B it was still relatively easy to tell the difference, with the XRS not so much; I often had to wait for them to move or stand up for a confirmation.
For larger targets that offered more contrast with their background (painted IPSC 50% silhouettes, etc) I wouldn't have had any issues with the XRS glass clarity.
Aside from glass quality both tracked well and got the job done, neither scope would be a limitation on making a hit in that department. The S&B had the better feeling turrets and zero stop arrangement. The ERS/XRS locking mechanism wasn't my favorite and there was a little play in the detents, not to mention the zero stop isn't as straightforward as the S&B. Mag ring on my S&B 3-20 is extremely stiff, even moreso when it's cold-- may send it in for that eventually. The 75Y minimum parallax of the XRS was an irritating limitation but not a dealbreaker given the intended use of the XRS whereas the S&B 3-20 has a much more versatile 25M minimum parallax.
As others have said though, the difference in glass quality between an XRS and the S&B is much less than say the difference between a 4-16 PST and the S&B. There is still a difference... just not as pronounced. Even though I own several S&B's I'd say they are past the point of diminishing returns on the dollars spent vs performance scale whereas the XRS at its usual street price is probably right at the sweet spot of dollars spent vs performance.