Exactly, JB, your experience isn't misleading you. There are many variations to deal with in what we enjoy and as I've said many times, rimfire is always more complex than centerfire. One of these complexities is manifest in the use of a suppressor, although many don't believe it's true....some will say, "you just have to build the rifle right."
I love the many times I'm able to use a suppressor; I have an issued patent for a suppressor with more to accomplish in this area, but I mostly stopped using a suppressor on rimfires a long time ago. Most of my 22's don't even have threaded muzzles. If quiet is what you're looking for based on what you're doing, that's great, but if consistent accuracy and precision is your goal, a suppressor isn't the way to go. I'm not saying there aren't and won't be some rifle and suppressor combos that won't work well, because some do, but largely, a lot will not work well.
MB