Powder not stored in a sealed container will be subject to the environmental conditions and the moisture content will change over time until it reaches equilibrium with its environment. As powder loses its moisture, its burn rate increases and visa-versa. With an increase in burn rate comes an increase in velocity. So, if you're highly tuning your cartridges for accuracy, like competing at a high level, changes in burn rate/velocity is a problem. Some people are confounded when they find their velocity has increased with a particular load using the same powder from the same jug and not realizing it's because their powder had dried out some. And likewise, there's been people complaining about their powder being slower not realizing their highly humid environment that they reload in has added moisture to their powder over time.
Where YOU have a problem leaving your powder in you CM after reloading will depend on how long it sits there before your next reloading. An extremely dry environment will act like a dry sponge and pull moisture out of powder pretty quickly. I know this because I also live in an extremely dry environment and have measure and watched what happens with my powder I I use it.
For example, new bottles (1 pounders) of Retumbo that I got had a moisture content of ~52%. A month later as I had less than 1/2 lb left in the bottle I was using; the moisture content was at ~39% and I was getting MV's substantially faster. The environment I reload is inside my home where the humidity runs pretty constant at around 29% (outdoors it can be as low as 7% this time of year, but that's not an issue for reloading). I pour what I need into a hopper and when I'm done it goes right back into the bottle and is sealed tight until the next reloading session. You'd think I wouldn't loose much moisture doing this, but apparently, I do. So, now I use humidor moisture packs in my powder containers to maintain a certain moisture level, especially in my 8lb jugs, and that has helped keep my loads much more consistent over time, which is important to me as I tend to be on the OCD side of things with it comes to precision reloading.
If precision reloading isn't really your focus, then I feel you really don't need to worry about leaving your powder in your CM between loading sessions. It'll still go bang.