If it were just plinking ammo, I wouldn't worry about it.
If you're trying to load target type ammo I would segregate it if I were using multiple charge weights.
It probably won't matter too much if they're within 2 grs. as you say, but I have OCD and that's how I would do it.
If you do load them all up with a single charge weight, I would at least group them by headstamp and fire them in that order.
I try and do 5 shot denominations (typical 5 shot groups,) so if I could do 30 Winchester brand and 20 BB, then that's fine, or 25 & 25, or 35 and 15 for a total of 50, in a box. I wouldn't want to do 37 Winchester and 13 BB, for example, but I'm just a hack, so what do I know?
That way, at least you're being somewhat consistent.
Chris