When a bullet encounters a crosswind, it turns slightly toward it like a weather vane. If you've ever flown in or seen a helicopter fly in a crosswind you'll notice it flying slightly crooked in orientation to it's forward path, same thing.
The aircraft isn't weather-vaning. It is a crab angle intentionally introduced by the pilot to maintain a straight flight path in a crosswind that would otherwise cause it to drift. The stronger the crosswind component, the more crab angle required to maintain the desired flight path. Since the bullet doesn't have a pilot, it has no choice but to drift.
You've got a torque factor on the bullet in flight. It's spinning, acts like a gyroscope. Aerodynamic jump is the same as gyroscopic precession. Right crosswind, bullet points right, gyroscopic factors push the nose of the bullet up. Opposite for left crosswind.