I carried an original XDM9 for several years and actually really enjoyed it. The grip felt great, the trigger was decent, it was smooth recoiling (despite the high bore axis), and 19+1 on top is hard to argue with (plus another 38 in spares)! I took two 4-day classes with it, and ultimately put north of 5,000 rounds through it with only a single stoppage that I attribute to the ammo (WWB).
However, I switched to Glock for a few reasons. For one, they're MUCH lighter. A G17 (direct comparison, size-wise, to the XDM) weighs almost 8oz less. Second, I can completely bench strip a Glock with just the armorer's tool, and that's it. With an XD/XDM, you need actual tools (punches, hammers, etc.), so the simplicity was hard to ignore. Add in the fact that I got a really good deal on my G19 as an LEO trade-in (and it's an early two-pin Gen3, which is kind of cool!), and I couldn't pass it up. Lighter, smaller, fewer parts, easier to work on. I put in a (-) connector and some AmeriGlo tritiums and had a local guy stipple it, and I've been rocking it that way for a decade, though a nice G48 recently replaced it as my EDC.
In short, if the XD/XDM works for your, run it. I've heard the stories about failures, and I can't discount them since I hear them from credible sources. But they're not incredibly frequent. I'd own another if I found one at a good price.