The extreme spread is something that I've tried to explain to even those with a good understanding of long range shooting. They know how important it is but don't realize on the 4k plus shots being attempted are pretty damn hard if not impossible to make because it takes just a few fps to make the difference like you said. Not to mention 1 or 2 mph of wind speed change, it's just not a realistic goal with a lot of the setups people are using.
The same phenomenon concerning ES can be seen by shooting a 22 rimfire at ELR distances for a 22, even with expensive ammo. Nevermind the ES, the slight wind blowing your bullet clean off the target is just as annoying. Then it occurs in your mind, why am I doing this, it's darn 22, lol. Fun to try, but...
Take it a step farther and pick any cartridge, it will have reasonable limits as to how far away it'l provide a decent hit ratio.
The same thing with a giant ULR rifle and cartridge at those ultra long distances. It get's to the point of being silly. A large amount of disposable income involved makes for great experimentation and some success but the purpose is getting fuzzy other than saying you hit a large target at 5000Y after a large volley of rounds.
Of note though is, that the giant ULR rifle and cartridge, at distances that would be hard for hitting steel with, for example say a 300WM, is substantially easier to hit with for the giant, the 300WM has done petered out. As long as consistently low ES and good accuracy exist you'll have a better chance hitting with a giant higher BC bullet that blows less in the wind. That's what I discovered with my 375CT, it had about half the wind drift as my 30 cal and extended reliable hits another 500 or so yards on the same sized steel, that is if the 375 didn't "hangfire"". I never took it past 2500Y, but on the other hand, I saw how much harder it was to hit the same steel at 2500Y vs 2000Y, so 3000Y would double or triple the difficulty and it had better be calm out.