I found that using fire-formed brass made a large positive difference on my LD results.
First, case capacity changed by a lot (probably by more than a grain of powder using Varget in a .308 case) even with running the case through a body die after the first firing. I would think your case capacity would increase even more given that you plan on only neck sizing.
Second, and I think most important, neck tension was noticeably different. I could feel the difference while seating bullets and it showed in my SD and ES numbers once I fired the rounds. The neck tension on my virgin Lapua brass was very tight. With the previously fired brass, once the necks were blown out and then resized with a Lee collet die, there was a great deal more consistency in seating feel and velocity. I didn't do a controlled and careful test of virgin brass vs once-fired brass, but I'm talking about a difference in 5-shot group SDs of about 12 fps with the virgin brass vs. 5 fps on the once-fired brass. Could be a coincidence, as a 5-shot group isn't much of a sample, but given how obviously different the neck tension was while seating bullets, it seems to matter to me.
I don't like the idea of wasting money fire-forming, but, for me personally, I don't plan on ever doing LD on virgin brass again.
With a recent batch of virgin brass, I fire-formed with cheap bullets and about a 2/3 powder charge. I also won't be doing that again either, as when I ran the cases, which were mixed in with some previously-fired full-powered powder charge loads, through the body die, I could easily tell which cases had been fired with the reduced loads. I have to believe that case capacity in the lower-power loads has not increased as much as in the full-power loads.
In sum, I'm going to fire form all virgin brass with full-powered loads going forward before doing LD or using the rounds for any precision shooting.
Hope this helps.