I'd slowly piece a set together because you'll end up pitching almost everything in a starter set.
For a press, I'd recommend something high quality that lets you change dies easy without having to re-set them every time, because that can be a pain in the ass especially if you're not doing one big run of brass at a time/bouncing around between cartridges. The rockchucker is a staple and with the hornady bushings you can swap dies out fast, and you can get into one cheap. The Redding T7 turret is a great option and you won't have to worry about having to set up dies every time you switch. I would honestly recommend the Co-Ax because even if you want to get into progressives, the Co-Ax will aways be useful for the odds and ends as well as the smaller volume work. Die swaps are seconds and you won't have to reset, you don't need shellholders, it has a lot of mechanical advantage, and the spent primer catching system is probably the cleanest of the single stage presses. That being said, I still have a rockchucker laying around for when i have to get western and don't want to tear up the Co-Ax (literally spent the last 3 hours beating on that rockchucker with a 2lb hammer hydroforming dasher brass and it's no worse for the wear).
Also, buy some description of auto-thrower. The RCBS Chargemaster Lite is $225 and is worth every single penny in my book. By the time you buy a Powder Measure, a trickler, and a scale, you're most of the way to the chargemaster and the convenience is almost priceless.
I'd research hard into trimming/case prep tools. Case prep by hand sucks the big one. With a lot of this stuff, you'll get something thinking it'll work and you'll not like it and go on searching for a better option and have the old shit sitting in a cabinet for the rest of eternity. If you see yourself reloading in any volume whatsoever, I would strongly urge you to look into a high quality case trimmer and just skip the manual tools. They're spendy but it will save you so much time.
As mentioned, redding are a sure bet for dies. Definitely get dedicated de-priming dies. This helps keep your sizing dies a lot cleaner.
For the little stuff, just pony up. Will a $10 powder funnel work? Definitely. Are you the type of person that will eventually want a funnel with a caliber-specific bushing set? If so, go ahead and spend the extra $50 now, you'll be $10 ahead and not randomly stumble on that stupid funnel and think back on all the money you've wasted.
My $.02