I've been to Japan 10 or 12 times. The 6 most memorable and impressive sites I visited were the Imperial Palace and Tsukiji's fish market (Tokyo), Kinkaku-ji, Kiyomizu-dera, Ryoan-ji (all in Kyoto), and Todai-ji (Nara).
I've been to the Imperial Palace 3 times, twice before 9/11. In those days, you were allowed to wander anywhere except the Emperor's quarters, and the "private" rock gardens and ponds weren't off limits then. (Used to be you could walk through the buildings' hallways, too, and see many of the rooms and furnishings where different samurai classes and staff were quartered.) The last time I went was prolly 6-8 years ago and most of the buildings are off-limits now, too. The Pine Corridor was accessible then (at least, where oral history says it was, was). Still, it's an impressive tour if you have the time; the area is expansive and just imagining the power wielded in the ancient world to get such huge structures built to precision in such short a time without power tools is a boggle. Just the outside moat is impressive.
The fish market in Tsukiji is amazing to see; ya gotta be there *early*-- about 4am to see the market while it's working. Biggest fish market in the world; you can't see the horizon to a wall standing under a roof. I forget-- bigger than 5 (or was it 9) football fields? I forgot... best udon in Japan can be found in the surrounding street restaurants, probably because the fish is so fresh there.
Kinkaku-ji, the gold-leafed temple, and Kiyomizu-dera, the Buddhist temple built without nails are both big photo ops. So is Todai-ji, one of the biggest wooden buildings in the world (housing the Daibatsu).
Ryoan-ji, the Zen temple's rock garden is 'just there'; nothing really impressive except for all the manicuring it takes to keep it up to "standards". It's 'way outta the way from everything else as I recall, so it may not appeal, but it's another photo op.