Without empirical evidence, I wouldn't call that a "huge difference".lol..I dunno for me it just felt a lot more stable and everything seemed to run much smoother (much like how it feels when I use WinXP x64.)
The only notable thing I see in Vista 64 (iirc) is the (for lack of my memory of the exact term) the dynamic entry points code loads into, so that hackers cannot target code based on the idea that it will load into the same memory addresses every single time.
Watching MS' XP-64 troubles, Vista 64 will be more of a wait than even my migration to Vista 32-bit will be. I just don't trust that good driver development and 32-bit app compatibility will be there. I hope they prove me wrong, though.