I have been running a 64-bit operating systems for a fair while now, windows server 2003 x64 and windows xp 2003 x64. I know drivers can be a bit of an issue for some devices. However this complaint against 64-bit applications isn't a problem as everyone makes it out to be, which usually I find that the complaint comes from those who don't use a 64-bit operating system.
The support for emulating 32-bit applications is actually quite good from what I have seen. I haven't had any major issues with installing anything except for adobe and they had a simple work around since the installer didn't support the characters (). So I had to change the installation directory from \Program Files(x86)\ to \Progra~2\ which solved the problem which has later been resolved with their newer media. The 64 bit editions of windows support both 32-bit and 64-bit applications. I fortunately have been able to have my firewall and antivirus software being produced for 64-bit windows, like symantec antivirus and outpost. However I have had to use microsoft antispyware instead of ewido(avg antispyware now) and not install spysweeper. While spysweeper didn't sadden me too much as it they made new innovations to it to make it slower and resourceless efficient. I hardly get spyware any more, tho I do hope thats not due to windows defender not doing it's job.
I think the greater and more logical concern with any upgrade is with applications being compatible with vista, antispyware, antivirus and firewalls and similar programs will all need to be made to run on vista. There probably isn't a lot of OS dependant programs that average people run that won't install that probably hasn't been updated for compatibility to windows xp x64 2003 that I have seen. Anything that requires a driver like many of the cd/dvd authoring programs have been updated and supported for awhile too. Which is one type of program that would matter.
Since it supports both 32-bit and 64-bit applications side by side, it's not a problem and I do have vmware workstation installed, I can install 32/64 bit operating systems if needed. Which I do have vista ultimate x64, windows xp x64 and server 2003 x64 installed. Fortunatly installation through vmware is pretty quick too.
I guess most people do expect windows x64 to be quite faster with everything they do. Which usually it doesn't impress people when it's minimal, however I find it like raid. Many people claim the benchmarks aren't great and the performance isn't worth it. However everything I do does feel faster than before. Which I find is often true with windows x64. Which my O&O disk defragmenter x64 does perform quite a bit better if thats an improvement to x64 itself lol. I even find myself using Internet Explorer 7 x64 quite a bit more now too since that runs pretty well for a lot of my browsing.
I have quite a few applications that were either made natively for 64-bit or had support where it can take advantages of 64-bit. Not in it's entirety, but being able to utilize higher yields of memory and other support which I believe adobe creative suite 2 has and CS3 will further iterate on that.
I even have few games I have installed and tried in native 64-bit like unreal tournament 2004, half-life 2, farcry. I have been meaning to find others to try like Shadow Ops: Red Mercury, however I don't play a lot of different games, mainly steam based games, so it was kind of a novelty thing to try farcry in 64-bit. Half-Life 2 and Unreal Tournament 2004 are quite good games which were fun to try in 64-bit mode which it seems valve had taken a big step further in making 64-bit HL2 which they had converted and made a pretty large GCF file you have to download to play. Unreal Tournament 2004 just replaces an executable. However it runs pretty smooth anyway.
I guess I am lucky most of my hardware works under 64-bit windows, which is another reason I like purchasing nvidia motherboards, creative labs sound blaster cards. They both creative drivers for their products if a new version of windows comes out. Which I think is quite nice of creative labs to do so. They have updated many of their cards to even support EAX4 like audigy 1 and 2, and even release drivers for the new operating systems. The only thing I have had a problem with as printer drivers for my HP officejet, which has 64-bit drivers, but they have been in beta for a long time. However hopefully they will eventually finish them since they support a wide range of products. While when vista is being sold, they should hopefully have products like printers that have support for both versions of vista.
I simply use another computer and have the beta drivers installed so I can do printing over the network.
I guess there are other drawbacks, but it hardly degrades any productivity or gaming that I do. If anything, I can alt tab out of steam games faster