Quite a few questions:
Reading the specs on your motherboard:
I see that it takes up to 16GB of DDR2 8500 800/667 ram. It also support Dual Channel (as do most boards these days) Since you are running a 32bit OS then it won't make sense to put more than 4GB in since that is all that will be recognized. If you plan on moving up to a 64bit then you can get more at that point.
Just looking briefly online I found that you can get around 4GB for $100 on Newegg.
The question of what and how much you should get really depends on what you are planning on doing with your machine. If you are going to be running things that will present a heavy load to your system then it is a good idea to put as much RAM as you can in. However if this is a machine that you are just going to use for word processing then that will be overkill. based on your video card I'm guessing you are going to be gaming on it and in that case I would go with the most RAM with the best quality I could.
As I mentioned, 4GB is around $100. $97 to be exact and that doesn't include shipping. So yes you could order one 2GB stick now and another down the road. I would recommend getting at least 2GB if you are going to be doing any gaming.
As far as the video cards go, from what I've read there doesn't seem to be a lot of difference between the two. One may have features that are better for certain games but overall they appear to be fairly evenly matched.
The 3870 got an overall score of 10399 where the 9600GT got a 9925.
In a FPS Benchmark at 1680x1050 the 3870 had 464.9 where the 9600GT had a 434.
In both tests it looks like 3870 out paced the 9600... but only by a little.
As far as your power supply... The size of your board won't matter much here. What will matter is the type of case you have. If your case is a micro ATX then you will need a microATX power supply so it will fit. If you have a full ATX case then you should get a full ATX power supply.