Hi,
I just so happen to have an almost identical router (662) sitting around, on a shelf not being used though, and most of the clients that my last work had those (660s) too...
If I remember correctly, when you first log in you should be able to see the DNS settings that the router is using - although depending on the version you might need to click "WAN" from the menu...
Are you trying to set up a static IP address for your XP computer, or do you want to set up a DHCP reservation..? Personally, I would suggest going with a reservation - as it is simpler to deal with in the longer term - but if you are intending to not have a DHCP server running in your LAN, then you can do that too...
By the way, LAN is Local Area Network, and includes anything that is inside your network - router/s, computers, network attached printers.... WAN is Wide Area Network, and means anything that isn't within your LAN.....