Various manufacturers and makers of monitoring software name their categories differently. While there are generally acceptable or self descriptive terms, sometimes they are still confusing.
For the first item you listed, CPU surface temp, it is unlikely that what they really meant was that you would be measuring the actual surface of the CPU. What it most likely indicates is the temperature read by a thermistor that is mounted near the surface of the CPU itself. Where this thermistor is placed is up to the system builder. Often times it is mounted to the heatsink as close to the area as possible where the CPU and heatsink actually make contact with each other. But realistically, it could be mounted anywhere or to anything. For instance, you could use that thermistor to measure your video card temperature instead of the CPU if you wanted to. This is an especially attractive option if your CPU has an on-die temperature measuring mechanism that can be used to provide CPU temps. In addition, some motherboards used a thermistor that was mounted on the motherboard, sticking up through the hole of the socket the CPU went in to. The theory behind these was that the thermistor could make contact with the bottom of the CPU and thereby approximate its temperature.
The second item you listed, CPU on die temp, is the measurement provided by a diode type device that is actually mounted inside the CPU core itself. This is the most accurate way to measure internal (true) CPU temperature in my opinion, and a whole lot better than using an externally mounted thermistor. However, not all CPU's have an internal diode for measuring temperature. This is especially true of older CPU's. In that case, you would have to resort to using the thermistor mentioned above to measure CPU temps.
Your third item, System temp, is most often associated with the operating temperature of the motherboard chipset.
As for what the reasonable maximums any of these temperatures can be is entirely dependent on what you are measuring. Different CPU's and chipsets for instance have different tolerences, and even the same types of CPU's can have different tolerences depending on their construction. For instance, an original Pentium III versus a Coppermine Pentium III, or an original Tbird versus one of the newer AMD offerings. Keep in mind though that in my experience I almost always experienced errors and flaky operation, no matter what CPU and chipset I was using, long before I got close to the manufacturers stated maximum temperature for the devices. Granted, they didn't burn up or fail completely, but they certainly didn't resemble anything like a stable machine either.
So, what you are left with is a generally agreed upon acceptable range of temperatures with which you can run the system and have it provide you with stable operation. This is of course somewhat subjective. Obviously you want to keep them all as cool as possible, or at the very least you want to prevent them from obtaining a temperature at which they become unstable or risk failure. That of course is going to depend on the components you are using, what you are using to cool them, and ambient case/room temperatures. It may also depend on how hard (hot) you are willing to push your components and how much risk you want to take.
For example, lets say that you had a CPU with a maximum rated temperature of 60 degrees Celcius. Lets also say that this CPU runs fine under full load and at a temperature of 45 degrees celcius. If you could cool it to 38 degrees without much effort, would you do so? Some people would, others would call it good enough. Personally, I want as much difference as I can reasonably obtain between the actual running temperature and the maximum its rated for before failure. This of course is a bit of a tradeoff between cooling options and their costs for me.
As for the theory about running a CPU hotter and having that reduce its longevity, I personally believe it to be true. The degree to which it effects the CPU is probably the biggest area of debate though. Running a CPU at high temperatures, or with increased voltage, can lead to a condition called Electromigration. If you thought my post was long already, explaining Electromigration would kill ya <lol>. So, you can read about it here:
http://www.csl.mete.metu.edu.tr/Elec...ation/emig.htm
The debate comes from the length of time it takes for Electromigration to happen or to become a factor (if at all). There are those camps that say "who cares?". "I don't intend to keep my processor for more than a year anyway, so why do I care that it won't last for the full 10 years of its expected life because of how I treat it"? There are other camps that feel you should attempt to maximize the life of the unit, and take steps to ensure they have done what they could to make conditions for the CPU optimal throughout its use. And there are other folks, like me, that fall somewhere in between.
I encourage you to read the article about Electromigration as its pretty straight forward and explains things really well. Might also help to list what CPU and heatsink/fan combo you are using to get some insight into what people think is an acceptable temperature, or so that you could at least do some comparisons.
Hope this helped. ~Target
hardware
prices 
Free Scan: Update Your PC's Outdated Drivers to Optimize Performance 
It's just that I see so many different contradictory stories told by various people I really want to find some accurate and reliable sources on this. 
