The RS4 is pretty much exactly as you described. Performance first. Comfort second. If you want sheer speed, there are usually some mild to moderate trade offs. Note,
AFAIK, it is only available with a manual, assuming transmission choice is a concern.
I can't even fit in an Audi TT, so I don't have much to add about the TTS. Like you said, it is more of a sports car, but it is also a cramp little car inside if you value comfort. That might not sound like of a problem now, or even during a test drive, but consider how it could feel after driving a few hours.
Of the models mentioned, the RS4 is the prestige car. Limited production and built for sport performance. Just know what you are buying, as there is a high probability of an used RS4 being ran hard.
About other brands, you might also want to take a glance at Mercedes CLK-class models. The W209 platform is far from slouching, and the AMG variants can certainly deliver impressive performance. You can also source a seven-speed automatic, so you might even actually get 20+ MPG on the highway.