My brother has been recently visiting and, just for fun, we did a short test to see how worth it is high-res audio to listen to music. He has worked for many years in a recording studio and now repairs and restores professional audio equipment, so has a trained ear and is used to work with hi-res audio, which is commonly used for mastering.
What I did was taking a couple of 48Khz and 192KHz, 24-bit tracks, and compress them with iTunes into AAC VBR 256 Kbps, the standard for iTunes Plus. So the changes from the originals were downsampling to 44.1Khz, reducing the bit-depth from 24-bit to 16-bit, and lossy compression. When I did this, I didn't pay attention to the order in which iTunes had placed the tracks, so when using the iPad as remote I genuinely did not know which one was the original and which the compressed one.
To his credit, he ended up being able to determine which was the original, but it took a while of going back and forward, constantly changing from one passage to another, and lots of concentration. He looked for specific sounds that would be more susceptible of being affected by the compression. I honestly couldn't tell the difference—even though I noticed a couple of extremelly minor differences, I could have not said that in one case the quality was inferior to the other. And he would not have been able to tell without very close A/B comparison either.
This was with my hi-fi setup, no headphones. And in my living-room, which is far from ideal for a test like this in comparison with a studio. Also, there are tracks that suffer more from compression than others, and I don't think that what I used were difficult ones—Getz/Gilberto, and Sting. But still, I am impressed by the compressed tracks that iTunes produces.