Acoustics make a bigger difference than most people allow for and, for me, acoustics covers a broad range of factors including the shape and size of the room and the positioning of the speakers therein, the nature of the surfaces in the room (flooring, wall coverings, soft/hard furnishings, blinds/curtains), and even the resonant properties of the construction. For example I am convinced that some modern walls cause extra resonances in the bass frequencies and that where this is the case (my own room, for example) it needs to be taken into account with careful speaker selection (Eg. B&Ws are mostly a nightmare in my room, whatever the amplification, because they tend to have a big bass).
Regarding the whole concept of accuracy, I think it is something you can keep going round in circles with. Ultimately it can only come down to reproducing the recording accurately because there are many factors out of our control as listeners during the recording, mastering and distribution processes. To be truly accurate to the recording you would have to listen to it in the studio where it was recorded and mastered and using the same equipment. As this selection of equipment, and the acoustics of the studio, would be different for different recordings you are never going to achieve total, consistent accuracy with a domestic hi-fi system whether judged by measurement or by listening.
Ultimately, it has to come down to how your ears perceive the sound and whether you actually enjoy listening to it. If you enjoy the process of tweaking to get the absolute best out of a system, or to get it sounding accurate to a live sound in the broad sense (and in the way that you personally perceive it), then what is the harm in that? Each to his own, but please don't try to kid anybody that accuracy is as simple as it is so often made out to be.