With me, it used to be "It's the sound, stupid!" Now I fully admit the the looks are just as important (they always were). However, if willing to stray from the core brands, you can get the best of both - it just takes more effort but hell, that can be half the fun of acquiring new stuff.
My Totem Arros are as small as towers get but still give a more than decent sound (Art Stilettos are a similar size) & I do get comments from visitors along the lines of "My wife could live with those!" It's usually "live" rather than "love". Looks are a matter of taste - very subjective - but most people with any sense of aesthetics know when something is just butt ugly or not.
As to why so many women approach HiFi with their hand on their hips, I'm not sure. Maybe some feel the need to assert themselves regarding male hobbies as part of the rebalancing of power in a relationship or simply a matter of being house proud - who knows. Perhaps if men did more housework, their spouses would use their arms to embrace them!