Thanks all for the responses, glad this is generating interest. I'll respond to JoeSlim in particular
I apologise in advance for the size of the text, want to try and cover everything.
The situation is, I think, one that has an answer. Cable x is either better than y, or it isn't, given some agreed criterion. I think most people most of the time agree on most of the aspects they want to hear from their music or film, so we have some criteria to measure against.
But we know that people look for positive bias, that they have moods, drink wine on some nights, pair equipment together, etc. So we need some way of removing all of these factors to just leave x and y. And we have one such method: science. Anything else is insufficient. It's like someone saying "well homeopathy works for me" - they are welcome to hold that opinion, but they aren't welcome to their own truth. If someone says, to take an example, that this Nordost Valhalla cable is better than this Bell Wire, they aren't entitled to their own truth.
They are, of course, welcome to say "it sounds to me like Valhalla is better than Bell" and, barring a brain scan, I would take their word that this is truthful
but that's not the question I want answering. I want the cable to justify it's cost, be advertised truthfully, and justify itself over simply the placebo affect with a cheaper cable.
As such, things like expertise, passion, practise, experience etc in cable listening is deserving of respect and worth discussing on an artistic level, but can never rise above anecdote.
You rightly ask how we could do this experiment. Off the top of my head, maybe something like this. We get the two cables, connect them to identical systems, in identical rooms in which the cables cannot be seen. No-one involved knows which is which. This test is repeated on several days, with several types of music, with several people, and finally the whole thing repeated on several systems so as to cover the system matching without having to consciously match them. The result would have to be statistically significant, using perhaps a two sigma standard deviation.
Of course, this would be expensive, time consuming, boring and a general pain in the ***
so what I recommend is that every cable article in the world be prefaced by the sentence "everything that follows is a personal opinion, to which we can assign no likelyhood of truth, or guarantee of lack of bias" . I'm of course not saying that reviews and advice are useless - I will be on this very forum mining as much as I can for my upcoming system
I just want reviews, technical articles, etc to be clear in the level of certainty they carry.