davedotco said:
Double blind and ABX testing are two different techniques, i believe they are quite different in psycological terms, thogh matt49 will put us straight on that.
In both cases the test equipment is set up in such a way that all variables are excluded, apart from the the items being compared, ie in the case of an amplifier test the tone controls would be set flat, the output levels carefully matched and care taken that both amplifiers are working well within their designed capabilities.
In a double blind test the test subjects are played excerts of music on one system, then on the other. They are simply required to say which one they think sounds best. Importantly, the person carying out the test does not know which amplifier is which at any point.
An ABX test is subtly different, in this case music is played on one system then the other, then a third time on either the first or the second system. The subject as asked whether the third sample is a repeat of the first or the second sample.
This is repeated as often as possible, with as many subjects as possible and the results calculated statistically, but I believe differently in the two cases.
There are also variations on these tests where tests are inserted where the two excerts are played on the same system. ie there should be no difference, and the results of these tests analysed too. All of these results can be (and have been) subjected to rigorous statistical analysis.
All of this analysis is of interest to academics but the ordinary hifi enthusiast simply wants to know whether he can tell a difference or not and the answer is often not.
Perhaps more importantly the enthusiast gets to see just how tiny the differences can be between, apparently, quite different products.
I think there are subtle variations on ABX tests but the one I've participated in we were not told exactly what we were testing. We were told that we were going to listen to various pieces of music more than once, that we could take notes, request repeats discuss what we thought, advised to concentrate on one particular aspect when comparing etc. Not strickly ABX but a varient. We were told at the end that we were listening to 3 different music server solutions including a bog standard HDD at varying bit depths.
My understanding of a double blind test is an ABX test (or very similar) where the person conducting the test is also unaware of the exact order of the tests though they will probably have to know what's being tested! The idea being that the testers cannot deliberately or sublimably influence the subjects or be seen to influence the subjects. Sighted formal tests are a complete joke that no self-respecting music lover should defend if they have any respect for impartiality.
I, like many have conducted sighted
comparisons at home. These do require a degree of caution to reduce Expectation Bias such as:
Deciding what we want to achieve & to be honest with ourselves - leave that axe in another room!
Acknowledging the principle of bias exists & the probability that no one is immune
Change one thing at a time
Undo any changes
Redo the changes
Not to take the process too seriously - have breaks & have fun
Consider long-term tests that last days, not minutes if possible
In conducting these comparisons with other interested parties, consider introducing blind tests (if only to gauge what difference it makes!)
My suspicion is the the shorter the test period the greater is the likelihood for confusion which may seem counter-intuitive & might also be wrong! The reason for suggesting it is because I've noticed differences months after changing something when I haven't been listening for them but have got confused when listening over minutes such as in a showroom. My opinion is that any who thinks they have perfect ears, are never biased & can pick out the the make of connectors being used are probably fooling no one apart from themselves.