Why are blind equipment tests bad?

[Big Chris]

JoelSim:
Yep, they're not cheap.

And my little 'un knows he's not allowed near my pride and joy. In fact within 5 feet of it. He's been taught well.

ÿ

It's amazing what an electrified fence can do.

;-)

 

[Anonymous]

aliEnRIK:
ÿ

I'll see what I can dig up in time. But the fact remains that blind listening tests are dubious at the best of times.

And I said I couldnt be bothered as I was at work and really didnt have the time for a decent sweep of the net so I simply googled and put that in within 1 minute. Is that ok or are you gonna shout at me again?

That simply isn't true, Rik, blind testing can be useful. Remember we're talking about all HiFi equipment and components, not just a narrow section of the market. As mentioned in my first post, blind, sighted and extensive sighted all have their place, and are all equally valid. If there is mischievous blind testing, I'm sure the same can be said for sighted.

Anyway, this thread seems to be stuck in a loop now.ÿ

 

[Anonymous]

idc:Mr Modesty:idc:

The conditions of this test are similar to the James Randi cable challenge, they are designed to be virtually impossible to pass. Here Richard Clark will do everything to make all of the amplifiers sound the same and then asks for 24 correct responses, 100% correct. If he and James Randi are so convinced of the similarity between hifi prodcuts, surely that similarity should be present all of the time in all circumstances?



I would have thought you golden eared audiophiles would easily manage 100% success when comparing a £200 amp to a £2000 one. If the bar was set too low it would be too easy to win by chance alone.

Not if the challenger sets both amps so as they sound the same. I have auditioned amps from Rega, Roksan and NAD one after the other with the same music, same volume and same other kit and there were clear differences. If the test is kept very simple and all of the challengers terms and conditions are removed I am quite sure me, you, everyone on this forum would hear a difference.

Are you saying that terms and conditions affect your hearing?

 

[Anonymous]

chebby:
I liked the bit about plugging valve-amps into an equaliser to 'tweak' the response until it sounded normal before testing started!

The guy obviously had no idea why some people buy valve amps.

It's to remove the obvious differences caused by the uneven frequency response of some designs.

 

[Andrew Everard]

This thread takes ever more bizarre twists.

 

[matthewpiano]

Mr Modesty:chebby:
I liked the bit about plugging valve-amps into an equaliser to 'tweak' the response until it sounded normal before testing started!

The guy obviously had no idea why some people buy valve amps.

It's to remove the obvious differences caused by the uneven frequency response of some designs.

That frequency response is one of the main characteristics of a particular design!! Removing those characteristics renders the whole test completely pointless!!

 

[Andrew Everard]

Oh good, I'm glad it's not just me who is baffled by this idea of using filtering to make all the amplifiers sound the same, then proving that they all sound the same by blind listening...

 

[matthewpiano]

To be honest I'm baffled by quite a bit of what lies within this thread.

 

[Anonymous]

"That frequency response is one of the main characteristics of a particular design!! Removing those characteristics renders the whole test completely pointless!"

It gives the biggest clue to the listener, but unless you are agreeing with Mr Clark (that all amplifiers sound the same when they are engineered to have a flat response) I don't see how it makes it pointless. Frequency responses are almost universally engineered to be as flat as possible, so 99% of amplifiers won't need any eq.

 

[idc]

I have heard and owned enough amps to know that they do not sound the same, though some sound more similar than others. So if you get a group of similar sounding amps, or put amps through equalisers to make them sound similar then it will become harder to differentiate between them

I would rather take a middle ground approach that without doctoring the amps sound and using the same source, speakers etc, whether sighted or not amps will sound different to one extent or another. When I auditioned the Rega Mira vs Roksan Kandy vs NAD (similar price don't remember the numbers) the NAD had a greater difference in sound to the Mira and Roksan which to my ear sounded similar. On that occasion I took the Mira home because it coped better with the badly recorded tester tracks I use for auditioning.

I have got a new headphone amp to replace my old one. Both were from the same manufacturer and I can hear clear differences. I presently use an ipod as my source. It has a wide variety of EQ settings on it. I could fiddle with those settings to make both amps sound very similar, so that unless you had a prolonged listening session it would be hard, if not impossible to differentiate between them. As it is I now find it easier to tell what tracks are lossless and what tracks are at a lower bit rate.

If either the Clark or Randi blind tests were conducted in the downstairs listening room at Loud & Clear in Glasgow, and for the Clark test unadultorated Rega, Roksan and NAD amps were used, or for the Randi test the Naim reference kit was used and the various cables swapped, I have no doubt they would be paying out regularly.

 

[the record spot]

Rega Mira - £500, Marantz 6010-KI, £400. Tried them both out using the same components elsewhere some years ago, the only thing to change were the amps. Had them on extended loan and there was a mile wide sonic difference. Rega - warm, not too dissimilar from my then Arcam Alpha 5 (hence why I didn't purchase as there was no major difference between the two). Marantz - clear, open, detailed. My kind of sound.

End of story and blows the argument out the water for me as you could hear the difference clearly enough whether or not you could see the damn things!

 

[Anonymous]

Not really sure what the argument is any more, but to me there are far more differences between different amps than different CD players, so, if the tests were between CD players, I'd almost bet on people not being able to tell the difference. After all, remember:ÿ

"What's in a name? That which we call a roseÿ

By any other name would smell as sweet."

 

[idc]

Disagree Tarquinh. This time an audition at Richer sounds between CDPs; a Sony (something) Arcam Alpha 7 and a Marantz (something). Again clear differences. My wife was with me at both that and the amp auditions, I have asked her and she is clear that she remembers hearing differences between each bit of kit. The CDP audition was just after we first met and it was one of our first dates. She admitted afterwards that at first she had though it was a load of nonsense. But on hearing each CDP the differences were obvious.

The reason why I think that this discussion is worthwhile is that we are producing evidence that sighted tests are worthwhile and meaningful. So far the evidence for the use of blind tests as suggested by Clark and Randi are not backed by any experience and I certainly do not understand why they are meaningful.

 

[chebby]

The amplifier section in my old Arcam Solo-Mini and the Rega Luna amp I had until a few years ago were both good as were my older amps (years ago) like the QED A-240SA or various NAD amps before that.

None of them though have had the almost insane grip on rhythm or the clarity and ease of the amp I use now. The Solo-Mini was smooth and detailed and the Luna and QED amps were both punchy and 'fast' and the NADs had their own unique trademark 'big' confident sound despite some rough edges.

I can confidently say that despite any overlaps in their various qualities (good or bad) that I would easily be able to pick this Nait 5i out of the bunch (blindfold) if I heard them all again even with the same speakers and at the same volume.

I don't care if my amp is exhibiting 'character' or has been imbued with qualities that might make some guy with a signal generator/anechoic chamber and oscilloscope wince a bit.

It does sound different, I can tell, and I do like it.

 

[Anonymous]

Fair enough, but I don't think many would dispute that the both are useful. I don't know the blind tests to which you refer, but they're only two out of many, and not, from what I've read here, any kind of benchmark.

I've done many a blind test in my time, though nearly always related to wine (used to work in a wine growing region in Oz) and none of the vignerons got upset if their wines weren't liked. Mind you, it was interesting to hear them denigrating their own wine then seeing their expressions when the label was revealed.

Funnily enough, it was after taking home an Arcam Alpha 7 and comparing it with my then Sony something that made me think the difference wasn't as noticeable as with amps. The one I bought, the Rega was slightly differently voiced. Again, not saying there isn't a difference, just that it's not as marked. ÿ

 

[Anonymous]

"I don't care if my amp is exhibiting 'character' or has been imbued
with qualities that might make some guy with a signal
generator/anechoic chamber and oscilloscope wince a bit".

The big question is whether that "character" is purely attributable to different frequency response or if there are other factors involved.

The number of failed participants in the Clark test suggests the former, in which case, good news, one amplifier of a certain power output is likely to sound very similar to another, regardless of price.

 

[basshead]

idc:
The CDP audition was just after we first met and it was one of our first dates. She admitted afterwards that at first she had though it was a load of nonsense. But on hearing each CDP the differences were obvious.

a cdp audition was a date? good to see romance is still alive!....... she did become your wife though so maybe i'll try a hifi audition next time i have a date

 

[idc]

basshead:idc:

The CDP audition was just after we first met and it was one of our first dates. She admitted afterwards that at first she had though it was a load of nonsense. But on hearing each CDP the differences were obvious.

a cdp audition was a date? good to see romance is still alive!....... she did become your wife though so maybe i'll try a hifi audition next time i have a date

It is a little bit different and if are left on your own in the audition room well.........................clicky !

 

[chebby]

Mr Modesty:The big question is whether that "character" is purely attributable to different frequency response or if there are other factors involved.
The number of failed participants in the Clark test suggests the former, in which case, good news, one amplifier of a certain power output is likely to sound very similar to another, regardless of price.

Paraphrasing 'Frasier' from one of his episodes....

At Cornell University they have an incredible piece of scientific
equipment known as the Tunneling Electron Microscope. Now, this
microscope is so powerful that by firing electrons you can actually see
images of the atom, the infinitesimally minute building blocks of our
universe. Mr Modesty, if I were using that microscope right now, I still
wouldn't be able to locate my interest in the Clark test.

 

[Anonymous]

chebby:
Paraphrasing 'Frasier' from one of his episodes....

At Cornell University they have an incredible piece of scientific
equipment known as the Tunneling Electron Microscope. Now, this
microscope is so powerful that by firing electrons you can actually see
images of the atom, the infinitesimally minute building blocks of our
universe. Mr Modesty, if I were using that microscope right now, I still
wouldn't be able to locate my interest in the Clark test.

I tried to watch an episode of Frazier once. Is it supposed to be funny?

 

[Simon Lucas]

Mr Modesty:I tried to watch an episode of Frazier once. Is it supposed to be funny?

If this thread has taught us one thing, it's that 'funny' is a relative term.

 

[professorhat]

Indeed, astonishingly, some folk think I'm funny...

 

[laserman16]

Simon Lucas:

'funny' is a relative term.

 

[chebby]

Mr Modesty:I tried to watch an episode of Frazier once. Is it supposed to be funny?

You take us to the point immediately. As with all things (including comedy) it is a matter of personal taste and tolerance - or acceptance - of what other people like. From the tone of your reply it seems you cannot accept what other people may find funny because you disapprove of it.*

I think similarly some of us cannot accept the huge variety in what people find enjoyable about their hifi's. Like those test methodologies, everything must be brought into line and made uniform, like that super linear, mythical budget amp that beats all-comers. (Once they have all been tweaked, equalised and filtered and hobbled to sound the same!)

Some people are too literal in outlook to 'get' fiction for instance ("but it's all lies and didn't really happen") or poetry or history ("but they're all dead so why does it matter?"). Like the little boy who is made angry by the magician's act because the 'magic' is an affront rather than just accepted as entertainment whether the trick is understood or not. Willing suspension of disbelief. ("Mummy! That cow is really two people in a costume.Tell them! Tell them!")

There are simple developmental tests done at around 4 - 5 years old involving something like a model of a hill and a house with a little model person looking towards the scene. The child is asked to draw what the figure can 'see'. The figure is then moved to the other side of the 'hill' facing the child and he or she is then asked to draw again what the figure is 'seeing' this time. The house should be obscured and the shape of the hill reversed.

It shows whether a child has developed the ability to see another person's point of view yet. Usually by infant school age they have. However there are always a few who will never pass the test. They can never imagine another person would not share the same view of everything as themselves.

They go on to do things like blind testing of hifi amplifiers with rigged entry criteria and results despite no-one else really being very interested. Had they been cleverer they could have spent 32 years berating God and religious faith in print like Richard Dawkins (a total bore but an incredibly rich and famous and shrewd one, who saw that neo-Darwinism, genetics, and religion were 'hotter' issues than home entertainment equipment!)

Being able to change my mind (like about Naim, which I hated the sound of originally) sort of proves to me I have one. I guess after proving myself wrong about my own pre-conceived notions - held for decades - about a brand then I really cannot justifiably have a problem about other people enjoying whatever they like whether it costs £200 or £20,000.

*Yes, I actually do like a bit of Frasier now & then and we have all the boxed DVD sets because my wife and kids love it also. Is there a Clark test on TV comedy entertainment too?

 

[Anonymous]

chebby:
Had they been cleverer they could have spent 32 years berating God and religious faith in print like Richard Dawkins (a total bore

The God Delusion is very interesting and very witty.

Have you read it?