Why are blind equipment tests bad?

[chebby]

It's all to the good in one way though.

More members, more posts, more hits, more views, more interest from advertisers.

I am still waiting for someone to provide references (or links) to a publication that reported on any independent blind test that ever proved that all amplifiers over £200 sound the same.

Author, date, publication... anything?

(3rd time of asking. It seems very well known because lots of people quote it, but no-one is able to reference it.)

 

[Andrew Everard]

chebby:More members, more posts, more hits, more views, more interest from advertisers.

Nah, apparently it doesn't work like that - we only get more advertising on the website if we write nice reviews they like. Allegedly.

 

[aliEnRIK]

chebby:

I would still like the references for (or links to) the independent blind test(s) that proved all hifi amplifiers over £200 sound the same.

It has been asserted so many times on this forum but I have never seen anyone back it up with a reference to a full report in a publication (or provide a link).

I have to assume such a test is hearsay or a hifi 'urban myth' otherwise.

I really cant be ar*ed looking for it but ill quote this ~ "A notable example is the blind listening test conducted by Stereo Review that concluded that a pair of Mark Levinson monoblocks, an output-transformerless tubed amplifier, and a $220 Pioneer receiver were all sonically identical. ("Do All Amplifiers Sound the Same?" published in the January, 1987 issue.)"

Now if you wish to contact 'stereo review' or go looking for the jan 87 issue then be my guest. Even if I COULD find it, there will always be people crying fowl so its kinda pointless me even trying.

 

[aliEnRIK]

Heres an ABX SITE at least indicating the tests do exist

 

[John Duncan]

chebby:
It's all to the good in one way though.

More members, more posts, more hits, more views, more interest from advertisers.

I am still waiting for someone to provide references (or links) to a publication that reported on any independent blind test that ever proved that all amplifiers over £200 sound the same.

Author, date, publication... anything?

(3rd time of asking. It seems very well known because lots of people quote it, but no-one is able to reference it.)

Google "do all amplifiers sound the same" and follow your nose...

 

[John Duncan]

JohnDuncan:chebby:
It's all to the good in one way though.

More members, more posts, more hits, more views, more interest from advertisers.

I am still waiting for someone to provide references (or links) to a publication that reported on any independent blind test that ever proved that all amplifiers over £200 sound the same.

Author, date, publication... anything?

(3rd time of asking. It seems very well known because lots of people quote it, but no-one is able to reference it.)

Google "do all amplifiers sound the same" and follow your nose...

Or do as Rik suggests, since the article is available in pdf........

 

[chebby]

aliEnRIK:I really cant be ar*ed looking for it but ill quote this ~ "A notable example is the blind listening test conducted by Stereo Review that concluded that a pair of Mark Levinson monoblocks, an output-transformerless tubed amplifier, and a $220 Pioneer receiver were all sonically identical. ("Do All Amplifiers Sound the Same?" published in the January, 1987 issue.)"

Ok so you can't be a@@ed looking for it but you can be a@@ed to quote it as gospel, despite the fact that the study is 22 years old and £200 back then is probably worth nearer £600 or £700 today (maybe more) by whatever measure. (House prices, wages etc.)

The onus is upon those who call upon the support of such trials/studies in making assertions to provide the evidence, and at least provide something up-to-date and relevant to todays hifi and todays prices and today's technology.

That test was published only 4 years after CD first became available to the public in the UK. Hardly means much now.

 

[Alec]

But hearing about the 92K thing on the radio last night really was funny. First time ive ever found richard nacon amusing. Ill wager...and hope, actually...that its the last, too.

Just who is richard nacon?! I think i probably meant bacon, but who knows.

 

[the record spot]

chebby:

Ok so you can't be a@@ed looking for it but you can be a@@ed to quote it as gospel, despite the fact that the study is 22 years old and £200 back then is probably worth nearer £600 or £700 today (maybe more) by whatever measure. (House prices, wages etc.)
The onus is upon those who call upon the support of such trials/studies in making assertions to provide the evidence, and at least provide something up-to-date and relevant to todays hifi and todays prices and today's technology.

That test was published only 4 years after CD first became available to the public in the UK. Hardly means much now.

£200 in 1987 is worth around £400 today. According to this site here: http://www.thisismoney.co.uk/historic-inflation-calculator which I thought was rather interesting.

In the case of the current debate, it's probably the same as getting a used kit dealer to take a Pioneer A400, A Leak Stereo 20 refurbished of course, and a current McIntosh sooper-dooper amp at several £'000s. All else being equal, the results might be interesting.

 

[John Duncan]

the record spot:chebby:

Ok so you can't be a@@ed looking for it but you can be a@@ed to quote it as gospel, despite the fact that the study is 22 years old and £200 back then is probably worth nearer £600 or £700 today (maybe more) by whatever measure. (House prices, wages etc.)
The onus is upon those who call upon the support of such trials/studies in making assertions to provide the evidence, and at least provide something up-to-date and relevant to todays hifi and todays prices and today's technology.

That test was published only 4 years after CD first became available to the public in the UK. Hardly means much now.

£200 in 1987 is worth around £400 today. According to this site here: http://www.thisismoney.co.uk/historic-inflation-calculator which I thought was rather interesting.

In the case of the current debate, it's probably the same as getting a used kit dealer to take a Pioneer A400, A Leak Stereo 20 refurbished of course, and a current McIntosh sooper-dooper amp at several £'000s. All else being equal, the results might be interesting.

In which case, if anybody wants to come round and compare my "£400 in today's money" Arcam Alpha 2 to, well, anything at all and see if they can tell the difference....

 

[chebby]

the record spot:
£200 in 1987 is worth around £400 today. According to this site here:
http://www.thisismoney.co.uk/historic-inflation-calculator
which I thought was rather interesting.

Interesting but inflation is not the only measure of the relative value of money now and 22 years ago.

My house was worth about £30k when I bought it in 1985 and about £45k at some point in 1987.

Even allowing for the drop in value of houses recently (I think it is about £170k now) you can see that inflation is not everything.

Average wages have certainly gone up by (a bit) more than double in the last 22 years and so has hifi. (Some hifi has increased in price by as much as 20 percent just in the last year alone!)

A Rega Planar 3 in 1998 cost £274 and now costs £390 (£398 if you allow for the VAT difference).

I bought one around 1987 and remember it was about £180 so that would almost tally with the increase in average (male) weekly wage over the same period. (£232 to £521 or 2.24x)

[Edit. Correction. That should be 'median' not 'average'.]

Using the 2.24X measure then we should be talking about... "do all amplifiers over £448 sound the same?" (I don't believe they do but at least it brings the question up-to-date).

 

[the record spot]

I think the drive in house prices in an unsuitable comparison chebby, driven as they are by more than just component part costs, exchange rate mechanisms and the like (I won't say greed, estate agents and chancers pushing up the value of property as "an investment" than "somewhere to live"...okay, okay, politics over).

And yes, there are the increases of the last year, etc, however, in relative terms, it's a reasonable indicator. You're still talking around double the money, and I think an entry level amp of then is more or less an entry level amp now (Cambridge 640A for instance is now not far off the £400 point as opposed to the £250 it started at).

The Mark Levinson - Pioneer - Leak might be a good test though; interesting comparison for sure. Count me in on that one if anyone sets something up!

 

[chebby]

the record spot:I think the drive in house prices in an unsuitable comparison chebby...

Which is why I settled on average (median) earnings as a better yardstick. I just wanted an example (houses) of something that represents a major long term financial commitment for most people that does not obey simple inflation over the 1987 - 2009 period. It certainly affects the disposable income of many hifi buyers.

Put it this way: If I made us all live on beans on toast every day and asked the wife to pay all the bills instead of me, then I could afford a decent mid-fi system every month (before the divorce papers are served) because I bought a house in 1985. Someone who bought the same house last year (with the same household income) could only buy an iPod ! That is the difference it makes.

 

[the record spot]

I'm missing your point Chebby; I think you've established that the purchase of a house impacts whatever else you may or may not be able to afford and that the drivers influencing the pricing of other items are affected by different forces than those involved in property price increases?

If you bought a place 20 years ago, your salary will have gone up, your disposable income likewise and how you choose to spend your money is another issue entirely compared to someone who's taken out a mortgage on 6x (or whatever) their salary for their studio flat and can't afford the tin of beans you mentioned...

 

[idc]

knightout:

The amplifier blind test that indicated they all sounded the same. Richard Clark amplifier challenge? $10,000 prize

Due to the restrictions on the comparison, particularly in my opinion the ones in bold I am not suprised no one won.

Cost to take the test is $200.00. $500.00 for people representing companies. Payable in advance

All amps must be brand name, standard production, linear voltage amplifiers. The amplifiers in the test must be operated within their linear power capacity. Power capacity is defined as clipping or 2% THD 20Hz to 10kHz, whichever is less.

The levels of both left and right channels will be adjusted to match to within .05 dB. Polarity of connections must be maintained so that the signal is not inverted. Left and Right cannot be reversed. Neither amplifier can exhibit excessive noise. Channel separation of the amps must be at least 30 dB from 20Hz to 20kHz.

All signal processing circuitry (e.g. bass boost, filters) must be turned off, and if the amplifier still exhibits nonlinear frequency response, an equalizer will be set by Richard Clark and inserted inline with one of the amps so that they both exhibit identical frequency response. The listener can choose which amplifier gets the equalizer. The EQ is most likely to be used when comparing a tube amplifier to a solid state amplifier

If our speakers are not acceptable, the listener can provide any commercially available speaker system as long as it uses dynamic drivers. The actual measured impedance cannot exceed the rated load impedance of the amplifiers tested.

No test signals can be used - only commercially available music.

A test session will consist of 12 A/B sequences. Passing the test will require a positive identification of each amp for all 12 sequences. The listener must pass two complete sessions of 12 comparisons. Passing the test means 24 correct responses.*

The conditions of this test are similar to the James Randi cable challenge, they are designed to be virtually impossible to pass. Here Richard Clark will do everything to make all of the amplifiers sound the same and then asks for 24 correct responses, 100% correct. If he and James Randi are so convinced of the similarity between hifi prodcuts, surely that similarity should be present all of the time in all circumstances?

 

[JoelSim]

knightout:

So blind testing and for that matter taking measurements with test equipment does not work with hifi because these are designed to remove subjectivity and bias. The appreciation of music is subjective, and the reviewers are unbiased so these methods are not needed. When measurements or blind testing disagrees with reviewers it is due to the methods being flawed. When reviewers apear to be biased it is due to unfortunate but happy coincidence.

Claims made by the magazine are in no way an extention of advertising and are impartial opinion. If a reviewer says they can hear a clear difference between digital optical audio connectors, or see a clear difference between hdmi cables, that is completely different to manufactures claiming their products improve performance and rightly the Advertising Standards Authority is powerless to call for proof, even if manufactures then refer people to these impartial opinions.

Any requests for reviewers to validate their opinions with data or double blind testing, is a attack on their integrity, and will rightly cause self righteous indignation.

Well that has put my mind at rest, all is well in the world of hifi reviews. It was justÿsilly me.ÿ

I believe cables could be easily double blind tested with a ABX Switcher or Comparator with AB and source direct. But this would obviously be a pointless and worthless exercise. So I understand why you dont bother.

The ASA are a powerless organisation...you can run what you like in the press and they have no powers at all. The BACC/RACC on the other hand require approval before anything is broadcast. No clock number, no commercial.

IMHO we have to concede knowledge to the people who have heard hundreds of systems over many years. Having said that there is always room for personal preference. What you like ie a reviewer is not the same as AN Other reviewer. The kit you are used to at home must play a part in what you hear with 'new' systems. In that way nothing is 'correct', it is merely opinion, albeit an experienced viewpoint.

ÿ

ÿ

 

[Anonymous]

idc:
The conditions of this test are similar to the James Randi cable challenge, they are designed to be virtually impossible to pass. Here Richard Clark will do everything to make all of the amplifiers sound the same and then asks for 24 correct responses, 100% correct. If he and James Randi are so convinced of the similarity between hifi prodcuts, surely that similarity should be present all of the time in all circumstances?



I would have thought you golden eared audiophiles would easily manage 100% success when comparing a £200 amp to a £2000 one. If the bar was set too low it would be too easy to win by chance alone.

 

[Andrew Everard]

Mr Modesty:I would have thought you golden eared audiophiles would easily manage 100% success when comparing a £200 amp to a £2000 one. If the bar was set too low it would be too easy to win by chance alone.

Nice one, Cyril!

 

[JoelSim]

Talking about houses, I am a chancer Chebby, and the value of my flat is probably less than when I bought it.

Having said that, I may put it down as my main home, buy a couple of toilet seats, get the moles evacuated from the garden, get the wisteria removed...oh and let my parents live there...

Oh to be an MP rather than a mere mortal

ÿ

 

[chebby]

JoelSim:Talking about houses, I am a chancer Chebby, and the value of my flat is probably less than when I bought it.

Don't worry it all comes right in the end. Just after buying our house we saw interest rates soar to 15.8 percent and we only had my income. Oh and our first child was born two days before we moved in!

I remember deck chairs and hand-me-down furniture for the months it took to save up for some half decent stuff of our own piece by piece. Unemployment was around 4 million at the time so there was no pressure!

At least we dodged all that negative equity stuff in 1989. When the property bubble burst, the value of our home dropped back to a level still comfortably above what we started out at.

When we had our first kid we had already realised we would be poor for the next 20 odd years so our expectations were not too high. Assume the worst and anything else is a bonus was almost the family motto for a while.

I forgot what disposable income was until about 7 years into the mortage then we had another kid and the next recession in 1992 just to stop us getting complacent!

 

[JoelSim]

Yep, they're not cheap.

And my little 'un knows he's not allowed near my pride and joy. In fact within 5 feet of it. He's been taught well.

ÿ

 

[idc]

Mr Modesty:idc:

The conditions of this test are similar to the James Randi cable challenge, they are designed to be virtually impossible to pass. Here Richard Clark will do everything to make all of the amplifiers sound the same and then asks for 24 correct responses, 100% correct. If he and James Randi are so convinced of the similarity between hifi prodcuts, surely that similarity should be present all of the time in all circumstances?



I would have thought you golden eared audiophiles would easily manage 100% success when comparing a £200 amp to a £2000 one. If the bar was set too low it would be too easy to win by chance alone.

Not if the challenger sets both amps so as they sound the same. I have auditioned amps from Rega, Roksan and NAD one after the other with the same music, same volume and same other kit and there were clear differences. If the test is kept very simple and all of the challengers terms and conditions are removed I am quite sure me, you, everyone on this forum would hear a difference.

 

[chebby]

I liked the bit about plugging valve-amps into an equaliser to 'tweak' the response until it sounded normal before testing started!

The guy obviously had no idea why some people buy valve amps.

 

[idc]

That was the part that had me thinking, am I missing something or is this not a blatant act to make all of the amps sound the same!

 

[aliEnRIK]

chebby:
aliEnRIK:I really cant be ar*ed looking for it but ill quote this ~ "A notable example is the blind listening test conducted by Stereo Review that concluded that a pair of Mark Levinson monoblocks, an output-transformerless tubed amplifier, and a $220 Pioneer receiver were all sonically identical. ("Do All Amplifiers Sound the Same?" published in the January, 1987 issue.)"

Ok so you can't be a@@ed looking for it but you can be a@@ed to quote it as gospel, despite the fact that the study is 22 years old and £200 back then is probably worth nearer £600 or £700 today (maybe more) by whatever measure. (House prices, wages etc.)

The onus is upon those who call upon the support of such trials/studies in making assertions to provide the evidence, and at least provide something up-to-date and relevant to todays hifi and todays prices and today's technology.

That test was published only 4 years after CD first became available to the public in the UK. Hardly means much now.

I'll see what I can dig up in time. But the fact remains that blind listening tests are dubious at the best of times.

And I said I couldnt be bothered as I was at work and really didnt have the time for a decent sweep of the net so I simply googled and put that in within 1 minute. Is that ok or are you gonna shout at me again?