Subjective/objective testing /AB / AB/X, thoughts.

[Overdose]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

Is that statement a joke?

What I meant was, if one correctly and consistently hears differences in a sighted test is that just a fluke? By correct I mean one would come to the same conclusions in a blind test.

So you believe that's just not possible? Again, a simple Yes/No.

No, it's not a fluke, it's expectation bias.

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

 

[Overdose]

The certainty is what discrepancy an individual finds between sighted and unsighted listening. Whilst this is not important to some, others may like to know that they can justify their expenditure based on sound quality alone.

There can be no more certainty from a comparison of sighted and unsighted listening than there can be from unsighted listening alone -- unless I've misunderstood your point.

The discrepancy being the results of a sighted test versus a blind test, ie there will certainly be a difference between the two test results.

Edit: Perhaps my first sentence seemed ambiguous.

 

[manicm]

The certainty is what discrepancy an individual finds between sighted and unsighted listening. Whilst this is not important to some, others may like to know that they can justify their expenditure based on sound quality alone.

There can be no more certainty from a comparison of sighted and unsighted listening than there can be from unsighted listening alone -- unless I've misunderstood your point.

The discrepancy being the results of a sighted test versus a blind test, ie there will certainly be a difference.

A long winded answer when a simple Yes would have sufficed, so in my example I would 'certainly' be wrong if I made preference clear. Case closed then ladies and gentlemen.

 

[John Duncan]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

 

[chebby]

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

Not all that glistens is worth a listen?

 

[DocG]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

that you're trying to upset Matt? :silenced:

 

[Overdose]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

I know you like shiny things JD, but all that glitters is not gold.

We can deduce that a blind test would have levelled the playing field and the audible merits of the equipment on test would have been isolated.

 

[John Duncan]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

I know you like shiny things JD, but all that glitters is not gold.

We can deduce that a blind test would have levelled the playing field and the audible merits of the equipment on test would have been isolated.

So cover amps up and they all sound ****, is that right?

Could we alternatively have deduced that it actually does sound a bit ****?

It's 'glisters' by the way.

 

[Overdose]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

I know you like shiny things JD, but all that glitters is not gold.

We can deduce that a blind test would have levelled the playing field and the audible merits of the equipment on test would have been isolated.

So cover amps up and they all sound ****, is that right?

Could we alternatively have deduced that it actually does sound a bit ****?

If you want to make a call on a piece of audio equipment in a sighted stting and in isolation, all well and good, but if you want to compare it to something similar, then you would have a much better chance of an acurate and balanced assessment by testing unseen.

You'd never make a good Sherlock Holmes by the way.

 

[chebby]

It's 'glisters' by the way.

I know, but that only rhymes with things like 'blisters' and 'sisters' and 'twisters'.

 

[John Duncan]

It's 'glisters' by the way.

I know, but that only rhymes with things like 'blisters' and 'sisters' and 'twisters.

That was not directed at you, whom I recognised had made a change in order to rhyme with 'listens'.

 

[chebby]

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

It ain't worth the bling if it don't have that swing (doo-ah, doo-ah, doo-ah, doo-ah ..., etc.)

 

[John Duncan]

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I heard the Devialet thingie thingie at WHF and I thought it looked great but sounded a bit ****. What do we learn from this?

I know you like shiny things JD, but all that glitters is not gold.

We can deduce that a blind test would have levelled the playing field and the audible merits of the equipment on test would have been isolated.

So cover amps up and they all sound ****, is that right?

Could we alternatively have deduced that it actually does sound a bit ****?

If you want to make a call on a piece of audio equipment in a sighted stting and in isolation, all well and good, but if you want to compare it to something similar, then you would have a much better chance of an acurate and balanced assessment by testing unseen.

You'd never make a good Sherlock Holmes by the way.

But if all amps are so close to each other that ABX testing will show that one should choose one on the basis of functionality and aesthetics, how can one (sighted and in isolation) sound a bit ****? Can it only be down to the fact that it was sighted? Or do some amps actually sound a bit ****?

 

[CnoEvil]

Come on Cno, are you (or anyone else) willing to take up the gauntlet and try an ABX test? (sorry to single you out again Cno but we had this conversation before about a year ago and you said that you'd try it one day in the future after you'd had your back operation.)

I can tell it will please you immensely to know, that today I (just for the slagging I'd get) compared a £150 P/C (plugged into a cheap extention lead) with a £550 one (plugged into the wall).......and it sounded like the amp (Electro ECI-3) had been upgraded.

Oh well, there's that expectation bias kicking in again.

BTW. I'm still waiting for you to take up the odd challenge as well.....somehow, one never gets round to it.

 

[matt49]

The discrepancy being the results of a sighted test versus a blind test, ie there will certainly be a difference between the two test results.

Edit: Perhaps my first sentence seemed ambiguous.

So if you do a sighted comparison of A and B and prefer B, then do a blind comparison of the same and prefer B -- then what's the difference between the test results? Sorry if I've misunderstood again.

 

[John Duncan]

The discrepancy being the results of a sighted test versus a blind test, ie there will certainly be a difference between the two test results.

Or, to paraphrase matt49, if I do a sighted test between your system and mine, and then do a blind test, will there be a difference between the two results?

 

[matthewpiano]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

Is that statement a joke?

What I meant was, if one correctly and consistently hears differences in a sighted test is that just a fluke? By correct I mean one would come to the same conclusions in a blind test.

So you believe that's just not possible? Again, a simple Yes/No.

No, it's not a fluke, it's expectation bias.

If you really want to separate audio equipment based on sound quality alone, then you need to do it blind, so that brand name, price and aesthetics do not skew your opinion.

I think that one word is crucial. I know we are all interested in equipment that sounds good, but I doubt there are many of us who don't also care about how it looks, how it feels to operate and everything tied up with pride of ownership.

I do also agree with the rest of your statement. For what it's worth I've often felt if we could cover over the maker's names on pianos, some customers might choose differently. Not necessarily cheaper options, but certainly different ones not influenced by name.

 

[Thompsonuxb]

I so prefer these types of topics over the "Which amp for my speakers" threads.

Anyhoo, I'm with David@Frankharvey - the diffences can only be truely appreciated over a prolong period - and CNOevil regards testing. Myself I disregard most of these scienetific test as they over complicate what should be an extremely simple excercise - listening to two or more pieces of equ and deciding which you prefer. I'm of the trust your own ears camp ( worry not if you are near deaf or can hear frequencies only dogs should be able to hear.)

I also think "in my opinion" any sort of level matching is nonesense as amps behaive & control speakers differently at a given volume setting (what I've found in my own personal experience). Expectation bias is also a nonesense due to the fact it can only work once in any given test.

Oh, to the person who mentioned "power chords making no difference" I can only guess you have not experimented with them and have no knowledge of the profound differences they make. I for one was genuinly suprised at the degree of difference they do make - more than interconnects I found but are cost inhibited.

Keep the test simple, listen and trust your ears - if you hear difference, then there are differences - sceince has no place in the simple joy of listening to music.

 

[John Duncan]

The discrepancy being the results of a sighted test versus a blind test, ie there will certainly be a difference between the two test results.

Or, to paraphrase matt49, if I do a sighted test between your system and mine, and then do a blind test, will there be a difference between the two results?

Further, could you tell us the mechanism by which you came to the conclusion that your system is 'better' than others available to you?

 

[CnoEvil]

For what it's worth I've often felt if we could cover over the maker's names on pianos, some customers might choose differently. Not necessarily cheaper options, but certainly different ones not influenced by name.

This is a great point, that often doesn't get raised enough.....brand snobbery......or even confining your listening to just the big names.

I started a thread today about a very unknown brand of speakers, and though they were pretty impressive, TBH. I'll be surprised if they generate much interest.

 

[Overdose]

But if all amps are so close to each other that ABX testing will show that one should choose one on the basis of functionality and aesthetics, how can one (sighted and in isolation) sound a bit ****? Can it only be down to the fact that it was sighted? Or do some amps actually sound a bit ****?

There are nearly always exceptions. That design was obviously especially incomptetant.

 

[steve_1979]

BTW. I'm still waiting for you to take up the odd challenge as well.....somehow, one never gets round to it.

I haven't forgoten about the ravioli. It's still on my to do list.

 

[busb]

So sighted tests are absolutely wrong? If I knowingly listen to a Rotel amp and a Cyrus amp of similar power and I have a clear preference would I be so wrong?

I think some common sense needs to kick in here. We don't go into a Hi Fi shop blindfold - their insurance wouldn't cover the potential damage for starters. The testing most of us are ever going to get involved in is informal - with friends, with nobody else, in shops etc. So most of our comparisons are going to be sighted & fairly ad hoc. If we are willing to admit that our senses can fool us, being forewarned is being forearmed.

When it comes to more formal testing to establish if, as an example, amplifiers of similar spec can be distinguished from each other - blind testing will remove most of the doubt that other factors are influencing the outcome - either subliminably or otherwise. The difference between the two is that the only people who need convincing with the informal tests are ourselves - all we can can do on forums is to invite others to repeat what we've done and draw their own conclusions.

 

[John Duncan]

But if all amps are so close to each other that ABX testing will show that one should choose one on the basis of functionality and aesthetics, how can one (sighted and in isolation) sound a bit ****? Can it only be down to the fact that it was sighted? Or do some amps actually sound a bit ****?

There are nearly always exceptions. That design was obviously especially incomptetant.

I thought the same about your exact speakers in the same room. What does that tell us? Was that design incompetent?

 

[CnoEvil]

Oh, to the person who mentioned "power chords making no difference" I can only guess you have not experimented with them and have no knowledge of the profound differences they make. I for one was genuinly suprised at the degree of difference they do make - more than interconnects I found but are cost inhibited.

Good man, I agree completely.

.....and now it's time to put on our straight jackets and get back on the bus.