Subjective/objective testing /AB / AB/X, thoughts.

[fr0g]

I don't wish to take sides or get overly involved with this argument, but isn't it all being taken a bit too seriously on both sides?

How important is all this, really? Its only a form of home entertainment, a way of listening to music! Why not enjoy 'knowing' what you 'know' and take advantage of the knowledge, and just leave those who don't 'know', and who are happy not 'knowing' to continue enjoying their take on the matter?

All everybody seems to be doing is winding each other up and that can't be productive for anybody.

I agree it can get a bit serious, and I have been guilty myself. Speaking personally, I think it just riles me when people make unsubstantiated claims in advertisements, or word of mouth, that are generally just guff and take people's money for the pleasure - that's when it really grates and I have a hard time not wading with my size 11's.

Another persona thing that I find is that it's odd that in some realms of hifi, like other walks of life, the faith some people have in things that can be proven to have no effect, yet people still believe they do. not many electronic devices that I can think of that people have that level of faith, even if in the presence of all the proof saying otherwise.

Agreed.

Personally I'd like to see the back of the typical line "Sounds good with MP3, but really comes alive if you use lossless".

And I guess the "Good Samaritan" comes out in many of us. Not wanting to see others "taken advantage of", either by the salesman who suggests buying "more expensive HDMI cables" or whatever.

Once you are determined that something is a rip-off, then you feel the need to spread the word. Even if sometimes you got that one wrong...

 

[John Duncan]

IMO there are three possible explanations for this:

1. the differences are objectively much smaller than many people think they are

Almost certainly the case.

2. blind tests aren’t designed and conducted with sufficient rigour to make their results reliable

That's true, but even unreliable results, given enough people tend to "lead to a conclusion". And if they are VERY positive they can lead to a high statisctical probability

3. people aren’t very good at taking blind tests

I agree that such testing can affect the test by the very fact we are being tested, but again, back to the sample size I think it can be ignored. In the end you're ears are still your ears, and any differences that can't be made out when you're not looking are either not there or are too small to worrry about...surely?

Can I ask a slightly different question from the one originally posted? Namely, is there a list of things that we can agree on disagreeing upon? eg cables (analogue and digital), lossy v lossless, support furniture, creams and pessaries, amplifiers, speakers...

It might make more sense of the conversation.

 

[Alec]

But I don't know anyone's opinion on pessaries.

 

[John Duncan]

But I don't know anyone's opinion on pessaries.

And if you did, you'd tell them to shove their opinions up their EDITED?

 

[fr0g]

Can I ask a slightly different question from the one originally posted? Namely, is there a list of things that we can agree on disagreeing upon? eg cables (analogue and digital), lossy v lossless, support furniture, creams and pessaries, amplifiers, speakers... It might make more sense of the conversation.

I think it may vary a bit person to person.

My view

All properly built digital cables give identical results.

Analogue cables "can" affect the sound, but again, properly built ones sound "pretty much" the same. I wouldn't even look at the brand or model any more.

Speaker cables the same. So long as they haven't got some wild electrical property (very low resistance, very high inductance etc) sound almost exactly the same.

Mains cables sound the same (with the inteference caveat)

Amplifiers need to be sufficiently powerful (Watts and current) for the speakers. I feel there is little to no difference between amps that measure similarly

You can get a DAC, CD player, or a digital streaming device for under £300 that is equally as good as any £10000+ player.

The picture and sound of any Blue ray player played through HDMI will be identical (caveat that the picture is stored in a lossy format and "could" be unpacked differently)

Unless you live on a bouncy castle, specialist rubber supports will make zero difference.

Digital (of any sort) given a competent DAC is higher fidelity than ANY vinyl playback given the same amp/speakers

Probably a few more..

 

[John Duncan]

In other news, I have just googled pessary and it turns out that I am a poor judge of anatomy.

 

[steve_1979]

In other news, I have just googled pessary and it turns out that I am a poor judge of anatomy.

:rofl:

 

[Alec]

In other news, I have just googled pessary and it turns out that I am a poor judge of anatomy.

Oh dear. In that case, your opinion is of no significance.

 

[John Duncan]

In other news, I have just googled pessary and it turns out that I am a poor judge of anatomy.

Oh dear. In that case, your opinion is of no significance.

This should come as no surprise to regular readers.

 

[matt49]

That's true, but even unreliable results, given enough people tend to "lead to a conclusion". And if they are VERY positive they can lead to a high statisctical probability

If poor test design skews a result, then the result must be excluded, regardless how many people have taken the test or how many times it's been done. If poor test design tends towards a null result, every such test will produce a null result. All the null results will look impressively consistent, but they'll be meaningless.

I agree that such testing can affect the test by the very fact we are being tested, but again, back to the sample size I think it can be ignored. In the end you're ears are still your ears, and any differences that can't be made out when you're not looking are either not there or are too small to worrry about...surely?

That's not what I had in mind. (Sorry, I was trying to be brief so I didn't fill in the detail.) First thing: ears. If people have poor HF hearing (as many men over 40 do) they have to be excluded from the test, otherwise they'll bias the test towards a null result. Pre-test screening is essential. Second: ears again. It's not the case that differences in hearing are only down to our ears. Hearing happens through the ears in the brain. (This is another argument for screening, BTW: you should exclude everyone who presumes that there are no differences to be heard, because consciously or not they're likely to skew the result towards null.) But my main point concerns our ability to take tests successfully. If there's a difference to be heard between two amps, say, then it seems obvious to me that in a blind test you're more likely to hear the difference if you're very familiar with the sound of the two amps and are used to hearing them under blind conditions. This effect is very well attested in other fields, for instance, in blind tasting of wine. (And before the inevitable howls of protest, yes, it is possible to learn to be a better blind taster of wine: I've seen these people in action many times producing freakishly accurate results.)

 

[John Duncan]

matt49, you seem well versed in this sort of stuff - I seem to remember that you have some sort of grounding to back it up; remind me?

 

[CnoEvil]

matt49, you seem well versed in this sort of stuff - I seem to remember that you have some sort of grounding to back it up; remind me?

Whatever you think, or think you know, or even know you think you know......sooner or later (probably sooner), somebody who knows they know, wiil tell you that you mightn't have bothered, as you don't in fact know what you think you know, and even worse, know what you know. :?

Now, I think I need a little lie down, unless of course, I'm wrong. :wall:

 

[busb]

Are the die hard subjectivists willing to test anything blind?

I doubt it but most are not diehard objectivists either.

Personally I found it very eye opening to test lossless files against low res MP3. The differences are so incedibly minor that it makes you realise that technology had this whole area sussed a while ago. If it takes serious concentration to tell 320kb apart from 128kb (which it does) then you can start to question how much money you really need to be spending.

That'a why I'm happy to use iTunes. I have used some lossless when ripping but most is done at 320k VBR. All the HD material I've heard has been well-recorded. What we all need is not so much high bit rates but competent recording in general.

 

[Overdose]

It's easy to over analyse this blind testing stuff.

Simply removing the equipment from sight to behind a curtain or using a sheet to cover it up should suffice and the 'test' environment might as well be a living room with a few mates or just yourself and an understanding helper. It need not be any more complicated than this to show at a very basic level that various bits of hifi equipment are far more similar than some would have one believe.

Take the links I highlighted as an example, they quite effectively demonstrate various aural illusions and to listen to them or take part on your Pc in your front room, would not be considered very scientific by any definition, but they do effectively highlight the falibility of our senses in very ordinary conditions.

 

[manicm]

I think the problem here may be the fact that we "know" that an MP3 (of whatever bitrate) IS in fact not as good as a lossless file. The same with CD v High res.

So we know for a fact that the higher res, higher bitrate file is better.

What we seem not to be able to accept is that for the most part, our hearing is simply not good enough to distinguish them.

It's pride imo.

Nevermind

 

[manicm]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

 

[the record spot]

All the HD material I've heard has been well-recorded. What we all need is not so much high bit rates but competent recording in general.

Oh brother, you got that right. Most salient point of the whole thread.

 

[matt49]

matt49, you seem well versed in this sort of stuff - I seem to remember that you have some sort of grounding to back it up; remind me?

For professional (academic) reasons, I do a lot of reading of psychological and psychiatric research. And then I write and talk about it. Some people read my stuff; some students have to listen to it. Poor sods.

 

[matt49]

It's easy to over analyse this blind testing stuff.

Simply removing the equipment from sight to behind a curtain or using a sheet to cover it up should suffice and the 'test' environment might as well be a living room with a few mates or just yourself and an understanding helper. It need not be any more complicated than this to show at a very basic level that various bits of hifi equipment are far more similar than some would have one believe.

Take the links I highlighted as an example, they quite effectively demonstrate various aural illusions and to listen to them or take part on your Pc in your front room, would not be considered very scientific by any definition, but they do effectively highlight the falibility of our senses in very ordinary conditions.

I agree. So long as you're not expecting any degree of certainty from these tests, they can be very interesting.

 

[Overdose]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

Is that statement a joke?

 

[the record spot]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

I guess it depends on whether you are comparing an amp (for example) at £3000 and one at £500. If they sound the same, why would you buy the more expensive one.

On the other hand, I tend to reckon you have different buying considerations as a consumer who's potentially looking to buy into more expensive gear - not just audio - but build, marque, desing style as well as purely for audio reasons. In the same way that somone might buy a more expensive watch.

 

[manicm]

No-one's disputing the merits of ABX/blind testing, but if one correctly chooses options in a sighted test is that just a fluke? We cannot trust ears except in a blind test?

And now, since some believe in absolutes answer a simple Yes/No will suffice, no grey areas please.

Is that statement a joke?

What I meant was, if one correctly and consistently hears differences in a sighted test is that just a fluke? By correct I mean one would come to the same conclusions in a blind test.

So you believe that's just not possible? Again, a simple Yes/No.

 

[Overdose]

It's easy to over analyse this blind testing stuff.

Simply removing the equipment from sight to behind a curtain or using a sheet to cover it up should suffice and the 'test' environment might as well be a living room with a few mates or just yourself and an understanding helper. It need not be any more complicated than this to show at a very basic level that various bits of hifi equipment are far more similar than some would have one believe.

Take the links I highlighted as an example, they quite effectively demonstrate various aural illusions and to listen to them or take part on your Pc in your front room, would not be considered very scientific by any definition, but they do effectively highlight the falibility of our senses in very ordinary conditions.

I agree. So long as you're not expecting any degree of certainty from these tests, they can be very interesting.

The certainty is what discrepancy an individual finds between sighted and unsighted listening. Whilst this is not important to some, others may like to know that they can justify their expenditure based on sound quality alone.

 

[manicm]

So sighted tests are absolutely wrong? If I knowingly listen to a Rotel amp and a Cyrus amp of similar power and I have a clear preference would I be so wrong?

 

[matt49]

The certainty is what discrepancy an individual finds between sighted and unsighted listening. Whilst this is not important to some, others may like to know that they can justify their expenditure based on sound quality alone.

There can be no more certainty from a comparison of sighted and unsighted listening than there can be from unsighted listening alone -- unless I've misunderstood your point.