Which is best - A DBT to test subjectivity.

[cheeseboy]

can't remember where I came across this, but I found it quite interesting. Apologies if it's been posted before (probably has)

Here's a dbt to test subjectivity. So it's not a test to see which is which and if they can detect a change between a certain cable, it's a simple which do you think subjectively sounds better. Kind of both sides of the coin this one I feel that can satisfy both camps. Well, probably not, but at the very least, a bit of an eye opener.

The spelling isn't great, but the guy is spanish. Still, be interested to know what people think of this one as it's a definite trust your ears test

http://matrixhifi.com/ENG_contenedor_ppec.htm

 

[chebby]

That test has been posted here in the forum a few times in the past.

As recently as this month ....

http://www.whathifi.com/forum/hi-fi/amp-to-run-atc-11-speakers

... and at least as long ago as 2011...

http://www.whathifi.com/comment/518129#comment-518129

 

[Thompsonuxb]

It's still an interesting test and proves one man's distortion free hi-fi sound is another man's boring low-fi and visa versa.

Its a preference test - my nephew for example thinks my set lacks bass and top end crispness.

I think his is a distorted shouty mess.

My sister inlaw prefers the sound produced from the speakers of her LED laden bombastic Sony boom box to the excellent even sounding Sony 176s's I gave her when I upgraded.

There are no hard rules how things should be enjoyed.

Well that's my conclusion.

 

[ellisdj]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

That was tthe case for me in my blind test of much less than the changes there - I still identified the same system 3 times.

The only thing they missed off in this test by the looks of it was time for the testers to listen to each system sighted before covering it over. It would be interesting to see if they could pick out the better system once they actually heard what it sounded like

 

[rainsoothe]

is this that "unbiased" test where when the test subject got 5/6 right, they redid the test 'caus of some bad setup thing? And with mostly poorly recorded pop material?

 

[Vladimir]

That matrix audio club test has been posted in every amp/cable/source debate and hasn't changed anyones mind.

 

[Vladimir]

It's still an interesting test and proves one man's distortion free hi-fi sound is another man's boring low-fi and visa versa.

No it doesn't. It proves many percieved differences in audio are expectation bias due to marketing and groupthink.

 

[Vladimir]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

Picked as best sounding because it was the easiest to identify. Are you trying to spin that as the cheapy was picked because it was horrid but people misinterpreted it as better?

 

[davedotco]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

Picked as best sounding because it was the easiest to identify. Are you trying to spin that as the cheapy was picked because it was horrid but people misinterpreted it as better?

I have read around this test, articles and comments from people who were present. After some teething troubles with earlier attempts, the methodology for the test was as written in the original article.

The listeners were simply told that they were listening to two systems, no details of the equipment type or cost was given and they were simply asked which one they preferred, classic 'double blind, A/B test'. In this case it was not an ABX test, which is not clear from the article.

In this case the listeners simply had to pick the system that sounded best to them, the speakers are revealing, the room treated to a reasonable degree but neither the listeners as individuals or as a group could consistently hear a difference, the results being entirely random.

On a personal level, I have taken part in organised blind tests (for Hi-fi Choice and other magazines) and can testify to the difficulty in hearing differences between supposedly very different components in level matched blind tests. These tests were not rigorous enough to offer cast iron proof, but they sure taught me how easily my ears could be fooled, 'night and day' differences that I was positive existed between components 'known' to be very different virtually disappearing when tested blind.

 

[ellisdj]

I dare say we are all in agreement the expensive system should sound better enough to easily be indentified but that wasnt the case in that room on that day to some of those people

I think it does prove 2 things - 1. It shows that buying expensive boxes doesnt guarantee great sound - I think it highlights this more than anything.

2. it proves you could give a pal £500 and get them to buy you a system and cover it up for ever and listen and be happy it sounds as good as £12000 worth. Not so sure on this one

They did treat the room, I respect them for that - but they most heavily treated the wrong wall - which is shame - I dare say as a compromise as the shelves were already there.

I also wonder why they used book shelf speakers and not a full range speaker.

Bit of a left field consideration but they powered the systems off a nasty mains extension - was this a limiting / equalizing factor of the test. Something else to look at and consider

 

[cheeseboy]

That test has been posted here in the forum a few times in the past.

As recently as this month ....

http://www.whathifi.com/forum/hi-fi/amp-to-run-atc-11-speakers

... and at least as long ago as 2011...

http://www.whathifi.com/comment/518129#comment-518129

thought it might have been. Cheers for the heads up.

 

[cheeseboy]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

ellis, sometimes I wonder if you do actually read things that are posted? It wasn't about picking out a system, it was about which system people preferred/enjoyed listening to more.

 

[davedotco]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

ellis, sometimes I wonder if you do actually read things that are posted? It wasn't about picking out a system, it was about which system people preferred/enjoyed listening to more.

Best not to argue CB.

Some people have infalible hearing that is not even remotely troubled by the psycoacoustic issues that affect the rest of humanity. In fact it is so good that it can tell you what is wrong with a system or a test methodology without actually being present!

In my experience such certainty in these matters comes only from the truly uninformed. Why bother learning about test methodology, the difference betweem A/B and A/B/X testing or even getting involved in properly conducted blind testing when you 'know' you are right?

 

[ellisdj]

It doesnt suprise me the cheap system was picked out by quite a lot of the listeners - It probably was the easiest to identify.

ellis, sometimes I wonder if you do actually read things that are posted? It wasn't about picking out a system, it was about which system people preferred/enjoyed listening to more.

It was about picking out a system - which one preferred.

I did read it as them having to pick out the better system and maybe wrote slightly wrong what I meant as a result - but still its the same thing the cheaper system probably stood out more is what I really meant - the testers were drawn to that sound as a result thus thinking its the better of the 2. This is what happened to me in a similar scenario and I chose the "worst" option as my preference also

 

[RobinKidderminster]

The matrix site is interesting. If the tests are to be fully believed, and no one has disputed its honesty, then how rediculous are the comments of those wishing to pull it apart.

For me it shows how strong expectation bias & placaebo effects our judgement. Those ignoring it do so at the mercy of their wallets and attract the ridicule of their peers.

 

[cheeseboy]

the testers were drawn to that sound as a result thus thinking its the better of the 2.

no, they chose the one they preferred, not which one they thought was most expensive. maybe you should read it again as it doesn't seem to have sunk in.

 

[RobinKidderminster]

How hard the critics try!

 

[ellisdj]

Cheeseboy it has sunk in your missing what I am saying - they choose which one they prefer there is nothing unclear about that -I am saying I can see why a high % of the tester could easily be drawn to the worst sounding system as it likely stood out more. More added to the sound - gives a false dynamic to bass especially which is therefore easier to be attracted to and easy to identify when going back and fourth between the 2 - and easy to assume its the better of the 2.

In actual fact The better system you would expect a cleaner presentation from, quite possibly leaner as a result will stand out less of the two as it will have less added to the sound.

Maybe you have never experienced this and dont know what I mean by this

 

[RobinKidderminster]

Cheeseboy it has sunk in your missing what I am saying - they choose which one they prefer there is nothing unclear about that -I am saying I can see why a high % of the tester could easily be drawn to the worst sounding system as it likely stood out more. More added to the sound - gives a false dynamic to bass especially which is therefore easier to be attracted to and easy to identify when going back and fourth between the 2 - and easy to assume its the better of the 2.

In actual fact The better system you would expect a cleaner presentation from, quite possibly leaner as a result will stand out less of the two as it will have less added to the sound.

Maybe you have never experienced this and dont know what I mean by this

Firstly 'a high percentage' - I know your maths is shandy but it was roughly 33/33/33. In other words - completely random.

Secondly you are suggesting a coloured big bassy sound fooled the listeners to make this preference. I don't see pubescent teens but reasonably mature music lovers.

Some of your arguments are so very hard to take seriously

 

[ellisdj]

How hard the critics try!

Robin talking to himself again.

Reason 1 why they got that room wrong

Reason 2

Reason 3

I still accept the results but for purposes of factuality it looks like they got that room quite wrong to me based on the above reasons alone. There was no data to show otherwise - which is a shame

 

[ellisdj]

Cheeseboy it has sunk in your missing what I am saying - they choose which one they prefer there is nothing unclear about that -I am saying I can see why a high % of the tester could easily be drawn to the worst sounding system as it likely stood out more. More added to the sound - gives a false dynamic to bass especially which is therefore easier to be attracted to and easy to identify when going back and fourth between the 2 - and easy to assume its the better of the 2.

In actual fact The better system you would expect a cleaner presentation from, quite possibly leaner as a result will stand out less of the two as it will have less added to the sound.

Maybe you have never experienced this and dont know what I mean by this

Firstly 'a high percentage' - I know your maths is shandy but it was roughly 33/33/33. In other words - completely random.

Secondly you are suggesting a coloured big bassy sound fooled the listeners to make this preference. I don't see pubescent teens but reasonably mature music lovers.

Some of your arguments are so very hard to take seriously

You have clearly never heard it either - hence why you say this - there is nothing not to believe - the speakers tested in that spot in the room would have had very limited bass response - so anything adding "phat" to the sound could easily be considered as better.

 

[RobinKidderminster]

Missing the point yet again. The room treatment has little to do with a comparitive test. As you have previously mentioned - its a control environment. So frustrating! There may well be critism of these tests but I'm sorry you don't get it.

 

[RobinKidderminster]

@ellisdj

It would seem that some folk don't always see eye to eye with your views.

No one woulld dispute that you have built an excellent HC using good equipment and serious room treatment. No one would dispute the importance of room treatment and I for one believe this to be (probably) the most important aspect of good cinema/hiring sound. However, (I guess) 90% of enthusiasts and probably more here are unable to invest in this kind of build and we struggle to make improvements in a dual-purpose lounge/listening room. I would therefore advocate that in our 'real world' we are interested to understand how to improve our systems without extensive room treatment, exotic power supply systems or cabling.

It seems that many of your comments throw red herrings. The test in this thread is not a scientific experiment in controlled conditions where measurements are essential (those very measurements that you have claimed are impossible to make) but it is a simple blind test comparison which shows that a group of enthusiasts can not agree which of two systems sounds best. It attempts to do this in a fairly typical room.

Please stop confusing statements which do nothing to uphold your beliefs but simply invite critism of your irrelevant monologue.

I realise this comment will wash over your head but 'its good to talk'.

 

[ellisdj]

Robin there is no confusing comments intended, and its nothing to do with my own system - its to do with what I have learned in doing it. You should read all those articles on that site - and watch the videos - lots to learn

The rooms effect completely alters what the speakers should be doing, so far away from the design its unreal - therefore i also ruins what the electronics and cabling etc are doing before it - how can it not be?

So really how are you supposed to be able to tell accurate differences in such an environment - its made a lot more difficult obviously.

What I think is very apt for the average hifi hobbist is that this test does quite clearly show that just buying expensive boxes is not the Major Impactor to improved sound quality. This is the biggest thing to come out of the test if you ask me.

 

[RobinKidderminster]

Fair comment.

Can you give a percentage for the rooms treatment? 1-2% or better?