An interesting experiment

[steve_1979]

As PH has pointed out, they're not my readers. I also think you misjudge what people want from the magazine. Badly.

Fair enough. Just a suggestion.

 

[professorhat]

(c) When the idea of tricking WHF readers with listening tests was last suggested (possibly by yourself given the number of times you have done this now)

This is the first time that I have ever suggested that WHF could run a test to see what difference expectation bias can have on how we hear things.

True, but it's a run of many threads that have the same basic agenda i.e. attempting to show people hearing / seeing things that are all in their mind and that WHF is basically misguided in publishing reviews they publish. A quick search on your name shows this very quickly.

But anyway, if it makes you happy to do this, who am I to argue, but as Lee H has pointed out, there is a certain futility in all these threads.

 

[BenLaw]

I've also done a study lately and prepared a graph with my findings.

Your y axis scale is much too short tho

 

[Alec]

(c) When the idea of tricking WHF readers with listening tests was last suggested (possibly by yourself given the number of times you have done this now)

This is the first time that I have ever suggested that WHF could run a test to see what difference expectation bias can have on how we hear things.

True, but it's a run of many threads that have the same basic agenda i.e. attempting to show people hearing / seeing things that are all in their mind and that WHF is basically misguided in publishing reviews they publish. A quick search on your name shows this very quickly.

But anyway, if it makes you happy to do this, who am I to argue, but as Lee H has pointed out, there is a certain futility in all these threads.

But the point of the threads is correct, repetitive, boring and useless though they are. It's sad indeed that even you are now so frustrated by them you feel you have to become a mag lacky. You could just do what I always fail to do and ignore them.

"But I believe I'm a walking contradiction...".

 

[Alec]

I've also done a study lately and prepared a graph with my findings.

HA! And your sig is mildly amusing too.

EDITED for spelling, seeing as one feels one has to point that out every time one edits now, lest the establishment think I'm amending my argument in any way.

 

[professorhat]

Pointing out common business sense is hardly being a "mag lacky". But you're right, best bet would be not to get involved. I think I'm just bored on this conference call at the moment and this was slightly more entertaining...

 

[tino]

Try this thought experiment ...

1. You go to a blind auction to buy one of two identical looking pictures, one is an original Van Gogh Sunflower painting, the other is a very close copy painted by Rolf Harris.

2. You pay £25million for the Rolf Harris copy because you can't tell the difference and you were led to believe it's probably a Van Gogh

3. Later you are told you paid £25million for a copy of an original

How do you feel?

a. Content because you can't tell the difference

b. Very content because you have an original Rolf Harris painting

c. A bit miffed, because you would have bought the Van Gogh if you had known even though they very similar

d. Absolutely gutted because you could have had the original for the same price

e. Don't know

 

[tino]

[UNPUBLISHED]

Dupe post :doh:

 

[BenLaw]

e

 

[Alec]

f

 

[BenLaw]

g

 

[John Duncan]

 

[Alec]

http://www.youtube.com/watch?v=0P4A1K4lXDo

EDIT - I enjoyed that.

 

[tino]

 

[Clare Newsome]

"And when did you last conduct an ABX listening test with your father?"

 

[CnoEvil]

"And when did you last conduct an ABX listening test with your father?"

....my lot don't listen to their father, let alone ABX with him! :wall:

 

[busb]

Why?

An experiment showing how expection bias can effect how we hear things would make an interesting and informative article for your readers?

<snipped>

Prof, (c) is a very good point and it not what any magazine would want to get involved in, why would they? Maybe if they wanted to shoot themselfs in the foot.

Personally i think it should be done within the forum, as i put earlier in the thread. I do think abx testing has flaws, which to be credible shouldn't have. If it was 100% acurate all the time it may be taken more seriously. This is why (a) i dont really believe in it and (b) i dont really believe in it.

It doesn't matter what magazine is approached - absolutely none would be stupid enough to risk the whole HiFi bubble bursting if there was any risk that such ideas as all well-made interconnects, similarly spec'd products etc sounded the same. Although a number of people do believe this, I'd bet that ABX testing would "prove" it. I share your scepticism that ABX testing is fairly pointless but I cannot come up with any convincing reasons why. It seems perfectly logical that the shorter the period of time we compare segments of tests the better but is it? Is long term memory where our brains are intuitively continuing to remember these niceties are actually more reliable? It would be rather interesting (to me, at least) if some differences are proved to exist but are so small that spending huge amounts of money obtaining then actually appreciating them becomes pointless to all but the excessively rich.

Maybe not quite in the same league but heavier than air flight was considered to be imposible to the point that only nutty people would persist in trying such folly. So I doff my cap to anyone who persists in asking those more difficult questions which may not be possible to answer but still need to be asked again & again. Does anyone know if minds haven't been changed by such repetition or are just assuming so? Is it not easy enough to ignore such threads? It just seems that the same people have to throw in their less than complimentary remarks ad naseum every time such questions are asked.

 

[steve_1979]

320 v 128... can you tell? (10 times in a row)

Ok, I've just had a try at this one and it's much harder than expected. I can tell the difference some of the time but not at others. It seems a bit easier to spot the difference with complex music than it is with simple music but even when I can tell the difference 128kbps still sounds surprisingly close to 320kbps.

I'm only guessing here but could it be that a simple waveform containing only one frequency can be accurately recreated using a low resolution low bit rate MP3 but a modulated waveform containing several different frequencies needs a higher resolution MP3 to be accurately recreated?

Maybe somebody with a better understanding of the technology could expand on this?

I've been told elsewhere that simple music is sometimes harder to compress than complex music. When two sounds of a similar frequency are played at the same time the loudest will mask the quieter one and the quieter one can be thrown away with no noticable difference. But if there's only one sound being played there's nothing to throw away.

Apparently 'Tom's Diner' by Suzanne Vega was used in the development of MP3 because it's difficult not to lose any sound quality when this particular track is compressed. I've had a go at comparing it at 128 and 320kbps and I couldn't hear any difference between them.

In fact I can't hear any difference between most music at 128kbps and 320kbps and even when I can the difference is so small it's very difficult to tell.

 

[MajorFubar]

It seems perfectly logical that the shorter the period of time we compare segments of tests the better but is it?

Not for me. I have in the past returned and exchanged HiFi which I thought was brilliant to start with but then it turned out I just couldn't live with some aspect of the sound that wasn't apparent when I bought the product, or if it was apparent, it wasn't initially annoying. Same goes with MP3/FLAC/etc ABX tests. Also such ABX tests of music I'm not compltely familiar with are pointless for me, like going to a dealer's to audition equipment using music I don't know. If I'm not truly familiar with the music, I don't intially know how to listen to listen to it judge what differences there are, unless it's really chalk and cheese.

 

[shooter]

Busb, as steve says above, some music compresses better than others, not sure why but lossy compression differs to lossless. So to be fair we would have to pick a wide selection of music and genre but doing so risk listener fatigue as we go through it, its easy to switch off, concentration is tiring over long periods and bordom is another factor.

And as MajorF above, he found over time familiarizing himself bought the greater reward, i agree to an extent but from a personnel angle, switching between i can pick up differences, its over time that i can tryly evaluate what is heard. Initially we may not differences, but on longer listens becomes aparent there are so how would you you set this up? Is there a middle ground where we all can come to a conclusion?

How about our hearing, everyone is different, just do an online hearing test and this becomes apparent. How do we get over this if in this experiment 3 in a the group of 10 have greater listening abaility, how do you evaluate that or if its not possible to find 10 people with identicle listening?

Equipment and format has to be leveled and matched religiously before a serious attempt is made to asses real quality differences. How do we level MP3's that have no specific bit rate to FLAC that does? And what equipment do we use, 20k set up or 1.5k? Your more likely to have greater transparency in a 20k system but as the majority of people dont have these system then a 1.5k may be fairer?

There are lots of lab tests out there to show MP3 inferior to lossless and from my 1 download experiance could hare differences and i will do some more 320 rips and play them back and compare to lossless, though a few more bods would make a better day.

 

[busb]

Oh I agree with the three of you. As for compression, photography is the same: the more complex a scene, the greater the file size which is why a sequence of jpg files will not be the same size. As I'm unconvinced that I will ever hear the difference between Apple lossless & 320k VBR, it's the latter format I rip at. Although disc space is hardly expensive, it goes against the grain filling it needlessly & would prefer to keep one version per song for all playback devices.

As for ABX testing, as has been mentioned: it needs to be done properly by such a body as either an audio codec company or university where participants are given medium quality copies of the some or most of the files to be used well in advance & given plenty of breaks during the actual sessions. However, many will wonder what would be proved & the tests would need to not just try to prove a positive, otherwise the results will be skewed. Anyway, ABX testing has been done to death on this forum!

 

[Fred_Barker]

What makes this experiment interesting is that it highlights how much of a difference expectation bias can make.

Take for example the first test where I played same music file twice. They all thought that it sounded clearer when I told them they're listening to a file that has better sound quality. They heard what they expected to hear even though there wasn't really any difference.

This experiment isn't just about MP3 vs FLAC. It's also about how expectation bias will cloud peoples judgment... Think about it.

:?

A funny old thing 'auditioning' hifi - once upon a long time ago, I compared a group of various speakers and amps, all using the same source, switched instantly from one to another, through all of the combinations. They all sounded really good and quite different, but I couldn't decide which one I liked the sound of the best...so I departed the studio...Sometime several years later, 99.999% oxygen free copper connectors and speaker cables, were brought into an already confusing mix of choices. Quite a large number of my CD's, old and new, sound pretty grim on systems of all prices ranges, but I still just like the music.

 

[chebby]

.

 

[Alantiggger]

Why?

An experiment showing how expection bias can effect how we hear things would make an interesting and informative article for your readers?

<snipped>

Prof, (c) is a very good point and it not what any magazine would want to get involved in, why would they? Maybe if they wanted to shoot themselfs in the foot.

Personally i think it should be done within the forum, as i put earlier in the thread. I do think abx testing has flaws, which to be credible shouldn't have. If it was 100% acurate all the time it may be taken more seriously. This is why (a) i dont really believe in it and (b) i dont really believe in it.

It doesn't matter what magazine is approached - absolutely none would be stupid enough to risk the whole HiFi bubble bursting if there was any risk that such ideas as all well-made interconnects, similarly spec'd products etc sounded the same. Although a number of people do believe this, I'd bet that ABX testing would "prove" it. I share your scepticism that ABX testing is fairly pointless but I cannot come up with any convincing reasons why. It seems perfectly logical that the shorter the period of time we compare segments of tests the better but is it? Is long term memory where our brains are intuitively continuing to remember these niceties are actually more reliable? It would be rather interesting (to me, at least) if some differences are proved to exist but are so small that spending huge amounts of money obtaining then actually appreciating them becomes pointless to all but the excessively rich.

Maybe not quite in the same league but heavier than air flight was considered to be imposible to the point that only nutty people would persist in trying such folly. So I doff my cap to anyone who persists in asking those more difficult questions which may not be possible to answer but still need to be asked again & again. Does anyone know if minds haven't been changed by such repetition or are just assuming so? Is it not easy enough to ignore such threads? It just seems that the same people have to throw in their less than complimentary remarks ad naseum every time such questions are asked.

Indeed.

 

[datay]

What makes this experiment interesting is that it highlights how much of a difference expectation bias can make.

Take for example the first test where I played same music file twice. They all thought that it sounded clearer when I told them they're listening to a file that has better sound quality. They heard what they expected to hear even though there wasn't really any difference.

This experiment isn't just about MP3 vs FLAC. It's also about how expectation bias will cloud peoples judgment... Think about it.

:?

Quite a large number of my CD's, old and new, sound pretty grim on systems of all prices ranges, but I still just like the music.

Well done. The sound of your gritted teeth as you listen must be distracting, try biting down on an isolation pad.