An interesting experiment

[steve_1979]

Lossless FLAC vs 320kbps MP3 experiment

 

[steve_1979]

Here are details of a 'lossless vs MP3' experiment that I have carried out on three different people. Each time results are the same. The experiment consists of three separate blind A/B listening tests and an ABX test.

1. For the first test I played a series of two identical MP3 files but lied and said that one of them was lossless. Most of the time they prefered the sound of the fake lossless file over the MP3 file even though it was really the same identical MP3 file.

2. For the second test I swapped the files around. I told them that the MP3 was lossless and that the lossless was an MP3. Most of the time they prefered the sound of the MP3 file over the lossless file.

3. For the third test I played the lossless and MP3 files again and told the truth about which one was which. Most of the time they prefered the sound of the lossless file over the MP3 file.

4. For the fourth test I let them do an ABX test using Foobar and nobody managed to tell the difference between the MP3 and lossless files.

:?

 

[chebby]

5. I undid the padlocks and released them.

 

[Clare Newsome]

Did you use your favourite £15 speaker ;-)

 

[tino]

Could they tell you were using active speakers? :shifty:

 

[steve_1979]

5. I undid the padlocks and released them.

No not yet. I need to keep them locked up for a bit longer so that I can run some speaker cable experiments on them... :twisted:

 

[steve_1979]

Did you use your favourite £15 speaker ;-)

Hehe.

No the AVI Neutron's.

 

[John Duncan]

Now do it again with 128k.

 

[John Duncan]

And then do it on actives, since presumably all that passive crossover distortion will prevent you hearing any difference anyway.

 

[Overdose]

Have you considered trying to find out at what point you could reliably tell the difference?

 

[John Duncan]

Oh, and one last thing - did any of them go on a second date?

 

[John Duncan]

Have you considered trying to find out at what point you could reliably tell the difference?

Me or him? If me, it starts to show itself about 256 and below. 128k is obvious to me on most stuff.

 

[steve_1979]

Have you considered trying to find out at what point you could reliably tell the difference?

Good idea. I might give that a try sometime.

 

[steve_1979]

Oh, and one last thing - did any of them go on a second date?

 

[Overdose]

Have you considered trying to find out at what point you could reliably tell the difference?

Me or him? If me, it starts to show itself about 256 and below. 128k is obvious to me on most stuff.

Him

I have my iTunes library at 256 VBR, but have never critically compared bitrates, although this rate is supposed to be indistinguishable from lossless and provides a slightly smaller file than 320Kbps.

 

[John Duncan]

I've tried, though it's quite difficult to do in a consistent way. On the whole though, I'm perfectly happy listening to Spotify at (mostly) 320k, but am also happy to rip CDs to lossless since I'm not worried about file sizes.

 

[steve_1979]

Have you considered trying to find out at what point you could reliably tell the difference?

Me or him? If me, it starts to show itself about 256 and below. 128k is obvious to me on most stuff.

It's exactly the same for me too.

There is sometimes subtle difference at 256kbps. At 128kbps the difference is usually pretty obvious but music still sounds pleasant enough to listen to.

 

[Overdose]

I've tried, though it's quite difficult to do in a consistent way. On the whole though, I'm perfectly happy listening to Spotify at (mostly) 320k, but am also happy to rip CDs to lossless since I'm not worried about file sizes.

I think nowadays, space is generally not an issue for music storage, unless were talking mobile devices. My iPod 160, is nearly full of 256 files, but I don't want to lose any fidelity by compressing further, as the iPod makes a good source as is.

Apple, are you listening? Bigger HDDs in the iPod please. I'll take an Archos next, when I run out of room (12GB to go) and I don't think that it'll be long.

Edit: A 250GB SSD iPod would be nice.

 

[6th.replicant]

Curiously, or not, ALAC vs 320 AACs is most obvious in the car, which doesn't have a 'fancy' system: Alpine headunit + iPod Classic + 8 OEM speakers (Blaupunkt), which, incidently, have very thin (2mm?) speaker cable :?

 

[steve_1979]

What makes this experiment interesting is that it highlights how much of a difference expectation bias can make.

Take for example the first test where I played same music file twice. They all thought that it sounded clearer when I told them they're listening to a file that has better sound quality. They heard what they expected to hear even though there wasn't really any difference.

This experiment isn't just about MP3 vs FLAC. It's also about how expectation bias will cloud peoples judgment... Think about it.

:?

 

[John Duncan]

Maybe they were too scared to say anything. Was the music Huey Lewis?

 

[bigblue235]

What makes this experiment interesting is that it highlights how much of a difference expectation bias can make.

Take for example the first test where I played same music file twice. They all thought that it sounded clearer when I told them they're listening to a file that has better sound quality. They heard what they expected to hear even though there wasn't really any difference.

This experiment isn't just about MP3 vs FLAC. It's also about how expectation bias will cloud peoples judgment... Think about it.

Or, alternatively, they just said that the supposedly better quality one sounded better as you'd told them it was better, and they didn't want to look like a div.

Anyway, tune in next week, when a random AVI forum member will try to prove a point that didn't come from the gospel according to Ashley. They're going to prove by just how many units the ADM 9 outsells 'redundant' hi-fi separates.

In all seriousness, I too rip to lossless as I have plenty of HDD space, but I keep meaning to make a separate library at 320kbps for the iPods as I can hear no difference between 320k and lossless. But I came to that conclusion just using my lugholes, which are far from infallible.

 

[steve_1979]

Or, alternatively, they just said that the supposedly better quality one sounded better as you'd told them it was better, and they didn't want to look like a div.

They were genuinely surprised when I told them that they had listened to the same file twice. They really did believe that they could hear a difference.

 

[Alec]

Maybe they were too scared to say anything. Was the music Huey Lewis?

Fair dos, that was funny.

 

[bigblue235]

Or, alternatively, they just said that the supposedly better quality one sounded better as you'd told them it was better, and they didn't want to look like a div.

They were genuinely surprised when I told them that they had listened to the same file twice. They really did believe that they could hear a difference.

Maybe, but my missus nods appreciatively when I drag her along to demos. Anyone would think she was genuinely impressed. But she just likes to back up my assertions when I'm wasting money, she's just being sweet.

She's quite good at that 'pretending she's impressed' thing. Thankfully.