An interesting experiment

[char_lotte]

Maybe they were too scared to say anything. Was the music Huey Lewis?

Fair dos, that was funny.

Agreed.

 

[fr0g]

Have you considered trying to find out at what point you could reliably tell the difference?

Me or him? If me, it starts to show itself about 256 and below. 128k is obvious to me on most stuff.

It's exactly the same for me too.

There is sometimes subtle difference at 256kbps. At 128kbps the difference is usually pretty obvious but music still sounds pleasant enough to listen to.

Rip to 190 VBR LAME. Then try the Foobar experiment. I bet you cannot pass the ABX test yourself.

At 320 MP3, AAC, OGG I don't believe anyone can tell the difference for real, but I stand to be corrected with a 20/20 Foobar test

 

[RobinKidderminster]

Am I missing something or does it matter about the source and the software conversion. I am not disputing nor supporting these results but I wonder if such tests have more variables than just speakers. It is an interesting experiment which deserves consideration.

Cheers

 

[Frank Harvey]

But she just likes to back up my assertions when I'm wasting money, she's just being sweet.

She's quite good at that 'pretending she's impressed' thing. Thankfully.

:rofl:

 

[John Duncan]

Maybe they were too scared to say anything. Was the music Huey Lewis?

Fair dos, that was funny.

Enough with the creepy flirting already.

 

[steve_1979]

It's exactly the same for me too.

There is sometimes subtle difference at 256kbps. At 128kbps the difference is usually pretty obvious but music still sounds pleasant enough to listen to.

Rip to 190 VBR LAME. Then try the Foobar experiment. I bet you cannot pass the ABX test yourself.

At 320 MP3, AAC, OGG I don't believe anyone can tell the difference for real, but I stand to be corrected with a 20/20 Foobar test

I haven't done much experimenting with any bitrates other than 320kbps. I haven't actually tried doing any proper ABX tests using 256 or 190kbps MP3's so it's possible that the occasional subtle difference that I think I can hear at 256kbps is really just expectation bias clouding my judgement.

I'll try a few ABX tests in Foobar using LAME ripped VBR 190 and 256kbps MP3's. I'll report back later with the results.

 

[proffski]

Here are details of a 'lossless vs MP3' experiment that I have carried out on three different people. Each time results are the same. The experiment consists of three separate blind A/B listening tests and an ABX test.

1. For the first test I played a series of two identical MP3 files but lied and said that one of them was lossless. Most of the time they prefered the sound of the fake lossless file over the MP3 file even though it was really the same identical MP3 file.

2. For the second test I swapped the files around. I told them that the MP3 was lossless and that the lossless was an MP3. Most of the time they prefered the sound of the MP3 file over the lossless file.

3. For the third test I played the lossless and MP3 files again and told the truth about which one was which. Most of the time they prefered the sound of the lossless file over the MP3 file.

4. For the fourth test I let them do an ABX test using Foobar and nobody managed to tell the difference between the MP3 and lossless files.

:?

Importantly, what kind of 'music' did you use as your reference?

How and where was it recorded?

 

[matthewpiano]

Interesting test and a good question there from proffski. I've found that on music with simpler textures, high bit-rate MP3 files and even 256k iTunes+ files can sound as good as lossless (to any extent that the human ear can actually tell). It is in music with more textural complexity where I find even 320k MP3 files can start to show their weakness, in my experience.

I agree that the expectation bias aspect of the test is intruiging though I'm sure there will be plenty of psychology investigations out there that could throw considerable light on this area.

 

[steve_1979]

Importantly, what kind of 'music' did you use as your reference?

How and where was it recorded?

They were all recorded from CD using LAME.

I used songs from Michael Jackson, Alan Parsons, Supertramp and some Mozart tracks from the Amadeus movie. I consider all of these to have very good sound quality with top notch recording and mastering (though my taste in music is questionable).

For variation I used some music which was very simple and some music which was very complex.

 

[steve_1979]

Am I missing something or does it matter about the source and the software conversion. I am not disputing nor supporting these results but I wonder if such tests have more variables than just speakers.

That's a good question.

Based on what I've read about this subject on the internet it is possible for the encoding software used to have a small effect on the sound of the MP3 file.

Apparently the first MP3 encoders are noticeably worst than modern MP3 encoders. Any modern MP3 encoder should now be high enough quality for it to be impossible to tell any difference between them. LAME is arguably the best MP3 encoder available though.

When it comes to MP3 playback all devices will read the file in the same way. The only exception to this is that some old MP3 players can't read read VBR MP3's and have to be used with CBR MP3's. All modern MP3 players will play both CBR and VBR MP3's though.

It is an interesting experiment which deserves consideration.

Thanks Rob.

I think that the expection bias element of this experiment is the most interesting part and deserves consideration. The MP3 vs lossless thing is also interesting but this has been done to death a million times already.

 

[paradiziac]

On my system I can hear differences between analogue and digital cables, mains cables and equipment supports, but not between a lossless rip and mp3 version at 320kbps.

Perhaps I could go through my collection and find a specific section of a track that does reveal a difference, but why? I stream the highest available bit rates and buy/rip CD/lossless only.

 

[steve_1979]

It's exactly the same for me too.

There is sometimes subtle difference at 256kbps. At 128kbps the difference is usually pretty obvious but music still sounds pleasant enough to listen to.

Ok I've done the tests and my findings mirror exactly what fr0g has said.

I cannot pass an ABX test comparing lossless to MP3's at either 256 or 192kbps. At 128kbps I can tell the difference but it is still sounds very good even at such a low bit rate.

Rip to 190 VBR LAME. Then try the Foobar experiment. I bet you cannot pass the ABX test yourself.

At 320 MP3, AAC, OGG I don't believe anyone can tell the difference for real, but I stand to be corrected with a 20/20 Foobar test

I haven't done much experimenting with any bitrates other than 320kbps. I haven't actually tried doing any proper ABX tests using 256 or 190kbps MP3's so it's possible that the occasional subtle difference that I think I can hear at 256kbps is really just expectation bias clouding my judgement.

I'll try a few ABX tests in Foobar using LAME ripped VBR 190 and 256kbps MP3's. I'll report back later with the results.

Ok I've done some tests and my findings mirror exactly what fr0g has said.

I cannot pass an ABX test comparing lossless to MP3's at either 256 or 192kbps. At 128kbps I can tell the difference but it's still sounds very good even at such a low bit rate.

 

[RobinKidderminster]

A little off topic maybe but in ripping cds to flac and mp3 plyed on a decent sony parts of tracks have had serious timing issues. ie sounds like the cd is spinning too fast. PC, cd drive, software, player issue? Have yet to sort it but certainly my rips have brought tears. Yes there is a play on words there but it is Sunday.

Cheers

 

[Overdose]

Ok I've done some tests and my findings mirror exactly what fr0g has said.

I cannot pass an ABX test comparing lossless to MP3's at either 256 or 192kbps. At 128kbps I can tell the difference but it's still sounds very good even at such a low bit rate.

If when I get around to it I find the same, I'll be saving a fair amount of space. Is it likely that any further compression, down to 190 VBR would result in any additional problems, or would the result be the same as ripping from CD straight to 190 VBR would you think?

 

[fr0g]

Ok I've done some tests and my findings mirror exactly what fr0g has said.

I cannot pass an ABX test comparing lossless to MP3's at either 256 or 192kbps. At 128kbps I can tell the difference but it's still sounds very good even at such a low bit rate.

If when I get around to it I find the same, I'll be saving a fair amount of space. Is it likely that any further compression, down to 190 VBR would result in any additional problems, or would the result be the same as ripping from CD straight to 190 VBR would you think?

Pure guess, but I imagine CD to 190 VBR will be better than 320 to 190 VBR. But I doubt you'll notice it. (Unless you are golden eared and listen extensively to 3 second clips of Kraftwerk).

 

[fr0g]

this is probably worth a re-post...

320 v 128... can you tell? (10 times in a row)

 

[basshound]

I suspect a test of 16bit files and 24bit files would have the same results.

 

[Overdose]

So far as I have tested, this is the case for me. 16bit vs 24bit.

 

[bigblue235]

Steve, regarding your expectation bias thing, isn't it just the same as new cables, new CD players, new anything else? If you want there to be a difference, there's more likely to be one. It is interesting, but I don't think it's a new concept.

 

[steve_1979]

Steve, regarding your expectation bias thing, isn't it just the same as new cables, new CD players, new anything else? If you want there to be a difference, there's more likely to be one. It is interesting, but I don't think it's a new concept.

Expectation bias isn't a new concept, it's been known about for ages. My experiment is just another demonstration that shows how easy it is to fool people into thinking that they can hear differences that aren't really there.

This is why it's important to use blind rather than sighted testing whenever you're demoing hifi equiptment. If you can see what you're listening to you may be biased without even realising it.

 

[John Duncan]

this is probably worth a re-post...

320 v 128... can you tell? (10 times in a row)

Isn't that - to coin a phrase - boiling a fr0g?

 

[bigblue235]

Expectation bias isn't a new concept, it's been known about for ages. My experiment is just another demonstration that shows how easy it is to fool people into thinking that they can hear differences that aren't really there.

This is why it's important to use blind rather than sighted testing whenever you're demoing hifi equiptment. If you can see what you're listening to you may be biased without even realising it.

So why are you just bringing up something that's well known about and suggesting we all need to think about it now? Is it just the basis for banging the blind testing drum? I think those that agree, will. Those that don't, won't.

I know I was joking earlier about the AVI forum thing, but couldn't you maybe make a thread about something which isn't pushed over there, just for a change?

 

[steve_1979]

So why are you just bringing up something that's well known about and suggesting we all need to think about it now? Is it just the basis for banging the blind testing drum?

I undertook the experiment because I wanted to verify peoples claims about whether or not it's possible to hear a difference between MP3 and FLAC. The results of my experiment were interesting so I decided to share them with the people here (after all this is a hifi forum). Just because the results suggest that expectation bias played a big part in what people heard it doesn't mean that I'm banging any drums.

I know I was joking earlier about the AVI forum thing

This thread has absolutely nothing to do with AVI. This thread is about MP3 vs FLAC comparisons and how expectation bias can effect what you hear.

Your comments don't come accross as joking either. If you're not interested in my threads why do you keep reading them and posting in them? You just make quips about AVI without ever saying anything constructive the thread topics. I've always been polite and friendly to you yet you take every opportunity to verbally attack and be rude to me.

couldn't you maybe make a thread about something which isn't pushed over there, just for a change?

Like this one? http://www.whathifi.com/forum/hi-fi/x-mini-ii-portable-speaker-review

 

[bigblue235]

I undertook the experiment because I wanted to verify peoples claims about whether or not it's possible to hear a difference between MP3 and FLAC. The results of my experiment were interesting so I decided to share them with the other members of this forum (after all this is a hifi forum). Just because these results suggest that expectation bias played a big part in what people heard it doesn't mean that I'm banging any drums.

This thread has absolutely nothing to do with AVI. This thread is about MP3 vs FLAC comparisons and how expectation bias can effect what you hear.

Your comments don't come accross as joking either. If you're not interested in my threads why do you keep reading them and posting in them? You rarely have anything constructive to say either, you just make quips about AVI. I've always been polite and friendly to you yet you take every opportunity to verbally attack and be rude to me.

I apologise if it comes across that I'm being rude, genuinely. I'm not intending to be rude. I do frequently intend to be a little sarcastic, partly to get my point across without being rude and partly for fun/banter. If you consider that to be an 'attack', well, I don't see it that way. The team here are fairly quick at moderating any comments that are genuinely rude, so I think if my posts were as offensive as you seem to find them then they'd be gone.

Look at the thread though. There were a few quips aimed at you before I even commented. Do you not think, given that several folk in this thread have made references to the same stuff as I have, that maybe there's something in it? You've made numerous references to blind testing, ABX, expectation bias (in bold, underlined, italics and sometimes in bold and underlined!) and so on. Stuff that What Hi-fi and us 'audiophools' don't tend to get into much, but stuff that's often the foundation of discussions on t'other forum. That's what I meant about banging the drum.

You and Max seem determined to sell us the ways and beliefs of the AVI world. I frequently see both of you challenged about some technical point, and the usual defence you have is "Martin Grindrod says..." or some other info which is on that forum. It's a wee bit cult-like. It's as if you're trying to convert us!

If you're going to keep on from one angle, people will have digs at you about it. I mentioned where I live a few times, and before I knew it PP was accusing me of working for the Tourist board. I derserved that, it was all in good fun, and I realised I was maybe going on about it a bit much, so I tried to be a bit more aware of it. Likewise, if I'm taking the piss about AVI stuff a bit too much, I need to be aware of that too.

Like this one? http://www.whathifi.com/forum/hi-fi/x-mini-ii-portable-speaker-review

Oh, there is one. Jolly good! (Yes, just sarcasm!) Anyway, hopefully you can see there's no ill will, please don't get your knickers in a twist about it, but if you want to discuss it I'd be happy for you to have my email address.

 

[busb]

Expectation bias isn't a new concept, it's been known about for ages. My experiment is just another demonstration that shows how easy it is to fool people into thinking that they can hear differences that aren't really there.

This is why it's important to use blind rather than sighted testing whenever you're demoing hifi equiptment. If you can see what you're listening to you may be biased without even realising it.

So why are you just bringing up something that's well known about and suggesting we all need to think about it now? Is it just the basis for banging the blind testing drum? I think those that agree, will. Those that don't, won't.

I know I was joking earlier about the AVI forum thing, but couldn't you maybe make a thread about something which isn't pushed over there, just for a change?

Wouldn't it make more sense not to participate in threads that you don't approve of or are not interested in? My view is that steve_1979 is willing to ask the big questions that some people on this forum only answer is to is to be flippant. I share his desire to get to the bottom of whether or not expensive cables make the difference some claim, if 24/192 really sounds better than redbook & at what bitrate threshhold we cannot hear differences.

Where I differ is that I don't have much faith in ABX testing telling us anything useful about SQ - I ernestly wish it did. I prefer Wadworth's Bishops Tipple to Fullers London Pride. If I was to participate in ABX tasting, I'd bet I'd fail for exactly the same reasons (short-term memory inaccuracy) not because my preference is bogus but because the test methodology is flawed IMO. Proving it is flawed is just as difficult as proving otherwise. Shotgun ABX tests spread over short time periods don't prove anything but do give sceptics plenty of fuel to burn. So what if the concept of expectation bias isn't new - it's still a valid observation & probably points to many differences are so close to call as debating how many fairies can dance on the head of a pin.