Blind Listening Tests are Flawed

[steve_1979]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

Indeed

:cheers:

 

[steve_1979]

Clicky

Randi is to cable threads what Hitler is to fourms in general.

(for the lawyers - purely in terms of, eventually, given enough time, Randi will be mentioned in a cable thread. I make no other associations between Randi and Hitler )

Godwin's Law - Eventually, given enough time, AVI will be mentioned in any WHF thread.

 

[manicm]

I think you are confusing the amount of data that can fit on a CD to its dynamic range.

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

The length of a CD has no effect on sound quality.

I did not say that absolutely, what I did say is that an extreme CD length may place additional strain on the laser and thus may affect playback. Is that so far-fetched?

 

[Anderson]

I think you are confusing the amount of data that can fit on a CD to its dynamic range.

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

The length of a CD has no effect on sound quality.

I did not say that absolutely, what I did say is that an extreme CD length may place additional strain on the laser and thus may affect playback. Is that so far-fetched?

I had a feeling that the answer to your question would be yes, wiki appears to agree with me, in that a standard CD can hold 80 minutes of music without any compression.

http://en.wikipedia.org/wiki/Compact_disc

 

[manicm]

Anderson - why do you insist on putting words in my mouth?? Where did I ever mention compression? Do you agree that a CD player is a mechanical device? Its transport consists of a motor and laser. And depending on the condition of a disc, those lasers are not infallible.

My ultimate point is that, since CD was developed in the late 70s, and launched around 1982, the 16/44 format was not chosen because it was deemed the absolute best, it was dictated by the technology limitations of the day. And that is the pure and simple fact - I'm not debating whether it's good enough or not - that's been argued to death.

 

[the record spot]

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

With respect, this is a nonsense. I have several CDs that go beyond 75 minutes, none of which results in this issue on playback. None.

No, this is not nonsense, it is my experience - both discs run over 78 minutes and my CDP was noisier than usual - are you saying I'm imagining that? It may not hold true on your player but it is on mine.

The conclusion that your player is experiencing noise thanks to a CD whose running time is longer than 78 minutes doesn't bear out across the board. Why would a CD player's laser malfunction in its reading ability? If my players have never done it, and there's been a few, surely it's less to do with the "problem" you're attaching to CD when it's more likely a flaw in your player?

 

[the record spot]

Anderson - why do you insist on putting words in my mouth?? Where did I ever mention compression? Do you agree that a CD player is a mechanical device? Its transport consists of a motor and laser. And depending on the condition of a disc, those lasers are not infallible.

My ultimate point is that, since CD was developed in the late 70s, and launched around 1982, the 16/44 format was not chosen because it was deemed the absolute best, it was dictated by the technology limitations of the day. And that is the pure and simple fact - I'm not debating whether it's good enough or not - that's been argued to death.

...and it's a "limitation" that's more than capable to handle high quality playback. One of the reasons why I ditched vinyl a couple of years ago.

 

[Overdose]

I think you are confusing the amount of data that can fit on a CD to its dynamic range.

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

The length of a CD has no effect on sound quality.

I did not say that absolutely, what I did say is that an extreme CD length may place additional strain on the laser and thus may affect playback. Is that so far-fetched?

Yes, a laser is either working or not. Given time it will eventually wear out, but it wouldn't be strained reading a more full CD.

 

[manicm]

The conclusion that your player is experiencing noise thanks to a CD whose running time is longer than 78 minutes doesn't bear out across the board. Why would a CD player's laser malfunction in its reading ability? If my players have never done it, and there's been a few, surely it's less to do with the "problem" you're attaching to CD when it's more likely a flaw in your player?

I too have had about 4 disc players. I'm not bluntly blaming the CD or my player for what I'm experiencing - and it's not that loud or malfunctioning - but I repeat it's my experience.

Also, I have Roger Waters' Amused To Death which is around 76 minutes long and it doesn't happen. My point is that a CDP is very much a mechanical device, and perhaps some are less tolerant than others, as the motor/laser combinations are sourced from different manufacturers.

Another example is my Pioneer DVD player - which was very sensitive to mildly scratched DVDs, which my subsequent Blu-ray players would play with ease.

Do you remember DualDiscs? When they were released many hifi companies warned customers about them, because many CD mechanisms from different makers were built to withstand specific tolerances, and these discs may have exceed those tolerances.

I will repeat my statement that the CD, and indeed perhaps any mechanical mediums are far from perfect.

 

[ifor]

I expect many have read this before, but it's new to me.

http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial?page=1

I'm open minded when it comes to the interconnect and speaker cable debates. I've never bothered, yet, to make comparisons, either blind or otherwise, but I do get exceedingly irritated by fundamentalist non-believers. What won't they just relax back to a position of extreme scepticism?

I do believe that double blind ABX is probably flawed, but that doesn't make me a believer. Curiosity got me googling and I ended up at the page for which I've given the link.

I'm deeply concerned that you might think that the article makes any sense.

Chris

Why be concerned? It was perfectly well written in a language I know well; it is perfectly understandable. Did I agree with it all? I would say I'm closer to no than probablly not. The bit that interested me was the bit about the Swedish BC. The bit that not a single person here, or in the comments below the article, has mentioned.

 

[the record spot]

The conclusion that your player is experiencing noise thanks to a CD whose running time is longer than 78 minutes doesn't bear out across the board. Why would a CD player's laser malfunction in its reading ability? If my players have never done it, and there's been a few, surely it's less to do with the "problem" you're attaching to CD when it's more likely a flaw in your player?

I too have had about 4 disc players. I'm not bluntly blaming the CD or my player for what I'm experiencing - and it's not that loud or malfunctioning - but I repeat it's my experience.

Also, I have Roger Waters' Amused To Death which is around 76 minutes long and it doesn't happen. My point is that a CDP is very much a mechanical device, and perhaps some are less tolerant than others, as the motor/laser combinations are sourced from different manufacturers.

Another example is my Pioneer DVD player - which was very sensitive to mildly scratched DVDs, which my subsequent Blu-ray players would play with ease.

Do you remember DualDiscs? When they were released many hifi companies warned customers about them, because many CD mechanisms from different makers were built to withstand specific tolerances, and these discs may have exceed those tolerances.

I will repeat my statement that the CD, and indeed perhaps any mechanical mediums are far from perfect.

I doubt anybody's saying they're perfect, but vinyl has more issues afflicting it - iffy vinyl, cartridge alignment, vertical tracking, increased distortion with longer LPs over 25 miutes, tape had wow and flutter, radio had reception issues, or needed a good aerial to mitigate the former, internet radio suffers from streaming issues,

Compared to those, CD probably has the least issues.

 

[davedotco]

Red book standard CDs are limited to 74 minutes by design, longer recordings can be accommodated by tightening the pitch slightly. if this is done several things can happen.

For example, a player that adheres strictly to Red Book standard will read such a disc, get a running time in excess of 74 minutes which it 'knows' to be impossible, and refuse to ply citing an error. This was not uncommon amongst early players.

Data steams that are more closely packed may be more susceptible to damage, causing excessive use of error correction, which might affect SQ.

It is hard to see why this would cause the mech to have to work harder but some mechs in some players, Naim for example, often seem to work harder than others. I have no explanation for this.

 

[Craig M.]

The bit that interested me was the bit about the Swedish BC. The bit that not a single person here, or in the comments below the article, has mentioned.

It's a shame they didn't blind test him first to see if he could spot it that way.

 

[davedotco]

Some years ago I met a chap who could not only pick compressed audio from uncompressed audio but could identify the codecs being used in each case.

He did this by training himself to listen for certain 'tells', compression effects created by the different codecs, he wasn't listening to the music at all. In recent years codec designers have reduced these effects and such a 'trick' is now much harder to pull off.

For what it is worth, in the heyday of FM radio, when a live BBC Radio Three concert was considered one of, if not the, best sound quality available in the home, few people realised that they were listening to a music signal that was restricted in bandwidth to 15khz and delivered to the transmitter using a 14 bit pcm encode/decode system.

 

[manicm]

In recent years codec designers have reduced these effects and such a 'trick' is now much harder to pull off.

Yet somehow I preferred earlier versions of the Lame MP3 encoder.

 

[davedotco]

Some years ago I met a chap who could not only pick compressed audio from uncompressed audio but could identify the codecs being used in each case.

He did this by training himself to listen for certain 'tells', compression effects created by the different codecs, he wasn't listening to the music at all.

And did he tell by blind testing?

Most definitely.

But as I said it was more or less a party trick, he said mp3 files of that period were particularly easy, a sort of 'background chatter' that could be heard at low levels, he showed me on a couple of obvious examples and although it seemed clear enough at the time I never got the knack.

I still cannot tell the difference between Ogg Vorbis (at 320 kbit/sec), downloaded from Spotify and FLAC files of the same disc, both from my hard drive. Maybe a better system....... :?

 

[hammill]

In recent years codec designers have reduced these effects and such a 'trick' is now much harder to pull off.

Yet somehow I preferred earlier versions of the Lame MP3 encoder.

That is ok. Loads of people prefer valve to transistor even though valve amps tend to be less faithful to the original recording. As long as you enjoy it who cares?

 

[davedotco]

In recent years codec designers have reduced these effects and such a 'trick' is now much harder to pull off.

Yet somehow I preferred earlier versions of the Lame MP3 encoder.

Quite possible. Different people react differently to different 'distortions', happens all across hi-fi.

I have never really streamed to a really good system in my home so can not say how good the modern codecs are, any attempt at such a comparison is usually done on headphones, never that satisfactory in my view.

 

[matt49]

I still cannot tell the difference between Ogg Vorbis (at 320 kbit/sec), downloaded from Spotify and FLAC files of the same disc, both from my hard drive. Maybe a better system....... :?

In my limited experience (an hour or two of comparison using Foobar ABX earlier this week) it is indeed very difficult. I lined up some files ripped to both ALAC and Ogg Vorbis 320kbps. I did two ABX runs of 10 passes each. On both runs I got the ABX right 100% on the first 5 passes. But after that I just fell to pieces and ended up with 7 out of 10. I don't know if it was listening fatigue or cognitive confusion or just that I'd got lucky on the first five passes. But I suspect that your friend's "party trick" could be done with Ogg Vorbis at 320 kbps, assuming you picked the right music and spent a lot of time practising.

The system was PC via USB > modded MDAC > Hifiman HE500s.

I'll give it another go when I have the time and energy.

 

[davedotco]

I still cannot tell the difference between Ogg Vorbis (at 320 kbit/sec), downloaded from Spotify and FLAC files of the same disc, both from my hard drive. Maybe a better system....... :?

In my limited experience (an hour or two of comparison using Foobar ABX earlier this week) it is indeed very difficult. I lined up some files ripped to both ALAC and Ogg Vorbis 320kbps. I did two ABX runs of 10 passes each. On both runs I got the ABX right 100% on the first 5 passes. But after that I just fell to pieces and ended up with 7 out of 10. I don't know if it was listening fatigue or cognitive confusion or just that I'd got lucky on the first five passes. But I suspect that your friend's "party trick" could be done with Ogg Vorbis at 320 kbps, assuming you picked the right music and spent a lot of time practising.

The system was PC via USB > modded MDAC > Hifiman HE500s.

I'll give it another go when I have the time and energy.

That is kind of what I would expect for an experienced enthusiast with good equipment, by the way, were you listening for changes in the music or for 'artifacts'?

For me this comes back to the question of music or hi-fi. I am happy to listen to and discuss hi-fi and I can be pretty damn critical, but for me that is the point of the excersise, the 'fun' of hi-fi as a hobby.

Listening to music is, for me, a very different thing. Firstly I am kind of wierd, I hate listening to the same recordings over and over again, I just can't stand to hear the same thing played in exactly the same way time after time, simple as that.

So lacking the funds to buy the enormous amount of 'new' (to me) music that I would ideally like, I play Spotify a lot. I know that this is not the best fidelity available but as we have discussed, it really is pretty good and it gives me the choice that i like.

Which brings me on to the so far, unasked question, how good does your system have to be before Spotify's limitations start to become objectional, not barely detectable as discussed, but actually objectional enough to effect everyday listening?

So far I have come to the tentative conclusion that on most rock and popular music the limiting factor is most likely to be the recording, not the 'transmission' system but that said, I find I am playing less classical music than usual. My current system is pretty basic, so that is most probably the issue, but I can't help wondering if the sound quality 'losses' via Spotify might be partly to blame....... :?

 

[TrevC]

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

With respect, this is a nonsense. I have several CDs that go beyond 75 minutes, none of which results in this issue on playback. None.

No, this is not nonsense, it is my experience - both discs run over 78 minutes and my CDP was noisier than usual - are you saying I'm imagining that? It may not hold true on your player but it is on mine.

It could be the CD itself, check for fingerprints on the outer part of it. The longer a CD is, the closer it plays to the edge.

 

[TrevC]

I think you are confusing the amount of data that can fit on a CD to its dynamic range.

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

The length of a CD has no effect on sound quality.

I did not say that absolutely, what I did say is that an extreme CD length may place additional strain on the laser and thus may affect playback. Is that so far-fetched?

Yes, that is nonsense.

 

[manicm]

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

With respect, this is a nonsense. I have several CDs that go beyond 75 minutes, none of which results in this issue on playback. None.

No, this is not nonsense, it is my experience - both discs run over 78 minutes and my CDP was noisier than usual - are you saying I'm imagining that? It may not hold true on your player but it is on mine.

It could be the CD itself, check for fingerprints on the outer part of it. The longer a CD is, the closer it plays to the edge.

Let me repeat, the disc plays back perfectly. But my observations were twofold:

1. My player was noisier than usual.

2. I thought a few songs could have sounded smoother.

 

[manicm]

I think you are confusing the amount of data that can fit on a CD to its dynamic range.

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

The length of a CD has no effect on sound quality.

I did not say that absolutely, what I did say is that an extreme CD length may place additional strain on the laser and thus may affect playback. Is that so far-fetched?

Yes, that is nonsense.

Why? Especially when redbook and CDPs were originally designed to play no more than 74 minutes. If you're going to dismiss my suspicions then substantiate.

Also, if you believe CD playback is always 100% reliable, then it lays waste to the claim that digital playback is inherently superior. Wasn't this Linn's claim and many others here for ditching CDs for streaming?

 

[Alec]

CD playback is digital playback