Blind Listening Tests are Flawed

[manicm]

What "additional"research is needed pray tell?

You can prouduce an identical copy of a waveform from a digital copy that has twice the sample rate as the maximum frequency in the waveform. It's not like it's a tough one.

Also, it's considered a "theorem". It has been proven from multiple sources and is essentially considered "fact".

The "only" vaguely possible need for more than 44.1 KHz is the claim that people can sense frequencies above the 22 KHz limit. A claim that has never been successfully proved, unlike Nyquist-Shannon, which has.

In the very same link that was given to me on the Nyquist theorem, that website seems to support higher sampling rates, NOT because it claims we can hear the higher frequencies, but that the need for aliasing, which simplistically speaking can introduce distortions, is mitigated. If you'd care to read further there, and as Cno also stated essentially.

 

[RobinKidderminster]

If.I cant hear a difference in a blind test then the difference ain't there, to me. Those who often critisise blind tests, often hear subtle differences in equipment using comparitive memory. They have better ears and brains than I. There seems to be a fear of these trials in 'believers' and a belief in trials for the non believers. These debates are somewhat pointles zzzzzzzz

The corollary of that is that if one can genuinely and truly hear a difference during a sighted audition, then the difference is true and no blind test is required. No-one here is detracting from the merits of blind testing as I can recall, but its supporters become violently reactionary when one supposes it's not the be-all and end-all of testing.

And I repeat these days, when many dealers are just trying to shift units it's extremely difficult to do good blind testing. And after all one doesn't walk in blindfolded.

Quite the opposite (I think?). I would argue that no 'sighted' test is of much value unless there are 'huge' differences. Expectation bias is not imo a weakness but merely a feature of our brains. Now I am confused!

 

[manicm]

Its is the 2001 one and its all around 7 so its has lots of compression applied, As said before their is a USA version thats around 13. NOt hard took about 20 seconds.

It doesn't matter, I wasn't complaining about compression or loudness, but rather that the one particular track did not sound as smooth as I expected. It had a subtle hardness to it. I've only listened to it once anyway, and need to repeat.

 

[andyjm]

Cno's quote at the bottom of his posts say it all. "Everything has been said before, but since nobody listens we have to keep going back and beginning all over again"

I have explained on this forum a number of times (as have others) about why 16/44.1 is just fine for audio recordings. The engineers at Philips back in the 70s weren't fools, and didn't hit upon the redbook standard by chance.

The links to papers on why 16/44.1 isn't great have nothing to do with the sampling rate or frequency not capturing ALL of the audible information, they revolve around the design of anti alias brick wall filter on the dac output. This can be overcome by over sampling techniques that move the aliases further up the frequency spectrum.

16/44.1 is really all you need.

 

[manicm]

Quite the opposite (I think?). I would argue that no 'sighted' test is of much value unless there are 'huge' differences. Expectation bias is not imo a weakness but merely a feature of our brains. Now I am confused!

If you are confused it's your own doing (or undoing I should say). I had my first CD player for 11 years, retaining the same amp and speakers, I bought my first DVD player and was appalled at its CD playback. Ditto for a NAD C521i CDP I briefly acquired - rubbish.

My expectations, rather naively, was that the Pioneer DVD deck would sound as good as my old Technics CDP. It didn't. It was rubbish. Ditto the NAD. And after 11 years, I didn't need any form of blind testing to tell me otherwise, so glaringly obvious was the difference. Because I had 11 years of ear training from my Technics.

So please, pretty please, don't dare tell me I have 'expectation bias' any other such malaise. Go and fob that off on someone else.

 

[manicm]

The links to papers on why 16/44.1 isn't great have nothing to do with the sampling rate or frequency not capturing ALL of the audible information, they revolve around the design of anti alias brick wall filter on the dac output. This can be overcome by over sampling techniques that move the aliases further up the frequency spectrum.

16/44.1 is really all you need.

Yes, Technics tried to 'overcome' it with their subsequent MASH technology, turned out to be a damp squib. Ditto Pioneer's whatsitsname - can't remember for the life of me.

Most of them turned out to be no more than pretty acronyms.

 

[alchemist 1]

TWADDLE !.......... :wall:

 

[the record spot]

One of the reasons they limited the resolution to 16/44 was because of capacity. If they could have gone higher they almost certainly would have.

I very recently bought Bee Gees The Record compilation, and both CDs are packed with over 78 minutes of audio. My CD player was definitely noiser than usual. This had to be down to the longer length. Also, Too Much Heaven, while not harsh, was not sounding as smooth as I expected. I'm not saying this is absolutely because of the longer length, but I'm suspecting so.

The CD medium was in no way perfect.

With respect, this is a nonsense. I have several CDs that go beyond 75 minutes, none of which results in this issue on playback. None.

 

[RobinKidderminster]

Quite the opposite (I think?). I would argue that no 'sighted' test is of much value unless there are 'huge' differences. Expectation bias is not imo a weakness but merely a feature of our brains. Now I am confused!

If you are confused it's your own doing (or undoing I should say). I had my first CD player for 11 years, retaining the same amp and speakers, I bought my first DVD player and was appalled at its CD playback. Ditto for a NAD C521i CDP I briefly acquired - rubbish.

My expectations, rather naively, was that the Pioneer DVD deck would sound as good as my old Technics CDP. It didn't. It was rubbish. Ditto the NAD. And after 11 years, I didn't need any form of blind testing to tell me otherwise, so glaringly obvious was the difference. Because I had 11 years of ear training from my Technics.

So please, pretty please, don't dare tell me I have 'expectation bias' any other such malaise. Go and fob that off on someone else.

I am undone. I do dare to tell u that we ALL have expectation bias but as I said it is not a 'malaise' but something we all have. In reference to blind/sited trials I wholly agree that we dont need blind testing to hear significant differences in kit. Equipment in any given 'room' can be compared with other equipement and certainly extended listening is essential in careful selection of equipement. I would however argue, that finding subtle differences we should rely on blind abx testing.

 

[Covenanter]

Absolutely it's relevant that's it's from 1928. I mean Isaac Newton gave us theories in the seventeenth century and everyone's realised he was a ******* idiot.

I will repeat it was just a theory, and nearly 90 years on no-one has done additional research??? Comparing it to Newton is preposterous. BTW just gotta love your profanity (it shows up explicitly while commenting).

It isn't really a theory like Einstein's theories! It is a theorem and there is a mathematical proof which I could derive when I was at Uni 40 years ago. So unless there is an error in the maths, which is actually rather simple in the scheme of things, it is rock solid. The likelihood of there being an error in the maths is vanishingly small so I think you can take it as being fine.

Chris

PS This is a link to a Wiki page that shows the maths:

http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem

 

[the record spot]

The links to papers on why 16/44.1 isn't great have nothing to do with the sampling rate or frequency not capturing ALL of the audible information, they revolve around the design of anti alias brick wall filter on the dac output. This can be overcome by over sampling techniques that move the aliases further up the frequency spectrum.

16/44.1 is really all you need.

Yes, Technics tried to 'overcome' it with their subsequent MASH technology, turned out to be a damp squib. Ditto Pioneer's whatsitsname - can't remember for the life of me.

Most of them turned out to be no more than pretty acronyms.

Any good mastering engineer will get round the brickwalling issue, see also over compression, excess limiting and so on.

This issue isn't an issue and hasn't been for years until the labels opted to put out some of their releases mixed and mastered up to the max. This is not the fault of the medium as anyone who's heard how good well mastered music on a digital carrier (take your pick) sounds.

 

[lindsayt]

OK, this 16 bit 44.1Khz format.

Let's ask a few questions and give it an extreme test.

Let's say I want to capture a 20khz sine wave. How is that transferred into 16/44.1khz format? Can anyone show me what the bit pattern would look like for each cycle on our 20khz sine wave? Ideally shown as a graph?

But we've left out a big variable. The volume of the sine wave when it was recorded or when it will be played back at a 1:1 ratio of recorded to playback volume.

So, let's say I want to capture a series of 20khz sine waves in 3 db steps, starting at 30 dbs and going all the way up to 120 dbs. I did say this was an extreme test. That's 30 different volume levels. Are there enough bits per second to properly capture all of our 30 different 20khz sine waves in 16/44.1 format?

Let's go 1 step further. I now want to record 20khz triangular waves at my 30 different volume levels. This is a wave that on an oscilloscope would like a straight line drawn from peak to trough. How would 16/44.1 format cope with that? And would the actual bit pattern fully differentiate these triangular waves from the sine waves?

And please don't refer me to some boring paper on Nyquists theorem. I want to see the bit pattern for all of these, or possibly a graphical representation of the bit patterns.

I bet you, there aren't enough bit patterns to go round, or that the shape or size of either my sine or triangular waves get distorted due to a lack of bits per second.

 

[Anderson]

Thank you for the link. It puts paid to zealots who espouse blind-testing as the only valid method. Linn have their tune-dem method which is not necessarily blind. And of-course there are other sighted/blind methods too.

It doesn't, you lot are a lot like religious folk, keep the faith.

 

[CnoEvil]

Thank you for the link. It puts paid to zealots who espouse blind-testing as the only valid method. Linn have their tune-dem method which is not necessarily blind. And of-course there are other sighted/blind methods too.

It doesn't, you lot are a lot like religious folk, keep the faith.

What else would you expect from a forum full of blokes, all of whom think they're right! :shifty:

 

[steve_1979]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

 

[BenLaw]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

I thought you'd have used a different word.

 

[steve_1979]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

I thought you'd have used a different word.

The word I wanted to use started in 'b' and ended 'ks' but the swear filter automatically changed it to the word 'rubbish'. :roll:

 

[davedotco]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

This thread is like a load of virgins talking about sex.

People who do not understand the math telling us why Nyquist-Shannon is wrong, people who have never taken a third party conducted blind test telling us why they are flawed, people who do not understand the scientific definition of theorem or the implementation of anti-aliasing filters all having their say, pointless.

There really are only two things you need to know about blind testing, they are....

Blind testing of hi-fi equipment, scientifically rigorous enough to prove anything is very rare indeed, it is even possible that the scale of such testing could skew the results so badly as to render them meaningless. This argument will go on forever.

More informal blind testing, properly level matched, will show anyone who takes part just how small the differences in hi-fi components are when evaluations are made using just our ears. Any reasonably compedent, third party conducted test will show this and it is pretty clear that few of the contributers to this thread have ever taken part in such a test. Such tests are not rigorous enough to prove anything, but they are enlightening for those taking part.

 

[lindsayt]

davedotco, do you understand the maths behind the Nyquist theorem? And if so can you please explain it laymans terms, because I've never seen it explained in laymans terms. I'm quite happy to admit that I'm a virgin when it comes to mastering the Nyquist theorem.

And do you understand how the analog to digital conversion with 16 bit 44.1khz format actually works in terms of sampling audio signals and converting it to a load of 0's and 1's? And if so can you please explain this in laymans terms, inparticular how it captures and records the vast range of frequencies, volumes and signal shapes that you get with music?

 

[Overdose]

And do you understand how the analog to digital conversion with 16 bit 44.1khz format actually works in terms of sampling audio signals and converting it to a load of 0's and 1's? And if so can you please explain this in laymans terms, inparticular how it captures and records the vast range of frequencies, volumes and signal shapes that you get with music?

@lindsayt (and anyone else who might be interested)

THIS might be helpful.

 

[Overdose]

Recording is done at a minimum of 48KHz usually and more often than not at higher rates still, such as 96KHz and 192KHz.

Triangular and sawtooth waves, whilst problematic, are much more effectively captured at these elevated sampling rates.

Sine waves are not a problem and regardless of frequency are captured acurately if they are below the Nyqvist frequency, filtering takes care of the correct reconstruction of the wave.

Graphs are a red herring and do not illustrate what actually happens, but in essence, complicated mathematics take care of the reconstruction of the sine waves.

Presumably more mathematical wizadry reconstructs the triangular, square and sawtooth waves with a combination of other sine waves, but that's not something that I've not yet looked into.

 

[andyjm]

Let's go 1 step further. I now want to record 20khz triangular waves at my 30 different volume levels. This is a wave that on an oscilloscope would like a straight line drawn from peak to trough. How would 16/44.1 format cope with that? And would the actual bit pattern fully differentiate these triangular waves from the sine waves? And please don't refer me to some boring paper on Nyquists theorem. I want to see the bit pattern for all of these, or possibly a graphical representation of the bit patterns. I bet you, there aren't enough bit patterns to go round, or that the shape or size of either my sine or triangular waves get distorted due to a lack of bits per second.

Lindy,

You need to think about this differently. Your mental image of a 20KHz triangular wave is incorrect.

Any shape waveform can be broken down into a series of sinewaves at different amplitude, frequency and phase. Rather like white light that is in reality whole spectrum of colours, anything but a pure sine wave is in fact a whole spread of frequencies.

Your 20KHz triangular wave example has frequency components that are odd harmonics to the base frequency, so 60KHz, 100KHz and so on all the way to infinity. We can only capture frequencies below 20KHz, so the only component of a 20KHz triangular wave the will be captured is its base 20KHz sine wave.

Another way to think of this is that a sampling A2D converter has an input filter to cuttoff frequencies above 20KHz to avoid aliasing. If you passed your 20KHz triangular wave through this filter, all that would come out is a pure sine wave - and that is what would get sampled.

 

[Broner]

Profanity on a forum like this is never funny - it's banal and inane.

EDITED

This thread is like a load of virgins talking about sex.

People who do not understand the math telling us why Nyquist-Shannon is wrong, people who have never taken a third party conducted blind test telling us why they are flawed, people who do not understand the scientific definition of theorem or the implementation of anti-aliasing filters all having their say, pointless.

There really are only two things you need to know about blind testing, they are....

Blind testing of hi-fi equipment, scientifically rigorous enough to prove anything is very rare indeed, it is even possible that the scale of such testing could skew the results so badly as to render them meaningless. This argument will go on forever.

More informal blind testing, properly level matched, will show anyone who takes part just how small the differences in hi-fi components are when evaluations are made using just our ears. Any reasonably compedent, third party conducted test will show this and it is pretty clear that few of the contributers to this thread have ever taken part in such a test. Such tests are not rigorous enough to prove anything, but they are enlightening for those taking part.

It depends on what you wish to prove from the outset. There are quite a bit of hifi enthusiastics here who know that they can hear the difference between a couple of cables. A blind test does prove whether their claims are justified. Also, if you take together all the available reports of the blind tests that have been executed, a very, very clear pattern starts to emerge indicating that the species that can hear the difference between two proper cables has yet to evolve.

 

[Covenanter]

davedotco, do you understand the maths behind the Nyquist theorem? And if so can you please explain it laymans terms, because I've never seen it explained in laymans terms. I'm quite happy to admit that I'm a virgin when it comes to mastering the Nyquist theorem.

And do you understand how the analog to digital conversion with 16 bit 44.1khz format actually works in terms of sampling audio signals and converting it to a load of 0's and 1's? And if so can you please explain this in laymans terms, inparticular how it captures and records the vast range of frequencies, volumes and signal shapes that you get with music?

The maths isn't difficult but it does involve integral calculus and if you haven't studied that then it would be hard for you to understand the details. Any first year maths student at Uni could follow the maths.

There are three fundamental ideas though which are easy to understand:

- Any waveform can be broken down into a series of sine waves - there is something called the Fourier Transform which does this

- Any waveform can be completely represented by samples taken at twice the frequency of the highest sine wave frequency present in the waveform - this is the Nyquist-Shannon Theorem - the key word here is "completely", no extra information is gained by sampling at higher rates

- The waveform can be reconstructed from the samples by low pass filtering

Chris

 

[proffski]

He would say that of course.

Of course blind tests don't work because you can't tell whats being tested, so how can you tell if its a $1,000 cable or a $1 one? As these guys found out: http://consumerist.com/2008/03/03/do-coat-hangers-sound-as-good-monster-cables/

Someone who does not agree: http://seanolive.blogspot.co.uk/2009/04/dishonesty-of-sighted-audio-product.html

And someone else: http://www.bostonaudiosociety.org/bas_speaker/abx_testing.htm