Subjective/objective testing /AB / AB/X, thoughts.

[jcbrum]

My position is this; imagine you are standing looking at an Aardvark, while typing so on a forum. Then people on the forum queue up to tell you that it's not possible, as they don't exist in your country. You reply that it is, as you are looking directly at it.

If you were a foo cable merchant, I wouldn't believe you either, because you talk nonsense.

JC

 

[jcbrum]

It's a credidibility problem.

Incredible claims need to be tested.

JC

 

[matt49]

If you were a foo cable merchant, I wouldn't believe you either, because you talk nonsense.

JC

No arguing or nastiness, simply discussion. Is it possible?

Seems not.

 

[MakkaPakka]

Are the die hard subjectivists willing to test anything blind?

Personally I found it very eye opening to test lossless files against low res MP3. The differences are so incedibly minor that it makes you realise that technology had this whole area sussed a while ago. If it takes serious concentration to tell 320kb apart from 128kb (which it does) then you can start to question how much money you really need to be spending.

 

[the record spot]

It's a credidibility problem.

Incredible claims need to be tested.

JC

Incredible spelling too methinks...

 

[the record spot]

If you were a foo cable merchant, I wouldn't believe you either, because you talk nonsense.

JC

What a miserly, mean-spirited little man you can be sometimes.

 

[fr0g]

Personally I found it very eye opening to test lossless files against low res MP3. The differences are so incedibly minor that it makes you realise that technology had this whole area sussed a while ago. If it takes serious concentration to tell 320kb apart from 128kb (which it does) then you can start to question how much money you really need to be spending.

I think the problem here may be the fact that we "know" that an MP3 (of whatever bitrate) IS in fact not as good as a lossless file. The same with CD v High res.

So we know for a fact that the higher res, higher bitrate file is better.

What we seem not to be able to accept is that for the most part, our hearing is simply not good enough to distinguish them.

It's pride imo.

 

[BenLaw]

It's a credidibility problem.

Incredible claims need to be tested.

JC

Incredible spelling too methinks...

Yes, it's meant to be 'incredidible'.

 

[CnoEvil]

My position is this; imagine you are standing looking at an Aardvark, while typing so on a forum. Then people on the forum queue up to tell you that it's not possible, as they don't exist in your country. You reply that it is, as you are looking directly at it.

If you were a foo cable merchant, I wouldn't believe you either, because you talk nonsense.

JC

Thank you for making my point clearer in one sentance, better than I did in a whole spiel.

 

[altruistic.lemon]

If you were a foo cable merchant, I wouldn't believe you either, because you talk nonsense.

JC

The man's on a charm offensive today

 

[John Duncan]

Can we keep this civil please? It's a perfectly reasonable topic for debate and needn't end up with me having to close it down.

 

[matt49]

Are the die hard subjectivists willing to test anything blind?

First, although I'm sceptical about blind tests of hifi, that's because I don't consider them to be sufficiently rigorous, for reasons I've stated repeatedly on this forum. So I wouldn't decribe myself as a subjectivist, let alone a 'diehard' one (whatever that means).

Second, I've done two blind tests. One was of (lossy) AAC files vs (lossless) ALAC files. The other was a heavily modded Sonos ZP90 vs an unmodded one. In both cases I scored about 80% in the 'X' part of the test. However, I don't think this counts for much, as the tests weren't properly rigorous (see above).

If it takes serious concentration to tell 320kb apart from 128kb (which it does) then you can start to question how much money you really need to be spending.

This is an important point, though I draw the opposite conclusion. During blind tests and 'normal' listening, the same acoustic phenomena are experienced. If something is felt to be wrong in a blind test, it will still be wrong in normal listening. Your response may be different: instead of consciously attending to the wrongness, you'll experience it unconsciously. The wrongness will still cause you to get less pleasure from listening.

 

[steve_1979]

Are the die hard subjectivists willing to test anything blind?

I've asked the same thing before several times and I can't remember a single occasion when any of them have tried it.

Come on Cno, are you (or anyone else) willing to take up the gauntlet and try an ABX test? (sorry to single you out again Cno but we had this conversation before about a year ago and you said that you'd try it one day in the future after you'd had your back operation.)

If you're willing to try comparing a 320kbps MP3 to a lossless WAV file I'll be happy to help you through each step of the process if you're not sure what to do. It's quite simple and I can take you through it in every detail if you want. I could even post screen shot pictures and web links showing you exactly what program needs to be installed and which buttons you need to press to run an ABX test.

 

[jcbrum]

Those who seem to think I have been rude or offensive to Cno (JD?) should carefully re-read my posts.

I was indulging his hypothesis. I said 'IF' you were a cable foo-merchant.

JC

 

[MakkaPakka]

I've posted this link before. No effort required at all 128kb MP3 vs 320kb MP3 with near-instant switching all from a webpage. This is a good place to start since the difference should be obvious.

www.mp3ornot.com

I failed when I tried on my laptp via headphones. I was 4/4 when I tried it on my proper setup and am confident I'd be pretty much 100% if I hadn't got bored, however; it's not easy - I have to replay several times to pick out the very subtle differences that are there.

 

[Overdose]

First, although I'm sceptical about blind tests of hifi, that's because I don't consider them to be sufficiently rigorous, for reasons I've stated repeatedly on this forum. So I wouldn't decribe myself as a subjectivist, let alone a 'diehard' one (whatever that means).

I'm not quite sure how rigorous a test needs to be to detect the large differences reported between equipment. Simply asking someone to listen to various pieces of equipment after removing the visual stimulous will give at least some indication that those differences are exagerated in the first place. It certainly seems to be the case so far.

Obviously more subtle differences would need to be more carefully tested and then the testing needs to be more rogorous, but at what point do you stop? If you have to sit in special conditions to have a remote chance of correctly detecting the subtle differences, then the point is moot, as ones own listening environment at home is going to be a much less controlled and more noisy.

Re your 80% score when testing the audio files and your modded ZP90, only you can decide if the test was rigorous enough for you to satisfy your curiosity. If you only scored 20%, I'm guessing it wouldn't be.

The key to the argument/discussion, as with most linked topics re differences, is what is realistically audible, not what is absolute. Remember that whatever is audible can be measured, but not all that can be measured is audible.

Simply trying out the variety of simple online tests available regarding hearing sensitivity, bias and illusion. should be enough to make people question their buying habits with regard to hifi equipment.

 

[John Duncan]

I've asked the same thing before several times and I can't remember a single occasion when any of them have tried it.

As pointed out earlier, I have been able to satisfy myself with simple A/B testing that I cannot tell the difference between 320 and lossless, thus saving myself the palaver of installing Windows. Other people's experiences may differ, of course.

 

[BigH]

I've posted this link before. No effort required at all 128kb MP3 vs 320kb MP3 with near-instant switching all from a webpage. This is a good place to start since the difference should be obvious.

www.mp3ornot.com

I failed when I tried on my laptp via headphones. I was 4/4 when I tried it on my proper setup and am confident I'd be pretty much 100% if I hadn't got bored, however; it's not easy - I have to replay several times to pick out the very subtle differences that are there.

Just tried it on my computer sound system 4/4 a piece of cake, no need to replay and only a few seconds for each sample.

How about some for mp3 (320) v 16 bit v 24 bit?

 

[the record spot]

Those who seem to think I have been rude or offensive to Cno (JD?) should carefully re-read my posts.

I was indulging his hypothesis. I said 'IF' you were a cable foo-merchant.

JC

Unfortunately, the tense you employed suggests otherwise, as you'd more accurately have continued using the past and not moved onto the present one in your post.

 

[John Duncan]

Those who seem to think I have been rude or offensive to Cno (JD?) should carefully re-read my posts.

There you go again, presuming I'm a EDITED...

 

[davedotco]

First, although I'm sceptical about blind tests of hifi, that's because I don't consider them to be sufficiently rigorous, for reasons I've stated repeatedly on this forum. So I wouldn't decribe myself as a subjectivist, let alone a 'diehard' one (whatever that means).

I'm not quite sure how rigorous a test needs to be to detect the large differences reported between equipment. Simply asking someone to listen to various pieces of equipment after removing the visual stimulous will give at least some indication that those differences are exagerated in the first place. It certainly seems to be the case so far.

Obviously more subtle differences would need to be more carefully tested and then the testing needs to be more rogorous, but at what point do you stop? If you have to sit in special conditions to have a remote chance of correctly detecting the subtle differences, then the point is moot, as ones own listening environment at home is going to be a much less controlled and more noisy.

Re your 80% score when testing the audio files and your modded ZP90, only you can decide if the test was rigorous enough for you to satisfy your curiosity. If you only scored 20%, I'm guessing it wouldn't be.

The key to the argument/discussion, as with most linked topics re differences, is what is realistically audible, not what is absolute. Remember that whatever is audible can be measured, but not all that can be measured is audible.

Simply trying out the variety of simple online tests available regarding hearing sensitivity, bias and illusion. should be enough to make people question their buying habits with regard to hifi equipment.

My view remains the same, taking part in any reasonably well organised blind test can be extremely instructive to any hi-fi enthusiast.

Differences often considered to be 'night and day' can simply disappear in blind tests and the shear difficulty in picking even quite large differences can be, to say the least, pretty interesting.

If you have never been involved in such a test it is really hard to explain just how difficult it is to pick even 'known' differences, particularly if the listener is experienced and used to hearing differences in normal, informal testing.

As I have said many times, I do not believe that blind testing is the definitive answer to anything in hi-fi, however experiencing it can really help the listener bring a sense of perspective and reality to more 'normal' auditioning.

 

[the record spot]

I've asked the same thing before several times and I can't remember a single occasion when any of them have tried it.

As pointed out earlier, I have been able to satisfy myself with simple A/B testing that I cannot tell the difference between 320 and lossless, thus saving myself the palaver of installing Windows. Other people's experiences may differ, of course.

Count me in on that. In my case you can take that down to 188kbps. My external hard drive has a mixed bunch of tracks on it and they all sound good.

 

[matt49]

The key to the argument/discussion, as with most linked topics re differences, is what is realistically audible, not what is absolute. Remember that whatever is audible can be measured [...]

Certainly, whatever is audible can in principle be measured. That doesn't mean we're actually able to measure what's audible. For instance, experiments on jitter in human voices have shown that listeners give more reliable assessments of the amount of roughness/jitter than measurements do. Whilst we are of course prone to all manner of auditory illusions, human hearing is exceptionally sensitive, especially in the frequency range of the human voice (for obvious reasons).

[...] but not all that can be measured is audible.

True, but human auditory experience extends well beyond the range of 12Hz to 20KHz that's normally cited in these discussions. That range represents what humans can consciously hear. We can experience subliminally frequencies both higher and lower than this range. The same applies to our ability to experience changes in volume.

 

[fr0g]

I've asked the same thing before several times and I can't remember a single occasion when any of them have tried it.

As pointed out earlier, I have been able to satisfy myself with simple A/B testing that I cannot tell the difference between 320 and lossless, thus saving myself the palaver of installing Windows. Other people's experiences may differ, of course.

Count me in on that. In my case you can take that down to 188kbps. My external hard drive has a mixed bunch of tracks on it and they all sound good.

I think this makes sense.

I have read in various places that a 190 Kbps VBR rip using LAME is pretty much transparent. I'd agree. I have mine set to 220 VBR just in case. In the case of AAC it can be a bit lower.

 

[jcbrum]

Those who seem to think I have been rude or offensive to Cno (JD?) should carefully re-read my posts.

I was indulging his hypothesis. I said 'IF' you were a cable foo-merchant.

JC

Unfortunately, the tense you employed suggests otherwise, as you'd more accurately have continued using the past and not moved onto the present one in your post.

Cno's hypothesis was presented in the present tense. As was my reply to his hypothesis.

It is clear what I meant, whatever anyone wishes to read into it.

JC