london based double blind listening test anyone?

[Glacialpath]

There are well established approaches for tests of this nature. Have a look at:

http://en.wikipedia.org/wiki/ABX_test

Of course ABX is a good way of testing. My suggestion is just another way I would like these tests to be considered.

After all if in the first 3 passes of the songs people only add one or two more things to their lists but then add ten more things to the new list after the change has been made I would say that was a pretty conclusive result.

If they still don't add anything after a change to the set up has been made then it either concludes they can't hear as well as they thought or there is no difference.

 

[matt49]

But in fairness, without both sides present, you may as well just have a piss up :cheers:

I can imagine two reasons why one might want to have a mix of cable-believers and cable-sceptics participating in the test. One reason makes sense, the other doesn't.

You might want to have both sides participating so that the two sides can keep one another honest regarding the design and conduct of the test. This seems like a very good reason.

Alternatively you might want both to participate so that the results reflect a spectrum of attitudes. This would be a very bad reason.

Imagine if the cable-believers managed repeatable positive results in the test, and the cable-sceptics produced null results. What would the conclusion be? The obvious conclusion would be that the cable-sceptics were unconsciously predisposed by expectation bias to produce a null result. Their results would have no bearing on the test. There's really no point in having them as participants.

This is why properly conducted tests of this kind usually involve prior screening of participants. Obviously you would screen out anyone who had impaired hearing. You’d also exclude any people who were predisposed by their expectations to produce a null result.

:cheers:

Matt

 

[Excitable Boy]

Count me in...could be really interesting.

 

[fr0g]

But in fairness, without both sides present, you may as well just have a piss up :cheers:

I can imagine two reasons why one might want to have a mix of cable-believers and cable-sceptics participating in the test. One reason makes sense, the other doesn't.

You might want to have both sides participating so that the two sides can keep one another honest regarding the design and conduct of the test. This seems like a very good reason.

Alternatively you might want both to participate so that the results reflect a spectrum of attitudes. This would be a very bad reason.

Imagine if the cable-believers managed repeatable positive results in the test, and the cable-sceptics produced null results. What would the conclusion be? The obvious conclusion would be that the cable-sceptics were unconsciously predisposed by expectation bias to produce a null result. Their results would have no bearing on the test. There's really no point in having them as participants.

This is why properly conducted tests of this kind usually involve prior screening of participants. Obviously you would screen out anyone who had impaired hearing. You’d also exclude any people who were predisposed by their expectations to produce a null result.

:cheers:

Matt

I disagree with your analysis.

I would imagine if there was* found to be an audible difference between cables, and only the believers heard it, they would put it down to the non-believers having golden ears, which to them would explain a thing or 2. I imagine anyone interested in the test would try as hard as they could to spot differences.

Most non-believers after all, are still audiophiles, and they still crave the best sound, and they still swap boxes.

*not that I would expect any differences to be heard by anyone with cables that are of sufficient quality unless there are some with purposefully odd electrical properties.

 

[RobinKidderminster]

Good luck lejockey. I would ignore some of these 'helpful' suggestions, state your rules and go for it. :cheers:

 

[RoyD]

I'd like to attend as well as I'm based in London too.

 

[John Duncan]

Are we talking about analogue cables (speaker and interconnect) only?

I can provide a couple of pairs of moderately exotic (£15-£20/m) speaker cables, but none of my interconnects cost more than fifty quid.

If it's digital cables, I'm not getting involved

 

[AlmaataKZ]

I am in, subject to dates.

can bring Benchmark DAC1 HDR, some interconnects, atc scm 7

tests can take time so need to set a realistic day plan. More test may mean more than one day.

good detailed agreed procedure will help achieve reputation

 

[Glacialpath]

Are we talking about analogue cables (speaker and interconnect) only? I can provide a couple of pairs of moderately exotic (£15-£20/m) speaker cables, but none of my interconnects cost more than fifty quid. If it's digital cables, I'm not getting involved

To me it is a test of cables. So all types of cables. There doesn't seem much point in only testing interconnects.

 

[Hi-FiOutlaw]

Can i bring this mains cable along?

 

[Hi-FiOutlaw]

Be very carefull, there may be some pitfalls... :twisted:

 

[plastic penguin]

Be very carefull, there may be some pitfalls... :twisted:

He could also be saying:

"I am talking to you in the cabinet room of this house. This morning my ambassador, the wife, handed you a final note stating that, unless you get out of that stupid box by 11 o'clock, a state of eviction will exist between us.

"I have to tell you now that no such undertaking has been received, and that consequently these cables and instruments will implode."

 

[hifikrazy]

Hopefully ellisdj, Thompsonuxb and hifikrazy will volunteer from the pro cable side.

i do hope so, but no point in inviting ellisdj, he's already stated that even if he had proof of differences, he would choose not to believe it. :shifty:

But in fairness, without both sides present, you may as well just have a piss up :cheers:

I would certainly be keen to participate but unfortunately I'm 10,500km away

 

[Happy_Listner]

Conducting a double blond test at your house? How old are these women?

 

[lejockey]

@happy listener judging by your profile picture, too old for you

I am pleased that the response has been good so far. When I get home from work, and can sit in front of my laptop and not on my phone, I shall write a bit more.

 

[matt49]

I disagree with your analysis.

I would imagine if there was* found to be an audible difference between cables, and only the believers heard it, they would put it down to the non-believers having golden ears, which to them would explain a thing or 2. I imagine anyone interested in the test would try as hard as they could to spot differences.

Most non-believers after all, are still audiophiles, and they still crave the best sound, and they still swap boxes.

*not that I would expect any differences to be heard by anyone with cables that are of sufficient quality unless there are some with purposefully odd electrical properties.

I don't understand the sentence beginning 'I would imagine ...'. Can you rephrase?

You seem to be saying that the sceptics (among whom I count myself) are more immune to expectation bias than the believers. What grounds do you have for that? I can't think of any. Scepticism is a belief like any other, and is therefore a source of bias.

 

[Broner]

I disagree with your analysis.

I would imagine if there was* found to be an audible difference between cables, and only the believers heard it, they would put it down to the non-believers having golden ears, which to them would explain a thing or 2. I imagine anyone interested in the test would try as hard as they could to spot differences.

Most non-believers after all, are still audiophiles, and they still crave the best sound, and they still swap boxes.

*not that I would expect any differences to be heard by anyone with cables that are of sufficient quality unless there are some with purposefully odd electrical properties.

I don't understand the sentence beginning 'I would imagine ...'. Can you rephrase?

You're not the only one and I also follow the reasoning in your previous reaction. Participants need to be screened so that a possible positive result from people who believe cables make a difference, are not offset by those who don't. Not-wanting to hear a difference can be an important factor that could influence the outcome. The results should show how 'believers', 'sceptists' and those two groups combined perform. Depending on how many swaps are done, it might even be interesting to see individual results. In any case, it will be interesting to hear the stories: what were people actually thinking? Was it easy to detect differences? Did their opinions change in any way?

 

[Glacialpath]

This is becoming very complex but we want this doen properly. So how would you go abouot screening the attendies and what characteristics would be considdered to be acceptable and whcih one not acceptable to the test.

I still believe the test has to be done scientifically but also some tests that only involve our ears and perseption of what we hear. You can't have one without the other.

What set ups can be agreed on and would all of us be able to get involved with swapping kit/cables.

The whole argument for for and against need to be laid out so everyone knows what is involved and being looked at. Are there only some components of a system the benifit from better cables or does the whole system beinfit. My suspicion is Amp aren't affected by mains cables or mains supply but CDPs DAC, Stearmers Blu-Ray players and the like are. I say this as the experiment I did with my guitar head made no difference regarding changing the mains block and cable and tuning the volumes up full against stock mains block and cable and turning the mains up to full. There was no difference either audibly or on the wave form I recorded. This test was only performed once so can not be conclusive. The USB cable was also a stock one. Before when I did a cable experiment on my guitar rigg the USB cable was a Chord Company one and there were differences but only whne the guitar lead was changed and not when the mains was changed.

 

[davedotco]

I think that this is becomming unnecessarily complex. Producing a scientifically viable study is way beyond the capabilities of a bunch of audio enthusiasts in a limited time.

Best to keep it simple, maybe two or three speaker and interconnects and see if anyone can reliably pick the difference.

I have said this before but I consider a simple test of this type to be instrumental in showing enthusiasts just how difficult this can be. Not saying whether differences will be heard or not, just that it will show just how difficult this can be once the visual clues are removed.

 

[drummerman]

I think that this is becomming unnecessarily complex. Producing a scientifically viable study is way beyond the capabilities of a bunch of audio enthusiasts in a limited time.

Best to keep it simple, maybe two or three speaker and interconnects and see if anyone can reliably pick the difference.

I have said this before but I consider a simple test of this type to be instrumental in showing enthusiasts just how difficult this can be. Not saying whether differences will be heard or not, just that it will show just how difficult this can be once the visual clues are removed.

A well made point.

It also underlines somewhat what I've said earlier.

We do listen with our ears and eyes. So if it makes you feel better to have spent a little bit more on your cables then why not.

regards

 

[Broner]

So how would you go abouot screening the attendies

That’s pretty simple. You ask: do you think an X-type of cable (depending on what you are testing) can make an audible difference in sound, provided that all the cables are designed according to generally accepted good manufacturing processes? (you can elaborate a bit more on what exactly constitutes GMP for a specific cable)

The respondent can then say: yes, no or I don’t know / I’m not sure.

and what characteristics would be considdered to be acceptable and whcih one not acceptable to the test.

Let’s say you test 2 cables. You could let people first knowingly listen to cable A. After that, you repeat it with cable B. You could even repeat this for a couple of times. Afterwards, when the blind listening test starts, you can ask people whether they are listening to cable A and B during a number of swaps. If you want a double blind test, you could even get someone involved who doesn't talk to the participants and is not involved in the processing of the data. With respect to your question ‘what characteristics one should listen to’: I’d say that’s not relevant. It’s about hearing a difference and any characteristic will do. (Please note that this is a very simple description of something that can be made very complex, depending on what it is you are testing and how reliable you want your results to be).

Fortunately, many other people have already done something similar and their descriptions of their set ups can be used to model the test.

With respect to what Dave & Drummerman have said, I guess it also depends on whether other people should be able to learn about the results on this forum. If yes, then following some procedures is absolutely necessary to prevent any possible confusion upfront and discontent afterwards. The last thing you want is that participants start having a fight on the forum when the results are presented.

By the way, I don’t think that following a couple of procedures, such as asking the participant whether he’s a believer or a skeptic, would make things very complex. If you undertake such a thing, there are a couple of minimal provisions you can take to prevent some obvious problems, and in my opinion that adds to the value of the test without making it difficult. The results may still not be of scientific value, but could be interesting enough to write a little summary about on Whathifi (with pictures of course, gotta have pictures).

In any case, I would always let 1 person work out a basic set up (preferably someone with an academic background in a discipline where such tests and statistical analyses are common practice, e.g. someone who has studied psychology, sociology or medicine), who sends out a draft and allows for comments or suggestions. This person then finalizes the draft and hopefully everyone is still happy to join in.

 

[davedotco]

Broner,

I would strongly believe that any test within the means of an amateur group like this, would have so many loopholes in terms of procedure or analysis that no one not present would accept the results.

To be honest, I would expect many of those taking part would have 'issues' too. Having taken part in third party conducted blind tests in the past, hearing differences unsighted is extremely difficult, the best we could reasonably hope for is to show those taking part just how small the real differences are and how the absolute certainty of sighted tests is, to put it kindly, not real.

 

[Vladimir]

I think that this is becomming unnecessarily complex. Producing a scientifically viable study is way beyond the capabilities of a bunch of audio enthusiasts in a limited time.

Best to keep it simple, maybe two or three speaker and interconnects and see if anyone can reliably pick the difference.

I have said this before but I consider a simple test of this type to be instrumental in showing enthusiasts just how difficult this can be. Not saying whether differences will be heard or not, just that it will show just how difficult this can be once the visual clues are removed.

And video everything for youtube!

 

[davedotco]

I think that this is becomming unnecessarily complex. Producing a scientifically viable study is way beyond the capabilities of a bunch of audio enthusiasts in a limited time.

Best to keep it simple, maybe two or three speaker and interconnects and see if anyone can reliably pick the difference.

I have said this before but I consider a simple test of this type to be instrumental in showing enthusiasts just how difficult this can be. Not saying whether differences will be heard or not, just that it will show just how difficult this can be once the visual clues are removed.

And video everything for youtube!

If you are coming bring pizza, lasagne is far too messy...... 8)

 

[Broner]

Hi Dave,

It depends on what you want to achieve. To rely on a couple of procedures doesn’t need to mean that people external to the test should see it as proof of whatever the outcome is, but it can help people understand that this test was taken with good intentions and that some obvious pitfalls have been avoided that could otherwise diminish any value of the test. (the suggestion done by Matt earlier was an obvious one in my opinion if it’s the idea to use the results and write a little piece on this forum).

Please note though that I haven’t suggested anything complex so far. But just as ‘blind swapping’ is inherent to a blind test, I tend to think that following several other procedures (how long is each cable listened to for how many times? Are people allowed to talk with each other? Is the data processed in any way and if so, is it also done separately for believers and skeptics?) can also be important for even the most basic exercise, because they can help to bring clarity to the undertaking, thereby preventing possible discontent, and allow for posting some results on a forum such as this.

In any case, it’s not so much about following the ‘right’ procedures, but about being in agreement about how things are going to take place, and what can possibly be concluded afterwards.