On your reviews could you give us a rating for sound quality not taking price into account ?

[Snooker]

It really would be interesting to have a blind review test of sound quality as I am sure many of the cheaper end stuff would sound better than the really expensive stuff, I beleive you could buy a complete system for £1,000-£2,000, and have no or extremely little further improvement the higher in cost you went ?

 

[Clare Newsome]

Co-incidentally, we've just conducted a 'Big Question' feature where we did a blind test of Blu-ray players at different price levels - results in next issue....

 

[shooter]

Interesting Clare. Is this blind testing a one off or an idea that yout thinking of taking up on a regular basis over more components.

 

[Andrew Everard]

Interesting Clare. Is this blind testing a one off or an idea that yout thinking of taking up on a regular basis over more components.

The Big Question is a regular feature.

 

[ClaudioC]

This is a very good suggestion! I would also ask you to consider comparisons of stuff within different price ranges so as to get an idea if it is worth spending more on some equipment (e.g. Is MA Bronze BX2 at £ 250 much better to Dali Zensor 1 at £200 and much worse to B & W 685 at £385? - which is the best value for money ?)

Thanks

 

[Anonymous]

It really would be interesting to have a blind review test of sound quality as I am sure many of the cheaper end stuff would sound better than the really expensive stuff, I beleive you could buy a complete system for £1,000-£2,000, and have no or extremely little further improvement the higher in cost you went ?

I don't think that review setup would work. Say you had a pair of MA bookshelves that got 80% as a score from the perfect sound. The next model that comes out, if its better, will get 81% or more. Eventually, after several revisions, they'll be up to the 90's and closing in to the 100, making it look on paper like they're close to the higher end £10,000+ models, when in reality they're worlds apart.

Plus, if the really high end stuff gets better and takes the magic 100, every other score needs to be revised etc...

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

 

[Lee H]

It really would be interesting to have a blind review test of sound quality as I am sure many of the cheaper end stuff would sound better than the really expensive stuff, I beleive you could buy a complete system for £1,000-£2,000, and have no or extremely little further improvement the higher in cost you went ?

I don't think that review setup would work. Say you had a pair of MA bookshelves that got 80% as a score from the perfect sound. The next model that comes out, if its better, will get 81% or more. Eventually, after several revisions, they'll be up to the 90's and closing in to the 100, making it look on paper like they're close to the higher end £10,000+ models, when in reality they're worlds apart.

Plus, if the really high end stuff gets better and takes the magic 100, every other score needs to be revised etc...

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

I think your objections are trivially easy to address. If one were to take the best speaker you could find and said that was a 100 (or 50 or 736 it does not really matter) then all you would do when a better speaker came out would be to give it a score higher than 100. Who says it had to be a percentage?

Then what's your frame of reference? I pick the mag up as a first time reader and something has a score of 1142. What does that tell me?

 

[hammill]

It really would be interesting to have a blind review test of sound quality as I am sure many of the cheaper end stuff would sound better than the really expensive stuff, I beleive you could buy a complete system for £1,000-£2,000, and have no or extremely little further improvement the higher in cost you went ?

I don't think that review setup would work. Say you had a pair of MA bookshelves that got 80% as a score from the perfect sound. The next model that comes out, if its better, will get 81% or more. Eventually, after several revisions, they'll be up to the 90's and closing in to the 100, making it look on paper like they're close to the higher end £10,000+ models, when in reality they're worlds apart.

Plus, if the really high end stuff gets better and takes the magic 100, every other score needs to be revised etc...

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

I think your objections are trivially easy to address. If one were to take the best speaker you could find and said that was a 100 (or 50 or 736 it does not really matter) then all you would do when a better speaker came out would be to give it a score higher than 100. Who says it had to be a percentage?

 

[professorhat]

This is a very good suggestion! I would also ask you to consider comparisons of stuff within different price ranges so as to get an idea if it is worth spending more on some equipment (e.g. Is MA Bronze BX2 at £ 250 much better to Dali Zensor 1 at £200 and much worse to B & W 685 at £385? - which is the best value for money ?)

Thanks

That's what the Group Tests are all about. However, I would still say you should audition all three and make your own mind up as to what constitutes the best value for money, as money has different value to different people.

 

[nopiano]

That's what the Group Tests are all about. However, I would still say you should audition all three and make your own mind up as to what constitutes the best value for money, as money has different value to different people.

And there is nothing that says you will necessarily prefer the £250 models to the £200 ones. In fact, many would choose the cheaper speakers and spend the difference on a better source or cables.

On the broader original post, a numerical system might work, but what constitutes the make-up? I might value depth of field as 20% of the overall rating, but you might see that as just 5%. You might rate bass extension 25%, for me 10%. And that's before you go about assessing anything!

I regret it's never going to happen. It would be pretty useless to read that a docking speaker was scored at 12 out of 100, or 87 out of 736, wouldn't it?! I think I recall one USA mag saying that 100 represented the current state of the art, so hopefully, over time, that became better, but everything dropped relative to that absolute sound.

However, I share the frustration that an item is apparently marked down for sound because of price! Or marked up when the price falls. Better, in my view, to see 5 stars as roughly equating to "one of the best currently around", 4 stars as "above average for the class", and so on.

 

[the record spot]

Oddly enough, back in the "old days" (ahem...), review scores were made up of three elements, of which sound was one. This meant you could see that a product might have a four star review overall, but the sound quality was reckoned to be worthy of five. Must be going back nearly ten years since that scoring format last appeared.

 

[busb]

My personal opinion is that blind or double blind testing has its place but is flawed, albeit less flawed than sighted testing. The problem with such testing is that it's somewhat transient in that it normally happens as a single event or just one day. I've often listened to stuff like CDPs or amps in showroom & have ended up thinking that the equipment I've heard sounds essentially the same! Other times such as a month ago, I listened to 3 amps that all sounded different (to my relief & were a MF M6, a Supernait & Primare i32). I'm still not sure if the reason I heard differences was due to the fact that my ears are tuned to critical listening mode or not, during this period of heightened interest. I've also bought equipment to be initially not that impressed by. However, over a period of months , I noticed detail I've not heard before on familiar recordings. Would blind testing have given me similar insights? I think not.

I suggest the following procedure to test the effectiveness of double blind testing as a form of calibration - the deliberate & controlled introduction of defects. One could introduce noise, harmonic or intermod distortion, channel imbalance, volume shifts, cross-talk & frequency filtering to name a few. If music lovers/HiFi buyers have trouble hearing measurable differences, what chance have they of hearing subtle variations - such as the difference between interconnect cables that measure the same?

A few years back when I participated on unmoderated Usenet HiFi forums, some people would insist that only sighted testing was valid. Someone conducted an experiment with speaker cables where the listeners invariably preferred the fat ones over thin cables - you've guessed - the same cables were used throughout the test (I would want to see the setup!) This forum is a model of politeness compared to Usenet - some people got very personal!

As for also giving marks for absolute SQ - I think we need to know if spending a fortune actually does give better sound or not & by how much. This would involve trying to quantify how much spending twice the amount gives in improvement. Twice as good is a very subjective term whereas twice the cost is very measurable! Maybe we should choose equipment by looks, build quality, upgradability etc as well as having faith in our own ears rather than relying on (often contrary) reviews or sales people!

Paul B

 

[Anonymous]

making it look on paper like they're close to the higher end £10,000+ models, when in reality they're worlds apart.

True, I've heard a few £10k+ systems sounding far far worse than much more 'modest' systems.

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

The law of dimishing returns happens a lot lower than £4k!! In fact I've heard £1k systems at hi-fi shows besting a £30k+ systems. Also low-power + high efficiency often sounds best and simply costs less than the usual brute force approach.

 

[the record spot]

Dimishing returns? Kicks in a heck of a lot lower than 31k - try some WAV files of well mastered discs through (in my case) a 32Gb Touch and some good in-ear 'phones.

 

[manicm]

I may be in a minority here - but ultimately I don't see any advantages of blind testing over sighted testing - after all we don't listen to music blindfolded or always in the dark. And my hifi decisions have always been sighted and non the worse for it. And I think it's because I 'hunt' down specific characteristics in the sound - so blindfolding would have no advantage.

 

[Alec]

I may be in a minority here - but ultimately I don't see any advantages of blind testing over sighted testing - after all we don't listen to music blindfolded or always in the dark. And my hifi decisions have always been sighted and non the worse for it.

Though you can't actually know that with 100% confidence. No, you can't. Not that it matters a jot

Andrew has said before, OP, that they wont do that because it would be like saying "you, pauper, cannot afford the good one", or words to that effect. So he believes that the more expensive stuff is better.

 

[Anonymous]

I may be in a minority here - but ultimately I don't see any advantages of blind testing over sighted testing - after all we don't listen to music blindfolded or always in the dark. And my hifi decisions have always been sighted and non the worse for it. And I think it's because I 'hunt' down specific characteristics in the sound - so blindfolding would have no advantage.

Blind testing doesn't mean you have to be blindfolded, just that you don't can't see the equipment you're testing. It has a major advantage in that your brain cannot tell you that the nicer looking or more expensive item is better...which it does without you wanting it to, due to expectation, pride etc (placebo). You can still seek out "specific sound characteristics", because you only need your ears for that.

Check this out. It's a horizon video showing the McGurk effect. What we hear is closely related to what we see...

http://www.youtube.com/watch?v=gCvdZhrEmm4&feature=related

 

[hammill]

Then what's your frame of reference? I pick the mag up as a first time reader and something has a score of 1142. What does that tell me?

The table that currently gives a star rating give a points rating ordered by rating, so the highest value is the best. Can be split by price if you like. Hardly difficult to do.

 

[Anonymous]

Blind testing is the last thing many hi-end manufacturers would want to see.

Their favourite questions is: What hi-end amplifier can I get for £1k

And not: What's the best amplifier I can get for £1k

Competing on sonic merit along is a marketing man's worst nightmare. Especially for cables, that strange mixed up world of make believe where even their own adverts contradict themselves.

 

[Andrew Everard]

Blind testing is the last thing many hi-end manufacturers would want to see.

Their favourite questions is: What hi-end amplifier can I get for £1k

And not: What's the best amplifier I can get for £1k

Competing on sonic merit along is a marketing man's worst nightmare. Especially for cables, that strange mixed up world of make believe where even their own adverts contradict themselves.

Good to see agenda-driven twaddle at last elevated to an art-form – it's long overdue. :doh:

 

[Clare Newsome]

Getting back to our reviews - rather than what 'marketing' people think (irrelevant to our reviewers) - we're currently looking at new ways of presenting the information gathered from our comparative testing.

Traditional magazine space restrictions mean we concentrate on relaying the core information on a product; there's more detail we could be serving up. How/when we could provide that information - in a range of forms - is under examination.

We've been doing this for 35 years now :cheer: , and still going strong- review ratings and styles have continued to evolve over that time and will doubtless do so in our multimedia future.

Keep the constructive feedback coming - all useful.

 

[busb]

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

Really? I for one am not convinced at all by this assertion! It's as if you are saying that elevated prices are a guarantee of quality? I do wonder at times if many folk choose a price range they can afford then buy on magazine's or HiFi salesman's recommendations rather than actually listening themselves. How many salesmen will tell a potential customer to spend less than they think they need to? It's a shop's mandate to maximise profit without being blatantly dishonest. Shops also generally don't recommend what they don't sell. I don't approach HiFi shops with cynicism but with a degree of scepticism as I do with magazine opinions - particularly regarding cables.

 

[Mr Morph]

A syst costing £2,000 will have no chance standing up to the quality of high end actives or even a system costing £3-£4,000. I think to hit the wall of diminishing returns, you'd need to spend £4,000. After that, things get a biharder to justify.

Really? I for one am not convinced at all by this assertion! It's as if you are saying that elevated prices are a guarantee of quality? I do wonder at times if many folk choose a price range they can afford then buy on magazine's or HiFi salesman's recommendations rather than actually listening themselves. How many salesmen will tell a potential customer to spend less than they think they need to? It's a shop's mandate to maximise profit without being blatantly dishonest. Shops also generally don't recommend what they don't sell. I don't approach HiFi shops with cynicism but with a degree of scepticism as I do with magazine opinions - particularly regarding cables.

+1. Just because something is expensive doesn't mean it's any good. I've encountered a lot of hi-end stuff that sounded pretty bland. And I've also encountered some realtively cheap well engineered stuff that sounded a million bucks. Concentrate on the sound and you can't go wrong.

 

[Anonymous]

That's certainly what I've found Mr. Morph!

IME buying the most expensive kit is rarely a path to good audio.

 

[manicm]

I may be in a minority here - but ultimately I don't see any advantages of blind testing over sighted testing - after all we don't listen to music blindfolded or always in the dark. And my hifi decisions have always been sighted and non the worse for it. And I think it's because I 'hunt' down specific characteristics in the sound - so blindfolding would have no advantage.

Blind testing doesn't mean you have to be blindfolded, just that you don't can't see the equipment you're testing. It has a major advantage in that your brain cannot tell you that the nicer looking or more expensive item is better...which it does without you wanting it to, due to expectation, pride etc (placebo). You can still seek out "specific sound characteristics", because you only need your ears for that.

Check this out. It's a horizon video showing the McGurk effect. What we hear is closely related to what we see...

http://www.youtube.com/watch?v=gCvdZhrEmm4&feature=related

I don't agree, so are you telling me that a nicer looking Blu-ray player might trick me into perceiving as the better player? Maybe, but then I'll be looking at the telly and not the player.

Likewise when listening to music I'm not looking at the equipment, certainly not the amps or sources anyway. Seems to me this is thumb-sucking.