What do you read into WHFSAV star review system?

[Andrewjvt]

A few points.

- We have always said our reviews are a guide and you should do your own research whenever possible. Prefer something else? That's OK! 

- We review as a team. In reality this means at least two people see/hear/use every product we review. That second person is often our technical editor, Ketan (who has a mechanical engineering BSc and used to work at Hi-Fi World, seeing as some people asked!).

- We argue about plenty of products, because there can always be elements of personal preference, but we come to a consensus view based on what we think is the right recommendation for the average consumer. Reading the full review instead of just looking at the star rating is always recommended.

- We meet with the industry and manufacturers to discuss their products all the time. Companies often insist on bringing kit to our test rooms and talking us through them so this is quite common.

- We don't take any notice of who is and who isn't buying advertising (for the millionth time). Most of the editorial team wouldn't even know.

- The review team isn't all young men. Some of us aren't very young, others certainly aren't men. All of us have a solid understanding of how the products we review work. We're not all electrical engineers but we absolutely don't believe that's necessary. Learning what to listen for with audio products, and look out for with video products, is certainly something you can be taught and improve, which is why new reviewers take a lot of training. We don't get people off the street who are reviewing £10,000 amps a week later.

- We have always been a mainstream publication, with an audience of very knowledgable enthusiasts (you lot) and casual buyers. Writing for both audiences is a challenge, and it's part of the reason why we try to keep it thorough, knowledgeable and informative, without being overly technical. That said, some of our reviews - such as the Temptations section - certainly don't shy away from getting techy.

- We built new test rooms two years ago (which you can read about here), which we took time to make more 'real world' and less like a test room. Our aim has always been that our recommendations are relevant to pretty much everyone, and you shouldn't require an industry-grade listening room to get the same results. 

- Lastly (because I need to catch a train), we're doing our best! We are on the same side as you! We're all into music, movies and technology, and we're lucky enough to play around with new gear and write about it. And we endeavour to give reliable reviews and decent advice. Simple as that. 

We will hopefully have some more events at What Hi-Fi? HQ in the near future so more people can see the rooms and meet the team. So look out for those.

Any more feedback, gratefully received.

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

 

[QuestForThe13thNote]

When I was a dealer, I offered advice that could be backed by a straightforward demonstration, now that I am not, I offer advice on technical or practical matters only and leave the subjective drivel to others.

By that token subjective was ok when you profited from it ... ?

I have got to shoot out so might not see a reply - just so Dave doesnt get offended - that was meant tongue in cheek but also a bit provocative as well for benefit of the conversation - I would call it Banter

No problem Ellis, it is a pretty good point.

As I have tried to make clear on numerous occasions, subjective evaluations are fine for the person concerned, but do not transfer well to other people with different experiences.

As a dealer, you would try and 'read' a customer and work out the demonstration that would be the most effective, there is of course a commercial angle. Occasionally I would get someone in the shop that I simply thought would be a waste of time demonstrating for, so rightly or wrongly I made up my mind and moved the customer on.

One thing became both clear and very frustrating, was the number of customers who simply would not hear anything that contradicted the gospel that the WHF (and some other) reviews became (I'm thinking late 90s here). You could demonstrate, quite clearly in many cases, that the latest 5* wonder was clearly outperformed by other product yet the customer would not accept the results of his own listening. One of the obvious responses was that it was 'confusing' and needed 'thinking about'.

A lot of people would then discount the evidence of their own ears and buy the 5* product elsewhere, what was amusing was when they revisited the shop after a month or two, the conversation invariable starting with the line, "I don't think I'm quite getting the best from my system....".

this is really quite ridiculous as on the one hand you say that everyone unto others has subjective arguments or views on making recommendations on hi fi, which is right, and you'd prefer to keep it to technicalities (which is fine), so you must accept that people's decisions are their own. But then you are critical when they choose what hi fi 5 star as not being their own judgements because they don't agree with you and you think are just because of what hi fi, and not their actual judgements. Was it you trying to get them to buy more expensive stuff if as you say you don't demo stuff for customers.

I'd expect every customer has the right to a demo, as they may be a potential customer and if it were my business I wouldn't care, all I'd want to extol on them is enthusiasm for hi fi. If you weren't doing that, sound likes a shop I'd avoid!!

You really don't get this, do you!

People interact with a hi-fi system in a highly subjective way, it's important that you react to the setup in a way that works for you. This will help to maximise the enjoyment that you get from a system but this is personal to you, no-one else. I am not remotely critical of people choosing whatever setup they like but choosing one because someone else likes it often does not work.

Let somebody else, a reviewer perhaps or maybe someone on a hi-fi forum, choose your system for you and the result is often dissapointing. Sit down with a competent dealer, play some setups, express your views and refine your choice, that is the way forward. I have supplied many systems that would not have been to my choice, but suited the customer, that's just how it is

No buying a system because someone likes it is often a very good reason to buy, as it's rated and well regarded and shows a lot do. The reviews like what hi fi help with this, as I say many people review at the mag, also looking at other mags who rate it too. So a good measure of being good already. Buying a system that isn't reverred but you get to the shop and the dealer tries to convince you otherwise, and you think it sounds best of all you hear, sounds to me like dealer selling usually for own motive. You'd have to be very cautious. When you hear people coming out of a dealer buying rega speakers for instance instead of pmc's, you have to wonder on this selling thing as almost everyone thinks the similar pmc's are universally better, as reflected by almost every review in comparison. You might be an exception but caution is needed

What I normally do is just get them to keep quiet and I listen and most dealers youd have to count your fingers after shaking their hand, such is how they are solely concentrated on shifting a certain brand, or highest margin etc. So a lot of what they say on what's best, has no value for me. It's all commercially driven. What is credible is when they say it's fantastic, as do reviews, as do other customers. And no you don't have a better ear than the mags, or the customer, goes back to elitist thing. I despise this kind of dealer knows best thinking you have. Nor is the dealer particularly specialised to tell anoyone on what sounds better. How can they be when I'm listening too and make comparisons on speakers they have at similar prices. I'm only really interested how many pairs of speakers for instance they sell of a brand I like against another, what price they can do, but I'd doubt sometimes that selling numbers info is even given truthfully. Hi fi is such a hard business for dealers they just have to do whatever to get sales. Often you wouldn't get the true reality of the speaker if you heard it for a very short time in a dealer, as opposed to doing that, comparing what the reviews say, what other buyers say, making your own judgements on if the reviews are right or not. I listen for at least an hour and make comparisons and I'm a right pain to them I'm sure. That's why reviews are so important and are not to be dismissed just to visit a dealer. Absolutely the worst way of just relying on the dealer. I'd come out with some right crap if I just did that. Buying stuff is a balancing act of juggling stuff and reaching conclusions that you like with what others say, and these five star reviews are very important as an opinion as a part of this process, especially as they are doing it all day long. How many dealers review their own stuff if only to form an opinion themselves. Hardly any, as they just sell what makes best returns and sell accordingly.

but interesting reading Joes comments, as pretty much all I've been saying at relying on these reviews is reflected in these comments and reviews as opinions.

 

[chebby]

What I normally do is just get them to keep quiet and I listen and most dealers youd have to count your fingers after shaking their hand ...

So you've just accused 'most' dealers of being crooks?

 

[davedotco]

A few points.

- We have always said our reviews are a guide and you should do your own research whenever possible. Prefer something else? That's OK!

- We review as a team. In reality this means at least two people see/hear/use every product we review. That second person is often our technical editor, Ketan (who has a mechanical engineering BSc and used to work at Hi-Fi World, seeing as some people asked!).

- We argue about plenty of products, because there can always be elements of personal preference, but we come to a consensus view based on what we think is the right recommendation for the average consumer. Reading the full review instead of just looking at the star rating is always recommended.

- We meet with the industry and manufacturers to discuss their products all the time. Companies often insist on bringing kit to our test rooms and talking us through them so this is quite common.

- We don't take any notice of who is and who isn't buying advertising (for the millionth time). Most of the editorial team wouldn't even know.

- The review team isn't all young men. Some of us aren't very young, others certainly aren't men. All of us have a solid understanding of how the products we review work. We're not all electrical engineers but we absolutely don't believe that's necessary. Learning what to listen for with audio products, and look out for with video products, is certainly something you can be taught and improve, which is why new reviewers take a lot of training. We don't get people off the street who are reviewing £10,000 amps a week later.

- We have always been a mainstream publication, with an audience of very knowledgable enthusiasts (you lot) and casual buyers. Writing for both audiences is a challenge, and it's part of the reason why we try to keep it thorough, knowledgeable and informative, without being overly technical. That said, some of our reviews - such as the Temptations section - certainly don't shy away from getting techy.

- We built new test rooms two years ago (which you can read about here), which we took time to make more 'real world' and less like a test room. Our aim has always been that our recommendations are relevant to pretty much everyone, and you shouldn't require an industry-grade listening room to get the same results.

- Lastly (because I need to catch a train), we're doing our best! We are on the same side as you! We're all into music, movies and technology, and we're lucky enough to play around with new gear and write about it. And we endeavour to give reliable reviews and decent advice. Simple as that.

We will hopefully have some more events at What Hi-Fi? HQ in the near future so more people can see the rooms and meet the team. So look out for those.

Any more feedback, gratefully received.

I have, on occasion, defended the honesty of your review process, particularly against those who suggest that it is tied to advertising as I know that is simply not the case.

Personally I feel that the magazine has always done well when writing for the more casual buyer but less well when addressing the more committed enthusiasts. The depth and detail (some of it technical) that the enthusiasts would ideally want may well be beyond the scope of a mainstream publication like WHF.

In some respects the more enthusiastic readers take some of your writing far too seriously, I'm not sure what you can do about this as the relatively simple, straitforward reviews suit the more casual, mainstream audience but less so the enthusiast.

The big problem for WHF (and other mags) is that you are part of the industry, and the industry very definitely has it's own mindset, a sort of 'groupthink' that is all pervasive.

I make that observation as someone who has been part of the industry for many years but am now what I refer to as a 'civilian'.

 

[QuestForThe13thNote]

What I normally do is just get them to keep quiet and I listen and most dealers youd have to count your fingers after shaking their hand ...

So you've just accused 'most' dealers of being crooks?

in a kind of jokey way with the comment about counting fingers, that I don't trust them so far as what they think is best. It works for me, that I don't go in having a relationship with them on trust, just being friendly, then I think it's works better and I can make all my own judgements. They just put the stuff in front of me and I make comparisons, and a couple of local dealers let me borrow stuff anyway.

 

[QuestForThe13thNote]

A few points.

- We have always said our reviews are a guide and you should do your own research whenever possible. Prefer something else? That's OK!

- We review as a team. In reality this means at least two people see/hear/use every product we review. That second person is often our technical editor, Ketan (who has a mechanical engineering BSc and used to work at Hi-Fi World, seeing as some people asked!).

- We argue about plenty of products, because there can always be elements of personal preference, but we come to a consensus view based on what we think is the right recommendation for the average consumer. Reading the full review instead of just looking at the star rating is always recommended.

- We meet with the industry and manufacturers to discuss their products all the time. Companies often insist on bringing kit to our test rooms and talking us through them so this is quite common.

- We don't take any notice of who is and who isn't buying advertising (for the millionth time). Most of the editorial team wouldn't even know.

- The review team isn't all young men. Some of us aren't very young, others certainly aren't men. All of us have a solid understanding of how the products we review work. We're not all electrical engineers but we absolutely don't believe that's necessary. Learning what to listen for with audio products, and look out for with video products, is certainly something you can be taught and improve, which is why new reviewers take a lot of training. We don't get people off the street who are reviewing £10,000 amps a week later.

- We have always been a mainstream publication, with an audience of very knowledgable enthusiasts (you lot) and casual buyers. Writing for both audiences is a challenge, and it's part of the reason why we try to keep it thorough, knowledgeable and informative, without being overly technical. That said, some of our reviews - such as the Temptations section - certainly don't shy away from getting techy.

- We built new test rooms two years ago (which you can read about here), which we took time to make more 'real world' and less like a test room. Our aim has always been that our recommendations are relevant to pretty much everyone, and you shouldn't require an industry-grade listening room to get the same results.

- Lastly (because I need to catch a train), we're doing our best! We are on the same side as you! We're all into music, movies and technology, and we're lucky enough to play around with new gear and write about it. And we endeavour to give reliable reviews and decent advice. Simple as that.

We will hopefully have some more events at What Hi-Fi? HQ in the near future so more people can see the rooms and meet the team. So look out for those.

Any more feedback, gratefully received.

I have, on occasion, defended the honesty of your review process, particularly against those who suggest that it is tied to advertising as I know that is simply not the case.

Personally I feel that the magazine has always done well when writing for the more casual buyer but less well when addressing the more committed enthusiasts. The depth and detail (some of it technical) that the enthusiasts would ideally want may well be beyond the scope of a mainstream publication like WHF.

In some respects the more enthusiastic readers take some of your writing far too seriously, I'm not sure what you can do about this as the relatively simple, straitforward reviews suit the more casual, mainstream audience but less so the enthusiast.

The big problem for WHF (and other mags) is that you are part of the industry, and the industry very definitely has it's own mindset, a sort of 'groupthink' that is all pervasive.

I make that observation as someone who has been part of the industry for many years but am now what I refer to as a 'civilian'.

its a magazine for the mass market quite obviously, which I think is quite good for quality hi fi stuff too, as they can be less 'up themselves' than all the technical people and look at it as enthusiasts rather than someone with a measuring device. Much more ear pricking and credible for me.

Im sure what hi fi aren't concerned about your comment about readers taking it too seriously. I suspect this is your view that people buy just off of what hi fi reviews, which I must say is doing people a disservice. And I'm sure your point is a little too disingenuous here too as what hi fi I'm sure would be delighted if people take the articles seriously as opinion and rely on iT if reviews are correct against customer opinion. It shows they are doing their job properly.

More power to 'group thinks' and lots of different ones. More to balance up against more biased dealers and manufacturers interested too commercially and mags just to what is good (if people can think there is no commercial reasons to five star a review!)

Brown envelope in the post please what hi fi. Lol.

 

[newlash09]

Buy blind like me. And almost 90% of my kit has been bought based on 5 star reviews here. And I haven't been disappointed with any of them. Since I haven't done comparative listening like the most of you. I continue to be happy with my stuff. Ignorance can be bliss. But Iam still thankfull for the present review system, that has never let me down. And continues to help millions around the world like me .

From my perception of their star system, 5 star product is one which might not blow away everyone, but will not disappoint anyone . Basically is an all rounder.

4 star can suit some better than a 5 star product. But not for everyone. Some might be disappointed, unlike the 5 star product that will offend none.

So I always go for 5 stars. And never been disappointed. Cheers

 

[steve_1979]

...our technical editor, Ketan (who has a mechanical engineering BSc and used to work at Hi-Fi World, seeing as some people asked!).

That's good to know. Thanks.

Any more feedback, gratefully received.

All things said and done I do still thoroughly enjoy reading your reveiws so you guys must be doing something right.

 

[Blacksabbath25]

I do not follow the 5 star ratings in what hifi magazine I used to when I was younger but not now as I've found out the only real way of finding the perfect hifi setup is going for a demo .

the 5 star ratings are only a guide and shouldn't make anyone buy blind because the product has a 5 star rating we also live in a modern world and we have far more information at our hands with a press of a button ( THE INTERNET ) and magazines then demo then you make your ( CHOICES ! )

its all so very important to trust in what you like personally regardless of cost weather it's cost you £10 or £10.000 it's your choice at the end of the day no one else's but yours .

what hifi 5 star ratings are the reviewers opinions only I no lots of stuff they have reviewed and they give 3 or 4 star ratings but when I've had a demo on some of those products I've really liked them and can't understand why they got 3 or 4 star rating but that's only my opinion and that's all it is .

but what troubles me about the 5 star ratings is what does it do for the sales of a product that only got 3 stars or the company that made that product ? Because if it's a small company it could make them go out of business because of a 3 star rating .

none of the hifi equipment I own has a 5 star rating I brought my equipment because my ears likd it and nothing else .

 

[steve_1979]

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

+1 on both points.

Nothing too in-depth re the technical infomation as that would be a bit boring. But basic information such as speakers frequency range including the all important +/-dB rating would be useful and easy enough for Mr Average Joe public to understand.

 

[CnoEvil]

I do not follow the 5 star ratings in what hifi magazine I used to when I was younger but not now as I've found out the only real way of finding the perfect hifi setup is going for a demo .

the 5 star ratings are only a guide and shouldn't make anyone buy blind because the product has a 5 star rating we also live in a modern world and we have far more information at our hands with a press of a button ( THE INTERNET ) and magazines then demo then you make your ( CHOICES ! )

its all so very important to trust in what you like personally regardless of cost weather it's cost you £10 or £10.000 it's your choice at the end of the day no one else's but yours .

what hifi 5 star ratings are the reviewers opinions only I no lots of stuff they have reviewed and they give 3 or 4 star ratings but when I've had a demo on some of those products I've really liked them and can't understand why they got 3 or 4 star rating but that's only my opinion and that's all it is .

but what troubles me about the 5 star ratings is what does it do for the sales of a product that only got 3 stars or the company that made that product ? Because if it's a small company it could make them go out of business because of a 3 star rating .

none of the hifi equipment I own has a 5 star rating I brought my equipment because my ears likd it and nothing else .

My amp was Hi Fi Worlds top Integrated Amp, awarded in Jan 2011......after I had bought it. *music2* *yahoo*

http://www.bm.rs/Musical%20Fidelity/Musical%20Fidelity%20AMS%2035i%20-%20Award%20HiFi%20World%20January%202011.pdf

Also made it into Stereophile's top Amp group.

Just sayin'. *angel*

 

[jjbomber]

Brown envelope in the post please what hi fi. Lol.

They may have a few of them in the office this week. Good timing.

 

[davedotco]

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

+1 on both points.

Nothing too in-depth re the technical infomation as that would be a bit boring. But basic information such as speakers frequency range including the all important +/-dB rating would be useful and easy enough for Mr Average Joe public to understand.

No chance.

Measuring a speakers frequency response is not a trivial undertaking, if that is not done, I fail to see the relevance of quoting such things.

Blind testing is never going to happen either, this would totally mess with the whole hi-fi 'group think' and would be very distructive of the way the industry works. Even simple level matching of say, amplifiers or dacs would be a step too far as the established hierarchy comes crashing down as a result of the findings.

 

[QuestForThe13thNote]

The problem blacksabbath is that dealers only sell one of a number of brands which they have allegiances too. I think it would be quite possible to go to a few dealers and buy a product based on what they have that doesn't stand up to what is best amongst those best products out in the market, to individual taste.. The dealer product may be that best product though! Following this is where reviews come in. Not relying on one mag but many, and also many consumers opinions demoing between what a dealer may have and these five star rated product a dealer may or may not have. So if you read people consistently saying that brand x, which a dealer has, is not as good as brand y, which is a five star review and not in the dealer, I'd ask why do these people think that, what does the review say and is the review credible and related to what people say. Where can I hear that 5 star product. It may be that you decide the 5 star isn't the best, but in the process you've found the best within time you have.

if you hear the five star product and it competes best for you with say 5 other products at a similar price, you know it's a credible review. This takes time rather than the 'I'll go to the dealer and choose from a few products' approach. I know loads of people who buy like this, but I think if I'm spending lots I'll take my time and I've probably been to 5 or 6 dealers before I choose on speakers or fundamental parts of the system. So I know I've got the best for my cash and my tastes. I generally find mags are consistent on speakers, less so on electronics, so my speakers are always 5 stars (not that I've chosen on that, but best possible sq) .

 

[steve_1979]

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

+1 on both points.

Nothing too in-depth re the technical infomation as that would be a bit boring. But basic information such as speakers frequency range including the all important +/-dB rating would be useful and easy enough for Mr Average Joe public to understand.

No chance.

Measuring a speakers frequency response is not a trivial undertaking, if that is not done, I fail to see the relevance of quoting such things.

Blind testing is never going to happen either, this would totally mess with the whole hi-fi 'group think' and would be very distructive of the way the industry works. Even simple level matching of say, amplifiers or dacs would be a step too far as the established hierarchy comes crashing down as a result of the findings.

Yeah I know. Doesn't stop me from asking ever now and then though.

 

[QuestForThe13thNote]

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

+1 on both points.

Nothing too in-depth re the technical infomation as that would be a bit boring. But basic information such as speakers frequency range including the all important +/-dB rating would be useful and easy enough for Mr Average Joe public to understand.

No chance.

Measuring a speakers frequency response is not a trivial undertaking, if that is not done, I fail to see the relevance of quoting such things.

Blind testing is never going to happen either, this would totally mess with the whole hi-fi 'group think' and would be very distructive of the way the industry works. Even simple level matching of say, amplifiers or dacs would be a step too far as the established hierarchy comes crashing down as a result of the findings.

It would be destructive if you had reviewers second guessing on what speakers were from blind tests, hedging their bets, and reviews coming out wrong for this. It would be stupid and would probably mean dealer sales would fall through the roof. If you believe in this davedotco, i can't believe you have thought this out properly. Has it ever occurred to you there is more than one 'group think' because products get rated and reviewed differently depending on what mags you read, from what countries etc. Also the plethora of customer opinion. It just has zero credibility.

 

[ellisdj]

Did a WHF Blind test which proved isolation matters to things that are not microphonic - yet people still dispute that is does.. thats why they dont bother

 

[davedotco]

I'd like more in-depth technical information on the insides and also blind testing between products to eliminate expectation bias

+1 on both points.

Nothing too in-depth re the technical infomation as that would be a bit boring. But basic information such as speakers frequency range including the all important +/-dB rating would be useful and easy enough for Mr Average Joe public to understand.

No chance.

Measuring a speakers frequency response is not a trivial undertaking, if that is not done, I fail to see the relevance of quoting such things.

Blind testing is never going to happen either, this would totally mess with the whole hi-fi 'group think' and would be very distructive of the way the industry works. Even simple level matching of say, amplifiers or dacs would be a step too far as the established hierarchy comes crashing down as a result of the findings.

It would be destructive if you had reviewers second guessing on what speakers were from blind tests, hedging their bets, and reviews coming out wrong for this. It would be stupid and would probably mean dealer sales would fall through the roof. If you believe in this davedotco, i can't believe you have thought this out properly. Has it ever occurred to you there is more than one 'group think' because products get rated and reviewed differently depending on what mags you read, from what countries etc. Also the plethora of customer opinion. It just has zero credibility.

You still don't get it, do you.

There are plenty of shades of opinion within the industry but everyone conforms to the basics of subjective evaluation and constant improvements that promotes the industry across all outlets, be it media or retail.

As a supplier or retailer you need to play the game to be successful, the crazy thing being that as part of the 'group' you do not even realise that you are doing so, you simply become part of 'zeitgeist'.

As a magazine you are not going to sell many copies if you say that this months new releases are no better than those you wrote about last month and as a dealer you rapidly learn that the first question from most customers is "what's new", very rarely do you get "what's good".

 

[davedotco]

Did a WHF Blind test which proved isolation matters to things that are not microphonic - yet people still dispute that is does.. thats why they dont bother

We used to do such tests at hi-fi shows back in the day and sold a lot of tables on the back of it.

Sadly it is a pretty un-sexy topic these days, dac chips for example are far sexier even though the differences are minute in many cases.

 

[QuestForThe13thNote]

Im not sure you get it . I think almost everyone thinks what's best for the cash, a basic tenet of economics to get your money's worth for your hard earned.

 

[davedotco]

Im not sure you get it . I think almost everyone thinks what's best for the cash, a basic tenet of economics to get your money's worth for your hard earned.

There is a saying about knowing the cost of everything and the value of nothing.....

 

[QuestForThe13thNote]

You sound like a really aggrieved disgruntled ex hi fi shop owner, although I symphathise if it was a tough market to be in.

 

[SemiChronic]

Found this link to review or Oppo Sonica which includes a mention of the Benchmark 2Dac,

Dont know if youve already seen it, might be an interesting read for you.

http://www.goodsound.com/index.php/equipment-reviews/820-oppo-digital-sonica-digital-to-analog-converter

 

[SemiChronic]

Found this link to review or Oppo Sonica which includes a mention of the Benchmark 2Dac,

Dont know if youve already seen it, might be an interesting read for you.

http://www.goodsound.com/index.php/equipment-reviews/820-oppo-digital-sonica-digital-to-analog-converter

 

[SemiChronic]

Forgot to add, Sonica was tested with a Hegel360, which might be of particular interest to you (Thats if you havent already sold your soul to actives)