What do you read into WHFSAV star review system?

Page 2 - Seeking answers? Join the What HiFi community: the world's leading independent guide to buying and owning hi-fi and home entertainment products.
Q

QuestForThe13thNote

Guest
The thing is you aren't biased based on the review. It just depends if you accept it but that pre supposes you haven't tested other hi fi products yourself and made own mind up ie what hi fi have got you hook line and sinker. But I suspect they haven't.
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
It is worth noting that the reviewers are, in the main, young men with no technical training and as far as I know have no practical training in how to listen.

They work as a group with members joining and leaving on a regular basis, though I think the 'groupthink' changes very slowly, if at all. Direct contact from equipment manufacturers/suppliers is not allowed but they are part of 'the industry' and their views and opinions are very obviously formed by that. The result is very definitey a bias towards the mainstream, both in terms of equipment and also attitudes.

WHF reviews are fairly predictable though, as indeed are the star ratings which are of course a huge marketing tool both for the magazine and the industry as a whole.

On a different note, WHF's listening methods tend to favour equipment that is on the 'bright' or 'forthright' side of neutral, not universal but quite noticeable over a period. This may help explain why a lot of people buying recommended product end up on forums like this complaining about 'brightness' or a lack of 'warmth'.
 
Q

QuestForThe13thNote

Guest
Ive never heard someone say that you need training on how to listen. Being able to discern differences in hi fi just comes from experience of having done it accross budget, midrange, and premium stuff, whether youve owned it or not. Thats All. Assuming your ears are good too.

Of course reviewers views will be formed by what manufacturers say of their products. And of course there is a bias to mainstream, mass market, or well known but non mass market audiophile stuff from likes of naim, PMC etc. If there wasnt then how would what hi fi generate advertising revenues as a business. If they were reviewing a hi fi amp of a type that doesnt sell much, but it is equally as good as a more well known offering, the magazine wouldnt generate necessary advertising revenue.

I agree they do tend to go for detail at the budget end, but for mid range and better, some of the reviews are right.
 

ellisdj

New member
Dec 11, 2008
377
1
0
Visit site
You definately can be trained as a listener but you dont need to be a technical engineering expert to be so - why would you need to be, they are completely different skill sets.
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
ellisdj said:
You definately can be trained as a listener but you dont need to be a technical engineering expert to be so - why would you need to be, they are completely different skill sets.

Harman are one of the few companies that publisize the way that they train their developement personel, going to the trouble to release 'training' packages for the enthusiast to try at home, 'How to listen' being the most well known.

Of course there are some golden eared individuals whose ability to circumvent psycological biases allows them to pass down to us mortals evaluations that are virtually 'ex cathedra', so no training required here.
 
Q

QuestForThe13thNote

Guest
When you go from talking about a scoring system and one product performs sonically better than another, its weird how it goes straight onto you saying Daved that some peoples listening is psychological. We are talking about something that performs well or not, out of any psycholigical sphere, surely?. For someone who has worked in hi-fi like you say you have, its very strange to me that you hold the attitude you do. You must have come accross stuff that performs widly better at the same price points. No? I have and Ive never even worked in the industry. But you seem to say most stuff is the same, is the message I get from you.
 

avole

New member
Jul 15, 2016
17
0
0
Visit site
steve_1979 said:
Al ears said:
What do you read into WHFSAV star review system?

My logical brain tells me that it's mostly meaningless. Nothing more than an inaccurate represention of someone with no scientific quilifications or background giving a subjective opion on a very oversimplified 1 to 5 scale. This has the potential to be made even more innacurate by the fact that the the magazine requires advertising income from hifi companies to stay in profit and could easily be tempted to add or remove stars.

However I'm ashamed to say that my emotional brain sees 5 starts and immediately thinks "That must be good" or it sees 4 stars and thinks "That must be rubbish".

I am, alas, a stupid emotional idiot.
Right on two counts. Apart from the lack of any real measurements, there's also the fact WHF does not do blind testing. Therefore, even if two items sound exactly the same, sight =bias, whether it be from the brand, the opinions of others, how the thing looks etc.

The stars are meaningless.
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
You really do not understand do you?

My views in this matter are simple.

Sighted, subjective testing is no way to evaluate equipment such that this evaluation is of value to another person. Thats it.

You disagree with that basic premise, I no longer feel obligated to attempt to cure you of what I consider to be your illusions.
 
S

SemiChronic

Guest
Andrewjvt, im curious as to what your system is these days.

Re: the Sonica, if i was looking for dac, i doubt i would have auditioned it on the strength of the WhatHifi review.
 

CnoEvil

New member
Aug 21, 2009
556
13
0
Visit site
davedotco said:
Sighted, subjective testing is no way to evaluate equipment such that this evaluation is of value to another person. Thats it.

I can certainly understand that POV....but personally I wouldn't go quite that far, as I think that a review has its place, as being part of the research process, provided:

- It is not taken as gospel, but used as one tool for creating a shortlist.

- You learn how to read between the lines; can home in on the described characteristics and take the subjective assessment with a pinch of salt.

- The reviewer can describe accurately what he hears.

- You can, if possible, pick a reviewer whose taste aligns as closely as possible with your own.

I agree with what you said about WHF liking a more forward detailed sound....which may be due to their highly treated listening rooms.
 

Native_bon

Well-known member
Nov 26, 2008
180
4
18,595
Visit site
davedotco said:
It is worth noting that the reviewers are, in the main, young men with no technical training and as far as I know have no practical training in how to listen.

They work as a group with members joining and leaving on a regular basis, though I think the 'groupthink' changes very slowly, if at all. Direct contact from equipment manufacturers/suppliers is not allowed but they are part of 'the industry' and their views and opinions are very obviously formed by that. The result is very definitey a bias towards the mainstream, both in terms of equipment and also attitudes.

WHF reviews are fairly predictable though, as indeed are the star ratings which are of course a huge marketing tool both for the magazine and the industry as a whole.

On a different note, WHF's listening methods tend to favour equipment that is on the 'bright' or 'forthright' side of neutral, not universal but quite noticeable over a period. This may help explain why a lot of people buying recommended product end up on forums like this complaining about 'brightness' or a lack of 'warmth'.
wonder why it took so long for the obvious to be pointed out. Nail on the head.
 
Q

QuestForThe13thNote

Guest
Couldn't be more wrong. Presumably on that basis it would be testing objectively using a multimeter etc that nobody would then know none the wiser, how it sounds. I don't think you can improve on current review system. Seems very ill thought out to me, around commercial realities as in my post above.
 

Andrewjvt

New member
Jun 18, 2014
99
4
0
Visit site
SemiChronic said:
Andrewjvt, im curious as to what your system is these days. 

Re: the Sonica, if i was looking for dac, i doubt i would have auditioned it on the strength of the WhatHifi review.

My system is a work in progress but my pride and joy for the minute is my benchmark dac3 hgc combined with some cheap but very effective JBL studio monitors (temporary for now)
The reason I'm interested in the sonica is because it uses the new pro sabre dac chip but is much cheaper than my benchmark dac3 hgc.

The benchmark uses the pro es9028 chip but the sonica uses the pro es9038

So that's why I'm interested in a comparison between the 2.
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
Subjective audio testing/reviewing as it is currently practiced is largely a work of imagination. It is often fun, occasionally informative but as a guide for establishing the relative merits of hi-fi equipment it is largely worthless.

Some reviews are helpful, I find 'long term' reviews quite informative and the more in depth articles that involve setup and system matching can be helpful but the 'crash bang' style of WHF and others is pretty much useless.

Cno, I know it is rare for you to be argumentative so this is said with the utmost respect, but I suggest that you think again about what you said, the idea that a bunch of young journalists (of modest ability) can, when listening to product on a production line basis, with no technical or other checks and balances offer a view that is of real value is quite absurd.
 

CnoEvil

New member
Aug 21, 2009
556
13
0
Visit site
davedotco said:
Cno, I know it is rare for you to be argumentative so this is said with the utmost respect, but I suggest that you think again about what you said, the idea that a bunch of young journalists (of modest ability) can, when listening to product on a production line basis, with no technical or other checks and balances offer a view that is of real value is quite absurd.

I have been around this game almost as long as you have and have (personally) found reviews to be useful....but always use my own judgement to come to a conclusion. The reviews I take seriously are generally not from WHF....but even they get it right occasionally. *biggrin*
 
Tend not to read much into any mag or internet review. As regards WHFI, in the main, my findings tend to chime with theirs. That said, there have been a few components over the years where I've thought: "Really? don't recognise that trait or this trait..."

For instance, they only gave the Usher S520 3 stars, but when I dem'd them a few years ago with an Arcam A18, they sounded top drawer. In fact I would still say they are my favourite sub £500 standmounter.

More importantly, though, if it wasn't for this forum and the WHFI mag, I wonder what systems we would have?
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
CnoEvil said:
davedotco said:
Cno, I know it is rare for you to be argumentative so this is said with the utmost respect, but I suggest that you think again about what you said, the idea that a bunch of young journalists (of modest ability) can, when listening to product on a production line basis, with no technical or other checks and balances offer a view that is of real value is quite absurd.

I have been around this game almost as long as you have and have (personally) found reviews to be useful....but always use my own judgement to come to a conclusion. The reviews I take seriously are generally not from WHF....but even they get it right occasionally. *biggrin*

I do not get to play with hi-fi that much these days, so it needs to be understood that this refers to a period about 10 (or more) years ago.

At that time I worked with equipment virtually every day, it was our way to sell equipment by demonstration, and customers would very often bring in their own equipment for comparitive demonstrations.

The number of 5* and other highly regarded products that simply failed to 'cut the mustard' was huge, the demonstrations were easy to do and the results sadly predictable. Naturally this gave me a somewhat sceptical view on the industry and the magazines that are an integral part of it.

Sure there are some well written and informative reviews in 'other' magazines and though often entertaining to read I feel that they have little if any value when it comes to evaluating equipment for my own use.
 
Dave and Cno are two of my favourite contributors here, and ones I often agree with or learn from.

On the point about the youthful folk at WHF, I was that age when I worked on Saturdays in a Hi-Fi store. I suppose I had pretty strong opinions back then, and they were formed from regular exposure to established favourites and new models. And I think, to be fair, that's how the WHF writers work. I agree it's bereft of science or rigour, but then most domestic listening is. There are some older people there too!

We all seem to agree that the star system is bit arbitrary, but it isn't really any different to record, TV, film or concert reviews in the newspaper. Five stars is the most familiar scale, even though a 'better way' might be possible. I think it's here to stay!
 
Q

QuestForThe13thNote

Guest
CnoEvil said:
davedotco said:
Cno, I know it is rare for you to be argumentative so this is said with the utmost respect, but I suggest that you think again about what you said, the idea that a bunch of young journalists (of modest ability) can, when listening to product on a production line basis, with no technical or other checks and balances offer a view that is of real value is quite absurd.

I have been around this game almost as long as you have and have (personally) found reviews to be useful....but always use my own judgement to come to a conclusion. The reviews I take seriously are generally not from WHF....but even they get it right occasionally. *biggrin*

this kind of talk that journalists can't review with a rudimentary technical knowledge gives hi fi a bad name. Hi fi snobbery. It kind of means that I or any other bod, who is not a hi fi specialist with huge technical knowledge, cannot come to a view on products. This is obviously not true. And who is to say journos don't have technical knowledge. I suspect as they are doing it as a job they have to have it more than most to be credible and for job retaining purposes.

anyone can do these reviews if they are reviewing sound quality and have prior experience of other products, as well as other features. For what Daved often talks sense, I think this is way far of common sense. Hi fi isn't exclusive, music which hi fi does, is all inclusive. And both can be gauged by all.
 
The bottom line is: Any professional review is ONLY designed as a guide. Having visited WHFI Towers, they have had designed acoustically treated demo rooms for hi-fi, AV/TV and a small room for testing things such as the SQ of iPad, phones etc etc. I would guess they dem products (various formats) for around 4-5 hours most days.

Where these reviews can fall flat is we don't live in specially treated acoustic living room. Some people have wooden and/or suspended flooring with minimal furnishings, while others, such as myself, have carpets, thick sofa, heavy duty curtains.... others will have something inbetween - a bit of both.

Hi-fis needs to be tailored to the respective room and not to a certain review. IMHO this is where the disparity kicks in between the professional review and real world listening.
 

ellisdj

New member
Dec 11, 2008
377
1
0
Visit site
Years ago I bought a Cyrus CD6S based on the 5 star what hifi review - I thought it was ok but I wasnt getting a sound I was happy with - foolishly I blamed the CD player and moved it on.

Later I heard the exact same CD player at WHF towers and could hear straight away why they gave it 5 stars as a review.

In a way the 5 star setup makes sense more to me now than before - some products really are great and you just really love them and they win 5 stars. Others are still good but dont quite make you love them 4 stars. Maybe thats not how they do it but that makes more sense to me now
 

davedotco

New member
Apr 24, 2013
20
1
0
Visit site
QuestForThe13thNote said:
CnoEvil said:
davedotco said:
Cno, I know it is rare for you to be argumentative so this is said with the utmost respect, but I suggest that you think again about what you said, the idea that a bunch of young journalists (of modest ability) can, when listening to product on a production line basis, with no technical or other checks and balances offer a view that is of real value is quite absurd.

I have been around this game almost as long as you have and have (personally) found reviews to be useful....but always use my own judgement to come to a conclusion. The reviews I take seriously are generally not from WHF....but even they get it right occasionally. *biggrin*

this kind of talk that journalists can't review with a rudimentary technical knowledge gives hi fi a bad name. Hi fi snobbery. It kind of means that I or any other bod, who is not a hi fi specialist with huge technical knowledge, cannot come to a view on products. This is obviously not true. And who is to say journos don't have technical knowledge. I suspect as they are doing it as a job they have to have it more than most to be credible and for job retaining purposes.

anyone can do these reviews if they are reviewing sound quality and have prior experience of other products, as well as other features. For what Daved often talks sense, I think this is way far of common sense. Hi fi isn't exclusive, music which hi fi does, is all inclusive. And both can be gauged by all.

Once again your basic lack of understanding is breathtaking.

I have never suggested that ordinary enthusiasts "cannot come to a view on products", anyone who has an interest can listen and come to a conclusion that is meaningful for them. Everyone is encouraged to listen for themselves and choose equipment that works for them, this is a fundamental tenet of this forum.

What they can not due is to extend their conclusions to other people who will not share their experience, preferences and prejudices. This simply does not and can not work.

For a review to be valuable for a third party it needs to have been produced according to a set of standards that are known, these may be technical, procedural or what have yo, this gives the third party reader a basis to relate the reviewers findings to his own requirements.
 

TRENDING THREADS

Latest posts