What do you read into WHFSAV star review system?

I have noticed something of a trend whilst reading the reviews of late from WHF, particularly when it comes to digital devices.

In the past I have found that many of their reviews have been pretty similar to mine when I have auditioned kit but these days their findings are becoming very much at odds to my own particularly in the field of digital equipment and DACs in particular.

Most of their speaker reviews in the past, at least the ones I have actually been lucky enough to audition have, to me, seemed quite accurate.

I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....
 
D

Deleted member 2457

Guest
Al ears said:
I have noticed something of a trend whilst reading the reviews of late from WHF, particularly when it comes to digital devices.

In the past I have found that many of their reviews have been pretty similar to mine when I have auditioned kit but these days their findings are becoming very much at odds to my own particularly in the field of digital equipment and DACs in particular.

Most of their speaker reviews in the past, at least the ones I have actually been lucky enough to audition have, to me, seemed quite accurate.

I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....
I think it's just What HiFi now, not the sound and vision?
 

insider9

Well-known member
Quite agreed Al it's been coming. The star rating system is long overdue a revamp. With general public better informed than ever current star rating simply doesn't have a right to carry on for much longer until WHF credibility is damaged.
 

chebby

Well-known member
Jun 2, 2008
1,253
26
19,220
Visit site
I don't read the reviews here any more TBH.

The backgrounds of the reviewers were journalism, business, politics, hospitality etc. when I last looked, with no technical/scientific/engineering disciplines represented (except for Ketan I think).

I have a small collection of 30+ years old magazines and many of the articles/reviews are a good afternoon's read with informed comment on technical issues included. I'd go as far as to say much of it was (still is) educational in a way that modern reviewers/contributors just can't hack. There were one or two publications - back in the day - that were a bit 'out there'/'off with the faeries' (anything by Jimmy Hughes especially) and/or Linn-Naim 'house' publications but, overall, i'd prefer to see some more decently crafted writing rather than the now - seemingly ubiquitous - cut 'n paste, bite-size gobbets served up by marketing departments (advertorials).

This site's quality of content started it's impression of a stricken submarine around the time Andrew Everard left. Self-admittedly not a 'technical' person he still managed to write some pretty good 'how to' guides and entertain us too. There was always a story and plenty of wit. He seemed to go the extra mile.

The following 'crie de coeur' seems to sum it up about his care for the witten word and the integrity of his writing even as applied to articles in consumer electronics magazines ...

https://andreweverard.com/2014/08/22/is-no-one-reading-the-internet-any-more/

I can't see him cut and pasting a manufacturer's marketing department's effluvia as a news item or an article.
 
gel said:
Al ears said:
I have noticed something of a trend whilst reading the reviews of late from WHF, particularly when it comes to digital devices.

In the past I have found that many of their reviews have been pretty similar to mine when I have auditioned kit but these days their findings are becoming very much at odds to my own particularly in the field of digital equipment and DACs in particular.

Most of their speaker reviews in the past, at least the ones I have actually been lucky enough to audition have, to me, seemed quite accurate.

I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....
I think it's just What HiFi now, not the sound and vision?

This purely demonstrates I am a long-termer who doesn't read the front cover..... why the lost the SAV bit is beyond me as that's what most of the magazine is composed of these days. ;-)
 
insider9 said:
Quite agreed Al it's been coming. The star rating system is long overdue a revamp. With general public better informed than ever current star rating simply doesn't have a right to carry on for much longer until WHF credibility is damaged.

I am not so at odds with the use of a star system per se its just their total divergence from the findings of other reviewers.
 
chebby said:
I don't read the reviews here any more TBH.

The backgrounds of the reviewers were journalism, business, politics, hospitality etc. when I last looked, with no technical/scientific/engineering disciplines represented (except for Ketan I think).

I have a small collection of 30+ years old magazines and many of the articles/reviews are a good afternoon's read with informed comment on technical issues included. I'd go as far as to say much of it was (still is) educational in a way that modern reviewers/contributors just can't hack. There were one or two publications - back in the day - that were a bit 'out there'/'off with the faeries' (anything by Jimmy Hughes especially) and/or Linn-Naim 'house' publications but, overall, i'd prefer to see some more decently crafted writing rather than the now - seemingly ubiquitous - cut 'n paste, bite-size gobbets served up by marketing departments (advertorials).

This site's quality of content started it's impression of a stricken submarine around the time Andrew Everard left. Self-admittedly not a 'technical' person he still managed to write some pretty good 'how to' guides and entertain us too. There was always a story and plenty of wit. He seemed to go the extra mile.

The following 'crie de coeur' seems to sum it up about his care for the witten word and the integrity of his writing even as applied to articles in consumer electronics magazines ...

https://andreweverard.com/2014/08/22/is-no-one-reading-the-internet-any-more/

I can't see him cut and pasting a manufacturer's marketing department's effluvia as a news item or an article.

I quite agree. Some of the most interesting observations and critique that I have read have come from the 'pen' of Mr. Everard and I still look forward to reading his latest 'words of wisdom'.
 
D

Deleted member 2457

Guest
Al ears said:
gel said:
Al ears said:
I have noticed something of a trend whilst reading the reviews of late from WHF, particularly when it comes to digital devices.

In the past I have found that many of their reviews have been pretty similar to mine when I have auditioned kit but these days their findings are becoming very much at odds to my own particularly in the field of digital equipment and DACs in particular.

Most of their speaker reviews in the past, at least the ones I have actually been lucky enough to audition have, to me, seemed quite accurate.

I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....
I think it's just What HiFi now, not the sound and vision?

This purely demonstrates I am a long-termer who doesn't read the front cover..... why the lost the SAV bit is beyond me as that's what most of the magazine is composed of these days. ;-)
I think I can remember WHF saying they want to go back to their roots.
 

jjbomber

Well-known member
Al ears said:
I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....

The same happened with the original Oppo DVD player, when WHF gave it 3 stars and everyone else gave it 5 stars. WHF soon fell into line and gave it 5 stars from then on.
 
jjbomber said:
Al ears said:
I was most interested to read their recent review of the Oppo Sonica DAC as I own an Oppo Universal player that may soon be relegated to my home cinema set-up. The review appears to pretty much pan this device and gives it three stars. Amazing I thought as just about ever other major publication including the likes of Mr. Andrew Everards' (who's opinions I value) simply rave about what a wonderful sounding multi-functional piece of equipment it is....

The same happened with the original Oppo DVD player, when WHF gave it 3 stars and everyone else gave it 5 stars. WHF soon fell into line and gave it 5 stars from then on.

Funny that. You sometimes wonder just who they have listening to these things.

I have just read their latest review of the Rega RP6 turntable which basically translates as 'it's really great but only if you connect it to a top dollar system' and the Ania cartridge which is brilliant ... as long as it's attached to a top end phono preamp , if you don't ithen its not... ;-)
 
Q

QuestForThe13thNote

Guest
Might that be a legitimate thing they found on testing with the rega?
 
Q

QuestForThe13thNote

Guest
I think any comparison with the audiolab m dac plus is a bit unfair in my point above on reflection, but the point is the same with similar priced stuff.

but what hi fi seem to be comparing the oppo sonica dac with the Cambridge audio cxn as a streamer come dac, as is the oppo. With the oppo at £800 and ca at £700. So that's the listening comparison to make in what hi fi comparative ratings of one against the other, as well as all the other stuff the products do too. Maybe the dual dacs of the Cambridge audio help it but I think the sabre dacs are very well regarded, so an interesting comparison.
 
Q

QuestForThe13thNote

Guest
I think the point of a rating system is always someone's view and it could be just the person who does the review, although I suspect what hi fi do have more than one person make the decisions. So you will never get a rating system right.

To you, you might think the oppos sonica dac will sound superb but just as a question to you, would you ever try or hear the audiolab m dac plus against the Oppo. This model what hi fi rate as 5 star. Assuming it met the same requirements for you and for arguments sake it is better sounding, and has more facilities etc over the oppo, then you can understand the star rating system has to be comparative. If it doesn't I'd agree it could be a wrong review with a caveat as below on system matching. If a product is reviewed and it beats the currently available competition then it gets 5 stars, if a model comes along and isn't as good then it has to be rated on performance against that best product of the same price.

a lot of people say I don't believe what hi fi because their product performs well to them even though not rated 5 stars, but I doubt they have made all the comparisons like what hi fi have, who do this as a job. The buyer doesn't have time to do it understandably.

also it has to be born in mind what you are actually testing the item in a system with. If the system will reveal the sonic benefits of a dac over another, then it's worthwhile having the better dac at the same price. If the system isn't so revealing, it doesn't matter. Just buy on functions and look etc. So I suspect this accounts a lot for why people don't get what hi fi ratings. But I think what hi fi should test the kit in similar systems and prices the dacs are intended to be used with. I'm not sure they do? But if they do and the system is the same for each test, and it's a decent system, what hi fi are very much in a position to say one dac is better than the other at the same price, certainly on sq, if the case, and therefore use comparative relative star ratings.

I don't tend to go on what hi fi alone. If they big up a product and all other mags do, then I'd be a fool to ignore this. But also important is what dealers sell most of, and what owners say about products once they have bought, and what they say about trialing two products against each other. Also in this context, the system they have, so you can take a view as to whether they (and you in deciding to buy) will or will not get the benefits of the dac purchase. For instance if they've budget speakers but are expecting to be able to make a comparison between the two pricey dacs, and that their speakers would not reveal these differences, and they say this particular dac is just as good, one can understand why.
 
S

SemiChronic

Guest
Andrew Everard's review of the Cambridge 851 series definitely helped me decide.
 

lindsayt

New member
Apr 8, 2011
16
2
0
Visit site
I read the September 2017 What Hi-fi review of the SME 15a yesterday. I thought that it was a good review that summed up the strong and weak points of these upper end SME turntables.

The 4 star rating of the SME 15a is fair enough. If you value detail retrieval above everything else, 5 stars might be appropriate. If you value dynamics and emotional involvement over everything else you might give it 3 or fewer stars. Therefore 4 stars is about right, with a recommendation to read the text in the review and if still interested, listen to it for yourself in a comparative demo and make your own mind up.

As for reviews of other products not agreeing with personal listening tests - well there are several possible reasons for that. The most innocent ones being:

1 different reference components used in different comparative demos resulting in different strengths and weaknesses being highlighted.

2 different synergy. EG power amp not happy with low impedance speakers in one system whilst sounding great with easy load speakers in another.

We can all have our own personal rating systems. Mine is much stricter than What Hi-fi's.

And it's a badge of honour if most or all of your system has never received any stars from What Hi-fi because it's never been reviewed by them.
 
I'm not sure whether seeing an otherwise well regarded product get 3 stars from WHF reassures me of their independent thinking, or makes me wonder what they are listening for.

They themselves define 3 stars as "Worth a look" while 5 stars means "One of the best".

My recollections from a day's visit to the previous WHF HQ and listening facilities were the emphasis on getting a collegiate view, never writing a review without others' input. And I think they always listen on both a reference system and a budget appropriate one.

Re the Rega Planar 6, I can imagine that if it is quite a subtle performer then in a budget system the Planar 2 or 3 might be a better match for a much lower cost. But for the Oppo DAC/streamer it is less obvious how it failed to impress - your point, Al, about digital components. The system synergy might be relevant there, I guess. Or maybe the unit supplied was below par (though with some items the same piece goes around many critics). Perhaps there are digital "funnies' we don't yet fully understand or have labels for?

Personally, I'd like an outright ranking system, though I realise that would be much more challenging. The scale would be pretty irrelevant but if a high performing and costly speaker, say, rated 120, a budget pair be might be 40. An outstanding budget pair might reach 45. Then a new, superior highend model might achieve 125. And so on. The current "performance and price" rating is far easier, as it relies on peer comparisons, and a price drop can equal an extra star.

The bottom line is surely that no single review should hold too much weight, and many items are quite system dependent for optimum performance - which is where a dealer can help so much.
 

Andrewjvt

New member
Jun 18, 2014
99
4
0
Visit site
nopiano said:
I'm not sure whether seeing an otherwise well regarded product get 3 stars from WHF reassures me of their independent thinking, or makes me wonder what they are listening for.  

They themselves define 3 stars as "Worth a look" while 5 stars means "One of the best".  

My recollections from a day's visit to the previous WHF HQ and listening facilities were the emphasis on getting a collegiate view, never writing a review without others' input.  And I think they always listen on both a reference system and a budget appropriate one. 

Re the Rega Planar 6, I can imagine that if it is quite a subtle performer then in a budget system the Planar 2 or 3 might be a better match for a much lower cost. But for the Oppo DAC/streamer it is less obvious how it failed to impress - your point, Al, about digital components.  The system synergy might be relevant there, I guess. Or maybe the unit supplied was below par (though with some items the same piece goes around many critics).  Perhaps there are digital "funnies' we don't yet fully understand or have labels for?

Personally, I'd like an outright ranking system, though I realise that would be much more challenging.  The scale would be pretty irrelevant but if a high performing and costly speaker, say, rated 120, a budget pair be might be 40.  An outstanding budget pair might reach 45.   Then a new, superior highend model might achieve 125.  And so on.  The current "performance and price" rating is far easier, as it relies on peer comparisons, and a price drop can equal an extra star.   

The bottom line is surely that no single review should hold too much weight, and many items are quite system dependent for optimum performance - which is where a dealer can help so much.  

I'll be soon testing the sonica dac v my benchmark dac3 to see if I can hear a difference
 
Q

QuestForThe13thNote

Guest
It's not meaningless because its opinion. It's to take from it what you will. So If there is an item on there that is very good and you'd agree with it, as there will be chances are across many products, you agree with the opinion of the reviewer as to its rating.

And as an opinion it will be subjective and a review can't be anything other, with the impossibility of testing sound quality, and weighing up the variables that makes one good functionally wise and another not so being impossible to score rate against the other or too complicated to do so.

and if I was reviewing stuff I'd have a five star rating system. Not 100 percent or 10 stars. All you really need to know is it in opinion of reviewer, best at what it is, near the best or below average, and the star rating can achieve that.

I don't think what hi fi have any allegiances and are independent to the extent they can say what they want. They are journalists with freedom of speech after all. You usually find the people who advertise have good products, so they can advertise free of fear of concern about a bad review. At the end of the day, advertising is advertising and serves a purpose, not just the reviews of the products.
 

steve_1979

Well-known member
Jul 14, 2010
231
10
18,795
Visit site
Al ears said:
What do you read into WHFSAV star review system?

My logical brain tells me that it's mostly meaningless. Nothing more than an inaccurate represention of someone with no scientific quilifications or background giving a subjective opion on a very oversimplified 1 to 5 scale. This has the potential to be made even more innacurate by the fact that the the magazine requires advertising income from hifi companies to stay in profit and could easily be tempted to add or remove stars.

However I'm ashamed to say that my emotional brain sees 5 starts and immediately thinks "That must be good" or it sees 4 stars and thinks "That must be rubbish".

I am, alas, a stupid emotional idiot.
 

Andrewjvt

New member
Jun 18, 2014
99
4
0
Visit site
steve_1979 said:
QuestForThe13thNote said:
It's not meaningless because its opinion. It's to take from it what you will.  So If there is an item on there that is very good and you'd agree with it, as there will be chances are across many products, you agree with the opinion of the reviewer as to its rating.

I'm the same with computer games. 90% = must be great. 85-90% score = worth a look. Less than 85% = rubbish.

Even though I've played high rated games and hated them  or played low rated games and enjoyed every minute the first thing I look at in a review is the score then from that point on I'm biased the that big number at the bottomof the page.

 

Same for me regards video games.
Good example

Wonder how many people are put off the sonica dac because of the 3 star review?
 
steve_1979 said:
Al ears said:
What do you read into WHFSAV star review system?

My logical brain tells me that it's mostly meaningless. Nothing more than an inaccurate represention of someone with no scientific quilifications or background giving a subjective opion on a very oversimplified 1 to 5 scale. This has the potential to be made even more innacurate by the fact that the the magazine requires advertising income from hifi companies to stay in profit and could easily be tempted to add or remove stars.

However I'm ashamed to say that my emotional brain sees 5 starts and immediately thinks "That must be good" or it sees 4 stars and thinks "That must be rubbish".

I am, alas, a stupid emotional idiot.

Perhaps we all are which is why we spend so much time asking dumb questions, or reading them, on this forum... ;-)
 

steve_1979

Well-known member
Jul 14, 2010
231
10
18,795
Visit site
QuestForThe13thNote said:
It's not meaningless because its opinion. It's to take from it what you will. So If there is an item on there that is very good and you'd agree with it, as there will be chances are across many products, you agree with the opinion of the reviewer as to its rating.

I'm the same with computer games. 90% = must be great. 85-90% score = worth a look. Less than 85% = rubbish.

Even though I've played high rated games and hated them or played low rated games and enjoyed every minute the first thing I look at in a review is the score then from that point on I'm biased by the that big number at the bottom of the page.
 

TRENDING THREADS

Latest posts