WHF Star Rating System

[Snooker]

I must admit I find the star rating system where it takes into account the price of the system to be misleading

Surely all you need to do is give it an overall percentage rating based on the actual sound quality etc, and maybe indicate that a system around:-

30% Poor 40% Below Average 50% Average 60% Good 70% Very Good 80% is Excellent 90% Superb and 100% Outstanding

 

[CnoEvil]

You could also argue, that if you are buying a £1k amp, you want to know how it sits against other £1k amps. How it fares against the SQ of a £10k Bryston, may not seem so relevant (to some).

IMO. Both systems have merit.

 

[jjbomber]

Sometimes the price comparisons are just plain stupid. Look how often phones are listed as free or £0, as you can buy them on monthly contract. The £270 OnePlus One wipes the floor with top of the range Samsungs, Sony Apple etc. They should be 3 star on a per pound basis but 5 star at £0. In fact, all phones at £0 should be 5 star.

Other times they can be useful. Products are normally available at the tested price, so we have a reference point. A product may be 4 star at £1000 but may be offered at £750. We now to adjust for the price and that will beat a 5 star at £750, or at least worth an audition.

Finally, the stars are only someone's opinion. There have been some howlers on here. For example Oppo DVD getting 3 stars even though it is award winning everywhere else. On the other hand, the ipad 3 was award winning yet pulled after 6 months by Apple. Actually, take all Apple reviews with a pinch of salt, if not an entire tablet of salt.

So the system is not perfect but it works.

 

[BIGBERNARDBRESSLAW]

So the system is not perfect but it works.

And that's a fact!

 

[lindsayt]

With my speakers I prefer my c£1k new amp to the £10k Bryston.

So on my star rating system, the Bryston would get 1 star out of 5.

There are loads of speakers where I'd prefer the Bryston. So I can understand other people giving it 5 stars and giving my amp 1 star...

The What Hi-fi star system is only a very broad indication of what they thought of a product in the context it was reviewed in. Nothing more. Nothing less.

I've never taken any notice of What Hi-fi star ratings when I've bought hi-fi equipment, but I can understand why other people are happy to be guided somewhat by their star ratings.

 

[drummerman]

WHS&V's rating system is very simplistic and deliberately so. It is aimed Joe Public and there is absolutely nothing wrong with that. The mixed content speakes volumes. They cover a larger area of entertainment electronics than most of the other publications and something has to give.

Most seasoned audiophiles will probably rather read other more hifi orientated magazines/reviews with more elaborate ratings and measurements. - Joe Public and newbies are simply not interested in that and in the unlikely case they are, the choice is out there.

 

[Al ears]

WHS&V's rating system is very simplistic and deliberately so. It is aimed Joe Public and there is absolutely nothing wrong with that. The mixed content speakes volumes. They cover a larger area of entertainment electronics than most of the other publications and something has to give.

Most seasoned audiophiles will probably rather read other more hifi orientated magazines/reviews with more elaborate ratings and measurements. - Joe Public and newbies are simply not interested in that and in the unlikely case they are, the choice is out there.

Ditto.

There are more complicated systems around but the 5 star ratings used by this publication are, to all intents and purposes, the only way for them to go about it for Mr. Average. It is a long and well-established system and any change to it would still have it's own inherrent flaws.

 

[nima]

WHF 5 Star Rating System is really: almost every review gets 5 stars.

 

[Laurens_B]

So who are these guys, Joe Public and Mr. Average, I have never seen them around, yet all the reviews are specially for them!! I will send an official complaint.

 

[jjbomber]

So who are these guys, Joe Public and Mr. Average, I have never seen them around, yet all the reviews are specially for them!! I will send an official complaint.

They do exist ... according to the man on the Clapham Omnibus.

 

[SteveR750]

Sometimes the price comparisons are just plain stupid. Look how often phones are listed as free or £0, as you can buy them on monthly contract. The £270 OnePlus One wipes the floor with top of the range Samsungs, Sony Apple etc. They should be 3 star on a per pound basis but 5 star at £0. In fact, all phones at £0 should be 5 star.

Other times they can be useful. Products are normally available at the tested price, so we have a reference point. A product may be 4 star at £1000 but may be offered at £750. We now to adjust for the price and that will beat a 5 star at £750, or at least worth an audition.

Finally, the stars are only someone's opinion. There have been some howlers on here. For example Oppo DVD getting 3 stars even though it is award winning everywhere else. On the other hand, the ipad 3 was award winning yet pulled after 6 months by Apple. Actually, take all Apple reviews with a pinch of salt, if not an entire tablet of salt.

So the system is not perfect but it works.

The idea is to rate a bunch of products within a price band, just like the awards do. If all phones are FOC, they are all in the same price band, and the relative star rating remains valid. For years, long before the online reviews the mag had the buying guide at the back, which was always split into distinct price bands (and might still be but it's been a long time since I bought a copy)

 

[Al ears]

So who are these guys, Joe Public and Mr. Average, I have never seen them around, yet all the reviews are specially for them!! I will send an official complaint.

They do exist ... according to the man on the Clapham Omnibus.

Ah! That was Fred Blogs....

 

[BigH]

WHF 5 Star Rating System is really: almost every review gets 5 stars.

Not true but most get 4 or 5 stars, its about promoting hifi not reviewing it.

 

[CnoEvil]

With my speakers I prefer my c£1k new amp to the £10k Bryston.

So on my star rating system, the Bryston would get 1 star out of 5.

There are loads of speakers where I'd prefer the Bryston. So I can understand other people giving it 5 stars and giving my amp 1 star...

The What Hi-fi star system is only a very broad indication of what they thought of a product in the context it was reviewed in. Nothing more. Nothing less.

I've never taken any notice of What Hi-fi star ratings when I've bought hi-fi equipment, but I can understand why other people are happy to be guided somewhat by their star ratings.

The Bryston was simply an example.....you can choose any £10k amp that you want, in which case you will almost certainly get a much better sound.

 

[BigH]

WHS&V's rating system is very simplistic and deliberately so. It is aimed Joe Public and there is absolutely nothing wrong with that. The mixed content speakes volumes. They cover a larger area of entertainment electronics than most of the other publications and something has to give.

Most seasoned audiophiles will probably rather read other more hifi orientated magazines/reviews with more elaborate ratings and measurements. - Joe Public and newbies are simply not interested in that and in the unlikely case they are, the choice is out there.

Ditto.

There are more complicated systems around but the 5 star ratings used by this publication are, to all intents and purposes, the only way for them to go about it for Mr. Average. It is a long and well-established system and any change to it would still have it's own inherrent flaws.

I don't think so. Other publications like car mags and camera reviews (dpreview) have much more sophisated scoring systems and they seem to work. Why are there so few bad reviews in hifi mags? I don't think they are true reviews, its about promoting hifi and advertising.

 

[BIGBERNARDBRESSLAW]

This has definitely been discussed in some depth before, but the powers that be are not interested in changing the system.

Personally, I'd like all products to receive marks out of hundred, with 5 categories within the hundred with marks out of 20, and with no adjustment for the price of the product, just totally based on quality. Then we can all see which is the best product (in the eye of the reviewers), and make our purchasing decisions based on what we can afford.

Far too many products are receiving 5 star reviews, and this will continue to happen until the review system is changed.

 

[Infiniteloop]

With my speakers I prefer my c£1k new amp to the £10k Bryston.

So on my star rating system, the Bryston would get 1 star out of 5.

There are loads of speakers where I'd prefer the Bryston. So I can understand other people giving it 5 stars and giving my amp 1 star...

The What Hi-fi star system is only a very broad indication of what they thought of a product in the context it was reviewed in. Nothing more. Nothing less.

I've never taken any notice of What Hi-fi star ratings when I've bought hi-fi equipment, but I can understand why other people are happy to be guided somewhat by their star ratings.

The Bryston was simply an example.....you can choose any £10k amp that you want, in which case you will almost certainly get a much better sound.

Careful. Trev C. might be around......

 

[drummerman]

WHS&V's rating system is very simplistic and deliberately so. It is aimed Joe Public and there is absolutely nothing wrong with that. The mixed content speakes volumes. They cover a larger area of entertainment electronics than most of the other publications and something has to give.

Most seasoned audiophiles will probably rather read other more hifi orientated magazines/reviews with more elaborate ratings and measurements. - Joe Public and newbies are simply not interested in that and in the unlikely case they are, the choice is out there.

Ditto.

There are more complicated systems around but the 5 star ratings used by this publication are, to all intents and purposes, the only way for them to go about it for Mr. Average. It is a long and well-established system and any change to it would still have it's own inherrent flaws.

I don't think so. Other publications like car mags and camera reviews (dpreview) have much more sophisated scoring systems and they seem to work. Why are there so few bad reviews in hifi mags? I don't think they are true reviews, its about promoting hifi and advertising.

Mr Average is unlikely to read dpreview. He or she uses his or her phone to take pictures ... .

There are bad reviews in hifi magazines. Just read the latest HifiWorld and their review of the newest Sony portable 'high resolution' player, just one example.

Not getting five stars/globes but 'only' four is probably often enough for folks to completely dismiss a perfectly good product, so there you have your damnation.

I am not saying for a moment that they're always right but to imply that there is some conspiracy theory is so ... 90's.

 

[lindsayt]

The Bryston was simply an example.....you can choose any £10k amp that you want, in which case you will almost certainly get a much better sound.

I almost certainly won't. Google "Jeff Korneff amplifier".

 

[matt49]

Careful. Trev C. might be around......

No, he's gone caravaning with Vladimir and davedotco.

 

[ID.]

Not getting five stars/globes but 'only' four is probably often enough for folks to completely dismiss a perfectly good product, so there you have your damnation.

This is would be the worst part. Why settle for 4 stars when there are 5 star products?

Then we have people joining the forums asking why their combination of 5 star products sounds carp.

Then the confusion when one publication rates a product highly but WHF doesn't, or vice versa. Then the screaming of their soul as they realize they have to spend time looking for products they like the sound of and that there is no god giving ultimate rankings of which products are best. The plaintive wails as people post threads about how can WHF give such a high/low rating to something they love (I was shocked to see a recent headphone review give 3 stars when I was sure it was a 4-5 star product, and didn't see the same issues the reviewers did with bass performance). But none of these are solved by using a different rating system.

Like any magazine/rating system, you have to first understand the criteria, consider it in partnership with the written review, and then use it to create a list of products you want to audition (WHF themselves also encourage this approach and never claim that their ratings/reviews are a guarantee of satisfaction). The biggest failings would be to rule out products with fewer stars or ignore the advice of an experienced dealer - particularly with respect to system matching - just because you read a review/rating somewhere.

 

[drummerman]

Not getting five stars/globes but 'only' four is probably often enough for folks to completely dismiss a perfectly good product, so there you have your damnation.

This is would be the worst part. Why settle for 4 stars when there are 5 star products?

Then we have people joining the forums asking why their combination of 5 star products sounds carp.

Then the confusion when one publication rates a product highly but WHF doesn't, or vice versa. Then the screaming of their soul as they realize they have to spend time looking for products they like the sound of and that there is no god giving ultimate rankings of which products are best. The plaintive wails as people post threads about how can WHF give such a high/low rating to something they love (I was shocked to see a recent headphone review give 3 stars when I was sure it was a 4-5 star product, and didn't see the same issues the reviewers did with bass performance). But none of these are solved by using a different rating system.

Like any magazine/rating system, you have to first understand the criteria, consider it in partnership with the written review, and then use it to create a list of products you want to audition (WHF themselves also encourage this approach and never claim that their ratings/reviews are a guarantee of satisfaction). The biggest failings would be to rule out products with fewer stars or ignore the advice of an experienced dealer - particularly with respect to system matching - just because you read a review/rating somewhere.

The Headphone example is a good point in case; Headphone sound is arguably mostly defined by the ear seal (or lack of it) with our differently shaped ears. Just try shifting your earphones a little bit or lift them away. Headphones are sometimes updated by simply changing the earpads.

Very much the same goes for room interaction. A huge contributor to the end effect. - I am not stating anything new here.

It can certainly explain differences in perceived sound quality, apart from personal taste and that includes reviewers. - In WHS&V's favour, they have accousticically treated rooms and have therefore a perhaps more neutral platform to work from than some others.

Measurements on the other hand are undisputable.

I am in between the subjective and objective camp, appreciating both. I read technical reviews and can interpret them but know full well that that is not the be it and end all. There are enough unknown factors to potentially significantly change the end result.

As for cables, well ...

(did I ever mention the excellent mains leads I use?)

 

[chris_bates1974]

I've found this thread really interesting. I must admit I have never really paid attention too much to the number of stars, just had a listen myself. So, out of interest, I just looked up the WHF review for my speakers (the most recent new purchase I've made) and found they were given only 3 stars. Based solely on that, sales would be pretty low!

It took a little time to get them set up prefectly in my hifi room, but my personal opinion is that they deserve more. But, it's my personal opinion.

I'd stress that the single most important review would be the one you make yourself when you hear the stuff you are interested in.

 

[Al ears]

WHS&V's rating system is very simplistic and deliberately so. It is aimed Joe Public and there is absolutely nothing wrong with that. The mixed content speakes volumes. They cover a larger area of entertainment electronics than most of the other publications and something has to give.

Most seasoned audiophiles will probably rather read other more hifi orientated magazines/reviews with more elaborate ratings and measurements. - Joe Public and newbies are simply not interested in that and in the unlikely case they are, the choice is out there.

Ditto.

There are more complicated systems around but the 5 star ratings used by this publication are, to all intents and purposes, the only way for them to go about it for Mr. Average. It is a long and well-established system and any change to it would still have it's own inherrent flaws.

I don't think so. Other publications like car mags and camera reviews (dpreview) have much more sophisated scoring systems and they seem to work. Why are there so few bad reviews in hifi mags? I don't think they are true reviews, its about promoting hifi and advertising.

Mr Average is unlikely to read dpreview. He or she uses his or her phone to take pictures ... .

There are bad reviews in hifi magazines. Just read the latest HifiWorld and their review of the newest Sony portable 'high resolution' player, just one example.

Not getting five stars/globes but 'only' four is probably often enough for folks to completely dismiss a perfectly good product, so there you have your damnation.

I am not saying for a moment that they're always right but to imply that there is some conspiracy theory is so ... 90's.

Yes, the Sony review was quite illuminating considering it's other recent offerings but nice to see something from the big-boys criticised so markedly.

I, for a great number of years, ran a system in which there was not a single piece of 5 star kit and it sounded great. People must realise that these rankings are purely some for of reference in relation to other the reviewer has heard in the past, they are based on individual pieces of equipment and can, in no way, give any idea what that piece of equipment will do when incorporated into a system.

You might have an all singing-all-dancing 5 star rated CD player or the like but integrate that into the wrong system and its all going to sound carp.

The secret to a good system is compatibility, or synergy some call it. Therefore star ratings in this instance are pretty meaningless.

 

[Snooker]

I don't think they are true reviews, its about promoting hifi and advertising

******************************************************************************************************************************

I agree with the above quote, and that if you did give the reviews scores in the 100% range purely based on sound quality etc, and not taking into acount the price that some of the cheaper stuff would be better than some of the much dearer stuff, and of course this would be bad for sales regarding the much dearer stuff, I have no doubt that this is true

You could easily devise a new rating system for the above, its quite easy really

I may be wrong about in this one particular example, but please read the information below:-

(Just out of interest the Denon Ceol N7 got 5 stars, but the Denon Ceol N8 got 4 stars, and am sure the audio electronics have not changed at all, whilst the current Marantz MRC610 got 5 stars, I may be wrong but it apears they are perhaps trying to "promote" the Marantz more than the Ceol, and the new current Denon Ceol N9 has not been reviewed yet !)