Flawed ratings system
- Thread starter drummerman
- Start date
Mind, compared to the **** interface of this forum (my thread has dissappeared) the ratings system is quite good!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Crxxx
Crxxx
Al ears
Well-known member
Without re-typing the thing (i am currently cursing the little guy maintaining this site in Bangladesh) ...
... HifiNews&RR system of '%' is much better. Nothing, I think, ever scored a full 100, that would be perfection which, even at the level of equipment they review (and measure) nigh on impossible (impossible?)
This mag (and HifiWorld, although they at least bench test) dish out full stars/globes etc by the bucket load even though flaws are almost always discovered and commented on (read the review of the new Muso).
In the case of HifiWorld, these shortcommings have quite frequently been significant (imho) making it even more questionable why they would give something full marks.
I guess, at least the faults are pointed out in the review (and often show in the measurements).
WH needs a different system. Even ten starts would be better but if they lack the space to fit them in (or time to type them out) adopt the % system ... imho.
... HifiNews&RR system of '%' is much better. Nothing, I think, ever scored a full 100, that would be perfection which, even at the level of equipment they review (and measure) nigh on impossible (impossible?)
This mag (and HifiWorld, although they at least bench test) dish out full stars/globes etc by the bucket load even though flaws are almost always discovered and commented on (read the review of the new Muso).
In the case of HifiWorld, these shortcommings have quite frequently been significant (imho) making it even more questionable why they would give something full marks.
I guess, at least the faults are pointed out in the review (and often show in the measurements).
WH needs a different system. Even ten starts would be better but if they lack the space to fit them in (or time to type them out) adopt the % system ... imho.
We've discussed this many, many times and done a lot of research on it. Every time, the vast majority of readers say they want us to stick with the five-star rating system, as it's clear and simple to understand, especially in overseas markets where we also operate.
We experimented with 10-star ratings on our sister title Autocar once, and everyone hated it, so we went back to five. I know some games titles use the % system, but in reality what would be the difference, say, between a score of 79% and 82%?
Half stars are one option we've considered, but does it really help buyers if a product gets 4.5 stars rather than 4? Not sure it does. And the star rating system is something we use across all our brands (WhatCar?, Autocar?, Stuff and What Hi-Fi?) so it wouldn't make much sense to change it unilaterally for one brand and not the others.
And for the record, we have cut back on the number of five-star reviews we give out.
We experimented with 10-star ratings on our sister title Autocar once, and everyone hated it, so we went back to five. I know some games titles use the % system, but in reality what would be the difference, say, between a score of 79% and 82%?
Half stars are one option we've considered, but does it really help buyers if a product gets 4.5 stars rather than 4? Not sure it does. And the star rating system is something we use across all our brands (WhatCar?, Autocar?, Stuff and What Hi-Fi?) so it wouldn't make much sense to change it unilaterally for one brand and not the others.
And for the record, we have cut back on the number of five-star reviews we give out.
Al ears
Well-known member
drummerman said:
Yep, I do like their percentage system. All these so called defects mentioned can be just down to the particular reviewer. Some things considered wrong might just happen to sound just right to other listeners. All reviews can be guidelines only and this is why you have to get out and audition yourself. No other way of doing it.
Not sure why anyone would argue about a finer scale of scoring but if market research has shown that that is the case who am I to argue against it 
drummerman said:Not sure why anyone would argue about a finer scale of scoring but if market research has shown that that is the case who am I to argue against it
And I guess the cross-brand consistency greatly simplifies those tricky Ford Mondeo/Arcam amplifier comparisons! ;-)
Blacksabbath25
Well-known member
yes a very clever ratings system which most people will only buy because of a 5 star ratings only what happen's to the amps ,cd players , speakers that only get the 3 stars will people buy them ? as they may think the 3 star poducts are rubbish . i myself used to go by the 5 star ratings system in the past but not anymore as there is some great hifi out there that no one looks at maybe because 5 stars means more to them then there own choice's *ok*Andy Clough said:
Blacksabbath25 said:
Ive also brought the roksan kandy k2 based on reviews only to find out it was not as good as i thought or read.
Sounds sweet until the music gets demanding or the speakers dip then became grainy.
I have no problem with the reviews but i think there should be more measurements and also more info on internals that the specs miss out. Also if it gets 4 or 5 stars. At least have the balls to compare it to other amps and saywhy or how.
Then say compared to this its a 5 star but say spend 200 more and you could have this that is so much better. Insert reason etc.
Not just excellent product. Ir this new design is much better than the one it replaces.
Why? What does it do so much better? How much difference between old and new. This amp is good but if you like party volume rather buy this one because ..........
Its like the reviews have become so vague. Like reading a star sign
It would be interesting to know the difference 79% had on sales compared to 81% or whatever. 4 vs 5 stars clearly has a massive impact. Just look at everyone who turns up on he forums asking only about combinations of 5 star kit. Heck, as far as I know WHF isn't sold here in Japan but I've seen little signs on kit indicating that it got full marks from WHF, or whatever source the shop or manufacturers want to use to help promote their kit.
I think the system is OK, but it does result in making very strong divisions between levels. Keeping it simple is all good and well, but even then readers don't understand how it works and we get frequent threads asking about why cheaper kit got 5 stars and something twice the price only got 4, demonstrating that they don't understand that the ratings are compared to other kit in the same price band.
On a certain level, people understand numbers, even if they are out of 10 or 100 while trying to glean an overall impression through a combination of stars and the actual content of the review as well as other kit in a similar price bracket can be challenging for some.
I think the system is OK, but it does result in making very strong divisions between levels. Keeping it simple is all good and well, but even then readers don't understand how it works and we get frequent threads asking about why cheaper kit got 5 stars and something twice the price only got 4, demonstrating that they don't understand that the ratings are compared to other kit in the same price band.
On a certain level, people understand numbers, even if they are out of 10 or 100 while trying to glean an overall impression through a combination of stars and the actual content of the review as well as other kit in a similar price bracket can be challenging for some.
i have no problem with what hifi ratings, i have bought kit on their reviews and found them to be accurate...five pairs of speakers..three amps..one pre amp..Andy Clough said:
Frank Harvey
Well-known member
Have you never come across a product that isn't good enough for 5 stars, but is too good for 4?Andy Clough said:
I rate movies on IMDB and Letterboxd - the latter uses a 5 star system and the former a 10 star system, and as Letterboxd allows half stars, it's easy for me to give equivalent ratings on both sites. 5 (and only full) stars seems a little limiting, unless you're going to drag everything down to a basic level and say that a product falls into that, that, that, that, or that category. At the other end of the scale, I think that % is a little to precise, although if you felt that one of two competing products are on an even keel, but you just feel that one is slightly better or has the edge, you can give it a few more % - not possible within 5 full stars.
The problem is any rating system is flawed. If you take an absolute stance and say a 5-star product represents the best they've ever heard at any price, then the products at a tenth of the price which still represent the best VFM in their own sector are going to look a bit pants scoring 2 stars. But if you take a relative stance, it can be difficult to judge how the best products in one price category compare with products in the next.
I reckon if you adopt a stance that you have to double the price to really notice an appreciable difference, then perhaps with the exception of a few giant killers such as the A400 amp (I'm not sure what its modern equivalent would be in terms of how game changing it was), you won't go far wrong.
I reckon if you adopt a stance that you have to double the price to really notice an appreciable difference, then perhaps with the exception of a few giant killers such as the A400 amp (I'm not sure what its modern equivalent would be in terms of how game changing it was), you won't go far wrong.
My beef was simply with the practise of giving full marks to things that have flaws. To me that makes no sense. The five star/globes etc rating simply doesn't allow to differentiate enough but market research has apparently shown that is exactly what folks want
Good enough
Good enough
jjbomber
Well-known member
The 5 star system is part of the What Hi-Fi brand and really it is part of hi-fi history, whether we like it or not. Changing it would be a shame.
Hi-Fi World globe system is just plain dumb. Rating in globes... Now they should switch to % rating system IMO.
Hi-Fi World globe system is just plain dumb. Rating in globes... Now they should switch to % rating system IMO.
Al ears
Well-known member
Vladimir said:
Stars, globes, apples..... what's the difference? Five of anything should be sufficient to rate an item of equipment.
Al ears said:
Which is what their market survey apparently showed.
However, if 5 is perfect (and it clearly isn't, read the new Muso review but there are many more) that leaves 4. Very simplistic imo and a bit of a 'dumb down'.
Hey ho
Al ears
Well-known member
drummerman said:Al ears said:
Which is what their market survey apparently showed.
However, if 5 is perfect (and it clearly isn't, read the new Muso review but there are many more) that leaves 4. Very simplistic imo and a bit of a 'dumb down'.
Hey ho
I believe the dumbing down has it's purpose. Can you see newbies getting on with a percentage system? This forum would be a nightmare. Not that it already isn't at times ;-)
Al ears said:drummerman said:Al ears said:
Which is what their market survey apparently showed.
However, if 5 is perfect (and it clearly isn't, read the new Muso review but there are many more) that leaves 4. Very simplistic imo and a bit of a 'dumb down'.
Hey ho
I believe the dumbing down has it's purpose. Can you see newbies getting on with a percentage system? This forum would be a nightmare. Not that it already isn't at times ;-)
I guess so and the shortcomings are, after all, pointed out in the review.
Aaaanyway, they ain't going to change it.
Similar threads
TRENDING THREADS
-
-
-
Is hi-fi really getting better? Our technical editor ponders its progression
- Started by What Hi-Fi?
- Replies: 16
-
-
-
Integrated amplifier with HDMI arc and self turning on on Optical input
- Started by sirfabio
- Replies: 2
-
Facebook
Twitter
Instagram