Five stars and five stars

[Rich111]

Hi all, I have been a What hifi reader for a long time and I think that it is time that the way the equipment is being rated
must change. For example how would I know which pair of speakers is better between one getting five stars during a test
and selling at $200 compared to another one getting 4 stars but being 10 times more expensive. I think that they all
should be tested equally and rated out of 100 because the actual method is misleading.Meaning if one speaker has
scored 89/100 it is better than another one scoring 80/100 even if the one scoring 89 is 5 times cheaper. I have browsed
through a German Hifi mag some time ago (even if I dont understand German) and this is the way they were rating their
equipment and I found it the best method to rate equipment, really helping customers looking for new gear and knowing
which speaker is really better in terms of sound quality than others irrespective of their price tag

 

[Cypher]

I agree with you. The german magazine "Audio" has the best rating system.

 

[Singslinger]

Interesting proposal but I think most people would want to know what gear is best for particular price ranges.

 

[simonlewis]

For example how would I know which pair of speakers is better between one getting five stars during a test and selling at $200 compared to another one getting 4 stars but being 10 times more expensive.

Well you could go to a retailer & listen to which speakers sound better with your system.

 

[BigH]

It is a good point. Is a Marantz Pearl Lite cd rated at 4 stars better than a £500 5 star player like the Rega? The Marantz was £900 but now around than £670.

 

[BigH]

For example how would I know which pair of speakers is better between one getting five stars during a test and selling at $200 compared to another one getting 4 stars but being 10 times more expensive.

Well you could go to a retailer & listen to which speakers sound better with your system.

Easier said than done, many dealers do not stock the speakers you want to hear, demo rooms are much different than yours, rooms between different dealers are not the same either. Only way to compare is home demo which is not easy to arrange.

 

[BigH]

Interesting proposal but I think most people would want to know what gear is best for particular price ranges.

Yes but what happens if prices change. For instance the Marantz 6004 is now around £225 and the Pearl Lite is around £670 down from £1,000.

 

[Singslinger]

Interesting proposal but I think most people would want to know what gear is best for particular price ranges.

Yes but what happens if prices change. For instance the Marantz 6004 is now around £225 and the Pearl Lite is around £670 down from £1,000.

Make sure the range is wide enough perhaps? Like "£500-1000" then "£1,000-1500" and so on. Not sure what bands WHF use but it's something along those lines.

Then again, I have read reviews describing a particular bit of kit as being the best the reviewer has ever heard, regardless of price. I guess reviewing and rating are inexact sciences and trying to place a simple guide like number of stars to help readers isn't as easy as it might look.

 

[CarlDW]

The Pioneer VSX-922 was rated a 3-star product at £500 but a 5-star product when the price dropped to £250, so the price point of a product can affect its star rating... although I wouldn't expect WHF to re-review all products that decrease in price. A simple star rating within a prescribed price bracket seems to be the easiest eay of identifying the better products out there.

 

[MajorFubar]

I can see the issue, but I'm not sure that a single grading system is more beneficial.

For example let's say the rating was out of 100 and Joe Bloggs is looking for a new pair of budget standmounts. If only the world's best speakers are ever going to get 90+, it's not particularly inspiring for him if he buys WHF and sees that the pair of budget standmounts he's got his eye on are rated 15 at best. How can he know how good that is?

Also, manufacturers are greatly proud of products which receive five-star WHF ratings, emblazoning the WHF 5* logo on their adverts and brochures. The hope of receiving it and being able to display it is one key reason why a manufacturer submits its products to the magazine. But if the new pair of £1,000 Tannoy floorstanders get awarded 30/100, it doesn't look particularly great, and so the kudos of receiving a five-star review would be lost. This would lead to fewer manufacturers submitting products for review and less advertising revenue.

 

[BIGBERNARDBRESSLAW]

The German magazine scoring system looks the best to me.

The 5 star system is just not detailed enough. There seems to be so much variation in quality of 5 star awarded items, even within the same price bracket, that I, and I'm sure others, find it difficult to work out the best product for our individual needs.

For instance, the GT50 has been awarded 5 out of 5, 100%, a perfect product apparently. Well, it isn't is it?

If marks were awarded out of ten within specific areas, and then totalled for a final score, we could see it's overall score, and also see clearly it's shortcomings, and whether, despite those shortcomings, it still offered good value for money.

So say the GT50 got 90/100, but only 4/10 for screen uniformity, we'd see clearly where it's major flaw was.

I can't see that this kind of system would be more time consuming, and it would improve the accuracy of reviews hugely in my opinion, so why not?

 

[BIGBERNARDBRESSLAW]

Also, manufacturers are greatly proud of products which receive five-star WHF ratings, emblazoning the WHF 5* logo on their adverts and brochures. The hope of receiving it and being able to display it is one key reason why a manufacturer submits its products to the magazine.

But to give something 5/5, perfect marks, for something that is way short of perfect, is just misleading.

 

[nopiano]

Everything has to have stars or blobs these days. Meals, cars, films, restaurants, hifi, TVs, etc.

Which? magazine do break down their scores into so many percent for several headings. But WHF offer an eye-catching headline rating, which is hardly surprising given its marketplace. Only an idiot, or someone with a very short attention span, would think that a random assortment of 5 star products will make a great system. But at least 4 or 5 stars tells you they are pretty decent items or even about the best for the money.

That's why if you get 4 stars at £900, you might get 5 stars at £750. And obviously a Pearl Lite Marantz is superior to a regular model at half the price or less (especially as they keep reducing the price).

With my purist hat on, I'd love an absolute scale with 100 being state of the art. But as already noted, "new budget bookshelf speaker cracks 19/100" isn't exactly a marketing person's dream, is it?

As ever, context is everything. A 5 star DAC is no use to you if you actually need a 4 star CD player!

 

[BigH]

I think a higher score rating would help, if you look at speakers the largest rating is 5 stars at 145, 4 stars are 127, 3 stars are 36, and 2 stars 8, no 1 stars?

IMHO there are too many 5 stars, I would expect 5 stars to less than 20% not nearly 46%.

A well know camera review website rates cameras out of 100, broken down into about 10 categories and it seems to work very well.

 

[BigH]

Everything has to have stars or blobs these days. Meals, cars, films, restaurants, hifi, TVs, etc.

Which? magazine do break down their scores into so many percent for several headings. But WHF offer an eye-catching headline rating, which is hardly surprising given its marketplace. Only an idiot, or someone with a very short attention span, would think that a random assortment of 5 star products will make a great system. But at least 4 or 5 stars tells you they are pretty decent items or even about the best for the money.

That's why if you get 4 stars at £900, you might get 5 stars at £750. And obviously a Pearl Lite Marantz is superior to a regular model at half the price or less (especially as they keep reducing the price).

With my purist hat on, I'd love an absolute scale with 100 being state of the art. But as already noted, "new budget bookshelf speaker cracks 19/100" isn't exactly a marketing person's dream, is it?

As ever, context is everything. A 5 star DAC is no use to you if you actually need a 4 star CD player!

I don't think it will work like that as the best bookshelf would get 19/20 for value even if it only got 10/20 for sound. The best £2,000 speaker may get 19/20 for sound but only 14/20 for value, as a £800 speaker maybe nearly as good.

When nearly half the reviews get 5 stars it is of little use to consumers, good for manufacturer.

 

[nopiano]

I don't think it will work like that as the best bookshelf would get 19/20 for value even if it only got 10/20 for sound. The best £2,000 speaker may get 19/20 for sound but only 14/20 for value, as a £800 speaker maybe nearly as good.

But what is 'value'? Surely it is some ratio of Sound Quality to Price. So you are double-counting on the SQ!

A good budget speaker is better value than any multi-thousand pound speaker, I agree there!

 

[William Kelsey]

Five stars and five stars
Five stars suggests to me that items are measured / evaluated on a quality / Value-For-Money quintile scale i.e. 20% for each star, with no item getting zero stars. As such, a five star item rates between 80 / 81% and 100% (with mid-point about 90.5%). I would also suggest that such ratings would correspond to price (RRP or typical price?) in a similar quintile way. So if an item gets reduced in price from one quintile to another, it would seem appropriate to assess irs worth / rating in its new price range, often suggesting that a decrease in price quintile might increase its quality / VFM rating.

Hope this makes sense.

 

[William Kelsey]

Five stars and five stars
Five stars suggests to me that items are measured / evaluated on a quality / Value-For-Money quintile scale i.e. 20% for each star, with no item getting zero stars. As such, a five star item rates between 80 / 81% and 100% (with mid-point about 90.5%). I would also suggest that such ratings would correspond to price (RRP or typical price?) in a similar quintile way. So if an item gets reduced in price from one quintile to another, it would seem appropriate to assess its worth / rating in its new price range, often suggesting that a decrease in price quintile might increase its quality / VFM rating.

Hope this makes sense.

 

[BigH]

Five stars and five stars
Five stars suggests to me that items are measured / evaluated on a quality / Value-For-Money quintile scale i.e. 20% for each star, with no item getting zero stars. As such, a five star item rates between 80 / 81% and 100% (with mid-point about 90.5%). I would also suggest that such ratings would correspond to price (RRP or typical price?) in a similar quintile way. So if an item gets reduced in price from one quintile to another, it would seem appropriate to assess its worth / rating in its new price range, often suggesting that a decrease in price quintile might increase its quality / VFM rating.

Hope this makes sense.

Yes but is a £1,000 cd player with 4 stars better than a 5 star £500 player, you don't know but if you had a sound rating and say the £1,000 got 16/20 and the £500 got 14/20 you would have a better idea. As I said above about 45% of speakers and it seems cd players get 5 stars so how can you choose between them? With hifi I think there is a lot of diminishing returns so it does not always follow that a £2,000 item is better than a £1,000 item.

 

[Native_bon]

Yes i agree with the first post.. Three things:

1) Value for money in percentage.

2) Sound Quality in percentage.

3) Determine price range of categories... for S/Q & Value for money... Lets say a range of £500 to £1500. This way you would be able to tell which serves your needs & best value to sound ratio. Also, if for some reason a £200 Hifi product could compete in that range of price then by all means it should also be included.

 

[Rich111]

YesI think we should all rate them out of 100 but rate them in their repective price categories as well. Yes it is
time consuming but it will really help potential buyers and bring someting really positive to what hifi mag.

 

[ID.]

I think this has been discussed, and answered by the magazine, in numerous threads already.

I think they should rate it through interpretive dance, but until they adopt my revolutionary, and clearly superior approach, I don't have much trouble interpreting the results by accounting for (1) the price bracket, (2) the text of the review and (3) the star rating, especially as I'm going to want to listen to the kit myself and find out what suits my tastes.

 

[Ajani]

I'm sure ID is right and this topic has been discussed before, but let me give my views anyway:

As an audiophile I'd love to see an absolute rating system like the German mag. I imagine 100 would be live music (or whatever) and 0 would be the sound my system makes when it is unplugged/disconnected.

HOWEVER, as other posters have noted there are limitations to such a scale.

1) An absolute rating doesn't tell me whether the product performs well in its price class. If a $500 amp get a rating of 40 is that above or below average for a $500 amp? I'd have to be familiar with the average rating for amps to make that decision. So a simple 5 star rating tells me that the amp is one of the best options in its class.

2) Such a scale is more difficult to measure accurately. How does someone determine whether a product should be rated 81 or 82 out of 100? We're talking about HiFi, not a college exam. So again it's easier to use broader categories like 1 - 5 stars.

3) Consumers don't want to buy a product rated 40/100. That sounds like junk, even though the product could be excellent value for money and well above the average rating for products in its price range. They'd rather buy something rated 5/5.

That said, there are some issues with 5 star rating system:

1) Is a 4 star $700 amp better than a 5 star $450 one? No way to tell from the ratings as they are in different price classes.

2) I think most HiFi mags (including WHF) grade too easily. Which is why there seem to be so many 5 star reviews. If 3 is average, then the majority of products reviewed should be rated 3 stars. Very very few products should receive either 1 or 5 stars.

Others in this thread have proposed various hybrid systems, but those also tend to get confusing. The more variables you measure make the review more useful but also more difficult to interpret. So one could argue that you might as well just read the narrative and not bother with the rating (as some US mags do).

 

[fr0g]

An absolute system cannot work.

Why?

Time... Is an 80% speaker reviewed in 2013 better than a 75% speaker reviewed in 2010? Really? Did the team compare them side by side?

In other words, there is no absolute that can possibly be accurate and fair. People will pick up the magazine and say, look, your 2009 NAD amp only got 71%, whereas this Marantz reviewed in this weeks issue got 74%....therefore this one is better.

And how many points would you leave unused for future product improvements? 5%, 10%, more??

And given the differences in electronic equipment at even a budget level and above, most reviews would end up in a clump at 80-95% anyway

The only fair method is based on the nearest equivalent products, which generally means pricepoint.

While I flinch and squirm at some reviews myself, a 4/5 star review tells me there is obviously nothing major wrong with a product if I'm in a rush and want a list of possible audition candidates at my price point.

Why fix what isn't (badly) broken?

 

[BigH]

Yes but the whole point of rating products is help people choose between them. If you take cd players they have rated 111, 51 5 stars, 45 4 stars, 14 3 stars an 1 2 stars. So 86.49% get 4 or 5 stars, so that means you will be unlucky to pick a bad one. Similar for speakers and power amps. Int. amps which many say are all similar have wider ratings?

If you keep the 5 stars how about using 1/2 marks, so some of those 5 stars would get 4.5. You read comments for 5 stars speakers, like not the most refined, treble can be harsh, well to me they should not be 5 stars.