Is 5-stars enough?

[Teeza]

What Hi-Fi has been using 5 stars to rank “comparable” equipment for as long as I can remember but have you ever considered any other ways to sort the good from the mediocre?

The current system has the advantage of being simple but it also has some serious drawbacks. It does not allow a direct comparison of a budget item with a mid-range item for example. Nor does it allow distinction between competing products that all get 5 stars. Finally it does not allow comparison over time as the best of the best today is still only 5 stars but no doubt much better than the best of the best 5 years ago (that I might be considering second hand).

All of this makes compiling a short list for audition harder work than could reasonably be expected when buying the mag.

Here is my straw man: keep the stars (at least for the time being) but add a relative score (compared to your reference kit) out of 100, at the foot of the review text (& new column in the tables). That should provide a good way determine the relative value for money across the range

 

[Clare Newsome]

Thanks for the feedback - link here to a big recent thread on ratings system you might find useful.

 

[Alec]

sorry to add more fuel to this one but any system making the buyers guide even bigger, frankly, scares me and makes me a bit

 

[Andrew Everard]

Been round this track too many times before, and for all the talk about straw men and other debating stuff we're sticking with what we have for now.

 

[Davro83]

I thought this and think this also. I mean say one product costs £200 and another costs £5000 and they both have 5 stars and i have £5000 to spend. Can i buy the £200 one and save loads of money? You need a system so we know what the best products are. Alot of items get 5 stars and you give somethings 5 stars because it costs little, or you dont give 5 stars because it costs too much. You need a rating system where a score is given, regardless of price so we know whats the best and whats not.

 

[Andrew Everard]

Al, you need to remember is that the star rating is on sound/vision/performance per pound, and then it's all clear.

 

[Clare Newsome]

And also remember that we publish both price-segmented Best Buys each issue, and online (and Award winners) that clarify the incremental steps up in performance....

 

[Alec]

Y'see!? Simpull, innit?

 

[Anonymous]

While What Hi Fi has it's place in pointing it's customers in the direction of what on the market place I for one don't buy into the star rating.

Only your ears can determine how good a product is.

My mate is a musician and got a call from a music mag. He was told more or less to put in a good advert for the album and his album would be looked at in a positive way.

I hear the same comment from sales staff at Hi Fi shops when listening to new equipment. Don't just buy something because it has five stars. Over the years I have heard five star beaters in a shop when getting a demo.

Recently a mate was kind enough to bring his Yamaha DSP-AX863SE over to my house for a listen. Yes it is a good av receiver. Then I had a demo of a Denon - Onyko and a Marantz. The Onkyo was too bright for my system while the Denon and Marantz had a warmer sound. I went for the Marantz SR-6003 for its sound and features. Now my mate is going to sell his Yamaha for the Marantz which can be bought for £630 on the net.

As I wrote above the star system can put you in a direction but it does not mean it's the best so always demo.

 

[Clare Newsome]

Yep, demoing kit is key - another point we stress every single issue (and up here, too...)

And BTW, I don't know how it works on music mags (or indeed other hi-fi mags), but advertising certainly doesn't influence reviews here on WHF - plenty of example of big advertisers who get poor reviews and indeed non-advertisers that get five stars and win Awards. Such are the joys of editorial independence and integrity!

 

[Big Aura]

the one little suggestion I would make (and I think you do this in the mag, but i don't have one at work to check) is that it would be good if the online reviews carried a date. Trying to differentiate between two products might be easier done if you had a bit of visibility as to when each got its star rating. Otherwise, I think you guys are the best.

 

[chebby]

Like any good system it should go up to 11.

 

[Anonymous]

Big Aura:the one little suggestion I would make (and I think you do this in the mag, but i don't have one at work to check) is that it would be good if the online reviews carried a date. Trying to differentiate between two products might be easier done if you had a bit of visibility as to when each got its star rating. Otherwise, I think you guys are the best.

Good point - star ratings are a bit bogus because they don't take into account people's actual systems (different combinations of kit will always sound different) but a date would be really useful.

 

[Anonymous]

randomdelusion:
Big Aura:the one little suggestion I would make (and I think you do this in the mag, but i don't have one at work to check) is that it would be good if the online reviews carried a date. Trying to differentiate between two products might be easier done if you had a bit of visibility as to when each got its star rating. Otherwise, I think you guys are the best.

Good point - star ratings are a bit bogus because they don't take into account people's actual systems (different combinations of kit will always sound different) but a date would be really useful.

Guys, lots of "nice to have's" but only one really viable request. A date on reviews. Can this be done WHFSV?

 

[Clare Newsome]

Ahead of you all there - it is already on every review here online (in the content management system) and appears when you do searches. Under the advanced search options for every category, you can also sort the reviews by earliest to latest (or vice versa) so you know which order products were reviewed in.

But yes, I agree the date should be more clearly displayed, too - it was on our original spec (hence us inputting it for every review!), but somehow didn't get coded. It's now on the fix list....

 

[Anonymous]

Clare,

I'm not trying to run down What Hi Fi as I buy the mag quite regularly myself and have went on to hear and purchase recommendations through out the years of reading the mag.

From reading the forum it seems many buy a five star product before even hearing the equipment first hand.
Each to there own I suppose

Personally I don't see why the star rating's needs to be changed. What Hi Fi is a good reference point in which to look when going to buy equipment.

 

[Teeza]

Interesting to note that: 1. I am not the first to raise this question 2. minds at WHF seem closed to further debate 3. balance of replies favour keeping the star system 4. stars are no substitute for listening.

I did not ask for the start ratings to me removed nor changed but supplemented. I did ask for a way of assisting the compilation of short list (say 3) for audition.

Take my own requirements as an example: I wanted to audition two amps (the Yamaha 763 and 863) but don't seem to be able to in the near future. So what should I look at instead? A quick review of the kit listed here, on-line, in the price range 251-600 reveals a total of 15 amps. One of them is a 1 star but the other 14 are all either 4 or 5 stars. On the assumption that a 4 star amp in the 401-600 band is potentially better than a 5 star amp in the 251-400 (double) band then I have "short" list of a dozen to audition. Totally impractical.

What would you suggest I replace my original short list with? And more importantly how will you choose them? What will your criteria be for culling up to 10 4/5 star products?

 

[Anonymous]

In the end, this just goes to show the value of supertests. Big well populated tests where all the players go up against each other.

The BluRay test last month was very good. No doubt the ranks will be filled when the rest of the high end toys come in. And then, as now, I'll be itching to see a supertest.

I always wanted What Hi-Fi to give a % rating of outright ability. But this isn't always a science. There's plenty of art to hi-fi and av and I'm beginning to see that trying to make it utterly scientific would remove the editorial interest from reviews. It would also be utterly impractical and lead to the most henious arguements. Just imagine!

So let be a bit realistic here. What Hi-Fi is a fun mag and a great guide. They're not going to change because the formula works. And I for one, have changed my mind and hope some of you come round too.

 

[Anonymous]

Teeza don't by pass the Marantz SR-6003 for demo. I kid you not when I say nothing can touch it in its price range for movies and music.

I'm really surprised What Hi Fi have not given it the once over as it's readily available and has been for a while.

 

[Clare Newsome]

Teeza:
Interesting to note that: 2. minds at WHF seem closed to further debate

Certainly not - and as anyone who's a regular on these Forums will know, we certainly respond to feedback.

As for cutting down your shortlst, i'd start with a list of your particular requirements and then read through those reviews you've singled out (including tech-specs, but not obsessively so) to find reference to those requirements.

Something may have, for example, got four stars because we felt it was a tad pricey - if you can find a bargain deal on it now, however, and it fits your requirements, add it to your shortlist.

Conversely, if something is highly rated but we single out a slight flaw in an area that's key to you - maybe a specification short-coming, or a sonic quirk - then you can discount it.

Hope this helps!

(PS: We haven't reviewed the Marantz as we haven't had review units in yet, despite repeated requests).

 

[Anonymous]

Half stars would be nice. Then again would they be half empty or half full?

 

[Davo2008]

Does that mean that when you gave the Onkyo 876 4 stars at £1,000 and now it's available for £599, you'd give it 5 stars?

 

[Big Aura]

"We are all in the gutter, but some of us are looking at the stars"

 

[Clare Newsome]

Davo2008:Does that mean that when you gave the Onkyo 876 4 stars at £1,000 and now it's available for £599, you'd give it 5 stars?

Depends on what else you could buy for that price these days, but at £599 (and as long as you're happy with its upfront presentation) it's certainly an even more tempting proposition!

 

[Davo2008]

Thanks Clare. By the way, did that £3.99 cable ever turn up? Don't forget to let us know when the test has been done!