Test Review Model quality question for What Hi-FI team

[markyd]

It has always amazed me how many Magazine reviews are very different to end users experiences. Take Sony TV's and their ubiquitous backlight issues - the latest generation of sets seems no exception.

I'd like to think that most reviewers are impartial and give commendation based on merit - but the odds of always getting a fantastic test model, are surely influencable?

I know most LCD panels are graded at source. A vast majority fall into the mid-range and budget standards, and a few high quality ones are reserved for high end applications. Is it not possible that the manufacturers can select particularly well performing batches based on serial numbers etc, and assign them as specific 'review approved' models.

Is there any way that review magazines and websites that are sent selected higher-than-averagely-performing review samples - to attain a potentially very lucrative favorable review?

I'm not a conspiracy theorist, but it doesn't seem out of the realms of feasibility...

 

[Clare Newsome]

Well considering the large number of Sony review sets we've had in that have had backlight issues, it's safe to say their test models seem representative....

If we have any doubts about a product, we can - and do - source kit from standard dealer stock.

 

[Andrew Everard]

Though of course if I were working in product PR/loan stock logistics, I'd make sure the product was at least performing to spec before sending it out for review...

 

[AEJim]

Yeah, it's a tricky one Andrew - we (well the old management in particular) often used to send out PP samples in a desperate hurry to meet awards deadlines and it came back to bite us on more than one occasion!

Remember the original Aego-T sub you had? It was very poor due to faulty wadding placement (stapled to the top of the inner cabinet!), discovered on return so we had to have "a strong word" with the production line and re-submit for review later... The original Neo 1's you had - 3 star review, basically some nice fellow in production decided a 90 degree angle on the magnet assembly was a little tricky so he softened it slightly, resulting in lost magnetic flux and weakened bass/mid output so a very "bright" speaker.

Nowadays I always make sure we at least try the speaker out before we send it! This still makes me nervous as I don't want to look like we're fiddling anything by sending pre-opened boxes, but we only send main production run product rather than PP now so there are rarely any issues. The only exception was the recent Pro Sub/Sat that literally came back from our PR guy and straight out - I wasn't in the office that day but we were sending the AE22's anyway so I took a gamble and just told the warehouse guys to put them in too (after clearing it with Simon Lucas of course!) and that seemed to work out pretty well review-wise!

I can see why people would try to cheat the system as reviews are pretty crucial to good sales but I think the fall-out if they were ever caught would be huge, probably enough to finish a company in such a specialist industry so is enough deterrent on its own. Aside from that most people in this industry are pretty good guys on the whole, each confident in their own product and what they're doing - that's not to say you don't hear stories though...

 

[Andrew Everard]

Not really tricky at all, and we're more than happy to get 'open box' products for review if it means the manufacturer/distributor is checking them over before sending them. Saves lots of excuses and recriminations later.

Can't imagine a car company sending a vehicle to one of our sister titles without first putting a few miles on it to make sure all is well.

 

[AEJim]

Andrew Everard:
Not really tricky at all, and we're more than happy to get 'open box' products for review if it means the manufacturer/distributor is checking them over before sending them. Saves lots of excuses and recriminations later.

Can't imagine a car company sending a vehicle to one of our sister titles without first putting a few miles on it to make sure all is well.

Yup, that's why we only send actual production now - we generally get half a dozen review sets out from the first batch, do a buzz & rattle test then a quick measure to make sure they're within tolerance, usually a brief listen if I'm not too busy as well. I just always feel like it's lack of trust in your own product somehow - that what we send out to the customer should be exactly what we send the reviewers, you're totally right that it would be a little risky to send blind though. I don't recall ever having to reject something in the pre-review test though so am happy enough with that, Reference products built in the UK go through these procedures as a matter of course anyway.