Page 1 of 2 12 LastLast
Results 1 to 50 of 56

Thread: has anyone written to Choice mag about their flawed testing?

  1. #1
    Senior Member
    Join Date
    Mar 2007
    Posts
    145

    has anyone written to Choice mag about their flawed testing?

    Gene Cafe Coffee Roaster $850 - Free Beans Free Freight
    Gday all
    I was thinking about writing to Choice after yet another frustrating read with the latest round of Espresso machine testing in the magazine I got today. I should have known better really, and not read it, but its like watching a car crash - you know its going to be bad but just have to look...
    They make prominent mention of the fact that they only got an ok taste score for the Silvia, and that 4 of the most expensive machines rate at the bottom of their test. Everyone here no doubt is quite aware that you cant make espresso with stale beans without some kind of help (eg double-floor baskets on entry level machines) but the average consumer is probably thinking that a good machine is a waste of money after reading the writeup.

    IMO its very poor on their part to omit mentioning that machines without pressurised baskets *must* be used with fresh beans, which are probably no more expensive than the supermaket beans and that fresh beans will *completely* change the results of the test. The blokes from Barista Basics should be well aware of that Im sure and Id have thought theyd want it explained for the sake of protecting their credibility - maybe Choice is afraid of upsetting the supermarkets if they started recommending people dont buy there?

    Their only mention of bean choice was "Better results can be achieved with better quality beans" but that on its own wont mean anything to an uninformed consumer as to how the results would be affected.

    Anyway I dont normally (or ever) write to magazines, but before I write to explain how they could make the testing not so completely flawed, so that their readers can make a more informed decision, I thought Id check if anyone else here has already done that...???

  2. #2
    A_M
    A_M is offline
    Guest
    Join Date
    Mar 2006
    Posts
    6,381

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 614D4D524750141B71220 link=1299838896/0#0 date=1299838896
    Anyway I dont normally (or ever) write to magazines, but before I write to explain how they could make the testing not so completely flawed, so that their readers can make a more informed decision, I thought Id check if anyone else here has already done that...???
    I have been known to comment and feedback on many web pages when I find errors and have commented as to teh content as to reviews on teh odd occasion. Even as far as comments to newspapers etc.

    Some times you get a Ack but often not.

    So what is the worse that can happen... No response; I say GO for it ;)

  3. #3
    Senior Member Dennis's Avatar
    Join Date
    Feb 2007
    Posts
    4,512

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 537F7F607562262943100 link=1299838896/0#0 date=1299838896
    The blokes from Barista Basics...
    Mate, have you checked out their website? Go there, part with $250, and 5 hours later you too can walk out with your head held high, because you will walk out with a "Master Barista" piece of paper.

    Sorry, know its OT, but am so sick of shonky "Barista Training" courses. >:(

  4. #4
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    I take the theory that all current affairs, and "generalist" style magazines (such as choice) are misinformed 9/10 times. The things is, you only notice when its something you specialise in. Assume they are always blowing smoke out their a**. They can always find an "expert" to feed them information that sounds good.

    They are there to make money by selling magazines. Take this scenario - basically all fridges last forever, so take a round up of fridges, say one of them is the best, and everyone goes and buys it. They are happy with it, and assume they were led well by Choice. They then take everything as gospel.

    If you ignore them as much as theyll ignore you, itll probably work well :)

  5. #5
    Senior Member
    Join Date
    Nov 2010
    Posts
    369

    Re: has anyone written to Choice mag about their flawed testing?

    How many coffee drinkers in Australia?

    Probably several million.

    There are about 14,000 Australians on this site (assuming there are a few non Australians that are members).

    Does this mean that less than 2% of coffee drinkers really care enough to pay attention to detail?

    For the remaining 98% of people cheap machine and cheap coffee will keep them happy.

    Actually a proportion of those people probably drink instant coffee!

    After all there is a lot of indifferent to poor coffee that is sold and the fact that people can make a living from producing rubbish suggests that there must be customers that are happy to buy rubbish.

    Is choice written for the 98% of people who are too lazy or too stupid to research something for themselves?

  6. #6
    Senior Member
    Join Date
    Jul 2006
    Posts
    150

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 133239393E24570 link=1299838896/2#2 date=1299840465
    Quote Originally Posted by 537F7F607562262943100 link=1299838896/0#0 date=1299838896
    The blokes from Barista Basics...
    Mate, have you checked out their website?* Go there, part with $250, and 5 hours later you too can walk out with your head held high, because you will walk out with a "Master Barista" piece of paper.

    Sorry, know its OT, but am so sick of shonky "Barista Training" courses.* >:(
    Years ago, Before i started working in espresso I got one of these courses as a gift. I walked out after the coarse feeling a little stupider and feeling sorry for the other people in the class who thought they had learned something.

  7. #7
    Senior Member Rusty's Avatar
    Join Date
    Apr 2004
    Location
    Carlingford NSW
    Posts
    691

    Re: has anyone written to Choice mag about their flawed testing?

    Hey... they get it right when they include cord storage as a pro or con* :D

  8. #8
    TC
    TC is offline
    .
    Join Date
    Oct 2004
    Location
    Melbourne
    Posts
    14,665

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 597E787F720B0 link=1299838896/6#6 date=1299966460
    Hey... they get it right when they include cord storage as a pro or con* :D
    Just goes to prove that they should stick to reviewing vacuum cleaners ::)

  9. #9
    Senior Member Rusty's Avatar
    Join Date
    Apr 2004
    Location
    Carlingford NSW
    Posts
    691

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 326D756368636F666665656D616E000 link=1299838896/7#7 date=1299971273
    Just goes to prove that they should stick to reviewing vacuum cleaners
    Are you suggesting they suck ;)

  10. #10
    Sleep is overrated Thundergod's Avatar
    Join Date
    Jul 2006
    Location
    Sydney
    Posts
    10,496

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 5269766E6374060 link=1299838896/5#5 date=1299956958
    Years ago, Before i started working in espresso I got one of these courses as a gift. I walked out after the coarse feeling a little stupider and feeling sorry for the other people in the class who thought they had learned something.
    I did a course a few years ago, not to become a barista (though my certificate says I am now) but to get a basic understanding of the machines so I could make an informed choice about what I was going to buy for myself.

    Despite my father having an espresso machine in his shop about 50 years ago, I was a little young at the time to know my way around it. ;)

    So for me the course was a good starting point.
    It did its job and I came away with a better idea of what I wanted in a machine.

    I do remember feeling strange being the only one there not trying to get into the industry. :o

    Since then Ive learned a lot.
    I think it would have been a lot more difficult had I not learned those basics.
    CS and a few industry professionals have built on that base and I know a little bit more now.

    For those that dont know, I landed a job in a cafe for a short while, not on the strength of the certificate but on the passion I showed for coffee.


  11. #11
    Senior Member Rusty's Avatar
    Join Date
    Apr 2004
    Location
    Carlingford NSW
    Posts
    691

    Re: has anyone written to Choice mag about their flawed testing?

    I was recently in the market for a couple of split air-cons so I took a three months sub of Choice Online.

    I was hopeful that it had improved over the ten years since I last subscribed but that was misplaced optimism Im afraid :-[

    I just looked at their review of 14 domestic grinders and to my surprise they each had a score assigned under the heading "Timer". There was no explanation of what this meant but apparently the whole 14 had it ::)

  12. #12
    Senior Member
    Join Date
    Jul 2006
    Posts
    150

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 755254535E270 link=1299838896/8#8 date=1299972662
    Quote Originally Posted by 326D756368636F666665656D616E000 link=1299838896/7#7 date=1299971273
    Just goes to prove that they should stick to reviewing vacuum cleaners
    Are you suggesting they suck* ;)
    Dont want to hijack this thread, but........
    This WAS quite a few years ago so I dont want to mention names but where I was sent was a bit wacky, and lots of Barista Basics were not even touched on.

    It was pretty much a full day, and people in the group mostly had no experience.* A few were opening their own cafe.* I had been making coffee at home and doing some roasting for a few years.

    My issues were - No discussion of cleaning and maintenance, no discussion of bean types or flavours, or the roasting process (beyond showing us some greens).* No mention of espresso flavours orhow to get a good shot beyond the 30ml in 30sec rule (and it was presented as a RULE).* No one was encouraged to try the espresso shots!!!!* :o* :o :o :o :o :o :o
    (Didnt taste that great anyway, probably stale beans and they had these san marino machines that seemed to be running real hot).

    I remember clearly when we were starting pulling shots that I asked about ristrettos and tried to talk about my experiece at home pulling ristrettos and how experimenting with grinding finer and/or dosing higher gives a good result, the instructer nodded and loudly told everyone aw yeah - you get a ristretto shot by pushing this button - the same thing as if someone orders a weak coffee.**


    I felt that there was way too much time spent on things like latte art (mostly making pretty looking etched pictures and spiderweb designs with choc syrup).* No time was spent talking about the flavours of coffee.* Nothing was mentioned about how a dirty group, or stale beans, or how leaving ground* coffee in the doser can make coffee taste bad.* Nothing was mentioned about not re-heating milk over and over and food safety issues like that.

    Mainly there was no emphasis on quality, seemed like all the important stuff was pushed through to cover the tafe certificate requirements, and then you get to spend a few hours making pretty looking zig zag patterns with chocolate and a toothpick.*

  13. #13
    Junior Member
    Join Date
    Oct 2010
    Posts
    15

    Re: has anyone written to Choice mag about their flawed testing?

    Being in the market soon, I looked at the article and wondered where they are coming from.

    Quote Originally Posted by 0B544C5A515A565F5F5C5C545857390 link=1299838896/7#7 date=1299971273
    they should stick to reviewing vacuum cleaners
    My father has been getting choice for as longer than I can remember. Many of the reviews suck, and they have become more superficial over time, especially the last decade or so. However we need someone representing we consumers, and there aint many who do that.
    Im sure the more feedback, or lost customers choice gets, the more theyll adjust.*

    So Cooper69S, go for it. (how do I link the name without a quote?)


  14. #14
    Coughy
    Join Date
    Apr 2007
    Location
    Adelaide Hills
    Posts
    79

    Re: has anyone written to Choice mag about their flawed testing?

    I had a laugh, Ive commented on a few of their machine tests, pointing out some various numbers of flaws in their methodology (beans, grinders, baskets...) and ok maybe they got a bit upset at my pickyness....but they deleted my comments!!!!

  15. #15
    Sleep is overrated Thundergod's Avatar
    Join Date
    Jul 2006
    Location
    Sydney
    Posts
    10,496

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 052238293E0F232A2A4C0 link=1299838896/13#13 date=1300074277
    they deleted my comments
    So whats the point of letting you comment in the first place?

  16. #16
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 1824392228293E2B23284C0 link=1299838896/14#14 date=1300080228
    So whats the point of letting you comment in the first place?
    Harvest 100 positive comments, 100 negative comments, delete the negative comments, and everyone can see how wonderful you are :)

  17. #17
    Senior Member
    Join Date
    Oct 2010
    Posts
    483

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 4956544E57425554433B0 link=1299838896/15#15 date=1300087254

    Harvest 100 positive comments, 100 negative comments, delete the negative comments, and everyone can see how wonderful you are :)
    Not quite. Leave in a handful of negative comments that are easily addressed so you can demonstrate your commitment to continuous improvement. ;)

  18. #18
    Senior Member
    Join Date
    Nov 2006
    Posts
    1,673

    Re: has anyone written to Choice mag about their flawed testing?

    There was a thread here a couple of years ago rtegarding this very thing and one of the editors tried to make excuses for their poor performance.

    Was a very good stoush at the time ;D I couldnt find it so it may have been culled.

    Basically I cancelled my subscription based on their flawed reviews on coffee machines as I dont think they apply much rigour to their testing of all products.

  19. #19
    Sleep is overrated Thundergod's Avatar
    Join Date
    Jul 2006
    Location
    Sydney
    Posts
    10,496

    Re: has anyone written to Choice mag about their flawed testing?

    Id subscribed for years but it was after a couple of obviously poor reviews of coffee machines (Silvia lost points for no cord storage) that I too cancelled my subscription.

  20. #20
    Mal Dimal's Avatar
    Join Date
    Oct 2004
    Location
    Warwick, QLD
    Posts
    16,878

    Re: has anyone written to Choice mag about their flawed testing?

    I believe that the fact of their reticence to receive positive critique of the information contained in any of their reviews speaks louder than anything else might. I tried many, many years ago to offer factual scientific assistance with a review I read at that time. All I got for my efforts was an abrupt equivalent get lost statement from their then public relations spokesperson and that was enough for me.

    As TG and Cindy, I cancelled my subscription right then and there.... >:(

    Mal.

  21. #21
    Senior Member
    Join Date
    Mar 2007
    Posts
    145

    Re: has anyone written to Choice mag about their flawed testing?

    finally got around to writing to them, probably a bit wordy, and Im sure some of you could have written a better critique, but oh well...*
    ccd it to the "Barista Basics" mob too for good measure

    Ive been a choice subscriber for many years and I enjoy the tests and reviews that you do.* However, Ive noticed a recurring problem with your espresso machine tests due to your choice of beans for testing, which bothered me enough that I felt compelled to write to you about it (I’ve never written to a magazine about anything before).* This response shouldn’t be seen as a reader “upset because their machine didn’t rate well”, as although I have used machines at every price level including some that you have tested, my current machine is outside the range tested in this comparison.

    Basically, the tests are very significantly biased towards low end machines due to their use of pressurised (dual-floor) filter baskets.* As I’m certain your testers are aware (as they must be as barista trainers), it is nigh impossible to extract good tasting espresso from stale coffee beans using a traditional espresso machine.* Anyone who has ever compared fresh beans with typical supermarket-sourced coffee will know by experience that the quality of espresso extraction starts to deteriorate noticeably after the beans reach an age of 3-4 weeks after roasting.* Makers of low end machines obviously realise that typical buyers of entry-level machines will buy their beans from the supermarket (even though fresh beans bought online or from a local café or roaster are often no more expensive), so they supply the machines with pressurised baskets which slow down extraction of stale coffee and give the espresso the appearance of having crema.*

    So your test results are valid in a certain limited sense – such as if fresh beans weren’t readily available, or for people who think buying coffee from anywhere other than the supermarket would be too much hassle.* In reality though, fresh coffee can be had by anyone, anywhere, with roasters delivering coffee beans to-the-door within a couple of days of being roasted if you’re not able to pick them up from the roaster yourself.* *It is undeniable that if the test was done using freshly ground fresh coffee beans, that traditional style espresso machines, using standard single-floor filter baskets, would produce better tasting espresso than machines using a double-floor basket.* *

    You did mention in the review that better results can be achieved by using better quality beans.* Being quite a vague statement, the simple and likely interpretation of that comment is that better beans will improve the taste of the espresso to the same degree regardless of which machine is used.* Without any further clarification, there is no way for an uninformed reader to know that only fresh beans are quality beans,* and that good fresh beans would totally change the results of the test, moving good machines equipped with single-floor filter baskets much further up the list.* *

    This is a major omission, as based on the results you’ve given, people will be led to believe that spending more on a higher level machine is a total waste of money, convincing them instead to buy a basic cheap machine.* I know because I went through this process several years ago, buying a mid-range Sunbeam machine after having been impressed by the review in your magazine, only to realise later that much better results can be achieved with a better machine, a good grinder and fresh beans.* So people (including myself) end up wasting money on low-end machines, in the false belief that the more expensive machines produce inferior coffee.* *Anyone who has used different types of machines will know this from experience, and I’m surprised that your testers didn’t make a point of mentioning this in the review, if only for the sake of protecting their credibility.

    Testing machines with supermarket coffee, which were not designed for use with supermarket coffee, and then rating them as ‘ok’, ‘poor’ or ‘borderline for making coffee’ is unreasonable in my opinion.* I would urge you to include some more balanced results, using fresh coffee, in future tests so that your readers are aware that although espresso quality isn’t determined by the price of the machine, spending more on the machine *can* yield superior results as long as they’re prepared to source suitable beans.*

  22. #22
    Senior Member Rusty's Avatar
    Join Date
    Apr 2004
    Location
    Carlingford NSW
    Posts
    691

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 436F6F706572363953000 link=1299838896/20#20 date=1300288956
    finally got around to writing to them, probably a bit wordy, and Im sure some of you could have written a better critique, but oh well...
    ccd it to the "Barista Basics" mob too for good measure
    Well done!

    A good critique of the situation very well composed :)

  23. #23
    TC
    TC is offline
    .
    Join Date
    Oct 2004
    Location
    Melbourne
    Posts
    14,665

    Re: has anyone written to Choice mag about their flawed testing?

    A really well written letter Cooper69 [smiley=thumbsup.gif]

    I would doubt that it will make much change though because when it comes to espresso machines, Choice seems to target the lowest and uneducated demographic.

    The testing is flawed. You take a machine and rate it according to the features it has and then weight them more heavily than the output of the machine- i.e. the coffee.

    I perceive a problem with some of the testing panel too in that their company is in the business of creating 3 hour trained baristas and then selling them the goods that they use in the course that gave them a certificate proclaiming them to be "trained".* My opinion is that* its not about the coffee, rather the buck and certainly that was my experience a few years ago when I had to have the dubious pleasure of pulling shots for them at an event.....and in the words of Forrest, "thats all I have to say about that".* ::) :-X

    Without a change in testing panel and a revised brief from Choice, I cant see that much will change. Its sad that they dont run this test at anything more than basic level and seemingly employ testing panel members who are prepared to sell out.

    2mcm

  24. #24
    Senior Member
    Join Date
    Apr 2008
    Posts
    115

    Re: has anyone written to Choice mag about their flawed testing?

    Nice work Cooper69S

    Be interesting to see if you get any response!

  25. #25
    Senior Member
    Join Date
    Mar 2007
    Posts
    145

    Re: has anyone written to Choice mag about their flawed testing?

    got a fairly cheerful response. They say they do use fresh beans, but it doesnt explain how that contrary to what you would expect, basic dual-floor basket machines consistently win their tests...

    apparently theyre going to make more prominent mention in the future of the importance of fresh beans though, so thats one good thing at least...


    Dear Geoff,

    Thanks for your well worded comments.

    When an espresso machine is supplied with single floor and dual floor filters, we use the single floor filter. As you’ve noted, manufacturers came up with the idea of dual floor filters so that you end up with crema, regardless of what type of beans you use. You can use pre-ground coffee from a supermarket and still end up with crema, however, we dont think the resulting coffee will be palatable. We grind the coffee beans just before making the shots for the taste test regardless of what type of filter we use.

    A bean that is nationally available is through supermarkets gives us a level playing field in which to test all espresso machines - aside from pod delivery machines. We believe the beans we get are within 3-4 weeks freshness. You mention that people often dont want to hassle of going to a bean supplier outside of their supermarket - this is true; some people are not even aware that better quality beans are offered outside of their local supermarket - we surveyed our subscribers and found that most of them source their beans from the supermarket. As you know, it’s the freshness of the bean that is a priority. This is not as prominent on our article as Id like, so Ill update this.

    We reiterate in every magazine article we publish on the espresso machines that using a better bean will give a better result, and youll find it on the front page of our online article at the top of the page – and I’ll make it more prominent with the freshness comment.

    We link directly to appliance reliability scoring, which we direct to consistently in our comments when commenters ask for direction between brands. These results show a great direction for consumer on what brand to lean towards, and away from - they are not our judgements, they are subscriber assessments. I hope this goes some way to answering your queries.

    If you have any further queries do not hesitate to contact Choice on 1800 069 552 or (02) 9577 3399.

    Regards,
    CHOICE Consumer Information

  26. #26
    Super Moderator Javaphile's Avatar
    Join Date
    Dec 2004
    Location
    Earth!
    Posts
    15,667

    Re: has anyone written to Choice mag about their flawed testing?

    As you’ve noted, manufacturers came up with the idea of dual floor filters so that you end up with crema, regardless of what type of beans you use. You can use pre-ground coffee from a supermarket and still end up with crema
    No, you end up with fake crema.

    We believe the beans we get are within 3-4 weeks freshness.
    Three to four week old beans are fresh?!

    Of course its not like theyll use truly fresh beans properly prepared otherwise none of their paying advertisers machines would stand a chance against a real espresso machine.


    Java "Someones ignorance is showing" phile

  27. #27
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 4F647364756D6C6960050 link=1299838896/25#25 date=1300420707
    Three to four week old beans are fresh?!
    You misread. They said "We believe". As far as Im concerned, "Believe" is about as strong as "assume" or "think".

    ie they dont really know, so they just bought what their misinformed users will buy.

  28. #28
    Senior Member
    Join Date
    Oct 2010
    Posts
    483

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 504F4D574E5B4C4D5A220 link=1299838896/26#26 date=1300433104
    e they dont really know, so they just bought what their misinformed users will buy.
    Which makes it a valid test for their target audience.

    I never subscribed to Choice, but I have taken a few of their free trials and used their reports for some white goods purchases. Mainly I looked at their reliability testing and running costs rather than anything else as I didnt really trust them. Seems I was right.

  29. #29
    Senior Member
    Join Date
    Nov 2010
    Posts
    369

    Re: has anyone written to Choice mag about their flawed testing?

    I believe in ****

    Does this make it true?

    Probably not in everybody elses eyes.

    Different people have different selection criteria

    Some people want a machine that makes good coffee but the point that often gets lost in this situation is that it is the operator that makes good coffee.

    The best equipment on the planet that is used poorly will only give average to poor results.

    Some people want a machine with cord storage or that looks good. ::) :-/

  30. #30
    Senior Member
    Join Date
    Mar 2007
    Posts
    145

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 474C4D7C4D424A514D230 link=1299838896/28#28 date=1300473209
    Different people have different selection criteria

    Some people want a machine that makes good coffee
    some? I think all people who buy a machine want it to make good coffee, otherwise they wouldnt buy it.* of course everyone has a different definition of good though

    Quote Originally Posted by 474C4D7C4D424A514D230 link=1299838896/28#28 date=1300473209
    but the point that often gets lost in this situation is that it is the operator that makes good coffee.

    The best equipment on the planet that is used poorly will only give average to poor results.
    I dont think that point is lost here.* you buy a machine that fits your budget and other requirements and makes coffee that youre happy with.* *The idea of this testing is that the same operator is using each machine, which *should* make it a level playing field.* To the average reader, theyll see that choice has chosen "professional barista trainers" to do the testing.* surely that means that theyre getting the absolute best out of each machine doesnt it???* so these tests conclude that with the "best" operator, the cheap machine still produces a (much) better brew than the expensive machines.* We all know that you can get a good coffee from a cheap machine, but what we also know is that contrary to their test results, you *can* get a better coffee from some of the more expensive machines they tested (all else being equal).

    One point that highlights the problem, is that of the 4 very similar Sunbeam machines in the test, EM3600, EM3800, EM4800C and EM5600, they rank the EM5600 as best of all tested for taste (score 75%), EM3800 3rd (70%), but the 4800 and 3600 are right down the bottom in 22nd and 23rd place (both 35% taste score), despite these last machine being rated with better coffee temperature scores and better temperature consistency scores than the first 2.* Youd think thered be a red flag there somewhere that somethings up with the testing...

    anyway Ive said my bit, got their response, and had now better stop worrying about it for the sake of my sanity...

  31. #31
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 220E0E110413575832610 link=1299838896/29#29 date=1300503964
    some? I think all people who buy a machine want it to make good coffee, otherwise they wouldnt buy it.* of course everyone has a different definition of good though
    I know people who have bought machines because they are "shiny", put them on their bench and only use them when they have guests (maybe once a month). They put the preground frozen coffee on the basket, dont tamp, and turn the machine on until its full enough in the mug to add a splash of milk.

    Some people buy coffee machines simply because they think they need to have one (consumerism fail).

  32. #32
    Senior Member
    Join Date
    Nov 2010
    Posts
    369

    Re: has anyone written to Choice mag about their flawed testing?

    The marketing people suggest there is a thing called "a public display" such as people drink cheap alcohol in private but if the boss is visiting they bring out the good stuff.

    I have a fancy coffee machine I must be ***** (even if I drink instant most days and dont know how to make coffee with it).

    To make good coffee you have to pay some attention to detail.

    It seems many people this is too hard or they cant be bothered.

    Is this confirmed by the number of dumbed down products that the only redeeming feature is that they are easy (and maybe quick)?

    After all junk food is quite popular but many people that produce junk food would conceded it id not the finest tasting or most nutritious food.

  33. #33
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 6269685968676F7468060 link=1299838896/31#31 date=1300506073
    Is this confirmed by the number of dumbed down products that the only redeeming feature is that they are easy (and maybe quick)?

    After all junk food is quite popular but many people that produce junk food would conceded it id not the finest tasting or most nutritious food.
    People, are intrinsically lazy. I personally rarely eat fast food and cook everything I can from scratch (tomato sauces, pizza bases etc). Most people I know only cook at home because its cheaper. If they had more money, theyd never cook themselves.

    Same goes for coffee. Theyre lazy, but theyll go to great lengths to impress others ie "Look what Ive got". If people lived 20 years behind their time, theyd feel like kings. I feel this is why a lot of elderly people can live on a shoestring budget and be happy, and conversely, why a lot of middle aged and younger people are so damned stressed all the time.

  34. #34
    Senior Member
    Join Date
    Aug 2007
    Posts
    440

    Re: has anyone written to Choice mag about their flawed testing?

    The one thing I think the average consumer out there needs to realise is that, "They can produce quality coffee equal to the best cafes at home."

    Im almost certain people beleive that it is difficult and near on impossible to make a coffee as good as the cafes. I think they buy their machines and buy the best of the supermarket quality. They refine the process and learn to get the best out of what theyve got and accept that as a great alternative to quality coffee from a cafe.

    Perhaps that is where the education needs to start. Once they know that they might be a little more critical when they read reviews and buy that recommended machine. They should let their taste buds decide. For most it is enough to say that I did the research and bought the best machine and the best grinder because choice said so.

  35. #35
    Senior Member
    Join Date
    Jun 2010
    Posts
    107

    Re: has anyone written to Choice mag about their flawed testing?

    Justin B, I think it has to do with what tickles people. I am interested in coffee roasting and drinking. I am also interested in guitars and modifiying them, and also in bicycles and bicycle racing. I am happy to invest a lot of time and whatever dollars the wifey allows me into these things because I get a lot of pleasure from these things.

    Most other things I dont care much to learn the ins and outs. Im sure I could make great cheese at home. Im sure I could bake healthier and tastier bread and biscuits. Im sure I could build a better computer than I can buy for the same $$. But I cant be stuffed. There is only so much free time you can use, and I channel mine into those things I get the most pleasure from.

    Your mileage may vary.

  36. #36
    Senior Member
    Join Date
    Aug 2007
    Posts
    440

    Re: has anyone written to Choice mag about their flawed testing?

    Not entirely sure what you are getting at but I guess this is Coffeesnobs and I do feel that you are missing the point. The thread was about coffee machines and Choices reviews. Presumably the person reading the review and buying the machine has decided to take that step. What I dont understand is why people would want to be half arsed when they do it. It doesnt take any more effort to make a quality coffee as it does a rubbish coffee.

  37. #37
    Senior Member
    Join Date
    Jul 2008
    Location
    Port Fairy
    Posts
    2,993

    Re: has anyone written to Choice mag about their flawed testing?

    Going back on topic ::)

    Seems Choice have just tested milk on consumers and sure enough they cant pick the difference between cheap and expensive. Coles and Woolies must be laughing all the way to the bank for the free plug for cheap milk at the farmers expense YET again.

    Put the milk to some real scrutiny WHAT IS ACTUALLY IN THE CHEAP STUFF to make it sort of like normal milk.

  38. #38
    Senior Member
    Join Date
    Nov 2010
    Posts
    448

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 65626669616B7E6E6960070 link=1299838896/36#36 date=1300785593
    Seems Choice have just tested milk on consumers and sure enough they cant pick the difference between cheap and expensive.
    Did they put it on Coco Pops (because thats how most of their reader base probably use it)?

  39. #39
    Senior Member
    Join Date
    Jul 2008
    Location
    Port Fairy
    Posts
    2,993

    Re: has anyone written to Choice mag about their flawed testing?

    Not sure about the CoCo pops but so long as their researchers were wearing their Today Tonight white lab coats to ensure accuracy of the results I suppose we must believe them ;D

  40. #40
    Senior Member
    Join Date
    Aug 2007
    Posts
    440

    Re: has anyone written to Choice mag about their flawed testing?

    Thing with milk is that often it does taste fine chilled and out of the fridge. Heat it up and thats where the difference is in my opinion. Most people arent making coffees with their milk though.

  41. #41
    Member
    Join Date
    Dec 2010
    Posts
    67

    Re: has anyone written to Choice mag about their flawed testing?

    An interesting range of inputs in this thread. One point that has not been mentioned is the criterion of complexity of use and the need for a fundamental understanding of a process that is inherent in any piece of equipment or technology. My guess is that Choice itself has issues with this and its possible that they make certain unstated assumptions about the wider populations inherent tendency to favour simplicity.

    An interesting case study in my own family might clarify what Im saying: my brother in law is a barrister who can afford expensive toys, and he bought a Sunbeam espresso machine primarily because it was easy to operate and it warms up very quickly (like 5 minutes). He would regard my Silvias warm up time of 25 minutes as a huge problem, and his pod-based coffee making makes absolutely no demands on him to understand what is a good coffee extraction and the processes that produce a good shot which leads to great coffee. For him, the range of options would never have included the Silvia because it actually requires one to master a range of steps, processes, and understandings of coffee-making parameters that most would regard as to much effort.

    Hence Choice reviews tend to cater for the masses; if they presented a more balanced review, there is just as much chance that many could buy a Silvia and produce awful coffee because they will never understand how it works.

  42. #42
    Senior Member
    Join Date
    Aug 2010
    Location
    Newborough, Latrobe Valley, Victoria
    Posts
    210

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 416D6D726770343B51020 link=1299838896/29#29 date=1300503964
    some? I think all people who buy a machine want it to make good coffee, otherwise they wouldnt buy it.* of course everyone has a different definition of good though
    Ignorance is bliss, and most coffee drinkers out there drink instant, so pretty much any machine with any beans, even stale pre-ground supermarket beans, is an improvement - I should know, I was there only 18 months ago!* I know Barista Basics has been mentioned... did they run the tests, or just assist Choice?* You can kinda understand if in-house staff at choice (i.e. jack of all trades, master of none types) can stuff up - or misprepresent - this sort of test, but you would hope an "expert" in the field would get it right.

    Many years ago I worked for Harvey Norman, and Choice rarely failed to raise eyebrows with their recommendations - clearly the products were assessed by an average Joe, not someone intimately familiar with the product.* It was funny, you would get certain types come in to purchase an item, Choice mag under their arm, and basically say "I want this one, cos Choice told me its the best".

    P.S. I saw the packet from the beans they used for the Choice test, and I can confirm that the beans were fresh... the Best Before date on the back of the pack was months away* ;D

  43. #43
    Senior Member
    Join Date
    Mar 2008
    Posts
    554

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 45404C48444F4916210 link=1299838896/41#41 date=1301003743
    but you would hope an "expert" in the field would get it right.
    You could think that, but what passes as an "expert" can be deception at best. EXAMPLE:
    There is a food critic and wine expert in Israel who is widely known and respected in Europe from all accounts. His name is Daniel Rogov. A few years back he published online an article he wrote for some mag or presentation on coffee. It was ridiculous. I went to his website and replied, breaking down the article, point by point. It took some time. There were statements like this, in this case referring to espresso: "Steam is forced through the coffee..."

    My response pissed off a load of his devoted followers (more accurately, devotees, or even his "flock"), and eventually he took the article down and closed and removed the comment area on the article.

    My belief was that he had some intern(s) do the research and they used sources that were at least 30 or 40 years old. he just took the info, did* a quick reread and edit, and posted it with his name on it.

    I really got his goat when I told him that there were more identified taste elements in espresso than in wine.* Anyway, the original article is available HERE on the Wayback machine...

    I suppose that we all have to be careful not to fall inline with "false authority."


    (fixed the link - Andy)

  44. #44
    Senior Member
    Join Date
    Aug 2010
    Location
    Newborough, Latrobe Valley, Victoria
    Posts
    210

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 6C5F505A476179103E0 link=1299838896/42#42 date=1301005837
    You could think that, but what passes as an "expert" can be deception at best.
    Thats why I used the inverted commas around "expert". The vibe I picked up from other comments is that they are not at the top of the field OR possibly are happy to have their name bought to add some perceived authority to Choices article (I hope thats not being too harsh - again, Im not familiar with them).

  45. #45
    Sleep is overrated Thundergod's Avatar
    Join Date
    Jul 2006
    Location
    Sydney
    Posts
    10,496

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 6F747E757C6F797C717C7F7C1D0 link=1299838896/40#40 date=1301002347
    he bought a Sunbeam espresso machine primarily because it was easy to operate and it warms up very quickly (like 5 minutes). He would regard my Silvias warm up time of 25 minutes as a huge problem, and his pod-based coffee making makes absolutely no demands on him to understand what is a good coffee extraction and the processes that produce a good shot which leads to great coffee.
    Just because the machine says its ready doesnt mean it is.
    There was a Sunbeam where I worked and the first thing I did on arrival every morning was to turn it on so that 30 minutes later it would be ready to make a decent coffee.

    From the rest of the description about your B-I-L I doubt hed care to try get the best out of his machine as he seems to be content with its minimum output standard.

    His loss. :(

  46. #46
    Junior Member
    Join Date
    Nov 2008
    Posts
    28

    Re: has anyone written to Choice mag about their flawed testing?

    Quote Originally Posted by 173B3B243126626D07540 link=1299838896/0#0 date=1299838896
    Anyway I dont normally (or ever) write to magazines, but before I write to explain how they could make the testing not so completely flawed, so that their readers can make a more informed decision, I thought Id check if anyone else here has already done that...???
    Yep, and its a pity more ppl dont take the time to do so. So do so! :) Choice seems to have lost much of their integrity in recent years.

  47. #47
    Senior Member
    Join Date
    Mar 2007
    Posts
    145

    Re: has anyone written to Choice mag about their flawed testing?

    I have finally had a positive response from Choice about these tests after posting a lengthy coment on their website after the last test, and now it looks like future tests might be a little bit more accurate than they have been so far.

    You make very valid points about the coffee we use in our tests. The beans we use are purchased from the supermarket. We use these beans because they are readily available for everyone. However, access to fresh coffee has become quite simple nowadays so we are going to review our test method when our next test comes up, particularly the issue of the coffee beans used.
    To answer a couple of your other questions, we adjust the grind and dose to suit each machine and we always use the single floor filter basket if both types are supplied.
    Regards, Rebecca
    I have no idea how they ever expected to get a decent coffee from supermarket beans without using dual-floor baskets. (with dual floors, you can maybe get a passable coffee by non-CS standards). Also, she says they used single floor when both were available. Given the dual floor is designed for exactly what they were doing, it would surely have made sense to use it???

    Also theyve done roaster tests and theyre giving Andys Behmor and green beanbay some publicity so well have to be nice to them now :P

  48. #48
    Senior Member
    Join Date
    Oct 2011
    Location
    Gold Coast
    Posts
    223

    Re: has anyone written to Choice mag about their flawed testing?

    I dont know how they are now but Choices testing was always considered by professional statisticians to be flawed. Most product testing didnt follow accepted practises used in good experimental and survey design. As a result their tests would never be considered repeatable. A colleague who was employ for a while in their testing program was told cost of operations limited their testing options. I doubt if things have changed.

    The one time we used their results (to choose a front loader washer) it turned out a big dud. It functioned perfectly as designed but just didnt spin fast enough and couldnt wash effectively. Later models of the same brand are nearly identical but designed to spin faster so we know it wasnt our technique. But according to Choice it topped the charts. It wasnt cheap either.

    I dont trust anything they have to say. Real world info and feedback from peer oriented forum like CS are far superior.

  49. #49
    Senior Member Mariner's Avatar
    Join Date
    Jan 2012
    Posts
    217

    Re: has anyone written to Choice mag about their flawed testing?

    I think that a lot of people may be missing the point here... We are snobs who have a MUCH higher standard of what we expect of a coffee, do we not? How many people reading this forum have gone to a friends house and had a coffee that is absolutely awful - but their friend thinks is very good and quite acceptable, OR, even to a cafe and had a poor cup???

    It takes a love and passion for what we are interested in to make great coffee IMHO - traits that MOST people will never share with us (case in point, most of my friends). It is quite easy to turn out a good shot when you have taken the time to practice and learn the required skills to do so. There is far more effort and work required to make good coffee than bad.....the cleaning regime I now undertake takes up far more time than most people would be willing to undertake to enjoy good coffee. It is not hard work for me or for any of you I am sure but, it would be for most.

    Quote Originally Posted by 594248434A594F4A474A494A2B0 link=1299838896/40#40 date=1301002347
    Hence Choice reviews tend to cater for the masses; if they presented a more balanced review, there is just as much chance that many could buy a Silvia and produce awful coffee because they will never understand how it works.
    Well said. Choice Mag. are not trying to sell coffee machines to snobs like us who are more discerning. As far as I know people here would just read a review out of interest would they not? I would not buy a $10,000 mountain bike if I just wanted to ride around the block. I would not buy a $5000 sextant to navigate a yacht around the harbour....get my drift. However, if I bought a cheap bike and discovered that I really liked riding it around the block and my interest was sparked then maybe my next bike would be a much better quality. Maybe if I enjoyed navigating and wanted to sail a ship around the world I would probably enjoy and need a very good sextant to do so...

    It is very easy for any of us to look at the world through our own perspective of the universe - I do all the time. Do I agree with Cooper69s? Absolutely I do! (Very well written letter by the way). But, I think that for people who from their opinions about espresso machines through Choice magazine, the article was probably quite Ok.

    By the way, I have an EM6910 that makes me about 7-8 double shots a day when I am home and I most certainly cannot wait to upgrade it to a better one. ;)


  50. #50
    Senior Member
    Join Date
    Oct 2011
    Location
    Gold Coast
    Posts
    223

    Re: has anyone written to Choice mag about their flawed testing?

    Behmor Brazen - $249 - Free Freight
    I wouldnt say Choice caters for the masses or they would have enough money to employ valid testing regimes and I certainly dont think everyone who enjoys a good coffee is a member of this forum. Choice subscription and cs membership are obviously not mutually exclusive or this thread would not exist.

    Interestingly the choice website recommends using freshly roasted beans and grinding just prior to brewing.
    Well done cooper69s !



Page 1 of 2 12 LastLast

Similar Threads

  1. Choice chooses choice machines
    By robusto in forum General Coffee Related...
    Replies: 0
    Last Post: 30th September 2005, 03:39 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •