Text Size:-+
09.01.2009

Gold Medals Do Not Mean Good Wine: Actual Proof?

SFIWC-2009-Double-Gold-Medal-2 .gifBear with me while I get this out of the way:

I told you so.

I've taken a lot of flak here at Vinography for my stance on the competition, state and county fair medals that wineries like to make a big deal about. I think they're all bunk -- useless to the consumer, and a waste of money for the wineries trying to win them. For reference, I suggest you look at my posts entitled: Stop The State Fair Madness and Wine Competitions are One Big Racket.

My opinion has been based up until this point on purely anecdotal evidence that I can summarize quite simply. I taste thousands of wines a year at large wine tasting events, where (annoyingly) many producers advertise (and proclaim) their medal winning wines. The vast majority of the time, these wines aren't any good. And quite to the contrary, some of the time, these wines are positively awful.

But now, we have a modicum of statistical evidence to support my contention, thanks to the work of some folks at the American Association of Wine Economists. These folks have just published a paper entitled: "An Analysis of Concordance in 13 U.S. Wine Competitions," which not only demonstrates what I believe to be pretty clear statistical evidence for my previous claims, they also manage to cite a study I was unaware of suggesting that gold medals don't really increase sales to consumers in the first place!

The paper outlines an analysis of, among other things, 2,440 wines that were entered into three or more wine competitions around the country. 47 percent of these won gold medals (that fact along should ring alarm bells), but of those, 84 percent won ZERO medals (not even a bronze) in other competitions. Which means that while these wines may have been rated as among the very best wines in one competition, they were rated as below average in another. Even taking into account the differences in the field of competition, this is a rather damning indictment of the quality, relevance, and value of these competitions and their awards.

One of the interesting details of the study was that the only place that these competitions were concordant in their evaluations of wines were of wines that they did not like. There were groups of wines that consistently received no award or a bronze medal at these competitions. This suggests, as the author of the study has apparently published elsewhere, that the judges at these events really only agree on what they don't like.

The competition results that were studied in this report include:

Dallas Morning News Wine Competition
San Francisco Chronicle Wine Competition
Grand Harvest Awards
Jerry D. Mead's New World International Wine Competition
West Coast Wine Competition
Pacific Rim International Wine Competition,
San Diego National Wine Competition
Riverside International Wine Competition
Los Angeles County Fair
International Eastern Wine Competition
Orange County Fair Wine Competition
San Francisco International Wine Competition
California State Fair Wine Competition.

For anyone unfamiliar with this competition circuit, these are essentially the largest and most prestigious wine competitions in America, many of which employ lots of wine professionals as judges. According to the paper, wineries spend more than a million dollars on entry fees every year for these competitions.

So I'll say it again: stop the madness. I know that small wineries need every little bit of help they can get to sell their wines and get some attention from consumers, but these competitions are a lousy way to do that, and the awards do more to prop up the egos of those who enter than help consumers make relevant buying decisions.

Even with this paper in hand, I expect a volley of stones. Read it first, and then fire away.

An Analysis of the Concordance Among 13 U.S. Wine Competitions -- 161k PDF.

Comments (66)

Blake wrote:
09.01.09 at 9:55 PM

Alder: Editorially, I agree with you. I never mention gold medals when writing a story about a winery. But from the position of the wineries, particularly unknown ones, entering a few competitions makes sense. Some consumers do pay attention to medals.
Also, to be fair, I'm not sure competitions disagree more than critics. Compare Parker and Wine Enthusiast, for example (I know you have).
I have been a judge at competitions and I always try to be diligent. And I have had lengthy arguments with other judges on my panel over wines. Wine's just like that.
I personally don't find gold medals a compelling reason to buy a wine. But if every consumer was like me, wow, what a different world -- lots of zombie movies but no unoaked Chardonnay.

Here, here!
Down with Wine Competitions!
Up with Consumers Judging Wine 4 Themselves!

Fred wrote:
09.01.09 at 11:52 PM

Bravo, Alder, bravo. Far too many wineries mistake medals for marketing.

cj wrote:
09.02.09 at 12:34 AM

How right you are, Alder !!
It is no different in Europe. And most of the time it is a vain attempt to do something for the USP. But please excuse : Do you really believe everything Mr. P.... writes about Bordeaux ? Not every consumer has the chance of tasting the wine that interests him. But do we need "Advocates", or "Ambassadeurs" or whatever you name these self-proclaimed "Gurus" ?? The only taste that counts is your own !!!

Pedro Benito Sáez wrote:
09.02.09 at 1:31 AM

What happens is that some wineries send different wines to the competitions that the ones that are sold to the public, in order to win. The solution for the organizations will be to buy the wines at the wine shops on the other hand I don’t think they will be interested in bear that cost.

Most of the income from wine magazines and media comes from the wineries that specialises in marketing and spend important amounts of money in advertising, which in general are the ones that tend to win these competitions.

Steve Raye wrote:
09.02.09 at 3:24 AM

I think this is also a matter of perspective. If it were only about the wine quality, your...and your readers' points...are certainly valid. But for those of us in the business of marketing and selling wine, we look at competitions from the practical side.

Quite commonly, a retailer won't put a new wine in without a reason. And one reason retailers and distributors "accept" to validate a wine's quality is some sort of award. Granted your comments about the validity or value of the awards being next to useless is certainly true. But if we can't get the wine on the shelf in the first place, it's irrelevant. It's sort of the same deal with ratings. Retailers and distributors want to see third party validation of a wine's quality whether it be Parker, Wine Spectator, Wine Enthusiast, BTI or even the Grundy County Fair. And a related point, to promote a wine on shelf, the most useful tool next to a salesman recommending it personally, is a shelf talker. Short, simple, graphic communication to the consumer in the form of a big number or a big medal, gives them the confidence to choose a wine. Consumers aren't in a position to evaluate whether this rating is more valid than that, or this competition more prestigious than that. But they do need some validation that THIS IS A GOOD WINE. And medals do that. And unfortunately, when it comes to ratings, retailers are no more discriminating than consumers.

So sure, from your perspective, medals...and by extension ratings...don't have much validity. But commercially and practically speaking they do have great value as a communication tool for consumers.

So the real dichotomy here is between validity and commercial utility, and we have to look at both sides before throwing the baby out with the bathwater.

And as a friend of mine in the business told me, he gets asked a lot: "What's the best bottle of wine?" His answer..."The one I just sold."

1WineDude wrote:
09.02.09 at 4:29 AM

I've read the report, and while I might agree that competitions offer little more than entertainment value, the stats. and science behind the report are crap and don't at all support the conclusions reached in the report!

I've written an article about that, it will post tomorrow morning ET on my blog. The AAWE ought to be ashamed of themselves, in my opinion, on publishing what I think are pretty substantial errors in logical reasoning.

Those gold medals might be bunk, biut NOT for the reasons stated in the report.

09.02.09 at 6:33 AM

Of course, there's the other dirty secret about wine competitions: the many judges who aren't organoleptically or sensory trained, which means that their opinions are as valid as the guy off the street, which may give it more weight...

Speaking as one who is called upon to judge: I agree with Alder, and I've said it many times in print and on my blog. But I've also learned the power of inertia--try to change course in any institution and you will have to expect a long, nasty ride.

Alder wrote:
09.02.09 at 7:49 AM

Steve,

Thanks very much for the comments. Excellent points to be sure. As someone who has the luxury of being an armchair critic to this whole process, I think we should look for a solution that might be relevant to both the consumer and the winery. This may be the best we have at the moment, but in my opinion, it sucks.

Craig Camp wrote:
09.02.09 at 8:54 AM

Could not agree more. These medals have been worthless for selling to professional trade buyers (sommeliers, top retailers) for twenty years. They are just not something serious wineries should participate in.

09.02.09 at 8:54 AM

Alder, I also conduct a sometime-jihad against medal madness. But I still judge at the Michigan Wine Competition. Why?

Those of us in emerging regions with smaller wineries realize that our winemakers have few ways to reach their potential customers. The mainstream wine media -- and most wine bloggers -- ignore us. In-state print media are dying or their writers unschooled, and mass electronic media have no airtime for wine. Retail distribution is spotty, and marketing support largely unavailable. At least here in Michigan, in-store tastings are illegal.

It's easy to put competitions' flaws in the crosshairs; I do, regularly. But they're not just a marketing tool for wineries. The hype can also expose consumers to smaller, local wineries they might not otherwise be able to learn about.

Gary wrote:
09.02.09 at 9:05 AM

I agree with everyone including Steve and WineDude. How many Double Gold or 95WS have we had in the past and wondered “What have they been Drinking?” The systems may be flawed, but they are the only offered. I expect Alder to come up with the cure!

Pete wrote:
09.02.09 at 10:47 AM

Alder, you assume wine competitions' inability to consistently identify quality wines should concern wineries. In doing so, you miss the point of why wineries participate in competitions. Surely you don't think they're looking for objective, well-informed assessment of their wines. Wineries enter competitions for the same reason they send wines to Enthusiast, Spectator, Advocate or any of the other critics: they see marketing value in doing so. Heimoff, Laube and Parker may arrive at vastly different views of a wine, but if just one of them plants a big fat 90+ kiss on it, the winery is happy. (Well, sure they'd much rather it be Parker or Laube, but they'll take Heimoff, too.) Likewise with competitions. Coming up empty in SF and Riverside doesn't mean the winery can't crow about the Triple Sweepstakes Super Duper Gold it got in San Diego.

I should add that in my years doing wine PR I was unenthusiastic about competitions—but not because I thought the judging was inconsistent (which I did, having judged myself). I simply believed winning medals made little impression on consumers, media and the trade. The reason I *did* sprinkle around a little bit of money on competitions? Many are run by wine critics—B-level critics, generally, but critics with audiences nonetheless. Entering their competitions was a way of showing them a little love.

09.02.09 at 11:01 AM

As an amateur winemaker and judge at home wine competitions, I and my fellows have always stressed constructive feedback to the winemakers so they can make better wine. "This wine sucks!" is not constructive. Constructive are details on the scoresheets like "vegetative", "could handle more oak" and "shows off the fruit well. Thank You". I try to offer food pairings when I judge wines.
We award medals and stickers for bottles. But we remind ourselves at our training sessions each year that the scoresheets and the details we add - not just numbers on a scale - are what we are about.
You're right, Alden. Most, if not all, of the pro judging is about selling bottles on store shelves. But judging can be about making better wine and giving ego strokes when some garage winemaker gets it done well.

09.02.09 at 11:01 AM

Pete,

You are confusing wine critics with wine competitions. They are not the same animal at all--they don't follow the same evaluation procedures either.

09.02.09 at 12:15 PM

I'd like to hear from the organizers... Alder thanks for putting yourself out there. Admittedly, receiving four gold medals for one of our wines didn't hurt sales - but I say, go to the tasting room and try before you buy! Or, take a recommendation from a friend who knows your taste.

kevin keith wrote:
09.02.09 at 12:19 PM

Having judged at these types of wine competitions, I have made it a point to find out what wines my panel scored well, and what my panel panned. It's interesting to go back to the medal results and find out everything my panel "medalled" came up with nothing, and the ones we chose as terrible, ended up with the Gold. Supposedly, at least at this particular Festival, their is a quota for golds, silvers and bronzes. Nice, eh? Nothing like a bit of electioneering.

Tom Wark wrote:
09.02.09 at 12:27 PM

For the record, showing off medals in Tasting Rooms DOES sell more of the medal winning wines. This isn't even debatable. I've seen it happen over and over.

That said, I find this extraordinarily hard to believe, if only because the conclusion is so harsh and so definitive:

"I taste thousands of wines a year at large wine tasting events, where (annoyingly) many producers advertise (and proclaim) their medal winning wines. The vast majority of the time, these wines aren't any good."

Pete wrote:
09.02.09 at 12:35 PM

Thomas, your comment strikes me as a non sequitur. If you'd like to comment on the actual point that I was making -- that wineries will spend resources on wine competitions based on their sense of those competitions' marketing value, not based on how consistently those competitions judge quality wines -- I'd certainly be interested in hearing that.

Alder wrote:
09.02.09 at 12:37 PM

Tom,

I'm sorry to say, it's quite true. I can't tell you how often I have this experience:

I walk up to a table at a large public wine tasting like Family Winemakers, ZAP, Rhone Rangers, etc. and as someone is pouring me a wine they tell me how many gold medals it just won at some fair or competition, and the majority of the time, that wine is merely mediocre.

I think this is partially because many of the wines that REALLY ARE good either have no need to send their wines to these competitions, or they, like some of the winery folks have commented above have the confidence to find better ways to sell their wines than such "beauty pageants."

In short, the quality of wines at many of these competitions I believe to be artificially low, by virtue of self selection.

Now that's a pretty harsh statement, and yes, of course there are plenty of exceptions, but in general it's what I believe to be true. Again, anecdotal evidence in my experience.

I'm sure medals do sell more wines in tasting rooms. Outside of those rooms is where I have doubts about their value and role in consumer choice.

09.02.09 at 12:37 PM

Couldn't agree more. As a newcomer to the industry (last year's first harvest is in the barrel -- so we're THAT new), one of the key ingredients of our marketing plan is to NOT take our wine to competitions. like the wine we make, this is a plan that will not suit everyone, and that's fine. the industry needs all types to make it work. so with no medals, how do we hope to sell our wine? with the individual story behind it, the adherence to our own values that translate into wines made from our own grapes -- distinct wine we truly believe in, mainly -- and of course, all the people tired of the hullaballoo and false promises that medals promote (you know who you are). the only medal we hope to ever achieve is that we stuck to our vision and conviction and did not follow the gold medal crowd.

Pete wrote:
09.02.09 at 12:47 PM

Tom, I don't doubt that medals in the tasting room can point visitors toward particular wines. When I worked in a tasting room, I found most visitors shockingly desperate to be told what wines to buy. But do you generally find competitions helpful for wine marketing/PR that is not aimed at influencing tasting-room sales but instead is aimed at driving restaurant and shop placements and sales? Like Alder, I'm skeptical in most cases.

Mark wrote:
09.02.09 at 12:53 PM

I agree with you there. Too often it is third tier or worse wineries using these medals to sell their products when most established top end wineries won't go near one of the competitions. If the public was more educated about the percentage of wines that are considered gold star, they might think differently about the award.

Tim D wrote:
09.02.09 at 12:56 PM

I'm still confused by the need for wine lovers to figure out a way to quantify a COMPLETELY subjective experience like enjoying wine. 100-point scales, quality-to-price ratios, medals at wine competitions, etc. all attempt to assign an objective value to wine, which is essentially an aesthetic thing.

I agree that results from wine competitions are somewhat arbitrary, as are scores from critics, opinions of friends, recommendations of the sommelier at the restaurant, etc. The real trick if figuring out how arbitrary they are and how much faith to put in them. Is a gold medal from the OC Fair worth as much as a 90-points score from Parker? I guess it depends on whether you like alcohol and residual sugar. The sommelier's recommendation might be worth something, but if he recommends a wine you've tasted and not enjoyed, then how much value does it have?

It all becomes subjective, when it comes to wine enjoyment.

Alder wrote:
09.02.09 at 1:20 PM

Tim,

I'll go halfway down that road with you to subjectivity, but no farther. Yes, wine is an aesthetic experience, but that doesn't mean that it is not possible to create some pretty well accepted critical standards for what makes a wine good and great. And of course there are a number of pretty objective scientific standards for what makes a wine "sound."

The reason that people need to "quantify a completely subjective experience" as you put it is the same reason that critics need to exist in the first place: because there's a lot to be gained by trusting someone who has a lot more experience in a particular field than you do to make recommendations. Consumers (of many aesthetic experiences) rely on critics all day long to save them time and energy -- why try all the wines in the supermarket when you can have someone point you to what they think are the best ones.

The scores from the critics that know what they're doing are far from arbitrary. Your tastes may not align with a particular critic's (that's why there are multiple critics, after all) but because you disagree on certain (or all) wines doesn't mean their opinions are arbitrary.

Tim D wrote:
09.02.09 at 1:54 PM

Alder,

I guess I'm only a little farther down that road than you. I agree with everything you say, including the value of critics for things of an aesthetic nature. But what I think is arbitrary is the assigning of numerical value (like medals at a competition) that critics give to wines. These values are abitrary - they exist only in the mind and purview of the critic assigning them and are not comparable to other critics or other numbers like price for that matter. They only look like numbers that can be compared like things that are actually objectively measured, but they are really just a code for the subjective thoughts of the critic. I can believe that a critic likes or dislikes a wine or that he/she thinks that wine A is better than wine B, but when they assign a number to wines they are problematically asserting that their palate is an instrument with more precision, consistency, and reliability than is plausible.

Thanks for the good conversation, I hardly ever get to talk about the shortcomings of language and numbers to convey proper meaning. Which I like talking about almost as much as wine.

Alder wrote:
09.02.09 at 2:08 PM


Tim,

We're close to being on the same page. There's a reason that my scoring system is deliberately "fuzzy" -- I too don't know the difference between a 92 and a 93 point wine. But I guess I'm having a hard time with your use of the word arbitrary. If you started saying "approximate" instead I think I'd agree with you fully.

But the suggestion that the scores a critic assigns are random, and have no relation to anything else, I can't fully appreciate. Imprecise to be sure, and an artificial way to try and impose some sort of valued ranking to something that exists in a fluid spectrum of experience, most definitely. But scoring systems used by critics have an internal logic, with values that are relative to each other, and in fact that can be correlated with other critics (most major wine critics have a great deal of agreement in their scores for the world's top wines). But more so than concordance with other critics, scores are most useful to consumers as relative gauges of a critics enthusiasm for a particular wine. With a little trial and error, a consumer can easily learn how to use a critic's opinions to guide them to something they will like.

Wine Doctor wrote:
09.02.09 at 2:37 PM

I am an Australian winemaker about to undergo my 3rd harvest in California and my 33rd overall. I am a technically trained wine show judge and have participated in competitions all over the world, including one in California.
The legendary Len Evans in Australia established arguably the toughest and strictest controlled wine show judging system in the world. Each competition he chaired operated with carefully selected and experienced technical judges, and his system must take much of the credit for the overall quality of Australian wine. Each wine is assessed on a universal scoring system with many judges participating at more than one competition. The judging criteria are very tough and any faults are punished. Gold medals are often not given if a class is weak and the percentage of golds awarded is normally quite low. If you win it, then you have earned it.
Now to my experience in California. It was at a large and well 'regarded' competition and the inital briefing session clearly stated...."give as many gold medals as you because it is good for marketing". The panel I judged with was chaired by a representative from a large liquor chain and the others consisted mostly of hospitality and media people. Lovely people but without a technical understanding of wine it was impossible to reach any agreement on any particular wine's quality.
A great example was the "Chairman" being very supportive (read demanding) of a Chardonnay and how beautifully integrated the oak was. My notes indicated the wine had "pinking" (even obvious to the rest of the panel when shown), was hard, phenolic and two dimensional without the support of any oak. I scored the wine as faulty but it managed to be warded very highly, even rising to "Best in Class". Interestingly, the wine was unveiled later and was, in fact, an unoaked Chardonnay.
Fundamental errors and misunderstanding of what makes a gold medal wine is turning many wine competitions into comedy shows with very little substance and diminishing consumer (and winemaker) confidence.

09.02.09 at 3:21 PM

Pete,

I'm just saying that wine critics and wine competitions is not the same subject. In fact, I assume that wine critic scores probably do help sell a hell of a lot more wine that competition medals--I don;t know for sure, but I assume it because I believe each appeals to a different audience.

The thing I don't think Gold Medals do that critic's scores seem to do is to create a viral buzz and price creep.

Knowing how each evaluation is done, I oddly think the competition results is a better reflection of what the consumer might experience than the often "at odds" views of wine critics (except when a competition has a goal for a certain number of awards, which really pisses me off).

At competitions, judges are not supposed to be looking for what they like, but the merits of the wine. Critics approach wine in a completely subjective manner.

Having said all that, I will never understand why consumers need to be told what to like, but if I owned a winery, I would understand how to use the awards.

Jason wrote:
09.02.09 at 3:58 PM

Has anyone mentioned the obvious - that pedigreed wineries don't submit their wines to competitions not because they don't have to, but because they have too much to lose by being bested by a cheap wine?

When higher profile wineries do submit their wines to competitions, you'd be surprised by how often they show poorly.

As flawed as competitions are, they are at least a level playing field for wineries that have no chance at getting noticed by the wine media. I'll take that over sending a wine to a famous critic or publication where they say they taste wine blind, but are really more interested in maintaining the status quo of regional pedigrees (and associated ad revenue).

I can't tell you how many times I've been underwhelmed by a 95 point wine. How is that system any better than the competition circuit?

barbara wrote:
09.02.09 at 4:16 PM

Well said Alder.

Alder wrote:
09.02.09 at 4:25 PM

Jason,

Competitions may be a level playing field, but the rules of that field, and the referees are often badly flawed. Which is the point of this article.

Not all wine critics take advertising from wineries, and critics get far more press and attention when they bash established pedigrees than when they merely continue the trend of high scores.

The fact that you don't like a wine that a critic scored 95 points has absolutely no bearing on:

1. the degree to which that score may be consistent with other critics
2. the degree to which that score may be internally consistent with that particular critics other scores for the same wine, or for other wines of similar quality
3. the likelihood (my claim) that professional critics are way more reliable judges of wine quality than fairs and competitions

If you've been underwhelmed by 95 point wines then you either need to find a different critic to follow (or better yet, pay no attention to scores anymore), or realize that you may just not like certain kinds of wine (I'm never going to agree with any 100 point score for Sherry or Port -- they're just not my style).

tom merle wrote:
09.02.09 at 5:21 PM

Just to clarify, the AAWE didn't write the critique, Bob Hodgson--the winemaker/owner of Fieldbrook Winery (since 1976) up in Humboldt County--did in the AAWE's Journal. Bob is also retired from Humboldt State Univ. where he taught statistics among other subjects (it will be interesting to see whether Dude can pick apart his methodology). It is the companion piece to his assessment of consistency among judges within one competition ( http://bit.ly/13Cgyb ), where he found replication of judging results to be down at the 10% level, akin to chance, just like the concordance article.

As some of you know, I believe there is an effective technique for arriving at an assessment of quality, whether according to personal preference or an attempt at determing objective traits. It relies on the "Wisdom of Crowds", ie., the tasting notes and ratings at CellarTracker because of their relatively large number of "judges". The better wines, putting aside QPR, such as Copain and Chasseur pinots, receive upwards of 20 reviews and scores over a period of time and in different contexts, which are then averaged. A 90+ on this website really means something, especially when it comes to buying decisions. Alas, while their data base is huge, relatively few of the wines produced get this kind of treatment.

09.02.09 at 5:33 PM

I think you're on to a very important issue. Sorry you didn't include the annual April Indiana Wine Fair in your list of competitions because we stress in this event that medals are awarded by consumers, not by professional judges. Wine medals are much like art medals; the judges often praise work that many average citizens just don't get. OK, so we need both. But when you're shelling out a few bucks for a bottle of wine, you want to like it when you get it home. (And, by the way, I marvel at how producers can still turn out decent wines at less than $15 a bottle given the costs of everything from real estate to equipment and machinery, printing and marketing. But let's keep the dialogue going. Medals mean something to producers; flavor and taste mean something to consumers. Are they the same thing?

Kevin wrote:
09.02.09 at 5:35 PM

When I was a retail buyer, I abhorred when the distributor / supplier told me about the medals something had won. It was basically a guarantee I wouldn't care for the wine. I also didn't care for them to tell me the press it had received, since it was bound to bias me.

Yes, I used shelf talkers and believe you me, they sell wine. There were wines I personally didn't care for, but if it had 90 points and sold for a good price, you bet I was interested. Wines like that, along with 1.5L (biggest sellers in the store) allowed me to buy wines I liked that had no press and I could hand sell.

I think the issue with all reviews comes down to the fact people have an enormous number of wine choices, wine is the most expensive (outside of low end) of options in the alcohol world and they have to rely on something to make a choice. Yes, their own palates are the best judges, but how available are wines for sampling to the general public?

Now that I'm no longer a complete insider, I depend upon certain wine buyers, reviews (primarily from Josh Raynolds and Stephen Tanzer) and the occasional tasting to make a decision. I'm dedicated that way, where as the average consumer is not. They just want something good, which is often quite different than what a wine geek would think of as being good. 2 Buck Chuck alone proves that fact.

I have not yet read the report, but look forward to getting into it tomorrow, along with 1WineDude's write up. Meanwhile, I'll be finishing off my Sineann Pinot Gris.

Jason wrote:
09.02.09 at 5:58 PM

I, and most of us in the industry, look at medals like a performance in any one game of the World Series - it's just one performance where things may go your way or not. The next night your team may lose, even though it was the same team that won the night before. That's the nature of competition and it may or may not be a reflection on it's ability to ascertain objective criteria. No one expects one good performance in a competition to be the defining critique of your wine. Wineries of course may exploit this one win, which is what you take issue with. But I have to say that this happens just as frequently with scores from critics. I agree that one good showing in the World Series is nothing to get excited about. But a consistent good showing is significant. I'd respect that more than a single good score from a single critic.

Anonymous wrote:
09.02.09 at 6:32 PM

I completely agree. The plethora of gold medals that adorn relatively common wines demonstrates that wine appreciation is subjective and largely determined by individual preferences and ability to discern the subtleties of each wine. While I don't claim to be a professional, I do find some "gold medal" wines to be quite ordinary and question the legitimacy.

Mike Dunne wrote:
09.02.09 at 7:07 PM

Not without irony, Bob Hodgson's research was undertaken in hopes of improving wine competitions, not getting rid of them, as the conclusions of his most recent paper and his earlier study are being interpreted, at least in this forum.

To be sure, wine evaluation is subjective. No format in that pursuit is perfect. However flawed, wine competitions theoretically have an edge on individual analysis in that they bring to the task three to five individuals per panel. While some judges may be undisciplined and easily distracted, more are earnest sommeliers, educators, retailers and winemakers who sincerely want to find and recognize exceptionally expressive wines. They taste them blind, often in cloistered conditions that encourage deliberation and the free exchange of views. One judge may see a flaw that others overlook, another may be able to draw upon his or her experience to provide enlightenment about pedigree, manipulation, evolution and the like. Nonetheless, a gold medal should be given only so much weight, but more weight than ought to be given wines in a laundry list of recommendations from a critic who tastes wines open, all in a short span, in an environment as alien to contemplation as a crowded pavilion at Fort Mason.

That said, I'm perplexed by a couple of Hodgson's conclusions, and that may be because I'm no statistician. In reading his analysis, however, I don't think he answered his original question: "Do gold medals reflect a measure of quality, or are they simply related to the number of competitions entered?"

What's more, he flatly states that "a wine's performance in one competition is not correlated with its performance in another." Yet, his own tables and figures show that six wines each won three gold medals in a spectrum of competitions, and 20 each got two gold medals. Not the most impressive correlation, granted, but correlation nevertheless.

Point of disclosure: I judge at several wine competitions about the country, and sometimes I'm even paid for joining a panel.

ned hoey wrote:
09.02.09 at 7:27 PM

Like many aspects of our world today, something that in the past was a reasonably scaled and sensible pursuit, has incrementally over time become something distorted from its humble origins.
There are too many competitions, too many critics, too much scoring etc now. The more there is the harder it becomes to keep it meaningful. Also it has become a diversion from simply enjoying wine.

Frankly I'm uncomfortable with wine competitions and scoring. How often does weightless, delicate, subtle, understated beauty "win" over depth, power, intensity and concentration?

09.03.09 at 7:11 AM

"There are too many competitions, too many critics, too much scoring etc now. The more there is the harder it becomes to keep it meaningful. Also it has become a diversion from simply enjoying wine."

Ned,

At about 1,000 and still counting, there's likely too many wine blogs, too.

LarrytheWineGuy wrote:
09.03.09 at 8:15 AM

Alder states, "I'm never going to agree with any 100 point score for Sherry or Port -- they're just not my style."

That is just too juicy to ignore. Alder I respectfully ask what your stylistic preference has to do with the substantive quality of a wine? Isn't any wine category worthy of a possible perfect review? Shouldn't a wine pundit be capable of putting aside a bias to objectively review a wine? Shouldn't a wine pundit recognize a great Port or Sherry from an ordinary port or sherry?

Also, I would love to see a similar study on Aussie and NZ wine competitions. Is the competition system flawed or is it simply a lack of experience and competence on the part of American judges? Who judges the judges?

Mark wrote:
09.03.09 at 8:55 AM

I agree that these competitions are pure bunk, because no matter how flawed the actual process is, in the end there is no right and no wrong wine that exists in this world. Everyone has different tastes for Pete’s sake!

But I do find it hard to believe that gold medals have no marketing benefits for driving sales at the retail level. That’s like saying a high score awarded by WS, RP, W&S or even Vinography are just as worthless. A shelf-talker that sings a wine’s praise is certainly going to be more alluring to a shopper over a wine that is unable to speak for itself. Why else would these ridiculous competitions continue to take place? Gold medals and high scores do more than just feed a winery’s ego. These wine competitions play a vital role in most state’s three-tier system that most certainly helps a winery move its inventory and for retailers to find willing buyers.

I will say though, if these competitions give out medals so freely, couldn’t they at least give each judge one that has to endure 100 or more “bad wines” before tasting one that they actually like?

Alder wrote:
09.03.09 at 9:19 AM

Mark,

Part of the reason that medals may play less of a role in selling a wine at retail is that quite often the information about what medals a wine has won don't make it to the retail channel. The winery that won the medals knows which wines have won, and as Tom references, they trumpet that info loud and clear in their tasting rooms to good effect. However by the time the wine is sold to the distributor, and then again to the retailer, it's not like every bottle comes with a sticker on it that says "Gold Medal: Sonoma County Fair 2009." Most retailers (and distributors) rely on the wine critics' ratings when they are deciding how to label their wines or to create shelftalkers. (As an aside -- I wonder whether distributors give a rip about medals when they are deciding to represent a wine or not).

I spend a good deal of time in wine stores wherever I am, and I rarely see medals mentioned on shelf-talkers or by retailers when they're telling me how good a wine is.

And then there's the consumers themselves. Has any retailer ever heard a consumer come in and ask for a gold medal winning Syrah? Of course not. They want the 90+ point Syrah.

Medals are a very limited sales tool outside of a winery's tasting room.

09.03.09 at 9:28 AM

LarrytheWineGuy,

Your point is well taken here. That's what I meant when Is aid that critics approach wine in a completely different manner than a judge at a competition. The former is looking for what he or she likes; the latter is trying to evaluate the merits of the individual product.

The goal of the critic is not be objective; the goal of the judge is to try to be objective. Different animals, indeed.

Richard wrote:
09.03.09 at 9:31 AM

Alder,

I agree completely with you, but what alternative do we have? As a very small boutique wine maker who uses a custom crush, the competitions and magazines, like Wine Spectator and Wine Advocate are the only way to get my wines noticed. Granted, I have sold a lot on "word of mouth" but that only goes so far.

Also, you don't comment on the almost rigged nature of some of the competitions - my wine won a competition (place and name will remain anonymous) and the results were never published because the sponsor was too fearful of the backlash from the "big wineries."

So, while I agree, I continue to enter comps in the hope of getting a "gold" to brag about or an high score to put on my website. Because, as others have stated, this is the barometer for a wine buying public that likes wine, but doesn't quite know what to buy - a gold medal or high score gives them something to go on.

Richard.

Alder wrote:
09.03.09 at 9:44 AM

Larry,

There is no such thing as a purely objective review until those electronic noses they're developing can start offering point scores on wines. You can't remove the psychological/subjective/emotional component from wine evaluation. And context, is very important. Please see my essay on the subject: Appreciating Wine in Context.

I just don't like Port or Sherry. I don't drink them, and I don't review them. I don't feel like I have the knowledge or experience with them to properly form a useful opinion for readers. Can I tell the difference between a mediocre port and a great one? Sure. But I've had people give me some of the world's greatest ports, and I still don't really like them, even though I recognize qualities in them that differentiate them from the $12 bottle of Dow's I can buy at Safeway.

But I definitely don't have enough context and experience to be able to proclaim a particular wine "perfect" or whatever we all can agree a 100 point score means. There's an emotional elation that must be produced for a wine to get 100 points (Parker himself says that the 100 point score is not so much about one increment better than 99 points as much as it is a feeling). I doubt I will ever have that elation when drinking Port and Sherry. Of course, my tastes could change over time, but for now, that's where I'm at.

Joe Gargiulo wrote:
09.03.09 at 10:03 AM

Yarrow: dismissing the entire professional wine comp process as "bunk" is an oversimplification. Your decision to post a graphic of a double gold medal captures the essence of where the process has evolved: today's double gold medal equals yesterday's gold -- i.e. gold ain't what it used to be.

The study at the center of this debate does not account for differences in regional palate: note that two of the comps, Dallas and Intl Eastern, would expectedly produce dramatically different results than Western comps. This represents a potential 15% error.

Blake (wine comp variation equals wine publication variation), Dunne, Goldberg (observation about wineries in emerging regions) and a few other posters have a realistic understanding while other posters make me wonder what planet they've been living on. Some have a penchant for the obvious (e.g. restauranteurs/retailers are not wowed by medals, but they do buy wines w/ good scores from pubs such as WS and Parker.)

Thomas P's discrediting of judges sans proper training ignores an important fact: many judges have 30-40 years of experience tasting wine, and that trumps anything learned in a component wine tasting class.

The message of the poster saying "down w/ wine comps; let consumers judge for themselves" would be great if consumers took the time to learn, but they want the easy way out as expressed by a poster making an analogy with consumers using mass media advertising to guide their purchase of household items.

I wasn’t aware of special "wine comp bottlings" on the winery level, but I have heard that Bordeaux chateaux have "Parker barrels". Thanks: I needed to be more cynical.

Finally, the best awards from wine comps (dbl gold, best of class/show/sweepstakes), when used in conjunction with overlapping good ratings from wine critics, will positively result in a sound wine purchase for a consumer.

Disclosure: I represent Vineyard & Winery Management magazine, the producer of West Coast Wine Comp, Grand Harvest Awards, and Intl Eastern Wine Comp. My opinions are strictly my own, and are not made in support of my client.

Randy wrote:
09.03.09 at 10:44 AM

The worst wine I felt I ever produced was accidentally submitted to a very prestigious medal competetion and received a double-gold. At that point, I knew the silly cometetions were not for my company. The alc was 16.+%, waaaaay too much oak and the tannins were too aggressive for Zin. Yuck.

Chhers.

09.03.09 at 12:00 PM

Joe,

How are you?

I respectfully disagree with you. Sometimes, 40 years of tasting wine can lead decidedly to definite biases. The idea behind wine evaluation should be to establish as many objective measures concerning a wine's individual merits as possible and that includes organoleptics, for which one must have at least minimal training.

As you know, I serve as a wine judge. I have witnessed first-hand the clash between organoleptic training and no training with things like excessive v.a. etc. You are aware I'm sure how winemakers on a competition panel often clash with non-winemakers because the former may find faults that the latter doesn't.

Critics can get away with it because, as I've said, they evaluate more what they like about a wine and not so much about the wine's technical merits--unless it is truly offensive.

tom merle wrote:
09.03.09 at 4:13 PM

The reason that CellarTracker (like other consumer evaluation sites like TripAdvisor) offers the best means of determining the elusive concept of quality is precisely because the members have little interest, for the most part, in organoleptics. Nor have they much training in the sense that TP means the term. They are seeking pleasure first and foremost, not technical integrity.

This represents an important distinction. If competitions are entered by wineries primarily to sway purchasing decisions by consumers, then they want such competitions, even if there is considerable difference among the palettes on various panels, to identify wines the judges like. The medals reflect degrees of appeal, and therefore should provide guidance for the wine drinker who is putting down the bucks, not receiving samples.

09.03.09 at 7:29 PM

Tom,

How the hell are panels of wine judges going to determine what thousands of consumers might like or want?

I don't understand that concept at all.

To me, the point of wine evaluations (I hate the word competition) is to recognize achievement--not to sell wine to unsuspecting consumers who can't make their own palate decisions; and if this is the goal of running a "competition" then they either should be abolished or they should send a bill to the winning wineries for the valuable PR they achieve.

I liken wine "competitions" to the annual county fair--a chance to give credit to those who work hard to achieve the closest thing to perfection that they can. The marketing is left to the producers, not to the judges.

tom merle wrote:
09.03.09 at 8:37 PM

Thomas,

From your response it would appear your haven't read all the previous comments, which I assume is not the case. How then to explain your POV i.e., to dismiss the mercenary reason that most wineries enter competitions. No one is asking judges to determine the tastes of a large number of consumers. Who could defend this position?

Rather, a gold medal from some wine industry luminaries helps consumers to make a selection among the 100s of wine bottles that they see before them at the store or online about which they would otherwise have no information. So like would be TV or cell phone buyers they seek out items that have received high marks in various review forums.

This is not to say that the county fair motivation as you describe it isn't also there on the part of the winemakers, but they aren't usually the ones writing the checks for hundreds of dollars in registration fees. This is the bill the participating wineries receive in a one for all, all for one cooperative effort to bring home the bacon.

09.04.09 at 7:10 AM

Sorry, Tom, I misunderstood you.

In my zeal to believe that the Gold Medals are there to reward the producer for achievement and not to tell the consumer what to buy, I keep forgetting that the promotion end of the thing IS the reason for the whole event.

To take the conversation full circle, I think critics' scores do more to move wine than competition medals, and have probably played out all that I can say on this matter.

LarrytheWineGuy wrote:
09.04.09 at 7:32 AM

Alder,
I appreciate the sentiment that a wine must have positive visceral appeal. However as Thomas P. stated this positive visceral appeal should be ignored through the vast majority of a well run wine judging. A wine pundit responds to a wine based on personal appeal while a wine judge has a more objective standard based on hygienic cleanliness, varietal typicity, intensity of aroma, balance of structural components, intensity of flavor, length of flavorful and textural finish. Maybe this illustration will aid my argument. As a lover of Brunello di Montalcino I can appreciate the style of modern producers and score such wines highly in a personal review. However, in a judging of Brunello di Montalcino I am less likely to reward this same wine because it does not meet the objective standards for a high quality Brunello di Montalcino. I will agree that US wine competitions sorely lack an objective approach and most judges at these competitions are responding viscerally rather than objectively hence the major inconsistencies. Subjective preferences in a judging are reserved for the highest awards such a double gold, best in category or best of show. Again, I would offer the Aussie system of judging as the gold standard. It would be fascinating to have a similar study of the major Aussie competitions.

Arthur wrote:
09.04.09 at 9:29 AM

Alder,

As always, I second LarrytheWineGuy's arguments.

Alder wrote:
09.04.09 at 9:36 AM

Larry,

Uh oh. You're sounding like a Sensory Analyst. Your comments imply that a) there are universal, objective, measurable standards for varietal typicity, intensity of aroma, balance of structural components, intensity of flavor, and length of flavorful textural finish; and b) that these are the only things that wine should be evaluated on.

I reject both premises.

Not that I don't agree that these considerations are part of an evaluation of a wine -- of course they are. Just like editing is part of the evaluation of the quality of a movie. But you're suggesting the equivalent of giving out oscars based on whether the shots are in focus, the editing is tight, the continuity is fluid, and the plot conforms to accepted standards for the genre. That's soulless and antithetical to everything that makes wine wonderful.

Of COURSE there have to be some standards by which we judge wines. I'm not arguing for some relativistic, gut-level, 100% personal evaluation of wine, but I am most definitely arguing against wine being just the sum of its parts.

I've had plenty of technically perfect wines that I would never want to spend my money on, and would never recommend anyone else drink either.

09.04.09 at 10:09 AM

Bravo, bravo! I whole heartedly concur with your assessment. Boulderdash I say! Medals only mean one thing - the wrong place at the right time.

09.04.09 at 10:47 AM

"I've had plenty of technically perfect wines that I would never want to spend my money on, and would never recommend anyone else drink either."

Alder,

This possible only if you don't like the wine, which has nothing to do with evaluating it for its merits, and all to do with evaluating it as it meets with your personal merits. Nothing wrong with this activity, per se, but it also has less to do with the wine than it does with the evaluator.

Take a second look at Amerine's writings.

Bobby wrote:
09.04.09 at 3:54 PM

There is not much of a difference between submitting wine to a competition looking for a result vs. submitting it to a critic, or even untrained wine blogger for a result. You basically get an opinion on a wine. Is Alder's opinion more valuable then a panel like this: http://www.sfwinecomp.com/judges.html

I'm sure this panel tastes hundreds of thousands of wines per year vs a bloggers hundreds or even thousand.

Competitions are just one more way for brands to break through the clutter of thousands of wines. Basically getting a review from a blogger or a medal from a competition are worth the same thing, whatever you can get out of it. Whocares, its one more opinion and you know what they say opinions are like, a@@hol@s, everyone has one.

09.04.09 at 3:58 PM

If it were not so short sided, the evisceration of wine competitions with this post and the responses would almost be humorous. Of course wine competitions are not perfect.

Apparently only a person who can rate a wine “about 9.5” on a 10 scale or give a wine a 92 point rating is capable of telling a consumer what they should drink. It would also be ridiculous to think that wineries enter their wines for any other reason than to give them something use as a marketing tool. Last time I checked, wineries were actually business hoping to make a profit for the work and money that goes into making the vast selection of wines we all enjoy. If winning a medal helps them do that, fine.

I cannot speak for every competition in the US and I am sure that some are far less reputable than others. The competitions I have been involved with as one of these "unqualified" evil professional judges does not make me draw the same conclusion as many have formed here.

Wine judges do not want to give medals to wines that do not deserve it. Yes, it is subjective but wines that are flawed due to VA, Brett, etc… do not normally make it in a reputable competition. Wines will receive medals that I may not like and you may not like. RP and Alder will also recommend wines that don’t match my palate.

The wine industry needs every tool it can get. If you are such a snob that you think only wines you like should be sold then I am sorry for you. There is such a thing as differing tastes. Remember that white zinfandel is one of the top selling varietals on the market. Should we ban it because you only drink fine dry red wine? I know many people who love Two Buck Chuck. I still let them come to dinner on occasions.

Lighten up people! The only palate you are ever going to think is perfect is your own. I know mine is!

Blind Muscat wrote:
09.09.09 at 1:04 PM

A little late to jump in, but . . . I'm not sure anyone has commented on the Hodgson study itself that Alder cites as the starting point for his rant, and I would submit that the methodology of his approach is problematic at best. For all its numbers, it would never make its way into a seriously-peer-reviewed technical journal.

What he has done is superimpose a "rigorous" statistical analysis on a series of non-controlled experiments -- tasters who have not been uniformly trained, procedures that are not set up to account for things like tasting order, all the many competitions analyzed have different approaches, etc. It's a laughable mismatch -- if people didn't take it so seriously.

Thus, his findings are not news, or at least not useful news. If the point is that wine competitions, like all wine tastings, are not rocket science, we already knew that. But if all the irrelevant math is supposed to give us a new level of understanding of the obvious, it fails.

Alder wrote:
09.09.09 at 1:43 PM

Muscat,

Thanks for the comments, though I don't agree. The statistical analysis has been applied to not to the individual competitions, but to their implied premise as a group. Competitions exist in principle to help consumers make choices. They are supposed to point out the good wines from the bad. Yes, every competition is different, with different fields, different judges, etc. But if they really work the way they claim to work, or more importantly how everyone ASSUMES they work, then it is not unreasonable to expect the wines that do well in one competition to do well in another. The fact that their results show not moderate, not weak, not even tenuous, but ZERO concordance of evaluation across 13 of the largest competitions is, in fact, a valid point. Every other critical outlet that evaluates wines that is not a competition has well established concordance between them. While the levels at which they may appraise wines vary, there is an incredible amount of consistency between WS, WE, W&S, RP, JR, BH, and the other outlets. Which is why their grand disagreements over say Pavie, or Kosta Browne Pinot Noir become fodder for gossip.

Stevie wrote:
09.10.09 at 8:20 AM

I'm fascinated by this "controversy" but remain puzzled by peoples reactions. I think that there is no such thing as a universal "wine of quality" that appeals to everyone's tastes. To me this article about gold medals underlines something that all of us have already known forever. Contests and ratings scales etc. should be viewed merely as helpful guides and not the last word on a wine's quality.

syrahfan wrote:
09.11.09 at 6:24 AM

Muscat,
You state that the study is not valid because the competitions do not use uniformly-trained evaluators, have inconsistent approaches, etc. But that misses the point of the study.

If the study looked at gold-medal award rates by staging rigorous, controlled, consistently-evaluated competitions, that would only investigate the question whether competitions *could* have consistent results, if they were all controlled that way.

But of course they're not. The point of the study was to investigate whether the *actual* competitions *do* have consistent results.

The factual finding of the study is simply this: that the correlation between different competitions is consistent with the awards being random. The "usefulness" of this is not a scientific question; that is a matter of personal choice, or marketing.

(No, I wasn't involved in the study, I'm just an intersted reader. Great blog Alder, first time poster.)

courtlandistan wrote:
11.18.09 at 5:14 PM

Fully agree. I have a Gold Medal winning 06 Dornfelder from Mokelumne in a glass right now. It's good and I couldn't tell you if the Dornfelder field at the OC Fair was that deep. I don't think it stands alone with a Gold, but in comparison to shoddier wines, maybe.

10.23.14 at 2:54 PM

Po?t carrément intéressant

Comment on this entry

(will not be published)
(optional -- Google will not follow)
Yes
 

Type the characters you see in the picture above.

Buy My Book!

small_final_covershot_dropshadow.jpg A wine book like no other. Photographs, essays, and wine recommendations. Learn more.

Follow Me On:

Twitter Facebook Pinterest Instagram Delectable Flipboard

Most Recent Entries

US 2014 Vintage - Early, Fast, Eventful Vinography Images: Big Shadow Come Explore The Essence of Wine with Me in Healdsburg: October 30th, 2014 Vinography Unboxed: Week of October 5, 2014 Another Idiotic California Law Screws Wineries Vinography Images: Vineyard Reflections The Fake Tongue Illusion and Wine Tasting 2014 Wine & Spirits Top 100 Tasting: October 21, San Francisco Cool Beauty: Tasting the Wines of the Western Sonoma Coast Vinography Images: Shaggy Companions

Favorite Posts From the Archives

Masuizumi Junmai Daiginjo, Toyama Prefecture Wine.Com Gives Retailers (and Consumers) the Finger 1961 Hospices de Beaune Emile Chandesais, Burgundy Wine Over Time The Better Half of My Palate 1999 Királyudvar "Lapis" Tokaji Furmint, Hungary What's Allowed in Your Wine and Winemaking Why Community Tasting Notes Sites Will Fail Appreciating Wine in Context The Soul vs. The Market 1989 Fiorano Botte 48 Semillion,Italy

Archives by Month

 

Required Reading for Wine Lovers

The Oxford Companion to Wine by Jancis Robinson The Taste of Wine by Emile Peynaud Adventures on the Wine Route by Kermit Lynch Love By the Glass by Dorothy Gaiter & John Brecher Noble Rot by William Echikson The Science of Wine by Jamie Goode The Judgement of Paris by George Taber The Wine Bible by Karen MacNeil The Botanist and the Vintner by Christy Campbell The Emperor of Wine by Elin McCoy The World Atlas of Wine by Hugh Johnson The World's Greatest Wine Estates by Robert M. Parker, Jr.