Looking for Bias in All the Wrong Places

, the wine and spirits columnist for Canada’s Globe and Mail newspaper, wine critics from any country with a significant wine industry (but especially America and Australia) can’t be trusted to rate their own country’s wines without artificially inflating the scores.

Crosariol claims that when he is out buying wine based on scores (that this particular wine critic buys his wine based on scores should be your first sign of ignorance) he deducts points if the reviewer is reviewing a wine from their own country.

What a load of crap.

Crosariol’s article starts out well enough:

“This will come as no surprise to most of you: Wine columnists are biased. Some have a special fondness for smooth oak and high alcohol. Others delight in herbaceous cabernet francs, still others in the sweat and guts of an old-school Barolo. As readers, we adjust, mentally deducting or adding a few points here and there when we’re wise to the critic’s unbridled enthusiasm for a particular style. Taste is taste, and sometimes that’s a critic’s biggest draw (depending on the reader).”

It’s hard to argue with that. But shortly thereafter, Crosariol goes off the rails.

“But too often the bias is political. It’s clear to me that many wine scorers give too much credit (read inflated scores) to products from their own countries,” he writes.

It would be one thing if Mr. Crosariol had done some analysis of critics scores in major publications, and offered a set of conclusions from which to argue his point. But we must trust his gut, and ride along with his outrage as he suggests that an American judging American wine is like having a whole panel of American judges at the Olympics judging gymnastics.

Actually, Mr. Crosariol, an American judging American wines is sort of like having a set of American judges preside over the U.S. Olympic Trials to determine who gets on the team. Judging at the Olympic games is about choosing the best athletes in the world from among a tiny pool of athletes that have already been heavily judged in their own countries. What’s more, the purpose of judging in the Olympics is to create a force ranked list of the best competitors. There can only be one gold medal. It’s a zero sum game with incredibly high stakes, in which all the competing countries stand to lose.

There is no analogue in the wine world (don’t get me started on wine competitions). When a critic gives a wine a 94 point score, he or she may be evaluating the wine based on an understanding of other comparable wines, but that 94 does not limit the number of 94 point scores for other wines that can be given out, nor does it presume to place that wine in a specific ranking of the world’s wines. Even when a critic doles out a magic 100 or 20 or 10 or whatever happens to be the top end of their particular scoring system, that doesn’t mean they think it is the best wine in the world.

Crosariol’s analogy is as shallow as the rest of his rant. His implication that every (non-British) wine reviewer sits down, tastes a wine, decides how good it is, and then tacks on a few extra points because it comes from their own country would be hysterically funny if it weren’t so insulting to hard working people.

There are basically two kinds of wine critics in the world. Those who professionally review wines on a global basis, and those whose work is restricted to a specific region or country. There are far more of the latter than the former.

It’s perfectly reasonable to expect that a wine reviewer who focuses solely on the wines of a specific region might evaluate those wines on a slightly different scale than someone who is evaluating those wines in the context of the global wine trade, but this is different than having a bias. If you are a South African wine reviewer, and all you do is evaluate South African wines, you might like them more than, say Stephen Tanzer, when he comes to town and reviews the wines between his trips to Burgundy and Australia.

It also happens to be impossible to prove (or even claim) that such a critic is biased, because they haven’t scored wines from other countries in a way that would allow you to point to a bias. But even supposing their scores for a given wine are higher than outsiders’, suggesting that this is the result of some nationalistic bias insults anyone who takes their job seriously as a professional. It seems to me that the worst accusation you could tar such a critic with might be naiveté and lack of perspective.

The other kind of wine critic, of which there are a far fewer number in the world, who regularly evaluates wines from every country, produces a body of work that could clearly and easily show bias, if that were what someone was looking for. For instance, you could look at the scores of one Dr. Jay Miller, American, who works (though not much longer) for the Wine Advocate. He happens to review wines from America (Washington and Oregon) as well as wines from Spain, Chile, and Argentina. All Mr. Crosariol would have to do would be to take his scores, drop them into an excel spreadsheet, and find out if a higher percentage of American wines got higher scores than those of other countries.

And what do we learn when we actually spend the ten minutes of effort to do this? That Crosariol is more interested in making lazy provocative statements than offering his readers intelligently reasoned arguments.

Jay Miller, who has been criticized in some quarters as one of America’s most flagrantly generous scorers, gave roughly 63% of the nearly 500 red wines he recently scored from Argentina’s 2008 vintage 90 points or higher. And his scores for the same vintage of American red wines? 64% scored 90 points or higher.

In the title of his piece Crosariol asks why critics can’t choke down their biases. But I’m left wondering whether Crosariol would be willing choke down his ego and admit that he’s being ridiculous?