Much hay has been made in recent weeks as the wine world grapples with the implications of ChatGPT specifically, and Large Language Models (LLMs) generally, on the world of wine.

We’ve seen the predictable articles with their equally predictably hyperbolic headlines describing the end of wine writing as we know it. We’ve seen the wine articles written by ChatGPT, and then the articles describing how crappy ChatGPT is at writing about wine.

Those are all expected and understandable reactions to any disruptive technology that appears poised to make a significant impact on society.

But make no mistake about it, ChatGPT can’t hold a candle to most of the professional wine writers whose work you’ll find in the pages of a wine magazine, either in terms of quality of prose, factual accuracy, or depth of analysis.

Yet.

LLMs like ChatGPT may quickly (the cynical might say they already have) reach the point of being able to write content at the level of your average wine industry marketer or average enthusiast wine blogger.

I’m in no position to predict when these LLMs are going to get good enough to generate reading material about wine that I want to consume, but given how fast they’re developing, it seems only a matter of time before they get close.

But will they get all the way?

The one thing ChatGPT will never be able to do (without some very creative human hijinks) is visit a winery, interview the winemaker, and walk through the vineyards making observations as it goes. It won’t come up with interesting questions to ask that winemaker, and press her on exactly why she decided to co-ferment Pinot Noir, Trousseau, and Gamay, for instance.

All those things require analysis, judgment, and insight that, for now and for possibly a good while, are well beyond the reach of LLMs such as ChatGPT. Don’t just take my word for it. Ask Noam Chomsky.

What these LLMs are very good at, however, is hoovering up data, and using (very sophisticated) statistics to turn it into plausible bits of information structured in seemingly quite creative (but really derivative) ways. The statements such algorithms can “make” are only as good as the information that the model is trained on. Which is to say, about as good as the Internet itself.

How good is the internet itself? Well, most of us spend a good deal of time there, and if we’re honest, it’s OK, but not great. But, in many cases, it’s quite good enough.

In terms of qualifying what good enough means, in a recently published whitepaper it was revealed that GPT-4, the next iteration of ChatGPT just “passed” the Intro Sommelier, and the theory portions of the Certified Sommelier, and the Advanced Sommelier tests from the Court of Master Sommeliers.

Here are some sample questions from the Advanced Sommelier Exam:

Which of the following is integral to the existence of flor?
a. High humidity levels
b. Anaerobic environment
c. 18% or higher alcohol by volume d. 12% or lower alcohol by volume

Name the most planted grape variety in each of the following AOCs:
a. Entre-Deux-Mers:
b. Barsac:
c. Fronsac:

Order the listed AOCs from lowest allowable maximum yield to highest allowable maximum yield:
Bourgogne Passetoutgrains
Bonnes Mares
Nuits-Saint-Georges
Chambolle-Musigny Les Amoureuses 1er Cru

Where must German Süssreserve be sourced from?

What major historical role did Duke Philip the Bold play in the Côte d'Or?

Now it’s not entirely clear from what I have been able to read in the technical background of this “experiment” who did the grading of the answers provided by GPT-4, but let’s assume for the moment that the exam was graded with the methods and level of rigor we could expect if we took the exam.

So these models now “know” enough to be able to pass the theory portion of the Advanced Sommelier exam, putting them right on the threshold of Master Sommelier level, an achievement I’ve watched friends and colleagues work for years to achieve.

On the one hand, that’s remarkable. In terms of being an extraordinary technical feat of engineering. On the other hand, it also proves that pretty much anyone could pass these exams if they could Google the answers. Same goes with the AP History exam and the Uniform Bar Exam (both of which GPT-4 also passed with flying colors).

That’s not what GPT-4 is doing in the slightest, but the LLM has been trained on and has ingested material available on the internet, which is essentially the same stuff we might find if we Googled intelligently.

ChatGPT will soon master factual information (including how much Completer is actually planted in Switzerland), provided that we figure out a way to ensure that factual information is mostly correct, and prevent those who prefer “alternative facts” from adding too much noise on top of the signal.

But as we all know (including the ER doctor who fed his patient notes to ChatGPT recently asking for diagnoses) simple facts aren’t always enough.

What the doctor learns in residency, what the sommelier learns on the floor, and what wine writers learn in the vineyards can’t be equaled by simply ingesting more data. The fact that GPT-4 can pass the second-most rigorous written theory examination in wine (I’m sure the MW exam will be next) does make me wonder, however, the weight we place on the ability to regurgitate facts when it comes to wine education and its attendant certifications.

The same, of course, can be said for education more broadly. At what point does the accumulation of factual knowledge become a useless exercise in the face of what a given profession, situation, or task requires? Certainly more than a little factual wine knowledge is important to be able to be an excellent, or even a world-class sommelier or wine writer. But that is far from everything that is required to deliver either a fabulous wine experience or a compelling narrative.

ChatGPT is not going to be replacing our favorite sommeliers or wine writers anytime soon (any more than will the latest taste-profiling app), because it can’t do what they do best.

But when an LLM can ace the hardest exams that the gatekeepers of the wine industry put up as indications of ultimate expertise, we have to ask how well the educational curriculae and learning paths that we have established for wine reflect the realities of life.

Illustration and comic created in collaboration with MidJourney AI.