• Home
  • About
  • Blog
  • The Feynman Technique
  • Food & Wine Pairing
  • Work With Tim
  • Contact
Tim Gaiser, Master Sommelier

Thinking beyond the Numbers

1/30/2013

9 Comments

 
Picture
Over the last ten-plus years, I’ve taught a classed called Mastering Wine II at the Rudd Center in St. Helena. In most respects, MWII, as I like to call it, is a second level wine survey class organized by focusing on a different grape variety or varieties each of the five days. The schedule alternates between lectures and tastings with two field trips thrown in to keep everyone from falling into a deep classroom trance. Over the years I’ve morphed the class into a professional tasting seminar with the express purpose of making sure that the students exponentially increase their ability to taste by week’s end.

During the week, I assign a discussion topic for after lunch with the lofty goal of trying to solve or at least examine some of the wine industry’s biggest issues. Tuesday’s topic is about the use of the 100 point scale in wine. I split the class into two groups and assign one group the task of coming up with everything good about using numerical scores for wine. The other group’s assignment is come with everything wrong about it. Here’s a sampling of answers from hundreds of students and dozens of classes over the last decade (Remember, these are student answers and not mine).

100 Point Scale - Pros:

·Helps wineries sell wine. 

·Helps PR and marketing people sell wine.

·Shelf-talkers in retail shops that use scores help sell wine and make it easier for consumers to find quality wines.

·Helps collectors buy futures.

·Provides an impetus for winemakers and wineries to make better wines, i.e., better wines theoretically should get better scores.

·Provides an easy-to-understand system for consumers to understand.

·Creates a level playing field for comparing wine quality. 

·Creates instantaneous markets for regions, appellations and wineries that were previously non-existent (Priorat a perfect example).


100 Point Scale - Cons:

· The entire scale is not used—scores practically always fall between 70-92 points. 

·A score reflects the opinion of either a single taster or is an average of several different tasters.

·Scores presuppose we all like the same style of wine.

·Wines with high scores tend to be riper styles with higher alcohol and more use of new oak. Delicate wines don’t get high scores as often.

·All grape varieties are not created equal; certain grapes/wines such as Sauvignon Blanc rarely get high scores like Cabernet Sauvignon.

·Winemakers have consciously have adjusted their winemaking to produce riper, richer wines to get higher scores. As a result, winemaking and wine styles are becoming homogenized. 

·Wine is a living, ever-changing thing; how can a score based on a single tasting experience accurately reflect a wine?

·There is such a thing as a perfect 100 point wine.

·What’s the difference between wines rated at 89 and 90 points?

·Scores presuppose a precision in wine that does not exist.

·Scores don’t take context into account—how the wine will ultimately be enjoyed. Will a given wine be consumed with food? By itself? 

The last point is always my favorite. While I certainly get the use of scores as an attempt to convey a wine’s quality, they don’t take context into account. Last year I wrote a post on context and wine called “The Beautiful Imprecision of Wine” (http://www.timgaiser.com/1/post/2012/6/the-beautiful-imprecision-of-wine.html). I put forth that there are three variables in any wine experience: the wine, the taster and the context. I argued that the context was by far the most important variable because it took into account everything from the time of day the wine was tasted, the glassware, the wine temperature, the temperature of the tasting room, how many wines were being tasted, the order of the wines in the tasting and much, much more. Change any one of these variables and you change a taster’s experience of a given wine.

In case you’re wondering, the first group of students always does a good job coming up with the benefits of using the 100 point system. However, when we get to the cons everyone inevitably piles on and it’s usually not pretty. After hashing it out for the better part of an hour, I ask the students to do one last thing; come up with a system that can convey the most important aspects of a well-made wine to a consumer without using any kind of numerical score, be it a 20 points, 100 points, or whatever. 

After much deep thinking and serious discussion, every class comes up with a strikingly similar system using the most vital components of wine such as sweetness/dryness, intensity of flavor, acidity, tannin, and the use of oak. From there, the students take each component and set extremes as baselines, as in a little of something versus a lot. For example, with sweetness or dryness they list “bone dry” at one end with "dessert" at the other; for intensity of flavor they usually list “delicate” and “intense.” And so on. Then the extremes for each component are filled in either by using low, medium, high, and variations thereof or with numbers in increments of three, seven, or nine (Remember, no base ten!). The results look something like this:

Intensity of flavor: Delicate 1   2   3   4   5   6   7  Intense

Sweetness/dryness: Bone dry 1   2   3   4   5   6   7  Dessert

Body: Light-bodied: 1   2   3   4   5   6   7   Full-bodied

Acid: Light acidity  1   2   3   4   5   6   7    Tart acidity

Tannin: No tannin 1   2   3   4   5   6   7   Astringent

Oak: No oak 1   2   3   4   5   6   7   100% new oak

At this point the students all sit back feeling pretty good. After all, they’ve just come up with a fairly creative system that manages to convey a great deal about a wine without using a numerical representative. I end the session by giving them the bad news: their brilliant system was created and used quite some time ago. 

In the summer of 1994, dear friend and fellow Master Peter Granoff opened the cyber doors to Virtual Vineyards, not only the first online wine retail “shop” but also the very first online retailer of any kind. Peter and his brother-in-law Robert Olson started Virtual Vineyards on a server literally in Olson’s garage. To avoid using numerical scores, Peter devised a system almost exactly like the one above to convey the most important information about every wine in the VV portfolio. His system used intensity of flavor, body, sweetness/dryness, acidity, tannin, oak, and complexity represented in one-through-seven increments. 

As with so many other online ventures of the time, Virtual Vineyards grew slowly at first. I was the 13th employee joining the company in April of 1996. Then with the help (or curse depending on how you look at it) of tens of millions of dollars of VC funding, Virtual Vineyards grew dramatically becoming the original version of wine.com before acquiring a competitor in mid-2000 and ultimately being unplugged by VC’s in April of 2001. I will say no more. But aside from being remarkably ahead of its time—there is nothing like it even today—Virtual Vineyards managed to sell over $50 million of wine in five years using Peter’s system and without ever using a single numerical score. 
Picture
Finally, on the subject of wine and numbers I will leave you with this; in December I had the opportunity to join five other fellow Master Sommeliers from the US, UK, and Germany in Istanbul to taste Turkish wines and meet Turkish winemakers. It was a great opportunity to have a first look at the country’s wines. Our tastings were scheduled over three days with the fourth day spent meeting with the winemakers to discuss our impressions of their wines as well as opportunities for export to the U.S. It goes without saying that in a country that is 90% Muslim, the Turkish government is very conservative about the production and enjoyment of alcoholic beverages. No surprise then that it’s remarkably challenging to distribute and sell wine inside the country with the government clearly forcing winemakers to sell their wine on the export market. 

Several days before our departure our contact in Istanbul e-mailed the tasting schedule, information about hotels and the tasting venue, as well as the news that we would be scoring the wines on a 100 point scale. Further, our contact would be tracking and calculating the validity of our scores by using the Pearson Correlation Coefficient for every 10th wine tasted. I’ve included it below for your curiosity and enjoyment (Sorry that the scan is a bit muddled). 
Picture
Further, all six Masters would be tested randomly for reliability during the three days by having a selected wine served multiple times at random intervals during each tasting session. A Modified Fisher (one-way random average measures), whatever that is, was used to compute the “intra-class correlation.” Again, the formulas are listed below. 
Picture
I must confess that Algebra II in my junior year of high school marked the end of my math career. From then on it was a steady academic diet of a music, world history, and literature dotted with the odd volleyball or tennis class. That said, how did the numbers actually work out using the complex logarithms listed above? I’m not completely sure but I can tell you that several times during the three days a quick check among the six of us tasters found that our scores were almost always within 3-4 points of each other with rare exceptions. And that’s not bad for numbers.
9 Comments
Bob (Los Angeles wine industry professional)
3/9/2013 02:42:57 pm

Caltech physics professor Leonard Mlodinow wrote a book titled "The Drunkard's Walk." In chapter 7, titled "Measurement and the Law of Errors" he addressed the issue of wine critics/reviewers who assign "points" in ranking wines.

In 2009, he contributed an essay to The Wall Street Journal on the subject. See below for his comments about Robert Parker priding himself for consistently being "within a 2-3 point deviation" on his scores for wines tasted repeatedly over time.

(Sorry, no "tracking and calculating the validity of . . . scores by using the Pearson Correlation Coefficient.")

Bob


From The Wall Street Journal “Weekend” Section
(November 20, 2009, Page W6):

“A Hint of Hype, A Taste of Illusion;
They pour, sip and, with passion and snobbery, glorify or doom wines.
But studies say the wine-rating system is badly flawed. How the experts fare against a coin toss.”

[Link: http://online.wsj.com/article/SB10001424052748703683804574533840282653628.html]

Essay by Leonard Mlodinow

. . . teaches randomness at Caltech. His most recent book is
"The Drunkard's Walk: How Randomness Rules Our Lives."


Acting on an informant's tip, in June 1973, French tax inspectors barged into the offices of the 155-year-old Cruse et Fils Frères wine shippers. Eighteen men were eventually prosecuted by the French government, accused, among other things, of passing off humble wines from the Languedoc region as the noble and five-times-as-costly wine of Bordeaux. During the trial it came out that the Bordeaux wine merchants regularly defrauded foreigners. One vat of wine considered extremely inferior, for example, was labeled "Salable as Beaujolais to Americans."

It was in this climate that in the 1970s a lawyer-turned-wine-critic named Robert M. Parker Jr. decided to aid consumers by assigning wines a grade on a 100-point scale. Today, critics like Mr. Parker exert enormous influence. The medals won at the 29 major U.S. wine competitions medals are considered so influential that wineries spend well over $1 million each year in entry fees. According to a 2001 study of Bordeaux wines, a one-point bump in Robert Parker's wine ratings averages equates to a 7% increase in price, and the price difference can be much greater at the high end.

Given the high price of wine and the enormous number of choices, a system in which industry experts comb through the forest of wines, judge them, and offer consumers the meaningful shortcut of medals and ratings makes sense.

But what if the successive judgments of the same wine, by the same wine expert, vary so widely that the ratings and medals on which wines base their reputations are merely a powerful illusion? That is the conclusion reached in two recent papers in the Journal of Wine Economics.

Both articles were authored by the same man, a unique blend of winemaker, scientist and statistician. The unlikely revolutionary is a soft-spoken fellow named Robert Hodgson, a retired professor who taught statistics at Humboldt State University. Since 1976, Mr. Hodgson has also been the proprietor of Fieldbrook Winery, a small operation that puts out about 10 wines each year, selling 1,500 cases

A few years ago, Mr. Hodgson began wondering how wines, such as his own, can win a gold medal at one competition, and "end up in the pooper" at others. He decided to take a course in wine judging, and met G.M "Pooch" Pucilowski, chief judge at the California State Fair wine competition, North America's oldest and most prestigious. Mr. Hodgson joined the Wine Competition's advisory board, and eventually "begged" to run a controlled scientific study of the tastings, conducted in the same manner as the real-world tastings. The board agreed, but expected the results to be kept confidential.

There is a rich history of scientific research questioning whether wine experts can really make the fine taste distinctions they claim. For example, a 1996 study in the Journal of Experimental Psychology showed that even flavor-trained professionals cannot reliably identify more than three or four components in a mixture, although wine critics regularly report tasting six or more. There are eight in this description, from The Wine News, as quoted on wine.com, of a Silverado Limited Reserve Cabernet Sauvignon 2005 that sells for more than $100 a bottle: "Dusty, chalky scents followed by mint, plum, tobacco and leather. Tasty cherry with smoky oak accents…" Another publication, The Wine Advocate, describes a wine as having "promising aromas of lavender, roasted herbs, blueberries, and black currants." What is striking about this pair of descriptions is that, although they are very different, they are descriptions of the same Cabernet. One taster lists eight flavors and scents, the other four, and not one of them coincide.

That wine critiques are peppered with such inconsistencies is exactly what the laboratory experiments would lead you to expect. In fact, about 2

Reply
Randy Caparoso
2/26/2014 06:06:40 am

BTW, Tim, this is the most reasonable explication for not following point-score systems I have read. Thanks! R

Reply
Bob Henry
2/26/2014 06:59:39 am

Tim,

Citing your observation . . .

"· The entire scale is not used -- scores practically always fall between 70-92 points."

. . . this question: Some wine scales go from 50 to 90 points (Robert Parker for non-age-able wines) and 50 to 100 points (Robert Parker for age-able wines).

What score do we give to water?

More seriously. If a wine is totally flawed, it should receive a score of zero.

~~ Bob

Reply
Tim Gaiser
2/26/2014 07:49:28 am

Hi Randy, thanks for reading and glad you enjoyed the post.

Reply
Bob Henry
2/26/2014 06:58:34 am

Tim,

Citing your observation . . .

"· The entire scale is not used -- scores practically always fall between 70-92 points."

. . . this question: Some wine scales go from 50 to 90 points (Robert Parker for non-age-able wines) and 50 to 100 points (Robert Parker for age-able wines).

What score do we give to water?

More seriously. If a wine is totally flawed, it should receive a score of zero.

~~ Bob

Reply
Bob Henry
2/26/2014 06:58:40 am

Tim,

Citing your observation . . .

"· The entire scale is not used -- scores practically always fall between 70-92 points."

. . . this question: Some wine scales go from 50 to 90 points (Robert Parker for non-age-able wines) and 50 to 100 points (Robert Parker for age-able wines).

What score do we give to water?

More seriously. If a wine is totally flawed, it should receive a score of zero.

~~ Bob

Reply
Bob Henry
2/26/2014 06:58:53 am

Tim,

Citing your observation . . .

"· The entire scale is not used -- scores practically always fall between 70-92 points."

. . . this question: Some wine scales go from 50 to 90 points (Robert Parker for non-age-able wines) and 50 to 100 points (Robert Parker for age-able wines).

What score do we give to water?

More seriously. If a wine is totally flawed, it should receive a score of zero.

~~ Bob

Reply
Bob Henry
2/26/2014 06:59:09 am

Tim,

Citing your observation . . .

"· The entire scale is not used -- scores practically always fall between 70-92 points."

. . . this question: Some wine scales go from 50 to 90 points (Robert Parker for non-age-able wines) and 50 to 100 points (Robert Parker for age-able wines).

What score do we give to water?

More seriously. If a wine is totally flawed, it should receive a score of zero.

~~ Bob

Reply
Bob Henry
2/26/2014 10:33:41 am

Tim,

Mea culpa for the redundant replies. A computer glitch.

~~ Bob

Reply



Leave a Reply.

    Tim Gaiser

    My thoughts on wine and more. I hope you enjoy.


    Get blog posts delivered by email. Enter your address:

     Subscribe in a reader


    Categories

    All
    All
    Books
    Dining
    Exam Preparation
    Food
    Food And Wine
    Other
    Recently Tasted
    Recently Tasting
    Recordings
    Restaurants
    Spirits
    Studies
    Tasting
    Tastings
    The Rest
    Wine
    Wine Events
    Wineries
    Wine Service
    Winesoftheworld
    Wines Of The World

Powered by Create your own unique website with customizable templates.