On The Relationship Between Reviewers

This post is a bit of a departure for this blog, but I decided that the snatch of math it contains pushes it just over the line of suitability. It was several years ago that my then fellow graduate student, Ilana Deluca, neice of Giorgio Deluca, got me into the habit of trying out new restaurants on Friday night. Neither of us had a significant other at the time, and we enjoy each other's company, so we'd eagerly try and find cuisine that both fit our minimal budgets and tantalized our tongues; if no such establishment fit the bill, we'd grab a bottle of red wine and wait patiently at Angelica Kitchen (Ilana's a veggie-oriented individual and I am, I hope, accomodating). Since then, sampling New York City Restaurants (and those in other locales when possible) has become a minor obsession of mine. I do tend to rely heavily on reviews from Zagat, Michelin (since they began weighing in on the subject again), Frank, and Adam. Thus, I was excitedly awaiting the release of the new publications from both Michelin and Zagat. However, I've long wondered about the relationship between the two scoring systems, a musing that I know I'm not alone in. Since I had access to both data sources, I thought I'd do an extremely simple bit of analysis to explore this topic. The graphic above, described below, is the result.

I started with the list of Michelin-starred restaurants, and looked up the Zagat FOOD rating only for these places (quibble about this if you like, I considered more in depth analysis by some sort of combination of scores for Food, Decor & Service, which may actually be forthcoming, but this seemed a best first-pass). I had thought initially that I'd find the starred restaurant with the lowest Zagat Food score and use this as a sort of cut-off, using only restaurants with Food scores with this value or higher. However, the lowest Zagat Food score for a restaurant on the starred-list is 22, and there are fully 784 restaurants on the Zagat.com site with Food scores of 22 or greater. So, I decided to limit myself to the 88 restaurants that receive a 26 for food or better (42 of these have Michelin stars). Then I simply plotted these restaurants as dots on a graph with number of Michelin Stars as the ordinate and Zagat Food score as the abscissa. Because of the overlap, I scaled each of the 16 resulting points on the graph by the number of entries at each set of coordinates. The coloring is simply for a little jazz-up. Finally, I performed a linear and exponential fit to the data, which were identical. By this I mean, I found the line and the exponential curve which came closest to matching up with the data points in the least-squares sense. Interestingly, these both predicted that as the Zagat Food scores goes up, the number of Michelin stars goes down! This is obviously an artifact of the inclusion of just as many restaurants with high Zagat Food scores and no stars as those with stars. What this does show, I'm sure to nobody's surprise, is that these scales are really not strongly related. Here's another version of the figure with a smaller dynamic range on the dot size, but with numbers of restaurants at each point explicitly printed on the graph.





As a closing note, it is a well known fact that averaging the guesses of many non-experts is often a better estimate of some parameter than those of a few experts. Sir Francis Galton first famously demonstrated this at a livestock fair with the weight of a bull as the parameter. This would seem to suggest that Zagat's rating system should be trusted as it is the amalgamation of the votes of all those who care to contribute whereas the Michelin guide relies on a smaller number of experts. I do not say this as some sort of definitive endorsement of the Zagat Guide, but rather as food for thought, which goes great with... dinner!