Has there been much research on rating inflation?

Question

Magnus Carlsen's draw in yesterday's round of the 2012 London Chess Classic assured that his rating in the next published FIDE rating list will surpass Kasparov's previous record of 2851. I've seen/heard passionate chess fans debate the relative merits of Carlsen's rating achievement versus Kasparov's versus, say, Fischer's. To be clear, that's not what I'm after here.

One crucial element of such discussions is the notion of whether Elo ratings in general have undergone inflation over time: are there so many more 2700+ grandmasters today than there were 20 years ago because of a general rise in playing strength, or just because of some overall inflationary trend in the numbers? I'm also not trying to solicit bare opinions about whether or not that's so. What I am interested in knowing:

What serious research attempts have been made to answer the empirical question as to whether FIDE Elo ratings have naturally inflated over time because of something other than a rise in overall strength in the player pool?

The Wikipedia entry on the Elo rating system has a little bit to say about the matter, and also points to an article by Jeff Sonas of Chessmetrics. In addition to any pointers to work by others, I, for one, would also welcome an answer that gives a clear, concise summary of Sonas' main points.

Answer 1

I am surprised that the paper "Intrinsic Chess Ratings" by Ken Regan and Guy Haworth hasn't been posted yet. It is exactly what's asked for, serious research into rating inflation. PDF

Basically they got games from three periods (1976-1979, 1991-1994, 2006-2009), in several rating ranges (e.g. both players within 10 points of 2200, within 10 points of 2300, etc), and excluded types of games that might be anomalous, like team matches. Read the paper, it looks quite thorough.

Then they compared the games systematically with Rybka 3.

Some sentences from the conclusion:

We conclude that there is a smooth relationship between the actual players’ Elo ratings and the intrinsic quality of the move choices as measured by the chess program and the agent fitting. Moreover, the final sfit values obtained are nearly the same for the corresponding entries of all three time periods.

In my view, it's quite solid evidence against the existence of rating inflation.

Answer 2

I poked around some. You've probably seen these pages, but I'll post them anyway:

a. This page will interest you. It includes a photocopy of a letter from Elo himself stating the possibility:

Thus over time the rating scale could drift unless some measures are taken to stabilize it.

He further mentions that the ratings scale has no anchor, no fixed point. Compare to an athlete who runs a race in an hour; an hour now is the same as an hour 50 years ago. Time is such a fixed point.

b. Also, hasn't the 'inflation' question already been answered by recent revelations of high ratings coming out of isolated areas? See the "Pool of Players" section of this page for an allusion to the issue. Additional support, though it is not scholarly nor particularly informative. Search for "isol". Here's another anecdote showing what happens with isolated populations (and another candidate for the 'why are chess players crazy' thread!) I didn't fact-check it but should be easy enough to do.

c. The Elo wiki article talks about inflation as if it's an accepted fact.

d. Here's a germane article about inflation, and the followup. Look at that smoking gun in 1986!

Answer 3

In absolute terms, Carlsen 2012 for sure is a stronger player than Kasparov 1985.

If Carlsen 2012 travelled in time played a match with Kasparov 1986, Carlsen would defeat Kasparov. This is simply because the technology-assisted preparation is a lot more efficient, and Carlsen has also an edge in opening theory, because he has the accumulated knowledge 1987-2012 that Kasparov does not have.

However, Kasparov is probably a stronger player than Carlsen. If we take the FIDE Top 100 List for June 2000 (the oldest one that can be obtained), we see that Kasparov with 2849 Elo competes with an average of 2641 for the 99 followers (Elo distance 208 points) while Calsen in Fide Top 100 for December 2012 with a 2848 Elo competes with an average of of 2702 for his 99 followers (Elo distance of 146 points).

Elo is about the difference of points, not about absolute values (100 points of difference for Elo mean that player A is 2 times better than player B, 200 points means 4 times better, and so on. So with that list, it meant that Kasparov was on average more than 4 times better than all his 99 followers, while Carlsen is probably less only 3 times better than the average of his 99 followers.

If we take the list were Kasparov has the maximum distance with his 99 followers and compare that distance with the best for Carlsen, we will be able to determine which player was actually the greatest, because with 99 data points, outliers (like another genius) get mitigate it.

I wonder however if Carlsen or Kasparov really care about who was better.