23

Magnus Carlsen's draw in yesterday's round of the 2012 London Chess Classic assured that his rating in the next published FIDE rating list will surpass Kasparov's previous record of 2851. I've seen/heard passionate chess fans debate the relative merits of Carlsen's rating achievement versus Kasparov's versus, say, Fischer's. To be clear, that's not what I'm after here.

One crucial element of such discussions is the notion of whether Elo ratings in general have undergone inflation over time: are there so many more 2700+ grandmasters today than there were 20 years ago because of a general rise in playing strength, or just because of some overall inflationary trend in the numbers? I'm also not trying to solicit bare opinions about whether or not that's so. What I am interested in knowing:

What serious research attempts have been made to answer the empirical question as to whether FIDE Elo ratings have naturally inflated over time because of something other than a rise in overall strength in the player pool?

The Wikipedia entry on the Elo rating system has a little bit to say about the matter, and also points to an article by Jeff Sonas of Chessmetrics. In addition to any pointers to work by others, I, for one, would also welcome an answer that gives a clear, concise summary of Sonas' main points.

  • Another thing to think about is inflation in USCF ratings. There has been, and periodically the USCF makes adjustments much to the horror of the players. Since the USCF and FIDE use the same system, fundamentally, I'd be surprised if inflation could affect the USCF and not FIDE. – Tony Ennis Dec 9 '12 at 12:55
  • 2
    The systems are not the same, for instance the USCF has rating floors which are clearly an inflationary factor. – RemcoGerlich Feb 12 '13 at 13:53
19

I am surprised that the paper "Intrinsic Chess Ratings" by Ken Regan and Guy Haworth hasn't been posted yet. It is exactly what's asked for, serious research into rating inflation. PDF

Basically they got games from three periods (1976-1979, 1991-1994, 2006-2009), in several rating ranges (e.g. both players within 10 points of 2200, within 10 points of 2300, etc), and excluded types of games that might be anomalous, like team matches. Read the paper, it looks quite thorough.

Then they compared the games systematically with Rybka 3.

Some sentences from the conclusion:

We conclude that there is a smooth relationship between the actual players’ Elo ratings and the intrinsic quality of the move choices as measured by the chess program and the agent fitting. Moreover, the final sfit values obtained are nearly the same for the corresponding entries of all three time periods.

In my view, it's quite solid evidence against the existence of rating inflation.

  • 1
    Thanks for posting this, I also came to share this. This is the only line of research that has compared players to an objective standard. All the arguments I have seen for rating inflation are subjective and generally anecdotal. On a personal note, I don't think the fact that Morphy was probably 2300 takes away from my appreciation of his games or his skill relative to his competitors at the time. – Sam Copeland Jul 1 '13 at 14:20
12

I poked around some. You've probably seen these pages, but I'll post them anyway:

a. This page will interest you. It includes a photocopy of a letter from Elo himself stating the possibility:

Thus over time the rating scale could drift unless some measures are taken to stabilize it.

He further mentions that the ratings scale has no anchor, no fixed point. Compare to an athlete who runs a race in an hour; an hour now is the same as an hour 50 years ago. Time is such a fixed point.

b. Also, hasn't the 'inflation' question already been answered by recent revelations of high ratings coming out of isolated areas? See the "Pool of Players" section of this page for an allusion to the issue. Additional support, though it is not scholarly nor particularly informative. Search for "isol". Here's another anecdote showing what happens with isolated populations (and another candidate for the 'why are chess players crazy' thread!) I didn't fact-check it but should be easy enough to do.

c. The Elo wiki article talks about inflation as if it's an accepted fact.

d. Here's a germane article about inflation, and the followup. Look at that smoking gun in 1986!

  • I hadn't seen the page from a., thanks for that. Regarding b., I'm unaware of whatever you're referring to; can you elaborate? – ETD Dec 9 '12 at 13:37
  • 2
    I'd argue that without an actual anchor, it's impossible to accurately adjust; in the end, we're just anecodatally adjusting towards some arbitrary value. – Daniel B Dec 10 '12 at 15:46
  • Possibly. But adjusting ratings to yield a similar distribution curve would probably be a good start. For example, some years ago, the USCF adjusts ratings so the average club player was a 1500. I don't know if they still do that. – Tony Ennis Dec 10 '12 at 23:48
  • 1
    @TonyEnnis Sure, and I think that's probably as good as it will get, for now. Specifically, I mean: what happens if the "average club player" today is actually better than 50 years ago? It's not like we can get them to play against players from the past... So we're left with estimating player strength somehow and adjusting. Perhaps with computer programs (run on a standard, prescribed platform), we could have some kind of unbiased, lasting anchor. But even this would have issues, such as the discovery of strategies that work well against the benchmark program, etc. – Daniel B Dec 11 '12 at 10:35
5

In absolute terms, Carlsen 2012 for sure is a stronger player than Kasparov 1985.

If Carlsen 2012 travelled in time played a match with Kasparov 1986, Carlsen would defeat Kasparov. This is simply because the technology-assisted preparation is a lot more efficient, and Carlsen has also an edge in opening theory, because he has the accumulated knowledge 1987-2012 that Kasparov does not have.

However, Kasparov is probably a stronger player than Carlsen. If we take the FIDE Top 100 List for June 2000 (the oldest one that can be obtained), we see that Kasparov with 2849 Elo competes with an average of 2641 for the 99 followers (Elo distance 208 points) while Calsen in Fide Top 100 for December 2012 with a 2848 Elo competes with an average of of 2702 for his 99 followers (Elo distance of 146 points).

Elo is about the difference of points, not about absolute values (100 points of difference for Elo mean that player A is 2 times better than player B, 200 points means 4 times better, and so on. So with that list, it meant that Kasparov was on average more than 4 times better than all his 99 followers, while Carlsen is probably less only 3 times better than the average of his 99 followers.

If we take the list were Kasparov has the maximum distance with his 99 followers and compare that distance with the best for Carlsen, we will be able to determine which player was actually the greatest, because with 99 data points, outliers (like another genius) get mitigate it.

I wonder however if Carlsen or Kasparov really care about who was better.

  • 3
    Your argument about Kasparov being a stronger player than Carlsen rests on comparing each to the next 99 best players. You note, correctly, that Elo ratings are relative, but your argument makes a second, unstated assumption, namely that the next 99 players today are of the same average playing strength as the next 99 in Kasparov's heyday. If that second assumption is not true, then you're comparing Kasparov and Carlsen to different standards. You need to find a pool of people who are the same today as in Kasparov's day. That pool is probably your average beginner, not super-Grandmasters. – Thucydides411 Jan 24 '15 at 20:25
5

Elo's system had two components. One was independent of history, the other was not. His system for creating a "performance rating" over the course of an event or a period of time had no historical component to it; it was simply a measure of performance over the specified time. (Memory fails me on this point, but I think when he was calculating the ratings for FIDE this was the method he used.)

However the Elo system as used by federations around the globe does have a historical component, in that ratings are calculated by calculating a delta, a change from the previous rating.

The historically-based system has a natural tendency towards deflation. The system is a closed system, with no new points being created. So new players come in, take points from established players, and then exit (through death or retirement) before returning all those points back to the next batch of rising players.

Many ideas have been tried to compensate for this, some working better than others. Add to this the commercial pressure in the USCF of the early 70's to make ratings rise faster (the rather cynical view was that players would buy a book from the USCF and play in a tournament, their rating would go up, encouraging them to buy another book, etc.) and inflation was a real thing at some points in history.

Since Elo's system was based on a normal (bell) curve, it's nonsense to try and gauge inflation by measuring either extreme; the extremes are more likely to be affected by the total number of players being rated than by changes in actual strength or any sort of inflation.

1

I have a simple idea. Let's take a chess computer (hardware + software) that had it's rating measured 20 years ago, via play with other chess computers with known ratings that they had 20 years ago. Now let's measure it's rating now (exactly the same hardware plus exactly the same software), via play with modern chess computers, with known today's rating. The difference of two measurements would constitute rating inflation for the past 20 years. Simple enough?

  • It would more or less compute the rating inflation for computers, not for human players. Humans play differently against computers than amongst themselves. – Glorfindel Dec 2 '16 at 19:30
0

First, you have to define what you mean by best. For example, does best mean you are the most dominant player for your era? Or does it mean that the quality of your player is superior to all other players. And if quality is what you mean, then how do you define quality?

Paul Morphy was probably the most dominant player. For example, when he was 12 years old he defeated a top ten player (Lowenthal) in a match 3-0. According to Edo and chessmetrics he was probably already one of the best players in the world at the age of 12! At the age of 21, he played against a simultaneous against 5 top ten players (Bird, Barnes, Boden, De Reviere, and Lowenthal) and scored 3-2.

However, most would argue that dominance is a poor indicator of who is best. After all, Morphy has been described as the first modern chess player. His competition was weak compared with subsequent champions.

Another definition that has been used is quality of play. However, this definition also has a lot of problems. In the 1900 hundreds , a number individuals argued that Steinitz or Lasker were the best players of all time arguing that their knowledge of opening and modern theory would make them superior to players from the past. However, Louis Paulsen made some very clever arguments against this hypothesis. He argued that Morphy (who had a photographic memory and memorized the Louisana bar code by the age of 19) if brought back to life would learn openings and modern theory within a year and be able to compete successfully against modern chess players.

Regan argues that modern chess players who have access to chess computers and modern training methods play more like computers than players of the past. That’s no surprise because they were trained by computers but does that mean that modern players are really better? This begs the question what would Fischer or Capablanca do if they had access to modern computers?

In addition, Professor Regan’s analysis computer strikes me as rather incomplete as it just involves a few five year periods and the players included in the analysis are not mentioned. A more thorough computer analysis by professors Matej Guid and Ivan Bratko found that in fact Capablanca played more like a computer than modern players! https://en.chessbase.com/post/computers-choose-who-was-the-strongest-player-. However, Guid and Bratko noted that there is a problem with concluding from this that Capablanca was a better player. Perhaps his rather sedate style led to fewer positions where he would be likely to blunder. Therefore, his blunder percentage was lower but he was also putting less pressure on his opponents than more aggressive players were. In fact, Capablanca had a high draw percentage compared with his contemporaries.

In contrast, a highly tactical player such as Kasparov might be penalized by his playing style which was more likely to lead to highly tactical positions where computers are especially good a finding errors. In fact, computers tend to perform better against tactical players than positional or in particular closed position players where tactics play a lessor role. Thus, computer analysis that relies on the number of computer detected errors is likely to favor sedate closed position players. In contrast, an aggressive player like Kasparov may make more tactical errors than some other players because he sought very complex positions but his opponents will make even more!

Therefore, you need an error weighting system that doesn’t just calculate the percentage of errors per 100 moves (which is basically what Regan and Guid and Bratko did ). Instead, you need to calculate the difference between your error rate and your opponents error rate. After all, chess is about committing fewer errors than your opponent. Putting pressure on your opponent to induce more errors is considered a good quality.

However, my revised calculation method leads to another problem which is these computer analyses don’t take into consider the strength of your opponent. For example, perhaps Larson achieves a very high chessmetrics rating because his aggressive (optimistic) style led to dominance over lower rated players. However, he had trouble in games against players of equal rating. Other players have frequently argued that he was too optimistic in his play against other high rated players. To avoid this problem, computer error checking analysis should only look at games against strong competors (e.g., the top 10, 20 or 100 players). However, that still doesn’t address the problem of increasing strong competition over time.

Can the problem of increasing quality of play be corrected by looking at back ratings such as Chessmetrics? Actually, I prefer the Edo back rating system http://www.edochess.ca/ because the statistical assumptions are better. For example, Chessmetrics assumes a player’s peak rating occurs when they are 40 years old. I doubt that is true for everyone and many players give up chess before that age or their play was only top notch for a few years (e.g., Harry Nelson Pillsbury, Charousek, Fischer, Morphy, Rubinstein, Fine). Unfortunately, Edo only compares players ratings from 1811 to 1920. According to Edo, Capablanca and Morphy are rated the two highest players from this era. According to Chessmetrics, Capablanca and Lasker were the two best players (Morphy doesn’t even make the top ten.) According to Chessmetrics, Zukertort, Steinitz, Tarrasch, Lasker, Pillsbury, Maroczy, Marshall, Janowsky, Chigorin, Schelecter, Blackburne, Duras, Teichmann, Neumann, Vidmar, Gunsberg, Rubinstein and Burn were better than Morphy. There are many other discrepancies between these two rating systems.

If innovation leads to dominance within a specific chess era over time and it becomes increasingly difficult to innovate over time as the strength of the competition increases you can’t measure true dominance by just looking at the match records of the top 30 players. That is, it’s a lot harder for Magnus Carlsen to dominate his opponents than it was for past champions. If you look at back ratings it’s easy to see that the magnitude of the difference between the top players’ ratings has been decreasing over time. So I believe an Edo type statistical model that takes into consideration the difficulty to dominate over time would be a better approach than what has been tried previously. For example, Fischer was a pretty dominant player for his era because he won 20 games in a row. What was Kasparov or Karpov longest winning streak compared with this winning streak? According to Seirawan, their longest winning streaks were seven games.

Of course, I’m not claiming that winning streaks are a good metric. I’m just arguing that dominance by ratings or in individual matches against other top players is a useful metric that isn’t explicitly taken into consideration in current back rating systems.

So my dream analysis is that you use Edo ratings based on a database that only includes the top 20 or 30 players from each five year period. After you complete this analysis you reweight your results by a dominance factor. That is, more recent players get a bonus factor that is calculated by estimating the trajectory of difficulty of dominating over time (the decrease in rating disparities between top 30 players over time). Next, you would validate this analysis by comparing players percentage of chess computer calculated blunders their opponents make minus their own blunders. If this invalidates the above, then you need to reweight according to the computer error checking analysis if it shows there is a tendency for more recent top players to play more accurately even after my dominance factor is taken into consideration.

My guess based on my eyeballing this, is that Kasparov would do very well. But that’s just a guess.

  • 2
    This doesn't appear to answer the question. – Herb Wolfe Feb 2 '17 at 5:47
  • My point is you can't answer the question about rating inflation until you define chess ability. I reviewed research attempting to adjust for rating inflation or attempting to determine how varies chess champions abilities vary over time (which is what rating inflation is all about). I believe the problem is that researchers haven't really identified their assumptions about what they believe chess ability is. In my opinion, without defining chess ability, you can't answer the question of whether chess ability changes with time or say anything about rating inflation. – ToddM Feb 2 '17 at 6:13

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.