The Historical Comparison of Players

Question

Arpad Elo described (chess life 1962) his ratings as

"the measurement of the position of a cork bobbing up and down on the surface of agitated water with a yard stick tied to a rope and which is swaying in the wind".

Thus the Elo Ratings system is more a measure of Chess players' relative strength against contemporaries rather than one with a historical context.

Is there a way to adapt the Elo system to give it historical context or is there a superior way to compare the relative strengths of players throughout history?

Answer 1

There is Chessmetrics, and it attempts to assign inflation adjusted ratings, even create elo-type rating for players of earlier period.

I am personally not a big fan of trying to compare great players across time. But it could be useful if you want to see who the top 10 players were in 1910 to 1920 time period.

Answer 2

The biggest obstacle to a project like this is players in any given time period are rated relative to the other players of that time period. This leads to two distortions of the mathematics that are non-trivial to compensate for:

1) Outliers. Given a particular player who is very far above his contemporaries, it's difficult to assign an accurate rating. Paul Morphy was winning constantly, and so it's hard to know just how far above his contemporaries to position him, to name one example. If a player were to lose 2 games in thirty, for example, it's not possible to know with any degree of certainty that if the number were increased to 60 he would lose 4. We can guess, but that's all we can do.

2) Size of the player pool. If we assume player skill to follow a normal distribution, then the distance between extremes will be wider given a larger pool. This doesn't indicate that a player at the extreme of a smaller pool is any less skilled than a player at the extreme of a larger pool, though the number assigned will be higher for the larger pool.

I doubt the question will ever permit itself to be solved mathematically, leaving us all plenty of room to argue for decades to come.

Answer 3

This Wikipedia article gives good overview on this question: http://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history

I personally like the method of comparing with engine play (One can lookup the work of Matej Guid and Ivan Bratko on this). This one also has flaws, however not the one currently described in Wikipedia - the authors have also measured and proved that it doesn't matter which engine is used or what is the evaluation depth, the relative comparisons holds cross different engine or evaluation depths. The drawbacks of this method would come from the style of play, as a more computer correct play would be favored versus more exploitive (from game theory point of view) play, which this method cannot determine.

Here is another page that gives a short summary of the method, as well as has an interesting calculation that was done, basically showing that the "strength" this method produces correlates with the ELO strength of the modern players:
http://chess-db.com/public/research/qualityofplay.html