Weighted adjustment for relative Elo

Question

I was looking at my biggest upsets the other day and realized that the result was unsatisfying. Most of them were 500-600 point upsets when I was still really low rated (i.e. 800 beating a 1300), while the upsets I consider the greatest have been the 150-200 point ones scored as an A player or expert. Similarly, it would be a much bigger upset if a 2650 were to beat Levon Aronian than it is for a 1500 to beat a master. It seems like the higher ratings go, the greater the points matter - a 100 point difference matters a lot more the higher up you go.

Can anyone come up with a weighted or logarithmic way of representing Elo ratings on some sort of scale so that this is accurately represented and players can be compared in a consistent manner, or does such a scale already exist?

Answer 1

I think the phenomenon that you're describing is due to variability in play at lower levels. Elo is defined based upon the statistical likelihood that one player will beat the other.

Here is the exact formula to get the expected score for a player (rating = Ra) vs an opponent (rating = Rb):

This means that an 800 player is theoretically as likely to beat a 1000 player as a 2200 player is to beat a 2400 player.

Obviously this is not the case because at the lower levels, there is a much higher variance in the results. This difference is captured quite well in the Glicko Rating System. Basically, a player has a rating as well as a ratings deviation (RD) which represents how well established a player's rating is. So in your case when you were rated 800, most likely you were closer to 1100 or 1200 strength but your rating had not yet caught up to your strength. It is one of the fundamental flaws in all rating systems that the rating always lags the strength. If the USCF used the Glicko system, then these early wins would have resulted in a far greater rating jump than actually happened, and your opponent would also have lost fewer points.

So to summarize, while the rating systems are not perfect and your strength will frequently be above your rating, an 800 beating a 1000 and a 2600 beating a 2800 are both equally likely from a purely mathematical standpoint.

Answer 2

You can build whatever scale you want... but as has been said, ratings are already calculated so that a particular ratings difference will produce a particular expected score. Adjustment will probably just skew things, especially if the adjustment is to the degree you suggest and based solely on the rating numbers (as opposed to, say, the fact that your 800s rating was provisional, and it went up significantly in the few tournaments it took for your rating to become established.)

I think you overestimate how uncommon it is for a 2650 to beat a 2820 or so. If there is a 170 point gap in ratings, the lower person is expected to score about 27-28%. Although much of that score is from draws, the lower player does win sometimes. To find out how often, I looked at a database with 127K games in it. I filtered it for games in which a 2800+ played someone 2650 or below. There were 230 such games. Of those, 16 were won by the weaker side. That's about 7%.

Or perhaps you underestimate how hard it is for a 1500 to beat a master. I don't have a database of lower rated players vs masters, but I looked at the USCF games history of a 2309 rated FM from my state. He's played 48 games against people rated under 1500 since they started keeping track in late 1991, and lost zero. He's played 104 games against people rated 1500-1699 and lost 4. Against people rated 1700-1999, he's lost 33 of 589, or about 5.6%.

Yes, I'm mixing USCF and FIDE ratings by comparing the two, but nevertheless I would say that a 1500 beating a master is actually a bigger upset than a 2650 (which is almost enough to be in the world's top 100 list) beating a 2820.

But you didn't ask about that, so I'll set it aside now. You asked about possible formulas. One formula that might be in the spirit of what you seem to intend (heavily weighting the amount of the upset based on the opponent's rating, and affecting all areas of the rating scale) is:

A = D * 2^((R/300)-5)

"A" is the adjusted amount of the upset, "D" is the difference in the ratings, and "R" is the opponent's rating. The 2 means the adjusted upset doubles for a given amount of opponent's rating (if it was a 3 it would triple instead), and the 300 represents the amount needed to make that change. (The 5 is just for scale.) So, by this formula, for every 300 points the opponent is rated, the adjusted amount of the upset doubles.

Personally I think that's way too steep (a difference in 1500 opponent's rating points means it's adjusted by a factor of 32, and I don't think you can ever say a 10 point difference is the same as a 320 point difference) but it seems to fit what was wanted. This formula would make a 2650 beating a 2820 slightly better than a 1500 beating a 2200, and would make an 1800 beating a 1950 better than an 800 beating a 1300.

Answer 3

You could come up with many more complicated ways to measure performance but that would not adequately fix the INCONSISTENCY and VARIABILITY of lower rated players.

Furthermore, ratings are not at all accurate at low levels because there is insufficient mixing of the competition. Top GMs play themselves which is a much much smaller group than those rating club level players worldwide or even in one large country. For that matter a small country should have more weak players than the total of ALL GMs in the world.

And ratings only estimate PAST PERFORMANCE, which as the ads say is no guarantee of future performance. Kids improve, a lot; and GMs get old and usually slightly weaker.

Answer 4

The weakness in the Elo rating system is that it relies on "scoring", that is, replacing trinomial chess game outcomes with binomial "scores" (win=1, lose=0, draw=1/2). Scoring causes some game outcome information to be lost. I made an empirical measure of the information loss and found it to be significant. I have proposed a two-dimensional measure of performance that is more accurate because it contains more information.