Elo rating only makes sense based on the population of players. So I wonder, has Stockfish's Elo been calculated in a pool including humans or human-calibrated engines?
We can take the TCEC ratings as "FIDE ratings" because some time ago there was an effort to calibrate the TCEC ratings as such. They did this by benchmarking against Fritz in Bahrain, which was assigned Kramnik's elo rating at the time he played (and drew) that man vs. machine match.
Do remember that the rating of any engine compared to a human will depend on things like the hardware used, the time control, and so on. Ratings against other engines are straightforward in comparison.
Here is one approach to find the approximate stockfish's FIDE Elo.
- Let Magnus play it at classical time control in a 24-game match.
- What is the result? Let's say (1.5-22.5) in favor of Stockfish. Magnus got 3 draws. You can do your estimate here.
- Since Stockfish has no FIDE Elo rating yet, FIDE may grant Stockfish a rating equal to the highest in the current rating list, and that is 2864. So now it will be Magnus (2864) vs Stockfish (2864).
- Let's update the rating.
rd = 2864 - 2864 = 0
In 24 games, the expected score is 12 or 50% of 24 games because the rd is zero.
rn = ro + K * (actual_score - expected_score) where: rd = rating difference rn = new rating ro = old rating K = development coefficient, say 10 actual_score = 1.5 (Magnus), 22.5 (Stockfish)
rn_magnus = ro_magnus + K * (actual_score - expected_score) rn_magnus = 2864 + 10 * (1.5 - 12) rn_magnus = 2759
rn_stockfish = ro_stockfish + K * (actual_score - expected_score) rn_stockfish = 2864 + 10 * (22.5 - 12) rn_stockfish = 2969
- Magnus played some games against humans and he got back to 2850
- Magnus play a second match with stockfish for 24 games. It will be Magnus (2850) vs Stockfish (2969)
- Let's say the result is (2.0 - 22.0) in favor of Stockfish, Magnus got 4 draws.
- Let's update the new rating.
magnus_rd = 2850 - 2969 = -119 actual_score = 2.0 expected_score = 0.335 * 24 or ≈ 8 rn_magnus = ro_magnus + K * (actual_score - expected_score) rn_magnus = 2850 + 10 * (2.0 - 8) = 2790
stockfish_rd = 2969 - 2850 = 119 actual_score = 22.0 expected_score = 0.665 * 24 or ≈ 16 rn_stockfish = ro_stockfish + K * (actual_score - expected_score) rn_stockfish = 2969 + 10 * (22.0 - 16) = 3029
- Do this many times until the gap between the highest human rating and stockfish rating reaches 700 or more.
At this moment rd = 3029 - 2790 = 239, this is far from 700, so a match with the engine is still reasonable.
Also Magnus can still gain some rating points against human opponents. So the approximate rating of Stockfish depends on the rating of the highest human player or top human players and how these players will perform against Stockfish.
Formula to get the approximate expected score given the rating difference.
expected_score ≈ 1 / (1 + 10 ^ (-rd/400))
We can build a simulator with some assumptions and stop the simulator when rd is 700 or more. Then get the final rating of Stockfish.