Elo rating only makes sense based on the population of players. So I wonder, has Stockfish's Elo been calculated in a pool including humans or human-calibrated engines?
Stockfish 15 is about 3620 rating on the FIDE scale. See the ratings in TCEC Cup 10 which played with a Stockfish that is slightly stronger than Stockfish 15.)
We can take the TCEC ratings as "FIDE ratings" because some time ago there was an effort to calibrate the TCEC ratings as such. They did this by benchmarking against Fritz in Bahrain, which was assigned Kramnik's elo rating at the time he played (and drew) that man vs. machine match.
Do remember that the rating of any engine compared to a human will depend on things like the hardware used, the time control, and so on. Ratings against other engines are straightforward in comparison.
Do you have any info on the TCEC benchmark match? Was it only one game? Aug 29, 2022 at 7:57
@qwr you can find all 100 games here: tcec-chess.com/…– AllureAug 29, 2022 at 8:58
Oh I was thinking about the Kramnik match. Afaict Kramnik played against engines in 2002 and 2006 which is ancient in computer terms, but this does answer the question Aug 29, 2022 at 16:09
@qwr The Fritz in Bahrain used is the ancient 2002 version of the engine.– AllureAug 29, 2022 at 16:13
Here is one approach to find the approximate stockfish's FIDE Elo.
- Let Magnus play it at classical time control in a 24-game match.
- What is the result? Let's say (1.5-22.5) in favor of Stockfish. Magnus got 3 draws. You can do your estimate here.
- Since Stockfish has no FIDE Elo rating yet, FIDE may grant Stockfish a rating equal to the highest in the current rating list, and that is 2864. So now it will be Magnus (2864) vs Stockfish (2864).
- Let's update the rating.
rd = 2864 - 2864 = 0
In 24 games, the expected score is 12 or 50% of 24 games because the rd is zero.
rn = ro + K * (actual_score - expected_score) where: rd = rating difference rn = new rating ro = old rating K = development coefficient, say 10 actual_score = 1.5 (Magnus), 22.5 (Stockfish)
rn_magnus = ro_magnus + K * (actual_score - expected_score) rn_magnus = 2864 + 10 * (1.5 - 12) rn_magnus = 2759
rn_stockfish = ro_stockfish + K * (actual_score - expected_score) rn_stockfish = 2864 + 10 * (22.5 - 12) rn_stockfish = 2969
- Magnus played some games against humans and he got back to 2850
- Magnus play a second match with stockfish for 24 games. It will be Magnus (2850) vs Stockfish (2969)
- Let's say the result is (2.0 - 22.0) in favor of Stockfish, Magnus got 4 draws.
- Let's update the new rating.
magnus_rd = 2850 - 2969 = -119 actual_score = 2.0 expected_score = 0.335 * 24 or ≈ 8 rn_magnus = ro_magnus + K * (actual_score - expected_score) rn_magnus = 2850 + 10 * (2.0 - 8) = 2790
stockfish_rd = 2969 - 2850 = 119 actual_score = 22.0 expected_score = 0.665 * 24 or ≈ 16 rn_stockfish = ro_stockfish + K * (actual_score - expected_score) rn_stockfish = 2969 + 10 * (22.0 - 16) = 3029
- Do this many times until the gap between the highest human rating and stockfish rating reaches 700 or more.
At this moment rd = 3029 - 2790 = 239, this is far from 700, so a match with the engine is still reasonable.
Also Magnus can still gain some rating points against human opponents. So the approximate rating of Stockfish depends on the rating of the highest human player or top human players and how these players will perform against Stockfish.
Formula to get the approximate expected score given the rating difference.
expected_score ≈ 1 / (1 + 10 ^ (-rd/400))
We can build a simulator with some assumptions and stop the simulator when rd is 700 or more. Then get the final rating of Stockfish.
I proposed something like this (humans and engines playing in the same pool) in my original question before I had to edit it out to be not "too broad". But instead of stockfish, weaker engines around his rating that can than face progressively stronger other engines. Aug 31, 2022 at 17:05