2

Elo rating only makes sense based on the population of players. So I wonder, has Stockfish's Elo been calculated in a pool including humans or human-calibrated engines?

Related: How accurate are chess engine ratings?

2 Answers 2

4

Stockfish 15 is about 3620 rating on the FIDE scale. See the ratings in TCEC Cup 10 which played with a Stockfish that is slightly stronger than Stockfish 15.)

We can take the TCEC ratings as "FIDE ratings" because some time ago there was an effort to calibrate the TCEC ratings as such. They did this by benchmarking against Fritz in Bahrain, which was assigned Kramnik's elo rating at the time he played (and drew) that man vs. machine match.

Do remember that the rating of any engine compared to a human will depend on things like the hardware used, the time control, and so on. Ratings against other engines are straightforward in comparison.

4
  • Do you have any info on the TCEC benchmark match? Was it only one game?
    – qwr
    Aug 29, 2022 at 7:57
  • @qwr you can find all 100 games here: tcec-chess.com/…
    – Allure
    Aug 29, 2022 at 8:58
  • Oh I was thinking about the Kramnik match. Afaict Kramnik played against engines in 2002 and 2006 which is ancient in computer terms, but this does answer the question
    – qwr
    Aug 29, 2022 at 16:09
  • @qwr The Fritz in Bahrain used is the ancient 2002 version of the engine.
    – Allure
    Aug 29, 2022 at 16:13
0

Here is one approach to find the approximate stockfish's FIDE Elo.

  1. Let Magnus play it at classical time control in a 24-game match.
  2. What is the result? Let's say (1.5-22.5) in favor of Stockfish. Magnus got 3 draws. You can do your estimate here.
  3. Since Stockfish has no FIDE Elo rating yet, FIDE may grant Stockfish a rating equal to the highest in the current rating list, and that is 2864. So now it will be Magnus (2864) vs Stockfish (2864).
  4. Let's update the rating. rd = 2864 - 2864 = 0
    In 24 games, the expected score is 12 or 50% of 24 games because the rd is zero.
rn = ro + K * (actual_score - expected_score)

where:
rd = rating difference
rn = new rating
ro = old rating
K = development coefficient, say 10
actual_score = 1.5 (Magnus), 22.5 (Stockfish)
rn_magnus = ro_magnus + K * (actual_score - expected_score)
rn_magnus = 2864 + 10 * (1.5 - 12)
rn_magnus = 2759
rn_stockfish = ro_stockfish + K * (actual_score - expected_score)
rn_stockfish = 2864 + 10 * (22.5 - 12)
rn_stockfish = 2969
  1. Magnus played some games against humans and he got back to 2850
  2. Magnus play a second match with stockfish for 24 games. It will be Magnus (2850) vs Stockfish (2969)
  3. Let's say the result is (2.0 - 22.0) in favor of Stockfish, Magnus got 4 draws.
  4. Let's update the new rating.
magnus_rd = 2850 - 2969 = -119
actual_score = 2.0
expected_score = 0.335 * 24 or ≈ 8
rn_magnus = ro_magnus + K * (actual_score - expected_score)
rn_magnus = 2850 + 10 * (2.0 - 8) = 2790
stockfish_rd = 2969 - 2850 = 119
actual_score = 22.0
expected_score = 0.665 * 24 or ≈ 16
rn_stockfish = ro_stockfish + K * (actual_score - expected_score)
rn_stockfish = 2969 + 10 * (22.0 - 16) = 3029
  1. Do this many times until the gap between the highest human rating and stockfish rating reaches 700 or more.

At this moment rd = 3029 - 2790 = 239, this is far from 700, so a match with the engine is still reasonable.

Also Magnus can still gain some rating points against human opponents. So the approximate rating of Stockfish depends on the rating of the highest human player or top human players and how these players will perform against Stockfish.

Formula to get the approximate expected score given the rating difference.

expected_score ≈ 1 / (1 + 10 ^ (-rd/400))

We can build a simulator with some assumptions and stop the simulator when rd is 700 or more. Then get the final rating of Stockfish.

2
  • I proposed something like this (humans and engines playing in the same pool) in my original question before I had to edit it out to be not "too broad". But instead of stockfish, weaker engines around his rating that can than face progressively stronger other engines.
    – qwr
    Aug 31, 2022 at 17:05
  • Magnus would lose every game May 12, 2023 at 22:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.