9

I want to create something that will select an appropriate Stockfish difficulty setting automatically given the player's rating. The straight Stockfish v11 that I've downloaded has 20 difficulty settings, and various other settings for controlling thinking time. Is there a resource that has approximate ratings for the levels? If that doesn't exist, what method would be recommended for finding the approximate ratings of chess engines when I'm starting from scratch?

EDIT: Sopel gave a link some time ago, that answers the problem, although they did it in comment form so I can't mark it as the correct answer

Improve this question
7
  • Not sure how you could really compare a human rating with a computer difficulty level. When you play a computer you pretty much are just playing against yourself. You are playing against your own knowledge and experience. In any given position you are trying to use your own experience to play the best move possible. The level of the computer really only determines how much you will be punished if you don't make the best possible move.
    – musefan
    Jun 5, 2020 at 10:08
  • The design of the system I'm building won't really have a good way for the "player" to select the difficulty of the engine, and the "player" will only be able to play against engines. For my purposes, I want to know the rough equivalent ELO of the stockfish levels so that I can calculate adjustments to ELO when the player wins/loses, and so that I can select an appropriate rated engine for the next game. "Player" in quotation marks because the system is not going to have one human player
    – day
    Jun 5, 2020 at 12:35
  • 1
    github.com/official-stockfish/Stockfish/pull/2225 is probably the best you'll get
    – Sopel
    Jun 8, 2020 at 22:33
  • 1
    Answer from sopel actually looks like the best answer. The files changed section in the link given describes changes to search.cpp, which contains the exact relation. Proving table for the relation given in the summary for peace of mind. Post link as an answer and I'll tick it.
    – day
    Jun 10, 2020 at 6:33
  • 1
    Here is a list of engine strenght. I think it's quite realistic. ccrl.chessdom.com/ccrl/4040 Hope it helps.
    – David
    Jan 12, 2021 at 11:39

3 Answers 3

Reset to default
3

Stockfish 16 UCI_Elo was calibrated with CCRL.

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)
   1 master-skill-19    :  3191.1   40.4   940.0    1707    55
   2 master-skill-18    :  3170.3   39.3  1343.0    2519    53
   3 master-skill-17    :  3141.3   37.8  2282.0    4422    52
   4 master-skill-16    :  3111.2   37.1  2773.0    5423    51
   5 master-skill-15    :  3069.5   37.2  2728.5    5386    51
   6 master-skill-14    :  3024.8   36.1  2702.0    5339    51
   7 master-skill-13    :  2972.9   35.4  2645.5    5263    50
   8 master-skill-12    :  2923.1   35.0  2653.5    5165    51
   9 master-skill-11    :  2855.5   33.6  2524.0    5081    50
  10 master-skill-10    :  2788.3   32.0  2724.5    5511    49
  11 stash-bot-v25.0    :  2744.0   31.5  1952.5    3840    51
  12 master-skill-9     :  2702.8   30.5  2670.0    5018    53
  13 master-skill-8     :  2596.2   28.5  2669.5    4975    54
  14 stash-bot-v21.0    :  2561.2   30.0  1338.0    3366    40
  15 master-skill-7     :  2499.5   28.5  1934.0    4178    46
  16 stash-bot-v20.0    :  2452.6   27.7  1606.5    3378    48
  17 stash-bot-v19.0    :  2425.3   26.7  1787.0    3365    53
  18 master-skill-6     :  2363.2   26.4  2510.5    4379    57
  19 stash-bot-v17.0    :  2280.7   25.4  2209.0    4378    50
  20 master-skill-5     :  2203.7   25.3  2859.5    5422    53
  21 stash-bot-v15.3    :  2200.0   25.4  1757.0    4383    40
  22 stash-bot-v14      :  2145.9   25.5  2890.0    5167    56
  23 stash-bot-v13      :  2042.7   25.8  2263.5    4363    52
  24 stash-bot-v12      :  1963.4   25.8  1769.5    4210    42
  25 master-skill-4     :  1922.9   25.9  2690.0    5399    50
  26 stash-bot-v11      :  1873.0   26.3  2203.5    4335    51
  27 stash-bot-v10      :  1783.8   27.8  2568.5    4301    60
  28 master-skill-3     :  1742.3   27.8  1909.5    4439    43
  29 master-skill-2     :  1608.4   29.4  2064.5    4389    47
  30 stash-bot-v9       :  1582.6   30.2  2130.0    4230    50
  31 master-skill-1     :  1467.6   31.3  2015.5    4244    47
  32 stash-bot-v8       :  1452.8   31.5  1953.5    3780    52
  33 master-skill-0     :  1320.1   32.9   651.5    2083    31

There was some discussion on improving Stockfish levels to play more humanlike https://github.com/official-stockfish/Stockfish/issues/3635

dav1312 provides this ELO rating list, but it's not listed how it was derived:

0    1347
1    1490
2    1597
3    1694
4    1785
5    1871
6    1954
7    2035
8    2113
9    2189
10    2264
11    2337
12    2409
13    2480
14    2550
15    2619
16    2686
17    2754
18    2820
19    2886

I think lichess could very well keep track of rating based on all the people that play it like for other bots: https://lichess.org/player/bots. I don't think they do, though.

Improve this answer
1
  • In the absence of the ability for me to mark sopel's comment as the correct answer because they didn't submit it as an answer, I'll award it to this one as it's very close to the same essence
    – day
    Jul 27, 2022 at 6:49
3

I am rated 1410 USCF. I played all 8 Stockfish levels this past weekend, once with each color. My record was 11-5. I was 2-0 vs Stockfish 1-5, 1-1 vs Stockfish 6, and 0-2 vs Stockfish 7 and 8.

Improve this answer
2
  • The most concrete answer, based on experience. Only that lacks the reflection time, the hardware were it runs (MHz and MB of RAM) and if pondering was active, to be more complete.
    – djnavas
    Jan 5, 2021 at 3:53
  • 1
    Sopel gave the best answer some months back here but they did it as a comment that I can't mark as the accepted answer. The lichess stockfish bots only have 8 discrete difficulty options and do not surface where on the actual stockfish difficulty setting scale from 1-20 they correspond to, rendering a winrate ELO analysis of them useless for the purposes of determining strength of a direct interface to the engine.
    – day
    Jan 5, 2021 at 13:29
1

They have a development pipeline doing testing based on one of the early versions. I saw a graph of such test in some heavy loading page from Next Chess Move website:

The development page gathering the test linked from the Fishtest wiki:

And for completion:

Improve this answer
5
  • 2
    These resources seem to be for evaluating optimisations to the engine in development, I don't see any tests evaluating the engine set to levels 1-19, not sure that fishtest would be receptive to having levels 1-19 put in the system given their stated goals
    – day
    Jun 8, 2020 at 2:51
  • I have not looked into the details, but if I were testing for possible regression or improvements, i would use the whole dynamic range available (both sides), I would expect that somewhere is that mass of data there should be a way to get some approximate ELO relationship to the parameter volume explored. There was no answer. I hope there will be a better answer later on. I am also curious. Do you know the stated goals?
    – dbdb
    Jun 9, 2020 at 2:27
  • 1
    The results in the first two links don't seem to have any examples of stockfish set to any level besides the highest one. The stated goals of fishtest by the github wiki are for development testing, making the best version of stockfish better. I'm not sure how much it would be appreciated if I inserted levels 1-19 into the pool, and I'm pretty sure the lowest stockfish levels are in the triple digits and I don't think that throwing them in a pool with 3300 rated engines will help me find ratings to the precision I need.
    – day
    Jun 10, 2020 at 6:36
  • I don''t know where to put this. But it appears that "play with computer" on lichess web sites has some associations to its rating system. I wonder on what basis. But maybe you already know. Finding out how they do it, could provide for a way to ELO. However, there is the question for any paring pool based ratings of defining that pool.
    – dbdb
    Aug 7, 2021 at 1:07
  • 1
    Sopel already provided the best answer in a comment under the question, but they never actually wrote it up as an answer and stack exhange doesn't allow me to mark just a comment as the correct answer. Lichess engine levels have nothing to do with the actual stockfish setting levels, but they do use the resource sopel linked for estimation.
    – day
    Aug 8, 2021 at 3:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.