6

I want to create something that will select an appropriate Stockfish difficulty setting automatically given the player's rating. The straight Stockfish v11 that I've downloaded has 20 difficulty settings, and various other settings for controlling thinking time. Is there a resource that has approximate ratings for the levels? If that doesn't exist, what method would be recommended for finding the approximate ratings of chess engines when I'm starting from scratch?

EDIT: Sopel gave a link some time ago, that answers the problem, although they did it in comment form so I can't mark it as the correct answer

6
  • Not sure how you could really compare a human rating with a computer difficulty level. When you play a computer you pretty much are just playing against yourself. You are playing against your own knowledge and experience. In any given position you are trying to use your own experience to play the best move possible. The level of the computer really only determines how much you will be punished if you don't make the best possible move.
    – musefan
    Jun 5 '20 at 10:08
  • The design of the system I'm building won't really have a good way for the "player" to select the difficulty of the engine, and the "player" will only be able to play against engines. For my purposes, I want to know the rough equivalent ELO of the stockfish levels so that I can calculate adjustments to ELO when the player wins/loses, and so that I can select an appropriate rated engine for the next game. "Player" in quotation marks because the system is not going to have one human player
    – Davidos
    Jun 5 '20 at 12:35
  • 1
    github.com/official-stockfish/Stockfish/pull/2225 is probably the best you'll get
    – Sopel
    Jun 8 '20 at 22:33
  • 1
    Answer from sopel actually looks like the best answer. The files changed section in the link given describes changes to search.cpp, which contains the exact relation. Proving table for the relation given in the summary for peace of mind. Post link as an answer and I'll tick it.
    – Davidos
    Jun 10 '20 at 6:33
  • 1
    Here is a list of engine strenght. I think it's quite realistic. ccrl.chessdom.com/ccrl/4040 Hope it helps.
    – David
    Jan 12 at 11:39
2

I am rated 1410 USCF. I played all 8 Stockfish levels this past weekend, once with each color. My record was 11-5. I was 2-0 vs Stockfish 1-5, 1-1 vs Stockfish 6, and 0-2 vs Stockfish 7 and 8.

2
  • The most concrete answer, based on experience. Only that lacks the reflection time, the hardware were it runs (MHz and MB of RAM) and if pondering was active, to be more complete.
    – djnavas
    Jan 5 at 3:53
  • 1
    Sopel gave the best answer some months back here but they did it as a comment that I can't mark as the accepted answer. The lichess stockfish bots only have 8 discrete difficulty options and do not surface where on the actual stockfish difficulty setting scale from 1-20 they correspond to, rendering a winrate ELO analysis of them useless for the purposes of determining strength of a direct interface to the engine.
    – Davidos
    Jan 5 at 13:29
1

They have a development pipeline doing testing based on one of the early versions. I saw a graph of such test in some heavy loading page from Next Chess Move website:

The development page gathering the test linked from the Fishtest wiki:

And for completion:

5
  • 2
    These resources seem to be for evaluating optimisations to the engine in development, I don't see any tests evaluating the engine set to levels 1-19, not sure that fishtest would be receptive to having levels 1-19 put in the system given their stated goals
    – Davidos
    Jun 8 '20 at 2:51
  • I have not looked into the details, but if I were testing for possible regression or improvements, i would use the whole dynamic range available (both sides), I would expect that somewhere is that mass of data there should be a way to get some approximate ELO relationship to the parameter volume explored. There was no answer. I hope there will be a better answer later on. I am also curious. Do you know the stated goals?
    – dbdb
    Jun 9 '20 at 2:27
  • 1
    The results in the first two links don't seem to have any examples of stockfish set to any level besides the highest one. The stated goals of fishtest by the github wiki are for development testing, making the best version of stockfish better. I'm not sure how much it would be appreciated if I inserted levels 1-19 into the pool, and I'm pretty sure the lowest stockfish levels are in the triple digits and I don't think that throwing them in a pool with 3300 rated engines will help me find ratings to the precision I need.
    – Davidos
    Jun 10 '20 at 6:36
  • I don''t know where to put this. But it appears that "play with computer" on lichess web sites has some associations to its rating system. I wonder on what basis. But maybe you already know. Finding out how they do it, could provide for a way to ELO. However, there is the question for any paring pool based ratings of defining that pool.
    – dbdb
    Aug 7 at 1:07
  • Sopel already provided the best answer in a comment under the question, but they never actually wrote it up as an answer and stack exhange doesn't allow me to mark just a comment as the correct answer. Lichess engine levels have nothing to do with the actual stockfish setting levels, but they do use the resource sopel linked for estimation.
    – Davidos
    Aug 8 at 3:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.