6

I want to create something that will select an appropriate Stockfish difficulty setting automatically given the player's rating. The straight Stockfish v11 that I've downloaded has 20 difficulty settings, and various other settings for controlling thinking time. Is there a resource that has approximate ratings for the levels? If that doesn't exist, what method would be recommended for finding the approximate ratings of chess engines when I'm starting from scratch?

5
  • Not sure how you could really compare a human rating with a computer difficulty level. When you play a computer you pretty much are just playing against yourself. You are playing against your own knowledge and experience. In any given position you are trying to use your own experience to play the best move possible. The level of the computer really only determines how much you will be punished if you don't make the best possible move. – musefan Jun 5 '20 at 10:08
  • The design of the system I'm building won't really have a good way for the "player" to select the difficulty of the engine, and the "player" will only be able to play against engines. For my purposes, I want to know the rough equivalent ELO of the stockfish levels so that I can calculate adjustments to ELO when the player wins/loses, and so that I can select an appropriate rated engine for the next game. "Player" in quotation marks because the system is not going to have one human player – Davidos Jun 5 '20 at 12:35
  • 1
    github.com/official-stockfish/Stockfish/pull/2225 is probably the best you'll get – Sopel Jun 8 '20 at 22:33
  • Answer from sopel actually looks like the best answer. The files changed section in the link given describes changes to search.cpp, which contains the exact relation. Proving table for the relation given in the summary for peace of mind. Post link as an answer and I'll tick it. – Davidos Jun 10 '20 at 6:33
  • Here is a list of engine strenght. I think it's quite realistic. ccrl.chessdom.com/ccrl/4040 Hope it helps. – David Jan 12 at 11:39
1

They have a development pipeline doing testing based on one of the early versions. I saw a graph of such test in some heavy loading page from Next Chess Move website:

The development page gathering the test linked from the Fishtest wiki:

And for completion:

3
  • 1
    These resources seem to be for evaluating optimisations to the engine in development, I don't see any tests evaluating the engine set to levels 1-19, not sure that fishtest would be receptive to having levels 1-19 put in the system given their stated goals – Davidos Jun 8 '20 at 2:51
  • I have not looked into the details, but if I were testing for possible regression or improvements, i would use the whole dynamic range available (both sides), I would expect that somewhere is that mass of data there should be a way to get some approximate ELO relationship to the parameter volume explored. There was no answer. I hope there will be a better answer later on. I am also curious. Do you know the stated goals? – dbdb Jun 9 '20 at 2:27
  • The results in the first two links don't seem to have any examples of stockfish set to any level besides the highest one. The stated goals of fishtest by the github wiki are for development testing, making the best version of stockfish better. I'm not sure how much it would be appreciated if I inserted levels 1-19 into the pool, and I'm pretty sure the lowest stockfish levels are in the triple digits and I don't think that throwing them in a pool with 3300 rated engines will help me find ratings to the precision I need. – Davidos Jun 10 '20 at 6:36
1

I am rated 1410 USCF. I played all 8 Stockfish levels this past weekend, once with each color. My record was 11-5. I was 2-0 vs Stockfish 1-5, 1-1 vs Stockfish 6, and 0-2 vs Stockfish 7 and 8.

2
  • The most concrete answer, based on experience. Only that lacks the reflection time, the hardware were it runs (MHz and MB of RAM) and if pondering was active, to be more complete. – djnavas Jan 5 at 3:53
  • Sopel gave the best answer some months back here but they did it as a comment that I can't mark as the accepted answer. The lichess stockfish bots only have 8 discrete difficulty options and do not surface where on the actual stockfish difficulty setting scale from 1-20 they correspond to, rendering a winrate ELO analysis of them useless for the purposes of determining strength of a direct interface to the engine. – Davidos Jan 5 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.