4

I want to make some statistics by analyzing the same position, multiple times, at different levels of Stockfish. Precisely, I would like Stockfish to analyze a specific position at different "Skill Level" but what I've noticed is that there is the "hash table" which keeps in memory the previous analyses. Somewhere I've found that if I specify the parameter game=object() the hash table has been reset but I'm not sure about that. What I want is that the second analysis is completely independent from the first one.

skill_levels=[2,10,20]
for i in range(len(skill_levels)):
    engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
    engine.configure({"Skill Level": skill_levels[i]})
    board = chess.Board(fens[f])
    dummy_move = engine.play(board, chess.engine.Limit(time=0.1), game=object()).move
    result = engine.analyse(board, chess.engine.Limit(time=1), game=object(),root_moves=[dummy_move])
    engine.quit()

How do you do this properly?

Second question: I suppose that limiting the engine on time chess.engine.Limit(time=1) I will get different results depending on the specs of the PC where Stockfish is running. How do I make the results somehow independent of that? Considering to use an average pc (i5 8th generation, 16gb of ram) which depth should I chose chess.engine.Limit(depth=20) the get more or less the same results of chess.engine.Limit(time=1)?

3
  • When you analyze the position by skill level what search info you would like to get? Is it the bestmove, the evaluation, nodes, nps?
    – ferdy
    May 27 at 1:18
  • the evaluation, precisely in Cp or Mate. But is that important?
    – Giorgio
    May 27 at 7:23
  • 1
    It is important because of your posted code. In that code you did not extract the score. I will post answer later on how you will get the score.
    – ferdy
    May 27 at 8:11

1 Answer 1

3

Use engine.play because you are using skill level, note use info parameter to get the score. You don't need to worry about the hash table because you quit the engine for every skill levels.

Method 1

Code
import chess.engine
import logging

# Uncomment to see the log in console.
# logging.basicConfig(level=logging.DEBUG)


stockfishpath = 'sf15.exe'
fens = [
    'r4rk1/5pp1/5q2/p2n2p1/2R3P1/7P/P1Q2P2/2R2BK1 w - - 0 1',
    'rn1qkbnr/pp2pppp/2p5/3p1b2/3P4/4PN2/PPP2PPP/RNBQKB1R w KQkq - 0 1'
]
skill_levels=[2,10,20]
analysis_time_sec = 2
hash_mb = 128

for fen in fens:
    print(f'FEN: {fen}')

    for s in skill_levels:
        engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
        engine.configure({"Skill Level": s})
        engine.configure({"Hash": hash_mb})
        board = chess.Board(fen)
        result = engine.play(board, chess.engine.Limit(time=analysis_time_sec), info=chess.engine.INFO_SCORE)
        engine.quit()

        bestmove = board.san(result.move)
        score_cp = result.info['score'].relative.score(mate_score=32000)
        score_mate = result.info['score'].relative.mate()

        print(f'skill level: {s}')
        print(f'score cp: {score_cp}')
        print(f'score mate: {score_mate}')
        print(f'bestmove: {bestmove}')
        print()

Output
FEN: r4rk1/5pp1/5q2/p2n2p1/2R3P1/7P/P1Q2P2/2R2BK1 w - - 0 1
skill level: 2
score cp: 57
score mate: None
bestmove: Rd1

skill level: 10
score cp: 57
score mate: None
bestmove: Rc6

skill level: 20
score cp: 45
score mate: None
bestmove: Rc6

FEN: rn1qkbnr/pp2pppp/2p5/3p1b2/3P4/4PN2/PPP2PPP/RNBQKB1R w KQkq - 0 1
skill level: 2
score cp: 34
score mate: None
bestmove: Nh4

skill level: 10
score cp: 34
score mate: None
bestmove: Bd3

skill level: 20
score cp: 23
score mate: None
bestmove: Bd3

Regarding your second question do the following:

  1. Run the engine
  2. Set engine options you like, e.g
    setoption name Hash value 128
  3. Send the command
    go movetime 1000
    that will search the engine for 1000 ms or 1 sec. Sample output:
info string NNUE evaluation using nn-6877cd24400e.nnue enabled
info depth 1 seldepth 1 multipv 1 score cp 30 nodes 20 nps 10000 tbhits 0 time 2 pv e2e4
info depth 2 seldepth 2 multipv 1 score cp 84 nodes 45 nps 22500 tbhits 0 time 2 pv e2e4 a7a6
info depth 3 seldepth 3 multipv 1 score cp 37 nodes 191 nps 63666 tbhits 0 time 3 pv c2c4 a7a6 e2e4
info depth 4 seldepth 4 multipv 1 score cp 158 nodes 264 nps 88000 tbhits 0 time 3 pv c2c4
info depth 5 seldepth 6 multipv 1 score cp 49 nodes 1449 nps 241500 tbhits 0 time 6 pv e2e4 d7d5 e4d5 d8d5 d2d4
info depth 6 seldepth 6 multipv 1 score cp 36 nodes 3954 nps 395400 tbhits 0 time 10 pv e2e4 c7c5 c2c3 b8c6
info depth 7 seldepth 9 multipv 1 score cp 37 nodes 6269 nps 447785 tbhits 0 time 14 pv e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 g8f6
info depth 8 seldepth 12 multipv 1 score cp 30 nodes 11049 nps 502227 tbhits 0 time 22 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 b7c6
info depth 9 seldepth 13 multipv 1 score cp 30 nodes 15347 nps 495064 tbhits 0 time 31 pv e2e4 e7e6 d2d4 d7d5 b1d2 g8f6 e4e5 f6d7 c2c3 c7c5 g1f3
info depth 10 seldepth 14 multipv 1 score cp 37 nodes 23259 nps 540906 tbhits 0 time 43 pv e2e4 c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1
info depth 11 seldepth 14 multipv 1 score cp 45 nodes 40064 nps 556444 tbhits 0 time 72 pv e2e4 c7c5 g1f3 e7e6 g2g3 d7d6 d2d4 c5d4 f3d4 g8f6
info depth 12 seldepth 15 multipv 1 score cp 38 nodes 57767 nps 571950 tbhits 0 time 101 pv e2e4 c7c5 g1f3 b8c6 f1b5 c6d4 b5e2 e7e6 f3d4 c5d4 e1g1 a7a6
info depth 13 seldepth 15 multipv 1 score cp 46 nodes 97551 nps 602166 tbhits 0 time 162 pv d2d4 d7d5 g1f3 g8f6 c2c4 e7e6 g2g3 f8e7 f1g2 e8g8 e1g1 d5c4 b1d2
info depth 14 seldepth 20 multipv 1 score cp 38 nodes 152736 nps 606095 tbhits 0 time 252 pv d2d4 d7d5 g1f3 g8f6 c2c4 c7c6 b1c3 g7g6 e2e3 f8g7 f1e2 e8g8 e1g1 d5c4 e2c4
info depth 15 seldepth 17 multipv 1 score cp 43 nodes 214622 nps 614962 tbhits 0 time 349 pv e2e4 c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 b8d7 e1g1 g7g6 f1e1 g8f6 c2c3 f8g7 d2d4 c5d4
info depth 16 seldepth 23 multipv 1 score cp 33 nodes 413957 nps 619696 tbhits 0 time 668 pv e2e4 c7c5 g1f3 d7d6 f1b5 b8d7 e1g1 a7a6 b5d7 c8d7 f1e1 e7e5 c2c3 g8e7 d2d4 e7g6 d4e5 d6e5 c3c4 f8e7 b1c3 e8g8 c3d5
info depth 17 seldepth 24 multipv 1 score cp 35 nodes 525134 nps 621460 tbhits 0 time 845 pv e2e4 c7c5 c2c3 g8f6 e4e5 f6d5 d2d4 c5d4 c3d4 b8c6 g1f3 e7e6 b1c3 d7d6 f1d3 d6e5 d4e5 f8b4
info depth 18 seldepth 24 multipv 1 score cp 35 nodes 619893 nps 618655 hashfull 37 tbhits 0 time 1002 pv e2e4 c7c5 c2c3 g8f6 e4e5 f6d5 d2d4 c5d4 c3d4 b8c6 g1f3 e7e6 b1c3 d7d6 f1d3 d6e5 d4e5 f8b4
bestmove e2e4 ponder c7c5

So in my PC the depth reached in single thread for 1 sec search is 18.

Method 2

Code
"""
Evaluate positions with skill levels and get the correct score
of the move corresponding to the skill level. Also Applicable
to uci engines that does not show correct score for the move when playing suboptimal moves.
"""

import chess.engine
# import logging

# Uncomment to see the log in console.
# logging.basicConfig(level=logging.DEBUG)


def analyze_position(stockfishpath, board, root_moves=None, skill_level=20, hash_mb=16, time_sec=1):
    """
    Analyze board position and return bestmove, score_cp and score_mate.
    """
    engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
    engine.configure({"Skill Level": skill_level})
    engine.configure({"Hash": hash_mb})
    result = engine.play(board, chess.engine.Limit(time=time_sec), info=chess.engine.INFO_SCORE, root_moves=root_moves)
    engine.quit()

    score_cp = result.info['score'].relative.score(mate_score=32000)
    score_mate = result.info['score'].relative.mate()
    bestmove = result.move

    return bestmove, score_cp, score_mate


def main():
    stockfishpath = 'sf15.exe'
    fens = [
        '3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1'
    ]
    skill_levels=[2,10,20]
    analysis_time_sec = 1
    hash_mb = 128

    for fen in fens:
        print(f'FEN: {fen}')

        for s in skill_levels:
            board = chess.Board(fen)
            bestmove, score_cp, score_mate = analyze_position(
                stockfishpath, board, root_moves=None, skill_level=s,
                hash_mb=hash_mb, time_sec=analysis_time_sec)

            # Restart the engine to get the actual score of the bestmove returned when skill level is not best.
            # This is because the current stockfish (v15) does not report the correct score based on the bestmove it returned.
            # We will use the searchmoves <move> command to restrict the engine to search only this move.
            if s < 20:
                # We will set the max skill level because we are after on the correct score of the given move.
                max_s = 20
                root_move = [bestmove]
                bestmove, score_cp, score_mate = analyze_position(
                    stockfishpath, board, root_moves=root_move, skill_level=max_s,
                    hash_mb=hash_mb, time_sec=analysis_time_sec)

            print(f'skill level: {s}')
            print(f'score cp: {score_cp}')
            print(f'score mate: {score_mate}')
            print(f'bestmove: {board.san(bestmove)}')
            print()


if __name__ == '__main__':
    main()
Output
FEN: 3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1
skill level: 2
score cp: 31996
score mate: 4
bestmove: Kc7

skill level: 10
score cp: 31995
score mate: 5
bestmove: Bb6+

skill level: 20
score cp: 31997
score mate: 3
bestmove: Qd1
5
  • thank you very much for your reply. That's exactly what I needed. But I have a problem, probably I cannot understand the result properly. Consider this position '3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1' this is what I get FEN: 3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1 skill level: 2 score cp: 31997 score mate: 3 bestmove: Kc7 why at skill level 2, the socore mate is 3 but the best move is Kc7 which does not leat to a mate in 3? Sumilarly for skill level = 10
    – Giorgio
    May 30 at 18:49
  • The engine suggests a weak move but the score it sends is not based from the suggested move this is a bug in the engine. Do you also want an accurate score based on the given move when skill level is not best? I will add new code later to my answer to determine accurate score of the move sent by the engine when skill level is not best.
    – ferdy
    May 30 at 23:17
  • Added method 2 in the answer to get the correct score of the bestmove returned when skill level is not best or 20.
    – ferdy
    May 31 at 0:08
  • it's perfect! thank you! Where does the 'mate_score=32000' come from?
    – Giorgio
    Jun 1 at 16:22
  • 32k is from sf source code.
    – ferdy
    Jun 1 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.