How do you nerf Stockfish to analyze the same position multiple times?

Question

I want to make some statistics by analyzing the same position, multiple times, at different levels of Stockfish. Precisely, I would like Stockfish to analyze a specific position at different "Skill Level" but what I've noticed is that there is the "hash table" which keeps in memory the previous analyses. Somewhere I've found that if I specify the parameter game=object() the hash table has been reset but I'm not sure about that. What I want is that the second analysis is completely independent from the first one.

skill_levels=[2,10,20]
for i in range(len(skill_levels)):
    engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
    engine.configure({"Skill Level": skill_levels[i]})
    board = chess.Board(fens[f])
    dummy_move = engine.play(board, chess.engine.Limit(time=0.1), game=object()).move
    result = engine.analyse(board, chess.engine.Limit(time=1), game=object(),root_moves=[dummy_move])
    engine.quit()

How do you do this properly?

Second question: I suppose that limiting the engine on time chess.engine.Limit(time=1) I will get different results depending on the specs of the PC where Stockfish is running. How do I make the results somehow independent of that? Considering to use an average pc (i5 8th generation, 16gb of ram) which depth should I chose chess.engine.Limit(depth=20) the get more or less the same results of chess.engine.Limit(time=1)?

When you analyze the position by skill level what search info you would like to get? Is it the bestmove, the evaluation, nodes, nps? — ferdy, Commented May 27, 2022 at 1:18
the evaluation, precisely in Cp or Mate. But is that important? — Giorgio, Commented May 27, 2022 at 7:23
It is important because of your posted code. In that code you did not extract the score. I will post answer later on how you will get the score. — ferdy, Commented May 27, 2022 at 8:11

ferdy · Accepted Answer · 2022-05-31 22:52:22Z

Use engine.play because you are using skill level, note use info parameter to get the score. You don't need to worry about the hash table because you quit the engine for every skill levels.

Method 1

Code

import chess.engine
import logging

# Uncomment to see the log in console.
# logging.basicConfig(level=logging.DEBUG)


stockfishpath = 'sf15.exe'
fens = [
    'r4rk1/5pp1/5q2/p2n2p1/2R3P1/7P/P1Q2P2/2R2BK1 w - - 0 1',
    'rn1qkbnr/pp2pppp/2p5/3p1b2/3P4/4PN2/PPP2PPP/RNBQKB1R w KQkq - 0 1'
]
skill_levels=[2,10,20]
analysis_time_sec = 2
hash_mb = 128

for fen in fens:
    print(f'FEN: {fen}')

    for s in skill_levels:
        engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
        engine.configure({"Skill Level": s})
        engine.configure({"Hash": hash_mb})
        board = chess.Board(fen)
        result = engine.play(board, chess.engine.Limit(time=analysis_time_sec), info=chess.engine.INFO_SCORE)
        engine.quit()

        bestmove = board.san(result.move)
        score_cp = result.info['score'].relative.score(mate_score=32000)
        score_mate = result.info['score'].relative.mate()

        print(f'skill level: {s}')
        print(f'score cp: {score_cp}')
        print(f'score mate: {score_mate}')
        print(f'bestmove: {bestmove}')
        print()

Output

FEN: r4rk1/5pp1/5q2/p2n2p1/2R3P1/7P/P1Q2P2/2R2BK1 w - - 0 1
skill level: 2
score cp: 57
score mate: None
bestmove: Rd1

skill level: 10
score cp: 57
score mate: None
bestmove: Rc6

skill level: 20
score cp: 45
score mate: None
bestmove: Rc6

FEN: rn1qkbnr/pp2pppp/2p5/3p1b2/3P4/4PN2/PPP2PPP/RNBQKB1R w KQkq - 0 1
skill level: 2
score cp: 34
score mate: None
bestmove: Nh4

skill level: 10
score cp: 34
score mate: None
bestmove: Bd3

skill level: 20
score cp: 23
score mate: None
bestmove: Bd3

Regarding your second question do the following:

Run the engine
Set engine options you like, e.g
setoption name Hash value 128
Send the command
go movetime 1000
that will search the engine for 1000 ms or 1 sec. Sample output:

info string NNUE evaluation using nn-6877cd24400e.nnue enabled
info depth 1 seldepth 1 multipv 1 score cp 30 nodes 20 nps 10000 tbhits 0 time 2 pv e2e4
info depth 2 seldepth 2 multipv 1 score cp 84 nodes 45 nps 22500 tbhits 0 time 2 pv e2e4 a7a6
info depth 3 seldepth 3 multipv 1 score cp 37 nodes 191 nps 63666 tbhits 0 time 3 pv c2c4 a7a6 e2e4
info depth 4 seldepth 4 multipv 1 score cp 158 nodes 264 nps 88000 tbhits 0 time 3 pv c2c4
info depth 5 seldepth 6 multipv 1 score cp 49 nodes 1449 nps 241500 tbhits 0 time 6 pv e2e4 d7d5 e4d5 d8d5 d2d4
info depth 6 seldepth 6 multipv 1 score cp 36 nodes 3954 nps 395400 tbhits 0 time 10 pv e2e4 c7c5 c2c3 b8c6
info depth 7 seldepth 9 multipv 1 score cp 37 nodes 6269 nps 447785 tbhits 0 time 14 pv e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 g8f6
info depth 8 seldepth 12 multipv 1 score cp 30 nodes 11049 nps 502227 tbhits 0 time 22 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 b7c6
info depth 9 seldepth 13 multipv 1 score cp 30 nodes 15347 nps 495064 tbhits 0 time 31 pv e2e4 e7e6 d2d4 d7d5 b1d2 g8f6 e4e5 f6d7 c2c3 c7c5 g1f3
info depth 10 seldepth 14 multipv 1 score cp 37 nodes 23259 nps 540906 tbhits 0 time 43 pv e2e4 c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 d8d7 e1g1
info depth 11 seldepth 14 multipv 1 score cp 45 nodes 40064 nps 556444 tbhits 0 time 72 pv e2e4 c7c5 g1f3 e7e6 g2g3 d7d6 d2d4 c5d4 f3d4 g8f6
info depth 12 seldepth 15 multipv 1 score cp 38 nodes 57767 nps 571950 tbhits 0 time 101 pv e2e4 c7c5 g1f3 b8c6 f1b5 c6d4 b5e2 e7e6 f3d4 c5d4 e1g1 a7a6
info depth 13 seldepth 15 multipv 1 score cp 46 nodes 97551 nps 602166 tbhits 0 time 162 pv d2d4 d7d5 g1f3 g8f6 c2c4 e7e6 g2g3 f8e7 f1g2 e8g8 e1g1 d5c4 b1d2
info depth 14 seldepth 20 multipv 1 score cp 38 nodes 152736 nps 606095 tbhits 0 time 252 pv d2d4 d7d5 g1f3 g8f6 c2c4 c7c6 b1c3 g7g6 e2e3 f8g7 f1e2 e8g8 e1g1 d5c4 e2c4
info depth 15 seldepth 17 multipv 1 score cp 43 nodes 214622 nps 614962 tbhits 0 time 349 pv e2e4 c7c5 g1f3 d7d6 f1b5 c8d7 b5d7 b8d7 e1g1 g7g6 f1e1 g8f6 c2c3 f8g7 d2d4 c5d4
info depth 16 seldepth 23 multipv 1 score cp 33 nodes 413957 nps 619696 tbhits 0 time 668 pv e2e4 c7c5 g1f3 d7d6 f1b5 b8d7 e1g1 a7a6 b5d7 c8d7 f1e1 e7e5 c2c3 g8e7 d2d4 e7g6 d4e5 d6e5 c3c4 f8e7 b1c3 e8g8 c3d5
info depth 17 seldepth 24 multipv 1 score cp 35 nodes 525134 nps 621460 tbhits 0 time 845 pv e2e4 c7c5 c2c3 g8f6 e4e5 f6d5 d2d4 c5d4 c3d4 b8c6 g1f3 e7e6 b1c3 d7d6 f1d3 d6e5 d4e5 f8b4
info depth 18 seldepth 24 multipv 1 score cp 35 nodes 619893 nps 618655 hashfull 37 tbhits 0 time 1002 pv e2e4 c7c5 c2c3 g8f6 e4e5 f6d5 d2d4 c5d4 c3d4 b8c6 g1f3 e7e6 b1c3 d7d6 f1d3 d6e5 d4e5 f8b4
bestmove e2e4 ponder c7c5

So in my PC the depth reached in single thread for 1 sec search is 18.

Method 2

Code

"""
Evaluate positions with skill levels and get the correct score
of the move corresponding to the skill level. Also Applicable
to uci engines that does not show correct score for the move when playing suboptimal moves.
"""

import chess.engine
# import logging

# Uncomment to see the log in console.
# logging.basicConfig(level=logging.DEBUG)


def analyze_position(stockfishpath, board, root_moves=None, skill_level=20, hash_mb=16, time_sec=1):
    """
    Analyze board position and return bestmove, score_cp and score_mate.
    """
    engine = chess.engine.SimpleEngine.popen_uci(stockfishpath)
    engine.configure({"Skill Level": skill_level})
    engine.configure({"Hash": hash_mb})
    result = engine.play(board, chess.engine.Limit(time=time_sec), info=chess.engine.INFO_SCORE, root_moves=root_moves)
    engine.quit()

    score_cp = result.info['score'].relative.score(mate_score=32000)
    score_mate = result.info['score'].relative.mate()
    bestmove = result.move

    return bestmove, score_cp, score_mate


def main():
    stockfishpath = 'sf15.exe'
    fens = [
        '3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1'
    ]
    skill_levels=[2,10,20]
    analysis_time_sec = 1
    hash_mb = 128

    for fen in fens:
        print(f'FEN: {fen}')

        for s in skill_levels:
            board = chess.Board(fen)
            bestmove, score_cp, score_mate = analyze_position(
                stockfishpath, board, root_moves=None, skill_level=s,
                hash_mb=hash_mb, time_sec=analysis_time_sec)

            # Restart the engine to get the actual score of the bestmove returned when skill level is not best.
            # This is because the current stockfish (v15) does not report the correct score based on the bestmove it returned.
            # We will use the searchmoves <move> command to restrict the engine to search only this move.
            if s < 20:
                # We will set the max skill level because we are after on the correct score of the given move.
                max_s = 20
                root_move = [bestmove]
                bestmove, score_cp, score_mate = analyze_position(
                    stockfishpath, board, root_moves=root_move, skill_level=max_s,
                    hash_mb=hash_mb, time_sec=analysis_time_sec)

            print(f'skill level: {s}')
            print(f'score cp: {score_cp}')
            print(f'score mate: {score_mate}')
            print(f'bestmove: {board.san(bestmove)}')
            print()


if __name__ == '__main__':
    main()

Output

FEN: 3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1
skill level: 2
score cp: 31996
score mate: 4
bestmove: Kc7

skill level: 10
score cp: 31995
score mate: 5
bestmove: Bb6+

skill level: 20
score cp: 31997
score mate: 3
bestmove: Qd1

thank you very much for your reply. That's exactly what I needed. But I have a problem, probably I cannot understand the result properly. Consider this position '3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1' this is what I get FEN: 3K4/8/8/p2k4/pp1B4/N5N1/P2Q4/8 w - - 0 1 skill level: 2 score cp: 31997 score mate: 3 bestmove: Kc7 why at skill level 2, the socore mate is 3 but the best move is Kc7 which does not leat to a mate in 3? Sumilarly for skill level = 10 — Giorgio, Commented May 30, 2022 at 18:49
The engine suggests a weak move but the score it sends is not based from the suggested move this is a bug in the engine. Do you also want an accurate score based on the given move when skill level is not best? I will add new code later to my answer to determine accurate score of the move sent by the engine when skill level is not best. — ferdy, Commented May 30, 2022 at 23:17
Added method 2 in the answer to get the correct score of the bestmove returned when skill level is not best or 20. — ferdy, Commented May 31, 2022 at 0:08
it's perfect! thank you! Where does the 'mate_score=32000' come from? — Giorgio, Commented Jun 1, 2022 at 16:22

Stack Exchange Network

How do you nerf Stockfish to analyze the same position multiple times?

1 Answer 1

Method 1

Code

Output

Method 2

Code

Output

Your Answer

Not the answer you're looking for? Browse other questions tagged
engines
stockfish
computer-chess
python
or ask your own question.

Hot Network Questions

How do you nerf Stockfish to analyze the same position multiple times?

1 Answer 1

Method 1

Code

Output

Method 2

Code

Output

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged enginesstockfishcomputer-chesspython or ask your own question.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
engines
stockfish
computer-chess
python
or ask your own question.