3

When I let Stockfish get to a certain depth (say depth 33), it gives some evaluation. Then I refresh the position (while leaving Stockfish on) and when it gets to the same depth it gives a different evaluation.

I know it uses previous analysis stored in a hash table to get to the same depth quicker the second time around, but why it can give a different evaluation puzzles me. Previous analysis should only improve search speed for a deterministic engine.

For anyone familiar with Stockfish's source code, any reasons this could be?

Improve this question
5
  • The answers in randomness in engine play aren't clear as why when reaching the same search depth, in two or more analysis of the same position, Stockfish give different evaluations. Even more, at a greater depth of analysis the invariant movements between the calculated variants are less and less, until reaching a minimum of 31 plays deep.
    – djnavas
    Sep 1, 2019 at 8:18
  • 2
    This question is not a duplicate. It is assuming a deterministic search without timing and multi-threading related non-determinism, so the answers from the other thread do not apply here.
    – Fabian Fichter
    Sep 1, 2019 at 9:09
  • Short answer: Stockfish is not God
    – David
    Sep 2, 2019 at 7:51
  • @David An entity giving consistent evaluations doesn't imply it's god :)
    – Inertial Ignorance
    Sep 2, 2019 at 8:42
  • it also depends on how you run stockfish, if you stop it manually by clicking on a graphical interface then you risk not stopping at exactly the same point in the eval.
    – Purefan
    Apr 11, 2022 at 20:53

1 Answer 1

Reset to default
7

If Stockfish was a pure minimax/alpha-beta searcher with all leaves having a predetermined depth, your assumption that the search result should be reproducible would hold true, even with a different state of the hash table. That is because the result in minimax/alpha-beta only depends on the evaluation function, but not on the order in which the moves are searched, which only has an impact on speed.

However, in fact Stockfish is using a lot of forward pruning, reduction and extension techniques that are state/history dependent, so the results of the search of a subtree depends on the state/history with which it entered the subtree. Therefore, starting with hash table entries that influence move ordering (and can actually return a different evaluation than searching the subtree again) leads to a different result of for a search of same depth.

There is one more detail in the Stockfish case why even with pure alpha-beta search re-searching the position again would give different results. The evaluation of draws is pseudo-randomized with small fluctuation from zero in order to avoid some search artefacts in case of repetition draws, and this pseudo-random number depends on the node count of the current search. Therefore, using a different move ordering due to the hash entries would lead to different pseudo-random numbers, and hence potentially a different search result.

Improve this answer
4
  • 1
    Very interesting, I wasn't aware of forward pruning. I know alpha-beta engines do move ordering (the ability to do this is the main advantage of a transposition table). But what you're saying is that the subtrees of the lowest ranked moves are ignored altogether, and not just looked at last?
    – Inertial Ignorance
    Sep 1, 2019 at 9:16
  • Yes, this is why modern chess engines can sometimes be blind to solutions of quite artificial studies, because they might just prune the best move (e.g., because it is sacrificing a piece for seemingly nothing). However, these pruning techniques only apply to small remaining depths, so as the total depth grows, it gets more and more unlikely that the critical move is pruned, so the search should still converge to the correct move.
    – Fabian Fichter
    Sep 1, 2019 at 9:21
  • But the details of the pruning, reduction and extension techniques would probably go beyond the scope of this thread. chessprogramming.org/Selectivity should be a starting point, otherwise you can maybe ask it as a new question.
    – Fabian Fichter
    Sep 1, 2019 at 9:31
  • Makes sense. I figured the pruning would only happen when the engine is near the depth limit, since if a move doesn't have a high evaluation at that point it's most likely bad.
    – Inertial Ignorance
    Sep 1, 2019 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.