What do chess engine evaluation scores (+1.3, -3.2) really mean?

Question

I know that chess engine evaluations like +1.3 intuitively mean that White is evaluated to have an advantage equivalent to 1.3 pawns of material, or that -3.2 means that Black is evaluated to have an advantage equivalent to a little over a knight or bishop. But I am struggling to pin down what "having an advantage equivalent to" really means or how it could be implemented.

Such evaluations do not simply count the material on the board, but rather might translate into such an advantage in future. For example, an evaluation of -3.2 might be because Black, through optimal play, can force the capture of a white Bishop "for free" in the next couple of moves.

Other cases might not be so clearly linked to material advantage. For example, although currently level in material, Black, through optimal play, may gain a strong tactical advantage in future due to the white King being open.

My doubt here is that an engine will inevitably turn a -3.2 advantage into a future win for Black, even if the full path to checkmate is not clear. So the further we look into the future, the stronger the advantage will inevitably become for Black.

Hence my confusion. It seems that there is some "future looking" aspect to the evaluations, but depending on how far into the future we look, the evaluation for the current position changes.

Trying to "backwards-engineer" the concept, it seems that maybe there is a fixed window of turns somehow in which the advantage is considered, but this again seems material-based whereas an advantage might never translate into a material advantage in the short term. Perhaps another way would be to consider how much material must be given to the other side right now in order to make the position equal, but then there's the question of what material, and where to place that material.

Chess engines (not based on supervised learning at least) must implement this evaluation function in some way, and I guess that there is some standard definition in order to make the scores of different engines comparable (though I might be wrong).

My question is: is there a standard/typical, rigorous definition for these chess engine evaluations, and if so, what is that definition?

Answer 1

One thing you must understand is that any chess engine using negamax/minimax with alpha-beta pruning and other heuristics uses a static evaluation function at the leaves of the search-tree. This function returns terminal values (e.g. win-in-0, loss-in-0, draw-in-0), or estimates of 'position value' when the game has not ended, and these values propagate up the search-tree. The engine eventually chooses a move that maximizes the guaranteed value of a leaf that it can reach in that tree no matter what the opponent does. Now, of course there is no doubt about win-in-k or loss-in-k values; if any correct engine evaluates a position to have such a value, it means it has proven that the indicated side can guarantee a win within k moves. The question is what do other values mean.

SF uses some big value M (≈64) to indicate almost sure win (overwhelming advantage), which shows up especially when one side will have an unstoppable pawn promotion and the other side has too little material. As per the above explanation, if SF evaluates a position to have value ≈ M or ≈ −M, that value implies guaranteed almost sure win for the indicated side. SF also has other 'tiers' for different levels of confidence.

But besides these coarse 'tiers' SF also has fine-tuned heuristic factors that are evaluated to add little bits of 'positional value' to each side, such as king-danger and bishop-pair. These do not correspond directly to material, but the values have been tuned so that SF makes the correct decision with the highest frequency that its designers could achieve using those heuristic factors. At least this is what is happening with SF10.

In fact, although each piece is given a value (piece-value-bonus), even this value changes over time (piece-value-mg to piece-value-eg), and the square it is on is also taken into account (piece-square-bonus), along with many other things. So it is hard to even say that +1.3 represents roughly 1.3 pawns. Not at all, really.

So there is not really any material meaning to an evaluation of +1.3 by SF10. Rather:

• SF10 has searched a tree T and found that if both players only make moves given in that tree T then White can guarantee to reach a leaf node of T with static evaluation value at least +1.3 and Black can guarantee to reach a leaf node of T with static evaluation value at most +1.3.
• In extensive testing, it is found that generally SF10 performs better with the weights it uses in its heuristics than with other weights. It does often mean that a position with higher static evaluation than another position is more likely to be better for White, but it can be very wrong, for example in this position:

[Title "Bad for Black"]
[FEN "q1k5/PpP2ppp/1Pp5/2N5/8/8/7P/K7 w - - 0 1"]

The static evaluation for this position (you can try using the link above) is solidly in Black's favour, but it is wrong! This shows that the static evaluation of leaves of the search-tree can be very inaccurate. However, this problem is mitigated strongly by the search-tree itself! Since SF never plays based only on static evaluation only, its chosen move tends to be correct simply because incorrectly evaluated leaves are rare. The primary approach is quiescence search, which extends the search-tree at nodes that are deemed to be not quiet (e.g. just after check or capture), and hence usually avoids erroneous evaluation based on a misleading material balance in the middle of a piece exchange. This approach does not help in the position I showed above, and SF10 depth 20 on Lichess evaluates it at ≈ −3.

But SF10 will rarely get into such problematic positions in the first place! This is because quiescence search ensures that the vast majority of leaf nodes of the search-tree have roughly correct relative static evaluations (and here all the tiny bits help), and so choosing the best move based on that tree not only almost always turns out to be the best move but also almost always avoids subtrees where the future search would have many such problematic positions at the search-tree leaves! For instance, to get to the above illustrated position Black not only has to hide the queen in a corner but also get the king stuck! This is very heavily disfavoured by many of the heuristic factors including king-danger and piece-mobility at earlier moves, so SF10 would never even have to evaluate such problematic positions in the first place.

Answer 2

Chess engines search the game tree and return the evaluation of the best final position in the tree. The evaluation corresponds to the static evaluation (i.e. the evaluation function of the engine evaluated on that position), regardless of whether it is a NN evaluation, a NNUE evaluation, or a handcrafted evaluation function. There is no "future-looking". Evaluation functions cannot look into the future; that is what the search algorithm does.

I'm guessing a lot of your confusion stems from the fact that engine evaluations have long passed the point of just counting material. It factors in the locations of the pieces, king safety, pawn structure, whether there are opposite color bishops on the board, and a whole lot of other factors.

Finally, some evaluation functions don't actually return a +x.xx centipawn eval. They return a win percentage, that is, an estimate of how likely it is that they will win from that position. That estimate is then converted to centipawns with a formula that depends on the engine, e.g. Leela's conversion from win % to centipawns is cp = 90 * tan(1.5637541897 * q), where q = win probability + draw probability*drawscore - loss probability [1]. If you are having trouble understanding centipawn eval, it might be more helpful to think of it as a win percentage.

[1] "drawscore" is a factor which allows you to influence how Leela treats draws. A negative drawscore for example pushes the engine to avoid draws, because draws would add a negative amount to the eval. It is similar to contempt in traditional engines.

Answer 3

The evaluation is based on general results from the engine’s tablebase. It thinks by positional means, and looks forward a few moves. It looks in its programmed tables and sees, ‘Ah, an open file is worth about .5 pawns (for example).’ Then it sees if it is also in this case a .5 advantage, and using this for many more details in the position, the computer calculates the approximate advantage for white or black.