What are the scales of the numerical evaluations of the strongest engines based on?

Question

Different engines have different scales for their numerical evaluations (for example, Houdini's evaluations are usually much lower than Stockfish's evaluations). This is because different engines use different units in their evaluation functions.

.

The Fritz 12 instructions manual says:

Chess engines evaluate positions with the help of a numeric value. The evaluation is expressed in pawn units, always from the point of view of White. If the program is displaying a value of +1.30, this means that it considers the white position to be better by the equivalent of 1.3 pawns. If White is actually a pawn up, then the additional 0.3 is the result of positional considerations (mobility, deployment of pieces, king safety, pawn structure, etc.).

This means that Fritz 12 evaluations are calibrated so that +1.00 is equivalent to being 1 Pawn up. (I'm not sure if this is still true for Deep Fritz 14 though...)

.

ChessBase says:

Houdini 4 uses calibrated evaluations in which engine scores correlate directly with the win expectancy in the position. A +1.00 pawn advantage gives a 80% chance of winning the game against an equal opponent at blitz time control. At +2.00 the engine will win 95% of the time, and at +3.00 about 99% of the time. If the advantage is +0.50, expect to win nearly 50% of the time.

This means that Houdini 4 evaluations are calibrated so that +1.00 is equivalent to having 80% chance of winning the game.

.

So I already got the answer to my question for Fritz 12 and Houdini 4. I would like to know the same thing for other engines (for as much engines as possible), preferably strong engines like Stockfish, Komodo, Gull, Critter, Rybka, Hiarcs, Junior, Shredder, etc...

Answer 1

There is nobody in the world can answer you confidently. It requires deep source-code level understanding of the engines. However, some of the engines you mentioned are close-source, which means it's simply not possible to get an answer for all engines.

However, I can answer about Stockfish, which I have studied. Stockfish doesn't attempt to optimize evaluation score just to make it consistent to what a human thinks. Stockfish has a framework for parameter tuning (SPSA) but the overall purpose is to increase strength. The score reported from Stockfish is no way consistent and tuned to what a grandmaster expects. Stockfish tends to report a score larger than Houdini in absolute term. This means, Stockfish is less conservative and therefore a stronger engine than Houdini.

Think like this, in a position where you have two chocies, one a simple draw that exchanges all pieces and the other is a queen vs three minor pieces. Both scenarios are considered theoretically drawn. Houdini would have reported zero or close to zero for both scenarios. While this is theoretically correct, the scheme doesn't encourage the engine to try harder for the queen vs thee minor pieces scenario where the engine has better chances to outplay a weaker opponent.

On the other hand, Stockfish, by being less pessimism, will have no trouble finding the queen vs minor pieces scenario. This is one of the reasons Stockfish has been proven a stronger engine than Houdini in all rating list.

In short, the score reported from Stockfish and lots of other engines is meaningless. They just guide the engine to make the best decision but it doesn't mean much other than that. Houdini's evaluation has been tuned, most likely for marketing purpose - you sell an engine better if it meets the users' expectation. But meeting a chess user expectation doesn't mean it'll make the engine play better, as Stockfish has proven, it makes the strength worse.

Therefore, there is no such thing as scaling in chess engine scoring. Some engines do, like Houdini, but this is just a trick to make you buy it. Chess engines don't need a scaling scheme to play strong.