Chess Stack Exchange is a question and answer site for serious players and enthusiasts of chess. It only takes a minute to sign up.
Sign up to join this community
Fritz 12 with Rybka gave a friend of mine an evaluation of +3 for white in this endgame position,
[Title "White to move"]
[FEN "8/8/8/3k3p/3P3K/2P3R1/8/7r w - - 0 1"]
which turned out to be a draw. But I've heard that +3 from a computer means a guaranteed win with perfect play. Did I just hear wrong? How should computer evaluations be interpreted in general? What do opening advantages of <.5 even mean?
There are a couple of things here.
First, each program is going to have its own way of evaluating positions so the scores can't be directly compared. For example, I was running StockFish against Rybka recently and found Stockfish's scores were about double those of Rybka. I was surprised by this, but it's pretty clear that a score of 1 doesn't always mean "1 pawn." I think what we should look at is how the score changes. Another curiosity I saw yesterday (while answering one of your other questions, coincidentally) was that Stockfish's evaluation algorithm doesn't like odd numbers very much. In fact most scores were multiples of .04. Given that the magnitude of the value is arbitrary, I would not assume any certain value means "a sure win" unless the machine is claiming it found a mate.
Second, end game tablebases were created because solving end games requires a lot of search depth. Computers playing at tournament speeds just don't do it well. I was working through a different game a few days back and announced on this site that one side had an advantage. Ed used a tablebase to show there was no mystery left in the position - it was theoretically drawn. Of course, there's a huge difference between drawn-with-perfect-play and drawn; the players have to find the right moves.
The small value generally given to White in the early stages of the game basically means white can claim more valuable real estate. For example, on move 1, White can claim e4 and attack e5 and f5. Black can counter. But then White can play Nc3 and attack/bolster a4, b5, d5, and e4. But Black can counter. So it means very little.
Finally, to answer the question in your subject line - the evaluations are very trustworthy as they are based on hard facts and an impressive search depth. Of course, the machines aren't infallible. But we b-players must remember that Stockfish (or Rybka) play at GM strengths on modest hardware. On the best common hardware, they estimate their ratings at FIDE 3200. This is so high that only the best humans have a slight chance of not losing.
Consider what this means; I (USCF 1650-ish) have no chance against a person (say, USCF 2050) who has no chance against a person (say, USCF 2450) who has a no chance against a person (say, USCF 2850) who has a sliver of a chance against a top-flight commercial program (FIDE 3200).
Thus, when Stockfish says one move is better than another, I usually take it at face value. When I hook up the endgame tablebases this thing's gonna start announcing mate-in-30s, lol.
Different engines have different "scales" for their numerical evaluations. For instance, in a typical middlegame position with plenty of play left, when Houdini says +2.00 or better, it is highly probable that White has a winning advantage (though even here I've included qualifications for a reason). But consider: one could modify the source code of Houdini and double the absolute values of all numbers involved in evaluations; one gets an engine of identical strength that produces identical play, but now +4.00 means what +2.00 used to mean. This illustrates that one shouldn't expect a uniform numerical threshold across engines that typically indicates a winning advantage.
More than this, though, it's important to understand that a numerical engine evaluation of a position (as opposed to an outright declaration of inevitable mate) never strictly translates to "a won game," even for a single, fixed engine. A key point is that numerical evaluations have no clear-cut "meaning" in broad chess terms, and are rather just a substitute for sentient thought that is used to mechanically guide an engine toward generally desirable outcomes by influencing which move it selects at each point in the game; in this light, what is ultimately most important to an engine's play is just the difference in evaluation assigned to potential moves, rather than anything about the absolute values involved. The numbers are useful to the engine itself, which needs something that concrete in order to make a decision for one move over another, but we humans shouldn't be too quick to read more meaning into the magnitudes involved with thoughts like "+X means a win."
In particular, the further and further we get toward an endgame as opposed to a middlegame, the less we can use a rule of thumb (like my +2.00 for Houdini in middlegames above) about a certain threshold being enough for a win. One key reason for this is the difficulty that engines have recognizing fortresses, where an abundance of extra material still isn't enough to win. For instance, when I feed Stockfish this position,
[FEN "5k2/6p1/5r2/3Q3P/4K3/8/8/8 w - - 0 1"]
after a couple minutes thought it is giving an evaluation of about +7.00, and in a typical position, when Stockfish says that, you almost certainly do have a win on your hands. Nevertheless, this is a dead draw, and a human can see this easily once the fact is realized that Black can just shuffle the rook between f6 and h6, and so (1) the h-pawn is useless, and (2) the white king will never be able to help the white queen attack. Eventually, Stockfish will recognize a draw here too, once it butts up against 50 moves, say, or finally runs out of different moves to try and finally can't avoid a repetition, but those events are way down the search depth line.
The endgame position from your earlier question that you linked to is akin to this sort of fortress, in that the extra connected passed pawns White has there are nice and all, but ultimately not quite enough to win in that position. If an engine were to calculate for enough time to see as much information as is contained in tablebases, then its evaluation would come down to 0, but in the meantime, its evaluation algorithm has nothing better to go on than to give a + for that extra material (that it doesn't yet know is meaningless).
I think this picture describes the situation quite well. It was created from 400k games, and considers only plain piece material.
