Fritz 12 with Rybka gave a friend of mine an evaluation of +3 for white in this endgame position, which turned out to be a draw. But I've heard that +3 from a computer means a guaranteed win with perfect play. Did I just hear wrong? How should computer evaluations be interpreted in general? What do opening advantages of <.5 even mean?
There are a couple of things here.
First, each program is going to have its own way of evaluating positions so the scores can't be directly compared. For example, I was running StockFish against Rybka recently and found Stockfish's scores were about double those of Rybka. I was surprised by this, but it's pretty clear that a score of 1 doesn't always mean "1 pawn." I think what we should look at is how the score changes. Another curiosity I saw yesterday (while answering one of your other questions, coincidentally) was that Stockfish's evaluation algorithm doesn't like odd numbers very much. In fact most scores were multiples of .04. Given that the magnitude of the value is arbitrary, I would not assume any certain value means "a sure win" unless the machine is claiming it found a mate.
Second, end game tablebases were created because solving end games requires a lot of search depth. Computers playing at tournament speeds just don't do it well. I was working through a different game a few days back and announced on this site that one side had an advantage. Ed used a tablebase to show there was no mystery left in the position - it was theoretically drawn. Of course, there's a huge difference between drawn-with-perfect-play and drawn; the players have to find the right moves.
The small value generally given to White in the early stages of the game basically means white can claim more valuable real estate. For example, on move 1, White can claim e4 and attack e5 and f5. Black can counter. But then White can play Nc3 and attack/bolster a4, b5, d5, and e4. But Black can counter. So it means very little.
Finally, to answer the question in your subject line - the evaluations are very trustworthy as they are based on hard facts and an impressive search depth. Of course, the machines aren't infallible. But we b-players must remember that Stockfish (or Rybka) play at GM strengths on modest hardware. On the best common hardware, they estimate their ratings at FIDE 3200. This is so high that only the best humans have a slight chance of not losing.
Consider what this means; I (USCF 1650-ish) have no chance against a person (say, USCF 2050) who has no chance against a person (say, USCF 2450) who has a no chance against a person (say, USCF 2850) who has a sliver of a chance against a top-flight commercial program (FIDE 3200).
Thus, when Stockfish says one move is better than another, I usually take it at face value. When I hook up the endgame tablebases this thing's gonna start announcing mate-in-30s, lol.
Different engines have different "scales" for their numerical evaluations. For instance, in a typical middlegame position with plenty of play left, when Houdini says +2.00 or better, it is highly probable that White has a winning advantage (though even here I've included qualifications for a reason). But consider: one could modify the source code of Houdini and double the absolute values of all numbers involved in evaluations; one gets an engine of identical strength that produces identical play, but now +4.00 means what +2.00 used to mean. This illustrates that one shouldn't expect a uniform numerical threshold across engines that typically indicates a winning advantage.
More than this, though, it's important to understand that a numerical engine evaluation of a position (as opposed to an outright declaration of inevitable mate) never strictly translates to "a won game," even for a single, fixed engine. A key point is that numerical evaluations have no clear-cut "meaning" in broad chess terms, and are rather just a substitute for sentient thought that is used to mechanically guide an engine toward generally desirable outcomes by influencing which move it selects at each point in the game; in this light, what is ultimately most important to an engine's play is just the difference in evaluation assigned to potential moves, rather than anything about the absolute values involved. The numbers are useful to the engine itself, which needs something that concrete in order to make a decision for one move over another, but we humans shouldn't be too quick to read more meaning into the magnitudes involved with thoughts like "+X means a win."
In particular, the further and further we get toward an endgame as opposed to a middlegame, the less and less reliably we can even use a rule of thumb (like my +2.00 for Houdini in middlegames above) about a certain threshold being enough for a win. One key reason for this is the difficulty that engines have recognizing fortresses, where an abundance of extra material still isn't enough to win. For instance, when I feed Stockfish this position,
after a couple minutes thought it is giving an evaluation of about +7.00, and in a typical position, when Stockfish says that, you almost certainly do have a win on your hands. Nevertheless, this is a dead draw, and a human can see this easily once the fact is realized that Black can just shuffle the rook between f6 and h6, and so (1) the h-pawn is useless, and (2) the white king will never be able to help the white queen attack. Eventually, Stockfish will recognize a draw here too, once it butts up against 50 moves, say, or finally runs out of different moves to try and finally can't avoid a repetition, but those events are way down the search depth line.
The endgame position from your earlier question that you linked to is akin to this sort of fortress, in that the extra connected passed pawns White has there are nice and all, but ultimately not quite enough to win in that position. Once an engine were to calculate for enough time to see as much information as is contained in tablebases, then its evaluation would come down to 0, but in the meantime, its evaluation algorithm has nothing better to go on than to give a + for that extra material (that it doesn't yet know is meaningless).
Well, computers generally evaluate based on position and material. As far as material goes
Each player starts the game with 1 King, 1 Queen, 2 Rooks, 2 Knights, 2 Bishops, and 8 pawns (sorry for being obvious here, but bear with me). Thus, each player starts with 1*0 + 1*9 + 2*5 + 2*3 + 2*3 + 8*1 = 39 points (as far as material goes).
Material is the baseline for the evaluation. At the first move of a game, both Black and White have 39 material points, and the difference is 0. Since it is White's move, there is a slight advantage which is usually reflected in the 0.1ish range.
Next, the evaluation will look at what is forced. Which is why, when there is an imminent mate, it will simply say m4 for the evaluation. It will gauge also how close to forcing a move the current position is in.
The reason that the evaluation comes back as 3 there, is that the computer believes that Black's pawn on the h file is going to be lost, leaving White up by 2 pawns, and more importantly, that White also can more than likely force an extra move on Black, which does not seem to change the end result which is that Black has an endless amount of checks and White cannot successfully remove Black's King from the pawns.
Chess software engines use heuristic evaluation functions in order to evaluate each position.
An heuristic is something that works in a certain domain (in our case chess position evaluation), but the reason why it works can not be proven scientifically (at least not completely).
The simplest heurist you can think of for chess is material.
Here is an example of heuristic function: h(input chess position) = number of white pieces - number of black pieces
Of course this is a very primitive way to evaluate a chess position, but it gives you already some indications.
In real chess engines programmers have embedded the usual common chess strategic knowledge (weak squares, bishop pair and so on) in the heuristic evaluation function, but since these concepts are not real 'rules' (though they have been tested by strong players in chess games) a chess engine evaluation is intrinsically 'wrong'.
I think this picture describes the situation quite well. It was created from 400k games, and considers only plain piece material.