The principal variation is always wrong!

Question

I read in an internet blog that when analyzing my games with an engine, it is often useless to copy the principal variation to my annotations, because (they say) it is well known that the principal variation is almost always wrong. By this, they mean that the principal variation often does not contain the best moves from both sides, as one would expect.

I am thinking about this and I am trying to understand what they really mean and why is it so. I can see two reasons:

1) If I analyze at 20 plies depth, then the fifth move in the principal variation was probably analyzed only at 10 plies depth, so it is less reliable than the first move. This probably does not apply very much to tactical positions, where, thank to quiescence search, positions are analyzed deeper when needed.

2) The engine is optimized to find the best move, but it doesn't care to find the best principal variation. This can be relevant when alpha-beta pruning is combined with aspiration window heuristic and incremental search.

Imagine that the engine first analyzes at 10 plies, and it finds that the best move is evaluated at +0.50 and the other moves are all less than +0.20. When increasing the search depth to 18 plies, the engine can analyze the best move with an aspiration window between +0.20 and +0.40, asking the search to find the exact value only if lies in the interval, or a cutoff value if it lies outside. If the search finds that the best move is valued more than +0.40, the engine only has to verify that the other moves are valued less than 0.40 and this can prove that the candidate best move is actually best without figuring out the actual value of the best move, nor the exact principal variation.

In the example, to prove that the best move is valued more than +0.40, at some point of the principal variation some second best move from the side to move can be sufficient to prove that the variation is better than +0.40, so the principal variation will contain that second best move, and the absolute best move for that position does not need to be determined.

See below for a more clear example of this.

My questions are:

1) Do you think I understand the reasons for the sentence "The principal variation is almost always wrong", or is there some other important reason that I am missing?

2) When analyzing my games with Stockfish or another popular engine, is there a way to turn off the optimizations I discussed above, and force the engine to compute the actual precise value of the best move, and a precise principal variation?

EDIT

Let me give a concrete example, otherwise what I wrote above is not so clear. This new example is super simplified, just to show the principle. Consider this case:

[FEN "1k3r2/pp6/3p4/8/8/n5B1/5PPP/5RK1 w - - 0 1"]

1.Bxd6+ Kc8 2.Bxa3

Here it is clear that the best move is 1.Bxd6+, since no other move wins a piece. Assume the engine suspects from previous analysis that 1.Bxd6+ is the only move that wins a piece. It analyses 1.Bxd6+ with an aspiration window in the interval from +1.00 to +2.00, asking for the exact value of the moves if it lies in the interval, or a cutoff it the moves wins at least two pawns. After every possible Black's reply (1...Ka8, 1...Kc8) it tries the move 2.Bxa3 first, evaluates it at +4.00, goes out of the window and returns this cutoff (without any need to try 2.Bxf8 at all). At the end, the engine happily concludes that 1.Bxd6+ is worth at least +4.00. Then the engine tries the other possible first moves by White, does not find anything worth so much, and correctly concludes that 1.Bxd6+ is the best move by White. But it returns an incorrect evaluation of +4.00 and an incorrect Principal Variation 1.Bxd6+ Kc8 2.Bxa3. The program never needed to find the exact evaluation to find the best move.

The thing I would like to write in my annotations is, instead, that this move has an evaluation of +6.00, with Principal Variation 1.Bxd6+ Kc8 2.Bxf8

So the fact that the engine is optimized only to find the best move, gives problems if I want to use it for analysis. And the mistake can be big.

I know that this example will not manage to confuse Stockfish, it was just an illustration, but I think this kind of problems can really happen, and probably this is what the blog post mentioned above wanted to say.

Answer 1

It is difficult to say anything about it without reading the blog post.

Regarding your thoughts:

Evaluations of +0.2, +0.26, +0.32 are for all practical purposes the same and basically mean that the position is equal. If you use another engine you will very likely find different numbers around zero.

Computer engines are not good for deciding on one opening variation over another, since in practical play there are more important factors than what the computer thinks is a centipawn advantage.

Above all, for practical play you want to pick a variation that you are comfortable playing. Also in many cases you want to avoid variations where you are forced to make 20 precise moves in a row (or be lost otherwise) while your opponent has a large margin for potential errors. Depending on the tournament situation sometimes you might also pick more drawish lines over risky double-edged lines.

So, all in all, using a computer engine for opening preparation in the way you suggest does not make much sense. It is better to use a database of human games, which will tell you how successful humans are in various variations, how risky (draws vs. decided games) it is, etc.

A computer engine is still useful for checking how to punish non-theory moves or for finding novelties (typically only done by top players).

Answer 2

No. I don't think so. The point of aspiration windows to introduce better alpha-beta cutoffs.

In your example, if your search gives +4.00 within your aspiration window that would be the best evaluation from your engine.

concludes that 1.Bxd6+ is worth at least +4.00.

No. +4.00 is the score, there is no such thing as "worth at least".

We only talk about "at least" when the score is outside the window. In this case, you'd have to do a new search with a wider window. Aspiration windows doesn't make your score less reliable, it makes the search quicker by introducing more cutoffs.

1. Principal variation is the moves that the engine considered the best. Of course, it's not the only way to play a position. To me, "The principal variation is almost always wrong" is not making any sense. There're many ways to play chess.

2. You shouldn't because aspiration windows is not causing you any problem. But if you insist, you could disable the aspiration margin in the source code, and then recompile from source.

Do you get the point? If you fail at your aspiration window, you'd have to do a new search. Otherwise, everything works. Please discard the "principal variation is always wrong" theory.