How not to use a chess engine?

Question

How not to use a chess engine? What are the bad practices when using a chess engine, and the good ones? How trustworthy are their suggestions and evaluations?

Answer 1

What's the purpose of this question & answer?

I see a lot of misuse of engines in this community. I see topics where people do opening "analysis" by copy pasting engine outputs. Even worse, I saw opening "analysis" by copy pasting in first move!

Lots of beginner in this community believe that engines give best possible move in every position, because they beat human players.

I tried to explain why this is wrong under several topics shortly, but I see short explanations are not enough, people don't want to believe some stranger saying that using the engine in such ways is wrong without explaining with every detail. So I, as a computer science student and hobby player(used to play OTB but quit it) wrote this very long, tiring answer, in hope of changing wrong beliefs.

Notes: This is a very long answer. I have probably a lot of grammar mistake and typo. Also it may be possible to explain these in a more fluent or in a shorter way. So, if you think you can improve this answer, please suggest edits, I would highly appreciate it.

How does an engine work? What are these numbers in analysis window?

Engines evaluate all positions using different metrics(which is very closely related to engines playing strength)

For example: Engines has predefined values for materials, like 1 points for a pawn, and 3 points for a bishop etc.

But that's not all of course, they use more advanced metrics too. For example: A passed pawn 0.2 points, bishop pair 0.1 points etc.

So, it evaluates all positions like this, and inserts into a tree(if you want to learn about tree's more, please take a look at this paper).

But, as the memory is not infinite, if computer runs out of memory, engine stops right? So engine has to use memory carefully. How? It deletes some nodes (positions) from tree which are not promising (has bad (it may not be easy to decide which is bad) evaluation score).

Longest possible path in the tree is the depth of the analysis. As every node represents a position, depth of 2 equals 1 move.

What are the strengths and weaknesses of engines comparing to humans?

Engines are absolutely better at tactics comparing to humans. Because tactics are

• convertible to concrete (material, including checkmate) advantages in short-term (generally in less than 10 moves)
• computers have overwhelmingly more memory comparing to humans (Can any chess player memorize and evaluate 10^3 different continuations in any position in 1 minute? No. But computers today can do it for more than 10^12)
• computers don't make calculation mistakes(practically), but humans do.

Humans are absolutely better at strategy and positional evaluations. Because:

• after all, engines are using metrics to evaluate which are absolutely open to question, for example there is no correct metric for bishop pair, or pawn.

• a master chess player is far more better when assessing metrics(machines are stupid)

• humans are overwhelmingly stronger at pruning, our search tree is far more smaller(we can eliminate a lot of unnecessary positions, computers fall short in this)

Let me demonstrate these in this position, which a 1200 Elo player can assess a draw(rightfully) in 3 seconds.

 [Title "Komodo-9.3 depth=48, -17.61"]
 [FEN "rr2k3/6b1/8/p1p1p1p1/PpPpPpPp/1P1P1P1P/8/4K3 b - - 0 1"]

Engine says it's -17.61, after seeing 24 moves ahead. Why? Is it that stupid? Can't it understand black can't progress?

Yes! It is that stupid! And you are the one who needs to understand it's draw, not the engine, it's a tool for you!

-17.61 is result of the evaluation metrics. Is the evaluation wrong then? No, evaluation is correct, if you interpret it correctly. The engine sums up metrics, which results -17.61, because black has a lot of material advantage. Interesting thing is that much material advantage isn't enough in this position and that's the part you should assist the engine with your intelligence.

Evaluation scores are not position assessments!!!

There are only two things can be considered as an assessment in an engine's output:

• Mate in x
• Draw by 0.00

Other than these two all evaluation score outputs are opinion-based. Like in the position above -17 evaluation can be assessed as a draw, it's very unlikely but possible.

So, engines are really far from perfect, that's why we have tablebases. Tablebases are sources of the absolute truth of a given position. A tablebase gives an output which tells the position is either mate (in x moves) or draw. State of the art tablebases has covered all positions having at most seven pieces.

How to interpret evaluation scores?

As I mentioned earlier, engines do their best to use limited memory and computational power. Because these are limited, engines tell users that how deep they went.

Imagine that you are a commander (engine) of an army, and you are in an open terrain. Your army has 10k soldiers. You have 10km (depth of search) line of sight, and you see enemy forces 9km ahead, you guess (evaluate) their count as roughly 100 soldiers, and you decide to attack. You and your army starts to run towards enemy. But after getting 3kms closer to enemy forces (gaining 3km more line of sight), you suddenly see 100k more enemy, and you suddenly decide that this battle is lost.

This was an analogy of horizon effect problem. There are times, in which the key move happens out of the horizon of the engine, and engine evaluates the position falsely. This problem happens lot more in endgame, because in endgame, comparing to middlegame and opening, positions evolve slowly, more moves needed to get same amount of progress. Remember the long manoeuvres of endgames. That's why engines are using tablebases (precalculated, sure as death assessments of positions which have less than 8 piece on board), to fight with horizon effect.

Let's return to position above and clarify why we can so easily assess it as draw, but computer don't give 0.00?

Engine has calculated 24 moves deep, nice. But we are calculating all possible continuations, until the end! We are not faster than computers, but we are quicker than them. Because we understand, after checking several moves, all continuations are trivial, and prune them all. We only calculate few positions, and assess it as draw. But engine calculates tons of, because its not clever enough to understand its trivial to calculate all these positions.

Now, how not to use chess engines?

• As an endgame god, like it knows it all
• As an opening god, like it knows it all
• As a strategy god, like it knows it all
• As a tactical god, like it knows it all(it generally knows it all, but there are exceptions, because of the horizon effect)

But to use as:

• A foolproof mate searcher
• The best tactician in the world
• A questionable advisor in strategical/positional evaluations

Don't forget that analysis and match are different tasks. Humans can't beat engines on matches doesn't mean it's the same in analysis. Actually these are very distinct tasks, because analysis aims to find the best move with infinite resources, but match aims to win with finite resources, and for winning, it's enough to play better than opponent. All competitive engines are tuned for matches, to get highest possible success in engine tournaments.

Read also

• A nice article about chess engines, relevant to this topic.

• A very instructive article about interpreting engine evaluations.

• A related topic: Why do chess engines sometimes miss good moves (or take forever to spot)?

• Another related topic: Computer evaluations: How trustworthy are they?

---

Examples of Engine Fails, for Unsatisfied Readers

More examples of positions, which state-of-the-art engines can't find the correct move, which was found by human-beings

[Event "Linares "]
[White "Topalov, Veselin"]
[Black "Shirov, Alexei"]
[FEN "8/8/4kpp1/3p1b2/p6P/2B5/6P1/6K1 b - - 0 47"]

1...Bh3!!-+ {Shirov played this} ( 1...a3? {Komodo-9.3 with 50 ply, 5GB Hashtable, evaluates -2.58} )

Analysis of Shirov's game