10

I hope this question hasn't been asked before.

I am looking for some examples where, say, the latest Stockfish or Komodo engine (or possibly some of the top NN engines) evaluates a position as plus or minus x for some time--possibly oscillating slightly near that point during that period--and then there is a big increase or drop in their evaluation.

What might be a suitable "for some time" period? I want to say more than a minute, but not more than 1 hour. Also, what is a "big" change in evaluation? I am not so sure about that, but let's say something like 3.0.

I am asking this question because, from my (very limited) experience, any dramatic changes in an engine's evaluation occur during the first 20 or so seconds. If an engine cannot find the deep tactics (or the deep positional play) during those first seconds then it is because the search tree has been pruned. (Am I right about this? I don't much about how chess engines operate.)

Edit: The drastic change in the evaluation should be a consequence of the introduction of a move not previously considered by the engine.

Improve this question
1
  • 1
    I remember a couple of analysis positions where that happened, let me see if I can find some. However on the note of pruning: pruning a line is not permanent, in fact if you let an engine calculate forever it will eventually search the entire tree without any pruning. (basically, the more you search, the less you prune early in the search tree; however you obviously gain new deeper parts of the tree that can be pruned)
    – koedem
    Nov 26, 2020 at 0:25

3 Answers 3

Reset to default
7

This happens often enough in engine tournaments that viewers have coined the terms "boom" and "moob" (for "eval explodes away from 0" and "eval implodes towards zero" respectively).

Here's the latest example of a boom from TCEC.

[FEN "8/kp6/6B1/p3p3/3r3P/q5P1/2Q2PK1/8 w - - 0 59"]

Leela, playing White, boomed from -0.46 the move before to -1.91 after playing 59. h5. That means this position (or one of the moves slightly prior) very likely meets your criteria, if you let Leela think for a sufficiently long time.

You can also see from the principal variations of both engines that Leela did not anticipate Komodo's move on move 58 - Leela predicted 58...Qd5+, but Komodo played 58...Qxa3.

If you're looking only for booms of more than +3, off the top of my head, this was another memorable boom.

[FEN "7Q/8/P3n1p1/2p3k1/6b1/5pb1/8/5K2 w - - 1 59"]

Leela, playing White, evaluated the position after 59.a7 as -0.06, then boomed to -14.28 after 59...Bf5 (Leela predicted 59...Nc7).

Edited to add: here's an example of a Stockfish moob from -1.38 to 0.00 after thinking for 4 minutes on TCEC hardware.

[FEN "r4k1r/p1pb2bp/2pp1nq1/4P3/3P3Q/2N5/PPP3PP/R4RK1 w - - 3 16"]

1.Rae1 Bf5

Stockfish had the Black pieces and its eval dropped after ...Bf5. Link to game with both engines' PVs if you want to see those.

Improve this answer
2
  • Yes, it's booms and moobs that I am interested in. (Thanks, by the way. I was not aware of these "terms".) How long did it take Leela to boom in the second one though? The latest Stockfish on my laptop finds mate in 22 in less than a minute. Do you have any examples where a boom (or moob) occurs after 2-3 minutes of evaluation?
    – the_fox
    Nov 27, 2020 at 16:36
  • @the_fox the second one used a (by now) very old Leela net, which had very noticeable weaknesses. That's why Stockfish finds the mate quickly and the Leela missed it completely. The time taken for an engine to boom / moob can be quite a while because that time taken is completely dependent on the hardware you're running the program on. Nonetheless, I'm adding another example, this time of a Stockfish moob which took several minutes on TCEC hardware (i.e. on your hardware, Stockfish will probably take longer to moob).
    – Allure
    Nov 27, 2020 at 18:21
6

This is a complicated position I had in a correspondence game. White to play and win. The engine evaluation rises and falls, occasionally hitting 0 again. Then it stays at roughly 1.5 for a long while until finally finding the breakthrough about 20 minutes in. Unsurprisingly, considering the full line is pretty insane.

[FEN "r5r1/1np3bk/1p1p4/3P4/bpP2PPp/P3N2P/1q1BQ2K/4R1R1 w - - 0 33"]

1. g5 Rge8 2. Qh5+ Kg8 3. Rg2 bxa3 4. Nf5 Rxe1 5. Bxe1 Bc2 6. Nh6+ Kh8 7. Ng4+ Kg8 8. Nf6+ Kf8 9. Nd7+ Ke7 10. Qg4 Nc5 11. g6 Rh8 12. Re2+ Kd8 13. Nb8 Ba4 14. Nc6+ Bxc6 15. dxc6 Qf6 16. Bf2 Re8 17. Ra2 Qe6 18. Bxh4+ Kc8 19. Re2 Kb8 20. Rxe6 Rxe6 21. Qg2 Re3 22. f5 Be5+ 23. Kg1 Rb3 24. Qa2 Bd4+ 25. Kf1 Rf3+ 26. Kg2 Rb3 27. f6 Rb2+ 28. Kf3 Rxa2 29. g7 Rg2 30. Kxg2 a2 31. g8=Q+ Ka7 32. f7 a1=Q 33. f8=Q Qg1+ 34. Kf3 Qd1+ 35. Kf4 Ne6+ 36. Qxe6 Qf1+ 37. Ke4 Qxf8 38. Qe7 Qxe7+ 39. Bxe7
Improve this answer
8
  • You may have to reset the move counter.. Start at 1. Idk if that's the only way but it seemed to work.
    – mowwwalker
    Nov 26, 2020 at 1:31
  • I should have been more careful in the phrasing of the problem. The move g5 doesn't quite fit the bill of what I am looking for because it is one of the original moves suggested. The evaluation does change by more than 3.0 if you let the engine think quite a bit, but what I am really looking for is a drastic change in the evaluation suggested by a move not previously considered.
    – the_fox
    Nov 26, 2020 at 1:42
  • 1
    @mowwwalker that worked indeed, thanks. A bit annoying but what can you do.
    – koedem
    Nov 26, 2020 at 2:54
  • @the_fox I see. There certainly are examples for that behavior too, however I don't know a fitting position off the top of my head. Let's see if I remember one later.
    – koedem
    Nov 26, 2020 at 2:55
  • 2
    @GS-ApologisetoMonica Good point, tried that now. Unfortunately while many positions on the way take a bit of time to switch the move, none of them take very long (so the accumulated time presumably simply contains the time of all switches along the way plus alternatives which also have to be calculated of course). A better way to find such a position is probably one of those short puzzles that Stockfish doesn't find and then wait until, well, it finds it. But the time it takes for something like that might vary a lot / be rather long.
    – koedem
    Nov 26, 2020 at 12:43
1

Adding another answer since it meets every requirement by the OP and I believe they'll be interested.

[FEN "4nr2/5ppk/pn3q2/3ppNRP/1pr1P3/1P3P2/1KP3Q1/R7 w - - 0 42"]

Game 99 of the Season 20 TCEC Superfinal

Leela had +0.03 eval with this principal variation:

41....d5 42. bxc4 Nxc4+ 43. Kb1 dxe4 44. fxe4 Na3+ 45. Ka2 Qe6+ 46. Kb2 Nc4+ 47. Kb1 Rg8 48. Rg6 Qd7 49. Ka2 Qa4+ 50. Kb1 Qd7

And it boomed to +10.59 after 42. Rg6 and 3 minutes 27s thought.

Of course, it is TCEC hardware - their Leela ran at 92.1 knps. It's extremely unlikely OP (or anyone) can get anywhere near this speed. With weaker hardware, Leela's eval will be different.

However it is worth emphasizing that OP's actual question (about the engine changing its mind during the search) is not so clear, because Leela could very well suffer from tunnel vision and completely miss 42. Rg6. That's not to say that Leela never changes its mind, it does, but it's not as simple as "run Leela for 3 minutes and it will boom". That's especially the case since what Leela sees in 3 minutes is heavily dependent on what hardware you run Leela on.

Here's an article by GM Matthew Sadler on a different blunder by Leela in the TCEC Season 20 superfinal illustrating the difficulties with the question, and possibly providing another interesting position. To quote:

Leave “my Leela” long enough (16.2M nodes searched) and its evaluation plummets: it sees that 79…Nb5 loses to the 80.Nf5 sacrifice and its assessment of its new top move (79…Bb5) is just 27.9% expected score (1 win, 3 draws and 6 losses). A big fall!

However, if we look at “TCEC Leela’s” evaluation at this point…

…it gives a WDL breakdown of 11.3% White win, 88.7% Draw and 0% Black win (0.16 is the conversion of this expected score to centipawns which the TCEC does to be able to compare easily with Stockfish). That’s just a slight disadvantage. Why is that?

Well, “TCEC Leela” moved 79…Bb5 very quickly, taking just 34 seconds for its move.

In that short time, it searched 2.3M nodes (10M nodes less than my 24 mins’ effort) and this was presumably not enough for it to see all the dangers in the position!

Position in question (I have taken the position one move earlier to have the board "right side up". White played 79.Bd3):

[FEN "5r1k/2nbq1rp/3p2p1/2pPp1P1/p3P3/P3NPQ1/1P2B2R/K6R w - - 3 79"]
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.