10

I was reading about the Lc0 engine and its evolution. Then took a glimpse at the games between Lc0 and the notorious Stockfish. I am not a professional or anything remotely close, but this particular game seems very odd to me. According to chess.com:

The Lc0 engine got off to a fast start in the bonus games, beating the reigning Computer Chess Champion Stockfish despite Lc0 starting down a pawn in the odds-chess position. In typical Leela style, the engine underpromoted to a rook to drive the win home.

[FEN "CCCC 1: Rapid Rumble (15|5) Bonus Games"]
[Date "2018.10.03"]
[White "Lc0 17.11089"]
[Black "Stockfish 220818"]
[Time "09:33:10"]
[TimeControl "900+5"]

1. f4 Nh6 2. f5 Nxf5 3. Nf3 Nh6 4. Ng1 Ng8 5. e4 e6 6. Nc3 d5 7. d4 Bb4 8. e5
Qh4+ 9. g3 Qd8 10. a3 Bf8 11. Be3 Ne7 12. Nf3 Nf5 13. Bg5 Be7 14. Bxe7 Nxe7 15.
Bd3 h6 16. b4 b6 17. O-O O-O 18. Ne2 c6 19. a4 c5 20. c3 Nbc6 21. Qd2 Bd7 22.
Rae1 Rc8 23. b5 Na5 24. Nf4 cxd4 25. cxd4 Qc7 26. Rc1 Qd8 27. Rxc8 Qxc8 28. Nh5
Nc4 29. Qf4 Ng6 30. Nf6+ Kh8 31. Qc1 Na5 32. Qd2 Nc4 33. Qc1 Na5 34. Bxg6 Qxc1
35. Rxc1 gxf6 36. Bh5 Nc4 37. exf6 Kg8 38. Ne5 Nxe5 39. dxe5 Rc8 40. Rxc8+ Bxc8
41. Be2 d4 42. Bf3 h5 43. Bxh5 Bb7 44. Kf2 Be4 45. Ke1 d3 46. Kd2 Kf8 47. Ke3
Bh7 48. Bd1 Ke8 49. g4 Kd8 50. Kd2 Ke8 51. Ke3 Kf8 52. Kf3 Ke8 53. Kf2 Kd7 54.
Ke3 Kc7 55. h4 Kc8 56. h5 Kd7 57. Kf3 Ke8 58. Kf2 Kf8 59. Ke3 Kg8 60. Kf4 Kh8
61. Kf3 Bg8 62. Ke4 Bh7+ 63. Kf4 Kg8 64. Ke3 Kf8 65. Kf2 Be4 66. Ke3 Bh7 67. Bb3
Kg8 68. Bc4 d2 69. Kxd2 Be4 70. Be2 Bb7 71. g5 Be4 72. Bd1 Kh8 73. Kc3 Kg8 74.
Be2 Bf5 75. Kd2 Bb1 76. Bd1 Kf8 77. Kc3 Ke8 78. Kb2 Bh7 79. Kc3 Be4 80. Kd2 Bf5
81. Ke3 Bb1 82. Kf3 Kf8 83. Ke3 Ke8 84. Kd4 Kf8 85. Kc3 Bh7 86. Be2 Kg8 87. Bg4
Be4 88. Bd1 Kh8 89. Kd2 Bh7 90. Bg4 Kg8 91. Bd1 Kf8 92. Bb3 Ke8 93. Bc4 Kd7 94.
Be2 Ke8 95. Kc3 Kf8 96. Bd1 Bb1 97. Bg4 Be4 98. Kd2 Kg8 99. g6 fxg6 100. Bxe6+
Kf8 101. h6 g5 102. Ke3 Bg6 103. Bd5 g4 104. Kf4 g3 105. Kxg3 Ke8 106. e6 Kd8
107. Kf4 a6 108. bxa6 Be8 109. a7 Kc7 110. a8=Q Kd6 111. h7 Kc5 112. h8=R Bd7
113. Ke5 Kb4 114. Rh3 Bxa4 115. Kd4 Bd1 116. Bc6 Bb3 117. f7 b5 118. f8=Q# (118.
f8=B#) 1-0

Ok, I can understand that Lc0 gives a pawn in advance —maybe to show off its superiority, but the rest of the game is full of unbelievably stupid moves. And ironically, those amateur moves are coming from both sides. For example, even for someone like me, 9. g3 Qd8 is hard to believe.

I also can understand that in rapid games, stupid moves are likely to happen. But these are chess engines who can find almost perfect moves in a fraction of second (well, according to my personal experience with Stockfish). So time limit cannot be such a heavy barrier. Then why are they playing dumb like this? Is it intentional or something?

3
  • 8
    Typically in computer chess tournaments, the engines are given fixed openings designed to reduce draw percent, and add variety. May 3, 2020 at 16:27
  • As I was scrolling quickly through the game I judged the queen move (8. ..Qh4+) to be about worsening white's pawn structure. One of the answers explains it is a credible attack that fails due to a trap that is hidden a few moves. This got me wondering why you thought this move was hard to believe.
    – hkBst
    Apr 30 at 17:10
  • I couldn't find the quote just now but I recall Carlsen saying something like "computers play like they don't understand anything, then they win." May 4 at 12:41

3 Answers 3

9

This is a real game, not a joke, and Stockfish got outplayed. You only quote one example so we'll focus on that, but first, note that the first four moves - 1. f4 Nh6 2. f5 Nxf5 3. Nf3 Nh6 4. Ng1 Nh8 - was part of the "opening book". In other words, the engines were forced to play those moves. Leela isn't giving away a pawn for free to show her superiority, she was given a pawn-down handicap position. Left to her own devices, Leela will never play 1. f4.

Secondly, what exactly happened on 9. g3 Qd8? Well, this quote is revealing:

But these are chess engines who can find almost perfect moves in a fraction of second (well, according to my personal experience with Stockfish). So time limit cannot be such a heavy barrier.

Engines can and do make mistakes! Like humans, their first impressions can be wrong, and they might change their mind as they think longer. Check out this question. If you look at the games in that question, engines can be just as blind as humans. Sure, the refutations are likely too deep for a human to spot (although it's possible if you know there's "something" in the position, see example), but another engine can.

In this case Stockfish was seduced by the apparent forced win of material after 8...Qh4+. It sees that 8...Qh4+ 9. g3 Qe4+ wins the h1-rook, so it thinks that White's best response is 9. Ke2, which is obviously bad news for White. In fact even today with a much more powerful Stockfish, Lichess Stockfish at depth 28 is still giving 8...Qh4+ as the best move, and predicts 9. Ke2.

But then if you execute 8...Qh4+, Stockfish suddenly realizes that 9. g3 is entirely playable. That's because 9. g3 Qe4+ 10. Kf2 Qxh1 11. h3 traps the Queen. It's not obvious how, but it does - 11. h3 Bxc3 12. bxc3 Qe4 12. Nf3 (threatening Bd3) Qg6 (intending to meet Bd3 with ...f5) 13. Nh4 and the Queen is trapped.

How can an engine as strong as Stockfish miss this Queen trap? It's got to do with pruning. Some moves an engine simply doesn't consider very seriously when it first looks at a position. Technically, the evaluation function says that White's position has worsened dramatically,* so White will never be crazy enough to play 9. g3, so the engine should prune 9. g3 ("not look further" - see alpha-beta pruning). Stockfish will still look at the move eventually, but only after searching the main moves to greater depth. Note this is not a bad thing - for moves like 9...Qxh2?? there's no hidden tactical motif that justifies looking further, so pruning 9...Qxh2 is correct.

And this is what happened. I left Lichess running as I was writing this answer, and at depth 29 Stockfish's principal move changed to 8...Ne7 with a corresponding fall in eval; 8...Qh4+ is no longer one of its three principal moves either.

If you are interested in this you might also want to check out Anatomy of a Blunder by GM Matthew Sadler.

*Keep in mind that the evaluation function produces an evaluation of the position without searching. Without searching, the position after 10...Qxh1 looks awesome for Black, since Black is up a rook, the Queen is not trapped, is generating threats, and White is unable to castle. It's only upon searching a few ply further that the engine discovers that White wins the Queen.

0
9

Basically, the first four moves were implemented to start a game without the f2 pawn, i.e., with an imposed handicap. The game starts at move 5 with a French Defense set-up for Black. I'm surprised that Lc0 can win against Stockfish with a pawn down. I would probably have chosen to play e5 than e6, with the idea to install a Bishop in c5 and try to take advantage of the open g1-a7 diagonal.

I was also intrigued by some of the positions and their seemingly dumb moves, but some of them are not that dumb!

You mentioned a first puzzling moment after 9.g3

Lc0 has a pawn down and Stockfish plays to win with Qd8! By taking in h1 Stockfish could be even worst. Then, we may question the Qd8-h4-d8 sequence that has the advantage of weakening the kingside but loses two tempi. Reacting by c5 instead of Qh4 was probably more natural.

[FEN "rnb1k1nr/ppp2ppp/4p3/3pP3/1b1P3q/2N3P1/PPP4P/R1BQKBNR b KQkq - 0 1"]
[title "Position after 9. g3"]


9... Qd8! {was played and Lc0 has a pawn down and Stockfish plays to win with Qd8!} (9... Qe4? {is the obvious move but} 10.Kf2 Qxh1 (10...Bxc3 11.bxc3 Qxh1 12.h3!! is a transposition) 11.h3!! 11...Bxc3 12.bxc3 Qe4 13.Nf3! {White can maybe play for a win here because I don't see how the Queen escapes!} (13.Bd3 Qh1 14.Bf1=))

The second one is after the massive exchanges 41.Be2

White has winning chances in this endgame and Stockfish tries to find some counterplay by playing 41... d4. Assume Black waits and plays Bb7-Ba8 without playing d4. Then, the winning plan for White is to bring the King to h5 to collect the h6 pawn.

[FEN "2b3k1/p4p2/1p2pP1p/1P1pP2B/P7/6P1/7P/6K1 w - - 0 1"]
[title "Position after 40... Bxc8"]

1.Be2! d4 (1...Bb7 2. Kf2 Ba8 3.Kf3 Bb7 4.Kg4 Ba8 5.Kh5 Kh7 6.Bd3 Kg8 7.Kxh6+-)

Update. There are still some positions where like @polfosol I can't understand Lc0 moves! For instance, after 78... Bh7, the move 79.Kc3 seems indeed quite "dumb"! After 79.Bc2! the win is very easy (at least for a human ;-).

[FEN "4k3/p4p1b/1p2pP2/1P2P1PP/P7/8/1K6/3B4 b - - 0 1"]
[title "Position after 78... Bh7"]
0
1

Games between engines often have absurd finishes, particularly when one side is material ahead. It has not been programmed to look for the shortest win, and so plays more or less any move that is still winning. I think sometimes it will give away some of its excess material to reach a position that is in its tablebase. Games between two computers are often quite ridiculous after one side obtains a winning advantage and your example is typical.

7
  • Engines do play for the shortest mate - at least the traditional ones did at the time the OP was asked. Leela is unique, she often "trolls" its opponents since it's programmed to look for the move with highest win probability, and when every move wins then the "win probability" is virtually the same for every move and the choice is semi-random. There've been lots of effort to make Leela not troll so much - see Moves Left Head, lczero.org/blog/2020/05/lc0-v0.25-has-been-released
    – Allure
    Apr 22 at 4:17
  • I imagine that different programmers take different attitudes toward efficiently finishing off a totally won endgame No, chess engines always do the same thing, which is make the move that maximizes the evaluation function. The problem is that for NN engines, in totally winning positions (or totally losing positions) the evaluation function for NN engines return virtually identical results almost regardless of the suggested move. Leela is a NN engine, which is why you see moves like 79. Kc3 and 112. h8=R. If Stockfish had the white pieces it would go for the shortest mate.
    – Allure
    Apr 24 at 23:23
  • Sorry, I dont see your point. The computer always maximises the evaluation function for sure but the programmer chooses the evaluation function to prioritize what has most value. Are we somehow talking past each other?
    – Philip Roe
    Apr 25 at 3:36
  • Are you familiar with neural networks (NNs)?
    – Allure
    Apr 25 at 3:41
  • I think I know the general idea but my understanding is that after they have trained themselves their workings are inscutable
    – Philip Roe
    Apr 25 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.