Why is Leela so good at beating Stockfish in the French/Slav?

Question

I notice Lc0 is especially good against Stockfish in French / Slav pawn structures. However, I don't understand why.

Here're a couple of games where Leela smoothly outplays Stockfish. Can anyone explain what Stockfish is doing wrong?

Link to this game displaying Lc0 & Stockfish PVs / evals for each move.

[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[Event "CCC11 Finals (30|5)"]
[Date "2019.12.19"]
[Round "1"]
[White "Lc0"]
[Black "Stockfish"]
[Result "1-0"]
[ECO "C04"]
[GameDuration "01:12:46"]
[GameEndTime "2019-12-19T07:33:45.057 PST"]
[GameStartTime "2019-12-19T06:20:58.414 PST"]
[Opening "French"]
[PlyCount "247"]
[TimeControl "1800+5"]
[Variation "Tarrasch, Guimard Main line"]

1. e4 e6 2. d4 d5 3. Nd2 Nc6 4. Ngf3 Nf6 5. e5 Nd7 6. Nb3 a5 7. a4 b6 8. h4 Bb7 9. c3 Qc8 10. h5 h6 11. Be3 Ba6 12. Nc1 Nd8 13. Rh3 Qb7 14. Rg3 b5 15. Nd3 Nc6 16. Be2 bxa4 17. Qxa4 Rb8 18. Bc1 Bb5 19. Qd1 Qb6 20. Kf1 Ne7 21. Nh4 c5 22. Nxc5 Nxc5 23. dxc5 Bxe2+ 24. Qxe2 Qxc5 25. Kg1 Qb5 26. Qd1 Nc6 27. Nf3 Qb6 28. Nd4 Bc5 29. Be3 Bxd4 30. cxd4 Kf8 31. Qg4 Rh7 32. Bxh6 Qxd4 33. Bxg7+ Ke8 34. h6 Rxb2 35. Qxd4 Nxd4 36. Rh3 Nb3 37. Ra4 Nd2 38. g4 Nc4 39. g5 Rb7 40. Rc3 Ke7 41.Kg2 Rb8 42. Bf6+ Kd7 43. Kg3 Rb1 44. Kh3 Rg1 45. Kh2 Rb1 46. Kg2 Rb5 47. Kg3 Nd2 48. Kg4 Nc4 49. Ra1 Kc6 50. Kh5 Kc5 51. g6 fxg6+ 52. Kxg6 Rbb7 53. Bg7 Rxh6+ 54. Bxh6 Kd4 55. Rg3 Rb8 56. Bg7 Ke4 57. Re1+ Kd4 58. Kf7 Kc5 59. Kxe6 a4 60. Kf5 Re8 61. e6 Nd6+ 62. Kg6 d4 63. e7 Rg8 64. Kh7 Ra8 65. Rg4 d3 66.Rd1 Re8 67.Rxd3 Nb7 68. f3 Kb5 69. f4 Rxe7 70. f5 Re1 71. f6 Rf1 72. Rdd4 a3 73. Rb4+ Kc6 74. Ra4 Nd6 75. Rxa3 Kb5 76. f7 Nxf7 77. Kg6 Nd8 78. Rb3+ Kc5 79. Rc3+ Kb5 80. Rc8 Nc6 81. Re8 Kc5 82. Rg5+ Kd6 83. Bf6 Rf2 84. Rf5 Ra2 85. Kf7 Ra7+ 86. Kf8 Rd7 87. Rh5 Rb7 88. Rh6 Rb5 89. Kg7 Rb1 90. Kg8 Kc5 91. Rh7 Rb3 92. Rh8 Kd6 93. Kf7 Rb7+ 94. Kg6 Rb3 95. Kf5 Kd7 96. Re4 Rf3+ 97. Kg5 Kd6 98. Rh6 Kd5 99. Re1 Kc4 100. Rh7 Rb3 101. Rc1+ Kd5 102. Rd7+ Ke6 103. Rd2 Rb5+ 104. Kg6 Ne5+ 105. Bxe5 Rxe5 106. Rg1 Ke7 107. Rgg2 Re6+ 108. Kg7 Re5 109. Rde2 Kd6 110. Rxe5 Kxe5 111. Kf7 Kd4 112. Ke6 Ke4 113. Rg4+ Kd3 114. Ke5 Ke3 115. Rd4 Kf3 116. Re4 Kg3 117. Rf4 Kh3 118. Kf5 Kg3 119. Kg5 Kh3 120. Rg4 Kh2 121. Kh4 Kh1 122. Kg3 Kg1 123. Rf4 Kh1 124. Rf1# 1-0

Another game where Stockfish just manages to hold the draw.

[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[Event "CCC11 Finals (30|5)"]
[Site "?"]
[Date "2019.12.18"]
[Round "1"]
[White "Lc0"]
[Black "Stockfish"]
[Result "1/2-1/2"]
[ECO "B12"]
[GameDuration "01:31:03"]
[GameEndTime "2019-12-18T23:02:39.927 PST"]
[GameStartTime "2019-12-18T21:31:35.992 PST"]
[Opening "Caro-Kann"]
[PlyCount "384"]
[TimeControl "1800+5"]
[Variation "Advance Variation"]

1. e4 c6 2. d4 d5 3. e5 Bf5 4. Ne2 e6 5. Ng3 Bg6 6. h4 h6 7. h5 Bh7 8. Bd3 Bxd3 9. Qxd3 c5 10. c3 Ne7 11. f4 Na6 12. Ne2 Nc6 13. a3 c4 14. Qc2 Qd7 15. Nd2 O-O-O 16. Nf3 Be7 17. g4 Rdf8 18. Be3 Nc7 19. f5 Kb8 20. Ng3 a6 21. O-O Bd8 22. Rf2 Re8 23. Raf1 Rhg8 24. Kg2 Ka8 25. Bc1 Kb8 26. Kh3 Ka8 27. f6 gxf6 28. exf6 Rh8 29. Ne5 Nxe5 30. dxe5 Nb5 31. Bf4 Bb6 32. Qd2 Bxf2 33. Rxf2 Na7 34. Bxh6 Nc6 35. Bf4 Qd8 36. Nf1 Nb8 37. Nh2 Nd7 38. Nf3 Nc5 39. Qe3 Ne4 40. Re2 Reg8 41. Qd4 Qc7 42. Be3 b5 43. Rg2 Rh7 44. Kh4 Rgh8 45. Rg1 Qd7 46. Ra1 Rb8 47. Ng5 Nxg5 48.Kxg5 Qc7 49. Kf4 a5 50. Rh1 Rbh8 51. Kg3 Kb7 52. Rb1 a4 53. Rh1 Ka6 54. Rh4 Kb7 55. Bg1 Kc8 56. Rh3 Rg8 57. Be3 Kd7 58. g5 Rgh8 59. Kg4 Rg8 60. Rh4 Kc8 61. Bf2 Kd7 62. Bg1 Kd8 63. Be3 Kc8 64. Rh2 Kb8 65. Rh1 Kc8 66. Rh4 Kd7 67. Bf2 Kc8 68. Rh1 Kb7 69. Bg1 Ka6 70. Rh2 Rhh8 71. Be3 Rc8 72. Rh4 Rcg8 73. Rh3 Kb7 74. Rf3 Kc8 75. Qc5 Qxc5 76. Bxc5 Kd7 77. g6 Ke8 78. Kg5 Rh7 79. Rf1 Rhh8 80. g7 Kd7 81. Kh4 Rh7 82. Kg5 Kc6 83. Bf8 Kc7 84. Rd1 Kd7 85. Bb4 Ke8 86. Be7 Rhh8 87. Bc5 Rh7 88. Ra1 Kd7 89. Be7 Ke8 90. Bb4 Rhh8 91. Bc5 Kd7 92. Bf8 Rh7 93. Bd6 Kd8 94. Bf8 Kd7 95. Rd1 Rhh8 96. h6 Rh7 97. Kh5 Kc6 98. Rg1 Kd7 99. Rg5 Ke8 100. Bd6 Kd7 101.Bb4 Kc6 102. Ba5 Kc5 103. Bc7 Kc6 104. Bd6 Kb7 105. Rg1 Kc6 106. Rd1 Kd7 107.Bc5 Kc6 108. Bf2 Rc8 109. Rg1 Rg8 110. Bd4 Kc7 111. b4 cxb3 112. Rg4 Kb7 113.Bf2 Kc7 114. Rd4 Rb8 115. Be3 Kd7 116. Bg1 Rd8 117. Rf4 Ra8 118. Rg4 Rc8 119. Rd4 Rb8 120. Bf2 Kc6 121. Rf4 Ra8 122. Bd4 Rb8 123. Rg4 Rg8 124. Rg5 Kc7 125. Rg4 Kc6 126. Rg3 Kb7 127. Rg2 Kc6 128. Rg4 Kc7 129. Rg5 Kc6 130. Rg4 Kb7 131. Rg3 Kc8 132. Rg6 Kd7 133. Rg3 Ke8 134. Rg4 Kd8 135. Rg5 Kc7 136. Rg3 Kd7 137.Rf3 Rc8 138. Rf2 Re8 139. Rf4 Rc8 140. Rf2 Kc6 141. Rf4 Kd7 142. c4 bxc4 143.Bc3 Rg8 144. Ba1 Kc6 145. Rd4 Rc8 146. Bc3 Rb8 147. Bb2 Kd7 148. Rg4 Rg8 149.Rd4 Kc6 150. Bc3 Rc8 151. Rg4 Rg8 152. Bd4 Kc7 153. Bb2 Kc6 154. Rd4 Kc5 155.Ba1 Kc6 156. Rd2 Kc5 157. Rd1 Rb8 158. Bd4+ Kc6 159. Rg1 Rg8 160. Rg5 Kc7 161.Rf5 Rc8 162. Rf1 Kc6 163. Rd1 Kb5 164. Bc3 Kc6 165. Ba1 Kd7 166. Rg1 Rg8 167.Rd1 Kc6 168. Bb2 Rc8 169. Rd2 Kd7 170. Bc3 Rg8 171. Ba1 Kc6 172. Rd1 Rc8 173.Rd2 Rg8 174. Rd1 Re8 175. Bc3 Rg8 176. Rd2 Ra8 177. Rd1 Rg8 178. Bb2 Kc5 179.Rd4 Kc6 180. Rd2 Rc8 181. Rg2 Rg8 182. Bd4 Kb5 183. Rg5 Kc6 184. Rg1 Kc7 185.Rg2 Kd7 186. Rf2 Ke8 187. Rd2 Kd7 188. Rg2 Kc7 189. Rg4 Kc6 190. Rg2 Kb5 191.Rg1 Ka5 192. Rg2 Rb8 1/2-1/2

The only common observation I've made is that it seems these games involve Stockfish not castling kingside. If the opening book forces it to do so, it doesn't get into scrapes like this as much. Example game where Stockfish drew a French with kingside castling without too much drama. It seems in the first game that a lot of action happened on the queenside while Stockfish declined to castle kingside (which makes sense, doing that seems foolhardy at best against a Rook on g3), but White's superior kingside space eventually led to a win in material. Game 2, the queenside was less of a battleground, but again White's superior kingside space led to a big advantage.

What plans should Stockfish have used in either game? Where did Stockfish go wrong?

I realize that, of course, Stockfish and Leela are way way better than the best humans, so I'm not looking for concrete lines here - just general plans that Stockfish should have tried. In particular (and especially for correspondence chess players), at which point should one look for improvements in Black's play?

Answer 1

Well, it is a small sample, but assuming that there are a lot more games like these, I think it could be the following things.

First, I am not sure when we first humans first decided that space was an advantage, but for as long as I have been playing, it has been a known factor. Both of these openings cede space compared to double-king-pawn openings and double-queen-pawn openings. It is notable that in both games above, white played e5 taking the space that was offered.

I think what we are seeing at such a high level is confirmation that space is maybe a larger factor than we previously believed, possibly even decisive. If one program can dismantle such a strong opponent with its one major advantage being space, it may just be THE single biggest reason. I have always loved openings that give me a space advantage, and have had very good results, but of course, we are human, and cannot execute like Leela can, so we don't win every time proving the value beyond all doubts like the machines soon may.

I also think that having that space may lead directly to more activity, and the cramped king-sides in both these games were expertly exploited. I think that in both games, black did not castle simply because it was already tactical suicide, and not castling, in and of itself, was not the reason for the loss...the damage was already done. I am not sure that all variations of these openings would see that as decisively as these games since not all allow e5.

Hopefully, that is a start to answering your question.

Answer 2

Because of the enormous skill difference between these computers and humans, any kind of analysis will inevitably be post-hoc. We can tell ourselves stories about how "Stockfish should have [insert plan]," (and I'm sure some people here will) but ultimately I think that any story we could come up with would be flawed at the level of Leela/Stockfish. This isn't to say our ideas are worthless, just that they are very likely to not tell the whole story.

I recognize that this is a bit of a "Who are we to judge god?" response to the problem of evil, but I stand by it. In the world of chess, gods are real.

---

That being said, I do think I can provide a (speculative) answer to the question stated in the title: why is leela so good at beating stockfish in the french caro kann? I realize this isn't exactly the answer you were going for, but I think it's interesting

Classical engine (eg stockfish) work by (smart) brute force. They leverage their computational power to compute many, many positions and beat you with calculation. In the 80s, early 90s, if you were a decent player, you could abuse this fact by steering the game into positions where calculation is less helpful. This usually meant closed positions with an unclear path to victory--without an obvious way to improve the position within the next x moves, computers don't know what to do, and make silly decisions.

But that was awhile ago. Computers are now much faster, and our algorithms much stronger. We've optimized the hell out of our engines, and it shows in the performance. However, I suspect classical engines still suffer from the same problems as before: in positions with no clear path forward, calculation just isn't as useful.

In particular, look at the position after move 9 in the first game. What do you do as black? Stockfish can certainly play this position very, very well (due to years of optimization), but I doubt to the same standard as it does open ones. Leela, being (partially) a neural net, doesn't rely as heavily on calculation, and (if we're anthropomorphizeing) has a better grasp of long-term "thinking."

So this would be my guess: leela does better in french/caro kann positions because they have unclear lines of progress. This plays into leela's strengths and Stockfish's weaknesses.

---

I would be curious to see if there are any examples of stockfish losing to leela in open, tactical games. The previous section would benefit from attempted falsification.

Answer 3

Generally speaking, Leela tends to have a better "intuition" and Stockfish is very good at brute force calculations. So in a structure like the French/Caro-Kann, where calculation becomes less important and strategy becomes more important, Leela will tend to do better.