5

What is current use status of Leela for practical player? Is it already working well as UCI? Could anybody give some insight on strength with various thinking times, especially in analysis mode? As leela uses different thinking process, I must ask also for multiPV search. Is the speed down of many PVs still as devastating as with other engines?

I have desktop with 6x3.2Ghz CPU. As graphics card like RTX2080ti is 'just' around 1300€ with good use outside of chess too, I wonder, what performance change would it mean.

With mentioned hardware, how long would approximately leela have to think to compare to Stockfish?

Stockfish 0.1s Stockfish 1s Stockfish 10s Stockfish 1min Stockfish 10min Stockfish 1hour

1
  • 5
    Leela is currently leading Stockfish in TCEC super final by 5 points, so it would seem that it is competitive with traditional engines (to say the least).
    – Qudit
    Commented May 18, 2019 at 5:34

2 Answers 2

6

Lc0 is fully compatible with UCI. It's strength depends a ton on what hardware it's being run on. On any hardware (tested down to Raspberry Pi), it will be superhuman (over 2900 elo). With a RTX 2060 ($200ish) it should be roughly even with SF on 8 cores (although you may need to mess with settings a bit to get it there). A 2080ti puts decently (20 or so elo) ahead of 24 cores of SF. The current MultiPV mode is only a display tweak. It isn't weaker, but it doesn't give as good analysis of the non-primary lines. If you have more questions, the best place to ask is probably the Lc0 discord.

2
  • Thank you for benchmark information. By GTX 2060 you mean RTX 2060 using FP16 backend, right?
    – hoacin
    Commented Jun 6, 2019 at 9:42
  • Correct. Updated to be more clear Commented Jun 6, 2019 at 15:16
6

Can't comment on hardware but will say something about Leela's strengths and weaknesses -

If you look through the games from the TCEC superfinal, sometimes Leela is brilliant. For example in this position from game 61:

[fen "r1bbnr2/pp1n1q1k/3p4/2pPp1pP/2P1PpP1/2NQ1N2/PP2BB2/2KR2R1 w - - 10 26"]

Stockfish (White) evaluated this as +0.63, while Leela was at -2.01. From the human point of view, I suspect most will prefer Black. That's because White has nothing on the kingside while Black is more capable of playing on the queenside. The game proved Leela right - it took a lot more moves for Stockfish to realize it was inferior, and even more moves after that for Stockfish to resign (it is a fantastic defender), but the fact that Leela saw it was superior so much earlier than Stockfish is very impressive. There are many similar games from the superfinal - Leela seems very capable of outplaying Stockfish positionally, often converting a Stockfish advantage into one of its own, even if the game eventually ends in a draw.

Another advantage of Leela is multi-PV. MCTS engines get this for free, while Stockfish (which runs on AB) needs to invest computational power. If this matters for you then that's an additional advantage for Leela.

However the infamous "Leela blunder" is still a thing. Take this position from game 81:

[fen "4rr2/2pb2bk/1p1p4/2nP3P/p1P1p3/P1N1Bq2/1P3N2/1K2Q1RR w - - 2 26"]

Leela just played 25...Rae8 and showed a +0.67 eval. It thinks its position is inferior, but not by that much. Its principal variation went 25. .. Rae8 26. h6 Be5 27. Rg7+ Kh8 28. Ka2 Rf7 29. Nfd1 Nd3 30. Qh4. Stockfish had the same moves, but saw that 28. Qg1! wins, and its eval shot up to +7.68 one move later. Leela remained blissfully unaware until Stockfish played Qg1, and resigned shortly afterwards. Stockfish virtually never makes this kind of one-move blunder.

Another problem with Leela is that it doesn't seem to defend very well. When Stockfish loses, it defends grimly and it takes a long time for Leela to grind a win. On the other hand when Leela loses, it often implodes. In the ongoing TCEC Champions bonus, its loss to Houdini 3 (!!) is illustrative. Houdini 3 is an engine many years old, rated hundreds of elo below Stockfish (and ostensibly Leela). But Leela hated the starting position so much it managed to lose. Comparatively you would need to play hundreds if not thousands of games before Stockfish loses to Houdini 3.

Finally a noticeable weakness of Leela's is that it isn't very good at beating inferior opponents. For example in the ongoing TCEC Champions bonus, which includes engines from many years ago, Leela is performing worse than 1-year old versions of Stockfish. In this, Leela is kind of similar to Petrosian - it doesn't win many games, but it's good in match play.

In the end, I think Leela is inconsistent. In some positions it's much stronger than Stockfish, and in others it's just worse. As a practical player, I'd say use both. Their relative strengths and weaknesses should be pretty apparent. As a heuristic, trust Stockfish if the position is highly tactical, or if it's an endgame. Otherwise trust Leela (but run Leela's principal variation through Stockfish to eliminate the "Leela blunder").

EDIT: Will probably expand on this later, but GM Matthew Sadler wrote a piece on the relative differences between Stockfish and Leela based on the 15th TCEC superfinal which you might find interesting (especially games 43/44) In a nutshell, Stockfish is comfortable going to a slightly inferior endgame and defending endlessly, while Leela hates that playstyle and prefers to seek active play.

5
  • Your comments about Multi-PV are somewhat misleading. The reason multi-pv doesn't weaken leela is that it's version of it isn't very good at what multi-pv is supposed to do (ie provide better analysis of weaker lines). Commented Jun 6, 2019 at 15:19
  • @OscarSmith you sure? If the node that Leela visits most is the line it plays, then the node that Leela visits second-most is the second-best line, and so on.
    – Allure
    Commented Jun 6, 2019 at 21:08
  • Yes, the problem is that since the 2nd best line for leela often has considerably fewer visits (10% of main line or less), everything past the 2nd best line is fairly under-analyzed. Commented Jun 6, 2019 at 21:17
  • 1
    Ideally there would be 3 modes for multi-PV, off, cosmetic, and analysis. Leela supports 1 and 2, most other engines support 1 and 3 Commented Jun 6, 2019 at 21:18
  • I decided not to expand too much on the edit, since GM Sadler's article & the Wikipedia section are, I think, pretty good.
    – Allure
    Commented Jul 5, 2019 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.