13

I’m a big fan of chess cheat detection apps, like Irwin that was created by lichess.org.

But I heard that there is a new bot called Leela that is teaching herself how to play chess, and to think in a human-like way. Is that true, and are there any ways to detect it?

  • 3
    I hope so because cheaters need to be banned. – Qudit May 7 at 9:43
  • 4
    I think this should stay open. It's very relevant to online chess. – Qudit Jun 13 at 23:40
11
+50

Leela works by performing an exceptionally sophisticated positional evaluation at a relatively shallow search depth, whereas most chess engines work by performing a simple evaluation at as deep a search as possible. In theory this should produce a more positional style of play, and it does seem to be an effective strategy versus today's best conventional chess engines.

So in general, the moves selected by Leela will be different from those selected by Stockfish (Irwin's reference engine) in key positions. But the differences in move evaluation may be small from Stockfish's point of view; it's not that Stockfish thinks Leela's moves are bad per se, just that it doesn't reliably choose them for itself, because it doesn't see their unique advantages relative to a move that it has deeply researched tactically.

However, there are other characteristics of engine-users' play that differ from that of strong human players, and are observable and measured by Irwin.

Move timing statistics are an important one, and that probably won't change with Leela. Good human players recognise key moves, where many possibilities of nearly equal apparent strength exist, and plan ahead during them, taking more time. Then they play quickly when their opponent responds in the way they expected, or in easy positions where there is only one obvious good (or even legal) move. An engine user doesn't do this - for every move, they must enter the opponent's move into the engine, and wait a few seconds to see what to do.

Another characteristic which Irwin checks for is large variations in playing style, either between games or during a game. A player who plays some games extremely well with very few mistakes, but then plays other games (at the same time control) rather badly with several blunders, is probably a weak player who uses an engine for his good games. This is especially true if he seems to play his strong games against players with high ratings. Or a player might find himself losing a game, then get out the engine, in which case he will show blunders and a declining position, followed by extremely accurate play which salvages the game.

In short: yes, I believe Irwin can detect engine users, even if the engine in question happens to be Leela.

  • 2
    +1. The part about play style is the important one. Irwin can also detect if someone has a GM help them for a day (i.e. a series of games, a single one won't be conclusive enough) - their play will be completely different, and that's what matters. – Annatar Jun 14 at 5:47
  • 1
    I will still be able to argue thank I went drunk to the games where I played poorly – David Jun 14 at 7:03
1

Unfortunately, if the cheater is smart enough, he's never going to be caught!

No matter how strong the cheat detector may be, if you only use the engine to help you out in one critical position, there is not enough data to get you caught.

Finally, the only thing that can be proven is similarity between your play and the engine's play, which does not constitute proof of cheating

  • I think cheat detector can catch him from only one move if this move rate was better than her other ordinary moves + comparing it with engine moves and the time that takes to done ? – shar Jun 17 at 7:02
  • 3
    No - a poor player could make an exceptionally "good" move purely by chance. Examining only one move would be insufficient evidence. Irwin looks at a wide body of evidence, gathered over the course of at least several games. – Chromatix Jun 17 at 7:51
  • @Chromatix Great point, and that's exactly why a good player who is also a smart cheater will always get away with it – David Jun 17 at 7:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.