2

Even with all the compute in the world, chess engines cannot compute very deeply. So, they have to make use of pruning heuristics and discard moves from the analysis.

Seems possible that even the best heuristics will mistakenly discard good moves, and conversely go down bad paths that initially look good from a heuristic standpoint.

Have researchers tried to identify when this tends to happen, and if this can be taken advantage of, similar to adversarial examples in Deep Learning? E.g. an expert with deep understanding of a chess engine internals could concoct a series of moves to force the chess engine to make bad moves.

I play chess.com occasionally, and I've noticed something like this happening, where the chess engine will evaluate the same move as exceedingly good, and then as exceedingly bad.

Improve this question
6
  • 3
    There are certain engines out there which are supposedly designed to play against Stockfish in particular. Though I haven't heard much about them in the past couple years. It's probably harder now that Stockfish uses a neural net for its evaluation, as opposed to hand-crafted heuristics.
    – Inertial Ignorance
    Nov 2, 2022 at 21:37
  • I did notice the use of neural networks, which also led to the question about adversarial examples. The imagery deep learning networks are easily fooled by minor image tweaks. I was wondering if the chess engine neural networks are similarly brittle.
    – yters
    Nov 2, 2022 at 21:42
  • 2
    @yters In general, neural nets that are trained though self-play reinforcement are much more robust against adversarial inputs than supervised models constrained by the size and quality of the annotated datasets on which their learning was based. That said, potential weaknesses can still remain due to the relation between feature encoding and the horizon effect on learning rate. For example, early Alphago-inspired go engines would tend to misjudge ladders, whereas recent stronger engines avoid the problem by encoding the presence/absence of...
    – Will
    Nov 3, 2022 at 11:33
  • 1
    ...ladders as additional input features (providing the model with new "vocabulary" so to speak). So if I were to look at finding such "exploits", I'd look at (lack of) feature encoding first.
    – Will
    Nov 3, 2022 at 11:35
  • How would this look like in practice? Would you say an agent that memorizes the games Stockfish loses and then repeats them ad infinitum (only works since chess engines are semi-deterministic) is an adversarial agent?
    – Allure
    Nov 9, 2022 at 2:14

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.