21

There seems to be an established "chess canon", whereby general principles exist for long term strategy in the game. There is a well-characterized set of openings, a small subset of which are even played, and even in middle to endgame one can identify named structures, attacking/defending pieces, point values, etc. High level players and chess engines will occasionally break "the rules" - but on a move-to-move basis, not consistently throughout an entire game. In others words, it seems like even though chess has room for a lot of creativity, that creativity lies within a well-established theory of what one might consider "basic technique".

I am thinking of a project involving training an AI to play chess, but with a loss function that discourages common moves for that position - i.e. a chess engine that performs incredibly well but as unlike a human as possible. But before I launch into training, I'm wondering if such a thing is even possible. Chess theory is great for teaching humans how to play, but is it possible that it is the only way to play chess at the level that top humans do? Are there any obstacles to the possibility of a completely different playstyle that would consistently break every rule in the book but still beat top players? Or does modern positional analysis rigorously rule out other strategies?

I'm a total amateur at chess, just a machine learning engineer with an interest in the game - so I apologize if this question is trivial.

  • 7
    There already are radically different but equally effective playstyles for chess. In many positions, you have the choice to play like an attacking maniac like Mikhail Tal, or a calm and collected positional player like Anatoly Karpov. Certain players prefer certain choices. However, in some positions, there is no choice - the position might dictate exactly how you must play. – NoseKnowsAll May 5 at 23:28
  • 6
    Alternatively, you can already say that there are radically different but equally effective playstyles for chess openings. Specifically, compare and contrast how Hypermodern openings ( en.wikipedia.org/wiki/Hypermodernism_(chess) ) attack the center from the wings, while the Classical school of chess dictates that you should control the center. – NoseKnowsAll May 5 at 23:30
  • 6
    The playstyle of modern engines actually already is distinct enough from "what humans would do" that top players are able to identify if someone is just letting an engine play against them in an online game with very high precision. – Annatar May 6 at 7:03
  • Good players don't base their game on "principles", but on accurate calculation and evaluation – David May 6 at 7:52
  • 11
    The fact that AlphaZero and Leela were trained from scratch with absolutely no input from humans on strategy, and ended up playing opening that were absolutely staples at current human top level seems to suggest that what we've invented is probably close to optimal. – RemcoGerlich May 6 at 12:32
20

It's definitely possible there are strategies we don't know about. However, training your engine to specifically play contrary to all we know isn't the way to get there. For example, what would you do if one of your pieces were threatened? Our current knowledge says to not give it up for no reason, so would your AI therefore decide to give it up (because it's technically mathematically possible there could be some hidden idea 30-40 moves down the road?). You'd do better with random moves.

A better way to unlock hidden strategies is through a more bottom up approach. For example, AlphaZero learned on its own (without human knowledge), just using the rules of the game to guide it. From this, we saw a new style of playing that was basically unknown. But trying to get this new style of playing wasn't the primary goal.

Basically, it's fine to disregard human knowledge, but it's not fine to actively train an AI to play opposite to this knowledge. Here you're just trading one well established knowledge base for a vastly poorer one.

| improve this answer | |
  • 4
    I think OP does not want the AI to base it's moves only on playing contrarily to what humans do. As far as I understand, they want the AI to make nearly optimal moves, but with preference towards the moves that are unhuman (in situations when many movese are approximately equally good, as far as its possible to determine). – Jakub Konieczny May 6 at 13:12
  • What kind of play style did AlphaZero develop? – BruceWayne May 6 at 15:49
  • 7
    @BruceWayne: Anna Rudolf has analyzed many AlphaZero games on her YT channel. Mostly it gives up pieces in favor of gaining territory and to barricade in opponent's pieces into a corner where they're useless. – Nyos May 6 at 16:33
  • 2
    agadmator has also gone over a ton of AlphaZero's games: youtube.com/user/AGADMATOR/search?query=alphazero – BlueRaja - Danny Pflughoeft May 7 at 5:56
6

In principle this is pretty easy. Get a database of 10 million human games, re-score the positions with a chess program (Lc0 or Stockfish), then train a Neural net to prioritize a combination of score and not picking the human selected move. Once you had the data, this could be done in a few weeks of GPU time, and should produce a computer that makes relatively weird moves, and could probably be super-human in strength.

| improve this answer | |
5

I am thinking of a project involving training an AI to play chess, but with a loss function that discourages common moves for that position - i.e. a chess engine that performs incredibly well but as unlike a human as possible.

Of course it would be possible to program an AI (or even an ordinary computer with no intelligence) to play the most unpopular moves for each position. However it would also be the single worst performing engine in the world. Complete beginners would love playing against it because they would win quite a few games.

If you stop and think about it for a few seconds you will realize that most of the time most of the people try and play the moves which give the best results. The more these moves become known the more they are played. The really bad moves that lead to quick losses tend to be played the least.

This conclusion requires knowledge of human nature and little or no chess knowledge.

| improve this answer | |
  • "people try and play the moves which give the best results" - Yes, but people aren't perfect. People can see ~20 moves ahead, maximum, and usually less. So what if there are strategies that only play out in the long game - i.e. 30+ moves later? Could such things exist, and could one build a whole playstyle out of them? – Nico A May 5 at 23:33
  • 3
    @NicoA Humans don't calculate 20 moves ahead (i.e. precise moves), but they can see much further (i.e. long-term implications of positional features, and especially fortress recognition). In fact, that is the last remaining advantage humans still have over conventional engines, which are strictly limited to their search depth (but are so much better during that window that this easily outweighs their relative weakness beyond it in a human vs. engine match). – Annatar May 6 at 6:57
  • "However it would also be the single worst performing engine in the world." This is incorrect. It is entirely possible and common to train a machine learning algorithm to take multiple criteria into account. Hypothetically, let's say that giving it a preference for the most uncommon moves made it play 10% worse. If it plays 1000% better than the best humans, this doesn't matter. – Jehan May 7 at 6:45
4

Yes

But only actual experimentation will tell us if AlphaZero has left a meaningful amount of the chess space unexplored. Chess, like all games, boils down to two fundamental components:

  1. Explore the space of possible moves
  2. Evaluate the "goodness" of each such move

AlphaZero uses a technique called Monte Carlo Tree Search to perform 1, and Deep Convolutional Neural Network to perform 2. And Bob's your uncle! Ok, that is a gross oversimplification, but let me explain briefly how each piece works.

Search

Games have a set of states and a set of moves which transition between the states. Since they usually have a single start state, and for simplicity, we usually model this set of states as a tree (rather than a graph). "Looking ahead" just means traversing layers down the "game tree". For a game like chess, which a high branching factor, you end up with a tremendous number of states in only a few plies, so much work goes into avoiding as many state nodes as possible (by trying to identify obvious losers and bailing on that path or following strong paths preferentially). MCTS goes for depth over breadth by "playing out" a particular move very deeply into the game. Since there are a limited number of moves for which one can perform this computationally expensive operation, it does so for only a random subset of possible moves. However, the subset is not chosen with a uniform distribution. The moves which are explored can be weighted by any heuristic you like. More about that later.

Evaluation

If you're looking at a winning state, it is easy to give an evaluation. If you're not looking at a winning state, you have to decide if it's "good" or "bad". This is where the neural network comes in. Right after you make a move, it's hard to tell whether the move had a positive or negative effect on your winning potential. But it's much easier to determine after the game is over. So when the game is complete, you now have a win/loss signal for all the board states in that game. This is now trivial training input for deciding whether each of those board states was "good" or "bad" for you, and you can backpropagate that signal through the neural network which gives you the "good/bad" signal for each board state. This is the "deep learning" portion of AlphaZero.

Because this is the stateful portion of AlphaZero, you could also say this is where the "knowledge" is, albeit in a rather abstract form which isn't really accessible to inspection or query.

Bias

If we assume that AlphaZero starts out by making uniformly random choices during MCTS, then it is fair to say that it explores the chess space without any particular bias. The DCNN may inform the MCTS layer, causing it to follow "good" paths rather than "bad" ones, but from what I've read, it prefers to follow "unknown" paths rather than "known". Basically, AlphaZero uses every game to increase its knowledge about chess rather than just trying to win the current game. Because it focuses on covering the space rather than just greedily following the most promising path, it learns much more broadly than chess engines with a non-stateful search algorithm, which must use their search cycles as economically as possible.

For this reason, one should suspect that AlphaZero is not leaving a lot of promising paths out of its repertoire. And, as others have noted, it's style is already quite different from most grandmasters, and it continues to teach us new things. Even so, there are free parameters in the AlphaZero design, which you may tune to extract different outcomes.

Strategy

If you were to create a "most alien AlphaZero", you could take the basic architecture, and then train it on human games. However, this training would build a secondary network called the Human Predictor. The goal of the HP network would simply be to predict the moves most likely to be played by a human for a particular board state.

Now, when you execute the MCTS phase, instead of preferring "moves I haven't explored before", you prefer "moves which humans wouldn't make", leaving all the rest the same as AlphaZero. This should basically implement what you are looking for. My guess is that the play will end up looking quite similar to AlphaZero in areas where AZ already plays in an alien style, and will be inferior to AZ in games where AZ overlaps more with "human style".

The reason should be obvious: AZ has no restrictions on the kinds of moves it makes. When it plays differently from humans, it isn't because it's trying to. It's because it found, through exhaustive trial and error, that those moves are simply better. And not just better against humans, either: better even against itself! It's possible that one could train AZ to beat humans even more efficiently by using the HP module described above to model the counter-moves that the opponent is likely to make during the MCTS phase. Such a modified AZ may find that "pure" AZ is overly conservative because it gives its human opponents too much credit.

It's also possible to train a "Stockfish killer" variant by simply using Stockfish itself to provide the counter-moves during the MCTS phase. You would need to do this strictly during the learning phase, because you wouldn't have the computes to do this during a live tourney, but effectively, the DCNN would learn to play exclusively against Stockfish, rather than AZ, and skip any counter-play that a different engine might provide. Then, during a tourney, the MCTS could revert to the standard algorithm, since the DCNN has already captured the essence of Stockfish and stuffed it in a bottle.

Anyway, sounds like a fun experiment. Good luck!

| improve this answer | |
3

The question is, equally effective against whom? You are correct that there tend to be "optimal" or "near-optimal" moves when engines play against engines. And when grandmasters play in long time control games, their moves often resemble those of engines. But when playing against lower-rated humans, or even against GM's in fast time control, there is a lot more flexibility in play. It just isn't possible for a human brain to calculate every possible variation, and so you can "get away with" not always playing the technically best move.

So in general, you don't have to operate under the assumption that your opponent is a 3500-rated chess engine. Against a human, you can play in a variety of styles that assume they might miss something, causing you to gain an advantage. This can take many different forms:

  • Offbeat openings: Avoiding their opening prep and playing something outside their comfort zone.

  • Playing a "trap": Allowing the opponent an opportunity to grab material that ultimately leads to a loss or a much worse position.

  • Time pressure: Refusing trades and keeping the position very complex to force them to calculate and burn their clock.

  • Limiting their options: Playing attacking moves that require very precise calculation to defend against. If they slip up, the game is over.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.