How is Alpha Zero "more human"?

Question

I have a possibly naive question about AlphaZero. I have seen it described as playing in a "more human" style than other computers, but whatever it does, it gains about 100 ELO points by doing it. Kasparov, and many others, have claimed that a strong human in collaboration with a computer will beat a strong computer (perhaps by about 100 ELO??). So an obvious question is, how would AlphaZero compare with a "centaur" combination?

Having only looked at a few games, what I notice is that most computers play wide open games that maximize their own mobility, but AlphaZero seems very concerned to limit the opponents mobility. In a human player I would describe this as a matter of style, not more or less human.

Answer 1

Page 5 in the paper has your answer:

... AlphaZero compensates for the lower number of evaluations by using its deep neural network to costs much more selectively on the most promising variations - arguably a more "human-like" approach to chess ...

"selectively" is the key word. What does that mean? Let's use this following position for our example:

https://chess24.com/en/read/news/london-classic-5-caruana-shows-how-it-s-done

This is a recent game won by Caruana in 2017 London Chess Classic. The White bishop is under attacked, and you know you have to move it. But where?

Possibilities (not losing a piece):

• Bh4
• Be3
• Bd2
• Bc1

What was Caruana thinking?

I felt like I would lose at some point, but when I saw, 25.Bc1 I suddenly started to get a bit more optimistic about my chances. I realised my position is bad, but at least I had a plan and that was really all I needed for some confidence in this position. When I saw this b3, c4 the position is double edged and I have some chances.

This is human thinking, and a "human move". Caruana hadn't considered Bh4, Be3 and Bd2 because they "looked" bad. He had been focusing only and only on the Bc1 move.

Humans play chess very selectively, we discard unreasonable moves because we don't have time to examine all possibilities equally.

• We discard Bh4 because it release the tension on the h6 pawn
• We discard Be3 because it blocks the two white rooks on the third rank
• We discard Bd2 because it blocks the White queen to the king side

That's what AlphaZero trying to claim in the paper. They claim their algorithm, although slower than Stockfish, is able to selectively pick better moves than Stockfish in the search. While Stockfish is faster, it wastes time on bad moves. AlphaZero is slower, but it's more precise (like what Caruana was doing).

For example, AlphaZero might spent 80% resources on Bc1, and 20% on all other bishop moves. Stockfish might give 25% for each move (Bh4, Be3, Bd2, Bc1).

Answer 2

Most strong engines emphasize looking very deeply, at the expense of having a superficial evaluation function. In the AlphaZero paper, they say that Stockfish looks at 70 million positions per second.

Human grandmasters look at very few positions indeed compared to engines, but they have a better feeling who is better in a given position.

AlphaZero looked at only 80,000 positions per second, so it spends much more time in its evaluation function.

That's the sense in which they meant "more human like", nothing more.

Answer 3

AlphaZero already seems to play like a regular "centaur" -> correspodence GM with an engine assistance.

As an FM I'd get much more enjoyment of playing AlphaZero vs a regular engine.

One comparison would be it plays like Karpov would with perfect tactics. (Game 9 AlphaZero plays a piece down for 15moves which is very Tal like).

It is not just style, AlphaZero gives an impression of understanding positions better than Stockfish.

AlphaZero also does not suffer from Horizon Effect that ALL chess engines had suffered from until now. Time and again it is able to correctly evaluate a position more moves down than Stockfish does.

Here's an example:

[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[White "AlphaZero"]
[Black "Stockfish"]
[Date "2017.12.05"]
[Result "1-0"]
[Event "Alphazero vs Stockfish: AlphaZero - Stockfish"]
[Site "https://lichess.org/study/EOddRjJ8"]
[UTCDate "2017.12.06"]
[UTCTime "05:53:49"]
[Variant "Standard"]
[ECO "C11"]
[Opening "French Defense: Steinitz Variation #2"]
[Annotator "https://lichess.org/@/Spreek"]

1. d4 e6 2. e4 d5 3. Nc3 Nf6 4. e5 Nfd7 5. f4 c5 6. Nf3 cxd4 7. Nb5 Bb4+ 8. Bd2 Bc5 9. b4 Be7 10. Nbxd4 Nc6 11. c3 a5 12. b5 Nxd4 13. cxd4 Nb6 14. a4 Nc4 15. Bd3 Nxd2 16. Kxd2 Bd7 17. Ke3 b6 18. g4 h5 19. Qg1 hxg4 20. Qxg4 Bf8 21. h4 Qe7 22. Rhc1 g6 23. Rc2 Kd8 24. Rac1 Qe8 25. Rc7 Rc8 26. Rxc8+ Bxc8 27. Rc6 Bb7 28. Rc2 Kd7 29. Ng5 Be7 30. Bxg6 Bxg5 31. Qxg5 fxg6 32. f5 Rg8 33. Qh6 Qf7 34. f6 Kd8 35. Kd2 Kd7 36. Rc1 Kd8 37. Qe3 Qf8 38. Qc3 Qb4 39. Qxb4 axb4 40. Rg1 b3 41. Kc3 Bc8 42. Kxb3 Bd7 43. Kb4 Be8 44. Ra1 Kc7 45. a5 Bd7 46. axb6+ Kxb6 47. Ra6+ Kb7 48. Kc5 Rd8 49. Ra2 Rc8+ 50. Kd6 Be8 51. Ke7 g5 52. hxg5 { 1-0 Black resigns. } 1-0

AlphaZero plays the king to center 16. Kxd2! in a middle game correctly judging that Black will not be able to take an advantage of it.

It is able to correctly evaluate a piece sacrifice 30. Bxg6! while regular engines are unable to see that they are lost for a number of moves.

32. f5 is quite nice too.

There are other examples such as exchange Sacrifice in Game 3.

Answer 4

It is as easy to jump on a bandwagon saying Alpha-Zero's play is 'more' human than previous computer chess programs as it is to jump on the opposite wagon and say Alpha-Zero's play is entirely 'alien'. It's not clear that Alpha-zero's play is 'more human' especially given our human tendency towards anthropomorphism.

Chess as a Struggle of the (human) Mind

But in chess is this tendency true? Magnus Carlsen once spoke about how 'traditional' computers in general lack human creativity saying:

"Chess is all about the struggle between human minds. That’s what makes it exciting. Computer chess is mechanical, dry and bland. The moves are very strong, of course, but there is no style. If you try to play against a chess computer, not only will you lose with a very high certainty, but you will also be bored in the process.

Magnus Carlsen didn't see evidence of human styles of play in traditional chess computers. So lets examine if Alpha-Zero's recent accomplishment has undone this perspective and moved us towards something more reminiscent of ourselves.

If by 'human-like' you mean play 'exhibiting behavior more likely to appeal to our sense of anthropomorphism' does Alpha-zero's style seem more human? How do we really test this subjective myopic humans like to project upon non-human things? Lets ask - does the algorithm 'selectively pick better' or exhibit 'more human creative choice' in its style of play?

The algorithm's creators indicate that unlike Stockfish which uses an Alpha-Beta search algorithm, Alpha-Zero employs a Monte-Carlo tree search (MCTS) algorithm which accepts as input a weighted parameters θ built up from previous outcomes ~ Page 3. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm).

So the algorithm doesn't exhibit choice at all. It actually engages in a random but probabilistic Monty-carlo search where the possible search paths available to it are increasingly prejudiced by previous outcomes. Did Alpha-zero choose to optimize its style of play this way or was that the choice of its programmers?

Does Alpha-zero always have all possible moves available to it for consideration or are some moves prejudiced algorithmically in a way that mimics experience which can be interpreted by humans anthropomorphically?

Initially it had all moves available to it so its 'style' was entirely random. However as it's search is increasingly and optimally constrained by previous success or failure its style is actually changing towards the mode its programmers have shackled it with. Is this 'more human' though? Compare this to Magnus Carlesen who will sometimes choose less optimal moves because they are more creative:

Magnus Carlsen: “I appreciate creating something unique”

Chess as a Struggle of the (alien) Mind

Humans can chose the criteria that drives their own style of play (for example I often chose impulse and error in my own style). Many see Alpha-zero's play in both chess and go as decidedly Alien. Nick Hynes, a grad student at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) observes:

“What we’re seeing here is a model free from human bias and presuppositions: It can learn whatever it determines is optimal, which may indeed be more nuanced that our own conceptions of the same. It’s like an alien civilization inventing its own mathematics which allows it to do things like time travel...”

Likewise GM Peter Heine Nielsen told Chess.com:

"After reading the paper but especially seeing the games I thought, well, I always wondered how it would be if a superior species landed on earth and showed us how they play chess. I feel now I know."

It seems that most react to Alpha-zero's emergent style of play as 'alien play', and not as 'more human'.

Therefore there's reason to disagree with the answers above that say 'yes'.

Answer 5

This is an incredibly interesting time to be alive.

Chess computers starting from the 1970s have been minimax-tree based search algorithms using alpha-beta pruning. These programs got stronger and stronger both because of advances in computer speed and parallelism and because of improvements in the heuristic eval function used to prune branches and select leaf nodes. But people have long noticed how materialistic and boring computer play is, and many people (myself included) thought it was impossible to encode "human" intuition into software.

But have you seen these games?

AlphaZero is exhibiting incredibly beautiful play, including several examples of material sacrifice for long-term positional advantage. This is reminiscent of some of the most beautiful games from human masters, but with unrivaled technical accuracy as well. This is the first example I've seen in my life of something that's computer-generated and also has deep beauty.

---

The Centaur Claim:

I've heard Garry say this many times, but it's just not true. Or at least, it won't be true any longer with AlphaZero on the scene.

Imagine this: there is a piece sac that has 10,000 relevant continuations, where 5,000 of them are purely tactical (yet mostly unrelated to one another) and another 5,000 that are mostly positional (yet mostly unrelated). How could a human sift through all of these variations without making a mistake? If AlphaZero can now look at these highly creative moves, what contribution could a human possibly make?

---

The Last Frontier:

There is one place left where brute-calculation will still beat deep neural nets: endgames. There is no amount of intuition that will beat a tablebase. But the endings that require a tablebase (because a search tree can't go deep enough to just calculate the right move) are pretty rare. And you could just plug a tablebase into AlphaZero, but that would destroy the purity of a "self-taught" engine, right?

Answer 6

Since humans lack ability to search deep, like traditional computer chess programs (fritz, stockfish et al), they create 'strategic principles' or thumb rules (center control, development, king safety) and concepts or tricks that are applicable in vast variety of situations in different ways, such as sacrifice, rooks connected, bishop pair, specific endings e.g how to corner the king with a rook and a pawn.

I think that alpha zero has independently reinvented many such concepts (percepts and concepts) and has also learnt tons of new ones - because its knowledge was not required to be built upon human evaluation functions and the strong minmax search which always assumes the opponent is a genius.

Of course, such principles themselves conflict in some situations, that is why various opening plays and pitfalls are carefully studied - eg don't develop queen too soon.

On the other hand, humans also notice that once you lose one piece (without exchange) you weaken your forces so they are extremely careful not to lose a piece without a compensation.

I think that Alphazero's play has liberated computer chess (and human chess) from the slavish fear of losing small material and overreliance on opening books and piece values.

Alphazero games show things like the 'strategic principles' like center control, development, space, initiative are far more important if your opponent is sloppy. In other words, 'sacrifice' is not really sacrifice but trading off a piece for gain in initiative, position, directed move.

Alphago (not the zero) relied on human evaluation, but alphazero sets up the entire chain of evaluation to 'search or simulation' as a single end to end process and comes up with totally new way of playing.

If you think about it, great masters of the past like Morphy, Fischer, Kasparov have been applauded for typically this kind of -counter-intuitive- play where they are not bounded by written-on-stone evaluation by take advantage of special situations that emerge. I think alpha zero's games have such 'wow' factor to it.

Why neural networks. While computer programs that use symbolic representation and discrete search can only use 'one' way of thinking, neural networks can parallelly process situations with alternate, conflicting evaluations and flip to the more valuable view in later layers.

Answer 7

More human in the sense that the moves it plays seem to coincide more or less with an human approach : play for long-term advantage, positional sacrifices, piece activity. There is an apparent convergence with human chess knowledge and accepted strategic principles refined over the centuries (e.g. it "discovered" many same openings). This is remarkable given the fact that AlphaZero has not been seeded with human-constructed chess knowledge.

But the similarities end here. AlphaZero takes it to the next level and does it better, and in ways humans have never conceived. AlphaZero possesses "superhuman" capabilities to quote the paper : "AlphaZero achieved a superhuman level of play [...]" (https://arxiv.org/pdf/1712.01815.pdf). In addition it doesn't have the weaknesses inherents to humans beings: concentration issues, fear, tiredness, feelings, intuition, etc. that limit humans. And its silicon brain allows for tactical combinations beyond human capabilities when necessary.

Answer 8

I want to say thank you to all who have responded to this question, often with subtlety and insight. The chief difference in the responses, it seems to me, is in the interpretation of the word human.

AlphaZero does not play human chess in the sense of oversights and miscalculations, but its "thought" process seem to correspond, in a heightened form, to how I think that most strong players think. You draw up, fairly quickly, a list of "candidate moves" that you would like to play, and for the strongest players this list is amazingly accurate, even playing something like a recognizably sensible game in one minute. The rest of the time is spent on asking, which of the moves on that list really work? Petrosian said that he felt most on form when the move he eventually played was the one he first thought of. We all know how satisfying it is when the move we most wanted to play turns out to be tactically playable. I can relate to the AlphaZero algorithm much more easily than I can to AlphaBeta search, which does not at all resemble what goes on in my head.

What seems most interesting is how the machine was able, by self-play, to recognize the promising candidates. That is where the potential lies for real revolution. I wonder whether this is only possible for domains like chess and go, where the objectives can be clearly defined. But I find it striking that AlphaZero seems to display purposeful play, but Stockfish has no idea what is going on.

Answer 9

The way I understand neural networks, A0's real advantage is its superior evaluation of board positions. This evaluation incorporates both short term tactical knowledge (which in a sense serves as a multiplier of the number of positions examined) and a superior evaluation of strategic value.

Answer 10

One thing I feel the whole discussion has missed is that A0 can play chess, shogi and go, all very well and all from self-training. This is much more human. Furthermore, in go it has revealed deeply new ideas to the top players (as I understand it). Other engines are very task-specific, A0 seems otherwise. I'd like to see it play chess960.

Answer 11

I don't think there is anything 'human' about Alpha. It just used much stronger hardware and played higher quality chess. The good opening moves it finds(for example, to fianchetto king side with Bg2) are fully due to its simulated opening book. Concepts that impressed me and that I have formulated in 'The Secret of Chess': http://davidsmerdon.com/?p=1970 , which Alpha uses for the first time among top engines, are advanced longer chains, for example the d4-e5-f6 chain that trumped a whole piece in the Bg6 sacrifice game, and central backward-makers, as seen in the French Defence games between both engines. Both concepts involve searching to great depths, and probably here Alpha was helped by its tremendous hardware. Otherwise, I see nothing human about its play. Many of the games were, admittedly, beautiful.