8

I am very interested in how AlphaGo resp. AlphaZero works. It seems to me, the related Google Papers are very dense and not easy to read.

Is there any textbook or lecture that explains on a technical level how it all works? I.e. starting with neural networks/deep learning, Monte Carlo Tree Search until reaching at AlphaGo/AlphaZero?

I have a master in computer science, but I never worked with artificial intelligence/machine learning.

Right now it looks to me that all the related information are scattered around, and I don't have time to listen to (online) lectures about machine learning that contain maybe only 10% on what is relevant to understand AlphaGo...

Can someone point me in the right direction?

Improve this question
2

3 Answers 3

Reset to default
10

EDIT

@unutbu's link in the comment is a good introductory read.

Solid understanding for AlphaZero most likely require a quantitative degree (PhD?). Are you asking for a crash course in AlphaZero?

Please note unless you invest significant amount of your time, nothing I say will work. There is no book that can possibly cover everything. You will need to work very hard.

Let's give a try. Crash course for AlphaZero.

1. Basic understanding in machine learning

Countless number of books. But if you don't have time, I'd recommend Professor Andrew Ng's machine learning courses on Coursera.

Your keywords: training sets, test sets, stochastic gradient descent, learning rate, GPU, cost function, cross entropy.

2. Deep neutral networks

You need to understand about neural networks. I'd recommend Professor Ian Goodfellow's deep learning book if you're serious. If you don't have time, please again follow Professor Andrew Ng 's online course on Coursera. You don't have to go though all chapters (but you should!).

YouTube has many quick introduction to neural networks, try them.

Your keywords: neuron, layers, weights, bias, mini-batch, activation.

3. Monte Carlo Tree Search

You should understand what Monte Carlo is. Books about Monte Carlo is everywhere on Amazon. Read the wikipedia about MCTS if you don't have time.

Your keywords: sampling, expansion, simulation, rollouts, backpropagation.

3. Reinforcement learning

Your keywords: policy gradient, gradient descent, learning rate

5. Chess board representation

The paper itself is simplest. The model encodes the board state (e.g. pieces) as a series of one-hot encoding binary values.

6. MCTS vs alpha-beta

enter image description here

Improve this answer
1
  • 4
    thanks very much for the detailed answer. So it seems that unfortunately there is no straight forward guide. My point is that of course as a CS major I know about neural networks, but not the specific kind of deep networks that were used in Alpha*. I roughly know about MCTS as introduced by Bernd Brügmann, but not the specific kind of adaptation for Alpha*. Same for reinforcement learning. For example Andrew Ng's course on ML is very broad and covers stuff like PCA/LDA, which as far as I know is completely irrelevant here. But I guess, I have to work it out myself then ;-)
    – ndbd
    Dec 11, 2017 at 10:55
7

I don't have enough reputation to comment, but AlphaGo Zero Explained In One Diagram is pretty good.

I also really like this tutorial.

Note that the first link doesn't describe when to create (expand) nodes. That part can be a little confusing. This link may help.

Improve this answer
1
  • That's good. I like it. +1.
    – SmallChess
    Jan 9, 2018 at 14:37
3

This presentation summarizes the journey to AlphaGo

This is a presentation of the underlying theory

This is the book

Plus, I guess, anything on deep convolutional neural networks.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.