1

Is there any fundamental difference (in algorithms etc.) between Alphazero training by playing against itself and when playing a match against Stockfish? Or were the match games basically treated as training game with the only diferrnce that half of the moves were coming from outside?

Also, did they use the same hardware during training and match?

1

AlphaZero trained itself by self-playing. It used the gradient descent algorithm for convergence (equation 1 in the paper). Please note the training phase had nothing to do with Stockfish.

Later, Google matched the "fully trained" AlphaZero against Stockfish. It's like you practice chess with yourself for four months, participated in a tournament then won all your games!

Hardwares were different in training and playing:

Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using 5,000 first-generation TPUs (15) to generate self-play games and 64 second-generation TPUs to train the neural networks.

and

... in chess, shogi and Go respectively, playing 100 game matches at tournament time controls of one minute per move. AlphaZero and the previous AlphaGo Zero used a single machine with 4 TPUs...

  • I understand the hardware part, but am still not sure whether the algorithms were the same. Using your analogy, if I play a tournament game that would still teach me something so could be considered "training" as well. Did the Alphazero vs Stockfish games improve the strength (i.e. the neural network) of Alphazero? – user1583209 Dec 26 '17 at 7:30
  • @user1583209 No. The match games between AlphaZero and Stockfish had nothing to do with training. It wouldn't have improved the models directly. – HelloWorld Dec 26 '17 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.