AlphaZero trained itself by self-playing. It used the gradient descent algorithm for convergence (equation 1 in the paper). Please note the training phase had nothing to do with Stockfish.
Later, Google matched the "fully trained" AlphaZero against Stockfish. It's like you practice chess with yourself for four months, participated in a tournament then won all your games!
Hardwares were different in training and playing:
Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using 5,000 first-generation TPUs (15) to generate self-play games and 64 second-generation
TPUs to train the neural networks.
... in chess, shogi and Go respectively, playing
100 game matches at tournament time controls of one minute per move. AlphaZero and the previous AlphaGo Zero used a single machine with 4 TPUs...