When AlphaZero learned on its own that a queen has a value of 9, a pawn 1, knight 3, etc., then did it also need to learn on its own that a piece has a value? Or was it that the concept of value was baked into its code and it only had to figure out on its own what the values are. The value of a queen is not strictly 9, but it is relatively more valuable than a pawn.
-
25Pieces don't have a value though!– DavidMay 31, 2021 at 20:26
-
18It didn't learn on it's own that a "queen has a value of 9" because she doesn't. That's a highly simplistic rough evaluation humans have given her with no other context as a guiding tactic for new players.– Adam BarnesJun 1, 2021 at 6:40
-
Not strictly 9, but only that a queen is worth more than a pawn, for example. Of course, exceptions exists, for example if the pawn can deliver a mate. It is very clear that Alphazero values each pieces differently, as this is prime requisite in position evaluation. If Google baked this notion of piece value, then Alphazero only had to find on its own the piece value, which could be dynamic based on the demands of the position. Hence, the notion of piece value is a-priori.– daparicJun 1, 2021 at 11:25
-
1I think you meant stockfish, alphazero doesn't assign a numeric value for pieces only for boards– RainbJun 2, 2021 at 4:22
-
2The "Zero" in alpha zero is because it was "given no domain knowledge except the rules". en.wikipedia.org/wiki/AlphaZero. MuZero was a variant that didn't even know the rules! en.wikipedia.org/wiki/MuZero– Mooing DuckJun 2, 2021 at 15:32
|
Show 1 more comment
7 Answers
In short, no there was no explicit concept of relative piece importance coded in. But the position evaluation is based on a neural network which possesses weights which indirectly account for the pieces in the position.
That said, in this AlphaZero paper they use self-play games to fit a linear model that predicts the game outcome from the difference in numbers of each piece only, result:
-
1My original question was, did alphazero also have to learn that values can be placed under a piece. And the answer to this is no. This concept that a numeric value can be placed under a piece is pre-programmed. That is, it is already been told a-priori that there are empty buckets out there that it needs to fill-in. Otherwise, it cannot construct from out of the blue, these empty buckets. However, Alphazero can place optimal values under each piece based on a given position and what it learned.– daparicJun 22, 2021 at 5:00
-
@eigenfield: AlphaZero was not taught that and it doesn't do it at all. It only evaluates entire boards, and doesn't do anything piece by piece. There are no buckets. The values in this answer were computed by humans based on scores AlphaZero gives to various positions with and without material differences, they're not numbers that AlphaZero uses internally at all. Jun 6, 2022 at 19:31
-
Then who taught alphazero the requisite conditions for a win? Why will it strive for a win? Why not for a lose? From these basic ponderings, it is very clear that some initializations has to exists. This is the question.– daparicJun 7, 2022 at 3:16
-
If a computer can correctly judge the "Penrose position", then I, for one, welcome our new overloads. But this is not happening yet. Therefore, putting too much trust on A.I. is a dangerous concept.– daparicJun 7, 2022 at 3:18
-
It will strive for a win because if it wins a game it will receive a positive reward in reinforcement learning, so it will be more likely to repeat actions that led to that win in the first place. (But that topic is beyond this question.) Jun 7, 2022 at 9:24
It sort of did. "Sort of" because once you examine how neural networks work it's not clear what AlphaZero is actually learning.
AlphaZero has a neural network evaluation function. How that works is that it takes as input the position on the board (along with other things like whether a pawn can capture en passant, whether castling is possible, etc) and converts that into a list of candidate moves to play as well as the probability of winning. There is no "queens are worth 9 pawns" input, there is only "there's a queen on this square". Of course, queens usually have quite an impact on a position, so the neural network will quickly learn that having queens strongly correlates with a high probability of winning, but nobody actually knows how much value the neural network ascribes to the queen.
What was definitely not given to AlphaZero was the idea that each piece has a value. Remember, it was only given the position on the board.
-
Actually, the neural part doesn't "converts that into a list of candidate moves". It simply evaluates each position. The MCTS part then lists the candidate moves and search the tree of potential moves.– JeffreyJun 1, 2021 at 1:49
-
6@Jeffrey are you sure? I am pretty certain that AlphaZero's neural network also provides a "move probability" for the next move, which is also how the MCTS decides which branch to search first.– AllureJun 1, 2021 at 1:54
-
5Ok, I'll take it back. It outputs both an evaluation and a policy over moves: "a continuous value of the board state vθ(s)∈[−1,1] from the perspective of the current player, and a policy → pθ(s) that is a probability vector over all possible actions." from web.stanford.edu/~surag/posts/alphazero.html– JeffreyJun 1, 2021 at 2:01
-
1
The short answer is: No.
AlphaZero learns to evaluate only the position. The position consists of all the pieces and their placement on the board (plus castling and en-passant information). There is no way to distinguish the material's value from the position's strength -- which is a really beautiful idea and offers a lot of learning potential for humans. AlphaZero chooses its moves to maximize the (positive) value of the position. (The checkmate-win has the maximal positive value, the checkmate-loss has the maximal negative value.)
The information about the value of single pieces is stored nowhere. It can be potentially obtained by comparing position evaluations when removing/adding pieces to the board. But it will also influence the position's strength which cannot be separated from the material's value itself. So in best case some average value of pieces (when averaging over many positions) might be estimated in this way.
One central piece of alphazero is the neural evaluation function. This function takes the board as input and produces a value as output. The only input is what pieces are on the board and where.
So, you can see it as the programmers enforcing this rule: the value of a position depends on which pieces are placed where.
This is what the AI "knew" as it as born, if you will. It knew that pieces being present and located relative to each other had value. Now, one possible way to score the board is a linear combination of "piece value" times "number of pieces of that kind". But in practice, the network learned a much more complex and non-linear function of the pieces present on the board.
It certainly could do what we often use: queen times 9 + knight times 3 + ... but it learned much better than that.
https://medium.com/applied-data-science/alphago-zero-explained-in-one-diagram-365f5abf67e0
The deep neural network on the left gives you a hint of how much more complex the evaluation function can be, over this simplistic linear evaluation.
-
5@eigenfield Yes, AlphaZero can identify this notion. What AlphaZero knows about the rules originally is that it can "play a move" by choosing a start square and an end square. Then it can "read ahead": when it selects a move in its head, it is provided with the resulting board position, or with a warning if the move isn't legal. Originally it didn't even know how pieces were supposed to move - it had to learn through trial and error that bishops move diagonally and rooks orthogonally, etc.– StefJun 1, 2021 at 8:38
-
3
-
2@Stef: I don't believe that. Are you sure that it had to learn castling and en passant by trial and error, not to say the 50-move rule? If those rules were not programmed into it, which of them did it fail to learn? Jun 1, 2021 at 10:11
-
2@user21820 I seemed to recall the input included some kinds of flags on the game position, for instance answering the question "would this move repeat the position?" or "is castling still allowed in this position?", but it's possible it has a more than that. Input is described in Deepmind's paper on arxiv, starting page 12, "Domain Knowledge".– StefJun 1, 2021 at 13:37
-
1@user21820 So there is indeed a boolean input flag for each castling, but it's up to AlphaZero to understand what this flag means.– StefJun 1, 2021 at 13:43
From the comments below, maybe a good summary of this answer:
Alphazero doesn't necessarily respect piece value the way we do, but both Alphazero's experience and our piece valuation (which is effectively our human experience) are approximations of same the balance of power for a given board state.
Piece value is a form of bias, by inherent definition of saying that one piece is worth more than other. It's an artifact of how humans reason about the strength of a certain position/play. Our piece valuation, when taken as literal truth, suggests that a queen is always worth more than e.g. a knight or a pawn.
Part of what makes machine learning so interesting is that having an unbiased objective view can fundamentally shake things up. For example, in certain strategies you could do more with a knight than a queen, therefore throwing out the idea that a queen has more value than a knight.
However, this is just a small subset of strategies and positions, among million others. You already touched on this point yourself, when you commented:
Not strictly 9, but only that a queen is worth more than a pawn, for example. Of course, exceptions exists, for example if the pawn can deliver a mate.
Humans aren't able to mentally track different individual piece valuations for millions of different positions or plays. Therefore, humans use a valuation that works in most cases. In most cases, a queen is more useful than a knight. In most cases, a knight is more useful than a pawn. This is why our piece valuation is the way that it is.
However, machines don't have these limitations. They are much more able to track different piece valuations for each distinct board situation, and therefore a machine doesn't need to rely on some arbitrary global piece value, as it is a needlessly imprecise measurement.
For a machine, it makes a lot more sense to simply calculate each piece's value on the fly, rather than relying on some half-baked approximation.
Different strokes for different folks. Machines are better at this, and therefore they don't need the oversimplified crutch that our piece valuation represents.
In order to understand machine learning, and how the machine "thinks", you need to sidestep your own bias and preconceived observations. Unsurprisingly, sidestepping human bias is by far the biggest challenge in machine learning - more so than the raw learning algorithm in and of itself.
That being said, if you asked Alphazero to play a billion games, look at the victories, asked it to re-simulate those games after taking away one random piece, and then asked which type of piece had the biggest impact on turning a victory in a defeat, then it would very likely tell you that it's the queen.
That is sort of the same as saying "the queen is the most valuable piece", but in a much more pedantic and deeply analytical way. It is pedantic to us, but for Alphazero this isn't pedantry but rather necessary accuracy. One man's pedantry is another man's precision.
Our human piece valuation system isn't wrong, it's just very crude and unrefined.
-
4@eigenfield You may be misunderstanding. Human preconceptions aren't a bias to how AlphaZero works, they're a bias to our understanding of AlphaZero, as you can tell by your own question, which assumes from human bias that AlphaZero thinks pieces have relative value.– MagmaJun 1, 2021 at 11:51
-
3@eigenfield: The balance and tactics involving chess is significantly more fine-grained and nuanced than humans having missed the tactic of "marching [your] king to the center of the board and deliver a win by so doing". Humans have reasonably approximated a good overall strategy, but not to the extent that a machine can. Those "main lines" you point out are effectively "the human approximation". The fact that Alphazero independently came up with a similar approach is proof that humans did a reasonably good job at approximating, i.e. we were on to something, but with less detail.– FlaterJun 1, 2021 at 11:53
-
3@eigenfield: Phrasing matters here. Alphazero doesn't necessarily respect piece value the way we do, but both Alphazero's experience and our human piece valuation (which is effectively our human experience) are approximations of same the balance of power for a given board state.– FlaterJun 1, 2021 at 12:25
-
4@eigenfield "Pieces have relative values, this is truth#1" [Citation needed] This seems to be where your bias comes in, and where you are fundamentally misunderstanding things. Alphazero does not start with "truths" like this programmed into it, and it does not necessarily simplify its neural model to conform to it (unlike humans!). This seems to be more of an issue of you not understanding neural networks than anything else. Jun 1, 2021 at 13:22
-
4@eigenfield: "The fact that it played well, means it learned those values like a human did." Playing the game well is not measured by adherence to the piece valuation. You're conflating two very different things here. Think of it this way: a cheetah uses its legs to move forward very fast. A tuna can move forward just as fast, but it has no concept of legs. Or a zeppelin and a helicopter both hover in the air, but for very different reasons. You're confusing doing something (moving forward, hovering, winning at chess) with how you do something (leg movement, buoyancy, chess strategies).– FlaterJun 1, 2021 at 13:23
AI engineer here.
The answer to this question is hiding in the name: AlphaZero. This refers to "zero prior knowledge of the game". AlphaZero can also play Go.
Now, it might be a bit strong to say that AlphaZero has absolutely no knowledge about chess. AlphaZero knows that it's a turn-by-turn board game, and that the 3 possible outcomes are a win for either side or a draw. (unlike Go, which has no draw). But AlphaZero had to figure out itself how pieces contributed to winning.
Since AlphaZero is a Deep Neural Network, it can quickly learn highly complex valuation functions. That is to say, it can learn a valuation function which includes how well each piece cooperates with other pieces.
What might also be surprising is how Google got started with these valuations. They simply started with entirely random values! The learning algorithm just tweaked the values step by step, and gradually won more and more games.
-
So just as I originally asked, they baked the concept of the existence of a value to a piece. And then, through its learning process, it learned these values depending on the demands of the position. What would have happen if the value existed on the square and not on the pieces? If they baked the notion of value to the pieces, then it would be oblivious that the value is in the squares. So, they will then re-baked it again and tell it where to value. That is, requiring human interference. This is what my question was.– daparicJun 3, 2021 at 6:06
This is a short answer, but hopefully it helps.
Consider how AlphaZero values pawns. It throws them away in disgust sometimes, usually to open up files or diagonals on which to plant menacing rooks and bishops. AlphaZero doesn't care one bit about the nominal value a piece may have (I'd argue most human players also don't care about a piece's value).
-
1Indeed so. There are times when a pawn, due to its board position, is worth far more than a queen. Jun 2, 2021 at 20:15
-
All of you have misunderstood the question. How would alphazero adapt, if the rule is such that, the quicker you move-respond, then one points will be added to you. And then, at the end of the game, these more-respond points gets added and compared and the bigger points wins. Hence, the piece values no longer matters here, as the condition for a win is as I described. Can alphazero find out this condition for a win?– daparicJun 7, 2022 at 3:41
-
@eigenfield I'm not sure what you mean. But I don't think AlphaZero cares about the value of a piece, that is, it doesn't look a queen and say "it's worth 9", instead it evaluates an entire position (not individual pieces), and then plays the move that maximises its expected win probability, at least, I think that's how it works.– stevecJun 7, 2022 at 3:47