I am referring to this picture.
What do the individual neurons in the ANN do? Do they represent criteria for the evaluation, like for example material, pawn structure, king safety and so on?
-
I think the best way to answer this question is with an analogy: what do the individual neurons in a human brain do? In general it will be more or less impossible to understand a neural network by looking at each individual node trying to decipher what that node does exactly, since all an individual node does is emit an output signal based on the inputs coming into it.– ScoungedFeb 5, 2022 at 13:42
-
1They simply calculate an activation function (sigmoid, relu, etc).– SecretAgentManFeb 5, 2022 at 14:33
Add a comment
|
1 Answer
I will extend a bit on Scounged's comment, which in principle suffices as an answer.
Indeed, each neuron is nothing more than a little funtion computer that takes all inputs, multiplies them with a weighting, and "fires" if the resultat is above some level (e.g. by a sigmoid, but 0/1 would work as well - anything nonlinear works).
The ineffability of NN always has been a main criticism against them. It is typical "emergent" behavior, and any NN that is large enough can no longer be split into sensible sub-units, even if the emulated function can. (It is very possible that a NN that simulates the KR/K mating technique is still simple enough for such an analysis.)
"Understanding NN" is cutting-edge research. "Understanding" as in "understanding the math behind it" is common knowledge and accessible to anyone with average math and computer science abilities, but "understanding" as in "understanding how it solves a task" is difficult.
It would be an interesting experiment to kill a specific neuron of Leela and see what happens, but I bet you could kill a specific neuron in my brain and nothing will happen to my ELO. Specifically, I do not expect that Leela suddenly neglects king safety, since Leela already has too many neurons.
answered Feb 5, 2022 at 16:57
