I am referring to this picture.
What do the individual neurons in the ANN do? Do they represent criteria for the evaluation, like for example material, pawn structure, king safety and so on?
Chess Stack Exchange is a question and answer site for serious players and enthusiasts of chess. It only takes a minute to sign up.Sign up to join this community
I will extend a bit on Scounged's comment, which in principle suffices as an answer.
Indeed, each neuron is nothing more than a little funtion computer that takes all inputs, multiplies them with a weighting, and "fires" if the resultat is above some level (e.g. by a sigmoid, but 0/1 would work as well - anything nonlinear works).
The ineffability of NN always has been a main criticism against them. It is typical "emergent" behavior, and any NN that is large enough can no longer be split into sensible sub-units, even if the emulated function can. (It is very possible that a NN that simulates the KR/K mating technique is still simple enough for such an analysis.)
"Understanding NN" is cutting-edge research. "Understanding" as in "understanding the math behind it" is common knowledge and accessible to anyone with average math and computer science abilities, but "understanding" as in "understanding how it solves a task" is difficult.
It would be an interesting experiment to kill a specific neuron of Leela and see what happens, but I bet you could kill a specific neuron in my brain and nothing will happen to my ELO. Specifically, I do not expect that Leela suddenly neglects king safety, since Leela already has too many neurons.