Are there any chess engines that can adapt and 'learn' how to play better with an opponent? One with certain system parameters that can be adjusted while playing so the computer can 'respond' better to a human player?


Learning to play better chess by itself is a part of reinforcement machine learning (temporal difference learning). A simple google will give you some academic papers. My recommended papers (that I've read myself):

While the programs are different. They share the similar idea:

The idea is like this:

  1. Come up with an evaluation function that you think appropriate
  2. Implement a search algorithm (mini-max, nega-max etc)
  3. Initialise the weights in the evaluation function. The weights would be your features
  4. Play a game against itself
  5. Record the result of the game and compute the error.
  6. Apply the following TD-Leaf equation to change the weights

enter image description here

  1. Play more games until you believe your values have converged

The equation updates the weight vector by the gradient, scaled by a constant (learning rate). The errors would be the temporal difference (d(t)) in the equation. They would need to be added for all moves until the end of game, scaled by the distance from t. Please google TD-Leaf if you want to know more.

While there have been some serious attempts at applying the TD-Leaf algorithm, none of which was strong enough to compete with Komodo and Stockfish. I think that's because:

  • Lack of determination. The authors wrote their TD-Leaf chess engines for their research. Once that was done, they moved on to something else, something much more profitable for them. For example, the NeuroChess program was written by a very well-known AI export. He works for Google and I think Google has more challenging task for him.
  • Weak search algorithm
  • Not challenging anymore. A human tuning chess engine can easily defeat the best grandmaster. Why would we waste time and resources to further improve it? Why not spend the money on Go, where the best human player has yet to be beaten?

Some remarks:

  • While it's possible to train a model from random weights, we don't want to do this because it would lead to extremely long training times. We can give starting values with some human-knowledge, or bootstrapping (read the Giraffe paper for details).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.