In a nutshell, the standard approach to tuning a chess engine's parameters is to:
- Define the parameters
- Give the parameters nominal (starting) values
- Run the engine to see how it performs
- Tune the parameter values to try to improve its performance
Then repeat Steps 3 and 4 until you've reached your goal for performance.
The usual approach to doing this is to set up a laboratory where engines face off in engine tournaments. Multiple games are used in which the engine plays both colors. The main tournaments of interest involve running an engine with parameter value set A against the same engine with parameter value set B.
As you can probably guess, results from this approach are heavily dependent on:
- The parameters chosen
- How the parameters are specified
- How the parameter values are varied throughout the testing
- How the engines are run (limited ply-depth, limited time, sensitivity, etc)
This approach also consumes a lot of time.
A more recent (and innovative approach) was developed in 2010 by researchers using Genetic Algorithm techniques to a) specify the parameters, and b) tune the parameter values. The investigators first ran an engine with a starting, nominal set of parameter values against a set of grandmaster games to see if it could effectively choose the "best move". The "best move" was defined as the move the grandmaster made*. Wherever it failed to do so was recorded. Then, another parameter value set was tried, and relative performance vs the prior run determined.
Then, a programmatic approach to combining the parameter values was tried, using the Genetic Algorithm principle of survival of the "fittest". Here, "fittest" means the one that generates output that most closely matches the ideal. (It also happens to be a pun on the statistical technique of "least squares fit" regression, a technique used to judge the quality of the approximation.)
Only after engine parameters have been found that can mimic a GM reasonably well does the actual engine tournament phase begin. In this phase, different parameter value sets are once again pitted against each other, this time directly. Genetic Algorithm improvement techniques are applied to generate successively better generations of the engine.
In this research project, 36 parameters were used, including all of the material values of the pieces, and many of the more common strategic evaluation criteria, such as backward pawns, weak squares, bishop pair, and so on. However, the researchers added some new parameters, such as "king pressure", "mobility" values for each kind of piece, rook on a file adjacent to the king, rook on a semi-open file, rook attacking the king on the a-/b-/g-/h-file, separation between a passed pawn and the defending king, and more.
Unfortunately, the researchers don't elaborate on how they came up with this suite of parameters, and what alternate parameters they may have tested and rejected. It would be reasonable to assume that they began with a much larger set, and determined (through trial and error) which ones had the greatest effect on performance, and which ones were either insignificant or derivative, and so could be dropped.
If this sounds like it might be useful, you can find the research here.
*A caveat about a phase of the approach that the researchers used is in order. In his Introduction to Understanding Chess Move by Move, John Nunn chose "...hard fought games between strong grandmasters..." to illustrate his themes. He then adds:
Readers may be quite surprised to see the number of question marks that adorn the games in this book. Surely, you might think, with just thirty games to select, it should have been easy to find some sound games. However, I can assure you that it was not. ... it is possible to find fault with virtually any complex, hard-fought game... I have never felt that my play was anywhere near completely accurate, so I personally don't find these revelations distressing. However, some may find it hard to admit that chess as played by human beings is less accurate than previously thought.
The point that Dr Nunn raises suggests that the researchers' initial approach to setting the engine parameters by requiring them to imitate grandmaster moves may be flawed because human play is flawed. In fact, it's well established that engines already play better than humans.
Therefore, perhaps a better approach to setting the initial parameters would be to match a new engine against a superior existing engine.