I would like to have Stockfish (or another strong engine) automatically analyze a very large database of games (>1M games), so that the analysis is as strong as possible while making the best use of my computational resources. A naive approach would be to run Stockfish on every position in every game to a certain depth, and then report the evaluation. But this would waste CPU cycles on calculating trivial positions for way too long, for example positions in which a simple recapture is obviously the only not-losing move. How should I determine how long I should let the engine think in each position?
11
-
Can you run a shadow search and check the magnitude of the evaluation? If it's over a certain threshold, run a deeper analysis. The hash-table should help you in the next search.– ABCDOct 19, 2015 at 23:26
-
Also, you might want to run the analysis across a cluster.– ABCDOct 19, 2015 at 23:26
-
Hi @StudentT, thanks for the comments. Yes, I intend to run this across a cluster. What do you mean by a shadow search? And how can I use the hash-table in the next search?– Big DoggOct 19, 2015 at 23:37
-
Sorry, I meant shallow search. I misspelled. I'll type a proper answer.– ABCDOct 19, 2015 at 23:40
-
You might want to consider merging the games into a variation tree before you start evaluating positions. Otherwise you will invariably evaluate the same opening lines many times.– BlindKungFuMasterOct 20, 2015 at 8:07
|
Show 6 more comments
1 Answer
Run a shallow search (you might want to add multi-pv). Check the magnuitude of the lines if you want to re-run a proper search. Although this sounds weird, but this is a very common strategy in Stockfish (eg: move reductions). Your won't waste your previous search because the hash-table should speed up your next search. You can read more here.
For example, you wouldn't want to waste CPU cycles in a position where white is a rook ahead.
Otherwise, I don't see a non-trivial way to detect if a position is "interesting" to search.
