3

I would like to have Stockfish (or another strong engine) automatically analyze a very large database of games (>1M games), so that the analysis is as strong as possible while making the best use of my computational resources. A naive approach would be to run Stockfish on every position in every game to a certain depth, and then report the evaluation. But this would waste CPU cycles on calculating trivial positions for way too long, for example positions in which a simple recapture is obviously the only not-losing move. How should I determine how long I should let the engine think in each position?

  • Can you run a shadow search and check the magnitude of the evaluation? If it's over a certain threshold, run a deeper analysis. The hash-table should help you in the next search. – HelloWorld Oct 19 '15 at 23:26
  • Also, you might want to run the analysis across a cluster. – HelloWorld Oct 19 '15 at 23:26
  • Hi @StudentT, thanks for the comments. Yes, I intend to run this across a cluster. What do you mean by a shadow search? And how can I use the hash-table in the next search? – Big Dogg Oct 19 '15 at 23:37
  • Sorry, I meant shallow search. I misspelled. I'll type a proper answer. – HelloWorld Oct 19 '15 at 23:40
  • You might want to consider merging the games into a variation tree before you start evaluating positions. Otherwise you will invariably evaluate the same opening lines many times. – BlindKungFuMaster Oct 20 '15 at 8:07
1

Run a shallow search (you might want to add multi-pv). Check the magnuitude of the lines if you want to re-run a proper search. Although this sounds weird, but this is a very common strategy in Stockfish (eg: move reductions). Your won't waste your previous search because the hash-table should speed up your next search. You can read more here.

For example, you wouldn't want to waste CPU cycles in a position where white is a rook ahead.

Otherwise, I don't see a non-trivial way to detect if a position is "interesting" to search.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.