4

I have a chess GUI which I've recently been updating to do preemptive searches (that is, depth searches for all possible opponent moves) that I'd like to speed up, since the bot searches go to at least depth 16 for every move, and each move runs a new Stockfish 11 command line (with minimum settings) in a separate thread.

Now, I have some solutions for the threading issue, but one would involve knowing more chess math than I know now.

I noticed that, after a certain search depth, the PV's first move, and then first few moves, really don't change that often, and I'd like to know if there's a formula out there, similar to one that I used to calculate winning odds.

I considered the following:

  • Do a depth search of n on the position after hypothetical opponent move m.
  • Retrieve the pv for depth n.
  • Append it to a corpus.
  • Do a depth search of n+1 on the same position.
  • Retrieve the pv for depth n+1.
  • If a text prediction algorithm using the corpus will produce n+1[pv][0:x], such that x is sufficiently large (needs more math) to be reliable, cease the depth searches and recommend n+1[pv][0].

I'm thinking that there might be a simpler way of reaching a similar outcome, since pv[0] would have to arise every time in the text prediction.

What other Stockfish info might I want to consider in generating a statistical confidence that the recommendation need not search any deeper?

Improve this question
13
  • This sounds difficult to tackle as it depends a lot on the position. Generally speaking I would guess that a stable evaluation may imply more stable future PVs. However that's not a guarantee, even a stable eval might just be overlooking some deep tactics. However I think it's more likely to keep the PV than an already varying eval.
    – koedem
    Commented Nov 25, 2020 at 17:55
  • Another thing to look at may be the node counts for different moves. I don't quite understand how you intend to use this / how your setup works, however, if you have some search and the top move being looked at takes most of the nodes, i.e. all non-optimal moves are dealt with quickly, then I'd think it's more likely that the top move won't change. (as all other moves are "refuted" very quickly) Whereas if the non-optimal moves take a lot of nodes they may be on the verge of becoming the new best move if that makes sense.
    – koedem
    Commented Nov 25, 2020 at 18:00
  • Sorry for the many comments but I have so many questions about this setup. So you run a fresh independent search for each possible move? You are aware that this is massively inefficient? (especially if you "only" care about the root position evaluation and move, but even otherwise) Not only does that deprive oneself of transposition table speedups between moves but also any search speedups through search windows. (for instance if you want to know if move x is best you don't need to know the evaluation of all other moves, you just need to know that they're worse than x; much faster to verify)
    – koedem
    Commented Nov 25, 2020 at 18:27
  • @koedem, so the naive preemptive mover that I built has the bot evaluate at depth for all opponent moves. For instance, if we're at startpos and the bot is black, the function I run threads 20 Stockfish 11 UCI's. Each UCI searches at depth (let's say 20), and then returns the info at depth 20. The program then kills the UCI's and the threads. All of the info is then stored so that, when the human does make a move, the response is faster. The inefficiency you mentioned obviates any gains from that approach. Hence my quest for shortcuts to either close the UCI's earlier or use just one.
    – Joshua Harwood
    Commented Nov 25, 2020 at 19:04
  • 1
    @koedem, yes, I tend to build brute-force, naive programs, and then work on optimizing them.
    – Joshua Harwood
    Commented Nov 25, 2020 at 19:08

1 Answer 1

Reset to default
4

I did a study using stockfish 14.1 on all positions (6 games) played Hikaru in the 2022 FIDE Grand Prix Leg 1.

Procedure

  • Get all positions where Hikaru is to move.
  • Analyze those positions with stockfish 14.1 at different base depths and depths+1 and save the bestmoves.

Results

    base_depth  adv_depth  match(%)
0            8          9  77.06
1            9         10  80.09
2           10         11  85.28
3           11         12  83.55
4           12         13  85.71
5           13         14  84.42
6           14         15  92.64
7           15         16  85.28
8           16         17  88.31
9           17         18  89.61
10          18         19  87.01

The match column is the percentage that the bestmove in base_depth is similar to the bestmove in base_depth+1 or adv_depth.

The highest match is in base_depth 14 at 92.64%.

Plot
enter image description here

Run a linear regression on that data using statsmodel and comes up with the following formula and margin of error of +/-6% at 95% confidence level and an R-squared of 0.641.

margin of error = rmse * 1.96

Formula

match_rate_pct = 1.1218 x base_depth + 70.1277  [+/- margin of error]

Example

If base depth is 17 the approximate match_rate will be:

match_rate_pct = 1.1218 x base_depth + 70.1277
match_rate_pct = 1.1218 x 17 + 70.1277

match_rate_pct = 89.2% +/- 6%

min match_rate_pct = 83.2%
max match_rate_pct = 95.2%
Improve this answer
2
  • How interesting. It would be interesting to continue to higher depths and see where the move match plateaus, if that stage can be reached, since it seems to be dropping at the highest tested depths but surely we expect it to ultimately rise again to nearly 100%?
    – Mobeus Zoom
    Commented Apr 30, 2022 at 15:53
  • I think it would make more sense to do a linear regression on log(1-p).
    – Acccumulation
    Commented May 2, 2022 at 4:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.