How to get stockfish to play a selfmatch without clearing the hash table between games

Question

A while ago I asked a question about how to let Stockfish play 10 games with itself for a limited number of moves:

Python script to let stockfish selfplay 10 games from a given position

This question was brilliantly answered by the user Phonon.

Now I would have a followup question. Since the games are starting from the same position, I would like to avoid to clean the hash memory, as I understand Stockfish does every time it starts a new game. Ideally, even the first game should avoid the hash table clearing since I might have analyzed the position before starting the self match and I would like to keep the information saved in the hash memory.

Is there a way to do that?

Answer 1

I would like to keep the information saved in the hash memory. Is there a way to do that?

Yes, there is.

Stockfish is open source. So you can examine the code, modify it, recompile and rebuild.

What you need to do is:

1. Find where Stockfish creates and uses the cache
2. Modify the code to save the cache to disk when the game finishes
3. Modify the code to optionally (you won't want to do it for every new game) load the cache from disk for a new game

Answer 2

The script from that other answer that you've linked to does not appear to be resetting the hash between subsequent games.

Stockfish uses the UCI protocol, and engines that implement this protocol (most modern engines) will typically clear the hash table when it is issued the ucinewgame, but this is not required of engines that implement the protocol.

The documentation for Python Chess indicates that this is issued based on the game parameter which is supplied to the play() method.

game – Optional. An arbitrary object that identifies the game. Will automatically inform the engine if the object is not equal to the previous game (e.g., ucinewgame, new).

My assumption is thus that the ucinewgame command is not issued if this parameter is not specified. Since the script in the linked question does not specify the game parameter anywhere, nor does it appear to do anything else which would send the ucinewgame command, I do not believe that the hash table is being cleared out between games.

However, if you did want to clear out the hash between games, you would create a new object (type does not matter) every time that you start a game, and pass that into as the game parameter for each move of that game. Whenever the value of this parameter changes between subsequent calls to the method, the ucinewgame command would be sent to Stockfish, informing it that the next commands are for a new game and it should do whatever it needs to do to start a new game (typically this would involve clearing out the hash table).

---

I also don't see anything that sets the size of the hash table before the game starts. The specification for the UCI protocol specifies that the default size for the hash table should be very small, so this script would not be making very good use of this table as it is.

For Stockfish, the default hash size is 16 MB, and it allows values up to 33554432 MB (~33.5 TB). They specify that you should optimally set this value to the amount of RAM you have available on your machine - 1 or 2 GB. E.g. if you have 8GB of RAM, set a value of 8192 MB - 2048 MB = 6144 MB (allocating 6GB for the hash table).

To set this in Python, before the for loop, you would call engine.configure({"Hash": 6144})