What are the best practices for measuring search and evaluation speed?

Question

I understand that perft is the standard tool for measuring an engine's move generation speed. However, during a perft search you don't execute your search, move ordering or evaluation functions - you just generate all the legal moves up to a certain depth and make/unmake them on the board.

I have found perft tests very useful for optimizing my engine's move generation, but aside from some basic unit tests I don't have an equivalent framework for testing my search and evaluation.

Are there are similar tools, methods or best practices for testing and debugging these parts of the engine?

Answer 1

I've just recently started developing my own chess engine, so i am by no means an expert. But ill still share my thoughts.

I think you can't make a generalization about search speed, as each search engine utilizes different parameters for its evaluation, resulting in cutoffs in search performance. Additionally, certain engines incorporate extensions when they encounter a "good capture" or early exit conditions.

Most search engines provide an output in terms of nodes per second (nps) for measuring the combined speed of evaluation and search. Nevertheless, it's crucial to understand that the interpretation of this metric may differ across engines, as some may include Transposition Table Lookups and other edge cases in their nps calculation.

Another approach for measuring evaluation speed involves disregarding the search phase and focusing solely on static evaluations performed on predetermined board positions. However, this method can be challenging to interpret as well. This is because some engines employ a complex static evaluation process, which may take longer, while others opt for a simple static evaluation but rely on a complex move-ordering algorithms. Ultimately, it's difficult to determine which approach is more effective and thus difficult to say which is more important.