Unit Testing a chess board representation

Question

So this is a bit of outlier question.

I am writing a chess library, essentially from scratch. You can find much of the code here, with the intention of using it for a GUI and/or an engine. (The game is Grand Chess but for the purposes of this question it doesn't really matter.)

I am currently writing unit tests to verify that my functions work as expected. And I was wondering if there was some sort of library or database of suggested positions to test I can use and work from, categorized by whether they're Checkmate, Stalemate, Check, Legal, Illegal, etc.

tl;dr I am looking for a list of positions to unit test my code against.

You can find my current tests here, I add to them every few days. However, I want to make sure the tests are exhaustive before I go debugging the code. (Half of them fail currently).

Edit : to clarify : I am not looking for engine ("best move") tests. I am looking for board representation ("is this position checkmate") tests. I already have a few puzzles lined up for engine testing.

Answer 1

When reading your question, my gut reaction is that your scope is too complicated for unit testing. I recommend a quick read through the free e-book Unit Testing Succinctly. However, I have no experience writing chess code (maybe your paradigms are different) - although I do software for a living.

A unit test should be very simple and test a function that does 1 single thing. Then you can combine the functions with some reasonable expectation that they will work. For example, I would expect a unit test for each piece to determine if a particular move is legal. A unit test for each piece to determine if it is putting the king in check. A test for each piece to determine where it is attacking, etc.

Testing a position seems like a very complicated unit test and would be much harder to do thoroughly. Instead write smaller tests against smaller functions and then know that those individually work - evaluating a position is just a matter of iterating over the simple functions.

If you want to test a position for a good (not forced) move, I think unit test's will artificially limit the long term development and strength of your chess engine... a binary result of a unit test will force your engine to make the same move every time.

I'd also look at adding unit tests for 'most direct' path to a mate with known endgames. I'd look to add unit tests for traversing through known openings as well. Mid game unit tests will be much harder - maybe plugging in a position and evaluating that the engine produces a usable result (which is a binary response).

For the question of evaluating a set of positions for your engine, you may do far better putting this question on https://stackoverflow.com/ with the "chess" tag.

Answer 2

While this is an old question, I thought that the concept presented in this blog might be useful: http://scionsoftware.com/blog/write-tests-by-playing-chess

The idea is that you would play a chess game on a GUI and have a mechanism to capture the state of the chess board by serializing to a file.

You could name those files per the test cases and feed them into whichever test method you define: IsCheckmate; IsLegal; IsDraw

One of the main reasons you would want to use a natural UI to create these test cases, outside of ease of creation, is that enough conditions also rely on move counts: castling, en passant, draw conditions.

Answer 3

I don't know any unit-testing database for chess engine (in general), and yes, writing exhaustive unit test is almost impossible.

You can maybe explore alternative testing technic like property-based testing (QuickCheck in Haskell, I don't know the F# environment but it certainly exists something like that for F#), which can automatically generate a large amount of "position" and test them using property defined by you.

Hope this will help a bit :) !