9

So this is a bit of outlier question.

I am writing a chess library, essentially from scratch. You can find much of the code here, with the intention of using it for a GUI and/or an engine. (The game is Grand Chess but for the purposes of this question it doesn't really matter.)

I am currently writing unit tests to verify that my functions work as expected. And I was wondering if there was some sort of library or database of suggested positions to test I can use and work from, categorized by whether they're Checkmate, Stalemate, Check, Legal, Illegal, etc.

tl;dr I am looking for a list of positions to unit test my code against.

You can find my current tests here, I add to them every few days. However, I want to make sure the tests are exhaustive before I go debugging the code. (Half of them fail currently).

Edit : to clarify : I am not looking for engine ("best move") tests. I am looking for board representation ("is this position checkmate") tests. I already have a few puzzles lined up for engine testing.

  • You're coding a variant. Standard chess dataset won't work for you. So I'm afraid you're on your own. – SmallChess Sep 27 '16 at 8:03
  • @StudentT i am looking for something i can start from . a regular databsase would do just fine, since you know, i can edit it. – asibahi Sep 27 '16 at 8:04
  • 1
    There're MANY such test sets, are you happy for something like perft, tactical exercises in standard chess? – SmallChess Sep 27 '16 at 8:07
  • @StudentT I couldn't find anything via Google, thus why I am asking here. Any info would be helpful. – asibahi Sep 27 '16 at 8:08
  • What you want to test has little tp do with board representation. You'll want to test board representation after making/unmaking moves or importing positions. Detecting mate/stalemate requires an evaluation function, and testing a position for legality should be a function of its own which has much more to do than checkung the board representation. – Queeg Mar 18 '17 at 19:25
2
+50

When reading your question, my gut reaction is that your scope is too complicated for unit testing. I recommend a quick read through the free e-book Unit Testing Succinctly. However, I have no experience writing chess code (maybe your paradigms are different) - although I do software for a living.

A unit test should be very simple and test a function that does 1 single thing. Then you can combine the functions with some reasonable expectation that they will work. For example, I would expect a unit test for each piece to determine if a particular move is legal. A unit test for each piece to determine if it is putting the king in check. A test for each piece to determine where it is attacking, etc.

Testing a position seems like a very complicated unit test and would be much harder to do thoroughly. Instead write smaller tests against smaller functions and then know that those individually work - evaluating a position is just a matter of iterating over the simple functions.

If you want to test a position for a good (not forced) move, I think unit test's will artificially limit the long term development and strength of your chess engine... a binary result of a unit test will force your engine to make the same move every time.

I'd also look at adding unit tests for 'most direct' path to a mate with known endgames. I'd look to add unit tests for traversing through known openings as well. Mid game unit tests will be much harder - maybe plugging in a position and evaluating that the engine produces a usable result (which is a binary response).

For the question of evaluating a set of positions for your engine, you may do far better putting this question on https://stackoverflow.com/ with the "chess" tag.

  • Thanks for the answer. However, as you can probably tell from the comment conversation under the question, I'm not looking to test the engine algorithms. I didn't even get to those yet. I'm looking to test if my code recognizes whether a standing position on the board is checkmate, or a stalemate, or not, which is "just" counting legal moves and whether the king is in check. In fact since I posted that question I went on to compose a small collection of positions to use as unit test cases. I might post that as answer. (I'm completely surprised someone offered a bounty for this, tbh.) – asibahi Mar 22 '17 at 15:38
  • This doesn't answer the question. – SmallChess Mar 22 '17 at 15:45
  • 1
    Furthermore, the notes about unit testing is just wrong. – SmallChess Mar 22 '17 at 15:47
  • @asibahi I didn't have enough rep to post in that comment section, so I had to add an answer. I really think you will be better served in a programming forum (stackexchange) with this question. But on this comment... you already identified individual unit tests -> For each piece test is there a legal move. If iterating over that returns false for each piece, you have stalemate, or checkmate if you are in check. You don't need a vast collection of positions for that. By testing each piece individually for its current state, you can iterate over multiple pieces to evaluate the position. – Paul Mar 22 '17 at 17:08
0

While this is an old question, I thought that the concept presented in this blog might be useful: http://scionsoftware.com/blog/write-tests-by-playing-chess

The idea is that you would play a chess game on a GUI and have a mechanism to capture the state of the chess board by serializing to a file.

You could name those files per the test cases and feed them into whichever test method you define: IsCheckmate; IsLegal; IsDraw

One of the main reasons you would want to use a natural UI to create these test cases, outside of ease of creation, is that enough conditions also rely on move counts: castling, en passant, draw conditions.

0

I don't know any unit-testing database for chess engine (in general), and yes, writing exhaustive unit test is almost impossible.

You can maybe explore alternative testing technic like property-based testing (QuickCheck in Haskell, I don't know the F# environment but it certainly exists something like that for F#), which can automatically generate a large amount of "position" and test them using property defined by you.

Hope this will help a bit :) !

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.