3

I am currently programming a chess engine and running my own tests to decide what changes improve the engine's play. I am struggling to find a meaningful answers regarding the output of fast-chess engine matches. Here is a sample output from one of my tests:

Results of ChessEngine vs ChessEngine_Bugless (10+1, NULL, NULL, scrapedOpenings.epd): Elo: 277.00 +/- 40.90, nElo: 345.17 +/- 34.05 LOS: 100.00 %, DrawRatio: 17.00 %, PairsRatio: 22.71 Games: 400, Wins: 307, Losses: 42, Draws: 51, Points: 332.5 (83.12 %) Ptnml(0-2): [4, 3, 34, 42, 117]

I understand Elo, nElo, LOS and DrawRatio. But what does PairsRatio refer to? Also does Ptnml refer to pentanomial statistics, if so what are the five 'bins' the results are being put into?

In an attempt to run fair matches I have scraped some opening positions (5 to 10 moves deep) and fed them into stockfish and only kept those that were less than 100 centi-pawns advantage for either side (this is what scrapedOpenings.epd refers to)

1 Answer 1

3

(Not an expert, but I follow computer chess closely so I have some idea.)

In computer chess engines take turns to play the same opening. This is because left to their own devices, engines play a very narrow range of openings, and it makes sense that strong engines should be able to play different types of positions; furthermore for the very strongest engines, the opening position is almost surely a draw, so you'll need an untold number of games to tell if your patch works.

What is standard is to give some opening book to your two engines, and they take turns to play with White. If the opening is White-favored, then the goal (at least at high levels of strength) is to win with White, then draw with Black. If this happens, then the engine wins the opening pair 1.5-0.5. This is what PairsRatio probably refers to - how many openings are won by one engine compared to the other.

As for the five bins, because there are two games, the five possible results are:

  • Engine goes 2-0 (wins both games)
  • Engine goes 1.5-0.5 (wins one, draws the other)
  • Engine goes 1-1 (both games drawn, or wins one and loses the other)
  • Engine goes 0.5-1.5 (loses one, draws the other)
  • Engine goes 0-2 (loses both)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.