-1

I use Arena for tournaments between chess engines.

The interesting thing is that in different 64-bit multi-processor computers ranking is slightly different. I was expecting that they should be the same.

Do you have any idea why this happens?

As an example Booot 6.2 64-bit loses most of the games on a low end new PC. However in a stronger PC with newer 'Core i7' CPU it can even have the same score as Stockfish 8 (which most of the time is in top of tables).

Improve this question
2
  • How many games did you have?
    – SmallChess
    Commented Oct 11, 2017 at 22:14
  • 1
    Most of the time 4 games for each engine vs. All other engines. Like if I have 10 engines it sums up to 180 games. With time control of 2m+2s
    – Farhad
    Commented Oct 11, 2017 at 23:16

2 Answers 2

Reset to default
2

How many games did you play? Unless your tournament had enough games, your conclusion wasn't statistically significant.

It's common misconception that Stockfish would play better chess than weaker engines in a short computer chess tournament. This is wrong and wrong. Stockfish's higher ELO means it's expected to play better chess. Anything could happen in a short match or tournament.

Magnus Carlsen has the highest FIDE ELO, but that doesn't mean he wins everything. He'd lost to Bu in the 2017 World Cup in a 2-games mini-match, but do you really think Bu is a stronger chess player than the World Champion?

Improve this answer
2
  • 1
    I always have 40 to 1000 games depending on number of engine. In best case there is 2 games with white and 2 games with black. You are completely right about GMs and normal players. For chess engines they almost choose the same opening against each other, which makes the results so consistent. To avoid that I make them to start with certain openings and even in that case again they have same moves at least for most of the game. For example rybka vs. boot looses with black and has a draw with white all the time or at least 90% of the games even I make them to have more and more games.
    – Farhad
    Commented Oct 11, 2017 at 23:24
  • 1
    However as I explained it already, when I change the computers which have huge CPU clock difference with same architecture the results start to vary. I don't know is it something related to programming or compilation of programs under windows or etc...
    – Farhad
    Commented Oct 11, 2017 at 23:28
0

Briefly looking at the source code on https://github.com/booot76/Booot-chess-engine, this engine uses the walltime a lot. Lines like

If (game.time<>game.rezerv) and ((MilliSecondsBetween(game.StartTime,now))>(0.7*game.time)) then break;

suggests the walltime is not just tracked for verbose output but to make decisions. Presumably, this is to make the engine a good opponent for human players but makes its behaviour depend on CPU speed.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.