7

I've built a chess engine in Python which I've been testing against StockFish through the use of the Python-Chess package.

Testing against StockFish seems/is problematic, and as a result I'm looking for suggestions of engines to test against.

There are two major issues with testing against StockFish (the first much more significant than the next)

  1. StockFish is partially used in generating the training data for the engine's evaluation function.
  2. StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

The engines I'm hoping to play against need to have no serious bugs or problems, and ideally would play at levels similar or a bit below a human master, without spending too much time running them. UCI engines are prefered so that my engine can easily communicate with them through the Python-Chess API (note that this doesn't mean my engine supports UCI).

If it helps anyone to have some context, my engine is Batch First.

Improve this question

8 Answers 8

Reset to default
8

Since your engine supports UCI, I would suggest checking out lichess.org. You can make a BOT account there and test against several other BOTs and people as well! Also, you can get your BOT rated in different time formats!!

I must warn you that most of the BOTs on lichess are some version of StockFish or other strong engines but you can find ample other BOTs.

Among strong BOTs (apart from Stockfish ones), I would suggest checking out:

  1. LeelaChess: A neural net based BOT.
  2. ASMFish

There are many more BOTs on lichess, you can view them here on lichess BOT team.

You can also find many not so strong self-implemented engines that you might like testing against, for example this.

Improve this answer
5
  • That seems particularly enticing! Competing against a top-tier engine restricted to very little time is more appealing for obvious reasons, but I have concerns that the elo/give_time curve might not behave as I expect, and cause weird/undesirable results. When restricted to very little time, would top-tier engines play similarly to the less strong engines?
    – Sam Ragusa
    Commented Jul 7, 2018 at 2:30
  • But both the BOTs will restricted to little time, right? So I guess the optimal BOT would search more depth than the weaker BOT doesn't matter the time. Regarding performance given low time, I would suggest checking out this video, where a human defeat a strong engine in hyperbullet match streamerclips.com/twitch/penguingm1/RefinedAverageLaptopRedCoat.
    – Laschet Jain
    Commented Jul 7, 2018 at 7:03
  • Damn, I didn't knew you could setup a bot account there!
    – Fernando
    Commented Aug 16, 2018 at 18:35
  • @Fernando This feature was added around 2 months ago.
    – Laschet Jain
    Commented Aug 16, 2018 at 18:36
  • @LaschetJain I'm checking it right now, thank you!
    – Fernando
    Commented Aug 16, 2018 at 18:37
3

http://computer-chess.org/doku.php?id=computer_chess:wiki:lists:chess_engine_list

  • Arasan
  • GreKo
  • Crafty
  • Rodent
  • Toga etc
Improve this answer
2

To complement the list given by @SmallChess, you can find at https://www.computerchess.org.uk/ccrl/4040/ an ELO ranking of many chess engines.

In the 2300-2600 range ("similar or a bit below a human master"), and amongst open-source engines, you have :

  • K2 0.87 (2580 ELO)
  • Jazz 840 64-bit 4CPU (2552 ELO)
  • Maverick 1.5 64-bit (2522 ELO)
  • Winter 0.1.1 64-bit (2492 ELO)
  • Lozza 1.18 64-bit (2452 ELO)
  • Topple 0.1.1 64-bit (2360 ELO)

I haven't tested these engines myself, but they seem to fit your criterias.

Update : If you're looking more for engines in the 2000-2300 range, then these engines might suit you better :

  • Absolute Zero 2.4.7.2 64-bit (ELO 2294)
  • CT800 1.30 64-bit (ELO 2253)
  • Zevra 1.8.4 r650 64-bit (ELO 2236)
  • ZCT 0.3.2451 (ELO 2210)
  • Tunguska 1.0 64-bit (ELO 2133)
  • Cinnamon 2.0 64-bit (ELO 2054)
Improve this answer
3
  • Similar or a bit below human master would be 2000-2200 USCF or up to 2200-2300 FIDE.
    – Herb
    Commented Jul 6, 2018 at 17:22
  • 1
    @HerbWolfe I interpreted "master" as GM level (2500-2600), and "a bit below" as FM to IM level (2300-2500). The original question is a little unclear on what "master" level mean exactly (I would have commented about it if I could). I'll update my answer anyway, but it would be best if the author could confirm what he meant exactly.
    – Umlin
    Commented Jul 6, 2018 at 17:43
  • 1
    I left the term 'master' purposely vague, because my familiarity with ELOs and the level of play they represents is lacking. I used just below a master level because my main point of reference is a friend who's rank is right below master (though I'm unsure what his actual ELO is). Either way, your summarized list of engine's is appreciated!
    – Sam Ragusa
    Commented Jul 7, 2018 at 2:37
1

Try https://github.com/fernandotenorio/Tunguska And let me know if your Python engine beat mine! =D

Improve this answer
1

StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

You can use go depth x or go nodes y to get Stockfish to almost arbitrary strength levels.

Improve this answer
0

StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

Aside from the mentioned UCI depth option, Stockfish also supports different playing strengths. See Is there a list of approximate Elo ratings for each Stockfish level? for discussion on Elo.

Improve this answer
0

Here are a two suggestions:

  1. Micromax
  2. Sunfish
Improve this answer
-1

Here is a list of computer chess engine rankings. Elo's are provided.

http://www.computerchess.org.uk/ccrl/4040/

Improve this answer
2
  • 4
    you post an URL that was already posted like 15 hours ago, what's the point?
    – lenik
    Commented Jul 8, 2018 at 7:44
  • Guess I overlooked the original post. People can do with the post as they like(ie reuse, remove because it is duplicated etc.)
    – hangejj
    Commented Jul 8, 2018 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.