8

For the past couple of weeks, I've been making a chess engine, and right now I'm working on creating its search and evaluation phases.

To start off with, I made the evaluation phase a simple count of material so I could focus on the search phase, and then I would come back to create a more nuanced evaluation function. So right now, I have negamax, with alpha-beta pruning, iterative deepening, and quiescence searching implemented, which I'm pretty happy about.

But the problem is sometimes my engine still decides to willing blunder away pieces. Here's an example from a recent game, where the computer is black and it's black to move. For some reason, my engine decided that Ng4xf2 was the best move here. And then once it lost the knight for a pawn, it just went on as if it had made an equal trade.

Can this kind of blundering by a chess engine be explained away by a very naive evaluation phase, like my engine currently has? If it can be then, then of course the solution would be to create a much stronger evaluation function, which I'll do. But if it can't - and I must admit I don't see how it could - then I'm afraid there's a problem with the search phase, which would be a disappointing, but not impossible, setback.

I found this question, which seems to give an answer, but the reason why I still don't see how the evaluation function could be the culprit is that at the very least, in my mind, an evaluation phase based purely on material gain shouldn't make basic blunders where it loses material literally on the next move (especially with quiescence searching!)

Can anyone with experience in this area of AI programming offer any insight here? Also, let know if you need to see any code and I'll be more than happy to provide it.

Improve this question
3
  • 1
    I'm not 100% sure, but suspect that an eval function only based on material can be tricky. Just to get sure you're on the safe side, I'd think of adding piece square tables. They're easy to implement, and suddenly your engine will play much better.
    – emdio
    Jun 29, 2021 at 8:59
  • Hmm maybe so @emdio, but I'll double-check my search code first I suppose.
    – Christian Dean
    Jun 29, 2021 at 14:47
  • "And then once it lost the knight for a pawn, it just went on as if it had made an equal trade." So it cannot even count material correctly some times. Hence it is a clean bug.
    – user27863
    Jun 29, 2021 at 22:14

2 Answers 2

Reset to default
7

It's always a question at what depth you stop. Eg.g, with 5 plys, your engine might be happy with

  1. ... Nxf2 one pawn ahead
  2. Bxf2 Bxf2 still one pawn ahead
  3. Kxf2 Qxf3+ still one pawn ahead and even check!

and not see 4. Kxf3. Something like this chould be prevented e.g. by selectively expanding until no capture / no check is made. Even then, the evaluation might stop with the last move leaving your queen attacked.

Improve this answer
3
  • Quiescence search should address that and OP claimed to have that.
    – user27863
    Jun 29, 2021 at 22:11
  • Right. @adrasthea, so I don't really think search depth is the issue. I recently scrapped what I had and just decided to write things from scratch, and I've gotten good results. Currently the engine is playing fairly solid chess, and doesn't make stupid blunders anymore (even more exciting it seems to know when to make tactical sacrifices now) and so I think the bug came from how I used my transposition table, so I'm going to be adding that feature back in very carefully.
    – Christian Dean
    Jun 29, 2021 at 23:12
  • 1
    @ChristianDean: If you have not already done so, you may wish to create a suite of unit tests which run your evaluation function (and just the evaluation function) against a series of fixed positions. That way, if your evaluation function suddenly changes its opinion of one of those positions, you will be able to look at the change and judge for yourself whether it is intentional.
    – Kevin
    Jun 30, 2021 at 0:16
13

This exact bot has already been created. On lichess, it is simpleEval. You could potentially use it to debug a lot of your issues by seeing what this engine does against you in a given position.

Having played this bot several times, no that move Nxf2 would probably be incorrect. This seems like a bug on your end. Assuming you have enough depth in the bot's calculation, that type of a blunder would be the last possible way it blunders. Specifically, your bot should make incredibly terrible moves from a positional standpoint as long as it doesn't see any tactics that win material in the upcoming moves. I've seen "+10" positions where technically the bot is up material. This is much more likely to be the bot's response to a specific situation than sacrificing anything unless it finds a forced win of material back in the near term.

Sorry that you might have to revisit some code you thought was already working.

Improve this answer
5
  • 1
    Hmm ok, no problem. I'll look into some of what you said. Thanks!
    – Christian Dean
    Jun 28, 2021 at 22:55
  • 1
    @NoseKnowsAll I'm not convinced that simpleEval is the same thing at all. The engine is way too advanced to be looking at the material count 3 moves ahead and nothing else. My guess is the evaluation itself uses material count only (none of the other factors in Stockfish's evaluation) but is otherwise (pruning etc) full-fledged Stockfish. A real engine which looks (even 100) moves ahead purely materially should not, for example, have by default anything to avoid even one-move checkmate.
    – Mobeus Zoom
    Jun 29, 2021 at 19:00
  • 5
    @MobeusZoom I would expect an engine that looks only at material to evaluate the King as some absurdly large amount of material, i.e. +100. That would allow it to find checkmate without violating its "material count only" rule. (The alternative of evaluating the King as worth 0 seems rather silly to me.)
    – Brilliand
    Jun 29, 2021 at 19:39
  • @Brilliand That isn't actually how engines like Stockfish treat king safety, however; they have separate components for it. And simpleEval seems to me like a pretty involved fork of Stockfish to just be looking at material count n moves deep. It's been identified here that simpleEval has some odd behaviour: chess.stackexchange.com/questions/34933/…. (With a claim there may be a bug: there is no bug - simpleEval is a relatively strong agent and the reason is likely inclusion of more than it says on the tin.)
    – Mobeus Zoom
    Jun 29, 2021 at 21:28
  • @Brilliand Well that's how some engines operate, particularly ones that use king-capture to filter out pseudo-legal moves such as Sunfish (written in Python). Other engines however, including mine, simply detect when a mate has occurred in the minimax function (1. The king is in check, and 2. has no legal moves) and return a relatively infinite value, so that the engine is always looking for mate over say, capturing an enemy queen.
    – Christian Dean
    Jun 29, 2021 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.