I'm confused by UCI Pondering and Time Control

Question

I'm trying to talk to a UCI engine (right now Stockfish, but more generally to all UCI engines) to write my own computer-vs-computer program (I could use Fritz, but I really want to learn more about UCI and engines in general, so I'm trying to do it on the nuts and bolts level).

I've gotten to the point where I can do:

position fen r1b1kb1r/ppp2ppp/3p1q1n/4p3/3nP3/1PN2N2/PBPPQPPP/R3KB1R w KQkq - 0 7
go wtime 500000 btime 500000

And read

info nodes 811977 time 4620
bestmove f3d4 ponder e5d4

So I know my engine decided, after it's processing time of 4620 milliseconds, to move f3d4, and it expects the other player to play e5d4.

First question: do I immediately send go ponder to let the engine think? Or do I need to use other commands first?

The player plays a move and, hot dog, it's what the engine said it was going to ponder.

Second question: do I just send it a ponderhit or do I need to postition fen --new position-- first then send ponderhit?

I'd personally expect the engine to update it's internal state on a ponderhit, but it looks like the engine is supposed to rely on the GUI to handle all board state for the game, is this correct?

All the examples I've seen just do a single analysis, or run a single move then end, I don't know how to continue the game!

Third question: When I send a go command I send along with it the wtime and btime left on my game's clock. Does the engine actually use this data to decide how long to think on each move, or does it expect the GUI to tell it how to manage it's own time? It looks like the engine handles this thinking on it's own, but as I've never actually gotten to the NEXT move, I'm not sure if it's actually doing time management.

I know these are horribly newbish questions, but all the reading up I've found on the UCI engine seems to just talk about one move, or analyzing a move, and I haven't found any examples of multiple moves in sequence, so I'm not quite sure how to play THROUGH a game on the UCI command line.

Answer 1

1. First the GUI sends the position string with the ponder move as the last move in this string. Then the GUI sends "go ponder". This makes little sense to me, but UCI is very much a kludge. Note that the engine can actually ponder whatever move(s) it wants, but UCI tried to make it simple for basic implementations. From the protocol

   • ponder

     start searching in pondering mode. Do not exit the search in ponder mode, even if it's mate! This means that the last move sent in in the position string is the ponder move. The engine can do what it wants to do, but after a "ponderhit" command it should execute the suggested move to ponder on. This means that the ponder move sent by the GUI can be interpreted as a recommendation about which move to ponder. However, if the engine decides to ponder on a different move, it should not display any mainlines as they are likely to be misinterpreted by the GUI because the GUI expects the engine to ponder on the suggested move.

2. I think the GUI simply sends "ponderhit" when it is applicable.

The main reason not to resend the position string, but to only send "ponderhit" is to ensure that the engine continues its current search in the pondered position, rather than starting a new one.

• ponderhit

    the user has played the expected move. This will be sent if the engine was
    told to ponder on the same move the user has played. The engine should
    continue searching but switch from pondering to normal search.
  

In general, UCI tries to be "stateless", which is really it's big error IMO, as it makes everything natty. Pondering is a prime example of when the engine can really do whatever it wants (even the protocol says explicitly). The "ponderhit" command's action is determined by the "position fen ... moves" string that was already sent before the pondering began. So in that sense the GUI still sends the board state (this is completely awkward in my opinion, particularly if you want to ponder more than one move, but that's UCI for you).

3. Engines can/should use their own time management based on "wtime" (and/or btime/winc/binc). Again it's a bit confusing with UCI, as the GUI is supposed to be in somewhat total control (like with opening books), but then has some leeway when allowing the engine to allocate its own time (you can use "go movetime" if you want the GUI to control time management, assuming the engine complies).

You can look at a log of how a GUI communicates with an engine such as Stockfish.

GUI -> engine1: position startpos
GUI -> engine1: go wtime 100000 winc 1000 btime 100000 binc 1000
engine1 -> GUI: bestmove e2e4 ponder e7e6
GUI -> engine1: position startpos moves e2e4 e7e6
GUI -> engine1: go ponder wtime 98123 winc 1000 btime 100000 binc 1000
[user or other engine plays the expected e7e6 move]
GUI -> engine1: ponderhit
[engine keeps thinking]
engine1 -> GUI: bestmove d2d4 ponder d7d5

Or if the pondered move wasn't made, then a new "position" string is sent instead of "ponderhit".

Answer 2

Q1: You shouldn't send ponder back, because you don't know what the other player would make.

Q2: You'll need send back a new position (give a FEN or a move list). You're expected to keep the state of the engine yourself. You can also send ponderhit if the expected ponder move is made, but it's not absolutely necessary.

Q3: Stockfish has an algorithm to decide when to stop searching based on the time you give to it. Yes, you should give the clock time.

UCI is not a formal and well-documented protocol, it was invented by Shredder for their own GUI and engine. When they released the protocol, there was nothing better, soon after everybody used it.

I understand you don't like keeping state yourself, but this is not how UCI works. You can hack by getting a FEN string from Stockfish, make a move, then get a new FEN. This should work but it's very hacky and definitely not how Stockfish is designed.