Does a reinforcement learning style model actually need to be combined with a search algo to produce the best moves?

Question

I have recently been studying up on Machine Learning based chess engines and have begun to develop one of my own. I was wondering, realistically, doesn't the board technically contain all of the data used to predict any number of moves into the future? From what I can tell, models like AlphaGo use a search algorithm to pick which moves it should investigate further in order to make its move.

what if a RL model were to just be trained on the current position, not any other data besides what anyone could see from looking at the board. through training, would the model develop its own method of searching for future moves in order to maximize its reward potential in a game naturally?

I assume that if the model developed its own algo it would be significantly faster and more optimized at returning good moves with a depth level that would make it competent at the game.

Is there something I am missing?

Answer 1

Are you talking about static evaluations? No it’s not sufficient to play a good game. In chess, even a single pawn push might change the evaluation of the position. There are too many possibilities, so no engine will be successful without a search algorithm. What’s the point of a good pawn structure if your opponent can force a checkmate? A good model can guide a search algorithm better but it’s not a replacement.

Answer 2

Pure reinforcement learning with current neural network architectures for chess will, if done well, produce an opponent that is surprisingly strong; I can certainly see such a program beat master level players, especially if they are unaware of whom they are playing against. It will however fall far short of chess programs that incorporate some form of search. Typical machine learning algorithms (here: neural networks) are fundamentally limited by being algorithms that process a given input in a fixed time, i.e. they cannot by themselves take advantage of more time to think. Algorithms that combine good static evaluation with look-ahead, on the other hand, will get a lot better when allowed to do some search. Competitive programs therefore use search.

That said, it is possible to try out in practice just how well a naked neural network can play, for instance by playing LC0 while requiring an instant response at each turn. To illustrate, I have just generated a game of LC0 under such conditions against the hard level of Shredder Chess Online; I would estimate that that opponent is roughly low candidate master level:

[FEN ""]
[Event "Test game"]
[Date "2023.01.08"]
[Round ""]
[White "Player"]
[Black "LC0"]
[Result "1/2 - 1/2"]

1.e4 c5 2.c3 Nf6 3.e5 Nd5 4.Nf3 Nc6 5.d4 cxd4 6.Bc4 Nb6 7.Bb3 d5 8.exd6 
Qxd6 9.O-O Be6 10.Na3 Bxb3 11.Qxb3 e6 12.Nb5 Qd7 13.Nxa7 Rxa7 14.Qxb6 dxc3
15.Be3 Ra6 16.Qb3 cxb2 17.Qxb2 f6 18.Rfd1 Qc8 19.Rd3 Be7 20.Rb3 Na5 21.Rc1
Qd7 22.Rbc3 O-O 23.Rc7 Qd8 24.Qb5 Ba3 25.R1c3 Qd1+ 26.Qf1 Qxf1+ 27.Kxf1 
Bd6 28.Rd7 Rf7 29.Rc8+ Rf8 30.Rc1 Rf7 31.Rc8+ Rf8 32.Rc1 Rf7 33.Rc8+ 1/2-1/2

The Maiabots on lichess are, as far as I know, another example of pure policy network opponents that are publicly available. They do, however, play significantly weaker than the LC0 policy.