Guidance on a new idea for an engine design?

Question

for my A-level computing project I was going to do an investigation into a type of engine design which incorporates Neural Networks that I haven't seen elsewhere; as part of this project I need to base my specification around the recipients of the outcomes for investigative projects (hopefully you guys).

My current design idea is this:

• For a given position, A neural network will evaluate the position and spit out two values: how good it is for white/black, and a value representing how much will be gained from analysing this position further.

• All of this information (along with the depth) could then be combined to determine whether the engine should analyse to a layer further down in the decision tree or return the first value (how good the position is for white) so that it could then be used as a "leaf" in the decision tree that could then be analysed by mini-max (along with suitable pruning algorithms) to give an overall analysis of a given move.

• To train the neural network: I was going to play two networks against each other and analyse their evaluations of the certain moments in the game post-game to tell them (using back propagation) where they should look deeper or evaluate a certain position as better/worse based on who won the match. I think this method is similar to policy gradients. Another approach I was looking into was a genetic algorithm however I was sceptical of whether I have nearly enough processing power available to me seen as I'm doing this all on my laptop :(

I would be very grateful if you could answer these questions for me:

1. Have you seen any similar projects like this before?

2. What are any pitfalls that you could see with this method?

3. would you consider using a similar method in your engines? Why/ why not?

4. how would you alter this investigative project so that it could benefit you more personally?

5. Would you use the genetic algorithm approach?

Thanks a lot in advance for any replies!

Answer 1

I see three ways of tackling this:

• You can add a confidence value to you position evaluation function. And then use this to improve your tree search. This sounds amenable to mathematical analysis, you'd have to adapt min-max search to handle a trade-off between confidence and evaluation.
• You can output a distribution of evaluations instead of a single numerical value. And again use this information to improve tree search. That's similar to the one above.
• You directly output a "how much compute should be spent on this position"-value.

The second thing has been done in some of Deepmind's video game reinforcement learning projects. So you might want to take a look at that.

I haven't seen anything like this for turn-based board games. It certainly sounds like a sensible idea.

If you want to do the third thing you need to figure out how to capture and backpropagate "how much compute should be spent on this position". That doesn't sound trivial and is not guaranteed to work, but it would be pretty neat and it would tilt your work more towards Deep Learning kinda stuff.

No, you should not use genetic algorithms. It's almost always better to optimise your function non-randomly. In your case backpropagation is highly likely to be much more efficient.

One pitfall is that even backpropagation might not be efficient enough. If you train your networks from scratch by self-play on your laptop, it is possible that the learning never really takes off. Self-play doesn't provide enough information for efficient learning, which is why AlphaZero and co use humongous amounts of compute.

So it might be more sensible to plan your training in such a way that it can use existing games. Then evaluation and confidence can be learned without doing any tree-search yourself.