Though I am not able to test it myself, I am confident of the following conclusion:
An ensemble of engines should be able to beat the strongest individual engine
Here are my key assumptions:
- Given typical time controls used for benchmarks, the time 'lost' by having a thin pre-evaluator before the engines would be negligible. As such we can say that the engines in the ensemble will effectively have the same sum of resources as the top engine.
- Suppose the ensemble would want to run two engines at the same time, it would be able to give each of them half the resources.
- Suppose the ensemble would want to run engines consecutively, it would be able to give each engine the full resources. The ramp-up time is assumed to be small enough if this is only done a few times during a game (but prohibitive if this would be done every move).
Also note that the ensemble would be able to contain a copy of the strongest engine itself. At time of writing the engines following the strongest engine are not far behind so this is not a critical point, but otherwise it would become a question of 'how much rating would an ensemble gain over its strongest member' which is significantly harder.
Scenario 1: Engines are stronger in different phases of the game
An easy 'win' here would be if one engine had a stronger (interaction with) an opening book, and another with an engame tablebase. However, even assuming all engines can use the best resources in these area with equal efficiency, then it is still commonly said that certain engines are 'strong in the opening' or 'good at endgames'.
Let me make an additional assumption here:
- I assume that engines that are good at a certain phase, do not have a strong dependency on other phases to realize this.
So, the engine that would be good at endgames would not only be good at endgames it reached itself, but also at endgames that were reached by another engine.
Most straightforward solution: Identify a phase of the game, and let that be played by the engine which is strongest at it
In case we just distinguish between opening and endgame it would be trivial to define a wrapper for this, and there would be about 1 switch per game. Of course this could be extended if you have an engine which is 'great at pawn endgames' or 'very good at positional games' but at that point it would already become harder to identify which engine to choose without using significant resources.
Scenario 2: Engines can find critical continuations for critical moves
This scenario is what I was originally curious about. However, based on my assumptions the most straightforward way to get the opinion of several engines would be to let them run in parallel. Suppose we just use an ensemble of two engines given half the resources, then they would both be a bit weaker, let us make another assumption based on some references.
- In a typical setting strength scales logarithmic to the available resources, and halving them reduces engine strength by 50-100 elo
Now that is significant, honestly it may be too much. Suppose we just put two engines against eachother with a 70 elo difference, the expected value would be about 60-40. That is a lot to make up for, but though I was unable to find any data on this, it may still be possible. Basically this just needs to result in 1 brilliant move or 1 averted blunder to swing an entire game. It would not be trivial to decide which engine to listen to on each move, but as engines typically can output some basic statistics (like their evaluation of all possible moves, and how deep they checked each one) it would probably not be too hard to make a reasonable and still lightweight decision.
Possible alternate solution: Run engines in parallel and pick the best move each time.
Again this could be extended, a simple way would be to have 3 engines, and pick the move that 2 of them give, but I am not sure if splitting the resources even thinner would be worth it. Another interesting idea might be to give the strongest member of the ensemble the most resources, and have it being sanity checked by the member which is best at this. In this case the main engine might only lose 10 elo points due to reduced resources so 'making up for it' could be a lot easier. But again, it will remain tricky to select the right move.
A final thought would be if engines use CPU and GPU, then perhaps the strenght of individual engines may not scale down too much in each of these resources, so a mix where engine one gets 80cpu+20gpu and engine2 gets 20cpu+80gpu may leave the individual engines nearly as strong as when they had full resourses.
Especially when engines can be run one at a time, it should be able to get better performance, though there must also be ways to get better outcomes by running multiple in parallel. However, this is not trivial.
Taking the ensemble concept really to the next level would likely be possible with small changes in the engines, for example not only providing the expected value of a score, but also how confident they feel about their evaluation.