How often do Super GMs achieve 100% engine correlation?

Question

There has been speculation that anomalous values of a correlation metric¹ may hint to the use of outside assistance.

Example

Examination of Hans Niemann vs Matthieu Cornette surprises:

Hikaru Nakamura:

this looked like a perfect game

Yosha Iglesias:

one-hundred percent - wow!

---

Context:

• 90% Sébastien Feller (Paris 2010 - known to be cheating)
• 72-75% Correspondence World Champion (pre engine era)
• 72% Bobby Fischer during 20 consecutive winning streak
• 70% Carlsen at his best
• 69% Kasparov at his best
• 62-67% Super GMs
• 57-62% GMs

---

Question

How often do super Grand Masters score 100% in this correlation metric. For example, 1/1,000, 1/1,000,000 1/1,000,000,000 etc.

Notes

• Nakamura mentions:

Fischer had 0 games at 100% in his entire career

• Some methods for calculating the correlation statistic can be made to exclude book openings (and even more impressively book openings at the time of the game).

• It may be sensible to exclude short games (e.g. Junior Speed Chess Champion, Arjun Erigaisi, is said to have 1 game at 100%, however, it was a 10-move game; the shortness of this game is notable).

• In the days following the release of Yosha's video, there has been some confusion around the methods used to attain the alleged 100% engine correlation. Grand Master Ben Finegold dismisses the idea of them entirely:

A lot of the information on the internet which says Hans played 100% in all these games, that's all nonsense. ... None of that is true.

---

¹ Let's Check

Answer 1

It depends on how many people analyze his games. The more different engines are used to compare a player's move, the more likely a match with an engine moves.

There is even evidence that people (e.g. ChessBase user gambit-man) uploaded engine moves of many old and unusual engines to the cloud analysis of Niemann's games in order to make him look even more suspicious.

There has been speculation that anomalous values of a correlation metric1 may hint to the use of outside assistance.

ChessBase documentation states, that the correlation metric is not suitable for cheat detection and can be manipulated, as any user-generated cloud content.

Answer 2

According to this tweet:

I analyzed every classical game of Magnus Carlsen since January 2020 with the famous chessbase tool. Two 100 % games, two other games above 90 %. It is an immense difference between Niemann and MC. Niemann has ten games with 100 % and another 23 games above 90 % in the same time

Carlsen's two 100% games were his 9th round win in the 2022 Tata Steel tournament against Shakhriyar Mamedyarov and his 12th round draw! in the 2021 Tata Steel tournament against Wojtaszek. That's 2 games out of 107 analyzed.

Scoring 100% is harder against better opposition (the more your opponent blunders the easier it is to play the best move and vice versa).

Answer 3

Well, it is pretty easy to calculate. Let's do the exercise for Magnus Carlsen, who has an engine correlation of 70%, which seems reasonable given that similar strong players like Fisher and Kasparov achieved similar values.

So the probability that Carlsen makes 20 out of 20 moves like the engine is 0.7^20 = 1/1253, i.e. about 0.1%. However, games are typically longer. If we go for 30 out of 30 moves, the probability is 0.7^30 = 1/44366, i.e. about 0.002%.

So, if one longer 100% correlation game (30 out of 30 moves excluding opening prep) occurs in 40 000 games of Carlsen, that would be totally normal.

Now let's look at the probability that Carlsen has K longer 100% correlation games in a total of N games. Here, we need the binomial distribution. The probability is (1/44366)^K * (1-1/44366)^(N-K) * (N choose K) Let's consider an example: Let's say Carlsen plays N=1000 games and gets 5 times 100% correlation. The probability for that to happen is: (1/44366)^5 * (1-1/44366)^995 * 1000999998997996/2/3/4/5 = 1 in 21 trillions.

Now, many of Niemann's 100% games were not that long, which is in his favor. So let's go with 20 out of 20 moves like the engine and 6 out of 500 games having 100% correlation. We take the engine correlation of strong super GMs instead of Carlsen's, i.e. 67%. The probability for that to happen is (0.67^20)^6 * (1-0.67^20)^494 * 500499498497496/495/2/3/4/5/6 = 1 in 42 million.

The author teaches graduate courses on probability and statistical inference.