3

Question: Title question, basically: do engines have a larger advantage over a human at long or short time controls? Have any studies been done on this?

Context for the question: On a reddit thread yesterday someone pointed out that Andrew Tang beat Leela in an ultrabullet game around four years ago. I was astounded by this -- I would not have thought this possible. (Leela was rated around 3250 on Lichess at the time, so she had been trained quite a bit there) I mentioned this to a buddy of mine, and his contention was that humans will perform better against engines at shorter time control. I thought that was flat out wrong, but set out to prove that point.

Unfortunately, I was not able to find much in the way of hard data. This best I could find was this experiment done by the folks (guy?) who runs FGRL that compares search depth against time. It seems that doubling the amount of thinking time approximately increases the search-depth by 2-ply. I have no idea how search-depth corresponds to playing strength -- the best I could find was this paper from 2013 that suggests that an increase in search depth of 2-ply is an increase in playing strength of approximately 120 rating points.

So, my best guess at this point is that doubling the amount of thinking time increases the engines rating by around 120 rating points. When I made the argument that engines have a bigger advantage at shorter time controls, I did not expect a 2-ply increase in search depth to increase the playing strength by 120 rating points, I expected it to be considerably lower.

Grandmasters also play exceptionally well at short time controls. How much do they gain by doubling the time. Is it 120 rating points? I don't know. Do you? This is what made me doubt my initial conclusion, and prompted this question.

Is the 120 rating point increase in computer playing strength by doubling its thinking time a reasonable conclusion?

Improve this question
8
  • 2
    Hey @Dargscisyhp , I don't exactly now why it was closed, but I think your question is either not really clear or may be perceived as trolling. It should be quite obvious, that engines have a far greater advantage at short time controls over humans.
    – Hauptideal
    Feb 3, 2022 at 17:19
  • 1
    I believe that the machine specs which the engine is running on is a factor.
    – hb20007
    Feb 4, 2022 at 10:46
  • @Hauptideal It isn't obvious. Are you ignoring the counterexample the op gave in his first lines of Andrew Tang winning a 15s+0 game against Leela? Consensus is it's inconceivable for any human to win a classical game against Leela, even if they played thousands.
    – Mobeus Zoom
    Feb 6, 2022 at 21:07
  • @Dargiscisyhp I would say the question can do with some limiting of scope to the key issue. You also sound like you would be happy with an estimate of how either human or computer strength individually vary with time-control, but on SE this should not be wrapped into one question with asking how large the differential is.
    – Mobeus Zoom
    Feb 6, 2022 at 21:23
  • 1
    TLDR (thanks to oscarsmith for pointing it out) seems to be that Leela was an infant then and not a 'real engine'. At a 3250 rating on Lichess it should be substantially weaker than even Rybka (which might be 3250+ in FIDE classical on a human scale, and almost certainly better at blitz than slow controls), let alone any contemporary engine. It received opportunities for play solely on the basis of being an imitation effort of AlphaZero. We can guess Leela has absorbed thousands of rating points since the match with Andrew Tang.
    – Mobeus Zoom
    Feb 7, 2022 at 22:08

2 Answers 2

Reset to default
2

Historically, the answer is "no". Computers beat humans at blitz well before they beat humans at classical chess (example).

Humans seldom play serious matches with a time handicap, so I'm not aware if there are statistics on how much elo a player gains by having 120 more minutes. With computers, the allocated time is less important than the depth it searches to - the difference between letting a computer think for 2 days vs. 1 day per move is much smaller than letting a computer think for 10 seconds vs. 5 seconds per move.

If you look at depth, then the elo gain with increasing search depth is tremendous. It depends on the engine, but an order of magnitude estimate for going one move deeper is a couple hundred elo.

Improve this answer
2
  • 1
    I think the historical answer is probably wrong for modern engines. The Deep Blue era engines were missing most of the pruning heuristics modern engines have. Both SF and LC0 have effective branching factors <2 compared to about 15-30 for simpler engines. This means that newer engines have significantly better scaling
    – Oscar Smith
    Feb 7, 2022 at 16:32
  • 1
    @OscarSmith good point. Unfortunately we probably don't have better, more recent data =/
    – Allure
    Feb 9, 2022 at 4:34
1

This is kind of hard to answer because of how much better computers are than humans at all time controls, but according to https://chesspro-ru.translate.goog/guestnew/looknullmessage/?themeid=54&id=15&page=24&_x_tr_sl=ru&_x_tr_tl=en&_x_tr_hl=en-US&_x_tr_pto=wapp, there was a 100 game match between SF11 and Fruit, where SF had 8 seconds per game + .04 seconds per move compared to 140 minutes per game + 40 seconds per move for fruit. With these settings, SF was about 150 elo stronger than fruit, which is roughly super-GM strength. Since then, SF has gained approximately https://github.com/official-stockfish/Stockfish/discussions/3628#discussioncomment-1047728, so SF at 1/4th second per game should be roughly the same strength as Magnus at a few hours per move.

Improve this answer
6
  • While interesting, the question is not 'how much time odds do computers need to give humans to be equally strong' but 'do computers have more or less of a disadvantage, assuming equal time on both sides, at fast vs slow controls' (taking as given that computers will dominate any time-control, but how big is the Elo gap?).
    – Mobeus Zoom
    Feb 6, 2022 at 21:05
  • The problem is that at any fixed time control, it's pretty hard to even measure the elo difference between humans and computers.
    – Oscar Smith
    Feb 7, 2022 at 2:05
  • 1
    I agree, which is why I haven't provided an answer. I'm only pointing out you haven't made an attempt to answer the actual question of this thread, which remains extremely interesting. However you slice the maths, there is a contradiction between two facts: (1) short time-controls are viewed as tactics-dominated and computers were much more superior at them much more early than at slow time-controls; (2) these days top engines appear to be multiple clear iterations of 100% win-rate stronger than top humans at slow controls*. How then did Andrew Tang win a 15s+0 game vs Leela?
    – Mobeus Zoom
    Feb 7, 2022 at 21:54
  • *Legacy backtesting proves this - it's been show for example that Rybka 3 (which on a human scale should have been 3200-3300 by 2008) < Houdini 2 which lost a legacy match (if I remember right - can look for reference on Talkchess) against Stockfish 8 without a single draw in over 120 games. Stockfish 10 struggles to get draws now against Stockfish 14.
    – Mobeus Zoom
    Feb 7, 2022 at 21:57
  • 1
    I had an inkling the reason was something like that. (IIRC a slightly later prototype version of Leela scored only 0-2 draws in the bottom division of TCEC. This was just an early experiment in imitating AlphaZero then, not the serious competitor engine it is now. ) Then that forms an important part of an answer I should think
    – Mobeus Zoom
    Feb 7, 2022 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.