16

Computer chess engines have gotten better since Deep Blue beat Kasparov in 1997.

Did the algorithms get better, or were the improvements mostly due to the same algorithms running faster thanks to faster hardware, etc.?

If the former, are these algorithmic improvements public?

And if so, what were the improvements? Where can I read about them?

8
+100

Maybe you can take a look at TalkChess, a forum dedicated to computer chess. I found a recent thread that might be interesting for you: Progress in 30 years by four intervals of 7-8 years

A couple of matches between (former) top engines are played on the same hardware. The test suggests that in the recent years (2002-2017), the gain is mainly made by software improvements. In the test, Stockfish (2017) scored an impressive 94/100 against RobboLito (2009), while RobboLito, on its turn, crushed Shredder (2002) with 92/100.

An important remark: as parallel computing is not implemented in the older engines, the test was performed on a single core. As a result, the hardware gain by parallel machines is not measured. On the other hand, you could argue that parallel computing is also a software gain: it is not easy to design and implement an efficient and well-scaling parallelization for the search algorithm.

The Stockfish engine is open source, so the algorithmic improvements are public. A lot of documentation can be found on https://chessprogramming.wikispaces.com

  • This answers his assertion. Try to answer the question next time. – Fred Knight Aug 29 '17 at 13:48
  • 1
    Well, I believe I did answer the question: the gain is mainly made by algorithm improvements. Furthermore, I showed data that supports this claim (see link) and pointed out a possible shortcoming (no parallelization measured). – Maxwell86 Aug 29 '17 at 14:34
3

I can't speak for the algorithm used for Deep Blue, but I'm going to try and explain the improvements in chess programming. Speed is the greatest improvement. Deep Blue used multi-processor dedicated computers, so a comparison isn't really possible.

https://chessprogramming.wikispaces.com/ is a great source, but it's hard to navigate.

There are 3 main functions that are tweaked to improve a chess engine are the evaluation, move generation, and search functions.

Evaluation is the hardest to program, as there are many exceptions to the rules. With hard drive space getting cheaper, the eval function allows for more exceptions to be evaluated.

Move generation, along with making and unmaking a move, consumes a lot of memory because it has to be preformed so many times. The most common generation functions are mailbox, bitboard, 0x88, 8x8, extended boards (10x10, 10x12), and a predetermined move array/table (*I use an indexed move table). Current opinion is that bitboards are the faster, and using magic bitboards speed this by up to 30%. Dr. Robert Hyatt, professor and creator of cratfy chess engine, claims no significant speed increase.

The early search function was the primitive min-max functions. Basically were you try to maximize the score of the side to move and minimize the opponent's score. Alpha-Beta was the first improvement. They reduced the number of moves being searched by transposition table, cut-off values, aspiration windows, and history heuristics. These are depth-first searches. There is also the internal iterative deepening search which tries to search the "best" move(s) the deepest hoping that searching other moves will prove to be fruitless.

NOTE: My index table. GNUChess and Jester both use an index array to generate their moves. They initialize the engine by filling array with possible moves. The take the six pieces and compute the legal moves that are available from each square. So each piece had a [64][8] array. I took this idea and compressed it to two indexes and a table. The table holds a value which tells if the 16 moves are possible, one index holds the offset of the move, and the other holds the mask.

offset[] = {-8, -1, 1, 8, -9, -7, 7, 9, -17, -15, -10, -6, 6, 10, 15, 17};

mask[] = {1, 2, 4, 8, 16, 32, 64, 128, 256, ...};

Then the generation of a sliding move is as easy as looking up the validity of it's mask in it's allowable offsets against the move table.

  • 7
    I try not to reply to answers, but this is just.... Alpha-beta and bitboards were invented LONG before Deep Blue. I'm also pretty sure that board eval doesn't access HD in any sane engine (the latency is HUGE). Fourthly, I'm very skeptical that RAM size makes any real difference in your normal alpha-beta search implementation. – MaxB Aug 17 '17 at 4:21
  • Could you perhaps add some hyperlinks to some of the concepts you are discussing? As someone who is interested in the concept, but unfamiliar with the terminology, it's hard to follow because I don't know what a bitboard is or the Crafty Chess engine. – Thunderforge Aug 17 '17 at 4:23
  • I thought that I was clear in that I wasn't comparing to Deep Blue, but I was giving a brief history. The hard drive I was referring to is the program itself. Every time that a new eval concept in included into a chess engine, more code, and therefore more HD space is required. – Fred Knight Aug 17 '17 at 7:55
  • @Thunderforge, the one link I gave explains every aspect you could ever want dealing with chess programming, however I admit that it is hard to navigate. I learned by reading other's source codes, but the one which is most heavily commented is Dr. Hyatt's Crafty engine. I choose not to be too comprehensive due to space limitations and the differences across platforms and compilers. If, after reading the wiki chess page, you are still confused, ask the question and I'm sure many will provide a better answer. – Fred Knight Aug 17 '17 at 8:04
  • 1
    Every time that a new eval concept in included into a chess engine, more code, and therefore more HD space is required. Board eval functions are typically designed to fit in CPU cache. CPU cache << RAM << HD. HD size makes no difference. – MaxB Aug 17 '17 at 16:19
2

Did the algorithms get better?

Obviously, yes a little bit.

or were the improvements mostly due to the same algorithms running faster thanks to faster hardware and software?

Minor nit: If the algorithms got better then that is the software getting better so there is no "or".

Moore's Law tells us that processor speed will double roughly every 18 months. That means it has doubled roughly 13 times in 20 years. That makes modern processors somewhere in the region of 8,000 times faster. So, far and away the biggest improvement in engine performance is due to faster hardware.

If the former, are these algorithmic improvements public?

And if so, what were the improvements? Where can I read about them?

Well, it wasn't the former, it was the latter. Nevertheless the improvements are mostly open source and freely visible by downloading the sources for engines like Stockfish. Perhaps also worth giving the general Stockfish download link since the specific source code link will likely expire when version 9 comes out.

  • 2
    That means it has doubled roughly 13 times in 20 years. I think you're misquoting Moore's Law. It says nothing about processor speed. In fact, it hasn't doubled in a while. – MaxB Aug 17 '17 at 15:53
  • hardware and software I meant software as in the implementation of the algorithm (ASM vs C++), but I can see how it's confusing. Fixed. – MaxB Aug 17 '17 at 15:55
  • 1
    He Moore's law is correct, except that he include the phrase "in the next decade." This would have been in 1975, and he was correct. – Fred Knight Aug 17 '17 at 20:44
  • -1 because answer is incorrect - on the same hardware, current engines still crush formerly-top engines. – Allure Dec 13 '18 at 3:20
0

It's all about algorithms.

Taking on a human chess player took one of the most powerful computers in the world at the time. This brute force computing approach allowed Deep Blue to look around six to eight moves ahead. In a closely-fought contest, the machine eventually defeated Kasparov by 3 1/2 games to 2 1/2.

Six years later, Kasparov was involved in another contest of man versus machine. This time he played against Deep Blue’s successor, Deep Junior. The result was a drawn series at three games all. The biggest difference was that Deep Junior ran on a machine with about one per cent of the computing power of Deep Blue. Chess-playing algorithms had improved to the point of achieving virtually the same result with a hundred times less computing power.

  • 4
    Welcome to Chess! You wrote the main portion of your answer as if it was a quote; could you please provide a source? – Glorfindel Nov 15 '18 at 11:29
0

Disclaimer: not an expert.

The algorithms got better, and today's best engines running on 1995 (remember Deep Blue was 1999) hardware will handily beat Kasparov. As I understand it, there are two aspects of algorithms:

Search. If for example I take your queen with my queen, a human opponent will automatically look first at recapturing. For a computer however, it will evaluate every possible response to QxQ. Almost all of the time, this is wasted processing power. A good search algorithm cuts down all these "branches" since they're irrelevant anyway.

The standard search algorithm is alpha-beta pruning, and it was used in the earliest chess computers. I don't know if Deep Blue used alpha-beta pruning, but modern engines don't. As a result their searches are "unsafe" - they can miss, for example, that some move other than recapturing the queen would've won the game. However, it's rare that this happens, and in return they push their depths very high. ("Depth" is a technical term for how deep the engine searches, so e.g. an engine that searches to depth 30 is likely to beat one that only searches to depth 20, all other things being equal).

Evaluation. This is the other prong of engine code. Given a particular position, is it better for white, black, or equal? This can involve all sorts of functions, e.g.

  • If one side has extra material/space, give it a bonus to eval.
  • If white has an advanced knight supported by a pawn, give white a bonus to eval.
  • If black's king is stalemated, give white a bonus to eval.
  • If white has a rook on the 7th rank, give white a bonus to eval.
  • If it's an endgame (and there are algorithms to decide whether the position is an endgame) and both sides have opposite color bishops, impose a penalty to eval (i.e. push it towards 0.00).

Today's engines evaluate positions much better than Deep Blue.

As for whether the algorithms are public, Stockfish is currently the world's strongest engine and it's open source. You can download the code yourself from Github.

Not the answer you're looking for? Browse other questions tagged or ask your own question.