I am a developer currently working on software to compute endgame tablebases. The real bottleneck with calculating positions is not memory per se but the working memory and computing cycles needed to calculate a network for an exponentially-growing set of positions.
The number of positions grows exponentially according to the number of men on the board; this means that the working memory needed (whether it be RAM or hard drive reads, which are less expensive and more scalable but significantly slower) increases proportionately to A^N
where A
is a scaling constant (somewhat less than 64) and N
is the number of men in the tablebase. The Lomonosov Supercomputer used 92 TiB of RAM, which is sufficient for a 7-man tablebase; as dfan said in his comment, the jump is too big.
Furthermore, as more nodes are added to the tablebase the number of calculations increases faster than linearly to the number of nodes. For instance, accessing data in a Binary Search Tree (BST) is proportional to ln(M)
where M
is the number of nodes in the tree; since M = A^N
, to access each node of the tree once, A^N * ln(A^N)
operations are needed. There are several steps in the tablebase retrograde evaluation algorithm that scale similarly.
In short, memory increases exponentially and the number of operations increases faster than exponentially as more men are added. Computing has not advanced this far in (at the time of writing) 6 years.
Edit
Since this answer is getting a lot of traffic and I re-read my answer, I feel the need to add some clarification. Endgame tablebases are created with retrograde analysis, so deterministically calculating the outcome of one position requires knowledge of enough possible successor positions. As we know, this graph mushrooms quickly. While retrograde analysis is definitely parallelizable, one needs an enormous amount of working memory. However, it is common practice to allocate only 10s of GB of RAM to a single processor, so positions must be communicated over a network. This would require a lot of bandwidth, and multiple processors contending to make changes to the same table state would create synchronization problems that add overhead. Positions must be stored in RAM because disk read latency is horrendous (on the order of milliseconds instead of nanoseconds for cache or microseconds for RAM). It's possible some amount of pipelining or caching could amortize poor latency, but this is still a hard problem to solve at the required scale.
Tl;dr even with a big supercomputer, modern computer architecture is not amenable to this type of problem.