How to determine if a position is open or closed in a "rigorous" fashion?

Question

I am creating a set of chess positions to be used [as a data set] for a project. It's based on supervised machine learning, but I need to have labeled data (see more italicized text below) before any machine learning can be done. I wish to label each position as open or closed or possibly some combination of the two.

I started out with assigning each position to just "open" or "closed," but this wasn't clear for a lot of the positions. Then I decided to go with a third label, semi-open/semi-closed, which is like the middle and in between open and closed. I’ve labeled most of the positions, but there are many positions that I still haven’t comfortably labeled with a distinct class.

I know that open positions are characterized as having pawnless ranks, files, and diagonals, while closed positions have them clogged up with pawns. In open positions you can usually centralize your pieces, while closed positions give you a hard time moving an arbitrary piece to a random location on the board. There are many other related patterns, features, and qualities.

I can easily go with my gut feeling when assigning "closed" to a position that has 7 pawns on each side with each pawn blockading another, and "open" to a position that has pawns only on the flanks and in their starting squares. However, it's hard for me to classify some positions such as these two examples:

[White "Position1"]
[Black "Open/Semi/Closed/Etc"]
[FEN "8/5Qpk/p6p/P1b5/4pPn1/2Pp3P/1P1BbqP1/R3R2K w - - 0 1 "]

---

[White "Position2"]
[Black "Open/Semi/Closed/Etc"]
[FEN "3r1rk1/pp2pp1p/2p3pb/2P5/4P3/4nNPb/PP2P2P/R1RNK2B w - - 0 1 "]

What would you assign to the examples above and why? For example: one position is more open than the other due to characteristic XYZ, or the second position should not be considered closed since there's an open file, and so on.

Should I just ditch the whole binary-ish labeling (classification) and instead assign a variable number (regression) describing how open or closed a position is (e.g. 0% to 100% open?) Or is there still a way for positions to go into a few categories, which means that we can have "rigorous" definitions of the categories of openness? And "rigorous" as in having few contradictions and preferably can be applied by a human without too much difficulty.

Answer 1

As a chess player who currently studies computer science, I found this question very interesting so here's my thoughts on the topic. You already know how chess books categorize open and closed positions and you may also know that a semi-closed position is one with some of the characteristics of an open but not all. As you understand this makes the line between closed and semi closed positions not easy to be seen. So my recommendation is to use a number to describe the "openness" of a position(although 50% may be a problem). Now some things I thinks you might find useful for implementing you machine learning algorithm. Firstly, I will suggest you to take a look at this article https://chess24.com/en/read/news/how-do-chess-engines-think in the section about position evaluation. Now, some parameters you might use:

1. Number of pawns(a position with no pawns is apparently wide open).

2. Number of pawn moves(assuming a position with three pawns each,there's a big difference if they are on f2,g2,f7 and f7,g7,h7 respectively and if they are on c4,d5,e4 and c5,d6,e5.

3. As you see in 2 you may give an extra weight for pawn moves in the center. Generally it will be a good idea to give extra weight for anything open in the center. This will be more clear in 4.

4. Number of open and semi-open files. The ones in the center again should have an extra weight.

5. Open diagonals. Note that diagonals with one or more isolani should be counted almost as opens(As an example think about the Indian's bishop targeting d4 and b2 when the c pawn is not on c3). And you should of course take into consideration the length of the diagonal.

Note that essentially you don't judge the whole position, just the pawn structure. That's why count the available moves of every piece is a mistake. This just measures the mobility of the given piece and doesn't necessarily has something to do with the pawn structure. For example, in a position with no pawns a Knight on a1 has only two moves while in a closed King's Indian the f3 Knight has seven( only one is occupied by the King). Some of them drop the piece but they are legal moves nonetheless.

Hope that helps a bit!

Answer 2

In the end the distinction between open/closed position is subjective and as you noticed there will always be some overlap or positions that don't fit perfectly into either category.

What exactly do you need this for?

For me the typical closed position has blocked pawn chains, generally many pawns, few or no open files. Because of this, the play tends to be slow, positional. Typical plans include undermining the pawn chains from the side or pawn storms/attacks against the enemy king.

Conversely open position have fewer and mobile pawns, open files, more space for pieces and are generally more tactical.

Answer 3

I echo user 1583209; why are you doing this?

But just as an abstract exercise, how about this measure of open-ness?

Look at every piece on the board, and count the number of moves that it would have if there was nothing else on the board. Add this up for all pieces on both sides and call it $N_{empty}$.

Now count the number of moves/captures that each piece actually has and add these. Call this total $N_{actual}$. The ratio $N_{actual}/N_{empty}$ is a measure of the open-ness of the position.

I think this is cute, and maybe as good as anything else, but why are we doing this?

Answer 4

As in openings, positions would be regarded as closed or open depending on whether they were blocked by pawns or not. And as in the openings, these could be semi-open as well, which as you indicate would be an in-between type of situation. These distinctions are not necessarily fixed but can change depending on whether or not pawn exchanges are made to open up the position. In an open position, the pieces would have more freedom to roam and search for tactical combinations as opposed to a closed one where the opposing pieces couldn't come in contact right away and more maneuvering would take place behind the lines. I would think that it would be more reasonable in your classifications to stick to these more general categories than to try to use percentages to describe what will be more or less fluid situations. I would categorize both of the positions shown as semi-open since Black has more open lines of attack and freedom of movement for his pieces in both as opposed to White. In fact, if it's Black's move, he has a pretty forced mate in position number 1 with 1..., Qxg2+!; 2. Kxg2, Bf3+; 3. Kf1 (if 3. Kg3, Bf2#) Nh2#. And if Black moves first in position number 2, he forces mate with 1..., Rxd1+!; 2. Rxd1 (if 2. Kf2, Be3#), Nc2+; 3. Kf2 Be3#.

Answer 5

You cannot. They all have characteristics of open/closed between the two extremes so constitute a fuzzy type of set or a statistical sort of categorization. There is no way to declare absolute that a position is either open or closed when many (most?) are in the vague area of semiclosed/semiopen as nobody tries to define quarter or 3/4 closed/open or finer fractional/decimal categories.

Answer 6

If i had to create a generalization it'd be something like, when you have atleast two central pawns for each side and they are in contact with each other and can't be taken (say pawns on d4 e5 and e6 d5) that is a closed position. Positions ranging from there being 4 central pawns that are not in contact (e2 d3 and e6 d5 for example), to there being 2 central pawns (1 pawn for each side, like pawns on d4 and d6, or pawns on e4 and e5) are differing degrees of being semi closed. Positions with only 1 central pawn, or no central pawns on the board, I would consider open.

Answer 7

There are historical notions that made more sense at some point of history than now, when the whole chess community knowledge that can sit in books or other storage media, I think also online databases, has been substantially exploding as well as the number of players globally, and means to diffuse that knowledge. Some talk about holes in the chess theory. I think some thinking in the line of extending notions and adjusting them to how big chess could be that we might not have foreseen as a community in the past, could help make more automatable or useful metrics that would have value for any legal position, and what such metric might associate to odds given some 2 player pair of average "strength" for the full legal chess point of view with all players quality of move possible. Simpler version would be like the endgame table assumptions, for all legal positions giving its output under the assumption of perfect continuation.

But, even if now one can find many ways to dismiss such dichotomies for not being applicable to all positions, a good way to adjust that past intuition made feature, is to see it as the opposite bounds of some more, how can I say, fine grain notions of line mobility obstacle degree. I could see many ways to test or construct automatic metric, that would compound all line pieces immediate mobilities and potential. That last word, is to tie in to what should be kept on the open-closed dichotomy. That opening pawn moves might set the tone, are irreversible, and their mutual, relative and across the board configuration tend to change slowly, so that having had the spectacular version of either open (that we would all agree it is), or closed, would soon steer the gamut of plans left possible and plausible as winning plans.

One could call the various constructed quantitative functions of the full position information metrics measuring each properties.. it is even possible that the notions would behaver as a good feature in ML on the data analysis side, if it started very fine grain, defined for each line pieces specific placements given all the other pawn placements (why not also include other blocking material of own color, and opposite color soft occlusion.

In fact, it might be equivalent to just the basic input vector for Lc0. Which is directly and exactly encoding the mobility rules set.

That would be the full multidimension version of what closed and open might mean as an average yes or no criterion over all those ultimate fine grain mobilities for the line pieces. There might also be another angle that would not be about the immediate mobility. But that would need a better formalism to describe many possible future mobilities for fast pieces given slower evolution of the slower pawn structure. This is not a good answer. I am just pushing the notion to their limit. Not dismissing them. The "gut feeling" might actually be an internal wet implementation, of what you could simulate in such ML project. I like to think. Good question, good principle of question for chess theory elements, actually. But I would not reject notions that can't be automatize directly, and would rather extend the notion to the now available data from actual machine players too. That which start by exploring legal chess, might have more dynamic range for your type of project. I suggest also digging in its latent space entrails. If you can do that.

Answer 8

As a statistician with a PhD in psychometrics and an interest in chess, I find this fascinating.

I would not categorize the variable. Three levels is better than two but why have levels at all? You can give an openness rating from 0 to 100.

One way would be to have a bunch of good players (whoever you could get, or pay) to rate a bunch of positions on how open they are. Then you can look at various features (many suggested above) and do a regression.

Alternatively, you could just make up a formula intuitively.

Answer 9

The best way to classify the position would be not just "closed" or "open", but with a continuous scale. I would propose having a "completely" closed position at zero on the scale; it's much easier to define the most closed possible position (a KP endgame with 16 pawns total and no pieces) than the most open position which seems basically impossible to define rigorously.

Several factors should be taken into account when considering how closed a position is, the most important of those being number of pawns, pawn mobility, and then piece mobility (but weighted specially). There are also several other slightly less important things, such as how exactly to weight mobility (more on that later) and open/semi-open files.

Assuming the position with zero for a maximally closed position, we can first take (8 - number of white pawns) and (8 - number of black pawns), then raise them each to the power of some constant (likely between 1 and 1.5, say 1.4 for the sake of demonstrating the next concept), then add those together. We have the exponentiation to make a position with 8p4p more open than a 6p6p position, and also to make the difference in openness between 4p4p and 6p6p be greater than the difference between a 6p6p and 8p8p position.

The next thing to take into account is pawn mobility. Pawn captures, including en-passant, should be weighted to be more significant than just single pawn pushes, and double pawn pushes should be weighted to be pretty insignificant since being able to double push on top of the single push should not give that much more openness.

Based on the scale from the previous part where the maximum openness score is around 36, every single pawn push should be weighted at around 1, every double pawn push should be weighted at around 0.5, and every capture should be weighted around 1.5. An additional bonus between 0.5 and 1 should probably be added to moves by pawns on the c, d, e, or f files, that way the center being locked makes the position significantly more closed.

There's also the matter of mobility. In my opinion, non-pawn mobility should be weighted by piece type and the square that's being moved to. For example, the king should have no weight in terms of mobility and the queen should have little weight so the queen doesn't skew the openness of the position too much. This might be unnecessary however.

The main point in mobility is weighting the moves based on the pawns on the file that they move to. If the move goes to a square in any of the spaces in front of a friendly pawn on the move's file, an openness score of around 0.3 can be added. Note that if there is no friendly pawn then no score is added. Additionally, if the move goes to a square behind an enemy pawn on the file, an additional bonus of roughly 0.3 should be added.

I think this answer is already too long, so I won't talk about open files. Also note that I haven't really tested out these ideas, and I suspect that the mobility stuff should be done differently so endgames don't automatically get scored as more closed due to the decreased total pieces and hence decreased overall mobility. This can probably be done but I can't figure out how to do it easily without breaking the idea about zero on the scale.