What is a good PGN software for dealing with buggy PGN files?

Question

I'm scanning old club zines. Apart from a colleague of mine who has the habit of hübnering up nested annotations with a mix of {[()]} (if done correctly, this should be no problem for PGN standard, since {} defines a comment regardless of additional brackets) there is also easily a ( mistaken for { in OCR or the other way round, or I forget to change () (in the zine) to {} or vice versa.

Here is a random example of a bug from today:

[FEN ""]  
1. d4 Nf6 2. g3 c5 3. c3 cxd4 4. cxd4 d5 5. Nc3 Bf5 {comment} 6. Qb3 Nc6 7. Nf3 {comment} 7...Na5 8. Qa4+ Bd7 9. Qd1 {comment) 9...e6 10. Bg2 Bd6 11. O-O O-O 12. Ne5 {comment} 12...Rc8 {comment) 13. Bg5 Nc4 {comment} 14. Bxf6 gxf6 15. Nxc4 Rxc4 16. e4

As you observe, two closing ) should be }. (Spotted them both in the source code?)

Here are some options I already tried:

• My usual PGN software is ChessPad, which said "Bxf6 illegal move". But the error happened far before that.
• Caissa Online PGN (my other favorite tool) gives total nonsense.
• Lichess doesn't accept it at all.
• I'm too lazy at the moment to write a Python parser.
• Brian (comment below) suggested "abusing" the CSE PGN tool. Guess what, indeed it barfs exactly where it should (at the first error), and would be a viable option if I can't find the bug. I'm still not happy with writing a "fake" question just to parse my PGN files.

Thus I would like you to suggest good standalone PGN software which nails down the error(s) in buggy PGNs exactly where it happens, as the CSE parser seems to do. This includes illegal moves due to typos, which are also frequent. (Clearly, this is tricky: a naive approach in the given example might pair the first closing bracket with the one still open after Qd1, and thus find as "next" move 12...Rc8 and protest there.)

Answer 1

Have you tried Scid? I tested your pgn on Scid v. 4.7.0 (Database/Import File(s) of PGN games...) and these are the results:

1. When importing the game it provides the following message:

Importing from test.pgn...

Imported 1 game.

PGN errors/warnings:

(game 1/line 6) Invalid Result tag: ?

(game 1/line 8) Error parsing the tag pair FEN ""

(game 1/line 9) Failed to parse the move: Bxf6

(game 1/line 10) End of game, ignored the part after the last error.

2. And this is the parsed game:

Not perfect, but better than Caissa and LiChess. And if you have a bunch of games to import, it doesn't crash and tells you which ones are buggy.

Answer 2

I note that the accepted answer doesn't really address your question. Understandable, as your question probably was vaguer than you might have thought.

First, what is a buggy PGN file? I would define it as one that doesn't fulfil the requirements of the PGN specification. (The Wikipedia page https://en.wikipedia.org/wiki/Portable_Game_Notation links to a version of those requirements.)

But as the PGN specification mentions four or five different file formats, it's also a question of which of those you refer to. The most probable is the "Export format", which is intended for files output by software, as intended for interchange, as well as archiving.

Then, that particular format is incredibly strict. If you list the requirements, you'll find that line separators must be line-feed only (with some under-documented exceptions), and you'll find that movetext representation must use the most compressed notation, in which some spaces are forbidden. Section 8.2.1 says that space character must only appear between two adjacent symbol tokens. That means that movetext such as "3. d5 (3. Kf6" is buggy: '(' is not a symbol token, so the space between it and 'd5' is incorrect. Similarly, numeric annotation glyphs (those $1 that are found in annotated PGN games) are not symbol tokens either, so "1. e4 $1" (corresponding to "1. e4!"( is illegal. No space is allowed between e4 and $1!)

To add to that, PGN insists on byte equivalence: A PGN file, written by two different pieces of software are not allowed to differ by one single byte.

In addition those requirements, you may find that many PGN files found "in the wild" claim to be UTF-8 encoded. That is not allowed by PGN (which only allows ISO 8859-1 ... which requirement actually is impossible to fulfil), so those files immediately qualify as 'buggy PGN files'.

And there are instances where the PGN standard itself doesn't know what a token is or not ... which is as situation no standard specification should ever allow to remain.

So ... what is a buggy PGN file? My attempt at definition above leads to most PGN files out there being buggy in one aspect or another. (It would be interesting to create a number of test files that have such bugs, and see if some particular software finds them or not.)

My own software for dealing with buggy PGN is very simple. Once I find that whatever software I use has problems (typically found when the expected number of games differs from those actually accepted), I run the program

print "Send back to originator, and complain"

A number of attempts at correcting PGN started, but as far I know they were never finished.

A program called "PGNtrim" (? I think) claimed to do some cleaning up of bad PGN files, but I noted that after feeding a file of more than 69000 games, more than 1000 of them were silently dropped. It did identify an extraneous '}' in a game, though. (I find a very old forum thread on that program that ends up recommending using scid instead, so ... perhaps you got the best answer, after all.)

(Added: If the errors you found are due to hand corrections, the file immediately changes from "Export format" to "Import format". And so you should need to run the file through a suitable program to transform it to Export format ... in theory. In practice I know of no such software.)