4

I'm a big fan of pgn-extract, but I am open to other linux-friendly tools. I have a big pgn file (over 5 million games) and a set of ECO codes. My goal is to fetch all the games matching those ECO codes, but with removing or fuzzying the players names and ratings, the year played, and the game result.

I then plan to open a random game with Scid and analyze it, since removing the players' data and game result will help me avoid any bias.

So far Im using pgn-extract like this: pgn-extract -t filter_for_naked --noresults --notags --nocomments --noduplicates -bl20 --output naked.pgn my.pgn

But Scid is having problems opening the file. As expected each game is just the set of moves (no headers at all) and I suspect Scid requires at least some headers.

Any advise on how to use pgn-extract or Scid to achieve this?

7

Thanks for your compliment about pgn-extract (I am its author). You might be able to make the task a little easier by adding --linelength 1000 to the argument list in order to get each game on a single line, then use sed to add an unknown result to the end of the game and re-process the game with pgn-extract to add an empty 7-tag roster. Finally, use sed again to edit the White and Black tags.

The resulting command would look something like the following, although sed experts might be able to clean it up a bit:

pgn-extract -t filter_for_naked --noresults --notags --nocomments --noduplicates -bl20 --linelength 1000 my.pgn | sed -e 's/..*/& */' | pgn-extract -7 | sed -e 's/White "?"/White "AnonW"/;s/Black "?"/Black "AnonB"/' > naked.pgn

You will get some error reports for games that end in mate having inconsistent results but you can hide those in a logfile.

| improve this answer | |
  • Thank you so much David! both for pgn-extract and the working command, it gave me exactly what I was looking for. Big help in my chase for improvement. – Purefan 2 days ago
  • You are welcome! – kentdjb 2 days ago
3

Scid vs PC requires [Black "AN Other1"] [White "AN Other2"] [Result ""].
Since this was cloned from SCID I'm pretty sure SCID will be the same. I suggest you write a simple program to add these 3 header fields with the [Result] field matching the result at the end of the game.

| improve this answer | |
2

You can use sed '/Result/c[Result ""]' my.pgn > no_results.pgn to replace lines containing the string Result with whatever you want the line to be. You can adopt this for whatever else you want to replace. You want to keep the c.

| improve this answer | |
New contributor
Arne is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
1

You can always copy the pure text from a PGN manually, but that only works for one game.

For more games I am not sure, and unfortunately most chess software is for windows.

| improve this answer | |
New contributor
David Miedema is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.