I am trying to build up a training dataset to train a convolutional neural network (well, the pipeline will probably be a little more complex, but probably a CNN will be at its core) to convert handwritten scoresheets into PGN.

Does anybody have a (larger) body of scoresheet images + respective PGNs or knows which persons/entities could potentially provide those? Since score sheet formats vary widely between countries and federations I'd be interested in a great variety of them.

  • Tournament directors and chess coaches must be a good starting point. I don't keep the the handwritten scoresheets of my former pupils, but other coaches might. BTW, I suppose you are only interested in algebraic notation ? Only in english, or worldwide ?
    – Evargalo
    Aug 4, 2020 at 12:35
  • 1
    Thanks! For now I would be only interested in algebraic notation but I am rather language-agnostic as long as I could easily tag the images in a first step. I'd probably train the model on some specific language first and would then expand to others if this yields good results. I should probably consider setting up a website where people could upload their own scoresheets. Aug 4, 2020 at 12:43
  • Have you started the project? Do you intend on adding chess knowledge to the system, so that it will only consider legal moves to increase the accuracy of the model?
    – Hauptideal
    Jul 18, 2022 at 9:15

2 Answers 2


Does anybody have a (larger) body of scoresheet images + respective PGNs or knows which persons/entities could potentially provide those?

All FIDE rated tournaments where norms are available (so must have a minimum number of GMs, IMs and federations playing in the tournament) are required to submit PGNs of games played to FIDE to ensure the integrity of titles. This means that all such tournaments will have a stack of scoresheets which have to be entered into the computer and submitted to FIDE. This means that arbiters at such tournaments who have responsibility for inputting are likely to have large numbers of such scoresheets.

Aside: As somebody who has been the chief arbiter in a norm tournament I can tell you that such a program, if successful, would be very, very welcome.

Additionally some federations, leagues and tournaments have policies of trying to submit PGNs for FIDE rated tournaments where norms are not available. The English Chess Federation is one such and in it the 4NCL (4 nations chess league) are very good at publishing PGNs.

Entering scoresheets into the computer is often difficult and can be boring. Some people (me, for instance) are not that good at processing a stack of, say, 100 scoresheets. Other people are very good indeed. In England there is one star who is quite brilliant. That is International Arbiter and IM Jack Rudd. I don't know what he does with scoresheets after he is finished but if he doesn't throw them away he will likely have a very large pile. You will see him on the list of arbiters officiating at mega tournaments like Gibraltar and Isle of Man because he is so good at this.

To be useful your program will need to handle languages other than English, including languages which are written from right-to-left like Arabic, Hebrew and Thai.

I should add that arbiters are not particularly strict when it comes to enforcing rules around scoresheet legibility and entering the moves correctly. For instance, although the abbreviations for the pieces may be in your native language, the files must be written in English ("a" through "h") however many players use their own alphabets for these and I don't think I have ever come across an arbiter who enforces this.

Finally, here are a couple of scoresheets from a tournament where I was chief arbiter a few years ago.

Almost illegible scoresheet - descriptive

Surprisingly this is one of the more legible scoresheets from this player. He did, though, use descriptive rather than algebraic notation.

Here is a more legal one (algebraic)

enter image description here

I wish you a lot of luck with your project but I don't think you really understand the problem. Playing chess can generate very high levels of stress in the players and this often leads to problems with how they write the moves.

PS You can download the pgn file for the tournament these games came from from the FIDE website here.

  • 1
    Thanks a lot for the information, much appreciated! I am well aware that the project is very hard and has a high probability of failing. However, even a accuracy of only 40% would be a great achievement in comparison to the status quo where many an hour are wasted in entering these scoresheets each day (I've also done my share...). There has been great progress in the field of deep learning in the last 10 years. Also, in case of chess games information about the possibility and probability (engine evaluation) of a move can be taken into account in addition to plain text recognition. Aug 4, 2020 at 14:14
  • I am pretty sure that the problem will be solved eventually. In any case a first step would be the assembly of a large training dataset, regardless of who will eventually succed in putting together a good enough deep learning architecture. Aug 4, 2020 at 14:16

In case you haven't seen this, there was a dataset released in 2021:

Owen Eicher et al., Handwritten Chess Scoresheet Dataset (HCS), 1, ID:HCS_1, URL:https://tc11.cvc.uab.es/datasets/HCS_1:

The Handwritten Chess Scoresheet or HCS dataset is an annotated collection of chess scoresheet images from live chess events. It was created to aid handwriting recognition research of chess moves to automate scoresheet digitization process. This dataset contains 158 games comprised of 215 pages of scoresheets digitized using a standard cellphone camera in natural lighting conditions. These images are tightly cropped, and a standard corner detection-based transformation is applied to eliminate perspective distortion. The headers and footers were also cropped out from each image in order to maintain player anonymity. Each scoresheet possesses a total of 120 move boxes, though in most cases a given scoresheet contains some blank boxes.

Also attached (please check the "Ground Truth" section) are two text files which contain ground truth data for the scoresheets. A folder named "empty scoresheets" contains a few scoresheets with no handwriting on them. This is included in the dataset intentionally to provide a format for the chess scoresheets used, and to help with extracting individual text boxes from scoresheet images.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.