I did the test and got 50 squares in 30 seconds. Part of the reason was that it takes some time to move the mouse, but it seems you're quite close to me in speed. I'm also a titled player that can read games from books, so I don't think this is where your problem is.
I'd suggest practicing blindfold chess. Just try playing it a bunch of times with someone, and over time you should get better at it. This helps with reading games from just the notation, since it's the same type of skill being applied.
You mentioned strong players are familiar with theory. This is true, and it includes knowledge like typical opening moves and pawn structures. If the game you are going through (without a board) reaches a position with a structure you're familiar with, or features sequences of moves you've seen many times before, then it's easier to keep track of. If you asked a strong player to read through a game where the 32 pieces start in completely random squares on the board, they would have a harder time since nothing will be familiar.
So to improve your knowledge of theory, the answer here is to just study books on openings, middlegames, etc. Obviously the main purpose of this would be to improve your chess as a whole, and the increased abilities to read games will come as a byproduct.