I'm posting a sample answer to my question to give others an idea about what I'm thinking about.
First, you have to define what you mean by best. For example, does best mean you are the most dominant player for your era? Or does it mean that the quality of your player is superior to all other players. And if quality is what you mean, then how do you define quality?
Paul Morphy was probably the most dominant player. For example, when he was 12 years old he defeated a top ten player (Lowenthal) in a match 3-0. According to Edo and chessmetrics he was probably already one of the best players in the world at the age of 12! At the age of 21, he played against a simultaneous against 5 top ten players (Bird, Barnes, Boden, De Reviere, and Lowenthal) and scored 3-2.
However, most would argue that dominance is a poor indicator of who is best. After all, Morphy has been described as the first modern chess player. His competition was weak compared with subsequent champions.
Another definition that has been used is quality of play. However, this definition also has a lot of problems. In the 1900 hundreds , a number individuals argued that Steinitz or Lasker were the best players of all time arguing that their knowledge of opening and modern theory would make them superior to players from the past. However, Louis Paulsen made some very clever arguments against this hypothesis. He argued that Morphy (who had a photographic memory and memorized the Louisana bar code by the age of 19) if brought back to life would learn openings and modern theory within a year and be able to compete successfully against modern chess players.
The mathematician Kenneth Regan argues that modern chess players who have access to chess computers and modern training methods play more like computers than players of the past. That’s no surprise because they were trained by computers but does that mean that modern players are really better? This begs the question what would Fischer or Capablanca do if they had access to modern computers? The problem with computer comparisons is they don’t take the historical factor into account.
You need an error weighting system that doesn’t just calculate the percentage of errors per 100 moves (which is basically what Regan and Guid and Bratko did ). Instead, you need to calculate the difference between your error rate and your opponents error rate. After all, chess is about committing fewer errors than your opponent. Putting pressure on your opponent to induce more errors is considered a good quality.
If innovation leads to dominance within a specific chess era over time and it becomes increasingly difficult to innovate over time as the strength of the competition increases you can’t measure true dominance by just looking at the match records of the top 30 players. That is, it’s a lot harder for Magnus Carlsen to dominate his opponents than it was for past champions.
Can the problem of increasing quality of play overtime be corrected by looking at back ratings? I prefer the Edo back rating system http://www.edochess.ca/ because the statistical assumptions are better. According to Chessmetrics, Zukertort, Steinitz, Tarrasch, Lasker, Pillsbury, Maroczy, Marshall, Janowsky, Chigorin, Schlechter, Blackburne, Duras, Teichmann, Neumann, Vidmar, Gunsberg, Rubinstein and Burn were better than Morphy. That doesn’t ring true for me.
If you look at Edo or Chessmetrics back ratings it’s easy to see that the magnitude of the difference between the top players’ ratings has been decreasing over time. So I believe an Edo type statistical model that takes into consideration the difficulty to dominate over time would be a better approach than what has been tried previously.
So my dream analysis is that you use Edo ratings based on a database that only includes the top 20 or 30 players from each five year period. After you complete this analysis you reweight your results by a dominance factor. That is, more recent players get a bonus factor that is calculated by estimating the trajectory of difficulty of dominating over time (the decrease in rating disparities between top 30 players over time). Next, you would validate this analysis by comparing players percentage of chess computer calculated blunders their opponents make minus their own blunders. If this invalidates the above, then you need to reweight according to the computer error checking analysis if it shows there is a tendency for more recent top players to play more accurately even after my dominance factor is taken into consideration.
My guess based on my eyeballing this, is that Kasparov would do very well. But that’s just a guess.