Timeline for How do you calculate error bars on elo rating?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 21, 2023 at 6:09 | history | edited | David | CC BY-SA 4.0 |
fixed grammar
|
| Apr 15, 2023 at 11:08 | comment | added | David | @DM In this case it'd be the same since even if it was 1 win and 99 draws you wouldn't get a significant difference. The proportions test approximation is good enough. One could also try a t-test for normal distributions as a reasonable approximation | |
| Apr 15, 2023 at 1:11 | comment | added | D M | @David To get to the spirit of the question, would it be valid to just go to statology.org/one-proportion-z-test-calculator and fill in p=0.505 and n=100, look at the 95% confidence intervals, and plug them into the FIDE fractional-score-to-rating table to convert them to a ratings difference? Or does the 3 result possibility throw this calculation off? (If I do that, I get 95% C.I. = [0.4070, 0.6030] which gives me about -67 to +74.) | |
| Apr 14, 2023 at 9:12 | comment | added | David | @Allure with the information we have the best choice would probably be a proportions test (statology.org/one-proportion-z-test). Let p be the average score Stockfish would get against Leela and let's take p=0.5 as the null hypothesis. However since chess is a game with three possible results, it'd be more accurate to have the win/draw/loss count rather than just the final score. Then we can check if the number of wins is significantly bigger than the number of losses instead of just a test over the final score. | |
| Apr 14, 2023 at 9:07 | comment | added | Allure | It is possible to perform a test of statistical significance over the match results to determine if there's enough evidence to conclude that one engine/player is stronger than the other Do you know how to do this test, e.g. with the match in the OP as an example? | |
| Apr 14, 2023 at 8:56 | history | answered | David | CC BY-SA 4.0 |