For example, could one measure ACPL (Average CentiPawn Loss) over time to detect progress? Could masters look at the games of a player and determine they're playing better even if their rating isn't going up, assuming their losses aren't attributed to time trouble or poor internet connection?
There are a couple of things to track.
Most of us has opening repertoire, you can practice it say once a week and record how many of it that you can solve. It is like a puzzle and you have to find the best move or idea.
Middle game knowledge
Same as in the opening.
Same as in the opening.
Memorize/understand a whole game, these are games that are related to your style or opening and are full of instructive plays. Practice it say once a week, Can you perfect it?
You can use ACPL if you have a fix opponent. Obviously you cannot use human opponents because they can change. So the best candidate for a fix opponent is an engine. Choose an engine that is close to your strength. Play it at say 4 games per week, then measure your ACPL. If you beat it with ease, choose another stronger engine.
One metric that I quite like to use is accuracy. I'm not sure about other websites, but with the premium Chess.com subscription you can run an engine analysis on each game you play. With this analysis comes an accuracy percentage 0-100% e.g. if you played the perfect move each time you would have an accuracy of 100%.
Over the year I've been playing I've noticed that I've gone from an average accuracy of ~60% when I was rated 600 elo, to an average accuracy of 70%-80% at 1200 elo. It's quite a nice way to measure improvement because although you will likely have lower accuracy in games you lose, your accuracy is somewhat independent on the result of the game (you can still play amazingly and still lose because of 1 blunder).
By that I mean that if you want to track something other than actual rating, but that was based on rating instead, then it's possible. For example, you could give puzzles in increasing degrees of difficulty, but that difficulty would be based on rating.
Otherwise, everything in most games is based on rating. There are many rating systems and chess uses an ELO system. For example, in sports seasons they use W-L ratios or points systems, which are also a type of rating. Even calculation of an average centipawn loss is a rating although engines wouldn't really be accurate as chess is not solved yet.