10

A while ago there was a site made by NM Jonathan Hilton's father that did some analytics on player's USCF rating histories.

I thought it was a pretty good site, but it disappeared. There are still some tools like that floating around. I was wondering if there was some sort of USCF API, or how that sort of data might be grabbed.

4

I'm not aware of an API, but they way they have it set up it's pretty easy to gather data via a technique called screen/web scraping.

For example, a search on your name gives a link to a specific page with your member id in the URL. Then if you have a programming language/library that is DOM-aware, you can simply figure out where certain things will be, based on the design of the page. Using jquery selectors, and a couple of traversals, that page can be parsed to get the rating as such:

> $("td td td:contains(Regular Rating)").next().find("nobr").text();
"2072  
2012-03(Current floor is 1800)"

Nice rating, by the way.

It's a bit painful, and even more painful if they ever decide to change the design of the pages, but eventually you'll be able to parse out all the data you want, and then build a way of collecting data for each member and storing it in your own database for analytics. It wouldn't be that hard to do, just iterate through member ids, and you can get their names, states, ratings, etc.--as much is available on their screens.

Hope that helps.

  • And I just saw your website--you probably know all about scraping. Anyway, it would be cool if they offered an API, but I don't think that's the sort of business they're in. – Eve Freeman Sep 4 '12 at 4:01
  • I actually don't yet. Something to learn. Ah well, sounds like a fun weekend project! – Andrew Latham Sep 4 '12 at 4:35
3

I don't think that the USCF has a full API, but there are a few special url's that you can use for simplified screen scraping/data parsing:

In each of those "thin" pages, you can pass a member ID as a url parameter (for example thin3.php?11111111).

The server response is very very lightweight - no extraneous markup.

These pages were designed for cell phone browser use by TD's the day of the tournament if I remember correctly, but they're also useful for extracting data.

  • Nice find. Definitely better than parsing the markup. – Eve Freeman Sep 6 '12 at 19:08
  • Just a note, this will give you current rating and name by member id, or member id by name, but it doesn't give you any tournament history stats. – Eve Freeman Sep 6 '12 at 19:23
  • @WesFreeman the only way that I've been able to find tournament history stats (and, well, history) is by very methodical parsing of the msa pages. I might have some old php code that extracts data somewhere, I'll see if I can dig that up when I'm alone with my external hard drive later. – Andrew Sep 6 '12 at 19:56
  • They REALLY need some form of API. There are so many cool mashups that could be done... – Mongo Nov 5 '12 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.