Statistical Skier

Another interesting data-centric post appeared over at NCCSEF, and when it comes to data, I just can’t help myself but comment.

This time we’ve got some slides that seem to be trying to draw a relationship between WJC results and winning a medal at (I believe) the Vancouver Olympic Games. We’re only shown the (partial) results for six skiers, so I’m not sure what exactly the lesson is supposed to be.

We seem to be mixing sprint and distance results together as an indicator for future success. That seems strange to me, but I’m certainly not an expert in that sort of thing. We’ve also selected a curiously successful subset of Olympic medalists to examine. Absent is Pietro Piller Cottrer, who’s best (and only) result at WJC was 32nd (admittedly, a long time ago). Also missing is Aino-Kaisa Saarinen who’s WJC results were 15th and 23rd. How about Tobias Angerer (WJC: 18th, 26th, 28th)? On the other hand, we are shown Marcus Hellner, who’s WJC results were good but not spectacular: 15th and 21st.

The further information provided at the bottom regarding time to an athlete’s first podium also contains mostly skiers who achieved this feat fairly young, but then also two who did not (Gaillard and Rickardsson).

What am I to learn from this? That the right path is to podium at WJC (Northug), except when it isn’t (Bjørgen, Haag)? That the right path is to be successful early in your 20’s on the WC (Northug, Harvey), except when it isn’t (Gaillard, Rickardsson)?

When I read stuff like this, I’m left feeling mostly confused, like I’ve been presented a bunch of data, but that no one has gone to the trouble to transform this data into information. The reader is left alone, drifting in a sea of numbers, wondering what exactly was the author’s point.

I’m absolutely not going to argue with the idea that skiers who show considerable promise early on are more likely to develop into successful WC skiers. Indeed, I’m less interested in the nuts and bolts of what results mean at a given age than I am in effective and clear presentation of data.

I’ve written about connections between WJC results and medal on another occasion and I tried to emphasize the fact that when you look at all the data, there’s certainly a connection, but the different paths that skiers take toward success can vary so much that it’s difficult to create many useful generalizations just from the data.

But let’s revisit this idea with a few simple approaches and see if we can organize the data in a way that’s informative (and maybe interesting too!). First, I’m going to broaden the scope from medals to top ten results at either Olympics or World Championships. The problem with looking only at medalists is that there are just too few of them. Much can be learned by imitating a single good skier, but there’s always the danger that what worked for them only worked because of something unique about them, rather than having stumbled across some universal truth of skiing.

Let’s tackle the connection between WJC results and whether or not someone achieves a top ten result at the Olympics or World Championships. I fit a simple model (actually, not so simple; no OLS regressions here!) and plotted the model’s predictions for the probability of a top ten result at a major championship based on that athlete’s best result at WJC (sprint or distance):

Just to be clear, this is not based on raw data; this is the model’s view of the world.

At first glance this looks fairly promising. The confidence bands (shaded region) are fairly narrow, reflecting a reasonable degree of confidence in these estimates. The safest way to interpret these probabilities is in a relative sense, rather than an absolute one. So for instance, I wouldn’t put much stock in the claim that a man who’s best WJC result is 5th has exactly an 80% chance of a top ten, but it does seem fairly reasonable to claim that they are roughly twice as likely to have a top ten as someone who’s best WJC result is only ~16th.

How good is this model at actual prediction? Um…ok, I guess. It accurately predicted the presence or absence of a top ten result in ~75% of the 814 cases considered. I made the model considerably more interested in correctly predicting a top ten result than in correctly predicting the absence of a top ten result, since that’s the way I think our biases lean. This means that while it did a decent job of correctly identifying people who would go on to a top ten result, it accomplished this in part by incorrectly tagging a fairly large number of people who did not. Whether or not 75% accuracy is “good enough” is largely a judgement call and will differ from person to person.

WJC results seems like such a crude measuring stick, though. It’s such a small number of races! So let’s see what we get if instead we use the average of a skier’s best five FIS point races by the age of 20 (again, combining sprint and distance FIS points; I realize that’s not ideal, but I’m keeping things very simple). Same methodology yields the following graph:

Some important notes about interpreting this graph:

• Once again, the relative probabilities are more sensible than the absolute probabilities.
• It’s hard to have an average below 20 FIS points by age 20 without actually having a top ten result in an event roughly equivalent to the Olympics or WSCs. So 90% is probably a serious underestimate there.
• Be careful to remember that in this model an average of 40 FIS points may include 2 sprint races and 3 distance races, so it’s not really clear what an “average of 40 FIS points” means here.

How well was this model able to predict the data? Better. It was right ~82% of the time (compared to ~75% last time). All of this improvement comes from doing a better job of avoiding false negatives: this model incorrectly predicted a top ten result much less often than the previous one. On the task of correctly predicting a top ten result, both models fared about the same.

What I find interesting about this second model is that it reinforces my intuitive notion that something fairly important is happening when people start to drift below 50 FIS points.

[ad#AdSenseBanner]