Statistical Skier

I received an email recently asking the above question, and since responding to these sorts of things is often kind of challenging in an email I figured I’d just write it up here. I’m sure I’ve written about this sort of thing before, but it was probably a long time ago. Regardless, I’ve been looking at skiing results data for long enough now that I’ll admit to having some pretty strong priors when people ask me questions like this. Specifically, my gut reaction is to respond that whatever pattern you’re responding to is probably not going to be particularly evident if we start looking at it systematically. Then again, I’m sort of a statistical Debbie Downer so take that with a grain of salt.

Setting my depressing priors aside, let’s see what we see. So let’s start by taking all US women’s major international results over a sizable chunk of recent history, say since the 2013-2014 season. We’ll exclude pursuit starts, since they rather famously don’t compare well with anything else, and let’s also exclude events in USA/CAN to avoid having to worry too much about events with enormously expanded American starters. (This last one isn’t a huge number anyway.)

The FIS race calendar is mostly pretty predictable, but it does shift around from year to year. But let’s be simple about this and divide the races up by month. However, I’m going make a special allowance for the Tour de Ski and basically consider it a “month” by itself. Probably I’d guess that people who’ve done it might be ok with labelling it as a month’s worth of racing, anyway. So our “months” are Nov, Dec, TdS, Jan, Feb and Mar. That’s a little weird, but hopefully it makes sense.

Again, simple is the best place to start, so let’s just count up American women top 3 and top 10 results in those time periods since 2014:

Three things immediately jump out: a noticeable lack of top results in November (i.e. Ruka), an apparent spike in good distance results during the Tour de Ski and a very consistent rate of top results in sprinting after November.

Yes, yes, I hear you all shouting at me through the internet about needing to look at rates not counts, so let’s divide all these by the total number of results (i.e. how many US women finished a race in that time period):

See, now that’s kind of interesting! It’s almost like things completely reversed between the distance and sprint results. We still see pretty low rates of top results in November, but now the rate of top distance results is the more stable of the two and the rate of top sprint results has an obvious peak during the Tour de Ski.

Circling back to the original question, my Debbie Downer-ism was mostly correct in that there isn’t some very obvious peak in Nov/Dec that steadily declines throughout the season. But that doesn’t mean this was all a waste, of course.

What’s going on here, I think, is two things: I suspect the US may have made the Tour de Ski a focus for their season in ways that perhaps other nations have not, and if you do focus on the Tour and finish it you’re likely to be rewarded with some significantly reduced field strengths and sizes in the later stages.

Frankly, what’s most interesting to me about all this is the relative lack of success at the Ruka races in November. I don’t know if that’s the result of intentional strategic choices or if perhaps the races and formats haven’t really suited as many of the US women as other events have. Perhaps I’ll have to write a follow up that digs into that a little further.

The recent episode of the Devon Kershaw Show by FasterSkier (which I’ve been enjoying quite a lot) included some discussion of the noticeably large time gaps in the men’s classic sprint World Cup in Oberstdorf, Germany. Beginning at around 25:18 they discussed a variety of aspects of the large time gaps. I thought that some of the things they noted would benefit from some cursory looks at some data. In no particular order…

“We’ve seen this before from Klaebo”

Indeed we have. Over the past decade or so when Klaebo has won a sprint qualification the margin has averaged around 1.65% in classic and 1.12% in freestyle. His margin in Oberstdorf was 1.31%. So this was not even particularly extreme for him, in terms of a gap over second place.

30th place is 15 second back

That is a remarkable time gap for a World Cup sprint qualification, for sure. But how unusual is it for much of the top 30 field to be that far off the pace? Let’s take all major international (WC, OWG, WSC, and TdS) sprint races over the past 10 seasons and plot the top 30 qualifying %-back values for each race as a line.

That’s definitely unusual. One of the things that’s interesting about gauging performance in individual sports like xc skiing is that there’s a subtle distinction between a particular outcome arising from one person skiing really slowly and another person skiing really fast. And of course it could be a combination of the two.

Devon and Jason discussed the fact that if you’re 30th and 15 seconds back in qualification, it seems really unlikely that you have a realistic shot at winning. But I think that underplays somewhat how unlikely it is for anyone to win a WC sprint race when you qualify that slowly, regardless of the time gap.

Over ten seasons, men qualifying in 25th-30th have reached the podium only 16 times, or around 1.92% of the time. On the women’s side it has happened only 7 times or 0.8% of the time. Actually winning is obviously even rarer: 5 times for the men and only once for the women, for 0.6% and 0.1% respectively. So the folks qualifying in the back of the field are very unlikely to win, even with “normal” time gaps.

Of course, it is easy to get confused about the causal relationship here. Fast qualifying times will tend to be correlated with all sorts of other fitness characteristics that lead to good performance in the heats. It’s not the fast qualifying time itself that leads to better performances.

Let’s return, though, to the unusual time gaps themselves. Certainly some specific talented sprinters had rough days (Ustiogov and Iversen were mentioned in the podcast). But if you’re sitting in 30th, 15 seconds out in this particular case, should you be saying to yourself, “Geez, I skied pretty badly!” or should you be saying, “Geez, the top 3-4 guys really just had another gear today!”. (Or obviously somewhere in between.)

This is one of the problems with measuring performance based on only the winner. Devon talked (correctly, I think) in the podcast about how if you’re 15 seconds back from the winner in qualification that’s a pretty good signal that you’re not going to win. But, it is a potentially quite misleading signal about how you skied relative to your own performance history!

One approach I happen to like to use as an additional perspective is to calculate percent back values based on the median, or middle, skier. What do the percent back curves I plotted above look like if I recalculate them based on the skier who qualified 15th?

Now which part of the red curve looks more unusual? This look at the race would at least suggest that the top 4-5 men were the ones who had unusual races, rather than the the field as a whole. The back of the qualifying field seems a bit more behind 15th place than usual, but not nearly as dramatically.

Obviously, this doesn’t change the calculus for how close you are to winning. If you’re racing against people like Klaebo who can put down winning margins like this, it is small comfort to be told that you didn’t really ski any slower than you usually might. But it is useful to keep in mind when you see time gaps like this that it is quite possible that the folks in 25th skied just as fast as they did yesterday, a month ago, a year ago (maybe faster!). In other words, it can be a signal of how much work you have to go, but not necessarily that the work you’ve done so far isn’t helping or even making you slower.

How fast do podium finishers qualify?

I’m glad you asked! Let’s look at that with the following plot:

ECDF stands for “empirical cumulative distribution function” which is quite a mouthful and isn’t a graph that normal folks interact with regularly, but don’t panic.

This just graphs the cumulative proportion of podium finishers (y-axis) who qualify with a given percent back or better (x-axis). So if you pick a spot on the y-axis, say 25%, and move horizontally until you hit the curve you look down on the y-axis and see that ~25% of podium finishers qualified with a percent back of ~0.4% or better. Don’t forget the “or better”.

Similarly, ~50% of sprint podium finishers qualify at ~1.4% back or better. And so on. The vertical line each curve starts with represents the fact that ~20% of podium finishers actually win qualification (0% back).

Looking at this it seems that 1.5-2% back might be a good target to think about if you want to be landing on the podium in World Cup sprint races. If you can’t qualify at least that close, you’re probably not setting yourself up for a good shot at a podium.

I got an email the other day asking if I thought that the US had been improving in international races lately. There are obviously lots of ways to measure the progress of a national ski program, but the only one I really can comment on is simply race results. So these are the US major international race results (WC, OWG, WSC and TdS) over the past decade or so. I’ve removed pursuit (i.e. handicap) start distance races, as they tend not to be very representative of how athletes perform in any other circumstance.

The women’s sprint and distance results have obviously improved, as they transitioned from one skier doing consistently well (Kikkan) to a handful of skiers doing well. The men’s sprint results have declined slightly, most likely correlated with Andrew Newell getting older and then retiring. The men’s distance results have been pretty consistently in the 25th-75th range with very occasional top 10-15 results, basically always in mass start events.

So the short answer isn’t much different from what you might say without all the number crunching: the women have improved quite a bit, while the men’s results have been either treading water or getting slightly worse, in the aggregate.