I want to try to explain the conceptual problem for the integrity of runing performance because of the potential of a shoe to enhance performance by upwards of 2%.
Let me try to do this by asking you to consider the following two statements, A and B. What you need to know is that only one of these can be true.
So basically, pick one:
Eliud Kipchoge is physiologically unique, by far the greatest marathon runner in history. He has moved marathon running forward by over 1% in only two years. His world record is 1:16 faster than the next best guy, he broke the then-world record by 1:18, and his best five performance average is almost the same as the next fastest performance has ever only once. He is a never-seen-before RUNNER.
(Subtext to statement A: Repeat the above statement, but swap Eliud Kipchoge for “Mosinet Geremew”, or “Mule Wasihun” or “Getaneh Molla” or “Herpase Negasa” or “Galen Rupp” or even “Julien Wanders” and then just change the context as appropriate to ask “Is ABC a sub-2:03/2:04 guy/59 min half marathoner”, physiologically?
The Nike Vaporfly shoe is a technological marvel. It’s worth between 2% and 4% to the actual performance of the runner because of its effects on the physiology of running, specifically running economy. In the words of Nike’s Vice-President for running: “Our elite athletes are telling us that they are experiencing notable efficiency gains – the highest athletes are testing over 5%”
So which is it? A or B? Red corner or blue corner? Because it’s one or the other.
OK, in fairness, I suppose, I can allow you a loophole of sorts, and say that you can create a Hybrid Statement C, that looks something like this:
Kipchoge is an amazing runner, physiologically superior to the current generation, who gets only a small benefit from the shoe, not so large that it improves a time of 2:03:30 to 2:01:39.
My response to this: How do you know? You cannot possibly evaluate what you have been watching in the last year and a half, because we don’t know where on the spectrum the performance boost lies. And because you have no certainty of that, you cannot evaluate the physiology or the physiological capacity (the person, in otherwords) relative to their own contemporaries, this current generation of runners.
Adjusting for the performance boost
What we can do is ask the following theoretical question:
For any running economy benefit of a shoe, XYZ percent, what would be the equivalent performance of the 2:01:39 marathon world record had it been run in the previous “best shoe” (or current next best shoe)? (You can do this for any other marathon performance too. Take your pick)
Below is a rudimentary graph and supporting table, and I’m just showing three performances – the WR, Geremew in London and Rupp’s PB – and the “conversion” back to the next best shoe, depending where on this spectrum you want to land:
In the graph above, I’m showing the equivalent or adjusted performance based on a range of RUNNING ECONOMY improvements, since this metric is what has been measured in the lab studies on the shoes. Running economy and performance don’t necessarily walk step by step together, in part because the relationship between O2 use and speed is not linear, but curved, and in part because the faster you run, the more energy goes into overcoming air resistance, so the relationship changes a bit for faster runners.
So what I’ve done is to use a study by Kipp, in which they modelled the relationship and estimated for elite marathon paces, every 1% in running economy improvement would translate into about 0.65% improvement in actual performance (for slower runners, the performance improvement rises per unit of RE improvement)
So anyway, what does it show? Leaving out the NYT analysis of data because that looks at performance not RE, we can see the following examples:
Kipchoge’s world record of 2:01:39 is the equivalent of a 2:03:59 if the running economy benefit is 2.9%. 2.9%, by the way, is what a lab study by Barnes and Kilding found when comparing the Vaporfly to the next most economical shoe when its mass was matched to the VF. 2:03:59 was the world record in 2008, exactly, as it turns out (Gebrselassie in Berlin)
Then we move along to 4.01%, the economy benefit in the Hoogkamer study, and 4.2%, the benefit of the VF when you don’t match the mass of the next most economical shoe. Now the 2:01:39 becomes a 2:04:54 and a 2:05:04. As recently as 2015, pre-dating the advance provided by the shoe (again, depending where you pin the advantage on that spectrum), those times had been run at least 34 times. And finally, at 5%, the “maximum reported benefit”, Kipchoge’s world record is adjusted to 2:05:44, which had been done easily over 50 times prior to the shoe’s introduction.
Indeed, Kipchoge himself ran 2:04:00 Pre-VF. If the shoe is worth, say 2% in performance improvement (a 3.1% RE improvement, well within the realms of what lab studies suggest), then that 2:04:00 was worth a 2:01:32 with the benefit of the shoe! Which means, by Nike’s own funded research, Kipchoge’s best performance is actually the one that he did when his insoles were running sideways and trying to escape his feet.
Food for thought.
Similar things are true for Geremew. At a 2.9% Running Economy improvement, his 2:02:55 is worth a 2:05:17 in the next best available shoe. At 4%, the Nike funded study value, it’s worth 2:06:12.
As for Rupp, he’s a 2:08:32 runner by the standards of one study (2.9% economy benefit), and a 2:09:30 marathoner if wearing the next best shoe according to the Hoogkamer study. If Rupp is the super responder, then at 5% economy improvement, he’s worth a 2:10:21 in the next best shoe.
Julien Wanders, incidentally, with his 59:13, would be adjusted to 1:00:48 if the running economy benefit he gets is 4.01%
Responders, non-responders and gaps in the knowledge
Now, your next question should be, what about responders and non-responders? Valid point, I’d say. You can decide on the spectrum above and then still assume that a given runner, say Kipchoge, might be a non-responder to the shoe. That would put him at 0%, which means the advances in performance are runner-derived. Or Geremew is a 2:02:55 guy, and would have been four years ago too.
To that, I’d say the following – Kipchoge would be an outlier if he were a non-responder, and so the above scenarios have to apply unless he’s in a distinct and as yet undocumented minority. In the study of Hoogkamer, they tested 18 very good runners and found NOT A SINGLE PERSON whose running economy got worse at the higher speeds. Here’s the graph. The solid line is the 4.01% RE improvement mentioned above, but each grey line shows some smaller, some larger, but not one is worse:
“What I can say on the record is that the Vaporfly Next% provided a statistically significant improvement in running economy over the original Vaporfly 4% in our test subjects,” Hoogkamer says. “Overall, it just performed better.” (*in every single person at faster speeds – my addition)
Here’s the problem – performance integrity
No doubt a lot of people will have read this thinking “so what”? And I hear you, so what? Whether this matters to you is a question of your philosophy towards technology and innovation. We can all have our personal viewpoints on this in a civil way. You might welcome the advance, and say “go for it, I like seeing a sub-2:00”.
I also want to stress that we don’t really know where on the spectrum the benefit lies. What is the magnitude of XYZ percent? I don’t know for sure, but I don’t think it’s zero, and I don’t think it’s the 5% (though the latter may be true in some runners, as mentioned). But it’s something, and not knowing what that “something” is is the biggest part of the problem.
As far as viewpoints on this go, here’s mine – when I watch elite runners, I want to have some degree of assurance that I am seeing the best runner win. I don’t watch it the same way I watch Formula 1, knowing that the race is in substantial part won or lost in the laboratories through engineering and tech. A world where the difference between cars is larger than the difference in ability between the drivers.
I want the result of a footrace to be determined by physiology and psychology, both of which are earned through training (as opposed to doping, for instance). I want this because I find value in the comparison, both within the race (who wins, who is better?) and over time, when it comes to running performances. History matters, and times matter.
Now, on this latter point, it doesn’t matter that Kipchoge is 16 min faster than Jim Peters in the 1950s. Only an idiot is going to start a conversation genuinely saying “I wonder if Kipchoge is a better runner than Peters?” This is blindingly obvious – generations pass and things change, including technology but also knowledge. It is expected that generational advances drive performance improvements, in pretty much every sphere of life. Maybe it’s fun as a bant over a beer, like it’s fun to argue over whether Federer would beat Laver.
But that’s impossible – things move on, and innovation is normal. But when the innovation is so large and sudden, what happens is that it distorts the contribution made by the runner to performance. It leaves us unable to assess whether we are seeing a human advance, and a great human athletic feat, or one where the winner of the race might not even be the best runner in the race.
So we celebrate Kipchoge, deservedly (I love watching him run, irrespective of any questions or doubts over anything), but in order to do so, we need to engage a degree of cognitive dissonance, because whatever he is doing is not comparable to what runners were doing even 4 years ago, at the time HE HIMSELF was running. We can’t even compare Kipchoge to Kipchoge.
And if you’re an Olympic hopeful, running to qualify, for example, and you lose a race by 45 seconds and the technology stands to be worth 2:20, then I think this compromises the integrity of the sport and its fairness. What is important is that the technological advance is sudden enough that the distortion happens in the SAME generation.
For instance, I noted above that 2:04:54, which is what Kipchoge’s world record adjusts to in the next best shoe if you work on the Hoogkamer 4.01% improvement, was run by 34 athletes prior to 2015. That means that within the SAME generation, a period of 4 years, we cannot conclude that Kipchoge is superior to any one of those 34 runners, whether is the same, or possibly even worse, because tech has confounded our ability.
To me, this is both unfair, and unfortunate. You might well pin that performance improvement at 0%, and then Kipchoge is legitimately physiologically worth 2:01:39. But since you can’t, you cannot appraise the performance even in the narrow window of Kipchoge’s own generation. You can’t even do it for Kipchoge’s own career, where, bizarrely, his best run ever is a 2:04:00 achieved despite his shoes.
Anyway, that’s the issue. I get that innovation happens. I don’t want people to run barefoot, and I’m not a Luddite on these issue. I just can’t trust what I’m seeing, and where that’s usually because I don’t know who is doping and who is not, I now don’t know who is benefitting and by how much?
And a final thought – given that the new shoe launched on Sunday in the London marathon may have helped three men race to 38km in sub-2:03 pace, there may yet be some merit in Nike’s contention that this shoe is even faster than the previous one.
At 2:00:25 in an incredibly contrived race in Monza, the sub-2 hour barrier is in sight. It needs an extra 0.36%. So they can throw a bigger Tesla at the problem, maybe build a bigger clock, use more pacesetters.
Or maybe they can just use this new shoe, which might just be worth 0.4% more than the previous one, and hey presto, a sub-2 hour marathon. But what does it actually mean, when the runner doing it might be at the same physiological level as 30 men of only one generation ago…?