NUDGE: A data-driven attempt at reducing concussion risk in rugby. A process explained

Right, rugby (and other contact sport) friends, gather round and let me try to share with you some slides & ideas I presented at the @NFLResearch Conference on Collision sport in London last week. This set was on nudging safer behavior, it’s a thread. But it’ll have pictures…1/

2/ First, a philosophical issue that *SHOULD* influence actions (subtly). Change is hard. It’s resisted. That’s as true in the #NFL as it is in #rugby, and in any other sport. These quotes you see below are basically interchangeable – players, coaches, media etc – push back against change.

3/ However, we also can’t allow the tail to wag the dog – if risk is too high, change IS necessary. But how much? The key question, for me, is how you achieve “change without change?”, as this slide asks? Change X but not Y (the sport) (too much). That’s key to getting buy-in.

4/ The process of trying to navigate this balance starts by understanding how the injury occurs? What is the mechanism? Where are the most likely preventable injuries? From 2015-2017, we (@worldrugby, in partnership with @mattjcrossie who was with the RFU), studied 611 head injuries

5/ The study found, among other things, that tackles are the most numerous AND highest risk (these are different things) injury event. Not surprising. What *IS* surprising (slightly) & crucial, is that the TACKLER has head injury risk 2.6x larger than the ball carrier (see the data on the right, which shows the propensity of match activities, and the propensity for the tackler and ball carrier – the tackler is injured 1.4 times every 1000 tackles, while the ball carrier has the head injury 0.54 times every 1000 tackles.

6/ This is really important, because you should by now be asking “If the tackler is most at risk, then what change do you make to protect a player who instigates an action that he himself causes?. It’s easy(-ier) to protect the recipient, the ball carrier, not so easy the ‘doer’

7/ Your 1st thoughts might be 1) Tackler technique, & 2) Law change. Yep, ours too. But 1st, we still needed to understand EXACTLY what tacklers do that increases their risk. So we studied 464 Head injury tackles and 4000 tackles not causing injury.  The idea is to develop a ‘risk spectrum”, where we can put the characteristics that are more likely to cause injury on the right, and those less likely on the left.  Once we know that, we can work towards risk reduction.

8/ This is really important because once we have this risk spectrum, we can consider options, like banning the most dangerous actions, or trying to substitute high risk for low risk (both law related), or changing high risk to lower risk through technique intervention, as shown:

9/ I can’t stress enough that this propensity method is THE ONLY way to manage risk – if you don’t know injury risk per 1000 EVENTS, then when substituting one behavior for another, you cannot account for “exposure”. So doing it by time, number or % is flawed & may make it worse

10/ So we looked at about 16 characteristics of the tackle. Here is the high level summary, somewhat oversimplified, of the key “isolated” tackle characteristics that have high risk and low risk. Some are obvious (speed, direction), some less so. #rugby#injury #concussion

11/ Next step – science MUST step aside. Our job is to describe risk, control for confounders. Once a risk picture is known, rugby experts must advise on how to change, not docs & scientists. So we gathered a group of 15 for intensive discussions, asking “How do we shift left?”

 

12/ Those experts – coaches, players, referees, former players – said “No way do you shift the top 2. Too integral to performance & the sport”.

They said “Maybe the bottom 2, let’s think laterally”.

And they said “Focus on height & body position – it’s the teachable aspect”

 

13/ Sooo…now we have a mandate – get height down, and body position of tacklers bent, not upright. This is key, so let’s explore why it was identified. Heres’ the risk spectrum for TYPE OF HEAD CONTACT that injures the TACKLER. Look at the risk of head to head vs head to hip

14/ See that head to head contact injures the tackler once every 88 times (11.3/1000)? It’s 6.6 times more likely to cause head injury than head-to-hip contacts, and 22.6 times more likely than head to upper body. Key point: For safety, get heads out of one another’s “airspace”!

15/ In fact, when we group all head injuries as “high contacts” (legal, but above the sternum), & “low contacts” (below sternum), we see that the risk of head injury is 4.3 times higher when the tackler’s head is “higher”. Also, an upright tackler is 1.5 times more likely injured

16/ There’s another fascinating bit of data, which I won’t show in this thread, where there’s a kind of “game theory” involved, where it DOES NOT MATTER what the ball carrier’s body position is, the highest risk is always when the tackler is upright. So irrespective of what the BC is doing, we want to get the tackler bent at the waist.  For any tackler position, the risk is highest when the tackler is upright, so that’s the situation to avoid.

17/ So this is the status – how to get tackler lower? Again, this is not my job as a scientist, it’s the job of an expert – coach, player, referee. But the intent is clear – less of what you see on top (both heads in danger), more of what you see at the bottom (less risk to BOTH)

18/ Here are gifs of those same tackles (I know they’re not identical in context, but the principle matters). First, a tackler who is upright, heads share airspace, alignment poor, both heads in danger (especially tackler’s) and the result is an HIA. This is the high RISK situation

19/ Here is the desired alternative – it’s not a huge change, but the tackler’s head is lower, his alignment is better, & as a result, his head is in less danger (remember that risk spectrum?) and the tackler’s head is almost totally out of danger (aside from whiplash or ground)

20/ So again, we ask the experts “How do we get the tackler lower?”. And we discuss this for hours, lively debate. Some people think it was done flippantly, I can assure you it wasn’t. Every option was considered. Broadly we came up with three phases (hypotheses), as shown here:

21/ Phase 1 is to more strictly apply existing laws. That would tell players “Go lower, or be sanctioned harshly”. It’s a stick approach. But making the stick bigger & using it more often. The principle is to drive a desired behavior – lower tackling – by punishing the undesired behaviour

22/ Unfortunately, a lot of people in the media and even in rugby didn’t quite get that this approach would protect BOTH the tackler and ball carrier. They thought it was only the ball carrier (because that’s how law is written). This ended up ‘undermining’ the principle a bit.

23/ But this is what it looked like – apply laws more often – more penalties, more yellow cards, different behaviors, lower tackle risk. This all hinged on referees actually applying the sanctions proposed. It’s like taking the medicine – it may taste bad, but it’s good for you

24/ Your next question – did they do it? It was mine too! My next job was to track the high tackle pens & cards over the 2 seasons since the directive & law re-inforcement. Ideally, we’d assess ref behavior in totality – what they give, what they miss.  We would also assess actual player behaviour – that’s step three that you can see above.  The plan is to do this in 2019, after the next Rugby World Cup, which will allow us to compare one four year cycle to another, and see whether player behaviour has changed.  But for now, we can assess part of the picture – the penalties and cards that are given

25/ So, what we find? In year 1, globally, there’s a 64% increase in high tackle penalty rate, and a 41% increase in yellow cards for high tackles. As expected. So far so good. Problem is, the yellow cards didn’t increase everywhere. In one country, it went DOWN. There was also high variance between competitions in terms of increase – some increased by 30%, some by 150%.

26/ The biggest issue though, is that this “stick” (or medicine – pick your metaphor!) wasn’t being applied often enough, nor was it strong enough. Even AFTER the change, a penalty every match, and yellow every 9 matches doesn’t look like enough to drive behavior.

27/ Next step – LATEST season data suggests continued increase in sanction, but crucial fine print – we see penalties continue to increase by 76% (good), and yellow cards for high tackles went up 13% in 2017/18 vs 2016/17. Result: yellow is given every 11.6 high tackle penalties

28/ That penalty to card ratio is a really crucial metric, because it tells you how referees perceive *severity*. In other words, they have seen and punished the high tackle, now must decide whether to card it. The 11.6 is actually higher than BEFORE the high tackle directive.

29/ So, what this means is that referees are seeing and penalizing high tackles more often, but giving yellow cards RELATIVELY less often. It suggests compromise, which is to an extent human nature. The media backlash & undermining by perception in some places didn’t help this.  Of course, one possibility is that the types of high tackles are just less serious, and that players have changed their bevaiour and so the referees are correctly awarding high tackles without giving penalties.  I’d be very surprised if this is happening, given that there’s been a 76% increase in high tackle penalty – do we really believe that penalty numbers have almost doubled, and that the new penalties, the ones that didn’t exist before, are almost exclusively the less severe ones?  Or is it more likely that high tackles have increased, many of which should be carded, but are not?  Certainly, the latter seems more likely to me, but we won’t know this until we redo the analysis, or independently verify the referee decisions (which we are doing, by the way, and have some data suggesting that it’s the latter of the two options, but I can’t share that incomplete data just yet).

30/ So, our next step was consultation. We consulted again with the expert group, we spoke to referees and other coaches, and we got referee input (as crucial stakeholders – they’re the people who have to “pull the lever” we identified previously, after all…!

31/ The message we got then is that on field sanctioning and carding is asking a great deal of a ref given the speed of the game and the impact of those cards on matches. As a result, we went BACK to that expert group’s suggestions, and looked at Phase 2 – the High Tackle Warning

32/ The High Tackle Warning is a modified sanction – instead of being ON field, and punishing illegal behavior, it moves sanction OFF field, punishes high RISK behavior, and does so with a large educational component. It would be used more often, but with less severe impact

33/ Let me explain how it works – remember that we identified 2 key risk factors?
1) Head to head/shoulder contact for tacklers;
2) Upright tacklersThose are the 2 risk behaviors the HTW aims to sanction post-match. So there are only 2 questions to be asked, as shown below

34/ Let’s look at these 3 cases, one by one. In the first, the tackler is clearly upright. There is head contact, but it’s initiated by the BC ducking. So, there is NO HTW because there’s no clear head to head or head to shoulder contact. BC-iniated contact when ducking is key

35/ Next one. You’ve already seen it. The tackler is upright, there IS clear head contact, so it’s an easy call – HTW IS ISSUED. A key point – this tackler CAN choose to go high, and he’s fine as long as there’s no head contact. It puts the onus on tackers to take responsibility.  It says “Tackle upright and high if you want to stop ball movement, but if you err and there is head contact, even if it’s YOUR head and their shoulder or head then you will be liable for sanction”

36/ And finally, two tacklers here. The first guy is fine – he’s bent, no HTW. The second is upright, there IS head to head contact, so he too would be ISSUED WITH AN HTW. The message is this: If you are high & upright, and strike a high, there’s a post-match sanction.

37/ So, let me stress, at present there is NO HTW in the sport. This is a concept we are exploring and studying right now. It was however used at the U20 World Champs this year, and it was a 2 person, 3 step process, summarized in this slide at the bottom:

38/ At the U20 World Champs, this was used. Did it work? I’d love to say a loud “YES”, and in truth, I’m encouraged by the result – we had fewer concussions than in any tournament since the HIA was first used. This graph shows incidence of concussion per 1000 hours (y-axis) and the actual number of concussions in 30 matches on the graph.

39/ However, much as I’d like to “claim it”, it would be scientifically dishonest to do so – the tournament is only 30 matches, there are so few concussions, and so what you see in the above figure this may be normal variation. It’s encouraging, but it’s not proof. We need more data before we truly know.

40/ What is interesting, and our *thinking* at this time, is that the educational aspect around the HTW matters more than the actual HTW. Before the tournament, this concept was explained to every player, coach & referee. The message is delivered THROUGH the HTW & then repeated often

41/ In the U20 tournament, in 30 matches, 11 HTWs were given. Every one comes with the re-inforcement of the “upright, high contact is bad” message, and that is the most crucial thing, I think. I think (and this is my opinion again) that the crucial barrier has been communication, and reaching all the coaches and players who are ultimately responsible for changing their behaviour.  If I’m talking “levers”, then law change was the lever we wished to pull, the referees were the ones asked to do the pulling, but ultimately, it’s the behaviour of players and coaches that would be ‘moved’ by that lever.  The education aspect of the HTW is a way to get the message those people more frequently, and I hope more effectively, and what’s where the promise of this system lies.  Will it be enough?  That’s the question for the future.

42/ The premise of the HTW, by the way, was that if a player got 3 HTWs, they’d serve a one-match ban. But the true value might be that in giving them to player A, players B through Z hear the same message, over & over, constantly reinforced. We are exploring how best to do this.

43/ And finally, Phase 3 is a lowering of the height of legal tackles from the current shoulder line to the armpit/nipple. This was done in the U20 trophy, and the RFU are doing it in their Cup competition. It’s too early to even say what this is showing, so I won’t speculate on what that may show.

44/ What I WILL say is the main outcome of these height trials (for me, anyway), is logistical or practical. Can referees enforce the change? Does lowering the legal height create massive inconsistency in their decision-making (this is already already a challenge – if we increase the number of decisions the referee has to make by 50 tackles a match because the line is lower, would we see 20 controversial calls rather than 5?)

We discussed this option with our experts & all I’ll say for now is that there’s a reason it was suggested as the third phase out of the three!

45/ When you lower legal height, then any situation of a BC ducking (like the one I showed earlier, and this one below) creates a dilemma for the ref, because they’re all “high”, but they’re ball carrier initiated high contacts.  So now the referee has to make a judgment call, presumably on the degree of force used by the tackler in combination with where the tackler’s arm or shoulder strikes the ball carrier.  It might end up looking like that tackler has to jump out of the way, and this would be untenable to every player and coach.  This might be “too much change”, going back to the concept I introduced in tweets 2 and 3!

46/ So that’s it. My talk, in tweets! Sorry for the length. The key point is this: How do you change without change? Sometimes, what looks good in theory fails because the people who must implement can’t go as far as they need to. We are navigating this issue all the time.

47/ Sometimes we fail. We go too far, or not far enough. We fail to communicate the process & intent. I get that. We deal with humans. But what I hope you can tell is that we’re trying to change it without changing *it* (the ‘DNA’), and we’re doing it systematically.