Model, model on the wall, who's the most Immortal of them all?

Quantitatively rating the best NRL players 1980-2019

I recently wrote a blog post about the greatest AFL players of all time and used RAPM - a statistical method used in hockey and basketball to isolate player impact - to quantitatively rate every player who has played since the birth of the game in 1897. Somewhat surprisingly, the model's number one ranked player was Chris Judd who represented West Coast and Carlton in the 2000s, won a premiership, two Brownlow Medals and retired in 2015. There were plenty of quirks in the ratings - RAPM isn't great at isolating individual player impact in large teams - but overall I was pretty happy with the analytical experiment as it allowed cross-era comparisons going back beyond the modern era. I'm a bit of an NRL fan as well these days, and the source of my data, AFL Tables, has a smaller, less comprehensive equivalent, Rugby League Tables, so I figured it would be an equally interesting endeavour to re-run the same analysis on rugby league data.

Unlike the AFL however, it seems there is no publicly available record of team lists for rugby league games prior to 1980. This is a bit surprising, as 1980 wasn't that long ago. I've come to realise that rugby league fans are generally not as interested in statistics about their sport as AFL fans. This lack of interest is reflected in less historical data and an online analytics community that is miles behind the AFL. For most people, that's probably fine. As a fan of the game, you can appreciate the artistry of the play-makers, the brutality of the forwards, the multi-faceted coordination of a strong defence, etc without needing to read a bunch of count statistics such as missed tackles, line-breaks, and run metres. Often these statistics that are measured have little value, and may even be misleading. Errors are a good example. Good teams tend to commit more errors than poor teams. This apparent contradiction isn't because errors are good, but more likely reflects that good teams are more aggressive and take more risks, which leads to more errors (in addition to more tries). Canterbury Bulldogs are the perfect illustration of this. They have maintained one of the highest completion rates in the league in recent years, yet their offensive play is uninspiring and they regularly struggle to put up a decent score. Yet commentators continue to bang on about completion rates ad nauseum. So on one level I can understand the lack of interest in rugby league statistics. But sports statistics can be a great way to discover new things about the game, particularly if one doesn't have the nuanced understanding of the game that comes from watching it for decades . The winners of the recent NFL Big Data Bowl, completed groundbreaking reserach into NFL, yet they were European and had little to no knowledge of the game. In any case, the lack of team lists prior to 1980, makes an analysis on player impact over the full history of rugby league impossible. So I'll change the time-frame to be 1980-present, and unless pre-1980 team lists become available in the future, we can only guess how RAPM would've rated Dally Messenger, Clive Churchill and other greats of yesteryear.

A quick, non-technical recap of how RAPM works (one can't assume NRL fans can be bothered reading an AFL article): RAPM is a statistical method that attempts to isolate the impact of each player to their team using only team lists and team scores. It does this by comparing how the team did with them on the court (in an NRL context, with them playing) versus off the court (not playing), and then adjusting for the strength of their opponents and teammates. Players who play on winning teams are given credit for success, particularly if the team does worse without them. The converse holds for players on weak teams. Additionally, RAPM takes some convincing that a good player is especially good, or that a bad player is especially bad, so will not award large absolute impact numbers to a player without a substantial sample. A noted flaw of RAPM is that it is a noisy method and some ratings may be implausible or even wrong, if players switch teams at lucky moments (NBA example: Robert Horry, doubtless a good player, who won seven championship rings with three different teams, including two with Houston when Jordan was playing baseball and three with the Kobe-led Lakers). This proved problematic for AFL, where there are 22 players on each team (basketball by comparison has only five) and the isolating out the individual effects proved tricky. I'm hoping that with only 17 players per team in rugby league, the RAPM results are a little more robust.

As with AFL, I standardise the game scores so that they are the on the same scale as 2019 scoring (mean and standard deviation). This is necessary to account of prior eras having higher (or lower) rates of scoring which might inflate (deflate) the ratings. Okay. Onto the results...

Right off the bat, these ratings look much better than the AFL results. Andrew Johns and Cameron Smith are miles ahead of the pack, albeit along different planes. I'm a bit of a fan of this. Smith has long been known for his reliability in defence and the Storm have been near the top of the defensive power rankings for much of his career. Johns is regarded by many as the GOAT and his offensive rating reflects this. Other legends of the modern game feature prominently in the top right quadrant, including Cooper Cronk, Johnathan Thurston, Greg Inglis and Darren Lockyer. How about the NRL immortals? Well, most of them played prior to 1980, and thus can't be evaluated using RAPM, but Johns is 2nd behind Smith (himself a likely future immortal) with a rating of +3.3 points and Meninga ranks 58th with a rating of +1.6 points, which sounds low, but still places him in the 98th percentile.

How about the best players at each position?

I'm not able to qualitatively sense-check these results with the same degree of confidence I could for AFL, but there are a couple points I feel are worth additional commentary:

• Luke Patten ranking ahead of Greg Inglis feels like a gross mistake. Not only did GI score 50 more tries in fewer games, he also represented Queensland in 32 State of Origin Games and Australia in 39 Internationals. Patten by contrast, never played representative rugby league. Patten was lucky to play in successful teams during his career though, and given this is a noted flaw of RAPM, his rating is perhaps comprehensible, if not entirely credible.

• Billy Slater (+1.7, 99th percentile) rates very highly, but narrowly misses out on a top five ranking at fullback. This seems like a mistake.

• Danny Buderus ranking 4th greatest may be a touch high. It's likely that he benefits from playing the vast majority of his career with Andrew Johns at halfback and may be receiving some of the credit for Johns' brilliance (this would also explain why Johns isn't closer to or ahead of Smith).

• Sonny Bill notwithstanding, the second row top five seems a little suspect. Notable second-rowers such as Gordon Tallis (+1.3, 97th), Steve Menzies (+1.5, 98th), Nathan Hindmarsh (+1.5, 98th) and Sam Thaiday (+1.4, 97th) all rate very highly, just not quite in the top five.

• Front-rowers Shane Webcke (+0.5, 86th) and Petero Civoniceva (+0.1, 66th) seem to be very hard done by and probably deserve to be near the top of the prop rankings.

• Cronk > Thurston? This may be a little controversial, but I think I'm okay with it. Cronk probably has the edge in game management, particularly defensively. Finishing his career with a 'three-peat' not bad either!

A final point is that these ratings represent career averages. Players who retire early, near their peaks, will, holding all else constant, be rated favourably to those who soldier on admirably into their late thirties. And, depending on how the Melbourne Storm fare in this season (and possibly the next if he plays on), this may cause Smith to ultimately drop down to second. For now though, All Hail King Cameron!