Last weekend’s predictions fared a lot better than Round 1. From 1/7 we’ve gone to 4/7. That means by extrapolation this weekend we should get 100%…

Here’s how last weekend’s games happened:

This week the Superbru fan analysis turns up a few interesting results. As expected, there’s a healthy dose of fan bias, but it’s not unanimous: Rebels and Lions fans are actually expecting their teams to lose this weekend.

Here are the Rugby Scientists’ prediction, triangulated from predictions made from past seasons points tallies, previous game analyses, and adjusted Superbru forecasts. The forecasts make it look like we’re in for a weekend of close games, although for three of the games the margin, form, and Superbru predictions coincide: Hurricanes to beat the Force, Sharks to beat the Bulls, and Stormers to beat the Lions, all away.

Dr Ian Durbach

(Data supplied by Superbru)

 

 

Following on from last week, here is an update from Dr Ian Durbach! Thanks again to Ian for putting the article together and for the data from the guys at Superbru…

Dark days for the Rugby Scientists prediction team this past weekend. In case you missed it, here’s how our predictions fared. Games we got correct are in bold but you won’t need to search too hard – there’s only one.

There are always going to be some upsets that predictions get wrong, that’s what makes sport exciting to watch. Results like the Rebels win were totally unexpected. Only 2% of Superbru users got that one right, die-hard Rebels fans we’d suspect. Models get their power by being right on average, so we’re hoping ours will bounce back. The fact that the away teams won 6 of the 7 games is something to watch, particularly as home ground advantage plays a strong role in our predictions.

This week we’re adding two new elements to our model. Firstly, the good folks at Superbru have provided us not just with the overall prediction given by their users, but how those predictions differ between fans of the teams involved, and neutrals. From some of our previous research we know that fans tend to overestimate their team’s prospects by about 5 points. That means we can adjust for that bias, as well as the home ground bias we talked about last week, before making our own predictions.

Here’s what Superbru fans expect to happen this weekend. Take a look at how different the forecasts of the two sets of fans are. Nothing like some good old-fashioned fan bias!

In addition to the new fan predictions, we’ve now got some game results that we can use to take into account a team’s current form. The idea of “form” suggests that teams that win in one week will, on average, tend to keep on winning. Our previous research has suggested that for every 10 points a team wins by in one week, they tend to, on average, win by 2 points the next week. In other words, scoring 10 points this week is “worth” 2 points the next week. Take the upcoming game between the Chiefs and the Brumbies. Both teams won last weekend, the Chiefs by 5, the Brumbies by 44. In making predictions for this week, the Chiefs 5 points are now “worth” 1 point, the Brumbies 44 points are “worth” 9 points. So with just this information we’d predict the Brumbies to beat the Chiefs by 8 points. Of course this is a pretty crude way of making a prediction, but we’ll add it to our existing model and hope it improves our performance!

To recap, our predictions are now made up of three components: predictions made from data on how teams did in previous seasons, which gives an idea of overall team quality; predictions made from recent performances, which tells us about current form, and Superbru predictions, which captures other kinds of information like team selections, injuries, and so on. Our final prediction is an average of these three predictions, let’s hope it does a little better than last week!

What are your thoughts on our predictions? Are we better off tossing a coin or is there some value in the predictive model proposed in this article?

With SuperRugby beginning again this weekend, we’ve managed to get a true scientist, Dr Ian Durbach, to comment on the ability to predict outcomes. Ian holds joint positions as adjunct senior lecturer in the Department of Statistical Sciences at the University of Cape Town and a researcher at the African Institute for Mathematical Sciences. His research interests are human judgment and decision making, particularly the way we reason when our choices involve risk or uncertainty. Quite simply, he’s a really clever guy.

When we asked Ian if we could apply science to predicting game outcomes so that we could be consistently correct as rugby experts such as Nick Mallett, he came up with a plan. Ian explains in his own words:

Picture from supersport.com “Dont miss out, warns Mallett”

The model we’re using is cobbled together from a variety of sources, and uses a few relationships we’ve been able to establish by looking at past performance and past predictions.

First, we looked at past 4 years of Super 15 rugby and worked out how many points each team won or lost by, on average over all the games they’ve played. The Crusaders come top – they have an average margin of +9, meaning they win by an average of 9 points. The table below shows the ranking, along with the average log points – you can see there’s a good but not perfect relationship between point margins and log points.

In our model, we’ve chosen to weight recent years more heavily, which also has the effect of sucking the values of all our predictions towards 0. Those values are in the third column of the table. This is actually a desirable feature to have whenever we know that the past is not a perfect predictor of the future – we’ll spare you the details!

Average points differences can be used to come up with a quick and dirty prediction. Take this weekend’s game between the Crusaders and the Rebels. We know the Crusaders win by 6 points on average. When they play the bottom teams they’ll tend to win by more, and when they play other top teams they’ll tend to win by less. The Rebels lose by 8 points on average. When they play a top team like the Crusaders, they’ll tend to lose by more than this. You can get a crude prediction by subtracting the two points differences from one another: the Crusaders are predicted to win by 6–(–8) = 14 points. That’s a start.

Next, we found that over a number of years, playing at home is “worth” about 4 points. That is, if you knew nothing at all about an upcoming game, you should predict the home team to win by 4 points. We know the Crusaders are playing at home, so we up the prediction in their favour, by 4 points. They’re now predicted to win by 18.

This is a data-driven prediction. But we know that rugby pundits are pretty good at predicting games. We’ve looked at some past predictions made by 80 000 users of the Superbru game. The players predicted the outcome of 73% of Super 15 games correctly, and were on average 10 points from the actual outcome of the game. Not bad! We found the following Superbru predictions for the upcoming week, which we’ve contrasted with the predictions from our data-driven approach. On the whole its amazing how similar the predictions are. We predict different outcomes for the Lions-Hurricanes and Blues-Chiefs games, but both sets of predictions are for the games to be close.

An obvious weakness of the data-driven approach is that it ignores a whole bunch of relevant current information – off-season transfers, injuries, form, and so on. Some of this we’ll work on building into later predictions, but for now we can assume that Superbru users are clued up on current rugby events, and use their predictions as proxies for that information. So we’ll make an additional prediction, which is a combination of our naïve data-driven model and the Superbru predictions.

Before we do that, an interesting fact is that Superbru users tend, on average, to slightly underestimate the effect of playing at home. They think it is worth around 2 points, while we know that it is actually worth about 4 points. So, if we know Superbru users think about a game, we should adjust their predictions by 2 points in favour of the home team before combining them. For now we’ll combine them in the simplest possible way, but taking the average of the two.

That’s all we’re going to do for now. Once we start getting some information in on how the teams are performing, we’ll use this information to adjust our model further. Let’s see how we do!

Mallet seems to think Chiefs will win when he referred to the team being “good again”. Who do you think will be proved correct? The experts (rugby and Superbru/couch) or the scientists?  We’d like to know your thoughts before the Blues vs Chiefs match kicks off. Tell us in the poll or feel free to comment!