RIP RPI

[Guapo]

https://deadspin.com/the-ncaa-will-use-a-new-metric-to-screw-your-team-out-o-1828530815

 

[pj]

NCAA won’t apply NET metric to previous yrs for comparison

Now THAT is ridiculous. No need to see if this is a quality metric via back testing. Just trust us...

They don't want us to realize for several more years that it gets more P5 teams into the tourney, with higher seedings.

 

[uc1990]

Don’t all these ratings start with a preseason ranking and then go from there? Which gives the teams/conferences that always get the top pre season rankings an advantage.

 

[auror]

Don’t all these ratings start with a preseason ranking and then go from there? Which gives the teams/conferences that always get the top pre season rankings an advantage.

Most phase out all pre-season impact at some point in the season. For this system, the results before/during the season don't matter in the slightest, only when time to select for NCAA tournament, so there's no real reason to even include anything.

 

[gtcam]

auror said: ↑
They [NCAA] heavily outsourced the research.
Click to expand...
Ahhhhhh there's the NCAA I know and love. Passing the buck like an old pro.

I get where your coming from BUT this might be for the best - nobody at the NCAA is smart enough to add 2 + 2 never mind work on something like this

 

[Ewekon]

very interested to see how this turns out.

i had made a few attempts at developing a machine learning model based on team stats and kenpom to build my brackets in prior years.

the main issue i foresee with this is machine learning methods (neural networks in particular) are usually a black box in that it can be hard to explain why the model made a specific decision. so when this new model leaves your team out of the tournament it may not give you a clear reason why.

and for those that are always looking for the anti-uconn angle in ncaa decision making, perhaps they trained the model with a bias against uconn, to ensure lower rankings in the future.

Your first few paragraphs made you look smart. The last confirmed brilliance.

 

[willie99]

Gamblers have been working on this for generations, with no luck. Math just doesn't provide a solution, no matter how many variables they throw in the mix

It should be about the eye test and a little common sense, but it will still be about what schools or conferences are represented on the committee AND spinning the numbers they use

 

[auror]

Gamblers have been working on this for generations, with no luck. Math just doesn't provide a solution, no matter how many variables they throw in the mix

It should be about the eye test and a little common sense, but it will still be about what schools or conferences are represented on the committee AND spinning the numbers they use

You're right in that the sample sizes are too small and variance in the game too large to be perfect.

But the Eye test is much more flawed than something like KenPom (or hopefully NET), just due to the inputs. Are you going to watch every possession of every team in the entire country? Even just the bubble is probably 80 teams * 34 games * 2 hours = 225 days worth of footage without sleeping. Further, are you going to be able to apply context of relative opponent strength in real time? Are you further able to prevent confirmation and other biases?

There's a reason successful gamblers have had "systems" without relying on the eye test.

 

[Paesano]

This is not perfect but it is progress. The RPI was awful.

 

[Scrutiner]

NCAA won’t apply NET metric to previous yrs for comparison

Now THAT is ridiculous. No need to see if this is a quality metric via back testing. Just trust us...



Ahhhhhh there's the NCAA I know and love. Passing the buck like an old pro.

The wording, "won't apply the NET metric to previous years for comparison" implies they didn't apply it.

Whereas my cynical mind tells me they probably did back test it, but won't allow the results to be seen and used for comparison because it would expose the fact that there's bias built into it.

 

[UConnNick]

Anything Mark Emmert is involved with cannot possibly be an improvement.

 

[weyuo]

" The new ranking system was approved in late July after months of consultation with the Division I Men’s Basketball Committee, the National Association of Basketball Coaches, top basketball analytics experts and Google Cloud Professional Services."

How the #$bleep%$ did Google sleaze it's way into how the NCAA determines it's National Championship bracket?

 

[auror]

" The new ranking system was approved in late July after months of consultation with the Division I Men’s Basketball Committee, the National Association of Basketball Coaches, top basketball analytics experts and Google Cloud Professional Services."

How the #$bleep%$ did Google sleaze it's way into how the NCAA determines it's National Championship bracket?

Google Cloud Named Official Cloud of the NCAA

I remember making fun of this at the time.

 

[nelsonmuntz]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

 

[pepband99]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

Just as many mid majors game the RPI to their advantage in scheduling - it's a 2-way street. I don't mind a little bit of mystery here, mostly because i would be shocked if this doesn't closely follow something like Pomeroy, with some added factors like the points cap. It won't be that hard to figure out.

 

[Purple Stein]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

The real question is whether it’s adaptive/frequently tweaked. We want to read bias into the system, and time will tell whether we’re justified!

 

[dennismenace]

Kitty: We are waiting.

 

[auror]

The real question is whether it’s adaptive/frequently tweaked. We want to read bias into the system, and time will tell whether we’re justified!

 

[QDOG5]

Just as many mid majors game the RPI to their advantage in scheduling - it's a 2-way street. I don't mind a little bit of mystery here, mostly because i would be shocked if this doesn't closely follow something like Pomeroy, with some added factors like the points cap. It won't be that hard to figure out.

Just wondering how the mid majors gamed the RPI?

 

[CL82]

Kitty: We are waiting.

Conspiracy Kitty says: Google is a responsible corporate citizen and a fine, fine company. I support them in whatever the do.



[Whispers] Shhh - they are everywhere. Why do you think I wear the foil?

 

[raf46]

This could mean that coaches will tend to keep their starters in longer just to ensure a 10+ point victory. If you think back to the National Championship game in 2004, its crazy to think the final margin was 9 points. I think at some points we were up by almost 30.

yes we were up by 30. maybe a little more!

 

[auror]

Just wondering how the mid majors gamed the RPI?

Famously the Missouri Valley in 2006.

The have-nots play the RPI game, too. The Missouri Valley Conference famously instituted a scheduling mandate a decade ago to improve its overall RPI and, with it, the chances of receiving at-large berths in the NCAA Tournament (and the accompanying seven-figure payouts). Schools risked financial penalties if they didn’t comply.

The MVC rose from a one-bid league to receiving four in 2006, two of which reached the Sweet 16. The scheduling mandates were removed for unrelated reasons, and the MVC became a one-bid league again. But 75 percent of the conference’s revenue is generated from NCAA Tournament payouts, and those payouts are projected to drop from $5 million to $2 million. The scheduling mandates are back.

Post article from 2006

Elgin decided to make a change in early 2000 after seeing the Valley's nonconference strength-of-schedule ranking drop from eighth to 25th. The conference decided to withhold an annual $50,000 NCAA tournament distribution from programs that did not play a nonconference schedule consisting of opponents with a three-year average RPI of 149 or better. Two teams, Elgin said, did not make the cut. But Elgin ditched the policy after two years, and now the league office works in conjunction with teams to assemble nonconference schedules.

Consider the top six teams in the Valley and the ACC, all potential NCAA tournament teams. The ACC group played 43 games against top 50 competition, including 13 opponents outside the ACC. The Valley group played 68 games against top 50 competition, but only eight opponents outside the Valley. In games against top 50 teams outside their conference, the Valley group is 4-4 and the ACC group is 9-4.

 

[Chin Diesel]

So, instead of wanking over playing opponents with low RPI's, we're back to complaining that our opponents just suck?

 

[QDOG5]

That's some good stuff @auror. I graduated from Bradley and didn't know the MVC got 4 teams in the tourney in 05-06. Bradley had a surprise run to the sweet 16 behind 7 footer Patrick O'Bryant. He's near the top of the list of guys who should have stayed in college. Never sniffed the NBA. Eustachy says he got the idea on how to beat the RPI from Cal (surprising) but the articles don't flesh out the methodology. From what I gather the entire conference schedules tough ooc away games before the conference season and hopefully a team or two are lucky to get a win. Then they trade wins and losses in conference after that and the top tier of the conference keep their high RPI. Am I on the right path with my analysis. Also, those articles are quite dated and I'm guessing the major conferences figured out how to thwart them by not playing them early in the season.

 

[auror]

That's some good stuff @auror. I graduated from Bradley and didn't know the MVC got 4 teams in the tourney in 05-06. Bradley had a surprise run to the sweet 16 behind 7 footer Patrick O'Bryant. He's near the top of the list of guys who should have stayed in college. Never sniffed the NBA. Eustachy says he got the idea on how to beat the RPI from Cal (surprising) but the articles don't flesh out the methodology. From what I gather the entire conference schedules tough ooc away games before the conference season and hopefully a team or two are lucky to get a win. Then they trade wins and losses in conference after that and the top tier of the conference keep their high RPI. Am I on the right path with my analysis. Also, those articles are quite dated and I'm guessing the major conferences figured out how to thwart them by not playing them early in the season.

Yeah the important thing was to schedule tough road games, win a couple if possible, avoid scheduling the real bottom of the barrel teams, try to schedule teams that will get a lot of wins but won't necessarily but good enough to beat you, and then win ALL of your home games. And then in conference hope as many home teams win. The RPI was very screwed up with home/road weighting.

But the vortex thing is kinda key for conference-wide system hacking. You need a couple actual good teams in your conference to beat other good teams, and then you need your other teams to schedule so they don't lose home games to mediocre/bad teams in nonconf and then have a couple of your above average teams beat the actual good teams which makes them then look like good teams themselves. It needs a bit of a perfect storm to work, but then it's self-sustaining.

Luke Winn: Scheduleball: Colorado State, Pitt exploit weaknesses of RPI