RPI, SOS, Rankings, Good wins, Bad losses and conferences

[UcMiami]

In a thread this past week, someone posted the question of why there was such hatred for the SEC as a conference and it got me thinking about how teams get ranked and the selection process. I though I would take a stab at addressing my feelings which I think may be fairly representative. And it is current since we are mostly just waiting for the committee to do it's work and announce the seedings.

Most here are Uconn fanatics and by extension supporters of the old BE and now the ACC. And one of the things I have always liked about these two conferences is that their conference records reflected the quality of the teams in the conference. Uconn is dominant and you can count on one hand the number of losses to inferior teams in the last decade, but that character carries through to the top half of the two leagues as a whole. ND lost to Uconn and no one else, Louisville lost to Uconn, ND and no one else, Rutgers lost to the top three and no one else, Depaul, St Johns, WV played to their standards. In the current AAC the same is happening - USF takes care of their business, followed by Temple, and Memphis and Tulane. And in OOC they play real road games and schedules that are commensurate with their talent level and challenge them. So their OOC record is very similar to their IC record - you don't have bad teams coming into conference with 12-2 records and then going 4-12 in conference. The record OOC is as 'honest' as the record IC. And that carries down from the top to the very bottom of the league.

Those two qualities - consistency of performance and consistency of scheduling are rare in WCBB and stand out as rarer within conferences. If you look at the records IC vs. OOC of all of the major conferences down to the A10, the A10 is the only one that has the same kind of 'honesty' within those results. And if you look at the actual results in conference you see huge holes in the consistency within the conference hierarchy.

And that honesty and consistency is important when it comes to the committee work this week, both in selecting schools and in seeding them because they use 'dumb' data to evaluate - RPI and SOS, and they use a criteria of good wins and bad losses that is also based on that dumb data. And the RPI and SOS numbers are corrupted by the 'dishonesty' of the records. And the good wins and losses are corrupted by the inconsistency of performance.

Specific to the SEC which for a number of years has had very 'dishonest' scheduling and results, and very inconsistent performance within conference play, it makes a mockery of the data - teams come into league play with 80+ winning percentages and then struggle to play .500 in conference. Teams like KY which has flirted all year with a top ten ranking and will get a top 16 seed lose to not just unranked conference mates but to bad Vanderbilt and terrible Ole Miss teams. And the result of that is a pretty bad Auburn team suddenly has a great win on its record and will get into the NCAA because of it - this is a team that played a terrible OOC and still only went only 10-3 with losses to UVA and Marquette but was able to beat Florida, Missouri, and KY (all with inflated OOC records) because those teams are so inconsistent.

And if you look at the Pac12, unfortunately it is in a similar situation this year with a lot of 'dishonest' records that boosted their RPI to #1, and a fair amount of inconsistency.

People point to the quality of the middle of the conference as the excuse for 'inconsistency' in conference play. I point to the inconsistency and the dishonest scheduling and see it as the reason the middle of the conference looks a lot better than it really is. Any team can throw a wobble or two into a regular season, but KY for example threw in four between Jan 3 and Jan 31.

I may post something else on RPI and SOS and what makes them 'dumb' but the above is I think why a lot of people have disdain for the SEC specifically and why they question other conferences as well.

 

[Bgillon]

In a thread this past week, someone posted the question of why there was such hatred for the SEC as a conference and it got me thinking about how teams get ranked and the selection process. I though I would take a stab at addressing my feelings which I think may be fairly representative. And it is current since we are mostly just waiting for the committee to do it's work and announce the seedings.

Most here are Uconn fanatics and by extension supporters of the old BE and now the ACC. And one of the things I have always liked about these two conferences is that their conference records reflected the quality of the teams in the conference. Uconn is dominant and you can count on one hand the number of losses to inferior teams in the last decade, but that character carries through to the top half of the two leagues as a whole. ND lost to Uconn and no one else, Louisville lost to Uconn, ND and no one else, Rutgers lost to the top three and no one else, Depaul, St Johns, WV played to their standards. In the current AAC the same is happening - USF takes care of their business, followed by Temple, and Memphis and Tulane. And in OOC they play real road games and schedules that are commensurate with their talent level and challenge them. So their OOC record is very similar to their IC record - you don't have bad teams coming into conference with 12-2 records and then going 4-12 in conference. The record OOC is as 'honest' as the record IC. And that carries down from the top to the very bottom of the league.

Those two qualities - consistency of performance and consistency of scheduling are rare in WCBB and stand out as rarer within conferences. If you look at the records IC vs. OOC of all of the major conferences down to the A10, the A10 is the only one that has the same kind of 'honesty' within those results. And if you look at the actual results in conference you see huge holes in the consistency within the conference hierarchy.

And that honesty and consistency is important when it comes to the committee work this week, both in selecting schools and in seeding them because they use 'dumb' data to evaluate - RPI and SOS, and they use a criteria of good wins and bad losses that is also based on that dumb data. And the RPI and SOS numbers are corrupted by the 'dishonesty' of the records. And the good wins and losses are corrupted by the inconsistency of performance.

Specific to the SEC which for a number of years has had very 'dishonest' scheduling and results, and very inconsistent performance within conference play, it makes a mockery of the data - teams come into league play with 80+ winning percentages and then struggle to play .500 in conference. Teams like KY which has flirted all year with a top ten ranking and will get a top 16 seed lose to not just unranked conference mates but to bad Vanderbilt and terrible Ole Miss teams. And the result of that is a pretty bad Auburn team suddenly has a great win on its record and will get into the NCAA because of it - this is a team that played a terrible OOC and still only went only 10-3 with losses to UVA and Marquette but was able to beat Florida, Missouri, and KY (all with inflated OOC records) because those teams are so inconsistent.

And if you look at the Pac12, unfortunately it is in a similar situation this year with a lot of 'dishonest' records that boosted their RPI to #1, and a fair amount of inconsistency.

People point to the quality of the middle of the conference as the excuse for 'inconsistency' in conference play. I point to the inconsistency and the dishonest scheduling and see it as the reason the middle of the conference looks a lot better than it really is. Any team can throw a wobble or two into a regular season, but KY for example threw in four between Jan 3 and Jan 31.

I may post something else on RPI and SOS and what makes them 'dumb' but the above is I think why a lot of people have disdain for the SEC specifically and why they question other conferences as well.

Wow, very thoughtful analysis. I would love to see you follow this up with an analysis of the SEC teams' OOC ranking. I mention this rather than try to do more leagues. Then you could draw conclusions and thus validate your methodology.

 

[HGN]

In a thread this past week, someone posted the question of why there was such hatred for the SEC as a conference and it got me thinking about how teams get ranked and the selection process. I though I would take a stab at addressing my feelings which I think may be fairly representative. And it is current since we are mostly just waiting for the committee to do it's work and announce the seedings.

Most here are Uconn fanatics and by extension supporters of the old BE and now the ACC. And one of the things I have always liked about these two conferences is that their conference records reflected the quality of the teams in the conference. Uconn is dominant and you can count on one hand the number of losses to inferior teams in the last decade, but that character carries through to the top half of the two leagues as a whole. ND lost to Uconn and no one else, Louisville lost to Uconn, ND and no one else, Rutgers lost to the top three and no one else, Depaul, St Johns, WV played to their standards. In the current AAC the same is happening - USF takes care of their business, followed by Temple, and Memphis and Tulane. And in OOC they play real road games and schedules that are commensurate with their talent level and challenge them. So their OOC record is very similar to their IC record - you don't have bad teams coming into conference with 12-2 records and then going 4-12 in conference. The record OOC is as 'honest' as the record IC. And that carries down from the top to the very bottom of the league.

Those two qualities - consistency of performance and consistency of scheduling are rare in WCBB and stand out as rarer within conferences. If you look at the records IC vs. OOC of all of the major conferences down to the A10, the A10 is the only one that has the same kind of 'honesty' within those results. And if you look at the actual results in conference you see huge holes in the consistency within the conference hierarchy.

And that honesty and consistency is important when it comes to the committee work this week, both in selecting schools and in seeding them because they use 'dumb' data to evaluate - RPI and SOS, and they use a criteria of good wins and bad losses that is also based on that dumb data. And the RPI and SOS numbers are corrupted by the 'dishonesty' of the records. And the good wins and losses are corrupted by the inconsistency of performance.

Specific to the SEC which for a number of years has had very 'dishonest' scheduling and results, and very inconsistent performance within conference play, it makes a mockery of the data - teams come into league play with 80+ winning percentages and then struggle to play .500 in conference. Teams like KY which has flirted all year with a top ten ranking and will get a top 16 seed lose to not just unranked conference mates but to bad Vanderbilt and terrible Ole Miss teams. And the result of that is a pretty bad Auburn team suddenly has a great win on its record and will get into the NCAA because of it - this is a team that played a terrible OOC and still only went only 10-3 with losses to UVA and Marquette but was able to beat Florida, Missouri, and KY (all with inflated OOC records) because those teams are so inconsistent.

And if you look at the Pac12, unfortunately it is in a similar situation this year with a lot of 'dishonest' records that boosted their RPI to #1, and a fair amount of inconsistency.

People point to the quality of the middle of the conference as the excuse for 'inconsistency' in conference play. I point to the inconsistency and the dishonest scheduling and see it as the reason the middle of the conference looks a lot better than it really is. Any team can throw a wobble or two into a regular season, but KY for example threw in four between Jan 3 and Jan 31.

I may post something else on RPI and SOS and what makes them 'dumb' but the above is I think why a lot of people have disdain for the SEC specifically and why they question other conferences as well.

Miami , I don't think it's hatred. Hatred is too strong of a word. I believe it has to do with the fact that we associate the SEC with Tennessee. And most don't say hate UT , they just don't love'em. The reason goes way back. But that's a story for another day.

 

[EricLA]

Miami , I don't think it's hatred. Hatred is too strong of a word. I believe it has to do with the fact that we associate the SEC with Tennessee. And most don't say hate UT , they just don't love'em. The reason goes way back. But that's a story for another day.

Au contrere. I actually like Texas (UT). I hate Tennessee. HATE. Yep I said it. More than the Yankees, more than the Lakers, more than the Pats, and more than the Duke or Kentucky men's teams (combined).

 

[j66kicker]

Au contrere. I actually like Texas (UT). I hate Tennessee. HATE. Yep I said it. More than the Yankees, more than the Lakers, more than the Pats, and more than the Duke or Kentucky men's teams (combined).

I hate Tennessee AND Texas (Oklahoma alum). Fortunately it's only one color to keep track of.

 

[EricLA]

I hate Tennessee AND Texas (Oklahoma alum). Fortunately it's only one color to keep track of.

True, but the Tennessee orange is kind of like that throw up in a pumpkin orange. It's not the kind of orange you can sit with. Just ask this Alabama fan! LOL...

 

[UcMiami]

Miami , I don't think it's hatred. Hatred is too strong of a word. I believe it has to do with the fact that we associate the SEC with Tennessee. And most don't say hate UT , they just don't love'em. The reason goes way back. But that's a story for another day.

HGN - agree, it is more disdain. And less for the teams than for the mythology of it being a 'super strong' league in WCBB that plays exceptional defense. And that ends up being abbreviated into 'hate'. For TN specifically 'hate' is not a bad word for many on this board.

Eric - I love that clip, watch it again every time I stumble across it!

 

[UcMiami]

OK - round II - SOS and RPI. These numbers are calculated in various ways by various entities and how the NCAA calculates the ones they use is I believe somewhat shrouded in mystery, but I call them 'dumb' in that they are all predicated on W/L records of teams and the teams they play and the teams they play. They generally do not use MOV nor do they distinguish relative strength - they treat Baylor beating the #100 ranked team as significantly better than when they beat the #350 team and losing to the #100 ranked team as significantly better than losing to the #350 ranked team. There are systems (Massey and Sagarin) that create multiple rankings that try to address some of this by adjusting values for quality of MOV (not raw MOV) and by decreasing the significance of clearly inferior or superior team's wins and losses. How those calculations are configured is also somewhat mysterious, but it generally improves the results. [This is the first year that I have been pretty unhappy with some of their results as it appears the vagaries of WCBB have confused some of their calculations.]

The issue with college sports is that however long the season is that the points of contact between teams OOC is minimal - FBS football has 120+ teams and each team plays about 6 games OOC so the chance of two teams from different conferences playing a common opponent is very low - teams don't even play every team in their conference as the conference schedule is 8 games typically. In basketball for both genders the number of D1 teams rises to 350, teams play around 14 games OOC, so the chance of a common opponent is again minimal. The limited schedules and huge number of teams make all college team sports very difficult to do competitive strength analysis on. In professional sports with many fewer teams and typically longer seasons you have head to head competition in everything but football, and in football you have multiple common opponents 'OOC'. This makes all types of statistical analysis of relative strengths much more meaningful because you have many more points of contact

And with WCBB specifically the distribution of quality is very far from a straight line - at the top end it drops rapidly and by about #100 it is almost a straight line out to #350. Any rating system that treats WCBB teams in a linear progression form 1 to 350 is a disaster, and both simple SOS and RPI systems do just that. [Men's basketball is a little better because every year the most talented teams have attrition of the best talent while the less talented teams retain their older and better coached talent. It still isn't great, but the quality drops more slowly from the top through #50, and the line continues to descend much more noticeably through from #50 through maybe #150. The back end is still a flatline.]

So specifics - there are a number of teams in P5 conferences and the BE who play poor OOC schedules for whatever reason (money, travel, or cynical calculation.) Money because they only play home games, travel because they don't leave their state or their region, and cynical because they are chasing an inflated W/L record or aiming for 20+ wins which used to be a magical ticket to the NCAA but has lost some of it's cache more recently.) For really good teams it doesn't screw up RPI and SOS calculations too much - they win 90+% of the games in OOC and proceed to do the same IC - it hurts the fan experience and probably the popularity of the sport but it doesn't mess to much with the committee work at NCAA time. Where the problem with SOS and RPI comes in is when decent teams play bad OOC schedules and when bad teams play even worse schedules. This in effect inflates their SOS which in turn inflates their future IC opponents RPI by making their W/L record look much better than it really should be. And this RPI effect just keeps getting multiplied as the teams play each other again and again. They either get the 50% bonus of an inflated opponent's W/L record or the 25% bonus of an inflated opponent's opponent W/L record with every game IC they play.

How this plays out in reality - the Pac12 has the highest conference RPI and compiled a 127-27 OOC record according to Massey - very impressive! Lets look inside:
1. They played 33 road games - an average of 2.75 per team. If you drop Washington state and their aberrational 6 road games out of that equation the rest of the league averaged 2.5 road games.

2. They played 18 currently ranked teams in total or an average of 1.5 per team. To be fair they probably expected TN (x2) and UNC to be ranked when the games were scheduled but that is partly offset by FL and WV who they probably didn't expect to be ranked.

3. They played 29 P5 teams of which 12 were 'also rans' in their conferences. An average of 2.4 per team.

So ... against teams ranked in the top ten they went 1-5 (Cal beat Louisville in the first game of the season) and against ranked teams 11-25 they went 4-8 for a grand total of 5 - 13. The Pac12 record against unranked teams was 122-11

Against P5 teams - their brethren - they went 4-13 against ranked P5 but a nice 12-2 against the unranked variety for a total record of 16-15. Against non P5 they went 111-12.

So the bottom line of their stellar conference RPI is that they beat up on a bunch of non-P5 teams, played P5 teams even and lost 62% of their games against ranked teams - Why does this matter to anyone? Because that RPI in OOC carries over to individual teams as they play their IC schedule and gets them better seedings in the NCAA and helps a bubble team get in. If Charlie is correct and I think he probably is they will have four teams hosting the first weekend. And their OOC resumes: Oregon State is 0-1 against ranked teams and 0-2 against NCAA bound teams; ASU is 2-2 against ranked teams, and hasn't played another NCAA bound team; UCLA is 0-2 against ranked teams and 2-2 against NCAA bound teams; Stanford is 1-1 against ranked teams and 2-1 against NCAA bound teams. What makes each of their resumes more appealing is they have beaten each other in conference play and they all have those nice RPI numbers. And part of the reason they beat each other is that few of them really challenged themselves OOC and none of them travelled much, so when they go on the road against decent teams they struggle. Stanford and OrSt split home and home, UCLA split home and homes with OrSt and ASU - makes everyone look good! (or bad!)

The solution - find a way to truly tailor RPI and SOS to WCBB - it is a unique competitive environment that you cannot just plug analytics developed for other competition into. Add a quality of MOV to the calculation so worse teams playing competitively against better teams get a bump and the better team gets a demerit (but blow outs vs 15 point wins are valued the same). Do away with any linear expectations or ratings of opponents and quantify wins and losses based on much narrower ranges - a top 10 team playing a 50th ranked team should be valued the exact same as playing a 150th ranked team - if you do that it creates a sliding scale so for the 20th ranked team the scale moves further away to the 75th or 100th vs the 175 or 200th. And playing the 10th ranked team is valued as highly as playing the 5th ranked team because both are 'aberrational'.
And change the rules for the seeding committee so bad scheduling OOC precludes favoritism in seeding, and please put a rule in place that a conference record of .500 or below disqualifies you from an at large bid. Give a little more love to the good mid majors and stop rewarding mediocrity.

 

[UcMiami]

Wow, very thoughtful analysis. I would love to see you follow this up with an analysis of the SEC teams' OOC ranking. I mention this rather than try to do more leagues. Then you could draw conclusions and thus validate your methodology.

Not sure if round two does it for you? I used the Pac12 and cheated in terms of analysis by using broad brush and not looking up every opponent's RPI value or W/L record.

 

[toadfoot]

Great read Uc. I've been arguing most of these same points for several years, but never had the patience to sit down and write up a thorough analysis as you've clearly done.

 

[Bgillon]

Not sure if round two does it for you? I used the Pac12 and cheated in terms of analysis by using broad brush and not looking up every opponent's RPI value or W/L record.

Superb work. A great example of why this board is the best board for WCBB!!!

 

[Gate81]

Think Mike Neighbors of WA cracked the RPI code and helped the whole conference figure out how to get better rankings...so it's the system, not those who figure out how it works.

 

[Plebe]

OK - round II - SOS and RPI. These numbers are calculated in various ways by various entities and how the NCAA calculates the ones they use is I believe somewhat shrouded in mystery, but I call them 'dumb' in that they are all predicated on W/L records of teams and the teams they play and the teams they play. They generally do not use MOV nor do they distinguish relative strength - they treat Baylor beating the #100 ranked team as significantly better than when they beat the #350 team and losing to the #100 ranked team as significantly better than losing to the #350 ranked team. There are systems (Massey and Sagarin) that create multiple rankings that try to address some of this by adjusting values for quality of MOV (not raw MOV) and by decreasing the significance of clearly inferior or superior team's wins and losses. How those calculations are configured is also somewhat mysterious, but it generally improves the results. [This is the first year that I have been pretty unhappy with some of their results as it appears the vagaries of WCBB have confused some of their calculations.]

The issue with college sports is that however long the season is that the points of contact between teams OOC is minimal - FBS football has 120+ teams and each team plays about 6 games OOC so the chance of two teams from different conferences playing a common opponent is very low - teams don't even play every team in their conference as the conference schedule is 8 games typically. In basketball for both genders the number of D1 teams rises to 350, teams play around 14 games OOC, so the chance of a common opponent is again minimal. The limited schedules and huge number of teams make all college team sports very difficult to do competitive strength analysis on. In professional sports with many fewer teams and typically longer seasons you have head to head competition in everything but football, and in football you have multiple common opponents 'OOC'. This makes all types of statistical analysis of relative strengths much more meaningful because you have many more points of contact

And with WCBB specifically the distribution of quality is very far from a straight line - at the top end it drops rapidly and by about #100 it is almost a straight line out to #350. Any rating system that treats WCBB teams in a linear progression form 1 to 350 is a disaster, and both simple SOS and RPI systems do just that. [Men's basketball is a little better because every year the most talented teams have attrition of the best talent while the less talented teams retain their older and better coached talent. It still isn't great, but the quality drops more slowly from the top through #50, and the line continues to descend much more noticeably through from #50 through maybe #150. The back end is still a flatline.]

So specifics - there are a number of teams in P5 conferences and the BE who play poor OOC schedules for whatever reason (money, travel, or cynical calculation.) Money because they only play home games, travel because they don't leave their state or their region, and cynical because they are chasing an inflated W/L record or aiming for 20+ wins which used to be a magical ticket to the NCAA but has lost some of it's cache more recently.) For really good teams it doesn't screw up RPI and SOS calculations too much - they win 90+% of the games in OOC and proceed to do the same IC - it hurts the fan experience and probably the popularity of the sport but it doesn't mess to much with the committee work at NCAA time. Where the problem with SOS and RPI comes in is when decent teams play bad OOC schedules and when bad teams play even worse schedules. This in effect inflates their SOS which in turn inflates their future IC opponents RPI by making their W/L record look much better than it really should be. And this RPI effect just keeps getting multiplied as the teams play each other again and again. They either get the 50% bonus of an inflated opponent's W/L record or the 25% bonus of an inflated opponent's opponent W/L record with every game IC they play.

How this plays out in reality - the Pac12 has the highest conference RPI and compiled a 127-27 OOC record according to Massey - very impressive! Lets look inside:
1. They played 33 road games - an average of 2.75 per team. If you drop Washington state and their aberrational 6 road games out of that equation the rest of the league averaged 2.5 road games.

2. They played 18 currently ranked teams in total or an average of 1.5 per team. To be fair they probably expected TN (x2) and UNC to be ranked when the games were scheduled but that is partly offset by FL and WV who they probably didn't expect to be ranked.

3. They played 29 P5 teams of which 12 were 'also rans' in their conferences. An average of 2.4 per team.

So ... against teams ranked in the top ten they went 1-5 (Cal beat Louisville in the first game of the season) and against ranked teams 11-25 they went 4-8 for a grand total of 5 - 13. The Pac12 record against unranked teams was 122-11

Against P5 teams - their brethren - they went 4-13 against ranked P5 but a nice 12-2 against the unranked variety for a total record of 16-15. Against non P5 they went 111-12.

So the bottom line of their stellar conference RPI is that they beat up on a bunch of non-P5 teams, played P5 teams even and lost 62% of their games against ranked teams - Why does this matter to anyone? Because that RPI in OOC carries over to individual teams as they play their IC schedule and gets them better seedings in the NCAA and helps a bubble team get in. If Charlie is correct and I think he probably is they will have four teams hosting the first weekend. And their OOC resumes: Oregon State is 0-1 against ranked teams and 0-2 against NCAA bound teams; ASU is 2-2 against ranked teams, and hasn't played another NCAA bound team; UCLA is 0-2 against ranked teams and 2-2 against NCAA bound teams; Stanford is 1-1 against ranked teams and 2-1 against NCAA bound teams. What makes each of their resumes more appealing is they have beaten each other in conference play and they all have those nice RPI numbers. And part of the reason they beat each other is that few of them really challenged themselves OOC and none of them travelled much, so when they go on the road against decent teams they struggle. Stanford and OrSt split home and home, UCLA split home and homes with OrSt and ASU - makes everyone look good! (or bad!)

The solution - find a way to truly tailor RPI and SOS to WCBB - it is a unique competitive environment that you cannot just plug analytics developed for other competition into. Add a quality of MOV to the calculation so worse teams playing competitively against better teams get a bump and the better team gets a demerit (but blow outs vs 15 point wins are valued the same). Do away with any linear expectations or ratings of opponents and quantify wins and losses based on much narrower ranges - a top 10 team playing a 50th ranked team should be valued the exact same as playing a 150th ranked team - if you do that it creates a sliding scale so for the 20th ranked team the scale moves further away to the 75th or 100th vs the 175 or 200th. And playing the 10th ranked team is valued as highly as playing the 5th ranked team because both are 'aberrational'.
And change the rules for the seeding committee so bad scheduling OOC precludes favoritism in seeding, and please put a rule in place that a conference record of .500 or below disqualifies you from an at large bid. Give a little more love to the good mid majors and stop rewarding mediocrity.

You did a great job, but in analyzing the Pac-12's nonconference results, it's a bit misleading to just cite overall records vs. "ranked" teams to avoid discussing which Pac-12 teams got quality OOC wins and just how good were the OOC teams that beat Pac-12 teams. There's a big difference between losing to Notre Dame and losing to, say, Colorado State. So let's look at the key OOC games for the top 4 Pac-12 teams (against teams that are either in the tournament or potential at-large teams):

• Oregon State (#5 RPI, #6 Massey): lost (by 1 point) at Notre Dame; lost vs. Tennessee (injuries to key starters played a role in both games).
• Arizona State (#11 RPI, #9 Massey): lost (by 2 points) to South Carolina (neutral court); lost vs. Kentucky; won at Syracuse; won vs. Florida State. Also, suffered an unexpected loss to VCU (#69 RPI, #83 Massey).
  
• UCLA (#9 RPI, #10 Massey): lost (in overtime) to Notre Dame (neutral court); lost (by 3 points) vs. South Carolina; won vs. St. John's; won at Michigan.
• Stanford (#10 RPI, #11 Massey): lost at Texas; won vs. George Washington; won vs. Tennessee; won (in overtime) vs. Purdue (neutral court). Also, suffered unexpected loss vs. Santa Clara (#63 RPI, #65 Massey).

So collectively, these top 4 Pac-12 teams went 7-7 against tournament or bubble teams. It should be noted that 4 of these 7 losses came to Notre Dame and South Carolina, and by very small margins at that. But it's also true that 3 of the 7 quality wins came against teams on or near the bubble (St. John's, Michigan, Purdue). The best wins were ASU's victories over Syracuse and FSU. To me, probably the most revealing data point is how competitive all 4 games were with ND and SC.

Here are some of the key OOC results for other Pac-12 teams:

• Washington: lost vs. Syracuse (neutral court); lost vs. Oklahoma.
  
• USC: won vs. West Virginia (neutral court).
• Cal: won at Louisville; lost vs. Texas A&M (neutral court).
• Utah: lost at BYU; lost vs. Oklahoma (neutral court).
• Washington State: lost at Oklahoma State.
• Colorado: lost at Kentucky; lost vs. Florida (neutral court); lost vs. Missouri.

Of these games, the only ones that were at least mildly surprising were UW's home loss to Oklahoma, USC's win over WVU, and Cal's road win at Louisville.

All in all, I don't feel that the Pac-12 is overrated as a result of "glitches" in the RPI. In fact, as a whole the Massey ratings are even more complimentary of the Pac-12 than the RPI is. Washington, for example, is #33 in the RPI but #21 in Massey. Colorado is #202 in the RPI but #151 in Massey. It's also worth noting that Massey also has the Pac-12 as the top conference in the country, and the SEC as the second-best conference.

 

[UcMiami]

You did a great job, but in analyzing the Pac-12's nonconference results, it's a bit misleading to just cite overall records vs. "ranked" teams to avoid discussing which Pac-12 teams got quality OOC wins and just how good were the OOC teams that beat Pac-12 teams. There's a big difference between losing to Notre Dame and losing to, say, Colorado State. So let's look at the key OOC games for the top 4 Pac-12 teams (against teams that are either in the tournament or potential at-large teams):

• Oregon State (#5 RPI, #6 Massey): lost (by 1 point) at Notre Dame; lost vs. Tennessee (injuries to key starters played a role in both games).
• Arizona State (#11 RPI, #9 Massey): lost (by 2 points) to South Carolina (neutral court); lost vs. Kentucky; won at Syracuse; won vs. Florida State. Also, suffered an unexpected loss to VCU (#69 RPI, #83 Massey).
  
• UCLA (#9 RPI, #10 Massey): lost (in overtime) to Notre Dame (neutral court); lost (by 3 points) vs. South Carolina; won vs. St. John's; won at Michigan.
• Stanford (#10 RPI, #11 Massey): lost at Texas; won vs. George Washington; won vs. Tennessee; won (in overtime) vs. Purdue (neutral court). Also, suffered unexpected loss vs. Santa Clara (#63 RPI, #65 Massey).

So collectively, these top 4 Pac-12 teams went 7-7 against tournament or bubble teams. It should be noted that 4 of these 7 losses came to Notre Dame and South Carolina, and by very small margins at that. But it's also true that 3 of the 7 quality wins came against teams on or near the bubble (St. John's, Michigan, Purdue). The best wins were ASU's victories over Syracuse and FSU. To me, probably the most revealing data point is how competitive all 4 games were with ND and SC.

Here are some of the key OOC results for other Pac-12 teams:

• Washington: lost vs. Syracuse (neutral court); lost vs. Oklahoma.
  
• USC: won vs. West Virginia (neutral court).
• Cal: won at Louisville; lost vs. Texas A&M (neutral court).
• Utah: lost at BYU; lost vs. Oklahoma (neutral court).
• Washington State: lost at Oklahoma State.
• Colorado: lost at Kentucky; lost vs. Florida (neutral court); lost vs. Missouri.

Of these games, the only ones that were at least mildly surprising were UW's home loss to Oklahoma, USC's win over WVU, and Cal's road win at Louisville.

All in all, I don't feel that the Pac-12 is overrated as a result of "glitches" in the RPI. In fact, as a whole the Massey ratings are even more complimentary of the Pac-12 than the RPI is. Washington, for example, is #33 in the RPI but #21 in Massey. Colorado is #202 in the RPI but #151 in Massey. It's also worth noting that Massey also has the Pac-12 as the top conference in the country, and the SEC as the second-best conference.

I did't want to get that deep into the analysis, but since we are there - we are talking about teams that are going to be in the top sixteen hosting home games and they have a total of one win against FSU and one win against Syr in teams that are expected to get top four seeds - the last two teams into that group probably. They have competitive losses to really good teams, which is great, but they haven't beaten a single team that will get seeded above them. They have a win and a loss to TN likely a six seed, a win over St John's and George Washington both currently rated as 8 seeds and a win over Purdue a 10 seed and not sure why you include Michigan as they are not even bubble worthy. And if you want to claim credit for taking ND to overtime (I admit impressive) than you need to give demerits for needing overtime to beat a pretty poor Purdue team. ASU was being slated above OSU and Maryland until they laid a considerable egg in the Pac12 tournament.
I am not sure there are more deserving teams for the top four seeds, so I am not really arguing for dropping them down. I am just pointing out that the conference RPI is to me pretty suspect and the clearest picture of that is the following comparisons:
Washington OOC win 83%, IC 61%
Oregon OOC 100%, IC 50%
Utah OOC 73%, IC 44%
WashSt OOC 82%, IC 28%
Cal OOC 80%, IC 22%
AZ OOC 75%, IC 17%
CO OOC 45%, IC 11% (I give them a pass, they were pretty miserable OOC and IC)

Those signify to me pretty dreadful OOC scheduling Washington at a 22 point variance is OK - it isn't an exact science. But from there the variances run 50 points, 29 points, 54 points, 58 points, 58 points. And whether that scheduling is intentional or not, it is a great benefit to the conference as a whole.

I am too lazy to do a complete dive into MOVs but I did happen to watch MD another team who had an incredibly weak OOC - they were up with Uconn in the high thirties at the start of their IC schedule, they are now in the low 20s.

As far as Massey and Sagarin - they both having similar conference rankings and generally similar team rankings - and this is the first year I have followed there sites that I remember being really surprised with how certain schools have been ranked by them outside of what I consider reasonableness - not specific to Pac12 teams, but just a series of outliers. It is why I suspect that while there software is better to my mind than the standard computer rankings, I suspect it is the same software they use for the men's side which means it hasn't been tweaked specific to the vagaries of the women's competitive environment.

I expect the P5 to be the top five RPI conferences - they have more depth than other conferences and better top end quality than mid majors. I think by 'aggressive' scheduling and by the multiplying effect of conference play the variance between their teams and good mid major teams gets exaggerated in the current system. And I am not trying to pick on the Pac12 - I could have done similar analysis on any one of the P5. In fact the SEC probably has more egregious examples without the excuse of being hidden away behind the rocky mountains making travel more arduous.

 

[Plebe]

I did't want to get that deep into the analysis, but since we are there - we are talking about teams that are going to be in the top sixteen hosting home games and they have a total of one win against FSU and one win against Syr in teams that are expected to get top four seeds - the last two teams into that group probably. They have competitive losses to really good teams, which is great, but they haven't beaten a single team that will get seeded above them. They have a win and a loss to TN likely a six seed, a win over St John's and George Washington both currently rated as 8 seeds and a win over Purdue a 10 seed and not sure why you include Michigan as they are not even bubble worthy. And if you want to claim credit for taking ND to overtime (I admit impressive) than you need to give demerits for needing overtime to beat a pretty poor Purdue team. ASU was being slated above OSU and Maryland until they laid a considerable egg in the Pac12 tournament.
I am not sure there are more deserving teams for the top four seeds, so I am not really arguing for dropping them down. I am just pointing out that the conference RPI is to me pretty suspect and the clearest picture of that is the following comparisons:
Washington OOC win 83%, IC 61%
Oregon OOC 100%, IC 50%
Utah OOC 73%, IC 44%
WashSt OOC 82%, IC 28%
Cal OOC 80%, IC 22%
AZ OOC 75%, IC 17%
CO OOC 45%, IC 11% (I give them a pass, they were pretty miserable OOC and IC)

Those signify to me pretty dreadful OOC scheduling Washington at a 22 point variance is OK - it isn't an exact science. But from there the variances run 50 points, 29 points, 54 points, 58 points, 58 points. And whether that scheduling is intentional or not, it is a great benefit to the conference as a whole.

I am too lazy to do a complete dive into MOVs but I did happen to watch MD another team who had an incredibly weak OOC - they were up with Uconn in the high thirties at the start of their IC schedule, they are now in the low 20s.

As far as Massey and Sagarin - they both having similar conference rankings and generally similar team rankings - and this is the first year I have followed there sites that I remember being really surprised with how certain schools have been ranked by them outside of what I consider reasonableness - not specific to Pac12 teams, but just a series of outliers. It is why I suspect that while there software is better to my mind than the standard computer rankings, I suspect it is the same software they use for the men's side which means it hasn't been tweaked specific to the vagaries of the women's competitive environment.

I expect the P5 to be the top five RPI conferences - they have more depth than other conferences and better top end quality than mid majors. I think by 'aggressive' scheduling and by the multiplying effect of conference play the variance between their teams and good mid major teams gets exaggerated in the current system. And I am not trying to pick on the Pac12 - I could have done similar analysis on any one of the P5. In fact the SEC probably has more egregious examples without the excuse of being hidden away behind the rocky mountains making travel more arduous.

You're right about Michigan. I included that game because I remembered Charlie listing Michigan as among his first four or next four out, but I see that's no longer the case. But on the flip side, I technically should have included UCLA's win over James Madison, since Charlie has JMU above the at-large cut line in his projected seeding.

I agree with your last paragraph, in that any if we look under the hood of the OOC results of any other power-5 conference, we're going to find similarly unflattering impressions. Let's take a gander at the ACC, which, aside from Notre Dame, fared quite poorly. Here are the key OOC results of the top third of the ACC:

• Notre Dame: Lost at UConn; won vs. Oregon State, won vs. Ohio State, won vs. DePaul, won vs. Tennessee.
• Louisville: Lost at Kentucky; won at Michigan State; won vs. USF; lost vs. Purdue (neutral court); also suffered unexpected losses to non-tournament teams Western Kentucky, Dayton and Cal.
• Syracuse: Lost at Maryland; lost vs. Arizona State; lost at Tennessee; won vs. Washington (neutral court).
• Florida State: Lost to UConn (neutral court); lost at Arizona State; lost at Florida; won vs. Rutgers; won vs. Temple.
• Miami: Lost vs. Baylor (neutral court); won vs. Indiana (neutral court).

Altogether, these 5 teams went 10-10 against teams of at-large quality. However, since no one is questioning that Notre Dame is a solid No. 1 seed, it's more illustrative to examine the other four teams, which went 6-9 in key OOC matchups. Only 2 of the 9 losses were to projected No. 1 seeds (UConn and Baylor), and 3 of the 6 wins came over bubble teams (Indiana, Rutgers, Temple). The best OOC win by any of these 4 teams was Louisville's win over Michigan State.

 

[Jim R]

Have to echo Bgillon - great board work - Thanks for peeling back the onion!

 

[UcMiami]

Plebe - ND is rightly excepted from this because they played a strong OOC and ran their conference (again.) You rightfully point out less than stellar OOC for the rest.
I think where I see a difference is that:
Louisville had a terrible start to the season with a young team, but then proceeded to run their conference except for the loss to ND and a not bad loss in the conference tournament.
Syracuse had a really bad IC loss to UNC, but otherwise beat everyone else IC except for the two top teams and beat one of them in the ACC tourney to climb back into a likely top 16 seed.
When you look at FSU it does began to get ugly -
FSU - you can drop Rutgers from the good wins - they have no chance at the dance and Temple is borderline. Their only saving grace is that they were consistent IC beating everyone below them until they met Miami for a third time - never easy. But below them in the ACC this year is really not very good.
By the time you get to Miami you are really looking at a Washington type resume which we really didn't break down above - it is the great unwashed underbelly of P5 conferences that get bids because they play above .500 IC and have maybe a few OK OOC games that they sometimes win. They clog up the 6, 7, and 8 seed lines never in doubt of getting a bid - my only complaint with them is that they get preferential seeding because of their conference RPI strength displacing some perhaps more worthy mid-major teams to less advantageous seedings. And they usually get a few 'good' wins in conference because of the inconsistency of the IC teams above them.

When you look further down the ACC teams do pop out in the RPI/SOS game similar to the Pac12 -
VaTech at 12-1 OOC and 5-11 IC (though who knew they would beat TN! )
BC at 12-1 OOC and 2-14 IC (with I guess surprising wins at Purdue and Nebraska)

 

[toadfoot]

I was otherwise occupied when I saw your post last night so didn't have time to say much more than job well done, but when I looked into some of these same issues earlier this year, one thing that jumped out at me was home versus away for some of these conferences/teams. I don't think it's a secret that the P5 conferences have the power to dictate location to most OOC opponents. In other words, they have the power to say, "play at my arena or not at all".
Exhibit #1 is Baylor, who played zero OOC away games, but they're hardly alone. A couple of things the NCAA could do to address this issue are:

1. Require all OOC games to be based on home and home arrangements.
or
1a. Require that during any given year no fewer than 35% of OOC games be away and that over a 5 year span no fewer than 40% be away.

2. Any team not meeting the requirements of 1 or 1a be penalized during the seeding of the NCAA tournament such that they will be seeded no higher than #17 overall (and thus not be eligible to host).

3. In conference opponents RPI is less heavily weighted than OOC opponents which would lessen the impact of conferences fattening up their RPI during OOC play in order to game the system by improving SOS during conference play.

 

[UcMiami]

And just to return to the first line of the OP - a quick look at the SEC:
I leave SCar out of it - they played 4 top ten teams OOC (and Duke which they expected to be ranked) going 3-1 and then ran their conference - solid team solid schedule. And I'll leave TN out of it as well - they are just a mess and they played their usual solid OOC schedule with unusual results.
MissSt - 13-1 playing 2 ranked teams, 1 P5 team, 2 road games (1 next door at LaTech) and 11-5 IC - ranked as high as 7 until they started the classic SEC road loss syndrome with losses at Missouri, GA, and TAMU. Best win IC at a sometimes ranked FLA and home against Missouri.

TAMU - 10-3 OOC, 11-5 IC - 1-2 against ranked teams with two more games schedule against what they thought would be ranked teams Duke and Cal, 1 road game, 4 P5 games - meh. Ranked in the teens as high as #10 settling in at 19 currently. Just nothing to get excited about - losses at Ark and Fla - best win home to MissSt. End the year with two losses to KY, and Tenn

KY - 11-0 OOC, 10-6 IC - 2-0 against ranked teams, 3-0 against P5 - very inconsistent IC with losses at AUB and FLA and home to Vandy followed by a strong run to end the year with wins at MissSt and TAMU, and FLA in the tournament.

FLA - 12-1 OOC, 10-6 IC - 1-0 against ranked teams 3-0 against P5, four road games - typical SEC road wimps losing at Missouri, at UGA and at Auburn with a home loss to UGA as well. Best wins home to KY and TAMU.

UGA - 12-1 OOC, 9-7 IC 0-1 against ranked teams 0-1 against P5 three road games - all over the place! Bad losses to ALA, LSU, and Vandy - Good wins at FLA and home to MissSt and FLA

Missouri - 13-0 OOC, 8-8 IC No ranked teams, 2 P5 team, no NCAA teams, 3 road games. Bad loss at Ark and home to Vandy, good wins home MissSt, FLA. Ended the year with loss to AUB.

Auburn - 10-3 OOC, 8-8 IC No ranked teams, 2 P5 teams, 1 midMajor NCAA team?, 5 road games, no bad IC losses, good wins home to KY and FLA

The rest - Arkansas 5-8 OOC playing a really good OOC but unable to beat any decent teams - but then beats TAMU, TN, Missouri IC proving how inconsistent those teams are - if they had played Auburns OOC they might replace them in the Bubble with a likely bid.

Vandy 11-2 OOC, 5-11 IC - no ranked, no P5 - beat KY, Missouri, and UGA IC
ALA - 11-2 OOC, 4-12 IC - no ranked, 1 P5 - beat UGA and TN
LSU - 6-7 OOC, 3-13 IC - strong OOC for an injured team - Beat UGA and TN IC
OleMiss 8-5 OOC, 2-14 - no ranked, no P5, no NCAA - beat UK

What stands out in this is the incredible inconsistency of the teams in the top half of the league - and I think it comes down to being offensively challenged - pretty much every team has struggled to score 50 points in multiple games this year - so if a bad team in the league gets 'hot' and hits 35% of their threes, and 40% from 2 and 70% from the foul line they are probably going to win a game against what should be on paper a much better team.

Average fouls and turnovers for SEC conference games are probably each in the 30s, average shooting percentages at 40%, average threes at 30%

And that inconsistency gets teams in the dance - Auburn is not a better team than any of the other bubble teams and probably worse than most, but they have two 'top 20 wins' and the SEC RPI boost - the same with Missouri aided by a 13-0 OOC, and UGA with their 12-1 record.

 

[KnightBridgeAZ]

I agree with much of the above, with the following 3 comments:

- I get issues with power conference teams playing such a high percentage of games at home, but up until the last few years, UConn did it as well. I kept my win / loss BE spreadsheets from 2009 till the end of the BE and I am pretty sure that in all of them UConn was between 66% and much higher "home". In the last few years, that has changed, but then, so has WBB. So under Toadfoot's scheme, UConn would have been penalized.

- I accept wanting to tweak things to make ratings of teams more fair, but in fact, the game and talent levels are always changing. Nothing has highlighted this more than the shifting of conferences, from the major BE split to the minor switching it drove affecting so may other conferences. I'm not sure there is any simple solution.

- "On any given day" - not every upset is due to teams not being properly ranked or even being offensively challenged. Sometimes, it is just an upset.

Earlier Colorado was mentioned. They are changing coaches, FWIW. Yet another interesting opportunity as they were at one time quite decent.


-

 

[UcMiami]

So final informational post - thought I would look at Best win/lost of the top end of the field:
AAC
Uconn - #2 SC or ND - none
UFS - Temple - Memphis
ACC
ND - #6 OrSt - #1 Uconn
Lou - #14 Syr - WKY I guess
Syr - #8 Lou - UNC
FSU - #20 MIA - #25 FL
BE
DePaul - #19 TAMU - Loyola of Chicago
Seton Hall - #18 DePaul - Creighton
Big12
Baylor - #7 TX - OkSt
Texas - #13 Stanford - #24 Oklahoma
WV - #24 Oklahoma - USC or GONZ
OKSt - #4 Bay - ISU
OK - #7 TX - N-TX
BigTen
MD - #14 Syr - #9 OSU
OSU - #5 MD - NW
MSU - #9 OSU - PSU
IND - #16 MSU - NW or Ohio
Pac12
OrSt - #10 UCLA - TN
ASU - #10 UCLA - VCU or Cal
Stanford - #6 OrSt - Santa Clara
UCLA - #6 OrSt - Cal
SEC
SCar - #9 OSU - #1 Uconn
MsST - #21 USF - Missouri
TAMU - #15 MsST - Ark
KY - #8 Lou - Vandy
Fla - #12 KY - Temple or Vandy

 

[toadfoot]

I agree with much of the above, with the following 3 comments:

- I get issues with power conference teams playing such a high percentage of games at home, but up until the last few years, UConn did it as well. I kept my win / loss BE spreadsheets from 2009 till the end of the BE and I am pretty sure that in all of them UConn was between 66% and much higher "home". In the last few years, that has changed, but then, so has WBB. So under Toadfoot's scheme, UConn would have been penalized.

- I accept wanting to tweak things to make ratings of teams more fair, but in fact, the game and talent levels are always changing. Nothing has highlighted this more than the shifting of conferences, from the major BE split to the minor switching it drove affecting so may other conferences. I'm not sure there is any simple solution.

- "On any given day" - not every upset is due to teams not being properly ranked or even being offensively challenged. Sometimes, it is just an upset.

Earlier Colorado was mentioned. They are changing coaches, FWIW. Yet another interesting opportunity as they were at one time quite decent.


-

And if the rules I suggested were in place during the timeframe you mention I would have fully supported UConn being penalized. The reality is that regardless what has occurred in the past, the NCAA absolutely needs to do something to address the rampant manipulation of the RPI by some conferences/teams. The RPI as currently calculated is a joke!

 

[UcMiami]

I agree with much of the above, with the following 3 comments:

- I get issues with power conference teams playing such a high percentage of games at home, but up until the last few years, UConn did it as well. I kept my win / loss BE spreadsheets from 2009 till the end of the BE and I am pretty sure that in all of them UConn was between 66% and much higher "home". In the last few years, that has changed, but then, so has WBB. So under Toadfoot's scheme, UConn would have been penalized.

- I accept wanting to tweak things to make ratings of teams more fair, but in fact, the game and talent levels are always changing. Nothing has highlighted this more than the shifting of conferences, from the major BE split to the minor switching it drove affecting so may other conferences. I'm not sure there is any simple solution.

- "On any given day" - not every upset is due to teams not being properly ranked or even being offensively challenged. Sometimes, it is just an upset.

Earlier Colorado was mentioned. They are changing coaches, FWIW. Yet another interesting opportunity as they were at one time quite decent.


-

Nice thoughts -
Home schedules are often financial in terms of gate and financial in terms of travel expense. And I agree about Uconn's past record with particular reference to the thankfully short lived in season tournament they hosted - ugly. Otherwise they kept a fairly decent balance, always having about 6 good H/H series running including one trip that usually included two games whether west coast or SW/Texas or an out of market in conference tournament.

Agree about the 'on any given night' but particularly in the SEC it seems to be more road vs home as it used to be in the Big12.

Yeah - all the echoes of conference realignment haven't been absorbed yet. In particular as the new BE and the new AAC raided the Mid Majors the quality in the bottom end of conferences has been seriously changed too.

 

[baggerbob]

Early season wins shouldn't be weighted as high as end of season wins. Most coaches use the OOC games to establish which players can contribute to a teams success. You also have players coming back from injury, as Syracuse did with Sykes, came back from a second ACL tear. The team that lost those early games is not the one that is playing now.

 

[UcMiami]

Early season wins shouldn't be weighted as high as end of season wins. Most coaches use the OOC games to establish which players can contribute to a teams success. You also have players coming back from injury, as Syracuse did with Sykes, came back from a second ACL tear. The team that lost those early games is not the one that is playing now.

The committee supposedly does give added weight to late season form - except when they decide to ignore it. (Part of the issue with the whole selection and seeding is there is always a data set you can use to justify one school over another, and in the committee discussions after they reveal, they do just that - for this school we gave them a plus for RPI and ignored bad losses, for this one they had two good wins even though the RPI was terrible, of this one they had a really strong last month so we ignored the beginning of the season, but for this school they had a great first month so we ignored all the losses they piled up in February, etc. And of course the explanation changes each year - this year S curve is king next year it will be total distance.)

And the problem with early season is that is when most of the comparative evaluation is done in OOC - if you don't pay attention to that, then you are just looking at conference play in a vacuum. and that is part of what I think the issue is.

You can look at the terrible start with Louisville and go 'integration of lots of freshman' and most people do because the rest of the year has been stellar, and you look at Baylor's loss and go 'Johnson out' but ND Uconn and OrSt all lost starters for games and USF had two significant losses with starters out, and I am sure half the NCAA field had nicks and bruises during the course of the season - finally you are what the record says you are and you can't wiggle the data for everyone.