The committee Experts vs. the AP/Coaches amateurs.

[UcMiami]

So I am starting this post without actually doing the research first but I thought it would be an interesting comparison - Charlie and others doing bracketology dismiss the poles as being 'unscientific popularity contest' and the like - so how did the expert scientific committee do on the top 16 vs. the amateurs?
Committee
1 Seeds
Uconn - FF - 0
ND - FF - 0
Tenn - Sw16 - 2
SCar - Sw16 - 2
2 Seeds
Duke - R32 - 1
Stan - FF - 1
WVU - R32 - 2
Baylor - E8 - 0
3 Seeds
T A&M - E8 - 1
KY - Sw16 - 0
PSU - Sw16 - 0
Louis - E8 - 1
4 Seeds
UNC - E8 - 1
Nebraska - R32 - 1
Maryland - FF - 2
Purdue - R32 - 1
Higher than 4 seed getting to Sw16 = 4 (BYU12/LSU7/DePaul7/OKSt5)
Total = 19
... a point for each missed round loss or win - so a #1 losing in Sw16 = 2 points and a 4 seed making the FF = 2 points as well. a high point total is bad.

Ap and Coaches
Uconn - 0
ND - 0
Tenn - 2
Louisville - 1

Baylor - 0
Stanford - 1
WVU - 2
SCar - 1

Duke - 1
KY - 0
Maryland - 2
UNC/Neb - 1

Neb/UNC - 1/1
PSU - 0
A&M - 1
NCSt/Gonz - 2/2
Higher seed getting to SW16 = 4/4 (BYU8/8, OKSt6/5, LSU?/?, DePaul6/6)

Total = 19
So the amateurs come in exactly the same in this little unscientific exercise - both groups missed the cinderella which is understandable. But I actually think missing by a single round is a pretty good result, missing by two or more seems more egregious - this is sports and the probability factor in a 4-5 matchup or a 2-3 matchup isn't that big an upset. And the balance outside the top end is probably even. So maybe a more interesting look is at the top 2 seed results:
There the scores are in favor of the amateurs 7 to 8 and again if we consider a one round miss acceptable but a two round miss not - the experts missed 3 times the amateurs only 2.
If we carry that emphasis on missing by two round out to the full sixteen teams - amateurs 4
experts 5

 

[Fightin Choke]

Interesting exercise, but the selection committee has a layer of inflexibility that you're not considering. The fact that they cannot have conference rivals play each other until a certain point AND the fact that hosts must host makes their task more difficult. Thus Creme refers to procedural bumps. If you could just compare the selection committee's S-curve to the polls, then that would be a more apt comparison.

Over on RebKell, a great poster has compared several brackets created by using the "S-curve" of different rating services. He considered all 4 completed rounds thus far, and gives more points to later rounds. In his analysis (LINK), Sagarin has fared best thus far, with Sonny Moore and Omni tied for second and SporTheory in 4th. Both polls are tied for 5th with 2 other metrics. Massey is tied for 9th with Real Time RPI Power Rating and 2 others.

 

[UcMiami]

Yes there are procedural protocols that the committee deals with but those work downward from the top and the home site issue doesn't really affect that for 1-4 seeds or the assignment of seeds - they had one each 1-4. So it is much more likely that an LSU was moved from an eight seed to a seven than that Louisville was moved from a 2 to 3 seed etc. And I am not worried about position within seed line as that is pretty hard to judge with all the various issues you mention. So looking at just the final sixteen teams should reduce most if not all of your issues - LSU should still not have made it to the sweet sixteen and Scar should still have made it to the FF.
I actually think the final comparisons of 2 round variation is the most valid and again addresses you concern if there was in fact an adjustment of a 2 and 3 seed the result would only be 1 round difference 16 to 8.

What bothered me in the bracketology discussions was the dismissal of the two polls as being meaningless by Creme particularly but others as well and the praising of the strength of in depth analysis that the committee was making. Net result - the polls were just a touch more accurate in predicting the sweet sixteen, elite eight, and final four as this analytic process. It is all within margin of error, but maybe Creme will not be quite so dismissive of the opinions of voters in future? - nah!
Of note - the polls got 5 of 8 elite eight teams, the committee got 4 of eight and the committee missed by 2 rounds on all four, the polls only 2 of their three.

There is an error in my calculations above - Duke as a two seed should be a '2' for the committee not a '1'. The totals are actually
Committee 20, Polls 19
And on misses by two rounds:
top eight
Committee 4 Polls 2
And for all 16
Committee 6 Polls 4

 

[ThisJustIn]

They're "meaningless" in that they are not taken into account by the Committee. Not that they are "inaccurate."
'sides - several times folks on this board have dismissed the poll folks as morons ... guess that's the kettle calling someone something...

 

[whaler11]

The polls and the committee are not attempting to measure the same thing, so that they don't always line up doesn't really matter.

 

[UcMiami]

The polls and the committee are not attempting to measure the same thing, so that they don't always line up doesn't really matter.

Yes - but ... the committee is trying to identify in quadruplets the strength of the field and the polls are trying to identify the strongest 25 teams and rank them so they are trying to accomplish the same result. I just find it interesting that for this year looking at the top 16 teams arrived at by polling coaches and writers across the country to name their best 25 you ended up with arguably a better result than you got from a committee that spends an intense weekend debating amongst themselves the strengths and weaknesses of each team and comes to a 'consensus' from weighing all their data points.

 

[whaler11]

Yes - but ... the committee is trying to identify in quadruplets the strength of the field and the polls are trying to identify the strongest 25 teams and rank them so they are trying to accomplish the same result. I just find it interesting that for this year looking at the top 16 teams arrived at by polling coaches and writers across the country to name their best 25 you ended up with arguably a better result than you got from a committee that spends an intense weekend debating amongst themselves the strengths and weaknesses of each team and comes to a 'consensus' from weighing all their data points.

No the committee is seeding the teams based their resumes durimg the season. They aren't trying to predict the future.

Who has the best resume over the course of a season and who are the best teams at a point in time are not attempting to solve for the same thing

 

[UcMiami]

NO - not the best resume. The committee is trying to determine the best teams - Notre Dame had easily a better 'resume' than Uconn but Uconn was the overall number 1 seed. - They are not saying these 4 teams have the best resumes so they are #1 seeds. They are saying these four teams based on their performance are the four best teams in the country. That is why they have been know to look at injuries to key players late in the season and penalize teams in terms of their seeding - the resume didn't change but the team is no longer as good because XYZ player tore an ACL in the conference semifinal. Stanford's 'resume' was clearly far superior to SCar and to TN but the committee determined they were not as good (oops!)