The women’s D1 national championship meet starts on Wednesday. We’ve scored the psych sheet and examined how performance changes from the psych sheet at the big meet. It’s now time to combine to two into a forecast of the meet.
I ran a Monte Carlo simulation of the NCAA women’s meet minus diving (most top teams have 1 or 0 divers except Minnesota, and UCLA who have 3). The exact procedure was:
- Modify each swim on the psych sheet swim by a random percentage based on their team’s performance history. The percentage was drawn from a normal distribution with mean and standard deviation of the team’s previous time changes at nationals (for example, Georgia mean: .1%, sd: .91%). If a team had fewer than 20 swims in the previous 7 years at nationals, the entire field’s mean of .45% and standard deviation of 1% were used.
- Re rank the times based off the time changes
- Score the meet and check the order of the teams
- Repeat 50,000 times
The top teams unsurprisingly remain unquestioned under this procedure. Stanford won over 99.9% of the time. California was 2nd over 99.9% of the time. There were shakeups in the the top 10. The model gives NC State, 4th on the psych sheet, almost no chance of a top 4 finish based on swimming points. Instead, 84% of the time the model has them finishing somewhere between 7th and 12th. Georgia, 6th on the psych sheet, ends up in the top 4 in 68% of simulations, and the top 5 in 91%. Texas, 5th on the psych sheet, finishes 5th or higher based on swimming points in only 16% of the model runs. Virginia, 8th on the psych sheet, is 7th or better in 72% of simulations. A table with more of the results is below.
This methodology isn’t perfect. There’s no diving. It doesn’t include a chance of relay DQ’s. It also makes an assumption that past performance at nationals is predictive of future performance. This assumption appears reasonably valid based on the year by year time changes. Teams performances are correlated one year to the next. However, this simulation doesn’t include contingencies for teams drastically changing their past behavior. For example, NC State, #5 on the psych sheet, historically added an average of .72% at nationals. If suddenly they behave like Stanford and add only .01%, the model’s prediction of a <2% chance of a top 5 finish will probably be up ended (I’m not saying anything about NC State. Just a random example). For most teams, I think the assumption of behavior consistent with their history at this meet is valid, but there will probably be a couple teams who change their approach and have a historically novel result and break out from their expected finish here.
That’s the fun part: seeing which teams break expectations and do things we didn’t (or shouldn’t) predict. This post is about setting a baseline of expectations. The story of the meet will be which expectations are defied or fulfilled.
Simulated Places
The spaces with 0% are perhaps better marked as <1%. It happened, but it rounded to 0%. The blank spaces never happened. The full table continues past 20th place and 24 teams, but I cut it off for readability/relevancy reasons.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | |
| Stanford | 100% | 0% | ||||||||||||||||||
| California | 0% | 100% | 0% | |||||||||||||||||
| Southern Cali | 0% | 45% | 30% | 18% | 5% | 1% | 0% | 0% | 0% | 0% | 0% | 0% | ||||||||
| Georgia | 0% | 34% | 34% | 23% | 7% | 2% | 1% | 0% | 0% | 0% | 0% | |||||||||
| Texas A&M | 21% | 28% | 32% | 12% | 5% | 2% | 1% | 0% | 0% | 0% | 0% | 0% | ||||||||
| Virginia | 1% | 4% | 12% | 31% | 24% | 13% | 7% | 4% | 2% | 1% | 0% | 0% | 0% | 0% | 0% | 0% | ||||
| Texas | 1% | 4% | 11% | 25% | 22% | 14% | 9% | 6% | 4% | 2% | 1% | 1% | 0% | 0% | 0% | 0% | 0% | |||
| Michigan | 0% | 0% | 2% | 8% | 14% | 17% | 16% | 14% | 11% | 9% | 6% | 3% | 1% | 0% | 0% | 0% | 0% | 0% | ||
| NC State | 0% | 0% | 1% | 5% | 11% | 17% | 18% | 16% | 13% | 9% | 5% | 2% | 1% | 0% | 0% | 0% | 0% | |||
| Louisville | 0% | 0% | 1% | 4% | 9% | 14% | 16% | 16% | 15% | 12% | 7% | 4% | 2% | 1% | 0% | 0% | 0% | 0% | ||
| Indiana | 0% | 0% | 0% | 2% | 6% | 11% | 15% | 18% | 18% | 14% | 9% | 4% | 2% | 1% | 0% | 0% | 0% | |||
| Arizona | 0% | 0% | 0% | 2% | 4% | 7% | 9% | 12% | 14% | 15% | 14% | 10% | 6% | 3% | 2% | 1% | 0% | 0% | ||
| Wisconsin | 0% | 0% | 0% | 1% | 2% | 4% | 6% | 9% | 11% | 15% | 16% | 14% | 9% | 5% | 3% | 2% | 1% | 1% | ||
| Minnesota | 0% | 0% | 0% | 1% | 2% | 4% | 8% | 13% | 19% | 21% | 14% | 9% | 5% | 3% | 1% | 1% | ||||
| UNC | 0% | 0% | 0% | 0% | 1% | 1% | 3% | 6% | 11% | 15% | 16% | 15% | 12% | 9% | 6% | |||||
| Ohio St | 0% | 0% | 0% | 0% | 1% | 1% | 3% | 6% | 12% | 17% | 17% | 15% | 12% | 8% | 5% | |||||
| Tennessee | 0% | 0% | 0% | 0% | 0% | 0% | 1% | 3% | 7% | 12% | 15% | 16% | 16% | 13% | 8% | |||||
| Kentucky | 0% | 0% | 0% | 0% | 1% | 2% | 4% | 8% | 13% | 15% | 16% | 15% | 11% | 7% | ||||||
| Missouri | 0% | 0% | 0% | 0% | 0% | 1% | 1% | 3% | 6% | 10% | 13% | 16% | 17% | 14% | ||||||
| UCLA | 0% | 0% | 0% | 0% | 1% | 2% | 4% | 7% | 11% | 17% | 19% | |||||||||
| Auburn | 0% | 0% | 0% | 0% | 0% | 1% | 2% | 4% | 6% | 10% | 14% | 16% | ||||||||
| Arizona St | 0% | 0% | 0% | 1% | 2% | 6% | ||||||||||||||
| Florida | 0% | 0% | 0% | 0% | 0% | 1% | 2% | |||||||||||||
| Florida St | 0% | 0% | 0% | 0% | 1% | 3% |
Never thought I’d see Monte Carlo applied to swim meet predictions, but I love it! Please do this for the men too. What’s the issue behind excluding diving? It would be much more predictive if you could include. And for the men’s meet, that may prove critical for the top spot.
The swimming psych sheet is full of times and places. The diving psych sheet looks like this: http://i.turner.ncaa.com/sites/default/files/external/gametool/brackets/2017_womens_qualified_divers.pdf
Cool work!
Interesting that UVA has a much better probabilistic forecast than NC State given the results at the ACC Championships. Given that the Monte Carlo sample size seems sufficiently adequate, it’s definitely a fascinating result. Thanks for this great probabilistic exercise! This is really fun to review.
It’s because of UVA’s better performance history at nationals vs psych sheet times. They’ve added an average of .42% vs .72% for NC State. NC State is much improved this year and that .72% is based on only 39 data points (vs 218 for UVA), so perhaps they will reverse the trend of below average nationals performance vs seed. Some regression to the mean wouldn’t be surprising. https://staging2.swimswam.com/ncaa-d1-women-performance-nationals-vs-seed-time/
Ok, took me a second, but when you say increase relative to seed, lower numbers are better. This is why UVA performs better in your model. Thank you for the additional insight and clarification.
NB: I figured the results were mostly due to UVA being top heavy, which would be rewarded more against the better overall field at the NCAAs.
Neither UVA or NCSU’s coach has been there the past 7 years, and likely that’s true for a few others as well – so I’m not sure going back 7 years is as valid when there has been a coaching change or an changed emphasis on whether to peak for NCAAs or the conference meet.
What do you guys think on how many american records will be set? 6 individual and 2 relay would be my guess but I am not going to do any statistical analysis to back it up.
Yeah I agree 6 or so individuals, probably 6-8. But I think more relay records will go down. The 400 and 800 free are pretty sure bets cause stanford set them at PAC 12s and Simone Katie and lia Neal can all almost certainly go faster on them. Cal and fanford were within a few tenths of the 200 medley so one of them will probably get it. In the 400 medley both cal and stanford have a shot but it’s not really a sure thing. And the 200 free cal and Stanford were only a few tenths off again. So it’s very possible that al 5 relay records go down but I’ll guess 4 do, and the 5th is… Read more »
This is an incredible article. I think we can all appreciate such a well qualified analysis and such careful thoughts. It’s really easy to get carried away with data sometimes thinking the numbers have all the answers. This analysis has such a wonderful understanding of the meaning behind the data.
Incredible journalism. Thank you!
I’m curious about how these stats would hold up against the NCAA Pick ‘Ems field. It’ll be fascinating to see how accurate this turns out.
Interesting and provocative article! If this turns out to be accurate, perhaps in the future we will not need to swim the meet. Thank you for the hard work. Will be interesting to see how accurate it is. I believe in UGA’s ability to thrive at the meet and predict them to be top three.
Agree, they usually step up but 3rd is the best scenario! Not enough silver bullets in their gun to get 1st or 2nd. So yes, I agree 3rd at best and probably not worse than 5th.
I think the larger point is missed here. This is not a true competition. First and second are spoken for. The first position that there is any competiton for is third. What incentives are there for any schools to attend this competition? In the NCAA basketball tournament there are always upsets or results that were not expected. How exciting would the basketball tournament be if there was a 100 percent chance that a certain team would win? How many people would attend? If i were a college athletic director I would seriously look at this table then re-consider which womens sport I wished to compete in. I would look to put teams in a sport where I at least had… Read more »
Your half correct.
No brainer Stanfords gonna be uno as everyone already knows. But good to have this scientific confirmation. The Card studs dont even have to try all that hard. Will still come in 200pts ahead of the runnerup. They are that good!
However im rolling my eyes at the prediction for kal! Its well known they have a tendency to choke & lack mental stamina on the big occasion. Historically they add tons of time from conference too. Just a reflection of their coaching i guess.
Georgia or Texas will be the teams trailing far behind Nerd Nation. Can predict with mucho confidence kal wont even make the top 5!
You’ve got the Cal thing backwards. Historically Cal are top tier when it comes to how they perform at nationals vs conference. They are virtually tied with Stanford for the best performing big school. https://staging2.swimswam.com/ncaa-d1-women-performance-nationals-vs-seed-time/
There is some evidence that they Cal’s swimmers don’t improve much from one year to the next (https://staging2.swimswam.com/how-much-faster-do-swimmers-get-in-college/), but when it comes to performance at nationals vs conference, they’ve done quite well in the recent past. If you want to be a hater, be an informed hater.
No hater just stating the cold facts. You only have to look up how much SLOWER there “stars” were last year in March compared to Pac12s. im talking about Bilquist, Baker & co.
I rest my case!
It’s true that Cal had their worst year in 7 years last year. They added an average of .49% to their seed times; however, their 7 year average is .02%. Why do you think last year is more predictive to how they’ll swim this year than the 6 year’s before last year? Stanford’s worst year (adding .44% in 2011) was followed by their best year (dropping .45% in 2012). Texas A&M followed the best year (2012 dropping .41%) with their worst (2013 adding .6%). It’s silly to claim that one year equals a trend.
Is it possible that Cal swims poorly again this year? Yes. Is it possible for them to have a concentration of bad swims among highly… Read more »
Common sense woulda told you last years a much better indicator than going back a few years no? Look most of the swimmers from ’16 are still competing in ’17. Why compare to a totally different team from ’11 or ’12?
Besides kal only managed 3rd place last year. With their heavy hitters like Bootsma & Pelton who are now gone. Theres a reason why there recruiting has been so terrible after the Franklin disaster.
Stanford is heading in the opposite direction. They shoulda won last year cept for the dq. The Cardinal adds the top freshman class this year & next year. Greg Meehan brings back their Olympian diver & a stud freshman diver. Now throw in the… Read more »
If you score out the meet, Texas is not going to finish higher than 5th or 6th. They had a lot of great Dual Meet Swimmers, but almost no A-Finalists and don’t quite have enough star-depth to win relays.
You’re**
-Cal grad
Alabama football
UConn Women’s Bball
Patriots winning the AFC
US Olympic Swim Team
Not saying any of those (except UConn Women) are as much of a lock to win their respective competitions as much as Stanford is a lock to win this meet, I’m just saying that sports without parity are still pretty fun to watch
in general, more fun. Parity in almost every sport is boring. Fans are drawn to greatness.
By your logic, every college but U Conn should drop womens’ basketball.
I love the internet when other people’s contributions and perspectives challenge and broaden my point of view. Unfortunately, this is not one of those times.
Perhaps some readers may not be aware that the above line of reasoning is @Buckeye499’s favored refrain. He/she has written volumes on this same topic in the past. Each subsequent installment has been received with similar levels of unanimous enthusiasm by respondents. I suggest it may be appropriate at this point to just politely move on. You may find that @Coach’s comments may fall into the same camp.