Cullen: At the 2013 Sloan Sports Analytics Conference

Scott Cullen
3/8/2013 3:31:25 PM
Decrease Text SizeIncrease Text Size
Text Size

Stats nerds from around North America (and even some from beyond) gathered in Boston over the weekend for the seventh annual Sloan Sports Analytics Conference. The conference brings together sharp analysts, whiz kids and executives, who know the ins-and-outs about how teams operate in the real world and how analytics can be used to provide better information going forward.

This was my third year attending the conference and, as always, it provided fascinating insights.

(Last year's conference recap and the 2011 conference recap)

Since there are multiple panel discussions going on at all times, that means making choices. I have been known to choose a panel based on panelists moreso than the stated topic (not for nothing but, sometimes, the discussions can veer pretty wide from the title topic).

There were a couple of themes that stood out to me at the Sloan Conference this year.

The first is the role of luck in sports. Fans don't want to believe that the results of sporting events are luck-dependent but, statistically-speaking, they are. Some more than others, but fortune, either good or bad, has a role. That's hard to accept when there is a tendency to make sports results a measuring stick about someone's character, but there are times that a game's results are really about getting the bounces, either good or bad.

In a presentation titled Why You Don't Understand Luck (by Michael Maboussin), there were several strong points.

The first of which is the presence of outcome bias. People, in general, constantly judge on outcomes, especially in sports when "Scoreboard!" is the way to silence any argument, when they should focus on process. If a team is performing the right way, they are ultimately more likely to have their desired results. Maybe not in one game -- anyone can win on any given day -- but over the long haul.

Maboussin's conclusion was that the majority of people love stories, yet struggle with statistics. That leads to overweighing recent results, relying too much on perception and making risk-averse choices, even when the statistics suggest that the odds favour a bolder approach. The Patriots' infamous 4th-and-2 play against Indianapolis in a Sunday night game in 2009 was cited as an example of the right process that was loudly criticized because of the play's result

As skill improves, throughout a sport, Maboussin asserted, there is more luck required to reach exceptional levels. Using Ted Williams hitting .406 in '41 as an example, Williams was four standard deviations ahead of the league average when he hit .406. Four standard deviations above league average last season would have resulted in hitting .380 and the reason for that is there is a broader base of talent throughout the league. Consider the calibre of players in the NHL in the 1980s, when Wayne Gretzky was racking up 200-point seasons, compared to the league-wide talent base in the current game, with players in much better shape and playing at a faster pace.

There was also a panel on True Performance and Randomness, which included Nate Silver, who is a bit of a stats community rockstar after his forecasts for the 2012 U.S. Election. The premise of the panel was, essentially, that real skill is something that is repeatable. Winning at roulette, for example, is a very luck-driven exercise. It doesn't mean you can't win, especially in the short-term but, over time, the odds will catch up to you. On the other hand, a skill like shooting a basketball is something that can be repeated. If you let Steve Nash shoot free-throws, he's going to make somewhere around 90% of them because he's established that is his skill level for that exercise.

Another panel, entitled Revenge of the Nerds, was moderated by Michael Lewis (author of "Moneyball") and included Silver, Mavericks owner Mark Cuban, Houston Rockets GM Darryl Morey and San Francisco 49ers COO Paraag Marathe. They discussed analytics in MLB, NBA and NFL (like most panels at this conference, the NHL wasn't represented).

Cuban has an interesting view because he's so involved as an owner. He talked, at times, about how organizational culture can lead to acceptance of advanced stats, but it certainly helps when, in the Dallas Mavericks' case, it's the team owner that is on board with using analytics to gain an edge. Cuban said that they were using advanced stats to evaluate NBA players when he took over the Mavericks in 2000. Getting organizational buy-in remains one of the bigger challenges for those providing analytics. It's one thing for teams to pay someone to provide analysis and information, it's another to take that information and use it in an actionable way that can lead to better results.

When it comes to organizational buy-in, Cuban noted that coaches think they can get the best out of any player, no matter their rap sheet or track record. That's who they are as coaches. General managers must take a wider view but, Cuban asserted that, "The number one job for an NBA GM is not to win championships, but to keep their job." It doesn't much matter, then, what the analytics say if those in decision-making positions aren't inclined to use the information, whether it's for self-preservation or some other external factor, rather than winning a championship.

Another recurring theme of the conference is the ongoing challenge that those involved in analytics have when trying to get their message across to those in the game -- general managers, coaches and players.

Basketball coach Stan Van Gundy, who was a star on several panels, with strong, thoughtful opinions that could be tied to real-life experiences, said that he was fine getting reports filled with pages of data and so on, but that he needed to have it boiled down to a message that could be presented to his players. The players don't need to know that an opponent shoots 12% worse when he dribbles at least twice before shooting (for example), but knowing that they should prevent a catch-and-shoot situation can help with a team's defensive strategy.

As much as the NBA has accepted analytics as part of their operations, and 29 teams were represented at the Sloan Conference (the Lakers were the only team not there), it's still a growth industry. Morey believes that, "The NBA is nowhere compared to where we are going to be," predicting that offences will be totally different in 10 years.

One reason to anticipate further NBA statistical advances is the use of SportVu, which is a recent advancement in data tracking technology, with cameras installed in 15 NBA buildings and those cameras track every on-court movement for all players, providing a staggering level of detail about what is happening on the floor. It's one thing to know that a player is shooting 43% from beyond the arc, but what about his shooting percentage on catch-and-shoot threes when a defender is within two feet with a hand up?

Dare to dream that the NHL might be inclined to have similar tracking data, but that could be well down the road.

One of the most entertaining panels of the weekend was the Predictive Sports Betting Analytics Panel, moderated by Jeff Ma and included Chad Millman, the Editor in Chief for ESPN the Magazine, Haralabos Voulgaris, a professional NBA bettor and Matthew Holt, the Director of Race and Sports Data for Cantor Gaming.

Voulgaris (aka @haralabob) took delight in tweaking Holt, the Cantor representative. There were several entertaining back-and-forth exchanges between Voulgaris and Holt, as Voulgaris kept challenging Cantor to take the big bets that they claim they are willing to take. At least year's conference, Holt expressed that Cantor wasn't afraid of any bettor and would accept $1-million individual wagers, but Voulgaris said he was currently limited to $5,000/game; a sizeable bet, he acknowledged, but a far cry from Cantor's claims.

The challenge with any of the analytics panels at this conference is that no one wants to give away too much information. When Ma pressed Voulgaris on some of the basketball models he had been using in the past, Voulgaris responded, "Maybe people are still using those models," then, after a beat, "Why are you trying to take food off my plate?"

While there were surely numbers geeks in attendance who loved the idea of making it rich in sports betting like Voulgaris, it's not an easy existence. Even though he's regarded as a very successful bettor, it's hard work. "I'm pretty sure I've spent more on analytics than some (NBA) teams," said Voulgaris. At the same time, for all the analysis he's using, it doesn't seem to be disputed that Voulgaris has struggled betting the NBA over the last couple seasons -- that's a pretty long time to weather the storm while models play catch-up.

I did like the point that Voulgaris made that NBA teams should be measuring their own models against point spreads because it provides an objective measure for whether the model is providing accurate forecasts or not. Basically, if it's profitable, it's probably a pretty decent model.

There was one panel called Beyond Crunching Numbers: How to Have Influence had a message that was brought up in several different panels and that was in regards to the challenges faced by those in the analytics community getting their ideas accepted, even by the teams for which they work.

Cade Massey, the panel moderator, noted, "One of the certainties about analytics is that we realize how uncertain things are." That can make it difficult to sell an idea because, after crunching the numbers, it's easier to know about probabilities and likelihoods, not 100% guarantees and it's much easier to sell someone when you are 100% in favour of the decision.

On another panel, Van Gundy made good points about establishing trust with coaches, because if they can see the benefits, they're going to be more willing to work through the numbers.

To that end, Browns President Alec Scheiner noted (I don't recall whether with the Browns or in his previous job with the Dallas Cowboys) that he noticed one of the stats guys showing up for work every day in a team t-shirt or sweatshirt and he finally asked him about it, thinking it wasn't necessarily the most appropriate form of attire, but it was a conscious decision by the analyst to dress more like coaches, the people he was hoping would be able to apply data into on-field improvement. Scheiner summed up his point, saying, "You have to spend a lot more time thinking about how to get a message across than the actual message."

As Kirk Goldsberry noted, the need is for "better people, not at doing stats, but at communicating stats" to decision-makers.

It was mentioned more than a few times how important it was for analysts to understand the culture of the team and that, if analysts show some recognition of issues facing coaches and executives that could help them gain greater acceptance.

One place that analysts may want to avoid going with their analysis is anywhere near former Toronto Maple Leafs GM Brian Burke, who didn't miss an opportunity to slam the stats community, even though the panel he was on -- Break-Ups in Sports -- was pretty far removed from the advanced analytics discussion.

Burke repeated lines that he used on last year's Hockey Panel, including, "Statistics are like a lamp-post to a drunk. Useful for support, but not illumination." Or, "Moneyball never won a championship."

Take issue with them if you like, and many did, but it's convincing someone like Burke, or his replacement, that presents a challenge to those in the objective analysis community.

My Twitter feed was full of people jumping on Burke's stance on Moneyball, but there also seems to be some misinterpretation of Moneyball as a stats-only approach, rather than a way to find under-valued assets. It just so happened that under-valued statistics, most notably on-base percentage, were what the Oakland A's used to gain their edge. The approach, however, doesn't only require stats. If fat players were being undervalued, the new market-inefficiency relative to their overall production (no matter the sport), it would be a "Moneyball" approach to sign fat players to contracts that provided value.

San Antonio Spurs GM RC Buford offered similar sentiments, during the Basketball Analytics panel, saying, "I don't think analytics is limited to number-crunching." It really isn't. That's just the starting point, for those willing to start.

Statheads, the ones that actually want to make a difference in sports, don't want to apply data without any other information -- the whole point is that using data to increase knowledge leads to more complete information and a better decision-making process.

As a writer, I use numbers to tell stories and, whether we want to believe it or not, they do. If I give you the most basic height and weight info. on a player, who is 5-foot-7 and 155 pounds, you have an image of that player in your mind. When I tell you that the player is 6-foot-4, 245 pounds, then you have a different picture of that player. Maybe the pictures aren't entirely accurate, but that is the beginning of a story. With more details, those stories can become more vivid.

If, for example, you consider the Ryan O'Reilly story I wrote a few weeks back, there is plenty of standard data (goals, assists etc.), but I linked to his player card at because, for those that are interested in and understand advanced stats, you can click on that link and the data presented tells you a more complete story about O'Reilly's ability to drive puck possession. It's not necessarily the whole story, but you're getting the story in HD if you know how to nagivate through the information.

A few other notes from the conference:
Former Jaguars head coach (and current Broncos defensive co-ordinator) Jack Del Rio revealed that RB Maurice Jones-Drew brought up fantasy points at the end of a close game when Del Rio had asked him to kneel close to the end zone.

Former Colts GM Bill Polian said that Indianapolis used stats benchmarks to find players to fit their system. For example, after analyzing the track record of previous middle linebackers in the system, Polian determined that the Colts needed their middle linebacker to run at least a 4.76 40-yard dash to play the "Tampa Two" defensive scheme.

Another Brian Burke beauty: "The worst thing that ever happened to sports was talk radio, and the internet is talk radio on steroids with lower IQs."

Paraag Marathe on the future of NFL stats: "There is still a lot uncharted territory like mental aptitude and injury prevention."

* Much to my chagrin, there was no Hockey Analytics panel at the conference this year, but there were a few presentations and some good discussions among stats-minded hockey individuals that were there. I'll have another blog later this week to cover hockey topics.

Scott Cullen can be reached at and followed on Twitter at For more, check out TSN Fantasy on Facebook.

Cabbie on

New York-bred hip hop artist Action Bronson discusses his friendship with Kevin Love, his jumpshot, Mike Napoli's beard, obscure sports references and Derek Jeter's Brand Jordan commercial. More...

He has speed in his DNA, learn more about Olympic champion Donovan Bailey's nephew, Jaden and his success on the gridiron in the latest Powerade 24. More...

© 2017
All rights reserved.
Bell Media Television