Cullen: Hockey at the Sloan Sports Analytics Conference

Scott Cullen
3/11/2011 6:31:56 PM
Decrease Text SizeIncrease Text Size
Text Size

After detailing my time last week at the Sloan Sports Analytics Conference here, I had promised a look at more hockey-centric themes from the conference in a future blog.

This is that stats-oriented stick-and-puck blog.

Part of the reason for making my inaugural trek to the conference was that this year was the first year that they had a Hockey Analytics panel. I know baseball is at the forefront of the stats movement and that basketball has been getting on the fast track in recent years but, going to the conference, I really wasn't sure how NHL teams use advanced analytics.

Now that I'm home, I still don't know, but my suspicion is that it's not nearly enough.

In any case, we'll begin this recap with the Saturday morning panel, moderated by Katie Burke, daughter of Maple Leafs GM Brian Burke and the Chair of the Alumni Executive Board for the conference.

The panel consisted of:
Stan Bowman, the general manager of the Stanley Cup-champion Chicago Blackhawks;
Jeff Solomon, VP of Hockey Operations and Legal Affairs, Los Angeles Kings;
Dan MacKinnon, Director of Player Personnel, Pittsburgh Penguins;
Don Fishman, Assistant GM and Director of Legal Affairs, Washington Capitals;
Jim Price, President, RinkNet

Bowman acknowledged that he broke into the hockey business on the finance side of things and used statistics to help improve his value on the operations side.

Admittedly, however, hockey's statistical analysis isn't the most refined. Bowman noted that the initial way of evaluating players internally consisted of having the coach give a player a rating (out of five) after each game.

The trouble with that subjective rating was that coaches tended to overvalue grinders in that format. With no expectations to score, a grinder would get a good grade provided they did the requisite skating and hitting, whereas a skilled forward that was expected to score would tend not to generate good ratings unless he produced offensively.

While the Blackhawks still use the subjective measurement, it's been combined with statistical measures that help smooth out the ratings. Bowman wasn't specific in detailing what stats the Blackhawks use, which is understandable, but there was no hinting that the Blackhawks have some special stats that they keep on their own.

When you consider the stats revolution that is taking place in basketball, for example, it's easy to grasp from Dallas Mavericks owner Mark Cuban and Houston Rockets GM Daryl Morey (two of the more prominent execs at the Sloan Sports Analytics Conference) that their organizations keep stats that aren't readily accessible in the public domain, but there wasn't even any hint in that regard from the hockey panel, that their teams have, say, a metric for clearing the zone successfully while under pressure (for defencemen).

I found Solomon to be interesting because he was a former player agent and, from an agent's perspective, I could see advanced statistics really coming in handy when it comes to contract negotiation time and given the reputation of a baseball agent like Scott Boras for statistical preparation as a source of leverage when it comes time to conduct contract talks, I wouldn't be surprised if there are hockey agents that are more progressive on the stats front than their counterparts in NHL front offices.

To that end, Solomon seemed to be open to more statistical input, but acknowledged that the NHL seemed to be at least five years behind the other sports when it comes to using advanced statistical measures.

MacKinnon and Fishman had what amounted to team marquee value, since they represented those involved in HBO's 24-7 Capitals-Penguins. While it was clear that both thirst for more information, it didn't seem like they were heavy into advanced statistical measures.

When MacKinnon made the point that he doesn't even know the stats lines of the Penguins prospects, insisting that the projection required for prospects is too great to get overly concerned about their junior or college stats, I was at least gratified by Bowman's assertion that if a player is being projected as a scoring forward in the NHL, then they have to have a certain level of scoring accomplishment in junior.

That's not to say a productive junior player is guaranteed to score as a pro, but it's very, very rare for a player to score at a higher rate as he moves up the developmental ladder.

Additionally, I simply can't imagine that there can't be metrics, especially when combined with scouting information, to help measure the likelihood of a prospect reaching expectations in the NHL.

Remember when Angelo Esposito scored 98 points as a QMJHL rookie in 2005-2006, then fell to 79 points the next year and yet the Penguins still drafted him 20th overall in 2007?

Even before the knee injuries that seem to have derailed his prospect status, the dip in Esposito's production (to 69 points in 2007-2008) had to be some cause for concern; something that, with the right metric, the numbers may have brought to light before the decline was so apparent.

Admittedly, part of the trouble when dealing with the statistics of junior and college hockey players is that there is such a wide variance in the calibre of teammates and opposition.

I still expect that the kind of bright minds involved at this conference would be able to figure out ways to include strength of teammates and opposition into the metrics, so that numbers aren't just thrown out entirely.

This is a common challenge for those in analytics, trying to get good data included, rather than thrown away altogether because it doesn't necessarily have the all-encompassing answer in a single number. Making a decision that is even 10% smarter is still worthwhile, believe it or not.

One of the more frustrating moments, for me, during this panel was a sequence in which MacKinnon, Fishman and Bowman all took shots at the value of plus-minus, because it doesn't measure strength of opposition and some players get "easy minutes" and/or players may be benefitting from playing with stronger teammates.

This is an entirely valid concern, but hardly limited to plus-minus.

I get it, plus-minus is hardly the be-all and end-all of stats, but doesn't who a player plays with and against affect everything they do on the ice? Don't tell me that playing on Sidney Crosby's wing brings the same expectations for goal scoring, or even shots on goal, as it does for those skating alongside Mike Rupp, and that's before we even get to who they match up against on other teams.

My view is that all stats need to put into context if they are going to have value. Go back to Esposito's rookie season in the Q, when he was so highly-touted after scoring 98 points. He was the second-highest scorer on his team that season, a mere 54 (!) points behind Alexander Radulov. Given that context, wouldn't that undermine the pure total of 98 points all on its own?

Conversely, one of this year's first-round prospects, Matt Puempel of the Peterborough Petes, is having a productive year, scoring 69 points in 55 games, ahead of teammate Austin Watson, who was a first-round pick of the Nashville Predators last summer.

Puempel is also minus-33 on a brutal Petes team that has 13 skaters with a rating at minus-20 or lower. When put into the context of his team's overall troubles, Puempel's value as a draft prospect doesn't seem to be affected much (if at all) by his plus-minus.

Fishman did, however, use context to bring Alexander Ovechkin's struggles to light, saying that Ovechkin's even-strength production was still good, but that his real struggles have come on the power play, a unit that has struggled as a whole for Washington.

Ovechkin has six power play goals in 68 games this season, well behind the pace of last year's 13 power play goals in 72 games that was his previous career low.

One of the presenters that I met with at the conference was Brian Macdonald, an assistant math professor at West Point, who had devised an adjusted plus-minus statistic that accounted for teammates, opposition, zone start and could be used as a measure for special teams as well.

Anytime you're dealing with advanced stats for the first time, it's nice to at least see some results off the top that make sense and while discussion with Brian revealed some of the surprised that he thought might be flukes (like Jason Pominville among the top even-strength defensive wingers), there's something to be said for identifying value in typically underrated players like Jan Hejda and Mike Weaver on the defensive metric.

I still need to spend more time going through Macdonald's numbers, but I obviously like his idea of trying to account for strength of teammates and opposition to devise his numbers because it helps provide context.

Michael Schuckers, an associate statistics professor at St. Lawrence University, comprised a defense-independent goaltender rating, which attempted to remove the quality of shot distribution that each goaltender faces, evaluating goaltenders based on a league-wide shot distribution and quality.

Again, the numbers require some investigation, but the result showed Ryan Miller as the best in the league, with numbers that indicated that the Sabres effectively surrendered a league-average quality of shots against last season.

At the lower end of the scale, for last season, Tim Thomas and Jimmy Howard were a couple of the goaltenders that would have fared substantially worse if given league-average shot quality.

On the surface, the theory is interesting and given the rise of fielding independent pitching in baseball, defense independent goaltending in hockey makes sense as a measure too.

I also ran into Adam Gold, who runs Adam has done some stats work for the St. Louis Blues, but his pet project is much more ambitious, trying to change the way the league sets its draft order based on when a team is eliminated from the postseason.

For a league that has shown zero interest in changing an already massively-flawed standings model, Adam has a big hill to climb, but part of the beauty of the Sloan Conference is getting unique ideas like his.

One of the presentations I didn't get to -- and I most defintely regret -- was e-mailed to me after the fact (the beauty of TSN's reach in the hockey community), Does Decision Order Matter? An Empirical Analysis of the NHL Draft, by Michael Brydon and Peter Tingling, assistant business professors at Simon Fraser who focus on decision theory when examining NHL drafts from 1995 through 2003, revealed just how random the results of the NHL draft tend to be.

Their findings suggest that, while some teams may hold an advantage in early rounds, a lot of teams do no better than random chance at the draft table and, even without advanced statistical measures being utilized, could be improved upon simply with better decision-making processes.

Now, if an NHL team really wanted to go crazy, maybe they could use improved decision-making processes and advanced statistical data to really come up with a comparitive advantage over the opposition.

On the Hockey Analytics panel, MacKinnon seemed most interested in having an expanded statistical reach in junior and college hockey, tracking more than mere goals and assists, but there was no real suggestion about how that would occur.

Is it valuable enough information that NHL's Central Scouting would be responsible for increasing the statistical measures in developmental leagues?

There does seem like a desire for knowledge but, as it is with any sports, it's fair to question how analytics can be applied to give a team an advantage.

When one considers that the Dallas Mavericks, for example, have a full-time statistical analyst (Roland Beech) on the payroll, and estimates at the conference indicated that there may be 20 NBA teams with someone in charge of providing analytics, Mark Cuban's position seems appropriate. "The way I look at it, relative to the cost," said Cuban in a recent Time Magazine interview, "if (Beech) wins me one game, he's paid for himself."

While there surely are some NHL teams that use analytics, I'm not yet sure how widespread it is in practice.  This league still seems awfully old school in a lot of ways.  No, really, it does.

The Minnesota Wild hired former sports writer Chris Snow to take on a position that would help them address contract negotiations, arbitration, free agency and the draft, but he was let go when a new regime took over.

While some teams may have consulting done when it comes to analytics, it still seems like an area ripe for improvement in many cases.

As mentioned in my previous post on the conference, scouting information can be combined with analytics, so there really ought to be more efficient methods of doing business for NHL teams. If a team's analytics department is able to help with an arbitration case, a draft pick, a contract negotiation, a savvy trade, isn't any one of those things likely to be worth the salary of one full-time employee?

Maybe they're all doing it and we just don't know about it but, as I say, I'm suspicious that this is an under-utilized method of evaluation, both in terms of player and contract value.

Part of the reason may be this unfortunate point about analytics in hockey that was presented by Fishman. As analytics would seem like a prime tool when going to arbitration with a player, one of the downsides of the process is that the arbitrators tend not to know hockey particularly well so the statistics that end up being used are very basic.

That being the case, it means that even trying to convince an arbitrator that adjusted plus-minus is a much better true measure of a player's value seems like a battle not worth fighting.

Can a hockey team embrace this kind of analysis?

Under a salary cap system in which teams at the top are forced to make ruthless financial decisions all the time and teams at the lower end of the pay scale are run on tight budgets, how can teams not adopt something that would allow them to make better decisions with their money?

I have lots of questions, but still need answers. Maybe someone could provide some analytics on this for me.

Scott Cullen can be reached at and followed on Twitter at For more, check out TSN Fantasy on Facebook.

Cabbie on

New York-bred hip hop artist Action Bronson discusses his friendship with Kevin Love, his jumpshot, Mike Napoli's beard, obscure sports references and Derek Jeter's Brand Jordan commercial. More...

He has speed in his DNA, learn more about Olympic champion Donovan Bailey's nephew, Jaden and his success on the gridiron in the latest Powerade 24. More...

© 2017
All rights reserved.
Bell Media Television