Measuring Everything is for Baking: Part 2
To read part 1, click the link below.
https://drewcarlsonhp.substack.com/p/measuring-everything-is-for-baking
If you’re ready, let go for part 2.
The Chart that Ruined my Career
Monday afternoon. Junior year of college.
The team would get to the rink before practice and huddle near my stall.
My designated locker was near a corkboard my coach had put up.
Every Monday he posted “The Chart.”
In the offseason, he had learned a famous D1 coach was using “data” so he did what any great coach would do and…
Copied and pasted it without any context.
Sounds like a recipe for disaster, right?
Yep.
The chart was a plus-minus tracking system with different statistical categories.
Okay, this doesn’t seem that bad, what’s the issue Drew?
The chart tracked:
goals
assists
shots
turnovers
hits
and a few other categories I don’t remember
The Problem
Every category was measured differently. Hits could only be 0 or positive numbers. Turnovers were always 0 or negative points towards your cumulative score.
The chart wasn’t “weighted” for what impacted the game more. And you were incentivized to “play safe.”
For example, Player A scored 2 goals (+2), created 6 scoring chances(stat not tracked) but made 5 turnovers.(-5)
He ended up looking “worse” than
Player B had 10 hits(+10) and dumped or chipped every puck out or deep that he touched which lead to zero “turnovers.”
Player A impacts the game, but Player B is better according to this chart.
See my issue yet?
My coach lived and died by this chart with certain players.
I was a playmaking, creative forward that would have flourished if he tracked:
shot assists
chances created
# of possession pucks turned into attacking pucks
But we didn’t track those. I tried to make plays all the time, creating my fair share of “turnovers.”
My Finnish teammate was 2nd to last with a large minus number… He started calling it “the leaderboard” because our numbers in golf were good enough to win any tournament by a landslide.
I was the worst player according to his chart. And I started to get scratched because of “data” alone. (Mind you, I was leading the team in 5v5 scoring when I started getting scratched) I was obviously super happy, so I went in and talked to the coach about the chart.
He immediately felt attacked and said, “do you think you can coach this team better than me? To which I should have responded, “yes.”
But you know… Power dynamics.
My argument about “the chart” that he didn’t want to hear was three-pronged:
That D1 coach that created this chart could have had weighted metrics
(for example, goals are worth 5, hits are worth nothing, contact that forces a takeaway is worth something… But hits and goals should not be worth the same)
He could have kept track of it without living and dying by it ( a reference check, not a reason to scratch players)
He could have scrapped it after one of his assistants said it was stupid in a meeting(He knew the coach “invented it” from reading an article, but it might not have lasted that long. Theory is great until it sucks in practice)
But my user experience as a player was ruined because someone chose to “measure everything” and be data-driven instead of data-informed.
Data Tells you What “Has Happened”
Data tells you what has happened.
He had 4 shots. He had 10 hits.
Data is noise… level one according to Alan Hall:
Good Coaching is Beyond Level 1
Great, he had 4 shots.
Where were they from? Why were they from the outside? How can we teach the player to get more shots on the inside?
Does he need to manipulate defenders better on the rush?
Does he need to relocate better in the offensive zone to create space?
Does he settle for outside shots because he’s afraid to make one more move to inside ice?
How can I teach this in a way that will transfer and help the player?
This is “coaching.” It’s deeper than data.
The other thing is looking at a chart and being mad because of numbers…
Also known as, “not coaching.”
On your way out bonus:
Semi-Related(but totally awesome thread… click to see it all)
Like this post? Find yourself reading every week off my Twitter posts or someone sharing with you?
Sounds like you’re ready to get them sent right to your inbox…