Metrics: What They Do

For the first time I can recall, I actually made a New Year’s resolution this year. Don’t give me too much credit — I stole it.

At an NYE party, a friend asked what people’s resolutions were for the upcoming year. “If you only make resolutions once per year, you’ve got to cut down your cycle time,” I scoffed.

My experience is that a “one year” timeframe is hopelessly ineffective for any type of behavior change. Daily or weekly is the way to go for most of the common resolutions people make. And for “bigger” goals — change fields, start a business, move to your dream location, get that promotion — a year can be unrealistically short.

“I make the same resolution every year,” my friend countered. “I try to read more books than I read the previous year.”

And I had to admit, that was a GREAT resolution. My friend had been doing it for 5 years running and her yearly number had gotten quite impressive — even under a very generous estimate, I was less than halfway there. And I found I couldn’t remember exactly what I’d read in the past year. She could.

I was jealous. So I stole the resolution.

That meant I needed to keep track of what I read. I already run a monthly book club so I made an extra tab in the spreadsheet to track my personal reading. And since everything was in a spreadsheet already, I set up some formulas.

I’m tracking how many days have gone by, how many books I’ve read, and then calculating my reading velocity — aka how many books I’m on pace to read this year.

I started out by reading an entire book on January 1 (book club was meeting Jan 2 and I’d been putting it off b/c of visiting family). That set me on pace for a totally unachievable 365 books per year and I’ve been steadily regressing since then.

60–70 seems challenging, but achievable

The Good Things

Directly as a result of this tracking spreadsheet, I have been much more motivated to read. Previously, I had a vague general sense that I “should” be reading more books, but now I’m much more consistently making the choice to read over TV/video games/internet than I was before. Knowing that my spreadsheet is there, with its “estimated books this year” trending downward every single day, has made it much easier for me to choose books over other activities.

I’m also much more motivated to finish books. To prevent our household from going bankrupt, we read almost entirely from the library so I’m used to being on a time limit. However, I often do that trick of checking out a bunch of ebooks then switching my Kindle to Airplane Mode so I have them indefinitely. Now that I’m tracking my pace, that hasn’t been necessary (though I still do it — force of habit). Entering an extra row into my spreadsheet and watching the “estimated books this year” metric shoot up is an extra bit of satisfaction that I look forward to as I close out a book.

Finally, I added some rows to my book tracker to track genre and author demographics. Left to my own devices, I wind up reading a lot of sci-fi by American authors, so these provide a bit of a nudge to search out more diverse writers and read books I normally wouldn’t pick up.

Not doing so hot this year…

The Bad Things

The big one: I’ve been tempted to game my metric by picking shorter books over longer ones. I’d heard the 880-page Seveneves was great, and in general, I like big, fat sci-fi tomes, but I hesitated. Instead I picked up a YA novel my wife had recommended (The Hate U Give, which was also good!).

I’ve also been more tempted to speed through books. I’ve always been more into finishing books rather than abandoning them and this metric has made that tendency much more pronounced. If I’m close to halfway through and I’m not into the book, it’s very very tempting to skim the rest as quickly as possible instead of putting it down and picking up something better (did that recently with the universally-acclaimed but personally-detested Sing, Unburied, Sing).

I’ve had to sweat some corner cases, namely graphic novels. I read GNs from time to time and I honestly don’t know whether I should count them or not. Some are easy “nos” (the Adventure Time ones I read to my kids), but ones that I read for myself should maybe count?

But worst is an undercurrent of fear that I might let go and start gaming the metric. Just start racking up graphic novels, novellas, YA books, re-reading Zelazny’s books of Amber, etc. Or that I read a ton this year, when it’s still exciting, then next year the bar is set too high and I get de-motivated. That I come to appreciate books not for the joy of what they are, but because they add an extra line in a spreadsheet. And that would truly be sad.

What It All Means

I was familiar with Campbell’s Law going in (having spent 8 years teaching in the public school system), but I was still surprised to find it hard at work in a situation where the metric is something personal, known only to me, and attached to no stakes whatsoever.

People respond to incentives. Metrics create incentives and therefore, are powerful. For my book-reading project, an incentive to read more is great. The negative effects are largely controllable, and if they really start hurting me, it’s easy to abandon the metric and do something else (I did, after all, still read Seveneves, all 880 pages.)

But when we measure metrics in other contexts, in contexts where those metrics are tied to something tangible — money, job security, an idea of “success” — we have to be much more careful.

On one project, we were redesigning a site that had a dozen or so pages with almost identical content. They were all cross-linked, and many pages on the site had redundant links to these pages. We proposed a design that consolidated into fewer, more focused pages and a link system that made it clear where the links were going.

Easy, right?

Well, a key stakeholder up the chain had a yearly bonus tied to the amount of traffic those pages got. Any attempt to reduce the number of pages or the number of links pointing to them was dead in the water. No amount of testing results would get him to budge. In the post-mortem, we learned we were the third UX group to (unsuccessfully) attempt to redesign that portion of the site.

The metric did what it was supposed to — it made it easy for that stakeholder to consistently choose the metric over other options. It rewarded him for doing so, and punished him for doing otherwise. So be careful when you use metrics. Make sure you know want you’re getting into.

And if you want to start your own book-reading journal, here’s the template I use.

Thanks for reading! If you made it this far, please clap so others can see it as well.