Every Metric Is A Vanity Metric

Posted by Dr-Pete

Marketers can get caught up in specific metrics, focusing on those data points that make you look good in reporting, but don’t help you understand your performance. 

In this week’s episode of Whiteboard Friday, Dr. Pete discusses the vanity we bring to the metrics we track, and how to take a better, more realistic view of your results.

Click on the whiteboard image above to open a high resolution version in a new tab!

Video Transcription

Hi, everybody. Welcome to another edition of Whiteboard Friday. I'm Dr. Pete, the Marketing Scientist for Moz, and I want to talk to you today about vanity metrics. 

So I think we all have an intuition of what that means, but what I want to discuss today is I think we get caught up in this being about specific metrics. To me, the problem isn't the metrics themselves. The problem is the vanity. So I want to talk about us and what we bring to metrics, and how to do better no matter what the metric is. 

SEO metric funnel

So I want to start with this kind of simplistic SEO funnel of metrics, starting with ranking.

Ranking

Ranking via click-through rate delivers traffic. Traffic via conversion rate delivers leads or sales or conversions or whatever you want to call them, the money. Then beyond that, we might have some more advanced metrics, like lifetime value, that kind of get into revenue over time or profit over time. Naturally, over time we've moved down this funnel and kind of put our attention more at the bottom, at the bottom line and the dollars.

That makes sense. I think it's good that we've gotten away from metrics like hits. In the early days, when a page counted more because it had 200 images and 73 JavaScript files, that's not so great, right? We know now that's probably bad in some cases. But it's possible to hold that mirror up to any of these metrics and get caught up in the vanity.

I know we're used to this with rankings and traffic. We've all had customers that wanted to go after certain very specific head terms or vanity terms as we call them, that really weren't delivering results or maybe cost a lot or were very competitive.

Traffic

Traffic, okay, traffic is good. But if you've ever had a piece of viral content that went really big but ended up not driving any conversions because it had nothing to do with your site, you know that's not so great.

In fact, traffic by itself could be bad. You could be overloading your server. You could be stopping legitimate customers from buying. So bringing people to your site for no reason or the wrong people isn't that great. 

Sales and lifetime value

So I know it's easy to look at this and say, "Okay, but come on, sales. The bottom line is the bottom line." Well, I'll give you an example.

Let's say you have a big sale and you set everything to 50% off, and you bring in a ton of new sales and a ton of revenue. But let's say I tell you that your profit margins were 20%. Is that a good thing? You just cost yourself a lot of money. Now maybe you had another agenda and you're hoping to bring them back, or there's a branding aspect. But by itself we don't know necessarily if that's a great thing.

Just making more revenue isn't so great. Even profit or something like lifetime value, this is an example based in real life, but I'm going to change it a little bit to protect the innocent. Let's say you were a small company and you owned some kind of an asset. You owned some intellectual property, or you owned a piece of physical property and you sold that one year at significant profit, big margins.

Then you look and you say, "Wow, this year we made 50% profits, and next year we're going to try to make 70% based on that number." That would be a really terrible idea because that was a one-time thing, and you're not taking that into account. This is a bit of a stretch. But it's possible even to take profit or something like lifetime value or EBITDA even out of context, and even though it's a more complex metric or it's farther down the funnel, you could miss something important about what that number really means.

The three Rs

So that's the first thing. Is this a real result? Is that number going up necessarily good by itself? Without the context, you can't know that. The second thing where I think we really need to look at the entire funnel and not get focused too far down is repairs, fixing what's broken.

So let's say you track sales. Sales are going great. Everything is going well. Everybody is happy. The dollar bills are coming in. Then it stops, or it starts to drop significantly. If you don't know what happened above this, you can't do anything to fix it.

So if you don't know that your traffic dropped, if you don't know that your click-through rate dropped, and let's say your traffic dropped, you don't know why it dropped, which pages, which keywords, what rankings were affected, did you have lower rankings, or did you have rankings on less keywords, you can't go back and fix this and figure out what happened. So tracking that bottom line number isn't enough.

At that point, that has become a vanity metric. That's become something that you're celebrating, but you're not really understanding how you got there. I think we're all aware of that to a point. Maybe we don't do it, but we know we should. But the other thing I miss I think sometimes and that we miss is something I'm going to refer to as replication.

Yes, I tried a little too hard to get three R's in here. But this is repeating success. If something works and you get a bunch of sales, even if it's high margin, you get profitable sales, but you don't know what you did, you don't know what really drove that, where did the traffic come from, what was the source of that, was it specific pieces of content, was it specific keywords, what campaign was that tied to, you can't replicate that success.

So it's not just about fixing something when it's broken and when the dollars start to dry up, but when things go well, not just celebrating, but going back and trying to work up the funnel and figuring out what you did right, because if you don't know what you did right, you can't do it again. 

So three R's. Results, consider the context of the metric. Repairs, be able to work up the funnel and know what's broken. If things go well, replication. Be able to repeat your successes and hopefully do it again. 

So again, vanity, it's not in the metric. It's in us. You can have vanity with any of these things. So don't get caught up in any one thing. Consider the whole funnel.

I hope you can avoid the mistakes, and I hope you can repeat your successes. Thanks a lot, and I'll see you next time. Bye-bye.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!