Key Results Should Pass the “So What?” Test
Image by Someecards. Created by John2149306
For several years, I have been teaching people how to create great OKRs, and I have been able to test different approaches and see what works. I see my job not as providing training, but finding the best ways to help people learn. Think of it as running a series of A/B tests with thousands of people and being able to iterate based on what helps them write good OKRs.
There is one technique that started as a side comment—I didn’t even have a slide for it. I simply mentioned it briefly. But over and over, I saw people using this straightforward technique, and they kept telling me how useful it was.
The technique is called the “So What?” Test, and it is as simple as it is powerful. It can help you improve your OKRs by forcing you to create Key Results that add value. Here’s how to use it:
Imagine that your company has new board members and that you have 10 minutes to present your OKRs to them. They are very smart, aggressive, ambitious—and they don’t tolerate any BS.
To add more color, you can imagine that you are presenting your OKR to a character from your favorite TV show. If you’re a Game of Thrones fan, you can imagine you’re presenting to the dangerous queen Cersei Lannister. Or if you’re a fan of the TV show Billions like me, you can imagine billionaire investor Bob Axelrod (also known as Axe) and his former protegé Taylor Mason both invested in your company:
You: Last quarter we achieved all our OKRs.
Axe: What were your OKRs?
You: The first Key Result was to develop a new feature.
Taylor: So what?
You: Well, it’s available for our customers.
Axe: So what? If nobody is using it, why should we care?
You: Well, our second Key Result was to launch the marketing campaign, which we did.
Axe and Taylor: SO WHAT?!?!
Key Results should stand the “So What?” Test—they have to deliver value and benefits to your customers, your company, or your employees. Good Key Results make a difference.
Examples
Let’s see some examples of Key Results that do not pass the “So What?” Test:
- Create an engagement program. So what? Have we been able to actually increase engagement?
- Develop a new website. So what? Are we getting more customers? Has conversion improved? What is the benefit?
- Redesign the customer journey. So what? Have we positively impacted customer behaviors?
Using the “So What?” Test
The “So What?” Test can feel very aggressive and “in your face,” especially in the beginning. Asking “so what?” to your colleagues feels rude. But after a while, you realize that you are actually helping people, and the “So What?” Test becomes natural.
Key Results Have Numbers, But That is Not Enough
One great tip for spotting bad Key Results comes from former Google Vice President Marissa Mayer: “It’s not a Key Result unless it has a number.”
But having a number is not enough. Good Key Results make a difference—they stand the “So What?” Test. Beware of Key Results that simply count the number of things you delivered.
Here are some examples of Key Results that do not pass the “So What?” Test, even though they have numbers in them:
- Publish 5 blog posts. So what? Are those the right posts? Have we generated more leads or delivered value to our customers?
- Train 10 people. So what? Have they learned anything? Has their performance improved?
- Deliver 100% of my tasks. So what? How do we know those tasks generated value?
Focusing on Customer Behaviors
A few months ago, my friend Josh Seiden published a new book, Outcomes Over Output. In the book, Josh describes another great technique to create good Key Results, which is to focus on customer behaviors.
I highly recommend the book, which is a short read packed with actionable insights. In the book, Josh presents his definition of an outcome: an outcome is a change in human behavior that drives business results. To Josh, outcomes are “the changes in customer, user, employee behavior that lead to good things for your company, your organization, or whomever is the focus of your work.”
He explains: “We want our customers to log onto our site more often, or put an extra item in their shopping cart, or share an interesting article with a friend, or upload a picture, or complete a task in less time. What do all of these things have in common? They’re all measures of customer behavior. They might be small changes in a big system, but they are specific, and they allow our teams the flexibility to figure out the most efficient way to solve the problem, to deliver the behavior change that we seek, and to make a meaningful contribution to the impacts (revenue, profitability) that our executive leaders care about.”
Based on this definition, Josh tells us to ask: what is the customer behavior change that we are looking for?
To be clear, I don’t agree that every outcome can be translated into human behaviors. Some things can’t be easily mapped into behavior changes, like cost reduction, and trying to shoehorn them to fit Josh’s model would not add value. But Josh’s approach is very helpful to create customer-centric Key Results, and it is an important technique, especially for product teams.
Customer Behaviors and the “So What?” Test
We can use the “So What?” Test to help us select the right behaviors. After all, we need to find behaviors that drive business results, as we learned from Josh.
In the book, Josh lists one example of an OKR:
Successfully Launch our New Product
- 25 positive reviews in the app store on the first day
- 3 mentions on industry blogs before launch
- 1000 new user registrations in the first week
- 25% of new users convert to repeat users
Let’s see if those Key Results stand the “So What?” Test.
KR 1: 25 positive reviews in the app store on the first day. So what? It means people like our app, so this KR clearly passes the test. Just remember that the opinions of friends and family don’t count as measures of customer satisfaction, and asking them to post good reviews won’t help you achieve this KR.
KR 2: 3 mentions on industry blogs before launch. So what? Are those the right blogs? Are those mentions driving traffic to our app? This KR does not stand the test, and it should be reviewed.
Key Results 3 and 4 pass the “So What?” Test, as they show people registered and come back as repeat users. Depending on your strategy, you would also want to know if you are monetizing the app later on.
Conclusion
Writing good OKRs is hard, but by using simple techniques like the “So What?” Test or Josh Seiden’s approach, you can drastically improve your chances of succeeding.