Hire Great Guessers

Hire Great Guessers-http://ow.ly/2AEnQ Harvard Business Review (repost)
8:30 AM Thursday September 2, 2010
Analytics are now king. And they should be. (If you're not already convinced, read Competing on Analytics, one of the best HBR articles I've ever read). It's so much easier to collect and digest numbers on your business than it was even ten years ago.

No less than 5% of your payroll should go toward data analysis. Who is your customer? What is she buying? How often? After what event(s)? Which version of the product sells better? At which price point? Which version of the packaging is more appealing? Which salesperson is more effective with which customer cohort? What zip code is responding most to your ad? How quickly/reliably/effectively does your product accomplish its stated goal, or your vision for it? How satisfied are your customers?

Your analysts should be setting up systems to collect these data streams and then chugging through the numbers to help you drive the company.

However! Test a bad premise, and your analytics will be pretty useless. If your entire question is wrong (e.g. "Do customers prefer the 200-ounce burger to the 250-ounce burger?"), you'll get bad answers.

Marty Cagan, a thought leader on product development, wrote a piece distinguishing "product discovery" from "product optimization". He explains that product discovery — prototyping, user testing — is the right way to identify "significant new functionality" and product optimization — analytics-based A/B testing — is the correct way to "optimize the user experience and/or business results of an existing product." His implicit point is that analytics about the customer experience are a waste until you have some early feedback on the product or feature idea to know whether you are even in the zone of test-worthiness.

Marty is right. (Full disclosure: he advises my company, ReputationDefender.) However, there's more to it than his essay suggests. The jump from discovery to optimization requires good guesses. Without good guessers, the project is doomed. Good guessers know what is worth investigating in the first place. And they have strong instincts — usually coupled with a knack for scrappy, lightning-fast research — into where the best bets lie. They are great not just at product dev, but at hiring, market development, strategy, vendor selection, advertising, and market segmentation and definition.

Take our burger example. Let's imagine three versions of the story:
Version 1. A bad guesser may actually entertain the notion that a 12-pound burger will have market appeal. Fire him politely but immediately. The focus group he wants to set up will drain time and resources.

Version 2. If we modify the scenario and call it a repositioning exercise, in which a giant patty is tested as a "family burger," to be ordered and consumed like a family-sized pizza, it might be worth a second look. But it's still not an obviously great guess.

Version 3. But what if the idea was to make a twelve-ounce burger with small ridges at the edges, so that the condiments don't slide out quite as easily? That could be worth testing.

The bottom line is that, if you start with a B- premise, even the best testing, optimization, and iteration on the idea will only ever get you to a B+ result. You need A premises to get A++ outcomes. And the double bottom line is that you need people with great instincts — great guessers — to get you A-level premises.

This idea is getting some scientific traction. Sanjoy Mahajan has taught a course at MIT on guessing and published a more serious tome on the topic, appealingly called Street-Fighting Mathematics. He teaches that analysis paralysis kills effective and timely solution-finding and that intelligent estimation is key to unlocking problem-solving speed and, ultimately, accuracy. Ask him or his trainees a question like "how many burgers are sold in America?" and they'll rapidly come up with a few simple minor premises and calculations to generate a pretty good answer. It's sorta like the McKinsey interview, but with some MITness mixed in to make it more believable.

Even if you can case-interview your way to identifying good guessers, finding the best ones is going to be subjective, a matter of guessing. You're going to have to trust your own instincts, and that you have good instincts in the first place. Finding great guessers is the essential first 10% that makes the remaining testing-and-analytics 90% radically more or less effective. And while it's possible to teach the rigor of testing and analytics, it's harder to teach great guessing, which is more like trying to teach someone how to be romantic or creative. Though I run my company on analytics, most of the time, if forced to make a choice, I'd hire a great guesser.

Michael Fertik is a repeat Internet entrepreneur and CEO with experience in technology and law. He founded ReputationDefender in 2006 with the belief that citizens have the right to control and protect their online reputation and privacy. Michael recently co-authored Wild West 2.0 which quickly gained acclaim as an Amazon.com Number 1 Bestselling Internet book. He has been named a World Economic Forum Technology Pioneer for 2011

Kevin Brown www.kbsinsight.blogspot.com