Most of us believe we know what purpose our products serve. We assume we understand, we really know, our customers – what they want, and how and where they want it. After all, the amount of customer data at our fingertips is unprecedented. And yet, we’re not really getting any better at innovating for them. Renowned authority on disruptive innovation Clay Christensen, and Bob Moesta, a lean innovation expert who focuses on the creation, development and launching of new products and services, believe it’s because we’re not asking the right questions.

Christensen and Moesta are among the principal architects of the Jobs to Be Done theory, an innovation framework they’ve continued to develop, advance and apply to everyday business challenges. Since first introducing the concept in the mid-90s, they’ve come to one critical conclusion: understanding customers does not drive innovation success. Understanding customer jobs does.

“Over the past two decades, we’ve watched great companies fail time and again – and waste billions on go-nowhere R&D efforts – because they’re focusing on the wrong things,” the duo explains in a recent Harvard Business Review article. “Rather than looking at specific customer use-cases, they chase the false sense of security offered by sophisticated algorithms or market surveys, or they focus on technical improvements rather than customer needs.”

True customer insights – the kind that can drive breakthrough innovations – often come from unconventional sources; they reveal the unexpected, and provide clues that bridge the gap between what customers say and what they actually do.

The New York Times offers an intriguing case-in-point with Freshii, when it asked customers to share selfies posing with healthy, on-the-go snacks. The Toronto-based healthy fast food chain was stunned to see candy bars instead of kale chips and quinoa. But the surprising selfies shed light on an important realization: sometimes a sugary snack is a perfectly acceptable (even preferable) alternative to salad.

As Christensen and Moesta explain it, “Customers make the choices they make to bring a product or service into their lives not because they’re dying to purchase something, but because they have what we call a ‘Job to Be Done’ that arises in their lives. They’re struggling to make progress with something – in particular circumstances.”

With the potential to reframe industries, it’s a provocative – and pivotal – message that Christensen dissects and drives home in his forthcoming book, “Competing Against Luck: The Story of Innovation and Customer Choice” (Harper Business, October 2016), co-authored with his colleagues and collaborators Taddy Hall, Karen Dillon and David Duncan.

“When a consumer shows us an image of how they might hire kale in one circumstance and Snickers in another, it challenges us to think differently about how our products help customers make the progress they seek – not just what we expect them to seek in their lives.”