This post contains my notes for Continuous Discovery Habits: Discover Products that Create Customer Value and Business Value. The book is about identifying product opportunities. I like the ideas and the approaches the author recommends make sense to me, although they are more suitable for fast-moving companies and folks working more on the product side (vs infra).

Continuous Discover

Prerequisite mindsets

  • Outcome-oriented: measure success by impact rather than Outputs
  • Customer-centric
  • Collaborative
  • Visual: externalize your thinking by drawing and map what you know.
  • Experimental
  • Continuous

Definition of continuous discovery:

  • At a minimum, weekly touch points with customers
  • By the team building the product
  • Where they conduct small research activities
  • In pursuit of a desired outcome
  • Opportunity means customer needs, pain points, and desires collectively
  • Opportunity solution trees: a simple way of visually representing the paths you might take to reach a desired outcome.
  • The root of the tree is your desired outcome—the business need that reflects how your team can create business value.
  • Next is the opportunity space. These are the customer needs, pain points, and desires that, if addressed, will drive your desired outcome.
  • Then is the solution space. This is where we’ll visually depict the solutions we are exploring.
  • In the bottom are assumption tests. This is how we’ll evaluate which solutions will help us best create customer value in a way that drives business value.

Decision making

  • Villains of decision making (Chip and Dan Heath in book “Decisive”)
    • Looking too narrowly at a problem.
    • Looking for evidence that confirms our beliefs. This is commonly known as confirmation bias.
    • Letting our short-term emotions affect our decisions.
    • Overconfidence
  • Instead of asking “whether or not” , we’ll use compare and contrast

Focus on Outcomes over Outputs

Why outcomes?

  • The key distinction asking to solve a customer problem or to address a business need over traditional road maps is that we are giving the team the autonomy to find the best solution.
  • An outcome communicates uncertainty. It gives the team the latitude they need to explore and pivot when needed.
  • Managing by outcomes communicates to the team how they should be measuring success.

Different types of outcomes

  • Business outcome
    • Measures how well the business is progressing.
    • Starts with financial metrics (e.g., grow revenue, reduce costs), but also represents strategic initiatives (e.g., grow market share in a specific region, increase sales to a new customer segment).
    • Many however are lagging indicators. They measure something after it has happened. Therefore, we want to identify leading indicators that predict the direction of the lagging indicator.
  • Product outcome
    • Measures how well the product is moving the business forward.
    • Usually in control of the delivery team
    • Increases a sense of responsibility and ownership
  • Traction metric
    • Measures usage of a specific feature or workflow in the product.
    • Instances when it is appropriate to assign traction metrics to your team:
      • the team is junior
      • the product is mature and there are known critical traction metrics

Common anti-patterns

  • Pursuing too many outcomes at once.
  • Ping-ponging from one outcome to another.
  • Setting individual outcomes instead of product-trio outcomes.
  • Choosing an output as an outcome.
  • Focusing on one outcome to the detriment of all else.

Discover Opportunities

Building experience maps

  • Set the scope of the experience map. The scope should give you room to explore while staying focused on your outcome.
  • Start individually to avoid groupthink
  • Explore the diverse perspectives on the team
  • Co-create a shared experience map
  • Avoid common anti-patterns
    • Getting bogged down in endless debate.
    • Using words instead of visuals: drawing helps see patterns that are hard to detect in words
    • Moving forward as if the map is true: we need to test the understanding in customer interviews
    • Forgetting to refine and evolve your map as you learn more

Interview customers to discover opportunities and capture interview snapshots

  • The purpose of customer interviewing is not to ask customers what you should build, but to discover and explore opportunities.
  • Researches show that interview participants struggle to answer direct (factual) questions accurately, due to cognitive biases.
  • The key to interviewing well is to distinguish what you are trying to learn (your research questions) from what you ask in the interview (your interview questions).
  • The best way to learn about customers’ needs, pain points and desires is to ask them to share specific stories about their experience. You’ll need to translate your research questions into interview questions that elicit these stories.
  • The golden rule of interviewing is to let the participant talk about what they care about most. You can steer the conversation in two ways:
    • You decide which type of story to collect.
    • You can use your story prompts to dig deeper into different parts of the story.
  • An interview snapshot is a one-pager designed to help you synthesize what you learned in a single interview. It’s how you are going to turn your copious notes into actionable insights.
    • A quote that represents a memorable moment from their story. This might be an emotional quote or a distinct behavior that stood out. It’s a key to unlock your memory.
    • Quick facts of the customer. It builds the context.
    • Insights and opportunities.
  • Interview continuously to have continuous opportunities
  • Automate the recruiting process so you have people to talk to every week.
    • Recruit them while they are using the product or service, e.g. showing a survey on the website
    • Ask your customer-facing colleagues to recruit
    • Interview your customer advisory board
  • Common anti-patterns
    • Rely on one person to recruit and interview participants
    • Ask who, what, why, how, and when questions: these questions overwhelm participants. Use story-based interview questions that start with “Tell me about a specific time when…”
    • Interview only when you think you need it
    • Share what you learned by sending out pages of notes and/ or sharing a recording. Share your interview snapshots instead.
    • Stop to synthesize a set of interviews.

Map out and structure opportunities

  • Our goal is to address customer opportunities that will have the biggest impact on our outcome first.
  • Start by taking an inventory of the opportunities.
  • Keeping the opportunities in a tree instead of a backlog, with parent-children and sibling relationships
  • The value of breaking big opportunities into a series of smaller opportunities:
    • Allows us to tackle problems that otherwise might seem unsolvable
    • Allows us to deliver value over time
  • Common anti-patterns
    • Opportunities framed from your company’s perspective.
    • Vertical opportunities. Happens when a parent with one child. It is either because we have similar opportunities or we are missing sibling opportunities.
    • Opportunities have multiple parent opportunities. Happens when it’s framed too broadly.
    • Opportunities are not specific.
    • Opportunities are solutions in disguise. An opportunity usually has multiple solutions. A solution has only one solution.
    • Capturing feelings as opportunities.

Assess and prioritize the opportunity space

  • Focusing on one opportunity at a time. It allows exploring multiple solutions and setting up good compare-and-contrast decisions.
  • Use the following criteria to assess opportunities:
    • Opportunity sizing, i.e. how many customers are affected and how often? Size a set of siblings against each other, instead of sizing each opportunity precisely.
    • Market factors help us evaluate how addressing each opportunity might affect our position in the market. Some may be table stakes, while others might be strategic differentiators.
    • Company factors help us evaluate the strategic impact of each opportunity for our company, business group, or team.
    • Customer factors help us evaluate how important each opportunity is to our customers.
  • Don’t score and stack-rank opportunities. It is a messy and subjective decision and leave it as is. This leaves room for doubt and course-correct later.
  • Two-way door decisions
    • One-way door decisions: decisions that are hard to reverse. Be cautious, data-driven, and deliberate in our decision-making.
    • Two-way door decisions: decisions that are easy to reverse. It’s important that we frame our discovery decisions as two-way door, reversible decisions.
  • Common anti-patterns
    • Delay a decision until there is more data. The best way to prevent analysis paralysis is to time-box your decision and move on. Trust that you’ll course-correct if you get data down the road that tells you that you made a less-than-optimal decision.
    • Over-rely on one set of factors at the cost of the others.
    • Work backwards from your expected conclusion.

Discover Solutions

Ideate effectively

  • Brainstorming
    • introduced and popularized by Alex Osborn in “Applied Imagination”.
    • Osborn outlined four rules for brainstorming:
      • focus on quantity
      • defer judgment, and separate idea generation from idea evaluation
      • welcome unusual ideas
      • combine and improve ideas
    • Researches show individuals generating ideas outperformed brainstorming groups. The mitigating factors are:
      • People tend to work harder when working individually than when working in groups. This is called social loafing.
      • Brainstorming groups exhibited many of the common challenges associated with group conformity.
      • People lose ideas amid the chaos of everyone sharing ideas in rapid succession.
      • A common group trait known as downward norm setting — the performance of the group tends to be limited to the lowest-performing member.
  • Studies show that alternating between individual ideation and group sharing of ideas can improve the quality of ideas generated in subsequent individual ideation sessions.
  • Tips to get unstuck
    • Don’t try to spend an hour generating ideas. Ideate at different times and in different places.
    • Incubation occurs when your brain continues to consider a problem even after you’ve stopped consciously thinking about it. Incubation can be particularly helpful after hearing other people’s ideas. So, if you get stuck, sleep on it. Tomorrow will likely bring fresh ideas.
    • Look to analogous products for inspiration, and even unrelated domains for similar problems.
    • Think what your extreme users might need.
    • Consider wild ideas. Wild ideas can improve more feasible ideas.
  • Common anti-patterns
    • Not including diverse perspectives.
    • Generating too many variations of the same idea.
    • Limiting ideation to one session.
    • Selecting ideas that don’t address the target opportunity.
  • Evaluate ideas
    • “dot-vote” as a team: allot three votes per member, each person assign 1-3 votes to the idea that they think addresses the target opportunity, select the top three ideas that garnered the most votes.

Identify hidden assumptions

  • Types of assumptions: desirability, viability, feasibility, usability, ethical
  • Ways to generate assumptions
    • Story map your ideas
    • Conduct a pre-mortem
    • Use your opportunity solution tree to work backwards from your solution back to your outcome
    • Question potential harm
  • Assess and prioritize assumptions: put assumptions on a two-dimensional grid with x-axis being “strong evidence” on the left and “weak evidence” on the right, y-axis “more important” on the top and “less important” on the bottom. Start testing those in the top-most, right-most corner.
  • Common anti-patterns
    • Not generating enough assumptions.
    • Phrasing your assumptions such that you need them to be false.
    • Not being specific enough.
    • Favoring one category at the cost of other categories.

Test assumptions

  • Our goal is not to seek truth but to mitigate risk.
  • Common anti-patterns
    • Overly complex simulations. Design your tests to be completed in a day or two, or a week, at most. This will ensure that you can keep your discovery iterations high.
    • Using percentages instead of specific numbers when defining evaluation criteria. Be explicit from the get-go about how many people you will test with.
    • Not defining enough evaluation criteria. At a minimum, you need to define how many people to test with and how many will exhibit the desired behavior. Complex actions may require multiple measurements.
    • Testing with the wrong audience. Remember to recruit for variation. Don’t just test with the easiest audience to reach or the most vocal audience.
    • Designing for less than the best-case scenario. When testing with small numbers, design your assumption tests such that they are likely to pass. If your assumption test passes with the most likely audience, then you can expand your reach to tougher audiences. If you fail in the best-case scenario, your results will be less ambiguous. If your test fails with a less-than-ideal audience, someone on the team is going to argue you tested with the wrong audience, and you’ll have to run the test again.

Measuring impact

  • Common anti-patterns
    • Trying to measure everything. Start small and work toward measuring the impact of your product changes on your product outcome. And with time, work to strengthen the connection between your product outcome and your business outcome.
    • Hyperfocusing on your assumption tests and forgetting to walk the lines of your opportunity solution tree.
    • Forgetting to test the connection between your product outcome and your business outcome.

Managing the cycles

  • Common anti-patterns
    • Overcommitting to an opportunity.
    • Avoiding hard opportunities.
    • Drawing conclusions from shallow learnings. Discovery requires strong critical-thinking skills.
    • Giving up before small changes have time to add up. Over time, small changes can have big impact.

Show your work

  • Common anti-patterns
    • Telling instead of showing. There’s a cognitive bias called the curse of knowledge. Once we know something, it’s hard for us to remember what it was like not to have that knowledge. The key to avoid this is to slow down. Walk the stakeholders through what you learned and what decisions you made. Give them space to follow your logic and give them time to reach the same conclusion.
    • Overwhelming stakeholders with all the messy details. Tailor the detail and context.
    • Arguing with stakeholders about why their ideas won’t work. Use the discovery framework to help the stakeholder see where their idea does fit.
    • Trying to win the ideological battle instead of focusing on the decision at hand.

Start small, and iterate

  • Common anti-patterns
    • Focusing on why a given strategy won’t work (AKA “That will never work here”), instead of focusing on what is within your control.
    • Being the annoying champion for the “right way” of working. There is no “one right way” to do discovery. Don’t let perfect be the enemy of good.
    • Waiting for permission instead of starting with what is within your control.