Stop wasting people’s time
1) Lean Startup
a) Validated learning
c) Innovation accounting
2) Don’t launch a rocket ship (Don’t plan so much!)
a) Instead of making complex plans that are based on a lot of assumptions, you can make constant adjustments with a steering wheel called the “Build-Measure-Learn” feedback loop.
3) The most important assumptions entrepreneurs make (test riskiest first)
a) Value hypothesis – whether a product or service really delivers value to customers once they are using it
b) Also know as wild guesses and hypothesis
4) A true experiment follows the scientific method.
a) Clear hypothesis that makes predictions about what is supposed to happen.
b) Then test those predictions empirically.
5) Experiment is a product (most engineers go right to D)
a) Do consumers recognize that they have the problem you are trying to solve?
b) If there was a solution, would they buy it?
c) Would they buy it from us?
d) Can we build a solution for that problem?
6) Positive changes in metrics became the quantitative validation that our learning was real. (ie: validated learning)
The only way to win is to learn faster than anyone else.
a) Build – minimum viable product (MVP). The MVP is that version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time.
b) Measure – determine whether the product development efforts are leading to real progress
c) Learn – whether to pivot the original strategy or persevere.
d) Planning really works in the reverse order
i) Figure out what we need to learn
ii) Use innovation accounting to figure out what we need to measure to know if we are gaining validated learning,
iii) Decide what product we need to build to run that experiment and get that measurement
a) The lesson of the MVP is that any additional work beyond what was required to start learning is waste, no matter how important it might have seemed at the time
b) Challenge to traditional notions of quality
c) Does the company already know what attributes of the product the customer will perceive as worthwhile?
d) Infinitely better than mere speculation or whiteboard strategizing, because it provides a solid empirical foundation on which to build future products
e) The vision entrepreneurs keep in their heads is of a high-quality mainstream product that will change the world, not one used by a small niche of people who are willing to give it a shot before it’s ready.
f) MVPs range in complexity from extremely simple smoke tests (little more than an advertisement) to actual early prototypes complete with problems and missing features.
g) If you are building the wrong thing, optimizing the product or its marketing will not yield significant results.
9) Types of MVP
a) Customer interviews
b) Smoke test (also video)/presale (kickstarter) – Done with marketing materials. Customers are given the opportunity to preorder a product that has not yet been built. A smoke test measures only one thing: whether customers are interested in trying a product.
c) Prototype/Model/Popup shop – Small-scale investment
d) Concierge minimum viable product – make sure the first few participants had an experience that was as good as possible, completely aligned with vision
e) Wizard of Oz – customers believe they are interacting with the actual product, but behind the scenes human beings are doing the work.
10) Innovation Accounting works in three steps:
a) First, use a MVP to establish real data on where the company is right now.
b) Second, tune the engine from the baseline toward the ideal.
c) Third, pivot or persevere
11) Sign of the need to pivot: the decreasing effectiveness of product experiments and the general feeling that product development should be more productive
a) Actionable – demonstrate clear cause and effect (otherwise, vanity metric)
b) Accessible – make the reports as simple as possible so that everyone understands them
i) cohort-based reports are the gold standard of learning metrics: they turn complex actions into people-based reports
i) test the data by hand, in the messy real world, by talking to customers.
ii) those building reports must make sure the mechanisms that generate the reports are not too complex.
d) Innovation accounting: product prioritization decisions, deciding which customers to target or listen to, and having the courage to subject a grand vision to constant testing and feedback
12) Cohort Analysis:
a) Every single day measure product’s performance with a brand new set of customers.
b) Instead of looking at cumulative totals or gross numbers such as total revenue and total number of customers, one looks at the performance of each group of customers that comes into contact with the product independently
c) Once your efforts are aligned with what customers really want, experiments are much more likely to change their behavior for the better
d) Split testing is your most valuable tool
e) Sign of a successful pivot: the new experiments you run are overall more productive than the experiments you were running before.
13) MVP allows a startup to fill in real baseline data in its growth model—conversion rates, sign-up and trial rates, customer lifetime value, and so on
14) Compare two startups. The first company sets out with a clear baseline metric, a hypothesis about what will improve that metric, and a set of experiments designed to test that hypothesis. The second team sits around debating what would improve the product, implements several of those changes at once, and celebrates if there is any positive increase in any of the numbers.
15) Types of Pivots
a) Zoom-in – feature becomes the whole product
b) Zoom-out – whole product becomes a single feature
c) Customer segment – solving a real problem, but for different customers
d) Customer need – customers have a problem, just not what you originally thought
e) Platform – change from application to platform (or vice versa)
f) Business architecture – high margin, low volume <-> low margin, high vlume
g) Value capture (monetization) –
h) Engine of growth – viral, sticky, or paid
i) Channel – deliver same solution through a different channel (ie: direct)
Growth hypothesis – how new customers will discover a product or service
16) Engines of growth
a) Sticky – Freemium like Evernote
b) Viral – more users as a consequence of normal use, like Facebook
c) Paid – Value of customer > cost of acquiring customer
Optimizing the whole suboptimizes the pieces.
17) Lean startup —actionable metrics, continuous deployment, and the overall Build- Measure-Learn feedback loop—necessarily cause teams to suboptimize for their individual functions. It does not matter how fast we can build. It does not matter how fast we can measure. What matters is how fast we can get through the entire loop.