How Validation Actually Works
Every time I talk to a group of startups I find that many of them were taught to validate by, “Go out, share your idea with a bunch of people, and see what they think.”
By now, I’ve honestly seen over a hundred cases where this advice led someone to invest years of their time and often their entire savings into a project that had “early validation,” because they received “positive feedback” — and then it fails to sell or get off the ground. And it’s tragic because they could have figured it out in a month with the RIGHT kind of testing at the outset.
The short version is, you CANNOT trust (consumer) interview-based or survey-based self-reporting for validation at the value proposition level. The false positive rate is very, very high in these circumstances.
(You can do it on a successful in-market product to prioritize feature development, but that is a completely different data story.)
You NEED to run an experiment that tests for true behavior change.
The good news is, you also do NOT need money, a design, or development to test for behavior change — you just need a concrete, handmade prototype and to get your user to let loose with it in a realistic scenario.
The other good news is, the false negative rate is ALSO very high — the truth is that very few negatives are actual true disproof. In my career, about 5% of the projects I’ve worked on actually turn into complete dead-end negatives. Instead, understanding WHY a user rejected something is often the most fun, exciting, and important key to finding the pivot to the right solution.
