·6 min read·startup-validation · founders · base-rates · forecasting

The outside view: what data can and can't tell you about your idea

Base rates can sharpen your judgment about a startup idea and rank what to de-risk. They cannot tell you whether you will win. Here is where that line sits, and why it matters.

Contents

There are two ways to judge a startup idea. From the inside, you look at the specifics: your insight, your team, your plan, the thing you can see that others cannot. From the outside, you ignore all of that at first and ask a colder question: of all the companies that looked roughly like this one at the start, how many worked?

Daniel Kahneman spent a career showing that people lean almost entirely on the inside view, and that it is where the planning fallacy lives. The outside view, the base rate for the reference class you belong to, is the correction. For founders it is also the single hardest perspective to hold, because your whole reason for building is a belief that you are the exception.

This post is about exactly how much the outside view can do for you, and where it stops. Both halves matter, because the tools now selling founders "data-driven validation" tend to overreach on the first and stay quiet about the second.

#What the data can do

It can give you an honest prior. The base rates are not gentle. About half of new US businesses are gone within five years. Of venture-backed companies specifically, roughly three quarters never return their investors' capital, and in one dataset of more than 21,000 financings, 65 percent returned less than the money that went in. That is your starting line before anyone hears a word about your idea. The outside view's first job is to stop you from quietly assuming you start at even odds.

~75%
of venture-backed companies never return their investors' capital (Shikhar Ghosh, Harvard Business School)

It can rank what to de-risk. Reference-class forecasting, the formal version of the outside view, is Bent Flyvbjerg's fix for the optimism that sinks big projects: find the class of comparable efforts, look at how they actually turned out, and start from that distribution instead of your plan. Applied to an idea, it tells you which of your assumptions is carrying the most risk, so you spend your first months testing that one rather than the one that is most fun to build.

It can be trained and measured. Forecasting is not a fixed trait. In a multi-year government tournament, Philip Tetlock's Good Judgment forecasters beat a control group by more than 50 percent, and beat intelligence analysts with access to classified information by over 30 percent, using no secrets, just disciplined, calibrated, repeatedly-scored judgment. The lesson for a founder is not "hire a forecaster." It is that being honest and calibrated about probabilities is a skill, and the alternative, confident conviction, is not the same thing.

#What the data cannot do

It cannot name the winner. Startup returns are a power law, and that is not a detail, it is the whole shape of the thing. In one large dataset, about 6 percent of investments generated roughly 60 percent of the returns. The best investors are wrong on most individual bets by design, and the winners routinely looked like bad ideas at the time: an air mattress in someone's living room, a side-project messaging app, a video-conferencing tool entering a market owned by giants. Any model that could reliably pick the 6 percent in advance would not be sold to you for a monthly fee.

It cannot resolve genuine uncertainty by pretending it is risk. A hundred years ago the economist Frank Knight drew the line that still matters here: "risk" is uncertainty you can measure and put a number on; "uncertainty" proper is the kind you cannot, and treating the second like the first is where confident forecasts go to die. Much of what determines a new venture's fate, whether a behavior catches on, whether a competitor moves, whether the timing is right, is Knightian uncertainty. Nassim Taleb's point about black swans is the sharp end of this: the outcomes that matter most are rare, unpredictable in advance, and obvious only in hindsight. A tool that hands you a crisp "72 percent likely to succeed" has quietly converted uncertainty it cannot measure into a number that sounds like it can.

#The honest use of the outside view

So the outside view is not a crystal ball and not an excuse. It is a discipline that does two specific things: it replaces your optimistic prior with an honest one, and it points you at the assumption most worth testing next. What it hands back is not a verdict on your future. It is a sharper question and a cheaper way to be wrong.

The failure mode to avoid is the one every over-confident idea tool falls into: dressing uncertainty up as a precise score because a number feels like an answer. The honest version says the quiet part out loud. Here is the base rate. Here is the one risk that most moves it. Here is the experiment that would resolve it. And here, plainly, is what no amount of data can tell you, which is whether you are the exception.

That last part is not a weakness of the method. It is the reason the method is trustworthy. The job of the outside view is not to predict the future. It is to make sure you are not the turkey, the one who mistook an absence of bad news for good news.

For the specific ways an idea dies and the base rate behind each, see the seven ways your startup idea dies and the Founder's Attack-Surface Checklist.

#Sources

Keep reading