The Worst Outcome Is a Mediocre Success (summary)
Summary of an essay by Abraham Thomas, published in Pivotal on 21 September 2023. Read the full essay on Pivotal →
“The worst outcome is a mediocre success” is advice Abraham Thomas received as a founder: in startup experiments, a clear success and a clear failure both teach you something, while a small, ambiguous success teaches you nothing. The point of an experiment is learning, not the result itself, so experiments should be designed so that success and failure are unmistakable.
How are startup experiments like the scientific method?
Founders have to discover almost everything about their business, and one way to do it is the scientific method: form a hypothesis, run an experiment, confirm or disprove it, and iterate.
Why is a mediocre success the worst result?
Thomas’s example: hypothesis, “cold-calling customers will lead to sales”. You hire a couple of sales reps to cold-call 100 customers.
| Result | What you learn |
|---|---|
| 30 sales | The hypothesis is true: hire more reps and double down. |
| 0 sales | The hypothesis is false: move on to other channels. |
| 1–2 sales | Nothing. Is the method flawed, were the reps weak, the script wrong, the targets wrong, or were you lucky? You can’t tell whether to scale or stop. |
Why do people design experiments that produce mediocre successes?
- It is human nature to hedge: founders make the calls themselves, target only the best prospects, and offer sweetheart pricing. Each move raises the chance of a sale but tells you nothing about whether the channel scales.
- People prefer experiments they know will work, which yield almost no new information.
- Schools and large organizations reward consistency: “a steady 7 is better than 10s interspersed with 0s”. That is an anti-pattern in startups.
What should founders do instead?
Define success and failure clearly before running the experiment, and avoid “the messy middle”. Remember that a failed hypothesis means a successful experiment. Thomas notes that the essay is about tactics; mediocre success in strategy is a separate question, mostly about opportunity cost.
Related
- Full essay: The Worst Outcome Is a Mediocre Success, Pivotal, 21 September 2023
- Strong Opinions, Weakly Held (summary)
- The Perils of Prudence (summary)