The Matrix of MVP Tests

Definition

Rather than arguing whether a landing page "counts" as an MVP, Olsen reclassifies all such artifacts as MVP tests and sorts them along two dimensions: whether they test the product or the marketing, and whether they are qualitative (small-sample, learn "why") or quantitative (large-sample, learn "what" and "how many"). The resulting two-by-two matrix — qualitative marketing, quantitative marketing, qualitative product, quantitative product — gives every named MVP technique (Wizard of Oz, smoke test, fake door, five-second test) an unambiguous home.

In the Book

Chapter 7 resolves the "is a landing page a real MVP" dispute by noting both camps are really describing MVP tests with different learning goals, not competing definitions of one thing. Olsen details the landing page/smoke test as the canonical quantitative-marketing example: direct traffic to a page describing the unbuilt product, measure the sign-up conversion rate (his worked example: 250 of 1,000 visitors, a 25 percent rate), and use that number to validate demand before writing product code. The five-second test — showing customers a landing page for five seconds and asking what they remember — is the qualitative-marketing counterpart, testing whether messaging lands at all. He stresses that quantitative tests tell you what customers did and how many, while qualitative tests tell you why, and that teams over-relying on one type (usually quantitative) miss half the picture; mature teams alternate rounds of each as they iterate.

Why It Matters

Any validation effort benefits from asking two separate questions before choosing a method: am I testing the pitch or the thing itself, and do I need statistical confidence or rich explanation? Collapsing these into one binary ("is this a real test or not") produces exactly the unproductive debate the book dissolves — the matrix lets you pick the cheapest test that answers the specific question currently blocking you, in any field where you must learn before committing full resources to build.