Method category: Evaluative market experiment

How to Use This in GLIDR

The Broken Promise Smoke Test is a great way to test the inherent virality of an idea by seeing how many of your users refer someone to your product despite you telling them to keep it secret. 

In GLIDR, you should run an Experiment for this test with a hypothesis about the cutoff rate for how many referrals indicate a success. Then, add Evidence - Other with the data from your test once you've completed it. Finally, in the Analyze phase, assess how well this test worked and update your project accordingly.

Learn more about each of those aspects of GLIDR:

Article excerpted from The Real Startup Book

People naturally want to share ideas or products they love with their friends. This smoke test helps test how much of a natural/inherent virality lies in the idea itself. It helps identify high-potential ideas before you've built a product.

This smoke test puts up a small social barrier (the social pressure of a personal promise) to see if anyone would consider the product idea worthy of sharing regardless of how developed it is. It tests the virality coefficient, or the "Referral" in Pirate Metrics' AARRR model. It is arguably a social variation of the landing page MVP.

This test is most relevant for early stage founders, particularly when testing a variety of ideas. It helps identify which of a number of ideas actually have relative merit. Ultimately, it helps focus your resources on an attractive and buzzworthy product.

In terms of the mechanics, it works similar to a landing page. There is a dedicated web page with a call-to-action. On the thank you page or in follow-up communication, the copy explicitly asks the registrant NOT to share the idea with others. After some time passes, you compare the original list of people you contacted with the actual list of signups. Anyone who wasn't in your original list of people you told about about the idea is counted as a referral.

Referral rate = (total number of signups in a time period/total number of original people contacted)

If this is above 100 percent, then you are seeing referrals.

With this smoke test, you aren't testing conversion rate but referral rate. Has anyone referred the product despite being told that it's not to be shared?

To the extent possible, it's also worth noting who is being referred. This provides additional insight on the ideal target profile for that product idea. These referrals should ideally be interviewed to discover more about them.

Minimal: 1-2 day developer and designer time, or you can use a viral landing page template like the ones at KickoffLabs.com

(from http://artofcleverness.stfi.re/broken-promise/?sf=agvkwwg#ab)

Interpretation is framed by how you originally formulated the experiment, i.e., what is the cutoff value for the referral rate in order to make a go/no-go decision? Also, be clear on exactly who you count, both in the original and referred group. For example, immediate family like your mom doesn't count (as well as anyone she refers) unless she really is representative of your target market.