Method category: Evaluative market experiment

How to Use This in GLIDR

In a High Bar Smoke Test, you increase the difficulty or friction of the customer's onboarding to your product in order to gauge the seriousness of their intent to purchase.

In GLIDR, you will be running this test with an Experiment. First, create or connect ideas about customer conversation rate to an Experiment. Then, in the Run phase, conduct the test itself by releasing the process with increased friction, and once complete, add the results to Evidence - Other. In the Analyze phase, determine if this test was successful or not and update your project accordingly. 

Learn more about each of those aspects of GLIDR:

Article excerpted from The Real Startup Book
​

A High Bar test helps to gauge the customer’s willingness to pay without using any form of monetary payment. This particular form of smoke test is focused on having the customer go through a set of activities containing abnormal amounts of usability friction (e.g. a very long, complicated signup form) to gauge the customer’s desire for a particular solution. This can be combined with additional "pre-qualifying" to ensure that the customers who do take the action are exactly the right kind of customer for the product.

A high bar smoke test refers to using an additional behavioral filter to test actual purchasing behavior (even if you aren't charging money yet). Basically, you are assuming that a truly motivated customer will jump through any hoops you put in front of them.

This is a particularly useful technique, if:

It's worth establishing that customers are already "signing up" with a frictionless form of currency. For example, you are gathering a lot of prospect emails. You aren't sure how well this will translate into sales. Your conversion rate to email serves as a benchmark value. Assume you introduce additional steps (extra form fields, application criteria, additional meetings, and hurdles). If you continue to get a similar conversion rate, then this high bar is reached. The smoke test passes.

This technique corresponds with "lead scoring" in a B2B context. In short, you can prioritize the relevance of incoming sales leads based on their interactions with you. Usually, the three main criteria used include RFM (recency, frequency, and money). A prospect with recent interactions is more likely to buy from you. A prospect who's frequently "touched" you, i.e., opened emails, attended meetings, or fielded calls, is more likely to buy. A prospect who's already spent money elsewhere to solve the problem (or with you) are more likely to buy from you. This is already implemented in various CRM and email software packages.

An early stage B2B startup can run a high bar smoke test, which if passed can also serve to prescreen incoming leads. The more robust data collection process helps them prioritize future sales opportunities.

Ultimately, what matters is sales. This test helps provide a proxy for sales if it is not practical or possible to immediately sell. Even if your signup conversion rate falls, this might not proportionally impact your actual sales. If you weed out prospects for whom your product is irrelevant, this will not impact your sales.

Online: A few hours to a few days of developer time for a simple approach
Existing packages: Integration time
Offline: Depends on your existing approach/processes, and what you want to learn

This type of smoke test requires a lot of self-awareness and accountability, even compared to other smoke tests. If you get an unexpected result, there is a temptation to question your previous assumptions.

As with all smoke tests, do not ask about future behavior.