Method category: Evaluative market experiment
How to Use This in GLIDR
A Comprehension Test helps you figure out if your customer even understands your value proposition before you try testing if they are interested or not.
In GLIDR, you can run an Experiment to test an idea about a particular Value Proposition. Then, run your test with a representative target customer and add the results as Evidence - Interview for each user you test with, or one Evidence - Other for all of them if you have larger group. Finally, in the Analyze phase, decide if you hit your target metric for comprehension and if the test was successful.
Learn more about each of those aspects of GLIDR:
Article excerpted from The Real Startup Book
A comprehension test will evaluate whether the customer understands the marketing message explaining the value proposition. This eliminates a possible false negative bias on smoke tests where the customer indicates they do not want the value proposition when they actually do not understand it.
Does the customer understand the value proposition?
How could we explain the value proposition better?
Time Commitment and Resources
For B2C, it can take 1-2 hours offline or 24 hours online.
For B2B, participant recruitment times can vary widely.
Write out value proposition in 1-3 sentences.
Show the value proposition to a participant for a few moments, then remove it.
Ask them to explain the value proposition in their own words from memory.
If the participant’s explanation is roughly comparable to our own, we count that as a positive result. If not, then it’s a negative. For this sort of test, we generally want a sample size of about 20 people and a positive conversion of about 80 percent.
The conversion has to be very high because regardless of what our value proposition is, people should understand it.
Take note: if many of the participants use identical language to explain the value proposition back, it should be considered as possible alternative marketing messages.
Confirmation Bias: Overly enthusiastic entrepreneurs will sometimes over explain, correct, or nonverbally prompt the participant with the correct answer.
Invalid Target Audience: Participants do not need to be the target customers, but they must have the same level of language and vocabulary as the target customer, e.g., a junior marketing manager can be used instead of a chief marketing officer.
False Negative: When using online surveys such as FiveSecondTest, the distractions of an online test can often result in a higher than normal failure rate.
“Run a comprehension test before a landing page test or you won’t understand why it doesn’t work.” @TriKro