Overview

Method category: Evaluative market experiment

How to Use This in GLIDR

In a Video Smoke Test, you make a video showing off the product to test early adopter excitement for your product and its potential virality.

In GLIDR, this test would be an Experiment where your success metrics would focus on expected views, shares, and/or conversions. In the Run phase, you would create one piece of Evidence - Other to track the video metrics. Finally, in Analyze, look back on the results to determine if your smoke test was successful or still needs work.

Learn more about each of those aspects of GLIDR:

---

Video Smoke Test

Article excerpted from The Real Startup Book

In Brief

The video smoke test consists of using a trailer video to test excitement and virality/buzz when describing the product to an early adopter audience. Unlike a traditional trailer, the video should create the illusion the product is real, in order to measure how much buzz it generates. The technique is particularly useful where user behavior can be easily simulated or demoed before a fully functional version of the product is ready.

Helps Answer

  • How well does the audience understand the messaging around the product?
  • Does the audience find the value proposition compelling?
  • Is the product buzzworthy, and does the target audience want to share it with their friends?
  • What channels are the most responsive/viral?

Tags

  • B2C
  • Value proposition
  • Referral
  • Product
  • Evaluative
  • Video

Description

Movie trailers have existed for decades. Book trailers have been used to successfully launch books. With a video smoke test, however, you are using video to explicitly run an experiment.

The difference here is that:

  1. The product does not have to exist yet, particularly if the product concept is very new and different from existing technology.
  2. The goal is to test how early evangelists actually react.
  3. Usually the key metric to observe is the virality coefficient, which is somewhat easier than it was with classic movie trailers.

Often the production of a video will require comprehension testing. In high-tech industries, it's common to create products that have clear benefits to geeks but are incomprehensible to the general population. A video used for a smoke test should use concepts that could be understood by a young child.

According to Lee LeFever, a good video:

  • Establishes empathy
  • Is unique
  • Is easy to understand
  • Communicates a key insight
  • Answers why the product exists
  • Gets prospects to care or hooks into why they would care about it

A good video also delivers enough impact that prospects want to share it with their friends in order to:

  • Appear "in-the-know"
  • Help address or identify the problem mentioned in the video

This is measured via the virality coefficient, as mentioned in the Broken Promise smoke test. In practice, it's possible to check for video virality by using social media platforms like Facebook or Twitter, particularly if they are used by the target audience. It's easy to use them to gather baseline organic sharing metrics.

Then some thought and planning needs to be put into promotion. In particular, channel testing is an important part of testing virality. Which Facebook groups? Which subreddits? Where should you guest post? Posting the video should also take into account the news cycle on social platforms, i.e., when is the right time to post it?

To some extent, this technique is also embedded in crowdfunding platforms like Kickstarter and Indiegogo.

Time Commitment

Depends on the expected production value of the video. Basic videos can be recorded and edited in 1-2 days; however, there is always the risk that the video will fall short because it doesn't deliver on one of the key criteria above. A full production can take 1-3 weeks.

To be effective, promotion planning starts a few weeks before posting. Actual work required depends on how much you want to promote the video, in order to get valid test results. As a minimum, you'd want to schedule at least 5 working days. More realistically, it requires at least a part-time effort of a few hours a day from idea to video launch.

How To

  1. First define your hypothesis, particularly your success criteria and the metric you want to test.
  2. Assuming it's referral rate, determine how exactly you are going to measure and track this rate.
  3. Plan out the key story or message you want to get across in the video. Ideally this product story should connect to the prospect emotionally.
  4. Create a marketing plan, based on your channel testing plan or results, ideally centered around a medium which has many people in your target market, is quite active, and will allow for sharing of the video. The video itself could be on a landing page or embedded directly into another site, i.e., FB video.

The following is adapted from Tim Ferriss' detailed breakdown on how to create a viral video trailer:

  1. Storyboarding: create a paper based "comic book" version of how the video should play out, with particular focus on the narrative
  2. Hire a video team (if you have budget): professionals will help increase the production value of the video, and in many case, the perceived value of the product goes up also.
  3. Scout locations: Choose backdrops that reinforce your message or are at least familiar and relevant for the target audience.
  4. Shoot principal footage: Take a lot of shots.
  5. Editing: Choose the best shots and arrange them.
  6. Add music: The right music should add to the mood and intended emotional tone of the entire video. Also make sure it doesn't overpower other important audio, such as "spoken word" narration.
  7. Launch/Promote: Try to get it out in front of as many (relevant) prospects as possible.
  8. Analyze: Tally up your metrics and figure out what referral rate you achieved.

Interpreting Results

Referral rate is difficult to accurately track, because most referrals (93 percent) in traditional businesses happen offline. Your best bet is to track aggregate exposure and then total number of referrals. If you only need a high-level metric this will be enough to pass/fail your original hypothesis.

Potential Biases

View count vanity: Views from people outside your intended target audience is a vanity metric.

Field Tips

  • "Keep your video concise and snappy, even when doing a video smoke test." @LaunchTomorrow
  • Got a tip? Add a tweetable quote by emailing us: realbook@kromatic.com

Case Studies

References

Did this answer your question?