Concierge Test

The Concierge Test is a way to test your product in a manual and high-touch way with a small set of early adopters

Jordan Duff avatar
Written by Jordan Duff
Updated over a week ago

Overview

Method category: Generative product research

How to Use This in GLIDR

The Concierge Test is a way to test your solution in a manual and high-touch way with a small set of early adopters, in order to gain deeper and more personalized feedback that you can iterate on quickly.

In GLIDR, a Concierge Test is Research since it generates many unstructured observations and ideas for how to improve your product based on this small-batch activity. After you figure out the basic structure of your solution, create a piece of Evidence - Other for each user you are performing the Concierge Test on and continuously record learnings, observations, and iterations in the Evidence. Once you want to move to automate parts of the experience and build your scalable solution, move into the Analyze phase to synthesize what you learned and figure out what to do next.

Learn more about each of those aspects of GLIDR:

---

Concierge Test

Article excerpted from The Real Startup Book

In Brief

Concierge is a technique to test the solution of a customer problem by manually performing tasks as a service. It is typically not very cost effective but can provide detailed information about how a solution can be created and what minimum viable feature set should be included in an automated and optimized product.

Helps Answer

  • Does the solution solve a real customer problem?

  • How can the problem be solved?

  • What is the minimum feature set required to implement a solution?

  • What are the greatest problems in effecting a solution for the customer?

Tags

  • B2C

  • B2B

  • Qualitative

  • Value proposition

Description

In a concierge test the value proposition is delivered as a service. Like a hotel concierge the focus is on a highly customized customer facing service. For this method you need to perform the tasks manually, usually with a few customers as it is not cost efficient to scale. However, the high customer touch allows you to get quality feedback from the targeted segment. The key advantage is that services can be almost instantly adjusted at a very low cost. Hence iterations based on insights from customer feedback are easily done.

To conduct a successful concierge test you need a clear and well-formulated value proposition. As an evolution of problem-solution interview techniques the goal is to test the solution and figure out if it matches your customer’s expectations. Design your value proposition as a service with the leap-of-faith assumptions in mind. When using the service your customers should go through the same steps as they would later with your actual product.

Deliver your service manually in a customized and personal way. To avoid being overwhelmed, start with just a small batch of customers. At this point you do not need a single line of code or automation. Even though it is inefficient and time consuming keep in mind that the direct customer touch you can learn from is more valuable. While delivering your service keep collecting customer feedback and adjust your service accordingly.

After some time you learn about your customers' expectations and what is really valuable to them. Gradually automate the parts of your service that work. Be careful not to run your concierge test forever! Keep automating and expanding your service until you are not getting new major insights.

Time Commitment & Resources

Concierge tests can be the most time consuming method as it requires manually solving the customer problem. For a complex B2B IT solution, a concierge solution can be a complete consulting engagement lasting many months. For a consumer, it might be as simple as personally going shopping with a customer.

Similarly, the method can require substantial resources or nothing but a pen and paper. In the case of a B2B concierge service, it is often possible to charge for the solution up-front, which eliminates resource constraints.

How To

  • Write down the value proposition that needs to be tested.

  • Design the value proposition as a personalized, customer touching service.

  • Talk to potential customers (early adopters) and offer them your service in exchange for payment.

  • Execute the service by performing tasks manually even though it is not effective.

Interpreting Results

You will collect mostly qualitative data as you are delivering a manual service. You need to aggregate the data from all your current customers for the various aspects of your service. Use the insights to adjust your service accordingly.

The main benefit of this method is to generate ideas around the potential solution/product and identify obstacles to implementing that solution.

Potential Biases

  • False Positive Bias: This method does not serve to validate the solution as the manual component provides an extra value proposition of trust and responsiveness. Entrepreneurs can therefore mistake positive feedback on the service as validation of the product concept. When moving to an automated solution, the extra “human” value proposition is removed and the customer can reject the solution.

  • Sampling Bias: As the concierge test is manually performed you have to find a balance. On the one hand having too many customers can be overwhelming. You find yourself or your team just busy to deliver the promised service, which leaves very little time to analyze the data and use the insights to make adjustments. On the other hand, you have to make sure that your customer batch is not too small. Insights you get from just one or two customers might not be enough. You risk that the collected feedback is not representative for your customer segment you are targeting.

Field Tips

  • “A concierge test is an experience not a product.” @poornima

  • Got a tip? Add a tweetable quote by emailing us: realbook@kromatic.com

Case Studies

References

Did this answer your question?