Method category: Evaluative product experiment

How to Use This in GLIDR

A Wizard of Oz test involves setting up a way for humans to manually perform a task in your product that you later want to automate, but for now don't have the technology or other capabilities for doing so. The users don't know that humans are performing their tasks manually, so to them it feels like "magic"; later on, if it is successful, you can decide if you want to invest the time in automating these same features.

In GLIDR, a Wizard of Oz test is an Experiment that will involve quite a bit of preparation and set-up. In the Plan phase, decide what level of engagement, growth, or other user metric you want to hit. This is also when you should figure out how exactly you will simulate the "automated" aspect of your product and who will be performing these tasks. In the Run phase, keep track of your key metrics via Evidence - Other as you simulate your product features to your users. Once you've gathered sufficient Evidence, move it to Analyze and figure out which pieces of the product to begin automating, which to remove, which to add, etc.

Learn more about each of those aspects of GLIDR:

Article excerpted from The Real Startup Book
​

The Wizard of Oz (WOZ) test involves one or more individuals simulating fully functioning product features without the use of technology. All inputs, outputs, and algorithms are performed manually by humans, but without the knowledge of the customer. This allows entrepreneurs to test a complete solution and gain accurate customer feedback without the cost of building a complex, fully automated solution. It is often confused with concierge testing.

The WOZ test is a well-known experiment from the 1970s. It involves one or more humans, so called wizards, who manually simulate the functionality of a product. However, the wizards are hidden so that the user does not know that someone is manually doing the work.

WOZ is not a "quick and dirty" prototyping method as it requires a high commitment of resources. Depending on the test there can be more than one wizard (human) interacting "behind the curtains" with the customers. The time commitment also heavily depends on the task domain and the number of users exposed to the product. For instance, categorizing images by a wizard (human) is easier and faster than transcribing voice for speech recognition. Hence the time commitment can vary from days to even several weeks.

The WOZ test can provide valuable information, especially quantitative data. We can also gather information about the nature of interaction with our product. Summarize all observations and use the insights to adjust our solution accordingly.