In a Wizard of Oz experiment, a speaker believes that he is communicating with a computer, while in reality the computer is simulated by a human. Wizard of Oz experiments are particularly useful in the simulation of computer-based spoken language systems for a clearly limited task, e.g. information retrieval systems like telephone information services, train enquiries, automatic translation of spoken language, etc.
The setup of a Wizard of Oz experiment must be convincing. The speaker must neither over- nor underestimate the experiment. This requires that the speaker believes that a computer could potentially handle the task, and that the task is significant enough to require the use of a computer. Modifying the speech produced by the human impostor in such a way that it sounds machine-like, e.g. monotonous, mechanical, etc., may make the experiment more convincing. If the generation of a response to the speaker takes some time, give the speaker a convincing explanation of the delay to be expected; a progress indicator (which may be fake) on the computer screen will keep the speaker interested and waiting for a response.
A Wizard of Oz experiment should be developed in at least two phases: in the exploratory phase the human impostor is free (within the limits of the task) in the responses to the speaker, and all responses are logged. For the production phase, the log files are evaluated to result in an abstract script which covers the allowed and desired subtasks. The human impostor must then adhere to this abstract script for the experiment.
For a more thorough treatment of Wizard of Oz experiments, see Chapter 13.