next up previous contents index
Next: Legal aspects Up: Procedures Previous: Management

Wizard of Oz

  In a Wizard of Oz experiment, a speaker believes that he is communicating with a computer, while in reality the computer is simulated by a human. Wizard of Oz experiments are particularly useful in the simulation of computer-based spoken language systems  for a clearly limited task, e.g. information retrieval systems like telephone information services, train enquiries, automatic translation of spoken language, etc.

The setup of a Wizard of Oz experiment must be convincing. The speaker must neither over- nor underestimate the experiment. This requires that the speaker believes that a computer could potentially handle the task, and that the task is significant enough to require the use of a computer. Modifying the speech produced by the human impostor in such a way that it sounds machine-like, e.g. monotonous, mechanical, etc., may make the experiment more convincing. If the generation of a response to the speaker takes some time, give the speaker a convincing explanation of the delay to be expected; a progress indicator (which may be fake) on the computer screen will keep the speaker interested and waiting for a response.

A Wizard of Oz experiment should be developed in at least two phases: in the exploratory phase the human impostor is free (within the limits of the task) in the responses to the speaker, and all responses are logged. For the production phase, the log files are evaluated to result in an abstract script which covers the allowed and desired subtasks. The human impostor must then adhere to this abstract script for the experiment.

For a more thorough treatment of Wizard of Oz experiments, see Chapter 13.

Recommendations

  1. Carefully hide the human ``impostor''  from the speaker.
  2. Record both the speaker and the human impostor, and log all responses of the computer.
  3. Clearly explain to the speaker the limitations of the task and the computer.
  4. Develop a Wizard of Oz experiment in phases: in an exploratory phase get the experiment to work, in a production phase do the actual recordings.

 



next up previous contents index
Next: Legal aspects Up: Procedures Previous: Management

EAGLES SWLG SoftEdition, May 1997. Get the book...