To structure our overview of TTS assessment tests we will discuss a number of useful distinguishing parameters, which partly overlap with earlier attempted taxonomies [Van Bezooijen & Pols (1990), Pols (1991), Jekosch & Pols (1994), Goldstein (1995), see e.g.,] and explain the relationships among them, before dealing with any specific assessment techniques. Figure 12.1 illustrates the relationships among the various dichotomies that make up our taxonomy. It will be apparent from the figure that the dichotomies are non-orthogonal. For instance, tests that have been developed to evaluate specific modules within a speech output system (glass box approach ), will only be used in a laboratory setting. The subdivision of human listener tasks will only be relevant when tests involve human listeners; therefore no task variables will be specified under automated test techniques. In the subsequent paragraphs we will outline and discuss the taxonomy of speech output assessment techniques following as much as possible the structure of Figure 12.1. Note that the dichotomies used are intended as contrastive dimensions so as to create a multidimensional space within which speech output tests can be located. The terms involved in a dichotomy should not be construed as labels identifying mutually exclusive approaches to speech output evaluation.
Figure 12.1: Relationships among dimensions involved in a taxonomy of speech output evaluation methods. Any path from the root drawn to any terminal that does not cross a horizontal gap constitutes a meaningful combination of test attributes.