next up previous contents index
Next: Recommendations Up: Influencing factors Previous: Speaker population size and

Speaker purpose and other human factors


The motivation for which a speaker is using a system also influences considerably its performance profile. We first describe a possible typology of applicant speakers  as regards their objectives. Then we mention other relevant human factors.

When the user's goal is conformant with the purpose of the system, a cooperative  (registered) speaker  can be defined as an authorised applicant who is willing to be identified or as a genuine speaker  who intends to be verified positively. Their counterpart in the impostor population would be a well-intentioned impostor , i.e. an impostor having the goal of being rejected.gif

When the user's goal and the system's purpose are inverse, an uncooperative (registered) speaker   knows that he is being verified but wants the system to reject him.gif For instance, an uncooperative speaker  is likely to use natural or artificial voice masking   in order to remain anonymous. In contrast, an intentional impostor  has the clear goal of being identified or verified though he is not registered (violation ), or to be identified as somebody else (usurpation ).

Here, a distinction must be made among intentional impostors  depending on whether they previously have or have not been in contact with the voice of the authentic user whose identity they are claiming. We propose the term acquainted impostor  to qualify an intentional impostor  who has some knowledge of the voice of the authorised speaker, as opposedto unacquainted impostors , when the impostor has never been in contact with the genuine or authentic user. The degree of success of an acquainted intentional impostor   will ultimately depend on his imitation skills.

The term casual impostor  is often used to qualify speakers who are used as impostors in an evaluation, but who were not recorded with the explicit instruction to try to defeat the system. In the same way, the term casual registered speakers  can be used to refer to a population of registered speakers  who have not received an explicit instruction to succeed in being identified or verified positively.gif

Here again, variants appear, depending on the way the experimenter chooses the claimed identity of a casual impostor  in a verification experiment. A casual impostor  can be tested against all registered users systematically,gif  against all other registered speakers  of the same sex , against all other registered speakers  of the opposite sex, against k registered speaker  chosen at random, against the k nearest neighbours in the registered population, etc.

Whereas, in a first approximation, a population of casual registered speakers  may be relatively representative of a population of cooperative registered speakers  , no test protocol using casual impostors  can accurately approximate the behaviour of intentional impostors . In practice, a real impostor could try to vary his voice characteristics for a fixed identity along successive trials, until he succeeds in defeating the system, gives up, or until the system blacklists the genuine user. Or he may try as many registered identities as he can with his natural voice or a disguised voice, until he succeeds, gives up, or until the police arrives!

However, most laboratory evaluations  use speech databases which have usually not been recorded in a real-world situation. Therefore they do not model accurately either cooperativeness or intentional imposture,  and the impostor speakers are casual impostors . A frequent practice is to use an exhaustive attempt test configuration, for which each impostor is successively tested against each registered speaker . We suggest adopting a slightly different approach. Two distinct experiments should in fact be carried out: one for which each casual impostor  utterance is tested against all registered identities of the same sex , and a second one for which each casual impostor  utterance is tested against all registered identities of the opposite sex . The first experiment permits estimation of the rejection  ability of a system towards unacquainted intentional impostors   who would know the sex  of the genuine speaker , even though casual impostors  are almost well-intentioned impostors . The second experiment tests whether the system is really robust to cross-sex  imposture.gif We will refer to these configurations as a selective attempt against all same-sex  speakers and selective attempt against all cross-sex  speakers respectively. In a first approximation, the proportion of successful violations  does not depend on the number of registered speakers .

In addition, testing  each impostor utterance against its nearest neighbour in the registered population can give an indication of the system's robustness against intentional imposture . However, the result will be directly influenced by the registered speaker  population size. Therefore this approach is only meaningful in the framework of a comparative evaluation on a common database. This approach can be qualified as a selective attempt towards the nearest registered neighbour. Other selective attempts are possible, such as towards speakers of the same age  class, for instance.

To summarise, registered speakers  should be qualified as cooperative , casual or uncooperative , whereas a distinction should be made between well-intentioned , casual  or (acquainted/unacquainted) intentional impostors    . Only field  data can provide realistic instances of user behaviour.

Additionally, the general motivation and behaviour of the users can have an impact on the performance of a system: for instance, what are the stakes of a successful identification or verification, the benefits of an imposture, the feeling of users towards voice technology in general, etc. In evaluation, all these aspects influence the motivation of the user, and therefore the interpretation of the results.

next up previous contents index
Next: Recommendations Up: Influencing factors Previous: Speaker population size and

EAGLES SWLG SoftEdition, May 1997. Get the book...