The success of a product could, to some extent, be measured a posteriori. Some of the relevant figures are:
It may prove difficult to obtain some of these figures. In the context of secure access (either local or remote), one important aspect is the actual reduction of fraud. There is a cost/benefit tradeoff to examine.
The installation of a speaker verification system can be expected to have an impact on the population profile of the users. In particular, it can:
A service provider would indeed be much more interested in tools and guidelines to permit an a priori assessment. He must analyse the user needs and/or the cost/benefit tradeoffs of the application from (sometimes) conflicting perspectives. He could obtain some indications from an analysis of existing applications. He should compare alternatives initially on paper and eventually with simulations. In some situations (telephone services are good examples), a ``Wizard of Oz'' (WOZ) technique (see Chapters 3, 9, 13) could be set up to investigate the man-machine interface and the user acceptability. With such a technique, a real or simulated application is set up, but all decisions concerning speech recognition and/or user verification are taken by a human operator (``the man behind the curtain''). Fuzziness on these decisions could be introduced to simulate recognition errors.
Once the initial specifications of the product are set up, a prototype is realised and field trials are organised. This is a necessary step before launching a large scale application. The specifications should be flexible enough for possible readjustments. The prototype should allow for the monitoring (or at least the recording) of the user interaction. These recordings are used to fine tune the system.
The length of the speech material necessary for training and testing , the false rejection rate and the ergonomy of the implementation will influence user acceptability. For example, experimental evaluations indicate that some speakers (``goats'' ) may be rejected quite often. The test population should be large enough to show the distribution of false rejection . In fact, the overall false rejection rate of a system may not help determining its user acceptability. For instance, a system that rejects systematically 5% of the user population but accepts with no errors the remaining 95% may be better accepted than a system with a 5% false rejection distributed uniformly on the entire population. Therefore it may be more meaningful to evaluate the proportion of the client population for which the false acceptance rate is under a certain threshold, rather than the average false acceptance rate as is commonly done. To satisfy the ``goat '' population, a backup strategy (using a human operator for example) can be proposed.
With some applications (like banking), imposture could be discovered a posteriori. The system could record all the transactions, and the voice recordings could be used to trace the impostor . Legal issues related to these recordings and their use should be investigated carefully.
To conclude, voice verification technology is certainly at a stage where its future use is partly conditioned to the ability of the system manufacturers to convince the service providers and the end users of the advantages that can be gained from such techniques. This conviction can only be reinforced with the definition of well-established assessment tools.