Several commercial systems using speaker recognition techniques are now available on the market, mostly for speaker verification purposes. As with existing algorithms, it is quite difficult to compare them objectively. This fact stresses the need of standard evaluation procedures these systems: one possible approach is the definition of standard applications. Such a procedure should specify very accurately the nature of the applications, and integrate the speaker verification layer into a predefined structure of authentication layers. It is indeed clear from several examples, that the error rate on the pure voice verification algorithm can have very different overall consequences over the global score of an application, depending on the way voice verification is integrated with other verification layers. A drawback of such an approach is that the applications to which a system is targeted may be quite different from reference applications, and the evaluation figures may become somehow meaningless.
Performance is only one aspect of system assessment, and service providers will consider a number of other aspects:
Most of the aspects mentioned in this list are not possible to quantify. They are elements in the judgment that the service provider will take into account when deciding to retain a given technological solution for his application.