SAM_SCOR is a software tool for scoring results from
automated testing of recognition devices attached to the SESAM
workstation.
SAM_SCOR aims to cover the main classes of
recognisers, from small vocabulary speaker-dependent isolated-word systems to medium/large vocabulary speaker independent
continuous speech recognisers.
For isolated word recognisers, standard statistics
comprise correct recognition, miss, substitution, correct
rejection and false alarm scores. Confusion matrices and
similarity matrices are listed and the confidence for the
conducted experiments is estimated. Special attention is given
to the scoring of recogniser testing in which CVC type words are
applied.
Other features are:
a detailed listing of the different error
conditions during testing (miss causes)
intergroup and intragroup scoring
the option of using the McNemar test for
comparing recognisers that have been tested on the same
database
For connected/continuous speech recognisers, SAM_SCOR acts
as a preprocessor for running the NIST string alignment and
scoring program (Release 3.0, modified to run on SESAM, and
distributed along with SAM_SCOR with the kind cooperation of
David Pallett, NIST).
Apart from the result files from testing a recogniser,
SAM_SCOR needs as input a scoring control file in SAM file
format, specifying the general conditions and type of scoring to
be carried out. This file may easily be produced manually (in a
text editor) or be generated by applying the SAM software
package RISE. The output of SAM_SCOR is both a standard
documentation file, which is easy to interpret, and a DBMS file
in SAM file format, which relates recogniser performance
measures to speech material and speaker characteristics.
Developing lab:
Speech Technology Centre
Institute of Electronic Systems
Fredrik Bajers Vej 7
DK - 9220 Aalborg, Denmark
