Systems selected for the Biometric Technology Rally will be evaluated on three categories: effectiveness, efficiency, and satisfaction. Effectiveness refers to the performance of the biometric, including failures to acquire, process, and match. Efficiency refers to the time it takes for Rally Participants’ system to complete a biometric transaction. Satisfaction refers to the degree to which users like or dislike using the system relative to others. To determine how well each system addresses the three categories, several metrics will be calculated.
To be considered for participation in the Biometric Technology Rally, systems should be able to meet the maximally accepted value for each metric. However, Rally Participants should aim to have their systems meet or exceed rally goal metrics.
| Metric | Category | Definition | Maximum Accepted Metrics | Rally Goal Metrics |
|---|---|---|---|---|
| Average Transaction Time | Efficiency | The time volunteers spend using the system | 10 seconds | 5 seconds or less |
| Failure to Acquire Rate (FtAR) | Effectiveness | Percentage of transactions that result in a failure to acquire or process image captures within a given time interval | 5% after 20 seconds | Less than 5% after 5 seconds or less than 1% after 20 seconds |
| Percent Satisfaction | Satisfaction | Proportion of positive satisfaction scores as rated by participants immediately following station use | Greater than 90% | Greater than 95% |
| Matching | Effectiveness | Percentage of transactions providing correct identity within a given time interval. This metric will be calculated by the MdTF matching engine; Rally Participants may also report matching, but it is not required | Greater than 95% after 20 seconds | Greater than 95% after 5 seconds or 99% or greater after 20 seconds |
| Average Transaction Time |
|---|
| Category: Efficiency |
| Definition: The time volunteers spend using the system |
| Maximum Accepted Metrics: 10 seconds |
| Rally Goal Metrics: 5 seconds or less |
| Failure to Acquire Rate (FtAR) |
| Category: Effectiveness |
| Definition: Percentage of transactions that result in a failure to acquire or process image captures within a given time interval |
| Maximum Accepted Metrics: 5% after 20 seconds |
| Rally Goal Metrics: Less than 5% after 5 seconds or less than 1% after 20 seconds |
| Percent Satisfaction |
| Category: Satisfaction |
| Definition: Proportion of positive satisfaction scores as rated by participants immediately following station use |
| Maximum Accepted Metrics: Greater than 90% |
| Rally Goal Metrics: Greater than 95% |
| Matching |
| Category: Effectiveness |
| Definition: Percentage of transactions providing correct identity within a given time interval. This metric will be calculated by the MdTF matching engine; Rally Participants may also report matching, but it is not required |
| Maximum Accepted Metrics: Greater than 95% after 20 seconds |
| Rally Goal Metrics: Greater than 95% after 5 seconds or 99% or greater after 20 seconds |