Questions and Answers

Is the [January] webinar being recorded so Rally Participants can view it later?

The Rally Participant Training webinar was not recorded but the slides from this webinar will be available for your reference.

You mentioned a Slack channel. What's the name of the channel?

Please email to receive information regarding the Slack Channel.

Do you limit to two participants from a vendor during the VIP day?

Rally Participants may bring up to three individuals for the VIP day. If individuals will only be present for the VIP day they do not need to complete the NIH Human Subject Training.

Will there be internet access available for Rally Participants so they can do work while event is going on?

Rally Participants will not be provided internet access while at the MdTF. In the event that you will need internet access, please bring your own hotspot.

How soon will this webinar be available for viewing?

The slides should be available by 1/25 COB.

What is the email to request for slides?

The Slides are posted here. Please email to obtain the password.

What are the safety certifications for prototype devices?

Please use the link provided in the slides to verify the safety of your systems. Additional requirements may be brought up by the IRB and will be handled on a case-by-case basis.

What is the primary goal of the 2018 Biometric Technology Rally testing?

This Rally is primarily a test of unmanned/unattended collection processes. The Rally seeks improved ways to collect biometrics quickly and effectively.

What biometric modalities will be tested in the 2018 Biometric Technology Rally?

The Rally seeks to evaluate both face and combined face/iris biometric systems for DHS use-cases.

What data will be provided to Rally Participants?

Rally Participants will receive the following biometric data:

  • Before Testing (only to Rally Participants performing identifications):
    1. Gallery of face images for identifications.
  • After Testing:
    1. All biometric data collected by their systems.
    2. Gallery of face images collected during testing by MdTF staff.
    3. Gallery of iris images collected during testing by MdTF staff (only to Rally Participants providing Iris images).
    4. Report describing performance metrics for the Rally System relative to average.

How will DHS select the panel of biometric experts?

DHS plans to include representatives from DHS S&T, NIST, CBP, TSA, and DoD. The panel may also include representatives from critical infrastructure sectors.

Is this Rally a pre-requisite for commercial awards?

Not at this time.

Can a vendor submit white papers on more than one product?

Yes, vendors may submit white papers on more than one system. If a vendor has multiple systems selected to participate in the Rally, each system will be assigned to separate station.

When will DHS hold a Rally on the fingerprint modality?

This will be determined by emerging DHS priorities and available resources.

Will volunteers be provided any documents or devices for use during the Rally?

No. Volunteers will not be provided with a travel token (e.g. document or device).

Why have tokens been omitted from the Rally?

Rally Systems should be applicable to DHS use-cases where there is no token available. DHS is interested in seeing if a process based on a token today can be made token-less (e.g. UK Iris).

Does the Rally Participant system need to identify and handle unknown volunteers?

Yes. Only ~80% of the volunteers will be in the provided face gallery. Volunteers that are not in the gallery must be identified as such by the system.

Will biometric identification accuracy be evaluated for systems that do not submit identifications?

Yes. Biometric accuracy for systems that do not submit identifications will be calculated by matching submitted images on MdTF systems after the Rally.

Why have tokens been omitted from the Rally?

Rally Systems should be applicable to DHS use-cases where there is no token available. DHS is interested in seeing if a process based on a token today can be made token-less (e.g. UK Iris).

Does the Rally Participant system need to identify and handle unknown volunteers?

Yes. Only ~80% of the volunteers will be in the provided face gallery. Volunteers that are not in the gallery must be identified as such by the system.

Will biometric identification accuracy be evaluated for systems that do not submit identifications?

Yes. Biometric accuracy for systems that do not submit identifications will be calculated by matching submitted images on MdTF systems after the Rally.

Will iris images be provided for matching operations?

No. Iris images will not be provided to Rally Participants for matching operations. Following the Rally, submitted face and iris images will be matched on MdTF systems using multiple biometric matching algorithms and galleries.

How will an iris enrollment database be used?

All iris matching will be performed on MdTF systems after the Rally.

What is the specific traveler use case?

The Rally seeks biometric systems useful for various DHS biometric use-cases such as identifying previously encountered, or trusted individuals.

Can you provide a few examples of “known DHS use cases”?

The general use case of Rally systems will be to verify the identity of an individual. This may apply to travel and other security use cases. Some specific use cases may be security check points, or identifying individuals at restricted access-points.

Should Rally Participants consider Port Of Entry - land/air/sea?

Yes, these are some potential use cases for this activity. Others include physical access control systems for sensitive buildings or areas.

Should the system be unmanned/unattended?

Yes. Systems must be able to automatically submit at least one face image, through the API, in an un-attended operation mode. No Rally Participant staff will be present on the test floor during the Rally.

Is it assumed/mandatory for the volunteer to interact with a system or can they simply be instructed to walk through without stopping?

The volunteers will make good faith effort to perform tasks as indicated by the Rally Systems. Rally Systems are responsible for all signage/instructions that inform the volunteers of any required actions.

Is facial recognition of large, fast moving crowds across many cameras important?

This particular Rally is intended to evaluate the trade-offs between collection time and accuracy for individuals – not large groups. Individual volunteers will interact with test systems for less than 20 seconds each. Interaction/transaction times of less than 5 seconds are desirable. Your test system may also include multiple cameras, however, for the purpose of this test you will only be able to submit a limited amount of still photo and video imagery.

Who will provide the USB drive to transfer data, Rally Participants or the MdTF?

In order to receive the face gallery, the MdTF will provide a USB drive to transfer it to Rally Participants’ systems. After the gallery has been transferred, the USB drive will be given back to the MdTF.

For Rally Participants that are recording video, the MdTF will provide a USB drive to transfer video from Rally Participant’s systems to the MdTF.

To obtain the data your system collected during the Rally, Rally Participants are responsible for bringing a high-speed, encrypted data locker that is FIPS compliant to the MdTF.

When does the afternoon test session end?

The afternoon session does not have a strict end time. Each session will end when all human volunteers have left the MdTF.

Is a bitlocker encrypted hard drive (HD) sufficient for transferring data collected from the Rally?

Yes, a bitlocker encrypted HD is sufficient. Please ensure that any HD is FIPS compliant.

How do I access the CRADA?

The CRADA was emailed directly to Rally Participants.

When should the identification function be called? Before or after the capture API is called? The question is about the correct sequence of the function calls.

There is no correct sequence of function calls. All calls to the API can be thought of as independent operations, i.e. not related to one another. For example, a call to /v1/face-captures at time t=0 followed by a call to /v1/identifications at time t=1 does not imply the face in the first call was used to calculate the identification result in the second.

You mentioned "safe wavelengths" Can you clarify that? Can I assume NIR is ok?

Yes, NIR is considered safe depending on intensity. We will measure the intensity of the light off your system during installation, if applicable.

With the email acknowledgement due tomorrow (1/26), does today's webinar (and previous ones) constitute the SOW and Recipient Training materials?

Attesting to attending the webinar on 1/26 constitutes receiving Recipient Training materials. The SOW will be included in the CRADA document.

Will we have the ability to dial into our systems for remote diagnosis?

No, you will not be able to dial into your systems during the Rally. We will provide the capability to monitor your systems output via our backend. Any diagnosis will need to happen locally.

How does the MdTF verify that Rally Participants have used and tested the API? Will this be by the station ID that Rally Participants use when testing the API?

Once Rally Participants believe they have successfully submitted a test transaction with the API, please send an email to with the time of the transaction and the endpoints used.

Slide 18 of the Training Power point shows a picture of an example station. What do the green check marks on the monitors mean?

The green checkmarks on the monitors are for the Test Guides and ensures that the MdTF’s backend systems are functional.

How will the human volunteers go through each station? Will there be MdTF staff controlling the flow of human volunteers through each station?

Prior to the test, the human volunteers are provided with the instructions as shown on Slide 21 of the Logistics Webinar Slides. The MdTF will have staff at the front of each station scanning wristbands. The scanners are told to wait until the person prior has cleared the station before scanning the next individual. In this sense these staff members are responsible for the flow, however the pace is dictated by what happens in the station, i.e. the Rally Participant system.

What is the procedure for starting and stopping systems on test days? Is there preparation time for engineers to start hardware and software before human volunteers arrive?

You should arrive approximately 45-minutes before each test session. This will be the time that Rally Participants have to perform starting any hardware or software your system requires. After all human volunteers have exited the afternoon session, there will be a brief period for Rally Participants to power down their systems.

Can third parties observe Rally activities at the Maryland Test Facility?

Restrictions associated with the IRB-approved testing protocol and limitations of the MdTF physical infrastructure prevents us from supporting outside observers for Rally activities. There will be a technology demonstration day for US Government and critical infrastructure stakeholders (e.g. airports, airlines), please submit an expression of interest to for consideration. A post-rally webinar briefing the results is planned for the April/May timeframe that will be available for third parties participation.

Does the MdTF have an existing CCTV system or surveillance camera system in place, and will it accessible to Rally Participants

A camera system is in place, however, it will not be accessible by Rally Systems. The MdTF video system is used for diagnostics only. Rally Participants may request a copy of the MdTF video observations for their Rally System. Usage of this video will be limited to activities authorized by the testing protocol and informed consent documents approved by the IRB and signed by the volunteers.

Can Rally Participants assume they will be able to access a cloud-based environment such as AWS, Azure, or Openshift/Openstack?

Rally Systems will not have external internet access – all communication will be internal to the MdTF. Prior to the test the MdTF plans to host the MdTF Rally API in the cloud to validate/troubleshoot data submission before Rally System installation at MdTF.

Does all equipment need to fit within a 7' x 8' space?

Yes. However, if space permits, we may allow packaging materials to be stored elsewhere.

Is a physical gate required or recommended in the 7’x 8’ space?

A physical gate is not required. Participants are encouraged to develop creative approaches to optimize efficiency, effectiveness, and volunteer satisfaction. Please see the Rally Metrics page for more information.

Will the 7’ x 8’ space have a roof/overhead covering?

No. Rally Stations will be separated by partitions but will not have roofs or coverings above the partitions. Rally Participants will not be allowed to attach anything to the roof or ceiling of the facility. All equipment must fit within the 7’ x 8’ space.

What measures are in place to mitigate any adverse effects of MdTF API latency?

To reduce delays during the Rally, the MdTF API will be hosted on the local area network. The MdTF API only provides HTTP response status codes indicating success or error. Rally Systems do not need to wait for any reply from the API to complete a transaction. Rally Participants concerned with timing, may implement asynchronous API calls. To reduce any delays associated with image submissions, Rally Participants are encouraged to send the smallest images which they believe will result in high identification performance on MdTF systems.

Will volunteers be wearing hats or sunglasses?

During the rally no attempt will be made to introduce occlusions; however, volunteers may wear whatever they desire during testing at the MdTF. If you wish for these specific items to be removed, your system should provide appropriate instructions to volunteers. Please be aware that the testing dates will be during winter, so some volunteers may be wearing winter attire (scarfs, hats, etc.). Volunteers will be able to store their personal belongings but they may elect to bring it with them or wear coats, hats, and glasses during their visit.

Will the request to remove occlusion articles (sunglasses, etc.) count against the recognition capability or simply against volunteer satisfaction?

Occlusions (e.g. due to clothing or hairstyle) may be present during acquisition. These cases will not be marked separately and will be included in the calculated performance of Rally Systems. Any impact of instructions to remove said occlusion will be included in the Rally Station’s Transaction Time, Failure to Acquire Rate, and volunteers’ Satisfaction rating. Rally Participants should be mindful of how the human factors of their system’s process may impact Rally metrics.

Are Rally Participants limited on the type of camera used to acquire real-time images/video?

Rally Participants may use any kind of camera or cameras during the test as long. However, any all rally systems must meet Rally requirements and only data sent through the MdTF API will be used for evaluation. Further, all Rally Systems must meet standard safety regulations.

Rally Participants should consider the properties of the images acquired from their camera(s) and the time to it takes for systems to send images to the API. It takes approximately 600 milliseconds to send a pair of 512x512 iris images to the API and approximately 300 milliseconds to send a 512x512 face image. Any time required to send images to the API will be included in the Rally metrics

Should Rally Systems provide feedback to volunteers once the face has been captured?

There are no strict usability or signage requirements for Rally Systems; however, volunteers will not be instructed by MdTF staff on how to use each Rally System. MdTF staff will instruct volunteers that the purpose of each Rally System is to gather face/iris images and that they should attempt to use each system in accordance to any provided instructions. Rally Participants are encouraged to implement any usability components so that volunteers comprehend what to do, including when to exit the system.

Will any ID be requested from the volunteer?

Volunteers will have IDs to establish ground truth during the Rally. However, this ID will not be available to Rally Systems. Volunteers will not be equipped with any token or ticket by the MdTF and Rally Participants that are completing the optional identification requirements should not rely on a token or ticket being presented.

Is there any penalty or negative impact, if more than 3 images are captured for one person?

You may capture as many images per person as you like. However, the API will only accept 3 calls to the face-captures, iris-captures, and identifications endpoint per person. A 4th call will result in an error code being returned. Rally Participants who capture more images per person will need to have a triage capability to decide which ones to send.

Was it correct, the last image of the 3 submitted should be the best quality?

Yes, the last image submitted by each system should be of the best quality as this image will be used to evaluate Rally Participants’ system. As a corollary there is no downside to submitting low quality images early.

How are the Rally Metrics defined?

Definitions of the metrics are provided in the Webinar slide deck.

Will biometric accuracy be evaluated for systems that do not perform identification operations?

Yes. Biometric accuracy for all Rally Systems will be independently evaluated on MdTF systems after the Rally using ground-truthed galleries.

Will biometric accuracy be evaluated for iris systems?

Yes. Biometric accuracy for iris systems will be evaluated on MdTF systems after the Rally using ground-truthed iris image gallery.

Will the metrics be used to determine a winner or result in an award?

The first Biometric Technology Rally does not include a financial prize. Rally Participants will receive data and information to improve the performance of their systems. Rally Participants may choose to use specific DHS S&T test results to describe the performance of their systems to third parties. DHS S&T may use test results to identify systems for DHS operational tests or limited pilots and share results as market research with other government agencies. There is no guarantee of an RFQ or RFP after the Rally.

Will the results of facial and iris recognition performance be combined?

Rally metrics will not combine the results of facial and iris recognition; the computed True Identification Rate (TIR) for each biometric modality will be calculated separately.

What matching system will be used to calculate TIR for images received through the API?

We will use multiple commercial and government algorithms to compute TIR for face and iris images against relevant galleries.

Message Sent!
Error. Please try again.