Facial Recognition Technology
How facial recognition converts facial imagery into mathematical representations, compares a probe image against one or many reference images, produces candidate results, and assists law-enforcement investigations—and what agencies should understand about image quality, thresholds, human review, demographic effects, misidentification, watchlists, evidence, privacy, disclosure, and governance.
What this explainer does
Facial recognition technology uses computational analysis of facial imagery to compare faces. Law-enforcement systems may compare a photograph of an unknown person against a gallery of known photographs and return possible candidates for further investigation.
The technology can help investigators develop leads when a person's face appears in surveillance video, a photograph, digital evidence, or other imagery but the person's identity is unknown.
Facial recognition can be powerful, but its output is probabilistic. A returned candidate should not be treated as automatic proof of identity. Image quality, gallery composition, algorithm performance, thresholds, demographic effects, and human review can all affect the process.
A facial-recognition candidate is an investigative lead—not a positive identification.
Identity should be confirmed through independent evidence before arrest, charging, warrant affidavits, or other consequential action.
1. Overview
Facial recognition technology uses algorithms to measure and compare facial characteristics in digital images.
In a typical investigative application, law enforcement possesses an image of an unknown person. That image is submitted to a facial-recognition system and compared against a gallery containing photographs associated with known or previously identified individuals.
The system calculates similarity and may return one or more candidate photographs. A trained reviewer then evaluates those candidates and the underlying imagery before investigators determine whether further investigation is warranted.
2. Face Detection, Verification, and Identification Are Different
| Function | Question | Example |
|---|---|---|
| Face Detection | Is there a face in this image? | Software finds faces so they can be blurred, cropped, or analyzed. |
| 1:1 Verification | Are these two images likely the same person? | Comparing a live image with the photograph associated with a claimed identity. |
| 1:N Identification | Who might this unknown face be? | Comparing a surveillance image against thousands or millions of gallery images. |
3. How a 1:N Facial Recognition Search Works
4. The Probe Image
The image submitted for comparison is commonly called the probe image.
Probe quality can dramatically affect performance. A clear front-facing booking photograph presents a very different technical problem from a blurry surveillance frame showing part of a face at an angle.
Too few facial pixels can reduce usable identifying information.
Profile views or extreme head angles can reduce comparability.
Strong shadows, backlighting, darkness, or overexposure can alter facial detail.
Masks, glasses, hats, hair, hands, or other objects can obscure features.
Fast movement or poor camera exposure can degrade facial structure.
Repeated screenshots, messaging applications, and video compression can eliminate detail.
5. The Gallery Matters
A 1:N search compares the probe against a collection of reference images commonly called a gallery.
Different systems may use different galleries, such as mugshot repositories, agency booking photographs, missing-person files, authorized investigative collections, driver's-license photographs where permitted by law, or other authorized image repositories.
6. Similarity Scores Are Not Probabilities of Guilt
Face-recognition systems typically calculate a numerical measure of similarity between facial representations.
The meaning of the resulting score depends on the algorithm, threshold, gallery, configuration, and vendor.
7. Candidate Lists
Many law-enforcement 1:N systems return a ranked candidate list rather than one definitive identity.
The algorithm is effectively saying: these gallery images are among those most similar to the probe under this system's comparison method.
A candidate may appear first, second, fifth, or elsewhere in the returned list.
The system may exclude candidates falling below a configured similarity threshold.
Searching millions of images creates a different false-candidate environment than comparing two known images.
8. Image Quality Can Dominate the Result
Agencies sometimes focus heavily on algorithm accuracy while underestimating the importance of the probe image itself.
Surveillance imagery can be particularly difficult because cameras are frequently mounted for scene coverage rather than biometric identification.
Overhead cameras may capture the top of the head more clearly than facial features.
A face occupying only a small portion of the frame contains limited information.
The best frame may occur before or after the moment investigators initially notice.
9. What Does “Accuracy” Mean?
Facial-recognition performance cannot responsibly be reduced to one universal accuracy percentage.
| Measure | Question |
|---|---|
| False Positive | How often does the system incorrectly associate different people? |
| False Negative | How often does the system fail to recognize the same person? |
| False Positive Identification Rate | In 1:N searching, how often does a non-matching search yield an incorrect candidate above threshold? |
| False Negative Identification Rate | How often does the true person fail to appear appropriately in the candidate result? |
| Rank Performance | How often does the correct individual appear within the first 1, 5, 10, or other candidate positions? |
| Operational Error | How often do algorithmic, human, procedural, or investigative errors ultimately produce a wrong identification? |
10. Demographic Effects
Independent testing has demonstrated that facial-recognition algorithms can exhibit different error rates across demographic groups. Performance varies significantly among algorithms.
These differences are not uniform across every vendor or every type of error. Agencies therefore should evaluate the particular algorithm, version, operating environment, and use case they actually deploy.
Error differentials have been documented across population groups in facial-recognition evaluations.
Performance differences can occur across sex categories depending on algorithm and dataset.
Age and aging can affect comparison performance, particularly when reference and probe images are separated in time.
11. Human Review Is Essential—but Humans Can Also Make Errors
Human review is an essential safeguard because algorithms identify similarity rather than proving identity.
But human review must be structured. A reviewer can be influenced by expectations, rank order, contextual information, confirmation bias, or visual similarity among different people.
Reviewers should understand facial comparison and system limitations.
Reviewers should evaluate whether the probe is actually suitable for comparison.
The review process and basis for advancing or rejecting a candidate should be documented.
12. How Misidentification Can Happen
13. Watchlists and Real-Time Recognition
Some systems can compare faces observed by cameras against a predefined watchlist rather than waiting for investigators to submit a photograph after an event.
This changes the risk profile substantially.
People may be compared automatically without being individually suspected of wrongdoing.
Outdated, erroneous, overbroad, or poorly sourced entries can generate inappropriate alerts.
Real-time alerts can create pressure to act before meaningful verification occurs.
14. Facial Recognition in an RTCC
Facial recognition becomes particularly consequential when integrated into a Real-Time Crime Center.
An analyst could potentially receive imagery from a camera, perform facial recognition, obtain a candidate, search agency records, examine associated vehicles, check ALPR history, and communicate a possible identity to officers within minutes.
15. Facial Recognition and Body-Worn Cameras
Body-worn cameras create an enormous volume of facial imagery. Technically, that creates opportunities for both retrospective and potentially real-time facial analysis.
But adding facial recognition to BWC systems can transform a recording device into an identification sensor capable of analyzing large numbers of police-public encounters.
16. Fourth Amendment Considerations
There is no single Supreme Court decision establishing one comprehensive Fourth Amendment rule for all law-enforcement facial-recognition uses.
The constitutional analysis can depend on the source of the probe image, how the gallery was created, whether other protected information is accessed, the surveillance context, duration, location, and what officers do after receiving a candidate result.
Raises different questions from imagery obtained through a separate unlawful search.
Authority to maintain or search particular photographs can depend on statutes, policy, and source.
The facial-recognition lead does not eliminate the applicable requirement for reasonable suspicion or probable cause.
17. First Amendment and Protected Activity
Facial recognition can implicate expressive and associational interests when used at demonstrations, political events, houses of worship, journalistic activity, organizational meetings, or similar contexts.
18. The Candidate Is Not the Evidence That Proves Identity
Once facial recognition develops a candidate, investigators should seek independent evidence.
Height, build, tattoos, scars, clothing, or other features may corroborate or exclude the candidate.
Phone, ALPR, video, witnesses, transactions, or other information may connect the individual to the relevant place.
Known relationships, vehicles, addresses, conduct, admissions, or other evidence may independently support identity.
19. Discovery, Disclosure, and Litigation
Facial recognition can create discoverable or potentially relevant records beyond the final candidate name.
| Record | Why It May Matter |
|---|---|
| Original Probe | Shows the imagery actually available for comparison. |
| Processed Probe | Documents cropping, enhancement, rotation, or other preprocessing. |
| Candidate List | Shows which identities the system returned and their rank. |
| Scores | May document algorithmic similarity information. |
| Gallery | Identifies what population was searched. |
| Algorithm / Version | Establishes which system produced the result. |
| Reviewer Notes | Shows human evaluation of returned candidates. |
| Audit Log | Documents search, user, date, system, and related activity. |
20. How Agencies Should Validate Facial Recognition
Independent laboratory testing is valuable, but agencies also need to determine whether the technology is appropriate for their intended operational use.
21. Procurement Questions
| Issue | Agency Question |
|---|---|
| Algorithm | Which exact algorithm and version will the agency use? |
| Independent Testing | Has the algorithm been evaluated by NIST or another credible independent body? |
| Gallery | Who supplies the reference images and what populations are included? |
| Threshold | Who determines the similarity threshold and can it be changed? |
| Model Changes | Can the vendor change the algorithm without notifying the agency? |
| Search Records | Are probe images, candidate lists, scores, and audit records preserved? |
| Training Use | May vendor systems use agency images to train or improve models? |
| Subprocessors | Which third parties can access biometric data? |
| Data Ownership | Who owns templates, probe images, candidate information, and logs? |
| Audit Rights | Can the agency independently audit search use and system performance? |
22. Agency Governance Framework
Specify when facial recognition may and may not be used.
Consider whether use should be limited by offense, risk, investigative need, or other defined criteria.
Require minimum image-quality assessment before search.
Restrict candidate evaluation to appropriately trained personnel.
Expressly state that a candidate result is not positive identification.
Require additional evidence before consequential enforcement action.
Establish entry criteria, expiration, review, correction, and deletion procedures.
Apply heightened safeguards to protests, religion, politics, journalism, and other First Amendment activity.
Document legal authority, source, retention, and permitted uses of searchable photographs.
Record who searched what image, when, why, and what result was produced.
Coordinate preservation and disclosure requirements with prosecutors.
Reassess performance after material changes to algorithms, galleries, thresholds, vendors, or workflows.
23. Questions Every Agency Should Answer
24. Where Facial Recognition Is Going
Live camera streams may increasingly be compared against authorized watchlists.
Candidate identities may be rapidly correlated with vehicles, records, locations, and other sensors.
Systems may search hours or days of archived video for appearances of a particular face.
Facial features may be combined with clothing, gait, vehicles, and other attributes to reconstruct movement.
Facial identification could potentially be layered onto police-public encounter footage.
AI may increasingly combine faces with voice, gait, vehicles, location, and other attributes.
25. Key Terms
26. Related ShieldPST.ai Resources
Integrated cameras, ALPR, databases, sensors, AI, analysts, and operational intelligence.
Open explainer →AI-assisted video analysis, search, transcription, redaction, report generation, and evidentiary integrity.
Open explainer →Public-source investigation, online identity, automated analysis, and investigative verification.
Open explainer →Preservation, metadata, discovery, authentication, and digital evidence governance.
Open resource →Research Fourth Amendment and emerging police-technology decisions.
Browse case library →Return to the Shield Technology Reference Library.
Browse explainers →27. Selected Primary and Authoritative Sources
NIST's ongoing independent evaluation of one-to-many facial-recognition algorithms, including false-positive and false-negative identification performance.
Review NIST 1:N evaluation
NIST's evaluation program for one-to-one facial verification algorithms.
Review NIST 1:1 evaluation
NIST research and ongoing evaluation concerning measured performance differences across age, sex, race, and other demographic categories.
Review demographic evaluations
Major NIST study evaluating demographic differentials across facial-recognition algorithms and image collections.
Review NIST report
FBI description of the NGI facial-recognition search service, which provides authorized law enforcement with ranked potential candidates as investigative leads.
Review FBI NGI information
DOJ report discussing law-enforcement facial-recognition use and safeguards, including the principle that facial-recognition results alone may not serve as sole proof of identity and that identity must be confirmed through other investigation or analysis.
Review DOJ report
GAO review of federal law-enforcement use of facial-recognition services, policy, training, privacy, civil-rights, and civil-liberties safeguards.
Review GAO report
GAO review addressing federal facial-recognition use and the FBI's Facial Recognition Technology Use Policy Directive and training requirements.
Review GAO report
GAO examination of agency-owned and external facial-recognition systems, tracking, privacy, accuracy, and risk assessment.
Review GAO report
28. Key Takeaways
- Face detection, one-to-one verification, and one-to-many identification are different technologies and should not be treated as interchangeable.
- Law-enforcement facial identification commonly involves comparing a probe photograph of an unknown person against a gallery of known photographs.
- A facial-recognition system normally measures similarity; it does not independently prove identity.
- A returned candidate should be treated as an investigative lead rather than a positive identification.
- Investigators should independently corroborate identity before arrest, charging, search warrants, or other consequential action.
- Probe-image quality can materially affect performance. Resolution, pose, lighting, motion, occlusion, compression, and camera angle all matter.
- The gallery matters as much as the algorithm. Agencies should know whose photographs are searchable and under what authority.
- Similarity scores should not be translated casually into percentages of certainty that a person is the suspect.
- Accuracy should be evaluated separately for false positives, false negatives, one-to-one verification, one-to-many identification, gallery size, rank, and operational conditions.
- Independent testing has documented demographic differentials in facial-recognition performance, and those differences vary substantially among algorithms.
- Human review is essential but can itself introduce confirmation bias or automation bias unless reviewers are trained and follow structured procedures.
- Real-time watchlist recognition creates greater privacy and operational risk than a single post-event investigative search and should receive separate governance treatment.
- RTCC integration can compress the path from camera image to facial-recognition candidate to field action into minutes, increasing the importance of verification and clear communication of uncertainty.
- Agencies should separately decide whether facial recognition may be applied to body-worn camera recordings.
- Facial-recognition use does not eliminate the constitutional requirements governing stops, searches, arrests, home entries, or other enforcement actions.
- Policies should include heightened safeguards when facial recognition intersects with protests, religion, journalism, political activity, or other protected First Amendment conduct.
- Agencies and prosecutors should preserve enough information to reconstruct the facial-recognition process during discovery and litigation.
- The next major governance challenge is the shift from post-event identification of an unknown suspect to persistent, real-time identification across networked cameras.