TransFIRA overview: a frozen pretrained encoder embeds training images and compares each embedding to the class centers to derive its CCS and CCAS recognizability scores, which supervise a lightweight recognizability prediction head on a fine-tuned copy of the encoder; at right, example BRIAR Protocol 3.1 faces labeled with their CCS and NNCCS values and a histogram of the CCS and NNCCS cosine-similarity distributions.

Abstract

Biometric recognition in surveillance, video, and web imagery must contend with extreme variation in pose, blur, illumination, occlusion, and distance, where visual quality metrics fail to predict whether inputs are recognizable to the deployed encoder. Existing face image quality assessment methods rely on visual heuristics, curated annotations, or costly generative pipelines, leaving their predictions detached from the encoder’s decision geometry.

We introduce TransFIRA, a lightweight framework that grounds recognizability directly in embedding space. TransFIRA derives class-center similarity (CCS) and class-center angular separation (CCAS) from a pretrained encoder’s embeddings and learns to predict them from images without quality annotations. Using predicted CCS for template weighting and \( CCAS > 0 \) as a natural filtering threshold yields state-of-the-art face verification performance on BRIAR, raising TAR at \( 10^{-6} \) FMR from 0.085 with uniform averaging to 0.598. The predicted scores also reveal how image degradations affect encoder-specific recognizability.

TransFIRA+ extends this framework to cross-camera multimodal person identification in untrimmed surveillance video. In this setting, body recognizability scores saturate within a narrow range, making direct weighting effectively uniform. We introduce a post-hoc sigmoid calibration that transforms these saturated scores into informative weights without retraining. After calibration, the BRIAR-trained TransFIRA predictor improves fused face and body identification on two unseen multi-camera datasets, outperforming detector-confidence weighting and uncalibrated CCS-rank weighting. Together, TransFIRA and TransFIRA+ demonstrate that encoder-grounded recognizability generalizes across biometric modalities and recognition settings.

Method

TransFIRA+ builds on TransFIRA, which predicts recognizability, the likelihood that a deployed encoder will correctly identify an image, directly from that encoder’s embedding geometry.

Image Recognizability for Template Aggregation

For each image \(x_i\) with embedding \(z_i\), class-center similarity (CCS) is the cosine similarity to the center \(\mu_{y_i}\) of its own identity, and NNCCS is the similarity to the nearest competing center. Their difference defines class-center angular separation (CCAS):

\( CCS_{x_i} = \cos(z_i, \mu_{y_i}), \quad NNCCS_{x_i} = \max_{\,j \neq y_i} \cos(z_i, \mu_j), \quad CCAS_{x_i} = CCS_{x_i} - NNCCS_{x_i} \).

These labels come from the frozen encoder without quality annotations, and a lightweight head finetuned with the backbone learns to predict them from images. The predicted scores then guide template aggregation through two complementary steps:

  • Filtering: Images with predicted \( CCAS > 0 \) are retained, ensuring that only frames confidently recognized by the encoder contribute to the template.
  • Weighting: Each remaining embedding is weighted by its predicted CCS before averaging, emphasizing compact, reliable samples most representative of the class.

Calibrated Recognizability Weights

On body imagery, raw CCS concentrates near one, so normalized template weights are almost uniform. TransFIRA+ fits a monotone logistic map on the training identities, labeling CCS values one and NNCCS values zero:

\( g(s) = \operatorname{sigmoid}\!\left(\alpha \, \frac{s - \mu_s}{\sigma_s} + \beta\right), \quad \alpha \geq 0. \)

The map preserves the predicted ranking and spreads saturated scores across \([0, 1]\), turning them into informative aggregation weights without retraining the predictor.

Histograms of BRIAR body CCS and NNCCS before and after sigmoid calibration: raw similarities concentrate near one, while calibrated CCS moves toward one and NNCCS toward zero.
Sigmoid calibration of BRIAR body recognizability with a SemReID encoder trained on the same dataset. Left: raw CCS and NNCCS concentrate near one (mean 0.97), making direct weighting effectively uniform.
Right: calibration maps CCS toward one and NNCCS toward zero, spreading the similarities across \([0, 1]\).

Results

TransFIRA filters frames at \( CCAS > 0 \) and weights the rest by predicted CCS, achieving state-of-the-art face verification on BRIAR and IJB-C while also improving body verification.

Multimodal Video Identification

We evaluate cross-camera person identification in untrimmed security video on two nonpublic IARPA VideoLINCS datasets, DS0001 and VL-MEVID. The body recognizability predictor is trained only on BRIAR and applied without adaptation; face templates and fusion parameters are held fixed, so only body frame weighting and filtering vary.

Table of body-only and fused mAP, fused Rank-1, and positive coverage on DS0001 and VL-MEVID for each body weighting under no filter, a matched confidence filter, and the CCAS greater than zero filter; calibrated CCS gives the best body and fused mAP on both datasets.
Multimodal video identification with ECHO-BID. Results report body-only and fused mAP, fused Rank-1 (R1), and positive coverage (Cov.) as percentages. Only body filtering and weighting vary; face templates and fusion parameters remain fixed. The confidence filter matches the development-set retention of \( CCAS > 0 \).

Encoder-Grounded Explainability

Because prediction needs no ground-truth labels, the scores explain how unseen images appear to the deployed encoder. Degrading images and rescoring them measures how each degradation affects encoder-specific recognizability. Low-scoring samples can be flagged for recapture or used as hard training examples, and scores aggregated by camera or capture condition show where the encoder struggles.

An IJB-C face at four blur levels with ground-truth and predicted CCS and CCAS under each image.
Recognizability under Gaussian blur with ArcFace on IJB-C. Light blur leaves recognizability essentially unchanged, while moderate and heavy blur reduce it sharply, and predicted CCS and CCAS track the ground truth at every level.

BibTeX

TransFIRA+ is an extended version of TransFIRA (FG 2026). If you find this work useful, please cite both papers.

@article{Tu2026TransFIRAPlus,
    author  = {Tu, Allen and Narayan, Kartik and Gleason, Joshua and Xu, Jennifer and Meyn, Matthew and Goldstein, Tom and Patel, Vishal M.},
    title   = {TransFIRA+: Transfer Learning for Biometric Recognizability Assessment},
    journal = {arXiv preprint arXiv:2611.23442},
    year    = {2026},
    url     = {https://transfira.github.io/plus}
}
@InProceedings{Tu2026TransFIRA,
    author    = {Tu, Allen and Narayan, Kartik and Gleason, Joshua and Xu, Jennifer and Meyn, Matthew and Goldstein, Tom and Patel, Vishal M.},
    title     = {TransFIRA: Transfer Learning for Face Image Recognizability Assessment},
    booktitle = {Proceedings of the IEEE 20th International Conference on Automatic Face and Gesture Recognition (FG)},
    month     = {May},
    year      = {2026},
    pages     = {1-9},
    doi       = {10.1109/FG67764.2026.11556959},
    url       = {https://transfira.github.io/}
}