A secure and explainable multimodal biometric system using trust adaptive fusion for face and fingerprint

Why your face and fingerprint need a smarter lock

Our phones, laptops, and even office doors increasingly rely on fingerprints and face scans instead of passwords. But what happens if those biometric "keys" are copied or corrupted—and how can we trust computer models that decide who gets in? This paper presents a new way to combine face and fingerprint data that aims to be more accurate, more private, and more understandable to humans, offering a glimpse of how tomorrow’s digital locks might work.

Using two traits instead of one

Single biometric systems, such as fingerprint-only or face-only scanners, can struggle when the image is blurry, poorly lit, or partially covered. The authors build a multimodal system that uses both face and fingerprint together. Each image is first cleaned up: faces are detected and aligned so that eyes, nose, and mouth are in a standard position, while fingerprints are denoised, binarized, and cropped to the most informative central area of the ridges. This careful preparation makes later steps more robust to everyday issues like lighting changes, finger pressure, or background clutter.

Teaching a compact digital "sense" of identity

Instead of manually crafting features such as edges or handcrafted texture patterns, the system uses a modern deep-learning model called MobileNetV2, enhanced with a "channel attention" mechanism. In simple terms, this network learns which parts of an image matter most for telling people apart and which can be safely ignored. It produces a short numerical fingerprint—called an embedding—for each face and each fingerprint. These compact summaries are designed to be distinctive enough to separate individuals while still being efficient to store and process.

Letting the system decide which signal to trust

Real-world data are messy: a smudged fingerprint or a dimly lit selfie can mislead even a powerful model. To handle this, the authors introduce Trust-Adaptive Fusion (TAF). The system estimates how confident it is in each modality and converts these confidence levels into trust scores. Rather than simply stacking the face and fingerprint features, it gives more weight to the more trustworthy one and less to the poorer-quality one when forming a combined representation. This dynamic weighting helps the system remain accurate even when one source of information is degraded or partially missing.

Locking the data while still doing the math

Because biometric traits cannot be changed like passwords, protecting them is crucial. The system therefore never stores or compares raw features in the clear. Instead, it encrypts the fused feature vector using a specialized technique known as homomorphic encryption. This method allows the server to perform the comparison—essentially measuring similarity between a stored template and a new login attempt—while the data remain encrypted. Only the final similarity result is decrypted, meaning the underlying biometric template stays hidden even from the server doing the work.

Opening the black box

Deep-learning models are often criticized as opaque. To address this, the authors integrate a visualization method called Grad-CAM. For a given decision, Grad-CAM highlights the regions of the face or fingerprint image that most influenced the outcome. In face images, the system focuses on areas around the eyes, nose, and mouth, while in fingerprint images it concentrates on ridge endings and branching points, not on noisy backgrounds. These heatmaps help users and system designers verify that the model is relying on sensible cues rather than accidental artifacts.

How well it works and why it matters

The proposed system is tested on standard public datasets for faces and fingerprints and shows extremely low error rates: it very rarely mistakes an impostor for a genuine user or rejects a legitimate user. Crucially, these results hold even when all matching is done in the encrypted domain, indicating that strong privacy protections do not significantly weaken performance. For everyday users, the takeaway is that combining multiple biometric traits, weighting them by their quality, and protecting them with advanced encryption can make digital access both safer and more trustworthy—without requiring us to remember a single complex password.

Citation: Chitrapu, P., Morampudi, M.K. & Kalluri, H.K. A secure and explainable multimodal biometric system using trust adaptive fusion for face and fingerprint. Sci Rep 16, 14244 (2026). https://doi.org/10.1038/s41598-026-43252-x

Keywords: biometric authentication, face and fingerprint, privacy-preserving security, deep learning, explainable AI