Paper deep dive
PLGSA-Transformer: Periocular Landmark-Guided Attention with Occlusion-Adaptive Cosine Thresholding for Cross-Modal Masked and Unmasked Face Recognition
Dana A Abdullah
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/7/2026, 8:41:13 AM
Summary
The paper introduces PLGSA-Transformer, a cross-modal face recognition framework designed to handle facial mask occlusion. It combines Periocular Landmark-Guided Spatial Attention (PLGSA) using MediaPipe landmarks, a Hybrid CNN-Transformer architecture based on EfficientNetB3, and an Occlusion-Adaptive Cosine Threshold (OACT) for dynamic decision boundaries. Evaluated on 858 images from Zenodo MDMFR and Kaggle CelebA-HQ, it achieves 97.22% verification accuracy, outperforming prior methods like MUFM, HOG, and FSM.
Entities (14)
Relation Signals (12)
PLGSA-Transformer ā uses ā PLGSA
confidence 97% Ā· This paper proposes PLGSA-Transformer... First, Periocular Landmark-Guided Spatial Attention (PLGSA) uses MediaPipe landmarks...
PLGSA-Transformer ā uses ā OACT
confidence 96% Ā· Third, the Occlusion-Adaptive Cosine Threshold (OACT) is a jointly trained head that raises the matching threshold...
OACT ā adapts ā Cosine Similarity Threshold
confidence 95% Ā· OACT is a jointly trained head that raises the matching threshold in proportion to predicted occlusion severity.
PLGSA-Transformer ā uses ā EfficientNetB3
confidence 95% Ā· fusing them with EfficientNetB3 features via a learnable residual gate... Hybrid CNN-Transformer Branch reshapes feature maps...
PLGSA-Transformer ā evaluatedon ā Zenodo MDMFR
confidence 94% Ā· evaluated on 858 images from Zenodo MDMFR (60%), Kaggle CelebA-HQ masked collection (25%), and author-collected images (15%).
PLGSA-Transformer ā uses ā Multi-Head Self-Attention Transformer
confidence 94% Ā· processed by a two-layer Multi-Head Self-Attention Transformer encoder, enabling cross-regional dependency modelling.
PLGSA ā directsattentionto ā Periocular Regions
confidence 93% Ā· explicitly directing spatial attention toward the most discriminative visible regions... eye, brow, and forehead regions.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The widespread adoption of facial masks, accelerated by COVID-19 and mandated in security-sensitive settings, has exposed limitations of conventional face recognition systems. Existing approaches relying on fixed cosine thresholds, non-adaptive CNNs, and purely data-driven features fail to generalize when facial regions are occluded, creating a gap between lab performance and real-world deployability. This paper proposes PLGSA-Transformer, a cross-modal face matching framework with three contributions. First, Periocular Landmark-Guided Spatial Attention (PLGSA) uses MediaPipe landmarks to compute Gaussian heatmaps over the eye, brow, and forehead regions, fusing them with EfficientNetB3 features via a learnable residual gate to direct attention toward discriminative visible regions. Second, a Hybrid CNN-Transformer Branch reshapes feature maps into tokens processed by a two-layer Multi-Head Self-Attention encoder, enabling cross-regional dependency modelling. Third, the Occlusion-Adaptive Cosine Threshold (OACT) is a jointly trained head that raises the matching threshold in proportion to predicted occlusion severity. The model is evaluated on 858 images from Zenodo MDMFR (60%), Kaggle CelebA-HQ masked collection (25%), and author-collected images (15%), spanning both genders, ages 21-75, with varied mask types, trained via a unified loss combining contrastive verification, identity classification, and occlusion cross-entropy. PLGSA-Transformer achieves 97.22% pair verification accuracy with ROC AUC 1.0000, surpassing VGG-16-based MUFM (Abdullah et al., 2025; 95.0%), HOG classifiers (Adnan et al., 2020; 85.0%), and Feature-based Structural Measure (Shnain et al., 2017; 86.61%). These results confirm that encoding periocular geometry into attention, with Transformer modelling and occlusion-adaptive thresholds, yields a robust, scalable solution for cross-modal masked face recognition.
Tags
Links
- Source: https://arxiv.org/abs/2607.03581v1
- Canonical: https://arxiv.org/abs/2607.03581v1
Trouble viewing inline? Open PDF directly ā
Full Text
66,705 characters extracted from source content.
Expand or collapse full text
PLGSA-Transformer: Periocular Landmark-Guided Attention with Occlusion-Adaptive Cosine Thresholding for Cross-Modal Masked and Unmasked Face Recognition Dana A. Abdullah 1,2 1 Faculty of Engineering and Computer Science, Qaiwan International University, Sulaymaniyah, KRG, Iraq. 2 Department of Information and Communication Technology Center (ICTC)-System Information, Ministry of Higher Education and Scientific Research. ABSTRACT The widespread adoption of facial masks in public spaces, accelerated by the COVID- 19 pandemic and increasingly mandated across security-sensitive environments, has critically exposed the limitations of conventional face recognition systems. Existing approaches that rely on fixed cosine similarity thresholds, spatially un adaptive convolutional architectures, and purely data-driven feature extraction fail to generalize effectively when significant facial regions are occluded, creating a fundamental gap between laboratory performance and real- world deployability in surveillance, access control, and law enforcement applications. This paper proposes PLGSA-Transformer, a novel cross-modal face matching framework that addresses these limitations through three distinct and complementary contributions. First, I introduce Periocular Landmark-Guided Spatial Attention (PLGSA), which leverages MediaPipe facial landmark detection to pre-compute Gaussian heatmaps over the eye, brow, and forehead regions and fuses these anatomical priors with EfficientNetB3 convolutional feature maps through a learnable residual gate, explicitly directing spatial attention toward the most discriminative visible regions. Second, I propose a Hybrid CNN- Transformer Branch in which EfficientNetB3 spatial feature maps are reshaped into token sequences and processed by a two-layer Multi-Head Self-Attention Transformer encoder, enabling global cross-regional dependency modelling between periocular landmarks. Third, I present the Occlusion-Adaptive Cosine Threshold (OACT), a jointly trained occlusion- estimation head that dynamically scales the matching threshold per probe according to predicted occlusion severity: thresholdįµ¢ = base threshold Ć (1 + Ī² Ć occlusion scoreįµ¢). The model is evaluated on a comprehensive dataset of 858 images assembled from three independent sources: the Zenodo MDMFR dataset (60%), the Kaggle CelebA-HQ masked collection (25%), and real-world images directly collected by the author (15%), covering both genders, subjects aged 21 to 75 years, and varied mask types. All three contributions are trained end-to-end through a unified multi-task loss combining contrastive verification loss, identity classification loss, and binary occlusion cross-entropy. Experimental evaluation demonstrates that PLGSA-Transformer achieves a pair verification accuracy of 97.22% with a ROC Area Under the Curve of 1.0000, surpassing the original VGG-16-based MUFM model (Abdullah et al., 2025) (95.0%), HOG-based classifiers (Adnan et al., 2020) (85.0%), and the Feature-based Structural Measure (Shnain et al., 2017) (86.61%). These results confirm that explicitly encoding periocular landmark geometry into the attention mechanism, combined with global Transformer modelling and occlusion-adaptive decision boundaries, yields a robust and scalable solution for cross-modal masked and unmasked face recognition. Keywords: Masked face recognition; Periocular landmark attention; Hybrid CNN-Transformer; Occlusion- adaptive threshold; Cosine similarity; EfficientNetB3; Multi-head self-attention; Cross-modal face matching; Biometric security 1. Introduction Facial recognition has become one of the most widely deployed biometric technologies in modern society, underpinning applications that span public security and law enforcement (Russ et al., 2018; Jiang, 2020), automated access control, forensic identification, and user authentication in consumer devices. The success of these systems depends fundamentally on two core capacities: reliable extraction of discriminative facial features and accurate cross- condition identity verification (Jiang, 2020; Nsaif et al., 2021). The rapid maturation of Convolutional Neural Networks (CNNs) over the past decade has driven remarkable improvements in recognition accuracy under controlled conditions, establishing deep learning- based face recognition as a commercially viable and scientifically well-understood technology (Steck et al., 2024; Abdel-Hamid, 2021). The global outbreak of the COVID-19 pandemic in 2020 introduced a persistent and structurally novel challenge that exposed a critical vulnerability in deployed recognition systems: the mandatory and widespread use of facial masks (Hariri, 2022; Talahua et al., 2021). Facial masks systematically occlude the nose, mouth, and cheek regions, collectively concealing between 40% and 60% of the facial surface area on which conventional recognition models depend (Hsu et al., 2022; Alzu'bi et al., 2021). This occlusion pattern degrades recognition performance substantially, with studies reporting accuracy reductions of 20% to 40% across standard benchmarks under masked conditions (Hariri, 2022; Eman et al., 2023; Hsu et al., 2022). The problem is further compounded by the diversity of mask types ā surgical masks, N95 respirators, fabric coverings, and scarves ā each presenting different occlusion boundaries, textures, and coverage extents that are difficult to model comprehensively during training (Alzu'bi et al., 2021). Beyond the pandemic, masked face recognition is now a permanent operational requirement in environments such as hospitals, airports, and high- security facilities where facial coverings are mandated irrespective of public health status. The research community has responded to this challenge through several lines of investigation. Early approaches focused on adapting existing deep learning architectures ā VGG-16, ResNet-50, and AlexNet ā to extract features from the periocular region that remains visible above the mask line (Alzu'bi et al., 2021). Hariri (2022) demonstrated that the eye and brow zone retains sufficient discriminative information for recognition when the lower face is fully occluded, achieving competitive performance on pandemic-era masked datasets. Talahua et al. (2021) combined MobileNetV2 for mask detection with FaceNet for recognition, reporting 99.65% mask detection accuracy, though recognition performance on masked faces in open-set conditions remained substantially lower. Hsu et al. (2022) addressed cross- condition generalization by synthesizing masked versions of standard benchmarks including IJB-B, IJB-C, and MS-1MV2, employing ResNet-100 with Centre Loss and Angular SoftMax, demonstrating that synthetic masked training data can close the performance gap relative to human recognition. Parallel to CNN-based feature extraction, metric learning strategies have improved embedding separability under occlusion. Deng et al. (2019) proposed ArcFace, employing an Additive Angular Margin loss to maximize inter-class separability. Wang et al. (2018) introduced CosFace, reformulating the softmax loss as a cosine-based margin function. Lin et al. (2021) presented xCos, an explainable cosine metric for face verification. Deng et al. (2021) extended cosine-based learning to masked scenarios through MFCosface. Salim and SĆ¼rantha (2023) showed that embedding-based K-N and SVM classifiers can improve masked recognition without retraining, and Eman et al. (2023) achieved 97% recognition through a hybrid pipeline combining RPCA and Particle Swarm Optimization-enhanced K-N. Despite these advances, three fundamental limitations persist. First, no prior method incorporates explicit anatomical guidance into the spatial attention computation; existing mechanisms such as CBAM (Woo et al., 2018) rely exclusively on data-driven inference. Second, all prior work employs purely convolutional architectures whose locally constrained receptive fields prevent long-range spatial dependency modelling across the periocular zone. Third, every existing method applies a single fixed cosine similarity threshold uniformly to all probe images regardless of individual occlusion severity, which is theoretically unsound and empirically suboptimal. To address all three limitations, this paper proposes PLGSA-Transformer, combining Periocular Landmark-Guided Spatial Attention (PLGSA) using MediaPipe landmark heatmaps (Lugaresi et al., 2019), a Hybrid CNN-Transformer Branch built on EfficientNetB3 (Tan and Le, 2019), and the Occlusion-Adaptive Cosine Threshold (OACT). The main contributions are summarized as follows: (1) We propose PLGSA, the first method to inject pre-computed periocular Gaussian landmark heatmaps directly into the CNN spatial attention computation as an explicit anatomical prior, directing network attention to the anatomically guaranteed unconcluded zone above any facial mask, with a learnable residual gate that prevents instability during early training. (2) We introduce a Hybrid CNN-Transformer Branch in which EfficientNetB3 spatial feature maps are reshaped into token sequences and processed by a two-layer Multi- Head Self-Attention Transformer encoder, enabling global cross-regional dependency modelling that locally constrained convolutional architectures cannot achieve. (3) We present OACT, a jointly trained occlusion-estimation head that dynamically adapts the cosine matching threshold per probe image according to predicted occlusion severity, replacing the static threshold applied uniformly in all prior work. (4) We demonstrate that PLGSA-Transformer achieves 97.22% pair verification accuracy with ROC AUC 1.0000, surpassing MUFM VGG-16 (Abdullah et al., 2025) (95.0%), HOG-based classifiers (Adnan et al., 2020) (85.0%), and the Feature-based Structural Measure (Shnain et al., 2017) (86.61%). The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 presents the methodology. Section 4 describes the experimental setup. Section 5 reports and discusses results. Section 6 discusses limitations and future work. Section 7 concludes the paper. 2. Related Work 2.1 Masked Face Recognition The recognition of masked faces has emerged as a critical research direction following the global adoption of facial coverings. Alzu'bi et al. (2021) provided a comprehensive review of deep learning methods applied to this problem, identifying the periocular region as the primary source of discriminative information when the lower face is concealed. Hariri (2022) demonstrated that eye and brow features retain sufficient discriminative information for recognition even under complete lower-face occlusion, achieving competitive performance on pandemic-era evaluation datasets through targeted periocular feature extraction. More recently, Suravarapu and Patil (2025) reinforced this observation, applying attention-based deep- learning classifiers to hexagonal periocular regions of interest for person identification and further confirming the discriminative value of the periocular region above the mask line. Traditional feature extraction approaches established important baselines. Adnan et al. (2020) compared HOG, LBP, PCA, SURF, and Harris features combined with K-N classification, with HOG achieving the highest accuracy of 85.0%. Shnain et al. (2017) investigated structural similarity under varying conditions using FSM, SSIM, and FSIM, with FSM achieving 86.61%, significantly outperforming SSIM (57.73%) and FSIM (28.03%). Salim and SĆ¼rantha (2023) demonstrated that standard deep face embeddings evaluated with K-N and SVM classifiers can improve masked recognition by 9.09% without model retraining. Eman et al. (2023) achieved 97% recognition through a pipeline integrating RPCA occlusion handling and Particle Swarm Optimization-enhanced K-N feature selection. Deep learning-centric systems produced strong results on controlled evaluation sets. Talahua et al. (2021) combined MobileNetV2 for mask detection with FaceNet for recognition, reporting 99.65% mask detection accuracy, though open-set performance under genuine masked conditions was substantially lower. Hsu et al. (2022) synthesized masked training data from benchmark datasets, employing ResNet-100 with Centre Loss and Angular SoftMax, and demonstrated that synthetic masked training improves generalization across pose, illumination, and age variations. Abdullah et al. (2025) proposed the direct predecessor of the present work, the Masked-Unmasked Face Matching Model (MUFM), combining VGG-16 transfer learning, K-N classification, and cosine similarity for cross-modal pair matching, achieving 95% recognition accuracy. The present paper extends MUFM by replacing VGG-16 with an anatomy-guided hybrid CNN-Transformer architecture and introducing per-probe adaptive thresholding. Table 1 summarizes and compares the key methods reviewed in this subsection. Study Feature Extraction Method / Classifier Accuracy Key Limitation Shnain et al. (2017) Structural similarity (FSM, SSIM, FSIM) Threshold-based matching 86.61% (FSM) No identity embedding; occlusion not modelled Adnan et al. (2020) HOG, LBP, PCA, SURF, Harris K-Nearest Neighbour 85.0% No deep features; no mask- specific design Hariri (2022) VGG / ResNet periocular zone CNN + cosine similarity Competitive No anatomical attention; fixed threshold Study Feature Extraction Method / Classifier Accuracy Key Limitation Talahua et al. (2021) MobileNetV2 + FaceNet Cosine similarity 99.65%* *Mask detection only; lower under open-set masked conditions Salim and SĆ¼rantha (2023) Generic deep embeddings K-N / SVM +9.09% gain No mask-specific training or architecture Wang et al. (2018) ResNet CosFace (cosine margin loss) State-of-art Fixed margin; no occlusion adaptation Deng et al. (2019) ResNet ArcFace (angular margin) State-of-art Fixed threshold; no periocular focus Deng et al. (2021) ResNet + attention MFCosface (cosine margin) Improved No explicit anatomical guidance Eman et al. (2023) MobileNet + RPCA K-N with PSO selection 97.0% Complex pipeline; no global spatial attention Abdullah et al. (2025) VGG-16 (transfer learning) K-N + cosine similarity 95.0% No anatomical attention; fixed threshold Proposed (PLGSA- Transformer) EfficientNetB3 + Transformer PLGSA + OACT (adaptive cosine) 97.22% ā Table 1. Comparison of related methods in masked face recognition. Denotes mask detection accuracy; open-set masked recognition performance was substantially lower. 2.2 Spatial Attention Mechanisms in Face Recognition Attention mechanisms have become a standard component of deep learning architectures, enabling models to selectively weight feature map regions according to task- specific relevance. Woo et al. (2018) introduced the Convolutional Block Attention Module (CBAM), which sequentially applies channel attention ā weighting feature channels by their global importance and spatial attention weighting spatial positions through inter-channel relationships. CBAM has been adopted in face recognition pipelines and applied to masked face recognition to emphasize the periocular zone over occluded regions, demonstrating consistent accuracy improvements over non-attention baselines. However, a fundamental limitation of CBAM and related self-supervised spatial attention mechanisms is their exclusive reliance on learned, data-driven attention weights. Under high occlusion variability, where mask types and coverage extents differ substantially across identities and acquisition conditions, purely data-driven mechanisms must infer the periocular zone from statistical regularities in training data alone. This inference is unreliable when training data are limited or when test-time occlusion configurations differ from those encountered during training. No prior attention-based method incorporates explicit anatomical guidance into the spatial attention computation. The present work addresses this gap through PLGSA, which replaces purely data-driven spatial attention with a Gaussian heatmap derived from MediaPipe facial landmark localization (Lugaresi et al., 2019), ensuring consistent periocular focus regardless of mask type or occlusion extent. 2.3 Transformer Architectures for Visual and Face Recognition The Transformer architecture, introduced by Vaswani et al. (2017) for sequence-to- sequence natural language processing through the Multi-Head Self-Attention (MHSA) mechanism, has been adapted to visual recognition with significant success. Dosovitskiy et al. (2021) proposed the Vision Transformer (ViT), treating non-overlapping image patches as token sequences and demonstrating that pure Transformer encoders achieve recognition accuracy competitive with CNNs when pre-trained on sufficiently large datasets. The global self-attention mechanism enables each spatial position to attend to all others simultaneously, modelling long-range dependencies that CNN receptive fields are structurally unable to capture within a single layer. Transformers have been applied specifically to face recognition with notable results. Zhong and Deng (2021) demonstrated that a Face Transformer, which processes facial patch tokens through a pure Transformer encoder, can outperform standard ResNet-based face recognition models on multiple benchmarks by leveraging the global structural relationships between facial regions that self-attention encodes. However, applying pure Transformer architectures to cross-modal masked face matching faces a compound challenge: they require large-scale pre-training datasets, and their token sequences include both occluded and unconcluded regions with no mechanism for directing attention specifically toward the periocular zone. The present work addresses both constraints through a Hybrid CNN- Transformer Branch, where EfficientNetB3 (Tan and Le, 2019) provides locally grounded, data-efficient periocular feature extraction, and a two-layer MHSA Transformer encoder models global cross-regional dependencies over the resulting spatial tokens. 2.4 Metric Learning and Cosine Similarity Cosine similarity is the standard metric for measuring angular distance between face recognition embeddings (Steck et al., 2024), and forms the basis of the most successful modern face verification systems. Deng et al. (2019) proposed ArcFace, incorporating an Additive Angular Margin loss to maximize angular separability between identity classes and achieving state-of-the-art performance on large-scale benchmarks. Wang et al. (2018) introduced CosFace, which reformulates the standard softmax loss as a cosine-based margin function, establishing competitive results across multiple protocols. Both ArcFace and CosFace apply fixed angular margins uniformly across all training samples without adaptation to occlusion severity. Lin et al. (2021) advanced interpretability through xCos, an explainable cosine similarity metric that spatially localizes the facial regions most responsible for the verification decision. Deng et al. (2021) extended cosine-based metric learning to masked scenarios through MFCosface, incorporating synthetic mask generation and spatial attention to emphasize unconcluded periocular features during training. Despite these advances, a consistent limitation across all reviewed cosine-based methods is the application of a single global decision threshold uniformly to all probe images at inference time, determined once from a validation set without regard to the individual occlusion severity of each probe. The present work addresses this directly through OACT, which jointly trains an occlusion- estimation head with the embedding network and uses its per-probe score to scale the matching threshold proportionally. Beyond these foundational approaches, the field has continued to evolve rapidly. Recent surveys consolidate this progress: Alashbi (2025) reviews deep-learning architectures and feature-extraction strategies for masked face recognition, Sharma et al. (2025) organize the literature into occlusion-robust, occlusion-aware, and occlusion-recovery paradigms, and Sovet (2025) benchmarks modern recognition backbones under occlusion and other challenging conditions. Zhang et al. (2025) likewise survey the broader mask-detection-and- recognition pipeline. Among recent methods, Huang et al. (2024) employ domain adaptation to align masked and unmasked feature distributions, while Alzubi et al. (2025) combine a generative adversarial network with a dual-scale adaptive attention mechanism to reconstruct and match occluded faces, and Pillay et al. (2025) develop a hybrid CNN that pairs MobileNetV2 mask detection with a FaceNet-based recognition network for real-time masked- face recognition. These developments confirm both the growing importance of the task and the persistence of the three gaps identified below. 2.5 Summary of Gaps The literature reviewed across Sections 2.1 to 2.4 reveals three persistent and interrelated gaps that existing work has not collectively addressed. First, while periocular attention is universally recognized as essential for masked face recognition, no method has incorporated explicit anatomical landmark geometry directly into the spatial attention computation, leaving all attention mechanisms dependent on data-driven inference that may be unreliable under diverse occlusion conditions. Second, while Transformer architectures have demonstrated strong global spatial reasoning for face recognition, no prior work has applied hybrid CNN-Transformer modelling to the specific cross-modal task of matching masked and unmasked images within a periocular attention framework. Third, while cosine similarity and margin-based metric learning have substantially improved embedding separability, every existing method applies a fixed decision threshold uniformly to all probes regardless of their individual occlusion severity, which is both theoretically unsound and empirically suboptimal. PLGSA-Transformer directly and simultaneously addresses all three gaps through the unified framework described in Section 3. 3. Methodology This section presents the complete methodology of the proposed PLGSA-Transformer framework. The overall pipeline encompasses the following sequential stages: dataset collection, data pre-processing, periocular ROI extraction and PLGSA heatmap generation, the proposed hybrid model architecture incorporating the three novelties, and the multi-task training configuration. Figure 1 provides an overview of the proposed pipeline: a maskedā unmasked image pair of the same identity is mapped by a shared (Siamese) encoderāan EfficientNet-B3 backbone, the PLGSA module, a CBAM block, token projection with positional encoding, and two Transformer encoder blocksāto a 128-dimensional embedding, after which cosine similarity and the occlusion-adaptive cosine threshold (OACT) produce the matched or not-matched decision. Figure 1. Overview of the proposed PLGSA-Transformer pipeline. A maskedāunmasked image pair of the same identity is processed by a shared (Siamese) encoder: periocular ROI extraction and a MediaPipe landmark heatmap (Ļ = 10) feed an EfficientNet-B3 backbone, followed by the Periocular Landmark-Guided Spatial Attention module (PLGSA, N1), a CBAM block, a 1Ć1 token projection to 25 tokens (d = 256) with positional encoding, and two Pre-LN Transformer encoder blocks (N2, four heads); global average pooling yields a 128- dimensional L2-normalised embedding. Verification compares the gallery and probe embeddings using cosine similarity with the Occlusion-Adaptive Cosine Threshold (OACT, N3, Ī² = 0.08) to produce a matched or not- matched decision. Dashed orange borders mark the three novel contributions (N1āN3). 3.1 Dataset Collection One of the primary challenges encountered in this study was the limited availability of image datasets containing paired images of the same individual both with and without a facial mask. Datasets featuring identical faces in both masked and unmasked conditions are exceedingly rare in the public domain, as the simultaneous capture of the same subject under both conditions requires controlled data collection protocols. To address this constraint, an extensive search was conducted across multiple data repositories, and a comprehensive dataset of 858 paired images was assembled from three independent and complementary sources. The first source, the Zenodo MDMFR dataset (Ullah and Javed, 2022), contributed 60% of the total images and provided a wide range of masked and unmasked face pairs across diverse ethnicities, lighting conditions, and mask types. The second source, the Kaggle CelebA-HQ masked celebrity dataset (Dubey and Pawar, 2024), contributed 25% of the images and provided high-resolution paired images with varied pose and illumination conditions. The third source consisted of real-world images directly captured by the research author, contributing 15% of the total dataset and ensuring the inclusion of authentic in-the-wild acquisition conditions not present in laboratory-collected datasets. Figure 2 illustrates the proportional contribution of each source, and Figure 3 presents the dataset collection and integration process. Figure 2. Dataset collection sources. The dataset was assembled from three independent sources to ensure diversity across acquisition conditions, mask types, and subject demographics. Figure 3. Dataset collection and integration process. Images from all three sources were merged into a single integrated dataset and subsequently divided into training and testing splits at the identity level. The assembled dataset exhibits balanced demographic representation. As illustrated in Figure 4, 58% of subjects are male and 42% are female, with subject ages ranging from 21 to 75 years. Notably, 65% of subjects fall within the 21 to 50 year age range, reflecting the demographic distribution of the source repositories. The dataset encompasses a diverse range of mask types including surgical masks, N95 respirators, cloth face coverings, and transparent face shields, ensuring that the model is evaluated under realistic and varied occlusion conditions. Figure 4. Gender distribution of dataset images. The dataset contains 58% male and 42% female subjects, covering ages from 21 to 75 years across three independent data sources. 3.2 Data Pre-processing Following data collection, an eight-stage pre-processing pipeline was applied to all images to ensure consistency, quality, and suitability for model training. Figure 5 illustrates the complete pre-processing pipeline with all stages in sequence. Figure 5. Data pre-processing pipeline. Eight sequential stages transform raw collected images into a standardized, augmented, and normalized format suitable for model training. The eight pre-processing stages are as follows: ā¢ Folder Organization: Images were segregated into two separate directories ā one containing unmasked face images and one containing masked face images ā to ensure correct class labelling for supervised learning. ā¢ Image Resizing: All images were resized to a standardized resolution of 160 Ć 160 pixels to ensure consistent spatial dimensions and to comply with the EfficientNetB3 input requirements. ā¢ Image Format Conversion: Images in heterogeneous formats including JPEG, BMP, GIF, and WebP were converted to PNG format to ensure consistency in image quality and eliminate format-specific compression artefacts during training. ā¢ Labelling: Each image was annotated with its corresponding identity label and mask status (masked or unmasked) to support supervised contrastive and classification training. ā¢ Data Augmentation: To increase dataset diversity and reduce overfitting, augmentation operations including random horizontal flipping, brightness and contrast adjustment, saturation and hue variation, and scale jitter were applied. Critically, all spatial augmentations were applied identically to both the image and its corresponding PLGSA heatmap to maintain spatial alignment. ā¢ Normalization: Pixel values were normalized from the range [0, 255] to [0, 1] to accelerate model convergence and ensure numerical stability during gradient descent. ā¢ Noise Reduction: Gaussian filtering was applied selectively to reduce sensor noise and compression artefacts, improving the quality of edge and texture features extracted by the convolutional backbone. ā¢ Colour Space Conversion: Images were converted from RGB to the required input format for EfficientNetB3, with grayscale conversion applied where color information was not deemed task-critical. 3.3 Periocular ROI Extraction and PLGSA Heatmap Generation (Novelty 1) Prior to feature extraction, each input image undergoes periocular Region of Interest (ROI) extraction using MediaPipe facial landmark detection (Lugaresi et al., 2019). MediaPipe localizes 478 facial landmarks in normalized image coordinates. From these, 28 landmarks corresponding exclusively to the eye contours and brow arches are selected, as these anatomical structures are guaranteed to remain visible above any standard facial mask. The periocular bounding box is computed by taking the minimum and maximum coordinates of the selected landmark set, expanded by 25% horizontally and 50% vertically to include surrounding discriminative context. When landmark detection fails due to extreme occlusion or poor image quality, a fallback region corresponding to the top one-third of the image is used. The PLGSA heatmap encodes the spatial prior of unconcluded regions as a Gaussian density map. For each of the 28 periocular landmark coordinates (cx, cy) projected onto the target image resolution of H Ć W pixels, a two-dimensional Gaussian function is computed as given in Equation 1: ķŗ ( ķ„,ķ¦ ) =exp [ ā ( ( ķ„āķ ķ„ ) 2 +(ķ¦āķ ķ¦ ) 2 ) 2ķ 2 ] (1) where Ļ = 10 pixels is the Gaussian standard deviation. The final heatmap H is formed by taking the per-pixel maximum across all 28 individual Gaussian responses, producing a smooth spatial prior H ā [0, 1] of shape (H, W, 1). The heatmap is stored alongside its corresponding image and passed as a second branch input throughout all preprocessing, augmentation, and model inference stages. 3.4 Novelty 1: Periocular Landmark-Guided Spatial Attention (PLGSA) The PLGSA layer receives the convolutional feature maps F produced by the EfficientNetB3 backbone and the corresponding heatmap H. The heatmap is first bilinearly resized to match the spatial dimensions of the feature maps, producing H Ģ . A combined attention map A is then computed by summing a 1Ć1 convolutional response on H Ģ and a 1Ć1 convolutional response on F, as given in Equation 2: ķ“=ķ(Conv 1 ( ķ» Ģ ) +Conv 2 ( ķ¹ ) ) (2) where Ļ denotes the sigmoid activation function. The attended feature maps Fā² are then produced through a learnable residual gate parameterized by a scalar Ī³, as given in Equation 3: ķ¹ ā² =ķ¹Ć ( 1+tanh ( ķ¾ ) Ćķ“ ) (3) The scalar Ī³ is initialized to zero at the start of training, ensuring that the heatmap exerts no influence in the initial training steps and that the network begins from the standard convolutional baseline. As training progresses, Ī³ grows in proportion to the gradient signal, allowing the heatmap prior to strengthen only where it reduces the training loss. This residual formulation ensures training stability while providing a principled mechanism for incorporating anatomical spatial guidance. 3.5 Novelty 2: Hybrid CNN-Transformer Branch The hybrid embedding branch processes each input image-heatmap pair through a sequential pipeline of five components: the EfficientNetB3 backbone, the PLGSA layer, the CBAM attention module, the Transformer encoder, and the MLP projection head. The full model architecture is illustrated in Figure 6. Figure 6. PLGSA-Transformer proposed model architecture. The Siamese structure processes masked and unmasked image pairs through a shared branch incorporating all three novelties. The L2-normalised embedding together with the classification and occlusion heads enables joint training through a unified multi-task loss function. The EfficientNetB3 backbone (Tan and Le, 2019), pre-trained on ImageNet, accepts a 160 Ć 160 Ć 3 RGB input and produces spatial feature maps of shape (5 Ć 5 Ć 1536) through a sequence of mobile inverted bottleneck convolutions with squeeze-and-excitation blocks. The lower 50% of backbone layers are frozen during training to preserve low-level feature representations, while the upper 50% are fine-tuned on the masked face dataset. Following the backbone, the PLGSA layer applies the anatomical attention gate described in Section 3.4. The CBAM module (Woo et al., 2018) then applies sequential channel and spatial self-attention, producing refined feature maps that suppress uninformative background activations. For the Transformer component, the attended feature maps are projected to dāāįµįµĖ” = 256 channels through a 1Ć1 convolution and reshaped into a sequence of 25 tokens of dimensionality 256, one token per spatial location of the 5 Ć 5 feature map. Learned positional encodings are added to each token to encode spatial position. The token sequence is then processed by two Pre-Layer Normalization Transformer encoder blocks, each consisting of Multi-Head Self-Attention with 4 attention heads and a feed-forward network with Gaussian Error Linear Unit (GELU) activation and inner dimension 512, as described in Equations 4 and 5: MHSA ( ķ,ķ¾,ķ ) =softmax( ķķ¾ T ā ķ ķ )ķ (4) FFN ( ķ„ ) =GELU ( ķ„ķ 1 +ķ 1 ) ķ 2 +ķ 2 (5) A Global Average Pooling operation across the 25 token positions produces a single 256-dimensional vector. This vector is passed through a two-layer MLP projection head with dropout regularization (rates 0.40 and 0.30) to produce a 128-dimensional embedding, which is L2-normalised to unit length to enable cosine similarity computation. 3.6 Novelty 3: Occlusion-Adaptive Cosine Threshold (OACT) The OACT mechanism addresses the fundamental limitation of fixed decision thresholds in face verification. A lightweight two-layer MLP occlusion estimation head is appended to the anchor embedding and trained jointly through binary cross-entropy against the ground-truth mask status label (1 = masked, 0 = unmasked), as given in Equation 6: ķ ķ =ķ ( ķ 2 Ā·ReLU ( ķ 1 Ā·ķ ķ +ķ 1 ) +ķ 2 ) (6) where eįµ¢ is the 128-dimensional embedding of probe image i, and sįµ¢ ā [0, 1] is the predicted occlusion severity score. At inference time, the global base threshold Ī is determined once from the Youden index applied to the validation ROC curve. For each probe image, a per- probe adaptive threshold Īįµ¢ is then computed as given in Equation 7: ķ ķ =ķ ( 1+ķ½Ā·ķ ķ ) (7) where Ī² = 0.08 is a conservative scaling factor. Probes with high occlusion scores receive a proportionally relaxed threshold, acknowledging that their reduced periocular visibility justifies a more permissive matching criterion. Probes with low occlusion scores retain a threshold close to the global optimum. The occlusion head shares the backbone weights with the embedding network and incurs no additional inference cost beyond a single MLP forward pass. 3.7 Siamese Architecture and Multi-Task Loss The full model adopts a Siamese architecture in which both input branches share identical weights, enforcing that the embedding function produces comparable representations for both masked and unmasked images. Given an input pair (Iā, Iā) with corresponding heatmaps (Hā, Hā), the shared branch produces embeddings (eā, eā), from which the cosine distance is computed as given in Equation 8: ķ ( ķ 1 ,ķ 2 ) =1ā ķ 1 Ā·ķ 2 ā ķ 1 ā ķ 2 ā (8) The model is trained through a unified multi-task loss combining four components as given in Equation 9: ā=ā con +ķ 1 ā cls +ķ 2 ā occ +ķ 3 ā cen (9) The contrastive loss ā con encourages same-identity pairs to have low cosine distance and different-identity pairs to have distance exceeding the margin Ī¼ = 1.0, with additional weighting for hard negatives. The classification loss ā cls applies sparse categorical cross- entropy to prevent embedding collapse by maintaining identity discriminability in the classification head. The occlusion loss ā occ trains the OACT head through binary cross-entropy on the mask status label. The center loss ā cen pulls same-identity embeddings toward their class centroid, compacting intra-class variance and improving retrieval performance. The contrastive loss carries an implicit unit weight, while the remaining loss weights are set to Ī»ā = 0.30 (classification), Ī»ā = 0.20 (occlusion), and Ī»ā = 0.003 (center). 3.8 Training Configuration All experiments were conducted in the Google Collab environment using GPU acceleration (typically an NVIDIA T4 GPU on the Collab platform). The model was trained for a maximum of 150 epochs using the Adam optimizer with an initial learning rate of 5 Ć 10ā»āµ. To promote stable convergence, a learning-rate scheduler halved the learning rate whenever the validation loss plateaued for 15 consecutive epochs. Early stopping with a patience of 30 epochs was additionally employed as a standard regularization measure to prevent overfitting and to retain the best-performing model weights. Under this configuration the model converged and training halted automatically after 81 epochs, well within the 150- epoch budget, with the checkpoint achieving the lowest validation loss retained for all reported evaluations. The batch size was set to 8 due to GPU memory constraints imposed by the high- resolution heatmap inputs. Identity-level dataset partitioning was applied to ensure that no identity appeared in both the training and test sets, with 75% of paired identities allocated to training and 25% to testing. Gallery prototypes were constructed from all available unmasked images per identity using 49 augmented embeddings per image, and probe embeddings were computed as the mean of 9 augmented versions to reduce single-image embedding noise. 4. Experimental Setup 4.1 Dataset and Evaluation Protocols The proposed model was evaluated on the 858-image dataset described in Section 3.1, partitioned at the identity level such that 75% of paired identities were allocated to training and the remaining 25% to testing, with no identity appearing in both splits. This identity-disjoint partitioning ensures that the reported performance reflects genuine generalization to previously unseen individuals rather than memorization of training identities. Two complementary evaluation protocols were adopted. The first, pair verification, presents the model with pairs of images and requires a binary same-or-different-identity decision based on the cosine distance between their embeddings. The second, gallery-probe identification, constructs a gallery of unmasked identity prototypes and queries it with masked probe images, requiring the model to retrieve the correct identity by nearest-neighbor search in the embedding space. Figure 7 presents representative samples from the dataset, showing unmasked and masked image pairs alongside their corresponding PLGSA attention heatmaps. Figure 7. Representative dataset samples. Top row: unmasked periocular images; middle row: masked images of the same identities; bottom row: PLGSA attention heatmaps (Novelty 1) overlaid on the periocular region. The heatmaps consistently localize the eye and brow region across diverse subjects and mask types. The samples in Figure 7 confirm that the periocular ROI extraction correctly isolates the eye and brow region across diverse subjects, and that the PLGSA heatmap consistently localizes the eyes and eyebrows ā the anatomical structures guaranteed to remain visible above any standard facial mask. The heatmaps are correctly computed for both masked and unmasked images, validating the anatomical attention prior described in Section 3.3. 4.2 Implementation Details All experiments were implemented in TensorFlow and Keras and conducted in the Google Collab environment with GPU acceleration. The EfficientNetB3 backbone was initialized with ImageNet pre-trained weights, with the lower 50% of layers frozen to preserve general low-level visual features and the upper 50% fine-tuned on the masked face dataset. Training employed the Adam optimizer with an initial learning rate of 5 Ć 10ā»āµ, a batch size of 8, and a maximum of 150 epochs with early stopping (patience 30) and learning-rate halving (patience 15); under this configuration the model converged after 81 epochs, well within the 150-epoch budget. The complete hyperparameter configuration is summarized in Table 2. Parameter Value Parameter Value Input image size 160 Ć 160 Embedding dimension 128 Backbone EfficientNetB3 Transformer blocks 2 Attention heads 4 FFN inner dimension 512 Token dimension 256 Optimizer Adam Initial learning rate 5 Ć 10ā»āµ Batch size 8 Maximum epochs 150 Early-stop patience 30 Contrastive margin 1.0 OACT Ī² 0.08 Heatmap Ļ 10 px Train / test split 75% / 25% Loss weights (Ī»ā,Ī»ā,Ī»ā) 0.30, 0.20, 0.003 Periocular landmarks 28 Table 2. Complete hyperparameter configuration of the proposed PLGSA-Transformer model. 4.3 Evaluation Metrics Four metrics were used to assess performance. For the pair verification task, the Receiver Operating Characteristic Area Under the Curve (ROC AUC) measures the separability of genuine and impostor pairs across all possible decision thresholds, while the pair accuracy reports the classification accuracy at the optimal threshold determined by the Youden index ā the operating point that maximizes the difference between the true positive rate and the false positive rate. For the gallery-probe identification task, Rank-1 accuracy measures the proportion of probes for which the correct identity is retrieved as the single nearest gallery prototype, and Rank-5 accuracy measures the proportion for which the correct identity appears among the five nearest prototypes. A confusion matrix is additionally reported to characterize the distribution of errors in the pair verification task between false acceptances and false rejections. 5. Results and Discussion 5.1 Pair Verification Performance Figure 8 presents the complete pair verification results, evaluated over 858 same-person pairs and 858 impostor pairs. The model achieves a ROC AUC of 1.0000, indicating perfect separability between genuine and impostor pairs on the test set, with the Youden-optimal decision threshold located at a cosine distance of 0.355. At this threshold, the model attains a same-pair match rate of 97.2%, correctly accepting 834 of the 858 genuine pairs. Figure 8. Pair verification and gallery-probe evaluation over 858 same-person pairs and 858 impostor pairs. From left to right: ROC curve with the Youden-optimal threshold marked; gallery-probe distance distributions for Rank-1 correct and incorrect retrievals; occlusion score versus nearest-neighbor distance; and the pair verification confusion matrix (same-pair match rate 97.2%). The confusion matrix in Figure 8 provides detail on the error distribution. The evaluation comprises 858 different-identity (impostor) pairs and 858 same-identity (genuine) pairs; all 858 impostor pairs and 834 of the 858 genuine pairs were correctly classified, while the remaining 24 genuine pairs were incorrectly rejected as non-matches. This corresponds to a precision of 1.000, a recall of 0.972, and a specificity of 1.000. The complete absence of false positives ā different individuals are never accepted as the same person ā is particularly significant for security-critical applications, where a false acceptance is typically far more costly than a false rejection. The 24 errors are therefore conservative false rejections rather than security-compromising false acceptances. 5.2 Gallery-Probe Identification Performance On the more challenging open-set gallery-probe identification task, in which each masked probe must be matched against a gallery of 50 unmasked identity prototypes, the model achieves a Rank-1 accuracy of 78.0% (39 of 50 probes correctly identified as the nearest neighbor) and a Rank-5 accuracy of 96.0% (48 of 50 probes with the correct identity among the five nearest neighbors). The substantial gap between Rank-1 and Rank-5 accuracy reveals an important property of the learned embedding space: for the majority of misidentified probes, the correct identity is not distant but is merely ranked second or third. This is illustrated by the representative decision examples in Table 3, and holds across the full test set: of the eleven Rank-1 errors, nine place the correct identity at Rank-2 or Rank- 3, and only two probes place the correct identity substantially far from the top (at Rank-13 and Rank-31 respectively). This indicates that the periocular embedding successfully clusters each identity into a tight neighborhood, and that the residual errors arise predominantly from fine- grained confusion between a small number of visually similar periocular regions rather than from a failure to learn discriminative identity features. Probe Retrieved ID Cosine Sim. OACT Thr. Occ. Score True-ID Rank Decision 17 17 0.993 0.369 0.47 Rank-1 MATCHED 32 32 0.992 0.369 0.49 Rank-1 MATCHED 1 1 0.969 0.368 0.43 Rank-1 MATCHED 12 12 0.872 0.370 0.51 Rank-1 MATCHED 40 40 0.817 0.369 0.48 Rank-1 MATCHED 15 15 0.664 0.369 0.49 Rank-1 MATCHED 46 41 0.928 0.369 0.48 Rank-2 NOT MATCHED 18 21 0.894 0.369 0.47 Rank-2 NOT MATCHED 26 27 0.858 0.369 0.48 Rank-5 NOT MATCHED 11 7 0.806 0.369 0.49 Rank-3 NOT MATCHED 10 29 0.798 0.369 0.49 Rank-13 NOT MATCHED 25 24 0.884 0.370 0.51 Rank-31 NOT MATCHED Table 3. Representative per-probe decision examples drawn from the 50-probe test set. The upper six rows are correct Rank-1 matches spanning a wide range of cosine similarities (0.66 to 0.99); the lower six are representative errors, including two confident high-similarity errors retrieved at Rank-2 and two distant errors where the correct identity falls to Rank-13 and Rank-31. 5.3 Training Convergence Analysis Figure 9 presents the per-epoch training and validation pair accuracy over the course of training, which terminated by early stopping after 81 epochs. The validation pair accuracy rises over the opening epochs, peaking near 0.90 at around epoch 18, and thereafter stabilizes around 0.80 with only minor fluctuation and no sustained downward drift, indicating that the embedding space settles into a stable configuration rather than overfitting; the best-performing weights were restored at the end of training. The horizontal dashed line marks the calibrated test-set match rate of 97.2% obtained at inference (Figure 8, Section 5.1), shown here for reference, and the per-epoch validation accuracy lies below it for the reasons set out below. Figure 9. Per-epoch training and validation pair accuracy. The validation pair accuracy rises over the opening epochs, peaks near 0.90, and then stabilizes around 0.80, while the training pair accuracy fluctuates around 0.50. The horizontal dashed line marks the test-set match rate of 97.2% reported in Figure 8 (Section 5.1). This per- epoch accuracy is a running per-batch-threshold proxy and therefore lies below the calibrated test-set value, as explained in Section 5.3. A notable feature of Figure 9 is that the training pair accuracy fluctuates around 0.50 while the validation pair accuracy stabilizes around 0.80. This apparent discrepancy is a direct consequence of how accuracy is computed during training rather than an indication of underfitting: the training-time accuracy uses a dynamic per-batch mean distance as its decision threshold, applied to big batches of heavily augmented hard pairs, whereas the validation accuracy is measured on clean, un-augmented pairs. The same per-batch-threshold measurement also explains the gap between the validation pair accuracy of approximately 0.80 in Figure 9 and the 97.2% same-pair match rate reported in Section 5.1: at inference the verification decision is made with the calibrated Youden-optimal threshold and with embeddings denoised by test-time augmentation, namely gallery prototypes formed from 49 augmented embeddings per identity and probe embeddings averaged over 9 augmented views (Section 3.8). This difference in evaluation protocol, rather than any inconsistency between the two figures, accounts for the substantially higher performance reported at inference, and the stable validation accuracy maintained across training confirms that the model is well optimized. 5.4 Qualitative Analysis Figure 10 presents eight correct Rank-1 matches selected to demonstrate the robustness of PLGSA-Transformer across challenging real-world variations in age, facial expression, head pose, and accessories. Each panel reports, for one identity, the gallery (unmasked) image, the masked probe with its PLGSA attention overlay, the periocular crops compared, and the decision metrics. Every example is a correct Rank-1 match, and in each the PLGSA attention concentrates on the periocular region irrespective of the surrounding variation. Figure 10. Eight correct Rank-1 matches spanning diverse ages, expressions, poses, and accessories. Each panel shows, left to right, the gallery (unmasked) image, the masked probe with its PLGSA attention overlay, the periocular crops compared, and the decision metrics. These examples collectively show that the periocular embedding generalizes across substantial demographic and expressive variation. Correct matches are obtained across a wide age range, from the child (Identity 49) to elderly subjects (Identity 28). The model maintains correct identification under strong facial expressions, including a broad smile (Identity 28), anger (Identity 41), fear (Identity 34), and an open-mouth shout (Identity 42), despite the deformation these induce around the eyes and brows. Robustness to accessories is evident for Identity 49, who wears both eyeglasses and a cap, and for Identity 16, whose surgical cap conceals the hairline. The most demanding case is Identity 11, a near-profile probe pose: although the out-of-plane rotation reduces the cosine similarity to 0.739, the lowest among all matches, the correct identity is still ranked first, indicating tolerance to moderate pose variation. Across these examples the cosine similarities span 0.739 to 0.980, and the predicted occlusion scores remain within a narrow band of approximately 0.46 to 0.51, consistent with the low occlusion-score variance reported in Section 5.6. 5.5 Analysis of Challenging Cases Figure 11 presents the gallery-probe pairs that the system reported as NOT MATCHED, in which the nearest retrieved gallery identity differs from the probeās true identity. These correspond to the Rank-1 identification errors quantified in Section 5.2 and are presented here for transparency and to characterize the conditions under which periocular matching is most challenged. As in Figure 10, each panel shows the masked probe alongside the gallery identity with which it was most strongly associated, together with the PLGSA attention overlay, the periocular crops, and the decision metrics. Figure 11. Challenging cases reported as NOT MATCHED, in which the nearest retrieved gallery identity differs from the probeās true identity. Each panel reports the cosine similarity to the retrieved identity and the rank at which the probeās true identity was found. The cases span a range of expressions, ages, and mask types, including surgical masks, cloth masks, and an N95 respirator. Several observations emerge from these cases. First, the errors are driven by genuine inter-subject similarity in the periocular region rather than by a failure of the attention mechanism: the PLGSA heatmaps remain correctly localized on the eyes and eyebrows in every panel, and several probes attain a high cosine similarity to the incorrect identity ā up to 0.940 ā confirming that, once the lower face is occluded, two different individuals can present a strikingly similar eye-and-brow appearance. Second, despite these confusions the probeās true identity is generally retained at a close rank: in most cases it is the second-, third-, or fifth- nearest gallery entry, so the genuine identity is rarely eliminated from the shortlist even when it is narrowly outranked by a similar-looking individual. This behavior is consistent with the high Rank-5 accuracy reported in Section 5.2 and indicates that a re-ranking stage or a stronger margin-based metric, as discussed in Section 6, could recover many of these cases. It is instructive to contrast the two evaluation protocols. Identification requires the model to single out one identity from the entire gallery, a demanding task on which visually similar individuals may be confused. Verification, by contrast, asks only whether two given images depict the same person, and on this task the model produced no false acceptances whatsoever, corresponding to the zero false-positive rate reported in Section 5.1. Taken together, these results indicate that the learned representation is discriminative ā the model does not erroneously declare two different people to be the same identity ā and that the residual identification errors reflect the intrinsic difficulty of ranking many similar periocular regions rather than a deficiency in the features themselves. The diversity of the cases in Figure 11, which span different ages, expressions including frowns, surprise, and smiles, and several mask types, further confirms that these confusions are not confined to any single demographic group or acquisition condition. 5.6 Occlusion Score Analysis The occlusion-estimation head produces scores within a narrow band between 0.43 and 0.52 across all probes, with a mean of approximately 0.48. Consequently, the Occlusion- Adaptive Cosine Threshold adjusts the base decision threshold of 0.355 to a per-probe range of 0.367 to 0.370, an adjustment of approximately 3 to 4%, consistent with the conservative scaling factor Ī² = 0.08. The scatter plot of occlusion score against nearest-neighbour distance in Figure 8 shows that correctly and incorrectly matched probes are not cleanly separated along the occlusion-score axis. This low variance in the occlusion scores is a genuine limitation of the current model and is discussed further in Section 6. It indicates that the occlusion head, trained jointly through binary cross-entropy on the mask-presence label, has learned to assign a relatively uniform occlusion estimate to all masked probes rather than a finely graded severity score. While the OACT mechanism is theoretically sound and integrates cleanly into the multi-task framework without additional inference cost, its practical effect on the present dataset is modest, and its full benefit would be realized on a dataset exhibiting greater variation in occlusion severity together with explicit occlusion-intensity supervision. 5.7 Comparison with Prior Work Table 4 compares the pair verification performance of PLGSA-Transformer against the prior methods reviewed in Section 2. The proposed model achieves 97.2% pair verification accuracy, exceeding the 95.0% reported by the original VGG-16-based MUFM model (Abdullah et al., 2025), the 86.61% of the Feature-based Structural Measure (Shnain et al., 2017), and the 85.0% of HOG-based classification (Adnan et al., 2020). Beyond the improvement in pair verification accuracy, PLGSA-Transformer additionally reports Rank-1 and Rank-5 identification metrics that quantify open-set retrieval performance ā a more demanding evaluation protocol that prior work in this specific line of research has not addressed. Method Approach Pair Accuracy Adnan et al. (2020) HOG features + K-N 85.0% Shnain et al. (2017) Feature-based Structural Measure (FSM) 86.61% Abdullah et al. (2025) VGG-16 + K-N + cosine similarity (MUFM) 95.0% Proposed PLGSA + Hybrid CNN-Transformer + OACT 97.2% Table 4. Pair verification performance comparison with prior methods. The proposed model additionally achieves Rank-1 identification accuracy of 78.0% and Rank-5 accuracy of 96.0%, metrics not reported by prior work. 6. Limitations and Future Work Despite the strong performance achieved, two limitations of the present study warrant discussion and motivate clear directions for future research. First, the evaluation was conducted on a relatively small dataset of 858 images spanning a limited number of distinct identities. From the identity-disjoint test partition, this yields 858 genuine and 858 impostor verification pairs together with 50 gallery-probe identification queries; although the verification pairs are numerous, they are combinatorially derived from this small pool of identities rather than from independent subjects. The results are internally consistent and the identity-disjoint partitioning guarantees genuine generalization to unseen individuals, yet the perfect ROC AUC of 1.0000 should be interpreted in the context of this limited scale and diversity: it reflects strong separability on the available data rather than a guarantee of flawless performance at larger scale. Validation on substantially larger and more diverse benchmark datasets is therefore a priority for future work. Second, as analyzed in Section 5.6, the occlusion-estimation head produces low- variance scores, confined to a narrow band of approximately 0.43 to 0.52, which limits the practical impact of the OACT mechanism on the current dataset. Future work will address this by introducing explicit occlusion-intensity supervision ā for example, training against graded mask-coverage labels rather than a binary mask-presence label ā to encourage the occlusion head to learn a more finely discriminative severity estimate, thereby allowing OACT to exert a more meaningful per-probe adjustment. 7. Conclusion This paper presented PLGSA-Transformer, a novel cross-modal face matching framework for recognizing individuals across masked and unmasked conditions. The framework introduced three complementary contributions: Periocular Landmark-Guided Spatial Attention (PLGSA), which injects an explicit anatomical prior derived from MediaPipe facial landmarks directly into the spatial attention computation through a learnable residual gate; a Hybrid CNN-Transformer Branch, which combines EfficientNetB3 local feature extraction with global cross-regional dependency modelling through a Multi-Head Self- Attention Transformer encoder; and the Occlusion-Adaptive Cosine Threshold (OACT), which dynamically adjusts the matching threshold per probe according to a jointly trained occlusion- severity estimate. Evaluated on a dataset of 858 images assembled from three independent sources, the proposed model achieved a pair verification accuracy of 97.2% with a ROC AUC of 1.0000, surpassing the original VGG-16-based MUFM model and traditional feature-based baselines. On the more demanding open-set gallery-probe identification task, the model achieved a Rank- 1 accuracy of 78.0% and a Rank-5 accuracy of 96.0%, with analysis confirming that the learned periocular embedding clusters identities into tight neighborhoods such that the majority of Rank-1 errors place the correct identity at Rank-2 or Rank-3. While the low variance of the occlusion-estimation head limits the practical impact of the OACT mechanism on the current dataset, and the small test-set size warrants validation at scale, the results collectively demonstrate that explicitly encoding periocular landmark geometry into the attention mechanism, combined with global Transformer modelling, provides an effective, interpretable, and theoretically grounded solution for cross-modal masked face recognition. Future work will focus on larger-scale validation, graded occlusion supervision. References Abdullah, D.A., Hamad, D.R., Maolood, I.Y., Beitollahi, H., Ameen, A.K., Aula, S.A., Abdulla, A.A., Shakor, M.Y. and Muhamad, S.S. (2025) āA novel facial recognition technique with focusing on masked facesā, Ain Shams Engineering Journal, 16, article 103350. Abdel-Hamid, L. (2021) āRetinal image quality assessment using transfer learning: Spatial images vs. wavelet detail subbandsā, Ain Shams Engineering Journal, 12(3), p. 2799ā2807. Adnan, S., Ali, F. and Abdulmunem, A.A. (2020) āFacial feature extraction for face recognitionā, Journal of Physics: Conference Series, 1664(1), article 012050. Alashbi, A. (2025) āReview of masked face recognition based on deep learningā, Technologies, 13(7), article 310. Alzuābi, A., Abuarqoub, A., Quiam, F. and Subhan, F. (2021) āMasked face recognition using deep learning: A reviewā, Electronics, 10(21), article 2666. Alzubi, J.A., Pokkuluri, K.S., Arunachalam, R., Shukla, S.K., Venugopal, S. and Arunachalam, K. (2025) āA generative adversarial network-based accurate masked face recognition model using dual scale adaptive efficient attention networkā, Scientific Reports, 15(1), article 17594. Deng, H., Su, Y., Zhao, M., Li, J. and Deng, H. (2021) āMFCosface: A masked-face recognition algorithm based on large margin cosine lossā, Applied Sciences, 11(16), article 7310. Deng, J., Guo, J., Xue, N. and Zafeiriou, S. (2019) āArcFace: Additive angular margin loss for deep face recognitionā, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), p. 5962ā5979. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J. and Houlsby, N. (2021) āAn image is worth 16x16 words: Transformers for image recognition at scaleā, in Proceedings of the 9th International Conference on Learning Representations (ICLR 2021), Vienna, Austria. Dubey, S. and Pawar, P. (2024) Unmasking celebrities: Explore the faces behind the masks [Dataset]. Kaggle. Available at: https://w.kaggle.com/datasets/prosad123/unmask-my-celeb-a-hq (Accessed: 2024). Eman, M., Hamouda, A., Shams, M. and Alfonse, M. (2023) āInnovative hybrid approach for masked face recognition using pretrained mask detection and segmentation, robust PCA, and KNN classifierā, Sensors, 23(15), article 6727. Hariri, W. (2022) āEfficient masked face recognition method during the COVID-19 pandemicā, Signal, Image and Video Processing, 16(3), p. 605ā612. Hsu, G.S.J., Liu, C.H., Lin, C.A. and Su, B.H. (2022) āMasked face recognition from synthesis to realityā, IEEE Access, 10, p. 37938ā37952. Huang, Y.C., Rahardjo, D.A.B., Shiue, R.H. and Chen, H.H. (2024) āMasked face recognition using domain adaptationā, Pattern Recognition, 153, article 110574. Jiang, E. (2020) āA review of the comparative studies on traditional and intelligent face recognition methodsā, in Proceedings of the 2020 International Conference on Computer Vision, Image and Deep Learning (CVIDL 2020), Chongqing, China, p. 11ā15. Lin, Y.S., Xu, Y.T., Chen, H.W., Wang, L.Y. and Lin, Y.Y. (2021) āxCos: An explainable cosine metric for face verification taskā, ACM Transactions on Multimedia Computing, Communications, and Applications, 17(3s), article 94. Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., Lee, J., Chang, W.T., Hua, W., Georg, M. and Grundmann, M. (2019) āMediaPipe: A framework for building perception pipelinesā, arXiv preprint arXiv:1906.08172. Nsaif, A.K., Ali, S.H.M., Jassim, K.N., Nseaf, A.K., Sulaiman, R., Al-Qaraghuli, A., Wahdan, O. and Nayan, N.A. (2021) āFRCNN-GNB: Cascade Faster R-CNN with Gabor Filters and NaĆÆve Bayes for enhanced eye detectionā, IEEE Access, 9, p. 15708ā15719. Pillay, C., Joseph, S. and van Niekerk, B. (2025) āDevelopment of a face mask detection and masked facial recognition model based on a hybrid convolutional neural networkā, IET Image Processing, 19(1), article e70239. Russ, A.J., Sauerland, M., Lee, C.E. and Bindemann, M. (2018) āIndividual differences in eyewitness accuracy across multiple lineups of facesā, Cognitive Research: Principles and Implications, 3(1), article 26. Salim, R.J. and SĆ¼rantha, N. (2023) āMasked face recognition by zeroing the masked region without model retrainingā, International Journal of Innovative Computing, Information and Control, 19(4), p. 1087ā1101. Sharma, S., Khan, M.A., Mir, H.M. and Sharma, S. (2025) āA comprehensive survey on masked face recognition techniques using deep learning: motivations, research progress, and future challengesā, ICT Express, in press. Shnain, N.A., Hussain, Z.M. and Lu, S.F. (2017) āA feature-based structural measure: An image similarity measure for face recognitionā, Applied Sciences, 7(8), article 786. Sovet, A. (2025) āRobust face recognition under challenging conditions: a comprehensive review of deep learning methods and challengesā, Applied Sciences, 15(17), article 9390. Steck, H., Ekanadham, C. and Kallus, N. (2024) āIs cosine-similarity of embeddings really about similarity?ā, in Proceedings of the ACM Web Conference 2024 (W ā24), Singapore, p. 887ā895. Suravarapu, V.K. and Patil, H.Y. (2025) āPerformance evaluation of enhanced deep learning classifiers for person identification and gender classificationā, Scientific Reports, 15(1), article 28182. Talahua, J.S., Buele, J., CalvopiƱa, P. and Varela-AldĆ”s, J. (2021) āFacial recognition system for people with and without face mask in times of the COVID-19 pandemicā, Sustainability, 13(12), article 6900. Tan, M. and Le, Q. (2019) āEfficientNet: Rethinking model scaling for convolutional neural networksā, in Proceedings of the 36th International Conference on Machine Learning (ICML 2019), Long Beach, CA, p. 6105ā6114. Ullah, N. and Javed, A. (2022) Face mask detection and masked facial recognition dataset (MDMFR Dataset) [Dataset]. Zenodo. Available at: https://zenodo.org/records/6408603 (Accessed: 2022). Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L. and Polosukhin, I. (2017) āAttention is all you needā, in Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, p. 5998ā6008. Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J. and Liu, W. (2018) āCosFace: Large margin cosine loss for deep face recognitionā, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, p. 5265ā5274. Woo, S., Park, J., Lee, J.Y. and Kweon, I.S. (2018) āCBAM: Convolutional block attention moduleā, in Proceedings of the European Conference on Computer Vision (ECCV 2018), Munich, Germany, p. 3ā19. Zhang, J., An, D., Zhang, Y., Wang, X., Wang, X., Wang, Q., Pan, Z. and Yue, Y. (2025) āA review on face mask recognitionā, Sensors, 25(2), article 387. Zhong, Y. and Deng, W. (2021) āFace transformer for recognitionā, arXiv preprint arXiv:2103.14803.