Paper deep dive
Is Geometry Enough? An Evaluation of Landmark-Based Gaze Estimation
Daniele Agostinelli, Thomas Agostinelli, Andrea Generosi, Maura Mengoni
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/27/2026, 1:15:27 AM
Summary
This paper evaluates landmark-based gaze estimation as a lightweight, interpretable alternative to appearance-based deep learning models. The authors introduce a standardized pipeline to extract and normalize facial landmarks from three large-scale datasets (Gaze360, ETH-XGaze, and GazeGene) and train three regression models: a holistic MLP, a siamese MLP, and an XGBoost regressor. While landmark-based models show lower within-domain performance compared to ResNet18 baselines due to landmark detection noise, they demonstrate comparable generalization capabilities in cross-domain evaluations, suggesting that sparse geometric features are sufficient for robust gaze estimation.
Entities (7)
Relation Signals (3)
ResNet18 → comparedto → Siamese MLP
confidence 95% · in cross-domain evaluation, the proposed MLP architectures show generalization capabilities comparable to those of ResNet18 baselines.
Siamese MLP → evaluatedon → Gaze360
confidence 90% · We evaluate model performances in terms of the Mean Angular Error on the testing set of the same dataset used for training
MediaPipe → extractsfrom → GazeGene
confidence 90% · For this dataset, we implement the full processing pipeline: facial landmark detection
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Appearance-based gaze estimation frequently relies on deep Convolutional Neural Networks (CNNs). These models are accurate, but computationally expensive and act as "black boxes", offering little interpretability. Geometric methods based on facial landmarks are a lightweight alternative, but their performance limits and generalization capabilities remain underexplored in modern benchmarks. In this study, we conduct a comprehensive evaluation of landmark-based gaze estimation. We introduce a standardized pipeline to extract and normalize landmarks from three large-scale datasets (Gaze360, ETH-XGaze, and GazeGene) and train lightweight regression models, specifically Extreme Gradient Boosted trees and two neural architectures: a holistic Multi-Layer Perceptron (MLP) and a siamese MLP designed to capture binocular geometry. We find that landmark-based models exhibit lower performance in within-domain evaluation, likely due to noise introduced into the datasets by the landmark detector. Nevertheless, in cross-domain evaluation, the proposed MLP architectures show generalization capabilities comparable to those of ResNet18 baselines. These findings suggest that sparse geometric features encode sufficient information for robust gaze estimation, paving the way for efficient, interpretable, and privacy-friendly edge applications. The source code and generated landmark-based datasets are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.24724v1
- Canonical: https://arxiv.org/abs/2603.24724v1
Trouble viewing inline? Open PDF directly →
Full Text
42,760 characters extracted from source content.
Expand or collapse full text
Is Geometry Enough? An Evaluation of Landmark-Based Gaze Estimation Daniele Agostinelli 1 , Thomas Agostinelli 1 , Andrea Generosi 2 , Maura Mengoni 1 1 Department of Industrial Engineering and Mathematical Sciences, Universit ` a Politecnica delle Marche, via Brecce Bianche, 12, Ancona, 60131, Italy 2 Department of Science and Information Technology, Universit ` a Pegaso, piazza Trieste e Trento, 48, Napoli, 80132, Italy Abstract—Appearance-based gaze estimation frequently relies on deep Convolutional Neural Networks (CNNs). These models are accurate, but computationally expensive and act as “black boxes”, offering little interpretability. Geometric methods based on facial landmarks are a lightweight alternative, but their performance limits and generalization capabilities remain un- derexplored in modern benchmarks. In this study, we conduct a comprehensive evaluation of landmark-based gaze estimation. We introduce a standardized pipeline to extract and normal- ize landmarks from three large-scale datasets (Gaze360, ETH- XGaze, and GazeGene) and train lightweight regression models, specifically Extreme Gradient Boosted trees and two neural architectures: a holistic Multi-Layer Perceptron (MLP) and a siamese MLP designed to capture binocular geometry. We find that landmark-based models exhibit lower performance in within-domain evaluation, likely due to noise introduced into the datasets by the landmark detector. Nevertheless, in cross-domain evaluation, the proposed MLP architectures show generalization capabilities comparable to those of ResNet18 baselines. These findings suggest that sparse geometric features encode sufficient information for robust gaze estimation, paving the way for effi- cient, interpretable, and privacy-friendly edge applications. The source code and generated landmark-based datasets are available at: https://github.com/daniele-agostinelli/LandmarkGaze.git. Index Terms—Gaze estimation, Facial landmarks, Machine learning, Deep learning, Pattern recognition I. INTRODUCTION Gaze estimation is a critical task for applications in human- computer interaction, automotive safety, and healthcare mon- itoring. Common techniques are categorized mainly into model-based and appearance-based methods [1]. Model-based methods rely on anatomical priors to reconstruct 3D eyeball structures, typically employing dedicated hardware such as infrared cameras and light sources [2]–[4]. Although these approaches achieve high accuracy, hardware requirements and high costs often limit their deployment. In contrast, appearance-based methods estimate gaze directly from images of the eyes or face, making them more suitable for uncon- strained settings [1]. Since the introduction of deep learning for gaze estima- tion [5], the field moved from extractors of eye features (e.g., using asymmetric networks [6], pictorial features [7], or few- shot learning strategies [8]) to models leveraging full-face context (e.g., spatial attention [9], dilated convolutions [10], coarse-to-fine architectures [11], adaptive fusion [12], and transformers [13]). More recent studies explored unsupervised domain adaptation [14]–[17] and domain generalization [18]– [21]. The main challenges in these approaches remain the high computational costs and lack of interpretability, since most of the appearance-based models rely on computationally heavy Convolutional Neural Network (CNN) backbones. These limi- tations might be overcome by adopting geometric models that estimate gaze directly from facial landmarks. While landmarks are typically used for initial data normalization [22] or as auxiliary features that are combined with image data [23]– [25], some studies have explored the strategy of regressing gaze exclusively from landmarks. In contexts such as physical ergonomics or emotion recognition, similar approaches have succeeded to train efficient predictors on geometric features rather than raw image data, supporting the fact that structural information alone is sufficient for regression tasks (e.g., [26]– [28]). Early work in this direction suggested that 2D facial land- marks can be linked to 3D gaze through explicit geometric modeling, using 3D eye-face models [29], [30]. Building on this idea, Park et al. [31] detected detailed eye-region landmarks with a stacked-hourglass network and fed these features into a support vector regressor, explicitly combining learned landmark detection with traditional geometric estima- tion. Similarly, other studies proposed approaches to estimate gaze solely from landmarks in the eye region or relative facial keypoint coordinates extracted with pose estimators [32]–[34]. Despite these contributions, landmark-based gaze estimation remains underexplored and lacks a systematic assessment against modern appearance-based baselines. Most prior studies train on different synthetic data (e.g., UnityEyes [35]) and validate their methods in specific settings. As a result, it remains unclear to what extent landmarks alone can be used for accurate and robust gaze estimation, and what are the capabilities and the current limitations of this approach. In this study, we propose a pipeline to derive normalized landmark-based datasets from annotated face images avail- able in the literature, i.e., Gaze360 [36], ETH-XGaze [37], and GazeGene [38]. After analyzing the coverage of these datasets, we use them to train and evaluate Extreme Gradient Boosted decision trees (XGBoost) and Multi-Layer Percep- trons (MLPs), which we interpret by Permutation Feature Importance (PFI). We find that lightweight MLPs can achieve cross-domain performances comparable to image-based base- lines, suggesting that facial landmarks contain the information necessary for accurate and robust gaze estimation. In conclu- sion, we identify the precision of the landmark detector and the arXiv:2603.24724v1 [cs.CV] 25 Mar 2026 Data normalization Input image Gaze vector g Camera matrix K c Landmark detection with conditional padding (MediaPipe) Face mesh (478 pts) Confidence ≥ 0.8 Feature selection Select N landmarks u i (eyes and head anchors) Already normalized? Head pose estimation (Solve PnP) Rotation R h Translation t h Canonical 3D face model Transformations Norm. rotation R n Warping matrix M Normalized parameters Virtual camera K n Norm. distance d n Normalize landmarks Projection u ′ i = proj(M, u i ) Normalize gaze Rotation g ′ = R n g Extracted data U ′ , g ′ YES NO Fig. 1. Pipeline for the extraction of landmark-based dataset. Images are processed to extract landmarks and head pose, which are then used to normalize the data into a virtual camera space. quality of the datasets as the main bottlenecks of this approach. I. METHODS In this study, we propose and assess a framework for gaze estimation based only on landmarks. In Section I-A we develop a procedure to extract landmark-based datasets from three suitable datasets that are available in the litera- ture: Gaze360 [36], ETH-Xgaze [37], and GazeGene [38]. In Section I-B, we describe the proposed network architectures, training procedures and evaluation protocols. A. Extraction of landmark-based datasets We extract normalized landmark-based datasets from la- beled images following the pipeline summarized in Fig. 1. For each image, we detect dense facial landmarks, estimate the head pose, and normalize the features together with ground- truth gaze labels. 1) Facial landmark detection: We detect faces and extract landmarks using MediaPipe [39], which generates a dense mesh of 478 points, including 10 landmarks of pupils and irises. We discard detections with a confidence score below 0.8. To reduce detection failures on cropped face images (such as those in the GazeGene [38] and ETH-Xgaze datasets [37]), we apply symmetric black padding to the image (extending each side by 25% of the original dimensions). From the dense face mesh, we select a subset of N = 20 landmarks critical for gaze estimation: two stable head anchors (nose tip and glabella) and, for each eye, the pupil center, four iris extrema, and four eye contour landmarks (corners and eyelids extrema), as shown in Fig. 2. We denote byU =u i N i=1 the set of these 2D landmarks, where u i = [u i,x , u i,y ] ⊤ in the raw image coordinate system. 2) Head pose estimation: In each image, we estimate the head pose by aligning the detected landmarks to a canonical 3D face model with origin at the midpoint between the eyes and nose corners [40]. In particular, we use OpenCV [41] to solve the Perspective-n-Point (PnP) problem to determine the rotation matrix R h = h e (h) 1 |e (h) 2 |e (h) 3 i and the translation (a)(b) Fig. 2. Example image from the GazeGene dataset [38]: (a) original and (b) normalized images. In (b), the red arrow indicates the normalized gaze direction, cyan lines are the image principal axis, and green circles denote the N = 20 landmarks extracted from the MediaPipe face mesh [39] to represent gaze and head orientation: the right iris (indices 473–477) and eye contour (263, 362, 374, 386), the left iris (468–472) and eye contour (33, 133, 145, 159), and two head anchors given by the nose tip (1) and the glabella (9). vector t h that map the 3D face model to the camera coordinate system. 3) Data normalization: To remove variability in face dis- tance, position, and in-plane rotation, we map data to a normalized space following the procedure described in [22]. Using the estimated head pose (R h , t h ), we construct the normalized coordinate system e (n) 1 = e (n) 2 × e (n) 3 , e (n) 2 = e (n) 3 × e (h) 1 ∥e (n) 3 × e (h) 1 ∥ , e (n) 3 = t h ∥t h ∥ , (1) where ∥·∥ denotes the Euclidean norm. Then we can map the original image plane to the normalized image plane through the perspective warping matrix M = K n SR n K −1 c ,(2) where K c is the intrinsic matrix of the physical camera (either provided in the dataset or estimated from the image size), R n = h e (n) 1 |e (n) 2 |e (n) 3 i ⊤ is the rotation matrix to the normalized coordinate system, S = diag ∥t h ∥ d n , ∥t h ∥ d n , 1 scales the face to a fixed distance d n , and K n is the intrinsic matrix of the virtual camera defined by focal length f n and principal point (c x , c y ). In conclusion, for each raw image, we determine the normalized landmarks, U ′ = u ′ i N i=1 , via perspective projection of the raw landmarks U , i.e., u ′ i = v i,x v i,z , v i,y v i,z ⊤ ,v i = M u i,x u i,y 1 .(3) Analogously, we rotate the 3D ground-truth gaze vector g into the normalized frame as g ′ = R n g.(4) In this study, we configure the normalized camera with an image resolution of 448× 448 pixels (c x = c y = 224), a focal length f n = 960, and a normalized distance d n = 300 m (Fig. 2). 4) Dataset-specific processing: a) Gaze360 [36]: This dataset consists of raw images with variable resolution, for which ground-truth head poses are not provided. We first detect facial landmarks (without padding) and estimate the head pose by refining an initial Efficient PnP (EPnP) solution via the Levenberg-Marquardt method. Then, we normalize landmarks and ground-truth gaze vectors as detailed in Section I-A3; to this aim, we approximate the camera intrinsics by a pinhole camera model with the focal length set to the image width and the principal point at the image center. b) ETH-XGaze [37]: We use the pre-processed version of face images with a resolution of 448× 448 pixels, which are already normalized according to the procedure described in Section I-A3. Consequently, we bypass the head pose estimation and normalization steps, detecting the normalized landmark directly on the images (with padding). Follow- ing [37], we apply histogram equalization to the Y-channel for frames with an index greater than 524 to ensure consistency across varying lighting conditions. Finally, we convert the ground-truth labels, originally provided as pitch and yaw, into 3D unit vectors in the normalized camera coordinate system. c) GazeGene [38]: For this dataset, we implement the full processing pipeline: facial landmark detection (with padding), head pose estimation, and data normalization as described in Secs. I-A1, I-A2 and I-A3. We estimate the head pose by solving the Perspective-n-Point (PnP) problem using the OpenCV iterative solver. We initialize the solver with the ground-truth head pose provided by the dataset, but we compute the normalization matrix R n by the re-estimated pose to align the normalization warp and the detected features. B. Landmark-based gaze estimation In this study, we propose and evaluate three regression models to map the N = 20 normalized landmarks, U ′ , to the 3D gaze vector in the normalized camera coordinate system, g ′ ∈R 3 : a holistic MLP, a siamese MLP, and an XGBoost decision tree. Figure 3 shows the diagrams of the three architectures. 1) Feature representation: We transform the normalized coordinates U ′ into feature vectors, using geometric centering and scaling. a) Global features (f G ): For models processing the face as a whole, i.e., holistic MLP and XGBoost, we compute a single reference centroid c E as the midpoint of the four eye corners. We center all landmarks relative to c E and scale them by the normalized image width w = 448, resulting in the feature vector f G ∈R 40 that preserves the relative geometry of the entire face. b) Local features (f L , f R , f H , ∆c): For the siamese MLP, we define the centroids of the left and right eye corners, c L and c R . For each eye there are N e = 9 landmarks, which are centered relative to their respective centroid and scaled by w, yielding two local feature vectors, f L , f R ∈R 18 . The two remaining landmarks (head anchors) are concatenated in the vector f H ∈R 4 . Finally, we compute the relative position vector ∆c = (c R − c L ) /w. 2) Neural architectures: We design two deep learning architectures sharing a common residual block structure with hidden width D, which maps h ∈R D into y = h +F (h) ∈ R D , where the residual function F (·) is the sequence shown in Fig. 3d, namely, Linear(D→D) → BatchNorm → GELU → Dropout(p) → Linear(D→D) → BatchNorm → GELU → Dropout(p). We report the specific configurations for each network in Tab. I. a) Holistic MLP: This network processes facial features through a monolithic approach. The architecture consists of an input projection layer from the vector f G to a hidden width D = 256 (Linear(2N→D) → BatchNorm → GELU), followed by a stack of K = 3 residual blocks, and a regression head that reduces the dimension to D/2 before predicting the final 3D gaze vector (Linear(D→D/2) → GELU → Linear(D/2→3)). Figure 3a shows a diagram of this architecture. b) Siamese MLP: This architecture models the binocular nature of gaze using two parallel branches and a fusion stage (Fig. 3b). Two independent encoders process f L and f R in parallel, each mirroring the structure of the holistic model: an input projection layer followed by K = 3 residual blocks with width D = 64. To account for the spatial relationship between landmarks, we concatenate the latent vectors from the eye encoders with the relative position vector, ∆c = (c R − c L ) /w ∈R 2 , and the normalized coordinates of the head anchors, f H ∈R 4 . Finally, a fusion MLP predicts the gaze vector from these features (Fig. 3b). 3) XGBoost: To establish a non-deep learning baseline, we train an extreme gradient boosted (XGBoost) regressor using the global features f G . Unlike monolithic neural networks, this architecture employs an ensemble of decision trees trained se- quentially. Since gradient boosting regressors typically predict scalar values, we employ a multi-output strategy, wrapping three independent estimators for the three components of the gaze vector, g ′ = g ′ x , g ′ y , g ′ z , as shown in Fig. 3c. Each estimator is an additive model composed of K = 1000 decision trees, where each tree has a maximum depth of 6. TABLE I SUMMARY OF INPUT REPRESENTATION AND SPECIFIC CONFIGURATIONS OF THE MLPS. FeatureHolistic MLPSiamese MLP Input representationGlobal landmarks (f G )Local eyes (f L ,f R ), geometry (∆c,f H ) Hidden width (D)25664 (per branch) N. residual blocks (K)33 (per branch) Dropout (p)0.10.1 (d) Residual block Input h∈R D + Output y∈R D Linear (D→D) ↓ BatchNorm ↓ GELU ↓ Dropout (p) ↓ Linear (D→D) ↓ BatchNorm ↓ GELU ↓ Dropout (p) (a) Holistic MLP Normalized landmarks U ′ =u i N i=1 Global features Center relative to c E scale by w f G ∈R 2N Input projection Lin→BN→GELU (width D) · Residual block (width D) ×K · Regression Head Lin (D→D/2) ↓ GELU ↓ Lin (D/2→3) 3D gaze vector g ′ (b) Siamese MLP Normalized landmarks U ′ =u i N i=1 Left features Center rel. to c L scale by w f L ∈R 2N e Input projection Lin→BN→GELU (width D) · Residual block (width D) ×K · e L ∈R D Right features Center rel. to c R scale by w f R ∈R 2N e Input projection Lin→BN→GELU (width D) · Residual block (width D) ×K · e R ∈R D Context features Relative position ∆c = c R − c L w Head anchors f H Fusion MLP Linear (2(D + 1 + N − N e )→2D) ↓ BatchNorm ↓ GELU ↓ Dropout (p) ↓ Linear (2D→D) ↓ GELU ↓ Linear (D → 3) 3D gaze vector g ′ (c) XGBoost Normalized landmarks U ′ =u i N i=1 Global features Center relative to c E scale by w f G ∈R 2N GB regressor (g ′ y ) GB regressor (g ′ x ) GB regressor (g ′ z ) Multi-output 3D gaze vector g ′ Fig. 3. Architectural comparison of gaze estimation models. (a) Holistic Multi-Layer Perceptron (MLP): facial landmarks are processed through an input projection layer, a stack of K residual blocks and a final regression head. (b) Siamese MLP: facial landmarks are split into local eye regions, processed by two independent encoders, and fused with geometric context (relative eye positions ∆c and head anchorsf H ) via a fusion MLP. (c) XGBoost: a gradient-boosted tree approach using a multi-output regressor for the gaze vector components,g ′ = g ′ x , g ′ y , g ′ z , based on global landmark features,f G . (d) Residual block used in (a) and (b), featuring a residual connection around two sets of Linear, BatchNorm, GELU, and Dropout layers. 4) Training and evaluation: We train the MLPs using an AdamW optimizer (weight decay of 10 −4 , batch size of 64) and a ReduceLROnPlateau scheduler (factor of 0.5, patience of 4− 5 epochs) starting from a value of 10 −1 for the holistic model and 10 −3 for the siamese one. Both networks are trained for a maximum of 200 epochs with early stopping (patience of 15 epochs). We define the loss function as the angular error between the predicted gaze vector g ′ p and the ground-truth g ′ , i.e., E g ′ , g ′ p = arccos g ′ p · g ′ ∥g ′ p ∥g ′ ∥ .(5) This loss function maximizes cosine similarity, ensuring that the gaze direction is accurate regardless of the magnitude. We train the XGBoost model using the histogram-based tree method, minimizing the mean squared error, with a learning rate of 0.05 and a subsampling of 80% of training instances and 80% of features per tree. To evaluate the generalization capability of the models, we split each dataset into training (∼80%), validation (∼10%), and testing (∼10%) sets. In particular, for GazeGene, we use subjects 1-46 for training, 47-51 for validation and 52-56 for testing; for ETH-Xgaze, we randomly split the 80 annotated subjects into 64 for training, 8 for validation, and 8 for testing; for Gaze360, we adopt the same splitting as in the original study [36]. First, we train models using training and validation sets. Then, we evaluate model performances in terms of the Mean Angular Error on the testing set of the same dataset used for training (within-domain evaluation), and on the entirety of the other datasets (cross-domain evaluation). I. RESULTS In this section, we present the quantitative results of the proposed landmark-based framework. In Section I-A, we an- alyze the extracted landmark datasets compared to the original ones. In Section I-B, we evaluate the within-domain accuracy of the proposed regression models (holistic and siamese MLPs, and XGBoost). In Section I-C, we present a cross-domain evaluation to assess generalization capabilities, comparing our lightweight models with the ResNet18 convolutional neural network. In Section I-D, we analyze the results of the training process using the Permutation Feature Importance method. A. Analysis of extracted datasets We report in Tab. I the retention rates of the face detection process using MediaPipe [39] with a minimum confidence of 0.8, as described in Section I-A1. For all datasets, more than 70% of the images were processed successfully, with the highest rate achieved on GazeGene (82.61%). TABLE I DATASET RETENTION STATISTICS. THE RETENTION RATE INDICATES THE PERCENTAGE OF IMAGES WHERE MEDIAPIPE SUCCESSFULLY DETECTED THE FACIAL LANDMARKS WITH CONFIDENCE≥ 0.8. DatasetOriginal samplesExtracted samplesRetention rate Gaze360 [36]16993513265178.06% ETH-XGaze [37]75654055905773.90% GazeGene [38]100800083268782.61% To investigate the geometric bias introduced by the extrac- tion process, we compare the density distributions of gaze and head angles before and after processing. Figure 4 illustrates such distributions for all datasets, together with the density of the excluded samples, showing that the landmark detector is robust for central gaze directions but tends to fail at extreme angles. This implies a reduction in the effective ranges of the extracted datasets compared to the original ones. B. Within-domain evaluation In Tab. I, we compare the within-domain performance of the proposed models (holistic and siamese MLPs, and XGBoost) against the results reported in the literature for a ResNet18 architecture [1], [38], representing a standard baseline for convolutional neural networks. Both MLPs con- sistently performed better than the XGBoost model, with the siamese MLP achieving the best results among all the landmark-based methods. Notably, the ResNet18 baseline out- performs all other methods in the within-domain evaluation. This performance gap might be due to noise introduced in the extracted datasets, resulting from inaccuracies in the detection of facial landmarks and, consequently, head pose estimation. Indeed, qualitative inspection of the processed images reveals evident landmark detection errors (Fig. 5), suggesting that the performances of these models are limited by the precision of the extracted datasets. TABLE I RESULTS OF WITHIN-DOMAIN EVALUATION. PERFORMANCE FOR LANDMARK-BASED MODELS IS PRESENTED AS MEAN ANGULAR ERROR AND STANDARD DEVIATION (SD) IN DEGREES ( ◦ ), CALCULATED ACROSS ALL TESTED SAMPLES. RESULTS FOR THE RESNET18 ARE REPORTED FROM THE LITERATURE [38]. BOLD VALUES INDICATE THE BEST PERFORMANCE. Image-basedLandmark-based DatasetResNet18XGBoostHolistic MLPSiamese MLP GazeGene3.7613.49 (8.71)9.00 (7.10)8.91 (7.02) ETH-XGaze5.7311.63 (7.81)8.54 (6.71)8.50 (6.59) Gaze36012.2318.81 (14.38)14.95 (12.66)14.68 (12.70) C. Cross-domain evaluation We report in Tab. IV the results of the cross-domain evalu- ation. Both landmark-based MLPs demonstrate generalization capabilities comparable to the ResNet18 baseline [38], while the XGBoost architecture shows consistently lower perfor- mances. In fact, while ResNet18 generally achieves lower errors in within-domain settings, its performance degrades significantly in cross-domain settings. In contrast, landmark- based models exhibit a relatively lower performance drop. The siamese MLP maintains the highest accuracy among landmark methods in all cross-domain scenarios. D. Feature importance analysis To interpret the models trained on landmark-based datasets, we analyze the contribution of different facial landmarks to the gaze estimation task using Permutation Feature Importance (PFI). This method measures the dependence of the model on specific features by evaluating the performance degradation when those features are randomly permuted. In the following, we limit our discussion to the siamese MLP, as it achieves the best within- and cross-domain results among the three landmark-based approaches. We define feature groups based on the semantic structure of the input data: the iris (pupil center and iris contour), the eye contour (eyelids and corners), the head anchors (glabella and nose bridge), and the relative position vector. For any feature group f , we construct a validation set D f by permuting the values of the features within f in the original validation set D, while keeping all other features and target labels fixed. We define the importance score I f as the increase in the Mean Angular Error (MAE) relative to the baseline performance, i.e., I f = ̄ E (D f )− ̄ E (D)(6) where ̄ E(·) represents the mean of the loss function (5) on the given dataset. We repeat the permutation procedure 1000 times for each feature group and report the results in Fig. 6. 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Gaze Original 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Extracted 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Excluded No data Head 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) No data 0 2 4 6 1e4 0 2 4 6 1e4 0 2 4 6 1e4 0.0 0.5 1.0 1.5 1e3 Gaze360 (a) 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Gaze Original 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Extracted 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Excluded 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) Head 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) 0.0 0.5 1.0 1.5 1e4 0.0 0.5 1.0 1.5 1e4 0.0 0.5 1.0 1.5 1e4 0 1 2 1e4 0 1 2 1e4 0 1 2 1e4 GazeGene (b) 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Gaze Original 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Extracted 12060060120 Yaw ( ) 120 60 0 60 120 Pitch ( ) Excluded 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) Head 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) 804004080 Yaw ( ) 80 40 0 40 80 Pitch ( ) 0.0 0.5 1.0 1.5 2.0 1e4 0.0 0.5 1.0 1.5 2.0 1e4 0 1 2 3 1e4 0.0 0.5 1.0 1.5 1e3 0.0 0.5 1.0 1.5 1e3 0 2 4 6 1e3 ETH-XGaze (c) Fig. 4. Distributions of yaw and pitch angles (in degrees) for gaze (first row) and head pose (second row) across the three datasets - Gaze360 (a), GazeGene (b), and ETH-XGaze (c). Columns (left to right) show the distributions of the original, extracted, and excluded samples. Colors represent density values according to the reported colorbars. TABLE IV RESULTS OF CROSS-DOMAIN EVALUATION. FOR EACH MODEL ARCHITECTURE, ROWS AND COLUMNS REPRESENT THE TRAINING AND TESTING DATASETS, RESPECTIVELY. PERFORMANCE FOR LANDMARK-BASED MODELS ARE PRESENTED AS MAE (SD) IN DEGREES ( ◦ ), WHERE SD IS CALCULATED ACROSS ALL TESTED SAMPLES. RESULTS FOR THE RESNET18 ARE REPORTED FROM THE LITERATURE [38]. BOLD VALUES INDICATE THE BEST PERFORMANCE FOR EACH TEST DATASET. ModelResNet18XGBoost Train Test GazeGeneETH-XgazeGaze360GazeGeneETH-XgazeGaze360 GazeGene-12.8724.20-17.56 (10.51)17.31 (12.52) ETH-Xgaze 15.42-19.4718.83 (10.99)-24.28 (13.71) Gaze360 18.8018.51-25.89 (15.95)24.88 (17.10)- ModelHolistic MLPSiamese MLP Train Test GazeGeneETH-XgazeGaze360GazeGeneETH-XgazeGaze360 GazeGene-15.35 (11.00)17.29 (12.95)-13.85 (9.87)17.28 (13.04) ETH-Xgaze 15.16 (10.10)-21.86 (14.78)14.85 (10.03)-21.27 (14.67) Gaze36017.13 (11.52)20.13 (13.03)-15.99 (10.81)17.55 (11.38)- For GazeGene, the head anchors are the most critical feature group, followed by the irises with a bias towards the right one (likely due to the biased dataset distribution, as shown in Fig. 4). For ETH-XGaze, the irises and eye contours are the most important features, with the head anchors playing a secondary role. Finally, for Gaze360, the eye contours emerge as the primary contributors, followed by the irises. IV. DISCUSSION Wefindthatlandmark-basedmodelsunderperform appearance-based approaches in within-domain evaluations. We hypothesize that the lower within-domain performance is due to noise introduced during the dataset extraction, particularly inaccuracies in face detection and head pose estimation, which degrade the quality and coverage of Fig. 5. Examples of evident errors in face-landmark detection from different datasets (GazeGene [38], Gaze360 [36], ETH-XGaze [37]). Green circles denote the N = 20 landmarks extracted using MediaPipe [39]. These errors are likely due to low image resolution, face cropping, or extreme poses. Fig. 6. Permutation Feature Importance (PFI) analysis of the siamese MLP trained on ETH-Xgaze, GazeGene, and Gaze360. The importance of each feature group is measured by the increase in mean angular error ( ◦ ) when the corresponding input features are permuted while keeping others constant. Higher values indicate a greater reliance of the model on that specific feature group. Error bars represent the standard deviation over 1000 permutation cycles. the original training data, as shown in Section I-A. However, landmark-based MLP networks achieve cross- domain performances that are comparable to those of the computationallyheavierResNet18baseline.Therefore, MLPs experience a lower performance drop with respect to ResNet18, thus exhibiting an overall stronger cross-domain generalization. This suggests that facial landmarks encode the essential geometric information for accurate gaze estimation, while being less sensitive to domain-specific variations. Indeed, by operating on sparse geometric coordinates rather than on pixel intensities, landmark-based models are inherently invariant to factors such as illumination and skin tone differences. The siamese MLP, which explicitly models the binocular structure of gaze, produces the best results among landmark- based methods, suggesting that architectures designed to re- flect the underlying geometry can better capture generalized gaze rules than holistic or tree-based approaches. Beyond accuracy and generalization, the landmark-based framework offers practical benefits. First, replacing a deep convolutional backbone with simple MLPs dramatically re- duces the computational cost during regression, making it ideal for devices with lower computational capabilities; in fact, both approaches require an initial detection and normalization step, but the one based on landmarks removes the need for a heavy feature extractor after normalization. Second, storing or processing only landmark coordinates might also reduce the exposure of sensitive biometric data. Third, the landmark- based approach provides greater interpretability of trained models. In particular, by a Permutation Feature Importance (PFI) analysis, we show that models rely on different features, depending on the training dataset. For example, the model trained on GazeGene shows a bias towards the right eye, likely reflecting the underlying distribution of the synthetic data. This interpretability might serve as a diagnostic tool, allowing researchers to effectively assess the quality and biases of gaze datasets. The primary bottleneck of the proposed approach is the accuracy and robustness of the upstream landmark detector. As observed in the analysis of the extracted datasets, the detector fails or produces unreliable results at extreme gaze and head pose angles, leading to a reduction in the actual quality, range and size of the datasets. For the proposed pipeline, the retention rate is between 73% and 82% across the three considered datasets (Gaze360, GazeGene and ETH-XGaze). Moreover, inaccuracies in landmark detection propagate into the normalization procedure, likely contributing to the performance gap observed between our models and the ResNet18 baseline in within-domain tests. Therefore, we expect that future improvements in the reliability of landmark detectors would directly translate to performance gains in this framework. In conclusion, our results support the fact that synthetic data can be used successfully for geometric gaze estimators. In fact, in the landmark-based approach, photorealistic images are not necessary, as long as geometric features are preserved. Therefore, future work should focus on enhancing landmark detection and expanding datasets to cover broader geometric and demographic variation without requiring photorealism. V. CONCLUSION This study supports the viability of estimating gaze direc- tion solely from facial landmarks. By evaluating lightweight MLPs and decision tree ensembles against state-of-the-art appearance-based baselines, we show that geometric features alone contain sufficient information for accurate gaze estima- tion. Although current landmark detectors introduce noise that limits within-domain precision, the proposed siamese MLP architecture exhibits comparable generalization performances of significantly heavier ResNet18 models. These findings highlight a clear path toward hardware-efficient and privacy- preserving gaze tracking. By decoupling the regression task from raw pixel data, we reduce the computational cost and exposure of sensitive biometric information. DATA AND CODE AVAILABILITY Source code and extracted landmark datasets are available at https://github.com/daniele-agostinelli/LandmarkGaze.git. REFERENCES [1] Y. Cheng, H. Wang, Y. Bao, and F. Lu, “Appearance-based gaze estima- tion with deep learning: A review and benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, p. 7509– 7528, 2024. [2] E. Guestrin and M. Eizenman, “General theory of remote gaze estimation using the pupil center and corneal reflections,” IEEE Transactions on Biomedical Engineering, vol. 53, no. 6, p. 1124–1133, 2006. [3] D. W. Hansen and Q. Ji, “In the eye of the beholder: A survey of models for eyes and gaze,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 3, p. 478–500, 2010. [4] K. Takemura and K. Yamagishi, “A hybrid eye-tracking method using a multispectral camera,” in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).IEEE Press, 2017, p. 1529–1534. [Online]. Available: https://doi.org/10.1109/SMC.2017. 8122831 [5] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “Appearance-based gaze estimation in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. [6] Y. Cheng, X. Zhang, F. Lu, and Y. Sato, “Gaze estimation by exploring two-eye asymmetry,” IEEE Transactions on Image Processing, vol. 29, p. 5259–5272, 2020. [7] S. Park, A. Spurr, and O. Hilliges, “Deep pictorial gaze estimation,” in Proceedings of the European Conference on Computer Vision (ECCV), September 2018. [8] S. Park, S. D. Mello, P. Molchanov, U. Iqbal, O. Hilliges, and J. Kautz, “Few-shot adaptive gaze estimation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019. [9] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “It’s written all over your face: Full-face appearance-based gaze estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017. [10] Z. Chen and B. E. Shi, “Appearance-based gaze estimation using dilated-convolutions,” in Computer Vision – ACCV 2018, C. Jawahar, H. Li, G. Mori, and K. Schindler, Eds.Cham: Springer International Publishing, 2019, p. 309–324. [11] Y. Cheng, S. Huang, F. Wang, C. Qian, and F. Lu, “A coarse- to-fine adaptive network for appearance-based gaze estimation,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, p. 10 623–10 630, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/6636 [12] Y. Bao, Y. Cheng, Y. Liu, and F. Lu, “Adaptive feature fusion network for gaze tracking in mobile tablets,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, p. 9936–9943. [13] Y. Cheng and F. Lu, “Gaze estimation using transformer,” in 2022 26th International Conference on Pattern Recognition (ICPR), 2022, p. 3341–3347. [14] Y. Liu, R. Liu, H. Wang, and F. Lu, “Generalizing gaze estimation with outlier-guided collaborative adaptation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oc- tober 2021, p. 3835–3844. [15] Y. Wang, Y. Jiang, J. Li, B. Ni, W. Dai, C. Li, H. Xiong, and T. Li, “Contrastive regression for domain adaptation on gaze estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, p. 19 376–19 385. [16] Y. Bao, Y. Liu, H. Wang, and F. Lu, “Generalizing gaze estimation with rotation consistency,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, p. 4207–4216. [17] X. Cai, J. Zeng, S. Shan, and X. Chen, “Source-free adaptive gaze estimation by uncertainty reduction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, p. 22 035–22 045. [18] Y. Bao and F. Lu, “From feature to gaze: A generalizable replacement of linear layer for gaze estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, p. 1409–1418. [19] Y. Cheng, Y. Bao, and F. Lu, “Puregaze: Purifying gaze feature for generalizable gaze estimation,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, p. 436–443, Jun. 2022. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/19921 [20] M. Xu, H. Wang, and F. Lu, “Learning a generalized gaze estimator from gaze-consistent feature,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 3, p. 3027–3035, Jun. 2023. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/25406 [21] Z. Liang, Y. Bao, and F. Lu, “De-confounded gaze estimation,” in Computer Vision – ECCV 2024, A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol, Eds. Cham: Springer Nature Switzerland, 2025, p. 219–235. [22] X. Zhang, Y. Sugano, and A. Bulling, “Revisiting data normalization for appearance-based gaze estimation,” in Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ser. ETRA ’18.New York, NY, USA: Association for Computing Machinery, 2018. [Online]. Available: https://doi.org/10.1145/3204493.3204548 [23] K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Ma- tusik, and A. Torralba, “Eye tracking for everyone,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. [24] Y. Yu, G. Liu, and J.-M. Odobez, “Deep multitask gaze estimation with a constrained landmark-gaze model,” in Proceedings of the European Conference on Computer Vision (ECCV) Workshops, September 2018. [25] Y. Lei, S. He, M. Khamis, and J. Ye, “An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices,” ACM Comput. Surv., vol. 56, no. 2, Sep. 2023. [Online]. Available: https://doi.org/10.1145/3606947 [26] T. Agostinelli, A. Generosi, and M. Mengoni, “A novel approach for monocular rgb-based ergonomics monitoring in industrial workspaces employing synthetic datasets to train a deep learning model,” The International Journal of Advanced Manufacturing Technology, p. 1– 24, 2026. [27] A. A. Macedo, L. Persona, and F. Meloni, “Recognition of emotions through facial geometry with normalized landmarks,” in Brazilian Sym- posium on Multimedia and the Web (WebMedia). SBC, 2024, p. 257– 266. [28] A. Kumar, A. Kumar, and S. Gupta, “Machine learning-driven emotion recognition through facial landmark analysis,” SN Computer Science, vol. 6, no. 2, p. 120, 2025. [29] J. Chen and Q. Ji, “3d gaze estimation with a single camera without ir illumination,” in 2008 19th International Conference on Pattern Recognition, 2008, p. 1–4. [30] K. Wang and Q. Ji, “Real time eye gaze tracking with 3d deformable eye-face model,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. [31] S. Park, X. Zhang, A. Bulling, and O. Hilliges, “Learning to find eye region landmarks for remote gaze estimation in unconstrained settings,” in Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ser. ETRA ’18.New York, NY, USA: Association for Computing Machinery, 2018. [Online]. Available: https://doi.org/10.1145/3204493.3204545 [32] J. Oh, Y. Lee, J. Yoo, and S. Kwon, “Improved feature-based gaze esti- mation using self-attention module and synthetic eye images,” Sensors (Basel), vol. 22, no. 11, p. 4026, May 2022. [33] P. Her, L. Manderle, P. A. Dias, H. Medeiros, and F. Odone, “Uncertainty-aware gaze tracking for assisted living environments,” IEEE Transactions on Image Processing, vol. 32, p. 2335–2347, 2023. [34] E. E. Ye, J. E. Ye, J. Ye, J. Ye, and R. Ye, “Low-cost geometry-based eye gaze detection using facial landmarks generated through deep learning,” 2023. [Online]. Available: https://arxiv.org/abs/2401.00406 [35] E. Wood, T. Baltru ˇ saitis, L.-P. Morency, P. Robinson, and A. Bulling, “Learning an appearance-based gaze estimator from one million synthesised images,” in Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, ser. ETRA ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 131–138. [Online]. Available: https://doi.org/10.1145/2857491.2857492 [36] P. Kellnhofer, A. Recasens, S. Stent, W. Matusik, and A. Torralba, “Gaze360: Physically unconstrained gaze estimation in the wild,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019. [37] X. Zhang, S. Park, T. Beeler, D. Bradley, S. Tang, and O. Hilliges, “Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation,” in Computer Vision – ECCV 2020, A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Cham: Springer International Publishing, 2020, p. 365–381. [38] Y. Bao, Z. Wang, and F. Lu, “Gazegene: Large-scale synthetic gaze dataset with 3d eyeball annotations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. [39] C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann, “Mediapipe: A framework for perceiving and processing reality,” in Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR) 2019, 2019. [Online]. Available: https://mixedreality.cs.cornell.edu/s/ NewTitleMay1MediaPipeCVPRCV4ARVRWorkshop2019.pdf [40] E. Marchand, H. Uchiyama, and F. Spindler, “Pose Estimation for Augmented Reality: A Hands-On Survey,” IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 12, p. 2633 – 2651, Dec. 2016. [Online]. Available: https://inria.hal.science/hal-01246370 [41] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.