Paper deep dive
From Alignment to Prediction: A Study of Self-Supervised Learning and Predictive Representation Learning
Mintu Dutta, Ritesh Vyas, Mohendra Roy
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/18/2026, 1:28:29 AM
Summary
This paper introduces Predictive Representation Learning (PRL) as a distinct category of self-supervised learning (SSL) that focuses on latent-space prediction of unobserved data components from observed context. The authors propose a new taxonomy for SSL, positioning Joint-Embedding Predictive Architectures (JEPA) as a canonical framework. Through comparative analysis of BYOL, MAE, and I-JEPA, the study demonstrates that while reconstruction-based methods like MAE achieve high similarity, predictive methods like I-JEPA offer superior robustness and structural dependency modeling.
Entities (5)
Relation Signals (3)
I-JEPA → exhibitsbetterrobustnessthan → MAE
confidence 95% · I-JEPA achieves the best robustness... MAE... exhibits relatively weak robustness of 0.55.
Predictive Representation Learning → includes → Joint-Embedding Predictive Architecture
confidence 95% · We argue that Joint-Embedding Predictive Architecture(JEPA) can be considered as an exemplary member of this new paradigm.
Predictive Representation Learning → contrastedwith → Alignment-based Learning
confidence 90% · We propose a common taxonomy that classifies PRL along with alignment and reconstruction-based learning approaches.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Self-supervised learning has emerged as a major technique for the task of learning from unlabeled data, where the current methods mostly revolve around alignment of representations and input recon struction. Although such approaches have demonstrated excellent performance in practice, their scope remains mostly confined to learning from observed data and does not provide much help in terms of a learning structure that is predictive of the data distribution. In this paper, we study some of the recent developments in the realm of self-supervised learning. We define a new category called Predictive Representation Learning (PRL), which revolves around the latent prediction of unobserved components of data based on the observation. We propose a common taxonomy that classifies PRL along with alignment and reconstruction-based learning approaches. Furthermore, we argue that Joint-Embedding Predictive Architecture(JEPA) can be considered as an exemplary member of this new paradigm. We further discuss theoretical perspectives and open challenges, highlighting predictive representation learning as a promising direction for future self-supervised learning research. In this study, we implemented Bootstrap Your Own Latent (BYOL), Masked Autoencoders (MAE), and Image-JEPA (I-JEPA) for comparative analysis. The results indicate that MAE achieves perfect similarity of 1.00, but exhibits relatively weak robustness of 0.55. In contrast, BYOL and I-JEPA attain accuracies of 0.98 and 0.95, with robustness scores of 0.75 and 0.78, respectively.
Tags
Links
- Source: https://arxiv.org/abs/2604.13518v1
- Canonical: https://arxiv.org/abs/2604.13518v1
Trouble viewing inline? Open PDF directly →
Full Text
38,670 characters extracted from source content.
Expand or collapse full text
11institutetext: Department of Information and Communication Technology, School of Technology, Pandit Deendayal Energy University, Gandhinagar, Gujarat 382007, India 22institutetext: Corresponding Author: mohendra.roy@ieee.org From Alignment to Prediction: A Study of Self-Supervised Learning and Predictive Representation Learning Mintu Dutta Ritesh Vyas Mohendra Roy Abstract Self-supervised learning has emerged as a major technique for the task of learning from unlabeled data, where the current methods mostly revolve around alignment of representations and input recon struction. Although such approaches have demonstrated excellent performance in practice, their scope remains mostly confined to learning from observed data and do not provide much help in terms of learning structure that is predictive of the data distribution. In this paper, we study some of the recent developments in the realm of self- supervised learning. We define a new category called Predictive Representation Learning (PRL) which revolves around latent prediction of unobserved components of data based on the observation. We propose a common taxonomy that classifies PRL along with alignment and reconstruction-based learning approaches. Furthermore, we argue that Joint-Embedding Predictive Architecture(JEPA) can be considered as an exemplary member of this new paradigm. We further discuss theoretical perspectives and open challenges, highlighting predictive representation learning as a promising direction for future self-supervised learning research. In this study, we implemented Bootstrap Your Own Latent (BYOL), Masked Autoencoders (MAE), and Image-JEPA (I-JEPA) for comparative analysis. The results indicate that MAE achieves perfect similarity of 1.00, but exhibits relatively weak robustness of 0.55. In contrast, BYOL and I-JEPA attain accuracies of 0.98 and 0.95, with robustness scores of 0.75 and 0.78, respectively. 1 Introduction Self-supervised learning has revolutionized representation learning by enabling models to learn from vast amounts of unlabeled data. The initial approaches were based on the use of handcrafted learning objectives, which were actually grounded in the principle of invariance learning, whereas current SSL approaches are based on more principled learning objectives such as contrastive alignment and reconstruction. Modern objectives (contrastive/reconstruction) are more general-purpose and require less domain-specific engineering. Despite their success, most existing methods focus on aligning representations of observed data or reconstructing input signals, rather than modeling predictive structure. Again, some of the most successful current SSL methods are neither contrastive (using negative samples) nor reconstruction-based. This article presents the argument that predictive latent modeling represents the next generation of self-supervised learning evolution. Structure of this paper: The basic structure of this article comprises the following key elements: i. presents a unified taxonomy of SSL methods, i. introduces Predictive Representation Learning as a new category, i. positions JEPA as a canonical predictive SSL framework, iv. discusses theoretical implications from our empirical study and future research directions. 2 Background and Fundamentals of Self-Supervised Learning This section reviews the core principles of self-supervised learning, including representation collapse, architectural asymmetry, and learning signal design. We highlight the key dimensions along which SSL methods differ. 2.1 Alignment-Based Self-Supervised Learning Contrastive Learning: Contrastive learning is a prominent paradigm in self-supervised learning that focuses on learning discriminative representations by comparing samples within a training batch. The central objective is to maximize the similarity between representations of different augmented views of the same input, known as positive pairs, while minimizing the similarity between representations of different inputs treated as negative pairs. This objective is commonly formulated using contrastive loss functions such as the InfoNCE loss[1]. Representative methods including SimCLR [2] and Momentum Contrast (MoCo) [3] have demonstrated that contrastive objectives can produce high-quality representations that are competitive with supervised learning on downstream tasks [2],[3] Despite their effectiveness, contrastive methods typically rely on large batch sizes or memory queues to provide sufficient negative samples, and their performance is sensitive to the choice of data augmentations and sampling strategies. These practical constraints increase computational complexity and motivate the development of alternative self-supervised learning approaches. Non-Contrastive Alignment Methods: The class of non-contrastive alignment algorithms is a type of self-supervised learning algorithm in which representations are learned by aligning the representations of different transformations of the same input but without using any negative examples explicitly. Rather than forcing differentiation between data points, non-contrastive algorithms ensure that no collapse occurs by making specific architecture choices, such as asymmetric networks, predictor heads, and stop gradients. The BYOL paper first proposed this concept by showing that a student network can predict its target network’s representation without negative pairs [4]. SimSiam further simplified this approach by removing the momentum encoder while retaining a predictor and stop-gradient mechanism [5]. However, despite the efficiency that is gained by not employing contrastive learning, which helps to greatly minimize the computational cost of training as well as the requirement of using large batch sizes and memory banks, the learning target still revolves around aligning representations between views. 2.2 Reconstruction-Based Self-Supervised Learning Reconstruction self-supervised learning techniques learn representations through reconstruction of the input signal’s missing/corrupted parts. Self-supervised learning is thus introduced by purposefully corrupting the input signal such that the representation learned can recover the signal. Classical self-supervised representation learning was pioneered by autoencoder methods, which have seen substantial development with the introduction of masking in recent years. For example, Masked AutoEncoders (MAE) work on the premise of masking a substantial portion of the input patches and reconstructing them. [6]. Similarly, methods such as BEiT leverage token reconstruction objectives inspired by masked language modeling [7]. Although reconstruction-based SSL effectively exploits partial observability and has demonstrated strong empirical results, these methods operate in input space, which imposes a high-dimensional reconstruction burden and may bias learning toward low-level details rather than semantic abstraction. These limitations motivate alternative approaches that shift the learning objective from input reconstruction to prediction in representation space. 2.3 Predictive Representation Learning With recent developments in SSL, a new family of approaches is being discovered that is not only limited to learning invariance or reconstruction but also extends to prediction in representation space. Alignment-based models focus on learning invariance between multiple views, while reconstruction-based models restore the masked input through the learned representations. However, both approaches rely on exploiting the observations. The emerging prediction approach, however, focuses on learning using the unobserved/missing parts with the help of context. The increased focus on prediction indicates that there is an interest in exploring predictive structure in the data. This paper refers to this new family of models as PRL and treats them as a different class of SSL approaches rather than a newly proposed paradigm.[8]. Formal Definition: Predictive Representation Learning encompasses self-supervised methods that define learning objectives in latent space through prediction of unobserved representations. Formally, given an unlabeled sample x∼x , the data is partitioned into an observed context c(x)c(x) and an unobserved target t(x)t(x). A context encoder maps c(x)c(x) to a latent representation, while a target encoder produces a representation of t(x)t(x). Training then minimizes a distance metric between the predicted and target embeddings, often using a separate target encoder. Crucially, this formulation avoids both explicit negative sampling and reconstruction of the original input, distinguishing PRL from contrastive and generative SSL approaches [8]. PRL methods learn representations by predicting the latent embedding of t(x)t(x) from c(x)c(x). Core Properties Predictive Representation Learning methods share several defining properties. i. Supervision arises from latent-space prediction rather than instance discrimination or input reconstruction, i. The learning objective introduces directionality, as representations of unobserved components are predicted from observed context, i. Representational collapse is mitigated through architectural mechanisms such as asymmetric encoders, predictor networks, and stop-gradient or momentum updates, rather than contrastive negatives or explicit variance regularization, iv. By operating in representation space and emphasizing prediction under partial observability, PRL methods are well suited for scalable learning across vision, video, audio, and multimodal domains In this study, we categorize JEPA-style architectures as representative implementations of PRL within this taxonomy. 3 Joint Embedding Predictive Architectures 3.1 Architecture Overview Joint Embedding Predictive Architectures (JEPA) belong to a class of predictive self-supervised learning methods that learn representations by predicting latent embeddings of unobserved data components from observed context. Unlike alignment-based or reconstruction-based approaches, JEPA does not compare multiple views symmetrically or reconstruct input signals. Instead, it formulates learning as a directional prediction problem in representation space. A typical JEPA architecture consists of three main components: i. Context Encoder: Maps the observed portion of an input sample to a latent representation. i. Target Encoder: Produces a latent representation of an unobserved or masked portion of the input; its parameters are not directly updated by gradients. i. Predictor Network: Transforms the context representation into a prediction of the target representation. Architectural asymmetry between the context and target paths, often implemented using stop-gradient or momentum-based updates, plays a central role in preventing representational collapse. 3.2 Learning Objective Let x∼x denote an unlabeled data sample drawn from distribution D. The input is partitioned into: i. an observed context c(x)c(x), i. an unobserved target component t(x)t(x). The context encoder fθf_θ produces a latent representation: zc=fθ(c(x)).z_c=f_θ (c(x) ). (1) The target encoder fθ¯f_ θ, whose parameters are updated using stop-gradient or exponential moving average (EMA), produces: zt=fθ¯(t(x)).z_t=f_ θ (t(x) ). (2) A predictor network gϕg_φ estimates the target representation: z^t=gϕ(zc). z_t=g_φ(z_c). (3) The JEPA training objective minimizes the discrepancy between predicted and target representations: ℒJEPA=x∼[‖gϕ(fθ(c(x)))−sg(fθ¯(t(x)))‖22],L_JEPA=E_x [ \|g_φ (f_θ(c(x)) )-sg (f_ θ(t(x)) ) \|_2^2 ], (4) where sg(⋅)sg(·) denotes the stop-gradient operation. This formulation defines learning entirely in latent space and avoids both explicit negative samples and input-level reconstruction. 4 Comparative Analysis of SSL Paradigms This section compares alignment-based, reconstruction-based, and predictive SSL methods in terms of objectives, collapse avoidance, and scalability. 4.1 Architectural Comparison As shown in Fig. 1, contrastive and non-contrastive self-supervised learning methods rely on symmetric view-based architectures, with contrastive approaches additionally requiring negative samples or memory queues. Reconstruction-based methods employ encoder–decoder designs that operate in input space to recover masked signals. In contrast, JEPA adopts an asymmetric dual-path architecture that predicts latent representations of unobserved components from context, avoiding both negative samples and input reconstruction. Figure 1: Architectural comparison of Contrastive, Non Contrastive, Reconstruction based and Predictive Representation 4.2 Representative Loss Functions and Formulations 4.2.1 Contrastive Loss (InfoNCE) Contrastive loss functions are widely used in methods such as SimCLR and MoCo to learn discriminative representations. A commonly adopted formulation is the InfoNCE loss: ℒInfoNCE=−logexp(sim(zi,zi+)/τ)exp(sim(zi,zi+)/τ)+∑j≠iexp(sim(zi,zj)/τ),L_InfoNCE=- (sim(z_i,z_i^+)/τ ) (sim(z_i,z_i^+)/τ )+ _j≠ i (sim(z_i,z_j)/τ ), (5) where ziz_i and zi+z_i^+ denote representations of a positive pair, zjz_j denotes representations of negative samples, sim(⋅,⋅)sim(·,·) is a similarity measure such as cosine similarity, and τ is a temperature parameter. This loss prevents representational collapse through the use of negative samples and encourages instance-level discrimination. Non-Contrastive Alignment Loss (BYOL / SimSiam): Non-contrastive alignment methods eliminate negative samples and rely on architectural asymmetry and gradient blocking. A typical loss function is defined as: ℒNC=‖gϕ(fθ(x1))−sg(fθ¯(x2))‖22,L_NC= \|g_φ (f_θ(x_1) )-sg (f_ θ(x_2) ) \|_2^2, (6) where fθf_θ is the online encoder, fθ¯f_ θ is the target encoder updated using stop-gradient or momentum mechanisms, gϕg_φ is a predictor network, and sg(⋅)sg(·) denotes the stop-gradient operation. These methods avoid collapse through predictor asymmetry and gradient blocking, while the objective enforces symmetric alignment between representations. 4.2.2 Reconstruction Loss (Autoencoders / MAE) Reconstruction-based self-supervised learning methods derive supervision by recovering masked or corrupted inputs. A generic reconstruction loss can be expressed as: ℒrec=‖x−D(E(x~))‖22,L_rec= \|x-D (E( x) ) \|_2^2, (7) where x~ x represents a masked or corrupted version of the input x, E(⋅)E(·) is an encoder, and D(⋅)D(·) is a decoder. Collapse is avoided through reconstruction fidelity; however, these losses operate in input space and are sensitive to low-level signal details. 4.2.3 Predictive Representation Learning Loss (JEPA) Predictive Representation Learning methods, including JEPA-style architectures, define learning objectives in latent space through prediction of unobserved components. A representative loss is given by: ℒPRL=x∼[‖gϕ(fθ(c(x)))−sg(fθ¯(t(x)))‖22],L_PRL=E_x [ \|g_φ (f_θ(c(x)) )-sg (f_ θ(t(x)) ) \|_2^2 ], (8) where c(x)c(x) denotes the observed context, t(x)t(x) the unobserved target component, and the remaining terms follow standard notation. This loss predicts latent representations of unseen components, does not rely on negative samples or input reconstruction, and avoids collapse through predictive structure. Comparison is shown in Table. 4. While both Table 1 and Table 4 compare self-supervised learning paradigms, they serve complementary purposes. Table 1 provides a high-level conceptual and architectural comparison across methods, whereas Table 4 focuses specifically on loss functions and optimization characteristics, including objective formulation and collapse avoidance mechanisms. Table 1: Conceptual and architectural comparison Dimension Contrastive SSL Non-Contrastive SSL Reconstruction SSL Predictive Representation Learning Representative methods SimCLR, MoCo BYOL, SimSiam MAE, BEiT JEPA-style methods Learning signal Discrimination View alignment Input reconstruction Latent prediction Negative samples Yes No No No Operates in input space No No Yes No Operates in latent space Yes Yes Partially Yes Objective symmetry Symmetric Symmetric Directional Directional Prediction target Instance embeddings View embeddings Pixels / tokens Unseen representations Collapse prevention Negatives Architecture, stop-gradient Reconstruction loss Architectural prediction Computational cost High Moderate High Moderate Semantic abstraction Medium Medium Low–medium High World-model capability Weak Weak Limited Strong 4.3 Empirical Comparison: Alignment vs Reconstruction vs Predictive Learning To complement the conceptual analysis, we present an empirical comparison across three major self-supervised learning paradigms: alignment-based (BYOL), reconstruction-based (MAE), and predictive representation learning (I-JEPA). Evaluation Metrics: i. Augmentation Similarity (mean ± std) i. Occlusion Robustness (mean ± std) Table 2: Empirical Comparison of SSL Paradigms from our implementation and training results of BYOL, I-JEPA for their Allighment, PRL and MAE Method Augmentation Similarity Occlusion Robustness BYOL (Alignment) 0.988±0.0070.988± 0.007 0.750±0.0460.750± 0.046 I-JEPA (PRL) 0.950±0.0270.950± 0.027 0.788±0.0450.788± 0.045 MAE (Reconstruction) 1.000±0.0001.000± 0.000 0.550±0.0860.550± 0.086 Observations: The following observations are derived from the empirical evaluation of BYOL, I-JEPA, and MAE based on our implementation. i. MAE achieves perfect similarity due to pixel-level reconstruction, but exhibits significantly lower robustness. i. BYOL achieves high similarity through strong alignment, but its robustness is lower than predictive methods. i. I-JEPA achieves the best robustness, demonstrating superior ability to handle occlusion and partial observability. Key Insight: These results highlight a fundamental trade-off: alignment and reconstruction objectives optimize similarity, whereas predictive objectives improve robustness. Predictive Representation Learning (PRL) provides a better balance by capturing structural dependencies, leading to improved generalization. Figure 2: Comparision of the results from our custom implimentation and training: Comparison of augmentation similarity and occlusion robustness across BYOL, I-JEPA, and MAE. Predictive learning (I-JEPA) achieves superior robustness despite lower similarity. 4.4 Benchmark Results of JEPA Variants We summarize representative benchmark results from published JEPA-based models across modalities. These results are drawn from recent CVPR and ICLR works and demonstrate the effectiveness of predictive representation learning. Table 3: Published Benchmark Results for JEPA Variants Model Modality Dataset Evaluation Protocol Performance I-JEPA [9] Vision ImageNet-1K Linear Probe (ViT-H/14) 72.8% Top-1 I-JEPA [9] Vision ImageNet-1K k-N ∼ 73% V-JEPA [10] Video Kinetics-400 Linear Probe Competitive with VideoMAE VL-JEPA [11] Vision-Language Image-Text datasets Retrieval / Alignment Improved cross-modal retrieval Graph-JEPA [12] Graph OGB datasets Node Classification SOTA / Competitive Observations: From both the benchmark results (published till date) and from our custom implementation results, we may draw the following observations: i. I-JEPA achieves strong performance on ImageNet linear evaluation without relying on reconstruction or contrastive losses. i. V-JEPA demonstrates competitive performance with reconstruction-based video SSL methods such as VideoMAE. i. Multimodal and graph JEPA variants show that predictive objectives generalize across domains. iv. These results highlight that predictive representation learning can achieve competitive performance while operating entirely in latent space. 4.5 Implementations of Joint Embedding Predictive Architectures Joint Embedding Predictive Architectures have been instantiated across multiple data modalities and learning settings, demonstrating the generality of predictive representation learning. i. I-JEPA (Image JEPA): A non-generative self-supervised learning approach that predicts latent embeddings of masked image regions from observed context, enabling the learning of semantic visual representations without pixel-level reconstruction [9]. i. VL-JEPA (Vision–Language JEPA): Extends the JEPA framework to jointly model visual and textual modalities by predicting continuous language embeddings from visual context, supporting unified vision–language representation learning [11]. i. V-JEPA 2 (Video JEPA): A video-based extension of JEPA that predicts representations of future or masked spatio-temporal regions, facilitating improved modeling of temporal dynamics and long-range dependencies [13]. iv. graph-JEPA: Applies JEPA principles to graph representation learning by predicting embeddings of nodes or subgraphs from observed graph context, enabling predictive learning on structured, non-Euclidean data [12]. v. d-jepa:Integrates JEPA-style latent prediction with denoising and generative modeling perspectives, interpreting JEPA as a generalized prediction framework in continuous representation spaces [14]. vi. deq-jepa: A sequence-oriented JEPA variant that performs discriminative sequential prediction of representations, enabling richer modeling of ordered visual or temporal data [15]. vii. seq-jepa: Combines JEPA with autoregressive and equivariant representation learning to support world modeling and structured prediction in sequential environments [16]. 5 New Taxonomy of Self-Supervised Learning This study adopts a taxonomy that organizes self-supervised learning methods according to the nature of their learning objectives rather than specific model architectures or data modalities, as illustrated in Fig. 3. Existing approaches are categorized into alignment-based methods, reconstruction-based methods, and predictive representation learning methods. Alignment-based approaches derive supervision by matching representations of observed views, reconstruction-based approaches focus on recovering masked input signals, while predictive representation learning emphasizes latent-space prediction of unobserved components from context. This taxonomy clarifies conceptual differences across self-supervised learning paradigms and provides a structured framework for positioning predictive architectures such as JEPA within the broader SSL landscape. The conceptual differences are listed in Table. 1 Figure 3: New Taxonomy for SSL categorization Table 4: Loss functions and optimization characteristics Aspect Contrastive Non-Contrastive Reconstruction Predictive (PRL) Representative loss InfoNCE Alignment loss Reconstruction loss Latent prediction loss Learning signal Discrimination View matching Input recovery Context-to-target prediction Negative samples Required Not required Not required Not required Operates in input space No No Yes No Operates in latent space Yes Yes Partially Yes Objective symmetry Symmetric Symmetric Directional Directional Collapse avoidance Negatives Architecture, stop-grad Reconstruction fidelity Predictive inconsistency Sensitivity to low-level details Low Low High Low Encouraged representation Instance separation Invariance Local detail Structural dependency 6 Why Predictive Representation Learning Is a Distinct SSL Category Predictive Representation Learning (PRL) is best understood as a distinct category within self-supervised learning due to fundamental differences in its learning objective, supervision mechanism, and representational focus. Rather than extending existing alignment or reconstruction strategies, PRL reframes self-supervision around prediction of latent representations corresponding to unobserved data, leading to qualitative differences in how representations are learned. 6.1 Learning Objective Perspective Most alignment-based self-supervised methods whether contrastive or non-contrastive optimize objectives that enforce consistency between representations derived from different views of the same observed input. Reconstruction-based methods instead minimize discrepancies between original inputs and their reconstructed versions. In both cases, optimization is driven by signals derived from data that is directly available to the model. PRL departs from this formulation by defining learning as a prediction task in representation space, where the objective is to estimate the embedding of a data component that is not directly observed. This predictive formulation introduces directionality and shifts the emphasis from similarity or fidelity to anticipation of missing information. 6.2 Nature of Self-Supervision In contrastive learning, supervision arises from distinguishing between positive and negative sample pairs, while non-contrastive alignment methods rely on architectural constraints to match representations across views. Reconstruction-based approaches generate supervision by recovering masked or corrupted input signals. In PRL, the supervisory signal is derived from the consistency between predicted and target embeddings of unobserved data components. Because the target representation corresponds to information outside the model’s immediate input, learning is driven by predictive relationships rather than surface-level similarity or reconstruction accuracy. 6.3 Role of Partial Observability While reconstruction-based methods introduce partial observability through masking, their objective ultimately requires exact recovery of missing input elements. Alignment-based methods generally assume that each view provides a complete observation of the underlying sample. PRL treats partial observability as an inherent condition of learning rather than a temporary obstacle. The model is not required to recover missing inputs, but only to predict their representations, allowing multiple possible inputs to map to similar latent targets. This encourages abstraction and reduces sensitivity to low-level variations. 6.4 Collapse Avoidance Mechanisms A central challenge in self-supervised learning is avoiding representational collapse, where all inputs are mapped to identical or low-variance representations. Different self-supervised learning paradigms address this issue through distinct objective formulations and architectural constraints. This section summarizes the collapse avoidance mechanisms used in alignment-based, reconstruction-based, and predictive representation learning approaches. 6.4.1 Contrastive Self-Supervised Learning Contrastive methods prevent collapse by explicitly enforcing separation between representations of different samples. A commonly used objective is the InfoNCE loss [1]: ℒInfoNCE=−logexp(sim(zi,zi+)/τ)exp(sim(zi,zi+)/τ)+∑j≠iexp(sim(zi,zj)/τ),L_InfoNCE=- (sim(z_i,z_i^+)/τ ) (sim(z_i,z_i^+)/τ )+ _j≠ i (sim(z_i,z_j)/τ ), (9) where ziz_i and zi+z_i^+ denote representations of positive pairs, zjz_j denotes representations of negative samples, sim(⋅,⋅)sim(·,·) is cosine similarity, and τ is a temperature parameter. Collapsed representations yield identical similarities for positive and negative pairs, resulting in a high loss and thus being penalized during optimization [2, 3]. 6.4.2 Non-Contrastive Alignment Methods Non-contrastive alignment methods avoid collapse without relying on negative samples by introducing architectural asymmetry and gradient blocking. A typical objective used in methods such as BYOL and SimSiam is: ℒNC=‖gϕ(fθ(x1))−sg(fθ¯(x2))‖22,L_NC= \|g_φ(f_θ(x_1))-sg (f_ θ(x_2) ) \|_2^2, (10) where fθf_θ denotes the online encoder, fθ¯f_ θ the target encoder updated via stop-gradient or exponential moving average, gϕg_φ a predictor network, and sg(⋅)sg(·) denotes the stop-gradient operation. The asymmetry introduced by the predictor and gradient blocking prevents trivial constant solutions from becoming stable optima [4, 5]. 6.4.3 Reconstruction-Based Self-Supervised Learning Reconstruction-based methods implicitly prevent collapse by enforcing fidelity to the input signal. A generic reconstruction loss can be expressed as: ℒrec=‖x−D(E(x~))‖2,L_rec= \|x-D(E( x)) \|^2, (11) where x~ x is a masked or corrupted version of the input x, E(⋅)E(·) is an encoder, and D(⋅)D(·) is a decoder. Collapsed representations are insufficient to reconstruct diverse inputs, resulting in large reconstruction error [6, 7]. 6.4.4 Predictive Representation Learning Predictive Representation Learning (PRL) avoids collapse by defining learning as prediction of latent representations corresponding to unobserved data components. A general PRL objective can be written as: ℒPRL=x∼[‖gϕ(fθ(c(x)))−sg(fθ¯(t(x)))‖22],L_PRL=E_x [ \|g_φ (f_θ(c(x)) )-sg (f_ θ(t(x)) ) \|_2^2 ], (12) where c(x)c(x) denotes an observed context, t(x)t(x) an unobserved target component, fθf_θ and fθ¯f_ θ are context and target encoders respectively, and gϕg_φ is a predictor. In this setting, collapsed representations are unable to predict diverse unseen targets across samples, leading to high prediction error. Collapse is therefore avoided through the predictive structure of the objective rather than explicit negatives or reconstruction losses [8]. 6.5 Representational Emphasis Alignment-based approaches primarily learn invariances to data augmentations, while reconstruction-based methods often retain fine-grained details necessary for input recovery. In contrast, PRL emphasizes representations that encode structural and relational information, as effective prediction of unobserved components requires capturing dependencies across different parts of the data. Table 5: Conceptual differences between self-supervised learning categories Dimension Alignment-Based SSL Reconstruction-Based SSL Predictive Representation Learning Primary learning objective Representation alignment Input recovery Latent prediction Source of supervision Observed views Observed inputs Unobserved components Objective formulation Symmetric Directional Directional Learning space Latent space Input space Latent space Role of masking or augmentation View generation Exact reconstruction Partial observability Treatment of missing information Implicitly ignored Explicitly recovered Predicted in representation space Collapse avoidance mechanism Negatives or architectural constraints Reconstruction loss Predictive structure Representational focus Invariance to transformations Local signal fidelity Structural dependencies Sensitivity to low-level details Low High Low Suitability for world modeling Limited Limited Strong 7 Open Challenges and Future Directions of Joint Embedding Predictive Architectures (JEPA) Based on this study we find the following challenges for JEPA architecture: i. Theoretical Understanding of Predictive Objectives: A key open challenge is the lack of formal theoretical guarantees explaining why latent predictive objectives yield stable and informative representations, particularly in the absence of contrastive negatives or reconstruction losses. i. Long-Horizon Prediction and Temporal Abstraction: Extending JEPA to long-term temporal prediction remains challenging due to issues such as error accumulation, representation drift, and the need for hierarchical temporal abstraction. i. Multimodal and Cross-Modal Prediction: While JEPA naturally supports multimodal learning, designing effective cross-modal predictive objectives and balancing modality-specific and shared representations remain open problems. iv. Scalability and Architectural Design Choices: Understanding how model size, predictor capacity, masking strategies, and encoder asymmetry affect performance is an open challenge, particularly for large-scale deployment. v. Evaluation and Benchmarking: Current evaluation protocols emphasize downstream task performance, which may not fully reflect predictive capability. Developing benchmarks that directly assess latent prediction quality is an important future direction. vi. Robustness Under Partial Observability: Although JEPA is designed for partial observability, systematic evaluation of robustness to missing, noisy, or ambiguous context remains limited and warrants further study. vii. Integration with Embodied and Interactive Systems: Applying JEPA to embodied agents and interactive environments introduces challenges related to online learning, exploration, and coupling predictive representations with control. viii. Connections to Cognitive and Theoretical Frameworks: Clarifying the relationship between JEPA and predictive coding, energy-based models, and free-energy minimization remains an open area for interdisciplinary research. 8 Conclusion In this work, we propose a taxonomic analysis of emerging self-supervised learning frameworks in terms of their objective functions and the signals used to train them. Namely, alignment-based and reconstruction-based approaches are considered from a single perspective, enabling a comparative evaluation of their representational capabilities, bias-inducing priors, and weaknesses in modeling the observed data distribution. In the proposed taxonomy, Predictive Representation Learning (PRL) approach is described as an independent class of methods, where optimization targets for latent representations are determined via predictive objective functions when only a limited part of the data is observable. In this setting, Joint Embedding Predictive Architectures (JEPA) are introduced as a set of canonical models, which implement predictive consistency of latent representations conditioned on contextual features and targets’ representations. We also discusses the differences in the structure of predictive objectives from that of alignment or reconstruction losses, especially with respect to information efficiency, invariant transformations, and implicit regularizers. References [1] A. van den Oord, Y. Li, and O. Vinyals, “Representation Learning with Contrastive Predictive Coding,” Proc. Advances in Neural Information Processing Systems (NeurIPS), 2018. [2] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A Simple Framework for Contrastive Learning of Visual Representations. In: ICML (2020) [3] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum Contrast for Unsupervised Visual Representation Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2020. [4] Grill, J.-B., et al.: Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning. In: NeurIPS (2020) [5] X. Chen and K. He, “Exploring Simple Siamese Representation Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2021. [6] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2022. [7] H. Bao, L. Dong, and F. Wei, “BEiT: BERT Pre-Training of Image Transformers,” Proc. Int. Conf. Learning Representations (ICLR), 2022. [8] LeCun, Y.: A Path Towards Autonomous Machine Intelligence. Meta AI White Paper (2022) [9] Assran, M., et al.: Self-Supervised Learning from Images with a Joint Embedding Predictive Architecture. In: CVPR (2023) [10] A. Bardes, J. Ponce, and Y. LeCun, “Video Joint-Embedding Predictive Architecture,” arXiv preprint arXiv:2401.x, 2024. [11] Chen, D., Shukor, M., Moutakanni, T., Chung, W., Yu, J., Kasarla, T., Bolourchi, A., LeCun, Y., Fung, P.: VL-JEPA: Joint Embedding Predictive Architecture for Vision-Language. arXiv preprint arXiv:2512.10942 (2025) [12] Skenderi, G., Li, H., Tang, J., Cristani, M.: Graph-JEPA: Graph-Level Representation Learning with Joint-Embedding Predictive Architectures. ICLR Workshop or OpenReview preprint (2024) [13] A. Recasens, J. Carreira, L. Beyer, F. Strub, L. Kirsch, N. Savinov, M. Tschannen, A. van den Oord, and O. J. Hénaff, “V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, 2025, p. 1–10. [14] Chen, D., Hu, J., Wei, X., Wu, E.: Denoising with a Joint-Embedding Predictive Architecture (D-JEPA). arXiv preprint arXiv:2410.03755 (2024) [15] He, X., Sakai, S., Yuan, K., Padoy, N., Hasegawa, T., Sigal, L.: DSeq-JEPA: Discriminative Sequential Joint-Embedding Predictive Architecture. arXiv preprint arXiv:2511.17354 (2025) [16] Ghaemi, H., Muller, E., Bakhtiari, S.: seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models. arXiv preprint arXiv:2505.03176 (2025) [17] Z. Fei, M. Fan, and J. Huang, "A-JEPA: Joint-Embedding Predictive Architecture Can Listen," arXiv preprint, 2023. [18] Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging Properties in Self-Supervised Vision Transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021) [19] Gui, J., Chen, T., Zhang, J., Zhang, Q., Liu, Y., Wang, S., Wang, X., Huang, F.: A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2022) [20] Jing, L., Tian, Y.: Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2020) [21] Baevski, A., Hsu, W.-N., Xu, Q., Babu, A., Gu, J., Auli, M.: data2vec: A General Framework for Self-Supervised Learning in Speech, Vision, and Language. In: Proceedings of the International Conference on Machine Learning (ICML) (2022)