Paper deep dive
TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs
Muhammad Haseeb Aslam, Alessandro Koerich, Marco Pedersoli, Ali Etemad, Eric Granger
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/20/2026, 4:27:55 AM
Summary
The paper introduces TTSD-FAR, a parameter-efficient Test-Time Adaptation (TTA) framework for Large Video-Language Models (LVLMs) to handle missing modalities in Emotion Recognition (ER). It combines Test-Time Self-Distillation (TTSD), where a frozen teacher guides an adaptive low-rank student via latent representation alignment, with Fisher-Anchored Restoration (FAR). FAR monitors Fisher information stability to detect distributional drift and restores the student to the teacher's anchor, preventing catastrophic forgetting and performance degradation during continual adaptation. Experiments on MELD, DFEW, and BAH datasets demonstrate superior performance over entropy-based and RAG baselines under 0%-50% missing modality conditions.
Entities (11)
Relation Signals (9)
TTSD-FAR → consistsof → Test-Time Self-Distillation
confidence 95% · We address both jointly with Test-Time Self-Distillation (TTSD)... Stability is built into this same loop through Fisher-Anchored Restoration (FAR)
TTSD-FAR → consistsof → Fisher-Anchored Restoration
confidence 95% · Stability is built into this same loop through Fisher-Anchored Restoration (FAR)
TTSD-FAR → evaluatedon → MELD
confidence 95% · Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities
TTSD-FAR → evaluatedon → DFEW
confidence 95% · Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities
TTSD-FAR → evaluatedon → BAH
confidence 95% · Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities
TTSD-FAR → targets → LVLMs
confidence 92% · TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs
TTSD-FAR → appliedto → Emotion Recognition
confidence 90% · multimodal emotion recognition (ER) in the wild
Video-LLaVA-7B → usedinanalysis → TTSD-FAR
confidence 88% · We perform this analysis on the MELD dataset... using the Video-LLaVA-7B model.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild. ER is inherently multimodal, requiring a joint understanding of facial expressions, vocalizations, language, biosignals, and gestures. However, real-world deployment remains challenging: modalities may be missing or noisy at test time. Partial observations can be viewed as a distribution shift relative to the complete-modality distribution. SOTA TTA methods based on entropy minimization or perplexity reduction do not transfer to autoregressive LVLMs, while retrieval augmented generation (RAG) degrades when the observed modality is weak. Because no ground-truth supervision exists to verify individual updates, adaptation across this stream risks accumulating drift and degrading once the model departs from a reliable solution. An effective solution must therefore adapt to arbitrary missing-modality patterns and remain effective during continual adaptation. We address both jointly with Test-Time Self-Distillation (TTSD), a parameter-efficient framework in which a frozen teacher, trained on complete modalities, guides an adaptive low-rank student via self-distillation, updating only a negligible number of parameters. Stability is built into this same loop through Fisher-Anchored Restoration (FAR), which monitors Fisher information stability to detect convergence versus drift and restores the student toward the teacher's anchor when distributional shifts are identified. Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation over long adaptation horizons, where baselines without restoration progressively degrade while TTSD-FAR remains consistent.
Tags
Links
- Source: https://arxiv.org/abs/2608.18386v1
- Canonical: https://arxiv.org/abs/2608.18386v1
Trouble viewing inline? Open PDF directly →
Full Text
81,034 characters extracted from source content.
Expand or collapse full text
TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs Muhammad Haseeb Aslam Alessandro Koerich Marco Pedersoli Ali Etemad Eric Granger Abstract Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in-the-wild. ER is inherently multimodal, requiring a joint understanding of facial expressions, vocalizations, language, biosignals, and gestures. However, real-world deployment remains challenging: modalities may be missing or noisy at test time. Partial observations can be viewed as a distribution shift relative to the complete-modality distribution. State-of-the-art TTA methods based on entropy minimization or perplexity reduction do not transfer to autoregressive LVLMs, while retrieval-augmented generation (RAG) degrades when the observed modality is weak. Because no ground-truth supervision exists to verify individual updates, adaptation across this stream risks accumulating drift and degrading once the model departs from a reliable solution. An effective solution must therefore adapt to arbitrary missing-modality patterns and remain effective during continual adaptation. We address both jointly with Test-Time Self-Distillation (TTSD), a parameter-efficient framework in which a frozen teacher, trained on complete modalities, guides an adaptive low-rank student via self-distillation, updating only a negligible number of parameters. Stability is built into this same loop through Fisher-Anchored Restoration (FAR), which monitors Fisher information stability to detect convergence versus drift and restores the student toward the teacher’s anchor when distributional shifts are identified. Our experiments11 1 Our code is included in the supplementary materials and will be made public. on MELD, DFEW, and BAH under 0%–50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation over long adaptation horizons, where baselines without restoration progressively degrade while TTSD-FAR remains consistent. 1 Introduction The rapid advancement of LVLMs (14; 17; 13) has revolutionized multimodal learning tasks, achieving unprecedented performance. These models leverage transformer architectures with billions of parameters, pre-trained on massive video-text corpora, to develop rich cross-modal representations (28). LVLMs are promising for video-based multimodal ER (MER), where systems typically capture facial, vocal, and language cues to address subtle (or compound) expressions and high inter-subject variability (10; 4). Recent work shows that LVLMs can capture fine-grained emotional and health states and contextual relationships that traditional discriminative models fail to detect (5). However, their deployment in real-world applications confronts a fundamental challenge that has received surprisingly little attention: missing modality at test time. In practical video-based MER scenarios, modality absence is ubiquitous and inevitable. Inputs may suffer from audio corruption in noisy environments, transcription services may fail due to poor audio quality or privacy constraints, and sensor malfunctions can eliminate entire modalities during data collection (34). Traditional approaches address this through training-time strategies, either by training separate models for each modality combination or by employing modality dropout during training (22). Yet both approaches are impractical for LVLMs, as retraining billion-parameter models for each missing modality scenario is computationally prohibitive (requiring 102010^20 FLOPs for 7B parameter models), while training-time modality dropout degrades full-modality performance and cannot adapt to deployment-specific missing patterns (16). 29 pose the missing modality as a domain shift problem and propose a method for TTA. The authors claim that despite advancements in the missing modality literature, a common drawback persists: all state-of-the-art approaches necessitate expensive retraining of the multimodal model. This poses a substantial challenge, particularly in applications with: (i) extensive training data; (i) where the retraining process is prohibitively expensive; and (i) when source data is not available, making the aforementioned approaches impractical. As shown with recent foundation models (e.g., GPT-style architectures and large multimodal systems (17; 14)), retraining large language models (LLMs) and LVLMs is prohibitively expensive, requiring specialized hardware and weeks of distributed optimization. It is therefore impractical to retrain or fully fine-tune these models for every downstream task or deployment setting due to catastrophic forgetting, computational cost, and source data availability. This motivates the development of lightweight TTA methods for efficient customization without modifying the full parameter set. We investigate the domain shift caused by missing modality inputs from 3 different viewpoints: (i) UMAP visualization of the model’s internal representations; (i) MMD computations between the full and missing modality inputs of the same sample; and (i) the degradation of performance. We perform this analysis on the MELD dataset with 50% text-missing inputs using the Video-LLaVA-7B model. Figure 1 displays the representation geometry of the complete and text-missing inputs. The two colors correspond to the two classes. The ∙ represents the full modality inputs and × represents the missing modality input for the same data samples. It reveals that the full and missing inputs are not only offset, but that missing inputs also blur the decision boundary. With full-modality inputs, the two classes form well-separated clusters, whereas under partial observation, this separation collapses entirely, with both classes overlapping into a single indistinguishable region Quantitative analysis reveals a substantial internal distribution shift when modalities are missing, with an MMD of 0.8350 and cosine similarity of 0.6342 between complete and missing inputs for the same input samples. This indicates that the absence of modalities can significantly disturb the learned attention manifold, even though the semantic content remains unchanged. Lastly, the decline in performance shown in Tables 1-3 indicates that LVLMs experience a severe performance drop (F1-score 0.6149 → 0.4785) in the 50% text-missing case. Entropy-based TTA methods tend to drop towards near-random performance after adaptation. This observation is in line with the findings of 9, which showed that entropy minimization objectives are ill-suited for the TTA of LLMs. A second, less-addressed challenge in online TTA for LLMs and LVLMs is perpetual adaptation. In practice, two failure modes emerge. First, once the student LoRA has converged, continued updates introduce gradient noise that perturbs the learned representation without improving performance (35; 24). Second, test streams in real deployments are non-stationary: speaker identities, recording conditions, and missing-modality rates shift over time. Perpetual adaptation without drift awareness risks overwriting consolidated knowledge, structurally analogous to catastrophic forgetting (12). Monitoring the raw distillation loss is an unreliable criterion for either stopping or restarting: the loss can plateau due to sample difficulty rather than genuine parameter convergence and can spike due to hard within-distribution samples rather than a true distributional shift. This motivates two design choices: adaptation needs a supervision signal that fits autoregressive LVLMs, motivating latent alignment over entropy. A stopping signal grounded in the parameters themselves rather than the loss, motivating the Fisher information diagonal, whose shift from its value at convergence exposes genuine drift and lets adaptation pause and reactivate accordingly. The contributions of this paper are summarized as follows: (1) TTSD: a parameter-efficient TTA method tailored to autoregressive LVLMs under missing modalities, built around conditional latent representation alignment as a supervision signal that overcomes the failure modes of entropy-based objectives and retrieval methods. (2) FAR: a two-state stopping and drift-detection and reactivation mechanism for TTSD. It monitors the Fisher information diagonal to detect student convergence (active → anchored) and uses a Fisher Mismatch Index to detect test-stream distributional shift (anchored → active). (3) An extensive set of experiments on three challenging video-based ER datasets, namely MELD (25), DFEW (11), and BAH (6), with multiple missing ratios and across different modalities shows the effectiveness of our proposed TTSD-FAR method. Figure 1: UMAP of the internal representations. The cross (×) represents the missing modality inputs, and the circle (∙ ) represents the full modality inputs. 2 Related Works Learning with Missing Modalities. Multimodal learning with missing modalities has been extensively studied, with early works such as ModDrop (21), SMIL (16), and masked modality projection (MMP) (22) introducing train-time strategies to enhance robustness. 18 propose a multi-modal teacher for masked modality learning, improving semantic segmentation under missing modalities. Some methods have explored modality-invariant architectures where single-branch models trained to be invariant to modality absence exhibit robustness during both training and testing (30; 37; 8). 7 use prompt learning in specialized cross-modal transformer architectures to regenerate the missing modality. More recently, 3 proposed SMCIR, which detects sample-level modality missingness via an unsupervised entropy/mutual-information/similarity score (DMFD), then reconstructs missing modalities through context-guided multi-scale cross-modal attention (CMCG). Despite this progress, most methods rely on training-time strategies, either reconstructing missing modalities, simulating modality absence during training, or making custom architectural changes. These train-time methods are not directly transferable to LVLMs, which are expensive to re-train and whose generative decoders lack clear self-supervised objectives that correlate with generation quality under modality absence. Multimodal systems often exhibit an imbalance in modality strength, with some modalities providing stronger task-relevant signals than others. When the stronger modality is absent, models experience a significant performance drop, highlighting the need for mechanisms that preserve knowledge from complete multimodal observations when operating even with weaker modalities. Test-Time Adaptation. TTA and test-time training have emerged to handle distribution shifts at inference time without access to target labels (36; 35). TENT proposes entropy minimization by adapting normalization statistics at test time (33). EATA (23) improves entropy-based TTA by selectively updating model parameters using only reliable low-entropy samples and regularizing updates with a Fisher constraint to prevent forgetting. Many recent works analyze batch normalization for robustness under shift (20). A related line of continual-TTA work decides when to reset via drift detection rather than fixed schedules or entropy alone: periodic resets to pretrained weights curb long-horizon collapse (26). Mishra (19) introduced RDumb++, a principled extension of RDumb that introduced two drift-detection mechanisms, i.e., entropy-based drift scoring and KL-divergence drift scoring, together with adaptive reset strategies. Alternating domain construction was proposed by (15). 32 instead optimize auxiliary self-supervised losses during inference to improve adaptation. These methods are tailored to models that produce a single softmax distribution over a small, fixed label set, where output entropy is a direct, low-dimensional measure of prediction confidence. Self-supervised objectives and associated stopping signals do not transfer to LVLMs. 9 introduced test-time learning for LLMs (TLM), adapting LLMs during inference using only unlabeled test data via input perplexity minimization rather than entropy minimization or supervised fine-tuning. Adapting entire layers or normalization statistics in billion-parameter models further raises stability and efficiency concerns, motivating lightweight, parameter-efficient adaptation instead. Unlike EATA’s per-step Fisher regularization, TTSD-FAR uses the Fisher diagonal as a control signal that suspends and resumes adaptation entirely. Retrieval-Augmented Generation. RAG methods improve LLM and LVLM’s performance during inference by incorporating external knowledge. Relevant information from a knowledge base is retrieved during inference (2; 38; 27). RAG methods prove effective for tasks that require domain knowledge, but they rely heavily on the quality of retrieved information and incur additional computational latency. Precomputed vector databases or memory banks are also required for retrieval during inference (1). State-of-the-art methods to address missing modalities either require retraining with each modality combination or depend on discriminative learning objectives and specialized architectures to hallucinate missing modalities. The main issue faced by LVLMs for TTA is that there is no strong supervision signal available for adaptation. Entropy objectives do not provide meaningful supervision, and RAG schemes fail if the observed modality is too weak to retrieve correct class neighbors. This paper fills such a gap by proposing a novel approach for feature-level adaptation using dual LoRA adapters through self-distillation and FAR for convergence monitoring that allows stopping and resuming adaptation dynamically. 3 Proposed Methodology Let =v×tX=X^v×X^t denote the multimodal input space consisting of video and text modalities, where vX^v represents video frames and tX^t represents textual utterances. Let Y denote the output label space (e.g., emotion classes). We assume access to a pre-trained LVLM fθ:→,θ∈ℝd,f^θ:X ,\ θ ^d, trained on large-scale multimodal corpora. The model consists of (i) modality-specific encoders Ev:v→ℝn×dv,Et:t→ℝm×dt,E^v:X^v ^n× d^v,\ E^t:X^t ^m× d^t, and (i) a LLMθLLM^θ backbone :ℝ(n+m)×d→:R^(n+m)× d , which performs cross-modal fusion and autoregressive reasoning. We denote the frozen pre-trained backbone parameters by θ0θ^0 and the teacher and student LoRA weights by Δθtea=BteaAtea θ^tea=B^teaA^tea and Δθstu=BstuAstu θ^stu=B^stuA^stu, respectively. We further denote the vectorized student LoRA parameters as ϕ=vec(Astu,Bstu)∈ℝpφ=vec(A^stu,B^stu) ^p, where p is the total number of student LoRA parameters. Figure 2: Illustration of the proposed TTSD-FAR method. The solid green arrow shows the flow of information in the case of complete modalities, and the dashed red arrow shows the flow of information in the missing modality case. The solid green arrow shows both modalities, and the single dashed green arrow denotes the input (x~ x) with masked input. In the missing case, the model bypasses adaptation and only infers with the student LoRA. In the complete case, the model uses complete modality features from the teacher LoRA and masked modality features from the student LoRA, and distillation loss ℒdistillL_distill is calculated to match the student representation with the teacher representation. 3.1 Problem Formulation Latent Inference under Partial Observation. Let x=(xv,xt)∼Pfullx=(x^v,x^t) P_full denote complete multimodal inputs drawn from the training joint distribution. At test time, modalities may be missing. We introduce binary modality masks mv,mt∈0,1,m^v,m^t∈\0,1\, and define the partially observed input as x~=(x~v,x~t)=(mv⋅xv,mt⋅xt). x=( x^v, x^t)=(m^v· x^v,\;m^t· x^t). (1) The teacher’s intermediate representation at layer l is htea(x)=LLMlθ0+Δθtea(xv,xt)∈ℝs×d.h^tea(x)=LLM_l^θ^0+ θ^tea(x^v,x^t) ^s× d. (2) The missing-modality setting can be viewed as a partial observation problem, where the learner must infer a representation consistent with the latent multimodal structure learned from complete data. We interpret htea(x)h^tea(x) as a latent variable lying on a multimodal representation manifold Z. Under squared error loss, the Bayes-optimal estimator of the complete-modality representation given the observed input x~ x is the conditional expectation h∗(x~)=[htea(x)∣x~].h^*( x)=E\! [h^tea(x) x ]. (3) TTSD as Conditional Latent Reconstruction. The student adapter parameterized by Δθstu θ^stu produces hstu(x~)=LLMlθ0+Δθstu(x~v,x~t)∈ℝs×d.h^stu( x)=LLM_l^θ^0+ θ^stu( x^v, x^t) ^s× d. (4) TTSD minimizes the feature-level distillation objective ℒdistill=x∼Pfull[‖hstu(x~)−htea(x)‖22],L_distill=E_x P_full [ \|h^stu( x)-h^tea(x) \|_2^2 ], (5) whose minimizer satisfies hstu∗(x~)=[htea(x)∣x~]h^stu^*( x)=E\! [h^tea(x) x ], so that under a squared-error objective, the student is encouraged to approximate the conditional expectation of the teacher representation given partial input. However, perpetual adaptation of ϕφ across an unbounded test stream introduces two failure modes: (i) once Δθstu θ^stu has converged, continued gradient updates add noise without improving the learned representation; and (i) in non-stationary streams, parameter drift can overwrite previously consolidated knowledge. We therefore augment TTSD with Fisher-Anchored Restoration (FAR), a two-state mechanism that determines when adaptation should proceed and when it should be suspended, based solely on the geometry of the Fisher importance landscape. 3.2 Architecture Overview Our TTSD-FAR framework operates on a single base video-language model fθ0f^θ^0 with two distinct LoRA adapters. Figure 2 illustrates the architecture. The architecture enables efficient mode switching: Teacher mode: y^tea=fθ0+Δθtea(xv,xt) y^tea=f^θ^0+ θ^tea(x^v,x^t) (6) Student mode: y^stu=fθ0+Δθstu(x~v,x~t) y^stu=f^θ^0+ θ^stu( x^v, x^t) (7) 3.3 TTA: Student LoRA At test time, when encountering a sample xx with complete modalities, we perform online adaptation of the student LoRA through feature-level self-distillation from the teacher. By directly matching intermediate representations, the student adapter is encouraged to capture the internal geometry of the teacher representation space rather than imitating its final decisions. This preserves the diversity and richness of the teacher features and mitigates mode collapse, which is particularly critical under missing-modality scenarios. Given a layer l∈ℒl of the LLM backbone, the hidden representations are: hltea h^tea_l =LLMlθ0+Δθtea(Ev(xv),Et(xt))∈ℝs×d =LLM_l^θ^0+ θ^tea(E^v(x^v),E^t(x^t)) ^s× d (8) hlstu h^stu_l =LLMlθ0+Δθstu(Ev(x~v),Et(x~t))∈ℝs×d =LLM_l^θ^0+ θ^stu(E^v( x^v),E^t( x^t)) ^s× d (9) where s is the sequence length (the sum of video tokens and text tokens) and d is the hidden dimension. We minimize the mean squared error between student and teacher representations as: ℒdistill=∑l∈ℒαl⋅‖hlstu−sg(hltea)‖22L_distill= _l _l·\|h^stu_l-sg(h^tea_l)\|_2^2 (10) where sg(⋅)sg(·) denotes the stop-gradient operation to prevent backpropagation into the frozen teacher, and αl _l are layer-specific weights. Teacher features are extracted once per sample. To ensure complete isolation, features are detached from the teacher network because no gradient calculation is required in the teacher LoRA. Teacher and student forward passes do not share computational graph nodes, preventing gradient contamination. Without isolation, shared normalization statistics or attention caches can cause interference. Why Self-Distillation for Missing Modalities? The teacher, trained on complete modalities, has learned rich cross-modal features that fuse information from both vision and text. When a modality is missing at test time, the student can be viewed as solving an inverse problem: Δθstu=argminΔθx~‖stu(x~)−tea(x)‖22. θ^stu= _ θE_ x \|h_stu( x)-h_tea(x) \|_2^2. (11) 3.4 Fisher-Anchored Restoration (FAR) Continuous adaptation of ϕφ is not harmless: once the student has found a solution to Eq. (11), additional gradient steps on the same distribution degrade performance through noise accumulation, while distribution shifts in the test stream can silently overwrite consolidated knowledge. Monitoring ℒdistillL_distill directly is unreliable as a stopping signal since the loss can plateau not because the student has converged, but because the current batch of samples is uniformly hard. The correct stopping signal is the stability of the parameter importance landscape. Specifically, whether the curvature of ℒdistillL_distill with respect to ϕφ has stabilised. Symmetrically, drift should not be detected by raw loss spikes, which can be triggered by hard samples in a stationary stream. Instead, drift is detected by monitoring whether incoming gradients engage parameter dimensions that were unimportant during consolidation. FAR formalizes this intuition through two complementary Fisher-based statistics and a two-state mechanism: active (the student LoRA is being updated) and anchored (the student LoRA is frozen). Online Fisher Diagonal. After each adaptation step, the diagonal of the empirical Fisher information matrix over ϕφ is updated via an exponential moving average: Ft=βF⋅Ft−1+(1−βF)⋅(∇ϕℒdistill)⊙2F_t= _F· F_t-1+(1- _F)· ( _φ\,L_distill ) 2 (12) where (⋅)⊙2(·) 2 denotes element-wise squaring, βF∈(0,1) _F∈(0,1) is the decay parameter controlling the weight placed on historical estimates. The element Ft,jF_t,j approximates the expected squared gradient with respect to the j-th parameter in ϕφ, providing a running estimate of each parameter’s contribution to the distillation objective. FtF_t is updated only during the active state. Consolidation Index and the active → anchored Transition. The consolidation index (CI) measures the normalized ℓ1 _1 rate of change of the Fisher diagonal between consecutive adaptation steps: CIt=‖Ft−Ft−1‖1‖Ft‖1+εCI_t= \|F_t-F_t-1\|_1\|F_t\|_1+ (13) where ε>0 >0 is a small numerical stability constant. When CItCI_t is large, the importance landscape is still evolving, i.e., the student has not yet settled on a stable curvature configuration. As the student converges, CIt→0CI_t→ 0, indicating that the Fisher diagonal has stopped shifting. The ℓ1 _1 norm is chosen because FtF_t is a non-negative vector, and ℓ1 _1 gives the total mass shifted in the importance landscape, which has a direct interpretation as the number of parameters whose importance is still changing. The transition from active → anchored is triggered when CItCI_t remains below a threshold τfc _fc for K consecutive adaptation steps CIt<τfcCI_t< _fc. The student LoRA parameters ϕφ are then frozen; no weight updates are applied while in the anchored state, and the current FtF_t is stored as an anchor, Fanchor=FtF_anchor=F_t . Fisher Mismatch Index and the anchored → active Transition. While anchored, for each incoming complete-modality sample, the gradient of ℒdistillL_distill is computed at the frozen anchor parameters without applying a weight update: gt=∇ϕℒdistill|ϕ=ϕanchorg_t= _φ\,L_distill |_φ= _anchor (14) The Fisher mismatch index (FMI) for sample t is then FMIt=1p∑j=1pgt,j2Fanchor,j+εFMI_t= 1p _j=1^p g_t,j^2F_anchor,j+ (15) where p=|ϕ|p=|φ| is the total number of student LoRA parameters. The ratio gt,j2/Fanchor,jg_t,j^2/F_anchor,j is large when the current sample induces a strong gradient in parameter j, but that parameter had low importance during consolidation. This points to large gradients in directions the consolidated student found unimportant. The transition from anchored back to active is triggered when FMIt>τfmiFMI_\,t> _fmi exceeds a threshold. 4 Results and Discussion Table 1: Comparison of methods under progressively missing textual and visual modalities. We report F1 on the MELD dataset as the proportion of unavailable input increases. N/A: Perplexity Gen. is not applicable to the vision-missing case. Text Missing Vision Missing Method Full 10% 20% 30% 40% 50% 10% 20% 30% 40% 50% No Adaptation 0.6149 0.5926 0.5706 0.5390 0.5190 0.4785 0.6122 0.6025 0.5934 0.5825 0.5775 TENT (ICLR’21) – 0.1650 0.1680 0.1765 0.1855 0.1655 0.1658 0.1725 0.1655 0.1838 0.1826 EATA (ICML’22) – 0.2854 0.2690 0.2770 0.2736 0.2482 0.2950 0.2750 0.2745 0.2530 0.2382 RAG (ICCV’25) – 0.5640 0.5248 0.5120 0.4845 0.4660 0.5925 0.5830 0.5634 0.5567 0.5534 Perplexity Gen. – 0.5680 0.5460 0.5255 0.4938 0.4722 N/A N/A N/A N/A N/A TTSD-FAR (Ours) – 0.6115 0.5890 0.5625 0.5454 0.5124 0.6130 0.6075 0.6055 0.5970 0.5915 4.1 Experimental Methodology Datasets. We evaluate our approach on three widely used multimodal and video-based emotion recognition benchmarks: Multimodal Emotion Lines Dataset (MELD) (25), Dynamic Facial Expression in-the-Wild (DFEW) (11), and Behavioral Ambivalence/Hesitancy (BAH) (6), each presenting distinct challenges in terms of modality diversity, temporal dynamics, and subtle affective cues. Details of the datasets are provided in the supplementary material. We exclusively validate on emotion recognition, which is one of the few multimodal applications where text transcripts serve as a crucial, non-redundant input modality. This makes it an ideal testbed for missing-modality robustness, since dropping the text modality induces genuine information loss. Evaluation Protocol. This paper makes no assumptions about the training phase. TTSD-FAR only requires a model that works with multiple modalities at test time. To the best of our knowledge, this is the first TTA method that effectively adapts to missing modalities in LVLMs. TTSD-FAR updates the model only when it encounters modality-complete samples but generates predictions for all samples, regardless of the modalities they contain. Missing modality scenarios are simulated by randomly masking modalities at inference time, with missing ratios ranging from 0% (complete) to 50%. All baselines use the same frozen LVLM backbone for fair comparison. Detailed implementation settings for each dataset are provided in the supplementary material. 4.2 Results under Missing Modalities Tables 1– 3 summarize performance across MELD, DFEW, and BAH under progressively increasing missing-modality ratios. Across all three datasets, entropy-based TTA methods (TENT and EATA) consistently underperform, confirming that confidence-based objectives provide unreliable supervision for autoregressive LVLMs under severe modality shift. This observation is consistent with the findings of 9, who showed that entropy minimization does not provide reliable optimization signals for autoregressive language models at test time. RAG and perplexity-based generation remain competitive when the observed modality is sufficiently informative, but their performance degrades as the dominant modality becomes unavailable. On MELD (Table 1), removing text causes substantially greater degradation than removing vision (0.6149→0.47850.6149→ 0.4785 versus 0.6149→0.57750.6149→ 0.5775), indicating that textual information provides the dominant supervision signal for emotion recognition in conversational settings. Consequently, retrieval-based methods struggle once textual information is unavailable, while entropy-based methods rapidly collapse toward near-random performance. TTSD-FAR instead leverages complete-modality teacher representations to supervise a student operating on synthetically masked inputs, allowing it to remain close to the full-modality baseline across both missing-text and missing-vision settings. A similar trend is observed on DFEW (Table 2), where removing visual information induces a substantial performance drop because facial dynamics constitute the dominant modality. Entropy-based adaptation again fails, while RAG deteriorates as weaker visual representations reduce retrieval quality. TTSD-FAR consistently achieves the strongest performance across all missing ratios, demonstrating that feature-level self-distillation provides a substantially more reliable adaptation signal than confidence- or retrieval-based approaches. Results for the weaker text-missing scenario are included in the supplementary material. Results on BAH (Table 3) further demonstrate the generality of the proposed framework. Although entropy-based methods perform relatively better because BAH is a binary classification task, they remain consistently inferior to TTSD-FAR. RAG and perplexity-based generation improve upon the zero-shot baseline at lower missing ratios but gradually deteriorate as textual information becomes increasingly unavailable. Additional implementation details and supplementary experiments are provided in the supplementary material. Table 2: Comparison of methods under progressively missing visual modality. We report F1 on the DFEW dataset as the proportion of unavailable visual input increases. Method Full 10% 20% 30% 40% 50% No Adaptation 0.5549 0.5300 0.5078 0.4984 0.4837 0.4434 TENT – 0.1548 0.1553 0.1665 0.1658 0.1640 EATA – 0.2534 0.2778 0.2735 0.2845 0.2885 RAG – 0.5340 0.5048 0.5020 0.4800 0.4468 TTSD-FAR (Ours) – 0.5515 0.5300 0.5195 0.5015 0.4775 Table 3: Comparison of methods under progressively missing textual modality conditions for BAH dataset. Method Full 10% 20% 30% 40% 50% Zero-Shot 0.7142 0.6835 0.6742 0.6526 0.6215 0.6023 TENT – 0.4923 0.4834 0.4710 0.4572 0.4550 EATA – 0.5630 0.5467 0.5360 0.5310 0.5250 RAG – 0.6835 0.6730 0.6535 0.6215 0.5950 Perplexity Gen. – 0.6870 0.6745 0.6535 0.6310 0.6030 TTSD-FAR (Ours) – 0.7050 0.6925 0.6695 0.6405 0.6250 4.3 Computational Complexity Analysis Experiments show that TTSD-FAR performs well when adapting to missing modality conditions at test time. However, this performance gain comes at a computational cost. We distinguish the two cases and explain the computational overhead for both cases. i) Full modality case: TTSD-FAR requires two forward passes, one forward pass for obtaining the teacher features and one for obtaining the student features. i) Missing modality case: For the missing modality case, there is no computational overhead. The model uses the adapted student LoRA for prediction. We update only 44M parameters out of the 7B parameters of the Video-LLaVA 7B model. This results in only ≈ 0.629% of the parameter updates. It is also important to note that the total wall time for the adaptation scenario depends on the missing ratio. For the 50% missing case, 50% of the samples will pose no additional overhead; the remaining 50% of the input samples require two forward passes and backpropagation to the student LoRA. Consequently, the total wall time of the complete MELD test set without adaptation takes ≈ 1.5 hours, and the time taken with adaptation is ≈ 2.1 hours. 4.4 Mechanistic Analysis of FAR To isolate the contribution of FAR, we compare TTSD with and without the restoration mechanism across MELD and BAH under increasing missing rates. Figure 3: Effectiveness of the FAR module on the distillation results across different missing rates. As shown in Figure 3, the unbounded variant (TTSD w/o FAR), which continues to adapt the student LoRA on every incoming sample, consistently underperforms the FAR-governed version at every missing rate. The consistent FAR gain across both the 7-class (MELD) and binary (BAH) settings indicates that this stopping/restarting behavior is not task-specific; rather, it reflects a general property of self-distillation under non-stationary missing-modality streams. The three heatmaps in Figure 4 show how Fisher parameter importance is distributed across LoRA modules under two adaptation regimes. The left panel shows FanchorF_anchor, the Fisher diagonal stored at the moment FAR declared convergence, where importance is sharply concentrated in the early layers of the query and up-projection modules, with the remainder of the landscape several orders of magnitude lower; this sparsity indicates that the student identified a precise, low-dimensional parameter subspace sufficient to solve the missing-modality distillation task. The middle panel shows the equivalent Fisher diagonal accumulated by continuous TTSD without any stopping criterion. The importance landscape is substantially more diffuse, and the gradient mass is spread uniformly across all layers and modules, reflecting that continued adaptation beyond convergence dilutes the signal that was meaningful at true convergence. The right panel shows the log-scale difference, which is almost entirely red, confirming that FAR anchors at a moment of higher and more concentrated importance across most of the parameter space. Figure 4: Fisher importance heatmaps (log10 scale): TTSD-FAR (left), TTSD (middle), and difference map (right), confirming systematically higher importance under FAR. 4.5 Discussion Our experiments reveal two core challenges for adapting autoregressive LVLMs under partial modality observation: recovering missing semantic content and sustaining effective adaptation over long deployment. Retrieval-based adaptation assumes the observed modality provides a query informative enough to retrieve semantically relevant references. This assumption breaks down when the missing modality carries the dominant information, in which case the retrieved examples supply inconsistent or misleading supervision. Entropy minimization fails in a complementary way. It implicitly assumes confident predictions are correct predictions, but under substantial modality shift, autoregressive LVLMs often generate overconfident, semantically incorrect outputs. Minimizing entropy in this regime reinforces these errors, producing optimization drift rather than genuine adaptation, consistent with the findings of 9. TTSD-FAR addresses both failure modes by treating missing-modality adaptation as a conditional representation alignment problem rather than a confidence or retrieval optimization problem. The student is guided toward the frozen reference model’s complete-modality latent representation through feature-level supervision, recovering semantic information that the observed modality alone cannot provide and yielding substantially more robust adaptation under severe degradation. A separate issue is that continued optimization past convergence causes parameter diffusion, eroding useful representations. FAR’s adaptive consolidation addresses this by tracking optimization stationarity via Fisher geometry to detect convergence, then reactivating adaptation only when deviations from the consolidated geometry signal genuine distributional drift. This preserves learned representations while retaining the capacity to adapt when the input distribution actually shifts. 5 Conclusion TTSD-FAR is a parameter-efficient framework for adapting large video-language models to missing modalities during inference. Missing modalities induce substantial shifts in representation geometry within autoregressive LVLMs, a regime in which entropy-based test-time adaptation and retrieval-augmented generation both fail to provide reliable supervision. TTSD-FAR instead couples a frozen teacher, trained on complete modalities, with an adaptive student adapter that operates on masked inputs, aligning student and teacher representations via feature-level self-distillation without modifying the base model’s parameters. Fisher-Anchored Restoration governs this process through a cycle of consolidation and reactivation: adaptation is suspended once the student’s parameter importance landscape stabilizes, and resumed only when the Fisher geometry signals genuine distributional drift, preventing the parameter diffusion that unbounded adaptation otherwise induces. Extensive experiments across three emotion recognition benchmarks show that TTSD-FAR consistently improves performance under severe missing-modality conditions while preserving accuracy competitive with the complete-modality setting. Supplementary materials. Includes proof sketches, dataset description and implementation details, algorithm for the proposed TTSD-FAR method, and extended ablation studies. Appendix A Appendix Algorithm for TTSD-FAR.1 Theoretical Properties of FAR.B.1 Prop. 1: Fisher Stability as a Stationarity Signal.B.1 Proposition 2: No-Regret Reactivation.B.1 Proposition 3: FMI Hard-Sample Robustness.B.1 Additional Results.C Distilling from Base Teacher (no teacher LoRA).C Weaker Modality Missing Results.C Additional Ablations.D LoRA Rank Sensitivity.D Effectivity of TTSD on Small Video Language Model.D FAR Hyperparameters Sensitivity.D.1 Effect of the FMI Threshold.7 FAR State Dynamics over the Test Stream.D.1 Loss-Based Stopping vs. FAR.8 Datasets and Implementation Details.E Datasets.E Implementation Details.E FAR Parameters.10 Appendix B Algorithm for TTSD-FAR Algorithm 1 summarizes the adaptation loop for the LVLM using the proposed TTSD-FAR methodology. Algorithm 1 TTSD-FAR 0: Δθtea θ^tea, ϕ0 _0, η, βF _F, τfc _fc, K, τfmi _fmi, M, ε 1: ϕ←ϕ0φ← _0; F,Fprev←pF,F_prev 0_p; state←Activestate← Active; c←0c← 0; ℬ←∅B← 2: for each incoming sample x~ x do 3: if state=Activestate= Active then 4: hltea←sg(LLMlθ0+Δθtea(x))h^tea_l (LLM_l^θ^0+ θ^tea(x)); hlstu←LLMlθ0+Δθstu(x~)h^stu_l _l^θ^0+ θ^stu( x) 5: g←∇ϕ∑lαl∥hstul−hteal∥22g← _φ _l _l\|h^stu_l-h^tea_l\|_2^2; ϕ←ϕ−ηgφ←φ-η g 6: F←βFFprev+(1−βF)g⊙2F← _FF_prev+(1- _F)g 2 7: CI←‖F−Fprev‖1/(‖F‖1+ε)CI←\|F-F_prev\|_1/(\|F\|_1+ ); Fprev←F_prev← F 8: c←c+1c← c+1 if CI<τfcCI< _fc, else c←0c← 0 9: if c≥Kc≥ K then 10: ϕanchor,Fanchor←ϕ,F _anchor,F_anchor←φ,F; state←Anchoredstate← Anchored; ℬ←∅B← 11: end if 12: else 13: Infer with frozen ϕanchor _anchor no update 14: if x~ x is a complete-modality sample then 15: Mask x~ x synthetically →x~′→ x ; g←∇ϕℒdistill(x~′)|ϕanchorg← _φL_distill( x )|_ _anchor 16: FMI←1p∑jgj2/(Fanchor,j+ε)FMI← 1p _jg_j^2/(F_anchor,j+ ); push to ℬB (keep last M) 17: if FMI>τfmiFMI> _fmi then 18: ϕ,Fprev←ϕanchor,Fanchorφ,F_prev← _anchor,F_anchor; c←0c← 0; state←Activestate← Active; ℬ←∅B← 19: end if 20: end if 21: end if 22: end for B.1 Theoretical Properties of FAR This section provides an intuitive theoretical justification for the proposed Fisher-Anchored Restoration (FAR) mechanism. Rather than proving global optimality, our goal is to explain why the proposed two-state adaptation strategy avoids unnecessary parameter drift while remaining capable of re-adapting under distribution shift. Assumptions The following assumptions are standard in stochastic optimization and test-time adaptation. Assumption 1 (Local Smoothness). The feature-level distillation objective Ldistill(ϕ)L_distill(φ) is continuously differentiable with L-Lipschitz gradients, ‖∇L(ϕ1)−∇L(ϕ2)‖≤L‖ϕ1−ϕ2‖.\|∇ L( _1)-∇ L( _2)\|≤ L\| _1- _2\|. (16) Assumption 2 (Bounded Gradient Variance). The stochastic gradients satisfy [gt]=∇L(ϕt),E[g_t]=∇ L( _t), (17) and [‖gt−∇L(ϕt)‖2]≤σ2.E [\|g_t-∇ L( _t)\|^2 ]≤σ^2. (18) Assumption 3 (Piecewise Stationary Test Stream). The test distribution remains stationary between distribution shifts. This assumption is common in continual test-time adaptation, where environmental conditions remain approximately constant for finite intervals before changing. Proposition 1: Fisher Stability as a Stationarity Signal. Let FtF_t denote the online Fisher diagonal defined in Eq. (12), Ft=βFFt−1+(1−βF)gt⊙2,F_t= _FF_t-1+(1- _F)\,g_t 2, (19) where gtg_t is the stochastic gradient of ℒdistillL_distill at step t, taken coordinate-wise so that gt⊙2g_t 2 denotes element-wise squaring. Under Assumption 2 (bounded gradient variance) and Assumption 3 (piecewise stationarity), suppose CIt=‖Ft−Ft−1‖1‖Ft‖1+ε<τfcCI_t= \|F_t-F_t-1\|_1\|F_t\|_1+ < _fc (20) holds for K consecutive adaptation steps. Then the second moment of the stochastic gradient, [gt⊙2]E[g_t 2], has stabilized across those steps, in the sense of Eq. (9) below. This need not imply that ∇ℒ(ϕt)≈0 ( _t)≈ 0. Proof Sketch: From the recursion defining FtF_t, Ft−Ft−1=(1−βF)(gt⊙2−Ft−1).F_t-F_t-1=(1- _F) (g_t 2-F_t-1 ). (21) Substituting into the definition of CItCI_t, CIt=(1−βF)‖gt⊙2−Ft−1‖1‖Ft‖1+ε.CI_t= (1- _F)\,\|g_t 2-F_t-1\|_1\|F_t\|_1+ . (22) Since βF∈(0,1) _F∈(0,1) is fixed, CIt<τfcCI_t< _fc is equivalent, up to the constant (1−βF)(1- _F), to ‖gt⊙2−Ft−1‖1<τfc1−βF(‖Ft‖1+ε),\|g_t 2-F_t-1\|_1< _fc1- _F (\|F_t\|_1+ ), (23) i.e., the current squared gradient agrees with the running Fisher estimate in aggregate ℓ1 _1 mass. Taking expectations under Assumption 2, which gives [gt]=∇ℒ(ϕt)E[g_t]= ( _t) and ‖gt−∇ℒ(ϕt)‖2≤σ2E\|g_t- ( _t)\|^2≤σ^2, the per-coordinate second moment satisfies [(gt,j)2]=(∇jℒ(ϕt))2+Var(gt,j).E[(g_t,j)^2]=( _jL( _t))^2+Var(g_t,j). (24) Condition (5) small for K consecutive steps therefore forces [(gt,j)2]≈[(gt−1,j)2]≈⋯≈[(gt−K,j)2]∀j,E[(g_t,j)^2] [(g_t-1,j)^2]≈·s [(g_t-K,j)^2] ∀ j, (25) which holds under either of two disjoint cases: (a) stationary point: ∇ℒ(ϕt)≈0, ( _t)≈ 0, (26) Var(gt,j)≈σj2, (g_t,j)≈ _j^2, (27) (b) constant residual: ∇ℒ(ϕt)≈cj≠0, ( _t)≈ c_j≠ 0, (28) Var(gt,j)≈σj2−cj2(unchanged). (g_t,j)≈ _j^2-c_j^2 (unchanged). (29) Equation (7) alone cannot distinguish case (a) from case (b): both produce a stable FtF_t. What Eq. (7) does establish is that the distribution generating gtg_t, and in particular its second moment, has stopped changing over the K-step window. Remark (Informal): Consolidation Reduces Unnecessary Drift. Given Eq. (7), consider T additional steps taken while CItCI_t remains below τfc _fc. Writing ϕt+T−ϕt=−η∑i=1Tgt+i _t+T- _t=-η _i=1^Tg_t+i and applying Assumption 2, [‖ϕt+T−ϕt‖2]=η2∑i=1T[‖gt+i‖2]=O(Tη2σ2),E [\| _t+T- _t\|^2 ]=η^2 _i=1^TE [\|g_t+i\|^2 ]=O(Tη^2σ^2), (30) matching Eq. (5), regardless of whether case (a) or (b) above holds. In case (a), this accumulation is pure noise with no accompanying decrease in ℒdistillL_distill; in case (b), useful progress −ηTc-η Tc is being made, but at rate bounded by c, the residual gradient, which Eq. (7) shows is itself no longer detectably shrinking. In either case, continued updates trade a fixed, boundable amount of progress against unbounded, monotonically growing variance as T increases. FAR operationalizes the resulting tradeoff by freezing ϕφ once Eq. (2) holds for K consecutive steps, accepting the small residual-progress cost of case (b) in exchange for eliminating the variance growth in Eq. (9) until Eq. (15) signals that reactivation is warranted. Proposition 2: No-Regret Reactivation Assume the test distribution changes from PAP_A to PBP_B, and the induced gradient statistics satisfy FMIt>τfmi.FMI_t> _fmi. (31) Then FAR exits the anchored state and resumes gradient updates. Following reactivation, the optimization dynamics are identical to continuous TTSD adaptation up to a finite detection delay. Proof Sketch: During the anchored state, the student parameters remain fixed, ϕt=ϕanchor. _t= _anchor. (32) For each incoming complete-modality sample, FAR evaluates FMIt=1p∑j=1pgt,j2Fanchor,j+ϵ.FMI_t= 1p _j=1^p g_t,j^2F_anchor,j+ε. (33) When the test distribution remains stationary, the incoming gradients remain aligned with the previously consolidated Fisher geometry, resulting in relatively small mismatch values that remain below the reactivation threshold. Suppose the underlying distribution changes. The gradients then begin appearing along parameter directions that previously exhibited low Fisher importance. Consequently, gt,j2≫Fanchor,j,g_t,j^2 F_anchor,j, (34) for a subset of parameters, causing the Fisher mismatch index to exceed τfmi _fmi. Once reactivated, FAR performs the same parameter update as standard TTSD, ϕt+1=ϕt−ηgt. _t+1= _t-η g_t. (35) Therefore, after detecting a distribution shift, FAR follows the same optimization trajectory as perpetual adaptation. The only difference is that FAR avoids unnecessary updates during stationary periods, thereby reducing parameter drift without sacrificing future adaptability. Proposition 3: FMI Hard-Sample Robustness. Suppose the student LoRA is in the anchored state with stored baseline Fanchor≈F∗(P):=P[g⊙g]F_anchor≈ F^*(P):=E_P[g g]. Consider two scenarios that produce equally elevated distillation loss: P′[ℒdistill]=C⋅P[ℒdistill],E_P [L_distill]=C·E_P[L_distill], (36) where scenario (a) is a hard within-distribution sample xH∼Px_H P with ‖gH‖2=C⋅P[‖g‖2]\|g_H\|^2=C·E_P[\|g\|^2], and scenario (b) is a shifted distribution P′P with the same expected loss elevation. Under Assumptions 2–3, the FMI satisfies P[FMI(xH)]≈1,E_P[FMI(x_H)]≈ 1, (37) regardless of C, while P′[FMIt]≠1,E_P [FMI_t]≠ 1, (38) whenever F∗(P′)≠F∗(P)F^*(P )≠ F^*(P). Consequently, the FMI threshold τfmi _fmi is triggered selectively by genuine distributional shift and not by gradient magnitude alone. Proof Sketch: By definition of the diagonal Fisher, F∗(P)j=P[gj2]F^*(P)_j=E_P[g_j^2]. For a hard sample xH∼Px_H P, the gradient gHg_H is drawn from the same distribution P used to construct FanchorF_anchor, so P[gH,j2]=F∗(P)j=Fanchor,jE_P[g_H,j^2]=F^*(P)_j=F_anchor,j coordinate-wise — the magnitude C scales the gradient but is distributed across the same high-importance coordinates that defined FanchorF_anchor in the first place, leaving the ratio unchanged: P[FMI(xH)]=1p∑j=1pFanchor,jFanchor,j+ε≈1.E_P[FMI(x_H)]= 1p _j=1^p F_anchor,jF_anchor,j+ ≈ 1. (39) Under a shifted distribution P′P , gradients instead concentrate in coordinates j∈Sj∈ S where Fanchor,jF_anchor,j was small, so gt,j2≫Fanchor,jg_t,j^2 F_anchor,j for j∈Sj∈ S, giving P′[FMIt]≥ 1+|S|p⋅δflow+ε,E_P [FMI_t]\;≥\;1+ |S|p· δf_low+ , (40) where δ>0δ>0 is the shift magnitude in S and flow=maxj∈SFanchor,jf_low= _j∈ SF_anchor,j. As flow→0f_low→ 0, this bound diverges, so any finite τfmi _fmi eventually detects the shift. Sample difficulty within P (large C) therefore leaves FMI near 1, while structural change in where gradient mass concentrates (a genuine distributional shift) drives it above threshold. Appendix C Additional Results Distilling from Base Teacher (no teacher LoRA). When working with LLMs and LVLMs, it is often realistic to assume that no retraining, either full or parameter-efficient, is performed on the base model. To demonstrate the effectiveness of TTSD under this constraint, we conduct an experiment where distillation is performed directly from the base model weights without any teacher LoRA as shown in Fig. 5. In this setting, teacher representations are extracted from the frozen base model, resulting in the same optimization problem as Eq. 11 Table 4 shows the results for the case where the teacher embeddings are obtained directly from the base model weights. Since the same distillation mechanism is applied, the setup is effectively identical except that the teacher representations are extracted from the frozen base model rather than from a Teacher LoRA trained with the objective in Eq. 11. The results therefore follow a similar trend to the case where the teacher embeddings are obtained using the Teacher LoRA objective. Figure 5: TTSD-FAR without teacher LoRA. Base weights provide teacher representations. Table 4: Comparison of methods under progressively missing text conditions. Distilling from the Base model, i.e., no teacher LoRA. We report WF1 on the BAH dataset as the proportion of unavailable textual input increases. Method Full 10% 20% 30% 40% 50% No Adaptation 0.6341 0.5905 0.5798 0.5710 0.5662 0.5614 TENT – 0.4910 0.4840 0.4733 0.4501 0.4453 EATA – 0.5465 0.5367 0.5290 0.5245 0.5110 RAG – 0.5770 0.5720 0.5540 0.5448 0.5250 Perplexity Gen. – 0.5790 0.5695 0.5648 0.5488 0.5430 TTSD-FAR (Ours) – 0.6185 0.6120 0.5955 0.5910 0.5870 Weaker-Modality Missing Results DFEW - Text Missing at Test-Time - Weaker Modality Missing Table 5 reports the performance of different methods on the DFEW under progressively increasing levels of missing textual modality. As the proportion of missing text increases, the performance of the model without adaptation gradually degrades. Table 5: Comparison of methods under progressively missing textual modality conditions. F1 on the DFEW dataset as the proportion of missing textual input increases. Method Full 10% 20% 30% 40% 50% No Adaptation 0.5549 0.5530 0.5490 0.5365 0.5260 0.5235 TENT – 0.1655 0.1725 0.1790 0.1935 0.1833 EATA – 0.2635 0.2674 0.2835 0.2855 0.2943 RAG – 0.5530 0.5425 0.5265 0.5265 0.5210 Perplexity Gen. – 0.5535 0.5460 0.5350 0.5300 0.5210 TTSD-FAR (Ours) – 0.5540 0.5510 0.5444 0.5376 0.5278 However, the performance drop due to the missing modality is insignificant compared to the vision missing in the DFEW dataset. This is primarily due to the dataset structure. The DFEW dataset comprises of 16372 short clips, of which 6196 are non-English transcriptions and 4960 clip have no speaker utterance at all. This shows that even the full modality performance (0.5549) is already less reliant on the textual modality. Classical test-time adaptation methods such as TENT (33) and EATA (23) exhibit severe performance drops, suggesting that entropy-minimization-based adaptation strategies are not well suited for multimodal missing-modality scenarios. In contrast, retrieval-based augmentation (RAG) and generation-based approaches maintain performance because the observed modality is stronger in this case. Our proposed TTSD method consistently achieves the best performance under moderate to severe missing-text conditions (20%–50%), demonstrating improved robustness and stability when textual inputs are partially unavailable. These results indicate that TTSD effectively mitigates the negative impact of missing textual modality while preserving strong performance as modality degradation increases. BAH - Vision Missing at Test-Time- Weaker Modality Missing Table 6 reports results on BAH under vision missing conditions. The vision drop for the BAH dataset results in negligible performance drops even at severe missing percentages (50%). The 10% has no effect on the performance. TTSD is able to improve over the existing TTA methods in both weaker and stronger missing modality conditions. Table 6: Comparison of methods under progressively missing visual modality. We report Avg-F1 on the BAH dataset as the proportion of unavailable visual input increases. Method Full 10% 20% 30% 40% 50% No Adaptation 0.7142 0.7142 0.7095 0.7046 0.6950 0.6923 TENT – 0.4834 0.4846 0.4790 0.4645 0.4754 EATA – 0.5655 0.5428 0.5165 0.5095 0.5052 RAG – 0.7100 0.7055 0.7059 0.7110 0.6975 TTSD-FAR (Ours) – 0.7142 0.7125 0.7120 0.7125 0.7125 As shown in Table 9, the textual input is critical for Ambivalence detection. This explains why the text-missing case for the BAH dataset results in a much higher performance drop than vision-missing, as shown in Table 6. Another important observation is that in the vision missing case (Table 6) the RAG-based method is also able to improve, unlike the text missing case. This is because in the vision missing case, the observed modality (text) is stronger and is able to effectively retrieve relevant samples. Whereas, in the text missing case, the observed modality (vision) was not discriminant enough for effective retrieval. Appendix D Additional Ablations: LoRA Rank Sensitivity. We investigate the effect of LoRA rank on TTSD’s robustness to modality-incomplete inputs. As expected, performance degrades monotonically with increasing missing rates across all ranks; yet, the degradation pattern reveals a clear sensitivity to rank capacity. Lower-rank adaptation (r=4) suffers disproportionately under higher missing rates, exhibiting steeper and less stable declines compared to higher rank, suggesting that insufficient parameter capacity limits the model’s ability to compensate for missing modalities. Ranks r=8 and r=16 remain consistently close throughout, with only marginal gains from doubling the rank, indicating diminishing returns beyond r=8. These results suggest that r=8 strikes the best balance between adaptability, robustness, and computational complexity. The ablation shows that LoRA rank is a meaningful factor in missing-modality robustness. Figure 6: Impact of LoRA rank on distillation performance under varying missing-modality rates across three datasets. Effectivity of TTSD on Small Video Language Model (SVLM): To further show the effectiveness of the proposed TTSD method. We perform the experiment on a smaller video language model, MobileVideoGPT-0.5B (31). Table 7: F1 Score on the BAH dataset under progressively missing textual input for two models: Video-LLaVA-7B and MobileVideoGPT-0.5B. Model Method Complete 10% 20% 30% 40% 50% Video-LLaVA-7B No Adaptation 0.6341 0.5905 0.5798 0.5710 0.5662 0.5614 TTSD-FAR (Ours) 0.6185 0.6120 0.5955 0.5910 0.5870 MobileVideoGPT-0.5B No Adaptation 0.4889 0.4521 0.4413 0.4310 0.4218 0.4125 TTSD-FAR (Ours) 0.4685 0.4595 0.4510 0.4428 0.4385 For Video-LLaVA-7B, the baseline model without adaptation shows a gradual degradation in performance as the proportion of missing textual input increases. Applying TTSD consistently improves the results across all missing-modality levels, demonstrating that the proposed method effectively recovers useful representations when textual information becomes partially unavailable. A similar trend is observed for the smaller MobileVideoGPT-0.5B model. Although its overall performance is lower due to its reduced model capacity, TTSD still provides consistent improvements over the non-adapted baseline under all missing-modality conditions. These results indicate that TTSD is not restricted to a specific LVLM architecture and can improve robustness for both large and lightweight video-language models. D.1 FAR Hyperparameters Sensitivity Effect of the FMI Threshold: Figure 7: Effect of τfmi _fmi on reactivation events. The three panels show the FMI trace at τfmi _fmi 1.0 (top), 2.0 (middle) and, 3.0 (bottom). Figure 7 illustrates the sensitivity of FAR’s reactivation criterion to the choice of τfmi _fmi. Each panel plots the FMI trace computed during the ANCHORED state, with the horizontal dashed line indicating τfmi _fmi and vertical dashed lines marking each detected reactivation event. At τfmi=1.0 _fmi=1.0 (top), the threshold is low enough that it is crossed both by the largest, genuine spikes and by several smaller fluctuations in the 1.0–1.5 range, yielding nine separate reactivations. This demonstrates the failure mode of an overly permissive threshold, where the mechanism reacts to noise as frequently as to true distributional shift. At τfmi=2.0 _fmi=2.0 (middle), corresponding to the value used in our reported experiments, the threshold clears the minor fluctuations, while still detecting the two clearly dominant spikes, resulting in exactly two reactivations. This setting represents the balance adopted for the main results. At τfmi=3.0 _fmi=3.0 (bottom), the threshold is raised further such that even a second sizeable spike is no longer detected, leaving only the single largest event to trigger reactivation. This illustrates the opposite failure mode: an excessively conservative threshold trades false positives for false negatives, risking missed detections of genuine distributional shift. FAR State Dynamics over the Test Stream Figure 8 shows the ACTIVE/ANCHORED state trajectory of FAR across the test stream under three hyperparameter regimes. The middle panel corresponds to the values reported in the paper and supplementary (βF=0.99 _F=0.99, τfc=0.02 _fc=0.02, K=5K=5, τfmi=2.0 _fmi=2.0). The student remains ACTIVE for roughly the first 40% of the stream before the Consolidation Index stays below τfc _fc for K consecutive samples, triggering consolidation into ANCHORED. It remains frozen until a distributional shift shortly after, around the mid-point of the stream, raises the FMI above τfmi _fmi, triggering reactivation. A second cycle follows later in the stream, consolidating and then reactivating again, after which the student remains active for the remainder until the very end, where it transitions to the Anchored state. This produces two well-separated reactivation cycles. Figure 8: Timeline of FAR states (active-anchored) across the test stream. The top panel shows a single consolidation slightly past the mid-point of the stream, with no reactivation for the remainder. This is consistent with a substantially higher τfmi=5 _fmi=5, under which later shifts in the stream no longer raise the FMI enough to cross the threshold, leaving the student permanently frozen once anchored. The bottom panel uses a relaxed consolidation requirement with a lower reactivation threshold of τfmi=0.5 _fmi=0.5. Consolidation fires as soon as CI drops below τfc _fc, and the low τfmi _fmi causes ordinary gradient noise while anchored to be sufficient to trigger reactivation. These settings produce rapid, near-continuous alternation between ACTIVE and ANCHORED throughout most of the stream, resulting in the student being in the ANCHORED state for roughly 80% of the test stream. These two extremes motivate the reported configuration of K=5K=5 and τfmi=2.0 _fmi=2.0, which yields the stable, interpretable cycles shown in the middle panel rather than premature permanent freezing or unstable oscillation. D.2 Loss-Based Stopping vs. FAR Table 8 compares TTSD-FAR against two ablated variants on the BAH dataset (with settings similar to Tab. 4) under progressively missing textual input. i) plain TTSD, which adapts on every complete-modality sample without any stopping criterion, and i) TTSD-LS, which instead halts adaptation once the raw distillation loss plateaus with a patience of 5, after which the student remains frozen for the rest of the stream. TTSD-LS underperforms not only TTSD-FAR but also plain, unbounded TTSD at every missing ratio. This confirms the motivation given in Section 3.4: the raw distillation loss is an unreliable stopping signal, since it can plateau due to sample difficulty rather than genuine parameter convergence. Once frozen, TTSD-LS has no mechanism to detect that the test stream has since shifted and never resumes adaptation. It accumulates the disadvantages of stopping without any of the drift-awareness that FAR provides. In contrast, TTSD-FAR consistently outperforms both plain TTSD and TTSD-LS across all missing ratios, indicating that gating adaptation on the Fisher information geometry rather than on the loss itself is necessary to obtain the benefits of stopping without sacrificing the ability to resume adaptation when genuine distributional shift occurs. Table 8: Ablation comparing TTSD-FAR against plain TTSD (no stopping criterion) and TTSD-LS (loss-plateau stopping criterion) under progressively missing text conditions. We report WF1 on the BAH dataset as the proportion of unavailable textual input increases. Method Full 10% 20% 30% 40% 50% No-Adaptation 0.6341 0.5905 0.5798 0.5710 0.5662 0.5614 TTSD – 0.6065 0.6039 0.5887 0.5816 0.5728 TTSD-LS – 0.5910 0.5820 0.5730 0.5645 0.5560 TTSD-FAR (Ours) – 0.6185 0.6120 0.5955 0.5910 0.5870 Appendix E Datasets and Implementation Details Datasets MELD (Multimodal EmotionLines Dataset): MELD is a conversational emotion recognition dataset that extends the original EmotionLines corpus by incorporating synchronized audio, visual, and textual modalities (25). It contains approximately 1,433 multi-party dialogues and over 13,000 utterances extracted from the Friends TV series, each labeled with one of seven discrete emotions (Anger, Disgust, Sadness, Joy, Neutral, Surprise, Fear) and sentiment labels. The multimodal nature and conversational context make MELD a challenging benchmark for models that must integrate semantic, acoustic, and facial cues to infer emotion. DFEW (Dynamic Facial Expression in the Wild): DFEW is a large-scale dynamic facial expression dataset collected from more than 1,500 movies and comprising over 16,000 short video clips depicting naturalistic expressions captured under unconstrained conditions (11). Unlike posed or laboratory-controlled datasets, DFEW includes significant variations in pose, illumination, occlusion, and actor demographics, with each clip annotated for one of seven basic emotions. This dataset serves as a benchmark for dynamic facial expression recognition (DFER) in the wild, emphasizing robustness to real-world noise and temporal dynamics. BAH (Behavioral Ambivalence/Hesitancy Dataset): The BAH dataset is a recently introduced benchmark for recognizing ambivalence and hesitancy (A/H) in video recordings designed for behaviour change studies (6). It contains 1,118 videos recorded from 224 participants across diverse demographics, captured while participants responded to stimuli intended to elicit ambivalence or hesitancy. Frame- and video-level annotations highlight segments containing A/H cues, and the dataset also provides aligned face crops, audio transcripts with timestamps, and participant metadata. BAH’s focus on subtle and conflicting emotional states presents a distinct challenge compared to conventional discrete emotion recognition tasks, highlighting the need for models that can capture fine-grained behavioral cues across modalities and are able to perform well even under missing conditions. The dataset comes with a predefined train-test split. We present the results on the official test set. The performance measure reported is the F1 score. Table 9: Summary of prompt variations for zero-shot inference with corresponding video-level F1 scores. Prompt Type Prompt Avg F1 Simple Classify the emotion in the video as either Non-Ambivalent or Ambivalent. Respond with only one word. 0.2827 Definition 1 Definition: Ambivalence is the state of having contradictory or conflicting feelings or attitudes towards something or someone simultaneously. Classify the emotion in the video as either Non-Ambivalent or Ambivalent. Respond with only one word. 0.3326 Definition 2 Definition: Ambivalence and hesitancy is understood as the simultaneous experience of desires for change and against change. Classify the emotion in the video as either Non-Ambivalent or Ambivalent. Respond with only one word. 0.3772 Transcript + Def 1 Video transcript: transcript. Definition: Ambivalence is the state of having contradictory or conflicting feelings or attitudes towards something or someone simultaneously. Classify the emotion in the video as either Non-Ambivalent or Ambivalent. Respond with only one word. 0.6341 Transcript + Def 2 Video transcript: transcript. Definition: Ambivalence and hesitancy are understood as the simultaneous experience of desires for change and against change. Classify the emotion in the video as either Non-Ambivalent or Ambivalent. Respond with only one word. 0.3945 Implementation Details Missing modality Simulation: This section provides the details of the missing modality simulation for both vision and text missing conditions. The missing modality problem is a vast paradigm, and it is important to set the scope of the experimentation to effectively validate the method. The datasets used for the validation of the TTSD-FAR method all contain utterance-level annotations. For the text-missing scenario, the speaker utterance in the prompt is replaced by an empty string, and for the vision-missing scenario, zero-imputed frames tensor is fed into the model. Figure 9 shows the vision-missing and text-missing scenarios for the MELD dataset. Figure 9: Illustration of the missing modality MELD Dataset: We evaluate on the MELD test split using utterance-level metadata providing dialogue ID, speaker, transcription, and 7-class emotion labels (neutral, surprise, fear, sadness, joy, disgust, anger). Visual inputs are constructed from pre-extracted cropped and aligned facial frames, resized to 224×224224× 224 and uniformly sampled to T=8T=8, frames per utterance. Textual inputs are the raw utterance transcriptions from the metadata. Missing modality scenarios are simulated by replacing the utterance string with an empty string for text-missing conditions, controlled by a seeded random state (seed 42) for reproducibility. The model is prompted in instruction-format with the video and utterance as inputs, and asked to classify the emotion from the seven candidate labels. Student LoRA is adapted using the Adam optimizer with a learning rate 10−410^-4 with a batch size of 1, where gradient accumulation over 8 steps is equivalent to performing a single effective update per sample, updating only the student LoRA parameters r=8r=8 , α=16α=16, and dropout 0.3. DFEW Dataset: Since Video-LLaVA-7B is used in all experiments, the data loading and prompting structure remains the same as MELD. Student LoRA is adapted using the Adam optimizer with a learning rate 10−410^-4 with a batch size of 1, where gradient accumulation over 8 steps is equivalent to performing a single effective update per sample, updating only the student LoRA parameters r=8r=8 , α=16α=16, and dropout 0.3. All experiments use float16 precision. The results are reported on the Official Fold-1 test set. BAH Dataset: Similar to DFEW and MELD datasets, we perform the experiments for the BAH dataset using the VideoLlava model. However, the task of Ambivalence/Hesitancy detection is more complex than basic emotion recognition. Following Gonzales et al. (6), we use the prompting structure shown in Table 9 to get the best results. Table 9 shows high reliance on the textual input. Adding an accurate description of the task improves performance, and the best results are achieved with the definition and speaker utterance. Student LoRA is adapted using the Adam optimizer with a learning rate 10−410^-4 with a batch size of 1, where gradient accumulation over 8 steps is equivalent to performing a single effective update per sample, updating only the student LoRA parameters r=8r=8, α=16α=16, and dropout 0.3. All experiments use float16 precision. The results are reported on the official test set. Compute details: For all experiments, the student is being aligned to the teacher’s hidden representations, not its output distribution. We therefore fix generation to greedy decoding (do_sample=False) so that final predictions are deterministic given a fixed adapter state. Experiments were run on a machine with 4× NVIDIA A100-SXM4-40GB GPUs and CUDA version 12.8. The exact libraries and corresponding versions are included in the code supplement. FAR Parameters: βF _F, τfc _fc, K, and τfmi _fmi together control the ACTIVE/ANCHORED state machine. βF _F sets the EMA decay for the Fisher diagonal. τfc _fc and K jointly gate consolidation: CI must stay below τfc _fc for K consecutive adapted samples before the student is frozen. τfmi _fmi gates reactivation, triggering once FMI exceeds this threshold. M and ε play a minor role, affecting only reporting and numerical stability rather than the transition logic itself. Table 10: FAR hyperparameters Parameter Value Definition βF _F 0.99 EMA decay parameter for the online Fisher diagonal τfc _fc 0.02 Consolidation Index threshold K 5 No. of consecutive adaptation steps required τfmi _fmi 2.0 Fisher Mismatch Index threshold ε 10−810^-8 Numerical stability constant References Abootorabi et al. (2025) M. M. Abootorabi et al. Ask in any modality: a comprehensive survey on multimodal retrieval-augmented generation. In Findings of ACL, Cited by: §2. Asai et al. (2023) A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi Self-rag: learning to retrieve, generate, and critique through self-reflection. ICLR abs/2310.11511. External Links: Link Cited by: §2. Chen et al. (2026) J. Chen, J. Liu, S. Liu, W. Zhang, A. Li, E. Zhu, and X. Liu Sample-specific modality diagnosis and cross-modal enhancement for incomplete multimodal representations. Proceedings of the AAAI Conference on Artificial Intelligence 40 (24), p. 20154–20162. External Links: Link, Document Cited by: §2. Cheng et al. (2024) Z. Cheng, Z. Cheng, J. He, J. Sun, K. Wang, Y. Lin, Z. Lian, X. Peng, and A. Hauptmann Emotion-llama: multimodal emotion recognition and reasoning with instruction tuning. External Links: 2406.11161, Link Cited by: §1. Ge et al. (2024) M. Ge, D. Tang, and M. Li Video emotion open-vocabulary recognition based on multimodal large language model. External Links: 2408.11286 Cited by: §1. González-González et al. (2026) M. González-González, S. Belharbi, M. O. Zeeshan, M. Sharafi, M. H. Aslam, M. Pedersoli, A. L. Koerich, S. L. Bacon, and E. Granger BAH dataset for ambivalence/hesitancy recognition in videos for digital behavioural change. In ICLR, Cited by: Appendix E, Appendix E, §1, §4.1. Guo et al. (2024) Z. Guo, T. Jin, and Z. Zhao Multimodal prompt learning with missing modalities for sentiment analysis and emotion recognition. In Proc. ACL, p. 1726–1736. Cited by: §2. Havaei et al. (2016) M. Havaei, N. Guizard, N. Chapados, and Y. Bengio HeMIS: hetero-modal image segmentation. In MICCAI, Cited by: §2. Hu et al. (2025) J. Hu, Z. Zhang, G. Chen, X. Wen, C. Shuai, W. Luo, B. Xiao, Y. Li, and M. Tan Test-time learning for large language models. External Links: 2505.20633 Cited by: §1, §2, §4.2, §4.5. Huang et al. (2025) D. Huang, Q. Li, C. Yan, Z. Cheng, Z. Han, Y. Huang, X. Li, B. Li, X. Wang, Z. Lian, Z. Cheng, and X. Peng Emotion-qwen: a unified framework for emotion and vision understanding. External Links: 2505.06685, Link Cited by: §1. Jiang et al. (2020) X. Jiang, Y. Zong, W. Zheng, C. Tang, W. Xia, C. Lu, and J. Liu DFEW: a large-scale database for recognizing dynamic facial expressions in the wild. External Links: 2008.05924 Cited by: Appendix E, §1, §4.1. Kirkpatrick et al. (2017) J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114 (13), p. 3521–3526. External Links: ISSN 1091-6490, Link, Document Cited by: §1. Li et al. (2023) K. Li, Y. He, Y. Wang, Y. Li, W. Wang, P. Luo, Y. Wang, L. Wang, and Y. Qiao VideoChat: chat-centric video understanding. In arXiv preprint arXiv:2305.06355, Cited by: §1. Lin et al. (2023) B. Lin, B. Zhu, Y. Ye, M. Ning, P. Jin, and L. Yuan Video-LLaVA: learning united visual representation by alignment before projection. In arXiv preprint arXiv:2311.10122, Cited by: §1, §1. Lin et al. (2024) G. Lin, W. P. Huang, and H. Lee Continual test-time adaptation for end-to-end speech recognition on noisy speech. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Cited by: §2. Ma et al. (2021) M. Ma, J. Ren, L. Zhao, S. Tulyakov, C. Wu, and X. Peng SMIL: multimodal learning with severely missing modality. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI 2021), p. 2302–2310. Cited by: §1, §2. Maaz et al. (2024) M. Maaz, H. Rasheed, S. Khan, and F. Khan Video-ChatGPT: towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 12585–12602. External Links: Document Cited by: §1, §1. Maheshwari et al. (2024) H. Maheshwari, Y. Liu, and Z. Kira Missing modality robustness in semi-supervised multi-modal semantic segmentation. In Proc. IEEE/CVF WACV, p. 1020–1030. Cited by: §2. Mishra (2026) H. Mishra RDumb++: drift-aware continual test-time adaptation. arXiv preprint arXiv:2601.15544. Cited by: §2. Nado et al. (2021) Z. Nado et al. Evaluating prediction-time batch normalization for robustness under covariate shift. NeurIPS. Cited by: §2. Neverova et al. (2016) N. Neverova et al. ModDrop: adaptive multi-modal gesture recognition. In CVPR, Cited by: §2. Nezakati et al. (2024) N. Nezakati, M. K. Reza, A. Patil, M. Solh, and S. Asif MMP: towards robust multi-modal learning with masked modality projection. External Links: Document Cited by: §1, §2. Niu et al. (2022) S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan Efficient test-time model adaptation without forgetting. In Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, p. 16888–16905. Cited by: Appendix C, §2. Niu et al. (2023) S. Niu, J. Wu, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan Towards stable test-time adaptation in dynamic wild world. External Links: 2302.12400, Link Cited by: §1. Poria et al. (2019) S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria, and R. Mihalcea MELD: a multimodal multi-party dataset for emotion recognition in conversations. External Links: 1810.02508 Cited by: Appendix E, §1, §4.1. Press et al. (2023) O. Press, S. Schneider, M. Kümmerer, and M. Bethge RDumb: a simple approach that questions our progress in continual test-time adaptation. In Advances in Neural Information Processing Systems, Vol. 36, p. 39915–39935. Cited by: §2. Qian et al. (2025) H. Qian, Z. Liu, P. Zhang, K. Mao, D. Lian, Z. Dou, and T. Huang MemoRAG: boosting long context processing with global memory-enhanced retrieval augmentation. In Proceedings of the ACM on Web Conference 2025, W ’25, New York, NY, USA, p. 2366–2377. External Links: ISBN 9798400712746, Document Cited by: §2. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML 2021), p. 8748–8763. Cited by: §1. Ramazanova et al. (2025) M. Ramazanova, A. Pardo, B. Ghanem, and M. Alfarra Test-time adaptation for combating missing modalities in egocentric videos. External Links: 2404.15161 Cited by: §1. Saeed et al. (2024) M. S. Saeed, S. Nawaz, M. Z. Zaheer, M. H. Khan, K. Nandakumar, M. H. Yousaf, H. Sajjad, T. D. Schepper, and M. Schedl Modality invariant multimodal learning to handle missing modalities: a single-branch approach. External Links: 2408.07445 Cited by: §2. Shaker et al. (2025) A. Shaker, M. Maaz, C. Gou, H. Rezatofighi, S. Khan, and F. S. Khan Mobile-videogpt: fast and accurate video understanding language model. External Links: 2503.21782, Link Cited by: Appendix D. Sun et al. (2020) Y. Sun et al. Test-time training with self-supervision for generalization under distribution shifts. In ICML, Cited by: §2. Wang et al. (2021) D. Wang et al. Tent: fully test-time adaptation by entropy minimization. In ICLR, Cited by: Appendix C, §2. Wang et al. (2020) Q. Wang, L. Zhan, P. Thompson, and J. Zhou Multimodal learning with incomplete modalities by knowledge distillation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p. 1828–1838. Cited by: §1. Wang et al. (2022) Q. Wang, O. Fink, L. Van Gool, and D. Dai Continual test-time domain adaptation. In CVPR, Cited by: §1, §2. Zhang et al. (2022) M. Zhang, S. Levine, and C. Finn MEMO: test time robustness via adaptation and augmentation. In NeurIPS, Cited by: §2. Zhao et al. (2021) J. Zhao, X. Mao, and L. Chen Missing modality imagination network for emotion recognition. In AAAI, Cited by: §2. Zhao et al. (2024) Q. Zhao, R. Wang, Y. Cen, D. Zha, S. Tan, Y. Dong, and J. Tang LongRAG: a dual-perspective retrieval-augmented generation paradigm for long-context question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 22600–22632. External Links: Document Cited by: §2.