Paper deep dive
Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learning
Longfei Huang, Yang Yang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/2/2026, 11:53:05 PM
Summary
The paper introduces Gradient-Aligned Alternating Learning (GAAL), a novel paradigm for multimodal tabular-image fusion. GAAL addresses gradient conflicts between modalities by using an alternating learning strategy combined with uncertainty-based cross-modal gradient surgery. This approach decouples multimodal gradients and uses high-entropy samples to guide optimization, resulting in improved performance across both multimodal and unimodal tasks compared to state-of-the-art baselines.
Entities (5)
Relation Signals (4)
GAAL → evaluatedon → DVM
confidence 95% · Empirical experiments on widely used datasets reveal the superiority of our method... DVM [13] dataset.
GAAL → evaluatedon → SUNAttribute
confidence 95% · We conducted experiments on three datasets: Data Visual Marketing (DVM) [13], SUNAttribute [21], and CelebA [17].
GAAL → evaluatedon → CelebA
confidence 95% · We conducted experiments on three datasets: Data Visual Marketing (DVM) [13], SUNAttribute [21], and CelebA [17].
GAAL → outperforms → SoTA baselines
confidence 90% · GAAL can outperform existing SoTA baselines, achieving not only the best multimodal performance but also the highest unimodal accuracy.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal tabular-image fusion is an emerging task that has received increasing attention in various domains. However, existing methods may be hindered by gradient conflicts between modalities, misleading the optimization of the unimodal learner. In this paper, we propose a novel Gradient-Aligned Alternating Learning (GAAL) paradigm to address this issue by aligning modality gradients. Specifically, GAAL adopts an alternating unimodal learning and shared classifier to decouple the multimodal gradient and facilitate interaction. Furthermore, we design uncertainty-based cross-modal gradient surgery to selectively align cross-modal gradients, thereby steering the shared parameters to benefit all modalities. As a result, GAAL can provide effective unimodal assistance and help boost the overall fusion performance. Empirical experiments on widely used datasets reveal the superiority of our method through comparison with various state-of-the-art (SoTA) tabular-image fusion baselines and test-time tabular missing baselines. The source code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2604.01579v1
- Canonical: https://arxiv.org/abs/2604.01579v1
Trouble viewing inline? Open PDF directly →
Full Text
31,706 characters extracted from source content.
Expand or collapse full text
Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learning Longfei Huang1, Yang Yang1∗ 1Nanjing University of Science and Technology hlf, yyang@njust.edu.cn *Corresponding author. Abstract Multimodal tabular-image fusion is an emerging task that has received increasing attention in various domains. However, existing methods may be hindered by gradient conflicts between modalities, misleading the optimization of the unimodal learner. In this paper, we propose a novel Gradient-Aligned Alternating Learning (GAAL) paradigm to address this issue by aligning modality gradients. Specifically, GAAL adopts an alternating unimodal learning and shared classifier to decouple the multimodal gradient and facilitate interaction. Furthermore, we design uncertainty-based cross-modal gradient surgery to selectively align cross-modal gradients, thereby steering the shared parameters to benefit all modalities. As a result, GAAL can provide effective unimodal assistance and help boost the overall fusion performance. Empirical experiments on widely used datasets reveal the superiority of our method through comparison with various state-of-the-art (SoTA) tabular-image fusion baselines and test-time tabular missing baselines. The source code is available at https://github.com/njustkmg/ICME26-GAAL. I Introduction In recent years, tabular data is increasingly accessible in multimodal datasets, and its integration is crucial in various applications [2, 33, 1]. An emerging example is tabular-image fusion that involves integrating structured tables and images to provide a holistic understanding of subjects. Despite numerous achievements, existing tabular-image fusion approaches [27, 5] often follow multimodal joint learning, where gradient conflicts frequently arise. These modality gradient conflicts are primarily caused by the unified optimization objective in joint learning [20, 16]. This potentially misleads unimodal learning and results in suboptimal final performance. As shown in Figure 1(a), we visualize the cosine similarity between image gradients and multimodal gradients on DVM [13] dataset. Negative cosine similarity indicates the presence of conflicts. Fortunately, recent multimodal studies [25, 30, 26] have recognized this issue and attempted to provide solutions. An early representative work, OGM [22], balances strong and weak modalities by adjusting gradient magnitudes. Subsequent work, MMPareto [26], introduces Pareto methods from multi-task learning to regulate gradient directions. Beyond the above joint learning paradigm, other works [34, 12, 6] adopt an alternating learning paradigm to decouple combined gradient into unimodal gradients, achieving higher performance. To facilitate interaction, they adopt a shared-head strategy. However, the aforementioned methods mainly focus on alleviating gradient conflicts in unimodal encoders, at the expense of the facilitative role of gradient signals in cross-modal interaction layer. This issue becomes pronounced when directly transferred to more challenging tabular-image tasks. As shown in Figure 1(b), these methods suffer from significant performance degradation in the tabular-image fusion, with image performance even falling below that of unimodal learning. This can be attributed to two main issues. First, gradient modulation is inherently more challenging in joint learning, making it difficult to unleash unimodal potential effectively. Second, while alternating learning can liberate unimodal encoders, shared head suffers from gradient conflicts. MLA [34] uses gradient orthogonalization to encourage modal independence, neglecting overlaps and synergy among modal objectives. Figure 1: We visualize the gradient conflicts and evaluate the performance of existing multimodal solutions on DVM dataset. (a) Multimodal and image gradients often show negative cosine similarity in naive joint learning, indicating severe gradient conflicts. (b) Existing multimodal solutions underperform in tabular-image fusion, failing to fully exploit unimodal image potential. In this work, to overcome the above challenges, we propose Gradient-Aligned Alternating Learning (GAAL), a novel tabular-image learning paradigm that effectively addresses gradient conflict and strengthens the tabular-image fusion performance. Specifically, GAAL alternates modality-specific optimization to decouple multimodal gradients while capturing cross-modal interaction via a shared classifier, thereby performing relatively independent unimodal training. Then, we design uncertainty-based cross-modal gradient surgery that modulates modal gradients via quadratic programming (QP) [7, 18] to address gradient conflicts and synergy at shared classifier. This surgery computes gradients from high-entropy samples of the previous modality and projects the current modality gradient onto the previous gradient direction while minimizing their Euclidean distance. To this end, GAAL addresses gradient conflicts and facilitates interactions in tabular-image fusion. As shown in Figure 1(b), our proposed method GAAL improves both multimodal and unimodal performance. Our main contributions are outlined as follows: • We propose GAAL, a novel alternating learning algorithm that allows the model to explore unimodal information effectively, thereby improving tabular-image fusion. • We design an uncertainty-based cross-modal gradient surgery that guides modal optimization directions to address gradient conflicts and facilitate synergy at the interaction layer. • Experimental results demonstrate that GAAL effectively addresses gradient conflict to improve tabular-image fusion performance and achieves SoTA performance. Figure 2: The framework of the GAAL method. (a) Gradient-Aligned Alternating Learning that only one modality’s learner is updated at each step. Uncertainty-based cross-modal gradient surgery utilizes gradients from cross-modal hard samples to guide the optimization of the shared classifier for the current modality. (b) Uncertainty-based Gradient Guidance that samples hard examples from another modality to provide cross-modal gradient guidance. I Related Work I-A Tabular-image Fusion Tabular data, known for its structured and interpretable nature, is the focus of traditional machine learning and deep learning [8, 19]. Unlike free-form audio or textual data, structured tabular data mixes dense numerical and sparse categorical features with differing value ranges and semantics, and lacks clearly defined interrelationships [3]. Existing tabular-image fusion approaches focus on integrating cross-modal information by multimodal learning strategies. TIP [5] proposes a transformer-based table encoder and enhances the robustness of tabular-image models through contrastive learning and cross-attention fusion. Moreover, since obtaining real-world tabular data is often costly and challenging, other studies have explored settings where tabular data are unavailable at inference time. These methods transfer expensive tabular expertise to images during training to enhance the performance of image models at inference. Since our method can facilitate cross-modal interaction, it also maintains good performance even when test-time tabular data are unavailable. I-B Multimodal Gradient Conflict Gradient conflict, defined as negative cosine similarity between gradients, has been extensively studied in multiple domains [15, 28]. Recent studies [25, 26] have found that gradient conflicts also exist in multimodal scenarios. Such multimodal gradient conflicts can mislead unimodal optimization, ultimately resulting in suboptimal multimodal performance. To address this issue, some early works [25, 22] adjust the magnitude of modality-specific gradients based on unimodal performance to compensate for weak modality. Subsequent work MMPareto [26] introduces Pareto approach in multi-task learning to identify integrated gradient directions beneficial for all modalities. Moreover, since multimodal gradient conflicts mainly arise from the unified optimization objective in joint learning, recent methods adopt an alternating learning paradigm to decouple optimization objective. Although this framework can directly address gradient conflicts in the encoder, shared interaction layers across modalities still suffer from gradient conflicts. Notably, MLA also considers interaction layer, but it isolates modal gradients through orthogonalization, overlooking potential overlap and synergy among modal optimization objectives. Therefore, our method aims to address both gradient conflicts and cross-modal interaction. I Methodology In this section, we introduce the proposed GAAL, a tabular-image alternating learning framework that addresses gradient conflict to improve classification. To facilitate tabular-image interaction, we design an uncertainty-guided cross-modal gradient surgery, which utilizes uncertainty-based cross-modal gradient guidance to assist in optimizing unimodal samples. The architecture of GAAL is shown in Figure 2. I-A Preliminary Let =xiI,xiT,yii=1NX= \x_i^I,x_i^T,y_i \_i=1^N be a training set, where yiy_i is the label for the ii-th instance and N is the number of training data. xiI∈ℝH×W×3x_i^I∈R^H× W× 3 represents the image, and xiT∈ℝDx_i^T∈R^D represents the tabular description, where D is the number of tabular features. Each tabular input contains two kinds of attributes, i.e., the categorical features (such as the “Gender”) and the continuous features (such as the “Age”). Assume there are Y classes in total, and yi∈[Y]=1,…,Yy_i∈ [Y ]= \1,...,Y \. For the sake of simplicity, we use superscript m to indicate the module corresponding to a specific modality in this section, where m∈I,Tm∈ \I,T \. Notation is summarized in supplemental material. I-B Gradient-Aligned Alternating Learning With the rapid growth of deep learning, representative tabular-image fusion approaches [9, 5, 14] have adopted deep neural network (DNN) for multimodal learning. Following these methods, we utilize DNN to construct our tabular and image models. Specifically, We use ϕmφ^m as encoders to extract features um=ϕm(θm,xm)u^m=φ^m(θ^m,x^m), where θ the encoder parameters. We define the predictor ψ as a mapping from the latent feature space to the label space. For given tabular-image pairs x=[xI,xT]x= [x^I,x^T ], the tabular-image model can be written as f(x)=ψ([uI:uT])f(x)=ψ( [u^I:u^T ]). Therefore, the objective function can be written as: ℒ(x,y)=−1N∑i=1Nyi⊤log(f(x)i). L(x,y)=- 1N _i=1^Ny_i (f(x)_i). (1) For a given input data instance, the alternating paradigm selects and updates a specific modality learner fm(xm)=ψ(Θm,ϕm(xm)f^m(x^m)=ψ( ^m,φ^m(x^m) at each time, where ψ is a shared classifier and Θ denotes the parameters of the classifier. The objective function ℒm(xm,y)L^m(x^m,y) of each modality is independent. By utilizing alternating learning, we can successfully address gradient conflicts in the tabular and image encoders. Since interaction layer is shared, gradient conflicts may arise in shared head during alternating learning, overshadowing the previous modality’s optimization. Hence, to address this issue, we propose a cross-modal gradient surgery that adjusts the current gradients based on cross-modal gradients. Concretely, assuming the gradient of the current modality at step t is denoted as g, and the gradient from the previous modality is gpg_p. We define α as the angle between g and gpg_p. We consider g to be detrimental to the previous modality when cosα<0 α<0. The cosine similarity (⋅)S(·) between two gradients g and gpg_p is (g,gp)=g⊤gp‖g‖2‖gp‖2, (g,g_p)= g g_p \|g \|_2 \|g_p \|_2, (2) where ∥⋅∥2 \|· \|_2 denote the L2L_2 norm of vectors. We aim to project g with the constraint to obtain the modified gradient g~ g that is aligned with gpg_p: gp⊤g~⩾ϵ, g_p g ε, (3) where ϵε is a small constant that governs the alignment intensity. Inspired by [18, 4], we can formulate (3) as the following optimization problem: ming~12‖g~−g‖22s.t.gp⊤g~⩾ϵ. matrix _ g& 12 \| g-g \|^2_2\\ s.t.&g_p g ε. matrix (4) Since the objective function is a convex quadratic function and the constraint is a linear inequality, the problem reduces to a convex quadratic programming problem with a single constraint. Geometrically, g~ g is the projection of g onto gpg_p, and g~ g is the closest to g in Euclidean distance. We introduce a dual variable v to represent the gradient weight. By constructing the Lagrangian function and applying KKT conditions, we derive both the closed-form optimal solution as follows: g~=g+vgp, g=g+vg_p, (5) v=max(0,ϵ−gp⊤g‖gp‖22). v= (0, ε-g_p g \|g_p \|^2_2). Unlike forcibly orthogonalizing all modal gradients, our method targets only conflicting gradients without enforcing orthogonality, which promotes multimodal synergy. More details are provided in supplemental material. During alternating iterations, we use cross-modal gradients to guide current modality, which addresses gradient conflicts in the interaction layer and improves cross-modal interaction. I-C Uncertainty-based Gradient Guidance Through the above gradient surgery, we successfully address gradient conflicts in the interaction layer and improve cross-modal interaction. Subsequently, we aim to select cross-modal gradients to provide more effective guidance in gradient surgery. Since gradients corresponding to hard samples often exhibit larger magnitudes, encapsulate more informative learning signals, and play a more critical role in parameter updates. Therefore, we design Uncertainty-based Gradient Guidance, which selects the gradient direction of high-uncertainty cross-modal samples as the projection direction. This gradient selection not only enables cross-modal gradients to guide current modality but also utilizes current modality to assist in learning cross-modal samples. Learning hard samples effectively enhances the model’s attention to edge cases. Specifically, for a sample ximx_i^m in a batch, let pimp_i^m denote the model’s predicted class probability distribution. We evaluate the uncertainty of each sample by computing the entropy of its predicted probability distribution pimp_i^m within the batch: ℋ(pim)=−∑c=1Ypi,cmlogpi,cm, (p_i^m)=- _c=1^Yp_i,c^m p_i,c^m, (6) where ℋ(pim)H(p_i^m) is the probability entropy for sample i. Finally, we select the top of λmλ^m samples with the highest uncertainty from modality m as hard samples to compute the gradient gpg_p. ℐpm=Topλm(ℋ(pim)∣i∈ℬm), _p^m=Top_λ^m (H(p_i^m) i ^m ), (7) gp=1|ℐpm|∑i∈ℐpm∇θℒ(xim), g_p= 1|I_p^m| _i _p^m _θL(x_i^m), (8) where ℬmB^m denotes the batch of modality m, ℐpmI_p^m is the set of indices of the selected top-λmλ^m high-entropy samples, ℒ(xim)L(x_i^m) is the loss for sample i, and ∇θ _θ represents the gradient with respect to classifier parameters. By selecting cross-modal gradients from hard samples, the learning signal can be effectively focused, reducing dilution from easy samples and helping to avoid local optima. Our algorithm is summarized in Algorithm 1. To sum up, our method GAAL employs alternating learning with shared-head interaction. Subsequently, we aim to address gradient conflicts and enhance synergy in interaction layer by projecting conflicting unimodal gradients. GAAL first selects hard samples from the previous modality to compute cross-modal gradients. Before updating the interaction layer for the current modality, conflicting gradients are projected onto the cross-modal gradients. This promotes hard sample learning and cross-modal interaction while addressing gradient conflicts. Algorithm 1 GAAL Algorithm. 1: Input: Training set X. 2: Output: The learned DNN models for all modalities. INIT Initialize iteration t=1t=1. Initialize encoder parameters θIθ^I, θTθ^T and shared classifier parameters Θ . 3: repeat 4: if mod(t,2)=1mod(t,2)=1 then 5: Sample ∀xiI,xiT,yi∈∀ \x_i^I,x_i^T,y_i \ , 6: end if 7: Pick up a specific modality m∈[I,T]m∈ [I,T ] in order; 8: Calculate predictions piIp_i^I and piTp_i^T in (10); 9: Calculate modality m loss ℒmL^m in (1); 10: Update modality-specific encoder parameters θtm _t^m; 11: Calculate modality m classifier gradient g; 12: Calculate sample entropy ℋ(I,T)/mH^(I,T)/m in (6); 13: Select the top of λ(I,T)/mλ^(I,T)/m high-entropy samples; 14: Calculate modality (I,T)/m(I,T)/m classifier gradient gpg_p; 15: Calculate modified gradient g~ g in (5); 16: Update shared parameters Θ using g~ g; 17: Update t=t+1t=t+1; 18: until Converge or reach maximum iterations. I-D Model Inference During the inference phase, given an tabular-image pair xi=[xiI,xiT]x_i= [x_i^I,x_i^T ], we compute the corresponding unimodal logits fim(xim)f_i^m(x_i^m). The unimodal prediction pimp_i^m is: pim=Softmax(fim(xim)), p_i^m=Softmax(f_i^m(x_i^m)), (9) and the final multimodal prediction pip_i is then derived through a weighted average of these unimodal logits: pi=Softmax(12(fiI(xiI)+fiT(xiT))). p_i=Softmax( 12(f_i^I(x_i^I)+f_i^T(x_i^T))). (10) With harmonized tabular-image fusion, inference merely requires simple averaging fusion at the decision level to achieve superior prediction results. TABLE I: The accuracy results on DVM, SUNAttribute, and CelebA datasets. The best performance is bolded, and the second-best is underlined. The image performance of test-time tabular missing methods represents their multimodal performance. The “*” indicates the version that follows the same pretrained weight setting as test-time tabular missing methods. Method DVM SUNAttribute CelebA Multi Image Tabular Multi Image Tabular Multi Image Tabular Unimodal Learning Resnet50 - 0.8743 - - 0.8361 - - 0.8146 - MLP - - 0.8742 - - 0.8082 - - 0.7775 Tabular-image Fusion CF 0.8793 0.0058 0.8792 0.8278 0.5544 0.8298 0.7740 0.5654 0.7723 MF 0.8993 - - 0.8333 - - 0.7941 - - DAFT 0.9460 - - 0.8456 - - 0.8202 - - TIP 0.9545 - - 0.8612 - - 0.8224 - - Multimodal Gradient Conflict OGM 0.8778 0.0076 0.8780 0.8305 0.6457 0.8312 0.7771 0.5328 0.7756 MMPareto 0.9658 0.7549 0.8851 0.8475 0.7992 0.8243 0.8082 0.7941 0.7702 MLA 0.9668 0.7664 0.8505 0.8417 0.8047 0.8194 0.8134 0.7915 0.7913 DI-MML 0.9719 0.8499 0.8748 0.8475 0.8003 0.8212 0.8199 0.8130 0.7839 ReconBoost 0.9714 0.8499 0.8756 0.8498 0.8117 0.8224 0.8149 0.8133 0.7836 LFM 0.9731 0.8543 0.8848 0.8487 0.8075 0.8222 0.8115 0.8110 0.7613 GAAL 0.9917 0.9057 0.9191 0.8668 0.8452 0.8368 0.8273 0.8222 0.7922 Test-time Tabular Missing KD - 0.8390 - - 0.8382 - - 0.8118 - MFH - - - - 0.8312 - - 0.7507 - FMR - 0.8427 - - 0.8347 - - 0.8003 - MMCL - 0.8203 - - 0.8431 - - 0.8041 - CHARMS - 0.9175 - - 0.8661 - - 0.8220 - GAAL* - 0.9358 - - 0.8662 - - 0.8313 - IV Experiments IV-A Experimental Settings Dataset. We conducted experiments on three datasets: Data Visual Marketing (DVM) [13], SUNAttribute [21], and CelebA [17]. DVM contains 1,451,784 car images paired with tabular data. Following previous work [9], car models with less than 100 samples were removed, resulting in 286 target classes. SUNAttribute is constructed from 717 SUN dataset categories, each annotated with 20 scene attributes. CelebA is a facial attribute dataset containing 202,599 face images. More details are provided in supplemental material. Baselines and Evaluation Metric. We selected various SoTA baselines for comparison, including tabular-image fusion methods: CF [23], MF [24], DAFT [27], TIP [5]; test-time tabular missing methods: KD [11], MFH [29], FMR [32], MMCL [9], CHARMS [14]; multimodal gradient conflict methods: OGM [22], MMPareto [26], MLA [34], DI-MML [6], ReconBoost [12] and LFM [31]. Following the setting of CHARMS [14], we adopt accuracy as the evaluation metric. Implementation Details. Following MMCL [9], we employ a ResNet50 [10] as image encoder and MLP as the tabular encoder. Additionally, we perform a grid search for hyperparameters and employ early stopping to select the best model. All experiments are conducted on an NVIDIA RTX A6000. More details are provided in supplemental material. IV-B Experimental Results To demonstrate the effectiveness of GAAL, we compare it with multiple popular methods on the three datasets shown in Table I, with results reported as mean values. The “-” indicates that the corresponding method failed to produce results for the modality. In particular, the MFH [29] method fails to handle the complex multi-class classification tasks of the DVM dataset. In summary, from Table I, we can observe that: 1) Compared with unimodal learning, tabular-image fusion methods, multimodal gradient conflict methods and test-time tabular missing methods, GAAL can achieve better performance in almost all cases. 2) GAAL can outperform existing SoTA baselines, achieving not only the best multimodal performance but also the highest unimodal accuracy. 3) The accuracy of test-time tabular missing setting demonstrates that our method achieves the best performance and remains robust. More results are provided in supplemental material. (a). DVM (b). SUNAttribute Figure 3: Sensitivity to hyper-parameter λIλ^I and λTλ^T on DVM and SUNAttribute datasets. TABLE I: Results of ablation studies on DVM and SUNAttribute datasets. CGS UGG DVM SUNAttribute Multi Image Tabular Multi Image Tabular 0.9701 0.8552 0.8760 0.8375 0.7789 0.8103 0.9909 0.8931 0.9203 0.8536 0.8201 0.8459 0.9917 0.9057 0.9191 0.8668 0.8452 0.8368 IV-C Sensitivity to Hyper-Parameters We study the impact of the threshold λI,λT \λ^I,λ^T \ on GAAL performance, as shown in Figure 3. An appropriate λI,λT \λ^I,λ^T \ can effectively concentrate gradient signals, providing efficient cross-modal guidance for model optimization. Excessively large or small λI,λTλ^I,λ^T can have negative effects, either diluting gradient signals or ignoring data distribution. (a). DVM (b). SUNAttribute Figure 4: Convergence results of GAAL on DVM and SUNAttribute datasets. IV-D Ablation Study We perform ablation studies on the DVM and SUNAttribute datasets to analyze the impact of alternating learning, cross-modal gradient surgery (CGS), and uncertainty-based gradient guidance (UGG), demonstrating the effectiveness of our method. Experimental results are reported in Table I. From Table I, we can find that: 1) Alternating learning, cross-modal gradient surgery and uncertainty-based gradient guidance can boost multimodal performance in terms of accuracy. 2) While the unimodal performance of the method using all objectives may not always reach the highest level, it achieves a more balanced classification performance across modalities. Figure 5: Impact of constraint margin ϵε. IV-E Convergence We also present the convergence results of the model during training on the DVM and SUNAttribute datasets. The loss curves are shown in Figure 4. Since gradients from different modalities naturally decouple, the loss curves of unimodal consistently decrease without getting stuck. IV-F Impact of constraint margin ϵε We vary ϵε on the SUNAttribute dataset and summarize the results in Figure 5. When ϵε is non-negative, it encourages modality gradient alignment. In particular, setting ϵ=0ε=0 may induce gradient orthogonality, leading to independent updates that are suboptimal for cross-modal interaction. Our results show that properly adjusting ϵε improves performance. V Conclusion In this work, we propose GAAL, a novel tabular-image fusion paradigm designed to address multimodal gradient conflicts and facilitate interaction. Our approach adopts alternating learning with a shared classifier to decouple multimodal gradient. To further address gradient conflicts in interaction layer and improve interaction, we design uncertainty-based cross-modal gradient surgery, which utilizes cross-modal gradients to guide the optimization of current modality. Experimental results demonstrate that GAAL outperforms all methods on both image-tabular fusion and test-time tabular missing tasks. We hope this work motivates future research on multimodal challenges encountered in real-world scenarios, with a particular focus on tabular-image learning. Limitations: Our method focuses on tabular and image classification. Future work will extend the approach to regression and detection tasks. Acknowledgment This work was supported in part by the NSFC (62276131), and in part by the Natural Science Foundation of Jiangsu Province of China under Grant (BK20240081). References [1] J. N. Acosta, G. J. Falcone, P. Rajpurkar, and E. J. Topol (2022) Multimodal biomedical ai. Nature medicine 28 (9), p. 1773–1784. Cited by: §I. [2] P. Baltescu, H. Chen, N. Pancha, A. Zhai, J. Leskovec, and C. Rosenberg (2022) ItemSage: learning product embeddings for shopping recommendations at pinterest. In KDD, p. 2703–2711. Cited by: §I. [3] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci (2024) Deep neural networks and tabular data: A survey. TNNLS 35 (6), p. 7499–7519. Cited by: §I-A. [4] A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny (2019) Efficient lifelong learning with A-GEM. In ICLR, Cited by: §I-B. [5] S. Du, S. Zheng, Y. Wang, W. Bai, D. P. O’Regan, and C. Qin (2024) TIP: tabular-image pre-training for multimodal classification with incomplete data. In ECCV, p. 478–496. Cited by: §I, §I-A, §I-B, §IV-A. [6] Y. Fan, W. Xu, H. Wang, J. Liu, and S. Guo (2024) Detached and interactive multimodal learning. In ACM M, Cited by: §I, §IV-A. [7] M. Frank, P. Wolfe, et al. (1956) An algorithm for quadratic programming. Naval research logistics quarterly 3 (1-2), p. 95–110. Cited by: §I. [8] Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko (2021) Revisiting deep learning models for tabular data. In NeurIPS, p. 18932–18943. Cited by: §I-A. [9] P. Hager, M. J. Menten, and D. Rueckert (2023) Best of both worlds: multimodal contrastive learning with tabular and imaging data. In CVPR, p. 23924–23935. Cited by: §I-B, §IV-A, §IV-A, §IV-A. [10] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In CVPR, p. 770–778. Cited by: §IV-A. [11] G. E. Hinton, O. Vinyals, and J. Dean (2015) Distilling the knowledge in a neural network. CoRR abs/1503.02531. Cited by: §IV-A. [12] C. Hua, Q. Xu, S. Bao, Z. Yang, and Q. Huang (2024) ReconBoost: boosting can achieve modality reconcilement. In ICML, Cited by: §I, §IV-A. [13] J. Huang, B. Chen, L. Luo, S. Yue, and I. Ounis (2022) DVM-CAR: A large-scale automotive dataset for visual marketing research and applications. In Big Data, p. 4140–4147. Cited by: §I, §IV-A. [14] J. Jiang, H. Ye, L. Wang, Y. Yang, Y. Jiang, and D. Zhan (2024) Tabular insights, visual impacts: transferring expertise from tables to images. In ICML, Cited by: §I-B, §IV-A. [15] Q. Jiang, Z. Chi, and Y. Yang (2025) Interactive multimodal learning via flat gradient modification. In IJCAI, p. 5489–5497. Cited by: §I-B. [16] Q. Jiang, L. Huang, and Y. Yang (2025) Rethinking multimodal learning from the perspective of mitigating classification ability disproportion. In NeurIPS, Cited by: §I. [17] Z. Liu, P. Luo, X. Wang, and X. Tang (2015) Deep learning face attributes in the wild. In ICCV, p. 3730–3738. Cited by: §IV-A. [18] D. Lopez-Paz and M. Ranzato (2017) Gradient episodic memory for continual learning. In NeurIPS, p. 6467–6476. Cited by: §I, §I-B. [19] A. Margeloiu, X. Jiang, N. Simidjievski, and M. Jamnik (2024) TabEBM: A tabular data augmentation method with distinct class-specific energy-based models. In NeurIPS, Cited by: §I-A. [20] H. Pan and Y. Yang (2025) Coordinated uni-modal assistance for enhancing multi-modal learning. In ICME, p. 1–6. Cited by: §I. [21] G. Patterson, C. Xu, H. Su, and J. Hays (2014) The SUN attribute database: beyond categories for deeper scene understanding. IJCV 108 (1-2), p. 59–81. Cited by: §IV-A. [22] X. Peng, Y. Wei, A. Deng, D. Wang, and D. Hu (2022) Balanced multimodal learning via on-the-fly gradient modulation. In CVPR, p. 8228–8237. Cited by: §I, §I-B, §IV-A. [23] S. Spasov, L. Passamonti, A. Duggento, P. Lio, N. Toschi, A. D. N. Initiative, et al. (2019) A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to alzheimer’s disease. Neuroimage 189, p. 276–287. Cited by: §IV-A. [24] L. A. Vale-Silva and K. Rohr (2021) Long-term cancer survival prediction using multimodal deep learning. Scientific Reports 11 (1), p. 13505. Cited by: §IV-A. [25] W. Wang, D. Tran, and M. Feiszli (2020) What makes training multi-modal classification networks hard?. In CVPR, p. 12692–12702. Cited by: §I, §I-B. [26] Y. Wei and D. Hu (2024) MMPareto: boosting multimodal learning with innocent unimodal assistance. In ICML, Cited by: §I, §I-B, §IV-A. [27] T. N. Wolf, S. Pölsterl, C. Wachinger, A. D. N. Initiative, and A. I. B. L. flagship study of ageing (2022) DAFT: A universal module to interweave tabular data and 3d images in cnns. NeuroImage 260, p. 119505. Cited by: §I, §IV-A. [28] J. Wu and M. Harandi (2025) MUNBa: machine unlearning via nash bargaining. In ICCV, p. 4754–4765. Cited by: §I-B. [29] Z. Xue, Z. Gao, S. Ren, and H. Zhao (2023) The modality focusing hypothesis: towards understanding crossmodal knowledge distillation. In ICLR, Cited by: §IV-A, §IV-B. [30] Y. Yang, H. Pan, Q. Jiang, Y. Xu, and J. Tang (2025) Learning to rebalance multi-modal optimization by adaptively masking subnetworks. TPAMI 47 (6), p. 4553–4566. Cited by: §I. [31] Y. Yang, F. Wan, Q. Jiang, and Y. Xu (2024) Facilitating multimodal classification via dynamically learning modality gap. In NeurIPS, Cited by: §IV-A. [32] Y. Yang, D. Zhan, Y. Fan, Y. Jiang, and Z. Zhou (2017) Deep learning for fixed model reuse. In Proceedings of the AAAI Conference on Artificial Intelligence, S. Singh and S. Markovitch (Eds.), p. 2831–2837. Cited by: §IV-A. [33] C. Zhang, X. Chu, L. Ma, Y. Zhu, Y. Wang, J. Wang, and J. Zhao (2022) M3Care: learning with missing modalities in multimodal healthcare data. In KDD, p. 2418–2428. Cited by: §I. [34] X. Zhang, J. Yoon, M. Bansal, and H. Yao (2024) Multimodal representation learning by alternating unimodal adaptation. In CVPR, p. 27456–27466. Cited by: §I, §I, §IV-A.