Paper deep dive
HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning
Qiuyu Zhu, Yi Gao, Zhichao Wan, Mingyang Ma
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Although recent Multimodal Large Language Models (MLLMs) have advanced general product understanding, they implicitly encode product information into global embeddings, thereby limiting their ability to capture fine-grained attributes. This limitation hinders performance in tasks requiring precise attribute discrimination, such as distinguishing subtle material differences among visually similar products. To address this challenge, we propose HMGCLIP, a unified multimodal embedding framework. By constructing a heterogeneous hypergraph, we leverage hypergraph topology to mine structure-aware hard negatives and align multi-granular semantics at both relation and hyperedge levels. This design enables a dual-granularity inference mechanism that dynamically fuses attribute evidence for both fine-grained and coarse-grained downstream tasks. Furthermore, we release a comprehensive fine-grained e-commerce dataset to facilitate future benchmarking. Extensive experiments on this new dataset and the public MAVE benchmark show that HMGCLIP outperforms strong multimodal encoders, MLLMs, and e-commerce baselines, validating the superiority of HMGCLIP.
Tags
Links
- Source: https://arxiv.org/abs/2608.24467v1
- Canonical: https://arxiv.org/abs/2608.24467v1
Trouble viewing inline? Open PDF directly →
Full Text
39,141 characters extracted from source content.
Expand or collapse full text
HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning Qiuyu Zhu Yi Gao Zhichao Wan Mingyang Ma Email: zqy527291,xiheng.gy,wanzhichao.wzc,mingyang.mmy@alibaba-inc.com Affiliation: Alibaba International Digital Commerce Group Abstract Although recent Multimodal Large Language Models (MLLMs) have advanced general product understanding, they implicitly encode product information into global embeddings, thereby limiting their ability to capture fine-grained attributes. This limitation hinders performance in tasks requiring precise attribute discrimination, such as distinguishing subtle material differences among visually similar products. To address this challenge, we propose HMGCLIP, a unified multimodal embedding framework. By constructing a heterogeneous hypergraph, we leverage hypergraph topology to mine structure-aware hard negatives and align multi-granular semantics at both relation and hyperedge levels. This design enables a dual-granularity inference mechanism that dynamically fuses attribute evidence for both fine-grained and coarse-grained downstream tasks. Furthermore, we release a comprehensive fine-grained e-commerce dataset to facilitate future benchmarking. Extensive experiments on this new dataset and the public MAVE benchmark show that HMGCLIP outperforms strong multimodal encoders, MLLMs, and e-commerce baselines, validating the superiority of HMGCLIP. 1 Introduction Modern e-commerce platforms manage billions of products described by images, titles, structured attributes, and categorical hierarchies. Learning unified representations from heterogeneous signals is fundamental to product search, recommendation, attribute value extraction (PAVE), and duplicate product detection (Radford et al., 2021; Chia et al., 2022). Despite rapid progress in multimodal representation learning (Lin et al., 2025; Li et al., 2026), three limitations persist in the e-commerce setting. Figure 1: Comparison of fine-grained attributes among the query, ground-truth product and similar product. First, general vision-language models such as CLIP (Radford et al., 2021) are trained on web-scale image–text pairs and struggle with the fine-grained distinctions that e-commerce demands—for instance, differentiating between “mesh” and “leather” uppers, or “air cushion” and “flat rubber” soles, requires more than global visual alignment. Second, standard contrastive learning typically treats negative samples uniformly or samples them randomly from batches. This approach fails to provide the semantically confusable hard negatives necessary for sharpening decision boundaries at the attribute level (de Souza P. Moreira et al., 2025). Third, existing benchmarks often lack the structural richness of real-world e-commerce data—particularly category taxonomies, attribute key–value constraints, and higher-order product entity groups—making it difficult to evaluate hierarchical and compositional understanding. The core challenge is that e-commerce products are inherently compositional: their semantics arise from the interaction of visual appearance, textual descriptions, and structured attributes. As illustrated in Figure 1, monolithic encoders may fail to preserve fine-grained product distinctions when trained without explicit structural constraints. For example, a query sneaker with a pink-tinted outsole and a mesh upper may receive high similarity to a visually similar but semantically incorrect item, such as a sneaker with a beige outsole and a leather upper, because the two products share coarse visual cues including silhouette, lacing pattern, and overall color distribution. In large-scale retrieval, such global bias leads to ranking errors and retrieval ambiguity, as the model overlooks decisive attribute-level evidence such as material texture and outsole appearance. To address these limitations, we present HMGCLIP, a unified framework for multimodal e-commerce representation learning built on three insights: (1) e-commerce data naturally forms a heterogeneous hypergraph where products, images, attributes, and categories participate in higher-order relationships that extend beyond pairwise edges; (2) effective contrastive learning requires hard negative mining at multiple granularities—from coarse category-level to fine attribute-level; (3) learning a structure-aware unified embedding space enables robust zero-shot generalization across diverse downstream tasks, eliminating the need for task-specific fine-tuning. Our main contributions are summarized as: • Framework. We present a unified multimodal embedding framework for e-commerce that supports both fine-grained and coarse-grained downstream tasks, achieving robust generalization without task-specific encoder fine-tuning. • Method. We develop a multi-granularity contrastive learning paradigm that aligns relation-level and hyperedge-level semantics in a unified embedding space. By exploiting heterogeneous hypergraph topology, our method mines semantically confusable hard negatives and jointly aligns global product representations with local attribute semantics and higher-order groups. • Dataset. We release a fine-grained multimodal e-commerce dataset to fill the gap in existing benchmarks and facilitate future research. • Evaluation. Extensive experiments on our introduced dataset and the public MAVE benchmark demonstrate that HMGCLIP achieves strong and often state-of-the-art performance across attribute prediction and product classification tasks, validating its robustness and versatility. 2 Related Work E-commerce Representation Learning. Vision-language pre-training models—including CLIP (Radford et al., 2021), ALIGN (Jia et al., 2021), and BLIP-2 (Li et al., 2023)—have demonstrated that large-scale image–text alignment yields transferable representations for retrieval and recognition. This paradigm has been adapted to e-commerce through domain-specific contrastive pre-training and knowledge-enhanced multimodal fusion (Dong et al., 2022; Zhu et al., 2021b; Jin et al., 2023). Recent MLLM-based e-commerce models improve generative product understanding (Fu et al., 2025), but they are often optimized for task-specific prediction rather than reusable embedding spaces. A parallel line of work addresses product attribute value extraction via generative or instruction-following multimodal LLMs (Khandelwal et al., 2023; Brinkmann et al., 2024). Despite these advances, most methods treat products as isolated image–text pairs and underuse structured dependencies such as attribute-key/value constraints and co-occurring attributes. In contrast, HMGCLIP learns a task-agnostic unified space where products, categories, and attributes are represented as reusable nodes for cross-task generalization. Contrastive Learning with Hard Negative Mining. Prior contrastive learning improves representation quality through label-aware positives, prototypes, graph augmentations, and hard negative mining (Khosla et al., 2020; Li et al., 2021; Zhu et al., 2021a; Robinson et al., 2021). In e-commerce, however, negatives are not arbitrary: they are constrained by category hierarchies, attribute-key/value relations, and visual similarity. Hypergraphs provide a natural abstraction for modeling higher-order product relations beyond pairwise edges. (Yadati et al., 2019; Huang and Yang, 2021), and recent e-commerce graph methods have demonstrated the value of structured product relations for retrieval, recommendation, and attribute extraction (Wang et al., 2023; Hu et al., 2025; Hongwimol et al., 2026). Nevertheless, existing methods rarely unify relation-level hard negatives, hyperedge-level semantic alignment, and multimodal product representations within a single embedding framework. HMGCLIP addresses it by combining heterogeneous hypergraph-based hard negative mining with hierarchical multi-granularity contrastive learning for diverse e-commerce tasks. 3 Preliminaries Products in e-commerce are compositional entities whose semantics arise from heterogeneous fields, including images, text, structured attributes, and category information. For a product p, we denote its multimodal inputs as p=Ip,Tp,Ap,CpX_p=\I_p,T_p,A_p,C_p\, where IpI_p is the product image, TpT_p is the title and description, Ap=a1,a2,…,akA_p=\a_1,a_2,…,a_k\ is the set of structured aspects, i.e., attribute–value pairs , and CpC_p is the category. These modalities capture semantic information at different granularities, ranging from fine-grained physical properties to coarse-grained product roles defined by the platform taxonomy. Our goal is to learn a unified multimodal embedding function f(⋅):→ℝdf(·):X ^d that captures fine-grained semantics while preserving discriminative boundaries. We evaluate the learned representations on two downstream tasks: attribute prediction (Yang et al., 2022), which tests fine-grained discrimination, and product classification, which tests semantic clustering quality. Following prior works (Yang et al., 2022; Khandelwal et al., 2023), we formulate classification as an embedding-based matching problem. Specifically, given a query product p, the predicted attribute value or category is defined as the candidate whose embedding is most semantically similar to the product representation: y∗=argmaxy∈^sim(f(p),f(y)),y^*= _y∈ Ysim(f(p),f(y)), (1) where Y denotes the candidate set of aspects or categories, and sim(⋅,⋅)sim(·,·) represents cosine similarity. 4 Methodology Figure 2: Overview of HMGCLIP. (1) Construction of a heterogeneous hypergraph from product data and semantic annotations. (2) Multi-granularity contrastive learning over relation-level and hyperedge-level views to unify the semantic space. (3) Dual-granularity inference via retrieval over learned key-value and category libraries for aspect prediction and evidence-fused product classification. As illustrated in Figure 2, HMGCLIP consists of three stages: (1) Heterogeneous Hypergraph Semantic Construction, which organizes products, aspects, and categories into a structured semantic graph with pairwise relations and higher-order hyperedges; (2) Multi-Granularity Contrastive Representation Learning, which aligns entities at the node, relation, and hyperedge levels to learn a unified semantic space; and (3) Dual-Granularity Inference, which performs attribute prediction and evidence-fused product classification by retrieving from the learned aspect and category libraries. HMGCLIP captures both local semantic relations and higher-order semantic dependencies, enabling discriminative representations for downstream e-commerce tasks such as fine-grained attribute prediction and coarse-grained product classification. 4.1 Heterogeneous Hypergraph Semantic Construction We formulate a heterogeneous graph Wang et al. (2019); Zhu et al. (2025a); Zhu et al. (2025b) =(,ℰ)G=(V,E) to structure the relationships among products, aspects, and categories, where the node set is defined as =∪V=P . Here, product nodes P are connected to aspect nodes A and category nodes C through product-aspect edges (p,a)∈ℰ(p,a) and product-category edges (p,c)∈ℰ(p,c) , respectively. However, pairwise relations alone cannot capture the group-level semantic consistency in e-commerce catalogs. For example, products sharing the same aspect, such as “Waterproof,” often form coherent clusters across different categories. To model such higher-order dependencies, we extend the heterogeneous graph G into a heterogeneous hypergraph Antelmi et al. (2023); Wang et al. (2019) H=(,ℰ,ℋ)G_H=(V,E,H), where ℋH denotes the set of hyperedges. Specifically, we define two types of hyperedges: (1) For each aspect node a∈a , an aspect-level hyperedge ha=a∪p∈∣(p,a)∈ℰh_a=\a\∪\p (p,a) \ groups the aspect with all associated products, capturing fine-grained semantic consistency. (2) Likewise, for each category node c∈c , a category-level hyperedge hc=c∪p∈∣(p,c)∈ℰh_c=\c\∪\p (p,c) \ groups the category with its member products, preserving coarse-grained semantic coherence. By combining pairwise edges and higher-order hyperedges, HG_H provides a structural foundation for subsequent contrastive learning. 4.2 Multi-Granularity Contrastive Learning To learn fine-grained representations, we fine-tune a pre-trained multimodal encoder with e-commerce-specific structural priors through multi-granularity contrastive learning over relation-level and hyperedge-level views. Our method leverages structure-aware sampling to construct informative positive and negative pairs, enabling the model to align pairwise relations and higher-order semantic groups within a unified embedding space. 4.2.1 Relation-Guided Hard Negative Mining Standard in-batch negatives are often semantically distant and provide limited supervision for fine-grained discrimination. To address this, we mine hard negatives for pairwise relations from the heterogeneous graph. Specifically, for an anchor product pip_i and its ground-truth positive aspects i+A_i^+, we construct a hard negative set i−A_i^- by applying three constraints: (i) the candidate aspect must share the same aspect key as a positive aspect, such as “Material” or “Color”; (i) it must appear in the same category context as pip_i; and (i) it must not be a ground-truth aspect of pip_i. This yields semantically confusable negatives for the product-aspect relation, encouraging the model to learn sharper distinctions among directly related aspects. 4.2.2 Hyperedge-Guided Group Alignment Beyond pairwise supervision, we enforce group-level consistency via hyperedge alignment. For each anchor product, we define positives as co-occurring products within the same hyperedge and negatives as those outside it. This contrastive objective maximizes anchor-positive similarity while minimizing anchor-negative similarity, fostering compact and coherent clustering of semantically related products in the latent space. 4.3 Joint Relation-Level and Hyperedge-Level Contrastive Optimization To jointly capture fine-grained pairwise semantics and higher-order group structure, we optimize a unified contrastive objective over relation-level and hyperedge-level supervision. Specifically, we adopt the InfoNCE loss van den Oord et al. (2018) as the basic optimization framework. For an anchor sample i, let i+S_i^+ and i−S_i^- denote the sets of positive and negative samples, respectively. We define the positive and negative partition functions as: Zi+=∑j∈i+exp(i⊤j/τ),Z_i^+= _j _i^+ (h_i h_j/τ), (2) Zi−=∑k∈i−exp(i⊤k/τ),Z_i^-= _k _i^- (h_i h_k/τ), (3) where τ is the temperature parameter. The loss for a batch of N samples is formulated as: ℒcontrast=−1N∑i=1NlogZi+Zi++Zi−.L_contrast=- 1N _i=1^N Z_i^+Z_i^++Z_i^-. (4) We apply this loss to both relation-level and hyperedge-level views. Specifically, ℒrelL_rel is computed using product-aspect pairs with hard negatives, while ℒhyperL_hyper is computed using product groups within semantic hyperedges defined in Section 4.1. The overall objective is defined as: ℒ=ℒrel+λℒhyper,L=L_rel+ _hyper, (5) where λ balances the contribution of the two supervision signals. 4.4 Dual-Granularity Inference Upon completion of multi-granularity contrastive pre-training, the encoders are frozen. We propose a dual-path inference mechanism that adapts to task granularity by leveraging the aligned embedding space for both fine-grained aspect retrieval and coarse-grained category prediction. 4.4.1 Path I: Fine-Grained Retrieval For fine-grained aspect prediction, we directly utilize the multimodal embedding qz_q of the query product q to retrieve the best-matching aspect from the vocabulary A. The predicted aspect a^q a_q is identified by maximizing the cosine similarity between the query and candidate aspect embeddings: a^q=argmaxj∈q⊤j‖q‖‖j‖, a_q= _j z_q a_j\|z_q\|\|a_j\|, (6) where ja_j denotes the embedding of aspect candidate j. This non-parametric nearest-neighbor search leverages the discriminative embedding space to capture fine-grained semantic distinctions without requiring additional classification heads. 4.4.2 Path I: Evidence-Fused Coarse-Grained Retrieval For coarse-grained category retrieval, relying solely on the product’s intrinsic features may lead to ambiguity in distinguishing semantically similar categories. To address this, we construct an enhanced query representation by integrating specific semantic evidence into the global context. Semantic Anchor-Based Fusion. Initially, we utilize the set of top-K aspects retrieved in Path I to construct a robust semantic anchor. Let 1,2,…,K\a_1,a_2,…,a_K\ be the embeddings of these retrieved aspects. We obtain the aggregated aspect representation agga_agg via mean pooling. This aggregated vector serves as a contextual summary of the product’s key aspects. We then fuse the product’s intrinsic multimodal embedding qz_q with this semantic summary via linear interpolation: fused=αq+(1−α)agg,z_fused= _q+(1-α)a_agg, (7) where α balances the contribution of intrinsic features and aggregated semantic evidence. The final category prediction y^q y_q is obtained by retrieving the best match from the category candidate set C based on cosine similarity: y^q=argmaxc∈(fused)⊤‖fused‖‖. y_q= _c \, (z_fused) c\|z_fused\|\|c\|. (8) Residual Transformer-based Fusion. To capture fine-grained semantic interactions between product features and aspect evidence, we further introduce an evidence-enhanced residual fusion module. This module injects retrieved aspect evidence into the product representation via a residual connection, thereby enriching semantic details while preserving the geometric structure of the learned embedding space. Given the product embedding qz_q and the set of retrieved aspect embeddings 1,…,K\a_1,…,a_K\ defined previously, we form a field-level token sequence by combining the product token with its aspect evidence tokens: =[q,1,2,…,K]X=[z_q,a_1,a_2,…,a_K]. Then, we add learnable type embeddings to distinguish the product token from aspect tokens: ~0=q+prod, x_0=z_q+e_prod, (9) ~i=i+attr,i=1,…,K, x_i=a_i+e_attr, i=1,…,K, (10) where prode_prod and attre_attr are learnable type embeddings. The resulting sequence is denoted as: ~=[~0,~1,…,~K] X=[ x_0, x_1,…, x_K]. After that, we apply a lightweight Transformer encoder to model interactions between the product representation and the retrieved aspect evidence: [0,1,…,K]=TransformerEncoder(~).[h_0,h_1,…,h_K]=TransformerEncoder( X). (11) Through self-attention, the product token can selectively attend to category-relevant aspects, while aspect tokens can also contextualize one another. This enables interaction-aware fusion beyond mean pooling or scalar weighting. Instead of directly replacing the product embedding with the transformed token 0h_0, we use it to predict a residual correction, and obtain the final representation through residual composition: =norm(q+αrfθ(0)).z=norm\! ( [rgb]0,0,0z_q+ _rf_θ(h_0) ). (12) where fθ(⋅)f_θ(·) is a lightweight projection module, αr _r controls the aspect-aware correction strength, and norm(⋅)norm(·) denotes L2 normalization. This residual design keeps the original product embedding as the semantic anchor while allowing retrieved aspect evidence to provide a controlled correction. Thus, the fused representation remains compatible with the learned embedding space and can be directly matched against taxonomy-defined categories. Table 1: Performance of aspect (attribute) prediction and product classification tasks on both datasets. Model Internal Dataset MAVE Hit@1/MRR@1 Hit@3 MRR@3 Hit@5 MRR@5 Hit@1/MRR@1 Hit@3 MRR@3 Hit@5 MRR@5 Aspect Prediction SigLIP2 40.74 76.27 56.64 87.68 59.26 5.73 15.60 9.90 24.42 11.90 FashionCLIP 49.63 81.59 64.00 90.54 66.06 16.71 36.00 25.00 46.92 27.48 InternVL3.5-2B 35.29 72.09 51.59 83.53 54.25 5.29 14.90 9.31 24.30 11.43 Qwen3-VL-2B 37.88 72.86 53.46 84.00 56.03 6.01 14.68 9.64 21.68 11.20 GME-Qwen2VL 40.61 75.58 56.13 86.48 58.65 11.40 26.76 17.98 37.32 20.38 M-Embed 47.09 79.06 61.32 89.65 63.75 5.48 15.93 9.87 23.43 11.57 CASLIE-S 39.19 74.73 54.99 84.94 57.35 8.13 20.93 13.60 29.94 15.66 Qwen3-VL-Emb 54.10 85.27 68.12 92.74 69.86 15.56 33.33 23.25 45.38 25.97 HMGCLIP 75.38 94.70 84.27 97.61 84.95 24.61 46.16 33.93 58.32 36.69 Product Classification SigLIP2 0.17 0.41 0.28 0.72 0.35 1.05 1.18 1.11 1.23 1.12 FashionCLIP 39.79 60.53 48.94 68.51 50.76 75.66 85.33 80.03 89.20 80.89 InternVL3.5-2B 6.81 15.36 10.47 21.73 11.92 8.99 40.24 24.04 48.27 25.86 Qwen3-VL-2B 0.27 0.78 0.48 1.24 0.59 22.49 34.89 27.99 37.87 28.68 GME-Qwen2VL 24.25 45.14 33.36 55.68 35.78 14.73 25.18 19.12 32.51 20.79 M-Embed 2.58 4.69 3.48 6.43 3.87 9.54 12.54 10.83 14.37 11.25 CASLIE-S 9.46 19.45 13.72 25.38 15.07 12.32 42.69 26.10 50.97 28.01 Qwen3-VL-Emb 48.30 71.19 58.46 79.79 60.45 61.45 83.63 71.44 89.83 72.87 HMGCLIP 84.23 93.77 88.55 95.75 89.01 96.70 99.15 97.83 99.57 97.93 5 Experiments and Analysis 5.1 Datasets MAVE (Yang et al., 2022). From the original 2.2M products, we retain multimodal products with images and structured aspects, prioritizing items with at least three aspects and adding two-aspect samples under a per-category cap to reduce imbalance. Internal dataset. We construct a dataset derived from real-world product listings on a Southeast Asian e-commerce platform. The data encompasses titles, descriptions, images, category labels, and structured key–value pairs. Reflecting the region’s linguistic diversity, the dataset is multilingual, predominantly featuring English alongside Thai, Vietnamese, Indonesian, and Chinese. It is specifically designed to address two core challenges: coarse-grained category understanding and fine-grained aspect discrimination. 5.2 Baselines and Evaluation Metrics All methods are evaluated under the same candidate pools. We report Hit@K and MRR@K for both fine-grained aspect prediction and coarse-grained product classification. For our method, Qwen3-VL-Embedding-2B serves as the backbone and is further post-trained with heterogeneous multi-granularity contrastive learning. At inference, aspect prediction uses direct product-to-aspect matching, while category prediction adopts aspect-guided semantic fusion (§4.4.2). 5.3 Experimental Results Table 2: Qualitative comparison on aspect (attribute) prediction. ✓ denotes correct Top-1; × denotes incorrect. # Image Product Ground Truth Qwen3-VL-Emb HMGCLIP 1 PIN PIN LUCKY CHARM MANEKI-NEKO WHITE SOLAR CAT DECOR CHARM material: plastic × material: brass ✓ material: plastic 2 Summer New Fashion Loose Womens T-shirt Short Sleeve Patchwork Vest Top Bottomin… clothing decoration: button × clothing decoration: side slit ✓ clothing decoration: button 3 Bhuuno Grommet rèm cua so Rèm cua so Phong Cách Trang Trí Nọi Thát Tra… curtain material: polyester × curtain material: vinyl ✓ curtain material: polyester Table 3: Qualitative comparison on category classification. ✓ denotes correct Top-1; × denotes incorrect. # Image Product Ground Truth Qwen3-VL-Emb HMGCLIP 1 DHT11 DHT 11 High Accuracy Temperature and Humidity Moisture Sensor Detect 3.3V 5V Module… Tools & Home Improvement > Electrical > Electrical Circuitry & Parts × Home Appliances > Heating, Cooling and Ventilation > Humidifiers ✓ Tools & Home Improvement > Electrical > Electrical Circuitry & Parts 2 Kẹm hàn cell pin cuọn đon 1 mét kẹm kích thuóc 0.15*8m Tools & Home Improvement > Power Tools & Accessories > Power Tools Parts × Stationery, Craft & Gift Cards > Packaging & Cartons > Packaging Protection > Shrink Wrap ✓ Tools & Home Improvement > Power Tools & Accessories > Power Tools Parts 3 Mesh Art Supplies Bag Portable Art Supplies Organizer Large A3 Size Me… Stationery, Craft & Gift Cards > School & Office > Storage & Organizers > Filing & Document Presentation × Bags and Travel > Functional Bags > Library Bag ✓ Stationery, Craft & Gift Cards > School & Office > Storage & Organizers > Filing & Document Presentation To evaluate the performance of HMGCLIP, we compare it with all baselines on both tasks across both datasets. The results are presented in Table 1. Aspect Prediction. HMGCLIP outperforms all baselines, demonstrating superior fine-grained semantic capture. On the internal dataset, it achieves a Hit@1/MRR@1 of 75.38%, surpassing the strongest baseline (Qwen3-VL-Emb) by 21.28%. This margin highlights its enhanced ability to distinguish subtle aspect differences compared to general-purpose VLMs. The advantage persists across broader recall scopes, with a Hit@5 of 97.61% (nearly 5% higher than the second-best model). Furthermore, HMGCLIP generalizes well to the heterogeneous MAVE benchmark, achieving a Hit@1/MRR@1 of 24.61% and MRR@5 of 36.69%. By exceeding specialized models like FashionCLIP (Hit@1: 16.71%), our approach proves robust in capturing complex product-aspect correlations in open-world scenarios. In particular, graph-derived hard negatives expose the model to semantically confusable attribute values, leading to sharper product–aspect decision boundaries. Product Classification. HMGCLIP consistently dominates baselines in coarse-grained category distinction. On the internal dataset, it achieves a Hit@1 of 84.23%, exceeding the second-best model (Qwen3-VL-Emb, 48.30%) by over 35%. On the MAVE benchmark, it attains a Hit@1 of 96.70% and Hit@5 of 99.57%, outperforming FashionCLIP (Hit@1: 75.66%). These results validate our Dual-Granularity Inference built on Residual Transformer Fusion. The low performance of SigLIP2 is likely due to its pairwise sigmoid image–text objective and lack of explicit cross-modal fusion. By preserving critical semantic signals during multimodal integration, this architecture creates a robust feature space that reinforces category boundaries through aspect-level consistency. 5.4 Analysis of Evidence Fusion Strategies We study evidence fusion through a progressive design axis, ranging from uniform aggregation to anchor-preserving weighting, instance-adaptive weighting, and interaction-aware residual fusion for coarse-grained category retrieval. First, aspect evidence must be incorporated carefully: mean pooling slightly improves Hit@1 over the backbone, but degrades Hit@3 and Hit@5, suggesting that uniform aggregation can disturb the neighborhood structure of the product embedding space. Second, preserving the product representation as a semantic anchor is important. Manually increasing the product-token weight improves over mean pooling, indicating that aspects should correct rather than replace the global product representation. Third, instance-adaptive weighting further improves performance, showing that different products rely on different subsets of evidence. 5.5 Qualitative Analysis To complement our quantitative results, we present qualitative comparisons in Table 2 and Table 3, illustrating how HMGCLIP rectifies errors made by the strongest baseline, Qwen3-VL-Emb. Aspect Prediction. Table 2 presents the Hit@1 results, illustrating that HMGCLIP correctly retrieves the ground truth for semantically ambiguous fine-grained aspects where the baseline fails. For instance, it distinguishes material differences (“plastic” vs. “brass” for a charm; “polyester” vs. “vinyl” for curtains) and structural details (“button” vs. “side slit” on a T-shirt). These results validate the efficacy of our heterogeneous multi-granularity contrastive learning, which enhances fine-grained discriminability by aligning visual features with specific aspect granularities, thereby sharpening decision boundaries among confusable values. Product Classification. Table 3 demonstrates the advantage of our evidence-fused coarse-grained retrieval. While Qwen3-VL-Emb misclassifies items into semantically overlapping but functionally distinct categories (e.g., DHT11 sensor as “Humidifiers,” welding tape as “Shrink Wrap”), HMGCLIP accurately predicts specific sub-categories (“Electrical Circuitry & Parts,” “Power Tools Parts”). This improvement stems from our residual transformer fusion mechanism, which injects retrieved aspect evidence into the classification process. By leveraging this fused evidence, the model effectively disambiguates products sharing general contextual features, reinforcing category boundaries through aspect-level consistency. 6 Conclusion In this paper, we present HMGCLIP, a unified multimodal embedding framework for multi-granularity e-commerce representation learning. By leveraging a heterogeneous hypergraph as a structural prior, HMGCLIP establishes a multi-granularity contrastive learning paradigm that aligns relation-level and hyperedge-level semantics within a unified embedding space. This design enables a dual-granularity inference mechanism that achieves generalization across both fine-grained and coarse-grained tasks without requiring task-specific fine-tuning. We further release a comprehensive multimodal e-commerce dataset. Experiments on both datasets demonstrate that HMGCLIP achieves state-of-the-art performance in both tasks, validating its effectiveness and versatility. References Antelmi et al. (2023) A. Antelmi, G. Cordasco, M. Polato, V. Scarano, C. Spagnuolo, and D. Yang A survey on hypergraph representation learning. ACM Comput. Surv. 56 (1). External Links: ISSN 0360-0300, Link, Document Cited by: §4.1. Brinkmann et al. (2024) A. Brinkmann, R. Shraga, and C. Bizer ExtractGPT: exploring the potential of large language models for product attribute value extraction. In Information Integration and Web Intelligence: 26th International Conference, iiWAS 2024, Bratislava, Slovakia, December 2–4, 2024, Proceedings, Part I, Lecture Notes in Computer Science, Vol. 15342, p. 38–52. External Links: Document, Link Cited by: §2. Chia et al. (2022) P. J. Chia, G. Attanasio, F. Bianchi, S. Terragni, A. R. Magalhães, D. Goncalves, C. Greco, and J. Tagliabue Contrastive language and vision learning of general fashion concepts. Scientific Reports 12, p. 18958. External Links: Document, Link Cited by: §1. de Souza P. Moreira et al. (2025) G. de Souza P. Moreira, R. Osmulski, M. Xu, R. Ak, B. Schifferer, and E. Oldridge NV-retriever: improving text embedding models with effective hard-negative mining. External Links: 2407.15831, Link Cited by: §1. Dong et al. (2022) X. Dong, X. Zhan, Y. Wu, Y. Wei, M. C. Kampffmeyer, X. Wei, M. Lu, Y. Wang, and X. Liang M5Product: self-harmonized contrastive learning for e-commercial multi-modal pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 11589–11598. External Links: Link Cited by: §2. Fu et al. (2025) C. Fu, D. Zhang, Y. Lin, Z. Nie, X. Zhang, J. Liu, Y. Liu, W. Guan, P. Wang, J. Xu, and B. Zheng MOON embedding: multimodal representation learning for e-commerce search advertising. CoRR abs/2511.11305. External Links: Document, 2511.11305, Link Cited by: §2. Hongwimol et al. (2026) P. Hongwimol, H. Shang, C. Wang, Z. Wan, Y. Gao, Y. Li, L. Gui, W. Sun, and C. Yu AutoPKG: an automated framework for dynamic e-commerce product-attribute knowledge graph construction. External Links: 2604.16950, Link Cited by: §2. Hu et al. (2025) J. Hu, J. Gong, H. Shen, and H. Eldardiry Hypergraph-based zero-shot multi-modal product attribute value extraction. In Proceedings of the ACM on Web Conference 2025, p. 4853–4862. External Links: Document, Link Cited by: §2. Huang and Yang (2021) J. Huang and J. Yang UniGNN: a unified framework for graph and hypergraph neural networks. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, p. 2563–2569. External Links: Document, Link Cited by: §2. Jia et al. (2021) C. Jia, Y. Yang, Y. Xia, Y. Chen, Z. Parekh, H. Pham, Q. V. Le, Y. Sung, Z. Li, and T. Duerig Scaling up visual and vision-language representation learning with noisy text supervision. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 4904–4916. External Links: Link Cited by: §2. Jin et al. (2023) Y. Jin, Y. Li, Z. Yuan, and Y. Mu Learning instance-level representation for large-scale multi-modal pretraining in e-commerce. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 11060–11069. External Links: Link Cited by: §2. Khandelwal et al. (2023) A. Khandelwal, H. Mittal, S. S. Kulkarni, and D. Gupta Large scale generative multimodal attribute extraction for e-commerce attributes. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), p. 277–287. External Links: Link, Document Cited by: §2, §3. Khosla et al. (2020) P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan Supervised contrastive learning. In Advances in Neural Information Processing Systems, Vol. 33, p. 18661–18673. External Links: Link Cited by: §2. Li et al. (2023) J. Li, D. Li, S. Savarese, and S. Hoi BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 19730–19742. External Links: Link Cited by: §2. Li et al. (2021) J. Li, P. Zhou, C. Xiong, and S. C. H. Hoi Prototypical contrastive learning of unsupervised representations. In International Conference on Learning Representations, External Links: Link Cited by: §2. Li et al. (2026) M. Li, Y. Zhang, D. Long, K. Chen, S. Song, S. Bai, Z. Yang, P. Xie, A. Yang, D. Liu, J. Zhou, and J. Lin Qwen3-VL-Embedding and Qwen3-VL-Reranker: a unified framework for state-of-the-art multimodal retrieval and ranking. External Links: 2601.04720, Link Cited by: §1. Lin et al. (2025) S. Lin, C. Lee, M. Shoeybi, J. Lin, B. Catanzaro, and W. Ping M-embed: universal multimodal retrieval with multimodal llms. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 44215–44234. External Links: Link Cited by: §1. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 8748–8763. External Links: Link Cited by: §1, §1, §2. Robinson et al. (2021) J. Robinson, C. Chuang, S. Sra, and S. Jegelka Contrastive learning with hard negative samples. In International Conference on Learning Representations, External Links: Link Cited by: §2. van den Oord et al. (2018) A. van den Oord, Y. Li, and O. Vinyals Representation learning with contrastive predictive coding. CoRR abs/1807.03748. External Links: Link, 1807.03748 Cited by: §4.3. Wang et al. (2019) X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu Heterogeneous graph attention network. In The World Wide Web Conference, W ’19, New York, NY, USA, p. 2022–2032. External Links: ISBN 9781450366748, Link, Document Cited by: §4.1, §4.1. Wang et al. (2023) X. Wang, C. Wang, L. Li, Z. Li, B. Chen, L. Jin, J. Huang, Y. Xiao, and M. Gao FashionKLIP: enhancing e-commerce image-text retrieval with fashion multi-modal conceptual knowledge graph. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), p. 149–158. External Links: Document, Link Cited by: §2. Yadati et al. (2019) N. Yadati, M. Nimishakavi, P. Yadav, V. Nitin, A. Louis, and P. Talukdar HyperGCN: a new method for training graph convolutional networks on hypergraphs. In Advances in Neural Information Processing Systems, Vol. 32, p. 1511–1522. External Links: Link Cited by: §2. Yang et al. (2022) L. Yang, Q. Wang, Z. Yu, A. Kulkarni, S. Sanghai, B. Shu, J. Elsas, and B. Kanagal MAVE: a product dataset for multi-source attribute value extraction. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, p. 772–781. External Links: Document, Link Cited by: §3, §3, §5.1. Zhu et al. (2025a) Q. Zhu, L. Zhang, Q. Xu, K. Liu, C. Long, and X. Wang HHGT: hierarchical heterogeneous graph transformer for heterogeneous graph representation learning. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, New York, NY, USA, p. 318–326. External Links: ISBN 9798400713293, Link, Document Cited by: §4.1. Zhu et al. (2025b) Q. Zhu, L. Zhang, Q. Xu, and C. Long HierPromptLM: a pure plm-based framework for representation learning on heterogeneous text-rich networks. External Links: 2501.12857, Link Cited by: §4.1. Zhu et al. (2021a) Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, p. 2069–2080. External Links: Document, Link Cited by: §2. Zhu et al. (2021b) Y. Zhu, H. Zhao, W. Zhang, G. Ye, H. Chen, N. Zhang, and H. Chen Knowledge perceived multi-modal pretraining in e-commerce. In Proceedings of the 29th ACM International Conference on Multimedia, p. 2744–2752. External Links: Document, Link Cited by: §2.