Paper deep dive
Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision
Shaojie Zhang, Ke Chen
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and stochastic corruption. Existing deep constrained clustering (DCC) methods mainly target hard, expert-agnostic constraints, treating soft labels mostly numerically rather than semantically. We formalize this setting as uncertainty-aware probabilistic constrained clustering (UPCC), defining a canonical aleatoric target through a heterogeneous observation process and analyzing its conditional identifiability. We introduce ProbPair, an angular pairwise objective for probabilistic relations, and build ECI-PP, an estimator--corrector--integrator framework that refines imperfect supervision via belief estimation, correction, and reliability-aware integration. Across challenging probabilistic supervision settings, experiments on diverse benchmarks show that ECI-PP outperforms state-of-the-art DCC methods and remains robust with a shared default configuration.
Tags
Links
- Source: https://arxiv.org/abs/2608.12027v1
- Canonical: https://arxiv.org/abs/2608.12027v1
Trouble viewing inline? Open PDF directly →
Full Text
165,847 characters extracted from source content.
Expand or collapse full text
Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision Shaojie Zhang Ke Chen Affiliation: Department of Computer Science, The University of Manchester, Manchester M13 9PL, U.K. Email: shaojie.zhang,ke.chen@manchester.ac.uk Abstract Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and stochastic corruption. Existing deep constrained clustering (DCC) methods mainly target hard, expert-agnostic constraints, treating soft labels mostly numerically rather than semantically. We formalize this setting as uncertainty-aware probabilistic constrained clustering (UPCC), defining a canonical aleatoric target through a heterogeneous observation process and analyzing its conditional identifiability. We introduce ProbPair, an angular pairwise objective for probabilistic relations, and build ECI-P, an estimator–corrector–integrator framework that refines imperfect supervision via belief estimation, correction, and reliability-aware integration. Across challenging probabilistic supervision settings, experiments on diverse benchmarks show that ECI-P outperforms state-of-the-art DCC methods and remains robust with a shared default configuration. 1 Introduction Clustering is a fundamental tool for uncovering structure in data, yet its unsupervised nature often yields partitions that may not align with domain knowledge [33, 34, 55]; pairwise constrained clustering (C) addresses this by using weak instance-level pairwise supervision [10, 15]. Traditional C methods [46, 45, 32, 3, 2, 24], however, often struggle with high-dimensional and complex data, motivating deep constrained clustering (DCC). Existing DCC methods can be broadly organized into two paradigms: end-to-end DCC [20, 40, 54, 37], which reformulates clustering as a pseudo-classification problem with anchors, and deep constraint embedding [14, 38], which instead learns clustering-friendly representations from pairwise supervision and, in more recent advances, further moves pairwise learning from Euclidean to angular space to better reconcile positive and negative relations [55]. Despite these advances, the supervision setting remains largely idealized. Existing work typically assumes hard binary pairwise relations, whereas supervision in practice often lies on a continuum. Moreover, such real-valued pairwise labels may simultaneously reflect intrinsic aleatoric ambiguity, expert-conditioned and pair-dependent epistemic judgment [22], and additional stochastic corruption from annotation or recording pipelines. For example, when comparing two clinical cases, a recorded observation may reflect ambiguity in the cases themselves, systematic clinician judgment bias, and documentation or data-entry errors. These considerations motivate what we term uncertainty-aware probabilistic constrained clustering (UPCC): a more specific and realistic setting in which probabilistic pairwise relations arise from a heterogeneous observation process. In this paper, we formalize the UPCC setting and develop a corresponding solution. We specify an observation model that distinguishes the aleatoric target from expert-conditioned judgment and stochastic corruption, and analyze when this canonical target is identifiable on observable pairs. To learn under UPCC, we first introduce ProbPair, a deep constraint embedding approach linking probabilistic pairwise relations to angular representation learning. Built on it, we further develop a Estimator–Corrector–Integrator ProbPair (ECI-P) framework, in which Estimators provide model beliefs, Correctors perform structured correction and suppress residual discordance, and the Integrator learns from the resulting surrogate supervision toward the underlying aleatoric relation. Our main contributions are: (i) We formalize UPCC through a heterogeneous observation model and analyze identifiability of the canonical aleatoric target on observable pairs. (i) We introduce ProbPair, a deep constraint embedding approach linking probabilistic pairwise relations to angular embeddings. (i) We develop ECI-P, a practical framework for aleatoric relation learning under entangled supervision. (iv) We empirically show across diverse benchmarks that ECI-P outperforms state-of-the-art DCC methods and is robust under fallible, heterogeneous, and corrupted supervision. 2 Related work Beyond binary pairwise constraints. Existing C methods either extend binary constraints only numerically, without semantically modeling intermediate supervision, or assign intermediate values to the confidence of a pre-defined relation type rather than a unified probability. Advanced deep methods typically employ logistic losses [21, 55] compatible with real-valued constraints in [0,1][0,1]. However, end-to-end DCC methods [20, 54, 42, 43, 16, 37] remain tied to hard anchor semantics, with attendant anchor misalignment [49] and error propagation [9, 31], while deep constraint embedding methods [55, 38, 14] avoid anchors but still require a binary regime for their theoretical geometric guarantees. Likewise, the deep generative approach [34] encodes constraints as a signed prior over discrete assignments, precluding a unified treatment of real-valued relations. Earlier non-deep approaches considered intermediate supervision more explicitly: (i) probabilistic treatments [26, 25] encoded the activation probability of a positive same-cluster tie, but offered neither negative relations nor a unified notion of pairwise affinity; and (i) fuzzy-constraint formulations attach an intermediate belief [13], degree [47], or penalty [35, 36, 1, 28] only after a discrete relation type has been specified. Taken together, these lines of work do not address uncertain relational judgment expressed by probabilistic labels under a unified semantics. By contrast, our ProbPair formulation, built on deep constraint embedding, directly models such unified probabilistic supervision, accommodating both vague annotations in practice and inherently ambiguous aleatoric relations. Imperfect pairwise supervision. While imperfect annotations in C have long been recognized [39, 8, 56, 30, 5, 53, 33, 34, 37], more realistic observation settings remain underexplored. Existing studies treat simplified noise through constraint-set robustness [39, 56], coarse prior uncertainty over assignments [34], or expert-level reliability in semi-crowdsourced settings [5, 33]. VolMaxDCC [37] further introduces confusion-based structured annotation modeling beyond unstructured label degradation or flipping. However, it still lacks heterogeneous multi-expert modeling and remains restricted to discrete binary-noise settings, leaving expert-specific, pair-dependent epistemic distortion unmodeled. By contrast, we consider genuinely entangled observations, arising from intrinsic aleatoric relations, expert-conditioned epistemic judgments, and stochastic corruption. Such a setting is reminiscent of learning from crowds, where expert behavior is commonly modeled under a fixed class-label scheme through expert-specific confusion or output layers [11, 41], or through expertise coupled with instance difficulty [50, 48]. Yet these strategies rely on a restricted class-label channel and are therefore not directly transferable to deep constraint embedding, where a highly flexible pairwise learner can readily absorb structured distortion and random corruption into its learned geometry. We therefore develop ECI-P, a multi-role iterative framework built on ProbPair that uses model-side beliefs to correct structured distortion and filter residual inconsistency, yielding refined surrogate supervision that progressively guides learning toward the underlying aleatoric relation. 3 UPCC formulation and canonical target We study uncertainty-aware probabilistic constrained clustering (UPCC), where expert-indexed real-valued pairwise labels arise from a heterogeneous observation process. Let =xjj=1||X=\x_j\_j=1^|X| be a dataset to be partitioned into C clusters, and let =(ai,bi,ei,yi)i=1||C=\(a_i,b_i,e_i,y_i)\_i=1^|C| denote the observed soft constraints, where (ai,bi)(a_i,b_i) indexes the pair (xai,xbi)(x_a_i,x_b_i), ei∈1,…,Ee_i∈\1,…,E\ records the expert identity, and yi∈[0,1]y_i∈[0,1] is the observed pairwise relation. The formulation targets a canonical pairwise probabilistic relation, while a final hard partition is treated as a downstream summary induced by that relation. 3.1 Observation model We view each observed pairwise relation as arising from three factors: (i) an underlying aleatoric relation in the data (e.g., intrinsic clinical similarities between a common cold and the flu), (i) an expert-conditioned epistemic judgment built on that relation (e.g., a novice doctor systematically overrating their similarity due to limited knowledge), and (i) a subsequent stochastic corruption stage accounting for accidental recording noise (e.g., random clerical errors in medical records). Aleatoric relation. Let S be the set of all C-partitions of X, and let S∈S be a latent random partition. For any index pair (a,b)(a,b), define the aleatoric co-membership relation Rab⋆:=Pr(xa and xb belong to the same cluster under S∣)∈[0,1].R _ab:= \! ($x_a$ and $x_b$ belong to the same cluster under S )∈[0,1]. This quantity serves as the canonical target of the formulation and remains well defined even when the latent partition is intrinsically ambiguous. Epistemic judgment. Let σ(⋅)σ(·) denote the sigmoid function, and define its clamped inverse logitε(u):=logit(minmaxu,ε,1−ε) _ (u):= \! ( \ \u, \,1- \ ) with ε≪1 1. We also introduce a deterministic pair descriptor ϕab=ϕ(xa,xb) _ab=φ(x_a,x_b), which serves as the covariate through which expert-dependent effects are parameterized. For expert e, let me(ϕab)∈ℝm_e( _ab) denote a structured mean effect, and let ue,ab|e,ϕab∼(0,τe2(ϕab)),τe(ϕab)>0,u_e,ab e, _ab \! (0, _e^2( _ab) ), _e( _ab)>0, be a centered residual perturbation. The latent expert judgment is generated as ye,abjud:=σ(logitε(Rab⋆)+me(ϕab)+ue,ab)∈(0,1).y^jud_e,ab:=σ\! ( _ (R _ab)+m_e( _ab)+u_e,ab )∈(0,1). (1) Under this construction, the judgment channel has the logistic-normal density pjud(y∣Rab⋆,e,ϕab)=1y(1−y)φ(logit(y),logitε(Rab⋆)+me(ϕab),τe2(ϕab))p_jud (y R _ab,e, _ab )= 1y(1-y)\, \! ( (y);\, _ (R _ab)+m_e( _ab),\, _e^2( _ab) ) (2) for y∈(0,1)y∈(0,1), where φ(⋅,μ,s2) (·;μ,s^2) denotes the Gaussian density; see Appendix A.1 for the derivation. The same judgment channel also gives rise to the corresponding mean distortion in probability space, ϵ¯e,ab(Rab⋆,ϕab):=[ye,abjud−Rab⋆∣Rab⋆,e,ϕab]. ε_e,ab(R _ab, _ab):=E\! [y^jud_e,ab-R _ab R _ab,e, _ab ]. (3) Stochastic corruption. To capture unstructured recording noise beyond the latent judgment, we introduce a corruption indicator ce,ab∼Bernoulli(πe)c_e,ab ( _e), where πe∈[0,1) _e∈[0,1) is the corruption probability for expert e, and define the final observed relation by ye,ab∣ce,ab=0∼pjud(⋅∣Rab⋆,e,ϕab),ye,ab∣ce,ab=1∼Uniform(0,1).y_e,ab c_e,ab=0 p_jud(· R_ab ,e, _ab), y_e,ab c_e,ab=1 (0,1). Equivalently, letting [0,1](⋅)1_[0,1](·) denote the uniform density on [0,1][0,1], the conditional observation law is p(y∣Rab⋆,e,ϕab)=(1−πe)pjud(y∣Rab⋆,e,ϕab)+πe 1[0,1](y).p (y R_ab ,e, _ab )=(1- _e)\,p_jud (y R_ab ,e, _ab )+ _e\,1_[0,1](y). (4) 3.2 Canonical target and identifiability We now formalize the canonical target within a structural setup for identifiability. To obtain a finite-dimensional canonical estimand, we parameterize Rab⋆=rθ(xa,xb),me(ϕab)=mηe(ϕab),τe(ϕab)=τζe(ϕab),R _ab=r_θ(x_a,x_b), m_e( _ab)=m_ _e( _ab), _e( _ab)= _ _e( _ab), and collect the parameters into Ξ=(θ,ηee=1E,ζee=1E,πee=1E) =(θ,\ _e\_e=1^E,\ _e\_e=1^E,\ _e\_e=1^E). This parameterization makes the canonical target explicit under the specific observation family. Since ϕab _ab is introduced only as a deterministic pair descriptor without additional representational constraints, the identification of the canonical relation term rθ(xa,xb)r_θ(x_a,x_b) is governed by the structural conditions introduced below. We first fix the trivial additive ambiguity in the location decomposition: Assumption 3.1 (Centering on observable support). For each expert e with nonzero observation probability, (a,b)|e[mηe(ϕab)]=0E_(a,b) e[m_ _e( _ab)]=0, where the expectation is taken over the observable-pair distribution conditional on expert e. This centering removes any remaining expert-specific constant shift on the observable support: Lemma 3.2 (Gauge fixing). Suppose both mηee=1E\m_ _e\_e=1^E and mηe′e=1E\m_η _e\_e=1^E satisfy Assumption 3.1. If for each expert e with nonzero observation probability there exists a constant ke∈ℝk_e such that mηe(ϕab)=mηe′(ϕab)−kem_ _e( _ab)=m_η _e( _ab)-k_e for all observable triples (a,b,e)(a,b,e), then ke=0k_e=0 for every such expert e. The proof is given in Appendix A.2. The logistic-normal clean channel in Eq. 2, combined with the observation law in Eq. 4, yields the following injectivity property: Lemma 3.3 (Injectivity of the observation family). Let μ:=logitε(Rab⋆)+me(ϕab)μ:= _ (R _ab)+m_e( _ab), s:=τe(ϕab)s:= _e( _ab), and π:=πeπ:= _e. For the observation law rewritten as p(y∣μ,s,π):=(1−π)1y(1−y)φ(logit(y),μ,s2)+π 1[0,1](y),y∈(0,1),p(y μ,s,π):=(1-π)\, 1y(1-y)\, \! ( (y);μ,s^2 )+π\,1_[0,1](y), y∈(0,1), (5) the mapping (μ,s,π)↦p(⋅∣μ,s,π)(μ,s,π) p(· μ,s,π) is injective on ℝ×(0,∞)×[0,1)R×(0,∞)×[0,1). The proof is given in Appendix A.3. The canonical formulation is completed by the following structural separability condition on the location term logitε(rθ(xa,xb))+mηe(ϕab) _ \! (r_θ(x_a,x_b) )+m_ _e( _ab): Assumption 3.4 (Structural separability). Under Assumption 3.1, if for all observable triples (a,b,e)(a,b,e), logitε(rθ(xa,xb))+mηe(ϕab)=logitε(rθ′(xa,xb))+mηe′(ϕab), _ \! (r_θ(x_a,x_b) )+m_ _e( _ab)= _ \! (r_θ (x_a,x_b) )+m_η _e( _ab), then for each expert e there exists a constant ke∈ℝk_e such that for all such observable triples, logitε(rθ(xa,xb))=logitε(rθ′(xa,xb))+ke,mηe(ϕab)=mηe′(ϕab)−ke. _ \! (r_θ(x_a,x_b) )= _ \! (r_θ (x_a,x_b) )+k_e, m_ _e( _ab)=m_η _e( _ab)-k_e. Assumption 3.4 provides the key structural requirement for the identifiability argument; boundary cases clarifying this condition are discussed in Appendix A.4. Writing the support of observable pairs as obs:=(a,b):∃e such that (a,b,e) is observableP_obs:=\(a,b):∃ e such that (a,b,e) is observable\, we then have the identifiability result: Theorem 3.5 (Conditional canonical identifiability of the aleatoric relation). Let Ξ and Ξ′ satisfy the observation model and Assumption 3.1 and 3.4. Assume further that rθ(xa,xb),rθ′(xa,xb)∈[ε,1−ε]r_θ(x_a,x_b),\,r_θ (x_a,x_b)∈[ ,1- ] for all (a,b)∈obs(a,b) _obs. If the induced conditional laws of the observed response coincide under Ξ and Ξ′ for every observable triple (a,b,e)(a,b,e), then rθ(xa,xb)=rθ′(xa,xb)r_θ(x_a,x_b)=r_θ (x_a,x_b) for all (a,b)∈obs(a,b) _obs. Hence the canonical aleatoric relation Rab⋆=rθ(xa,xb)R _ab=r_θ(x_a,x_b) is identifiable on observable pairs. The proof is in Appendix A.5. Under the specific observation family, Theorem 3.5 gives conditional identifiability of the canonical aleatoric relation on observable pairs at the law level. In the practical single-observation regime, however, estimation still relies on the shared parametric structure imposed across pairs and experts rather than repeated observations of the same pair–expert combination. 4 Method We first introduce a probability-aware pairwise learning formulation that links probabilistic pairwise relations to clustering-oriented representations. Building on this formulation and motivated by the theoretical observation model, we develop a practical learning framework in which a surrogate supervision route guides learning from entangled supervision toward the aleatoric relation. 4.1 ProbPair: probability-aware pairwise representation learning We establish a representation learning formulation under probabilistic pairwise supervision. Given supervised pairs (ai,bi,yi)i=1||\(a_i,b_i,y_i)\_i=1^|C|, where yi∈[0,1]y_i∈[0,1] denotes the probabilistic relation for pair (xai,xbi)(x_a_i,x_b_i), we aim to learn clustering-oriented representations. To this end, we adopt a deep constraint embedding formulation with encoder fψ:→⊂ℝDf_ψ:X ^D and latent code zj=fψ(xj)z_j=f_ψ(x_j). We work in angular space, where bounded cosine similarity provides a natural geometric quantity for probabilistic calibration. To connect probabilistic pairwise relations with the latent representation space, we introduce the following probabilistic readout: y^aibi:=σ((cos(zai,zbi)−m)/T)∈(0,1), y_a_ib_i:=σ (( (z_a_i,z_b_i)-m)/T )∈(0,1), (6) where m∈ℝm and T>0T>0 are learnable parameters controlling the operating point and sharpness of the mapping. The resulting pairwise objective, referred to as the ProbPair loss, is given by ℒP=−1||∑i=1||[yilogy^aibi+(1−yi)log(1−y^aibi)].L_P=- 1|C| _i=1^|C| [y_i y_a_ib_i+(1-y_i) (1- y_a_ib_i) ]. (7) Unlike hard-constraint objectives, Eq. 7 directly accommodates non-binary relation strengths. As a result, the induced geometry need not collapse to a purely binary regime, but can preserve both confident and uncertain pairwise structure. We further include a reconstruction regularizer to avoid degenerate representations and preserve the global structure of the data. Since the representation is learned in angular space, reconstruction is performed from normalized latent embeddings, as in [55]. Specifically, with decoder gψ′:→g_ψ :Z and Norm(⋅)Norm(·) denoting ℓ2 _2 normalization, we decode from x^j=gψ′(Norm(zj)) x_j=g_ψ (Norm(z_j)) and use reconstruction loss ℒrec=1||∑j=1||‖xj−x^j‖22L_rec= 1|X| _j=1^|X|\|x_j- x_j\|_2^2. With λrec>0 _rec>0 balancing pairwise relation fitting and reconstruction regularization, the overall objective is ℒ=ℒP+λrecℒrec.L=L_P+ _recL_rec. (8) Minimizing Eq. 8 yields optimal normalized embeddings norm=Norm(zj)j=1||Z_norm\!=\!\Norm(z_j)\_j=1^|X| that are shaped by probabilistic pairwise relations while being regularized by reconstruction. These angular embeddings can then be used by a downstream clustering algorithm to obtain the final partition, and can also produce probabilistic relation estimates for instance pairs through Eq. 6. 4.2 Estimator–Corrector–Integrator ProbPair Building on the above formulation, we develop a practical framework motivated by the observation model. Rather than learning directly from entangled supervision, it progressively refines the supervision signal by correcting the structured observational component, screening the residual inconsistency, and feeding the refined supervision back into learning, thereby yielding an iterative Estimator–Corrector–Integrator ProbPair (ECI-P) design for learning toward the aleatoric relation. The overall ECI-P flow is summarized in Fig. 1. Figure 1: Overview of the ECI-P pipeline. The Estimators are trained with the ProbPair objective ℒL (Eq. 8) and produce out-of-fold beliefs y^ioof y_i^oof via the readout in Eq. 6. The Correctors take [y^ioof;ϕi][ y_i^oof; _i] as input, with ϕi _i the pair-i descriptor, and learn probability-space corrections Δ^i _i via Eq. 10, yielding corrected relations yicory_i^cor. Reliability-aware screening produces reliability weights wiw_i, and BC fusion forms the refined supervision yiBCy_i^BC via Eqs. 11 and 12. The Integrator learns embeddings from the refined supervision; its Integrator-side relations yiinty_i^int are fed back to the Estimators for iterative refinement, while the final embeddings intZ^int are clustered into the final partition. Estimator. To enable the subsequent learning of structured observational effects from entangled supervision, ECI-P first constructs a model-side reference that is less coupled to the raw observations. In flexible pairwise representation learning, beliefs fitted on the same constraints used for training can absorb noisy observations in-sample and are therefore unreliable for this role. We accordingly partition C into K disjoint folds (k)k=1K\C^(k)\_k=1^K and instantiate K non-shared Estimators Est(k)k=1K\Est^(k)\_k=1^K, where Est(k)=(fωk,gωk′,mk,Tk)Est^(k)\!=\!(f_ _k,g_ω _k,m_k,T_k) denotes the encoder, decoder, and fold-specific readout parameters. Each Est(k)Est^(k) is trained on ∖(k)C ^(k) by minimizing ℒL in Eq. 8, and then produces holdout beliefs on (k)C^(k) through Eq. 6. Aggregating these predictions over all folds yields a full set of out-of-fold beliefs y^ioofi=1||\ y_i^oof\_i=1^|C|, which serves as the model-side reference. To initialize the estimator, we weight each observed pair in ProbPair loss by an information-based decisiveness score, defined as κi=DKL(Bern(yi)∥Bern(y¯))(1−yi)DKL(Bern(0)∥Bern(y¯))+yiDKL(Bern(1)∥Bern(y¯))∈[0,1], _i= D_KL\! (Bern(y_i)\,\|\,Bern( y) )(1-y_i)\,D_KL\! (Bern(0)\,\|\,Bern( y) )+y_i\,D_KL\! (Bern(1)\,\|\,Bern( y) )∈[0,1], (9) where y¯=1||∑i=1||yi y= 1|C| _i=1^|C|y_i, DKL(⋅∥⋅)D_KL(·\|·) denotes the Kullback–Leibler divergence, and Bern(⋅)Bern(·) denotes the Bernoulli law. This weighting emphasizes more decisive judgments, which are typically expressed with greater expert confidence, and thereby stabilizes the initial out-of-fold beliefs. Corrector. Given the Estimators’ out-of-fold beliefs, the Corrector aims to learn structured expert-specific corrections in the observed supervision. These corrections are motivated by the epistemic component defined in logit space (Section 3.1), whereas for practical stability we work with its mean probability-space distortion ϵ¯e,ab(Rab⋆,ϕab) ε_e,ab(R _ab, _ab) (see Eq. 3), because small probability discrepancies near the 0/10/1 boundaries induce disproportionately large logit residuals. Although approximate and not preserving the variance structure encoded by τe(ϕab) _e( _ab), ϵ¯e,ab(Rab⋆,ϕab) ε_e,ab(R _ab, _ab) remains a reasonable proxy because it captures the structured mean effect induced by the epistemic channel, making the resulting Corrector a practical surrogate module. We take y^ioof y_i^oof as the current proxy for Raibi⋆R_a_ib_i and introduce expert-specific Correctors Coree=1E\Cor_e\_e=1^E in probability space, where each CoreCor_e is parameterized by a network qνeq_ _e on ℐe:=i:ei=eI_e:=\i\!:\!e_i\!=\!e\ with input [y^ioof;ϕi][ y_i^oof; _i]; here we instantiate ϕi=[xai⊙xbi;|xai−xbi|] _i=[x_a_i\! \!x_b_i;\,|x_a_i\!-\!x_b_i|] with ⊙ denoting element-wise product, as a simple and stable generic pair feature. Writing Δi:=y^ioof−yi∈(−1,1) _i:= y_i^oof-y_i∈(-1,1), the Corrector output is Δ^i:=qνei([y^ioof;ϕi])∈(−1,1) _i:=q_ _e_i([ y_i^oof; _i])∈(-1,1). Since Δi _i is an empirical target formed from the observed yiy_i and the proxy y^ioof y_i^oof, it may also contain stochastic corruption and proxy error in addition to structured epistemic effects. We therefore train each small expert-specific Corrector with regularization: ℒcor(e)=1|ℐe|∑i∈ℐe[ρ(Δ^i−Δi)+λcor|Δ^i|2].L_cor^(e)= 1|I_e| _i _e [ρ( _i- _i)+ _cor| _i|^2 ]. (10) Here, ρ(⋅)ρ(·) is the Huber loss and λcor>0 _cor>0 weights the ℓ2 _2 regularizer, which biases the limited-capacity Corrector toward shared learnable structure rather than arbitrary pair-specific fluctuations. Integrator. The Integrator, parameterized as Int=(fω,gω′,m,T)Int=(f_ω,g_ω ,m,T), is the learner that integrates the refined supervision into a clustering-oriented embedding. To build this refined supervision from the Corrector output, we first form a corrected relation yicory_i^cor, quantify the remaining inconsistency by gapigap_i, and convert it into a screening weight wiw_i, with γ>0γ>0 controlling the sharpness of down-weighting: yicor:=softclipξ(yi+Δ^i),gapi:=|softclipξ(y^ioof)−yicor|,wi:=(1−gapi)γ.y_i^cor:=softclip_ξ(y_i+ _i), _i:= |softclip_ξ( y_i^oof)-y_i^cor |, w_i:=(1-gap_i)^γ. (11) Here, softclipξ(u):=ξ−1softplus(ξu)−ξ−1softplus(ξ(u−1))softclip_ξ(u):=ξ^-1softplus(ξ u)-ξ^-1softplus(ξ(u-1)) with ξ>0ξ>0 softly maps values into (0,1)(0,1) while preserving gradient flow near the boundaries; applying the same map to y^ioof y_i^oof keeps the gap computation in the same bounded space. The residual gapigap_i quantifies the unexplained discrepancy that remains after structured correction. While the observation model associates this discrepancy with stochastic corruption and residual epistemic variation, gapigap_i may also reflect proxy error in y^ioof y_i^oof and imperfect Corrector fitting in practice. We therefore treat it uniformly as a reliability signal for heuristic screening through wiw_i. We then set ni:=n0win_i:=n_0w_i with n0>0n_0>0, and construct the Bayesian-confidence supervision yiBC:=(yi+niyicor)/(1+ni),y_i^BC:=(y_i+n_i\,y_i^cor)/(1+n_i), (12) which keeps yiy_i as a fixed prior while letting yicory_i^cor contribute according to its screened reliability. The Integrator then optimizes ℒL (Eq. 8) using yiBCy_i^BC as the pairwise target, with wiw_i multiplying each per-constraint term in ℒPL_P (Eq. 7) before averaging, yielding Integrator embeddings. Iterative refinement and deployment. ECI-P operates by passing supervision through the Estimator–Corrector–Integrator pipeline to produce, via Eq. 6, Integrator-side relations yiinty_i^int, and then feeding (yiint,wi)(y_i^int,w_i) back to the Estimators for iterative refinement. Warm-up initializes this loop by starting the Estimators from (yi,κi)(y_i, _i); subsequent rounds replace this with (yiint,wi)(y_i^int,w_i). At each round, the refreshed Estimator beliefs are passed again through the Corrector–Integrator pipeline to update yicory_i^cor, gapigap_i, wiw_i, and yiBCy_i^BC, thereby yielding a practical finite-stage refinement procedure that progressively improves the surrogate supervision toward the aleatoric relation. After refinement, the final normalized Integrator embeddings int:=Norm(fω(xj))j=1||Z^int:=\Norm(f_ω(x_j))\_j=1^|X| encode the learned pairwise relation structure; applying an unsupervised clustering algorithm to intZ^int then yields the final partition. Appendix B provides algorithmic details and complexity analysis. 5 Experiments 5.1 Experimental settings Our experiments evaluate ECI-P under UPCC along four axes: (i) comparison with DCC and probabilistic pairwise baselines, (i) robustness to expert quality, corruption, and multi-expert variation, (i) held-out diagnostics of internal dynamics, and (iv) sensitivity and ablation of key designs. Datasets. We adopt six image benchmarks (CIFAR100-20, CIFAR10 [23], FMNIST [51], ImageNet10 [4], MNIST [27], STL10 [7]) and two text benchmarks (Reuters subset [52], RCV1-10 [55]), covering varying class counts and class imbalance. Dataset details and splits are in Appendix C.1. Compared methods. We compare ECI-P with six baselines applicable to probabilistic supervision: VanillaDCC [21] as a basic logistic pairwise baseline, VolMaxDCC [37] with explicit noise modeling, CIDEC [54] and SpherePair [55] as state-of-the-art end-to-end DCC and deep constraint embedding methods, respectively, and ProbPair/Weighted ProbPair as direct ablations. ProbPair uses only ℒL in Eq. 8, while Weighted ProbPair additionally uses the information-based weight in Eq. 9. Constraint generation. To instantiate UPCC, we generate constraints with trained expert classifiers rather than deriving binary labels from ground-truth classes. Experts are trained on controlled labeled subsets; uniformly sampled constraint pairs assigned to expert e are given a judgment ye,abjud=⟨ae,be⟩∈[0,1]y^jud_e,ab= p^e_a, p^e_b ∈[0,1] from the predictive distributions, and the recorded yiy_i is then produced through the corruption channel in Section 3.1. For single-expert supervision, settings lv0.1/lv0.01/lv0.001 vary the labeled-data fraction used to train the expert. Multi-expert settings multi2/multi3/multi10 use multiple experts with complementary familiar/unfamiliar categories, where unfamiliar categories act as expert blind spots. See Appendix C.3 for the full procedure. Protocol. For a comparative study, we report clustering metrics (ACC/NMI/ARI) over five trials; embedding-based methods use K-means on learned representations, while end-to-end DCC baselines use native outputs. The main comparison uses the default lv0.01 single-expert and multi3 multi-expert regimes, both with corruption probability 0.30.3 and 9k constraints in total (33k per expert under multi3). To assess robustness, we vary single-expert quality, corruption probability, and multi-expert configuration with matched constraint budgets. To probe ECI-P beyond final clustering scores, we conduct held-out internal-signal diagnostics and sensitivity/ablation studies. Implementation. For fair comparison, we follow established fully connected architectures and the unsupervised pretraining protocol in [55, 37]: encoder-based methods use a 500500–500500–20002000 backbone with embedding dimension 1010 except 2020 for CIFAR100-20, while VanillaDCC and VolMaxDCC use their native 512512–512512 classifier architectures. For baselines, we use reported best settings where available; VolMaxDCC follows its original validation-based search with 1,0001,000 training samples held out. Except in sensitivity/ablation studies, ECI-P uses one global setting without dataset-specific tuning: 55 estimator folds, Corrector 6464–1616, λcor=0.5 _cor=0.5, γ=10γ=10, n0=10n_0=10, ξ=20ξ=20, and λrec=0.02 _rec=0.02. Full experimental details are provided in Appendix C to support reproducibility. 5.2 Experimental results Main comparison. Table 1 compares our ECI-P against baselines under the lv0.01 single-expert and multi3 multi-expert settings, both with corruption probability 0.30.3 and 9k constraints; additional constraint-budget results appear in Appendix D.1. Overall, ECI-P is strongest, ranking first in 4141 out of 4848 test entries (22 expert regimes×88 datasets×33 metrics) and within the top two in 46/4846/48, while one of the ProbPair-family methods is best in 45/4845/48, supporting the ProbPair formulation under UPCC. Under single-expert supervision, ECI-P outperforms the strong non-ProbPair baseline, SpherePair, in most entries (44–10%10\% absolute NMI margins); further comparisons without pretraining make this pattern clearer (Appendix D.2). Under multi-expert supervision, the NMI margin over SpherePair widens to 66–32%32\%. Appendix D.3 further compares baselines with an expert-aware ensemble wrapper. As an ablation signal, Weighted ProbPair shows a clearer advantage over ProbPair in the multi-expert setting, suggesting that information-based weighting is more useful when low-decisiveness constraints can indicate expert unfamiliarity. A remaining caveat is the severely imbalanced RCV1-10: SpherePair retains the best ACC, reflecting the advantage of its angular geometry for imbalanced pair structures, while ECI-P still gives the best NMI/ARI, indicating stronger grouping consistency. Table 1: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under lv0.01 and multi3 settings, with corruption probability 0.30.3 and 99k constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. Vanilla- DCC VolMax- DCC CIDEC SpherePair ProbPair Weighted ProbPair ECI-P (Ours) Single- expert lv0.01 CIFAR100 ACC 16.8, 17.0 46.1, 45.9 14.3, 13.6 50.2, 50.5 49.4, 49.5 50.3, 50.8 52.7, 52.9 NMI 13.5, 14.6 45.6, 46.1 11.0, 11.2 43.0, 44.2 43.2, 44.2 44.5, 45.7 49.2, 49.9 ARI 6.0, 6.3 30.4, 30.5 2.8, 2.3 32.1, 32.6 30.6, 30.9 31.5, 32.1 36.2, 36.5 CIFAR10 ACC 38.3, 38.7 82.5, 82.5 44.3, 44.0 84.9, 85.7 85.9, 86.0 86.6, 86.7 87.8, 87.8 NMI 29.9, 31.2 71.9, 72.2 36.1, 37.6 72.4, 74.1 75.0, 75.4 76.2, 76.8 79.0, 79.1 ARI 23.2, 24.0 68.3, 68.3 21.1, 21.3 71.2, 72.5 73.0, 73.2 74.3, 74.5 76.7, 76.8 FMNIST ACC 39.8, 40.2 66.5, 65.5 36.1, 35.7 72.0, 71.2 73.6, 72.5 74.6, 73.6 72.4, 71.1 NMI 30.1, 31.3 57.2, 56.8 29.7, 30.5 60.4, 60.9 62.9, 62.9 63.9, 64.0 65.7, 65.0 ARI 21.5, 22.2 46.4, 45.2 15.7, 15.6 53.0, 52.3 54.8, 53.7 55.3, 54.3 55.5, 53.9 ImageNet10 ACC 39.0, 41.5 87.4, 88.1 63.5, 68.3 85.1, 89.1 89.2, 90.5 89.1, 90.3 92.8, 93.1 NMI 27.6, 33.7 77.8, 79.2 47.2, 57.5 71.0, 79.2 80.1, 83.2 80.3, 82.7 88.1, 88.5 ARI 21.1, 25.2 75.2, 76.1 35.5, 42.2 70.7, 78.2 78.5, 80.7 78.4, 80.2 85.8, 86.2 MNIST ACC 38.9, 40.3 75.6, 76.9 47.0, 46.7 87.8, 89.1 89.0, 90.0 89.9, 90.8 90.7, 91.0 NMI 29.5, 32.2 59.3, 61.6 41.4, 43.7 75.4, 78.0 76.9, 79.0 78.3, 80.2 80.7, 81.6 ARI 23.1, 24.8 56.5, 58.7 24.7, 24.2 75.4, 77.8 77.4, 79.4 79.1, 80.9 80.8, 81.5 REUTERS ACC 69.7, 77.0 71.7, 76.8 72.9, 79.3 77.9, 81.8 78.2, 80.8 80.4, 82.7 85.1, 86.6 NMI 30.5, 43.3 34.3, 43.4 35.7, 48.1 48.7, 56.3 53.2, 58.3 54.5, 59.2 61.4, 64.4 ARI 36.8, 50.4 40.9, 50.4 41.8, 54.9 52.9, 61.1 55.5, 61.0 58.6, 63.4 66.6, 70.0 STL10 ACC 35.5, 37.8 79.0, 80.1 57.8, 63.3 76.1, 79.7 81.6, 83.4 81.6, 83.1 85.4, 85.8 NMI 22.9, 27.6 66.7, 69.3 39.3, 48.5 59.5, 66.5 68.2, 72.1 67.9, 71.6 75.8, 76.6 ARI 17.1, 19.8 62.8, 64.7 29.8, 37.3 57.2, 63.3 65.8, 68.9 65.8, 68.4 72.8, 73.4 RCV1-10 ACC 46.2, 46.4 61.3, 61.8 42.5, 44.3 71.2, 71.4 52.7, 53.0 55.2, 55.1 69.0, 68.9 NMI 15.5, 15.9 35.7, 36.7 29.5, 30.0 60.5, 61.3 53.8, 54.1 57.4, 57.5 65.5, 65.8 ARI 19.8, 20.1 44.7, 45.4 26.2, 28.3 61.0, 61.4 43.6, 43.7 47.1, 47.0 62.0, 62.0 Multi- expert multi3 CIFAR100 ACC 15.4, 15.8 42.8, 42.5 12.8, 12.1 44.3, 44.4 43.3, 43.4 46.1, 46.5 48.6, 48.8 NMI 12.8, 13.6 43.4, 43.8 10.0, 10.5 40.5, 41.5 40.0, 40.9 41.8, 43.0 48.1, 48.5 ARI 5.8, 5.8 27.1, 27.1 2.1, 2.0 27.1, 27.6 26.0, 26.2 28.0, 28.5 32.9, 32.9 CIFAR10 ACC 32.4, 32.8 76.5, 76.6 33.2, 31.6 73.8, 74.5 81.3, 81.7 81.4, 81.7 81.6, 82.1 NMI 29.1, 30.9 73.1, 73.2 28.7, 29.1 64.7, 66.5 71.5, 72.3 72.0, 72.6 77.8, 77.8 ARI 21.2, 22.4 65.8, 65.9 12.3, 12.1 59.0, 60.5 67.1, 67.8 67.9, 68.4 73.1, 73.0 FMNIST ACC 35.2, 35.6 59.4, 59.1 28.6, 28.8 61.0, 61.3 66.6, 66.7 68.0, 67.6 65.3, 64.8 NMI 31.4, 32.8 59.2, 59.0 28.3, 30.6 56.6, 57.8 60.7, 61.2 61.7, 62.0 66.0, 65.6 ARI 20.4, 21.3 45.0, 44.3 14.5, 15.4 46.8, 47.4 49.9, 50.0 51.9, 51.8 53.4, 52.8 ImageNet10 ACC 31.8, 34.2 89.9, 90.5 39.4, 38.8 80.7, 85.0 87.4, 89.2 90.1, 91.3 92.4, 92.6 NMI 22.0, 26.8 87.8, 88.7 31.0, 34.8 65.6, 73.8 76.1, 80.1 80.1, 83.0 87.7, 88.1 ARI 15.8, 18.7 83.9, 84.9 14.3, 15.6 63.4, 71.0 75.0, 78.3 79.8, 82.0 85.4, 85.7 MNIST ACC 32.2, 33.3 67.1, 67.8 31.8, 31.0 78.6, 80.0 80.5, 81.8 85.1, 86.1 84.7, 85.1 NMI 26.5, 29.3 57.5, 58.9 28.8, 30.2 70.8, 74.0 72.1, 74.7 75.0, 77.2 79.9, 80.7 ARI 18.4, 20.1 50.9, 52.1 13.2, 13.4 66.3, 69.1 68.7, 71.2 73.4, 75.5 77.1, 77.5 REUTERS ACC 55.3, 59.2 67.5, 68.0 57.3, 59.1 62.1, 69.9 74.5, 79.0 81.0, 83.9 88.3, 89.6 NMI 14.7, 22.8 45.4, 46.5 18.3, 25.7 25.6, 37.2 44.6, 52.8 51.5, 57.9 65.4, 68.7 ARI 19.2, 26.2 47.0, 47.9 20.4, 23.1 28.0, 40.7 50.2, 58.6 57.9, 64.2 72.9, 76.0 STL10 ACC 29.2, 30.7 71.0, 71.1 31.4, 31.6 67.8, 71.4 81.1, 83.0 83.1, 84.3 87.9, 87.7 NMI 21.5, 26.0 70.2, 69.9 24.9, 31.7 53.2, 59.7 66.1, 70.2 69.4, 72.0 78.3, 78.5 ARI 15.2, 17.7 59.5, 59.1 13.6, 16.9 47.8, 53.3 63.7, 67.1 67.4, 69.4 75.9, 75.8 RCV1-10 ACC 44.3, 44.6 48.3, 49.0 32.3, 32.0 55.7, 55.9 43.4, 43.5 47.8, 47.9 53.4, 53.3 NMI 14.3, 14.7 31.4, 32.1 14.1, 13.5 51.9, 52.7 46.6, 47.4 51.6, 52.0 58.9, 59.0 ARI 17.8, 18.0 30.2, 30.9 10.8, 10.3 43.6, 44.2 34.4, 34.7 38.2, 38.4 46.2, 46.1 Figure 2: Test NMI performance (mean± over 5 runs) of all models across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 99k constraints. Robustness across supervision conditions. We further stress-test all methods by varying one supervision factor in UPCC at a time. Around the 99k default supervision in Table 1, Fig. 2 reports test NMI as we vary (A) expert quality, (B) corruption probability, and (C) multi-expert configuration; ACC/ARI results are deferred to Appendix D.4. Across the three sweeps, the trends largely follow the expected supervision-quality ordering: stronger experts, lower corruption, and multi-expert settings with reduced expert blind spots (as detailed in Appendix C.3) generally yield better performance, while the gap between ECI-P and strong ProbPair-family baselines often narrows in easier regimes. The stability advantage of ECI-P is broadly visible across datasets, particularly on FMNIST, ImageNet10, and STL10, and is most pronounced under increasing corruption, where most baselines degrade sharply while ECI-P remains comparatively stable. Overall, these results are consistent with the role of our ECI framework in improving robustness to imperfect and heterogeneous supervision. Figure 3: Held-out diagnostic evolution over 500500 training iterations on MNIST under multi3 with corruption rate 0.30.3 (mean± over 5 runs). (A–C) Brier scores for estimated, corrected, and integrated relations; (D) residual discrepancy between corrected relations and estimator beliefs on uncorrupted constraints; (E,F) AUC/AP measuring the alignment between reliability-aware screening and injected corruption; (G,H) the same screening diagnostic with oracle-generated pairwise supervision before corruption. Held-out diagnostics. We examine whether ECI-P’s internal signals evolve as intended on sample-disjoint held-out constraints. Fig. 3 shows the MNIST case under multi3 with corruption rate 0.30.3, using benchmark labels only to form an external hard co-membership reference for diagnostics, rather than the canonical aleatoric relation R⋆R . Panels (A–C) report Brier scores of the estimated, corrected, and integrated relations, and panel (D) reports the residual discrepancy between corrected relations and estimator beliefs on uncorrupted constraints; lower is better. Panels (E,F) compare reliability-aware screening with the injected corruption indicator using AUC/AP, where higher values indicate better corruption separation. (G,H) repeat the same screening diagnostic with oracle-generated clean supervision before corruption, removing the expert epistemic uncertainty present in (E,F). The curves improve rapidly and then stabilize: relation Brier scores decrease, the post-correction residual discrepancy drops, and the screening signal aligns with corruption above chance, while the oracle-setting curves in (G,H) more closely track corruption as expected. Overall, these held-out trends are consistent with the intended behavior of the ECI framework, from internal relation estimates to reliability-aware screening; full cross-dataset diagnostics are in Appendix D.5. Sensitivity and ablation. We vary the main ECI-P design choices around the default configuration in Section 5.1 to assess sensitivity and ablate key components. The results (see Appendix D.6) show that the default setting is robust over broad ranges: larger K gives only mild gains relative to its cross-fitting cost, information-based warm-up is mildly helpful, small Correctors with moderate regularization are sufficient, and the screening and boundary parameters (γ,n0,ξ)(γ,n_0,ξ) are generally insensitive. Reconstruction strength allows tuning flexibility, but the shared default remains reliable overall. These results indicate that ECI-P is robust without dataset-specific hyperparameter tuning. 6 Conclusion This paper addressed a realistic probabilistic constrained clustering setting, where pairwise supervision entangles aleatoric, epistemic, and stochastic factors. We formalized the constraint-observation process and its conditional identifiability, introduced ProbPair for angular learning from probabilistic relations, and built ECI-P as a practical framework for such supervision. While existing methods largely rely on idealized constraints, ECI-P outperforms state-of-the-art alternatives and remains robust under fallible, heterogeneous, and corrupted supervision without dataset-specific tuning. These results come with several qualifications. The identifiability analysis is population-level and conditional; fully factor-specific finite-sample estimation would require stronger assumptions or additional observations beyond the current surrogate residual structure. Accordingly, the Corrector and reliability signal are practical surrogates rather than estimators of individual latent factors; a more theory-aligned extension would model structured correction, residual epistemic variation, and corruption-aware screening separately. Empirically, our experiments use controlled expert simulations, leaving real-world annotator-provided constraints as a natural next step. Finally, richer Corrector pair descriptors and principled stopping criteria may further improve robustness and training adaptivity. References [1] V. Antoine, B. Quost, M. Masson, and T. Denoeux (2014) CEVCLUS: evidential clustering with instance-level constraints for relational data. Soft Computing 18 (7), p. 1321–1335. Cited by: §2. [2] S. Basu, A. Banerjee, and R. J. Mooney (2004) Active semi-supervision for pairwise constrained clustering. In Proceedings of the 2004 SIAM International Conference on Data Mining, p. 333–344. Cited by: §1. [3] M. Bilenko, S. Basu, and R. J. Mooney (2004) Integrating constraints and metric learning in semi-supervised clustering. In Proceedings of the twenty-first International Conference on Machine Learning, p. 11. Cited by: §1. [4] J. Chang, L. Wang, G. Meng, S. Xiang, and C. Pan (2017) Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, p. 5879–5887. Cited by: §C.1, §5.1. [5] Y. Chang, J. Chen, M. H. Cho, P. J. Castaldi, E. K. Silverman, and J. G. Dy (2017) Multiple clustering views from multiple uncertain experts. In International Conference on Machine Learning, p. 674–683. Cited by: §2. [6] V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins (2018) Double/debiased machine learning for treatment and structural parameters. Econometrics Journal 21, p. C1–C68. Cited by: §B.2. [7] A. Coates, A. Ng, and H. Lee (2011) An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, p. 215–223. Cited by: §C.1, §5.1. [8] T. F. Covoes, E. R. Hruschka, and J. Ghosh (2013) A study of k-means-based algorithms for constrained clustering. Intelligent Data Analysis 17 (3), p. 485–505. Cited by: §2. [9] J. Dai, Z. Ren, Y. Luo, H. Song, and J. Yang (2023) Tensorized anchor graph learning for large-scale multi-view clustering. Cognitive Computation 15 (5), p. 1581–1592. Cited by: §2. [10] I. Davidson and S. Basu (2007) A survey of clustering with instance level constraints. ACM Transactions on Knowledge Discovery from data 1 (1-41), p. 2–42. Cited by: §1. [11] A. P. Dawid and A. M. Skene (1979) Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics) 28 (1), p. 20–28. Cited by: §2. [12] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) Imagenet: a large-scale hierarchical image database. In 2009 IEEE conference on Computer Vision and Pattern Recognition, p. 248–255. Cited by: §C.1. [13] I. Diaz-Valenzuela, M. A. Vila, and M. J. Martin-Bautista (2015) On the use of fuzzy constraints in semisupervised clustering. IEEE Transactions on Fuzzy Systems 24 (4), p. 992–999. Cited by: §2. [14] S. Fogel, H. Averbuch-Elor, D. Cohen-Or, and J. Goldberger (2019) Clustering-driven deep embedding with pairwise constraints. IEEE computer graphics and applications 39 (4), p. 16–27. Cited by: §1, §2. [15] G. González-Almagro, D. Peralta, E. De Poorter, J. Cano, and S. García (2025) Semi-supervised constrained clustering: an in-depth overview, ranked taxonomy and future research directions. Artificial Intelligence Review 58, p. 157. Cited by: §1. [16] J. Goschenhofer, B. Bischl, and Z. Kira (2023) ConstraintMatch for semi-constrained clustering. In 2023 International Joint Conference on Neural Networks (IJCNN), p. 1–10. Cited by: §2. [17] X. Guo, L. Gao, X. Liu, and J. Yin (2017) Improved deep embedded clustering with local structure preservation.. In IJCAI, Vol. 17, p. 1753–1759. Cited by: 1st item, §C.1, §C.2. [18] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778. Cited by: §C.5. [19] G. E. Hinton and R. R. Salakhutdinov (2006) Reducing the dimensionality of data with neural networks. Science 313 (5786), p. 504–507. Cited by: §C.5. [20] Y. Hsu, Z. Lv, J. Schlosser, P. Odom, and Z. Kira (2018) A probabilistic constrained clustering for transfer learning and image category discovery. In Proceedings of the CVPR 2018 Deep-Vision Workshop, Cited by: §1, §2. [21] Y. Hsu, Z. Lv, J. Schlosser, P. Odom, and Z. Kira (2019) Multi-class classification without multi-class labels. In International Conference on Learning Representations, External Links: Link Cited by: §C.2, §2, §5.1. [22] E. Hüllermeier and W. Waegeman (2021) Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning 110 (3), p. 457–506. Cited by: §1. [23] A. Krizhevsky G. Hinton et al. (2009) Learning multiple layers of features from tiny images. Cited by: §C.1, §C.1, §5.1. [24] B. Kulis, S. Basu, I. Dhillon, and R. Mooney (2005) Semi-supervised graph clustering: a kernel approach. In Proceedings of the 22nd International Conference on Machine Learning, p. 457–464. Cited by: §1. [25] M. H. Law, A. Topchy, and A. K. Jain (2004) Clustering with soft and group constraints. In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), p. 662–670. Cited by: §2. [26] M. H. Law, A. Topchy, and A. K. Jain (2005) Model-based clustering with probabilistic constraints. In Proceedings of the 2005 SIAM International Conference on Data Mining, p. 641–645. Cited by: §2. [27] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (11), p. 2278–2324. Cited by: §C.1, §5.1. [28] F. Li, S. Li, and T. Denœux (2018) K-cevclus: constrained evidential clustering of large dissimilarity data. Knowledge-Based Systems 142, p. 29–44. Cited by: §2. [29] Y. Li, P. Hu, Z. Liu, D. Peng, J. T. Zhou, and X. Peng (2021) Contrastive clustering. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35, p. 8547–8555. Cited by: §C.5. [30] H. Liu, Z. Tao, and Y. Fu (2017) Partition level constrained clustering. IEEE transactions on pattern analysis and machine intelligence 40 (10), p. 2469–2483. Cited by: §2. [31] S. Liu, Q. Liao, S. Wang, X. Liu, and E. Zhu (2024) Robust and consistent anchor graph learning for multi-view clustering. IEEE Transactions on Knowledge and Data Engineering. Cited by: §2. [32] Z. Lu and T. Leen (2004) Semi-supervised learning with penalized probabilistic clustering. Advances in Neural Information Processing Systems 17. Cited by: §1. [33] Y. Luo, T. Tian, J. Shi, J. Zhu, and B. Zhang (2018) Semi-crowdsourced clustering with deep generative models. Advances in Neural Information Processing Systems 31. Cited by: §1, §2. [34] L. Manduchi, K. Chin-Cheong, H. Michel, S. Wellmann, and J. Vogt (2021) Deep conditional gaussian mixture model for constrained clustering. Advances in Neural Information Processing Systems 34, p. 11303–11314. Cited by: §C.5, §1, §2, §2. [35] J. Mei, H. Lv, J. Cao, and W. Gong (2019) Pairwise constrained fuzzy clustering: relation, comparison and parallelization. International Journal of Fuzzy Systems 21 (6), p. 1938–1949. Cited by: §2. [36] J. Mei and H. Lv (2018) Semi-supervised fuzzy c-means regularized with pairwise constraints. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), p. 781–786. Cited by: §2. [37] T. Nguyen, S. Ibrahim, and X. Fu (2023) Deep clustering with incomplete noisy pairwise annotations: a geometric regularization approach. In International Conference on Machine Learning, p. 25980–26007. Cited by: 2nd item, §C.1, §C.2, §C.5, §1, §2, §2, §5.1, §5.1. [38] A. Q. Ohi, M. F. Mridha, F. B. Safir, M. A. Hamid, and M. M. Monowar (2020) Autoembedder: a semi-supervised dnn embedding system for clustering. Knowledge-Based Systems 204, p. 106190. Cited by: §1, §2. [39] D. Pelleg and D. Baras (2007) K-means with large and noisy constraint sets. In European Conference on Machine Learning, p. 674–682. Cited by: §2. [40] Y. Ren, K. Hu, X. Dai, L. Pan, S. C. Hoi, and Z. Xu (2019) Semi-supervised deep embedded clustering. Neurocomputing 325, p. 121–130. Cited by: 1st item, §C.1, §C.5, §C.5, §1. [41] F. Rodrigues and F. Pereira (2018) Deep learning from crowds. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §2. [42] M. Śmieja, Ł. Struski, and M. A. Figueiredo (2020) A classification-based approach to semi-supervised clustering with pairwise constraints. Neural Networks 127, p. 193–203. Cited by: §2. [43] T. Tian, J. Zhang, X. Lin, Z. Wei, and H. Hakonarson (2021) Model-based deep embedding for constrained clustering analysis of single cell rna-seq data. Nature communications 12 (1), p. 1873. Cited by: §2. [44] P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol (2008) Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine learning, p. 1096–1103. Cited by: §C.5. [45] K. Wagstaff, C. Cardie, S. Rogers, S. Schrödl, et al. (2001) Constrained k-means clustering with background knowledge. In ICML, Vol. 1, p. 577–584. Cited by: §1. [46] K. Wagstaff and C. Cardie (2000) Clustering with instance-level constraints. AAAI/IAAI 1097 (577-584), p. 197. Cited by: §1. [47] Z. Wang, S. Wang, L. Bai, W. Wang, and Y. Shao (2021) Semisupervised fuzzy clustering with fuzzy pairwise constraints. IEEE Transactions on Fuzzy Systems 30 (9), p. 3797–3811. Cited by: §2. [48] P. Welinder, S. Branson, P. Perona, and S. Belongie (2010) The multidimensional wisdom of crowds. Advances in Neural Information Processing Systems 23. Cited by: §2. [49] Y. Wen, S. Liu, X. Wan, S. Wang, K. Liang, X. Liu, X. Yang, and P. Zhang (2023) Efficient multi-view graph clustering with local and global structure preservation. In Proceedings of the 31st ACM International Conference on Multimedia, p. 3021–3030. Cited by: §2. [50] J. Whitehill, T. Wu, J. Bergsma, J. Movellan, and P. Ruvolo (2009) Whose vote should count more: optimal integration of labels from labelers of unknown expertise. Advances in Neural Information Processing Systems 22. Cited by: §2. [51] H. Xiao, K. Rasul, and R. Vollgraf (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747. Cited by: §C.1, §5.1. [52] J. Xie, R. Girshick, and A. Farhadi (2016) Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning, p. 478–487. Cited by: 1st item, §C.1, §C.1, §C.2, §5.1. [53] H. Zhang, S. Basu, and I. Davidson (2020) A framework for deep constrained clustering-algorithms and advances. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I, p. 57–72. Cited by: §2. [54] H. Zhang, T. Zhan, S. Basu, and I. Davidson (2021) A framework for deep constrained clustering. Data Mining and Knowledge Discovery 35, p. 593–620. Cited by: 1st item, §C.1, §C.2, §C.5, §C.5, §1, §2, §5.1. [55] S. Zhang and K. Chen (2025) Angular constraint embedding via spherepair loss for constrained clustering. In Advances in Neural Information Processing Systems 39, External Links: Link Cited by: 4th item, §C.1, §C.1, §C.2, §C.5, §C.5, §D.6, §1, §2, §4.1, §5.1, §5.1, §5.1, footnote 10. [56] X. Zhu, C. C. Loy, and S. Gong (2015) Constrained clustering with imperfect oracles. IEEE transactions on neural networks and learning systems 27 (6), p. 1345–1357. Cited by: §2. Appendix A Additional theoretical details A.1 Derivation of the logistic-normal judgment density Starting from Equation 1, define μe,ab:=logitε(Rab⋆)+me(ϕab). _e,ab:= _ (R _ab)+m_e( _ab). Conditionally on (Rab⋆,e,ϕab)(R _ab,e, _ab), we then have logit(ye,abjud)=μe,ab+ue,ab,ue,ab∼(0,τe2(ϕab)), (y^jud_e,ab )= _e,ab+u_e,ab, u_e,ab \! (0, _e^2( _ab) ), so that logit(ye,abjud)∼(μe,ab,τe2(ϕab)). (y^jud_e,ab ) \! ( _e,ab, _e^2( _ab) ). For a generic y∈(0,1)y∈(0,1), write v=logit(y)v= (y). By the change-of-variables formula, pjud(y∣Rab⋆,e,ϕab)=φ(v,μe,ab,τe2(ϕab))|dvdy|,v=logit(y),y∈(0,1).p_jud(y R _ab,e, _ab)= \! (v;\, _e,ab,\, _e^2( _ab) ) | dvdy |, v= (y), y∈(0,1). Using dvdy=dlogit(y)dy=1y(1−y) dvdy= d (y)dy= 1y(1-y), we obtain pjud(y∣Rab⋆,e,ϕab)=1y(1−y)φ(logit(y),logitε(Rab⋆)+me(ϕab),τe2(ϕab)),y∈(0,1),p_jud(y R _ab,e, _ab)= 1y(1-y)\, \! ( (y);\, _ (R _ab)+m_e( _ab),\, _e^2( _ab) ), y∈(0,1), which is exactly Equation 2. A.2 Proof of Lemma 3.2 Proof. Fix an expert e with nonzero observation probability. By assumption, mηe(ϕab)=mηe′(ϕab)−kem_ _e( _ab)=m_η _e( _ab)-k_e for all observable triples (a,b,e)(a,b,e). Taking expectation over the observable-pair distribution conditional on e yields (a,b)|e[mηe(ϕab)]=(a,b)|e[mηe′(ϕab)]−ke.E_(a,b) e[m_ _e( _ab)]=E_(a,b) e[m_η _e( _ab)]-k_e. Both expectations vanish by Assumption 3.1, hence ke=0k_e=0.11 1 The centering is imposed conditionally on each expert because the remaining additive ambiguity after fixing the total location is expert-wise: global centering would only fix the average shift across experts, not each expert-specific shift. Thus, under the present assumptions, global centering would not be sufficient for identifiability unless one further assumes that different experts observe sufficiently overlapping pair regions. Since e was arbitrary over experts with nonzero observation probability, the conclusion holds for every such expert. ∎ A.3 Proof of Lemma 3.3 Proof. By Equations 2 and 4, the observation law takes the form Equation 5. Assume that p(⋅∣μ,s,π)=p(⋅∣μ′,s′,π′)p(· μ,s,π)=p(· μ ,s ,π ) on (0,1)(0,1), where p(y∣μ,s,π)p(y μ,s,π) is the density defined in Equation 5. Since [0,1](y)=11_[0,1](y)=1 for y∈(0,1)y∈(0,1), equality of the two densities means that (1−π)1y(1−y)φ(logit(y),μ,s2)+π=(1−π′)1y(1−y)φ(logit(y),μ′,s′2)+π′(1-π)\, 1y(1-y)\, \! ( (y);μ,s^2 )+π=(1-π )\, 1y(1-y)\, \! ( (y);μ ,s ^2 )+π for all y∈(0,1)y∈(0,1). Write v=logit(y)v= (y), so that y=σ(v)y=σ(v) and y(1−y)=σ(v)(1−σ(v))y(1-y)=σ(v)(1-σ(v)). Multiplying both sides by y(1−y)y(1-y) gives (1−π)φ(v,μ,s2)+πσ(v)(1−σ(v))=(1−π′)φ(v,μ′,s′2)+π′σ(v)(1−σ(v))(1-π)\, (v;μ,s^2)+π\,σ(v)(1-σ(v))=(1-π )\, (v;μ ,s ^2)+π \,σ(v)(1-σ(v)) for all v∈ℝv . As v→+∞v→+∞, the Gaussian terms decay faster than e−ve^-v, whereas σ(v)(1−σ(v))=e−v(1+e−v)2∼e−v.σ(v)(1-σ(v))= e^-v(1+e^-v)^2 e^-v. Therefore, limv→+∞ev[(1−π)φ(v,μ,s2)+πσ(v)(1−σ(v))]=π, _v→+∞e^v [(1-π)\, (v;μ,s^2)+π\,σ(v)(1-σ(v)) ]=π, and similarly the right-hand side tends to π′π . Hence π=π′π=π . Subtracting the common contamination term then yields φ(v,μ,s2)=φ(v,μ′,s′2)for all v∈ℝ. (v;μ,s^2)= (v;μ ,s ^2) all v . Injectivity of the Gaussian family implies μ=μ′μ=μ and s=s′s=s . Therefore (μ,s,π)=(μ′,s′,π′)(μ,s,π)=(μ ,s ,π ), proving the claim. ∎ A.4 Boundary cases for structural separability Assumption 3.4 concerns the decomposition of the identifiable location term logitε(rθ(xa,xb))+mηe(ϕab) _ (r_θ(x_a,x_b))+m_ _e( _ab) into a canonical relation component and an expert-specific epistemic component. Two boundary cases clarify this requirement. (i) If the pair descriptor ϕab _ab is uninformative, for example constant over observable pairs for a given expert, then mηe(ϕab)m_ _e( _ab) reduces to an expert-specific constant and cannot capture pair-dependent epistemic effects; the intended epistemic component is then not represented by the chosen model class. (i) Conversely, if ϕab _ab is overly rich and the function class for mηe(ϕab)m_ _e( _ab) overlaps with that of logitε(rθ(xa,xb)) _ (r_θ(x_a,x_b)), then shared pair-dependent variation can be moved between the two terms while preserving their sum. For example, a sufficiently small zero-mean shift lying in this overlap can be added to the canonical location term and subtracted from the expert-specific term without violating centering or changing the observation law. The condition therefore rules out such interchangeability, ensuring that canonical aleatoric variation and structured expert-specific variation play distinct explanatory roles on the observable support. A.5 Proof of Theorem 3.5 Proof. Assume that the induced conditional laws coincide under Ξ and Ξ′ for every observable triple (a,b,e)(a,b,e). For each such triple, define μΞ(a,b,e):=logitε(rθ(xa,xb))+mηe(ϕab),sΞ(a,b,e):=τζe(ϕab), _ (a,b,e):= _ \! (r_θ(x_a,x_b) )+m_ _e( _ab), s_ (a,b,e):= _ _e( _ab), and define μΞ′(a,b,e) _ (a,b,e) and sΞ′(a,b,e)s_ (a,b,e) analogously. By Equations 4 and 5, the conditional law of the observed response for (a,b,e)(a,b,e) is exactly p(y∣μΞ(a,b,e),sΞ(a,b,e),πe),p (y _ (a,b,e),s_ (a,b,e), _e ), and similarly under Ξ′ . Since the two conditional laws coincide, Lemma 3.3 implies that for every observable triple (a,b,e)(a,b,e), μΞ(a,b,e)=μΞ′(a,b,e),sΞ(a,b,e)=sΞ′(a,b,e),πe=πe′. _ (a,b,e)= _ (a,b,e), s_ (a,b,e)=s_ (a,b,e), _e=π _e. In particular, logitε(rθ(xa,xb))+mηe(ϕab)=logitε(rθ′(xa,xb))+mηe′(ϕab) _ \! (r_θ(x_a,x_b) )+m_ _e( _ab)= _ \! (r_θ (x_a,x_b) )+m_η _e( _ab) for all observable triples (a,b,e)(a,b,e). Applying Assumption 3.4, for each expert e there exists a constant ke∈ℝk_e such that for all observable triples (a,b,e)(a,b,e), logitε(rθ(xa,xb))=logitε(rθ′(xa,xb))+ke,mηe(ϕab)=mηe′(ϕab)−ke. _ \! (r_θ(x_a,x_b) )= _ \! (r_θ (x_a,x_b) )+k_e, m_ _e( _ab)=m_η _e( _ab)-k_e. By Lemma 3.2, each kek_e on the observable expert support must be zero. Therefore, logitε(rθ(xa,xb))=logitε(rθ′(xa,xb))for all (a,b)∈obs. _ \! (r_θ(x_a,x_b) )= _ \! (r_θ (x_a,x_b) ) all (a,b) _obs. Since rθ(xa,xb),rθ′(xa,xb)∈[ε,1−ε]r_θ(x_a,x_b),r_θ (x_a,x_b)∈[ ,1- ] on obsP_obs, the clamped logit satisfies logitε(u)=logit(u) _ (u)= (u) on this interval and is therefore injective there. It follows that rθ(xa,xb)=rθ′(xa,xb)for all (a,b)∈obs,r_θ(x_a,x_b)=r_θ (x_a,x_b) all (a,b) _obs, which proves the theorem. ∎ Appendix B Algorithm and complexity analysis B.1 Algorithm We provide the detailed ECI-P procedure introduced in Section 4, summarized in Algorithm 1. Algorithm 1 ECI-P for learning under UPCC 1: training data =xjj=1||X=\x_j\_j=1^|X|; constraints =(ai,bi,ei,yi)i=1||C=\(a_i,b_i,e_i,y_i)\_i=1^|C|; number of experts E; number of estimator folds K; total training epochs NepN_ep; warm-up epochs NwuN_wu; hyperparameters λrec _rec (reconstruction weight), λcor _cor (Corrector regularization), ξ (soft-clipping sharpness), γ (screening sharpness), n0n_0 (Bayesian-confidence scale); clustering routine Clustering(⋅)Clustering(·) 2: Partition C into K disjoint folds (k)k=1K\C^(k)\_k=1^K and define ℐe:=i:ei=eI_e:=\i:e_i=e\ for each expert e 3: Initialize Est(k)=(fωk,gωk′,mk,Tk)k=1K\Est^(k)=(f_ _k,g_ω _k,m_k,T_k)\_k=1^K, Core=qνee=1E\Cor_e=q_ _e\_e=1^E, and Int=(fω,gω′,m,T)Int=(f_ω,g_ω ,m,T) 4: 5: Stage I: Warm-up 6: Compute decisiveness weights κii=1||\ _i\_i=1^|C| by Eq. 9 7: for n=1,…,Nwun=1,…,N_wu do 8: for k=1,…,Kk=1,…,K do 9: Update Est(k)Est^(k) on ∖(k)C\! \!C^(k) by gradient descent on Eq. 8 with yiy_i and weights κi _i 10: end for 11: end for 12: Aggregate holdout predictions from Est(k)k=1K\Est^(k)\_k=1^K to obtain y^ioofi=1||\ y_i^oof\_i=1^|C| via Eq. 6 13: for e=1,…,Ee=1,…,E do 14: Update CoreCor_e on ℐeI_e by gradient descent on Eq. 10 until convergence 15: end for 16: Compute yicor,gapi,wii=1||\y_i^cor,gap_i,w_i\_i=1^|C| by Eq. 11 17: Set ni←n0win_i← n_0w_i and compute Bayesian-confidence supervision yiBCi=1||\y_i^BC\_i=1^|C| by Eq. 12 18: for n=1,…,Nwun=1,…,N_wu do 19: Update IntInt by gradient descent on Eq. 8 with yiBCy_i^BC and weights wiw_i 20: end for 21: 22: Stage I: Iterative refinement 23: for t=Nwu+1,…,Nept=N_wu+1,…,N_ep do 24: Read out Integrator-side relations yiinti=1||\y_i^int\_i=1^|C| from the Integrator embeddings via Eq. 6 25: for k=1,…,Kk=1,…,K do 26: Update Est(k)Est^(k) on ∖(k)C ^(k) by gradient descent on Eq. 8 with yiinty_i^int and weights wiw_i 27: end for 28: Aggregate updated holdout predictions to refresh y^ioofi=1||\ y_i^oof\_i=1^|C| 29: for e=1,…,Ee=1,…,E do 30: Update CoreCor_e on ℐeI_e by gradient descent on Eq. 10 until convergence 31: end for 32: Refresh yicor,gapi,wii=1||\y_i^cor,gap_i,w_i\_i=1^|C| by Eq. 11 33: Set ni←n0win_i← n_0w_i and refresh Bayesian-confidence supervision yiBCi=1||\y_i^BC\_i=1^|C| by Eq. 12 34: Update IntInt by gradient descent on Eq. 8 with yiBCy_i^BC and weights wiw_i 35: end for 36: 37: Stage I: Clustering and unseen-instance assignment 38: Obtain the final normalized Integrator embeddings int←Norm(fω(xj))j=1||Z^int←\Norm(f_ω(x_j))\_j=1^|X| 39: Apply Clustering(int)Clustering(Z^int) to obtain the final partition S S 40: Compute cluster centroids μc\ _c\ from intZ^int and S S 41: for each unseen instance x~ x do 42: Compute z~int←Norm(fω(x~)) z^int (f_ω( x)) 43: Assign x~ x to the nearest centroid in μc\ _c\ 44: end for B.2 Computational complexity analysis For an Estimator/Integrator backbone (f,g,m,T) (f,g,m,T ), let cfc_f and cgc_g denote the per-instance training costs of the encoder f and decoder g, respectively, and let cm,T=O(1)c_m,T=O(1) denote the cost of updating the scalar readout parameters (m,T)(m,T). We write cpair=O(cf+cm,T)c_pair=O(c_f+c_m,T) for the amortized per-constraint cost of the ProbPair-style pairwise term, and crec=O(cf+cg)c_rec=O(c_f+c_g) for the per-instance reconstruction cost. Thus, one backbone epoch over constraints C and instances X costs O(||cpair+||crec)O\! (|C|c_pair+|X|c_rec ). In particular, the pairwise term scales with the number of constraints, while the reconstruction term scales linearly with |||X| as the usual autoencoder regularization cost. Notably, each Corrector Core=qνeCor_e=q_ _e is a small expert-specific MLP, rather than an encoder–decoder backbone, and its capacity is intentionally kept more limited than that of the Estimator/Integrator networks so that it learns only shared structured expert-specific patterns rather than arbitrary fitting. We write ccorc_cor for its amortized per-constraint update cost. Under this notation, with NwuN_wu warm-up epochs and NepN_ep total epochs, the warm-up stage costs O(Nwu(K+1)(||cpair+||crec)+Jcor||ccor),O\! (N_wu(K+1) (|C|c_pair+|X|c_rec )+J_cor|C|c_cor ), where JcorJ_cor is the average number of optimization passes used when fitting the Correctors. Each of the Tref:=Nep−NwuT_ref:=N_ep-N_wu subsequent refinement rounds costs O((K+1)(||cpair+||crec)+Jcor||ccor).O\! ((K+1) (|C|c_pair+|X|c_rec )+J_cor|C|c_cor ). The total training complexity is O((Nwu+Tref)(K+1)(||cpair+||crec)+(1+Tref)Jcor||ccor).O\! ( (N_wu+T_ref )(K+1) (|C|c_pair+|X|c_rec )+ (1+T_ref )J_cor|C|c_cor ). The dominant overhead comes from the K-fold cross-fitting of the Estimators, namely the repeated backbone updates scaled by (K+1)(||cpair+||crec)(K+1) (|C|c_pair+|X|c_rec ). This overhead is justified by the need for out-of-fold beliefs: the Estimators are introduced not as a redundant ensemble, but to provide model-side references decoupled from the specific constraints they are later used to assess, thereby reducing self-confirmation and overfitting to noisy supervision. This use of held-out predictions is analogous in spirit to debiased/double machine learning [6]: both reduce in-sample contamination, although ECI-P uses them to obtain cleaner belief proxies under noisy supervision rather than to pursue semiparametric efficiency. By contrast, the expert-specific correction stage does not scale as O(E||)O(E|C|), because each constraint is routed only to its corresponding expert module and ∑e=1E|ℐe|=|| _e=1^E|I_e|=|C|. Thus, the per-expert Correctors add only a linear-in-|||C| refinement cost, which is largely unavoidable under heterogeneous expert supervision. More broadly, the practical cost profile of ECI-P should be interpreted together with its training protocol: it does not require an additional expert-aware wrapper in multi-expert settings (see Appendix C.4), and unlike methods such as VolMaxDCC, its performance is not tightly tied to an extensive hyperparameter-search stage. Meanwhile, Appendix D.2 shows that ECI-P is less dependent on unsupervised pre-training, which can be advantageous in resource-limited settings, and an overall training-time comparison is reported in Appendix E. A discussion of the choice of K is provided in Appendix D.6. Appendix C Details of experimental settings We provide supplementary information for the experimental setup in Section 5.1, including the benchmark datasets, compared methods, constraint-generation procedure, experimental protocol, and implementation details. C.1 Datasets We use eight benchmark datasets covering image and text clustering tasks, with different numbers of classes and class-balance properties. Their details are as follows. CIFAR100-2022 2 CIFAR100 webpage: https://w.cs.toronto.edu/~kriz/cifar.html [23]: A 20-class version of CIFAR100 obtained by using the 20 superclasses as clustering targets. The original CIFAR100 dataset contains 60,000 real-world 32×3232× 32 color images from 100 fine-grained classes, grouped into 20 superclasses. In our experiments, the 20 superclasses are treated as ground-truth clusters, with 3,000 images per superclass. CIFAR1033 3 CIFAR10 webpage: https://w.cs.toronto.edu/~kriz/cifar.html [23]: A natural-image dataset containing 60,000 32×3232× 32 color images from 10 object categories, with 6,000 images per category. FMNIST44 4 FMNIST repository: https://github.com/zalandoresearch/fashion-mnist [51]: A fashion-product image dataset containing 70,000 grayscale 28×2828× 28 images from 10 categories. The official split contains 60,000 training images and 10,000 test images. ImageNet1055 5 ImageNet webpage: https://image-net.org/ [4]: A 10-class subset of ImageNet [12], containing 13,000 color images in total, with 1,300 images per class. MNIST66 6 MNIST webpage: http://yann.lecun.com/exdb/mnist/ [27]: A handwritten-digit dataset containing 70,000 grayscale 28×2828× 28 images from 10 digit classes. The official split contains 60,000 training images and 10,000 test images. Reuters subset77 7 Preprocessed Reuters subset: https://github.com/piiswrong/dec [52]: A text benchmark derived from the RCV1 corpus88 8 RCV1 webpage: https://trec.nist.gov/data/reuters/reuters.html, using tf–idf features over the 2,000 most frequent words. It contains four root categories, Corporate/Industrial (CCAT), Economics (ECAT), Government/Social (GCAT), and Markets (MCAT), treated as ground-truth clusters. The four categories contain 4,840, 3,470, 2,673, and 1,017 samples, respectively, giving a mildly imbalanced benchmark. The official split contains 10,000 training samples and 2,000 test samples. STL1099 9 STL10 webpage: https://cs.stanford.edu/~acoates/stl10/ [7]: An image dataset containing 13,000 96×9696× 96 color images from 10 object categories, with 1,300 images per category. RCV1-101010 10 Preprocessed RCV1-10 subset released with [55]: https://github.com/spherepaircc/SpherePairCC [55]: An imbalanced 10-category subset of RCV1 constructed from single-label articles in categories C14, C18, C313, C42, E21, E311, GDEF, GODD, GWELF, and M13. It contains 177,669 documents represented by tf–idf features over the 2,000 most frequent word stems. The class distribution is highly skewed, ranging from 903 documents in GWELF to 53,127 documents in M13, making it a benchmark for evaluating clustering under severe class imbalance. These benchmarks follow common evaluation practice in deep clustering and constrained clustering: MNIST, FMNIST, Reuters subset, CIFAR10, ImageNet10, STL10, and CIFAR100-20 have been widely used in prior deep clustering/DCC studies [52, 17, 40, 54, 37, 55], while RCV1-10 follows the imbalanced text benchmark introduced in [55]. For MNIST, FMNIST, and Reuters subset, we use the official train/test splits, yielding 60,000/10,000, 60,000/10,000, and 10,000/2,000 training/test samples, respectively. For CIFAR100-20, CIFAR10, ImageNet10, STL10, and RCV1-10, we randomly split each dataset into 80%80\% training and 20%20\% testing, yielding 48,000/12,000, 48,000/12,000, 10,400/2,600, 10,400/2,600, and 142,135/35,534 training/test samples, respectively. All probabilistic constraints for training are generated from the training split only. C.2 Compared methods We summarize the compared methods and their use under probabilistic pairwise supervision. Methods tied to hard binary constraints or discrete signed priors are not included, as they do not natively operate on the real-valued probabilistic supervision considered in UPCC. VanillaDCC. VanillaDCC [21] is an end-to-end DCC method based on the Meta Classification Likelihood (MCL) loss. Given the soft cluster-assignment vector j=(qj1,…,qjC) q_j=(q_j1,…,q_jC) for each instance xjx_j, the pairwise co-clustering probability of a constrained pair is Pico=aibi⊤P^co_i= q_a_i q_b_i . The MCL loss is ℒMCL=−1||∑i=1||[yilogPico+(1−yi)log(1−Pico)].L_MCL=- 1|C| _i=1^|C| [y_i P^co_i+(1-y_i) (1-P^co_i) ]. Although originally used for binary constraints, this logistic/BCE-form objective can directly take real-valued targets yi∈[0,1]y_i∈[0,1], so VanillaDCC serves as a simple end-to-end DCC baseline under probabilistic supervision. VolMaxDCC. VolMaxDCC [37] extends MCL by introducing an explicit noise-modeling mechanism through a confusion-adjusted co-clustering probability. Concretely, it replaces PicoP^co_i with Pi′co=aiBbi⊤P_i = q_a_iB q_b_i , where B is derived from a learnable confusion matrix, and adds a volume maximization term: ℒVolMax=−1||∑i=1||[yilogPi′co+(1−yi)log(1−Pi′co)]−λlogdet(⊤),L_VolMax=- 1|C| _i=1^|C| [y_i P_i +(1-y_i) (1-P_i ) ]-λ (Q Q), where Q is the assignment matrix. The first term allows the model to account for systematic annotation confusion, while the volume term encourages separated and distinguishable cluster assignments. We therefore include VolMaxDCC as the compared baseline with explicit noise modeling. At the same time, its design is still rooted in noisy binary supervision and assignment-level separation, whereas UPCC requires preserving a continuum of probabilistic pairwise relations rather than only correcting corrupted hard relations. CIDEC. CIDEC [54] extends the DEC/IDEC deep embedding clustering framework [52, 17] by incorporating pairwise constraints through the MCL loss. It uses an autoencoder to obtain latent embeddings, initializes C cluster anchors by K-means, and computes a soft assignment vector j=(qj1,…,qjC) q_j=(q_j1,…,q_jC) for each instance. Its unsupervised clustering term follows the DEC-style KL objective ℒDEC=∑j=1||∑c=1Cpjclogpjcqjc,L_DEC= _j=1^|X| _c=1^Cp_jc p_jcq_jc, where j p_j is the sharpened target distribution derived from j q_j. CIDEC combines this clustering loss with a reconstruction loss and the MCL pairwise constraint loss. In our setting, its MCL component uses the real-valued yiy_i directly through the same BCE-form target as VanillaDCC, making CIDEC a strong end-to-end baseline compatible with probabilistic supervision. SpherePair. SpherePair [55] is a deep constraint embedding method that learns normalized angular representations with an autoencoder. For binary hard constraints, it defines an angular BCE-form loss ℒang=−1||∑i=1||[yilogsaibi++(1−yi)log(1−shard,aibi−)],L_ang=- 1|C| _i=1^|C| [y_i s^+_a_ib_i+(1-y_i) (1-s^-_hard,a_ib_i) ], where saibi+s^+_a_ib_i is the must-link similarity and shard,aibi−s^-_hard,a_ib_i is the cannot-link transformed similarity: saibi+=12(cos(θai,bi)+1),shard,aibi−=12(cos(minωθai,bi,π)+1).s^+_a_ib_i= 12 ( ( _ z_a_i, z_b_i)+1 ), s^-_hard,a_ib_i= 12 ( ( \ω _ z_a_i, z_b_i,π\)+1 ). Here ω controls the negative zone of angular size π/ωπ/ω; the SpherePair theory fixes ω=2ω=2, giving the π/2π/2 negative-zone boundary and the corresponding conflict-free equidistant geometry for hard must-link/cannot-link constraints. Since this original formulation is defined for binary relation types, applying SpherePair to UPCC requires a continuous counterpart for intermediate labels. We therefore keep the same ω=2ω=2 angular scale and use ssoft,aibi−=12(cos(2θai,bi)+1)=cos2(θai,bi)s^-_soft,a_ib_i= 12 ( (2 _ z_a_i, z_b_i)+1 )= ^2( _ z_a_i, z_b_i) in the BCE-form loss with each real-valued yi∈[0,1]y_i∈[0,1]: ℒangsoft=−1||∑i=1||[yilogsaibi++(1−yi)log(1−ssoft,aibi−)].L_ang^soft=- 1|C| _i=1^|C| [y_i s^+_a_ib_i+(1-y_i) (1-s^-_soft,a_ib_i) ]. Under this soft extension, the attractive endpoint yi=1y_i=1 is minimized at θai,bi=0 _ z_a_i, z_b_i=0, while the fully repulsive endpoint yi=0y_i=0 is minimized at θai,bi=π/2 _ z_a_i, z_b_i=π/2, matching the orthogonal negative-zone boundary selected by the original ω=2ω=2 SpherePair geometry. For intermediate labels, the preferred pairwise angle varies continuously between these two endpoints; in the single-pair case, the optimum satisfies cosθai,bi=yi/(2−yi) _ z_a_i, z_b_i=y_i/(2-y_i). SpherePair also uses the normalized-embedding reconstruction regularizer ℒrec=1||∑j‖xj−x^j‖22L_rec= 1|X| _j\|x_j- x_j\|_2^2 with x^j=g(Norm(f(xj))) x_j=g(Norm(f(x_j))), which is also adopted by our ProbPair, Weighted ProbPair, and ECI-P. Thus, the SpherePair baseline in our experiments is the natural real-valued angular extension of the original hard-constraint method, while its formal geometric guarantee remains tied to the binary setting. ProbPair and Weighted ProbPair. ProbPair is the direct probabilistic pairwise baseline introduced in Section 4.1. It uses the probabilistic angular readout in Eq. 6 and minimizes the reconstruction-regularized objective ℒL in Eq. 8, without our Estimator–Corrector–Integrator refinement. Weighted ProbPair further applies the information-based constraint weight κi _i in Eq. 9 to the pairwise term: ℒWPP=−1||∑i=1||κi[yilogy^aibi+(1−yi)log(1−y^aibi)]+λrecℒrec.L_WPP=- 1|C| _i=1^|C| _i [y_i y_a_ib_i+(1-y_i) (1- y_a_ib_i) ]+ _recL_rec. Thus, ProbPair isolates the base probabilistic angular formulation, while Weighted ProbPair isolates the effect of decisiveness-based weighting. Comparing Weighted ProbPair with ECI-P further tests the contribution of the Estimator–Corrector–Integrator pipeline beyond static reweighting. C.3 Constraint generation Here, we detail how probabilistic pairwise constraints are generated in our experiments. We use trained expert classifiers as simulated judgment channels to produce probabilistic pairwise judgments and then apply the corruption channel in Section 3.1, thereby instantiating the UPCC observation process for constraint generation. Expert classifiers. For each dataset, simulated experts are trained on controlled labeled subsets of the training split and then used to generate predictive class distributions for queried pairs. The limited labeled-subset construction makes these classifiers expert-dependent judgment mechanisms, whose soft predictive distributions reflect both systematic bias and residual uncertainty. The subset selection rules for single- and multi-expert settings are described below, while the expert network architecture and optimization details are given in Appendix C.5. Single-expert settings. In the single-expert setting, one expert classifier is trained per dataset. We use three expert-quality levels, denoted by lv0.1, lv0.01, and lv0.001. For lvrr, the expert is trained using an r fraction of the labeled training samples from each class. The sampling is class-stratified: for every class, the number of selected samples is ⌊rnc⌋ rn_c , where ncn_c is the number of available training samples in class c, with at least one sample retained for each class. Thus, smaller values of r produce weaker experts and hence more epistemically distorted probabilistic judgments. Multi-expert settings. In the multi-expert setting, we construct a group of complementary experts. The three settings multi2, multi3, and multi10 use 22, 33, and 1010 experts, respectively. Each expert has a set of familiar classes and a set of unfamiliar classes. For all multi-expert settings, familiar classes use a high labeled-data fraction h=0.1h=0.1, while unfamiliar classes use a low labeled-data fraction l=0.0001l=0.0001. The class-level training fraction for expert e is therefore ρe,c=h,c∈ℱe,l,c∈e, _e,c= casesh,&c _e,\\ l,&c _e, cases where ℱeF_e and eU_e denote the familiar and unfamiliar class sets of expert e. The unfamiliar classes are assigned to make the experts complementary. Let the dataset contain C classes under the fixed label ordering used in expert generation. When the number of experts E satisfies E≤CE≤ C, the class list is split into E consecutive blocks that are as balanced as possible. Expert e takes the e-th block as its unfamiliar classes and treats all remaining classes as familiar. Equivalently, if C=qE+rC=qE+r with 0≤r<E0≤ r<E, then the first r experts have q+1q+1 unfamiliar classes and the remaining experts have q unfamiliar classes. For example, on a 10-class dataset, multi2 uses unfamiliar-class counts (5,5)(5,5), multi3 uses (4,3,3)(4,3,3), and multi10 gives each expert exactly one unfamiliar class. For CIFAR100-20, the corresponding counts are (10,10)(10,10), (7,7,6)(7,7,6), and (2,…,2)(2,…,2). For Reuters subset with four classes, multi2 gives (2,2)(2,2) and multi3 gives (2,1,1)(2,1,1). If E>CE>C, the experts are divided into full groups of size C, where each expert in a group is unfamiliar with one distinct class; any remaining experts form a partial group whose unfamiliar classes are sampled without replacement from the class list. This case occurs for Reuters subset under multi10. The resulting unfamiliar-class counts are summarized in Table 2. Table 2: Unfamiliar-class counts per expert under the multi-expert settings. These counts describe the blind-spot distribution used to construct complementary experts. Dataset type multi2 multi3 multi10 20-class datasets (10,10)(10,10) (7,7,6)(7,7,6) (2,2,2,2,2,2,2,2,2,2)(2,2,2,2,2,2,2,2,2,2) 10-class datasets (5,5)(5,5) (4,3,3)(4,3,3) (1,1,1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1,1,1) 4-class datasets (2,2)(2,2) (2,1,1)(2,1,1) (1,1,1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1,1,1) Pair sampling and expert assignment. All queried pairs are sampled randomly and uniformly from the training split. In the single-expert setting, every sampled pair is assigned to the single available expert. In the multi-expert setting, constraints are generated separately for each expert, and the total constraint budget is distributed evenly across experts unless otherwise specified. For a queried pair (xa,xb)(x_a,x_b) assigned to expert e, the trained expert outputs predictive class distributions ae p^e_a and be p^e_b. The clean expert judgment is computed as ye,abjud=⟨ae,be⟩=∑c=1Cpa,cepb,ce∈[0,1].y^jud_e,ab= p^e_a, p^e_b = _c=1^Cp^e_a,cp^e_b,c∈[0,1]. This value is high for similar expert-assigned class distributions, low when the expert separates the samples, and intermediate under uncertainty or overlapping class mass. Stochastic corruption. After the clean judgment is obtained, the recorded constraint value is produced through the corruption channel in Section 3.1. For each pair, an independent corruption indicator is drawn as ci∼Bernoulli(π)c_i (π), where π is the corruption probability used by the corresponding experiment. If ci=0c_i=0, the recorded label is the clean expert judgment: yi=yei,aibijud.y_i=y^jud_e_i,a_ib_i. If ci=1c_i=1, the clean judgment is replaced by an unstructured random value: yi∼Uniform(0,1).y_i (0,1). Thus, the final constraints combine three factors: data-dependent soft pairwise ambiguity expressed through expert predictive distributions, structured expert-dependent epistemic distortion from class-specific familiarity, and unstructured stochastic corruption from the uniform replacement channel. C.4 Experimental protocol Given the dataset details in Appendix C.1 and the constraint-generation procedure in Appendix C.3, we specify the experimental protocol for comparative evaluations, expert-aware ensemble references, held-out diagnostics, and sensitivity/ablation studies. Comparative evaluations. For clustering-performance comparisons, we report Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) over five trials. For embedding-based methods, including SpherePair, ProbPair, Weighted ProbPair, and ECI-P, the final partition is obtained by applying K-means to the learned representations, while end-to-end DCC baselines use their native clustering outputs. For the full main comparison, we use lv0.01 and multi3 with corruption probability 0.30.3 and total constraint budgets 33k/66k/99k; under multi3, these correspond to 11k/22k/33k constraints per expert. Robustness comparisons use a total budget of 99k constraints and vary one supervision factor at a time. For single-expert robustness, we use lv0.01 and corruption probability 0.30.3 as the reference setting: expert quality is varied over lv0.1/lv0.01/lv0.001 while fixing corruption probability at 0.30.3, and corruption probability is varied over 0.1/0.3/0.50.1/0.3/0.5 while fixing the expert level at lv0.01. For multi-expert robustness, we compare multi2, multi3, and multi10 specified in Appendix C.3; under the 99k total budget, these correspond to 4,5004,500, 3,0003,000, and 900900 constraints per expert, respectively. Moving from multi2 to multi10 increases both the number of experts and the average expert accuracy (Table 4), because the complementary blind-spot rule assigns fewer unfamiliar classes to each expert (see Table 2). Expert-aware ensemble references. Under multi-expert supervision, ECI-P uses expert identities through its expert-specific Correctors, whereas expert-agnostic baselines treat the merged constraints as a single supervision pool. As an additional reference, we also include expert-aware ensemble variants for baselines for multi-expert comparisons. For a baseline trained under an E-expert setting, we instantiate E members, each associated with one expert identity. All members are trained on the full merged constraint set =(ai,bi,ei,yi)i=1||C=\(a_i,b_i,e_i,y_i)\_i=1^|C|, but member e applies the multiplier αi(e)=1,ei=e,α,ei≠e,α∈[0,1], _i^(e)= cases1,&e_i=e,\\ α,&e_i≠ e, cases α∈[0,1], to the pairwise loss term of each constraint. Thus, α=0α=0 makes each member use only constraints from its associated expert, while α=1α=1 recovers the expert-agnostic merged-constraint training for every member. More generally, if the pairwise part of a baseline objective is decomposed into per-constraint terms ℓipair _i^pair, member e uses ℒℬ,pair(e)=1||∑i=1||αi(e)ℓipair,L_B,pair^(e)= 1|C| _i=1^|C| _i^(e) _i^pair, while all non-pairwise components of the baseline objective are kept unchanged. After training, each member produces a hard partition S^(e) S^(e). We aggregate the members through the co-association matrix Aabens=1E∑e=1E[S^(e)(xa)=S^(e)(xb)],A^ens_ab= 1E _e=1^EI\! [ S^(e)(x_a)= S^(e)(x_b) ], then form Dabens=1−AabensD^ens_ab=1-A^ens_ab and apply hierarchical clustering to obtain the final ensemble partition. The non-target expert weight α is selected separately for each dataset–baseline–multi-expert configuration using the reserved 1,0001,000-sample validation split: we search α∈0,0.01,0.05,0.1,0.5,1α∈\0,0.01,0.05,0.1,0.5,1\, run each candidate three times, and choose the value with the highest mean validation ACC before test evaluation. These ensemble variants give expert-agnostic baselines access to expert identities, but they also alter the training and aggregation structure through multiple expert-weighted members; they are therefore interpreted as auxiliary expert-aware references rather than fully symmetric replacements for the native baselines. Held-out diagnostics. The held-out diagnostics examine whether the internal quantities of ECI-P show trends consistent with their intended roles: Estimator-side beliefs, corrected relations, and integrated relations are compared with an external pairwise reference, and the post-correction gap is tested as a residual-discrepancy signal. To this end, we construct a diagnostic constraint set diagC^diag where |diag|=1000|C^diag|=1000 from the test split. These constraints are sample-disjoint from the training constraints and are used only for diagnosis, not for training, early stopping, or hyperparameter selection. For a held-out pair i=(ai,bi)i=(a_i,b_i) with ground-truth class labels tait_a_i and tbit_b_i, we define the external hard oracle oi⋆=[tai=tbi],o_i =I[t_a_i=t_b_i], which serves only as a diagnostic reference and is not identified with the canonical aleatoric relation Raibi⋆R_a_ib_i in Section 3. On diagC^diag, which is not assigned to any cross-fitting fold, the Estimator diagnostic belief is computed by averaging the predictions of all K fold-specific Estimators; we still denote this held-out Estimator belief by y^ioof y^oof_i for notational consistency. We then evaluate y^ioof y^oof_i, the corrected relation yicory^cor_i, and the Integrator belief yiinty^int_i against this external oracle using Brier scores. For any held-out score vi∈[0,1]v_i∈[0,1] and index set ℐI, we use Brier(v,ℐ)=1|ℐ|∑i∈ℐ(vi−oi⋆)2.Brier(v;I)= 1|I| _i (v_i-o_i )^2. The Estimator and Integrator diagnostics are computed on all diagnostic constraints, whereas Corrector-oracle alignment is evaluated on the clean subset ℐclean=i∈diag:ci=0I_clean=\i ^diag:c_i=0\. We also compute the clean post-correction discrepancy gapi,clean=|softclipξ(y^ioof)−yicor|,i∈ℐclean,gap_i,clean= |softclip_ξ( y_i^oof)-y_i^cor |, i _clean, to measure the residual disagreement between the Estimator belief and the corrected relation on uncorrupted held-out judgments. For corruption-screening diagnostics, the corresponding gap score is computed on all diagnostic constraints and used to rank corrupted records (ci=1)(c_i=1) against clean records (ci=0)(c_i=0), from which we report AUCcorruptAUC_corrupt and APcorruptAP_corrupt. Because this gap captures unexplained discrepancy after correction, it may reflect both stochastic corruption and residual epistemic uncertainty, rather than stochastic corruption alone. We therefore also include an oracle-supervision diagnostic, where the clean relation is given by oi⋆o_i before applying the same corruption channel; this provides a reference case in which expert-dependent epistemic distortion is removed. Sensitivity and ablation studies. Sensitivity and ablation studies are conducted for ECI-P under the default single-expert setting, namely lv0.01 with corruption probability 0.30.3. Each study is repeated over five trials and varies one factor at a time while keeping the remaining hyperparameters and the constraint-generation protocol fixed. ProbPair and Weighted ProbPair already provide direct ablation-style references for the base probabilistic pairwise objective and the information-based weighting strategy, as described in Appendix C.2. Here, we further examine ECI-P-specific design and sensitivity factors across its main components: the number of estimator folds K and the use of the information-based warm-up score κi _i in Eq. 9; the Corrector regularization weight λcor _cor and its capacity; the reliability-screening parameters γ, n0n_0, and soft-clipping sharpness ξ; and the reconstruction weight λrec _rec. C.5 Implementation We provide implementation details1111 11 Our source code is available on GitHub, and the URL is included in the supplementary materials. for the experimental pipeline, including the computing environment, data preprocessing, expert classifiers and generated constraints, model backbones, pretraining, optimization, and hyperparameter settings. Software and hardware. All model training code is implemented in Python 3.7 and PyTorch 1.5.11212 12 PyTorch 1.5.1: https://github.com/pytorch/pytorch/releases/tag/v1.5.1. For methods whose final prediction is obtained by K-means on learned representations, we use the K-means implementation in scikit-learn1313 13 Scikit-learn webpage: https://scikit-learn.org/0.19/documentation.html. For the hierarchical clustering step used by the expert-aware ensemble references in Appendix C.4, we use the fastcluster package1414 14 fastcluster library: https://pypi.org/project/fastcluster/. Each training run is executed on a single NVIDIA A100 40GB or NVIDIA H200 141GB GPU. Data preprocessing. We follow the same preprocessing pipeline as [55]. For Reuters subset and RCV1-10, we directly use the preprocessed tf–idf representations over the 2,0002,000 most frequent words or word stems, as described in Appendix C.1. For MNIST and FMNIST, the 28×2828× 28 grayscale images are flattened into 784784-dimensional vectors, following the standard fully connected input format used in prior deep clustering/DCC studies [40, 54, 34, 55]. For CIFAR10, CIFAR100-20, STL10, and ImageNet10, we adopt the unsupervised feature extraction strategy in [29]: a ResNet-34 model [18] is trained for 1,0001,000 epochs, and the resulting 512512-dimensional representations are used as input features. This image preprocessing follows the setting used in DCC studies [37, 55] and keeps the inputs compatible with fully connected architectures across datasets. The same preprocessed inputs are used for expert classifier training and for all compared clustering methods. Table 3: Test accuracy (%) of single-expert classifiers used for constraint generation. Dataset lv0.1 lv0.01 lv0.001 CIFAR100-20 73.08 65.23 46.58 CIFAR10 91.01 88.49 73.83 FMNIST 84.45 74.77 59.83 ImageNet10 96.27 92.62 87.62 MNIST 93.67 88.76 65.20 Reuters 92.90 83.60 49.75 STL10 90.04 84.31 72.77 RCV1-10 94.26 90.47 77.62 Table 4: Average test accuracy (%) of multi-expert groups used for constraint generation. Each value is the arithmetic mean over experts in the corresponding group. Dataset multi2 multi3 multi10 CIFAR100-20 41.23 51.90 67.36 CIFAR10 52.72 65.94 82.74 FMNIST 54.57 60.68 77.64 ImageNet10 73.87 76.73 90.80 MNIST 50.45 65.89 86.13 Reuters 47.58 63.02 71.42 STL10 53.98 65.67 83.22 RCV1-10 49.06 64.57 85.96 Figure 4: Distributions of recorded constraint values for representative generated-supervision settings. The top row shows lv0.01 single-expert supervision and the bottom row shows multi3 multi-expert supervision, both with corruption probability 0.30.3. Expert classifiers and generated constraints. The expert classifiers used in Appendix C.3 are implemented as MLP classifiers with two hidden layers of size 512512–512512, ReLU activations, dropout rate 0.20.2, and a final softmax classification layer. They are trained with cross-entropy loss using Adam, with learning rate 10−310^-3, batch size 256256, maximum 100100 epochs, and early stopping patience 1010 based on training accuracy. The resulting test accuracies of single experts are reported in Table 3, and the average test accuracies of multi-expert groups are reported in Table 4. These accuracies characterize the effective quality of the simulated experts produced by the subset-selection and blind-spot protocols in Appendix C.3. After the corruption channel in Section 3.1, these expert judgments yield the recorded constraint values used for training. Fig. 4 visualizes their per-dataset distributions for representative lv0.01 single-expert and multi3 multi-expert settings, both with corruption probability 0.30.3. Model backbones and pretraining. For encoder-based methods, we follow the fully connected autoencoder backbone used in [40, 54, 55]: the encoder has hidden layers 500500–500500–20002000, paired with a symmetric decoder when reconstruction is used. The embedding dimension is set to D=10D=10 for all datasets except CIFAR100-20, where D=20D=20, following [55]. VanillaDCC and VolMaxDCC use their native MLP classifier architectures with two hidden layers of size 512512–512512, and do not use autoencoder pretraining. For pretrain-capable methods, we use the two-stage stacked denoising autoencoder (SDAE) pretraining approach [19, 44], following the protocol used in [40, 54, 55]: hidden layers are first pretrained layer-wise as denoising autoencoders, followed by end-to-end autoencoder fine-tuning on the training split. Unless otherwise specified, all pretrain-capable methods use this pretrained initialization. For methods using angular reconstruction (SpherePair, ProbPair, Weighted ProbPair, ECI-P), decoding is performed from normalized embeddings as in [55]. In ECI-P, both the K Estimator backbones and the Integrator use the same autoencoder architecture described above. Each Estimator fold has its own encoder–decoder backbone and its own probabilistic readout parameters (mk,Tk)(m_k,T_k), while the Integrator has its own readout parameters (m,T)(m,T) in Eq. 6. These scalar readout parameters are parameterized as m=tanh(m~)m= ( m) and T=softplus(T~)T=softplus( T) for numerical stability within the cosine range, with the same parameterization used for each Estimator fold. When pretraining is used, the same pretrained autoencoder weights initialize the Integrator backbone and all Estimator backbones, while the readout parameters and Correctors are initialized separately. Unless otherwise specified, each expert-specific Corrector is an MLP with hidden layers 6464–1616 and a tanh output, which keeps the predicted probability-space residual within (−1,1)(-1,1). Optimization and hyperparameters. We use the reported or recommended hyperparameters for the baseline methods whenever available. Below, we summarize the optimization settings and implementation configurations for ECI-P and all compared baselines. • VanillaDCC. VanillaDCC optimizes the MCL loss ℒMCLL_MCL using Adam with learning rate 10−310^-3 and batch size 256256. Training runs for at most 500500 epochs, with early stopping triggered when the relative change in the soft cluster assignments on the training samples falls below 10−310^-3 for two consecutive checks. This assignment-change stopping rule follows common practice in deep clustering and deep constrained clustering [52, 17, 40, 54]. • VolMaxDCC. For VolMaxDCC, we follow its original optimization and validation-based hyperparameter-search procedure [37]. The learnable matrix B is parameterized elementwise as Bcc′=(1+exp(−Bcc′))−1B_c = (1+ (-B _c ) )^-1, where Bcc′B _c is initialized to 11 for c=c′c=c and to −1-1 otherwise. Optimization uses SGD with learning rate 0.50.5 for the network parameters and 0.10.1 for B′B , with batch size 128128. The VolMaxDCC geometric regularization weight is selected from 0,10−1,10−2,10−3,10−4,10−5\0,10^-1,10^-2,10^-3,10^-4,10^-5\ using the reserved validation split; for each dataset and supervision setting, we choose the value with the highest mean validation ACC over three validation runs. For multi-expert supervision, a shared selected value is used for the multi-expert settings defined in Appendix C.3, where the familiar and unfamiliar labeled-data fractions are fixed as h=0.1h=0.1 and l=0.0001l=0.0001. The selected values are reported in Table 5. Table 5: Selected VolMaxDCC geometric regularization weights. Values are selected by validation ACC over three runs for each dataset and supervision setting. Dataset lv0.1 lv0.01 lv0.001 multi CIFAR100-20 10−310^-3 10−310^-3 10−110^-1 10−210^-2 CIFAR10 10−310^-3 10−310^-3 10−110^-1 10−210^-2 FMNIST 10−310^-3 10−310^-3 10−210^-2 10−210^-2 ImageNet10 00 10−410^-4 10−110^-1 10−210^-2 MNIST 10−210^-2 10−310^-3 10−210^-2 10−210^-2 Reuters 10−210^-2 00 10−110^-1 10−110^-1 STL10 10−310^-3 10−310^-3 10−210^-2 10−110^-1 RCV1-10 10−410^-4 10−410^-4 10−510^-5 10−310^-3 • CIDEC. For CIDEC, we follow the authors’ recommended settings: the reconstruction/clustering trade-off is set to λ1=1 _1=1, and the MCL constraint-balancing parameter is set to λ2=0.1 _2=0.1. The C cluster anchors are initialized by K-means. Optimization uses Adam with learning rate 10−310^-3 and batch size 256256. Training proceeds for at most 500500 epochs, with early stopping applied when the relative change in the soft assignments falls below 10−310^-3 over consecutive checks. • SpherePair. For SpherePair, the reconstruction weight is set to 0.020.02, matching the setting used in [55]. In the original binary formulation, the negative-zone factor is fixed as ω=2ω=2, which yields the theoretically guaranteed π/2π/2 negative-zone boundary. As described in Appendix C.2, we use the real-valued extension of SpherePair by keeping the same ω=2ω=2 angular scale and replacing the original clamped cannot-link score with the continuous counterpart ssoft,aibi−=12(cos(2θai,bi)+1)s^-_soft,a_ib_i= 12 ( (2 _ z_a_i, z_b_i)+1 ). The observed yi∈[0,1]y_i∈[0,1] is then used directly in the BCE-form angular loss. Under this implementation, yi=1y_i=1 favors θai,bi=0 _ z_a_i, z_b_i=0 and yi=0y_i=0 favors θai,bi=π/2 _ z_a_i, z_b_i=π/2, while intermediate labels continuously interpolate between these two angular endpoints through the two BCE terms. Optimization uses Adam with learning rate 10−310^-3 and constraint mini-batch size 256256. The instance mini-batch size for reconstruction is set according to the number of constraint mini-batches, following [55]. Training runs for at most 500500 epochs, with early stopping after the first 100100 epochs if the relative change in the total loss remains below 0.10.1 for five consecutive checks. For the expert-aware ensemble reference in Appendix C.4, the expert-aware hyperparameter α is selected on the reserved validation split; the selected values are reported in Table 6. Table 6: Selected expert-aware ensemble hyperparameter α. Values are chosen by mean validation ACC over three runs and shared across the multi-expert configurations defined in Appendix C.3. Dataset SpherePair Weighted ProbPair CIFAR100-20 0.050.05 0.050.05 CIFAR10 1.001.00 0.050.05 FMNIST 1.001.00 0.500.50 ImageNet10 0.000.00 0.100.10 MNIST 0.050.05 0.500.50 Reuters 0.050.05 0.050.05 STL10 0.010.01 0.010.01 RCV1-10 0.010.01 0.050.05 • ProbPair variants and ECI-P (Ours). ProbPair, Weighted ProbPair, and ECI-P use the ProbPair-style readout in Eq. 6 and the reconstruction weight λrec=0.02 _rec=0.02. ProbPair optimizes Eq. 8, while Weighted ProbPair additionally applies the information-based weight κi _i in Eq. 9. For the expert-aware ensemble reference based on Weighted ProbPair in Appendix C.4, the expert-aware hyperparameter α is selected on the reserved validation split; the selected values are reported in Table 6. The shared base optimizer is Adam with learning rate 10−310^-3 and batch size 256256. The probabilistic readout parameters, including (m,T)(m,T) and the fold-wise (mk,Tk)(m_k,T_k) when present, are initialized as (0,0.1)(0,0.1) and optimized with learning rate 10−210^-2. We use a fixed outer training budget of 500500 epochs; principled stopping criteria for iterative refinement remain an open question for future study. For any weighted ProbPair-style objective, including Weighted ProbPair and the weighted Estimator/Integrator updates in ECI-P, the weight is applied inside the constraint average: ℒPPw=−1||∑i=1||wi[y~ilogy^aibi+(1−y~i)log(1−y^aibi)],L^w_P=- 1|C| _i=1^|C|w_i [ y_i y_a_ib_i+(1- y_i) (1- y_a_ib_i) ], where y~i y_i denotes the corresponding training target, such as yiy_i, yiinty^int_i, or yiBCy_i^BC. For ECI-P, we use one global hyperparameter setting unless explicitly varied in sensitivity or ablation studies: K=5K=5 estimator folds, λcor=0.5 _cor=0.5, γ=10γ=10, n0=10n_0=10, soft-clipping sharpness ξ=20ξ=20, and information-based warm-up enabled. The default warm-up trains the Estimators and Integrator for 5050 epochs before iterative refinement. The initial y^ioof y^oof_i is obtained by strict fold-heldout prediction, and later refinement rounds use refreshed Estimator beliefs under the Integrator-updated targets while retaining the same fold structure. Each Corrector update uses the shared base optimizer setting, with the Huber quadratic-to-linear transition in Eq. 10 fixed at 0.10.1, holds out 10% of its assigned constraints for internal validation, runs for at most 50 epochs, and stops early if the validation loss does not improve for 10 consecutive epochs. Between refinement iterations, Gaussian noise with standard deviation 0.010.01 is added to the final linear layers of the Estimators, Correctors, and Integrator to mildly perturb the next refinement step. Notably, the validation-based selections of the VolMaxDCC geometric regularization weight and the expert-aware ensemble hyperparameter α rely on ground-truth class labels on the validation split, which are typically unavailable in constrained clustering and are not part of the observable supervision under UPCC. In contrast, ECI-P uses one global hyperparameter setting across datasets and handles expert identities directly through expert-specific Correctors, without dataset-specific tuning or an additional hyperparameterized ensemble wrapper. Appendix D Additional experimental results D.1 Comparison under different constraint budgets Tables 7 and 8 report the full main-comparison results under the default lv0.01 single-expert and multi3 multi-expert regimes, with corruption probability fixed at 0.30.3. The total constraint budget is varied over 33k/66k/99k; under multi3, these correspond to 11k/22k/33k constraints per expert. All results in this subsection use pre-trained backbones for methods with a pretraining stage, while VanillaDCC and VolMaxDCC have no pretraining variant. Table 7: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the lv0.01 setting, with corruption probability 0.30.3 and 33k/66k/99k constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. Vanilla- DCC VolMax- DCC CIDEC SpherePair ProbPair Weighted ProbPair ECI-P (Ours) CIFAR100 3k ACC 14.7, 14.7 42.8, 42.5 14.9, 14.6 45.7, 45.7 44.5, 44.4 45.2, 45.2 48.1, 47.8 NMI 11.2, 11.8 42.5, 42.9 13.1, 13.6 42.8, 43.3 42.4, 42.9 43.4, 43.7 46.0, 46.5 ARI 4.0, 4.1 26.2, 26.0 2.6, 2.1 28.5, 28.5 27.6, 27.6 28.6, 28.5 31.3, 31.4 6k ACC 16.5, 16.6 45.4, 45.3 16.6, 15.6 49.0, 49.2 46.1, 46.4 48.3, 48.5 51.2, 51.1 NMI 12.6, 13.5 43.9, 44.5 15.0, 15.1 43.8, 44.9 42.4, 43.4 43.7, 44.5 48.3, 48.8 ARI 5.4, 5.6 28.2, 28.2 3.4, 2.9 31.5, 32.0 28.7, 29.0 30.2, 30.5 34.7, 34.7 9k ACC 16.8, 17.0 46.1, 45.9 14.3, 13.6 50.2, 50.5 49.4, 49.5 50.3, 50.8 52.7, 52.9 NMI 13.5, 14.6 45.6, 46.1 11.0, 11.2 43.0, 44.2 43.2, 44.2 44.5, 45.7 49.2, 49.9 ARI 6.0, 6.3 30.4, 30.5 2.8, 2.3 32.1, 32.6 30.6, 30.9 31.5, 32.1 36.2, 36.5 CIFAR10 3k ACC 34.6, 34.6 79.9, 79.8 54.1, 52.7 84.6, 84.9 84.8, 85.0 85.8, 85.7 85.5, 85.4 NMI 27.9, 28.1 69.3, 69.4 40.8, 40.7 74.3, 74.7 74.9, 75.3 76.0, 76.0 77.0, 76.9 ARI 20.2, 20.3 64.4, 64.4 23.0, 20.5 71.7, 72.0 71.9, 72.1 73.4, 73.2 73.5, 73.3 6k ACC 35.5, 35.7 84.7, 84.6 51.3, 51.0 84.8, 85.3 86.0, 86.0 86.5, 86.5 87.3, 87.2 NMI 27.5, 28.3 73.6, 73.7 39.8, 41.1 73.1, 74.2 75.6, 75.9 76.3, 76.5 78.6, 78.7 ARI 20.7, 21.2 71.0, 70.9 22.9, 22.8 71.5, 72.3 73.5, 73.5 74.3, 74.2 76.0, 76.0 9k ACC 38.3, 38.7 82.5, 82.5 44.3, 44.0 84.9, 85.7 85.9, 86.0 86.6, 86.7 87.8, 87.8 NMI 29.9, 31.2 71.9, 72.2 36.1, 37.6 72.4, 74.1 75.0, 75.4 76.2, 76.8 79.0, 79.1 ARI 23.2, 24.0 68.3, 68.3 21.1, 21.3 71.2, 72.5 73.0, 73.2 74.3, 74.5 76.7, 76.8 FMNIST 3k ACC 36.6, 36.5 63.3, 62.5 35.4, 34.4 71.1, 70.0 70.5, 69.5 71.0, 70.0 70.2, 69.2 NMI 31.1, 31.1 55.5, 55.2 31.6, 31.3 61.0, 60.7 61.0, 60.6 61.3, 61.0 64.0, 63.4 ARI 19.7, 19.5 44.4, 43.6 13.5, 12.4 53.2, 52.0 52.1, 51.0 53.1, 52.1 53.8, 52.6 6k ACC 38.6, 38.5 66.2, 65.1 33.8, 33.3 70.9, 69.8 72.7, 71.5 73.6, 72.5 71.3, 70.2 NMI 30.9, 31.3 56.7, 56.3 26.9, 27.1 60.6, 60.5 62.4, 62.1 63.1, 62.9 65.3, 64.5 ARI 20.7, 20.9 46.6, 45.5 12.2, 11.9 53.2, 52.1 53.9, 52.6 54.6, 53.5 55.1, 53.8 9k ACC 39.8, 40.2 66.5, 65.5 36.1, 35.7 72.0, 71.2 73.6, 72.5 74.6, 73.6 72.4, 71.1 NMI 30.1, 31.3 57.2, 56.8 29.7, 30.5 60.4, 60.9 62.9, 62.9 63.9, 64.0 65.7, 65.0 ARI 21.5, 22.2 46.4, 45.2 15.7, 15.6 53.0, 52.3 54.8, 53.7 55.3, 54.3 55.5, 53.9 ImageNet10 3k ACC 35.4, 37.3 86.7, 86.9 69.0, 71.2 88.0, 89.2 89.6, 89.8 90.0, 90.1 92.2, 92.5 NMI 27.7, 30.8 78.8, 79.4 53.9, 58.7 78.6, 81.4 81.8, 82.3 82.8, 83.3 87.7, 88.1 ARI 19.1, 20.8 74.1, 73.9 43.4, 47.1 76.8, 78.9 79.4, 79.6 80.5, 80.5 85.0, 85.4 6k ACC 40.6, 42.0 87.2, 87.5 67.0, 71.2 87.2, 89.3 89.7, 90.5 89.7, 90.2 92.7, 92.9 NMI 30.6, 35.2 78.3, 79.3 50.3, 58.7 75.4, 80.4 81.2, 83.3 81.6, 82.8 88.3, 88.5 ARI 23.2, 26.0 75.1, 75.6 40.2, 46.3 74.7, 78.7 79.3, 80.7 79.5, 80.0 85.9, 86.1 9k ACC 39.0, 41.5 87.4, 88.1 63.5, 68.3 85.1, 89.1 89.2, 90.5 89.1, 90.3 92.8, 93.1 NMI 27.6, 33.7 77.8, 79.2 47.2, 57.5 71.0, 79.2 80.1, 83.2 80.3, 82.7 88.1, 88.5 ARI 21.1, 25.2 75.2, 76.1 35.5, 42.2 70.7, 78.2 78.5, 80.7 78.4, 80.2 85.8, 86.2 MNIST 3k ACC 25.9, 26.1 65.6, 66.4 59.1, 56.5 87.3, 87.9 86.3, 86.6 89.6, 90.2 84.2, 84.6 NMI 16.0, 17.0 52.7, 54.1 51.9, 51.0 78.1, 79.3 75.5, 76.5 78.4, 79.7 75.0, 76.1 ARI 10.4, 10.8 47.8, 48.9 30.3, 26.7 76.8, 77.8 74.3, 74.9 78.5, 79.7 72.3, 73.1 6k ACC 33.3, 33.9 73.4, 74.4 59.2, 57.0 89.3, 90.1 88.0, 89.2 89.5, 90.1 89.5, 90.3 NMI 24.2, 25.8 58.4, 60.5 52.0, 52.1 78.2, 80.2 75.8, 78.1 77.8, 79.2 79.2, 80.3 ARI 18.4, 19.4 55.2, 56.8 29.8, 26.4 78.2, 79.9 75.8, 78.1 78.4, 79.7 78.9, 80.1 9k ACC 38.9, 40.3 75.6, 76.9 47.0, 46.7 87.8, 89.1 89.0, 90.0 89.9, 90.8 90.7, 91.0 NMI 29.5, 32.2 59.3, 61.6 41.4, 43.7 75.4, 78.0 76.9, 79.0 78.3, 80.2 80.7, 81.6 ARI 23.1, 24.8 56.5, 58.7 24.7, 24.2 75.4, 77.8 77.4, 79.4 79.1, 80.9 80.8, 81.5 REUTERS 3k ACC 53.2, 54.0 62.8, 63.2 74.4, 77.0 81.4, 83.4 79.6, 81.3 82.8, 84.4 86.0, 86.8 NMI 15.1, 17.9 25.6, 28.0 42.8, 49.8 54.5, 58.9 55.4, 58.5 57.8, 61.6 63.8, 65.6 ARI 17.0, 18.0 31.2, 32.4 50.4, 56.7 59.8, 64.4 58.9, 62.3 62.9, 66.1 69.4, 71.0 6k ACC 66.0, 70.6 72.5, 76.8 74.2, 79.3 80.1, 83.2 79.6, 81.5 82.2, 83.9 85.4, 86.4 NMI 27.7, 36.3 36.0, 44.9 38.2, 48.8 52.0, 58.5 55.0, 58.9 56.7, 60.4 62.4, 65.0 ARI 34.0, 42.6 42.4, 50.8 44.6, 55.6 56.8, 63.2 58.1, 62.0 61.5, 65.3 67.8, 70.3 9k ACC 69.7, 77.0 71.7, 76.8 72.9, 79.3 77.9, 81.8 78.2, 80.8 80.4, 82.7 85.1, 86.6 NMI 30.5, 43.3 34.3, 43.4 35.7, 48.1 48.7, 56.3 53.2, 58.3 54.5, 59.2 61.4, 64.4 ARI 36.8, 50.4 40.9, 50.4 41.8, 54.9 52.9, 61.1 55.5, 61.0 58.6, 63.4 66.6, 70.0 STL10 3k ACC 30.9, 31.5 72.8, 73.3 66.8, 69.4 80.7, 81.7 81.9, 83.0 83.3, 83.9 85.0, 85.3 NMI 21.1, 23.4 64.8, 66.5 49.9, 55.1 66.7, 69.1 69.7, 72.0 71.5, 73.0 75.3, 75.9 ARI 14.1, 15.0 56.3, 57.0 40.5, 44.8 64.2, 66.1 66.3, 68.2 68.8, 69.8 72.0, 72.5 6k ACC 30.4, 31.6 76.8, 77.5 62.7, 66.7 79.0, 81.7 81.6, 82.9 82.8, 83.9 85.4, 85.5 NMI 19.5, 22.2 65.8, 67.7 44.4, 51.4 63.2, 68.2 68.7, 71.6 70.2, 72.5 75.8, 76.3 ARI 14.3, 15.7 60.0, 61.3 34.1, 40.0 61.2, 65.5 65.9, 68.2 67.9, 69.6 72.7, 73.0 9k ACC 35.5, 37.8 79.0, 80.1 57.8, 63.3 76.1, 79.7 81.6, 83.4 81.6, 83.1 85.4, 85.8 NMI 22.9, 27.6 66.7, 69.3 39.3, 48.5 59.5, 66.5 68.2, 72.1 67.9, 71.6 75.8, 76.6 ARI 17.1, 19.8 62.8, 64.7 29.8, 37.3 57.2, 63.3 65.8, 68.9 65.8, 68.4 72.8, 73.4 RCV1-10 3k ACC 40.0, 40.1 46.3, 46.4 47.4, 48.7 65.1, 64.9 54.0, 53.8 52.6, 52.4 61.7, 61.7 NMI 12.3, 12.4 21.5, 21.9 33.3, 32.5 59.2, 59.6 54.2, 54.4 55.1, 55.3 60.6, 60.8 ARI 16.4, 16.5 21.2, 21.4 28.6, 29.1 56.1, 56.2 43.3, 43.1 43.5, 43.4 53.3, 53.5 6k ACC 52.4, 52.6 62.1, 62.4 42.3, 43.5 70.8, 70.7 51.5, 51.3 54.0, 53.9 63.7, 63.7 NMI 21.8, 22.2 36.0, 36.7 30.5, 30.5 60.6, 61.1 54.1, 54.2 57.5, 57.8 62.9, 63.0 ARI 28.0, 28.2 42.9, 43.3 24.9, 26.7 61.6, 61.8 42.3, 42.1 46.2, 46.1 55.9, 56.1 9k ACC 46.2, 46.4 61.3, 61.8 42.5, 44.3 71.2, 71.4 52.7, 53.0 55.2, 55.1 69.0, 68.9 NMI 15.5, 15.9 35.7, 36.7 29.5, 30.0 60.5, 61.3 53.8, 54.1 57.4, 57.5 65.5, 65.8 ARI 19.8, 20.1 44.7, 45.4 26.2, 28.3 61.0, 61.4 43.6, 43.7 47.1, 47.0 62.0, 62.0 Table 8: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the multi3 setting, with corruption probability 0.30.3 and 33k/66k/99k total constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. Vanilla- DCC VolMax- DCC CIDEC SpherePair ProbPair Weighted ProbPair ECI-P (Ours) CIFAR100 3k ACC 12.2, 12.4 43.7, 43.3 19.5, 18.4 42.0, 42.1 41.5, 41.6 43.1, 43.3 45.6, 45.3 NMI 9.5, 9.9 43.4, 43.6 20.5, 20.5 41.5, 42.1 40.2, 41.1 41.9, 42.4 45.0, 45.5 ARI 2.6, 2.5 26.8, 26.6 3.9, 3.2 26.1, 26.3 24.8, 25.3 26.7, 26.8 29.7, 29.8 6k ACC 14.2, 14.4 43.1, 42.5 17.4, 16.6 42.4, 42.4 42.0, 42.1 44.1, 44.4 49.1, 49.3 NMI 11.3, 12.2 44.3, 44.5 16.2, 16.3 40.0, 40.9 39.8, 40.6 41.1, 41.9 47.1, 47.7 ARI 4.6, 4.7 27.7, 27.5 3.2, 2.5 26.2, 26.5 25.2, 25.4 26.4, 26.7 32.1, 32.4 9k ACC 15.4, 15.8 42.8, 42.5 12.8, 12.1 44.3, 44.4 43.3, 43.4 46.1, 46.5 48.6, 48.8 NMI 12.8, 13.6 43.4, 43.8 10.0, 10.5 40.5, 41.5 40.0, 40.9 41.8, 43.0 48.1, 48.5 ARI 5.8, 5.8 27.1, 27.1 2.1, 2.0 27.1, 27.6 26.0, 26.2 28.0, 28.5 32.9, 32.9 CIFAR10 3k ACC 28.8, 28.8 76.0, 75.8 38.4, 36.8 74.8, 75.1 79.8, 79.8 82.0, 82.1 82.8, 82.5 NMI 25.2, 25.5 71.0, 70.9 32.3, 31.7 69.5, 70.2 72.0, 72.1 73.2, 73.3 76.0, 75.9 ARI 16.5, 16.6 63.3, 63.0 13.4, 11.9 63.0, 63.5 67.0, 66.8 68.7, 68.7 70.8, 70.7 6k ACC 33.6, 33.9 78.0, 78.1 36.5, 34.9 76.5, 76.8 80.7, 81.0 80.9, 81.0 84.0, 84.0 NMI 30.9, 32.0 72.6, 72.9 31.9, 31.9 66.9, 68.1 71.2, 71.6 71.9, 72.2 77.0, 77.1 ARI 22.4, 23.1 65.7, 65.8 14.2, 14.0 61.7, 62.5 66.2, 66.6 67.8, 67.9 72.6, 72.6 9k ACC 32.4, 32.8 76.5, 76.6 33.2, 31.6 73.8, 74.5 81.3, 81.7 81.4, 81.7 81.6, 82.1 NMI 29.1, 30.9 73.1, 73.2 28.7, 29.1 64.7, 66.5 71.5, 72.3 72.0, 72.6 77.8, 77.8 ARI 21.2, 22.4 65.8, 65.9 12.3, 12.1 59.0, 60.5 67.1, 67.8 67.9, 68.4 73.1, 73.0 FMNIST 3k ACC 33.7, 33.7 58.1, 57.7 30.1, 29.6 61.3, 61.0 60.9, 60.3 62.6, 62.5 60.1, 59.2 NMI 32.9, 33.0 56.7, 56.4 29.8, 30.1 58.1, 58.2 59.2, 58.8 59.1, 59.1 62.7, 61.9 ARI 20.7, 20.6 42.7, 42.1 14.1, 14.0 45.9, 45.6 46.4, 45.8 46.4, 46.2 49.5, 48.4 6k ACC 34.2, 34.3 58.2, 57.9 30.4, 30.1 65.4, 65.3 64.1, 63.8 66.1, 66.0 62.0, 61.1 NMI 32.1, 32.8 58.3, 58.1 28.1, 29.6 58.9, 59.7 59.6, 59.7 60.1, 60.3 64.3, 63.8 ARI 21.0, 21.2 44.4, 43.7 14.2, 14.7 48.7, 48.9 47.8, 47.7 49.6, 49.5 51.0, 51.0 9k ACC 35.2, 35.6 59.4, 59.1 28.6, 28.8 61.0, 61.3 66.6, 66.7 68.0, 67.6 65.3, 64.8 NMI 31.4, 32.8 59.2, 59.0 28.3, 30.6 56.6, 57.8 60.7, 61.2 61.7, 62.0 66.0, 65.6 ARI 20.4, 21.3 45.0, 44.3 14.5, 15.4 46.8, 47.4 49.9, 50.0 51.9, 51.8 53.4, 52.8 ImageNet10 3k ACC 33.9, 34.7 88.8, 88.9 50.7, 49.2 81.0, 81.5 87.8, 88.4 89.5, 89.7 91.5, 91.8 NMI 29.0, 31.4 85.9, 85.8 41.2, 43.3 72.7, 74.2 78.8, 80.3 81.6, 82.3 86.9, 87.3 ARI 19.4, 20.7 81.9, 81.7 21.7, 20.4 68.2, 69.1 76.7, 77.7 79.7, 79.9 84.1, 84.4 6k ACC 32.7, 34.6 87.6, 88.1 43.0, 41.6 81.8, 83.7 88.6, 89.4 89.6, 90.1 91.9, 92.1 NMI 25.3, 30.1 86.5, 87.1 35.4, 38.5 69.2, 73.5 78.3, 80.4 80.2, 81.6 87.2, 87.8 ARI 17.9, 20.9 81.9, 82.4 17.3, 17.9 66.3, 70.0 77.2, 78.6 79.3, 80.1 84.6, 85.0 9k ACC 31.8, 34.2 89.9, 90.5 39.4, 38.8 80.7, 85.0 87.4, 89.2 90.1, 91.3 92.4, 92.6 NMI 22.0, 26.8 87.8, 88.7 31.0, 34.8 65.6, 73.8 76.1, 80.1 80.1, 83.0 87.7, 88.1 ARI 15.8, 18.7 83.9, 84.9 14.3, 15.6 63.4, 71.0 75.0, 78.3 79.8, 82.0 85.4, 85.7 MNIST 3k ACC 27.0, 27.3 54.6, 55.4 44.9, 41.8 83.6, 84.4 81.2, 82.2 86.5, 87.2 80.9, 81.5 NMI 19.2, 20.5 48.5, 49.8 45.6, 44.1 75.2, 76.9 73.0, 74.7 75.2, 76.5 75.7, 77.1 ARI 11.5, 12.2 38.6, 39.5 20.4, 17.8 71.5, 72.4 69.4, 70.9 73.9, 74.9 71.9, 73.2 6k ACC 28.1, 29.0 62.5, 63.3 39.8, 38.1 78.8, 79.9 81.0, 82.3 86.5, 87.4 86.1, 86.4 NMI 22.0, 23.8 55.1, 56.6 39.2, 39.3 72.8, 75.3 72.2, 74.4 75.3, 76.9 79.8, 80.5 ARI 14.7, 15.7 47.1, 48.4 16.8, 15.4 68.2, 70.3 68.7, 70.8 73.9, 75.4 77.8, 78.3 9k ACC 32.2, 33.3 67.1, 67.8 31.8, 31.0 78.6, 80.0 80.5, 81.8 85.1, 86.1 84.7, 85.1 NMI 26.5, 29.3 57.5, 58.9 28.8, 30.2 70.8, 74.0 72.1, 74.7 75.0, 77.2 79.9, 80.7 ARI 18.4, 20.1 50.9, 52.1 13.2, 13.4 66.3, 69.1 68.7, 71.2 73.4, 75.5 77.1, 77.5 REUTERS 3k ACC 42.8, 41.4 66.9, 67.3 61.4, 62.1 72.8, 75.2 78.6, 79.9 75.7, 78.0 85.0, 86.3 NMI 3.2, 3.7 40.8, 42.5 26.3, 32.2 40.2, 44.6 48.0, 50.3 49.8, 53.4 62.0, 65.0 ARI 4.7, 4.0 42.8, 43.9 28.1, 28.9 45.2, 50.1 55.0, 57.5 53.8, 58.1 68.3, 71.1 6k ACC 51.4, 52.9 67.0, 68.2 58.7, 60.4 66.7, 72.9 78.6, 81.4 81.0, 83.4 87.3, 88.5 NMI 11.8, 15.9 44.3, 46.5 19.8, 27.0 32.8, 43.2 48.3, 54.0 52.9, 57.7 63.8, 67.2 ARI 14.0, 16.4 46.5, 48.7 22.6, 25.1 37.2, 48.2 54.5, 60.3 58.8, 63.6 71.3, 74.3 9k ACC 55.3, 59.2 67.5, 68.0 57.3, 59.1 62.1, 69.9 74.5, 79.0 81.0, 83.9 88.3, 89.6 NMI 14.7, 22.8 45.4, 46.5 18.3, 25.7 25.6, 37.2 44.6, 52.8 51.5, 57.9 65.4, 68.7 ARI 19.2, 26.2 47.0, 47.9 20.4, 23.1 28.0, 40.7 50.2, 58.6 57.9, 64.2 72.9, 76.0 STL10 3k ACC 25.6, 26.4 74.6, 74.8 47.4, 47.4 72.7, 73.4 79.2, 79.9 80.4, 80.8 85.7, 85.9 NMI 19.7, 21.7 71.5, 72.0 38.3, 41.5 61.7, 64.0 66.5, 68.4 68.9, 70.0 75.7, 76.3 ARI 12.3, 13.1 62.3, 62.8 21.8, 22.4 55.0, 56.4 61.6, 62.9 64.7, 65.3 72.2, 72.5 6k ACC 29.8, 31.2 72.7, 72.6 37.6, 37.2 67.7, 69.8 81.0, 82.0 83.1, 84.2 87.4, 87.5 NMI 22.3, 26.4 71.1, 71.2 31.2, 37.2 54.8, 58.8 67.4, 69.9 69.5, 72.0 77.5, 78.1 ARI 15.5, 17.7 61.1, 60.6 15.8, 18.2 48.6, 51.7 64.0, 65.7 67.4, 69.4 75.1, 75.4 9k ACC 29.2, 30.7 71.0, 71.1 31.4, 31.6 67.8, 71.4 81.1, 83.0 83.1, 84.3 87.9, 87.7 NMI 21.5, 26.0 70.2, 69.9 24.9, 31.7 53.2, 59.7 66.1, 70.2 69.4, 72.0 78.3, 78.5 ARI 15.2, 17.7 59.5, 59.1 13.6, 16.9 47.8, 53.3 63.7, 67.1 67.4, 69.4 75.9, 75.8 RCV1-10 3k ACC 29.2, 29.2 31.7, 31.9 33.5, 34.9 51.1, 51.0 47.9, 47.8 47.0, 46.9 55.3, 55.2 NMI 2.5, 2.6 23.0, 23.6 19.1, 18.3 51.2, 51.7 50.0, 50.3 51.4, 51.7 56.8, 56.9 ARI 3.2, 3.1 10.8, 11.1 14.3, 14.6 39.5, 39.6 37.8, 37.9 37.3, 37.4 45.6, 45.4 6k ACC 43.2, 43.4 46.3, 46.6 31.8, 32.0 51.2, 51.4 46.6, 46.6 46.9, 46.8 55.2, 55.0 NMI 13.7, 14.0 30.2, 30.7 13.5, 13.2 51.1, 51.7 48.1, 48.5 51.6, 51.8 58.4, 58.6 ARI 16.6, 16.7 26.3, 26.6 9.8, 9.7 40.9, 41.1 35.7, 35.8 38.0, 37.9 46.8, 46.8 9k ACC 44.3, 44.6 48.3, 49.0 32.3, 32.0 55.7, 55.9 43.4, 43.5 47.8, 47.9 53.4, 53.3 NMI 14.3, 14.7 31.4, 32.1 14.1, 13.5 51.9, 52.7 46.6, 47.4 51.6, 52.0 58.9, 59.0 ARI 17.8, 18.0 30.2, 30.9 10.8, 10.3 43.6, 44.2 34.4, 34.7 38.2, 38.4 46.2, 46.1 The main-text conclusion remains stable across budgets. Across the 144144 test entries in the two tables, ECI-P ranks first in 122/144122/144 entries and within the top two in 134/144134/144, while one ProbPair-family method is best in 137/144137/144 entries. The budget-wise pattern is not completely uniform, as expected: with only 33k constraints, all methods are less stable and the separation between methods is smaller, whereas at 99k several strong baselines also improve and narrow some ACC gaps. Even under these effects, ECI-P remains the leading method at every budget, ranking first in 38/4838/48, 43/4843/48, and 41/4841/48 entries under 33k, 66k, and 99k constraints, respectively. The advantage is clearest on structure-sensitive metrics. ECI-P gives the best NMI in 46/4846/48 entries and the best ARI in 43/4843/48, compared with 33/4833/48 for ACC. Since NMI and ARI better reflect grouping consistency than matched-label accuracy alone, this suggests that ECI-P most reliably improves the learned clustering structure. For example, on single-expert CIFAR100, ECI-P raises NMI from roughly 4444–46%46\% for the strongest non-ECI competitors to 4747–50%50\% across budgets; on multi-expert Reuters, it raises NMI from about 5353–58%58\% for Weighted ProbPair to 6565–69%69\%. Dataset-wise, ECI-P is best in all entries on CIFAR100, Reuters, and STL10, and in nearly all entries on CIFAR10 and ImageNet10. FMNIST and MNIST show more competition from Weighted ProbPair or SpherePair, especially on ACC and low-budget MNIST, but ECI-P remains strongest or near-strongest on NMI/ARI in most cases. The ProbPair variants show a consistent ablation pattern. Weighted ProbPair improves over ProbPair in most entries, and this effect is stronger under multi-expert supervision (67/7267/72 entries) than under single-expert supervision (62/7262/72 entries). This supports the interpretation that information-based weighting is especially useful when low-decisiveness constraints may reflect expert unfamiliarity. ECI-P further improves over Weighted ProbPair in 129/144129/144 entries, showing that weighting alone does not replace explicit estimation, correction, and integration. Notably, VolMaxDCC also provides a useful noise-aware reference among the non-ProbPair baselines. It improves over VanillaDCC and is competitive on some datasets, but reaches the top two in only 15/14415/144 entries. This suggests that explicit noise modeling is beneficial, yet its hard-noise-oriented formulation is not sufficient for the expert-conditioned probabilistic supervision in UPCC. The main caveat remains the severely imbalanced RCV1-10 benchmark. SpherePair remains the main ACC competitor there, consistent with the advantage of its angular geometry for imbalanced pair structures. Nevertheless, ECI-P gives the best NMI across all RCV1-10 budgets and regimes, and is often strongest in ARI. Thus, ECI-P corrects much of the weakness of plain ProbPair/Weighted ProbPair on this benchmark, while the ACC results indicate that severe class imbalance remains a limitation for the probabilistic formulation. D.2 Comparison without pretraining Table 9: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the lv0.01 setting with corruption probability 0.30.3 and 33k/66k/99k constraints. † indicates models without pretraining. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. Vanilla- DCC VolMax- DCC CIDEC† SpherePair† ProbPair† Weighted ProbPair† ECI-P† (Ours) CIFAR100 3k ACC 14.7, 14.7 42.8, 42.5 14.4, 14.2 44.0, 43.8 43.2, 43.0 42.0, 41.9 46.6, 46.5 NMI 11.2, 11.8 42.5, 42.9 11.2, 11.5 41.9, 42.4 39.7, 40.4 38.9, 39.5 45.5, 46.2 ARI 4.0, 4.1 26.2, 26.0 3.7, 3.5 27.5, 27.5 25.7, 25.8 24.9, 25.0 30.5, 30.8 6k ACC 16.5, 16.6 45.4, 45.3 12.3, 12.2 47.6, 47.8 44.7, 44.9 45.2, 45.5 49.3, 49.1 NMI 12.6, 13.5 43.9, 44.5 8.6, 9.4 42.9, 43.9 39.9, 40.8 40.2, 41.0 47.8, 48.3 ARI 5.4, 5.6 28.2, 28.2 2.7, 2.7 30.5, 30.9 26.9, 27.3 27.1, 27.6 33.9, 33.9 9k ACC 16.8, 17.0 46.1, 45.9 12.0, 11.9 48.9, 49.0 47.4, 47.5 47.8, 48.2 52.2, 52.5 NMI 13.5, 14.6 45.6, 46.1 8.1, 9.0 42.5, 43.6 40.9, 42.0 42.1, 43.1 49.3, 50.1 ARI 6.0, 6.3 30.4, 30.5 2.9, 2.9 31.0, 31.4 28.9, 29.2 30.0, 30.4 35.9, 36.3 CIFAR10 3k ACC 34.6, 34.6 79.9, 79.8 39.0, 39.0 80.4, 80.7 83.0, 83.1 82.8, 82.9 84.5, 84.4 NMI 27.9, 28.1 69.3, 69.4 29.1, 29.6 70.8, 71.1 71.8, 72.1 73.3, 73.6 76.0, 75.9 ARI 20.2, 20.3 64.4, 64.4 17.8, 17.7 67.2, 67.4 68.6, 68.7 70.3, 70.4 72.3, 72.1 6k ACC 35.5, 35.7 84.7, 84.6 35.7, 35.9 84.2, 84.5 84.1, 84.4 85.1, 85.3 87.0, 87.0 NMI 27.5, 28.3 73.6, 73.7 27.4, 29.1 71.6, 72.5 73.1, 74.0 74.3, 74.7 78.5, 78.5 ARI 20.7, 21.2 71.0, 70.9 18.6, 19.0 70.2, 70.8 70.8, 71.2 72.4, 72.6 75.8, 75.7 9k ACC 38.3, 38.7 82.5, 82.5 31.9, 32.1 83.3, 84.0 85.2, 85.6 86.1, 86.4 87.7, 87.7 NMI 29.9, 31.2 71.9, 72.2 23.9, 25.8 70.1, 71.7 73.9, 74.8 75.0, 75.8 79.1, 79.0 ARI 23.2, 24.0 68.3, 68.3 16.7, 17.3 68.9, 70.2 71.9, 72.5 73.5, 73.9 76.8, 76.7 FMNIST 3k ACC 36.6, 36.5 63.3, 62.5 36.3, 35.6 64.6, 63.6 58.0, 57.3 61.9, 61.3 67.3, 66.6 NMI 31.1, 31.1 55.5, 55.2 29.2, 28.9 57.7, 57.2 51.8, 51.4 53.2, 53.2 63.1, 62.4 ARI 19.7, 19.5 44.4, 43.6 15.7, 14.9 48.3, 47.3 40.1, 39.3 43.0, 42.6 51.8, 50.7 6k ACC 38.6, 38.5 66.2, 65.1 34.1, 33.6 64.7, 63.7 65.8, 65.0 66.8, 66.0 69.0, 67.7 NMI 30.9, 31.3 56.7, 56.3 27.5, 27.8 54.1, 54.1 55.8, 55.7 56.1, 56.2 64.2, 63.4 ARI 20.7, 20.9 46.6, 45.5 16.4, 16.1 45.9, 45.4 46.6, 45.8 47.3, 46.6 53.1, 51.6 9k ACC 39.8, 40.2 66.5, 65.5 34.7, 34.3 67.6, 66.9 69.6, 68.8 71.0, 69.9 72.4, 71.2 NMI 30.1, 31.3 57.2, 56.8 27.1, 28.0 55.5, 55.9 58.2, 58.6 59.2, 59.3 65.6, 64.7 ARI 21.5, 22.2 46.4, 45.2 15.9, 15.7 48.5, 48.0 49.7, 49.2 51.0, 50.1 55.5, 54.0 ImageNet10 3k ACC 35.4, 37.3 86.7, 86.9 50.8, 52.0 88.5, 89.8 87.3, 87.6 88.9, 89.1 92.5, 92.6 NMI 27.7, 30.8 78.8, 79.4 42.8, 46.5 78.8, 81.6 78.4, 79.3 80.6, 81.4 87.9, 88.2 ARI 19.1, 20.8 74.1, 73.9 30.1, 31.5 77.5, 79.7 75.7, 76.0 77.9, 78.2 85.5, 85.6 6k ACC 40.6, 42.0 87.2, 87.5 32.0, 33.6 87.0, 89.4 88.8, 89.8 89.6, 90.1 92.5, 92.8 NMI 30.6, 35.2 78.3, 79.3 23.3, 29.4 74.9, 80.2 79.2, 81.5 80.9, 82.3 87.8, 88.4 ARI 23.2, 26.0 75.1, 75.6 16.1, 19.4 74.3, 78.7 77.7, 79.4 79.3, 79.9 85.5, 86.0 9k ACC 39.0, 41.5 87.4, 88.1 31.9, 34.5 85.2, 88.9 88.4, 90.1 88.9, 90.0 92.6, 93.1 NMI 27.6, 33.7 77.8, 79.2 23.9, 32.4 71.1, 79.1 78.3, 82.2 79.4, 82.5 87.9, 88.5 ARI 21.1, 25.2 75.2, 76.1 17.1, 22.1 70.9, 77.9 76.8, 79.9 77.8, 79.8 85.6, 86.2 MNIST 3k ACC 25.9, 26.1 65.6, 66.4 41.6, 41.0 72.5, 73.2 67.6, 68.8 70.1, 70.7 80.0, 80.6 NMI 16.0, 17.0 52.7, 54.1 35.1, 35.6 61.5, 63.0 54.0, 55.9 57.2, 58.4 68.6, 69.6 ARI 10.4, 10.8 47.8, 48.9 19.6, 18.8 56.4, 57.7 48.7, 50.4 52.1, 53.1 64.9, 65.9 6k ACC 33.3, 33.9 73.4, 74.4 36.4, 36.3 76.8, 77.9 76.6, 77.5 82.4, 83.2 87.4, 88.0 NMI 24.2, 25.8 58.4, 60.5 29.6, 31.0 63.3, 65.2 62.3, 64.2 67.5, 69.1 75.6, 76.8 ARI 18.4, 19.4 55.2, 56.8 18.2, 18.0 60.1, 61.7 59.3, 60.9 66.5, 68.0 74.9, 76.1 9k ACC 38.9, 40.3 75.6, 76.9 36.0, 36.8 80.1, 81.3 82.2, 83.6 86.0, 87.2 88.7, 89.2 NMI 29.5, 32.2 59.3, 61.6 28.3, 30.9 67.0, 69.6 67.5, 70.2 72.2, 74.6 77.4, 78.5 ARI 23.1, 24.8 56.5, 58.7 19.1, 19.8 65.5, 67.7 66.4, 68.9 72.3, 74.6 77.2, 78.2 REUTERS 3k ACC 53.2, 54.0 62.8, 63.2 58.1, 56.0 76.5, 78.8 69.4, 71.5 71.6, 73.9 80.0, 80.7 NMI 15.1, 17.9 25.6, 28.0 26.2, 30.4 49.5, 54.7 44.4, 48.8 46.7, 51.2 56.3, 57.9 ARI 17.0, 18.0 31.2, 32.4 29.6, 29.7 54.0, 59.2 46.7, 51.3 48.7, 53.3 61.6, 63.2 6k ACC 66.0, 70.6 72.5, 76.8 45.1, 46.1 78.4, 81.2 75.6, 78.2 74.5, 77.1 83.7, 85.2 NMI 27.7, 36.3 36.0, 44.9 9.9, 14.1 50.3, 57.1 48.1, 53.5 48.1, 53.5 59.5, 62.7 ARI 34.0, 42.6 42.4, 50.8 11.3, 14.3 54.5, 61.1 52.6, 57.9 51.2, 56.3 65.0, 68.1 9k ACC 69.7, 77.0 71.7, 76.8 52.9, 54.5 77.5, 82.2 76.1, 80.1 75.3, 79.3 81.9, 83.1 NMI 30.5, 43.3 34.3, 43.4 16.8, 25.5 48.7, 57.0 49.5, 57.5 47.7, 54.6 57.6, 60.2 ARI 36.8, 50.4 40.9, 50.4 19.6, 25.5 52.6, 61.4 52.3, 60.0 51.3, 58.6 62.0, 64.7 STL10 3k ACC 30.9, 31.5 72.8, 73.3 49.2, 50.3 80.9, 82.9 80.3, 81.1 80.8, 81.5 84.2, 84.7 NMI 21.1, 23.4 64.8, 66.5 38.2, 41.9 67.2, 71.0 66.9, 69.2 67.5, 69.2 74.5, 75.1 ARI 14.1, 15.0 56.3, 57.0 27.0, 28.5 64.3, 67.8 64.0, 65.5 64.7, 66.0 71.0, 71.6 6k ACC 30.4, 31.6 76.8, 77.5 35.1, 36.8 78.7, 81.7 80.3, 81.9 81.5, 82.7 85.2, 85.7 NMI 19.5, 22.2 65.8, 67.7 26.8, 31.7 62.6, 68.4 66.3, 69.6 67.9, 70.7 75.2, 76.2 ARI 14.3, 15.7 60.0, 61.3 18.6, 21.3 60.4, 65.6 63.8, 66.4 65.7, 67.8 72.1, 73.0 9k ACC 35.5, 37.8 79.0, 80.1 31.8, 33.4 76.9, 80.8 80.3, 82.8 81.3, 82.7 85.3, 85.7 NMI 22.9, 27.6 66.7, 69.3 23.3, 28.8 59.7, 66.9 66.0, 70.8 67.1, 70.6 75.6, 76.3 ARI 17.1, 19.8 62.8, 64.7 16.4, 19.3 57.6, 64.1 63.6, 67.7 65.3, 67.8 72.4, 73.0 RCV1-10 3k ACC 40.0, 40.1 46.3, 46.4 39.4, 37.4 52.1, 52.2 45.2, 45.1 44.6, 44.6 55.5, 55.7 NMI 12.3, 12.4 21.5, 21.9 21.6, 21.4 51.5, 51.7 42.3, 42.5 43.9, 44.1 54.7, 55.1 ARI 16.4, 16.5 21.2, 21.4 18.3, 19.6 42.6, 42.7 34.5, 34.5 35.0, 35.0 47.6, 47.9 6k ACC 52.4, 52.6 62.1, 62.4 38.5, 37.0 62.0, 61.9 47.0, 47.1 48.2, 48.3 59.7, 59.9 NMI 21.8, 22.2 36.0, 36.7 15.1, 14.9 55.8, 56.2 44.1, 44.5 47.4, 47.4 58.5, 58.8 ARI 28.0, 28.2 42.9, 43.3 16.2, 16.1 52.8, 52.9 37.5, 37.6 39.0, 38.9 51.5, 51.7 9k ACC 46.2, 46.4 61.3, 61.8 41.6, 42.0 66.6, 66.8 45.7, 45.5 49.8, 49.9 64.1, 64.1 NMI 15.5, 15.9 35.7, 36.7 14.4, 14.9 58.0, 58.8 43.6, 44.0 49.7, 49.9 61.4, 61.6 ARI 19.8, 20.1 44.7, 45.4 15.8, 16.4 57.8, 58.4 35.6, 35.6 42.9, 43.0 56.0, 56.0 Table 9 repeats the single-expert comparison without the autoencoder pretraining stage for pretrain-capable methods, denoted by † in the table: CIDEC†, SpherePair†, ProbPair†, Weighted ProbPair†, and ECI-P†. These methods are trained from random initialization, while VanillaDCC and VolMaxDCC are unchanged because they have no pretraining variant. All other settings match Table 7: lv0.01 supervision, corruption probability 0.30.3, and 33k/66k/99k constraints. The main effect of removing pretraining is not a collapse of ECI-P†, but a sharper separation from the other pretrain-capable methods. ECI-P† ranks first in 68/7268/72 test entries and within the top two in 71/7271/72, compared with 59/7259/72 and 66/7266/72 in the pre-trained comparison. It is also best in every entry on seven of the eight datasets; the only non-best entries occur on RCV1-10 ACC/ARI at 66k/99k. This indicates that ECI-P† is less dependent on a favorable autoencoder initialization than the compared deep baselines. The contrast with SpherePair† is also clearer without pretraining. ECI-P† outperforms SpherePair† in 68/7268/72 entries, compared with 62/7262/72 in the pre-trained setting, and achieves the best NMI in all cases. Its absolute NMI margins over SpherePair† range from about 33–12%12\%, with visible gains on FMNIST, MNIST, and STL10. This suggests that the Estimator–Corrector–Integrator structure helps form a more reliable grouping structure when the representation is learned from scratch. The ProbPair variants show a more mixed response to random initialization. Weighted ProbPair† still improves over ProbPair† in most entries, so information-based weighting remains useful. However, Weighted ProbPair† no longer compares as consistently with SpherePair†: it beats SpherePair† in only 33/7233/72 entries, down from 53/7253/72 with pretraining. By contrast, ECI-P† improves over Weighted ProbPair† in all entries, showing that weighting alone is not sufficient when the representation starts from a weaker initialization. The remaining caveat is again the severely imbalanced RCV1-10 benchmark. SpherePair† keeps stronger ACC at 99k and stronger ARI at 66k/99k, consistent with the robustness of its angular geometry under severe imbalance. Nevertheless, ECI-P† gives the best NMI at every budget and substantially improves over ProbPair†/Weighted ProbPair†, indicating stronger grouping consistency despite the remaining imbalance-related ACC/ARI limitation. D.3 Comparison with expert-aware baseline extensions Table 10: Comparative performance (%) (ACC, NMI, ARI) across datasets for ECI-P, SPEA, and WPPEA under the multi3 setting, with corruption probability 0.30.3 and 33k/66k/99k total constraints. SPEA and WPPEA denote the expert-aware ensemble extensions of SpherePair and Weighted ProbPair, respectively. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. 3k 6k 9k ACC NMI ARI ACC NMI ARI ACC NMI ARI CIFAR100 SPEA 42.2, 43.2 41.7, 42.5 26.6, 27.0 41.0, 40.9 39.7, 40.9 25.5, 25.8 41.9, 42.3 39.2, 40.5 26.0, 27.1 WPPEA 43.1, 43.4 41.4, 42.2 26.4, 26.5 41.5, 41.4 40.4, 41.3 25.8, 26.1 46.4, 46.4 42.9, 43.8 29.8, 29.9 ECI-P 45.6, 45.3 45.0, 45.5 29.7, 29.8 49.1, 49.3 47.1, 47.7 32.1, 32.4 48.6, 48.8 48.1, 48.5 32.9, 32.9 CIFAR10 SPEA 76.6, 77.1 70.7, 71.5 64.0, 64.8 75.5, 75.9 67.9, 69.4 62.4, 63.5 74.1, 75.2 67.0, 69.0 61.3, 62.9 WPPEA 82.6, 82.5 74.1, 74.2 69.4, 69.2 81.2, 81.2 74.6, 74.9 69.6, 69.6 82.5, 82.8 74.2, 74.5 70.0, 70.3 ECI-P 82.8, 82.5 76.0, 75.9 70.8, 70.7 84.0, 84.0 77.0, 77.1 72.6, 72.6 81.6, 82.1 77.8, 77.8 73.1, 73.0 FMNIST SPEA 63.2, 61.6 60.3, 59.7 48.4, 46.9 63.5, 62.5 59.4, 59.9 48.5, 48.3 62.5, 62.5 58.6, 60.2 49.0, 49.4 WPPEA 64.5, 64.3 59.7, 59.8 47.9, 47.7 68.0, 68.3 62.1, 62.4 52.0, 52.0 71.8, 71.7 64.6, 64.8 55.2, 55.1 ECI-P 60.1, 59.2 62.7, 61.9 49.5, 48.4 62.0, 61.1 64.3, 63.8 51.0, 51.0 65.3, 64.8 66.0, 65.6 53.4, 52.8 ImageNet10 SPEA 81.7, 85.3 78.0, 79.8 71.8, 75.2 82.1, 86.6 77.8, 79.6 72.8, 75.8 81.2, 81.4 75.3, 76.7 70.6, 71.3 WPPEA 90.5, 90.8 83.8, 84.2 81.8, 82.0 90.9, 91.6 83.4, 84.8 82.2, 83.4 91.2, 92.1 82.4, 84.7 82.2, 83.8 ECI-P 91.5, 91.8 86.9, 87.3 84.1, 84.4 91.9, 92.1 87.2, 87.8 84.6, 85.0 92.4, 92.6 87.7, 88.1 85.4, 85.7 MNIST SPEA 80.5, 81.1 75.2, 76.7 70.7, 71.7 78.6, 79.6 72.7, 75.5 67.9, 70.2 77.5, 78.1 70.2, 73.7 65.7, 67.8 WPPEA 87.9, 88.7 77.2, 78.5 76.2, 77.4 85.8, 86.4 77.4, 79.0 75.5, 76.8 89.2, 90.4 78.1, 80.4 78.2, 80.3 ECI-P 80.9, 81.5 75.7, 77.1 71.9, 73.2 86.1, 86.4 79.8, 80.5 77.8, 78.3 84.7, 85.1 79.9, 80.7 77.1, 77.5 REUTERS SPEA 75.8, 76.1 41.8, 45.7 48.0, 50.2 72.0, 77.0 35.3, 43.8 40.9, 49.8 68.4, 77.0 30.6, 43.1 34.7, 49.6 WPPEA 82.9, 83.3 55.4, 58.3 62.4, 64.5 81.4, 83.7 53.6, 58.4 59.5, 64.4 77.7, 80.9 50.3, 56.0 54.8, 61.5 ECI-P 85.0, 86.3 62.0, 65.0 68.3, 71.1 87.3, 88.5 63.8, 67.2 71.3, 74.3 88.3, 89.6 65.4, 68.7 72.9, 76.0 STL10 SPEA 74.2, 76.7 66.5, 68.1 59.7, 61.1 73.0, 75.4 64.0, 67.3 57.4, 60.1 72.7, 72.9 62.5, 66.2 56.7, 58.7 WPPEA 84.0, 84.5 72.4, 73.8 69.0, 70.0 85.2, 85.6 73.7, 75.2 71.3, 72.0 85.1, 86.1 73.3, 75.4 71.1, 72.7 ECI-P 85.7, 85.9 75.7, 76.3 72.2, 72.5 87.4, 87.5 77.5, 78.1 75.1, 75.4 87.9, 87.7 78.3, 78.5 75.9, 75.8 RCV1-10 SPEA 60.2, 60.9 52.6, 53.1 48.9, 49.5 68.2, 63.5 54.8, 53.8 55.7, 51.4 71.2, 67.7 54.8, 54.4 58.8, 55.9 WPPEA 52.5, 47.6 52.6, 52.5 41.9, 38.3 54.6, 52.6 53.6, 53.7 45.0, 42.8 56.5, 58.4 53.5, 54.2 47.0, 49.2 ECI-P 55.3, 55.2 56.8, 56.9 45.6, 45.4 55.2, 55.0 58.4, 58.6 46.8, 46.8 53.4, 53.3 58.9, 59.0 46.2, 46.1 Table 10 compares ECI-P with SPEA and WPPEA under the multi3 setting, where SPEA and WPPEA denote our expert-aware ensemble extensions of SpherePair and Weighted ProbPair, respectively. As described in Appendix C.4, these extensions instantiate multiple expert-weighted members for a baseline and aggregate their partitions through ensemble clustering. The expert-weighting parameter α is selected on the reserved validation split, with the selected values reported in Table 6. Thus, the comparison should be read as an auxiliary reference rather than a native baseline comparison: SPEA and WPPEA use expert identities, but they also introduce validation tuning and ensemble aggregation, making them inherently uncomparable with the native ECI-P protocol in a strict sense. The extensions substantially strengthen the corresponding baselines, confirming that expert identities carry useful information in the multi-expert setting. WPPEA benefits clearly on datasets such as MNIST, FMNIST, and STL10, while SPEA gains competitiveness on Reuters and RCV1-10. However, these gains conflate two effects: using expert identities and ensembling multiple expert-weighted members. For this reason, the results are best interpreted as stronger expert-aware references, not as fully symmetric replacements for the native baselines in Table 1. Even against these stronger references, ECI-P remains the most reliable method overall. It obtains the best result in 55/7255/72 test entries and ranks within the top two in 68/7268/72. The advantage is clearest in NMI, where ECI-P is best in 23/2423/24 cases, indicating that its expert-conditioned design most consistently improves grouping structure. Dataset-wise, ECI-P is best in all entries on CIFAR100, ImageNet10, Reuters, and STL10, and remains strongest in most CIFAR10 entries. The main losses occur on FMNIST and MNIST, mostly on ACC/ARI, where WPPEA is particularly strong, and on RCV1-10 ACC/ARI, where the angular geometry of SPEA remains advantageous under severe imbalance. Overall, expert-aware ensembling makes strong baselines substantially more competitive, but ECI-P still provides a more direct and generally stronger way to use expert identities without relying on a validation-tuned ensemble wrapper. Figure 5: Full robustness results in ACC (mean± over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 99k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions. Figure 6: Full robustness results in NMI (mean± over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 99k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions. Figure 7: Full robustness results in ARI (mean± over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 99k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions. D.4 Robustness across supervision conditions Figs. 5, 6 and 7 report the full ACC, NMI, and ARI results corresponding to the robustness study in the main text. All three sweeps use a fixed budget of 99k constraints, and the central setting in each sweep, namely lv0.01 expert quality, corruption probability 0.30.3, and multi3, matches the default setting used in Table 1. Thus, these figures also cover the main-comparison results in Table 1, while additionally reporting the standard deviations omitted from the table for space. For the multi-expert sweep, we further include SPEA and WPPEA, the expert-aware ensemble extensions introduced in Appendix D.3, as stronger references that also exploit expert identities. Effects of supervision factors. Across the three sweeps, the results largely follow the expected supervision-quality ordering. In the single-expert sweep, stronger experts generally yield better clustering quality; in the corruption sweep, increasing corruption leads to clear degradation. In the multi-expert sweep, moving from multi2 to multi10 also tends to help the stronger methods, because the protocol progressively reduces expert blind spots and increases average expert accuracy, as quantified in Table 4. Since the total constraint budget is fixed throughout, the performance trends mainly reflect changes in supervision quality and heterogeneity rather than in the amount of supervision. Baseline behavior. The non-ECI baselines show a clear hierarchy as supervision becomes less reliable. Plain end-to-end methods such as VanillaDCC and CIDEC often nearly collapse under low-quality or heterogeneous supervision, especially in NMI and ARI, suggesting that cluster-assignment training is vulnerable when pairwise supervision is inconsistent. VolMaxDCC is usually more resilient, consistent with the benefit of explicit noise modeling. SpherePair and the ProbPair-family baselines are substantially stronger than the plain end-to-end methods, reflecting the robustness of geometric representation learning under noisy supervision. Nevertheless, corruption still causes visible degradation for the stronger baselines, particularly on FMNIST, ImageNet10, and STL10. ECI-P robustness. Across the supervision sweeps, ECI-P better preserves clustering quality than the native baselines, especially as expert quality decreases or corruption increases. This advantage is most consistent on NMI and ARI, suggesting that the correction-and-integration pipeline mainly improves the recovered grouping structure. ACC is more competitive and contains more exceptions, especially on FMNIST, MNIST, and RCV1-10, where Weighted ProbPair, SPEA, or WPPEA can sometimes match or exceed ECI-P. This is consistent with the main-comparison results (Appendix D.1): matched-label accuracy can favor angular or ensemble-based baselines on some datasets, even when ECI-P remains stronger or near-stronger on grouping-oriented metrics (NMI/ARI). The expert-aware extensions further clarify the role of expert identities: SPEA and WPPEA often improve over their native counterparts in the multi-expert sweep, confirming that expert identity information is useful under heterogeneous supervision. However, these extensions rely on validation-selected expert weighting and ensemble aggregation, whereas ECI-P uses expert-conditioned correction within the learning pipeline. Overall, the full results support the main-text conclusion that ECI-P better preserves clustering quality when supervision is fallible, heterogeneous, or corrupted. D.5 Held-out diagnostics Diagnostic protocol. The held-out diagnostics complement the clustering results by tracking the training dynamics of ECI-P’s internal quantities. Following Appendix C.4, we construct a sample-disjoint diagnostic constraint set diagC^diag from the test split, used only for diagnosis and never for optimization, early stopping, or hyperparameter selection. Benchmark labels are used only to define an external hard co-membership reference, rather than the canonical aleatoric relation R⋆R . All diagnostics use multi3 supervision with limited 33k training constraints in total (11k per expert) and corruption rate 0.30.3. To reduce the influence of a strong initial representation or a long warm-up phase, we use random initialization without unsupervised pretraining, give the Estimators and Integrator only one warm-up epoch, and then track the next 500500 training iterations, each corresponding to one epoch. We report mean± over 55 runs, recording at each iteration the Estimator out-of-fold relation y^ioof y^oof_i, the corrected relation yicory^cor_i, and the Integrator relation yiinty_i^int on diagC^diag. Figure 8: CIFAR100-20 held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 9: CIFAR10 held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 10: FMNIST held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 11: ImageNet10 held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 12: MNIST held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 13: Reuters held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 14: STL10 held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Figure 15: RCV1-10 held-out diagnostic evolution under multi3 with corruption rate 0.30.3 (mean± over 5 runs). Same panel definitions as in Fig. 3. Diagnostic metrics. For a held-out pair i=(ai,bi)i=(a_i,b_i) with benchmark labels tait_a_i and tbit_b_i, define the external hard co-membership reference oi⋆:=[tai=tbi].o_i :=I[t_a_i=t_b_i]. Let ℐclean=i∈diag:ci=0I_clean=\i ^diag:c_i=0\ be the uncorrupted held-out subset, and define the residual discrepancy after correction as gapi=|softclipξ(y^ioof)−yicor|.gap_i= |softclip_ξ( y^oof_i)-y^cor_i |. For any relation score vi∈[0,1]v_i∈[0,1] and index set ℐI, we use Brier(v,ℐ)=1|ℐ|∑i∈ℐ(vi−oi⋆)2.Brier(v;I)= 1|I| _i (v_i-o_i )^2. We report Brierest=Brier(y^oof,diag)Brier_est=Brier( y^oof;C^diag), Briercor,clean=Brier(ycor,ℐclean)Brier_cor,clean=Brier(y^cor;I_clean), Brierint=Brier(yint,diag)Brier_int=Brier(y^int;C^diag), and [gapi,clean]=|ℐclean|−1∑i∈ℐcleangapiE[gap_i,clean]=|I_clean|^-1 _i _cleangap_i. For reliability-aware screening, gapigap_i is used to rank held-out constraints, and we report AUCcorruptAUC_corrupt and APcorruptAP_corrupt against the known corruption indicator cic_i. These scores measure alignment with corruption rather than pure corruption estimation, since the reliability signal may also reflect difficult clean pairs, residual epistemic variation, and imperfect estimation/correction; nevertheless, corrupted records are expected to rank higher on average. As a reference, we also replace the multi3 expert-generated judgments by oi⋆o_i before applying the same 33k budget and corruption rate 0.30.3, thereby removing expert epistemic uncertainty from the clean supervision. Evolution of internal estimates. The full diagnostic curves are shown in Figs. 8, 9, 10, 11, 12, 13, 14 and 15. Panels (A–C) track the Estimator, Corrector, and Integrator relation scores through their Brier errors against the external hard co-membership reference oi⋆o_i . Across datasets, these errors typically decrease quickly in the early stage and then become relatively stable, with some late-stage fluctuation. This pattern is clearer on CIFAR100-20, CIFAR10, ImageNet10, MNIST, and STL10, while FMNIST and Reuters show more rebound after the initial improvement, and RCV1-10 is the most unstable case. Because these Brier scores use oi⋆o_i rather than the latent aleatoric relation Raibi⋆R_a_ib_i , small non-monotone changes should be read as diagnostic behavior rather than as direct evidence about latent-target recovery. Panel (D) gives a complementary correction-side view: the clean residual discrepancy usually drops early, indicating that corrected relations become more aligned with estimator beliefs on uncorrupted held-out constraints. Since the Estimator and Corrector co-evolve, this quantity should be interpreted together with (A–C), as evidence that the correction stage follows the improving estimator signal. Screening against injected corruption. Panels (E,F) evaluate whether the reliability signal aligns with the injected corruption indicator. Under multi3, AUCcorruptAUC_corrupt and APcorruptAP_corrupt are generally above their chance baselines, so corrupted records tend to be ranked higher by the screening signal. The alignment is clearer on most image datasets, especially CIFAR100-20, CIFAR10, ImageNet10, MNIST, and STL10, but weaker or noisier on Reuters and particularly RCV1-10. This limitation reflects the heuristic nature of the screening signal: unreliability may arise not only from stochastic corruption, but also from expert-side epistemic variation, imperfect estimation/correction, and partial self-confirmation during training. The oracle-reference curves in panels (G,H) support this interpretation, as they usually track corruption more closely than the multi3 curves when the clean supervision is generated from the hard oracle relation before corruption. Summary. Overall, the diagnostics provide consistent qualitative evidence for the intended behavior of ECI-P, although the curves are not uniformly monotone across datasets. The internal estimates improve mainly in the early stage, the correction discrepancy is reduced, and the screening signal remains meaningfully associated with injected corruption. RCV1-10 is the most challenging case, consistent with the broader experimental picture under severe class imbalance, where improvements in clustering structure are less uniformly reflected by the held-out diagnostic curves. Early stabilization further suggests that the ECI components often reach useful internal agreement before the fixed training horizon; however, we keep this schedule here and leave principled early stopping for future work. These diagnostics therefore provide practical held-out evidence for ECI-P’s intended behavior, while remaining proxy measurements rather than exact tests of latent aleatoric recovery or pure corruption detection. D.6 Sensitivity and ablation studies We further study the main ECI-P design choices around the default configuration specified in Section 5.1. All sensitivity runs use the default single-expert supervision condition, i.e., lv0.01 expert quality, corruption rate 0.30.3, and 99k constraints. To expose the effect of each design choice, we use random initialization without unsupervised pretraining and vary one factor while keeping the others fixed. Figs. 16, 17, 18, 19, 20, 21, 22 and 23 report the corresponding test ACC, NMI, and ARI. Estimator choices. Figs. 16 and 17 examine the Estimator-side choices. For the number of folds, increasing K from 22 to 2020 gives mild gains on several datasets, such as MNIST, FMNIST, CIFAR10, and ImageNet10, but the curves are mostly flat and the improvement is not proportional to the extra cross-fitting cost. This supports using K=5K=5 as a practical default: it avoids the weakest cross-fitting setting while keeping the main Estimator cost moderate. The information-based warm-up weight κi _i provides a small but consistent benefit. Enabling it is usually better or comparable across datasets, with visible gains on MNIST, FMNIST, Reuters, and CIFAR10, indicating that informative constraints are useful for stabilizing the early Estimator signal. This agrees with the main comparison in Table 1, where Weighted ProbPair generally improves over ProbPair by using the same information-based weighting principle. At the same time, the gains remain moderate, suggesting that ECI-P benefits from this design without being overly dependent on it. Figure 16: Sensitivity to the number of estimator folds K. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 17: Ablation of the information-based warm-up weight κi _i. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 18: Sensitivity to Corrector capacity. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 19: Sensitivity to the Corrector regularization strength λcor _cor. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Corrector choices. Figs. 18 and 19 study Corrector capacity and regularization. For capacity, we compare a large encoder-sized Corrector (500500–500500–20002000), a medium classifier-sized Corrector (512512–512512), and the default small Corrector (6464–1616). The three sizes perform similarly on most datasets, and this supports the default use of a small Corrector: it is cheaper and also limits the risk of fitting arbitrary pair-specific deviations rather than structured expert-conditioned correction. The regularization strength λcor _cor shows the same trade-off more directly. Compared with λcor=0 _cor=0, mild to moderate regularization can help on datasets such as FMNIST, CIFAR10, CIFAR100-20, and RCV1-10, but overly strong regularization is harmful on some datasets, most clearly Reuters when λcor _cor reaches 55 or 1010. Thus, the Corrector should not be left completely unconstrained, but it also should not be forced too strongly toward zero correction. The default λcor=0.5 _cor=0.5 lies in the stable middle range. Reliability screening and boundary handling. Figs. 20, 21 and 22 evaluate the parameters used after correction. The screening sharpness γ is robust over the tested range, especially for γ∈[5,20]γ∈[5,20]. Larger values are useful on some difficult datasets, notably RCV1-10 and also Reuters or MNIST in some metrics, indicating that stronger suppression of unreliable constraints can help when supervision is more fragile. The Bayesian-confidence scale n0n_0 is also stable overall. Larger n0n_0 improves RCV1-10 and slightly helps Reuters, suggesting that these datasets benefit from assigning more evidence to the refined supervision, whereas FMNIST does not benefit from the same increase. Thus, n0n_0 controls a real reliability trade-off, but the default value remains within a safe region. Finally, performance changes little as the soft-clipping sharpness ξ varies: most datasets change little from ξ=5ξ=5 to 5050, with only mild fluctuations on RCV1-10 and a few small metric-specific changes. This supports soft clipping as a robust boundary-handling design for keeping corrected relations in a valid probability range without relying on a finely tuned clipping sharpness. Reconstruction strength. Fig. 23 shows a noticeable dataset-dependent trade-off for λrec _rec. The default value λrec=0.02 _rec=0.02 follows SpherePair [55] and remains a reliable setting, but ECI-P often benefits from moderately stronger reconstruction. For example, MNIST improves steadily as λrec _rec increases, while Reuters and RCV1-10 prefer a moderate range around 0.050.05–0.20.2 before performance drops at larger values. In contrast, FMNIST and CIFAR10 do not benefit from overly large reconstruction weights, and ImageNet10 is almost unchanged. This suggests that reconstruction provides useful supervision-independent structure for out-of-fold estimation and final integration, but too much reconstruction can compete with clustering-oriented pairwise learning on some datasets. When validation-based tuning is available, λrec _rec is therefore a meaningful parameter to adjust; without such tuning, the shared default remains a fair and stable compromise. Figure 20: Sensitivity to the reliability-screening sharpness γ. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 21: Sensitivity to the Bayesian-confidence scale n0n_0. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 22: Sensitivity to the soft-clipping sharpness ξ. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Figure 23: Sensitivity to the reconstruction weight λrec _rec. Each panel reports test ACC, NMI, or ARI across datasets (mean± over 5 runs). Overall, the sensitivity and ablation results support the use of the default ECI-P configuration in the main experiments. Estimator, Corrector, screening, and soft-clipping choices are stable over broad ranges, while the more dataset-dependent reconstruction weight changes performance gradually rather than causing abrupt failure. The results therefore indicate that ECI-P is not hyperparameter-fragile, even though dataset-specific validation could still improve individual settings when such tuning is allowed. Appendix E Learning efficiency Table 11: Overall wall-clock training time for different methods on eight datasets under the multi3 multi-expert supervision setting with 99k constraints and corruption probability 0.30.3, measured on a single NVIDIA A100 40GB GPU. Methods marked with ∗ require hyperparameter tuning, and their corresponding times are underlined. CIFAR100-20 CIFAR10 FMNIST ImageNet10 MNIST Reuters STL10 RCV1-10 VanillaDCC 1m12s 1m10s 1m20s 0m45s 1m27s 0m37s 0m41s 1m51s VolMaxDCC∗ 10m23s 45m21s 1h43m18s 6m20s 1h35m30s 5m00s 3m06s 2h39m25s CIDEC 18m27s 19m26s 24m49s 5m03s 25m06s 5m53s 6m03s 1h07m26s SpherePair 18m27s 17m58s 22m25s 5m09s 22m54s 6m05s 5m18s 1h01m54s SpherePair (EA)∗ 3h53m10s 3h40m47s 4h30m35s 1h49m25s 4h33m48s 2h12m12s 1h51m31s 10h40m49s ProbPair 18m19s 17m42s 22m16s 4m57s 22m41s 5m49s 5m02s 1h01m36s Weighted ProbPair 18m24s 17m44s 22m23s 5m07s 22m56s 5m52s 5m06s 1h01m37s Weighted ProbPair (EA)∗ 3h54m42s 3h34m20s 4h24m15s 1h44m36s 4h22m09s 2h01m15s 1h46m31s 10h32m39s ECI-P (Ours) 29m48s 29m12s 34m01s 15m49s 34m39s 17m34s 15m59s 1h15m41s Overall training time. Appendix B.2 analyzes the computational complexity of ECI-P; here, based on the implementation in Appendix C.5, we report empirical wall-clock measurements for the complete training pipelines in Table 11. Each entry is measured under the multi3 setting with 99k constraints and corruption probability 0.30.3, and covers the full practical training procedure: (i) for methods requiring unsupervised pretraining, the reported time includes the pretraining stage; (i) for methods marked with ∗, it also includes three scans over the corresponding hyperparameter list and a final training run with the selected hyperparameter. When both pretraining and hyperparameter tuning are required, the timing includes pretraining on the training split excluding validation samples before tuning, followed by pretraining on the full training split before the final run. Runtime comparison. The runtimes in Table 11 should be interpreted alongside the clustering results in Appendix D.1. VanillaDCC is fastest, but mainly because collapsed assignment learning triggers early stopping; its performance in Table 8 is far below the competitive methods. VolMaxDCC has variable cost, becoming expensive on FMNIST, MNIST, and RCV1-10 due to validation-based hyperparameter search. For methods with autoencoder pretraining, including CIDEC, SpherePair, ProbPair, Weighted ProbPair, and ECI-P, overall costs remain on the same order, with pretraining as a shared pipeline component. ECI-P adds moderate overhead from estimator cross-fitting and correction/integration, but this overhead is modest relative to its empirical gains; in more resource-limited settings, for example when unsupervised pretraining is unavailable, Appendix D.2 shows an even larger performance advantage. The expert-aware SpherePair and Weighted ProbPair extensions are most expensive because ensemble construction and hyperparameter tuning scale with the number of experts. In contrast, Appendix B.2 shows that ECI-P incorporates expert identities through expert-conditioned correction without replicating the full training pipeline, so its leading cost scales mainly with the constraint budget rather than the expert count.