Paper deep dive
Candidate Retention for Abductive Learning
Hao-Yuan He, Yu Liu, Ming Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Abductive learning combines neural perception with symbolic reasoning, using explanations generated by abduction to supervise the perception model. Multiple valid explanations of the same symbolic target can assign conflicting labels to the same inputs. Common policies select a single candidate as a pseudo-label, which may reinforce mistaken assignments, or weight all candidates, which may spread supervision across competing labels. These risks motivate selecting a retained subset to balance supervision sharpness and model-mass coverage. To guide this choice, we bound the coordinate-level supervision error using retained uncertainty, discarded model mass, and model mismatch. For a fixed model and training pair, only the first two terms depend on the retained set. We propose Abductive Candidate Retention (ACR), which uses these terms to guide greedy additions, accepting a candidate when its recovered mass exceeds the increase in retained uncertainty. Experiments show that ACR improves concept accuracy over single-candidate baselines and A3BL in most evaluated aggregated mod-addition settings. Objective ablations support the joint use of uncertainty and posterior mass.
Tags
Links
- Source: https://arxiv.org/abs/2609.39561v1
- Canonical: https://arxiv.org/abs/2609.39561v1
Trouble viewing inline? Open PDF directly →
Full Text
47,474 characters extracted from source content.
Expand or collapse full text
Candidate Retention for Abductive Learning Hao-Yuan He Yu Liu Ming Li School of Artificial Intelligence, Nanjing University National Key Laboratory for Novel Software Technology, Nanjing University hehy, liuy, lim@lamda.nju.edu.cn Abstract Abductive learning combines neural perception with symbolic reasoning, using explanations generated by abduction to supervise the perception model. Multiple valid explanations of the same symbolic target can assign conflicting labels to the same inputs. Common policies select a single candidate as a pseudo-label, which may reinforce mistaken assignments, or weight all candidates, which may spread supervision across competing labels. These risks motivate selecting a retained subset to balance supervision sharpness and model-mass coverage. To guide this choice, we bound the coordinate-level supervision error using retained uncertainty, discarded model mass, and model mismatch. For a fixed model and training pair, only the first two terms depend on the retained set. We propose Abductive Candidate Retention (ACR), which uses these terms to guide greedy additions, accepting a candidate when its recovered mass exceeds the increase in retained uncertainty. Experiments show that ACR improves concept accuracy over single-candidate baselines andA 3 BLin most evaluated aggregated mod-addition settings. Objective ablations support the joint use of uncertainty and posterior mass. 1 Introduction Neuro-symbolic learning integrates neural perception with symbolic reasoning to learn from data and background knowledge. Abductive learning (ABL, Zhou 2019) combines neural perception with symbolic reasoning through abduction (Magnani, 2009): a perception model proposes latent concepts from raw inputs, and when these proposals are inconsistent with the observed evidence, abduction generates candidate concept assignments that entail it. These candidates then supervise the perception model. However, the background knowledge does not always determine a unique candidate. Consider SUM(,) = 3: the addition rule admits[= 0,= 3],[= 3,= 0], and other assignments. Because different candidates label the same images differently, they create ambiguity in the concept- level supervision. Classical ABL selects one candidate as a pseudo-label using the current model (Dai et al., 2019). A 3 BLnormalizes all valid candidates into a posterior and marginalizes it into soft labels (He et al., 2024b). Selecting one explanation yields sharp labels but can reinforce a mistaken assignment. Retaining more explanations preserves alternatives, while their disagreements can spread supervision across competing labels. We call the choice of which candidates to include candidate retention. A candidate’s contribution to supervision depends on the label support already retained. Two candidates with equal weights can therefore change supervision differently: one may reinforce supported labels, while the other adds competing labels. This combination-dependent effect motivates evaluating each addition through the supervision formed by the retained set. 1 arXiv:2609.39561v1 [cs.LG] 30 Sep 2026 Machine LearningLogical Reasoning Abductive reasoner Supervision construction Single-candidate Candidates Abductive Candidate Retention Candidates Full-candidate Candidates single / sharpselective / balancedfull / diffuse Instances Perception model Observed evidence Candidate set Knowledge base train with x f θ y s(x, y) KB c 1 c 2 c 3 c K · f θ (x) s(x, y)7→ ̃p ̃p c 1 1 c 2 0 c 3 0 c 4 0 c 5 0 ̃p c 1 0 c 2 0.36 c 3 0.46 c 4 0.18 c 5 0 ̃p c 1 0.15 c 2 0.28 c 3 0.32 c 4 0.17 c 5 0.08 ̃p Figure 1: Candidate retention in the ABL loop. Upper: abduction returnss(x,y); supervision construction marginalizes the retained candidates’ weighted labels to train the perception model. Lower: three retention policies with schematic candidate weights ̃p. Shading indicates which candidates are retained. We propose Abductive Candidate Retention (ACR), which evaluates additions through their recovered model mass and change in coordinate uncertainty (Figure 1). An upper bound on coordinate supervision error contains retained uncertainty, discarded model mass, and model mismatch. For a fixed model and training pair, only the first two depend on the retained set. LetM θ (R;x,y) denote the retained mass and U θ (R;x,y) the average coordinate uncertainty. We score a retained setRby J θ (R; x,y) = U θ (R; x,y) + 1− M θ (R; x,y). ACR starts from the highest-weight candidate and adds the candidate that most decreases this criterion, stopping when no addition improves it. Marginalizing the retained weights then supplies concept targets for the neural update. Our contributions: 1. We analyze candidate retention through the concept supervision induced by a subset and give an upper bound on its coordinate-level error. 2.We develop a greedy rule that evaluates added candidate mass together with changes in coordinate uncertainty, adapting the retained set’s composition and size. 3.We show concept-accuracy gains on most evaluated mod-addition settings, with controlled comparisons and training diagnostics, and evaluate ACR on chess attack and judicial sentencing. 2 Problem Setup LetXbe the input space,Za finite concept set, andYthe target space. In typical ABL tasks, a single symbolic target constrains multiple inputs jointly. Each training example therefore contains 2 an input sequencex= (x j ) m j=1 ∈X m and an observed targety ∈Y. The conceptz j ∈Zfor each input x j is hidden. ABL (Zhou, 2019) combines a perception model with background knowledgeKB. For each input x, the model assigns a probability f θ (k | x) to every concept k ∈Z. The knowledge base defines the candidates that entail the observed target: s(x,y) =c∈Z m : c∧ KB|= y. Training revealsy, but it does not identify whichc∈ s(x,y) matches the inputs. ABL uses these valid candidates to learn the hidden concepts. Example 1. ConsiderSUM(,)=3. The input contains two unlabeled digit images, and3is their observed sum. The valid candidates are[0,3],[1,2],[2,1], and[3,0]. Only one describes the image contents, but its digit labels are unavailable during training. 2.1 Existing ABL Supervision Classical ABL selects one valid candidate and uses it as supervision (Dai et al., 2019). It commonly ranks candidates by confidence or Hamming distance. Under the factorized perception model, candidate confidence is p θ (c| x) = m Y j=1 f θ (c j | x j ). A Hamming score measures the negative distance betweencand the predicted concept sequence. Writing either score generically as score θ , ABL selects ˆ c θ ∈ arg max c∈s(x,y) score θ (c; x). The selected candidate provides hard labels for updatingf θ . The updated model then reranks the candidates. A 3 BLkeeps all valid candidates and weights them by confidence (He et al., 2024b). Their total confidence is Z θ (x,y) = X c∈s(x,y) p θ (c| x). Normalizing by Z θ (x,y) gives the candidate posterior w θ (c| x,y) = p θ (c| x) Z θ (x,y) ,c∈ s(x,y). We assume s(x,y)̸= ∅ and Z θ (x,y) > 0. A 3 BLmarginalizes this posterior at each sequence position to construct soft concept targets for f θ . Classical ABL andA 3 BLprovide the single-candidate and posterior-marginalization reference policies. We next consider how the retained subset shapes concept supervision and use this information to guide selection. 3 Abductive Candidate Retention ACR scores retained sets, converts them into concept targets, and constructs them by greedy selection. Throughout this section,w θ (c| x,y) denotes a normalized model-induced distribution over the nonempty valid sets(x,y). The product posterior above is one choice; task-specific scores are given in Appendix B. 3 3.1 A Retention Criterion For a training pair, letR ⊆ s(x,y) be the retained candidate set. Conditioning the candidate distribution on R gives its retained mass and conditional distribution: M θ (R; x,y) = X c∈R w θ (c| x,y), q θ,R (c| x,y) = w θ (c| x,y) M θ (R; x,y) I[c∈ R], where M θ (R; x,y) > 0. At position j, the marginal of q θ,R is π θ,R,j (k) =Pr c∼q θ,R (·|x,y) (c j = k). Retained uncertainty averages one minus the largest marginal probability across positions: U θ (R; x,y) = 1 m m X j=1 1− max k∈Z π θ,R,j (k) . A lower U θ (R;x,y) gives sharper concept targets. The discarded posterior mass is 1− M θ (R;x,y). To relate the retained supervision to concept error, letP ⋆ (· | x,y) be the unknown true distribution over valid candidates. It is a point mass when each input sequence has one true candidate. The retained prediction at position j is ˆz R,j (x,y)∈ arg max k∈Z π θ,R,j (k). Its ideal risk is the expected coordinate error under P ⋆ : R ⋆ (R; x,y) = E c∼P ⋆ (·|x,y) 1 m m X j=1 I[ˆz R,j (x,y)̸= c j ] . SinceP ⋆ is unknown, we compare the induced supervision with this distribution through the following upper bound. Theorem 1. For any positive-mass set R⊆ s(x,y), R ⋆ (R; x,y)≤ U θ (R; x,y) + TV(P ⋆ ,w θ ) + 1− M θ (R; x,y), where TV denotes total variation distance and both distributions are conditioned on (x,y). Proof. WriteP=P ⋆ ,w=w θ ,q R =q θ,R , andM R =M θ (R), suppressing (x,y). The coordinate loss ℓ R (c) = m −1 P j I[ˆz R,j ̸= c j ] lies in [0, 1], so E P [ℓ R ]≤ E q R [ℓ R ] + TV(P,q R ). The first term equals U θ (R). OnR,q R =w/M R ≥ w; outsideR,q R = 0. The absolute differences sum to 1− M R on each part, so TV(w,q R ) = 1− M R . By the triangle inequality, TV(P,q R )≤ TV(P,w) + TV(w,q R ) = TV(P,w) + 1− M R . Combining the two inequalities proves the result. 4 Interpretation. Underq θ,R , the coordinate error equals retained uncertainty. Model mismatch and discarded mass bound the change in expected error when moving toP ⋆ . Thus retained mass describes coverage under the model distribution, and its relation to the true explanation depends on model mismatch. For a fixed model and training pair, the mismatch term is constant acrossR. ACR uses retained uncertainty and discarded mass, the two subset-dependent terms in the bound, to score candidate additions: J θ (R; x,y) = U θ (R; x,y) + 1− M θ (R; x,y). LetC K contain themin(K,|s|) highest-weight candidates, with fixed tie-breaking. Weights remain normalized overs, so discarded mass includes candidates outside this search pool. The full valid set globally minimizesJ θ ; the criterion alone does not favor partial retention. ACR uses it to evaluate individual additions along a greedy path from the highest-weight singleton and stops when no addition improves it. This local rule can yield intermediate subsets. Their benefit over the full set is evaluated empirically and is not guaranteed by the bound. 3.2 Retained Concept Supervision At roundt, the current parametersθ t and retained setR t define soft targetsπ t,j (k) =π θ t ,R t ,j (k) and massM t =M θ t (R t ;x,y). Holding these quantities fixed during the neural update gives the normalized cross-entropy loss L ret (θ;π t ) = m X j=1 X k∈Z π t,j (k)[− logf θ (k | x j )]. Marginalization pools label support across joint explanations. Several candidates can agree at one position while assigning different labels elsewhere; their weights add to the same target entry at that position. Every input in a sequence receives a target derived from the same retained distribution. A singleton produces one-hot targets, while retaining the full valid set recovers its full-distribution marginals. Intermediate sets preserve the alternatives that contribute to the selected supervision. In aggregated mod addition, the implementation uses the original retained weights to form the targets: a t,j (k) = X c∈R t w θ t (c| x,y)I[c j = k] = M t π t,j (k). Its loss isM t L ret (θ;π t ), withM t also fixed during the update. The conditional marginals specify the relative label distribution, and retained mass weights the training example. Appendix B gives the scoring and weighting conventions. The standard EM connection for fixed sets, product-posterior weights, and normalized targets is described in Appendix A.1. 3.3 Greedy Candidate Retention We constructR ⊆ C K by forward selection, starting with the highest-weight candidate. The set expands only when adding a candidate lowers the criterion. To see how a candidate changes supervision, writeM=M θ (R;x,y),w c =w θ (c | x,y), and π j = π θ,R,j . Adding c /∈ R yields marginals π + j = π θ,R∪c,j with π + j (k) = Mπ j (k) + w c I[c j = k] M + w c . 5 The candidate contributes mass to its assigned label at each position. The effect on uncertainty depends on where this support falls relative to the existing label masses. Two candidates with equal weights can therefore produce different changes in uncertainty. A weight-only top-krule ranks them independently of the current retained set. ACR evaluates their effects on the supervision already formed by that set, using the following change in the criterion. For c∈ C K \ R, define ∆ c J θ (R)≡ J θ (R∪c; x,y)− J θ (R; x,y) = U θ (R∪c; x,y)− U θ (R; x,y) − w θ (c| x,y). The first difference measures the change in retained ambiguity. The final term rewards the posterior mass recovered by c. At each step, we select the candidate with the smallest ∆ c J θ (R). We add it if this value is negative and stop otherwise. Algorithm 1 Greedy Candidate Retention Require: Pool C K and full-distribution weights w θ (c| x,y) Ensure: Retained set R 1: Choose c 0 ∈ arg max c∈C K w θ (c| x,y). 2: Set R←c 0 . 3: while R̸= C K do 4: Choose c † ∈ arg min c∈C K ∆ c J θ (R). 5: if ∆ c † J θ (R)≥ 0 then 6:break 7: end if 8: Update R← R∪c † . 9: end while 10: return R. Each accepted step strictly lowersJ θ and adds one candidate, so the algorithm terminates at a one-addition local optimum overC K . With maintained coordinate label masses, selection takes O(N 2 m) time in the worst case, whereN=|C K |≤ K. At roundt, the selectedR t defines the fixed targets in Section 3.2. Candidate weights and retention are recomputed after the neural update. 4 Experiments We empirically evaluate ACR through two questions: •How does candidate retention compare with the single-candidate and posterior-weighted policies when valid candidates disagree on hidden concepts? •What roles do the two terms in the objective play, and how does the retained set change during training? 6 Table 1: Character accuracy on digit addition (%; mean±std). N/A: did not finish within 24 hours. MNISTKMNIST Methodn = 1n = 2n = 3n = 4n = 1n = 2n = 3n = 4 DeepStochLog 99.00 ±0.13 98.88 ±0.02 98.94 ±0.15 98.98 ±0.13 94.00 ±0.73 94.06 ±0.30 94.00 ±0.29 93.93 ±0.49 NeurASP50.01 ±0.44 10.21 ±0.39 10.21 ±0.39 N/A10.01 ±0.58 10.05 ±0.51 N/AN/A DeepProbLog 97.42 ±0.29 N/AN/AN/A80.54 ±0.21 N/AN/AN/A ABL-hamming 98.49 ±0.22 98.65 ±0.08 98.63 ±0.12 60.41 ±40.27 91.48 ±0.54 92.43 ±0.65 63.51 ±36.91 47.74 ±37.13 ABL-confidence 79.31 ±41.35 79.32 ±41.61 77.10 ±43.53 79.22 ±41.65 86.32 ±25.40 79.80 ±34.91 72.76 ±38.75 54.84 ±42.97 A 3 BL98.92 ±0.15 99.03 ±0.09 98.68 ±0.17 98.42 ±0.08 94.84 ±0.25 94.31 ±0.21 93.16 ±1.00 92.97 ±0.44 ACR (ours)98.95 ±0.04 99.06 ±0.18 99.02 ±0.07 98.93 ±0.08 94.44 ±0.28 95.12 ±0.41 94.44 ±0.18 94.35 ±0.50 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test -3.1% (2,7) 12040 0.0 0.5 1.0 test +2.4% (2,9) 12040 0.0 0.5 1.0 test +65.0% (3,4) 12040 0.0 0.5 1.0 test +14.1% (3,5) 12040 0.0 0.5 1.0 test +59.1% (3,7) 12040 0.0 0.5 1.0 test +45.1% (3,8) 12040 0.0 0.5 1.0 test +72.0% (4,5) 12040 0.0 0.5 1.0 test +50.8% (4,7) 12040 0.0 0.5 1.0 test +48.6% (4,9) 12040 0.0 0.5 1.0 test +61.4% (5,6) 12040 0.0 0.5 1.0 test +60.6% (5,7) 12040 0.0 0.5 1.0 test +56.2% (5,8) 12040 0.0 0.5 1.0 test +54.9% (5,9) 12040 0.0 0.5 1.0 test +34.5% (6,7) 12040 0.0 0.5 1.0 test +33.7% (6,8) 12040 0.0 0.5 1.0 test +31.2% (6,9) 12040 0.0 0.5 1.0 test -0.1% (7,8) 12040 0.0 0.5 1.0 test +0.1% (7,9) 12040 0.0 0.5 1.0 test +0.1% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test -3.1% (2,7) 12040 0.0 0.5 1.0 test +2.4% (2,9) 12040 0.0 0.5 1.0 test +65.0% (3,4) 12040 0.0 0.5 1.0 test +14.1% (3,5) 12040 0.0 0.5 1.0 test +59.1% (3,7) 12040 0.0 0.5 1.0 test +45.1% (3,8) 12040 0.0 0.5 1.0 test +72.0% (4,5) 12040 0.0 0.5 1.0 test +50.8% (4,7) 12040 0.0 0.5 1.0 test +48.6% (4,9) 12040 0.0 0.5 1.0 test +61.4% (5,6) 12040 0.0 0.5 1.0 test +60.6% (5,7) 12040 0.0 0.5 1.0 test +56.2% (5,8) 12040 0.0 0.5 1.0 test +54.9% (5,9) 12040 0.0 0.5 1.0 test +34.5% (6,7) 12040 0.0 0.5 1.0 test +33.7% (6,8) 12040 0.0 0.5 1.0 test +31.2% (6,9) 12040 0.0 0.5 1.0 test -0.1% (7,8) 12040 0.0 0.5 1.0 test +0.1% (7,9) 12040 0.0 0.5 1.0 test +0.1% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test -3.1% (2,7) 12040 0.0 0.5 1.0 test +2.4% (2,9) 12040 0.0 0.5 1.0 test +65.0% (3,4) 12040 0.0 0.5 1.0 test +14.1% (3,5) 12040 0.0 0.5 1.0 test +59.1% (3,7) 12040 0.0 0.5 1.0 test +45.1% (3,8) 12040 0.0 0.5 1.0 test +72.0% (4,5) 12040 0.0 0.5 1.0 test +50.8% (4,7) 12040 0.0 0.5 1.0 test +48.6% (4,9) 12040 0.0 0.5 1.0 test +61.4% (5,6) 12040 0.0 0.5 1.0 test +60.6% (5,7) 12040 0.0 0.5 1.0 test +56.2% (5,8) 12040 0.0 0.5 1.0 test +54.9% (5,9) 12040 0.0 0.5 1.0 test +34.5% (6,7) 12040 0.0 0.5 1.0 test +33.7% (6,8) 12040 0.0 0.5 1.0 test +31.2% (6,9) 12040 0.0 0.5 1.0 test -0.1% (7,8) 12040 0.0 0.5 1.0 test +0.1% (7,9) 12040 0.0 0.5 1.0 test +0.1% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming 12040 0.0 0.5 1.0 test -3.1% (2,7) 12040 0.0 0.5 1.0 test +2.4% (2,9) 12040 0.0 0.5 1.0 test +65.0% (3,4) 12040 0.0 0.5 1.0 test +14.1% (3,5) 12040 0.0 0.5 1.0 test +59.1% (3,7) 12040 0.0 0.5 1.0 test +45.1% (3,8) 12040 0.0 0.5 1.0 test +72.0% (4,5) 12040 0.0 0.5 1.0 test +50.8% (4,7) 12040 0.0 0.5 1.0 test +48.6% (4,9) 12040 0.0 0.5 1.0 test +61.4% (5,6) 12040 0.0 0.5 1.0 test +60.6% (5,7) 12040 0.0 0.5 1.0 test +56.2% (5,8) 12040 0.0 0.5 1.0 test +54.9% (5,9) 12040 0.0 0.5 1.0 test +34.5% (6,7) 12040 0.0 0.5 1.0 test +33.7% (6,8) 12040 0.0 0.5 1.0 test +31.2% (6,9) 12040 0.0 0.5 1.0 test -0.1% (7,8) 12040 0.0 0.5 1.0 test +0.1% (7,9) 12040 0.0 0.5 1.0 test +0.1% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming MNIST KMNIST 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming Figure 2: Training curves for four representative modulus pairs per dataset. Annotations show ACR’s accuracy gain in percentage points over the strongest baseline. Full results in Appendix C. Section 4.1 compares retention policies on digit addition (Manhaeve et al., 2018) and aggregated mod addition (He and Li, 2025). Section 4.2 examines the two objective terms through ablations and retention dynamics. Section 4.3 examines applications to chess attack (Dai et al., 2019) and judicial sentencing (Huang et al., 2020). 4.1 Learning from Ambiguous Explanations We begin with two arithmetic settings where the same observed target admits multiple valid digit assignments. In digit addition, each example contains two sequences ofndigit images whose hidden labels form two integers; training observes only their sum (Manhaeve et al., 2018). Forn= 1,SUM(,)=3 reveals the sum3, while the image labels remain hidden. MNIST (LeCun et al., 1998) and KM- NIST (Clanuwat et al., 2018) supply the images forn∈1,2,3,4. We report character accuracy on the recovered digit labels, averaged over five runs. Asngrows, more hidden decisions must agree with the same target. For these arithmetic tasks, we compare ACR with the single-candidate ABL-hamming and ABL-confidence (Dai et al., 2019), andA 3 BL(He et al., 2024b). Digit addition also includes DeepProbLog, NeurASP, and DeepStochLog (Manhaeve et al., 2018; Yang et al., 2020; Winters et al., 2022). For aggregated mod addition, theA 3 BLruns use a top-32 cap and temperature 0.2, 7 w/o Mw/o UACR 0 20 40 60 80 100 MNIST Character accuracy (%) w/o Mw/o UACR 0 20 40 60 80 100 True-label weight (%) w/o Mw/o UACR 0 5 10 15 20 25 30 Retained size w/o Mw/o UACR 0 20 40 60 80 100 KMNIST Character accuracy (%) w/o Mw/o UACR 0 20 40 60 80 100 True-label weight (%) w/o Mw/o UACR 0 5 10 15 20 25 30 Retained size Figure 3: Task-wise ablation distributions across 19 modulus pairs per dataset. Columns: character accuracy, TLM, and retained set size. ACR w/o mass and ACR w/o uncertainty (w/o M and w/o U in the panels) remove the mass and uncertainty terms. retaining all candidates when at most 32 are available. Both ACR andA 3 BLmaintain high character accuracy across operand lengths, whereas the hard-selection methods have lower means or larger variation across runs (Table 1). On KMNIST, ACR exceedsA 3 BLforn≥2, with gains of 0.81–1.38 percentage points;A 3 BLperforms better at n = 1. Digit addition tests scaling in the number of hidden positions but uses a single rule per example. Aggregated mod addition introduces a harder coupling: a shared digit predictor must satisfy two modular rules simultaneously. For a modulusk, each example contains two one-digit images with hidden labels z 1 ,z 2 and one observed residue: y k = (z 1 + z 2 ) mod k. Each training example samplesk ∈ k 1 ,k 2 and all examples share one digit predictor, so MODSUM_3(,)=0andMODSUM_4(,)=3follow different modular rules within the same train- ing set. We evaluate 19 modulus pairs with 30,000 examples and three runs per pair (He and Li, 2025). The representative curves in Figure 2 reveal a difference in learning dynamics: ACR maintains high accuracy after the initial rise, whileA 3 BLdeclines after an earlier peak in some settings. Across the evaluated modulus pairs, ACR improves concept accuracy overA 3 BLin most settings. The gains vary across pairs; Appendix C reports the complete results, including small or negative gains. 4.2 Understanding Candidate Retention The retention objectiveJ θ balances retained uncertainty against discarded posterior mass. We study these roles through controlled ablations of the two terms and a training trace of the resulting 8 Table 2: Objective ablation averaged over all aggregated mod-addition settings (19 modulus pairs× 2 datasets). ACR achieves the highest accuracy and TLM; removing either term degrades both. MethodAcc. (%) TLM (%) Size ACR w/o mass20.017.88 1.00 ACR w/o uncertainty65.2240.15 18.76 ACR94.9894.25 16.68 supervision. Both use retained true-label weight (TLM) as a diagnostic. Letz ⋆ = (z ⋆ j ) m j=1 denote the ground-truth concepts; we define TLM θ (R; x,y) = 1 m m X j=1 X c∈R e w θ (c| x,y)I[c j = z ⋆ j ]. Here e w θ is the candidate distribution used in the experiments (Appendix B). TLM averages the retained weight assigned to the correct concept without renormalizing overR; it therefore reflects both retained mass and label quality. The ablations in Table 2 compare three retention policies under a shared training protocol. Without the mass term, the initial singleton already minimizes uncertainty, U θ = 0, so the strict- improvement rule accepts no additions. Without uncertainty, each positive-weight addition reduces discarded mass, and the rule retains the full search pool ofmin(32,|s|) candidates. Both controls use the same scoring and raw-weight supervision as ACR. Averaged over the aggregated mod-addition settings, ACR reaches 94.98% concept accuracy, compared with 20.01% for ACR w/o mass and 65.22% for ACR w/o uncertainty (Table 2). Removing either term also lowers retained true-label weight. ACR retains fewer candidates on average than ACR w/o uncertainty while assigning more weight to correct labels. Figure 3 shows how these differences vary across modulus pairs. The representative trace in Figure 4 follows the retained supervision on MNIST mod addition (3,4). Retained uncertainty rises early and then falls, while discarded mass approaches zero. In this trace, sharper coordinate targets coexist later in training with coverage of nearly all model-posterior mass. 4.3 Beyond Arithmetic Reasoning We next test ACR on two non-arithmetic tasks: chess attack and judicial sentencing, comparing with ABL-hamming, ABL-confidence (Dai et al., 2019), andA 3 BL(He et al., 2024b). In chess attack, MNIST images occupy board squares and map to six piece types (Dai et al., 2019). Training provides square locations and whether any pair attacks; piece labels remain hidden. We report attack and piece accuracy from a single run on about 60,000 training and 10,000 test boards. All methods share the split and 20-epoch schedule; ACR uses five warm-up epochs ofA 3 BLsupervision before switching to retention-based selection. Judicial sentencing takes judgment facts as input and predicts discrete sentencing factors, which a knowledge base maps to the observed prison term (Huang et al., 2020). We follow the task setup of Huang et al. and report Macro-F1 and Micro-F1, averaged over five runs. Warm-started ACR improves more on piece identification than on attack prediction (Table 3). Both marginalization-based methods outperform hard selection, while the judicial task leaves less room for adaptive retention to differentiate from A 3 BL . 9 0.0 0.5 1.0 Objective 151015 Epoch 0.0 0.5 1.0 TLM U μ 1 ¡M μ Figure 4: Training dynamics on MNIST mod addition (3,4). Top: retained uncertainty U θ and discarded mass 1− M θ ; bottom: TLM. Lines: medians; bands: interquartile ranges. Table 3: Results beyond arithmetic. Best in bold. Chess (%)Judicial MethodAttack Piece Macro-F1 Micro-F1 ABL-hamming 69.01 17.95 0.32470 0.70256 ABL-confidence 69.01 17.95 0.32470 0.70256 A 3 BL97.53 87.19 0.83224 0.89574 ACR (ours)99.31 98.29 0.83309 0.89870 5 Related Work In this section, we review the two lines of work most relevant to our method: neuro-symbolic learning and abductive learning. 5.1 Neuro-Symbolic Learning Neuro-symbolic methods connect perception and reasoning through architectures, loss functions, or explicit inference. Early systems embedded declarative rules in connectionist architectures (Towell and Shavlik, 1994; d’Avila Garcez et al., 2002). Logic-as-loss methods translate symbolic constraints into differentiable objectives (Xu et al., 2018; Roychowdhury et al., 2021); their behavior depends on the chosen relaxation (van Krieken et al., 2022; He et al., 2024a). Hybrid systems retain an explicit symbolic reasoner. DeepProbLog extends ProbLog with neural predicates (Manhaeve et al., 2018, 2021), NeurASP couples neural perception with answer-set programming (Yang et al., 2020), and DeepStochLog uses stochastic logic programs (Winters et al., 2022). They share a modular connection between neural perception and symbolic reasoning, with 10 different mechanisms for exchanging information. In ABL, abduction returns explanations that become labels for the neural update. This feedback interface makes the candidate-to-supervision step explicit: several valid joint assignments can supply different labels for the same inputs. ACR studies which of these explanations should contribute to the returned supervision and how their labels combine into concept targets. 5.2 Abductive Learning ABL turns abduction, inferring hypotheses that explain observations under background knowl- edge (Magnani, 2009), into a learning mechanism whose central aim is to make perception and reasoning mutually beneficial (Zhou, 2019; Dai et al., 2019). The neural model predicts concepts, abduction revises them under the knowledge base, and the revised candidates supervise the next neural update. When several candidates explain the same target, classical ABL selects the one closest to the current prediction, using Hamming distance or model confidence. This decision depends on model quality: at cold start, an unreliable prediction can select a poor pseudo-label, which subsequent updates may reinforce (Tao et al., 2024; He et al., 2024b). ABLSim mitigates this dependence with inter-sample similarity that guides consistency optimization before the classifier becomes reliable (Huang et al., 2021). Subsequent work extends the loop along three axes. (i) Knowledge scope: GABL enables abduction from inexact ground propositions (Cai et al., 2021), and related methods learn from raw data, semi-supervised signals, or inaccurate rules (Dai and Muggleton, 2021; Huang et al., 2020; Yang et al., 2024). The framework has also been applied to PDE discovery (Gao et al., 2025) and document recognition (Gao et al., 2024). (i) Identifiability: formal analyses connect learning reliability to knowledge-base structure (Tao et al., 2024; He and Li, 2025). (i) Reasoning control: C-ABL partitions the knowledge base into sub-bases introduced progressively (Hu et al., 2025). These advances improve what knowledge is available or how reasoning is structured, but leave a separate design choice: how much of the candidate posterior should return to the neural model as supervision. This choice matters because target-level consistency need not identify the intended latent concepts: different candidates can explain one target while assigning different concept labels, producing reasoning shortcuts (Marconato et al., 2023; Bortolotti et al., 2024).A 3 BLaddresses this ambiguity by weighting every valid candidate under the current model and marginalizing their coordinate labels (He et al., 2024b), avoiding an early single-candidate commitment. Classical ABL andA 3 BLtherefore define the single-candidate and marginalization endpoints. ACR develops the intermediate range through a greedy subset-selection criterion combining supervision sharpness and posterior-mass coverage. 6 Limitations and Future Directions ACR operates on the candidate set returned by abduction. Enumeration can become the bottleneck when this space grows combinatorially. The criterion uses unit coefficients, and task-dependent weighting remains unexplored. Our comparisons use fixedKand temperature; validation-selected settings and retained-size-matched controls would further characterize the benefit of the selection rule. The chess result also calls for repeated runs with matched warm-up schedules. Candidate retention alone does not establish identifiability of the intended concept semantics, which is central to reasoning shortcuts (Marconato et al., 2023). Future directions include jointly generating and retaining explanations, learning the relative weighting of uncertainty and mass, and extending retention to inaccurate rules or partial labels. 11 7 Conclusion Candidate retention shapes concept supervision through the labels that explanations support jointly. ACR evaluates each addition by the model mass it recovers and the change it makes to coordinate uncertainty. Our risk bound separates these two selection-dependent quantities from model mismatch; the greedy rule uses them to construct a retained set for each training pair. Experiments show concept-accuracy gains over single-candidate baselines andA 3 BLin most evaluated aggregated mod-addition settings. Objective ablations support the complementary roles of mass and uncertainty in retaining alternatives while keeping concept targets informative. References Bortolotti, S., Marconato, E., Carraro, T., Morettin, P., van Krieken, E., Vergari, A., Teso, S., and Passerini, A. (2024). A neuro-symbolic benchmark suite for concept quality and reasoning shortcuts. In Advances in Neural Information Processing Systems, volume 37, pages 115861–115905, Vancouver, BC, Canada. Curran Associates, Inc. Datasets and Benchmarks Track. Cai, L.-W., Dai, W.-Z., Huang, Y.-X., Li, Y.-F., Muggleton, S., and Jiang, Y. (2021). Abductive learning with ground knowledge base. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, pages 1815–1821, Virtual Event / Montreal, Canada. International Joint Conferences on Artificial Intelligence. Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., and Ha, D. (2018). Deep learning for classical japanese literature. CoRR, abs/1812.01718. Dai, W.-Z. and Muggleton, S. (2021). Abductive knowledge induction from raw data. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, pages 1845–1851, Virtual Event / Montreal, Canada. International Joint Conferences on Artificial Intelligence. Dai, W.-Z., Xu, Q., Yu, Y., and Zhou, Z.-H. (2019). Bridging machine learning and logical reasoning by abductive learning. In Advances in Neural Information Processing Systems, volume 32, pages 2815–2826, Vancouver, BC, Canada. Curran Associates, Inc. d’Avila Garcez, A. S., Broda, K. B., and Gabbay, D. M. (2002). Neural-Symbolic Learning Systems: Foundations and Applications. Perspectives in Neural Computing. Springer-Verlag, London, UK. Gao, E.-H., Ge, C., Jiang, Y., and Zhou, Z.-H. (2025). Discovering symbolic partial differential equation by abductive learning. In Advances in Neural Information Processing Systems, volume 38, pages 57359–57385, San Diego, CA, USA and Mexico City, Mexico. Curran Associates, Inc. Gao, E.-H., Huang, Y.-X., Hu, W.-C., Zhu, X.-H., and Dai, W.-Z. (2024). Knowledge-enhanced historical document segmentation and recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8409–8416, Vancouver, BC, Canada. AAAI Press. He, H.-Y., Dai, W.-Z., and Li, M. (2024a). Reduced implication-bias logic loss for neuro-symbolic learning. Machine Learning, 113(6):3357–3377. He, H.-Y. and Li, M. (2025). A learnability analysis on neuro-symbolic learning. In Advances in Neural Information Processing Systems, volume 38, pages 84923–84956, San Diego, CA, USA and Mexico City, Mexico. Curran Associates, Inc. 12 He, H.-Y., Sun, H., Xie, Z., and Li, M. (2024b). Ambiguity-aware abductive learning. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 18019–18042, Vienna, Austria. PMLR. Hu, W.-C., Li, Q.-J., Jia, L.-H., Ge, C., Li, Y.-F., Jiang, Y., and Zhou, Z.-H. (2025). Curriculum abductive learning. In Advances in Neural Information Processing Systems, volume 38, pages 174455–174483, San Diego, CA, USA and Mexico City, Mexico. Curran Associates, Inc. Huang, Y.-X., Dai, W.-Z., Cai, L.-W., Muggleton, S. H., and Jiang, Y. (2021). Fast abductive learning by similarity-based consistency optimization. In Advances in Neural Information Processing Systems, volume 34, pages 26574–26584, Virtual Event. Curran Associates, Inc. Huang, Y.-X., Dai, W.-Z., Yang, J., Cai, L.-W., Cheng, S., Huang, R., Li, Y.-F., and Zhou, Z.-H. (2020). Semi-supervised Abductive Learning and its application to theft judicial sentencing. In 2020 IEEE International Conference on Data Mining (ICDM), pages 1070–1075, Virtual Event / Sorrento, Italy. IEEE. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324. Lücke, J. (2019). Truncated variational expectation maximization. arXiv:1610.03113, version 3. Magnani, L. (2009). Abductive Cognition: The Epistemological and Eco-Cognitive Dimensions of Hypothetical Reasoning, volume 3 of Cognitive Systems Monographs. Springer, Berlin, Heidelberg. Manhaeve, R., Dumančić, S., Kimmig, A., Demeester, T., and De Raedt, L. (2018). DeepProbLog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems, volume 31, pages 3753–3763, Montréal, QC, Canada. Curran Associates, Inc. Manhaeve, R., Dumančić, S., Kimmig, A., Demeester, T., and De Raedt, L. (2021). Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence, 298:103504. Marconato, E., Teso, S., Vergari, A., and Passerini, A. (2023). Not all neuro-symbolic concepts are created equal: Analysis and mitigation of reasoning shortcuts. In Advances in Neural Information Processing Systems, volume 36, pages 72507–72539, New Orleans, LA, USA. Curran Associates, Inc. Roychowdhury, S., Diligenti, M., and Gori, M. (2021). Regularizing deep networks with prior knowledge: A constraint-based approach. Knowledge-Based Systems, 222:106989. Tao, L., Huang, Y.-X., Dai, W.-Z., and Jiang, Y. (2024). Deciphering raw data in neuro-symbolic learning with provable guarantees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15310–15318, Vancouver, BC, Canada. AAAI Press. Towell, G. G. and Shavlik, J. W. (1994). Knowledge-based artificial neural networks. Artificial Intelligence, 70(1–2):119–165. van Krieken, E., Acar, E., and van Harmelen, F. (2022). Analyzing differentiable fuzzy logic operators. Artificial Intelligence, 302:103602. Winters, T., Marra, G., Manhaeve, R., and De Raedt, L. (2022). DeepStochLog: Neural stochastic logic programming. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10090–10100, Virtual Event. AAAI Press. 13 Xu, J., Zhang, Z., Friedman, T., Liang, Y., and Van den Broeck, G. (2018). A semantic loss function for deep learning with symbolic knowledge. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5502–5511, Stockholm, Sweden. PMLR. Yang, X.-W., Shao, J.-J., Tu, W.-W., Li, Y.-F., Dai, W.-Z., and Zhou, Z.-H. (2024). Safe abductive learning in the presence of inaccurate rules. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16361–16369, Vancouver, BC, Canada. AAAI Press. Yang, Z., Ishay, A., and Lee, J. (2020). NeurASP: Embracing neural networks into answer set programming. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, pages 1755–1762, Virtual Event / Yokohama, Japan. International Joint Conferences on Artificial Intelligence. Zhou, Z.-H. (2019). Abductive Learning: Towards bridging machine learning and logical reasoning. Science China Information Sciences, 62(7):076101. 14 Appendix A Proofs A.1 Fixed-Support EM Connection This subsection considers the product-posterior choicew θ (c | x,y) =p θ (c | x)/Z θ (x,y) and normalized retained targets. For a fixed positive-mass set R, define logZ θ,R (x,y) = log X c∈R p θ (c| x). The following identity is standard in truncated variational EM (Lücke, 2019). Proposition 1. With product-posterior weights andRfixed during an update, computingq t =q θ t ,R performs the exact E-step for logZ θ,R (x,y). Holding q t fixed, an update that reduces L ret (θ;q t ) =−E c∼q t [logp θ (c| x)] performs a generalized M-step. Proof of Proposition 1. Proof. For any distribution q supported on R, the standard variational identity gives logZ θ,R (x,y) =F x,y (q,θ) + KL(q∥q θ,R ), whereF x,y (q,θ) =E c∼q [logp θ (c| x)] +H(q). Atθ=θ t , settingq=q θ t ,R makes the KL term zero and the bound tight: this is the exact E-step. Withq t fixed,H(q t ) is constant inθ, so reducing L ret (θ;q t ) =−E q t [logp θ (c | x)] raisesFand thus the likelihood bound, giving a generalized M-step. The likelihood statement concerns a fixed support and these normalized product-posterior targets. The task-specific score and raw-weight convention used for aggregated mod addition are specified below. B Experimental Protocol Hardware. All experiments were run on a server with two NVIDIA A100-PCIE GPUs (40 GB each) and two Intel Xeon Platinum 8358 CPUs. Repeated runs. Digit addition reports mean and standard deviation over five runs. Aggregated mod addition, its ablations, and retention dynamics each use three runs; Figure 4 shows medians and interquartile ranges. Chess attack uses one run with a 20-epoch schedule; ACR uses five warm-up epochs of A 3 BL supervision. Judicial sentencing averages over five runs. Comparison methods. Comparison methods use their official implementations, including the task-specific data splits and hyperparameters, following the evaluation protocols described above. For aggregated mod addition, ACR usesK= 32 and a temperature of 0.2, the same settings as A 3 BL . The chess warm-up schedule is stated alongside its results. 15 Reproducibility. The supplementary materials include data-preprocessing, training, and analysis code, together with task-specific configurations and instructions for reproducing all reported results. C Full Aggregated Mod-Addition Results We follow the aggregated mod-addition setup of He and Li (2025) and evaluate all 19 modulus pairs classified as learnable. Figures 5 and 6 report training trajectories for all pairs. ACR achieves higher accuracy thanA 3 BLon most pairs in both datasets. The largest gap in favor of the baseline occurs at (2, 7), where A 3 BL leads by about three points. 12040 0.0 0.5 1.0 test +0.1% (2,7) 12040 0.0 0.5 1.0 test +0.2% (2,9) 12040 0.0 0.5 1.0 test +57.6% (3,4) 12040 0.0 0.5 1.0 test +68.8% (3,5) 12040 0.0 0.5 1.0 test +36.5% (3,7) 12040 0.0 0.5 1.0 test +23.2% (3,8) 12040 0.0 0.5 1.0 test +67.8% (4,5) 12040 0.0 0.5 1.0 test +30.1% (4,7) 12040 0.0 0.5 1.0 test +26.8% (4,9) 12040 0.0 0.5 1.0 test +42.4% (5,6) 12040 0.0 0.5 1.0 test +37.7% (5,7) 12040 0.0 0.5 1.0 test +33.7% (5,8) 12040 0.0 0.5 1.0 test +35.4% (5,9) 12040 0.0 0.5 1.0 test -0.1% (6,7) 12040 0.0 0.5 1.0 test +0.0% (6,8) 12040 0.0 0.5 1.0 test -0.0% (6,9) 12040 0.0 0.5 1.0 test +0.0% (7,8) 12040 0.0 0.5 1.0 test -0.1% (7,9) 12040 0.0 0.5 1.0 test -0.0% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming Figure 5: Character-accuracy trajectories for all 19 MNIST modulus pairs. The bottom-right panel summarizes ACR’s test gain over the strongest baseline. 16 12040 0.0 0.5 1.0 test -3.1% (2,7) 12040 0.0 0.5 1.0 test +2.4% (2,9) 12040 0.0 0.5 1.0 test +65.0% (3,4) 12040 0.0 0.5 1.0 test +14.1% (3,5) 12040 0.0 0.5 1.0 test +59.1% (3,7) 12040 0.0 0.5 1.0 test +45.1% (3,8) 12040 0.0 0.5 1.0 test +72.0% (4,5) 12040 0.0 0.5 1.0 test +50.8% (4,7) 12040 0.0 0.5 1.0 test +48.6% (4,9) 12040 0.0 0.5 1.0 test +61.4% (5,6) 12040 0.0 0.5 1.0 test +60.6% (5,7) 12040 0.0 0.5 1.0 test +56.2% (5,8) 12040 0.0 0.5 1.0 test +54.9% (5,9) 12040 0.0 0.5 1.0 test +34.5% (6,7) 12040 0.0 0.5 1.0 test +33.7% (6,8) 12040 0.0 0.5 1.0 test +31.2% (6,9) 12040 0.0 0.5 1.0 test -0.1% (7,8) 12040 0.0 0.5 1.0 test +0.1% (7,9) 12040 0.0 0.5 1.0 test +0.1% (8,9) (2,7) (2,9) (3,4) (3,5) (3,7) (3,8) (4,5) (4,7) (4,9) (5,6) (5,7) (5,8)(5,9) (6,7) (6,8)(6,9) (7,8)(7,9) (8,9) 0 20 40 60 % test improvement vs best baseline Character accuracy ACR (ours)A3BLABL confidenceABL hamming Figure 6: Character-accuracy trajectories for all 19 KMNIST modulus pairs. The bottom-right panel summarizes ACR’s test gain over the strongest baseline. 17