Paper deep dive
Weakly supervised concept Bottleneck Learning for Robust Two stage Object centric visual reasoning
Sparsh Tiwari, Gesina Schwalbe, Bettina Finzel
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/30/2026, 2:46:55 AM
Summary
The paper introduces Dynamic Orthogonal Concept Bottleneck (D-OCB), a weakly supervised, object-centric slot-VAE framework for two-stage neuro-symbolic visual reasoning. D-OCB extracts human-aligned symbolic predicates using minimal annotations (1%-15%) by employing dynamic dimensionality allocation, fixed noise floors to prevent representation collapse, and orthogonalization penalties to reduce feature entanglement. It outperforms or matches end-to-end paradigms on CLEVR and CLEVR-Hans benchmarks.
Entities (10)
Relation Signals (9)
D-OCB → evaluatedon → CLEVR-Hans3
confidence 95% · evaluate on the CLEVR object-attribute benchmark... and its logical scene classification variants CLEVR-Hans3
D-OCB → evaluatedon → CLEVR-Hans7
confidence 95% · evaluate on the CLEVR object-attribute benchmark... and CLEVR-Hans7
D-OCB → evaluatedon → CLEVR
confidence 95% · We empirically evaluate on the CLEVR object-attribute benchmark
D-OCB → uses → Slot Attention
confidence 95% · routed through an iterative slot attention module to enforce a tokenization prior
D-OCB → uses → Dynamic Dimensionality Allocation
confidence 95% · D-OCB utilizes a novel dynamic dimensionality allocation mechanism that adaptively redistributes feature space size
D-OCB → evaluatedon → HAM10000
confidence 90% · real world melanoma detection application using the HAM10000 dataset
D-OCB → outperformsormatches → end-to-end paradigms
confidence 90% · matching or outperforming end-to-end paradigms
D-OCB → prevents → Representation collapse
confidence 90% · D-OCB eliminates the arduous manual tuning... effectively preventing representation collapse
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Two-stage neuro-symbolic architectures provide an elegant paradigm for visual problem solving by cleanly separating connectionist perception of predefined symbols from possibly later defined relational reasoning thereon. However, anchoring high-level predicates into visual frames typically necessitates annotations that are expensive to acquire. In this work, we introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB), an object-centric slot- VAE framework designed to extract human-aligned symbolic predicates under extremely weak supervision. D-OCB eliminates the arduous manual tuning of loss-balancing coef- ficients by dynamically learning optimal hyperparameter allocations during training. To infuse prior knowledge on independence of concept categories, in addition to standard re- construction self-supervision we penalize correlation across concept subspaces. Crucially, to combat the instability of very low supervision regimes, D-OCB incorporates a dynamic di- mensionality allocation mechanism; this adaptive formulation allows well-represented con- cepts to yield latent dimensions to underperforming concepts that are lagging behind, effectively preventing representation collapse and significantly improving overall concept accuracy. Through an extensive empirical evaluation, we demonstrate that our framework achieves high concept alignment and downstream visual reasoning accuracy using minimal label budgets, matching or outperforming end-to-end paradigms.
Tags
Links
- Source: https://arxiv.org/abs/2608.22584v1
- Canonical: https://arxiv.org/abs/2608.22584v1
Trouble viewing inline? Open PDF directly →
Full Text
56,251 characters extracted from source content.
Expand or collapse full text
Weakly supervised concept Bottleneck Learning for Robust Two stage Object centric visual reasoning Sparsh Tiwari Affiliation: University of Lübeck, Germany and Ulm University, Germany and University of Bamberg, Germany Gesina Schwalbe Affiliation: Bettina Finzel Affiliation: Abstract Two-stage neuro-symbolic architectures provide an elegant paradigm for visual problem solving by cleanly separating connectionist perception of predefined symbols from possibly later defined relational reasoning thereon. However, anchoring high-level predicates into visual frames typically necessitates annotations that are expensive to acquire. In this work, we introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB), an object-centric slot-VAE framework designed to extract human-aligned symbolic predicates under extremely weak supervision. D-OCB eliminates the arduous manual tuning of loss-balancing coefficients by dynamically learning optimal hyperparameter allocations during training. To infuse prior knowledge on independence of concept categories, in addition to standard reconstruction self-supervision we penalize correlation across concept subspaces. Crucially, to combat the instability of very low supervision regimes, D-OCB incorporates a dynamic dimensionality allocation mechanism; this adaptive formulation allows well-represented concepts to yield latent dimensions to underperforming concepts that are lagging behind, effectively preventing representation collapse and significantly improving overall concept accuracy. Through an extensive empirical evaluation, we demonstrate that our framework achieves high concept alignment and downstream visual reasoning accuracy using minimal label budgets, matching or outperforming end-to-end paradigms. 1 Introduction Neuro-symbolic architectures for visual tasks allow to combine the feature extraction capabilities of deep neural networks (DNNs) on raw signals with interpretable and efficient symbolic reasoning (13; 25). A central impediment within this paradigm is the classic symbol grounding problem, which asks how abstract symbols acquire semantic meaning beyond circular, text-based definitions and connect to perceptual boundaries in the physical world (40). Specifically, feature extractors have to aggregate distributed features across visual space into coherent, unified object-centric representations (5). Two-stage neuro-symbolic architectures offer a compelling solution when symbol annotations are available, by structurally decoupling perception of symbols from reasoning on them (37; 35). In the first stage, a perception network maps raw inputs onto an explicit concept bottleneck layer; in the second stage, a symbolic engine executes logic programs directly over these discovered predicates. This modularity ensures that predictions are fully traceable, reduces susceptibility to shortcut learning, yields faster optimization convergence, and permits the reuse of a static reasoning backend across disparate visual domains (21). Despite these advantages, traditional concept bottlenecks suffer from labeling requirements. Existing frameworks rely on full supervision to instantiate intermediate concepts, or turn to reinforcement learning settings with high sample variance (9). The use of background knowledge or programmatic constraints to provide weak supervision, where only parts of the training data requires costly labels, remains significantly underexplored (26). This particularly is the case for object-centric concept bottleneck architectures and for the use in neuro-symbolic architectures. Figure 1: Overview of our D-OCB architecture and training scheme To effectively utilize weak supervision for object-centric extraction, three concrete representational hurdles must first be resolved. First, generative bottlenecks trained with sparse labels frequently succumb to representation collapse (18), where unbounded variance optimization forces latent variables to degrade into uninformative noise. Second, without exhaustive dense annotations to explicitly isolate attributes, feature extractors naturally default to feature cross-correlation. This results in the entanglement of distinct conceptual properties (e.g., blending shape constraints into color representations) to exploit spurious visual shortcuts. Finally, traditional bottlenecks assign statically fixed vector sizes across all target concepts. This assumes uniform semantic complexity and inevitably causes capacity misallocation (16), where highly nuanced concepts are starved of representational bandwidth while trivial concepts waste feature dimensions. To address these vulnerabilities, we introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB 11 1 The codebase is available at https://github.com/sparshX1993/Dynamic-Orthogonal-Concept-Bottleneck.), illustrated in Figure 1. Rather than serving as a standalone, end-to-end reasoning architecture, D-OCB is explicitly framed as a weakly supervised predicate-grounding module (or perceptual bottleneck) designed specifically for two-stage neuro-symbolic pipelines. By structurally uncoupling perception from deduction, D-OCB anchors human-aligned symbolic predicates from minimal annotation budgets. It prevents representation collapse by anchoring per-concept variances against a calibrated noise floor alongside a variational auto-encoder reconstruction loss (58). To combat feature cross-correlation, D-OCB introduces an explicit penalty that orthogonalizes subspaces between different concept categories. Crucially, to prevent capacity misallocation , D-OCB utilizes a novel dynamic dimensionality allocation mechanism that adaptively redistributes feature space size during training based on concept convergence; this ensures stable representations before matching concepts via the Hungarian algorithm. The result is a specialized perception module capable of providing crisp, human-aligned predicate distributions to any frozen, off-the-shelf symbolic solver. Our key contributions amount to: • We introduce D-OCB (cf. Figure 1), a weakly supervised, two-stage object-centric visual reasoning framework that grounds semantic predicates using as little as 1%1\% of instance-level concept annotations. • We propose a structural approach for stable concept learning under weak supervision that effectively prevents rank collapse inside each concept block. • We formalize a dynamic dimensionality allocation protocol, entirely eliminating the need for manual latent-capacity tuning. • We empirically demonstrate on the CLEVR and confounded CLEVR-Hans3/7 benchmarks that D-OCB predicates drive independent symbolic reasoners to accuracies competitive with end-to-end differentiable paradigms, adding as little as 1%1\%-15%15\% concept supervision to the training data. 2 Related Work Our architecture is fundamentally motivated by the need to solve two classical cognitive hurdles simultaneously: the symbol grounding problem (40), which asks how discrete symbols acquire physical semantic meaning from unstructured visual boundaries; and the binding problem (5), which concerns how distributed features across visual space are aggregated into unified, coherent entities. To achieve this, the design of our D-OCB intersects three major paradigms in contemporary neuro-symbolic artificial intelligence: concept bottleneck models (CBMs), object-centric representation learning, and differentiable logical rule induction. CBMs and Generative Realignment. CBMs are interpretable architectures that factor downstream predictions through an intermediate bottleneck layer, where each individual output dimension is explicitly constrained to represent exactly one human-understandable concept(26; 20). Traditional formulations in this domain typically operate over holistic, image-level encodings and rely on dense, exhaustive supervision to map pixels onto categorical concepts (26; 32). To mitigate this data-scarcity issue (1), contemporary frameworks have introduced semi-supervised (17) and completely label-free (44) concept learning. Concurrently, efforts have been made to move beyond holistic representations by developing object-centric concept bottlenecks (56). Alongside these developments, recent frameworks have explored the integration of unsupervised generative constraints into the bottleneck layer (30). These generative approaches formalize how an image reconstruction loss can actively optimize concept alignment, prevent information leakage, and isolate target attributes (23; 29). D-OCB builds directly upon these generative and object-centric foundations but shifts the paradigm entirely to extremely sparse supervision regimes. By introducing a tuning-free dynamic allocation scheme and an explicit cross-correlation constraint, our work explores how balancing self-supervised reconstruction with minimal concept labels can preserve or even enhance grounding accuracy (8). Object-Centric Tokenization and Slot-Based Bottlenecks. To perform relational and compositional reasoning in multi-object scenes, perception models require an object-centric tokenization (5; 49). A recent and superior approach is to use slot attention mechanisms which decompose a raw visual frame into distinct, non-overlapping slot tokens (11), encouraging spatial specialization and compositionality (31). Despite their success in unsupervised scene decomposition (41), extending object-centric slots to interpretable concept layers introduces significant training instabilities (28). D-OCB resolves these limitations by introducing a strictly block-diagonal projection matrix as additional bias that forces uncorrelated concept subspaces. Neuro-Symbolic Reasoning and Two-Stage Pipelines. Neuro-symbolic AI seeks to combine the robust perceptual pattern recognition of deep neural networks with the rigorous deductive compositionality of symbolic reasoning backends (50; 47). While end-to-end logical frameworks like differentiable inductive logic programming (DILP) (21; 52; 51) optimize perception and deduction jointly via gradient descent, they remain highly vulnerable to Likelihood Blow-up (39) and Posterior Collapse and representation contamination. To make use of available ground truth concept labels for interpretability, and to enable systematic reuse, D-OCB operates as a modular, two-stage neuro-symbolic framework that structurally uncouples perceptual tokenization from downstream symbolic solver engines (12; 65). 3 Approach: Dynamic Orthogonal Concept Bottleneck End-to-end differentiable reasoning frameworks are highly susceptible to feature entanglement (38) and rely heavily on dense supervision. To overcome these inherent limitations, we propose a two-stage neuro-symbolic architecture that cleanly decouples object-centric visual perception from downstream logical reasoning. To ensure the perception module grounds continuous visual data into discrete, disentangled symbols under extreme weak supervision (1%1\%–15%15\%), we introduce D-OCB. For a formal task definition, let ∈ℝH×W×CX ^H× W× C define the input space of raw visual scenes, and let =(s1,…,sK)S=(s_1,…,s_K) represent an ordered specification of human-interpretable concept predicates. Our goal is to train an object-centric perception module g:→g:X that maps an input image x to a discrete valuation tensor c∈ℝN×Kc ^N× K, representing the presence of K concepts across N distinct object slots, utilizing an extremely limited label budget. These grounded concepts are then passed into a standalone symbolic solver f:→f:C to deduce the final downstream task solution y∈y (42). 3.1 Overview: Architecture and Optimization Objectives We first establish the overall architecture for object-centric perceptual grounding, mapping raw pixels to discrete symbolic predicates in a step-by-step manner.An input scene x∈ℝh×w×cx ^h× w× c of height h, width w, and c channels is processed by a feature extractor, such as DINOv2 (45), augmented with positional embeddings. The resulting continuous feature maps are routed through an iterative slot attention module (31) to enforce a tokenization prior, outputting N uninterpretable object slots: H=SlotAttention(Encoder(x))∈ℝN×D H=SlotAttention(Encoder(x)) ^N× D (1) where D represents the hidden slot dimension (typically D=128D=128). Following the extraction of these deterministic slots, the model maps the representations to a partitioned latent concept space Z∈ℝN×DtotalZ ^N× D_total. This concept space has a total feature dimensionality DtotalD_total, which is explicitly divided into independent subspaces for each target concept c∈c (e.g., shape, color), such that Dtotal=∑c∈DcD_total= _c D_c. The specific dimensionality DcD_c allocated to each concept is not static; it is dynamically adjusted during training (as detailed in Section 3.3) to provide more capacity to struggling concepts. To derive crisp concept predictions from these latent representations, each isolated concept subspace zc∈ℝDcz_c ^D_c per slot is fed into an independent prediction head (a 2-layer MLP). These heads output raw class logits, which are converted into continuous probabilities via a softmax function (for multi-class concepts like color) or a sigmoid function (for binary concepts like material/Size). Finally, an argmax operation (or standard thresholding) discretizes these soft probabilities into the definitive, crisp symbolic predicates (e.g., color(obj1, red)) that are ultimately passed to the downstream logical reasoner. To train this bottleneck effectively without manual hyperparameter engineering, we optimize an object-centric Variational Autoencoder (VAE) objective that balances a self-supervised reconstruction loss ℒreconL_recon, a concept prediction loss (67) ℒconceptL_concept, and a structural cross-correlation loss22 2 For more detailed breakdown of our loss function and its mathematical formulations, see supplementary material. (3) ℒorthL_orth: ℒtotal=ℒrecon+λ1ℒconcept+λ2ℒorth _total=L_recon+ _1L_concept+ _2L_orth (2) The scaling coefficients λ1 _1 and λ2 _2 are dynamically updated at each epoch based on the moving average of gradient variances. Crucially, the cross-correlation loss ℒorthL_orth minimizes the off-diagonal elements of the feature correlation matrix. This explicitly forces the learned concept subspaces to remain orthogonal and causally isolated from one another (48; 57), ensuring that weakly supervised concept signals are not overwhelmed by unconstrained reconstruction errors (64) and avoiding the feature interference common in standard dense projections (14; 24). 3.2 Generative Bottleneck: Calibrated Fixed Noise Floors A critical challenge in this architecture is the representation collapse common to generative bottlenecks (15). Generative models typically learn a per-sample variance (σ(x)σ(x)). When annotation is scarce, the unsupervised reconstruction loss ℒreconL_recon can force the network to map visual noise directly into this learned variance. This unbounded optimization causes a likelihood blow-up (39), which leads to severe posterior collapse (33). Consequently, the latent variables become mathematically non-identifiable (62), effectively destroying the semantic manifold. To prevent this, D-OCB discards per-sample learned variance entirely. Instead, we introduce a fixed noise floor σc _c specific to each concept subspace c: zc=μc(x)+σc⊙ϵ z_c= _c(x)+ _c ε (3) Optimized globally during a brief one-shot calibration phase on the labeled subset, σc _c is permanently frozen. This renders the signal-to-noise ratio (SNR=‖μc‖2/σc2SNR=|| _c||^2/ _c^2) mathematically stable and bars the decoder from encoding reconstruction noise into the variance. 3.3 Orthogonality: Concept Assignment and Dimensionality Allocation With the latent space cleanly orthogonalized by the cross-correlation loss and stabilized by the fixed noise floors, the remaining task is to map these dimensions to specific human-interpretable concepts and distribute capacity effectively. First, to explicitly assign specific latent dimensions to their corresponding ground-truth concepts (e.g., matching a subspace to concept categories “shape” or “color”), alignments are secured via the Hungarian matching algorithm (27). Second, to efficiently allocate latent capacity among these matched concepts without rank collapse (43), D-OCB replaces standard Singular Value Decomposition (60) with Fisher’s Linear Discriminant Analysis (7) (LDA). LDA projects running class centroids by computing a projection matrix W≔argmaxW|WTSBW||WTSWW|W _W |W^TS_BW||W^TS_WW| that explicitly maximizes between-class scatter SBS_B relative to within-class scatter SWS_W. By maximizing class separability, LDA remains invariant to unsupervised reconstruction noise. Combined with Johnson-Lindenstrauss random projections for distance-preserving weight transfers, this dynamic dimensionality allocation ensures that underperforming concepts receive the necessary representational capacity without the need for manual latent-capacity tuning. Figure 2: Correleation MATRIX for Clevr HANS7 Showing our DDA assign dimension from one concept to another for epoch 5 (left) to epoch 100 (right) 4 Experiments and Results We empirically evaluate the representational fidelity, reasoning capability, and data efficiency of D-OCB, aiming to answer the following questions: RQ1: Can D-OCB extract highly accurate and well-grounded visual predicates under extreme weak supervision (using only 1%1\%–15%15\% of the training dataset annotations)? RQ2: Are the extracted predicates sufficiently stable to drive an offline symbolic solver without requiring joint end-to-end logic backpropagation? RQ3: Does structurally decoupling perception from logical reasoning successfully prevent the exploitation of spurious correlations, thereby enabling D-OCB to achieve robust out-of-distribution (OOD) confounder resistance? Datasets. We evaluate on the CLEVR object-attribute benchmark (70,000 training images) (19) and its logical scene classification variants CLEVR-Hans3 (3,000 per class, 3 classes) and CLEVR-Hans7 (21,000 training images) (54) and a real world melanoma detection application using the HAM10000 dataset 33 3 find details on the choice of datasets in the technical supplementary (59) (8,012 training images). Hence, the considered sample budgets for 1%/15% weak supervision amount to a mere 700700/10.3k10.3k, 9090/1.35k1.35k, 210210/3.15k3.15k, 8080/1.2k1.2k samples for CLEVR, CLEVR3, CLEVR7, and HAM10000 respectively. In all weakly supervised configurations, the remaining majority of the training pool remains completely unannotated. Baselines. We benchmark our semantic tokenization pipeline against the following contemporary visual concept extraction architectures: the Concept Embedding Model (CEM) (66); the Probabilistic Concept Bottleneck Model (ProbCBM) (22); SlotFormer (4) for temporal object-centric mechanics; a Standard Slot Attention baseline (Vanilla+SA); SlotVAE-NO-KL (a deterministic ablation); and SlotVAE-KL (61), optimized under standard Kullback-Leibler regularizers (2; 63; 34).To rigorously evaluate out-of-distribution (OOD) reasoning and confounder resistance, we compare our modular D-OCB (coupled with DILP) against end-to-end and dense-supervision paradigms. These include: an end-to-end trained CNN (0% concept supervision); a CNN + DILP baseline (100% supervision); state-of-the-art end-to-end differentiable frameworks (α (52) and NEUMANN (53)); DeepProbLog (35) (trained jointly with 100% supervision); and a Ground Truth (Oracle) baseline. Evaluation Metrics. Perceptual quality and grounding alignment are assessed via concept accuracy (acc.) and Macro-F1 scores, computed by aligning predicted object slots to ground-truth annotations via Hungarian matching (27). Downstream task execution is measured via rule evaluation acc. and F1-scores (10). Hyperparameters. The architecture is optimized over 200 epochs with a batch size of 64 and a base learning rate of 4×10−44× 10^-4 utilizing a cosine annealing schedule (45). DDA capacity shifts occur every 5 epochs following an initial 30-epoch warmup gate, transferring features in discrete steps of Δ=8 =8 dimensions to maintain block-diagonal alignment. To prioritize semantic grounding, the concept classification loss is heavily weighted (λ1=8.0 _1=8.0, scaling to 12.012.0 in later phases) relative to the reconstruction baseline (λrecon=1.0 _recon=1.0). Finally, the object existence head employs a focal loss (γ=2.0γ=2.0, positive weight =2.5=2.5) to successfully counter slot sparsity. 4.1 RQ1: Visual Concept Extraction and Predicate Accuracy To evaluate the representational quality of our perception module, we measure its capability to ground continuous visual features into discrete symbolic predicates using the CLEVR and HAM10000 datasets under extremely weak supervision regimes (1%1\% to 15%15\%). Unlike traditional slot models that frequently exhibit performance collapse on complex attributes due to unconstrained reconstruction objectives hijacking the latent space topology, our D-OCB explicitly prevents features from dissolving into unstructured noise. Table 1: Concept classification acc. on the CLEVR dataset across varying levels of weak supervision. Results are reported as mean ± standard deviation over 5 folds. CNN Backbone DINO Backbone Sup. Method Size Shape Color Material Mean Size Shape Color Material Mean 1% Vanilla+SA 51.2±0.051.2± 0.0 49.3±0.049.3± 0.0 10.7±0.010.7± 0.0 62.2±0.062.2± 0.0 46.4±0.046.4± 0.0 72.3±0.072.3± 0.0 63.4±0.063.4± 0.0 15.8±0.015.8± 0.0 70.6±0.070.6± 0.0 57.3±0.057.3± 0.0 CEM 51.2±1.351.2± 1.3 33.9±0.533.9± 0.5 17.2±6.317.2± 6.3 50.9±0.550.9± 0.5 38.3±2.138.3± 2.1 73.4±2.273.4± 2.2 41.3±3.941.3± 3.9 12.3±0.112.3± 0.1 70.8±6.670.8± 6.6 49.5±2.649.5± 2.6 ProbCBM 48.2±3.948.2± 3.9 34.5±2.134.5± 2.1 20.9±8.220.9± 8.2 50.7±0.750.7± 0.7 38.6±3.338.6± 3.3 72.3±4.472.3± 4.4 38.4±2.238.4± 2.2 12.5±0.312.5± 0.3 67.0±7.467.0± 7.4 47.5±3.247.5± 3.2 SlotFormer 25.2±3.125.2± 3.1 25.0±2.025.0± 2.0 17.5±5.117.5± 5.1 50.1±1.150.1± 1.1 29.4±1.729.4± 1.7 48.0±2.848.0± 2.8 38.3±2.838.3± 2.8 17.7±3.917.7± 3.9 55.0±3.155.0± 3.1 39.8±2.139.8± 2.1 SlotVAE-NoVAE 52.8±0.052.8± 0.0 51.5±0.051.5± 0.0 8.9±0.08.9± 0.0 60.5±0.060.5± 0.0 48.6±0.048.6± 0.0 76.2±0.076.2± 0.0 62.6±0.062.6± 0.0 12.1±0.012.1± 0.0 75.6±0.075.6± 0.0 56.9±0.056.9± 0.0 SlotVAE-KL 62.0±4.862.0± 4.8 64.0±6.664.0± 6.6 16.9±3.716.9± 3.7 79.2±1.479.2± 1.4 55.0±3.255.0± 3.2 90.5±6.790.5± 6.7 80.7±6.480.7± 6.4 38.3 ± 0.2 93.6±3.293.6± 3.2 75.3±3.375.3± 3.3 D-OCB (Ours) 75.1 ± 16.1 67.5 ± 3.4 28.2 ± 5.1 84.5 ± 5.0 63.8 ± 7.1 94.4 ± 1.4 82.3 ± 8.6 36.7±6.336.7± 6.3 94.9 ± 2.3 77.1 ± 4.5 5% CEM 53.5±1.153.5± 1.1 35.9±0.635.9± 0.6 24.5±0.524.5± 0.5 51.9±1.051.9± 1.0 41.5±0.541.5± 0.5 80.2±2.980.2± 2.9 48.7±1.548.7± 1.5 12.4±0.212.4± 0.2 82.7±2.982.7± 2.9 56.0±1.056.0± 1.0 ProbCBM 48.1±5.248.1± 5.2 34.5±1.534.5± 1.5 25.0±0.925.0± 0.9 50.9±0.750.9± 0.7 39.6±1.639.6± 1.6 74.6±5.574.6± 5.5 46.8±4.946.8± 4.9 12.6±0.412.6± 0.4 79.8±3.979.8± 3.9 53.5±3.153.5± 3.1 SlotFormer 26.7±2.726.7± 2.7 26.1±3.226.1± 3.2 24.8±0.524.8± 0.5 50.3±0.850.3± 0.8 32.0±0.932.0± 0.9 46.4±3.146.4± 3.1 43.9±4.143.9± 4.1 14.1±1.814.1± 1.8 63.2±3.963.2± 3.9 41.9±2.441.9± 2.4 SlotVAE-KL 85.8±5.285.8± 5.2 68.5±4.568.5± 4.5 43.0±2.343.0± 2.3 91.9 ± 1.6 72.3±2.672.3± 2.6 91.0±7.991.0± 7.9 87.3±5.987.3± 5.9 62.3±0.162.3± 0.1 96.6±5.696.6± 5.6 84.1±4.184.1± 4.1 D-OCB (Ours) 91.8 ± 3.3 81.6 ± 1.5 58.5 ± 2.3 91.5±1.891.5± 1.8 80.8 ± 1.0 97.6 ± 0.0 95.1 ± 1.1 75.4 ± 1.5 97.1 ± 0.1 91.3 ± 0.4 15% Vanilla+SA 73.3±0.073.3± 0.0 66.5±0.066.5± 0.0 59.3±0.059.3± 0.0 69.7±0.069.7± 0.0 74.1±0.074.1± 0.0 77.8±0.077.8± 0.0 78.0±0.078.0± 0.0 63.9±0.063.9± 0.0 78.9±0.078.9± 0.0 70.8±0.070.8± 0.0 CEM 61.5±2.761.5± 2.7 37.0±1.237.0± 1.2 24.7±1.124.7± 1.1 53.0±0.953.0± 0.9 44.0±1.244.0± 1.2 86.2±1.286.2± 1.2 54.0±1.554.0± 1.5 12.7±0.412.7± 0.4 86.7±0.886.7± 0.8 60.0±0.460.0± 0.4 ProbCBM 51.4±4.651.4± 4.6 34.5±1.134.5± 1.1 25.0±0.525.0± 0.5 51.4±0.851.4± 0.8 40.6±1.340.6± 1.3 81.6±3.781.6± 3.7 48.6±4.848.6± 4.8 13.8±1.613.8± 1.6 82.7±2.082.7± 2.0 56.7±2.556.7± 2.5 SlotFormer 26.1±3.826.1± 3.8 25.7±3.325.7± 3.3 24.9±0.224.9± 0.2 50.0±1.050.0± 1.0 31.7±1.231.7± 1.2 47.3±2.747.3± 2.7 45.6±4.945.6± 4.9 16.7±3.716.7± 3.7 65.5±3.665.5± 3.6 43.8±2.743.8± 2.7 SlotVAE-NoVAE 75.5±0.075.5± 0.0 65.8±0.065.8± 0.0 61.2±0.061.2± 0.0 74.8±0.074.8± 0.0 66.5±0.066.5± 0.0 79.8±0.079.8± 0.0 78.8±0.078.8± 0.0 64.1±0.064.1± 0.0 76.2±0.076.2± 0.0 77.3±0.077.3± 0.0 SlotVAE-KL 91.6±5.791.6± 5.7 74.4±3.774.4± 3.7 52.9±1.852.9± 1.8 84.2±2.184.2± 2.1 75.8±2.475.8± 2.4 97.6 ± 8.2 92.6±4.392.6± 4.3 74.5±0.174.5± 0.1 95.9±6.495.9± 6.4 89.2±4.089.2± 4.0 D-OCB (Ours) 94.5 ± 1.6 88.6 ± 2.2 84.0 ± 0.7 93.4 ± 1.3 90.1 ± 0.9 97.6 ± 0.0 95.8 ± 0.3 84.6 ± 0.6 97.2 ± 0.1 93.8 ± 0.1 Table 2: Concept classification acc. of D-OCB on the HAM10000 dataset across varying levels of weak supervision. Results are reported as mean ± standard deviation over five folds. Backbone Sup. (%) Dx Localization Sex Age Mean CNN 1 55.3±0.055.3± 0.0 84.2±0.084.2± 0.0 22.1±0.022.1± 0.0 50.0±0.050.0± 0.0 52.9±0.052.9± 0.0 5 55.0±0.055.0± 0.0 84.4±0.084.4± 0.0 60.6±0.060.6± 0.0 61.7±0.061.7± 0.0 65.4±0.065.4± 0.0 15 59.3±0.059.3± 0.0 86.0±0.086.0± 0.0 64.3±0.064.3± 0.0 64.6±0.064.6± 0.0 68.6±0.068.6± 0.0 DINO 1 61.8±0.061.8± 0.0 82.5±0.082.5± 0.0 67.4±0.067.4± 0.0 60.2±0.060.2± 0.0 68.0±0.068.0± 0.0 5 64.4±0.064.4± 0.0 78.5±0.078.5± 0.0 69.4±0.069.4± 0.0 69.1±0.069.1± 0.0 70.4±0.070.4± 0.0 15 64.2±0.064.2± 0.0 85.5±0.085.5± 0.0 72.8±0.072.8± 0.0 70.9±0.070.9± 0.0 73.4±0.073.4± 0.0 As shown in Table 2 and Table 1, D-OCB significantly outperforms baselines, amongst others by the combination of VAE reconstruction loss (difference to NoVAE), and by dynamically assigning dimensions to concepts lacking accuracy and discarding per-sample learned variance (difference to SlotVAE-KL). E.g., it maintains a 73.4%73.4\% mean accuracy on HAM10000 with a DINO backbone at just 15%15\% supervision. D-OCB thus achieves competitive accurate concept alignment without the excessive need for labels and without manual hyperparameter tuning.Empirically, D-OCB actively prevents representation collapse to extract highly accurate semantic concepts using minimal label budgets, significantly outperforming unconstrained generative baselines giving a positive answer to RQ1. 4.2 RQ2: Framework-Agnostic Logical Reasoning Evaluation We now investigate whether the accurately extracted visual atoms are sufficiently stable to drive downstream symbolic reasoning engines—such as DILP (51), Decision Trees, Bayesian Networks, and the Neural-Symbolic Concept Learner (NS-CL) (37)—without requiring joint end-to-end logic backpropagation. To rigorously test this, we export the frozen valuation tensors generated by D-OCB to evaluate specific compositional and diagnostic rules, as detailed in Table 3. Table 3: Downstream evaluation rules for the CLEVR and HAM10000 datasets. ID Rule Type Description CLEVR R1 4-way Conjunction The scene contains an object that is simultaneously large, red, metal, and a cube. R2 3-way Conjunction The scene contains a small rubber sphere. R3 Spatial Relation A blue cylinder is spatially to the left of a red object (evaluated via 3D x-coordinates). R4 Cardinality The scene contains ≥2≥ 2 metal objects; requires the model to count distinct slots satisfying this constraint. R5 Proximity Relation A large green object is within a continuous Euclidean distance threshold (≤2.5≤ 2.5) of a yellow object. HAM R1 Simple The lesion is malignant. R2 Disjunctive The lesion is melanoma (mel) or basal cell carcinoma (bcc). R3 Conjunctive The lesion is melanoma located on the back (a diagnosis and location conjunction). The empirical results in Table 5 and Table 4 substantiate the universal compatibility and representational stability of D-OCB44 4 For full table with all rules see supplementary material. . Because the softly-valued probabilistic margins generated by D-OCB accurately preserve semantic boundaries even under minimal supervision budgets (1%1\% to 15%15\%), they seamlessly drive diverse, off-the-shelf symbolic reasoning frontends—spanning statistical methods (BN) (46), rule-based decision trees (DT)(6), and differentiable logic frameworks (DILP, NS-CL(36))—to highly accurate downstream task performance. Our results confirm that the softly-valued probabilistic margins generated by D-OCB maintain strict semantic boundaries, seamlessly driving diverse, frozen symbolic reasoners to accurate downstream performance. Table 4: Downstream reasoning acc. (%) averaged across rules R1-R5 on the CLEVR dataset. Results over multiple folds (complete table in the supplementary) Backb. Percept. Model Decision Tree (DT) Bayes Net (BN) NS-CL DILP 1% 15% 25% 50% 75% 1% 15% 25% 50% 75% 1% 15% 25% 50% 75% 1% 15% 25% 50% 75% CNN D-OCB (Ours) 63.1 70.8 72.0 71.9 69.8 85.0 71.6 72.1 71.8 71.8 62.2 71.0 71.5 71.3 71.2 67.1 97.0 97.1 97.4 97.1 SlotVAE-KL 56.3 33.3 66.6 67.7 71.6 52.9 32.7 83.3 84.5 85.5 45.9 31.4 50.1 74.0 74.2 45.0 47.1 69.5 80.8 82.1 SlotVAE-NoVAE 55.6 48.3 59.8 68.1 69.4 69.3 53.6 70.0 60.2 61.5 61.4 56.7 66.6 72.2 72.9 70.9 74.6 89.0 95.6 96.0 Vanilla+SA 51.3 51.8 54.1 55.5 58.9 51.4 53.6 53.9 53.8 52.4 62.2 60.3 63.8 62.7 66.8 64.2 76.1 85.2 85.2 92.1 DINO D-OCB (Ours) 67.9 68.3 70.3 74.0 69.9 55.6 71.1 71.0 71.0 71.0 70.0 70.8 70.6 70.6 70.8 93.9 96.9 97.3 97.4 97.6 SlotVAE-KL 65.4 72.8 67.5 69.9 70.5 86.6 72.2 87.7 88.1 87.5 64.8 71.8 71.4 70.5 70.6 71.7 95.3 81.8 82.0 81.0 SlotVAE-NoVAE 64.8 68.0 69.2 66.4 66.6 69.3 71.5 72.0 71.4 71.5 71.4 71.4 71.8 71.0 71.0 95.9 96.8 97.1 97.6 97.6 Vanilla+SA 62.2 68.8 68.9 68.5 66.3 68.6 71.3 72.0 70.7 71.5 65.3 71.0 71.3 70.5 70.8 90.6 96.8 96.2 96.5 96.5 Table 5: Reasoning acc. (%) of D-OCB vs. Slot-VAE baseline on HAM10000 (RQ2) for three diagnostic rules across different backbones, supervision levels, and reasoning frameworks. Baseline F1 scores have been converted to % for direct comparison. Mean ± std. dev. over 5 folds. B Sup% Model DILP (%) Decision Tree (%) Bayes Net (%) NS-CL (%) R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3 CNN 1 D-OCB (Ours) 31.8±0.031.8± 0.0 34.8±0.034.8± 0.0 86.6±0.086.6± 0.0 59.7±0.059.7± 0.0 69.2±0.069.2± 0.0 72.6±0.072.6± 0.0 76.6±0.076.6± 0.0 82.6±0.082.6± 0.0 92.5±0.092.5± 0.0 52.7±0.052.7± 0.0 60.2±0.060.2± 0.0 92.5±0.092.5± 0.0 Slot-VAE - - - 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 15.6±1.015.6± 1.0 18.6±1.018.6± 1.0 2.6±1.02.6± 1.0 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 15 D-OCB (Ours) 69.7±0.069.7± 0.0 72.6±0.072.6± 0.0 80.1±0.080.1± 0.0 63.2±0.063.2± 0.0 58.2±0.058.2± 0.0 72.1±0.072.1± 0.0 84.6±0.084.6± 0.0 82.6±0.082.6± 0.0 95.0±0.095.0± 0.0 69.2±0.069.2± 0.0 64.2±0.064.2± 0.0 84.1±0.084.1± 0.0 Slot-VAE - - - 27.2±3.027.2± 3.0 15.4±3.015.4± 3.0 0.4±1.00.4± 1.0 30.0±1.030.0± 1.0 18.5±1.018.5± 1.0 0.0±0.00.0± 0.0 43.3±0.043.3± 0.0 37.9±0.037.9± 0.0 8.8±0.08.8± 0.0 75 D-OCB (Ours) 82.6±2.582.6± 2.5 82.3±2.282.3± 2.2 92.4±3.392.4± 3.3 79.9±2.379.9± 2.3 73.8±3.973.8± 3.9 84.6±1.684.6± 1.6 86.4±2.386.4± 2.3 83.9±0.583.9± 0.5 93.5±1.193.5± 1.1 76.3±1.676.3± 1.6 73.1±3.973.1± 3.9 89.7±2.989.7± 2.9 Slot-VAE - - - 40.2±2.040.2± 2.0 18.2±1.018.2± 1.0 0.0±0.00.0± 0.0 49.4±1.049.4± 1.0 36.8±1.036.8± 1.0 0.0±0.00.0± 0.0 50.9±0.050.9± 0.0 43.5±0.043.5± 0.0 18.9±0.018.9± 0.0 DINO 1 D-OCB (Ours) 85.1±0.085.1± 0.0 78.1±0.078.1± 0.0 88.6±0.088.6± 0.0 73.6±0.073.6± 0.0 64.7±0.064.7± 0.0 78.1±0.078.1± 0.0 19.9±0.019.9± 0.0 79.6±0.079.6± 0.0 90.0±0.090.0± 0.0 84.1±0.084.1± 0.0 73.1±0.073.1± 0.0 92.0±0.092.0± 0.0 Slot-VAE - - - 37.0±3.037.0± 3.0 14.4±3.014.4± 3.0 0.0±0.00.0± 0.0 37.2±1.037.2± 1.0 23.6±2.023.6± 2.0 0.0±0.00.0± 0.0 48.9±0.048.9± 0.0 41.9±0.041.9± 0.0 10.4±0.010.4± 0.0 15 D-OCB (Ours) 87.1±0.087.1± 0.0 82.1±0.082.1± 0.0 94.5±0.094.5± 0.0 68.7±0.068.7± 0.0 61.7±0.061.7± 0.0 70.6±0.070.6± 0.0 87.1±0.087.1± 0.0 82.1±0.082.1± 0.0 91.5±0.091.5± 0.0 79.6±0.079.6± 0.0 65.2±0.065.2± 0.0 92.5±0.092.5± 0.0 Slot-VAE - - - 62.5±2.062.5± 2.0 55.1±2.055.1± 2.0 0.0±0.00.0± 0.0 37.3±2.037.3± 2.0 10.7±2.010.7± 2.0 0.0±0.00.0± 0.0 65.2±0.065.2± 0.0 60.4±0.060.4± 0.0 25.5±0.025.5± 0.0 75 D-OCB (Ours) 87.6±2.187.6± 2.1 87.4±2.987.4± 2.9 93.2±1.393.2± 1.3 83.1±5.783.1± 5.7 78.6±6.878.6± 6.8 90.9±1.290.9± 1.2 90.7±1.990.7± 1.9 87.9±2.787.9± 2.7 93.5±1.193.5± 1.1 85.9±2.485.9± 2.4 84.4±4.984.4± 4.9 92.5±0.792.5± 0.7 Slot-VAE - - - 70.9±1.070.9± 1.0 66.3±2.066.3± 2.0 11.8±4.011.8± 4.0 47.3±3.047.3± 3.0 19.8±2.019.8± 2.0 0.0±0.00.0± 0.0 70.7±0.070.7± 0.0 66.7±0.066.7± 0.0 31.0±0.031.0± 0.0 4.3 RQ3: Confounder Resistance and Out-of-Distribution Reasoning A critical vulnerability in end-to-end differentiable logic frameworks is their susceptibility to dataset biases, where the loss from the logical reasoner propagates directly into the perception layer. This phenomenon encourages the perception module to entangle distinct concepts—such as ‘shape’ and ‘color’—to exploit spurious correlations in the training data rather than learning the true underlying causal rules. To evaluate this hypothesis, we utilize the confounded splits of the CLEVR-Hans3 and CLEVR-Hans7 benchmarks (55), where the training sets contain deliberate confounders (e.g., specific shapes artificially co-occurring with specific colors) that are subsequently removed in the out-of-distribution (OOD) test splits. We compare our D-OCB coupled with Differentiable Inductive Logic Programming (DILP) against several end-to-end, two-stage and a Ground Truth (Oracle) baseline. Unlike fully end-to-end neuro-symbolic networks that backpropagate logic-validation errors directly into the perception backbone, D-OCB structurally decouples these processes by extracting softly-valued visual predicates under weak supervision (1% to 75%) for frozen downstream evaluation. As Table 6 demonstrates, this synergy of object-centric biases and symbolic reasoning effectively suppresses likelihood blowup. Furthermore, our CLEVR-Hans7 analysis confirms that Dynamic Dimensionality Allocation (DDA) autonomously optimizes concept subspace capacities to prevent representation bottlenecks55 5 For detailed cross-validation results, hyperparameter configurations, and full DDA ablation see supplementary material.. Even at minimal supervision budgets (1% to 5%), D-OCB paired with DILP (utilizing DINO) achieves robust out-of-distribution (OOD) accuracy with marginal generalization gaps (1.1±0.81.1± 0.8 on CLEVR-Hans3; 3.00±0.563.00± 0.56 on CLEVR-Hans7), matching densely supervised baselines and resisting confounders without explicit annotations. In RQ3, by isolating the perceptual bottleneck, D-OCB successfully suppresses likelihood blowup and ignores spurious training correlations, achieving robust OOD generalization that matches densely supervised paradigms. Table 6: Confounder Resistance on CLEVR-Hans3 and CLEVR-Hans7. Comparison of D-OCB against end-to-end architectures and dense-supervision baselines. We report train acc. (in-distribution), test acc. (out-of-distribution), and the OOD Gap (↓ ). Mean ± std. dev. over 5 folds. CLEVR-Hans3 Benchmark Method Sup. Train Acc ↑ Test Acc ↑ OOD Gap ↓ E2E CNN Basel. 0% 100.0±0.0100.0± 0.0 70.3±0.070.3± 0.0 29.7±0.029.7± 0.0 NeSy 100% 98.5±0.098.5± 0.0 81.7±0.081.7± 0.0 16.8±0.016.8± 0.0 NeSy-XIL 100% 100.0±0.0100.0± 0.0 91.3±0.091.3± 0.0 8.7±0.08.7± 0.0 α 100% 97.5±0.097.5± 0.0 97.5±0.097.5± 0.0 −0.0±0.0-0.0± 0.0 NEUMANN 100% 96.7±0.096.7± 0.0 97.4±0.097.4± 0.0 −0.8±0.0-0.8± 0.0 D-OCB + DILP (DINO) 1% 72.8±18.672.8± 18.6 71.2±18.271.2± 18.2 1.6±0.91.6± 0.9 5% 98.2±0.498.2± 0.4 97.0±0.497.0± 0.4 1.1±0.81.1± 0.8 15% 98.7 ± 0.4 98.4 ± 0.6 0.3 ± 0.3 D-OCB + DILP (CNN) 1% 36.0±2.336.0± 2.3 35.3±0.735.3± 0.7 0.8±1.70.8± 1.7 15% 73.9±1.773.9± 1.7 72.5±0.972.5± 0.9 1.4±2.11.4± 2.1 75% 98.3±0.398.3± 0.3 97.3±0.697.3± 0.6 1.0±0.81.0± 0.8 GT O. + DILP 100% 100.0±0.0100.0± 0.0 100.0±0.0100.0± 0.0 0.0±0.00.0± 0.0 CLEVR-Hans7 Benchmark Method Sup. Train Acc ↑ Test Acc ↑ OOD Gap ↓ E2E CNN Basel. 0% 69.13±0.0069.13± 0.00 51.43±0.0051.43± 0.00 17.70±0.0017.70± 0.00 CNN Baseline 100% 70.79±0.0070.79± 0.00 68.46±0.0068.46± 0.00 2.33±0.002.33± 0.00 DeepProbLog 100% 85.35±0.0085.35± 0.00 67.98±0.0067.98± 0.00 17.37±0.0017.37± 0.00 D-OCB + DILP (DINO) 1% 80.86±2.2880.86± 2.28 78.55±2.5778.55± 2.57 2.31±0.412.31± 0.41 5% 90.60±0.1290.60± 0.12 87.60±0.5687.60± 0.56 3.00±0.563.00± 0.56 15% 92.43±0.1692.43± 0.16 89.07±0.2789.07± 0.27 3.36±0.373.36± 0.37 D-OCB + DILP (CNN) 1% 22.76±4.1622.76± 4.16 22.21±3.9922.21± 3.99 0.55±0.440.55± 0.44 15% 76.91±3.6176.91± 3.61 69.82±4.4969.82± 4.49 7.09±0.887.09± 0.88 75% 92.30±0.3292.30± 0.32 87.26±0.3287.26± 0.32 5.05±0.435.05± 0.43 GT O. + DILP 100% 94.21 ± 0.00 93.56 ± 0.00 0.65 ± 0.00 5 Conclusion In this work, we proposed the Dynamic Orthogonal Concept Bottleneck (D-OCB) neural network, a visual feature extractor for neuro-symbolic reasoning that can leverage extremely weak supervision (1%1\% to 15%15\%) for human-alignment of the predicates. Crucially, D-OCB adds self-supervision via a reconstruction and a cross-correlation orthogonal loss, paired with a novel dynamic allocation of extracted features to the labeled human-interpretable concepts. This adaptive mechanism actively redistributes latent capacity, transferring dimensions from well-represented concepts to those lacking accuracy. Our empirical evaluations on CLEVR, CLEVR-Hans, and HAM10000 demonstrate that a trained D-OCB’s concept predictions are not only superior in low supervision accuracy, but also seamlessly drive diverse off-the-shelf symbolic reasoners to near-oracle performance, even without joint backpropagation. Furthermore, by the structural decoupling of perception from logical induction D-OCB-based neuro-symbolic pipelines achieve state-of-the-art out-of-distribution generalization against confounding factors. Ultimately, D-OCB establishes a highly stable, efficient, and computationally accessible visual-logical primitive layer for complex downstream reasoning tasks. As a remaining limitation, the architecture currently relies on a rigid slot attention mechanism, which necessitates the manual, a priori definition of the maximum number of object slots. Additionally, while the latent vector compression effectively isolates structural concepts, it inherently filters out fine-grained details, which can degrade performance on highly textured datasets where critical information is lost during bottleneck projection. Future work can address these constraints by exploring closed-loop abductive learning, wherein the downstream System 2 logical reasoner provides top-down error corrections to iteratively refine the System 1 perceptual module. acknowledgments-disclosure-of-funding.G.S. and S.T. acknowledge support through the project “chAI” funded by the German Federal Ministry of Research, Technology and Space (BMFTR), grant no. 16IS24058. References Agrawal et al. (2025) S. Agrawal, D. Vemuri, S. S. C. P, and V. N. Balasubramanian Walking the web of concept-class relationships in incrementally trained interpretable models. External Links: 2502.20393, Link Cited by: §2. Asperti and Trentin (2020) A. Asperti and M. Trentin Balancing reconstruction error and kullback-leibler divergence in variational autoencoders. External Links: 2002.07514, Link Cited by: §4. Bardes et al. (2022) A. Bardes, J. Ponce, and Y. LeCun Vicreg: variance-invariance-covariance regularization for self-supervised learning. In International Conference on Learning Representations, Cited by: §3.1. Biza et al. (2023) O. Biza, S. van Steenkiste, M. S. M. Sajjadi, G. F. Elsayed, A. Mahendran, and T. Kipf Invariant slot attention: object discovery with slot-centric reference frames. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, p. 2507–2527. External Links: Link Cited by: §4. Brady et al. (2025) J. Brady, J. von Kügelgen, S. Lachapelle, S. Buchholz, T. Kipf, and W. Brendel Interaction asymmetry: A general principle for learning composable abstractions. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, External Links: Link Cited by: §1, §2, §2. Breiman et al. (1984) L. Breiman, J. Friedman, R. A. Olshen, and C. J. Stone Classification and regression trees. Routledge. Cited by: §4.2. Chen (2017) G. Chen Latent discriminant analysis with representative feature discovery. Proceedings of the AAAI Conference on Artificial Intelligence 31, p. . External Links: Document Cited by: §3.3. Chen et al. (2019) R. T. Q. Chen, X. Li, R. Grosse, and D. Duvenaud Isolating sources of disentanglement in variational autoencoders. External Links: 1802.04942, Link Cited by: §2. Dai and Wipf (2019) B. Dai and D. Wipf Diagnosing and enhancing vae models. External Links: 1903.05789, Link Cited by: §1. Daniele et al. (2024) A. Daniele, T. Campari, S. Malhotra, and L. Serafini Simple and effective transfer learning for neuro-symbolic integration. External Links: 2402.14047, Link Cited by: §4. Didolkar et al. (2025) A. Didolkar, A. Zadaianchuk, R. Awal, M. Seitzer, E. Gavves, and A. Agrawal CTRL-o: language-controllable object-centric visual representation learning. External Links: 2503.21747, Link Cited by: §2. Dinu et al. (2024) M. Dinu, C. Leoveanu-Condrei, M. Holzleitner, W. Zellinger, and S. Hochreiter SymbolicAI: a framework for logic-based approaches combining generative models and solvers. External Links: 2402.00854, Link Cited by: §2. d’Avila Garcez and Lamb (2023) A. d’Avila Garcez and L. C. Lamb Neurosymbolic AI: the 3rd wave. Artificial Intelligence Review 56 (11), p. 12387–12406. External Links: ISSN 1573-7462, Document Cited by: §1. Goyal and Bengio (2022) A. Goyal and Y. Bengio Inductive biases for deep learning of higher-level cognition. External Links: 2011.15091, Link Cited by: §3.1. Heek et al. (2026) J. Heek, E. Hoogeboom, T. Mensink, and T. Salimans Unified latents (ul): how to train your latents. External Links: 2602.17270, Link Cited by: §3.2. Hou and Behdinan (2022) C. K. J. Hou and K. Behdinan Dimensionality reduction in surrogate modeling: a review of combined methods. Data Science and Engineering 7 (4), p. 402–427. External Links: Document, Link Cited by: §1. Hu et al. (2025) L. Hu, T. Huang, H. Xie, X. Gong, C. Ren, Z. Hu, L. Yu, P. Ma, and D. Wang Semi-supervised concept bottleneck models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 2110–2119. Cited by: §2. Jing et al. (2021) L. Jing, P. Vincent, Y. LeCun, and Y. Tian Understanding dimensional collapse in contrastive self-supervised learning. CoRR abs/2110.09348. External Links: Link, 2110.09348 Cited by: §1. Johnson et al. (2017) J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. B. Girshick CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, p. 1988–1997. External Links: Link, Document Cited by: §4. Kazhdan et al. (2021) D. Kazhdan, B. Dimanov, H. A. Terre, M. Jamnik, P. Liò, and A. Weller Is disentanglement all you need? Comparing concept-based & disentanglement approaches. CoRR abs/2104.06917. External Links: 2104.06917 Cited by: §2. Kikaj et al. (2025) A. Kikaj, G. Marra, F. Geerts, R. Manhaeve, and L. De Raedt DeepGraphLog for Layered Neurosymbolic AI. In ECAI 2025, p. 1551–1558. External Links: Document Cited by: §1, §2. Kim et al. (2023) E. Kim, D. Jung, S. Park, S. Kim, and S. Yoon Probabilistic concept bottleneck models. External Links: 2306.01574, Link Cited by: §4. Kim and Mnih (2018) H. Kim and A. Mnih Disentangling by factorising. In Proc. 2018 Int. Conf. Machine Learning, p. 2649–2658. External Links: ISSN 2640-3498 Cited by: §2. Kirsch et al. (2018) L. Kirsch, J. Kunesch, and D. Barber Modular networks: learning to decompose neural computation. In Advances in Neural Information Processing Systems, Vol. 31. Cited by: §3.1. Kishor (2022) R. Kishor Neuro-symbolic ai: bringing a new era of machine learning. International Journal of Research Publication and Reviews. External Links: Link Cited by: §1. Knab et al. (2026) P. Knab, D. Steinmann, C. Bartelt, K. Kersting, B. Schiele, T. Seidl, U. Schlegel, and W. Stammer What’s in the bottle? a survey and roadmap of concept bottleneck models. Transactions on Machine Learning Research. Note: External Links: ISSN 2835-8856, Link Cited by: §1, §2. Kuhn (1955) H. W. Kuhn The hungarian method for the assignment problem. Naval Research Logistics (NRL) 52. External Links: Link Cited by: §3.3, §4. Liu et al. (2025) H. Liu, R. Zhao, H. Chen, and J. Pajarinen MetaSlot: break through the fixed number of slots in object-centric learning. CoRR abs/2505.20772. External Links: Link, Document, 2505.20772 Cited by: §2. Locatello et al. (2019a) F. Locatello, G. Abbati, T. Rainforth, S. Bauer, B. Schölkopf, and O. Bachem On the Fairness of Disentangled Representations. In Advances in Neural Information Processing Systems, Vol. 32. Cited by: §2. Locatello et al. (2019b) F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Schölkopf, and O. Bachem Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. In Proceedings of the 36th International Conference on Machine Learning, p. 4114–4124. External Links: ISSN 2640-3498 Cited by: §2. Locatello et al. (2020) F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, and T. Kipf Object-Centric Learning with Slot Attention. In Advances in Neural Information Processing Systems, Vol. 33, p. 11525–11538. Cited by: §2, §3.1. Losch et al. (2021) M. Losch, M. Fritz, and B. Schiele Semantic Bottlenecks: Quantifying and Improving Inspectability of Deep Representations. International Journal of Computer Vision 129 (11), p. 3136–3153. External Links: ISSN 1573-1405, Document Cited by: §2. Lucas et al. (2019a) J. Lucas, G. Tucker, R. Grosse, and M. Norouzi Don’t blame the elbo! a linear vae perspective on posterior collapse. In Advances in Neural Information Processing Systems, Vol. 32. Cited by: §3.2. Lucas et al. (2019b) J. Lucas, G. Tucker, R. Grosse, and M. Norouzi Don’t blame the elbo! a linear vae perspective on posterior collapse. External Links: 1911.02469, Link Cited by: §4. Manhaeve et al. (2018) R. Manhaeve, S. Dumančić, A. Kimmig, T. Demeester, and L. D. Raedt DeepProbLog: neural probabilistic logic programming. External Links: 1805.10872, Link Cited by: §1, §4. Mao et al. (2019) J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu The neuro-symbolic concept learner: interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations, Cited by: §4.2. Mao et al. (2018) J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In Int. Conf. Learning Representations, Cited by: §1, §4.2. Marconato et al. (2023) E. Marconato, S. Teso, A. Vergari, and A. Passerini Not all neuro-symbolic concepts are created equal: analysis and mitigation of reasoning shortcuts. External Links: 2305.19951, Link Cited by: §3. Mattei and Frellsen (2018) P. Mattei and J. Frellsen Leveraging the exact likelihood of deep latent variable models. In Advances in Neural Information Processing Systems, Vol. 31. Cited by: §2, §3.2. Minsky (1991) M. Minsky Logical versus analogical or symbolic versus connectionist or neat versus scruffy. AI Mag. 12, p. 34–51. External Links: Link Cited by: §1, §2. Mosbach et al. (2025) M. Mosbach, J. N. Ewertz, A. Villar-Corrales, and S. Behnke SOLD: slot object-centric latent dynamics models for relational manipulation learning from pixels. External Links: 2410.08822, Link Cited by: §2. Muggleton (1991) S. Muggleton Inductive logic programming. New Generation Computing 8 (4), p. 295–318. External Links: ISSN 1882-7055, Document Cited by: §3. Noci et al. (2022) L. Noci, S. Anagnostidis, L. Biggio, A. Orvieto, S. P. Singh, and A. Lucchi Signal propagation in transformers: theoretical perspectives and the role of rank collapse. External Links: 2206.03126, Link Cited by: §3.3. Oikarinen et al. (2023) T. Oikarinen, S. Das, L. M. Nguyen, and T. Weng Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §2. Oquab et al. (2024) M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jégou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res. 2024. External Links: Link Cited by: §3.1, §4. Pearl (1988) J. Pearl Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann. Cited by: §4.2. Roth et al. (2025) S. Roth, L. Baur, D. Boer, and S. Kramer Enhancing Symbolic Machine Learning by Subsymbolic Representations. External Links: 2506.14569, Document Cited by: §2. Schölkopf et al. (2021) B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio Toward causal representation learning. Proceedings of the IEEE 109 (5), p. 612–634. External Links: Document Cited by: §3.1. Seitzer et al. (2023) M. Seitzer, M. Horn, A. Zadaianchuk, D. Zietlow, T. Xiao, C. Simon-Gabriel, T. He, Z. Zhang, B. Schölkopf, T. Brox, and F. Locatello Bridging the gap to real-world object-centric learning. External Links: 2209.14860, Link Cited by: §2. Shindo et al. (2021a) H. Shindo, D. S. Dhami, and K. Kersting Neuro-symbolic forward reasoning. External Links: 2110.09383, Link Cited by: §2. Shindo et al. (2021b) H. Shindo, M. Nishino, and A. Yamamoto Differentiable inductive logic programming for structured examples. CoRR abs/2103.01719. External Links: Link, 2103.01719 Cited by: §2, §4.2. Shindo et al. (2023) H. Shindo, V. Pfanschilling, D. S. Dhami, and K. Kersting α : thinking visual scenes as differentiable logic programs. Machine Learning 112 (4), p. 1465–1497. Cited by: §2, §4. Shindo et al. (2024) H. Shindo, V. Pfanschilling, D. S. Dhami, and K. Kersting Learning differentiable logic programs for abstract visual reasoning. Machine Learning 113. Cited by: §4. Stammer et al. (2020) W. Stammer, P. Schramowski, and K. Kersting Right for the right concept: revising neuro-symbolic concepts by interacting with their explanations. CoRR abs/2011.12854. External Links: Link, 2011.12854 Cited by: §4. Stammer et al. (2021) W. Stammer, P. Schramowski, and K. Kersting Right for the right concept: revising neuro-symbolic concepts by interacting with their explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3619–3629. Cited by: §4.3. Steinmann et al. (2025) D. Steinmann, W. Stammer, A. Wüst, and K. Kersting Object-centric concept-bottlenecks. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, p. 68899–68924. External Links: Link Cited by: §2. Suter et al. (2019) R. Suter, D. Miladinović, B. Schölkopf, and S. Bauer Robustly disentangled causal mechanisms: validating deep representations for interventional robustness. External Links: 1811.00007, Link Cited by: §3.1. Sutter et al. (2025) T. M. Sutter, Y. Meng, A. Agostini, D. Chopard, N. Fortin, J. E. Vogt, B. Shahbaba, and S. Mandt Unity by diversity: improved representation learning in multimodal vaes. External Links: 2403.05300, Link Cited by: §1. Tschandl et al. (2018) P. Tschandl, C. Rosendahl, and H. Kittler The HAM10000 dataset: A large collection of multi-source dermatoscopic images of common pigmented skin lesions. CoRR abs/1803.10417. External Links: Link, 1803.10417 Cited by: §4. Vasudevan and Ramakrishna (2017) V. Vasudevan and M. Ramakrishna A hierarchical singular value decomposition algorithm for low rank matrices. CoRR abs/1710.02812. External Links: Link, 1710.02812 Cited by: §3.3. Wang et al. (2024) Y. Wang, L. Liu, and J. Dauwels Slot-vae: object-centric scene generation with slot attention. External Links: 2306.06997, Link Cited by: §4. Wang et al. (2021) Y. Wang, D. M. Blei, and J. P. Cunningham Posterior collapse and latent variable non-identifiability. Advances in Neural Information Processing Systems 34, p. 5443–5455. Cited by: §3.2. Wang et al. (2023) Y. Wang, D. M. Blei, and J. P. Cunningham Posterior collapse and latent variable non-identifiability. External Links: 2301.00537, Link Cited by: §4. Watters et al. (2019) N. Watters, L. Matthey, C. P. Burgess, and A. Lerchner Spatial broadcast decoder: A simple architecture for learning disentangled representations in VAEs. CoRR abs/1901.07017. External Links: Link, 1901.07017 Cited by: §3.1. Wüst et al. (2025) A. Wüst, W. Stammer, H. Shindo, L. Helff, D. S. Dhami, and K. Kersting Synthesizing visual concepts as vision-language programs. External Links: 2511.18964, Link Cited by: §2. Zarlenga et al. (2022) M. E. Zarlenga, P. Barbiero, G. Ciravegna, G. Marra, F. Giannini, M. Diligenti, Z. Shams, F. Precioso, S. Melacci, A. Weller, P. Lio, and M. Jamnik Concept embedding models: beyond the accuracy-explainability trade-off. External Links: 2209.09056, Link Cited by: §4. Zbontar et al. (2021) J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny Barlow twins: self-supervised learning via redundancy reduction. In International Conference on Machine Learning, p. 12310–12320. Cited by: §3.1.