Paper deep dive
LinSlot: Exploiting Linear Representation hypothesis for unsupervised attribute discovery from slot based object representation
Sanket Gandhi, Utkarsh Giri, Varun Subramanium, Rohan Paul, Parag Singla
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/9/2026, 7:50:11 AM
Summary
The paper introduces LinSlot, a framework for unsupervised attribute discovery from slot-based object representations. It leverages the Linear Representation Hypothesis (LRH) to model objects as linear combinations of attributes (blocks). The method uses a two-stage inference process: first discovering object slots via Slot Attention, then inferring attribute blocks via Block Attention. The architecture optimizes an Evidence Lower Bound (ELBO) with reconstruction, slot, and block regularizers, demonstrating improved disentanglement and image editing capabilities over state-of-the-art methods.
Entities (8)
Relation Signals (7)
LinSlot → leverages → Linear Representation Hypothesis
confidence 95% · Our key contribution is leveraging the Linear Representation Hypothesis (LRH)...
LinSlot → uses → Slot Attention
confidence 92% · we use Slot Attention for the first stage...
LinSlot → uses → Block Attention
confidence 92% · ...and introduce Block Attention for the second stage
Slot Attention → discovers → object slots
confidence 90% · we first infer the object slots S using a slot-based object discovery method
Block Attention → infers → attribute blocks
confidence 90% · The block inference module should therefore infer the blocks using the slots as input.
LinSlot → optimizes → Evidence Lower Bound
confidence 90% · This architecture effectively optimizes the Evidence Lower Bound (ELBO)...
LinSlot → improves → DCI scores
confidence 88% · Our experiments on multiple datasets demonstrate improvements in DCI scores over state-of-the-art methods.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper studies the problem of learning disentangled representations of objects and their attributes from raw, unstructured image data. Slot-based methods have shown considerable success in unsupervised learning of object representations from images. Block-slot attention-based methods extend this framework to attribute representations by assuming a uniform factorization of object representations into attributes, which may be suboptimal and consequently limit the quality of the learned representations. We therefore investigate a framework for jointly discovering object and attribute representations. Our key contribution is leveraging the Linear Representation Hypothesis (LRH), which postulates that composable concepts can be represented as linearly additive subspaces in slot representations. Based on this insight, we propose a probabilistic model connecting images, slots (objects), and blocks (attributes). We present an architecture that leverages block attention to connect attribute representations to slots and incorporates LRH in both object and attribute representation spaces. This architecture effectively optimizes the Evidence Lower Bound (ELBO) of the proposed graphical model. Our experiments demonstrate (i) effective discovery of disentangled object and attribute representations, (ii) empirical evidence for LRH in slot space, and (iii) the ability to perform image editing owing to the disentangled and interpretable nature of the learned representations. Our experiments on multiple datasets demonstrate improvements in DCI scores over state-of-the-art methods.
Tags
Links
- Source: https://arxiv.org/abs/2610.10722v1
- Canonical: https://arxiv.org/abs/2610.10722v1
Trouble viewing inline? Open PDF directly →
Full Text
74,193 characters extracted from source content.
Expand or collapse full text
LinSlot: Exploiting Linear Representation hypothesis for unsupervised attribute discovery from slot based object representation Sanket Gandhi Affiliation: Yardi School of AI, IIT Delhi Utkarsh Giri Affiliation: Yardi School of AI, IIT Delhi Varun Subramanium Affiliation: Department of Computer Science and Engineering, IIT DelhiCorrespondence to: sanket.gandhi@scai.iitd.ac.in Rohan Paul & Parag Singla Affiliation: Yardi School of AI, IIT Delhi Affiliation: Department of Computer Science and Engineering, IIT DelhiCorrespondence to: sanket.gandhi@scai.iitd.ac.in Abstract This paper studies the problem of learning disentangled representations of objects and their attributes from raw, unstructured image data. Slot-based methods have shown considerable success in unsupervised learning of object representations from images. Block-slot attention-based methods extend this framework to attribute representations by assuming a uniform factorization of object representations into attributes, which may be suboptimal and consequently limit the quality of the learned representations. We therefore investigate a framework for jointly discovering object and attribute representations. Our key contribution is leveraging the Linear Representation Hypothesis (LRH), which postulates that composable concepts can be represented as linearly additive subspaces in slot representations. Based on this insight, we propose a probabilistic model connecting images, slots (objects), and blocks (attributes). We present an architecture that leverages block attention to connect attribute representations to slots and incorporates LRH in both object and attribute representation spaces. This architecture effectively optimizes the Evidence Lower Bound (ELBO) of the proposed graphical model. Our experiments demonstrate (i) effective discovery of disentangled object and attribute representations, (i) empirical evidence for LRH in slot space, and (i) the ability to perform image editing owing to the disentangled and interpretable nature of the learned representations. Our experiments on multiple datasets demonstrate improvements in DCI scores over state-of-the-art methods. 1 Introduction A core aspect of the human intelligence is its ability to form disentangled representations of the world (Fodor and Pylyshyn, 1988; Whitehead, 1927; Dehaene et al., 2022), without having any access to supervised data. Enabling deep learning models with such representations has been a long-standing goal and is desirable for a number of reasons, i.e., having properties such as interpretability, compositional generalization, and sample efficiency (Bengio et al., 2013; Greff et al., 2020; Lake et al., 2017; Goyal and Bengio, 2022). In this work, we are specifically interested in the task of joint disentanglement of objects and their attributes (such as color, shape, size etc.) in static images without any supervision. Extracting such information automatically from images, in an unsupervised manner, will go a long way in building models, that have above desired properties of generalizability, sample efficiency and robustness to noise. Recently proposed slot attention based methods have shown promising results on learning object-centric disentangled representation in images(Locatello et al., 2020b; Akan and Yemez, 2025a; Didolkar et al., 2025), and these methods have been effectively applied for downstream tasks such as image segmentation, compositional generation and VQA. Extending this work, block-slot attention (Singh et al., 2023; Stammer et al., 2024) based methods partition the learned object representations (slots) into attributes (factors) (e.g., color, shape) yielding a fine grained and interpretable understanding of the image generation process. Despite this progress, a fundamental question remains yet to be answered: is there a theoretical framework, in the form of a generative graphical model, which can guide the joint discovery of objects and their attributes in images in an unsupervised manner and provide a guarantee on the identifiability of latent factors under suitable assumptions? If so, what are these assumptions under which such identifiability of factors (up to a permutation) holds, and how can we design an encoder-decoder architecture based on the assumptions of our model, to effectively discover the objects, and their latent factors from a given set of images. Our solution to this problem is motivated by the linear representation hypothesis which has been used extensively for better understanding of latent factors in textual domains Park et al. (2024b). There have been some recent attempts to extend this work to the visual domains (Gao et al., 2025; Fel et al., 2026b), and that becomes our starting point. Specifically, we make an assumption that the object can be represented as a stochastic linear combination of its attribute values sampled from a given prior. Under this assumption, our graphical model defines a three-step sampling process: (1) sampling a concept index (e.g., red, blue, or green) and its corresponding value (e.g., the degree of redness) from a prior distribution; (2) constructing each object representation as a linear function of the sampled factor values; and (3) generating the image through a learned function of the resulting object representations. This graphical model enables derivation of an ELBO (evidence-lower-bound) formulation for optimizing the likelihood. Correspondingly, our architecture, for disentanglement, involves a two-step encoder, going from an image to corresponding slots using existing slot-attention based methods (Locatello et al., 2020b) in the outer loop, which formulates this as a problem of competing attention over image patches (2) followed by block discovery module, which first projects the slot to a set of latent dimensions, over which blocks attend via competing attention, for the discovery of latent factors. The factors are poised to be linear combination of underlying (learnable) concepts, in line with earlier work (Singh et al., 2023), albeit with strong justification arising from our graphical model, for using the linear combination, as a way of imposing mixture of Gaussian prior. Decoding works in reverse, by first constructing the slots from the discovered factors, followed by construction of the image from the decoded slot. We experiment with two variants of our approach, one using a decoder working directly on the slot representation, and the other one, where the internal structure of each slot, in terms of its constituent blocks, is exposed to the decoder. Entire architecture is trained in an end to end manner, with a curriculum, for the joint discovery of objects (slots) and their attributes (factors). We first show experimentally the validation of the linear structure in the slot space across the slots methods and datasets. We demonstrate the object manipulation via simple slot arithmetic of addition and subtraction. We perform extensive evaluation over three benchmark datasets to evaluate the efficacy of our approach both for object discovery as well as factor discovery. We find that our two stage pipeline not only results in improved factor discovery, it also improves the object discovery, compared to existing baselines, both in quantitative as well as qualitative evaluation. We also perform ablations demonstrating the contribution of various components our two-stage pipeline towards the efficacy of object and attribute discovery. We plan to release our code publicly on acceptance. 2 Related Work Object-Centric Learning: Unsupervised object discovery methods aim at learning disentangled object representations for scenes without supervision. Earlier methods used generative modeling of the scene and imposed the object level structure in the latents (Eslami et al., 2016; Kosiorek et al., 2018; Jiang* et al., 2020; Lin et al., 2020; Kossen et al., 2020; Stanic and Schmidhuber, 2019; Stelzner et al., 2019). These spatial attention methods struggle to find occluded and non-local objects. Scene mixture methods models scenes as composition of object images Greff et al. (2017); Greff et al. (2019); Burgess et al. (2019); Engelcke et al. (2020); Engelcke et al. (2021) - however inference is computationally challenging. Slot attention Locatello et al. (2020b) leverages competitive spatial attention over the image to allow slots to bind to objects in an image. Others (Kipf et al., 2022; Elsayed et al., 2022; Wu et al., 2023a) model temporal dynamics and discover objects from images. Recent efforts extend the approach to realistic scenes by incorporating more expressive decoders Singh et al. (2022a); Singh et al. (2022b); Jiang et al. (2023); Wu et al. (2023b) as well as using pre-trained image features (Caron et al., 2021) combined with a diffusion decoder Seitzer et al. (2023); Zadaianchuk et al. (2023); Singh et al. (2025); Akan and Yemez (2025b); Liu et al. (2025); Nguyen et al. (2026). Disentangled representation learning: Attribute discovery aims at learning independent factored representations such as colour, shape, texture etc. that explain generation of objects observed in images. Earliest approaches train an VAE (Kingma and Welling, 2014) based methods (Higgins et al., 2017; Kim and Mnih, 2018; Chen et al., 2018) but suffer from identifiability issues (Locatello et al., 2018; Khemakhem et al., 2019) necessitating the need for inductive biases to aid disentanglement. However with weak supervision it is guaranteed to recover attributes (Shu et al., 2020; Ahuja et al., 2022; Locatello et al., 2020a). Recently diffusion and transformer based methods were also proposed Yang et al. (2022); Yang et al. (2023). Enabling compositionality of representations also enabled disentanglement Jung et al. (2025). Linear Representation Hypothesis: Presence of the linear structure in the textual domain well established (Mikolov et al., 2013; Park et al., 2024a; Park et al., 2025; Jiang et al., 2024) even in LLMs. In vision domain Trager et al. (2023); Wang et al. (2023). However recovering such representations without supervision is difficult Locatello et al. (2018). Existing work uses Sparse Autoencoders (SAEs) for recovering the concepts from latent representation which is additive in concepts(Gao et al., 2025; Fel et al., 2026b). However block have to identify all the values of the specific factor and SAE atoms fail to capture such representations (Bhalla et al., 2026). One option is to enable grouping of SAE atoms (Fel et al., 2026a) but it assumes the presence of the attributes is sparse which is not true for our proposed generative model. 3 Background: Slot Attention Mathematically, given an input image X∈ℝH×W×3X ^H× W× 3 it is first passed through an encoder typically implemented as CNN to get the latent representation of the image, as a vector as ∈ℝNe×dinputE ^N_e× d_input, where NeN_e represents the number of spatial features, and dinputd_input represents their dimension. Following this, the each element in the slots set ∈ℝNs×dslotS ^N_s× d_slot is initialized by sampling from a normal distribution (μ,Σ)N(μ, ) with learnable parameters. These slots then attend to the input image features with learnable k, q and v projection functions (matrices) to get the attention matrix A as: A=Softmax(q()k()Tdslot,axis=’slots’)A= Softmax ( q(S)k(E)^T d_slot, axis='slots' ) (1) Additionally, the attention matrix A is further weight normalized along the ’axis’ dimension to ensure competitive nature of the attention. The update vector U=A∗v()U=A*v(E), resulting from the attention mechanism, is then fed to a recurrent neural network as input and previous slots as states. The output of the recurrent neural network is passed through an MLP to get the updated slot representation. This process is repeated for fixed number of iterations to get the final slot representation. Refer to Algorithm 1 in Locatello et al. (2020b) for a detailed pseudocode. 4 LinSlot In this section, we present our framework LinSlot for unsupervised factor representation learning from slots. We first formulate a generative model of images that exploits the linearity of the slot representation space (Section 4.1). We then establish identifiability of the proposed generative formulation under weak assumptions (Section 4.2). Next, we describe a deterministic inference procedure motivated by this formulation, which first discovers object slots using Slot Attention and then infers attribute representations using our Block Attention mechanism (Section 4.3). Finally, we describe the training objectives motivated by the resulting ELBO and our curriculum-based training procedure (Section 4.4). (a) Generative model. (b) Linear representation of slots. Figure 1: Overview of the proposed representation and generative model. 4.1 generative model In this section, we describe the generative model of the images considered in our work. We assume that each image consists of a set of objects, where each object is characterized by a set of factors of variation, such as color, shape, and size. Each factor is characterized by a fixed set of concepts, and each factor is associated with a particular concept from this set. For example, red, blue, and green are concepts for the color factor. The generative process proceeds in three steps: (1) for each factor of each object, a concept index is first sampled, after which the corresponding concept is sampled from its own prior and assigned to that factor. We assume the concept prior to be a normal distribution, which results in a GMM for the distribution of each factor; (2) the representation of each object is sampled conditioned on its factor values, and specifically, is given by the linear sum of the factor representations; and (3) the image is generated conditioned on the representations of all objects. Fig. 1(a) illustrates this generative process. Our problem statement then is: given a set of training images, a family of prior distribution p over concepts, an unknown function f mapping the sampled concept values to the corresponding object representation, and another function g mapping the set of object representations to the image representation, under what conditions, can we uniquely discover the (a) parameters of the concept prior (b) values of the underlying concepts upto a permutation, (c) inverse of the functions f, given fixed g? This leads to the following formulation and the corresponding identifiability result. Specifically, assume that the image X∈ℝH×W×3X ^H× W× 3 is generated by set of N latent object representations referred as slots decoded by S=s1,…,sNS=\s_1,...,s_N\. Each slot si∈ℝdslots_i ^d_ slot itself is independently generated by M latent factor (attribute) representations referred as blocks denoted by the ordered set ci=c1,i,..,cM,ic_i=\c_1,i,..,c_M,i\. Each jthj^th block cj,i∈ℝdblockc_j,i ^d_ block is sampled independently from it’s own specific prior which is shared across all the objects. Let C=c1,..,cNC=\c_1,..,c_N\. The generative process can be expressed by following joint distribution: pθ(X,,)=pθX(X|S)∏i=1N(pθS(si|ci)∏j=1MpθCj(cj,i))p_θ(X,S,C)=p_ _X(X|S) _i=1^N (p_ _S(s_i|c_i) _j=1^Mp_ _C_j(c_j,i) ) (2) Where θ=θX,θS,θCjj=1Mθ=\ _X, _S,\ _C_j\_j=1^M\ are the parameters of the distribution. We further assume the prior on the blocks to be a Gaussian mixture model (GMM) which are specific to the blocks. cj,i∼pθCj(Cj):=∑k=1Kjπk,j(μk,j,Σk,j)c_j,i p_ _C_j(C_j):= _k=1^K_j _k,jN( _k,j, _k,j) (3) Linear Structure of Slots: We also restrict the slot representations to be linearly additive in terms of some transformed block representations. That is si∼pθS(S|ci):=(μ(ci),ΣS)whereμ(ci)=∑j=1Mfj(cj,i)+bs_i p_ _S(S|c_i):=N(μ(c_i), _S)\ \ \ \\ \ where\ \ \ \ \ μ(c_i)= _j=1^Mf_j(c_j,i)+b (4) Such additive structure implies the presence of the global delta directions in slots space which represents a direction from one factor value to another value. For example in Fig. 1(b) the delta direction of the blue to red color and cube to cylinder shape denoted as δcolor[red→blue] _color[red→ blue]. These delta directions can be used to manipulate factors in the slot representation by simply vector addition. For example adding the delta direction δcolor[red→blue] _color[red→ blue] to any red object slot should change the object color to blue. 4.2 identifiability Informal Theorem 1 (Identifiability of the Blocks ). Under the proposed generative model and the assumptions that fjj=1M\f_j\_j=1^M are piecewise affine, Mdblock≤dslotMd_ block≤ d_ slot and Ki≠KuKwK_i≠ K_uK_w for any triplets i,u,w∈1,..,Mi,u,w∈1,..,M, then the blocks are identifiable upto translation and permutation given slots are fixed. The proof is provided in the A.1 which follows from the Kivva et al. (2022) where we first show that the concatenated blocks are also from GMM. Then the number of 4.3 inference We approximate the true posterior pθ(S,C|X)p_θ(S,C|X) with variational approximation qϕ(S,C|X)q_φ(S,C|X) due to intractability of the former. We further factorize the variational posterior distribution as qϕ(S,C|X)=qϕS(S|X)∏i=1NqϕC(ci|si)q_φ(S,C|X)=q_ _S(S|X) _i=1^Nq_ _C(c_i|s_i) (5) One benefit of having such inference modeling is that the slot posterior qϕ(S|X)q_φ(S|X) can be implemented as any slot based method and could also be pretrained. The block discovery method is not tied to the slot discovery as done in the block-slot attention which makes the optimization simpler. However inferring the blocks from slots is not trivial and we discuss it in following section. The factorization in Eq. 5 suggests a sequential inference procedure: given an image X, we first infer the object slots S using a slot-based object discovery method, and then infer the attribute blocks C conditioned on the inferred slots. In our implementation, we use Slot Attention for the first stage and introduce Block Attention for the second stage: X→Slot Attention→Block Attention.X Slot AttentionS Block AttentionC. (6) The block inference module should therefore infer the blocks using the slots as input. Note that the slot representation is additive in terms of transformed block representations. 4.3.1 Block inference using block attention Figure 2: Block Attention 1: Input: sis_i 2: Initialize c¯i∼(μC,ΣC) c_i ( _C, _C) 3: cj,i=c¯j,ic_j,i= c_j,i ∀j∈1,…,M∀ j∈\1,…,M\ 4: vk,i=(si⊤uk)ukv_k,i=(s_i u_k)u_k ∀k∈1,…,Np∀ k∈\1,…,N_p\ 5: for t=1t=1 to TbT_b do 6: cj,i=LN(cj,i)c_j,i= LN(c_j,i) ∀j∈1,…,M∀ j∈\1,…,M\ 7: Ai=Softmax(k(i)q(i)⊤dblock)A_i= Softmax ( k(V_i)q(c_i) d_block ) 8: Ai=Ai/Ai.sum(d=’input’)A_i=A_i/A_i. sum( d='input') 9: Ui=AiTv(i)U_i=A_i^Tv(V_i) 10: for j∈1,…,Mj∈\1,…,M\ do 11: uj,i=GRUj(st=cj,i,in=uj,i)u_j,i= GRU_j( st=c_j,i, in=u_j,i) 12: uj,i=uj,i+MLPj(uj,i)u_j,i=u_j,i+ MLP_j(u_j,i) 13: cj,i=Softmax(uj,iℳj⊤dblock)ℳjc_j,i= Softmax ( u_j,iM_j d_block ) M_j 14: end for 15: end for 16: return ic_i We propose a iterative refinement based inference algorithm for inferring blocks from slots. Blocks are refined with the attention between projections of the slots along fixed learnable directions and the blocks inferred so far. The attention is normalized such that the blocks compete to attend for a particular projection of the slot. Specifically, let u1,,..uNpu_1,,..u_N_p denote the NpN_p learnable vectors representing the latent space of projections where each uk∈ℝdslotu_k ^d_slot. Let the projections of the slot sis_i onto this space be denoted as iT=[v1,i,…,vNp,i]V_i^T=[v_1,i,...,v_N_p,i] where vk,i=(siTuk)ukv_k,i=(s_i^Tu_k)u_k. Given these projection vectors represented in the latent space, we infer the blocks corresponding to a slot sis_i by iteratively refining initial block set ¯ c_i with the attention on to these projection vectors. The attention is made competitive by normalizing along block axis (Line 7, Algorithm 2). In line 8, we normalize the attention matrix again along input (project vector) axis, normalization across ’input’ (projection vectors) dimension. UiU_i (line 9) denotes the update vector computed as a result of the attention mechanism. The updates corresponding to particular blocks cj,ic_j,i denotes as uj,i∈ℝdblocku_j,i ^d_block are used to refine the blocks from previous iterations with the GRU and MLP shown in the algorithm. Finally at the end of each refinement step, the blocks are also replaced as linear sum of the prototypes similar to block slot attention Singh et al. (2023) (line 13). Figure 3: Overview of the inference: The image is encoded into spatial features E, from which slots S are inferred via slot attention. For each slot, projected features v are used to iteratively refine M attribute blocks through competitive attention, with each block constrained to a linear combination of shared concept vectors. The resulting blocks serve as disentangled attribute representations. 4.4 Learning and Implementation The parameters θ and ϕφ can be learned by maximizing the evidence lower bound (ELBO) of the data log-likelihood. For the proposed generative model, the ELBO is given by: For the full derivation, please refer to Appendix A.2. logpθ(X)≥ p_θ(X)≥ qϕS(|X)[logpθX(X|)]⏟Reconstruction−DKL(qϕS(|X)∥pθS(|))⏟Slot regularizer E_q_ _S(S|X) [ p_ _X(X|S) ]_Reconstruction- D_KL\! (q_ _S(S|X) \|p_ _S(S|C) )_ [rgb]1,0,0Slot regularizer (7) −∑i=1NqϕS(|X)[DKL(qϕC(i|si)∥pθC(i))]⏟Block regularizer. - _i=1^NE_q_ _S(S|X) [D_KL\! (q_ _C(c_i|s_i) \|p_ _C(c_i) ) ]_ [rgb]0,0,1Block regularizer. The first term corresponds to the reconstruction objective, which measures the reconstruction error between the input images and the images reconstructed from the inferred slots. The second term is the slot regularizer, which encourages the inferred slots to be close to the linear combination of the transformed blocks. The third term is the block regularizer, which penalizes deviations of the inferred block distributions from their GMM priors. In our implementation, we optimize the image reconstruction loss and the slot regularization loss explicitly, while the block regularizer is enforced implicitly through the prototype memory. Furthermore, when the mixture assignment probabilities are inferred using cosine similarity, the block projection in the prototype space (Line 13, Algorithm 2) is equivalent to explicitly setting the block regularization loss to zero. See Appendix A.3 for the proof. Thus, we optimize the following loss: ℒtotal=ℒimg−recon+λℒslot−reg.L_total=L_img-recon+λ [rgb]1,0,0L_slot-reg [rgb]0,0,0. (8) We implement the slot-specific inference network qϕSq_ _S using Slot Attention and the block-specific inference network qϕCq_ _C using Block Attention (Section 4.3.1). The first loss is the image reconstruction loss. Depending on the decoder architecture, this loss can be implemented as an MSE loss for CNN-based decoders, a dVAE prediction loss for transformer-based decoders, a denoising loss for diffusion-based decoders, or a feature reconstruction loss. The second loss is the slot regularization loss, defined as ℒslot−reg=∑i=1N‖si−s^i‖L_slot-reg= _i=1^N \|s_i- s_i \| where sis_i denotes the slots inferred by Slot Attention and s^i s_i denotes the slots generated from the inferred blocks, given by s^i=∑j=1Mfj(cj,i)+b. s_i= _j=1^Mf_j(c_j,i)+b. We train the model in two phases. In the first phase, we train only the Slot Attention module using the image reconstruction loss. In the second phase, we introduce the block-specific modules and train the full model using the losses described above. For the slot-specific reconstruction loss, we found it useful to also use slots generated from the inferred blocks. Specifically, in each mini-batch, we use half of the slots inferred by Slot Attention and half of the slots generated from the inferred blocks. In 2nd phase, instead of sampling the initial slots from a normal distribution, we initialize them using linear combination of the transformed initial blocks. Specifically, the slot sis_i in Slot Attention is initialized as si=∑j=1mfj(c¯j,i).s_i= _j=1^mf_j( c_j,i). Line 2 of Alg. 2 for block initialization procedure. 5 Experiments Our experiments try to answer the following questions: (1) Are slot representations linear in terms of the object factors? (2) Is the proposed attention-based inference method capable of recovering objects and factors of objects? (3) How much is the model robust to its training pipeline and its architecture decisions? 5.1 Setup Methods: We evaluated various slot based methods for the validating the linearity in their representations. This includes the 1. Slot attention Locatello et al. (2020b) which uses SBD decoder Watters et al. (2019). 2. SLATE Singh et al. (2022a) which uses the transformer based decoder. We found that modifying SLATE with difference slot initialization mechanism and a small module before decoder improves the SLATE performance. Please refer to Appendix C.1. We name this baseline as SLATE+. 3. Slot-Diffusion Wu et al. (2023b) which uses the diffusion decoder. 4. CODA-base Nguyen et al. (2026) which uses the denoising diffusion loss from pretrained diffusion. We do not include the contrastive alignment loss from the original work. For evaluating the unsupervised object and attribute discovery we use Slot attention, SLATE+ and a block-slot attention based baseline SysBinder Singh et al. (2023). For our proposed attention based method LinSlot we use SLATE+ as slot base which we train in the first phase. To see the effect of the attention based inference method alone we also include our attention based sequential inference method with block decoder instead of slot decoder. We name it as LinSlotB. LinSlotB is trained end to end without slot regularizer. Datasets: We use three image benchmarks derived from Johnson et al. (2016); Karazija et al. (2021), provided by Singh et al. (2023) for unsupervised object and attribute discovery. These datasets consist of the multi object images with 128×128128× 128 resolution. The objects in these datasets are characterized by set of categorical and continuous attributes. CLEVR-Easy, CLEVR-Hard and CLEVR-Tex. Additionally for slot linearity validation we further use MOVi-C Greff et al. (2022) which The frames from video as used as images. We generate the paired version of the datasets where each datapoint is 4 images. The first two images are same except one object is different which have a single attribute different . The remaining two images are also same have all objects same except one object which are different in single attribute same as the attribute in first pair and the values of the changed attribute matches that from the first pair. For further details please refer to B.2 Metrics: We evaluate the linearity of slot representation by cosine similarity between the two estimations of the delta directions 4.1. That is subtraction of slot pair differing in only one attribute should be similar to the subtraction of different slot pair differing in same one attribute with similar values. We evaluate the attribute disentanglement quality of representation using the standard DCI metric Eastwood and Williams (2018), which measures Disentanglement, Completeness, and Informativeness. Additionally, we report the Foreground Adjusted Rand Index (FG-ARI) to quantify object discovery. FG-ARI measures the clustering agreement between predicted slots and ground-truth masks exclusively on foreground objects, providing as assessment of segmentation quality. Figure 4: Object Manipulation via delta directions: Delta direction is estimated by subtracting slot of object indicated from first image from slot of the object indicated in second image. This delta direction is then added to the object indicated in the third image. The fourth image is decoded by slots manipulated slots from third image. First three images are reconstruction. Table 1: Cosine similarity between two estimations of the same delta direction. CLEVR Easy CLEVR Hard CLEVR-Tex MOVi-C Method Slot Attn. SLATE+ Slot Attn. SLATE+ SLATE+ Slot Diff. CODA Same δ 0.6610 0.7465 0.6024 0.6511 0.5618 0.6356 0.2230 Different δ 0.2232 0.1647 0.1898 0.1493 0.1498 0.1719 0.1020 5.2 Linearity in Slots As mentioned above, the presence of linear structure in the slot space implies the existence of global delta directions 4.1. We test this by measuring the cosine similarity between two estimates of the same delta direction obtained from different object slots in two different images. For example, for the color factor, we compare δcolor[red→blue] _color[red ] estimated from two different object slots belonging to two different images. As a baseline, we measure the cosine similarity between delta directions corresponding to different factors. Table 1 shows the results across four datasets and four different methods. The cosine similarity between two estimates of the same delta direction (first row) is consistently higher than that between delta directions from different factors. This trend holds across all slot methods, including those using pretrained diffusion decoders, and across all datasets. These results suggest the existence of global delta directions and, consequently, linearity in the slot space, which is inherent to slot attention rather than specific to a particular dataset or model. On MOVi-C, CODA exhibits lower absolute values, which we attribute to errors in slot inference due to over- or under-segmentation. We also manipulate slots using these delta directions, analogous to word-embedding arithmetic, where r(king)−r(boy)+r(girl)∼r(queen)r(king)-r(boy)+r(girl) r(queen). Specifically, we modify a slot by adding a delta direction and decode it using the slot decoder to generate the manipulated image. Figure 4 shows the results. Changing a slot by adding a delta direction consistently changes the corresponding object factor in a predictable manner. This behavior holds from the weaker SBD slot decoder to the strongest diffusion-based decoder, suggesting that linearity is fundamental to slot representations. Please refer to Appendix C.2 for additional results showing tracing along the principal axes of delta directions. 5.3 Object and Factor Discovery Figure 5: Quantitative Results: Comparison of DCI and FG-ARI scores for our model and baselines across all datasets. Figure 6: Qualitative Results: Comparison of attribute swapping results between our model and SysBinder. Blue arrows indicate the objects participating in the swaps. Recon. denotes the image reconstructed by the model. Fig. 5 shows the DCI and FG-ARI scores for all the methods across all the datasets. Our attention-based methods with the slot decoder, LinSlot, and the block decoder, LinSlotB, perform better on the CLEVR-Easy and CLEVR-Hard datasets than all the baselines on the D, C, and FG-ARI scores. On CLEVR-Tex, LinSlot is on par with the best method on the D and C scores, whereas its FG-ARI score is comparable to the best number, which is achieved by LinSlotB. This highlights the effectiveness of our approach, which does not require specialized decoders or the extensive hyperparameter tuning required by block-slot attention. The I score is lower than the best-performing method on the CLEVR-Hard and CLEVR-Tex datasets. However, when changing the decoder to the block decoder, as in LinSlotB, the I scores match the best-performing values, suggesting the role of the decoder in the I score. The variance of both our methods is lower than that of the other baselines, suggesting lower sensitivity to the random seed. This suggests that our identifiability analysis is effective. Additionally, we perform a swapping experiment in which we swap the blocks between two objects from the same image and decode the swapped representations to see whether the factors change in the intended way. Fig. 6 shows the results of both our methods along with SysBinder on all three datasets using their best seeds. We observe that on CLEVR-Easy, all methods perform the swapping faithfully. On CLEVR-Hard, we observe that even the best seed of SysBinder struggles with the size factor. On CLEVR-Tex, however, we observe that all the methods perform the swapping faithfully. 5.4 Ablations Effect of Training Choices: Our method, LinSlot, is trained in two phases and uses both the slots generated from the inferred blocks and the slots inferred directly from the images for decoding. We ablate the impact of these training choices, as they are not trivial consequences of the generative model. The results are shown in Table 3. Removing either choice substantially affects the D and C scores, whereas removing the two-phase training and using slots inferred directly from the images leads to training collapse. GRU-MLP Sensitivity: We use a GRU and MLP combination in the refinement loop of the blocks. This choice was inspired by Block-Slot Attention Singh et al. (2023). We ablate these design choices; the results are shown in Table 3. We note that LinSlot is not sensitive to the choice of GRU and MLP, unlike Block-Slot Attention, highlighting robustness of our model. Table 2: Effect of Training Choices 2-Phase Both D C I FG-ARI ✓ ✓ 0.9686 0.7322 0.9621 0.9122 × ✓ 0.8466 0.5994 0.9487 0.9055 ✓ × —– —– —– —– × × 0.7592 0.6104 0.9368 0.9307 Table 3: GRU-MLP Sensitivity GRU MLP D C I FG-ARI ✓ ✓ 0.9686 0.7322 0.9621 0.9122 × ✓ 0.9807 0.7322 0.9634 0.9113 ✓ × 0.9710 0.7356 0.9619 0.9158 × × 0.9767 0.7468 0.9656 0.9189 6 Conclusion and Limitation In this work, we presented a generative framework for unsupervised object and factor discovery that exploits the linear additive structure of factors in slot representations. To the best of our knowledge, we are the first to demonstrate the Linear Representation Hypothesis (LRH) in slot space. We further proposed a sequential inference framework and derived its learning objectives from ELBO optimization. Experiments across datasets and slot-based methods provide consistent evidence for linear structure in slot representations, while our attention-based block inference effectively discovers both objects and their factors. Our model does not explicitly represent background or global properties such as lighting and style, which are treated as sources of uncertainty. Future work includes end-to-end identifiability, extension to video, and explicit modeling of global properties and object interactions. 7 AI use statement In this work, we used generative AI tools for grammar and punctuation refining of text. We have not used generative AI tools for creating ideas, writing core code of the model, writing proofs. Additionally, we used generative AI tools for creating plot code. We have reviewed all AI-assisted work. We verified the grammar and graph generated. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. References Ahuja et al. (2022) K. Ahuja, J. Hartford, and Y. Bengio Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), Cited by: §2. Akan and Yemez (2025a) A. K. Akan and Y. Yemez Slot-guided adaptation of pre-trained diffusion models for object-centric learning and compositional generation. In International Conference on Learning Representations, Cited by: §1. Akan and Yemez (2025b) K. Akan and Y. Yemez Slot-guided adaptation of pre-trained diffusion models for object-centric learning and compositional generation. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2. Bengio et al. (2013) Y. Bengio, A. Courville, and P. Vincent Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (8), p. 1798–1828. External Links: Document Cited by: §1. Bhalla et al. (2026) U. Bhalla, T. Fel, C. Rager, S. Feucht, T. Haklay, D. Wurgaft, S. Boppana, M. Kowal, J. Merullo, A. Geiger, and E. S. Lubana Do sparse autoencoders capture concept manifolds?. In Third Conference on Language Modeling, External Links: Link Cited by: §2. Burgess et al. (2019) C. P. Burgess, L. Matthey, N. Watters, R. Kabra, I. Higgins, M. M. Botvinick, and A. Lerchner MONet: unsupervised scene decomposition and representation. ArXiv abs/1901.11390. Cited by: §2. Caron et al. (2021) M. Caron, H. Touvron, I. Misra, H. J’egou, J. Mairal, P. Bojanowski, and A. Joulin Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), p. 9630–9640. External Links: Link Cited by: §2. Chen et al. (2018) R. T. Q. Chen, X. Li, R. Grosse, and D. Duvenaud Isolating sources of disentanglement in vaes. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, Red Hook, NY, USA, p. 2615–2625. Cited by: §2. Dehaene et al. (2022) S. Dehaene, F. Al Roumi, Y. Lakretz, S. Planton, and M. Sablé-Meyer Symbols and mental programs: a hypothesis about human singularity. Trends in Cognitive Sciences 26 (9), p. 751–766. External Links: ISSN 1364-6613, Document, Link Cited by: §1. Didolkar et al. (2025) A. R. Didolkar, A. Zadaianchuk, R. Awal, M. Seitzer, E. Gavves, and A. Agrawal CTRL-o: language-controllable object-centric visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1. Dilokthanakul et al. (2016) N. Dilokthanakul, P. A. M. Mediano, M. Garnelo, M. J. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan Deep unsupervised clustering with gaussian mixture variational autoencoders. ArXiv abs/1611.02648. External Links: Link Cited by: §A.3. Eastwood and Williams (2018) C. Eastwood and C. K. I. Williams A framework for the quantitative evaluation of disentangled representations. In International Conference on Learning Representations, External Links: Link Cited by: §5.1. Elsayed et al. (2022) G. F. Elsayed, A. Mahendran, S. van Steenkiste, K. Greff, M. C. Mozer, and T. Kipf SAVi++: towards end-to-end object-centric learning from real-world videos. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), Cited by: §2. Engelcke et al. (2021) M. Engelcke, O. P. Jones, and I. Posner GENESIS-v2: inferring unordered object representations without iterative refinement. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan (Eds.), Cited by: §2. Engelcke et al. (2020) M. Engelcke, A. R. Kosiorek, O. P. Jones, and I. Posner GENESIS: generative scene inference and sampling with object-centric latent representations. In International Conference on Learning Representations, Cited by: §2. Eslami et al. (2016) S. M. A. Eslami, N. Heess, T. Weber, Y. Tassa, D. Szepesvari, k. kavukcuoglu, and G. E. Hinton Attend, infer, repeat: fast scene understanding with generative models. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29, p. . Cited by: §2. Fel et al. (2026a) T. Fel, M. Kowal, M. Jacobs, D. Hazra, U. Bhalla, L. Sharkey, L. Bushnaq, S. Grant, T. Haklay, T. Icard, C. Rager, M. Pearce, D. Wurgaft, A. Swann, F. Doshi, S. Boppana, C. Tigges, N. Cammarata, T. Serre, V. Shyam, O. Lewis, T. McGrath, J. Merullo, E. S. Lubana, and A. Geiger Structuring sparsity: block-sparse featurizers capture visual concept manifolds. External Links: 2606.25234, Link Cited by: §2. Fel et al. (2026b) T. Fel, B. Wang, M. A. Lepori, M. Kowal, A. Lee, R. Balestriero, S. Joseph, E. S. Lubana, T. Konkle, D. E. Ba, and M. Wattenberg Into the rabbit hull: from task-relevant concepts in DINO to minkowski geometry. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2. Fodor and Pylyshyn (1988) J. A. Fodor and Z. W. Pylyshyn Connectionism and cognitive architecture: a critical analysis. Cognition 28 (1), p. 3–71. External Links: ISSN 0010-0277, Document Cited by: §1. Gao et al. (2025) L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2. Goyal and Bengio (2022) A. Goyal and Y. Bengio Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 478, p. . External Links: Document Cited by: §1. Greff et al. (2022) K. Greff, F. Belletti, L. Beyer, C. Doersch, Y. Du, D. Duckworth, D. J. Fleet, D. Gnanapragasam, F. Golemo, C. Herrmann, T. Kipf, A. Kundu, D. Lagun, I. Laradji, H. (. Liu, H. Meyer, Y. Miao, D. Nowrouzezahrai, C. Oztireli, E. Pot, N. Radwan, D. Rebain, S. Sabour, M. S. M. Sajjadi, M. Sela, V. Sitzmann, A. Stone, D. Sun, S. Vora, Z. Wang, T. Wu, K. M. Yi, F. Zhong, and A. Tagliasacchi Kubric: a scalable dataset generator. Cited by: §B.2, §5.1. Greff et al. (2019) K. Greff, R. L. Kaufman, R. Kabra, N. Watters, C. P. Burgess, D. Zoran, L. Matthey, M. M. Botvinick, and A. Lerchner Multi-object representation learning with iterative variational inference. ArXiv abs/1903.00450. Cited by: §2. Greff et al. (2017) K. Greff, S. van Steenkiste, and J. Schmidhuber Neural expectation maximization. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30, p. . Cited by: §2. Greff et al. (2020) K. Greff, S. van Steenkiste, and J. Schmidhuber On the binding problem in artificial neural networks. External Links: 2012.05208, Link Cited by: §1. Higgins et al. (2017) I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner Beta-VAE: learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, Cited by: §2. Jiang et al. (2023) J. Jiang, F. Deng, G. Singh, and S. Ahn Object-centric slot diffusion. In Advances in Neural Information Processing Systems, Vol. 36, p. 8563–8601. Cited by: §2. Jiang et al. (2024) Y. Jiang, G. Rajendran, P. K. Ravikumar, B. Aragam, and V. Veitch On the origins of linear representations in large language models. In Forty-first International Conference on Machine Learning, External Links: Link Cited by: §2. Jiang et al. (2016) Z. Jiang, Y. Zheng, H. Tan, B. Tang, and H. Zhou Variational deep embedding: an unsupervised and generative approach to clustering. In International Joint Conference on Artificial Intelligence, External Links: Link Cited by: §A.3. Jiang* et al. (2020) J. Jiang*, S. Janghorbani*, G. D. Melo, and S. Ahn SCALOR: generative world models with scalable object representations. In International Conference on Learning Representations, Cited by: §2. Johnson et al. (2016) J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. B. Girshick CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 1988–1997. Cited by: §B.2, §5.1. Jung et al. (2025) W. Jung, D. H. Lee, and S. Hong Disentangled representation learning via modular compositional bias. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §2. Karazija et al. (2021) L. Karazija, I. Laina, and C. Rupprecht ClevrTex: a texture-rich benchmark for unsupervised multi-object segmentation. ArXiv abs/2111.10265. External Links: Link Cited by: §B.2, §5.1. Khemakhem et al. (2019) I. Khemakhem, D. P. Kingma, and A. Hyvärinen Variational autoencoders and nonlinear ica: a unifying framework. In International Conference on Artificial Intelligence and Statistics, External Links: Link Cited by: §2. Kim and Mnih (2018) H. Kim and A. Mnih Disentangling by factorising. In Proceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause (Eds.), Proceedings of Machine Learning Research, Vol. 80, p. 2649–2658. Cited by: §2. Kingma and Welling (2014) D. P. Kingma and M. Welling Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, External Links: http://arxiv.org/abs/1312.6114v10 Cited by: §2. Kipf et al. (2022) T. Kipf, G. F. Elsayed, A. Mahendran, A. Stone, S. Sabour, G. Heigold, R. Jonschkowski, A. Dosovitskiy, and K. Greff Conditional object-centric learning from video. In International Conference on Learning Representations, Cited by: §2. Kivva et al. (2022) B. Kivva, G. Rajendran, P. Ravikumar, and B. Aragam Identifiability of deep generative models without auxiliary information. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. External Links: ISBN 9781713871088 Cited by: §A.1, §4.2. Kosiorek et al. (2018) A. Kosiorek, H. Kim, Y. W. Teh, and I. Posner Sequential attend, infer, repeat: generative modelling of moving objects. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31, p. . Cited by: §2. Kossen et al. (2020) J. Kossen, K. Stelzner, M. Hussing, C. Voelcker, and K. Kersting Structured object-aware physics prediction for video modeling and planning. In International Conference on Learning Representations, Cited by: §2. Lake et al. (2017) B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman Building machines that learn and think like people. Behavioral and Brain Sciences 40. External Links: Document Cited by: §1. Lin et al. (2020) Z. Lin, Y. Wu, S. Peri, B. Fu, J. Jiang, and S. Ahn Improving generative imagination in object-centric world models. In International Conference on Machine Learning, Cited by: §2. Liu et al. (2025) H. Liu, R. Zhao, H. Chen, and J. Pajarinen MetaSlot: break through the fixed number of slots in object-centric learning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §2. Locatello et al. (2018) F. Locatello, S. Bauer, M. Lucic, S. Gelly, B. Scholkopf, and O. Bachem Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning, External Links: Link Cited by: §2, §2. Locatello et al. (2020a) F. Locatello, B. Poole, G. Rätsch, B. Schölkopf, O. Bachem, and M. Tschannen Weakly-supervised disentanglement without compromises. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. Cited by: §2. Locatello et al. (2020b) F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, and T. Kipf Object-centric learning with slot attention. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 11525–11538. Cited by: §1, §1, §2, §3, §5.1. Mikolov et al. (2013) T. Mikolov, W. Yih, and G. Zweig Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, L. Vanderwende, H. Daumé I, and K. Kirchhoff (Eds.), Atlanta, Georgia, p. 746–751. External Links: Link Cited by: §2. Nguyen et al. (2026) B. Nguyen, Y. Takida, N. Murata, C. Lai, T. Uesaka, S. Ermon, and Y. Mitsufuji Improved object-centric diffusion learning with registers and contrastive alignment. External Links: 2601.01224, Link Cited by: §2, §5.1. Park et al. (2025) K. Park, Y. J. Choe, Y. Jiang, and V. Veitch The geometry of categorical and hierarchical concepts in large language models. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 76441–76463. External Links: Link Cited by: §2. Park et al. (2024a) K. Park, Y. J. Choe, and V. Veitch The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. Cited by: §2. Park et al. (2024b) K. Park, Y. J. Choe, and V. Veitch The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), External Links: Link Cited by: §1. Seitzer et al. (2023) M. Seitzer, M. Horn, A. Zadaianchuk, D. Zietlow, T. Xiao, C. Simon-Gabriel, T. He, Z. Zhang, B. Schölkopf, T. Brox, and F. Locatello Bridging the gap to real-world object-centric learning. In The Eleventh International Conference on Learning Representations, Cited by: §2. Shu et al. (2020) R. Shu, Y. Chen, A. Kumar, S. Ermon, and B. Poole Weakly supervised disentanglement with guarantees. In International Conference on Learning Representations, External Links: Link Cited by: §2. Singh et al. (2022a) G. Singh, F. Deng, and S. Ahn Illiterate DALL-e learns to compose. In International Conference on Learning Representations, Cited by: §C.1, §2, §5.1. Singh et al. (2023) G. Singh, Y. Kim, and S. Ahn Neural systematic binder. In The Eleventh International Conference on Learning Representations, Cited by: §B.2, §1, §1, §4.3.1, §5.1, §5.1, §5.4. Singh et al. (2022b) G. Singh, Y. Wu, and S. Ahn Simple unsupervised object-centric learning for complex and naturalistic videos. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 18181–18196. Cited by: §2. Singh et al. (2025) K. Singh, S. Scahub-Meyer, and S. Roth GLASS: guided latent slot diffusion for object-centric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2. Stammer et al. (2024) W. Stammer, A. Wüst, D. Steinmann, and K. Kersting Neural concept binder. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §1. Stanic and Schmidhuber (2019) A. Stanic and J. Schmidhuber R-sqair: relational sequential attend, infer, repeat. ArXiv abs/1910.05231. External Links: Link Cited by: §2. Stelzner et al. (2019) K. Stelzner, R. Peharz, and K. Kersting Faster attend-infer-repeat with tractable probabilistic models. In Proceedings of the 36th International Conference on Machine Learning, p. 5966–5975. External Links: Link Cited by: §2. Trager et al. (2023) M. Trager, P. Perera, L. Zancato, A. Achille, P. Bhatia, and S. Soatto Linear spaces of meanings: compositional structures in vision-language models. p. 15349–15358. External Links: Document Cited by: §2. Wang et al. (2023) Z. Wang, L. Gui, J. Negrea, and V. Veitch Concept algebra for score-based conditional model. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, External Links: Link Cited by: §2. Watters et al. (2019) N. Watters, L. Matthey, C. P. Burgess, and A. Lerchner Spatial broadcast decoder: a simple architecture for disentangled representations in VAEs. Cited by: §5.1. Whitehead (1927) A.N. Whitehead Symbolism, its meaning and effect. Page-Barbour lectures, Macmillan. External Links: LCCN 27023396 Cited by: §1. Wu et al. (2023a) Z. Wu, N. Dvornik, K. Greff, T. Kipf, and A. Garg SlotFormer: unsupervised visual dynamics simulation with object-centric models. In The Eleventh International Conference on Learning Representations, Cited by: §2. Wu et al. (2023b) Z. Wu, J. Hu, W. Lu, I. Gilitschenski, and A. Garg SlotDiffusion: object-centric generative modeling with diffusion models. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 50932–50958. Cited by: §2, §5.1. Yang et al. (2022) T. Yang, Y. Wang, Y. Lu, and N. Zheng Visual concepts tokenization. ArXiv abs/2205.10093. External Links: Link Cited by: §2. Yang et al. (2023) T. Yang, Y. Wang, Y. Lu, and N. Zheng DisDiff: unsupervised disentanglement of diffusion probabilistic models. In Thirty-seventh Conference on Neural Information Processing Systems, External Links: Link Cited by: §2. Zadaianchuk et al. (2023) A. Zadaianchuk, M. Seitzer, and G. Martius Object-centric learning for real-world videos by predicting temporal feature similarities. In Thirty-seventh Conference on Neural Information Processing Systems, Cited by: §2. Appendix A Theoretical Proofs A.1 Proof of identifiability theorem Dropping the slot index for simplicity, and letting c denote the concatenated factor vector, the generative model for a slot can be written as: s=f()+ϵwhereϵ∼(,ΣS)s=f(c)+ε ε (0, _S) (9) Where f()=∑i=1Mfi(ci)f(c)= _i=1^Mf_i(c_i) is also a piecewise affine function. The distribution of the concatenated vector c is a joint GMM: ∼∑k=1Kπ¯k(μ¯k,Σ¯k)c _k=1^K π_kN( μ_k, _k) (10) where the total number of components is K=∏i=1MKiK= _i=1^MK_i. Here, μ¯k μ_k and Σ¯k _k are formed by concatenating the corresponding component means μk,i _k,i and block-diagonalizing the covariances Σk,i _k,i across all factors i, and π¯k π_k is the product of the relevant mixing weights πi,k _i,k. According to the theorem established by Kivva et al. (2022), the latent variable c in this linear Gaussian model is identifiable up to permutation, scaling, and translation. Suppose we have two trained models that yield the identical marginal distribution for s: s s ∼(f(),Σ) (f(c), ) wherec∼∑k=1Kπ¯k(μ¯k,Σ¯k) c _k=1^K π_kN( μ_k, _k) (11) s s ∼(f′(c’),Σ′) (f (c'), ) wherec′∼∑k=1Kπ¯k′(μ¯k′,Σ¯k′) c _k=1^K π _kN( μ _k, _k) (12) Identifiability implies that the latent variables from the two models are related by: c=ΠDc′+bc= Dc +b (13) where Π is a permutation matrix, D is a diagonal scaling matrix, and b is a translation vector. We now argue that Π must be a block-permutation matrix; that is, it cannot distribute the dimensions of a single factor vector c′uc u from the second model across two different factors cic^i and cvc^v in the first model. Assume, for the sake of contradiction, that Π does split an factor. Without loss of generality, suppose a subset of dimensions from c′uc u is assigned to cic^i, and the remaining dimensions of c′uc u are assigned to a different factor cvc^v. Conversely, this means the factor cic^i in the first model is composed of scaled, permuted, and translated dimensions sourced from at least two independent factors in the second model (e.g., c′uc u and c′wc w). Let these source factors have KuK_u and KwK_w mixture components, respectively. Intuitively, combining dimensions from different independent factors necessitates a multiplicative expansion in the number of mixture components to properly model the new joint space (i.e., it requires Ku×KwK_u× K_w components). However, our target factor cic^i is strictly defined as a sum over only KiK_i components. Thus, mixing these dimensions would demand significantly more underlying components than the model’s structural capacity allows. Formally, because the source factors are independent, concatenating them forms a joint GMM with KuKwK_uK_w components: ci∼∑l=1KuKwπl′(μl′,Σl′)c^i _l=1^K_uK_wπ _lN(μ _l, _l ) (14) However, by our initial definition, cic^i itself is drawn from a GMM with exactly KiK_i components: ci∼∑l=1Kiπi,l(μi,l,Σi,l)c^i _l=1^K_i _i,lN( _i,l, _i,l) (15) Equating these two distributions yields: ∑l=1Kiπi,l(μi,l,Σi,l)≡∑l=1KuKwπl′(μl′,Σl′) _l=1^K_i _i,lN( _i,l, _i,l)≡ _l=1^K_uK_wπ _lN(μ _l, _l ) (16) This equivalence requires Ki=KuKwK_i=K_uK_w. However, we explicitly assumed that Ki≠KuKwK_i≠ K_uK_w for any triplet of distinct indices (i,u,w)(i,u,w). This is a contradiction. Therefore, Π cannot split factors across different blocks, proving that Π must be block-diagonal. A.2 Derivation of the ELBO Note that the generative process of the data is: pθ(X,,)=pθX(X|S)∏i=1N(pθS(si|ci)∏j=1MpθCj(cj,i))p_θ(X,S,C)=p_ _X(X|S) _i=1^N (p_ _S(s_i|c_i) _j=1^Mp_ _C_j(c_j,i) ) (17) We further define following joint two distributions pθS(|)=∏i=1NpθS(si|i)andpθC()=∏i=1N(∏j=1MpθCj(cj,i))p_ _S(S|C)= _i=1^Np_ _S(s_i|c_i)\ \ \ and\ \ \ \ p_ _C(C)= _i=1^N ( _j=1^Mp_ _C_j(c_j,i) ) (18) Further we also define the following joint distributions from variational posterior as: qϕC(C|S)=∏i=1NqϕC(ci|si)q_ _C(C|S)= _i=1^Nq_ _C(c_i|s_i) (19) The log likelihood of the data can be written as: logpθ(X) p_θ(X) =log(∫S∫CpθX(X|)pθS(|)pθC()) = ( _S _Cp_ _X(X|S)p_ _S(S|C)p_ _C(C)dSdC ) (20) ≥qϕ(S,C|X)[log(pθX(X|)pθS(|)pθC()qϕS(S|X)qϕC(C|S))] _q_φ(S,C|X) [ ( p_ _X(X|S)p_ _S(S|C)p_ _C(C)q_ _S(S|X)q_ _C(C|S) ) ] (21) =qϕS(S|X)[logpθX(X|S)]−DKL(qϕS(S|X)∥pθS(|))⏟slot regularizer =E_q_ _S(S|X) [ p_ _X(X|S) ]- D_KL (q_ _S(S|X)\,\|\,p_ _S(S|C) )_slot regularizer (22) −qϕS(S|X)[DKL(qϕC(|)∥pθC())]⏟block regularizer. - E_q_ _S(S|X) [D_KL (q_ _C(C|S)\,\|\,p_ _C(C) ) ]_block regularizer. (23) A.3 Concept Memory projection as Explicit block regularizer The block regularizer encourages the inferred block distribution to match the GMM prior. This regularizer can be factored at the object level and further decomposed into a sum of regularizers over individual blocks. We then use the result from Dilokthanakul et al. (2016); Jiang et al. (2016) for the KL divergence between a normal distribution and a GMM: DKL(qϕC(|)∥pθC()) D_KL (q_ _C(C|S)\,\|\,p_ _C(C) ) =∑i=1NDKL(qϕC(i|si)∥pθC(i)) = _i=1^ND_KL (q_ _C(c_i|s_i)\,\|\,p_ _C(c_i) ) (24) =∑i=1N∑j=1MDKL(qϕC(cj,i|si)∥pθCj(cj,i)) = _i=1^N _j=1^MD_KL (q_ _C(c_j,i|s_i)\,\|\,p_ _C_j(c_j,i) ) (25) Assuming that qϕC(cj,i|si)q_ _C(c_j,i|s_i) is a normal distribution, each term in the above sum corresponds to the KL divergence between a normal distribution and a GMM. Using Jensen’s inequality, this KL divergence can be upper-bounded by a weighted sum of KL divergences between the inferred distribution and the individual Gaussian components of the GMM. DKL(qϕC(cj,i|si)∥pθCj(cj,i)) D_KL (q_ _C(c_j,i|s_i)\,\|\,p_ _C_j(c_j,i) ) =−qϕC(cj,i|si)[log(pθCj(cj,i)qϕC(cj,i|si))] =-E_q_ _C(c_j,i|s_i) [ ( p_ _C_j(c_j,i)q_ _C(c_j,i|s_i) ) ] (26) =−qϕC(cj,i|si)[log(∑k=1Kjπk,j(cj,i,μk,j,Σk,j)qϕC(cj,i|si))] =-E_q_ _C(c_j,i|s_i) [ ( _k=1^K_j _k,jN(c_j,i; _k,j, _k,j)q_ _C(c_j,i|s_i) ) ] (27) Introducing an auxiliary assignment distribution γk,j(si) _k,j(s_i) satisfying ∑kγk,j(si)=1 _k _k,j(s_i)=1, the above expression can be upper-bounded as: DKL(qϕC(cj,i|si)∥pθCj(cj,i)) D_KL (q_ _C(c_j,i|s_i)\,\|\,p_ _C_j(c_j,i) ) =−qϕC(cj,i|si)[log(∑k=1Kjπk,j(cj,i,μk,j,Σk,j)γk,j(si)γk,j(si)qϕC(cj,i|si))] =-E_q_ _C(c_j,i|s_i) [ ( _k=1^K_j _k,jN(c_j,i; _k,j, _k,j) _k,j(s_i) _k,j(s_i)q_ _C(c_j,i|s_i) ) ] (29) ≤γk,j(si)DKL(qϕC(cj,i|si)||(μk,j,Σk,j)) ≤ _k,j(s_i)D_KL(q_ _C(c_j,i|s_i)||N( _k,j, _k,j)) (30) +∑kKjDKL(γk,j(si)||πk,j) + _k^K_jD_KL( _k,j(s_i)|| _k,j) (31) Using the assumption that the qϕC(cj,i|si)q_ _C(c_j,i|s_i) is normal distribution with mean cj,ic_j,i and additionally that it have identity covariance matrix. Also assuming all the Σk,j _k,j are also identity. Also ignoring the last term then the the best c~j,i c_j,i will be when the KL divergence is minimized. cj,i∗=min∑k=1Kjcj,iγk,j(si)‖cj,i−μk,j‖22c_j,i^*= _c_j,i _k=1^K_j _k,j(s_i)||c_j,i- _k,j||_2^2 (32) Which have the solution that cj,i∗=∑k=1Kjγk,j(si)μk,jc_j,i^*= _k=1^K_j _k,j(s_i) _k,j (33) Thus, the optimal block representation is a weighted combination of the GMM prototype vectors. In particular, if the assignment weights are given by γk,j(si)=exp(cosine-sim(cj,i,μk,j))∑kexp(cosine-sim(cj,i,μk,j)) _k,j(s_i)= ( cosine-sim(c_j,i, _k,j)) _k ( cosine-sim(c_j,i, _k,j)) and the prototype matrix is defined as ℳjT=[μ1,j..μKj,j]M_j^T= bmatrix _1,j\ \ ..\ \ _K_j,j bmatrix then the resulting representation corresponds to a projection of the block onto the prototype space. This establishes the connection between the concept memory projection and the explicit block regularizer. Appendix B Experiment Details B.1 Two Phase training Please refer to fig 7 where we show our two phase training pipeline. In phase one we train the slot attention and decoder on the slot method specific loss. In phase two we add the block specific modules and keep the decoder frozen. Table 4: Hyperparameters of training for CLEVR-Easy, CLEVR-Hard, and CLEVR-Tex datasets. Module Hyperparameter Dataset CLEVR CLEVR CLEVR Easy Hard Tex General Batch Size 64 64 64 Image Size 128 128 128 Training Steps 200K 260K 315K Encoder Learning Rate 0.0001 0.0001 0.0001 Decoder Learning Rate (only used in phase 1) 0.0003 0.0003 0.0003 Block Block Size (dblockd_block) 192 192 192 Refinement # Blocks (M) 8 16 8 # Concepts (KjK_j same across all j) 64 64 64 # Refinement Steps (TbT_b) 3 3 3 # Projections (NpN_p) 192 192 192 Slot Slot Size (dslotd_slot) 192 192 192 Attention # Slots (N) 4 4 6 # Refinement steps (TsT_s) 3 3 3 Figure 7: Training phases of the model. B.2 Additional Details of the datasets We use three image benchmarks derived from Johnson et al. (2016); Karazija et al. (2021), provided by Singh et al. (2023). These datasets consist of the multi object images with 128×128128× 128 resolution. The objects in these datasets are characterized by set of categorical and continuous attributes with details as below. Additionally for slot linearity validation we further use MOVi-C Greff et al. (2022) CLEVR-Easy dataset features scenes with 2-3 objects. Each object have 3 attributes: color with 8 possible choices, shape with 3 possible choices and position which is continuous. CLEVR-Hard, a more challenging dataset with images consisting of 2-3 objects. Each object has 5 attributes: color with 137, shape with 3 and material with 2 possible choices; size comes from fixed range and the position is continuous. CLEVR-Tex dataset consists of textured images where the objects and background both have realistic textures increasing the visual complexity. The images consist of 2-3 objects with each object having 3 attributes. Texture/Material have 57 possible choices, shapes have 4 possible choices and position is continuous. MOVi-C is the originally video dataset of objects rendered on the natural background. The frames from video as used as images. B.3 Creation of the paired Dataset Each datapoint of the paired dataset is a tuple t(l)=(X(l,1),X(l,2),X(l,3),X(l,4))t^(l)=(X^(l,1),X^(l,2),X^(l,3),X^(l,4)) of four images. The images X(l,1)X^(l,1) and X(l,2)X^(l,2) contain the same objects except one; without loss of generality, let it be the first object generated in both images. These two objects differ in exactly one attribute j⋆∈1,…,Mj ∈\1,…,M\, i.e. cj,1(l,1)=cj,1(l,2)c_j,1^(l,1)=c_j,1^(l,2) for all j∈1,…,M∖j⋆j∈\1,…,M\ \j \. Similarly, X(l,3)X^(l,3) and X(l,4)X^(l,4) contain the same objects except the first one, and these first objects differ only in attribute j⋆j : cj,1(l,3)=cj,1(l,4)c_j,1^(l,3)=c_j,1^(l,4) for all j∈1,…,M∖j⋆j∈\1,…,M\ \j \. Moreover, both pairs undergo the same change of attribute j⋆j : cj⋆,1(l,1)=cj⋆,1(l,3)c^(l,1)_j ,1=c^(l,3)_j ,1 and cj⋆,1(l,2)=cj⋆,1(l,4)c^(l,2)_j ,1=c^(l,4)_j ,1, while the first objects of the two pairs differ in at least one other attribute, ∃j≠j⋆:cj,1(l,1)≠cj,1(l,3)∃\,j≠ j :c^(l,1)_j,1≠ c^(l,3)_j,1. Appendix C Additional Results C.1 Slate modifications We modified the original SLATE Singh et al. (2022a) with initializing the slots as linear transformation of 2048 dimensional random vector sampled from learned gaussian rather than from just the learned normal distribution. Additionally we add a learnable linear projector on the slots before passing them to transformer slot decoder which makes 8 different projections of slots. The transformer decoder then uses this 8N8N tokens for decoding same as block decoder does. Refer table 5 which shows comparison on the CLEVR-Easy dataset. Table 5: SLATE vs SLATE+ Results on CLEVR-Easy Method FG-ARI SLATE 0.8166 SLATE+ 0.9369 C.2 Linearity in Slots Table 6: Stable rank and participation ratio of the delta-direction subspace of each factor (mean ± std over seeds). Dataset (model) Factor #Values Stable rank Part. ratio CLEVR-Easy (Slot Attention, dslot=64d_slot=64) Position cont. 3.21±0.383.21± 0.38 5.58±0.535.58± 0.53 Color 8 2.93±0.622.93± 0.62 4.91±1.464.91± 1.46 Shape 3 2.06±0.162.06± 0.16 3.70±0.483.70± 0.48 CLEVR-Hard (SLATE+, dslot=192d_slot=192) Position cont. 3.89±0.053.89± 0.05 8.08±0.048.08± 0.04 Color 137 3.82±0.113.82± 0.11 7.24±0.347.24± 0.34 Shape 3 2.79±0.192.79± 0.19 5.37±0.525.37± 0.52 Material 2 1.83±0.071.83± 0.07 3.18±0.213.18± 0.21 Size cont. 1.82±0.121.82± 0.12 3.12±0.343.12± 0.34 CLEVR-Tex (Slot Diffusion, dslot=192d_slot=192) Position cont. 3.71±0.173.71± 0.17 6.60±0.546.60± 0.54 Material 57 5.43±0.375.43± 0.37 11.12±0.2111.12± 0.21 Shape 4 2.81±0.172.81± 0.17 6.46±0.596.46± 0.59 Figure 8: Steering slots along principal axes of the delta space: We steer slots by adding a scaled multiple α of a principal axis computed from the delta directions associated with a particular factor. The slots corresponding to the objects highlighted by red boxes in the first column are steered, with α increasing from left to right. The resulting images exhibit meaningful and smooth changes along the corresponding factor, consistently across different objects, datasets, and methods, including Slot Attention, SLATE+, and Slot Diffusion. The presence of the delta directions in the slot space supports the linearity of the slot space. However, it does not reveal the structure of the factor manifold. One hypothesis is that the factor manifold is a linear subspace of the slot space itself. The question is then what the inherent dimension of this subspace is and how to find its basis. Assuming the factor manifold is a linear subspace, the delta directions of a specific factor span this subspace. Finding the principal axes of the delta directions then naturally gives a basis for it. Specifically, we estimate the delta directions from all slot pairs in the paired data where factor j is changed. We then center the directions by subtracting the mean delta direction and arrange them as the rows of a matrix Δj _j. Next, we compute the singular value decomposition Δj=UΣVT _j=U V^T. We choose the top ljl_j right singular vectors (the columns of V) as the principal axes, denoted as pj,1,…,pj,lj\p_j,1,…,p_j,l_j\. Here, ljl_j is the floor of the participation ratio PRj=(∑iσi2)2/∑iσi4PR_j= ( _i _i^2 )^2/ _i _i^4, where σi _i are the singular values of Δj _j. We then steer the slot s of an object in an image as s′=s+αpj,is =s+α\,p_j,i for each principal axis pj,ip_j,i. We visualize the resulting changes by decoding the steered slot together with the remaining, unmodified slots. The range of α is set by the minimum and maximum projections of all slots in the dataset onto pj,ip_j,i. Table 6 reports the stable rank and participation ratio of the delta spaces across different slot methods and datasets. The delta spaces consistently have substantially lower effective dimensionality than the full slot space, indicating that changes associated with individual factors occupy low-dimensional directions within the slot representation. We also observe a relationship between the number of possible values of a factor and the dimensionality of its delta space, as measured by both stable rank and participation ratio, with material on CLEVR-Tex being a notable exception. We further examine whether these low-dimensional directions correspond to semantically meaningful variations by steering slots along the principal axes. Figure 8 shows representative results across datasets and methods. Steering along the principal axes consistently changes the corresponding factor while preserving the other properties of the object, and produces smooth transitions across different objects and datasets. This behavior is also consistent across random seeds. Moreover, individual principal axes often correspond to interpretable variations within a factor. For example, the first color principal axis on CLEVR-Hard primarily changes the brightness or darkness of the object color, while the two principal axes associated with position on CLEVR-Tex correspond approximately to horizontal and vertical image position. These results suggest that factor-specific variations are concentrated in low-dimensional, approximately linear subspaces of the slot space, and that the principal axes of these subspaces capture semantically coherent directions. Together, these findings provide empirical support for our hypothesis that factor manifolds are approximately linear in the learned slot space.