Paper deep dive
Annotation-Free Furniture Codes: What They Encode, and How Far They Transfer
Benjamin Friedman
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/14/2026, 7:19:11 AM
Summary
This paper investigates whether a single self-supervised geometric token can replace human-annotated class labels and canonical pose conventions in 3D scene synthesis. Using a Finite Scalar Quantization (FSQ) point-cloud autoencoder trained on 3D-FUTURE furniture, the authors demonstrate that the resulting discrete codes naturally encode both fine-grained categorical identity and yaw orientation without explicit supervision. By manipulating the chamfer distance training target, they show that rotation awareness can be explicitly controlled, trading yaw encoding for improved class recovery. Cross-dataset evaluation on ShapeNet reveals that transferability is shape-dependent: box-like furniture aligns well with the source manifold, while organically-shaped items do not, though target-blind augmentation partially mitigates this gap.
Entities (8)
Relation Signals (7)
Finite Scalar Quantization (FSQ) → implements → AutoEncoder
confidence 95% · A Finite Scalar Quantization (FSQ) point-cloud autoencoder is chamfer-trained on placed 3D-FUTURE furniture...
AutoEncoder → trainedon → 3D-FUTURE
confidence 95% · chamfer-trained on placed 3D-FUTURE furniture with no labels or pose annotations.
Discrete Codes → encodes → Yaw Orientation
confidence 90% · Diagnostic probes recover... yaw (52.7±0.5°) from the codes alone.
Discrete Codes → encodes → Categorical Class Label
confidence 90% · Diagnostic probes recover fine-category (62.6±0.5%), super-category (85.6±1.3%)... from the codes alone.
Chamfer distance → controls → Yaw Orientation encoding
confidence 85% · Swapping the chamfer target from the rotated to the un-rotated point cloud collapses the yaw signal while raising class recovery...
Target-Blind Augmentation → improves → Cross-dataset Alignment
confidence 85% · and a target-blind augmentation partly closes the gap.
Box-like Furniture → transfersto → ShapeNet
confidence 85% · on an unseen dataset (ShapeNet), alignment is category-dependent: box-like furniture transfers, organically-shaped furniture does not...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Layout-based 3D scene synthesizers place each object using two human-annotated channels: a categorical class label and a canonical-pose convention. We ask whether a single self-supervised token derived from object geometry can replace both, and study such tokens directly as a representation, decoupled from any synthesizer. A Finite Scalar Quantization (FSQ) point-cloud autoencoder is chamfer-trained on placed 3D-FUTURE furniture with no labels or pose annotations. Diagnostic probes recover fine-category (62.6 +/- 0.5%), super-category (85.6 +/- 1.3%), and yaw (52.7 +/- 0.5 deg) from the codes alone. Swapping the chamfer target from the rotated to the un-rotated point cloud collapses the yaw signal while raising class recovery, showing the codes' rotation content can be set by the training objective. Scaling across asset libraries needs codes that transfer; on an unseen dataset (ShapeNet), alignment is category-dependent: box-like furniture transfers, organically-shaped furniture does not, and a target-blind augmentation partly closes the gap.
Tags
Links
- Source: https://arxiv.org/abs/2607.10461v1
- Canonical: https://arxiv.org/abs/2607.10461v1
Trouble viewing inline? Open PDF directly →
Full Text
41,764 characters extracted from source content.
Expand or collapse full text
Annotation-Free Furniture Codes: What They Encode, and How Far They Transfer Benjamin Friedman DLR Group Abstract Layout-based 3D scene synthesizers place each object using two human-annotated channels: a categorical class label and a canonical-pose convention. We ask whether a single self-supervised token derived from object geometry can replace both, and study such tokens directly as a representation, decoupled from any synthesizer. A Finite Scalar Quantization (FSQ) point-cloud autoencoder is chamfer-trained on placed 3D-FUTURE furniture with no labels or pose annotations. Diagnostic probes recover fine-category (62.6±0.562.6± 0.5 %), super-category (85.6±1.385.6± 1.3 %), and yaw (52.7±0.5∘52.7± 0.5 ) from the codes alone. Swapping the chamfer target from the rotated to the un-rotated point cloud collapses the yaw signal while raising class recovery, showing the codes’ rotation content can be set by the training objective. Scaling across asset libraries needs codes that transfer; on an unseen dataset (ShapeNet), alignment is category-dependent: box-like furniture transfers, organically-shaped furniture does not, and a target-blind augmentation partly closes the gap. Keywords: self-supervised 3D representations, point-cloud tokenization, finite scalar quantization, 3D scene synthesis, cross-dataset transfer 1 Introduction Layout-based indoor-scene synthesizers, autoregressive (ATISS [3]) and diffusion-based (DiffuScene [4], InstructScene [5]) alike, predict per object a categorical class index and an explicit yaw. Both are supervision channels requiring human work: a class taxonomy someone must define and assign, and a canonical-pose convention under which a predicted angle θ orients different meshes consistently. That convention is subtle, meaningful only if every mesh’s local frame aligns to a shared semantic “front,” and neither the 3D-FUTURE [6] nor 3D-FRONT [7] papers document how it was set. Object geometry, by contrast, is self-evident: point clouds are sampled from meshes with no human in the loop. A single self-supervised geometric token per object, a scene tokenizer standing in for the (class, angle) pair, is an attractive front-end, and one that could scale across asset libraries without the per-library taxonomy work labels demand. But a tokenizer is only as useful as what its tokens encode, a question logically prior to any generator: what a token carries, what determines it, and how far it survives a change of asset library. This paper characterizes the tokens themselves, independent of any downstream generator; the synthesizer is motivating context, not a deliverable here. We train a Finite Scalar Quantization (FSQ) [1] point-cloud autoencoder with a chamfer objective on placed (rotated) furniture point clouds (no class labels, no pose annotations) and study the resulting 500-entry discrete codes. Contributions. • Probe quantification. Small probes [9] recover both super-/fine-category and yaw from the one-hot code, well above random and modal baselines, averaged over n=3n=3 seed-and-split repetitions (Section 5). • Loss-target control (headline). A one-boolean intervention, replacing the chamfer target with the canonical (un-rotated) point cloud, collapses the yaw probe to modal baseline and raises class recovery. This quantifies, through the discrete bottleneck, that the rotation content is set by the supervision target, and shows both rotation-aware and rotation-invariant regimes are reachable from one fixed pipeline (Section 6). Only the rotated-target recipe is annotation-free; the canonical control uses the un-rotated mesh. • Cross-dataset code alignment (headline). Encoding ShapeNet furniture through the frozen autoencoder, cross-dataset alignment is category-dependent: box-like categories transfer, organically-shaped ones do not (Section 7). • Domain-robust augmentation. A generic digitization augmentation (with the target dataset never seen) partially closes the gap at zero within-dataset reconstruction cost (Section 8). We are explicit about scope. Every result here is a property of the codes. We do not train an end-to-end label-free synthesizer, report FID, or study scene placement; those are separate downstream questions. The contribution is a characterization of what a geometry-only shape vocabulary contains and how it behaves under a loss-target change and a dataset shift. 2 Related Work Discrete 3D shape representations. VQ-VAEs [2] have been adapted to 3D in AutoSDF [12], ShapeFormer [13], and 3DILG [14]; MeshGPT [15] learns a triangle vocabulary. FSQ [1] removes the learned codebook (no embedding table, no commitment loss, no EMA) via a fixed per-dimension grid, which is why we adopt it; we include a VQ-VAE comparison (supplementary). Point-BERT [16] and Point-MAE [17] use masked, label-free pretraining on point clouds; we share the geometry-only premise but target a one-token-per-object vocabulary and analyse its contents directly. Rotation in 3D representations. Two families dominate: equivariant architectures (Tensor Field Networks [18], SE(3)-Transformers [19], Vector Neurons [20]) that bake in group structure, and learned canonicalization (Canonical Capsules [21], ConDor [22]); Frame Averaging [23] obtains invariance by averaging any backbone over a frame. We ask a different question: for a fixed non-equivariant pipeline, to what extent is the codes’ rotation-awareness set by the training objective? We isolate it to the loss target (Section 6). Cross-dataset transfer. Whether a learned 3D representation transfers across asset libraries with different tessellation, sampling, and modelling conventions is a practical deployment question. We measure it directly in code space between 3D-FUTURE [6] and ShapeNet [8], and test a standard robustness augmentation (jitter/dropout/voxel-snap) as a domain-generalization [24] lever that never observes the target dataset. 3 Method Figure 1: Pipeline. A point cloud is encoded (PointNet), quantized by FSQ to a single 500500-entry geometric code, and decoded (FoldingNet); the only training signal is chamfer distance to a target. The chamfer target is the one knob we vary: the rotated input (primary recipe) or the canonical, un-rotated point cloud (control, Section 6). No class labels and no canonical-pose annotations are consumed. (The token is “∼9 9-bit” in the nominal sense: 500500 codes ≈8.97≈ 8.97 bits; ∼460 460 are exercised across rotated placements, ≈8.85≈ 8.85 bits effective, and ∼400 400/80%80\% over the canonical models alone; see supplementary.) 3.1 Autoencoder and codebook A PointNet encoder [10] maps an Nin×3N_in×3 input point cloud to a 128128-dim global feature, linearly projected to a dfsq=5d_fsq=5 FSQ latent, tanh -squashed and rounded to per-dimension levels [5,5,5,2,2][5,5,5,2,2] (∏iLi=500 _iL_i=500 codes), projected back to 128128 dims, and decoded by FoldingNet [11] to an Nout×3N_out×3 output cloud (Nin=Nout=512N_in=N_out=512 in all experiments) (Fig 1). FSQ uses the straight-through estimator; there is no learned codebook, commitment loss, or EMA. Total parameters: 193193k. The training signal is geometry only: chamfer distance between input and reconstruction. No class labels and no canonical poses are consumed during training. 3.2 Primary recipe and the loss-target control Each 3D-FRONT placement carries a yaw θ about the vertical axis. Let canP_can be a mesh’s canonical (un-rotated) point cloud and θ=Ry(θ)canP_θ=R_y(θ)P_can its placed version. Primary recipe (rotated-target). Input θP_θ, target θP_θ: to reconstruct the placed cloud the encoder must encode θ. Strictly annotation-free in the chamfer loss. Canonical-target control. Input Ry(Δθ)θR_y( θ)P_θ, target canP_can: the target is rotation-free, so the encoder is asked to produce features that decode to the canonical pose regardless of input rotation. This is the single controlled change (Δθ∈[−180∘,180∘] θ∈[-180 ,180 ]). Architecture, optimizer, and codebook are held constant across the two; only the chamfer target differs. 3.3 Probe protocol To measure what the codes carry we train small probes [9], 2-layer MLPs (hidden 64, dropout 0.1, Adam 3e−33e-3, 30 epochs), on a 500500-dim one-hot of the discrete code. P1 — class. 3D-FUTURE super-category (6 present) and fine-category (27 with ≥30≥ 30 placements). Metrics: top-1/top-5, macro-F1. Baselines: random and majority. P2 — yaw. A (cosθ,sinθ)( θ, θ) regressor (mean/median angular error) and an 88-bin classifier. Baselines: uniform-random (90∘90 ) and modal-yaw (∼80∘ 80 , from the strong axis-aligned prior). A class-conditional variant trains one regressor per super-category to isolate within-class rotation beyond the class-yaw prior. Split: 80/20 stratified by super-category, seed 42, held constant across recipes. We use the taxonomy only at evaluation (probe targets, purity); the model never sees it. 3.4 Cross-dataset alignment protocol We measure whether foreign-dataset geometry lands where native geometry does. For each ShapeNet [8] category we sample surface point clouds, normalize identically to the 3D-FUTURE canonical clouds, encode them at K∈0,90,180,270∘K∈\0,90,180,270\ through the frozen autoencoder, and compute the pre-quantization embedding distance from each ShapeNet cloud to the matching 3D-FUTURE super-category manifold, normalized by the native intra-category spacing. We report the ratio r=dShapeNet→native/dnativer=d_ShapeNet /d_native; r≈1r≈ 1 means foreign geometry is indistinguishable from native geometry, larger r means off-manifold. Both terms are encoded with the same checkpoint. 300300 clouds per category. r is measured on the pre-quantization latent, where distance is defined; the discrete codes we deploy show the same alignment (Section 7): a representation property, not a latent artifact. 4 Setup Furniture from 3D-FUTURE [6] (8,2298,229 seen models, ∼96 96k placements), N=512N=512 points. Adam, lr 10−310^-3, batch 128128, 1515 epochs, cosine annealing; validation chamfer on a held-out 9,6179,617-placement split. Each configuration trains in 77–1515 min on one A100. Cross-dataset clouds are drawn from ShapeNetCore furniture synsets. Code, configs, and per-run results will be released. 5 What the codes encode Class identity. Even with no label seen at training, per-code super-category purity sits well above the 16.7%16.7\% random baseline and rises monotonically with codebook size (a codebook-size sweep is in the supplementary material). We anchor the study at 500 codes (83.7%83.7\% purity), a vocabulary a downstream model could plausibly learn to predict over. Class and yaw, from the codes. The probes (Table 1, “rotated-target” column) recover fine-category at 62.6±0.562.6± 0.5 % (∼4.4× 4.4× majority over 2727 classes) and super-category at 85.6±1.385.6± 1.3 % (∼5× 5× random); yaw follows at 52.7±0.5∘52.7± 0.5 mean angular error (∼27∘ 27 better than modal). The codes carry both channels a synthesizer would otherwise read from annotation. Quantization cost. To calibrate these numbers we probe the continuous 55-dim pre-quantization latent on the same three models: it recovers super-category at 92.1±0.5%92.1± 0.5\,\% and yaw at 45.4±0.3∘45.4± 0.3 , an effective ceiling (the code is a deterministic function of that latent). Against the discrete probe on the same models (Table 1: 85.6%85.6\,\%, 52.7∘52.7 ), quantizing to 500500 codes costs 6.56.5\,p super-category and 7.3∘7.3 yaw: the codes retain most of the recoverable signal, and the discrete-vs-continuous gap, not the absolute number, is the price of a compact token. The yaw error is bimodal. The 52.7∘52.7 mean understates typical accuracy: the median is 10.2∘10.2 and most placements are recovered within 20∘20 , but ∼20% 20\,\% suffer near-180∘180 front/back flips on symmetric furniture that drag the mean up (Fig 2). A geometry-only code recovers yaw near-perfectly where orientation is unambiguous and fails where the shape is symmetric: expected behaviour, not a defect. The flips are front/back code collapse. The bimodality has a concrete mechanism. Encoding each mesh at 1212 yaws, we find the code is often invariant to a 180∘180 turn (code(θ)=code(θ+180∘)code(θ)=code(θ+180 )) and this collapse is category-structured: high for front/back-symmetric furniture (Pier/Stool 0.770.77, Cabinet/Shelf/Desk 0.670.67, Table 0.640.64) and low for clearly-oriented furniture (Sofa 0.140.14, Bed 0.130.13, Chair 0.030.03; symmetric objects also span far fewer distinct codes across the 1212 yaws, ∼5 5 vs ∼8 8). A collapsed code cannot separate the two orientations, so a code-based predictor guesses front/back at chance and errs by ∼180∘ 180 on half of those placements; the mean collapse rate (0.450.45) therefore predicts a ∼0.22 0.22 flip fraction, matching the ∼20% 20\% tail in Fig 2. The flips are geometry, not noise: symmetric objects share one code across their front and back placements. Figure 2: Distribution of per-sample yaw angular error (rotated-target model, one representative seed). Bimodal: a mode near 0∘0 (57%57\,\% of placements within 20∘20 ) and a second near 180∘180 (20%20\,\%, front/back flips on symmetric furniture). The mean (51.8∘51.8 ) is pulled up by the flip tail; the median is 11.5∘11.5 . (This seed; cf. the n=3n=3 52.7∘52.7 mean / 10.2∘10.2 median in Table 1, within noise.) Where the within-class rotation signal is strong. The class-conditional probe (Table 2) shows the rotation signal is strong for canonically-oriented classes (within-Chair 18.9∘18.9 , Sofa 18.6∘18.6 , Bed 25.8∘25.8 ) and weak for Tables and Storage (within ∼7∘ 7 of modal). This could reflect the classes’ yaw distributions rather than their geometry: the oriented classes see more varied training yaws in 3D-FRONT. A matched-yaw control (supplementary), re-probing every class under an identical uniform yaw distribution, confirms the split survives: it is geometric, not a distribution artifact (symmetric furniture maps θ and θ+180∘θ+180 to near-identical clouds, hence codes). The control equalizes the evaluation distribution, not the training one. Table 1: Probe-based quantification, mean ± std over n=3n=3 seed-and-split repetitions. Same architecture/optimizer/codebook; only the chamfer target differs. Class: higher better; yaw error: lower better. Probe / metric Random / baseline rotated-target canonical-target P1 — class identity (code → class) Super-cat top-1 (%) 16.7 / maj. 36.5 85.6±1.385.6± 1.3 89.8±0.189.8± 0.1 Super-cat macro-F1 0.17 0.745±0.0270.745± 0.027 0.808±0.0090.808± 0.009 Fine-cat top-1 (%) 3.7 / maj. 14.2 62.6±0.562.6± 0.5 68.3±0.968.3± 0.9 Fine-cat top-5 (%) 18.5 96.3±0.396.3± 0.3 98.2±0.298.2± 0.2 P2 — rotation (code → yaw, marginal) Mean angular error (deg) 90 / modal 80 52.7±0.552.7± 0.5 79.4±0.479.4± 0.4 Median angular error (deg) — 10.2±3.110.2± 3.1 81.9±0.981.9± 0.9 8-bin yaw top-1 (%) 12.5 / modal 28.8 61.5±0.761.5± 0.7 31.6±0.431.6± 0.4 Table 2: Class-conditional yaw probe: mean angular error (deg), lower better; “Δ modal” = improvement over the class-specific modal-yaw baseline (positive = within-class rotation beyond the class label). Mean over n=3n=3. Super-cat rotated-target canonical-target probe Δ modal probe Δ modal Chair 18.9±3.118.9± 3.1 +71.2+71.2 85.0±0.885.0± 0.8 +5.1+5.1 Bed 25.8±2.325.8± 2.3 +62.6+62.6 78.0±1.778.0± 1.7 +10.4+10.4 Sofa 18.6±1.118.6± 1.1 +72.6+72.6 84.4±1.384.4± 1.3 +6.8+6.8 Table 58.4±1.358.4± 1.3 +6.7+6.7 66.9±1.366.9± 1.3 −1.9-1.9 Storage 78.1±1.978.1± 1.9 +5.4+5.4 83.1±1.183.1± 1.1 +0.5+0.5 Other 54.2±5.154.2± 5.1 −1.2-1.2 55.6±2.655.6± 2.6 −2.6-2.6 6 The loss target controls the rotation encoding The primary recipe’s codes encode rotation (Table 1); where from? Information-theoretically it is nearly forced: the encoder receives the rotated cloud in both regimes, so under a canonical (de-rotated) target any retained orientation is penalised by chamfer, whereas a rotated target rewards it. So the supervision target should govern the rotation content. We quantify this through the discrete bottleneck and show both regimes (rotation-aware and rotation-invariant) are reachable from one fixed pipeline by flipping a single boolean, via the canonical-target control (Section 3.2): architecture, optimizer, codebook, and split are identical; only the chamfer target changes. Two caveats. First, only the rotated-target recipe is annotation-free: the canonical target requires the un-rotated mesh, which presupposes a human-defined canonical-pose convention. The invariant regime is therefore a control, not a second annotation-free recipe. Second, the annotation-free (rotated-target) codes encode yaw entangled with shape (revisited in the discussion). Figure 3: Pre-quantization embeddings of held-out meshes, each encoded at 88 yaws. Top (rotated-target, primary): points trace per-mesh rotation arcs (left, coloured by yaw) that also cluster by identity (right): codes encode shape and rotation. Bottom (canonical-target, control): the arcs collapse to identity-only clusters: the same encoder, same data, same held-out meshes; only the chamfer target differs. The rotation encoding is set by the loss target. The yaw encoding collapses. Under the canonical target the yaw probe drops to modal baseline (mean error 52.7∘→79.4∘52.7 \!→\!79.4 , 88-bin top-1 61.5%→31.6%61.5\%\!→\!31.6\%; Table 1), and the class-conditional signal zeroes out (Table 2). The rotation encoding is therefore data-induced, a consequence of training against a rotation-bearing target. Freed capacity improves identity. The same control raises class recovery: super-category top-1 85.6%→89.8%85.6\%\!→\!89.8\% (+4.2+4.2 p), macro-F1 0.745→0.8080.745\!→\!0.808, fine-cat 62.6%→68.3%62.6\%\!→\!68.3\%: capacity the primary recipe spent on orientation is freed for identity, confirming it was being spent on yaw. Reconstruction is unchanged within seed noise (chamfer 0.0079→0.00750.0079\!→\!0.0075), so the collapse is not a fitting failure. The training-distribution controls (supplementary) show this is robust to the input augmentation: joint-augmentation controls (input and target rotated together) hold reconstruction and utilization, while both canonical-target variants free utilization. Interpretation. For the yaw-only 3D-FRONT setting, changing one boolean in the loss switches the vocabulary between rotation-aware and rotation-invariant: the behavioural change one would otherwise seek by rebuilding the encoder to be equivariant, plus improved utilization. The canonical-target regime is best read as a learned canonicalization [21, 22] with a discrete bottleneck; we propose no new mechanism (the canonical pose is given by the dataset). The invariance is empirical, holding over the training augmentation’s support (yaw only, not full SO(3)), where an equivariant encoder [18, 19, 20] or frame averaging [23] would generalize by construction. 7 Cross-dataset code alignment Does a geometry-only vocabulary transfer to a different asset library? We encode ShapeNet furniture through the frozen 500-code autoencoder and measure per-category alignment to the native 3D-FUTURE manifold (Section 3.4). The answer is category-dependent (Table 3): box-like categories (cabinet, bookshelf, table, lamp) already align (r=0.9r=0.9–1.6×1.6× native spacing), whereas organically-shaped categories (bed, sofa, chair) sit 3.23.2–5.1×5.1× off the manifold. Five confounds were ruled out: discrete-vs-continuous latent, mesh quality, pool size (a size sweep is invariant), orientation (both datasets Y-up; extents matched), and category (measured across all seven). A continuous transfer gradient. r is a manifold-distance ratio; we tie it to a task by retrieving each ShapeNet query’s nearest native neighbours in code space (balanced 6-way gallery, chance 0.170.17) and asking whether the nearest is the correct super-category (Table 3, P@1). At the category level every r<3r<3 category retrieves correctly 33–5×5× above chance (0.550.55–0.780.78), while sofa and chair fall to or below it (0.070.07, 0.090.09); the two rankings agree (Spearman −0.79-0.79). Scoring each of the 2,0332,033 query meshes by its own distance ratio resolves the shape (Fig 4): P@1 declines smoothly and monotonically from ∼0.8 0.8 at r<1r<1 to near zero at r>6r>6, with no cliff, passing through chance around r≈3r≈ 3. The r<3r<3 cut is therefore best read not as a hard boundary but as the point where geometry-only retrieval decays to chance; transfer is a continuum, and the binary label a convenience: an r=1r=1 query retrieves far better than an r=2.5r=2.5 one, though both nominally “transfer.” This anchor is at the super-category level: r tracks whether a foreign query lands among the right kind of native furniture, not that it retrieves the right individual shape (fine-grained retrieval is left open). Figure 4: The cross-dataset transfer curve. Each of 2,0332,033 ShapeNet query meshes is scored by its own distance ratio r to the native manifold and binned; y is the fraction whose nearest native neighbour in code space is the correct super-category (P@1; band is Wilson 95%95\%). Retrieval degrades smoothly with r (a gradient, not a step) and reaches chance (0.170.17) near r≈3r≈ 3. Table 3: Cross-dataset code alignment. r=r= ShapeNet-to-native distance // native intra-category spacing, encode-native matched methodology, 300 clouds/category, 500-code baseline autoencoder. r≈1r≈ 1: foreign geometry indistinguishable from native. P@1: fraction of ShapeNet queries whose nearest native neighbour in the pre-quantization embedding is the correct super-category, over a balanced 6-way gallery (chance 0.170.17); it anchors r to a retrieval task. Spearman(r, P@1)=−0.79=-0.79. The transfers/gap column is a two-way discretization of a continuum: retrieval degrades smoothly with r (Fig 4), so the labels are a reading aid, not a hard boundary. ShapeNet cat Native target r P@1 verdict cabinet Cabinet/Shelf/Desk 0.9× 0.60 transfers bookshelf Cabinet/Shelf/Desk 1.0× 0.78 transfers table Table 1.5× 0.55 transfers lamp Others 1.6× 0.72 transfers bed Bed 3.2× 0.36 gap sofa Sofa 3.9× 0.07 gap chair Chair 5.1× 0.09 gap The discrete codes agree with the embedding. r and P@1 are defined on the continuous pre-quantization embedding, but the paper’s unit of study is the code, so we repeat the alignment on the discrete index. Predicting each ShapeNet query’s native super-category from its code alone (majority native vote over the balanced gallery) gives a discrete P@1 that tracks the embedding P@1 across categories (Spearman 0.930.93) and r (Spearman −0.75-0.75): transferring categories score 0.460.46–0.610.61, sofa and chair fall to chance (0.080.08, 0.050.05). Independently, each transferring category’s 500500-bin code histogram most resembles the correct native super-category, whereas sofa and chair’s do not. Because the discrete codes reproduce the alignment measured on the continuous latent, the cross-dataset result is a property of the codes themselves, not an artifact of measuring the pre-quantization embedding. Interpretation. The split tracks geometric stereotypy: box-like furniture is nearly identical in silhouette across libraries (a cabinet is a cuboid everywhere), so its codes are dataset-agnostic; chairs, sofas, and beds vary far more, and that variation is what a geometry-only code is sensitive to. A label-based vocabulary is more robust to this shift, but not for free: carrying labels across datasets means reconciling two taxonomies of differing coverage and specificity (does “chair” map to one bucket, or split across armchair / stool / dining-chair?), itself manual work. The contrast is thus a trade: geometry codes need no taxonomy alignment but drift off-manifold where shape varies most, while labels resist that drift only once a human has aligned the label spaces. We do not evaluate downstream retrieval or placement here. 8 A generic augmentation partially closes the gap Can the gap shrink without letting the encoder see the target dataset, i.e. preserving a genuine unseen-dataset test? We retrain on 3D-FUTURE with a generic digitization augmentation (denoising: input augmented, target clean) — coordinate jitter, random point dropout, and voxel-snap (a low-poly/tessellation proxy). The menu is standard 3D robustness augmentation, justified generically rather than tuned to ShapeNet; the encoder never sees ShapeNet. Figure 5: Cross-dataset alignment ratio r per ShapeNet category on the 500500-code rotated-target model (open = baseline; filled = ++digitization augmentation, the two training seeds joined by a bar; gray arrow = baseline → shift). r<3r<3 (shaded) is on-manifold. Box-like categories transfer robustly and are seed-stable; the shape-variable categories (bed, sofa, chair) shrink toward the boundary at zero within-dataset cost, but their two aug seeds straddle it; the per-category effect is seed-dependent (Table 4). The augmentation gives partial domain robustness at zero within-dataset cost (Table 4): validation chamfer is unchanged (0.00720.0072 vs the 0.00730.0073 baseline), and the already-transferring box-like categories stay put across both training seeds. For the shape-variable categories the effect is real but seed-dependent: augmentation pulls sofa, bed, and chair from their 3.23.2–5.1×5.1× baselines down toward the r=3r=3 boundary, but two augmentation seeds disagree on which of them crosses it (sofa 2.32.3/4.04.0, bed 3.43.4/2.42.4, chair 4.34.3/2.92.9). We therefore report the aggregate shift toward transfer, not a per-category ordering. Table 4: Domain-robust digitization augmentation vs the baseline, same encode-native diagnostic. Augmented VAEs never observe ShapeNet; within-3D-FUTURE val chamfer 0.0072≈0.00730.0072≈ 0.0073 baseline (no regression). All are 500500-code rotated-target models in the same regime, differing only in the augmentation. s1, s2 are two augmentation training seeds: box-like categories transfer robustly across both; the shape-variable categories (sofa/bed/chair) are seed-variable, hovering near the r=3r=3 boundary. Category baseline r aug r (s1) aug r (s2) note cabinet 0.9× 0.9× 1.0× transfers (stable) bookshelf 1.0× 0.9× 1.0× transfers (stable) table 1.5× 1.5× 1.6× transfers (stable) lamp 1.6× 2.0× 1.6× transfers (stable) sofa 3.9× 2.3× 4.0× seed-variable bed 3.2× 3.4× 2.4× seed-variable chair 5.1× 4.3× 2.9× seed-variable Reading. Style-level augmentation removes the part of the cross-dataset gap attributable to digitization differences (sampling density, tessellation, scan noise). Its effect on the box-like categories is null (they already transfer) and stable across seeds; on the three shape-variable categories it shrinks the gap on average but with enough seed variance that no single one reliably crosses r<3r<3. This is consistent with a residual, genuine silhouette shift that surface-style augmentation only partly removes, and points to multi-source training with a third held-out dataset as the honest next lever (deferred). We deliberately exclude training the encoder on ShapeNet: even with chairs held out, it would let the encoder adapt to ShapeNet style through the other categories, silently degrading a dataset-level unseen test to a class-level one. 9 Discussion and limitations What these results are. A characterization of a geometry-only shape vocabulary: it carries class and yaw recoverably; its rotation content is set by the loss target; and it transfers across datasets in a category-dependent way a generic augmentation partly repairs. The probe protocol (∼90 90k pre-extracted codes, under two minutes on one GPU) is cheap enough to serve as a tokenizer-quality unit test before any downstream use. Bearing on scene generation. We build no scene synthesizer, but both findings bear on one. A generator built on these tokens reads each in place of the (class, pose) annotations current layout-based models take per object, so the class and orientation it can condition on are bounded by what the token encodes, which our probes measure (Sections 5–6); the loss-target control makes that bound a design choice, not a fixed encoder property. The cross-dataset result (Section 7) is about scale: a multi-library generator would reuse one geometry-only vocabulary across asset libraries, and r marks per category where that reuse holds, and where it would silently degrade. Limitations. • Downstream out of scope. We show what the codes contain and how they transfer, not that a synthesizer built on them matches a label-supervised one; probes measure recoverability from the code, not predictability from autoregressive context. • Entangled pose and shape. The primary recipe packs yaw and shape into one ∼9 9-bit token, so pose is not independently addressable: re-orienting an object means jumping to a different code that may also change its shape: a genuine design problem for a layout-editable generator, not just an unrun experiment. The arcs in Fig 3 show yaw varies smoothly in the latent, but not that a controllable, globally factorized yaw axis exists. The canonical-target regime removes the entanglement but is not annotation-free. • Single dataset pair; one VAE. Alignment is measured 3D-FUTURE ↔ ShapeNet with one 500-code autoencoder; the augmentation is a single configuration (two seeds), seed-variable per-category on the shape-variable classes (Table 4). • Yaw only. 3D-FRONT placements rotate about the vertical axis; the loss-target control speaks to yaw, not full SO(3). • Within-class rotation confound (training-side). The matched-yaw control (supplementary) equalizes the evaluation yaw distribution but not the training one, so geometric observability and training-yaw variety are not fully separated (that would need retraining on yaw-balanced data). • No external calibration point. We report recoverability in absolute terms. Established SSL point encoders (e.g. Point-BERT / Point-MAE [16, 17]) are not drop-in baselines (per-patch, multi-token, and far higher-capacity than our single ∼9 9-bit code), so a fair use is a clearly-labeled capacity-mismatched ceiling (a pooled SSL embedding, probed), left to future work. • Uneven seed coverage. The core probe table (Table 1) carries n=3n=3 error bars; the follow-ups — continuous-latent reference, yaw-error distribution (Fig 2), retrieval P@1, and transfer curve (Fig 4, Wilson intervals over 2,0332,033 meshes) — are single-seed, as are the VQ and encoder-backbone comparisons. 10 Conclusion A chamfer-trained FSQ autoencoder, given no class labels and no canonical-pose annotations, produces codes from which a small probe recovers class (fine-category 62.6±0.562.6± 0.5 %, super-category 85.6±1.385.6± 1.3 %) and yaw (52.7±0.5∘52.7± 0.5 ): the two channels synthesizers read from annotation. Swapping the chamfer target (rotated → canonical) collapses the yaw encoding to modal baseline and raises class recovery, so rotation content is set by the objective. Across datasets alignment is category-dependent (box-like transfers; bed/sofa/chair sit 3.23.2–5.1×5.1× off-manifold), which a target-blind augmentation partly and seed-dependently closes. Scope is narrow (yaw only, one dataset pair, one autoencoder), and a label-free synthesizer on these codes remains future work. Appendix S1 Supporting analyses Codebook-size sweep. Weighted super-category purity rises monotonically with codebook size (76.1%76.1\% at 108 codes to 89.1%89.1\% at 3,1253,125, all far above the 16.7%16.7\% random baseline), while chamfer plateaus past 500 codes and the effective vocabulary scales sub-linearly (Table S1). We anchor the main study at 500 codes as a size a downstream model could plausibly learn to predict over. Table S1: Codebook-size sweep. Effective vocabulary (used codes) scales sub-linearly; chamfer plateaus past 500 codes; weighted super-category purity improves monotonically (random ≈16.7%≈ 16.7\%). Used/Util count distinct codes over the canonical (unrotated) models; rotation exercises more: the 500500-code baseline uses ∼ 460 (∼ 93%) over rotated placements, the set the probes encode. Config Codes Used Util Chamfer Sup. Purity 108-code (3,3,3,2,2)(3,3,3,2,2) 108 106 98.1 % 0.0097 76.1 % 500-code (5,5,5,2,2)(5,5,5,2,2) [baseline] 500 400 80.0 % 0.0079 83.7 % 1250-code (5,5,5,5,2)(5,5,5,5,2) 1,250 732 58.6 % 0.0077 86.4 % 3125-code (5,5,5,5,5)(5,5,5,5,5) 3,125 1,235 39.5 % 0.0069 89.1 % Bottleneck: FSQ vs VQ-VAE. At a matched 500-code budget, an out-of-the-box VQ-VAE [2] (EMA codebook, commitment 0.250.25, single seed) collapses to 7.57.5 % utilization (36 codes) under the rotated target vs FSQ’s 96.396.3 % (481±8481± 8; both measured over rotated placements in this matched-budget comparison, consistent with the baseline’s 93%93\% over placements and 80%80\% over canonical models, Table S1); the collapse propagates to every probe (super-cat top-1 68.468.4 vs 85.685.6). Under the canonical target, where the task needs less capacity, the gap narrows (utilization 22.222.2 vs 84.384.3 %; super-cat 86.886.8 vs 89.889.8). The loss-target rotation finding survives qualitatively under VQ (more rotation under rotated-target, more class under canonical), just at lower magnitudes, confirming it is a property of the objective, not the bottleneck. We do not claim the utilization gap as a result: codebook collapse is a well-documented VQ failure mode with well-known fixes (codebook reset [26], k-means init [27], lower commitment), and these are single-seed, out-of-the-box numbers. Read this only as a practitioner note (FSQ gave us high utilization with no such tuning), not as evidence that FSQ is fundamentally better. Encoder backbone. Swapping PointNet for DGCNN [25] at k=20k=20 leaves chamfer, utilization, and neighbourhood purity within run-to-run noise (chamfer 0.00790.0079 vs 0.00760.0076; utilization 8080 vs 8383 %); k=10k=10 underfits. Both encoders are non-equivariant and max-pool over 512512 points, which discards much of the local edge structure EdgeConv exposes; the comparison may differ at larger point counts. This is one seed: we report no evidence of a difference at this scale, not equivalence, which a single seed cannot establish. Matched-yaw control. The within-class rotation signal (main paper, Section 5) is stronger for canonically-oriented classes, but those classes also see more varied training yaws in 3D-FRONT. To separate geometry from distribution we re-probe every class under an identical uniform yaw distribution: each mesh rotated through 2424 evenly-spaced yaws, mesh-disjoint eval. The split persists (Table S2): Sofa/Chair/Bed recover orientation at 3131–43∘43 (uniform baseline 90∘90 ), while Table/Others/Cabinet/Stool sit at 7979–88∘88 , essentially baseline. With the distribution matched the difference is geometric: front/back-symmetric furniture maps θ and θ+180∘θ+180 to near-identical clouds (hence codes), so orientation is unrecoverable regardless of training. The control equalizes the evaluation distribution; the training distribution stays natural (fully removing that confound would need retraining on yaw-balanced data). Table S2: Matched-yaw control. Class-conditional yaw error (mean/median deg, lower better) with every class given an identical uniform yaw distribution (2424 yaws/mesh, mesh-disjoint eval; uniform baseline 90∘90 ). The strong/weak split of Table 2 (main paper) survives distribution matching, isolating it to geometric observability rather than yaw variety. Super-cat Matched mean Matched median Sofa 31.5 18.1 Chair 32.1 24.7 Bed 43.3 25.3 Table 79.1 71.3 Others 84.1 79.0 Cabinet/Shelf/Desk 88.1 86.5 Pier/Stool 88.3 88.1 Training-distribution controls. Table S3 gives the training-time chamfer and utilization for the primary recipe and the four augmentation/target controls discussed in Section 6 of the main paper. Table S3: Primary recipe (row 1) and four training-distribution controls at 500 codes. Rows 2–3 vary the input distribution (target == rotated input); rows 4–5 vary the supervision target (canonical). Chamfer/utilization are training-time; recoverability is in Table 1 (main paper). Aug Target Chamfer Util — (baseline) rotated 0.0079 80.0 % ±15∘± 15 joint rotated 0.0080 82.0 % ±180∘± 180 joint rotated 0.0089 66.4 % ±15∘± 15 canonical 0.0075 79.6 % ±180∘± 180 canonical 0.0075 87.6 % References [1] F. Mentzer, D. Minnen, E. Agustsson, and M. Tschannen. Finite scalar quantization: VQ-VAE made simple. In ICLR, 2024. [2] A. van den Oord, O. Vinyals, and K. Kavukcuoglu. Neural discrete representation learning. In NeurIPS, 2017. [3] D. Paschalidou, A. Kar, M. Shugrina, K. Kreis, A. Geiger, and S. Fidler. ATISS: Autoregressive transformers for indoor scene synthesis. In NeurIPS, 2021. [4] J. Tang, Y. Nie, L. Markhasin, A. Dai, J. Thies, and M. Nießner. DiffuScene: Denoising diffusion models for generative indoor scene synthesis. In CVPR, 2024. [5] C. Lin and Y. Mu. InstructScene: Instruction-driven 3D indoor scene synthesis with semantic graph prior. In ICLR, 2024. [6] H. Fu, R. Jia, L. Gao, M. Gong, B. Zhao, S. Maybank, and D. Tao. 3D-FUTURE: 3D furniture shape with texture. IJCV, 2021. [7] H. Fu, B. Cai, L. Gao, L.-X. Zhang, J. Wang, C. Li, Q. Zeng, C. Sun, R. Jia, B. Zhao, and H. Zhang. 3D-FRONT: 3D furnished rooms with layouts and semantics. In ICCV, 2021. [8] A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. ShapeNet: An information-rich 3D model repository. arXiv:1512.03012, 2015. [9] G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes. In ICLR Workshop, 2017. [10] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. In CVPR, 2017. [11] Y. Yang, C. Feng, Y. Shen, and D. Tian. FoldingNet: Point cloud auto-encoder via deep grid deformation. In CVPR, 2018. [12] P. Mittal, Y.-C. Cheng, M. Singh, and S. Tulsiani. AutoSDF: Shape priors for 3D completion, reconstruction and generation. In CVPR, 2022. [13] X. Yan, L. Lin, N. J. Mitra, D. Lischinski, D. Cohen-Or, and H. Huang. ShapeFormer: Transformer-based shape completion via sparse representation. In CVPR, 2022. [14] B. Zhang, M. Nießner, and P. Wonka. 3DILG: Irregular latent grids for 3D generative modeling. In NeurIPS, 2022. [15] Y. Siddiqui, A. Alliegro, A. Artemov, T. Tommasi, D. Sirigatti, V. Rosov, A. Dai, and M. Nießner. MeshGPT: Generating triangle meshes with decoder-only transformers. In CVPR, 2024. [16] X. Yu, L. Tang, Y. Rao, T. Huang, J. Zhou, and J. Lu. Point-BERT: Pre-training 3D point cloud transformers with masked point modeling. In CVPR, 2022. [17] Y. Pang, W. Wang, F. E. H. Tay, W. Liu, Y. Tian, and L. Yuan. Masked autoencoders for point cloud self-supervised learning. In ECCV, 2022. [18] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. arXiv:1802.08219, 2018. [19] F. Fuchs, D. Worrall, V. Fischer, and M. Welling. SE(3)-Transformers: 3D roto-translation equivariant attention networks. In NeurIPS, 2020. [20] C. Deng, O. Litany, Y. Duan, A. Poulenard, A. Tagliasacchi, and L. Guibas. Vector neurons: A general framework for SO(3)-equivariant networks. In ICCV, 2021. [21] W. Sun, A. Tagliasacchi, B. Deng, S. Sabour, S. Yazdani, G. Hinton, and K. M. Yi. Canonical capsules: Self-supervised capsules in canonical pose. In NeurIPS, 2021. [22] R. Sajnani, A. Poulenard, J. Jain, R. Dua, L. Guibas, and S. Sridhar. ConDor: Self-supervised canonicalization of 3D pose for partial shapes. In CVPR, 2022. [23] O. Puny, M. Atzmon, H. Ben-Hamu, I. Misra, A. Grover, E. J. Smith, and Y. Lipman. Frame averaging for invariant and equivariant network design. In ICLR, 2022. [24] C. Huang, Z. Cao, Y. Wang, J. Wang, and M. Long. MetaSets: Meta-learning on point sets for generalizable representations. In CVPR, 2021. [25] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph CNN for learning on point clouds. ACM Trans. Graph., 2019. [26] P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever. Jukebox: A generative model for music. arXiv:2005.00341, 2020. [27] A. Łańcucki, J. Chorowski, G. Sanchez, R. Marxer, N. Chen, H. J. G. A. Dolfing, S. Khurana, T. Alumäe, and A. Laurent. Robust training of vector quantized bottleneck models. In IJCNN, 2020.