Paper deep dive

Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans

Sizhong Qin, Ramon Elias Weber, Xinzheng Lu

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 86

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/22/2026, 6:26:08 AM

Summary

HouseMind is a multimodal large language model (MLLM) framework that unifies the understanding, generation, and editing of architectural floor plans. By utilizing discrete room-instance tokens via VQ-VAE and a unified sequence-modeling approach, the model bridges the gap between geometric layouts and symbolic reasoning, enabling efficient, controllable, and locally deployable architectural design.

Entities (4)

HouseMind · model · 100%RPLAN · dataset · 95%VQ-VAE · algorithm · 95%Qwen3-0.6B · model-architecture · 90%

Relation Signals (3)

HouseMind → evaluatedon → RPLAN

confidence 95% · To evaluate this unified formulation, we construct a benchmark... based on the RPLAN dataset

HouseMind → uses → VQ-VAE

confidence 95% · HouseMind discretizes layouts into room-instance tokens using a Vector-Quantized Variational Autoencoder (VQ-VAE)

HouseMind → basedon → Qwen3-0.6B

confidence 90% · All models share the same hierarchical tokenization and architectural backbone based on Qwen3-0.6B

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Architectural floor plan design demands joint reasoning over geometry, semantics, and spatial hierarchy, which remains a major challenge for current AI systems. Although recent diffusion and language models improve visual fidelity, they still struggle with coherent spatial reasoning and controllable generation. We present HouseMind, a multimodal large language model that unifies floor plan understanding, generation, and editing in one framework. We introduce discrete room-instance tokens to construct a unified vocabulary that bridges layouts and symbolic reasoning. With multimodal alignment and instruction tuning, the model synthesizes coherent, controllable layouts from text instructions. Experiments show how the framework achieves superior geometric validity and controllability while remaining efficient and locally deployable.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

85,757 characters extracted from source content.

Expand or collapse full text

Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans Sizhong Qin1,2 Ramon Elias Weber†2 Xinzheng Lu†1 1Tsinghua University 2UC Berkeley https://housemind.github.io/ †equal contribution Abstract Architectural floor plan design demands joint reasoning over geometry, semantics, and spatial hierarchy, which remains a major challenge for current AI systems. Although recent diffusion and language models improve visual fidelity, they still struggle with coherent spatial reasoning and controllable generation. We present HouseMind, a multimodal large language model that unifies floor plan understanding, generation, and editing in one framework. We introduce discrete room-instance tokens to construct a unified vocabulary that bridges layouts and symbolic reasoning. With multimodal alignment and instruction tuning, the model synthesizes coherent, controllable layouts from text instructions. Experiments show how the framework achieves superior geometric validity and controllability while remaining efficient and locally deployable. Figure 1: HouseMind learns the language of space by modeling outlines and rooms as spatial tokens. Through hierarchical tokenization and multimodal reasoning, it can understand, generate, and edit architectural floor plans from natural language prompts. 1 Introduction Generative models and artificial intelligence (AI) driven workflows are reshaping how buildings are designed [9, 11, 16, 59]. They have been applied at different stages of the design process, including schematic generation, interactive co-design, and performance optimization [31, 27]. Although large language models (LLM) have been proven to be exceptional in generalizing, understanding, and emulating language, code, and pixels, they currently lack the ability for semantic reasoning that is required for spatial design. For the design of buildings, specifically floor plans, patterns are not sequential, but embedded in complex relationships. Due to this, the generation of an architectural layout remains one of the most cognitively demanding tasks [41, 47]. It requires models to capture hierarchical and relational dependencies between functional spaces while maintaining geometric feasibility and semantic coherence. In this research, we propose a new model that decodes spatial relationships and enables spatial coherence from prompt to output. Recent advances in diffusion-based and autoregressive models have greatly improved the fidelity and diversity of layout generation. However, these approaches still face four major limitations. First, they often regard layout synthesis as a purely visual process, without explicit reasoning at the room-instance level, which can lead to plans that appear locally plausible but lack global spatial coherence, such as consistent adjacency or circulation relationships among rooms. Moreover, preference-aligned or large-scale vision–language models frequently behave as black-box generators, offering limited interpretability and spatial controllability. At the same time, existing frameworks struggle to unify understanding, generation, and editing within a single architecture, particularly under the geometric and semantic complexity of building layouts. In addition, most AI systems remain computationally demanding and difficult to deploy locally, restricting their integration into practical design workflows. In this work, we introduce HouseMind, an efficient and locally deployable multimodal model that unifies floor plan understanding, generation, and editing within one coherent framework. HouseMind discretizes layouts into room-instance tokens using a Vector-Quantized Variational Autoencoder (VQ-VAE) [36] and leverages an LLM for multimodal reasoning. By representing both geometry and semantics as discrete token sequences, HouseMind bridges the gap between symbolic reasoning and continuous layout geometry, enabling controllable and interpretable operations directly in the latent token space. Our approach offers three key advantages: (1) Fine-grained reasoning at the room-instance level allows text-guided control over spatial structure and semantic composition. (2) Unified multitask formulation: The same model handles understanding, conditional generation, and localized editing within a single sequence modeling framework. (3) Practical efficiency: The compact architecture enables real-time inference and on-device deployment while maintaining global coherence. To evaluate this unified formulation, we construct a benchmark encompassing all three tasks (understanding, generation, and editing) based on the RPLAN dataset [44]. Extensive experiments demonstrate that HouseMind achieves strong generalization across modalities and tasks, outperforming prior diffusion- and LLM-based baselines while maintaining geometric validity, semantic consistency, and practical deployability. 2 Related Work Learning spatial and functional patterns. Early data-driven approaches learn geometric and functional regularities of architectural layouts directly from data. GAN-based frameworks [22, 24, 25, 1, 50, 18, 35] enhance realism through adversarial and graph-constrained objectives, but often overfit to local geometries and lack global spatial semantics. Graph- or GNN-based approaches [7, 19, 45, 46, 3, 42] model room connectivity and hierarchy to improve relational reasoning, though discrete graph representations limit geometric fidelity and scalability. Diffusion-based methods [33, 53, 5, 8, 57, 38] generate diverse and stable results via iterative denoising, yet remain computationally costly and typically confined to single-task synthesis without high-level control. Incorporating structural and semantic reasoning. To go beyond pure pattern learning, recent studies integrate explicit structural constraints and semantic control into generative frameworks. Structure-aware representations such as wall graphs or hierarchical layouts [8, 34] enable relationally consistent generation, while reinforcement and multi-agent frameworks [32, 58, 20, 10] optimize functional objectives and circulation logic. Further bridging geometry and semantics, MaskPLAN [56] introduces a VQ-VAE–based attribute discrete latent model that encodes geometric attributes into visual tokens and reconstructs them via masked transformer autoencoding, offering an early attempt at controllable semantic generation. More recently, hybrid transformer–diffusion paradigms couple geometric decoding with structural reasoning for improved controllability [5, 53]. Despite these advances, existing methods still rely on handcrafted priors or task-specific constraints, limiting their generalization across diverse design contexts. LLM-driven multimodal design. Recent progress in LLMs and multimodal large models (MLLMs) introduces a new paradigm that connects textual intent with spatial reasoning. Tell2Design [14] establishes a benchmark linking textual descriptions and floor plan layouts. ChatDesign [15] and DStruct2Design [21] leverage LLM priors for layout synthesis. More recent frameworks such as LLM-based FloorPlan Design [29] leverage LLMs to translate natural language into vectorized floor plans via structured semantic parsing, while ChatHouseDiffusion [28] integrates language understanding with diffusion-based generation for enhanced controllability. Cross-modal agents like CARD [54] and Zeng et al. [55] unify generation and editing, and FloorPlan-LLaMa [51] and FloorPlan-DeepSeek [52] align multimodal reasoning with expert feedback for semantic understanding and next-room prediction. Overall, while these efforts greatly enhance interpretability and cross-task reasoning, most models remain modular. This motivates our unified multitask multimodal framework that jointly learns geometric, semantic, and topological representations for consistent reasoning across understanding, generation, and editing. 3 Problem Formulation An architectural floor plan with N rooms can be decomposed into two components: an outline xox_o that defines the global floor plan boundary, and a set of room instances xrii=1N\x_r_i\_i=1^N that describe the geometry and semantics of individual rooms: x=xo,xrii=1N.x\;=\;\\,x_o,\;\x_r_i\_i=1^N\,\. (1) Discrete representation. Both the outline and the rooms are quantized by two separate VQ-VAE encoders: o=Eo(xo),ri=Er(xri,xo), z_o\;=\;E_o(x_o), z_r_i\;=\;E_r(x_r_i,\,x_o), (2) where o=(z1(o),…,zmo(o)) z_o=(z^(o)_1,…,z^(o)_m_o) and ri=(zi,1(r),…,zi,mi(r)) z_r_i=(z^(r)_i,1,…,z^(r)_i,m_i) denote the sequences of discrete tokens obtained from the outline and the i-th room, respectively. Here mom_o and mim_i are the numbers of tokens for the outline and room rir_i, and zj(o)∈oz^(o)_j\!∈\!Z_o, zi,j(r)∈rz^(r)_i,j\!∈\!Z_r come from the learned codebooks oZ_o (outline) and rZ_r (room). Figure 2: Understanding: given a prompt, an outline, and an existing floor plan, the model outputs a textual description, a bubble diagram, and structured JSON capturing spatial semantics. Generation: given a prompt and an outline, the model produces a complete, coherent floor plan. Editing: given a prompt, an outline, and a reference floor plan, the model outputs an updated plan aligned with the editing intent. Structured tokenization. The entire floor plan is represented as an interleaved token sequence combining geometric and semantic information: Z=[o,ℓr1,r1,…,ℓrN,rN],Z\;=\;[\, z_o,\; _r_1,\; z_r_1,\;…,\; _r_N,\; z_r_N\,], (3) where ℓri∈ _r_i\!∈\!C is the semantic label token of room rir_i, and C denotes the set of all room categories. Unified task formulation. Given the structured sequence Z, the proposed HouseMind framework jointly addresses three core tasks as shown in Fig. 2: • Understanding: inferring room functions, spatial relations, and topological constraints directly from Z. • Generation: given a text specification s (e.g., “three bedrooms and one bathroom”) and outline tokens o z_o, the model autoregressively generates the layout: p(Z∣o,s)=∏tp(Zt∣Z<t,o,s),p\! (Z z_o,s )\;=\; _tp\! (Z_t Z_<t,\, z_o,\,s ), (4) where ZtZ_t denotes the t-th token in the sequence. • Editing: given an existing layout sequence ZsrcZ^src and a text instruction s, the model produces an updated layout ZtgtZ^tgt as p(Ztgt∣Zsrc,s)=∏tp(Zttgt|Zsrc,Z<ttgt,s),p\! (Z^tgt Z^src,s )\;=\; _tp\! (Z^tgt_t\, |\,Z^src,\,Z^tgt_<t,\,s ), (5) modifying only tokens relevant to the instruction while keeping others unchanged. In summary, HouseMind unifies floor plan understanding, generation, and editing as a single sequence-modeling problem over discretized outline and room tokens, enabling LLMs to jointly reason about spatial semantics and geometric structure. 4 Method Our framework, HouseMind, unifies geometric understanding, generation, and editing of architectural floor plans within a MLLM. It consists of two core components: (1) Room-Instance Tokenization, which discretizes structural layouts into compact spatial tokens using hierarchical VQ-VAE modules; and (2) Multimodal Alignment and Instruction Tuning, which aligns spatial and linguistic representations for unified reasoning, generation, and editing. Figure 3: Overall framework of HouseMind. The model is trained through a three-stage multimodal alignment and instruction tuning pipeline: (S1) Embedding Initialization establishes cross-modal compatibility between geometric and linguistic tokens; (S2) Multimodal Pre-training aligns text and spatial representations; and (S3) Instruction Tuning (SFT) enables task-aware spatial reasoning. 4.1 Room-Instance Tokenization To bridge continuous geometric layouts and discrete sequence modeling, we employ VQ-VAE modules to learn discrete representations of the floor plan outline and individual room instances. Each branch adopts an encoder, quantizer, and decoder structure with a CNN encoder, a transposed-CNN decoder, and its own learnable codebook. Outline discretization. A CNN encoder Eo(⋅)E_o(·) extracts latent features from the binary outline mask xox_o. These features are vector-quantized using the outline codebook o=ek(o)k=1KoZ_o=\e^(o)_k\_k=1^K_o: zj(o)=ekj⋆(o),kj⋆=argmink∥Eo(xo)j−ek(o)∥2.z^(o)_j=e^(o)_k_j , k_j = _k\! E_o(x_o)_j-e^(o)_k _2. (6) Here j=1,…,moj=1,…,m_o indexes the outline tokens. The decoder reconstructs the outline as x^o=Do(o) x_o=D_o( z_o), where o=(z1(o),…,zmo(o)) z_o=(z^(o)_1,…,z^(o)_m_o). This process converts geometric contours into a discrete vocabulary that captures the global building envelope. Conditional room discretization. Each room mask xrix_r_i is encoded jointly with the corresponding outline context. The conditional encoder Er(⋅)E_r(·) maps (xri,xo)(x_r_i,x_o) to latent features, which are then quantized using the room codebook r=ek(r)k=1KrZ_r=\e^(r)_k\_k=1^K_r: zi,j(r)=eki,j⋆(r),ki,j⋆=argmink∥Er(xri,xo)j−ek(r)∥2.z^(r)_i,j=e^(r)_k_i,j , k_i,j = _k\! E_r(x_r_i,x_o)_j-e^(r)_k _2. (7) Here j=1,…,mij=1,…,m_i enumerates tokens within room rir_i. The decoder reconstructs each room as x^ri=Dr(ri,xo) x_r_i=D_r( z_r_i,x_o), where ri=(zi,1(r),…,zi,mi(r)) z_r_i=(z^(r)_i,1,…,z^(r)_i,m_i). By conditioning room encoding on the outline, the model learns context-aware room representations that capture both geometry and spatial adjacency, forming a structured and interpretable spatial token sequence. 4.2 Multimodal Alignment and Instruction Tuning Built upon discretized spatial tokens, HouseMind integrates language and geometry through a three-stage multimodal alignment and instruction tuning pipeline (Fig. 3) that progressively enhances spatial reasoning, cross-modal understanding, and controllable generation. Stage 1. Embedding initialization. Spatial codebooks obtained from the VQ-VAE modules, including the outline codes oZ_o and room codes rZ_r, are incorporated into the language model’s vocabulary by assigning each code a unique trainable token embedding. This establishes a one-to-one correspondence between discrete spatial codes and the model’s textual tokens, forming a unified vocabulary that jointly represents geometry and language. Through this initialization, HouseMind ensures that spatial and linguistic symbols coexist within the same token space, allowing the MLLM to process geometric layouts and natural language seamlessly within a single autoregressive sequence. Table 1: Understanding results. Success: success rate; RMR: room match rate; LocAcc: room location accuracy; AreaDiff: room area difference (m2); AdjAcc: room adjacency accuracy; RelAcc: spatial relation accuracy. Method Success RMR LocAcc AreaDiff↓ AdjAcc RelAcc Time (s) LLaVA-v1.6-Mistral-7B-HF 1.000 0.616 0.225 3.649 0.134 0.056 ∼ 6 Qwen3-VL-8B-Instruct 1.000 0.698 0.347 5.837 0.382 0.128 ∼ 8 InternVL3.5-8B 1.000 0.847 0.546 12.234 0.469 0.157 ∼ 13 MiniCPM-V 4.5 0.996 0.904 0.492 13.765 0.597 0.208 ∼ 14 HouseMind-U 1.000 0.998 0.969 0.549 0.990 0.808 ∼ 3 HouseMind-O 1.000 0.998 0.925 0.655 0.954 0.738 ∼ 3 Table 2: Generation results. Micro/Macro IoU measure pixel-level overlap; SSIM [40] and PSNR quantify perceptual similarity; FID [6] and GED [30] evaluate distributional realism; Node F1 and Edge Overlap assess graph-level correctness. * denotes methods without released code; results are reproduced. Method Micro IoU Macro IoU SSIM PSNR FID↓ GED↓ Node F1 Edge Ovl. Time (s) Qwen-Image-Edit-2509 0.161 0.0621 0.721 12.4 156 – – – ∼ 240 Tell2Design 0.390 0.307 0.840 13.2 30.5 6.94 0.808 0.197 ∼ 15 ChatHouseDiffusion 0.589 0.521 0.866 14.9 11.3 2.36 0.985 0.710 ∼ 30 FloorPlanLLaMA* 0.607 0.511 0.874 15.5 49.3 2.68 0.922 0.574 ∼ 1 HouseMind-G 0.709 0.653 0.886 16.0 1.91 1.01 0.994 0.880 ∼ 2 HouseMind-O 0.710 0.654 0.887 16.1 1.89 1.03 0.994 0.880 ∼ 2 Table 3: Editing results. Δ and Δ measure editing precision (spatial and pixel-level change correctness); Micro/Macro IoU assess final layout quality; GED evaluates distributional realism; Node F1 and Edge Overlap assess graph-level consistency. Method ↓ Micro IoU Macro IoU GED↓ Node F1 Edge Ovl. Time (s) Before edit – – 0.880 0.821 3.06 0.934 0.740 – FLUX.1-Kontext-dev 0.053 0.0162 0.289 0.185 8.91 0.765 0.222 ∼ 240 Qwen-Image-Edit-2509 0.088 0.0074 0.567 0.429 7.96 0.915 0.426 ∼ 240 HouseMind-E 0.608 0.0019 0.855 0.823 0.467 0.998 0.934 ∼ 3 HouseMind-O 0.598 0.0022 0.844 0.813 0.653 0.997 0.908 ∼ 3 Stage 2. Multimodal pre-training. After establishing the shared vocabulary, the model is trained on large-scale paired data comprising textual descriptions, outline tokens, and room tokens. Using an autoregressive language-modeling objective, the model learns to predict the next token in mixed sequences of text and spatial tokens. This stage enables bidirectional alignment between language and geometry: the model learns to interpret textual spatial relations (e.g., “the kitchen is north of the living room”) and to reconstruct or complete geometric layouts from tokenized representations. Through this process, the model acquires a unified understanding of architectural semantics and geometry, serving as the foundation for subsequent instruction tuning and spatial reasoning. Stage 3. Instruction tuning (supervised fine-tuning, SFT). The final stage performs SFT on curated multimodal instruction data. It covers three core tasks: (1) Understanding: interpreting existing layouts to describe room topology and relations; (2) Generation: synthesizing plausible layouts from text and outlines; (3) Editing: modifying existing floor plans according to natural-language instructions. This SFT stage grants the model task awareness, spatial reasoning skills, and controllability, enabling consistent and prompt-driven design interaction. Each training sample is serialized into an interleaved sequence of text and spatial tokens, allowing the model to learn within a unified autoregressive framework. During inference, the same formulation supports tasks such as text-to-layout generation, layout interpretation, and instruction-based editing within a single unified architecture. 5 Experiments 5.1 Benchmark Construction and Data Processing We first construct a canonical JSON representation for each floor plan, encoding room type, area, centroid, and pairwise spatial relations. Based on these JSONs, Qwen3-30B-A3B [48] automatically generates two textual descriptions per sample: a simple version summarizing the layout and a detailed version including areas, positions, and relations. The two versions are mixed to form a linguistically diverse corpus that serves as the pre-training base for HouseMind. Built upon this base, we establish the first unified benchmark that jointly evaluates understanding, generation, and editing of architectural floor plans under consistent geometry, text, and evaluation protocols. The understanding and generation tasks are directly derived from the mixed corpus, while the editing subset extends it with controlled structural modifications such as adding or removing rooms to assess spatial controllability. In total, 2,308 samples are reserved as a shared test set for all three tasks, while the remaining 76,122 and 2,308 samples are used for training and validation, respectively. 5.2 Evaluation Protocols and Metrics We adopt a unified evaluation protocol across all tasks, measuring both pixel-level geometry and graph-level spatial consistency. Pixel-level metrics (e.g., Micro/Macro IoU, SSIM [40], PSNR, FID [6]) assess geometric and perceptual fidelity of the generated layouts, while structure-level metrics (e.g., Node F1, Edge Overlap, GED [30]) evaluate topological correctness and relational consistency among rooms. This ensures that the reported scores reflect only meaningful and structurally valid predictions. All evaluations are performed on 256×256256×256 color-mapped layouts that contain wall boundaries. We apply a graphics-based post-processing method to normalize all generated images to the same size and append wall boundaries. Results are averaged over the shared test set of 2,308 samples. Since this study aims to develop a lightweight and locally deployable room layout design method, all compared approaches in this section are implemented to run on a single NVIDIA RTX 3090 GPU, and inference time is reported in seconds per sample. Implementation details of other methods are provided in the supplementary materials. Figure 4: Qualitative comparison results of understanding, generation, and editing. For understanding tasks (U), HouseMind accurately identifies the number of rooms and their connections. For generation tasks (G), HouseMind preserves both the room layout and the overall outline consistency; the generation prompts are provided in the supplementary materials. For editing tasks (E), HouseMind accurately executes the specified modifications when the instructions are explicit. 5.3 Quantitative Analysis We train separate models for the three core tasks: HouseMind-U (understanding), HouseMind-G (generation), HouseMind-E (editing), and a unified variant, HouseMind-O (Omni), jointly trained on all tasks for better cross-task generalization. All models share the same hierarchical tokenization and architectural backbone based on Qwen3-0.6B [48], ensuring consistent and efficient multimodal reasoning. Understanding. Understanding architectural layouts requires not only recognizing room types but also reasoning about their spatial hierarchy and inter-room relationships, aligning visual geometry with textual semantics. Current vision–language models (e.g., LLaVA-v1.6-Mistral-7B-HF [17], Qwen3-VL-8B-Instruct [2], InternVL3.5-8B [39], MiniCPM-V 4.5 [49]) rely on large-scale multimodal pre-training and show strong visual perception, but they lack structured reasoning and spatial consistency at the room-instance level. To overcome these issues, HouseMind introduces explicit structural and semantic constraints for more accurate and consistent spatial understanding. As shown in Table 1, HouseMind achieves superior performance across all metrics, reaching perfect success and matching rates, indicating precise multimodal alignment between geometry and language. Compared with vision–language baselines, it improves room localization and adjacency accuracy by more than 40 absolute points and reduces the mean room area error from several square meters to below 0.6 m2m^2. Its relational reasoning accuracy of about 0.8 further demonstrates the model’s ability to infer complex spatial dependencies beyond direct visual cues. Moreover, HouseMind-O performs comparably to its individually trained counterpart, suggesting that the unified multitask architecture maintains both robustness and efficiency. Generation. Unlike general-purpose diffusion or image-based models (e.g., ChatHouseDiffusion [28], FloorPlanLLaMA [51]) and open-source multimodal image editing models (e.g., Qwen-Image-Edit-2509 [43], FLUX.1-Kontext-dev [12]), HouseMind maintains geometric validity, semantic consistency, and topological coherence throughout the generation process. For fair comparison, all methods are evaluated under a unified graphical post-processing pipeline that standardizes geometric and scale alignment, as detailed in the supplementary materials. Notably, FLUX.1-Kontext-dev failed to produce complete layouts owing to the CLIP encoder’s restricted token budget, while Qwen-Image-Edit-2509 was executed under int8 quantization to fit GPU memory constraints. As summarized in Table 2, HouseMind consistently outperforms existing methods across both pixel-level and graph-level metrics. It achieves Micro/Macro IoU scores of 0.71/0.65, improving IoU by over 10% compared with ChatHouseDiffusion, while reducing FID from 11.3 to 1.9, demonstrating significant gains in realism and spatial precision. Graph-based indicators further confirm that HouseMind generates layouts with superior room connectivity and adjacency consistency. The unified multimodal variant, HouseMind-O, achieves even more stable results across all metrics, validating the effectiveness of unified pre-training for multimodal spatial reasoning and generation. Editing. For the editing task, HouseMind jointly encodes textual editing instructions and the original layout under explicit geometric and semantic constraints, enabling precise additive and subtractive modifications while maintaining global spatial logic. Unlike general-purpose image-editing models, it performs structure-aware control rather than low-level pixel manipulation. As shown in Table 3, this design yields controllable and spatially consistent modifications with significantly higher editing fidelity compared to the post-edit ground truth. Since some editing instructions do not explicitly specify room positions or sizes, we also evaluate structural-level consistency. The nearly perfect Node F1 indicates accurate modification of room types, while the substantially lower GED and higher edge overlap relative to the pre-edit layouts demonstrate that HouseMind produces more coherent and semantically consistent spatial relationships after editing. In contrast, general multimodal image editing models fail to achieve comparable results and often degrade the original structural integrity. 5.4 Qualitative Analysis To further evaluate the effectiveness of our approach, we conduct a qualitative comparison using two state-of-the-art multimodal models, GPT-5 [26] and Gemini 2.5 Pro [4], as shown in Fig. 4. Understanding. LLaVA-v1.6-Mistral-7B-HF tends to output a fixed pattern and fails to produce complete layouts. Qwen3-VL-8B-Instruct generates almost all room types regardless of the actual input, resulting in a mismatch with the ground truth. InternVL3.5-8B and MiniCPM-V 4.5 achieve more accurate room prediction, yet they still suffer from noticeable errors in room size estimation and topological relationships. GPT-5 achieves a noticeable improvement, while Gemini 2.5 Pro produces relatively coherent bubble-like representations but still contains room-type errors. In contrast, HouseMind captures room types, sizes, and topological relations with high fidelity, exhibiting only minor local inaccuracies. Generation. Tell2Design is trained on manually annotated and rule-based datasets, which limits its generalization to out-of-distribution layouts. ChatHouseDiffusion leverages room outlines effectively and performs well on simple layouts but struggles with complex spatial configurations. FloorPlanLLaMA employs a VQ-VAE to encode the entire floor plan; while it preserves the approximate room count, it often fails to maintain boundary consistency and produces noticeable noise (the results shown are after post-processing). Qwen-Image-Edit generates outputs inconsistent with the given prompts. GPT-5 and Gemini 2.5 Pro produce generally reasonable room layouts but still fail to meet specific design constraints. By contrast, HouseMind generates results that align well with the textual descriptions and accurately conform to the given building outlines. Editing. The editing task is relatively simple; however, both FLUX.1-Kontext-dev and Qwen-Image-Edit-2509 tend to introduce irrelevant elements during generation. GPT-5 performs better and generally fulfills the editing instructions but slightly alters the existing layout, while Gemini 2.5 Pro performs somewhat worse. In contrast, HouseMind achieves precise, localized modifications without affecting unrelated regions, demonstrating fine-grained spatial control and superior structural consistency. 5.5 Ablations All ablation studies are conducted on the validation split to avoid biasing the held-out test set used for Tables 1–3. Validation cross-entropy (Eval Loss) is adopted as a concise indicator of cross-modal alignment, where lower values indicate stronger correspondence between text and spatial representations. As illustrated in Sec. 4.2, HouseMind follows a three-stage training pipeline. To evaluate the contribution of each stage, we compare four variants: removing both Stage 1 and 2 (w/o Stage 1&2), removing Stage 1 only (w/o Stage 1), removing Stage 2 only (w/o Stage 2), and the full model (Full). Table 4: Loss under different training-stage configurations. Model Train Loss ↓ Eval Loss ↓ w/o Stage 1 & 2 0.0729 0.0836 w/o Stage 1 0.0659 0.0840 w/o Stage 2 0.0712 0.0831 Full 0.0644 0.0830 Removing either Stage 1 or Stage 2 increases the evaluation loss, revealing weaker multimodal alignment. Without Stage 1, the model fails to ground spatial tokens within a stable embedding space, causing optimization instability. Without Stage 2, the backbone lacks higher-level text–layout correspondence even when embeddings are initialized. The complete three-stage pipeline achieves the lowest loss, demonstrating that Stage 1 ensures consistent token-level initialization, while Stage 2 refines global spatial reasoning. Together, this progressive alignment process underscores the necessity of coupling stable spatial grounding with high-level semantic fusion for robust multimodal understanding. It provides the foundation that enables HouseMind to generalize effectively across understanding, generation, and editing tasks. 6 Discussion Limitations. Despite promising performance, several limitations remain. (1) The current editing module focuses primarily on simple operations such as room addition and deletion, without supporting complex topological transformations. (2) Functional components like doors, windows, and furniture are not yet modeled, limiting the model’s applicability to detailed interior design. (3) The system’s behavior is not fully aligned with human design preferences and aesthetic constraints, leaving a gap between AI-generated layouts and professional design standards. Future Directions. Future work will address these issues from multiple perspectives. (1) Expand the dataset [37] and instruction set to include a broader variety of editing tasks, improving generalization across unseen layout patterns. (2) Integrate architecture-structure co-design [13] into the pipeline, enabling full-process generative design that spans architectural layout and structural configuration. (3) Incorporate performance evaluation metrics [23] and human-centered preference alignment [51], allowing the model to generate layouts that not only satisfy spatial logic but also meet safety, comfort, and sustainability criteria. 7 Conclusion HouseMind is a unified and lightweight framework that enables MLLMs to understand, generate, and edit architectural floor plans through hierarchical tokenization. By bridging geometric structures with linguistic reasoning, it achieves coherent, controllable, and interpretable spatial design. Its room-by-room reasoning paradigm aligns with how architects iteratively conceive and refine functional spaces in practice. Extensive experiments across understanding, generation, and editing tasks demonstrate consistent gains in accuracy, efficiency, and semantic fidelity. These results establish tokenization as a key mechanism linking large language models with spatial design intelligence, marking a crucial step toward human-aligned and performance-aware architecture–structure co-design. Acknowledgements This work was supported by the Beijing Municipal Natural Science Foundation (8252008), the Tsinghua University Initiative Scientific Research Program (2025Z03KYY001), and the National Natural Science Foundation of China (525B2130). References [1] M. Aalaei, M. Saadi, M. Rahbar, and A. Ekhlassi (2023-11) Architectural layout generation using a graph-constrained conditional Generative Adversarial Network (GAN). Automation in Construction 155, p. 105053. External Links: ISSN 0926-5805, Document Cited by: §2. [2] S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, W. Ge, Z. Guo, Q. Huang, J. Huang, F. Huang, B. Hui, S. Jiang, Z. Li, M. Li, M. Li, K. Li, Z. Lin, J. Lin, X. Liu, J. Liu, C. Liu, Y. Liu, D. Liu, S. Liu, D. Lu, R. Luo, C. Lv, R. Men, L. Meng, X. Ren, X. Ren, S. Song, Y. Sun, J. Tang, J. Tu, J. Wan, P. Wang, P. Wang, Q. Wang, Y. Wang, T. Xie, Y. Xu, H. Xu, J. Xu, Z. Yang, M. Yang, J. Yang, A. Yang, B. Yu, F. Zhang, H. Zhang, X. Zhang, B. Zheng, H. Zhong, J. Zhou, F. Zhou, J. Zhou, Y. Zhu, and K. Zhu (2025) Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §5.3. [3] M. H. Dupty, Y. Dong, S. Leng, G. Fu, Y. L. Goh, W. Lu, and W. S. Lee (2024-06) Constrained layout generation with factor graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 12851–12860. Cited by: §2. [4] G. Gemini Team (2025) Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. External Links: Link Cited by: §5.4. [5] A. Gueze, M. Ospici, D. Rohmer, and M. Cani (2023) Floor plan reconstruction from sparse views: combining graph neural network with constrained diffusion. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Vol. , p. 1575–1584. External Links: Document Cited by: §2, §2. [6] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, p. 6629–6640. External Links: ISBN 9781510860964 Cited by: Table 2, Table 2, §5.2. [7] R. Hu, Z. Huang, Y. Tang, O. Van Kaick, H. Zhang, and H. Huang (2020-08) Graph2Plan: learning floorplan generation from layout graphs. ACM Trans. Graph. 39 (4). External Links: ISSN 0730-0301, Link, Document Cited by: §2. [8] S. Hu, W. Wu, Y. Wang, B. Xu, and L. Zheng (2025-04) GSDiff: Synthesizing vector floorplans via geometry-enhanced structural graph generation. Proceedings of the AAAI Conference on Artificial Intelligence 39 (16), p. 17323–17332. External Links: ISSN 2374-3468, Document Cited by: §2, §2. [9] S. Jang, H. Roh, and G. Lee (2025-06) Generative AI in architectural design: Application, data, and evaluation methods. Automation in Construction 174, p. 106174. External Links: ISSN 0926-5805, Document Cited by: §1. [10] R. Kakooee and B. Dillenburger (2025-01) Enhancing architectural space layout design by pretraining deep reinforcement learning agents. Journal of Computational Design and Engineering 12 (1), p. 149–166. External Links: ISSN 2288-5048, Document Cited by: §2. [11] A. Khan, S. Chang, and H. Chang (2025-12) Generative AI approaches for architectural design automation. Automation in Construction 180, p. 106506. External Links: ISSN 0926-5805, Document Cited by: §1. [12] B. F. Labs, S. Batifol, A. Blattmann, F. Boesel, S. Consul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esser, S. Kulal, K. Lacey, Y. Levi, C. Li, D. Lorenz, J. Müller, D. Podell, R. Rombach, H. Saini, A. Sauer, and L. Smith (2025) FLUX.1 kontext: flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742. External Links: Link Cited by: §5.3. [13] H. Leng, Y. Gao, and Y. Zhou (2024-12) ArchiDiffusion: A novel diffusion model connecting architectural layout generation from sketches to Shear Wall Design. Journal of Building Engineering 98, p. 111373. External Links: ISSN 2352-7102, Document Cited by: §6. [14] S. Leng, Y. Zhou, M. H. Dupty, W. S. Lee, S. Joyce, and W. Lu (2023-07) Tell2Design: A Dataset for Language-Guided Floor Plan Generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki (Eds.), p. 14680–14697. External Links: Document Cited by: §2. [15] J. Li, Y. Luo, S. Lu, J. Zhang, J. Wang, R. Guo, and S. Wang (2024) ChatDesign: Bootstrapping Generative Floor Plan Design With Pre-trained Large Language Models. In Proceedings of the 29th International Conference of the Association for Computer Aided Architectural Design Research in Asia (CAADRIA), Vol. 1, p. 99–108. External Links: Document Cited by: §2. [16] W. Liao, X. Lu, Y. Fei, Y. Gu, and Y. Huang (2024-01) Generative AI design for building structures. Automation in Construction 157, p. 105187. External Links: ISSN 0926-5805, Document Cited by: §1. [17] H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee (2024-01) LLaVA-next: improved reasoning, ocr, and world knowledge. Note: https://llava-vl.github.io/blog/2024-01-30-llava-next/ Cited by: §5.3. [18] J. Liu, Z. Qiu, L. Wang, P. Liu, G. Cheng, and Y. Chen (2024-03) Intelligent floor plan design of modular high-rise residential building based on graph-constrained generative adversarial networks. Automation in Construction 159, p. 105264. External Links: ISSN 0926-5805, Document Cited by: §2. [19] Z. Lu, Y. Li, and F. Wang (2025-02) Complex layout generation for large-scale floor plans via deep edge-aware GNNs. Applied Intelligence 55 (6), p. 400. External Links: ISSN 1573-7497, Document Cited by: §2. [20] G. Luo, X. Zhou, L. Feng, J. Liu, P. Liu, Y. Liao, W. Shan, and H. Qi (2025-11) Controllable and flexible residential floor plan layout design based on multi-agent deep reinforcement learning with layout prior size and similar experience abandon. Advanced Engineering Informatics 68, p. 103702. External Links: ISSN 1474-0346, Document Cited by: §2. [21] Z. Luo, L. Lara, G. Y. Luo, F. Golemo, C. Beckham, and C. Pal (2024) DStruct2Design: data and benchmarks for data structure driven generative floor plan design. arXiv preprint arXiv:2407.15723. Cited by: §2. [22] Z. Luo and W. Huang (2022-10) FloorplanGAN: Vector residential floorplan adversarial generation. Automation in Construction 142, p. 104470. External Links: ISSN 0926-5805, Document Cited by: §2. [23] A. Meselhy and A. Almalkawi (2025-04) A review of artificial intelligence methodologies in computational automated generation of high performance floorplans. npj Clean Energy 1 (1), p. 2. External Links: ISSN 3059-2232, Document Cited by: §6. [24] N. Nauata, K. Chang, C. Cheng, G. Mori, and Y. Furukawa (2020) House-GAN: Relational generative adversarial networks for graph-constrained house layout generation. In European Conference on Computer Vision, p. 162–177. External Links: Document, ISBN 978-3-030-58452-8 Cited by: §2. [25] N. Nauata, S. Hosseini, K. Chang, H. Chu, C. Cheng, and Y. Furukawa (2021-06) House-gan++: generative adversarial layout refinement network towards intelligent computational agent for professional architects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13632–13641. Cited by: §2. [26] OpenAI (2025-10) Introducing GPT-5. Note: https://openai.com/index/introducing-gpt-5/ Cited by: §5.4. [27] S. Qin, H. Guan, W. Liao, Y. Gu, Z. Zheng, and H. Xue (2024-10) Intelligent design and optimization system for shear wall structures based on large language models and generative artificial intelligence. Journal of Building Engineering 95, p. 109996. External Links: ISSN 2352-7102, Document Cited by: §1. [28] S. Qin, C. He, Q. Chen, S. Yang, W. Liao, Y. Gu, and X. Lu (2024) ChatHouseDiffusion: prompt-guided generation and editing of floor plans. arXiv preprint arXiv:2410.11908. Cited by: §2, §5.3. [29] Z. Qiu, J. Liu, Y. Wu, P. Liu, H. Qi, H. Liang, and Y. Xia (2025-12) LLM-based framework for automated and customized floor plan design. Automation in Construction 180, p. 106512. External Links: ISSN 0926-5805, Document Cited by: §2. [30] A. Sanfeliu and K. Fu (1983) A distance measure between attributed relational graphs for pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics SMC-13 (3), p. 353–362. External Links: Document Cited by: Table 2, Table 2, §5.2. [31] A. Savov, A. Yoo, C. Lin, and B. Dillenburger (2025) Generalist Generative Agent: Open-ended design exploration with large language models. In Proceedings of the 30th International Conference of the Association for Computer Aided Architectural Design Research in Asia (CAADRIA), Cited by: §1. [32] P. Y. Su, X. Lin, W. S. Lu, F. Xiong, Z. Y. Peng, and Y. Lu (2024-12) Generative design for complex floorplans in high-rise residential buildings: A Monte Carlo tree search-based self-organizing multi-agent system (MCTS-MAS) solution. Expert Systems with Applications 258, p. 125167. External Links: ISSN 0957-4174, Document Cited by: §2. [33] P. Su, W. Lu, J. Chen, and S. Hong (2024) Floor plan graph learning for generative design of residential buildings: a discrete denoising diffusion model. Building Research & Information 52 (6), p. 627–643. External Links: Document, Link, https://doi.org/10.1080/09613218.2023.2288097 Cited by: §2. [34] J. Sun, W. Wu, L. Liu, W. Min, G. Zhang, and L. Zheng (2022-07) WallPlan: synthesizing floorplans by learning to generate wall graphs. ACM Trans. Graph. 41 (4). External Links: ISSN 0730-0301, Link, Document Cited by: §2. [35] H. Tang, Z. Zhang, H. Shi, B. Li, L. Shao, N. Sebe, R. Timofte, and L. Van Gool (2023-06) Graph Transformer GANs for Graph-Constrained House Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 2173–2182. External Links: Document, ISBN 979-8-3503-0129-8 Cited by: §2. [36] A. van den Oord, O. Vinyals, and K. Kavukcuoglu (2017) Neural discrete representation learning. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Cited by: §1. [37] C. van Engelenburg, F. Mostafavi, E. Kuhn, Y. Jeon, M. Franzen, M. Standfest, J. van Gemert, and S. Khademi (2025) MSD: A Benchmark Dataset for Floor Plan Generation of Building Complexes. In Computer Vision – ECCV 2024, A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol (Eds.), p. 60–75. External Links: Document, ISBN 978-3-031-73636-0 Cited by: §6. [38] S. Wang and R. Pajarola (2025-10) Eliminating Rasterization: Direct Vector Floor Plan Generation With DiffPlanner. IEEE Transactions on Visualization and Computer Graphics 31 (10), p. 7906–7922. External Links: ISSN 1941-0506, Document Cited by: §2. [39] W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shao, Z. Wang, Z. Chen, H. Zhang, G. Yang, H. Wang, Q. Wei, J. Yin, W. Li, E. Cui, G. Chen, Z. Ding, C. Tian, Z. Wu, J. Xie, Z. Li, B. Yang, Y. Duan, X. Wang, Z. Hou, H. Hao, T. Zhang, S. Li, X. Zhao, H. Duan, N. Deng, B. Fu, Y. He, Y. Wang, C. He, B. Shi, J. He, Y. Xiong, H. Lv, L. Wu, W. Shao, K. Zhang, H. Deng, B. Qi, J. Ge, Q. Guo, W. Zhang, S. Zhang, M. Cao, J. Lin, K. Tang, J. Gao, H. Huang, Y. Gu, C. Lyu, H. Tang, R. Wang, H. Lv, W. Ouyang, L. Wang, M. Dou, X. Zhu, T. Lu, D. Lin, J. Dai, W. Su, B. Zhou, K. Chen, Y. Qiao, W. Wang, and G. Luo (2025) InternVL3.5: advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Cited by: §5.3. [40] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (4), p. 600–612. External Links: Document Cited by: Table 2, Table 2, §5.2. [41] R. E. Weber, C. Mueller, and C. Reinhart (2022-08) Automated floorplan generation in architectural design: A review of methods and applications. Automation in Construction 140, p. 104385. External Links: ISSN 0926-5805, Document Cited by: §1. [42] R. E. Weber, C. Mueller, and C. Reinhart (2024-09) A hypergraph model shows the carbon reduction potential of effective space use in housing. Nature Communications 15 (1), p. 8327. External Links: ISSN 2041-1723, Document Cited by: §2. [43] C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, Y. Chen, Z. Tang, Z. Zhang, Z. Wang, A. Yang, B. Yu, C. Cheng, D. Liu, D. Li, H. Zhang, H. Meng, H. Wei, J. Ni, K. Chen, K. Cao, L. Peng, L. Qu, M. Wu, P. Wang, S. Yu, T. Wen, W. Feng, X. Xu, Y. Wang, Y. Zhang, Y. Zhu, Y. Wu, Y. Cai, and Z. Liu (2025) Qwen-image technical report. arXiv preprint arXiv:2508.02324. External Links: Link Cited by: §5.3. [44] W. Wu, X. Fu, R. Tang, Y. Wang, Y. Qi, and L. Liu (2019-11) Data-driven interior plan generation for residential buildings. ACM Trans. Graph. 38 (6). External Links: ISSN 0730-0301, Link, Document Cited by: §1. [45] T. Xia, A. Ledbetter, A. Bobe, J. Hofland, B. Krouwels, T. Wang, L. C. Siebert, P. Chan, and J. Yang (2024) Interactive ai for generative housing design based on graph neural networks and deep generative models. In Proceedings of the 2024 European Conference on Computing in Construction, p. 469–477. External Links: Document Cited by: §2. [46] Y. Xin, Y. Zhou, and Y. Liu (2025-08) Prompts to layouts: Hybrid graph neural network and agent-based model for generative architectural design. Automation in Construction 176, p. 106253. External Links: ISSN 0926-5805, Document Cited by: §2. [47] S. Yan, C. Wu, and Y. Zhang (2025) Generative design for architectural spatial layouts: a review of technical approaches. Journal of Asian Architecture and Building Engineering, p. 1–21. External Links: ISSN 1346-7581, Document Cited by: §1. [48] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §5.1, §5.3. [49] Y. Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He, Q. Chen, H. Zhou, Z. Zou, H. Zhang, S. Hu, Z. Zheng, J. Zhou, J. Cai, X. Han, G. Zeng, D. Li, Z. Liu, and M. Sun (2024) MiniCPM-v: a gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800. Cited by: §5.3. [50] Z. Ye, S. Liu, Z. Tian, Y. Chen, L. Zheng, and J. Chen (2025-01) Graph-RWGAN: A Method for Generating House Layouts Based on Multi-Relation Graph Attention Mechanism. Buildings 15 (19), p. 3623. External Links: ISSN 2075-5309, Document Cited by: §2. [51] J. Yin, P. Zeng, H. Sun, Y. Dai, H. Zheng, M. Zhang, Y. Zhang, and S. Lu (2025-07) FloorPlan-LLaMa: Aligning Architects’ Feedback and Domain Knowledge in Architectural Floor Plan Generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), p. 6640–6662. External Links: Document, ISBN 979-8-89176-251-0 Cited by: §2, §5.3, §6. [52] J. Yin, P. Zeng, J. Zhong, P. Li, M. Zhang, R. Luo, and S. Lu (2025) FloorPlan-deepseek (fpds): a multimodal approach to floorplan generation using vector-based next room prediction. arXiv preprint arXiv:2506.21562. Cited by: §2. [53] P. Zeng, W. Gao, J. Yin, P. Xu, and S. Lu (2024-06) Residential floor plans: Multi-conditional automatic generation using diffusion models. Automation in Construction 162, p. 105374. External Links: ISSN 0926-5805, Document Cited by: §2, §2. [54] P. Zeng, J. Yin, M. Zhang, Y. Dai, J. Li, Z. Jin, and S. Lu (2025-11) CARD: cross-modal agent framework for generative and editable residential design. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), p. 9304–9319. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §2. [55] P. Zeng, J. Yin, M. Zhang, J. Li, Y. Zhang, and S. Lu (2025-10) Unified residential floor plan generation with multimodal inputs. Automation in Construction 178, p. 106408. External Links: ISSN 0926-5805, Document Cited by: §2. [56] H. Zhang, A. Savov, and B. Dillenburger (2024-06) MaskPLAN: masked generative layout planning from partial input. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 8964–8973. Cited by: §2. [57] H. Zhang and R. Zhang (2025-09) Generating accessible multi-occupancy floor plans with fine-grained control using a diffusion model. Automation in Construction 177, p. 106332. External Links: ISSN 0926-5805, Document Cited by: §2. [58] X. Zhou, G. Luo, Y. Liao, L. Feng, J. Liu, H. Qi, and K. Li (2025-09) Automated aggregation of dwelling units and traffic cores in high-rise residential floor plans using genetic algorithm and multi-agent cooperative deep Q-network. Automation in Construction 177, p. 106329. External Links: ISSN 0926-5805, Document Cited by: §2. [59] X. Zhuang, P. Zhu, A. Yang, and L. Caldas (2025-06) Machine learning for generative architectural design: Advancements, opportunities, and challenges. Automation in Construction 174, p. 106129. External Links: ISSN 0926-5805, Document Cited by: §1. Appendix A Implementation Details A.1 Room-Instance Tokenization We employ two lightweight VQ-VAE branches (Fig. A.1) to discretize floor plan geometry into spatial tokens: an outline branch that encodes the global building boundary, and a conditional room branch that encodes each room conditioned on its corresponding outline. Specifically, the room encoder concatenates the outline mask as an additional input channel to preserve adjacency and boundary consistency. Together, these branches produce a hierarchical spatial vocabulary serving as the foundation for multimodal alignment and instruction tuning. Figure A.1: VQ-VAE tokenization framework. The outline branch encodes the global boundary, while the conditional room branch encodes each room with outline context to capture spatial relations. We study reconstruction fidelity with respect to token granularity and codebook capacity. As shown in Tables A.1 and A.2, token count strongly influences downstream sequence length and contextual reasoning ability: 2×22× 2 tokens underfit geometry, while 16×1616× 16 tokens generate unnecessarily long sequences. Hence, we focus on 4×44× 4 and 8×88× 8 configurations for practical balance. Table A.1: Outline branch: PSNR (dB) and SSIM under different token grids and codebook sizes. tokensN_ tokens Codebook PSNR SSIM 4×44× 4 256 20.003 0.911 4×44× 4 512 22.188 0.946 4×44× 4 1024 23.202 0.957 8×88× 8 256 35.898 0.998 8×88× 8 512 36.657 0.998 8×88× 8 1024 37.053 0.998 Table A.2: Room branch: PSNR (dB) and SSIM under different token grids and codebook sizes. tokensN_ tokens Codebook PSNR SSIM 4×44× 4 256 31.896 0.994 4×44× 4 512 33.106 0.995 4×44× 4 1024 34.506 0.997 8×88× 8 256 44.624 1.000 8×88× 8 512 48.467 1.000 8×88× 8 1024 51.769 1.000 Since each outline corresponds to multiple room instances, the room-level data volume is substantially larger, necessitating a smaller learning rate and fewer epochs for stable convergence. Considering the trade-off between fidelity and efficiency, 8×88× 8 tokens deliver clearly superior reconstruction to 4×44× 4 while avoiding the context-length overhead of 16×1616× 16. Although increasing the codebook size yields modest gains, a compact codebook of 256 effectively balances information density and vocabulary size. Concretely, both branches adopt a latent dimension of 256 with three downsampling layers. The outline VQ-VAE is trained for 50 epochs with batch size 256 and learning rate 3×10−43× 10^-4, while the conditional room VQ-VAE is trained for 30 epochs with batch size 256 and learning rate 1×10−41× 10^-4. A potential concern is whether the VQ-VAE discretization introduces an information bottleneck that limits downstream generation quality. To assess this, we first examine reconstruction fidelity. As the reconstruction PSNR is controlled around 40 dB, it is already close to a near-lossless level for layout geometry. Qualitative comparisons in Fig. A.1 show that reconstructed layouts are almost indistinguishable from the original inputs, indicating minimal information loss during tokenization. We further conduct a codebook size ablation study to evaluate whether generation performance is sensitive to token vocabulary capacity. Results are summarized in Table A.3. Across codebook sizes ranging from 256 to 1024, generation metrics (Macro IoU, FID, GED, Node F1, and Edge Overlap) remain largely stable. The performance variation is marginal, suggesting that the discretized latent space preserves sufficient structural information for downstream reasoning. These results indicate that the VQ-VAE module does not constitute a performance bottleneck in our pipeline. Table A.3: Codebook size ablation study Size Ma. IoU FID↓ GED↓ Node F1 Edge Ovl. 256 0.654 1.89 1.03 0.994 0.880 512 0.657 1.95 1.04 0.995 0.877 1024 0.657 1.97 1.00 0.994 0.881 A.2 Multimodal Alignment and Instruction Tuning LLM training is conducted on Ubuntu 22.04.5 LTS with an AMD EPYC 7K62 48-Core CPU (96 threads), 256 GB RAM, and an NVIDIA GeForce RTX 5090 GPU (32 GB VRAM). All experiments are implemented using the LLaMA-Factory framework with the Qwen3-0.6B backbone and FlashAttention-2 acceleration. To integrate spatial information, we extend the original tokenizer by manually appending a set of discrete outline and room tokens derived from the VQ-VAE codebooks, enabling the LLM to process geometric and semantic content within a unified vocabulary. Each multimodal sample contains up to 2048 tokens, including spatial tokens and textual instructions. The model is trained in three sequential stages following the pipeline described in Sec. 4.2: embedding initialization, multimodal pre-training, and instruction tuning. A cosine learning-rate schedule with 10% warm-up is adopted throughout. Early stages employ a small effective batch size (4 with gradient accumulation) to stabilize optimization under long-sequence multimodal inputs, while Stage 3 uses a larger batch size (16) for instruction-level generalization. Learning rates are set to 1×10−51× 10^-5 for Stages 1–2 and 2×10−52× 10^-5 for Stage 3, with 1, 2, and 3 epochs respectively. A.3 Baseline Methods To ensure a fair comparison, all baseline methods are reproduced or standardized under a unified training and inference pipeline. All models are evaluated at 256×256256× 256 resolution with wall-boundary restoration, and inference is conducted on a single RTX 3090 GPU (an Intel Xeon E5-2682 v4 CPU and 32 GB RAM). Understanding. For multimodal reasoning baselines, including LLaVA-v1.6-Mistral-7B-HF, Qwen3-VL-8B-Instruct, InternVL3.5-8B, and MiniCPM-V 4.5, we use their official checkpoints and serve them through vLLM for efficient inference. Each model receives the color-mapped layout and text instruction, producing structured descriptions for evaluating room-type classification, localization accuracy, and relational reasoning. The prompt template used for understanding is as follows: Prompt Template for Understanding: You are an expert in architectural layout understanding. The input is a color-coded floor plan image with an overall size of 18m × 18m. Each color approximately represents a functional area, and the full layout fits within this boundary. Color-to-room mapping: Light khaki or pale yellow → Livingroom (large open space) Orange → Masterroom (private sleeping area) Light salmon or red → Kitchen (food preparation area) Light cyan → Bathroom (toilet or shower area) Violet → Diningroom (area for meals) Plum → Storage (small enclosed area) Bright yellow → Commonroom (Secondroom / Studyroom / Childroom / Guestroom) Olive green → Balcony (semi-outdoor area) Black → Exterior wall or building boundary White → Front door, main entrance, interior walls, interior doors, and external background Example: Below is an example floor plan analysis. Input image: [example_floorplan.png] Output JSON: ’rooms’: [ ’idx’: 0, ’type’: ’LivingRoom’, ’area’: 33, ’width’: 6, ’height’: 9, ’position’: ’east’, ’idx’: 1, ’type’: ’SecondRoom’, ’area’: 12, ’width’: 3, ’height’: 4, ’position’: ’northwest’, ’idx’: 2, ’type’: ’MasterRoom’, ’area’: 12, ’width’: 3, ’height’: 5, ’position’: ’southwest’, ’idx’: 3, ’type’: ’StudyRoom’, ’area’: 11, ’width’: 3, ’height’: 4, ’position’: ’north’, ’idx’: 4, ’type’: ’Bathroom’, ’area’: 4, ’width’: 2, ’height’: 2, ’position’: ’west’, ’idx’: 5, ’type’: ’Kitchen’, ’area’: 4, ’width’: 2, ’height’: 2, ’position’: ’northeast’ ], ’edges’: [ ’room1’: 5, ’room2’: 3, ’relation’: ’right-of’, ’text’: ’Kitchen is right-of StudyRoom’, ’room1’: 5, ’room2’: 0, ’relation’: ’above’, ’text’: ’Kitchen is above LivingRoom’, ’room1’: 1, ’room2’: 3, ’relation’: ’left-of’, ’text’: ’SecondRoom is left-of StudyRoom’, ’room1’: 1, ’room2’: 4, ’relation’: ’above’, ’text’: ’SecondRoom is above Bathroom’, ’room1’: 4, ’room2’: 2, ’relation’: ’above’, ’text’: ’Bathroom is above MasterRoom’ ], ’description’: ’The floor plan centers around the spacious Living Room, with surrounding rooms arranged by clear spatial logic.’ Now follow this example format for the next images. Instruction: Provide both room (node) attributes and spatial (edge) relations in JSON format. Your task: - Identify all rooms (nodes) within the 18m × 18m plan. - For each room, estimate attributes: - idx (int): unique ID - type (str): functional category (e.g., LivingRoom, Kitchen) - area (float): area in square meters - width, height (float): dimensions in meters - position (str): coarse location in the 18m × 18m layout (e.g., north, center, southeast) - Infer spatial relations (edges) between rooms using ONLY the following edge types: [’left-above’, ’left-below’, ’left-of’, ’above’, ’inside’, ’surrounding’, ’below’, ’right-of’, ’right-above’, ’right-below’] - Each edge must: - reference valid room indices (room1, room2) - use exactly one relation from the list above as "relation" - include a short human-readable "text" Semantics (guidance): - "left-of"/"right-of"/"above"/"below": strict axis-aligned relations. - "left-above", "left-below", "right-above", "right-below": diagonal/oblique relations combining horizontal and vertical. - "inside": room1 is fully inside room2 (room2 acts as container). - "surrounding": room1 encloses or wraps around room2 (the inverse of "inside"). Output format (must be valid JSON; use double quotes): "rooms": ["idx": 0, "type": "RoomType", "area": 0.0, "width": 0.0, "height": 0.0, "position": "center"], "edges": ["room1": 0, "room2": 1, "relation": "left-of", "text": "Room0 is left-of Room1"], "description": "One-sentence summary of the layout." Requirements: - Use only the allowed edge types listed above. - All dimensions/positions are interpreted relative to the 18m × 18m plan. - Output a single valid JSON object and nothing else. Generation. Tell2Design and ChatHouseDiffusion are reproduced using their released implementations. Tell2Design performs direct text-to-layout generation, while ChatHouseDiffusion employs diffusion sampling with language conditioning. Since the official FloorPlan-LLaMA is not publicly available, we reproduce it using a VQ-VAE tokenizer and Qwen backbone. Floor plans are converted to grayscale and tokenized with 16×1616× 16 latent grids, codebook size 1024, latent dimension 256. The tokenizer reaches approximately 38.1 dB PSNR and 0.99 SSIM, confirming high geometric fidelity before language alignment. Qwen-Image-Edit-2509 is also included as a generation baseline. The prompt template used for generation is: Prompt Template for Generation: You are an expert in architectural layout generation. The input image is a black building outline. Please generate a complete flat color-blocked floor plan within the provided outline. - Keep the wall boundaries fixed and fill enclosed regions with appropriate colors. - Each room or functional area should be represented by a solid color block according to the color legend. - The generated layout must align with the spatial description below. - Avoid adding any text, labels, furniture, shadows, or perspective effects. Spatial Description: [SPATIAL_DESCRIPTION] Color Legend: Light khaki (RGB 238 232 170) → Livingroom (large open space), Entrance, Wall-in Orange (RGB 255 165 0) → Masterroom (private sleeping area) Light salmon (RGB 240 128 128) → Kitchen (food preparation area) Light cyan (RGB 173 216 210) → Bathroom (toilet or shower area) Olive green (RGB 107 142 35) → Balcony (semi-outdoor area) Violet (RGB 218 112 214) → Diningroom (area for meals) Plum (RGB 221 160 221) → Storage (small enclosed area) Bright yellow (RGB 255 215 0) → Commonroom / Secondroom / Studyroom / Childroom / Guestroom Black (RGB 0 0 0) → Exterior wall or building boundary White (RGB 255 255 255) → Front door / main entrance / interior walls / interior doors / external background Output Goal: Produce a visually clean and semantically accurate colored floor plan image that fits exactly within the input outline. Editing. For structure-aware layout editing, we evaluate FLUX.1-Kontext-dev and Qwen-Image-Edit-2509 using the original layout and a textual edit command. The prompt template for editing is: Prompt Template for Editing: You are an expert in architectural floor plan editing. The input image is a color-coded floor plan representing functional areas with flat color blocks. Your task is to edit the given layout based on the following instruction, while maintaining architectural coherence and color consistency. Editing Guidelines: - Follow the edit instruction carefully (e.g., add, remove, move, merge, resize rooms). - Preserve unrelated regions and overall wall boundaries. - Use the same color scheme for all room types as specified in the legend. - Maintain clean edges and closed regions without overlapping or blending. - Do not add any text, furniture, or decorations. - Ensure that all colors remain consistent with the color legend below. Edit Instruction: [EDIT_INSTRUCTION] Color Legend: Light khaki (RGB 238 232 170) → Livingroom (large open space), Entrance, Wall-in Orange (RGB 255 165 0) → Masterroom (private sleeping area) Light salmon (RGB 240 128 128) → Kitchen (food preparation area) Light cyan (RGB 173 216 210) → Bathroom (toilet or shower area) Olive green (RGB 107 142 35) → Balcony (semi-outdoor area) Violet (RGB 218 112 214) → Diningroom (area for meals) Plum (RGB 221 160 221) → Storage (small enclosed area) Bright yellow (RGB 255 215 0) → Commonroom / Secondroom / Studyroom / Childroom / Guestroom Black (RGB 0 0 0) → Exterior wall or building boundary White (RGB 255 255 255) → Front door / main entrance / interior walls / interior doors / external background Output Goal: Return an updated floor plan image with only the described edits applied, preserving the rest of the layout unchanged. Appendix B Evaluation Metrics We evaluate model performance for three tasks using unified geometric and structural metrics. Each floor plan is a 256×256256×256 color-coded layout corresponding to an 18×1818×18 m area. B.1 Understanding Metrics Success Rate (Success). Success=NsuccessNtotal,Success= N_successN_total, (B.1) where NsuccessN_success and NtotalN_total are the numbers of successful and total samples. Room Match Rate (RMR). RMR=1N∑i=1NI(ti=t^i),RMR= 1N _i=1^NI(t_i= t_i), (B.2) where N is the number of rooms, tit_i and t^i t_i are the ground-truth and predicted room types, and I(⋅)I(·) equals 1 if the condition is true and 0 otherwise. Room Location Accuracy (LocAcc). LocAcc=1N∑i=1NI(pi=p^i),LocAcc= 1N _i=1^NI(p_i= p_i), (B.3) where pip_i and p^i p_i denote the coarse positions of room i (e.g., north, south, east, west, or center). Room Area Difference (AreaDiff). AreaDiff=1N∑i=1N|Ai−A^i|,AreaDiff= 1N _i=1^N|A_i- A_i|, (B.4) where AiA_i and A^i A_i are the ground-truth and predicted areas (in m2). Room Adjacency Accuracy (AdjAcc). AdjAcc=|E∩E^||E∪E^|AdjAcc= |E∩ E||E∪ E| (B.5) where E and E E denote the ground-truth and predicted adjacency sets. Spatial Relation Accuracy (RelAcc). RelAcc=1|R|∑(i,j)∈RI(rij=r^ij),RelAcc= 1|R| _(i,j)∈ RI(r_ij= r_ij), (B.6) where R is the set of all adjacent ordered pairs with directional relations (e.g., left-of, above, below), and rijr_ij and r^ij r_ij represent their ground-truth and predicted relation types. B.2 Generation Metrics Micro IoU. IoUmicro=∑c=1C|Mcpred∩Mcgt|∑c=1C|Mcpred∪Mcgt|,IoU_micro= _c=1^C|M_c^pred∩ M_c^gt| _c=1^C|M_c^pred∪ M_c^gt|, (B.7) where McpredM_c^pred and McgtM_c^gt are binary masks for category c. Macro IoU. IoUmacro=1C∑c=1C|Mcpred∩Mcgt||Mcpred∪Mcgt|.IoU_macro= 1C _c=1^C |M_c^pred∩ M_c^gt||M_c^pred∪ M_c^gt|. (B.8) Macro-IoU balances room frequencies by averaging across categories. Structural Similarity (SSIM). SSIM=(2μxμy+C1)(2σxy+C2)(μx2+μy2+C1)(σx2+σy2+C2),SSIM= (2 _x _y+C_1)(2 _xy+C_2)( _x^2+ _y^2+C_1)( _x^2+ _y^2+C_2), (B.9) where μx,μy _x, _y are mean intensities, σx,σy _x, _y variances, and σxy _xy covariance. C1,C2C_1,C_2 are small constants for stability. Peak Signal-to-Noise Ratio (PSNR). PSNR=10log10⁡(L2MSE),PSNR=10 _10\! ( L^2MSE ), (B.10) where L is the maximum intensity (255) and MSEMSE is the mean squared error. Fréchet Inception Distance (FID). FID=‖μr−μg‖22+Tr(Σr+Σg−2(ΣrΣg)1/2),FID=\| _r- _g\|_2^2+Tr( _r+ _g-2( _r _g)^1/2), (B.11) where (μr,Σr)( _r, _r) and (μg,Σg)( _g, _g) are the feature means and covariances of real and generated layouts. Graph Edit Distance (GED). GED(Gr,Gg)=minπ∑o∈πc(o),GED(G_r,G_g)= _π _o∈πc(o), (B.12) where GrG_r and GgG_g are room–adjacency graphs, π is a sequence of edit operations (node/edge insertion, deletion, or substitution), and c(o)c(o) is the edit cost. We use the graph_edit_distance() function in NetworkX. Node F1. P=|Tpred∩Tgt||Tpred|,R=|Tpred∩Tgt||Tgt|,P= |T_pred∩ T_gt||T_pred|, R= |T_pred∩ T_gt||T_gt|, (B.13) NodeF1=2PRP+R,Node~F1= 2PRP+R, (B.14) where TpredT_pred and TgtT_gt are the sets of predicted and ground-truth room types, and P, R are precision and recall. Edge Overlap. EdgeOverlap=|Epred∩Egt||Epred∪Egt|,Edge~Overlap= |E_pred∩ E_gt||E_pred∪ E_gt|, (B.15) which measures the proportion of correctly predicted adjacency relations, reflecting structural consistency. B.3 Editing Metrics Edit-Detection IoU (Δ ). MgtΔ=T(Maftergt−Mbefore),MpredΔ=T(Mafterpred−Mbefore),M _gt=T(M^gt_after-M_before), M _pred=T(M^pred_after-M_before), (B.16) ΔIoU=|MgtΔ∩MpredΔ||MgtΔ∪MpredΔ|, = |M _gt∩ M _pred||M _gt∪ M _pred|, (B.17) where T(⋅)T(·) denotes thresholding of the difference map to obtain the changed region. During computation, pixels corresponding to wall-line areas are excluded to avoid non-editable regions affecting the metric. Diff-MSE (Δ ). ΔMSE=1N∑p=1N(MgtΔ(p)−MpredΔ(p))2, = 1N _p=1^N (M _gt(p)-M _pred(p) )^2, (B.18) where p indexes pixels and N is the total number of pixels. Structural Consistency. GED, Node F1, and Edge Overlap are reused to measure whether the edited layout maintains correct topology. B.4 Metric Significance Structural metrics (GED, Node F1, Edge Overlap) and class-balanced Macro-IoU capture spatial topology and functional balance. Perceptual metrics (SSIM, PSNR, FID) assess visual realism, while edit-specific metrics (Δ , Δ ) evaluate the controllability and precision of modifications. For floor plan tasks, structural metrics provide more meaningful insights into spatial reasoning and design consistency, while pixel-level metrics serve primarily as complementary references. Appendix C Post-processing and Effect Analysis C.1 Post-processing Pipeline To ensure clean topology, structural consistency, and standardized color encoding, all generated floor plans are refined through a unified three-stage pipeline implemented with PyTorch, OpenCV, and NumPy. This end-to-end process converts discrete token predictions into visually coherent, evaluation-ready layouts. Floor Plan Rendering. During the generation stage, model predictions in tokenized form are converted into complete color layouts through a rendering routine implemented in PyTorch. Conditioned on the building outline, each room is decoded into a binary occupancy map and overlaid in descending order of area to form the full layout. Each functional region is assigned a base color from the predefined legend, with slight random jitter to enhance visual distinction. The resulting layout contains black outer walls, white inner separators, and solid-colored functional areas, serving as the standardized input for the subsequent structural correction and standardization steps. Structural Correction. Ambiguous gray zones and small irregularities are first removed by replacing undefined pixels with the dominant color of neighboring regions. Morphological open–close operations are then applied within each connected component to smooth protruding edges while preserving room boundaries and overall spatial integrity. Standardization and Contour Refinement. After correction, each pixel is reassigned to the nearest entry in the predefined color map to ensure consistent semantic encoding across samples. Contours are redrawn with black outer walls and white inner separators to enhance visual clarity, and all layouts are resized to 256×256256× 256 pixels using nearest-neighbor interpolation to maintain discrete color values. This three-stage pipeline is uniformly applied across all generation and editing experiments for every compared method, ensuring consistent color alignment, structural fidelity, and fair quantitative evaluation under a unified protocol, as illustrated in Fig. C.1. Figure C.1: Post-processing examples. C.2 Effect of Structural Correction To evaluate the impact of the structural correction step, we compare generation metrics before and after applying it, while keeping the standardization process fixed for both settings. As shown in Table C.1, structural correction further improves geometric consistency and perceptual clarity by smoothing irregular edges and resolving ambiguous regions near boundaries. Table C.1: Impact of Structural Correction in HouseMind-O. Metric w/o Correction w/ Correction Micro IoU 0.710 0.710 Macro IoU 0.654 0.654 SSIM 0.887 0.887 PSNR 16.0 16.1 FID↓ 1.87 1.89 GED↓ 1.10 1.03 Node F1 0.994 0.994 Edge Overlap 0.873 0.880 The corrected results exhibit more coherent room boundaries and reduced local noise, as reflected by lower GED and higher Edge Overlap. Appendix D Result Analysis and Discussion D.1 Stability Analysis To evaluate the robustness and consistency of different models, we visualize the distribution of evaluation metrics across the entire test set using HouseMind-O. Figure D.1: Result distribution across tasks. HouseMind-O maintains stable, high-performance distributions in understanding and generation, with the bimodal ΔIoU in editing mainly caused by unclear spatial instructions. In the understanding task, HouseMind-O demonstrates accurate prediction of room types, room locations, and room adjacency relations. The Room Area Difference is generally controlled within 2m22\,m^2, indicating stable quantitative reasoning of room size. However, the prediction of spatial relations between rooms remains relatively weaker, mainly due to minor inconsistencies in the definition of inter-room positional relationships within the training dataset. In the generation task, pixel-level metrics exhibit more uniform distributions while maintaining consistently high values without significant outliers. For structural metrics, most generated floor plans achieve Node F1=1Node~F1=1, demonstrating nearly perfect alignment between predicted and target room types. More than half of the samples also reach Edge Overlap=1Edge~Overlap=1 and GED=0GED=0, indicating precise prediction of room topological relations. Only a few cases present slight deviations, which can be further improved in subsequent training stages. In the editing task, the distribution of ΔIoU shows a distinct bimodal pattern. This arises from a subset of editing prompts that do not explicitly specify the insertion position of the new room, leading to ambiguous geometric modifications. When the editing instruction clearly describes both the target room location and size, the predicted results are generally accurate and consistent with the ground truth. D.2 Pixel-Structure Coupling Analysis We further analyze the relationship between pixel-level accuracy (Macro IoU) and structural consistency (Edge Overlap) across three generation methods. As shown in Fig. D.2, FloorPlanLLaMA (our re-implementation), which encodes the entire layout as a single image sequence, and ChatHouseDiffusion, which performs global image-to-layout diffusion generation, both exhibit relatively strong correlations (r=0.70r=0.70 and r=0.63r=0.63, respectively) between the two metrics. This indicates that pixel-level alignment and structural accuracy are inherently interdependent in holistic generation. As models pursue higher pixel fidelity, their topological precision also increases; however, this often comes at the cost of reduced flexibility in structural reasoning, as they overfit to visual similarity. In contrast, HouseMind-O, equipped with room tokenization, shows a lower correlation (r=0.57r=0.57), indicating that the pixel and topological aspects are partially decoupled. Most data points lie in the upper region of the plot, where structural consistency remains high, even when pixel IoU slightly decreases. This reflects the model’s priority in maintaining accurate topological relations across rooms, which better aligns with practical architectural requirements. Overall, the reduced coupling in HouseMind-O demonstrates that room-wise tokenization enables the model to reason about spatial topology independently from pixel reconstruction, achieving structure-aware generation that is both controllable and semantically consistent. Figure D.2: Pixel–structure coupling analysis. Scatter plots show the correlation between Macro IoU and Edge Overlap for three generation paradigms. D.3 Methodological Comparison and Discussion Beyond quantitative metrics, we compare the underlying paradigms adopted by representative model families to clarify the advantages of the proposed HouseMind. General-purpose multimodal models such as GPT-5 integrate diffusion-based visual modules for open-domain image generation and editing. While these models excel in semantic reasoning and zero-shot transfer, they lack geometric and topological priors, leading to spatially invalid or unstructured layouts. Autoregressive text-to-layout generators like FloorPlanLLaMA encode entire floor plans as sequential token streams. This paradigm ensures high pixel fidelity and global semantic coherence but suffers from limited controllability and no editing capability, as the generation order constrains spatial reasoning. Diffusion-based holistic generators, exemplified by ChatHouseDiffusion, produce visually realistic and smooth layouts through iterative denoising. However, their reliance on pixel-level reconstruction often causes inconsistent topology and unstable room adjacency, which undermines functional correctness. In contrast, the proposed HouseMind adopts a room-tokenized multimodal reasoning paradigm, jointly modeling text, geometry, and topology within a unified LLM framework. This design enables room-wise controllability and structural preservation, achieving consistent topology even when visual pixel alignment fluctuates. Unlike holistic approaches, HouseMind can independently reason about spatial structure while maintaining semantic coherence across understanding, generation, and editing tasks. Overall, this comparison highlights a paradigm shift from visually driven diffusion or autoregressive generation toward structure-aware multimodal reasoning, representing a critical step toward advancing controllable architectural layout generation. Appendix E Prompts for Generation Cases The following prompts correspond to the generation results illustrated in Fig. 4. Each prompt describes the intended spatial relationships and functional layout used as textual input for the generation task. Prompt G1 The layout centers around a spacious Living Room, occupying the central area of the floor plan with an area of 26 square meters. To the west of the Living Room lies the Master Room, which connects directly to it and also extends slightly above the Balcony situated to the south. The Kitchen, positioned in the northwest corner, is smaller in size at 6 square meters and connects to both the Living Room on its right and the Bathroom directly below it. The Bathroom, located just northwest of center and slightly west of the Living Room, is adjacent to both the Kitchen above and the Master Room below, forming a vertical sequence of rooms along the western side. To the northeast, the Second Room sits comfortably next to the Living Room, sharing a direct right-of relationship with it. Finally, the Balcony, positioned directly south of the Living Room, spans the southern edge of the plan and lies below both the Living Room and the Master Room, which projects slightly over it from the northwest. All rooms are efficiently arranged, with clear adjacency and vertical alignment enhancing the flow between functional zones. Prompt G2 Create a house layout that reflects the described spatial logic. The layout centers on a spacious LivingRoom, which is flanked to the left by the SecondRoom and to the right by the MasterRoom. Above the LivingRoom, toward the northeast, sit the Kitchen and Bathroom side by side, with the Kitchen positioned to the left of and slightly above the Bathroom. The Bathroom lies directly above the MasterRoom, while the Kitchen extends slightly northeastward, with a narrow Balcony positioned to its right and above the MasterRoom. A larger Balcony stretches along the southern edge of the LivingRoom, connecting to both the MasterRoom and Kitchen via diagonal relationships. The overall arrangement creates a functional, tiered flow from the central living area to private and service spaces. Prompt G3 The layout centers on a spacious Living Room, occupying 36 square meters and positioned at the heart of the floor plan. To the north, the Kitchen (4 sq. meters) sits directly above the Living Room and is connected to a smaller Balcony (2 sq. meters) that extends above it, with the Kitchen positioned just below this northern balcony. To the northwest, the Second Room (10 sq. meters) lies above the Living Room and to the left of the Kitchen, with a diagonal connection beneath it to the northern balcony. Directly south of the Living Room is the Master Room (8 sq. meters), which also connects to a larger Balcony (6 sq. meters) situated below it. To the east of the Living Room lies the Bathroom (8 sq. meters), positioned to its right, with the Master Room lying diagonally below and to the left of it. The two balconies—smaller to the north and larger to the south—frame the upper and lower edges of the layout, with the northern balcony adjacent to both the Kitchen and Second Room, and the southern balcony extending beneath the Master Room and aligned with the Living Room’s southern edge. All rooms are arranged in a cohesive, vertically and horizontally connected sequence, with the Living Room serving as the central hub linking all other spaces. Appendix F Generalization and Real-World Deployment This section provides a more detailed analysis of the generalization ability and robustness of HouseMind beyond the standard in-distribution evaluation. F.1 Modeling Design for Generalization From a modeling perspective, HouseMind does not directly learn raw room distributions in pixel or coordinate space. Instead, spatial layouts are first discretized through a VQ-VAE encoder, which transforms continuous geometric configurations into structured spatial tokens. This representation abstracts away low-level coordinate noise and encourages the model to learn higher-level spatial relationships (e.g., adjacency, functional grouping, alignment patterns). By operating in this tokenized latent space, the LLM captures relational and compositional regularities rather than memorizing specific layout instances. This representation-driven learning mechanism enables the model to generalize beyond layouts that strictly follow the empirical training distribution. F.2 Out-of-Distribution Evaluation To further evaluate generalization, we construct a deliberately more challenging out-of-distribution (OOD) test set for generation. Compared with the standard test split, this OOD set includes: • Studio layouts (no separate bedroom); • Layouts with two living rooms; • Layouts including uncommon room types (e.g., wine cellar treated as storage); • Logically invalid configurations (e.g., centrally located balcony); • Irregular building outlines; • Prompts without explicitly specified room types. Representative examples are shown in Fig. F.1. Figure F.1: Results of HouseMind on out-of-distribution cases. Results indicate that HouseMind maintains structurally consistent generation across most uncommon layouts and room-type configurations. Even when room combinations deviate from typical training patterns, the model preserves global structural coherence and largely respects adjacency as well as functional grouping constraints. In addition, prompt paraphrasing does not significantly degrade structural validity, suggesting robustness to linguistic variations. Failure cases primarily arise under highly complex or mutually conflicting constraints. In such scenarios, the model tends to favor layouts that align with realistic architectural priors rather than strictly following logically inconsistent or physically implausible instructions (e.g., placing a balcony at the geometric center of the building). This behavior indicates that the model internalizes real-world spatial distributions and architectural regularities, which function as an implicit prior during generation. F.3 Robustness Enhancement in Deployment To further enhance robustness in real-world deployment, we implement several additional strategies in our public platform, available at https://housemind.ai-structure.com/: 1. Structured prompt guidance: predefined room type selections and prompt templates are provided to reduce ambiguity and improve the clarity of user inputs. 2. Intermediate prompt reformulation: an auxiliary LLM reformulates raw user instructions into structured representations that are better aligned with HouseMind’s token space. 3. Post-generation validation: automatic validation of room types, counts, and adjacency relations is performed, with regeneration triggered when inconsistency with the input prompt is detected. These mechanisms collectively improve robustness under noisy, incomplete, or partially inconsistent user inputs, while preserving structural validity and generative diversity.

Instant research discovery