Paper deep dive
Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers
Atsuyuki Miyai, Mashiro Toyooka, Zaiying Zhao, Kenta Watanabe, Toshihiko Yamasaki, Kiyoharu Aizawa
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/2/2026, 3:33:28 AM
Summary
The paper introduces Paper Reconstruction Evaluation (PaperRecon), a systematic framework for evaluating AI-driven scientific writing. It uses a benchmark called PaperWrite-Bench, consisting of 51 papers from top-tier venues, to measure presentation quality and hallucination rates in coding agents like ClaudeCode and Codex. The study reveals a trade-off where higher presentation quality often correlates with increased hallucination rates.
Entities (4)
Relation Signals (3)
PaperWrite-Bench â contains â 51 papers
confidence 100% ¡ PaperWrite-Bench, a benchmark of 51 papers from top-tier venues
PaperRecon â evaluates â AI-written papers
confidence 95% ¡ PaperRecon disentangles the evaluation of the AI-written papers into two orthogonal dimensions, Presentation and Hallucination
ClaudeCode â exhibits â Hallucinations
confidence 90% ¡ ClaudeCode achieves higher presentation quality at the cost of more than 10 hallucinations per paper on average
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper introduces the first systematic evaluation framework for quantifying the quality and risks of papers written by modern coding agents. While AI-driven paper writing has become a growing concern, rigorous evaluation of the quality and potential risks of AI-written papers remains limited, and a unified understanding of their reliability is still lacking. We introduce Paper Reconstruction Evaluation (PaperRecon), an evaluation framework in which an overview (this http URL) is created from an existing paper, after which an agent generates a full paper based on the overview and minimal additional resources, and the result is subsequently compared against the original paper. PaperRecon disentangles the evaluation of the AI-written papers into two orthogonal dimensions, Presentation and Hallucination, where Presentation is evaluated using a rubric and Hallucination is assessed via agentic evaluation grounded in the original paper source. For evaluation, we introduce PaperWrite-Bench, a benchmark of 51 papers from top-tier venues across diverse domains published after 2025. Our experiments reveal a clear trade-off: while both ClaudeCode and Codex improve with model advances, ClaudeCode achieves higher presentation quality at the cost of more than 10 hallucinations per paper on average, whereas Codex produces fewer hallucinations but lower presentation quality. This work takes a first step toward establishing evaluation frameworks for AI-driven paper writing and improving the understanding of its risks within the research community.
Tags
Links
- Source: https://arxiv.org/abs/2604.01128v1
- Canonical: https://arxiv.org/abs/2604.01128v1
Trouble viewing inline? Open PDF directly â
Full Text
141,412 characters extracted from source content.
Expand or collapse full text
Preprint. Under review. Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers Atsuyuki MiyaiMashiro Toyooka â Zaiying Zhao â Kenta Watanabe â Toshihiko YamasakiKiyoharu Aizawa The University of Tokyo https://agent4science-utokyo.github.io/PaperRecon_HP d Original PaperGenerated Paper Coding Agent LLM & Human Step0: PreprocessStep1: Writing Section-wise Eval Step2: Evaluation Figure 1: Paper Reconstruction Evaluation. Given minimal resources derived from a origi- nal paper, a coding agent reconstructs the full paper. The generated paper is then compared with the original paper to evaluate writing performance along two complementary axes: presentation quality and hallucination. Abstract This paper introduces the first systematic evaluation framework for quan- tifying the quality and risks of papers written by modern coding agents. While AI-driven paper writing has become a growing concern, rigorous evaluation of the quality and potential risks of AI-written papers remains limited, and a unified understanding of their reliability is still lacking. We introduce Paper Reconstruction Evaluation (PaperRecon), an evaluation framework in which an overview (overview.md) is created from an existing paper, after which an agent generates a full paper based on the overview and minimal additional resources, and the result is subsequently compared against the original paper. PaperRecon disentangles the evaluation of the AI-written papers into two orthogonal dimensions, Presentation and Hallu- cination, where Presentation is evaluated using a rubric and Hallucination is assessed via agentic evaluation grounded in the original paper source. For evaluation, we introduce PaperWrite-Bench, a benchmark of 51 papers from top-tier venues across diverse domains published after 2025. Our experiments reveal a clear trade-off: while both ClaudeCode and Codex improve with model advances, ClaudeCode achieves higher presentation quality at the cost of more than 10 hallucinations per paper on average, whereas Codex produces fewer hallucinations but lower presentation qual- ity. This work takes a first step toward establishing evaluation frameworks for AI-driven paper writing and improving the understanding of its risks within the research community. â Equal contribution 1 arXiv:2604.01128v1 [cs.CL] 1 Apr 2026 Preprint. Under review. 1 Introduction As recent AI tools, exemplified by coding agents, continue to advance, it is increasingly important to rigorously evaluate how they automate the research process and the potential risks they introduce, in order to ensure sustainable AI-driven scientific progress (Miyai et al., 2026). In particular, recent incidents involve submissions of AI-written papers to academic venues (Liang et al., 2025; ICLR Program Chairs, 2025; Neubig, 2025). These submissions are likely to continue increasing alongside the rapid growth of AI Scientists in recent years (Intology, 2025; Weng et al., 2026; Miyai et al., 2026). To advance scientific progress while preserving academic integrity, it is essential for the research community to continuously monitor the progress and risks of AI-driven writing based on rigorous and reliable evaluation. Evaluating the paper writing capability of AI agents is inherently challenging and has not been sufficiently addressed in prior work. Existing approaches have explored using AI reviewers to assess paper quality (Liu et al., 2024; Yamada et al., 2025; Weng et al., 2025; Zhu et al., 2025). However, these methods are inadequate, as they tend to assign higher scores to papers with more severe fabrications (Jiang et al., 2025; Miyai et al., 2026). While hallucinations in AI-written papers have been recognized, prior work has been limited to surface-level issues such as citation errors (Walters & Wilder, 2023; Ansari, 2026; Sakai et al., 2026) or individual hallucination cases (Yamada et al., 2025; Miyai et al., 2026), without enabling systematic evaluation. We propose Paper Reconstruction Evaluation (PaperRecon), the first evaluation framework for measuring the paper writing capabilities of AI agents. The overview of PaperRecon is shown in Figure 1. PaperRecon first compresses an existing paper into a structured summary, research_overview.md, which retains only essential information. Given this compressed representation, along with other minimal resources (tables, figures, bib file), agents are tasked with reconstructing the original paper. This workflow of generating a paper from minimal inputs is equivalent to isolating the writing component of existing AI Scientist systems (Yamada et al., 2025; Weng et al., 2026; Miyai et al., 2026). If agents can reliably reconstruct high-fidelity papers, this provides a strong signal of their writing capabilities. The key strength of PaperRecon as an evaluation framework lies in its ability to enable precise assessment through direct comparison with the original paper. In particular, it decomposes the notion of writing quality into two orthogonal dimensions: Presentation and Hallucination. Presentation is evaluated using rubric evaluation (Fan et al., 2024; Phung et al., 2023; Arora et al., 2025), assessing how faithfully the key elements of the original paper are preserved in the reconstructed paper. Hallucinations are assessed via agentic evaluation grounded in the original paper source, enabling fine-grained detection of factual inconsistencies. This design allows PaperRecon to jointly evaluate both presentation quality and factual correctness in a unified and reliable manner. For PaperRecon, we introduce PaperWrite-Bench, a benchmark constructed from 51 papers published after 2025. PaperWrite-Bench consists of papers from a diverse set of top-tier venues, including NeurIPS, ICLR, CVPR, ICCV, ACL, and ACMMM, providing broad coverage across research domains. This benchmark enables systematic and comprehensive evaluation of modern writing agents across realistic and diverse fields. We evaluate recent powerful and widely used agents, including Claude Code (Anthropic, 2025b), Claude Code Agent Teams (Anthropic, 2026), and Codex (OpenAI, 2025), across a range of underlying models, from Claude Sonnet 4 (Anthropic, 2025a) and Claude Sonnet 4.6 (Anthropic, 2026) to GPT-5 (OpenAI, 2025) and GPT-5.4 (OpenAI, 2026). Our experiments yield the following key findings: 1.Claude Code achieves higher presentation quality than Codex. Claude Code better captures the key elements required for scientific writing across sections. 2. Codex produces fewer hallucinations than Claude Code. While Claude Code exhibits more than 10 hallucinations per paper on average, Codex limits this to around 3. 2 Preprint. Under review. 3.Writing capability improves with model advances. This also suggests that Paper Reconstruction Evaluation serves as a reliable metric for tracking progress in writing ability. Our work makes the following contributions to the research community: â˘Paper Reconstruction Evaluation (PaperRecon): We propose the first evaluation framework for scientific writing, Paper Reconstruction Evaluation, which measures the quality of paper reconstruction from compressed representations, along with a detailed evaluation protocol. ⢠PaperWrite-Bench: We introduce PaperWrite-Bench, a benchmark constructed from recent papers across diverse research domains, enabling comprehensive evaluation of agentsâ ability to reconstruct papers from minimal information. â˘Quantitative analysis of presentation and hallucination: We provide a systematic evaluation of modern agents across both Presentation and Hallucination, quantify- ing how their capabilities evolve with model advances. Our results offer insights into the current state and trade-offs of AI-driven scientific writing. 2 Related Work AI-driven Research Automation. Recent progress in AI has accelerated efforts to automate various stages of the research process (Si et al., 2025b;a; Asai et al., 2026; Novikov et al., 2025; Weng et al., 2025; Villaescusa-Navarro et al., 2025; Mitchener et al., 2025; Zhuang et al., 2025; Lin et al., 2023; Gottweis et al., 2025), as well as end-to-end research pipelines (Lu et al., 2024; Intology, 2025; Tang et al., 2025; Miyai et al., 2026). However, recent work highlights the growing importance of carefully understanding the risks associated with increasingly capable AI scientists (Miyai et al., 2026). In particular, Miyai et al. (2026) reports risks across all stages of the research process, including idea generation, experimentation, writing, and review, and shows that the writing stage is especially prone to hallucinations such as inconsistencies with experimental results and fabricated content. In this work, we aim to accurately measure the writing capabilities of modern agents through a novel evaluation framework, Paper Reconstruction Evaluation. Evaluation of AI-written Articles. Evaluation of AI-written articles has been studied in domains beyond scientific papers (Yang et al., 2023; Fitria, 2023; Zhong et al., 2026; Shao et al., 2024). For example, prior work has long explored generating Wikipedia pages (Banerjee & Mitra, 2015; MinguillĂłn et al., 2017; Liu et al., 2018; Fan & Gardent, 2022; Shao et al., 2024). In addition, Zhong et al. (2026) evaluates how well AI systems can write essays by using real GRE writing prompts. However, there are few papers to evaluate AI-driven scientific writing. This difference is likely due to the complexity of scientific writing, which requires justifying the significance of the problem, situating the work within the existing research landscape, designing sound and valid evaluation protocols, ensuring reproducibility and verifiability, and maintaining alignment between claims and supporting evidence. These requirements made earlier AI systems struggle with scientific writing and its evaluation, making it substantially more challenging than tasks such as Wikipedia or essay writing. Hallucination and Evaluation of AI-written Papers. It is widely recognized that papers written by AI systems often contain hallucinations. However, existing studies have been limited to surface-level issues such as citation errors (Walters & Wilder, 2023; Ansari, 2026; Sakai et al., 2026) or individual hallucination cases (Miyai et al., 2026), without enabling systematic evaluation. This limitation largely stems from the absence of a well-defined protocol for assessing the substantive content of generated papers. As a result, existing evaluations of AI-written scientific papers have primarily relied on review-based assessment, focusing on whether a paper surpasses the acceptance threshold of academic venues (Lu et al., 2024; Yamada et al., 2025; Weng et al., 2025; Zhu et al., 2025). However, prior work has shown that AI reviewers often fail to detect hallucinations, leading to cases where papers with more severe hallucinations receive higher scores (Jiang et al., 2025; Miyai et al., 2026). While AI-Researcher (Tang et al., 2025) shares a similar perspective in generating new papers from source materials of existing research, it evaluates novelty, methodological 3 Preprint. Under review. Original PaperGenerated Paper Abstract Introduction Related Work +3, +2, +5 +4, +2, +5 +2, +2, +2 Avg. 3.0 Abstract Introduction Related Work rubric Original Paper Generated Paper Abstract Introduction Related Work Original Full Paper LLM Agent Original Full Paper Full Resources Code Figs, Tables Overview. md Supported Neutral Contradictory Supported Neutral Contradictory (a) Rubric Evaluation(b) Agentic Evaluation Presentation Evaluation Hallucination Evaluation Figure 2: Overview of PaperRecon evaluation pipeline. Our evaluation compares gen- erated papers against GT (original) papers along two complementary axes: Presentation evaluation with rubric and agentic hallucination evaluation validity, and empirical performance, making them fundamentally different from writing evaluation. Therefore, to accurately understand the risks of AI-driven writing, it is essential to move beyond review-based evaluation and establish an evaluation protocol that directly assesses both presentation quality and hallucination. 3 Paper Reconstruction Evaluation 3.1 Problem Definition Paper Reconstruction Evaluation (PaperRecon) is a framework for evaluating how accurately coding agents can reconstruct scientific papers. From each original paper, we extract the following information and provide it to the agent: (1)Research Overview: A Markdown file summarizing the motivation, method, and key experimental results of the paper. (2)Figure: Figures from the original paper with simplified captions (e.g., fig_method.jpg: Method Overview.). (3)Table: LaTeX source code of tables from the original paper with simplified captions (e.g., table_cb.tex: Main Result.). (4) References: The bibliography file of the original paper, where each entry is augmented with its abstract. (5) Code: The codebase associated with the original paper, if available. Given these inputs, the agent is tasked with reconstructing the original paper. The generated paper is then compared against the original from multiple perspectives to evaluate writing capability. Here, we reuse the original references instead of requiring agents to construct them from scratch, as accurate reference collection is itself a separate research problem. This design allows us to isolate and evaluate the core writing ability of the agent. 3.2 Evaluation Protocol We evaluate generated papers by comparing them against ground-truth (GT) papers along three complementary axes: Presentation evaluation with rubric, agentic hallucination eval- uation, and citation-level evaluation. Figure 2 shows the overview of presentation and hallucination evaluation pipelines. To compute these scores, we first perform section classi- fication and matching, then evaluate each matched pair. 3.2.1 Section Classification and Matching Papers organize sections differently, so we first extract all sections from both the GT and generated LaTeX files. Next, we map them into seven common categories: Abstract, Introduc- 4 Preprint. Under review. tion, Method, Benchmark Construction, Experiment, Related Work, and Conclusion. Classification first applies keyword-based rules (e.g., âpreliminaryââMethod, âablation studiesââEx- periment); sections that cannot be classified by rules are passed to an LLM for classification based on section name and content. When multiple raw sections map to the same category, they are merged. This classification is performed once and shared across all subsequent evaluation steps. 3.2.2 Rubric Evaluation To enable accurate and fine-grained evaluation, we adopt a rubric evaluation (Fan et al., 2024; Phung et al., 2023; Arora et al., 2025). We found in preliminary experiments that LLM-as-a-judge yielded low discriminative power in evaluation, whereas rubric evaluation enables more discriminative assessment. For each GT paper, we pre-construct a rubric that specifies key elements expected in each section, along with their relative importance and descriptions. Each rubric element corre- sponds to a concrete and verifiable point (e.g., âProblem motivation: improve reasoning in vision-language modelsâ for the Abstract, or âEvaluation covers both visual reasoning and general image understanding benchmarksâ for the Experiment section). These rubrics are initially generated from the GT paper using an LLM (i.e., GPT-5.4 (OpenAI, 2026)), and are subsequently reviewed and, when necessary, refined by the authors to ensure quality and correctness. For each section excluding the Conclusion section, we use an LLM judge with GPT-5.4 to evaluate how well the generated section covers each rubric element on a 1â5 scale: 5: the content is fully and accurately described with correct details; 4: the content is mostly described, with the core idea present but some details missing; 3: the content is partially described, with significant gaps or vagueness; 2: the content is barely mentioned, with only a superficial or indirect reference; 1: the content is completely absent from the generated section. In addition to text-based rubric elements, we incorporate figure- and table-level scores as part of the evaluation rubric. Figure Evaluation. We align GT and figures in the generated papers based on GT Tex file and assess their contextual appropriateness. If both the GT and generated papers reference a figure within the same section, a full context score of 5 is assigned. Otherwise, an LLM evaluates whether the figure is used in an appropriate context on a 1â5 scale. If the generated paper references the figure in any section (even if different from the GT), the LLM-derived score is used; if the figure is not referenced at all, it is assigned a score of 1. Table Evaluation. We extract tables from both GT and generated LaTeX files and perform matching using a hierarchical strategy: label matching, caption matching, and LLM-based matching. For each matched pair, an LLM evaluates numerical accuracy, structural align- ment, and content consistency, producing a match score on a 1â5 scale. Tables present in the GT but missing from the generated paper are assigned a score of 1. The final rubric score for each section is the average across all rubric elements (text, figure, and table items combined). The average number of rubrics for each section is as follows: Ab- stract: 10.3, Introduction: 13.3, Related Work: 12.6, Method: 14.2, Benchmark Construction: 14.6, and Experiment: 14.3. 3.2.3 Hallucination Analysis We identify factual errors via a two-stage, claim-level analysis. Stage 1: Claim Extraction. For each section (excluding the Conclusion), an LLM (i.e., GPT- 5.4) extracts all concrete and verifiable claims from the generated content and classifies each claim into one of the following categories: 1. Supported: directly stated in, or logically derivable from, the GT paper. 5 Preprint. Under review. 2. Neutral: not present in the GT paper, but a reasonable general statement or supple- mentary detail that does not contradict it (note that absence from the GT does not imply contradiction). 3. Contradictory: directly conflicts with specific information in the GT paper. For claims classified as contradictory, we further assign a severity level: major (e.g., incorrect numerical values, fabricated results, or incorrect method descriptions) or minor (e.g., overly strong generalizations, imprecise wording). Stage 2: Verification. All claims labeled as contradictory across sections are aggregated and re-evaluated with a coding agent (i.e., Claude Code with Sonnet4.6). The agent is provided with the GT paper resources (including LaTeX source, codebase, figures, and tables) and re-examines each flagged claim, potentially revising false positives to supported or neutral. This two-stage design reduces false positives while limiting the computational cost to a single agent invocation. Finally, we report the total counts of supported, neutral, major contradictory, and minor contra- dictory claims. 3.2.4 Citation-Level Evaluation While the primary objective of PaperRecon is to assess the content of generated papers, we also evaluate citation usage by comparing citation keys between the GT and generated papers using an F1-based metric. We extract all citation keys in both GT and generated LaTeX files, and compute precision, recall, and F1 score based on the overlap of citation key sets. We additionally detect hallucinated citations (keys cited in the generated paper but absent fromreferences.bib), missing citations (keys in GT but not in the generated paper), and extra citations (keys added by the prediction that are not in GT). 3.2.5 Overall Metrics We report the following aggregate metrics: (i) Avg. Rubric Score: Mean rubric score across all evaluated sections and elements (1â5 scale), including figure and table items. (i) Hallucination Counts: Total number of major contradictory claims detected across all sections after two-stage verification. (i) Citation Scores: Precision (valid cited / total cited), Recall (valid cited GT / total GT), F1 (harmonic mean), Hallucination (num of invalid citations). 4 PaperWrite-Bench 4.1 Benchmark Overview In this section, we introduce PaperWrite-Bench, a benchmark designed for PaperRecon. PaperWrite-Bench consists of 51 papers manually curated by the authors, selected from a diverse set of top-tier conferences, including ACL 2025, EMNLP 2025, CVPR 2025, CVPR 2026, ICCV 2025, ICLR 2025, NeurIPS 2025, ICLR 2026, and ACMMM 2025, covering a wide range of domains such as computer vision, natural language processing, machine learning, and multimedia processing. Among the 51 papers, 32 focus on proposing new methods, 12 introduce new benchmarks, and 7 combine both contributions. This diversity enables comprehensive evaluation of agent writing capabilities across different types of scientific papers. Prior work has constructed similar benchmarks for the reproduction of the experimental results by collecting papers, such as Exp-Bench (56 papers) (Kon et al., 2026) and PaperBench (20 papers) (Starace et al., 2025) (Kon et al., 2026; Hu et al., 2025; Starace et al., 2025). However, these benchmarks are largely based on papers published around 2024 and do not reflect more recent developments. Therefore, we curate PaperWrite-Bench from more recent sources to better reflect the current capabilities of modern agents. 6 Preprint. Under review. Table 1: Presentation evaluation. Rubric evaluation scores by model and section (1â5 scale). We observe that Claude Code achieves higher presentation quality than Codex. AgentModelAbs.Intro.Rel.Meth.Bench.Exp.Avg. CodexGPT54.003.582.322.893.253.533.26 CodexGPT5.44.063.872.723.513.793.643.59 ClaudeCodeSonnet44.103.882.483.233.633.663.49 ClaudeCodeSonnet4.64.374.123.083.693.844.003.86 ClaudeCode-TeamsSonnet4.64.284.053.073.623.993.973.82 4.2 Benchmark Construction Pipeline Constructionofresearch_overview.md.Foreachpaper,wegeneratea research_overview.mdusing GPT-5 (OpenAI, 2025), which summarizes the key in- formation required for reconstruction. To ensure quality, the authors manually verify that the overview contains sufficient information to faithfully reconstruct the original paper. On average, each file contains 463 words. Extraction of Tables, Figures, References, and Code. Using the arXiv source files, we extract tables and figures and store them in dedicated directories (tables/andfigures/). Following prior work (Liu et al., 2024), we provide structured reference information to the agent viatable_summary.txtandfigure_summary.txt, which include file paths and brief descriptions (e.g., the first line of the original caption). To ensure proper use of references, we augment each entry in the.bibfile with abstracts retrieved via the Semantic Scholar API. In addition, we include the associated codebase, when available, to facilitate accurate reconstruction of the proposed methods. When the README.md file in the code contained sections such as an abstract or introduction, those parts were manually removed. Template and Style File Construction. We extract the section structure of each original paper from the arXiv source and create atemplate.tex, along with the necessary style files. Thetemplate.texcontains only the section structure of the original paper, preserving the same section headings while leaving the content empty. Agents are instructed to generate the paper following this predefined structure. This design is motivated by the observation that section organization varies significantly across papers, making direct comparison between generated and original papers difficult. By fixing the section structure, we enable more accurate and consistent evaluation. Moreover, since defining the section structure is a simple preprocessing step, this pipeline remains practical, requiring only minimal human intervention before the writing task is carried out by the agent. 5 Experiments 5.1 Experimental Setup Agents. We evaluate three coding agents: Claude Code (Anthropic, 2025b) (single-agent), Codex (OpenAI, 2025) (single-agent), and Claude Agent Teams (Anthropic, 2026) (multi- agent). Claude Code is evaluated with Sonnet 4 (Anthropic, 2025a) and Sonnet 4.6 (An- thropic, 2026), Codex with GPT-5 (OpenAI, 2025) and GPT-5.4 (OpenAI, 2026), and Claude Agent Teams with Sonnet 4.6, resulting in five agent configurations in total. Writing Pipeline. The goal of this study is to understand how well the current agents can perform scientific writing under a simple setup and what risks arise in the process. Therefore, we adopt a deliberately simple writing pipeline. Our pipeline includes a compilation feedback loop, where LaTeX errors are returned to the agent for correction, as well as a page limit adjustment step following Liu et al. (2024); Yamada et al. (2025). 7 Preprint. Under review. Table 2: Hallucination evaluation. Each score represents the average number of hallucina- tions per paper in each section. We observe that Codex produces fewer hallucinations than Claude Code. AgentModelAbs.Intro.Rel.Meth.Bench.Exp.Total CodexGPT50.30.60.33.81.93.410.2 CodexGPT5.40.10.30.21.30.20.93.0 ClaudeCodeSonnet40.20.50.55.40.84.712.0 ClaudeCodeSonnet4.60.20.80.64.70.53.610.4 ClaudeCode-TeamsSonnet4.60.30.60.83.90.53.89.8 Table 3: Citation evaluation scores. AgentModelPrec.RecallF1Hal. CodexGPT50.890.270.390.0 CodexGPT5.40.860.430.560.0 ClaudeCodeSonnet40.750.240.343.5 ClaudeCodeSonnet4.60.830.580.670.2 ClaudeCode-TeamsSonnet4.60.840.560.660.2 5.2 Main Results We report rubric-based presentation evaluation scores in Table 1. For hallucination evalua- tion, we report the average number of hallucinations, defined as claims classified as major contradictory, in Table 2. Table 3 shows the citation evaluation results. We summarize the key findings below. [F1] Claude Code outperforms Codex in presentation quality. As shown in Table 1, Claude Code consistently achieves higher presentation scores than Codex across all sections, indicating a stronger ability to capture and articulate key scientific points. However, the best- performing agent, Claude Code with Claude Sonnet 4.6, reaches a score of 3.86, suggesting that there remains substantial room for improvement. [F2] Claude Code exhibits substantially more Hallucinations, while Codex significantly reduces them. Table 2 reports the average number of hallucinations per paper across sections. We observe a clear contrast: although Claude Code achieves higher presentation quality, it produces a large number of hallucinations, exceeding 10 per paper even with Claude Sonnet 4.6. In contrast, using GPT-5.4 (OpenAI, 2026) reduces hallucinations to around 3 per paper. These results reveal a clear trade-off between presentation quality and hallucination, highlighting the importance of evaluating both dimensions to accurately assess model performance. [F3] Codex produces fewer citation hallucinations than Claude Code. Table 3 reports the results on citation accuracy. Consistent with [F2], while Claude achieves higher Citation F1 scores, Codex produces substantially fewer hallucinated citations. This again highlights a trade-off between citation coverage and factual reliability. [F4] Writing capability improves with model advances. Our evaluation framework ac- curately captures performance gains from model improvements. We observe consistent increases in writing quality from Claude Sonnet 4 (Anthropic, 2025a) to Claude Sonnet 4.6 (Anthropic, 2026), as well as from GPT-5 (OpenAI, 2025) to GPT-5.4 (OpenAI, 2026), demonstrating that PaperRecon effectively tracks progress in writing capability. 5.3 Human Validation Presentation Validation. To validate the reliability of our proposed evaluation framework, we conducted a human correlation analysis using 72 pairs of generated papers. These pairs 8 Preprint. Under review. Table 4: Effect of research overview length. Comparison of default vs. long research overview as input to the writing agent. Scores are averaged across 12 papers. Rubric EvalâHallucinationâ OverviewDefaultLongDefaultLong Sonnet43.493.648.85.8 Sonnet4.63.834.179.82.3 Table 5: Analysis by paper type. Evaluation results grouped by conference type. Conf.# PapersRubricâHal.â ML213.588.3 CV213.6310.1 M53.4710.7 NLP43.776.0 were constructed from 12 source papers, each reconstructed by four agent configurations (i.e., Claude Code and Codex, each with two backbone LLMs), yielding six pairwise com- binations per source. We recruited three human reviewers with experience as reviewers at top-tier conferences, each of whom provided pairwise judgments (win, tie, or lose) for the 24 paper pairs. We then measured the KendallâsĎ b correlation between these human judgments and the rankings induced by our rubric-based evaluation scores. Our results demonstrate a strong and highly significant correlation (Ď b =0.578,p<0.001), indicating that the rubric-based evaluation aligns well with expert human judgment. We also observe discrepancies between rubric scores and human preferences, often driven by reviewersâ subjective biases such as a preference for conciseness over detailed explanations. These findings suggest that our evaluation framework can provide consistent and high-quality scoring while mitigating subjective biases inherent in human evaluation. Hallucination Validation. For hallucination validation, we focus on measuring precision by verifying whether claims classified as major contradictory are indeed incorrect. Exhaustively identifying all hallucinations is prohibitively labor-intensive; therefore, evaluating precision alone provides a reliable basis for model comparison. As this is a fact-checking task, the verification is conducted manually by the authors. From the evaluation data, we extracted 97 instances labeled as major contradictory in GPT-5, GPT-5.4 and Sonnet-4.6 papers and manually examined them. We find that 96% of these correspond to genuine contradictions or fabrications. This result indicates that hallucinations detected by our method are highly likely to be true hallucinations. 5.4 Analysis Effect of Research Overview Length. We investigate how the granularity of the research overview affects reconstruction quality by comparing default and long versions. The default overview provides a high-level summary (463 words on average), while the long overview includes more detailed descriptions of methodology and experimental design (1492 words on average). Each example is shown in Section B.1. Table 4 shows the results. Consistent with intuition, more detailed research overviews lead to higher presentation scores and fewer hallucinations. This also indicates that our evaluation metrics are able to accurately assess the quality of the papers. Performance by Conference Type. Table 5 shows the performance across different confer- ence types. Although there are differences in the number of papers, we observe that NLP conferences achieve the highest performance. To investigate the cause, we examined the original papers and found that NLP papers tend to focus more on findings-based research, with fewer complex mathematical formulations or methods compared to other fields. There- fore, we consider that it is ultimately necessary to evaluate writing progress separately for each research field. 6 Conclusion, Limitations and Future Work In this work, we introduce Paper Reconstruction Evaluation, the first systematic evaluation framework for AI-generated scientific papers. Along with PaperWrite-Bench, we conduct a comprehensive evaluation of the capabilities and risks of modern writing agents. We discuss the limitations of our approach and outline directions for future work below. 9 Preprint. Under review. Controlled Input Assumptions. Our framework provides structured resources, including figures, tables, and references, to the agent. This design reduces external dependencies such as retrieval and reference collection, and allows us to focus on evaluating core writing ability. Evaluating writing performance under more limited resources, including settings where models rely on external systems, is an important direction for future work. Limited Coverage of Diverse Writing Styles. Evaluating scientific papers is inherently challenging, as human writing is diverse and not fully captured by current LLMs. As a result, section-wise evaluation may not fully reflect overall quality. Developing more robust methods remains an important direction for future work. Ethics Statement This work studies the capabilities and risks of AI-driven scientific writing through a con- trolled evaluation framework. While our approach enables systematic assessment of presen- tation quality and hallucinations, it also highlights the potential for advanced AI systems to generate plausible but misleading scientific content. A key ethical concern is the misuse of such systems to produce fabricated or low-quality research papers that may bypass standard review processes. Our findings, particularly the trade-off between presentation quality and hallucination, underscore the importance of developing robust evaluation methods and safeguards against unreliable AI-generated content. Authorship Statement Atsuyuki Miyai served as the project lead and director of the entire initiative, overseeing all aspects from the conception of PaperRecon to its execution and paper writing. Mashiro Toyooka was primarily responsible for implementation, contributing to the de- velopment of the core codebase for agentic writing as well as the evaluation framework, in collaboration with Atsuyuki Miyai. Zaiying Zhao implemented the figure evaluation system and contributed to the writing of the paper. Kenta Watanabe contributed by carefully reviewing documents such as overview.md for completeness and proposing the Citation Score. Toshihiko Yamasaki mainly served in a managerial role, providing valuable suggestions to the project on a regular basis. Kiyoharu Aizawa provided regular and decisive guidance throughout the project, offering invaluable resource support that was critical to its successful execution. Acknowledgment We would like to thank Qing Yu, Satoshi Kosugi, and Jeonghun Baek for reviewing our generated paper. This work was partialy supported by JSPS KAKENHI 25H01164 and JST BOOST, Japan Grant Number JPMJBS2418. References Samar Ansari. Compound deception in elite peer review: A failure mode taxonomy of 100 fabricated citations at neurips 2025. arXiv preprint arXiv:2602.05930, 2026. Anthropic.System card:Claude opus 4 and claude sonnet 4.Tech- nicalreport,Anthropic,2025a.URLhttps://w-cdn.anthropic.com/ 6d8a8055020700718b0c49369f60816ba2a7c285.pdf. Accessed: 2025-09-14. 10 Preprint. Under review. Anthropic. Claude code. Technical report, Anthropic, 2025b. URLhttps://w.anthropic. com/claude. Accessed: 2026-03-14. Anthropic. Claude sonnet 4.6. Technical report, Anthropic, 2026. URLhttps://w. anthropic.com/claude/sonnet. Accessed: 2026-03-14. Anthropic. Claude code agent teams, 2026. URLhttps://code.claude.com/docs/en/ agent-teams. Accessed: 2026-03-30. Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin QuiĂąonero- Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775, 2025. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike Dâarcy, et al. Synthesizing scientific literature with retrieval-augmented language models. Nature, 2026. ISSN 1476-4687. Siddhartha Banerjee and Prasenjit Mitra. WikiKreator: Improving Wikipedia stubs automat- ically. In ACL, 2015. Angela Fan and Claire Gardent. Generating biographies on Wikipedia: The impact of gender bias on the retrieval-based generation of women biographies. In ACL, 2022. Zhiyuan Fan, Weinong Wang, Debing Zhang, et al. Sedareval: Automated evaluation using self-adaptive rubrics. In Findings of EMNLP 2024, 2024. Tira Nur Fitria. Artificial intelligence (ai) technology in openai chatgpt application: A review of chatgpt in writing english essay. In ELT Forum: Journal of English Language Teaching, volume 12, p. 44â58, 2023. Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025. Chuxuan Hu, Liyun Zhang, Yeji Lim, Aum Wadhwani, Austin Peters, and Daniel Kang. Repro-bench: Can agentic ai systems assess the reproducibility of social science research? In Findings of the Association for Computational Linguistics: ACL 2025, p. 23616â23626, 2025. ICLRProgramChairs.Iclr2026responsetollm-generated papersandreviews.https://blog.iclr.c/2025/11/19/ iclr-2026-response-to-llm-generated-papers-and-reviews/ ,2025.Accessed: 2026-03-30. Intology.Zochi technical report.2025.URLhttps://w.intology.ai/blog/ zochi-tech-report. Accessed: 2025-10-17. Fengqing Jiang, Yichen Feng, Yuetai Li, Luyao Niu, Basel Alomair, and Radha Poovendran. Badscientist: Can a research agent write convincing but unsound papers that fool llm reviewers? arXiv preprint arXiv:2510.18003, 2025. Patrick Tser Jern Kon, Jiachen Liu, Xinyi Zhu, Qiuyi Ding, Jingjia Peng, Jiarong Xing, Yibo Huang, Yiming Qiu, Jayanth Srinivasa, Myungjin Lee, et al. Exp-bench: Can ai conduct ai research experiments? In ICLR, 2026. Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, et al. Quantifying large language model usage in scientific papers. Nature Human Behaviour, p. 1â11, 2025. Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. Automated scholarly paper review: Concepts, technologies, and challenges. Information Fusion, 98: 101830, 2023. ISSN 1566-2535. doi: 10.1016/j.inffus.2023.101830. URLhttps://doi.org/ 10.1016/j.inffus.2023.101830. Available online 12 May 2023. 11 Preprint. Under review. Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. Generating wikipedia by summarizing long sequences. In ICLR, 2018. Zhihan Liu, Yubo Chai, and Jianfeng Li. Towards fully autonomous research powered by llms: Case study on simulations. arXiv preprint arXiv:2408.15512, 2024. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024. JuliĂ MinguillĂłn, Maura Lerga, Eduard Aibar, Josep LladĂłs-Masllorens, and Antoni Meseguer-Artola. Semi-automatic generation of a corpus of wikipedia articles on science and technology. Profesional de la InformaciĂłn, 26(5):995â1005, 2017. Ludovico Mitchener, Angela Yiu, Benjamin Chang, Mathieu Bourdenx, Tyler Nadolski, Arvis Sulovari, Eric C Landsness, Daniel L Barabasi, Siddharth Narayanan, Nicky Evans, et al. Kosmos: An ai scientist for autonomous discovery. arXiv preprint arXiv:2511.02824, 2025. Atsuyuki Miyai, Mashiro Toyooka, Takashi Otonari, Zaiying Zhao, and Kiyoharu Aizawa. Jr. ai scientist and its risk report: Autonomous scientific exploration from a baseline paper. TMLR, 2026. Graham Neubig. Post on x.https://x.com/gneubig/status/1989681438577336401, 2025. Accessed: 2026-03-16. Alexander Novikov, Ngân V Ě u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025. OpenAI. Gpt-5 system card. Technical report, OpenAI, 2025. URLhttps://cdn.openai. com/gpt-5-system-card.pdf. Accessed: 2025-09-14. OpenAI. Codex, 2025. URL https://openai.com/codex/. Accessed: 2026-03-30. OpenAI.Introducing gpt-5.4.March 2026.URLhttps://openai.com/index/ introducing-gpt-5-4/. Accessed: 2026-03-25. Sakuya Ota, Qing Yu, Kent Fujiwara, Satoshi Ikehata, and Ikuro Sato. Pino: Person- interaction noise optimization for long-duration and customizable motion generation of arbitrary-sized groups. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 10676â10685, 2025. Tung Phung, Victor-Alexandru P Ěadurean, JosĂŠ Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, and Gustavo Soares. Generative ai for programming education: Benchmarking chatgpt, gpt-4, and human tutors. In ICER, 2023. Yusuke Sakai, Hidetaka Kamigaito, and Taro Watanabe. Hallucitation matters: Revealing the impact of hallucinated references with 300 hallucinated papers in acl conferences. arXiv preprint arXiv:2601.18724, 2026. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. Assisting in writing wikipedia-like articles from scratch with large language models. In NAACL, 2024. Chenglei Si, Tatsunori Hashimoto, and Diyi Yang. The ideation-execution gap: Execution outcomes of llm-generated versus human research ideas. arXiv preprint arXiv:2506.20803, 2025a. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. In ICLR, 2025b. 12 Preprint. Under review. Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. Paperbench: Evalu- ating aiâs ability to replicate ai research. In ICML, 2025. Jiabin Tang, Lianghao Xia, Zhonghang Li, and Chao Huang. Ai-researcher: Autonomous scientific innovation. In NeurIPS, 2025. Francisco Villaescusa-Navarro, Boris Bolliet, Pablo Villanueva-Domingo, Adrian E Bayer, Aidan Acquah, Chetana Amancharla, Almog Barzilay-Siegal, Pablo Bermejo, Camille Bilodeau, Pablo CĂĄrdenas RamĂrez, et al. The denario project: Deep knowledge ai agents for scientific discovery. arXiv preprint arXiv:2510.26887, 2025. William H Walters and Esther Isabelle Wilder. Fabrication and errors in the bibliographic citations generated by chatgpt. Scientific Reports, 13(1):14045, 2023. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. Cycleresearcher: Improving automated research via automated review. In ICLR, 2025. Yixuan Weng, Minjun Zhu, Qiujie Xie, Qiyao Sun, Zhen Lin, Sifan Liu, and Yue Zhang. Deepscientist: Advancing frontier-pushing scientific findings progressively. In ICLR, 2026. Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066, 2025. Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, et al. Egolife: Towards egocentric life assistant. In CVPR, 2025. Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. Doc: Improving long story coherence with detailed outline control. In ACL, 2023. Yang Zhong, Jiangang Hao, Michael Fauss, Chen Li, and Yuan Wang. Ai-generated essays: Characteristics and implications on automated scoring and academic integrity. Educational Measurement: Issues and Practice, 45(1):e70013, 2026. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Deepreview: Improving llm-based paper review with human-like deep thinking process. In ACL, 2025. Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, and Jialiang Lin. Large language models for automated scholarly paper review: A survey. Inf. Fusion, 124 (C), December 2025. ISSN 1566-2535. doi: 10.1016/j.inffus.2025.103332. URLhttps: //doi.org/10.1016/j.inffus.2025.103332. 13 Preprint. Under review. Appendix A PaperWrite-Bench A.1 Statistics of PaperWrite-Bench Sources Table A presents statistics on the conferences included in PaperWrite-Bench Sources. As shown, ML and CV conferences constitute the majority, followed by multimedia and NLP conferences. Table A: Distribution of papers by research area. AreaConference# PapersSubtotal ML ICLR267 21 NeurIPS256 ICLR254 ICML252 AAAI252 CV CVPR2516 21ICCV253 CVPR262 MultimediaACMMM2555 NLP ACL253 4 NAACL251 Total5151 Table B: Representative rubric evaluation points per section (importance: high). Shown for a benchmark construction paper (EgoLife (Yang et al., 2025), CVPR25) as a concrete example. SectionEvaluation Point AbstractProject goal: egocentric life assistant with wearable AI glasses The abstract must state the overarching objective of EgoLife: building an AI-powered egocentric life assistant that accompanies users and improves personal efficiency through wearable glasses. IntroductionVision and motivation for life-oriented egocentric AI assistance The introduction should open with a motivating vision of an AI assistant embedded in daily life that provides personalized, long-term assistance. Related WorkPositioning within egocentric dataset evolution The section should situate EgoLife in the broader history of egocentric vision datasets, starting from early foundational collections. Benchmark ConstructionSeven-day multimodal data collection in EgoHouse with six participants The section must state that the benchmark is built from a week-long recording of six volunteers living in a custom environment with multimodal sensing. MethodOverall EgoButler architecture with two subsystems The method section must explain EgoButler with its two subsystems: EgoGPT for continuous clip captioning and EgoRAG for retrieval-augmented QA. ExperimentMain benchmark comparison outcome The section should present the unified comparison protocol via EgoButler and the main benchmark comparison outcome. 14 Preprint. Under review. A.2 Example of Rubrics Table B presents a subset of example rubrics for each section. The base paper is EgoLife (Yang et al., 2025). B Detailed Prompts B.1 Examples of research_overview.md research_overview_default.md (EgoLife) # EgoLife: Research Overview ## Title **EgoLife: Towards Egocentric Life Assistant** --- ## 1. Motivation Existing egocentric datasets and benchmarks capture short, single-person activities and miss week-long, multi-person social dynamics needed for real-life assistance. The field needs both a long-duration, multimodal, interpersonal dataset and a method capable of ultra-long-context reasoning. --- ## 2. Key Insight A life assistant must fuse egocentric visual-audio cues with long-horizon memory to answer practical, personalized questions. **Key idea:** Combine omni-modal clip understanding with hierarchical retrieval over week-long egocentric recordings. --- ## 3. Benchmark Design: EgoLifeQA EgoLifeQA is a multiple-choice QA benchmark built on the 266-hour EgoLife dataset (6 participants, 7 days), emphasizing ultra-long-term reasoning. It spans five task types: EntityLog, EventRecall, HabitInsight, RelationMap, and TaskMaster (3,000 QAs total). Each QA includes a "certificate length" (look-back requirement), with 2,003 questions requiring >2 hours of context. Evaluation is accuracy over categories, with evidence retrieval from week-long recordings. --- ## 4. Data Construction Pipeline - Data sources: Meta Aria glasses (video, audio, IMU, gaze), 15 exo GoPro cameras, and 2 mmWave devices in an instrumented "EgoHouse." - Collection: 6 participants cohabiting for 7 days (~8 h/day), synchronized via EgoSync; recordings segmented and aligned; primary language Chinese with English translations. - Filtering/cleaning: Privacy protection via EgoBlur; synchronization and denoising; temporal consistency checks. - Annotation: Transcripts via Whisper + diarization + human review; 5-min narrated clips (0.8x speed) -> merged by GPT-4o into dense visual-audio captions, then verified. - QA creation/QC: ~100K auto-generated QAs per participant -> manually filtered to 500 each (3K total), with distractors, audio-needed flags, and certificate lengths; SRT-based alignment and multi-pass human verification. --- ## 5. Proposed Method: EgoButler EgoButler integrates EgoGPT (clip-level omni-modal understanding) and EgoRAG (hierarchical retrieval for long-context QA). EgoGPT builds on LLaVA-OneVision with an audio branch (Whisper v3) and is instruction-tuned on EgoIT-99K (99K egocentric QAs), plus Day-1 personalization for identity-awareness. EgoRAG constructs multi-level memory (clip/hour/day summaries) and retrieves top-k evidence for answer generation. --- ## 6. Key Findings - EgoGPT achieves state-of-the-art on egocentric benchmarks: 75.4 (EgoSchema), 61.4 (EgoThink), and 33.4 (EgoPlan). - On EgoLifeQA, personalization boosts average from 33.1 to 36.0 and RelationMap from 29.6 to 33.6. - EgoRAG markedly improves long-context QA: >24h certificate length from 25.0 to 35.4; 15 Preprint. Under review. 6-24h from 26.8 to 38.9. - Caption quality is critical: human visual-audio captions reach 45.5 avg; audio-only models lag (27-28), visual-only better (31-34), best with audio+visual (36.0). - Benchmark shows 2,003/3,000 QAs need >2h context, validating the need for retrieval over week-long memory. --- ## 7. Contributions - EgoLife: a 266-hour, week-long, multimodal, ego-exo egocentric dataset with dense transcripts and visual-audio captions. - EgoLifeQA: 3,000 long-context, multiple-choice QAs across five life-assistant task categories with certificate-length metadata. - EgoButler: a two-stage system (EgoGPT + EgoRAG) for personalized, long-horizon egocentric QA; release of EgoIT-99K for instruction tuning. - Comprehensive analysis identifying personalization, caption quality, and hierarchical retrieval as key drivers of performance. --- ## 8. Takeaway EgoLife and EgoButler establish a practical path toward egocentric life assistants by pairing week-long multimodal data with a retrieval-augmented, omni-modal method for ultra-long-context reasoning. research_overview_long.md (EgoLife) # EgoLife: Research Overview ## Title **EgoLife: Towards Egocentric Life Assistant** --- ## 1. Motivation Egocentric AI assistants promise to augment daily life by recalling past events, tracking habits, and making personalized recommendations. However, existing egocentric datasets and benchmarks (e.g., EPIC-KITCHENS, Ego4D) primarily cover short to medium spans, single-person views, and narrowly scoped activities. They lack week-long coverage, rich interpersonal dynamics, and consistent multimodal capture -- all crucial for building assistants that can reason over ultra-long temporal horizons and understand social context. This work addresses two gaps simultaneously: (1) the lack of a longitudinal, multiperson, multimodal egocentric dataset, and (2) the absence of a benchmark that evaluates long-context, life-oriented assistance. It also contributes a method that integrates clip-level multimodal understanding with scalable long-context retrieval to answer week-scale questions. Taken together, the dataset, benchmark, and method move toward practical egocentric life assistants that operate over days, handle audio-visual inputs, and keep personalized, identity-aware memory. --- ## 2. Key Insight The core insight is that long-horizon egocentric assistance requires tightly coupled components: a personalized, omni-modal clip-understanding model and a hierarchical, time-aware memory system that supports efficient retrieval over week-level video. - Key idea: Fuse continuous ego-clip captioning (visual+audio) with hierarchical memory summarization and retrieval to answer long-context, life-oriented questions that require identity awareness and habit/event grounding across days. --- ## 3. Benchmark Design: EgoLifeQA ### 3.1 Overview EgoLifeQA is a long-context, life-oriented QA benchmark built on EgoLife, covering a week of shared living among six participants. It evaluates five capability axes -- object/entity logging, event recall, habit analysis, social relationship understanding, and task-oriented assistance -- via multiple-choice questions that require retrieving evidence across hours to days. The benchmark explicitly annotates whether audio is required and the look-back time ("certificate length"). Goals: - Evaluate long-term memory and retrieval over week-scale egocentric content. - Test personalization (identity recognition, social interaction patterns). 16 Preprint. Under review. - Assess multi-modal integration (audio + video). ### 3.2 Data Collection & Curation - Data sources: EgoLife dataset (Meta Aria smart glasses egocentric capture; 15 exocentric GoPro cameras; 2 mmWave devices). Primary language: Chinese; annotations translated to English. - Initial QA generation: For each participant, "visual-audio captions" (dense narrations post-processed by GPT-4o) were fed into GPT-4o with tailored prompts per question type to produce ~100K timestamped QA candidates. - Human-in-the-loop curation: Annotators reviewed candidates synchronized with video (SRT), retained only questions requiring >= 5 minutes of look-back, and prioritized longer dependencies and high real-world relevance. - Finalization: 1,000 questions per participant were pruned and refined to 500 per participant (3,000 total), with distractors authored for MCQ. Annotators labeled audio requirement and certificate length. Quality control included multi-round reviews and timestamp verification. - Annotation protocol: - Five categories (EntityLog, EventRecall, HabitInsight, RelationMap, TaskMaster) with structured prompts. - Multi-choice answers with evidence timestamps. - Minimum evidence window: >= 5 minutes before the question timestamp. ### 3.3 Task Definition & Evaluation Protocol - Input: Question, candidate answers (MCQ), and access to video-derived memory (captions, transcripts) plus retrieval. - Output: Single-choice answer. - Metrics: Accuracy (per category and overall), breakdown by certificate length (<2h, 2-6h, 6-24h, >24h) and by audio requirement. - Splits: 500 QAs per participant (6 participants -> 3,000 total). Evaluation reports per-participant and averaged. --- ## 4. Dataset Statistics EgoLife (data foundation for EgoLifeQA): - Participants: 6 volunteers cohabiting for 7 days in a fully instrumented house (EgoHouse). - Recording devices: - Egocentric: Meta Aria glasses (video, audio, IMU, gaze; synchronized). - Exocentric: 15 GoPro cameras for multi-view coverage. - Additional sensing: 2 mmWave devices; house 3D reconstruction with Aria Multi-MPS. - Duration: - ~300 hours captured; 266 hours retained post-cleaning. - Approximately 8 hours/day/person; minimum 6h/day enforced. - Language: Primarily Chinese; English translations provided. - Annotations: - Transcripts: Diarized, speaker-resolved; cross-participant overlap curated (initial merged ~50h reviewed, then split and refined). - Dense narrations: 361K phrases (avg 2.65 s), captured at 0.8x speed for high density. - Merged captions: 25K coherent segments (via GPT-4o-mini merging). - Visual-audio captions: Enriched with 1 FPS frames + transcripts, summarized and human-verified. - EgoLifeQA: - 3,000 QAs total (500 per participant). - Certificate length distribution includes 997 QAs with look-back <2h and 2,003 QAs >2h (many beyond 24h). - Audio requirement is labeled per QA. Comparisons: - Compared to EgoSchema, HourVideo, etc., EgoLife/EgoLifeQA uniquely target week-long, multiperson daily life with rich ego-exo, multimodal capture and social dynamics. --- ## 5. Proposed Method: EgoButler ### 5.1 Overview EgoButler integrates: - EgoGPT (System-I): A 7B vision-audio-language model for clip-level understanding and captioning, adapted to egocentric domain and personalized identity cues. - EgoRAG (System-I): A hierarchical, time-aware retrieval-augmented generation system that builds a multi-level memory bank (clip/hour/day) and retrieves evidence for ultra-long-context QA. Pipeline: 1) Continuous 30 s clip captioning (visual + audio) -> memory bank. 2) Hierarchical retrieval over day/hour windows -> top-k clip evidences. 3) Answer generation with retrieved context. 17 Preprint. Under review. ### 5.2 EgoGPT (Clip-Level Omni-Modal Understanding) - Base: LLaVA-OneVision (Qwen2-based, 7B). - Audio branch: Added following Ola-style design; Whisper Large v3 encodes audio; trained audio projection on LibriSpeech; finetuned jointly. - Training data: EgoIT-99K (Table: 9 egocentric datasets; 43.16 h; 1,529 videos incl. 686 with audio; 99.48K QAs across VC/AVC/MCQ/MRC/IQA). - Personalization: Additional finetuning on EgoLife Day-1 to calibrate identity cues and environment context. - Functions: - Dense visual-audio captioning for 30 s clips (1 FPS sampling in experiments for captioners). - Answering questions with retrieved evidence and identity-aware references. ### 5.3 EgoRAG (Long-Context Retrieval-Augmented Generation) - Memory bank M = (c_i, d_i, t_i) where: - c_i: clip features; d_i: textual descriptions (EgoGPT captions); t_i: hierarchical summaries (hourly/day). - Retrieval: - Coarse-to-fine: first retrieve by day -> hour summaries, then fine-grained clip selection. - Relevance score: s_i = Similarity(q, c_i) + lambda * Similarity(q, d_i). In main experiments lambda = 0 (text-only retrieval). - Select top-k clips (k typically small; e.g., 3) for evidence. - Response: - r = LLM(q, R), where LLM is EgoGPT or a stronger generator (e.g., GPT-4o) conditioned on retrieved evidence R. --- ## 6. Experimental Results ### 6.1 Setup - Baselines (egocentric VLM benchmarks): GPT-4v/4o, Gemini-1.5-Pro, LLaVA-Next-Video, LongVA, IXC-2.5, InternVideo2, Qwen2-VL, Oryx, LLaVA-OV, LLaVA-Videos. - Datasets: EgoSchema, EgoPlan-Bench, EgoThink; EgoLifeQA (Jake's 500-QA split in this version). - Inputs: 32 frames for standard benchmarks; 1 FPS for caption-based memory in EgoLifeQA. - EgoRAG: text-similarity retrieval (lambda=0), top-3 30 s clips; re-query with GPT-4o-mini; final answer generation by GPT-4o for fairness across captioners. ### 6.2 Key Findings - EgoGPT on benchmarks (32 frames): - EgoGPT (EgoIT-99K): EgoSchema 73.2, EgoPlan 32.4, EgoThink 61.7. - With personalization (+D1): EgoSchema 75.4 (+2.2), EgoPlan 33.4 (+1.0), EgoThink 61.4. - EgoLifeQA (average accuracy): - Gemini-1.5-Pro: 36.9; GPT-4o: 36.2; LLaVA-OV: 30.8. - EgoGPT (non-personalized): 33.1; EgoGPT (+D1): 36.0 (+2.9). - Category-wise (EgoGPT +D1): EntityLog 39.2, EventRecall 36.5, HabitInsight 31.1, RelationMap 33.6, TaskMaster 39.7. - EgoRAG effectiveness by certificate length: - EgoGPT vs. EgoGPT+EgoRAG: - <2h: 28.2 -> 27.2; - 2-6h: 29.1 -> 35.7; - 6-24h: 26.8 -> 38.9; - >24h: 25.0 -> 35.4. - Large gains for >= 2h look-back validate long-context retrieval benefits. - Caption quality matters (avg accuracy): - Narration-only: 31.5; Transcript-only: 29.6; Visual-audio captions (human-verified): 45.5. - EgoGPT memory banks: audio-only 27.2-28.1; visual-only 31.2-33.6; visual+audio 33.1-36.0. --- ## 7. Analysis & Insights - Personalization helps: Day-1 finetuning improves entity and relation questions, indicating identity cues and environment priors are beneficial -- but can lead to overfitting (e.g., misidentifying later people wearing similar colors). - Audio is useful but insufficient alone: audio-only captioning underperforms; best results require joint visual+audio. - Long-context retrieval is crucial: naive segmentation causes hallucinations; hierarchical memory and retrieval yield pronounced gains especially for >6h queries. - Error modes: - Speech/emotion understanding is limited by ASR-centric audio training (e.g., laughter/emotion recognition). - Single-pass retrieval lacks iterative reasoning; failures cascade when relevant clips aren't surfaced. - Social interaction nuance (e.g., subtle nonverbal cues) remains challenging. 18 Preprint. Under review. --- ## 8. Contributions - EgoLife dataset: 266 h retained from ~300 h of week-long, multiperson egocentric recordings with synchronized ego-exo capture (15 exo cameras, 2 mmWave), rich transcripts and dense visual-audio captions. - EgoLifeQA benchmark: 3,000 long-context MCQs across five categories, with certificate lengths up to >24h and audio-requirement labels, targeting practical life assistance. - EgoButler system: - EgoGPT: a 7B vision-audio-language model tuned on EgoIT-99K and personalized on EgoLife Day-1, achieving strong egocentric benchmark performance. - EgoRAG: hierarchical, time-aware memory and retrieval for ultra-long-context QA, significantly improving accuracy for long look-back queries. --- ## 9. Limitations & Future Work - Dataset scope: single week, six participants, primarily Chinese language; future expansions to more participants, diverse locales, and languages are planned. - Audio understanding: current training biases toward ASR; future models should integrate paralinguistic cues (emotion, prosody). - Personalization stability: day-1 finetuning can overfit identity cues; future work on robust identity modeling and lifelong adaptation is needed. - Retrieval reasoning: EgoRAG is single-pass without multi-hop refinement; integrating step-by-step, iterative retrieval and self-correction could improve robustness. - Privacy and deployment: despite anonymization tools (EgoBlur), real-world deployment requires stronger privacy-preserving methods and secure on-device processing. --- ## 10. Takeaway EgoLife establishes the first week-scale, multiperson, multimodal egocentric dataset and a long-context QA benchmark targeting life assistance. Coupled with EgoButler -- integrating personalized clip-level multimodal understanding (EgoGPT) and hierarchical long-context retrieval (EgoRAG) -- the project advances the frontier of egocentric AI toward practical, personalized assistants that reason across days, track habits, and understand social dynamics. B.2 Prompts for Creating Overviews Prompt for Generating Research Overview (Default / Method Paper) [System] You are an expert AI researcher. Your task is to read a research paper and generate a structured research overview in Markdown format. The overview should be comprehensive yet concise, capturing the key aspects of the paper. It will be used as input for an AI system that writes LaTeX papers, so accuracy and clarity are critical. Output ONLY the Markdown content, no preamble or explanation. [User] Read the following research paper and generate a concise research overview in Markdown format. Keep it SHORT and focused. Each section should be 2-4 sentences at most. Total length should be around 1500-2500 characters. Follow this structure: # [Paper Acronym/Name]: Research Overview ## Title ## 1. Motivation (1-2 sentences) ## 2. Key Insight (1-2 sentences, bold "Key idea:" one-liner) ## 3. Proposed Method (2-4 sentences) ## 4. Experimental Results (3-5 bullet points with numbers) ## 5. Contributions (3-4 bullet points) ## 6. Takeaway (one sentence) Prompt for Generating Research Overview (Long / Method Paper) [System] (same as above) [User] Read the following research paper and generate a detailed and comprehensive research overview in Markdown format. Be thorough: include technical details, formulas, specific numbers, and nuanced analysis. Total length should be around 4000-8000 characters. Follow this structure: 19 Preprint. Under review. # [Paper Acronym/Name]: Research Overview ## Title ## 1. Motivation (2-3 paragraphs) ## 2. Key Insight (bold "Key idea:" summary) ## 3. Proposed Method ### 3.1 Overview ### 3.2 [Component Name] (add subsections as needed) ## 4. [Additional Section as needed] ## 5. Experimental Results ### 5.1 Setup ### 5.2 Key Findings ## 6. Analysis & Insights ## 7. Contributions ## 8. Limitations & Future Work ## 9. Takeaway Prompt for Generating Research Overview (Default / Benchmark Paper) [System] (same as above) [User] Read the following research paper about a benchmark/dataset and generate a concise research overview in Markdown format. Keep it SHORT and focused. Each section should be 2-4 sentences at most. Total length should be around 1500-2500 characters. Follow this structure: # [Benchmark Name]: Research Overview ## Title ## 1. Motivation ## 2. Key Insight ## 3. Benchmark Design (tasks, data sources, scale, evaluation protocol) ## 4. Data Construction Pipeline (data sources; collection process; filtering/cleaning; annotation protocol; number of annotators and qualifications; inter-annotator agreement; quality control measures; automated/human-in-the-loop curation steps) ## 5. Key Findings (3-5 bullet points with numbers) ## 6. Contributions ## 7. Takeaway Prompt for Generating Research Overview (Default / Both Paper) [System] (same as above) [User] Read the following research paper that contributes BOTH a new method AND a new benchmark/dataset. Generate a concise research overview in Markdown format. Keep it SHORT and focused. Each section should be 2-4 sentences at most. Total length should be around 1500-2500 characters. Follow this structure: # [Paper Acronym/Name]: Research Overview ## Title ## 1. Motivation ## 2. Key Insight ## 3. Benchmark Design (tasks, data sources, scale, evaluation protocol) ## 4. Data Construction Pipeline (data sources; collection process; filtering/cleaning; annotation protocol; quality control measures) ## 5. Proposed Method (2-4 sentences, key components) ## 6. Key Findings (3-5 bullet points covering both method and benchmark) ## 7. Contributions ## 8. Takeaway Prompt for Generating Research Overview (Long / Both Paper) [System] (same as above) [User] Read the following research paper that contributes BOTH a new method AND a new benchmark/dataset. Generate a detailed and comprehensive research overview in Markdown format. Be thorough: include technical details, formulas, dataset statistics, specific numbers, and nuanced analysis. Total length should be around 4000-8000 characters. Follow this structure: # [Paper Acronym/Name]: Research Overview ## Title 20 Preprint. Under review. ## 1. Motivation (2-3 paragraphs) ## 2. Key Insight ## 3. Benchmark Design ### 3.1 Overview ### 3.2 Data Collection & Curation (data sources; collection process; filtering/cleaning; annotation protocol; annotator count and qualifications; inter-annotator agreement; quality control; automated/human-in-the-loop curation) ### 3.3 Task Definition & Evaluation Protocol ## 4. Dataset Statistics ## 5. Proposed Method ### 5.1 Overview ### 5.2 [Component Name] (add subsections as needed) ## 6. Experimental Results ### 6.1 Setup ### 6.2 Key Findings ## 7. Analysis & Insights ## 8. Contributions ## 9. Limitations & Future Work ## 10. Takeaway Prompt for Generating Research Overview (Long / Benchmark Paper) [System] (same as above) [User] Read the following research paper about a benchmark/dataset and generate a detailed and comprehensive research overview in Markdown format. Be thorough: include technical details, dataset statistics, specific numbers, and nuanced analysis. Total length should be around 4000-8000 characters. Follow this structure: # [Benchmark Name]: Research Overview ## Title ## 1. Motivation (2-3 paragraphs) ## 2. Key Insight ## 3. Benchmark Design ### 3.1 Overview ### 3.2 Data Collection & Curation (data sources and selection criteria; collection process; filtering/cleaning; annotation protocol; annotator count and qualifications; inter-annotator agreement; quality control; automated/human-in-the-loop curation) ### 3.3 Task Definition & Evaluation Protocol ### 3.4 [Additional Design Aspect] ## 4. Dataset Statistics ## 5. Baseline Evaluation ### 5.1 Evaluated Methods ### 5.2 Key Findings ## 6. Analysis & Insights ## 7. Contributions ## 8. Limitations & Future Work ## 9. Takeaway B.3 Prompts for Creating Rubrics Prompt for Extracting Evaluation Points (Rubric) from Each Section You are an expert paper reviewer. Given a section of an accepted top-tier conference paper, extract the key elements that are essential for reproducing this section's content. Each element should represent a distinct piece of information, claim, technical detail, or structural component that a generated version of this section MUST include to be considered faithful. Guidelines: - Extract 5-15 elements per section depending on length and complexity. - For Introduction: problem motivation, research gap, proposed approach overview, key contributions, paper structure. - For Method: each major component/module, key equations/formulations, design choices and justifications, training procedures. - For Experiment: datasets, baselines, evaluation metrics, main results, ablation studies, key findings. - For Related Work: main research areas covered, key distinctions from prior work. - For Abstract: core problem, approach, key results. 21 Preprint. Under review. - importance should be "high" for elements without which the section would be fundamentally incomplete, "medium" for important but not critical details, "low" for nice-to-have elements. - evidence should be a brief quote or reference to the specific part of the GT text. Respond in JSON format matching the schema exactly. Write element and description in English. Write evidence in the original language of the paper (usually English). B.4 Writing Prompts For the writing prompt, we assumed the writing process would be used as part of a practical AI Scientist system, and therefore referred the prompt from Jr. AI Scientist (Miyai et al., 2026). Writeup Agent Prompt Your goal is to write up the following idea: ```markdown research_overview_text ``` Note that idea_text represents a preliminary hypothesis and may not necessarily align with the experiments that were eventually performed. First, make sure to refer to the experiment data contained in table/ and figure/ folders. We have VLM-based table descriptions: ``` table_descriptions ``` We also have VLM-based figure descriptions: ``` plot_descriptions ``` To better understand the methodology and experiments, please also refer to: - code/ as the proposed method's code implementation Please read the current template.tex file and update it to produce a complete, coherent, and scientifically accurate paper. This must be an acceptable complete LaTeX writeup, suitable for a num_page-page column_type paper. Make sure to use the citations from the references.bib file and report results accurately based on the experimental data provided. IMPORTANT: references.bib can be very large. Do NOT read the entire file at once. Instead, use Grep to search for relevant citation keys or authors, then Read only the specific portions you need (using offset and limit parameters). Start by reading template.tex to understand the current state, then edit it to incorporate all the information above into a complete paper. Please note: For the bibliography, do not use the filecontentsreferences.bib environment. Instead, all citations should refer to an external file named references.bib. Reflection Prompt (LaTeX Error Fixing) Now let's reflect and identify any issues (including but not limited to). Your task is to read the current template.tex file and improve it based on the feedback provided below. 1) Are there any LaTeX syntax errors or style violations we can fix? Refer to the chktex output below. chktex results: ``` check_output ``` 2) Are there any LaTeX compilation errors? Refer to the tectonic compile output below. tectonic compile output: ``` compile_output 22 Preprint. Under review. ``` If there are errors reported above, please fix them directly. Read template.tex and edit it to address these issues. Focus especially on fixing compilation errors so that the paper compiles successfully. If no errors are reported, no changes are necessary. Page Limit Adjustment Prompt The main text (before'References') is currently main_pages pages. The target is page_limit pages. The paper is status. Please action to reach the target. Do NOT move content to or create an Appendix. Keep everything in the main text. Do not add or remove more than 1000 characters in this revision. Do not use filecontentsreferences.bib. B.5 Evaluation Prompts Rubric Evaluation: Section-Level Scoring [System Message] You are an expert paper reviewer. You are given: 1. A list of key elements (rubric) that should appear in a specific section of a paper. 2. The predicted section content to evaluate. 3. (Optional) Figure/Table context showing which visual assets are present or missing. For each element, score how well the predicted section covers it on a 1-5 scale: 5: Fully and accurately described. The element is present with correct details. 4: Mostly described. The core idea is present but some details are missing or slightly imprecise. 3: Partially described. The element is mentioned but with significant gaps or vagueness. 2: Barely mentioned. Only a superficial or indirect reference exists. 1: Not described at all. The element is completely absent from the predicted section. Respond in JSON format. For each element, provide: - "element": the element name (copy exactly from input) - "score": 1-5 - "reasoning": brief explanation [User Message] **Section: section_name** **Rubric (key elements to check):** - element_name (importance): description - ... **Predicted section content:** pred_content ### Figure/Table Context for this Section The following analysis describes the presence or absence of visual assets in this section compared to the Ground Truth: figure_table_context **Instructions:** 1. Evaluate each rubric element based on the text content. 2. CRITICAL: If a rubric element requires or refers to data/visuals, and the corresponding Figure/Table is reported as MISSING or has low match score in the context above, you MUST penalize the score for that element. Evaluate each element on a 1-5 scale. 23 Preprint. Under review. Hallucination Evaluation: Stage 1 â Claim Extraction & Classification (LLM) [System Message] You are an expert paper reviewer detecting factual errors in a predicted paper section. You are given: 1. The predicted section content. 2. The ground truth (GT) full paper content for reference. Your task is to identify all concrete, verifiable claims in the predicted section (e.g., specific numbers, method descriptions, experimental setups, results) and classify each into one of three categories: - supported: The claim is directly stated in or logically derivable from the GT paper. - neutral: The claim is NOT in the GT, but is a reasonable general statement, common knowledge, or supplementary detail that does not contradict the GT. This is NOT an error. - contradictory: The claim directly contradicts specific information in the GT paper. This is a factual error / hallucination. For contradictory claims, also assign severity: - major: Incorrect numbers, fabricated results, wrong method descriptions, misattributed findings -- errors that would mislead a reader. - minor: Overly strong generalizations, imprecise wording that slightly distorts meaning, minor numerical rounding issues. IMPORTANT: - Do NOT classify claims as contradictory simply because they are absent from the GT. Absence != contradiction. - Focus on claims that can be verified against the GT. Skip purely stylistic or structural observations. - Be thorough: extract ALL verifiable claims, not just a few. Respond in JSON with a list of claims. Each claim has: - "claim": the specific statement from Pred - "classification": "supported" | "neutral" | "contradictory" - "evidence": brief explanation - "severity": "major" | "minor" (only for contradictory, null otherwise) [User Message] **Section being evaluated: section_name** **Predicted section content:** pred_content **Ground Truth full paper (for reference):** gt_full_content Extract and classify all verifiable claims from the predicted section. Hallucination Evaluation: Stage 2 â Agent-Based Verification [System Message] You are a rigorous fact-checker performing a second-pass verification. A previous reviewer flagged the following claim as contradictory (factual error / hallucination) in a predicted paper section. Your job is to carefully re-examine whether this is truly a contradiction with the Ground Truth paper, or a false positive. Classify the claim into one of: - contradictory: Confirmed. The claim genuinely contradicts specific information in the GT paper. - neutral: False positive. The claim is absent from the GT but does NOT contradict it. Absence is not contradiction. - supported: False positive. The claim is actually supported by the GT paper. For confirmed contradictory claims, assign severity: - major: Incorrect numbers, fabricated results, wrong method descriptions. - minor: Overly strong generalizations, imprecise wording. Respond in JSON: - "classification": "supported" | "neutral" | "contradictory" - "severity": "major" | "minor" for contradictory, "none" for supported/neutral 24 Preprint. Under review. - "evidence": brief explanation of why you changed or kept the classification [User Message] The following N claims were flagged as contradictory by a previous reviewer. Re-examine EACH claim and classify it independently. ### Claim 1 - Claim: claim_text - Original evidence: original_evidence - Original severity: original_severity ... Your current working directory contains the Ground Truth paper resources: - The GT paper's LaTeX source (main.tex, gt_main.tex, or similar .tex files) - code/ directory with the original codebase (if exists) - figures/ directory with figure images - tables/ directory with table data (.tex files) Please read the relevant GT files to verify each claim. For claims about the implementation details or methods, check the code/ directory. For other claims, you can check main.tex/gt_main.tex and tables/. Return a JSON with a "results" array containing one entry per claim, in the same order. Note: The agent operates in READ-ONLY mode with access to Read, Glob, and Grep tools only. C Generated Papers We include the two generated papers based on PINO (Ota et al., 2025). 25 PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groups Sakuya Ota Qing Yu Kent Fujiwara Satoshi Ikehata Ikuro Sato Abstract Generating realistic multi-person interactions at scale re- mains an open challenge in human motion synthesis. Ex- isting methods either rely on shared text prompts that yield uniform, undifferentiated behaviors, or require retraining when new constraints are introduced. We present PINO (Person-InteractionNoiseOptimization), a training-free framework that composes arbitrary-sized group interac- tions from a pretrained two-person diffusion model (In- terGen). Our key insight is to decompose group dynam- ics into a sequence of pairwise generations anchored by a shared âpivotâ character. For each new person added to the group, PINO conditions the target character on the pivot via masked diffusion and optimizes the initial noise with differ- entiable physics-based penalties, including overlap avoid- ance, root position, movement region, facing orientation, and relative positioning. Per-pair text prompts enable fine- grained role assignment without shared-prompt ambigu- ity. For long-duration sequences, PINO employs motion inpainting with boundary smoothness penalties to extend interactions while preserving temporal coherence. Exper- iments on the InterHuman dataset demonstrate that PINO eliminates body overlap (Overlapâ0.000vs.0.119for In- terGen in the two-person case), dramatically reduces pen- etration volume (275.65vs.3112.72), and maintains com- petitive semantic quality (FID13.163vs.13.278). These benefits scale to groups of five or more persons without any additional training. 1. Introduction Synthesizing realistic human motion has broad applications in computer animation, video games, and robotics. While single-person text-to-motion generation has achieved im- pressive results [1,3,20,26], extending these techniques to multi-person interactions poses unique challenges: mo- tions must be jointly plausible, semantically consistent with natural language descriptions, and physically coherent (i.e., free of body penetration and interpenetration artifacts). Existing approaches to multi-person generation fall into three broad categories. First, methods based on Control- Net [25] (e.g., InterControl [22], FreeMotion [2]) adapt single-person or two-person networks through conditional guidance. Second, directly trained multi-person networks (e.g., Shan et al. [16]) are learned on multi-person datasets, limiting scalability. Third, diffusion-based composition methods such as PriorMDM [15] combine single-person priors via learned communication blocks. Across all cat- egories, a common limitation is that adding new characters or introducing new spatio-temporal constraints typically re- quires either retraining or significant model redesign. Inspired by Diffusion Noise Optimization (DNO) [7] and ProgMoGen [11], which demonstrated that optimiz- ing the initial noise of a diffusion modelârather than its weightsâenables zero-shot controllability for single- person motion, we extend this principle to the multi-person setting. Our approach, PINO (Person-InteractionNoise Optimization), leverages a pretrained two-person diffu- sion model (InterGen [10]) and composes arbitrarily large groups through sequential, pairwise generation guided by physics-aware noise optimization. The core idea of PINO is to treat group interactions as a chain of pairwise relationships anchored by a pivot charac- ter. When generating personk(k>2), we fix the motion of the pivot (person 1) via feature-space masking and gen- erate personkconditioned on the pivot using a per-pair text prompt. The initial noise for personkis optimized through a differentiable loss that combines: (i) anoverlap penalty enforcing minimum distance between all character pairs; (i) aroot position penaltycontrolling start/end positions; (i) amovement region penaltyconstraining spatial extents; (iv) afacing orientation penaltyspecifying relative body orientations; and (v) arelative position penaltygoverning pairwise spatial relationships. For long-duration generation, motion inpainting with a boundary smoothness penalty en- ables coherent sequence extension without retraining. Our contributions are as follows: â˘Atraining-freeframework that scales a pretrained two- person diffusion model to arbitrary group sizes via se- quential pairwise generation with pivot conditioning. â˘Physics-aware noise optimizationthat enforces overlap avoidance and fine-grained spatio-temporal controls en- tirely at inference time. 1 GT (line 118) misattributes shafir2024pri ormdm to ComMDM instead of âPriorMDM with learned communicati on blocks. Agent: Claude Code (Sonnet4.6) Rubric Score: 3.95 Major Contradiction: 13 â˘Per-pair prompt switchingthat enables nuanced role assignment and customizable behaviors within large groups, avoiding shared-prompt ambiguity. â˘Long-duration motion generationvia inpainting with smooth boundary transitions, enabling evolving interac- tions over time. Experiments on the InterHuman benchmark show that PINO achieves near-zero overlap even in five-person sce- narios, dramatic reductions in penetration volume, and competitive or superior FID scores compared to InterGen and FreeMotion, all without additional training. 2. Related Work Single-Person Motion Generation.Text-conditioned human motion synthesis has made rapid progress using variational autoencoders [12,13], transformer-based archi- tectures [3], and diffusion models [1,20,26]. MDM [20] introduced a classifier-free diffusion model for motion that predicts the clean sample at each denoising step. MLD [1] performs diffusion in a learned latent space to reduce computational cost. These works focus on single-person scenarios and serve as the backbone components for multi-person extensions. Controllable Motion Generation.Adding spatial con- straints to diffusion models without retraining is an active research direction. GMD [6] introduces imputation-based spatial guidance for pelvis trajectory control. OmniCon- trol [23] extends this to any joint at any time. ProgMo- Gen [11] demonstrates that open-set motion control tasks can be solved by optimizing the diffusion latent code using programmable constraint functionsâinspiring our noise optimization approach. DNO [7] further formalizes noise optimization as a universal prior for single-person motion editing, completion, and denoising. EDGE [21] proposes in-betweening for dance, a form of motion inpainting that is related to our long-duration extension. Multi-Person Interaction Generation.Classical meth- ods synthesize interactions by retargeting or blending cap- tured motion clips [8,17]. With the rise of learning-based approaches, more flexible interaction generation has be- come possible. ActFormer [24] uses a GAN-based trans- former for multi-person action-conditioned generation. In- terGen [10] introduces a two-person diffusion model with weight-sharing denoisers and mutual attention trained on the InterHuman dataset [10]. PriorMDM [15] explores compositional use of single-person priors through a learned communication block (ComMDM) for two-person interac- tion. InterHandGen [9] addresses two-hand interaction gen- eration using cascaded reverse diffusion. Role-aware in- teraction generation [19] distinguishes active and passive Method #Person in Network Design #Person in Training Data Order of Multi-Person Generation Method InterControl [22]OneOneSequentialControlNet FreeMotion [2]OneTwoSequentialControlNet Shan et al. [16]MultipleMultipleConcurrent Supervised Training ProgMoGen [11]OneTraining-free N/A (Single-Person) Noise Optimization Ours Two (or multiple) Training-freeSequential Noise Optimization Table 1. Comparison of recent multi-person interaction generation methods, including single-person method ProgMoGen [11]. roles via text conditioning. InterMask [5] uses collaborative masked modeling in discrete space for high-fidelity two- person generation. DiffCollage [27] composes large con- tent via factor graph representations of overlapping diffu- sion models. Scaling to Arbitrary Groups.Going beyond two per- sons, InterControl [22] adapts a single-person diffusion model to multi-person scenarios via zero-shot joint-contact guidance. FreeMotion [2] unifies single and multi- person synthesis through decoupled generation and inter- action modules but still relies on ControlNet adapters. Shan et al. [16] train a multi-person transformer diffusion model on large-scale datasets curated from internet videos. Table1compares these methods along key design dimen- sions. Unlike prior work, PINO requires no retraining, sup- ports per-pair prompts, and achieves physical plausibility through noise optimization. 3. Preliminaries Two-Person Diffusion Model.We build on Inter- Gen [10], a denoising diffusion probabilistic model [4] for two-person interaction synthesis. Given a text promptc, In- terGen learns the distribution over two-person motion se- quencesx=(x (1) ,x (2) )âR Tâ2âD , whereTis the number of frames andD= 262is the per-person fea- ture dimension (encoding joint positions, velocities, rota- tions, and foot contact). The model uses two weight-sharing transformer denoisers with mutual attention to capture inter- person dependencies. At inference, a DDIM sampler [18] denoises the joint motion from Gaussian noise conditioned on a CLIP [14] text embedding. Diffusion Noise Optimization.DNO [7] and ProgMo- Gen [11] demonstrated that, given a pretrained diffusion modelĎ Ď , one can optimize the initial noisez T âN(0,I) to satisfy a target objectiveLwithout modifying model weights: z â T = arg min z T L( Ë x 0 (z T )),(1) 2 GT misplaces ActFormer under single- person GAN-based motion generation instead of multi-person interaction generation. GT describes InterControl as ControlNet- based joint guidance, not zero- shot joint- contact guidance. Figure 1.Person-Interaction Noise Optimization (PINO).Starting from a two-person generation conditioned on a per-pair prompt, PINO adds characters incrementally. For each new person, person 1 acts as the pivot (reference): its motion is injected into the initial noise via feature masking, and the new personâs noise is optimized with physics-based penalties. The same procedure is repeated for long-duration extension via motion inpainting. where Ë x 0 (z T ) = Denoise Ď (z T ,c)is the sample generated by the full reverse diffusion process fromz T . Gradients flow through the entire denoising chain back toz T . This enables arbitrary differentiable constraints to be imposed at inference time. 4. Method 4.1. Overview PINO generates group interactions of arbitrary size in a two- stage pipeline (Fig.1): 1.Base generation:A two-person interaction (persons 1 and 2) is generated with InterGen conditioned on a text promptc 12 , with noise optimized to satisfy physics- based penalties. 2.Sequential extension:Each additional personk(kâ3) is generated conditioned on person 1 (the pivot) using a per-pair promptc 1k , with its noise optimized to avoid overlap with all previously generated persons. For long-duration generation, a third stage extends any two- person sequence using motion inpainting with boundary smoothness penalties. 4.2. Masked Pivot Conditioning When generating personk(kâ3) in interaction with the pivot (person 1), we fix person 1âs motionx (1) âR TâD and generate a new pair( Ě x (1) ,x (k) )using InterGen. The key is to constrain Ě x (1) to match the existing pivot motion. Following the inpainting strategy in the diffusion literature, we construct the initial noise as: z T =mâx (1) +(1âm)âĎ,ĎâN(0,I),(2) wheremis a binary mask selecting the feature dimensions corresponding to person 1. During each denoising step, the predicted person 1 features are replaced by the fixed pivot motion, ensuring the generated pair remains anchored to the existing group. Personkâs noise is left free to be optimized. 4.3. Physics-Aware Noise Optimization Starting from the masked initial noise, we optimize the free (personk) portion ofz T by minimizing a composite physics-based loss: L=L overlap +Îľ r L root +Îľ reg L region +Îľ o L orient +Îľ rel L rel ,(3) whereÎľ r ,Îľ reg ,Îľ o ,Îľ rel are scalar weights. We describe each penalty below. Overlap Penalty (L overlap ).Body interpenetration is a common artifact in multi-person generation. We penalize configurations in which any two charactersâ root (pelvis) positions on the XZ plane are closer than a minimum dis- tanced min or farther thand max : L overlap = 1 T T ! t=1 " max(0,d min ââp t 1 âp t k â) + max(0,âp t 1 âp t k ââd max ) # , (4) wherep t i âR 2 is the XZ position of the pelvis of personi at framet. Whenk>3, we additionally penalize per- sonkagainst all previously generated persons (excluding the pivot) using a separate distance thresholdd other . Root Position Penalty (L root ).To control start and/or end positions, we define a target root positiong i âR 2 for per- soniand penalize deviation: L root = max(0,âp Îľ i âg i â 2 âθ),(5) whereĎis the target time frame (start or end) andθis a tolerance margin. Movement Region Penalty (L region ).When a spatial re- gion constraint is specified (e.g., âperson must stay within a6mâ6m rectangle centered at the originâ), we penalize 3 GT defines the reference character as kâ (from an interaction list), not fixed to person 1; the pivot can change, so restricting it to person 1 is incorrect. GT allows kâ to be any prior character, not always person 1, so fixing the pivot to person 1 is incorrect. GT applies temporal inpainting at each denoising step and uses masked conditioning with noisy references, not as initial noise construction. GT (line 230) uses a noisy reference at each timestep, not a fixed clean pivot motion. GT defines overlap loss only for distances below δ, with no d_max penalty; the claim adds a non- existent farther- than- d_max term. frames where the root position violates the boundary: L region = 1 T T ! t=1 â[p t k /âR]¡dist(p t k ,R),(6) whereRis a circular or rectangular region and dist(¡,R)is the distance from the boundary. Facing Orientation Penalty (L orient ).To control relative facing directions between persons, we define a target cosine similarityËsbetween the forward-facing unit vectorsn t 1 and n t k derived from each personâs hip joints: L orient = 1 T T ! t=1 max(0,cos min ân t 1 ¡n t k ),(7) wherecos min is a threshold (e.g.,0.9) above which the con- straint is considered satisfied. Relative Position Penalty (L rel ).To specify the desired relative spatial arrangement between persons (e.g., âper- sonkstands to the left of person 1â), we define a target relative displacement vectorÎľ 1k âR 2 in the XZ plane and penalize deviation from it: L rel = 1 T T ! t=1 max $ 0,â(p t k âp t 1 )âÎľ 1k â 2 âθ rel % ,(8) whereθ rel is a tolerance margin allowing small devia- tions. This penalty complements the overlap penalty by not only preventing characters from being too close, but ac- tively guiding their arrangement into semantically meaning- ful configurations. Optimization.We optimizez T using Adam with learning rate1â10 â2 and early stopping whenL<Ď stop = 10 â3 , running up toN iter = 50iterations. Loss weights are fixed toÎľ r = 10,Îľ reg =Îľ o =Îľ rel =1across all experi- ments and group sizes. Gradients flow through the entire denoising chain by retaining the computation graph during the reverse diffusion. Gradient norms are clipped to1.0be- fore each parameter update to prevent instabilities that arise early in the optimization trajectory, when denoising paths are farthest from the data manifold. The optimization is performed on a single GPU and does not modify any model parameters. 4.4. Long-Duration Generation via Inpainting For long-duration scenarios, we use motion inpainting to extend a completed short sequencex [0,T 0 ] by an additional !Tframes. The extension shares an overlap window of T ov frames with the existing sequence; the noise for these Figure 2. Qualitative results of multi-person interaction generation with PINO. From top to bottom: three-person, four-person, and five-person scenarios. Each group is generated sequentially, with person 1 serving as pivot. Despite increasing group size, charac- ters maintain clear separation and semantically meaningful inter- actions. frames is initialized with the existing motion rather than pure Gaussian noise (analogous to Eq. (2) applied to tem- poral dimensions). An additional boundary smoothness penalty penalizes discontinuities in joint velocities at the transition frame: L smooth =â Ě x (i) T 0 â Ě x (i) T 0 +1 â 2 ,(9) where Ě x (i) t denotes the joint velocity of personiat framet. The overlap penalty from Eq. (4) is also applied through- out the extended sequence to maintain physical plausibil- ity. The extended sequence is obtained by concatenating the original and newly generated segments at the overlap boundary. 4.5. Per-Pair Prompt Switching A key advantage of the sequential pairwise generation framework is that each pair(1,k)can be conditioned on a distinct text promptc 1k describing the specific interaction between the pivot and personk. For example, in a five- person group, one might specify: â˘c 12 : âTwo people shake hands.â â˘c 13 : âTwo people high-five.â â˘c 14 : âTwo people bow to each other.â â˘c 15 : âTwo people dance together.â This fine-grained control is impossible with shared-prompt methods (e.g., InterGen applied to a single joint prompt for all persons), where all characters tend to perform the same 4 GT explicitly states: 'we set the learning rate to 0.003' explicitly states: 'conduct 100 optimizati on steps to yield optimal results', not 50 as claimed. GT states the boundary penalty minimizes acceleration, not velocity discontinuiti es; the claim is incorrect. or similar motion. Our ablation study (Section5.5) confirms the contribution of each loss component, including how the noise optimization enforces different spatial roles. 5. Experiments 5.1. Experimental Setup Dataset.We evaluate PINO on the InterHuman dataset [10], a large-scale collection of two-person in- teraction motion sequences (approximately 107M frames across 23,337 text-annotated interactions). For multi- person evaluation we compose two-person pairs from this dataset, and for five-person evaluation we extend the composition chain. Baselines.We compare against: â˘InterGen[10]: The pretrained two-person diffusion model used as our backbone. For multi-person scenar- ios, InterGen is applied with inpainting (denotedInter- Gen (inpainting)). â˘FreeMotion[2]: A ControlNet-based approach that sup- ports number-free multi-person generation. Our method is denotedPINO-InterGento indicate the use of InterGen as the base model. Metrics.We evaluate along two axes: â˘Non-semantic (physics) metrics:Overlap(fraction of frames with root distance below threshold),Penetration Volume(PenVol; cm 3 , volumetric interpenetration),Foot Skate(foot sliding artifact), andMax Acceleration(Max Acc.; physical plausibility of dynamics). â˘Semantic metrics:FID(Fr Ě echet Inception Distance, com- puted on motion features from a pretrained encoder),Di- versity(Div.), andR-Precision(Top-3; motion-text re- trieval accuracy). Implementation Details.All experiments use InterGenâs pretrained checkpoint without any fine-tuning. DDIM sam- pling runs forT ddim = 50steps with a classifier-free guid- ance scale of2.5. Noise optimization for each additional person requires a separate gradient-enabled forward pass through the full denoising chain. Distance thresholds for the overlap penalty are set tod min =0.5m andd max =2.0m for pivot pairs, andd other =0.3m for non-pivot pairs. For long-duration generation, the temporal overlap window is fixed atT ov = 30frames. All hyperparameters are held constant across group sizes and datasets, demonstrating the robustness of PINO to hyperparameter choices. 5.2. Two-Person Interaction Generation Table2reports results for two-person interaction genera- tion with overlap avoidance. PINO achieves essentially zero MethodOverlapâPenVol. (cm 3 )âFoot SkateâMax Acc.â GT0.029471.750.0990.074 InterGen0.1193112.720.1240.034 PINO-InterGen0.000275.650.1250.034 (a) Non-semantic metrics. MethodFIDâDiv.âR-Prec.â(Top3) GT0.983 7.9210.715 InterGen13.278 7.7930.674 PINO-InterGen13.163 7.9040.675 (b) Semantic metrics. Table 2. Evaluation of two-person interaction generation while avoiding overlap. MethodOverlap!PenVol.!Foot Skate!Max Acc.! InterGen (inpainting) 0.766 25215.780.0990.030 PINO-InterGen0.000785.010.0910.031 Table 3. Evaluation of multi-person interaction generation overlap (0.000) and dramatically reduces penetration vol- ume (275.65cm 3 ) compared to InterGen (0.119overlap, 3112.72cm 3 PenVol). Remarkably, the semantic quality is preserved or improved: FID drops from13.278(InterGen) to13.163(PINO-InterGen), and Diversity (7.904vs.7.793) and R-Precision (0.675vs.0.674) are both slightly better. Ground-truth (GT) statistics are provided as upper bounds on quality. Foot Skate and Max Acceleration remain on par with InterGen, showing that noise optimization does not in- troduce spurious dynamics. 5.3. Multi-Person Interaction Generation Table3compares PINO against InterGen with inpaint- ing for multi-person (three-person and beyond) generation. PINO eliminates overlap entirely (0.000vs.0.766) and re- duces PenVol by a factor ofâ32(785.01vs.25215.78), while also improving Foot Skate (0.091vs.0.099). Max Acceleration is slightly higher for PINO (0.031vs.0.030), but the difference is negligible. These results confirm that the sequential pairwise framework with noise optimization effectively prevents physical artifacts as the group grows. Scaling to Five Persons.Table4reports pairwise FID and overlap for groups of up to five persons (where pair (1,k)reports metrics for persons 1 andk). PINO consis- tently achieves low overlap across all pairs, reaching only 0.069for pair(1,5)compared to0.977for InterGen and 0.991for FreeMotion. FID is also lower for all pairs: e.g., 16.911vs.19.501(InterGen) and25.671(FreeMotion) for pair(1,5). Foot Skate and Max Acceleration are slightly higher than InterGenâs in some pairs, a minor trade-off for the dramatic reduction in physical artifacts. Fig.2illustrates the diversity of generated group interactions. 5 GT defines Overlap as a sequence- level probability (any pair, any frame below threshold), not a per- frame fraction. MethodPair FID!Overlap!Foot Skate!Max Acc.! FreeMotion(1,2) 19.383 0.2550.1120.034 (1,3) 25.184 0.8060.1220.038 (1,4) 25.812 0.9640.1270.040 (1,5) 25.671 0.9910.1300.042 InterGen(1,2) 13.100 0.1190.1260.034 (1,3) 19.414 0.7990.1250.036 (1,4) 19.386 0.9370.1240.038 (1,5) 19.501 0.9770.1240.039 PINO-InterGen (1,2)12.920 0.0000.1270.034 (1,3)17.170 0.0020.1350.040 (1,4)16.890 0.0190.1380.043 (1,5)16.911 0.0690.1420.045 Table 4. Evaluation of multi-person interaction generation. From pair (1,2), interactions are generated incrementally using 1 as a pivot. Overlap is among all individuals generated up to that step. Figure 3. Qualitative comparison of multi-person interaction gen- eration. InterGen (inpainting) produces severe body overlap arti- facts. PINO maintains clear separation between all persons across frames. MethodOverlap!PenVol.!Foot Skate!Max Acc.! InterGen (inpainting) 0.672 15622.160.0700.044 PINO-InterGen0.0311375.240.0450.043 Table 5. Evaluation of non-semantic elements in motion extension. 5.4. Long-Duration Motion Extension For the motion extension task, we evaluate PINO in two complementary ways. Non-semantic evaluation.Table5shows that the ex- tended motions generated by PINO have significantly lower overlap (0.031vs.0.672), lower PenVol (1375.24vs. 15622.16), and lower Foot Skate (0.045vs.0.070) and Max Acceleration (0.043vs.0.044) compared to InterGen in- painting. The boundary smoothness penalty effectively pre- vents discontinuities at the transition frame. Semantic evaluation.Table6evaluates semantic qual- ity by cropping extended sequences to match ground-truth lengths. PINO achieves a lower FID (18.930vs.20.390), zero overlap (0.0vs.0.089), lower Foot Skate (0.054vs. 0.081), and lower Max Acceleration (0.025vs.0.036). This confirms that noise-optimized inpainting not only avoids ar- tifacts but also maintains the semantic fidelity of the gener- ated motions. MethodFID!Overlap!Foot Skate!Max Acc.! InterGen20.390 0.0890.0810.036 PINO-InterGen18.9300.00.0540.025 Table 6. Evaluation of semantic elements in two-person motion extension. Extended motions are cropped to match the ground truth length for evaluation. MethodPos. Err.âOverlapâReg. Viol.âOrient. Err.â InterGen1.00.2920.5001.0 +L root 0.00.3330.9171.0 +L overlap 0.00.00.9581.0 +L region 0.0830.00.0431.0 +L orientation 0.0830.0430.0830.208 Table 7. Ablation study on motion penalties. Metrics include po- sitional errors (Pos. Err.â), overlap (Overlapâ), region violations (Reg. Viol.â), and orientation errors (Orient. Err.â). Positional errors are measured with a threshold of 20 cm, region violations with a threshold of 10 cm, and orientation errors with a threshold of 20 degrees. Lower values indicate better performance. 5.5. Ablation Study To understand the contribution of each penalty term, Ta- ble7reports spatio-temporal control metrics as losses are added incrementally. Starting from the vanilla InterGen baseline (all metrics at their worst), addingL root reduces positional error to0.0. AddingL overlap eliminates overlap to 0.0. AddingL region reduces region violations from0.958to 0.043. Finally, addingL orient brings orientation error from 1.0down to0.208âthe residual error reflects the inher- ent ambiguity of orientation in complex motions. Notably, each penalty targets its corresponding metric without sig- nificantly degrading others, validating the modular design of our loss function. 5.6. Runtime Analysis Noise optimization introduces additional inference time rel- ative to vanilla diffusion sampling. On a single NVIDIA A100 GPU, generating a two-person sequence ofT= 150 frames with PINO requires approximately90seconds, com- pared to under5seconds for standard InterGen. For a five-person group, the total generation time scales approx- imately linearly with group size, reaching roughly6min- utes. Although longer than baseline sampling, this overhead avoids any retrainingâa process requiring hours on the full InterHuman dataset. The per-person optimization steps are independent given the pivot motion and can therefore be parallelized across GPUs to reduce wall-clock time further. 5.7. Discussion Taken together, the experimental results reveal a consistent theme: PINOâs training-free noise optimization effectively decouples physical plausibility from semantic quality. Un- like approaches that enforce physical constraints through auxiliary loss terms during trainingâwhich risk overfitting 6 GT shows individual penalties degrade other metrics; improveme nts occur only when combined, not per- penalty. Figure 4. Visualization of the ablation study. Left to right: In- terGen baseline, addingL root ,+L overlap ,+L region ,+L orient . Each penalty progressively enforces its corresponding spatial constraint while preserving overall motion quality. the model to specific physics regimesâour inference-time optimization adapts to any spatio-temporal constraint ex- pressible as a differentiable penalty. This flexibility makes PINO particularly well-suited for applications requiring on- the-fly customization, such as interactive animation tools or scenario-specific motion synthesis for robotics simula- tion. Moreover, the ablation results in Section5.5confirm that each penalty term targets its respective artifact inde- pendently, which simplifies hyperparameter tuning: prac- titioners can selectively enable or disable individual con- straints without destabilizing the overall optimization. The consistent gains in physical plausibility metrics across two- person, multi-person, and long-duration settings further val- idate the generality of the pivot-anchored pairwise decom- position as a scalable interaction generation paradigm. One notable observation is that noise optimization acts as a form of post-hoc constraint satisfaction rather than a learned prior; consequently, the quality of generated mo- tions is bounded by the capacity of the underlying In- terGen model. When a text prompt specifies an interac- tion underrepresented in InterGenâs training distribution, the optimizer may require additional iterations to balance the semantic and physical objectives simultaneously. A promising direction for future work is to combine our inference-time optimization with lightweight fine-tuning on in-domain data, achieving improved semantic fidelity at lower computational cost. Additionally, the pairwise generation paradigm assumes group interactions can be adequately captured through bilateral relationships with a shared pivot. While this holds for common scenar- iosâgroup conversations, coordinated exercises, or team handshakesâit may be less appropriate for interactions re- quiring simultaneous coupling among three or more per- sons, such as a group lift, where higher-order dependencies cannot be cleanly decomposed into pairwise terms. It is also worth highlighting that PINOâs modular penalty design provides a natural interface for domain experts to specify interaction semantics. A choreographer, for in- stance, could define custom region and orientation con- straints to stage a group dance performance, while a robotics engineer could specify tight spacing and collision- avoidance distances for coordinated manipulation tasks. This versatility positions noise optimization as a general- purpose inference-time control mechanism for generative motion models, extending well beyond the academic bench- marks considered here. 6. Limitations While PINO achieves strong results, several limitations re- main. First, the overlap penalty operates on root (pelvis) distances and does not model the full 3D body mesh; in highly contorted poses, slight limb penetrations can still oc- cur. Second, noise optimization increases inference time compared to vanilla diffusion sampling, as gradients must be backpropagated through the entire denoising chain at each optimization iteration. Third, as the group size grows, the pivot-anchored approach may limit the diversity of per- sonkâs motion, since it is always conditioned on person 1. Exploring alternative graph topologies (e.g., ring or tree structures) for pivot assignment is left as future work. Fi- nally, the current framework generates all personsâ mo- tions in a fixed XZ ground plane, which may not capture all interaction geometries (e.g., stacked, airborne motions). Furthermore, backpropagating gradients through the full DDIM denoising chain requires retaining the entire com- putation graph in GPU memory, which can become a bot- tleneck for very long sequences (Tâ150) or large groups on memory-constrained hardware; gradient checkpointing strategies could mitigate this cost in future implementa- tions. 7. Conclusion We have presented PINO, a training-free framework for generating physically plausible multi-person interactions of arbitrary group size. By decomposing group dynam- ics into a sequence of pairwise diffusion generations an- chored by a pivot character, and by optimizing the ini- tial noise with differentiable physics-based penalties, PINO achieves near-zero body overlap and dramatically reduced penetration volumes without any additional training. Per- 7 pair text prompt switching enables fine-grained role assign- ment within large groups. A motion inpainting strategy with boundary smoothness penalties extends the framework to long-duration sequences. Extensive experiments on Inter- Human demonstrate consistent improvements in physical plausibility metrics while maintaining competitive seman- tic quality. Acknowledgements.Our code is based on InterGen [10]. Part of the code is adapted from ProgMoGen [11]. References [1]Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your com- mands via motion diffusion in latent space. 2023.1, 2 [2]Ke Fan, Junshu Tang, Weijian Cao, Ran Yi, Moran Li, Jingyu Gong, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Lizhuang Ma. Freemotion: A unified framework for number-free text-to-motion synthesis. 2024.1,2,5 [3]Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. 2022.1,2 [4]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denois- ing diffusion probabilistic models. pages 6840â6851, 2020.2 [5]Muhammad Gohar Javed, Chuan Guo, Li Cheng, and Xingyu Li. Intermask: 3d human interaction gen- eration via collaborative masked modelling.arXiv preprint arXiv:2410.10010, 2024.2 [6]Korrawe Karunratanakul, Konpat Preechakul, Supa- sorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. pages 2151â2162, 2023.2 [7]Korrawe Karunratanakul, Konpat Preechakul, Emre Aksan, Thabo Beeler, Supasorn Suwajanakorn, and Siyu Tang. Optimizing diffusion noise can serve as universal motion priors. 2024.1,2 [8]Taku Komura, Edmond S. Ho, and Rynson W. H. Lau. Animating reactive motion using momentum- based inverse kinematics.Computer Animation and Virtual Worlds, 16(3-4):213â223, 2005.2 [9]Jihyun Lee, Shunsuke Saito, Giljoo Nam, Minhyuk Sung, and Tae-Kyun Kim. Interhandgen: Two-hand interaction generation via cascaded reverse diffusion. 2024.2 [10]Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions. pages 1â21, 2024.1,2,5,8 [11]Hanchao Liu, Xiaohang Zhan, Shaoli Huang, Tai- Jiang Mu, and Ying Shan. Programmable motion gen- eration for open-set motion control tasks. 2024.1,2, 8 [12]Mathis Petrovich, Michael J. Black, and G Ě ul Varol. Action-conditioned 3d human motion synthesis with transformer vae. 2021.2 [13]Mathis Petrovich, Michael J Black, and G Ě ul Varol. Temos: Generating diverse human motions from tex- tual descriptions. 2022.2 [14]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning trans- ferable visual models from natural language supervi- sion. InICML, 2021.2 [15]Yoni Shafir, Guy Tevet, Roy Kapon, and Amit Haim Bermano. Human motion diffusion as a generative prior. 2024.1,2 [16]Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifeoma Nwogu, Guo-Jun Qi, and Mitch Hill. Towards open domain text-driven synthesis of multi- person motions. 2024.1,2 [17]Hubert P. H. Shum, Taku Komura, and Shuntaro Ya- mazaki. Simulating competitive interactions using singly captured motions. InProceedings of the 2007 ACM Symposium on Virtual Reality Software and Technology, pages 65â72, 2007.2 [18]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. InProceedings of the International Conference on Learning Repre- sentations (ICLR), 2020.2 [19]Mikihiro Tanaka and Kent Fujiwara. Role-aware in- teraction generation from textual description. 2023. 2 [20]Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-Or, and Amit Haim Bermano. Human motion diffusion model. 2023.1,2 [21]Jonathan Tseng, Rodrigo Castellon, and C. Karen Liu. Edge: Editable dance generation from music. 2023.2 [22]Zhenzhi Wang, Jingbo Wang, Yixuan Li, Dahua Lin, and Bo Dai. Intercontrol: Zero-shot human interac- tion generation by controlling every joint. InNeurIPS, 2024.1,2 [23]Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint at any time for human motion generation. 2024.2 [24]Liang Xu, Ziyang Song, Dongliang Wang, Jing Su, Zhicheng Fang, Chenjing Ding, Weihao Gan, Yichao Yan, Xin Jin, Xiaokang Yang, et al. Actformer: A gan- based transformer towards general action-conditioned 3d human motion generation. 2023.2 8 [25]Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. 2023.1 [26]Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. 46(6):4115â4128, 2024.1,2 [27]Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming yu Liu. Diffcollage: Parallel genera- tion of large content with diffusion models. 2023.2 9 PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groups Anonymous ICCV Submission Abstract Generating text-driven human interactions beyond two peo- ple remains difficult because interaction complexity grows combinatorially with group size, while existing models are typically trained for a fixed number of actors and often pro- duce collisions, shared-prompt ambiguity, or limited spa- tial control. We present PINO, a training-free framework that composes arbitrary-sized group interactions from a pretrained two-person diffusion model by optimizing the initial diffusion noise at inference time. PINO adds one new character at a time around a shared pivot person, reusing the pretrained model through masked condition- ing so previously generated motion is preserved while pair- specific prompts define the role of the newly added partici- pant. The optimization objective combines collision-aware and spatio-temporal penalties, enabling overlap avoidance, root-position control, region constraints, and orientation control without retraining the backbone model. PINO also extends interactions to longer durations through inpainting- based continuation with optimized unobserved segments. When instantiated on InterGen, PINO reduces overlap from 0.119 to 0.000 and penetration volume from 3112.72 to 275.65 in two-person generation while preserving seman- tic quality. In multi-person generation, it reduces overlap from 0.766 to 0.000 and penetration volume from 25215.78 to 785.01, and it continues to scale favorably up to five peo- ple. For motion extension, PINO improves both semantic fidelity and physical plausibility, reducing FID from 20.390 to 18.930 and overlap from 0.089 to 0.000. These results show that inference-time noise optimization is a practical route to controllable and physically coherent group inter- action synthesis without additional training. 1. Introduction Text-driven human motion generation has advanced rapidly with diffusion models for single-person motion and two- person interaction synthesis [2,6,11,14]. However, gen- erating realistic interactions for arbitrary-sized groups re- mains substantially harder than the two-person case. The main difficulty is not only modeling additional bodies, but preserving coherent semantics while preventing trivial fail- ure modes such as body overlap, interpenetration, and role ambiguity across multiple participants. Existing approaches address this challenge from two main directions. One line trains dedicated multi-person generators that directly model several people at once [1,9]. This offers expressive generation, but the architecture and training data are tied to the group distributions seen dur- ing training. Another line keeps pretrained priors fixed and applies inference-time control or optimization [5,7,12]. These methods improve flexibility, but they do not directly exploit the structure of human group interactions as a com- position of localized pairwise relations. Our starting observation is that many group interactions can be organized around a shared pivot person. For exam- ple, a central character may greet one person, guide another, and coordinate with several surrounding partners. This sug- gests a simple decomposition: instead of asking one model invocation to explain the entire group, we can sequentially compose pairwise interactions between the pivot and each newly added person. A pretrained two-person interaction diffusion model already contains a strong prior over pose re- alism, timing, and semantic interaction. The missing com- ponent is a mechanism that preserves this prior while en- forcing group-level physical consistency and user control. Based on this idea, we proposePINO(Person- InteractionNoiseOptimization), a training-free framework for long-duration and customizable group interaction gener- ation. PINO uses a pretrained two-person diffusion model as a frozen backbone, then optimizes the initial diffusion noise for each pairwise generation step. Masked con- ditioning keeps the pivot motion fixed, while the newly added character is synthesized from a pair-specific text prompt. The optimization objective is defined directly in motion space and includes penalties for overlap avoidance, start/end root positions, movement regions, and orientation. This retains the semantic prior of the pretrained model while giving the user precise inference-time control. PINO also supports long-duration interaction generation. Rather than retraining a dedicated long-horizon model, we extend a motion sequence by treating known frames as fixed conditions and optimizing the unobserved segment 1 Agent: Codex (GPT5.4) Rubric Score: 3.67 Major Contradiction: 3 Figure 1.Person-Interaction Noise Optimization (PINO).Starting from a pretrained two-person diffusion model, PINO generates an initial pair and then sequentially adds new participants around a pivot person through masked conditioning. The initial noise for the new pair is optimized with physics-aware and spatio-temporal penalties, enabling collision avoidance and controllable group composition without retraining. through inpainting. This preserves continuity while keeping the same inference-time control mechanism used for group composition. Experiments on InterGen-based generation show a con- sistent pattern: PINO yields the largest gains in physi- cal plausibility and scalability, while preserving or mod- estly improving semantic metrics. In two-person genera- tion, PINO eliminates overlap and sharply reduces penetra- tion volume without degrading FID. In multi-person gener- ation, the gap becomes larger, especially as the number of people increases. In motion extension, PINO improves both semantic fidelity and non-semantic realism. Our contributions are threefold. First, we introduce a training-free framework that composes arbitrary-sized group interactions from a pretrained two-person diffusion model through sequential pivot-based generation. Second, we formulate inference-time person-interaction noise opti- mization with collision-aware and spatio-temporal penal- ties, enabling customizable control without retraining. Third, we show that the same mechanism extends to long- duration interaction generation through inpainting-based continuation. 2. Related Work 2.1. Diffusion Models for Human Motion Diffusion models have become a standard backbone for text-to-motion generation because they produce diverse and realistic trajectories while supporting flexible condition- ing [3,10,11,14]. Subsequent work improved fidelity and controllability through masked modeling, latent-space gen- eration, or stronger priors [2,8]. These methods focus pri- marily on single-person motion and do not directly address multi-person physical constraints. PINO instead treats a pretrained motion diffusion model as a reusable prior and moves the control problem to inference time. 2.2. Interactive and Multi-Person Motion Genera- tion Two-person interaction generation has recently become fea- sible with specialized architectures and datasets, especially through InterGen [6]. More recent interactive generators further improve fidelity and reaction modeling, but they still target a fixed number of participants during training. For larger groups, FreeMotion [1] and Shan et al. [9] move to- ward open-domain multi-person generation, yet they model multiple people jointly and therefore inherit the difficulty of scaling network design and training data to larger groups. PINO takes a different route: it reuses a pretrained two- person model and composes larger groups sequentially, which makes the number of generated people an inference- time choice rather than a training-time design decision. 2.3. Inference-Time Control and Optimization Inference-time control has emerged as an attractive al- ternative to retraining motion generators. Guided Mo- tion Diffusion [4] and OmniControl [13] incorporate spa- tial constraints for single-person motion generation. Pro- grammable Motion Generation [7] and Diffusion Noise Op- timization [5] show that optimizing latent variables or dif- fusion noise can satisfy diverse motion-space constraints without task-specific retraining. In the multi-person set- ting, InterControl [12] uses joint-level interaction control for zero-shot synthesis. PINO is closest in spirit to these training-free methods, but differs in two ways: it explicitly decomposes group generation into pivot-centered pairwise interactions, and it uses the optimization to preserve physi- cal consistency among already generated participants while allowing prompt switching across pairs. 3. Method 3.1. Problem Setup We assume access to a pretrained text-conditioned two- person diffusion modelGsuch as InterGen [6]. Given a text 2 GT shows FreeMotio n uses single- person sequential generatio n, not joint multi- person modeling; grouping it with Shan et al. is incorrect. Method #Person in Network Design #Person in Training Data Order of Multi-Person Generation Method InterControl [12]OneOneSequentialControlNet FreeMotion [1]OneTwoSequentialControlNet Shan et al. [9]MultipleMultipleConcurrent Supervised Training ProgMoGen [7]OneTraining-free N/A (Single-Person) Noise Optimization Ours Two (or multiple) Training-freeSequential Noise Optimization Table 1. Comparison of recent multi-person interaction generation methods, including single-person method ProgMoGen [7]. promptp, the model maps an initial Gaussian noise tensor zto a motion sequence Ë X=G(z,p)containing two in- teracting people. Each person is represented by a sequence of motion features; in the InterGen instantiation, a single person uses 262 features per frame, and optimization is per- formed on 210-frame windows. Our goal is to generate a group ofNinteracting peo- ple, potentially with pair-specific prompts and motion con- straints, without changing the pretrained weights ofG.We also want the same mechanism to support motion extension beyond the native window length of the pretrained model. 3.2. Pivot-Based Pairwise Composition The key design choice in PINO is to organize group gener- ation around apivotperson shared across pairwise genera- tions. We first sample a two-person interaction from prompt p 1,2 to obtain the pivot motionX (1) and the second partici- pant motionX (2) . To add a new participantk, we invoke the same two-person diffusion model again with a new prompt p 1,k , but now one input slot is fixed to the pivot motion: Ë X (1,k) =G(z k ,p 1,k ;X (1) ,M),(1) whereMis a binary mask that preserves the pivot features and leaves the new participant unconstrained. This masked conditioning is implemented through the inpainting interface of the frozen backbone. Concretely, the known coordinates of the pivot motion are copied into the denoiser output at every reverse step, so optimization only modifies the unconstrained participant. The result is a new motionX (k) that interacts with the pivot while remaining compatible with previously generated people. Repeating this process sequentially yields groups of arbitrary size. This decomposition has two benefits. First, it avoids ask- ing a single prompt to explain the roles of all participants at once. Each pair can have its own prompt, which allows prompt switching and clearer role assignment. Second, it leverages a pretrained interaction prior that already mod- els close-range coordination, while group-level consistency is handled by optimization. The pivot assumption is also computationally useful. Keeping one actor fixed across gen- erations propagates a stable temporal reference into every new pair, which reduces the number of unconstrained bod- ies that the optimizer must coordinate simultaneously. In practice, this turns a hard many-body search problem into a sequence of smaller constrained updates while still allow- ing non-pivot participants to interact indirectly through the shared spatial layout. 3.3. Person-Interaction Noise Optimization For each generation step, PINO optimizes the initial dif- fusion noise rather than retraining or fine-tuning the back- bone. Letz 0 k be an initial random noise sample for the pair consisting of the pivot and a new target person. PINO solves z Ď k = arg min z k Ď ov L overlap +Ď root L root +Ď reg L region +Ď ori L orientation , (2) where all losses are computed on the decoded motion. Overlap loss.The default objective encourages the newly generated person to maintain a valid distance from the pivot and from all previously generated non-pivot participants. In the released implementation, this is expressed through root-distance penalties on the pelvis trajectory, with a min- imum distance threshold for collision avoidance and an op- tional maximum threshold to prevent degenerate separa- tions. When generating thek-th person, the loss also pushes that person away from already generated people other than the pivot. Root-position loss.To support temporal anchors, PINO can constrain the start and end root positions of one or both participants. This is useful when the user needs a character to enter or exit at a prescribed location, or when the motion must align with external timing constraints. Region loss.PINO can restrict a participant to remain in- side or outside a spatial region. The implementation sup- ports circular or rectangular regions on the ground plane, enabling scene-aware placement or motion zoning without retraining a scene-conditioned model. Orientation loss.An orientation penalty aligns the par- ticipantâs facing direction with a target direction. This is especially useful for prompts whose semantics depend on mutual attention, such as facing a pivot before approaching or gesturing. The optimization is fully differentiable because gradients propagate from the motion-space objective through the de- noising process back to the initial noise. In practice, we use Adam for up to 100 iterations with early stopping, following the optimization loop implemented in the released code. 3 GT defines L = L_overlap + L_control (including relative position), not a weighted four-term loss; the claim omits a key term and misstates the structure. Algorithm 1PINO inference for group composition 1:Generate(X (1) ,X (2) )âG(z 2 ,p 1,2 ) 2:fork=3toNdo 3:Samplez k and fix pivot motionX (1) with maskM 4:Optimizez k using Eq. (2) 5:DecodeX (k) âG(z Ď k ,p 1,k ;X (1) ,M) 6:end for 7:returnX (i) N i=1 3.4. Prompt Switching and Customizable Roles Because each pairwise generation step uses its own text prompt, PINO naturally supports prompt switching across participants. A single group motion can therefore mix se- mantically different relations around the same pivot, such as handshaking with one person while addressing or guid- ing another. This is difficult for shared-prompt approaches, which often blur individual roles when several people must simultaneously satisfy one global textual description. 3.5. Long-Duration Motion Extension The same formulation can extend interactions beyond the native duration of the pretrained model. We keep a subset of frames fixed as observed motion, mask them through the inpainting interface, and optimize the noise for the remain- ing interval. This reuses the pretrained prior for local mo- tion realism while allowing the unobserved portion to adapt to collision-aware and control-oriented objectives. In prac- tice, the known segment acts as a temporal anchor at the transition boundary, producing longer interactions without training a separate long-horizon model. 4. Experiments 4.1. Experimental Setup We instantiate PINO on InterGen [6] and denote the result- ing system asPINO-InterGen. The evaluation follows the InterHuman protocol used by the provided codebase and reports both semantic and non-semantic metrics. Seman- tic quality is measured by FID, diversity, and R-precision when available. Physical plausibility is measured by over- lap, penetration volume, foot skating, and maximum accel- eration. For multi-person generation, we compare against InterGen with inpainting-based composition and against FreeMotion [1] where available. Unless otherwise noted, PINO keeps the pretrained gen- erator frozen and applies optimization only at inference time. This matters because improvements cannot be at- tributed to additional supervised training or changes in backbone capacity. MethodOverlapâPenVol. (cm 3 )âFoot SkateâMax Acc.â GT0.029471.750.0990.074 InterGen0.1193112.720.1240.034 PINO-InterGen0.000275.650.1250.034 (a) Non-semantic metrics. MethodFIDâDiv.âR-Prec.â(Top3) GT0.983 7.9210.715 InterGen13.278 7.7930.674 PINO-InterGen13.163 7.9040.675 (b) Semantic metrics. Table 2. Evaluation of two-person interaction generation while avoiding overlap. MethodOverlap!PenVol.!Foot Skate!Max Acc.! InterGen (inpainting) 0.766 25215.780.0990.030 PINO-InterGen0.000785.010.0910.031 Table 3. Evaluation of multi-person interaction generation 4.2. Two-Person Interaction Generation with Over- lap Avoidance We first evaluate whether PINO can improve physical plau- sibility in the base two-person setting without sacrificing se- mantic quality. Table2shows that it does. PINO-InterGen reduces overlap from 0.119 to 0.000 and penetration volume from 3112.72 to 275.65. At the same time, FID slightly improves from 13.278 to 13.163, diversity increases from 7.793 to 7.904, and R-precision is marginally higher. These results are important because they isolate the ef- fect of noise optimization from the more complex multi- person setting. The gain is not coming from a different mo- tion prior; it comes from using the same pretrained prior more effectively under a physically informed objective. 4.3. Multi-Person Interaction Generation We next evaluate the main setting of interest: composing larger groups by sequentially adding participants around a pivot. Table3compares PINO-InterGen against InterGen with inpainting-based multi-person composition. PINO re- duces overlap from 0.766 to 0.000 and penetration volume from 25215.78 to 785.01, while also slightly improving foot skating. Maximum acceleration is nearly unchanged (0.031 vs. 0.030), indicating that the gain does not come from overly damped motions. To study scaling, Table4reports pairwise quality as the group grows from two to five people using person 1 as the pivot. PINO-InterGen consistently achieves the best FID for every pair and dramatically lowers overlap at each step. The advantage becomes larger as more participants are added: for pair(1,5), FID drops from 19.501 to 16.911 and overlap drops from 0.977 to 0.069 compared with InterGen. Earlier pairs remain near zero overlap, which indicates that sequen- 4 GT selects any prior individual k as reference, not always person 1; fixing the pivot and using pâ,k contradicts the algorithm. Figure 2. Comparison of the generated multi-person interaction. The visualization highlights that PINO produces cleaner spacing and fewer severe collisions when new participants are added sequentially around a pivot. MethodPair FID!Overlap!Foot Skate!Max Acc.! FreeMotion(1,2) 19.383 0.2550.1120.034 (1,3) 25.184 0.8060.1220.038 (1,4) 25.812 0.9640.1270.040 (1,5) 25.671 0.9910.1300.042 InterGen(1,2) 13.100 0.1190.1260.034 (1,3) 19.414 0.7990.1250.036 (1,4) 19.386 0.9370.1240.038 (1,5) 19.501 0.9770.1240.039 PINO-InterGen (1,2)12.920 0.0000.1270.034 (1,3)17.170 0.0020.1350.040 (1,4)16.890 0.0190.1380.043 (1,5)16.911 0.0690.1420.045 Table 4. Evaluation of multi-person interaction generation. From pair (1,2), interactions are generated incrementally using 1 as a pivot. Overlap is among all individuals generated up to that step. tial composition does not accumulate severe geometric fail- ures as quickly as the baseline. 4.4. Long-Duration Motion Extension We finally evaluate motion extension. In the semantic evaluation of Table5, PINO-InterGen improves FID from 20.390 to 18.930, reduces overlap from 0.089 to 0.000, low- ers foot skating from 0.081 to 0.054, and substantially low- ers maximum acceleration from 0.036 to 0.025. In the non- semantic extension benchmark of Table6, PINO likewise improves every reported metric, including overlap (0.672 to 0.031), penetration volume (15622.16 to 1375.24), and foot skating (0.070 to 0.045). Together, these results suggest that inference-time opti- mization is effective not only for adding more people, but also for extending interaction duration while preserving re- alism at the transition. MethodFID!Overlap!Foot Skate!Max Acc.! InterGen20.390 0.0890.0810.036 PINO-InterGen18.9300.00.0540.025 Table 5. Evaluation of semantic elements in two-person motion extension. Extended motions are cropped to match the ground truth length for evaluation. MethodOverlap!PenVol.!Foot Skate!Max Acc.! InterGen (inpainting) 0.672 15622.160.0700.044 PINO-InterGen0.0311375.240.0450.043 Table 6. Evaluation of non-semantic elements in motion extension. 5. Analysis 5.1. Ablation on Motion Penalties Table7isolates the role of different inference-time penal- ties. Starting from plain InterGen, adding the root-position loss eliminates positional error but does not address colli- sions and substantially increases region violations. Adding the overlap loss removes overlap entirely, confirming that the main physical gain in PINO is driven by explicit ge- ometric control rather than being an incidental byproduct of prompt decomposition. Adding the region loss produces the best region-violation score (0.043), while the orientation loss sharply reduces orientation error from 1.0 to 0.208. This ablation supports the design of Eq. (2): no sin- gle penalty solves all control requirements. Instead, the strength of PINO is that these constraints can be combined as needed at inference time, while the pretrained motion prior remains fixed. 5.2. Qualitative Observations Fig.2and Fig.4provide qualitative evidence consistent with the metrics. Compared with direct multi-person com- 5 Figure 3. Visualization of the ablation study. Different penalties affect different aspects of the generated motion, with overlap-oriented terms primarily improving physical plausibility and orientation/region terms improving controllability. position, PINO maintains clearer personal space and pro- duces less entangled body configurations. The visualiza- tions also suggest why pivot-based prompt switching is use- ful: the new participant can adopt a distinct role relative to the pivot instead of collapsing toward an averaged group behavior. 5.3. Discussion The empirical pattern across all benchmarks is consistent. PINOâs largest gains appear in collision-related metrics, es- pecially overlap and penetration volume, and the margin widens as the composition problem becomes harder. Se- mantic fidelity is preserved in the base two-person setting and improves in motion extension, which indicates that op- timizing the initial noise does not simply trade semantics 6 Figure 4. Qualitative results of multi-person interaction generation. PINO supports customizable pairwise interactions within a larger group while maintaining physically cleaner spacing. for geometry. At the same time, some motion smoothness metrics remain close to the baseline rather than improving dramatically, suggesting that PINOâs main benefit is better physical feasibility and controllability rather than a whole- sale change in motion style. 6. Conclusion We presented PINO, a training-free framework for arbitrary-sized and long-duration human interaction gener- ation. PINO composes a group around a pivot person using a pretrained two-person diffusion model, and it optimizes the initial diffusion noise with motion-space penalties to en- force physical plausibility and user control. Experiments show that this simple decomposition scales better than di- rect composition baselines, substantially reduces overlap and penetration, and supports motion extension without re- training. These results indicate that inference-time noise optimization is a strong and practical interface for turning pretrained interaction models into controllable group mo- tion generators. 7 MethodPos. Err.âOverlapâReg. Viol.âOrient. Err.â InterGen1.00.2920.5001.0 +L root 0.00.3330.9171.0 +L overlap 0.00.00.9581.0 +L region 0.0830.00.0431.0 +L orientation 0.0830.0430.0830.208 Table 7. Ablation study on motion penalties. Metrics include po- sitional errors (Pos. Err.â), overlap (Overlapâ), region violations (Reg. Viol.â), and orientation errors (Orient. Err.â). Positional errors are measured with a threshold of 20 cm, region violations with a threshold of 10 cm, and orientation errors with a threshold of 20 degrees. Lower values indicate better performance. 8 References [1]Ke Fan, Junshu Tang, Weijian Cao, Ran Yi, Moran Li, Jingyu Gong, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Lizhuang Ma. Freemotion: A unified framework for number-free text-to-motion synthesis. 2024.1,2,3,4 [2]Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. 2024.1,2 [3]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denois- ing diffusion probabilistic models. pages 6840â6851, 2020.2 [4]Korrawe Karunratanakul, Konpat Preechakul, Supa- sorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. pages 2151â2162, 2023.2 [5]Korrawe Karunratanakul, Konpat Preechakul, Emre Aksan, Thabo Beeler, Supasorn Suwajanakorn, and Siyu Tang. Optimizing diffusion noise can serve as universal motion priors. 2024.1,2 [6]Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions. pages 1â21, 2024.1,2,4 [7]Hanchao Liu, Xiaohang Zhan, Shaoli Huang, Tai- Jiang Mu, and Ying Shan. Programmable motion gen- eration for open-set motion control tasks. 2024.1,2, 3 [8]Yoni Shafir, Guy Tevet, Roy Kapon, and Amit Haim Bermano. Human motion diffusion as a generative prior. 2024.2 [9]Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifeoma Nwogu, Guo-Jun Qi, and Mitch Hill. Towards open domain text-driven synthesis of multi- person motions. 2024.1,2,3 [10]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. 2021.2 [11]Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-Or, and Amit Haim Bermano. Human motion diffusion model. 2023.1,2 [12]Zhenzhi Wang, Jingbo Wang, Yixuan Li, Dahua Lin, and Bo Dai. Intercontrol: Zero-shot human interac- tion generation by controlling every joint. InNeurIPS, 2024.1,2,3 [13]Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint at any time for human motion generation. 2024.2 [14]Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. 46(6):4115â4128, 2024.1,2 9