Paper deep dive

WorldBench: Evaluating LLMs on Three.js Voxel World Generation

Krish Bakshi

Year: 2026Venue: arXiv preprintArea: cs.GRType: PreprintEmbeddings: 75

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 10/9/2026, 7:39:46 AM

Summary

The paper introduces WorldBench, a benchmark and automated judge for evaluating Large Language Models (LLMs) on their ability to generate interactive 3D voxel worlds using Three.js. The authors argue that existing evaluation methods relying solely on code analysis or static visual snapshots are unreliable, often disagreeing on 32% of required items. WorldBench employs a dual-channel judging system that explores the running world (controlling time, orbiting, and using a navigator agent for close-ups) and reads the source code, verifying claims against the actual source text and measured pixel changes. The benchmark evaluates five frontier models (Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7, Gemini 3.1 Pro) on a prompt requiring a floating voxel island with specific biomes, physics, and cycles, scoring them on a 280-point ladder across five tests.

Entities (11)

WorldBench · benchmark · 100%Three.js · technology · 95%Claude Fable 5.1 · model · 90%Gemini 3.1 Pro · model · 90%GPT-6 Astra · model · 90%Grok 4.7 · model · 90%Kimi K3 · model · 90%Mutation Test · method · 90%Navigator Agent · component · 90%T3 Bench · benchmark · 85%WorldCoder-Bench · benchmark · 85%

Relation Signals (10)

WorldBench → evaluates → LLMs

confidence 95% · WorldBench: Evaluating LLMs on Three.js Voxel World Generation

WorldBench → uses → Three.js

confidence 95% · LLM-generated Three.js worlds.

WorldBench → evaluates → GPT-6 Astra

confidence 90% · We evaluate five frontier models: ... GPT-6 Astra ...

WorldBench → evaluates → Kimi K3

confidence 90% · We evaluate five frontier models: ... Kimi K3 ...

WorldBench → evaluates → Gemini 3.1 Pro

confidence 90% · We evaluate five frontier models: ... and Gemini 3.1 Pro.

WorldBench → evaluates → Grok 4.7

confidence 90% · We evaluate five frontier models: ... Grok 4.7 ...

WorldBench → evaluates → Claude Fable 5.1

confidence 90% · We evaluate five frontier models: Claude Fable 5.1...

WorldBench → usescomponent → Navigator Agent

confidence 90% · sending a navigator agent to frame each biome

WorldBench → usesmethod → Mutation Test

confidence 90% · A mutation test, in which we remove features by construction, shows that code-only judging gives full credit...

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated this http URL worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at this https URL

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

71,992 characters extracted from source content.

Expand or collapse full text

WorldBench: Evaluating LLMs on Three.js Voxel World Generation Krish Bakshi krishrajeshbakshi@gmail.com Abstract Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision–language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench. 1 Introduction Asked for a 3D world, a language model no longer returns a description or a single asset. It returns a program: one HTML file that builds terrain from noise functions, places forests from lookup tables, runs water and weather as particle systems, and turns night into day on its own clock. What that world is only exists while the program runs. Grading it means answering two questions at once: does the code build the thing, and does the running world show it? Neither answer implies the other. Code can build an animal at coordinates no camera ever sees, or define a weather system that never switches on. A render can show a feature that no identifier names, such as a terraced noise threshold that is a sandstone mesa. Automatic evaluators usually answer one of the two. Render-based benchmarks score a handful of fixed views with a vision–language model [2,5,10]. They see the result but not its cause, a still frame cannot show a river flowing, and the judge’s verdict is taken as given. For generated Three.js programs, Lu et al.[8]found screenshot-plus-VLM judging essentially uncorrelated with ground truth. Code-based evaluation reads the source, with a model or with patterns. It sees the cause but not the result, and pattern matching is easy to game: on our own early keyword test, a one-line file declaringconst pine=1,...,lava=9scored as highly as every real world. WorldCoder-Bench sidesteps both by requiring each program to expose its state through a fixed interface and checking it against expert-written contracts. That works for functional tasks. It does not fit an open-ended creative prompt: “the desert borders the jungle” or “autumn looks different from spring” is not a state variable anyone can specify in advance, and asking for an interface changes the task being measured. On five frontier models’ worlds, the two views disagree on 32% of required items. The code is there and the frames never show it in 105 of 365 cases, and the frames show it but the code was not found in 10. Disagreement alone does not say which view is right, so we test that on features we remove by construction (Section 5.3). We present WorldBench, a benchmark and judge for open-ended, LLM-written 3D worlds. One prompt asks for a floating voxel island with ten biomes in stated relations, physical water, lava and weather, and day/night and seasonal cycles. Five tests with fixed maxima, 280 points in total, score it item by item. The judge explores 1 arXiv:2610.10622v1 [cs.GR] 7 Oct 2026 READ the code as written(the cause) LOOK at the world as it runs(the result) IJ Generated world oneworld.html; prompt unchanged f Code reader quotes the lines that build each item case 3: t=n2>0.7? T.HROCK:T.HIGH; Explorer wide orbits + close views of each biome ¬ Really there? the quote must be text the source contains ¬ Really shown? biome confirmed blind; pixels decide what they can measure Ǒ Item verdict code ½ frames ½ each half earned only if its check passes →5 tests,/ 280 seen, no code found: search again code-reading judgesstop here: they trust the reader snapshot judgesstop here: a few fixed views, no checks Figure 1: The judge. One generated world is examined through two channels: the code as written and the world as it runs. Each channel’s evidence is checked against something the model under test cannot fake (the source text, measured pixels) before an item’s half is earned. Grey brackets mark where code-reading and snapshot judges stop. Thumbnails and the code line are from claude-fable-5-1. the running world (Figure 1): it takes control of the world’s own clock, measures which render is day instead of assuming it, orbits the island for wide views, and sends a navigator agent in for close views of each biome. It also reads the code for what it is looking at. Neither channel is trusted. A code quote counts only if it is text the source contains, a navigator’s “found it” is re-checked blind, and wherever a property is measurable (brightness, motion, colour shift), pixels decide instead of the visual judge. Our contributions: •A benchmark for open-ended, code-generated 3D worlds. The prompt asks for nothing on the grader’s behalf: one fixed prompt, a 280-point ladder of item-level tests, and a bounded, error-only generation protocol (Section 3). •A judge that explores and reads, and trusts neither, with every model-made claim backed by the source text or by measured pixels before it scores (Section 4). • Evidence for each design choice on five frontier models’ worlds: an agreement analysis, a close-versus- wide ablation, a mutation test with features removed by construction, and an error analysis over four error classes (Section 5). 2 Related work Evaluating generated 3D content. T 3 Bench [5] scores text-to-3D methods from multi-view renders, with a quality metric and an alignment metric in which a language model judges multi-view captions; it reports that current methods struggle most with surroundings and multi-object scenes, the kind of content our island asks for. GT23D-Bench [2] pairs a large annotated asset set with ten metrics validated by correlation with human judgment. SceneEval [10] evaluates indoor scene synthesis on explicit requirements (object counts, attributes, relations) and implicit expectations (support, collisions, navigability), using a vision–language model on fixed renders and geometric checks on the scene’s meshes wherever possible. Our split between required contents (WC003) and physical behaviour (WC004) follows the same explicit/implicit distinction. What differs is the 2 what is judgedlooks at renders reads code explores the world each verdict checked nothing added to output T 3 Bench3D asset✓ fixed views –✗✓ GT23D-Bench3D asset✓ fixed views –✗✓ SceneEvalindoor scene (objects, meshes) ✓ fixed views –✗◦ geometry where possible ✓ WorldCoder-Bench Three.js program, functional task ✗ ◦ scripted actions ✓ state contracts ✗ must expose a state object worldbenchThree.js program, open-ended world ✓ time, seasons, per-biome navigator ✓ source text + pixels ✓ Table 1: Where WorldBench sits among the evaluation benchmarks it cites (generators are discussed in the text).◦: partly. object being judged: these benchmarks score assets or scene data, while we score a program whose output exists only when it runs, and whose camera, clock and seasons we must drive ourselves. Language models that build 3D scenes. A line of systems uses language models to generate scenes rather than grade them. SceneCraft [6] writes Blender Python from a scene graph and refines it with a vision–language model that critiques the render. SceneX [12] plans procedural Blender commands over a curated asset library and demonstrates natural landscapes and seasonal changes, the content closest to ours. Reason-3D [1] uses large reasoning models for object retrieval and collision-aware placement, and SceneOrchestra [4] trains an orchestrator to emit whole tool-call trajectories instead of rendering after every step. These systems give the model scene tools and asset libraries. Our models get one prompt and plain Three.js and must write the procedural generation themselves, repairing from errors only [in the spirit of 3]. Benchmarks of LLM-written interactive worlds. WorldCoder-Bench [8] is the closest work: 2,026 tasks whose output, like ours, is one self-contained Three.js page. It runs each program in a headless browser, requires it to expose a standard state object, applies scripted actions, and checks state changes against hidden contracts written by experts. It also reports that screenshot-based judging fails on these programs. We share the medium and the scepticism about visual judges, but take the opposite task shape: one open-ended world instead of thousands of narrow functional tasks, with nothing exposed for the grader. Where it replaces visual judging with contracts, we keep visual judging and verify it, against the source text and against pixels. Table 1 compares the benchmarks side by side. Models as judges. Strong language models agree with human preferences often enough to serve as judges [11], but they carry biases, including a preference for their own family’s outputs [9]. Most judge pipelines respond by calibrating the judge against people. Our worlds are open-ended and synthetic, so there is no reference answer to annotate. We instead check each judgement against evidence the model under test cannot fake, and validate the judge by mutation [7]: features removed on purpose, where the correct verdict is known by construction. 3 The benchmark Figure 2 shows the whole flow, from prompt to score. This section covers the prompt, how worlds are produced, and how they are scored; Section 4 covers how each item’s evidence is gathered and checked. 3 1 GENERATE2 EVIDENCE 3 SCORE δ Prompt one open-ended spec for all ˎ LLM under test writes one HTML world IJ world.html must load as a Three.js world f Read the code quotes checked against the source Explore sightings checked against pixels Ô Score ½ code + ½ frames per item→/ 280 Figure 2: The evaluation flow. The model under test turns one fixed prompt into a singleworld.html, which must load as a Three.js world. Evidence is then gathered two ways, by reading the code and by exploring the running world, and each kind is checked (quotes against the source, sightings against pixels). One open-ended prompt. Every model receives the same prompt (about 2,000 words, reproduced in Appendix A). It asks for a single, self-contained HTML file that renders, with Three.js, a floating voxel island in a black void: ten biomes (snow mountains, snowy conifer forest, highlands, jungle, swamp, flowering grove, grassland plateau, coastal delta, desert, volcano) placed in stated relations to one another (the forest sits in front of the mountains, the swamp on the jungle’s far side, the volcano away from the desert); water, lava and particles that behave physically; a day/night cycle and four seasons; and a small on-screen legend. The prompt fixes relationships, not coordinates, and says nothing about how the world will be graded. There is no ground-truth world to compare against. The prompt asks for nothing on the grader’s behalf. Grading runs a copy with one clock patch (Section 4). Generation. We evaluate five models, each queried through its developer’s official API: Claude Fable 5.1 (Anthropic), GPT-6 Astra (OpenAI), Kimi K3 (Moonshot AI), Grok 4.7 (xAI) and Gemini 3.1 Pro (Google DeepMind); Table 8 lists the generation settings. Each world is one completion, sampled at temperature 1.0, continued if the response is truncated. Before grading, a bounded repair loop removes crashes that would otherwise hide the whole world: the file is parsed (node –check) and loaded in a headless browser, and any parse error or console error is sent back to the model, which may edit the file with small verified replacements, for at most five rounds. The model is told this budget up front. Repairs see only errors, never renders or scores, so generation never sees the grader’s view. The ladder. A world is graded by five tests with fixed maxima, 280 points in total (Table 2), so totals compare across models. Each test scores many small, checkable items rather than one holistic grade, and every item is decided by the judge described next. Throughout the paper, worked examples use one reference world, claude-fable-5-1, the highest-scoring world. WC002 is the one test of spatial reasoning: the prompt describes the island only through relations, and the world must turn them into a map. A model reads the whole source, finds where the layout decides each cell’s biome (region centres, boxes, bands, coast rules) and reports which biomes touch and how high each sits. It may list two biomes as touching only when the positions it found put them side by side, quoting those numbers. Fixed rules written from the prompt then score that report (Figure 3). The reference world scores 15 of 20 here. The reading is a model’s, so Section 6 checks it against the actual biome maps of four worlds. 4 The judge: explore and read, trust neither Figure 1 shows the idea. Reading looks at the cause: the program as written. Looking looks at the result: the program as it runs. An item scores only for the evidence that survives its check. Several models fill the judge’s roles; they and their settings are listed in Appendix F. Each check below exists because a simpler judge failed on real worlds; Appendix E lists those judges and how each failed. 4 REQUIRED BY THE PROMPT mountains forest highlands jungle swamp grove grassland delta desert volcano must touchone ofmust not touch height: mountains≻forest≻highlands≻jungle≻grassland≻delta DRAWN FROMclaude-fable-5-1(15/20; extracted links) mountains forest highlands jungle swamp grove grassland delta desert volcano requiredmissingforbiddenfails height: mountains≻forest≻volcano≻desert≻jungle≻highlands ≻grassland≻grove≻swamp≻delta Figure 3: WC002, placement, on the reference world. Left: the relations the prompt requires, as the grader encodes them. Right: the links the judge model read fromclaude-fable-5-1’s code, restricted to the pairs the rules mention. It is not a map of the island: node positions are the same fixed layout on both sides. Each biome is coloured by its WC002 verdict; the elevation order read from the code is printed below, with the one violation in red. The forest and highlands fail on a missing link, the highlands also sit below the jungle, and the swamp, desert and volcano each touch a biome they must not. Against this world’s actual biome map, two of these 19 pairs are misread: the forest and highlands do touch, and the desert does not touch the mountains (it still fails on its real border with the delta). testwhat it measuresevidencemax WC001 it is a voxel island: cubes on a grid, contained water, floating over a void code judge (probability + verbatim quote per item) and a visual bug hunt over the frames 40 WC002 all 10 biomes present and placed as asked (neighbours, elevation order) a biome counts only if its layout code is really in the source; a model reads neighbours and heights from the whole source; fixed rules score that reading 20 WC003 each biome’s required contents (peaks, cacti, mangroves, . . . ) and nothing that belongs elsewhere per item: verified code quote + navigated biome frame; disagreements re-searched in code 100 WC004 physics and motion: water in beds, things fall, rivers flow, particles drift per item: verified code quote + frames; motion counts only if a burst’s pixels changed 100 WC005 day/night and seasons: sun orbit, dimming, tint, season cycle code probe; change measured from pixels (brightness, frame difference, chromaticity) 20 total280 Table 2: The ladder: five tests with fixed maxima, 280 points in total. 4.1 Looking: exploring the running world The savedworld.htmlis not edited. Grading runs a copy whose clock is patched, as in Stage I, in a headless browser. An init script seedsMath.random, so a reload builds the same island, and reaches the scene and camera through Three.js’s own developer-tools hook. Looking runs as a four-stage pipeline: the world’s time is brought under control, views are taken, motion is recorded, and every close view is checked blind before it can be judged. Stage I: Controlling time. Every world keeps its own clock, written its own way. A language model patches that clock so it obeys an externally set time of day and season. The patch is tightly constrained: at most six edits, each under 600 added characters, each required to read the external time, so a patcher cannot add lighting the model never wrote. The patch’s mapping from requested time to noon is not trusted either. The world is rendered at four requested times and the brightest render, by mean luma, is taken as day (Figure 4, top). In the reference world, noon falls at a quarter of the cycle, where a fixed “half-way is noon” rule would have 5 chosen dusk. The four seasons are then rendered at day (Figure 4, bottom). Change over time is measured on these frames, not judged: mean luma for night, frame difference against day, and chromaticity shift for twilight tint and seasons. In the reference world the visual judge described the four season frames as “visuals remain identical”; their measured chromaticity shift from spring is 0.019, 0.035 and 0.024, and the measurement decides. TIME OF DAY: requestedwb_tod, measured mean luma (0–255) and difference from day 0.000.25 = day0.500.75 luma42.0Δday 11.4 luma102.0brightest⇒day luma36.8Δday 11.9luma30.7Δday 14.2 SEASON: requestedwb_seasonat day, measured chromaticity shift from spring spring summer autumn winter shift0.000(reference) shift0.019 shift0.035 shift0.024 Figure 4: Controlling time, reference world. Top: the world’s own clock, patched to the requested time of day; the brightest render (here requested time 0.25, not 0.50) becomes day for every later view, and each other render’s frame difference from day (Δday) is recorded. Bottom: the four seasons at day. The visual judge called these frames identical; the measured chromaticity shift from spring is non-zero for every season, and it is the measurement that scores the item. Stage I: Wide and close views. From day, the judge takes wide views (an overview, four orbits, and the four seasons) and close views of each biome. Close views come from a navigator agent, a tool-calling loop of observe→act→observe. Each turn it receives the page’s accessibility tree and its latest frame, and calls exactly one tool:click(a legend entry the world itself provides),orbit,pan,zoom,saveorgive_up, within an eight-step budget. Figure 5 shows the reference world’s longest episode: the navigator clicks the swamp’s legend entry, zooms in, orbits away and back to bring the water channels into frame, zooms again and saves (6 steps). Each turn keeps only the newest frame in context, so a long episode does not grow the prompt. Stage I: Motion bursts. A single frame cannot show motion, and a visual judge asked about motion across frames will report it from appearance alone. For each saved close view the judge therefore takes a still-camera burst: four frames 2 s apart with the camera fixed, then one zoom out and three more, and an overlay that paints every changed pixel. A visual “it moves” counts only if the burst’s pixels changed. Figure 6 shows the same motion item in two worlds. In one, rain falls and 38.4% of pixels change. In the other, the visual judge again reported “Vertical falling rain particles”, but 0.0% of pixels changed, so the claim is refused. A typical world yields about a hundred frames in total. Stage IV: Blind identification. The navigator’s claim that it found a biome is not trusted: in one world it saved the volcano as the swamp with full confidence. Every saved frame is re-taken with the page’s own interface hidden, so the clicked legend label cannot give the answer, and then identified blind: all of a world’s saved frames go to one shuffled call that is never told which biome a frame was meant to show. A frame counts as a biome’s close view only if the answer matches (Figure 5, middle row); otherwise it is dropped and that biome is judged from the wide views alone (Figure 5, bottom row). Appendix Figure 12 shows every close view of the reference world. 6 0 start: day preview, overview 1 click(legend “Swamp”) 2 zoom(+2) 3 orbit(100, 0) 4 orbit(−150, 50) 5 zoom(+2) 6 save(swamp), HUD hidden Blind identification every saved frame, shuffled in one call, with no biome named asked: which biome is this? answer:swamp 3matches: the frame counts as the swamp’s close view REJECTED, another world (deepseek-v4-pro): the navigator’s “found” is overruled Navigator save(delta) after 7 steps, confi- dence 0.7 note: “Shows green blocks and blue ocean in background” Blind identification answer:forest, not delta 7no match: the frame is dropped; the delta’s items are judged from the wide views only Figure 5: One navigator episode, reference world, swamp. Each frame is what the agent saw after the action below it; frames are re-rendered from the recorded actions with no model call (seeded world, pinned clock, so only animation phase differs from the original run). The saved frame (6) counts only after blind identification agrees. Bottom: a rejection from another graded world outside the five reported here; the navigator saved the delta with confidence 0.7, and the blind answer was forest. claude-fable-5-1, jungle: near burst, camera still 푡=0s푡=2s푡=4s 푡=6s changed pixels gpt-6-astra, jungle: near burst, camera still 푡=0s푡=2s푡=4s 푡=6s changed pixels visual judge,rain_shaft_fall: “blue rain particles fall downward across frames” measured:38.4%of pixels changed motion, frames half:3 visual judge,rain_shaft_fall: “Vertical falling rain particles”measured:0.0%of pixels changedmotion, frames half:7 Figure 6: Still-camera motion bursts for one WC004 item,rain_shaft_fall, in the jungles ofclaude-fable-5-1 (left) andgpt-6-astra(right). Red in the overlay marks pixels that changed anywhere in the burst. The visual judge reports falling rain in both; only the burst whose pixels changed earns the frames half of the motion credit. 4.2 Reading: the code as evidence Reading also runs in stages: quotes are proposed, verified against the source, repaired when badly copied, and searched again where the frames disagree. Stage I: Quoting. For each biome, a code probe lists the test’s items and must quote, for each one it claims, the source lines that build it. 7 capture: biome_highlands.png AB capture: biome_mountains.png C claude-fable-5-1/world.html 160else switch(b1) 161case 1: t=h>56?T.SNOW:(h>48&&n2>0.4?T.SNOW:T.ROCK); bre ... 162case 2: t=T.SNOWG; break; 163case 3: t=n2>0.7?T.HROCK:T.HIGH; break; A 164case 4: t=fbm(x*0.1+77,z*0.1,3)>0.68?T.JROCK:T.JUNGLE; ... 165case 5: t=T.SWAMP; break; 166case 6: t=n1>0.6?T.OLIVE:T.GRASS; break; . . . 195for(const p of JPONDS) stampPool(p.x,p.z,p.r,RIVER); 196 197const RIVERS=[ 198... 1:3.4,p:[[6,-84],[3,-74],[-3,-64],[-7,-54],[-13,-44],[- ... C 199k:'river',w:1.9,w1:2.4,p:[[0,42],[-8,49],[-14,57],[-18 ... 200k:'river',w:1.8,w1:2.4,p:[[1,48],[9,53],[14,60],[17,70 ... 201k:'river',w:1.2,w1:1.6,p:[[-20,70],[-12,80],[-8,92],[- ... . . . 448systems.push(new PSys(count:900,color:0xcfd8dc,size:3. ... 449systems.push(new PSys(count:700,color:0xb8c2a8,size:3. ... 450systems.push(new PSys(count:1600,color:0xa8c8f,size:0 ... 451... PSys(count:600,color:0xd0d8dc,size:3,opacity:0.22,box: ... B 452systems.push(new PSys(count:2200,color:0xa8c8f,size:0 ... 453systems.push(new PSys(count:1300,color:0xb8c8f,size:0 ... itemprobe’s code quotein src frame judge sayshalvespts AExposed rock/stonecase 3: t=n2>0.7?T.HROCK:T.HIGH; break; ✓“Grey stone patches on surface” 1.11 / 1.11 BMist particlessystems.push(new PSys(count:600, color:0xd0d8dc,size:3,opacity:0 ... ✓ not seen in any frame 0.56 / 1.11 CMeltwater streamno quote (probe: not in scope)✗ “Blue meltwater stream flowing down the mountain side.” 0.83 / 1.67 Figure 7: Three items, verified two ways (reference world; A and B in the highlands, C in the mountains). A: the quote is source line 163 verbatim and the frame shows stone, so both halves are earned. B: the quote is real (line 451), but no frame shows the mist, so only the code half is earned (error class E2, unobserved; Section 5.5). C: the frame shows the meltwater stream, but the code probe found nothing in the mountains and the recheck agent did not credit it either: the stream comes from a river table placed by coordinates (line 198), not from code that names the biome. Only the frames half is earned. Stage I: Verifying. A quote counts only if it is text the source contains: matching ignores whitespace and is done line by line, at least 80% of a quote’s lines must be found, and a line whose ending was garbled in copying still matches if its first 85% is verbatim. Invented code does not pass. Before this check, a model was credited for a plausiblescene.add(new THREE.Mesh(palmGeo,...))that appeared nowhere in the file. A quote must also come from that biome’s own code, and one quote cannot satisfy two items. Stage I: Repairing. Probes copy imperfectly, joining separate lines or abbreviating with an ellipsis. Rather than loosen the check, a quote repair step lets a model choose line numbers from a numbered window of the file, and rebuilds the quote from those lines, so it can select code but never write it. Stage IV: Rechecking. When the frames show an item but the probe found no code for it, an evidence agent searches the source with read-only tools (grep, line reads and a small set of text-only shell commands) over a sandboxed copy of the file, within ten tool calls. It is told that a sighting is a lead, not proof, and that inferring a feature from a neighbouring one (“there are pools, so there must be banks”) is not evidence. Only these disagreements are searched again. This is the one stage that waits for Looking, since it needs the frames. 8 4.3 Combining the two An item’s points are split evenly between the two channels (Figure 7). The code half is earned by a quote that passes the checks above. The frames half is earned when the visual judge reports the item in a blind-confirmed frame of that biome, or, where the property is measurable, when the pixels confirm it (Stages I and I). Items that must not appear in a biome (a palm in the desert, snow on the volcano) cost points when either channel finds them: a verified quote that builds one in the biome, or a sighting in its frames. Scoring. Within a test, each biome is worth 10 points, divided among its required items by weight푤 푖 , so item 푖 is worth 푝 푖 = 10 푤 푖 Í 푗 푤 푗 .(1) With푐 푖 ∈ 0, 1for a passed code check (Stage I of Reading) and푣 푖 ∈ 0, 1for a sighting in a blind-confirmed frame (Stage IV of Looking), the item earns 푔 푖 = 푝 푖 1 2 푐 푖 + 1 2 푣 푖 .(2) An item that must also move (WC004) splits its points between look and motion, and its motion half is split again between code and the burst: 푔 푖 = 푝 푖 h 1 2 1 2 푐 푖 + 1 2 푣 푖 + 1 2 1 2 푐 mot 푖 + 1 2 푚 푖 i ,(3) where푚 푖 = 1only if the visual judge reports motion on the expected axis and the burst’s changed-pixel share 푓 푖 ≥ 0.001. Each forbidden item푙that is coded or seen subtracts its share, and a biome cannot go below zero: 푆 푏 = max 0, ∑︁ 푖 푔 푖 − ∑︁ 푙 푝 푙 [푐 푙 ∨ 푣 푙 ] ,WC003 = 10 ∑︁ 푏=1 푆 푏 .(4) Item B in Figure 7, for example, has푐 = 1and푣 = 0, and earns 1 2 × 1.11 = 0.56. Appendix C gives the formulas for the other tests and the pixel measures. Because the halves are independent, code that never renders earns at most half, and so does a render the probe could not explain; Section 5 measures how often each happens. 5 Results Setup. We grade one world from each of the five models of Section 3, all with the same judge configuration. We ask three questions of the judge before reading the leaderboard: do its two channels disagree, does exploring matter, and does it notice when a feature is really gone? 5.1 Code and frames disagree on a third of items WC003 checks 73 required items per world, 365 in total. For each, the judge has a code verdict and a frames verdict (Table 3). They agree on 250 items and disagree on 115 (32%), between 26% and 41% per world. Disagreement is lopsided: in 105 items the code is found but no frame shows the item, against 10 in the other direction. A judge that only read code would credit 289 items (79%); one that only looked would credit 194 (53%). “Not seen” means not seen in our frames, which is either a real absence or an item our capture did not reach; Section 5.3 separates the two on items we removed ourselves, and Section 5.5 names the two disagreement directions as error classes E2 (unobserved) and E3 (unattributed). 9 code: foundcode: not found worldframes: seennot seenseennot seenitemsdisagree claude-fable-5-14719167327% gpt-6-astra4321277332% kimi-k-339212117332% grok-4-731264127341% gemini-3-1-pro24181307326% all184105106636532% Table 3: Code and frames verdicts on WC003’s required items, per world. “Not seen” means not seen in our frames. 5.2 Close and wide views see different things To test whether exploring matters, we re-ran WC003’s visual judge on the same saved frames with the navigator’s close views removed, leaving the overview and four fixed orbits, which is what a snapshot judge has. Prompt, items and judge are unchanged (Table 4). Close views see more required items (194 against 178), but not by a wide margin, and the two sets differ: 42 items are seen only up close and 26 only in the wide views. The close-only items are small contents (animals account for 12 of the 42, followed by flowers, tall grass, vines and rain shafts). The wide-only items are biome-scale layout and weather (swamp channels, river mouths, islands, flat basins, mist, ash). The code probe independently supports 81% of the extra sightings on both sides, so neither set is mostly hallucinated. Totals barely move (320.2 against 322.1 WC003 points over five worlds), but individual worlds do:grok-4-7loses 5.5 points without close views andgemini-3-1-pro gains 8.9, which swapskimi-k-3andgrok-4-7in the WC003 ranking. Exploration matters for coverage, not because any one camera is better. Two caveats apply: the close condition gave the judge two images per biome and the wide one five, and the navigator confirmed only 5 of 10 biomes in grok-4-7’s world. required items seenseen only byWC003 score worldnavigatedsnapshotsnavigatedsnapshotsnavigatedsnapshots claude-fable-5-148459676.2975.97 gpt-6-astra453716874.4870.52 kimi-k-341406560.1162.90 grok-4-735279165.0359.52 gemini-3-1-pro25292644.2553.14 all1941784226 Table 4: Close (navigated) vs. wide (fixed snapshot) views: same judge, items and saved frames; only the images given to the judge differ. 5.3 Removing features on purpose Since there is no reference answer to annotate, we create ground truth by construction, as in mutation testing [7]. In a copy ofclaude-fable-5-1’s world we remove features, then re-run capture and WC003 on the changed biomes (Table 5). One feature is deleted outright. Three are made dead code: the function that builds them stays in the file but is never called. A fourth is disabled in place (if(false)). The world is otherwise identical, including its random stream. The correct verdict for every removed item is absent. A code-only judge credits all four features whose code is still in the file, at full value, quoting real lines such as the keptfunction cactus(...)definition. Our frames half sees none of the cacti, mangroves or palms and cuts each to half credit: removed items keep 3.55 of their 7.11 points under our judge, against 5.44 under code-only judging. One removal slips through entirely: the disabled blizzard, where the probe quoted the switched-off line and the visual judge read drifting snowfall near the peaks as a blizzard. A snapshot-only judge is strictest here (1.67 points kept), but Section 5.2 shows it also misses real close-up items. Two findings bound these numbers. First, our code half still accepts code that never runs, so dead code earns half credit (Section 6). Second, single-run item verdicts are noisy: 7 of 32 items we did not touch changed verdict between 10 beforeafter removal (correct = 0) removed itemhowrealcode-onlysnapshot-onlyours summit_blizzarddisabled1.671.671.671.67 yellow_white_flowersdeleted1.67000 cacti_on_flatsdead code1.431.4300.71 mangrove_or_prop_rootsdead code1.431.4300.71 palm_silhouettesdead code0.910.9100.46 removed items still credited4 / 51 / 54 / 5 points still given (of 7.11)5.441.673.55 untouched items in the same 5 biomes whose verdict changed (judge noise): 7 of 32 Table 5: Mutation test on a copy ofclaude-fable-5-1’s world: points each judge still gives to items we removed. The correct value for every row is 0. the original run and this one. A sixth removal (volcanic ash) is excluded because the item also accepts eruption particles, and the volcano’s lava bombs, which we did not remove, still provide them; the judge was right to keep crediting it. 5.4 Leaderboard Table 6 gives the totals and Figure 8 the per-test profile.claude-fable-5-1leads with 210.79 of 280, followed bygpt-6-astra(199.73),kimi-k-3(191.36),grok-4-7(187.78) andgemini-3-1-pro(149.00). The order is not uniform across tests.claude-fable-5-1leads on contents (76.29) and physics (72.5). kimi-k-3is ahead on the voxel island (38/40) and on cycles (14/20), andgrok-4-7on placement (18/20); those gaps are a few points, and the placement column is a model’s reading of the source (Section 6). We read gaps of a few points between neighbours as ties, in totals as well, such as kimi-k-3 and grok-4-7. WC001 voxel island /40 WC002 placement /20 WC003 contents /100 WC004 physics /100 WC005 cycles /20 Total all tests /280 0% 25% 50% 75% 100% score, % of test maximum claude-fable-5-1gpt-6-astrakimi-k-3 grok-4-7 gemini-3-1-pro Figure 8: Per-test scores as a percent of each test’s maximum (maxima differ, 20 to 100 points), and the total out of 280. Exact values are in Table 6. 5.5 Error analysis We classify every point lost on WC003 and WC004 (378.3 points over the five worlds) by the state of its item’s two channels. The code channel is positive when a verified quote exists (Section 4.2); the frames channel is positive when a blind-confirmed frame shows the item (Section 4.1). This gives four error classes (Table 7), used with these names throughout. E1 dominates (48.0%). E2 is the second largest (32.4%), and it is exactly the set a code-only judge would have 11 modelWC001WC002WC003WC004WC005total /40/20/100/100/20/280 claude-fable-5-1361576.2972.511210.79 gpt-6-astra361674.4863.2510199.73 kimi-k-3381760.1162.2514191.36 grok-4-7351865.0358.7511187.78 gemini-3-1-pro371344.2544.7510149.00 Table 6: Leaderboard, ordered by total. Gaps of a few points between neighbours are within judge noise (Section 5.3). error classcode frames meaningshare of lost points E1 unimplemented✗no verified quote and no confirmed frame48.0% E2 unobserved✓✗the code is in the source; no capture shows the item32.4% E3 unattributed✗✓a frame shows the item; no quote verifies against the source 8.9% E4 property violation✓look confirmed in both; the item’s motion is not verified (the quoted motion code is not a per-frame update, or the burst’s pixels do not change) 3.9% penalties and gatesnot an error class: a wrong-biome feature coded or seen, the delta zeroed for an ocean over the void (WC003), or the island gate (WC004) 6.7% Table 7: Error classes for a lost item, by which channel’s evidence is missing. Shares are of all 378.3 points lost on WC003 + WC004 across five worlds, which is 1000 minus their WC003 and WC004 scores. credited, so a third of all losses lie in the gap this benchmark measures. E3 (8.9%) is the reverse gap, closed in part by the evidence-agent recheck; E4 is rare (3.9%), and the last 6.7% are penalties and gates, which no channel explains. The class definitions are operational, not causal: E1 is an upper bound on true absence, because it is the code probe’s verdict, and E2 includes items that exist but that capture did not reach (Section 6). By model. The class mix differs per world (Figure 9).gemini-3-1-pro’s losses are mostly E1.grok-4-7 has the most E2 points, partly a capture effect: the navigator confirmed only five of its ten biomes (Section 5.2). gpt-6-astra has the most E4 points. By biome. Wet and detailed regions lose the most points (Figure 10). The jungle is among the four hardest biomes for every model, and the grassland is the easiest for every model. On average jungle loses 53% of its points, delta 51%, swamp 51% and grove 50%, against 11% for the open grassland. 4 items are E1 for all five models, all small life (forest-floor cover and ferns, the grove’s pollinators and bees). 02040 60 80100120 claude-fable-5-1 gpt-6-astra kimi-k-3 grok-4-7 gemini-3-1-pro points lost on WC003 + WC004 (of 200 per world) E1 unimplementedE2 unobservedE3 unattributedE4 property violationpenalties and gates 22 19 7 51.2 19 23 99 62.3 28 25 17 77.6 3434 7 76.2 78 21 66 111.0 Figure 9: Lost WC003 + WC004 points per model, by error class (Table 7); each bar is 200 minus the world’s WC003 and WC004 scores. Green (E2, unobserved) is the code-only judge’s blind spot: code that exists but that no capture shows. Grey is penalties and gates, mostly kimi-k-3’s delta, zeroed for an ocean over the void. 12 mountainsforest highlandsjungle swamp grovegrassland deltadesert volcano claude-fable-5-1 gpt-6-astra kimi-k-3 grok-4-7 gemini-3-1-pro mean 20 27 314033 444 1830 9 23233040 67 27 5 31 39 27 17 21 39 44 45 38131003832 47 5050 64 22 63 9 33 27 15 41 34 68 77 8878 2774 3733 303143 535150 11 51 3423 0%80% share of the biome’s WC003 + WC004 points lost Figure 10: Share of each biome’s WC003 + WC004 points lost, per model and on average. The jungle is among the four hardest biomes for every model, and the open grassland is the easiest for every model. 6 Limitations Unreached code still earns half. The code half asks whether a quote is real text in the source, not whether that code ever runs. In the mutation test, three features whose functions were kept but never called still earned their code half, and a feature switched off withif(false)was quoted by its disabled line. Checking that quoted code is reachable from the world’s build or frame loop would close most of this gap. The visual judge can still be fooled. Verification narrows what the visual judge decides but does not remove it. Where no pixel measurement applies, a look-alike can pass: drifting snowfall near the peaks was accepted as a blizzard after the blizzard had been removed. Placement is a model’s reading. WC002’s neighbour list comes from a model reading the layout code, and unlike the item quotes it is not checked pair by pair. Against the actual biome maps of four worlds it is right on 61 of 76 rule-relevant pairs (of 19 each:claude-fable-5-117,kimi-k-314,gemini-3-1-pro14, and 16 onclaude-opus-5-5, a world outside the five reported here). The other two reported worlds, including grok-4-7, which tops placement, have no such map, so their placement scores are unchecked. The errors are mostly missed links, so a world can lose a placement point for a border it does have. Reading each world’s own biome map from the running page would make this exact, but the code that stores that map differs in every world. In a trial on the same four worlds, a model asked to locate it reproduced the exact map on its first attempt in two; in the other two it missed a biome or returned values it could not map. E2 (unobserved) mixes two causes. An item the frames do not show is either not rendered or not captured. The navigator confirmed all ten biomes in three worlds, nine in one, and five ingrok-4-7’s, so part of that world’s E2 points reflect capture, not the world. Noise and scale. We grade one world per model, from one prompt, once, and single item verdicts are noisy (Section 5.3). Totals are steadier, but we treat gaps of a few points between neighbouring models as ties and do not claim a ranking among them. The mutation test covers five removals in one world. Repeated generations and repeated grading would give the variance this paper does not report. No human labels, by design. The worlds are open-ended and synthetic, with no reference answer to annotate, so we validate the judge by construction rather than by agreement with people. Mutation shows whether the judge notices what we changed, not whether it agrees with a person on everything we did not. Judges and medium. Every judge role is a model from the same developer as one model under test (Appendix F). That model ranked last, but we did not measure the bias with a cross-family judge. The benchmark covers Three.js worlds only. 13 7 Conclusion A world written as code has to be graded twice, once for what the program says and once for what it shows, because the two disagree on a third of items, mostly code that no frame shows, and neither can say alone which is right. WorldBench grades both: it explores the running world close and wide, reads the code for what it is looking at, and lets an item score only for evidence that survives a check against the source or the pixels. Placement is the one reading we do not verify item by item; on four worlds it matched the biome map on 61 of 76 pairs (Section 6). Removing features on purpose shows the gain over reading alone, and the gap that remains: code that exists but never runs. References [1]Frédéric Berdoz, Luca A. Lanzendörfer, Nick Tuninga, and Roger Wattenhofer. Text-to-Scene with Large Reasoning Models. arXiv:2509.26091, 2025. [2]Xiao Cai, Sitong Su, Jingkuan Song, Pengpeng Zeng, Ji Zhang, Qinhong Du, Mengqi Li, Heng Tao Shen, and Lianli Gao. GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark. arXiv:2412.09997, 2024. [3] Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching Large Language Models to Self-Debug. arXiv:2304.05128, 2023. [4]Yun He, Kelin Yu, and Matthias Zwicker. SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation. arXiv:2604.19907, 2026. [5]Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng, Ran Yi, Juanzi Li, and Yong-Jin Liu. T 3 Bench: Benchmarking Current Progress in Text-to-3D Generation. arXiv:2310.02977, 2023. [6]Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A. Ross, Cordelia Schmid, and Alireza Fathi. SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code. arXiv:2403.01248, 2024. [7] Yue Jia and Mark Harman. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering, 37(5):649–678, 2011. doi: 10.1109/TSE.2010.62. [8] Shuo Lu, Yinuo Xu, Kecheng Yu, Siru Jiang, Yongcan Yu, Yubin Wang, Haitao Yang, Yuxiang Zhang, Bin Wang, Ran He, and Jian Liang. WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis. arXiv:2606.01869, 2026. [9]Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM Evaluators Recognize and Favor Their Own Generations. arXiv:2404.13076, 2024. [10]Hou In Ivan Tam, Hou In Derek Pun, Austin T. Wang, Angel X. Chang, and Manolis Savva. SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis. arXiv:2503.14756, 2025. WACV 2026. [11]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685, 2023. [12]Mengqi Zhou, Yuxi Wang, Jun Hou, Shougao Zhang, Yiwei Li, Chuanchen Luo, Junran Peng, and Zhaoxiang Zhang. SceneX: Procedural Controllable Large-scale Scene Generation. arXiv:2403.15698, 2024. 14 Appendix A The prompt Every model receives this prompt as its only instruction (typographic characters shown in ASCII). Build a single, self-contained HTML file that renders the world described below as an explorable 3D scene using Three.js (CDN import is fine -- no build step, no separate asset files). It must just open in a browser and run, with a free camera (e.g. OrbitControls) so a visitor can orbit, zoom, and pan around the whole world. Include a simple on-screen UI so a visitor can navigate and read the world easily (see UI section below). Output ONLY the complete HTML file -- no prose, no commentary, no code fences. Choose your own overall scale. What matters is relative placement, ecological causality, and clear visual reading from orbit -- not any particular absolute size or fixed elevation numbers. -------------------------------------------------------- THE WORLD -------------------------------------------------------- # Floating Biome Island with Ocean -- World Description ## Visual style (required) Minecraft-like voxel / block aesthetic throughout: - Terrain, cliffs, water, lava, trees, flora, and structures are built from chunky axis-aligned cubes and blocky stacked forms -- not smooth organic meshes, not photoreal terrain, not low-poly "soft" stylization. - Flat, solid face colors (Lambert / unlit-friendly); no PBR gloss, no detailed textures required. Pixel-chunk readability from orbit. - Trees and plants are stacked blocks / simple voxel silhouettes (trunk + canopy cubes), not billboards or smooth cones. - Weather and particles can be simple quads or small cubes, but the landmass itself must read as a voxel island. ## Overview A floating landmass hangs motionless in an infinite black void. Viewed from a high orbit, nearly the entire surface should be readable at once. Each biome needs room to read as its own region -- distinct transitions, not a muddled cluster. Flora and fauna should feel alive inside each biome without overcrowding the whole island. Much of the outer perimeter is bordered by shallow coastal ocean and sandy beaches before terrain rises inland. Elevations are exaggerated and dramatic relative to each other: peaks dominate, shelves sit mid-height, plains are low and open, cliffs drop sharply. The ocean is suspended with the island and ends at cliff edges where waterfalls plunge into darkness. ## World layout (relationships) Compose a coherent ecological sequence. Relative placement should feel like this (adapt freely as long as the relationships hold): 1. Snow mountains occupy the far / behind end of the island -- the final alpine massif at the cold edge. 2. Snowy conifer forest sits as elevated foothills in front of those mountains (between the peaks and the rest of the island). 3. Highlands occupy an elevated plain to the side of the snow forest -- between the mountain flank and the back side of the flowering grove -- then slope gently down into that pink grove. 4. Dense jungle is a large wet canopy region, with a backwater / swamp 15 marsh on the jungle's far side (away from the main plains outlet). 5. Flat grassland plateau is the open river plain that receives wet- corridor drainage and leads toward the coast. 6. Flowering grove is a moist pink basin off the plains / below the highland slope -- visually separate from green grassland. 7. Desert is a vast arid basin (south / rain-shadow side) with its own sandstone desert mountains -- never the volcano. 8. Volcano is a separate tall cone on volcanic ground only (far from the desert's river path). 9. Coastal ocean and beaches wrap much of the southern and western edge. ## Scale and density (qualitative) - Prefer a clearly legible island: biomes spaced so each reads as its own region from high orbit. - Avoid collapsing adjacent biomes -- especially grassland vs flowering grove, desert vs volcano, snow forest vs alpine peaks. - Within each biome, flora and fauna should feel abundant for that climate (dense jungle canopy; cactus across desert flats; pines in the snow forest; sparse scrub on highlands). - Mountains and the volcano dominate the skyline; desert flats and the grassland plateau sit lower between them. ## Fauna Voxel / blocky animals (stacked cubes, same aesthetic as the land) -- simple silhouettes are enough; place them where they belong: - Deer or similar grazers in jungle edges, grasslands, grove, conifers - Foxes or small arid / marsh creatures in desert and swamp - Occasional coastal or river life near beaches and deltas if it fits - Enough motion or presence to feel alive, not empty dioramas -- without overcrowding ## Ecological reasoning Layout and systems must follow ecological logic, not decoration: - Water stays in the wet corridor: mountain melt -> through / past the snow-forest foothills and wet highlands -> weave through jungle -> flat grasslands -> coastal delta / ocean. - Flowering grove is a moist offshoot below the highland slope / plains moisture -- not a stop on the way into the desert. - Desert is arid: vast flat basin with sandstone desert mountains; cacti across the flats; only fading dry washes; no through-rivers into the volcano; no perennial river cutting the desert core. - Volcano is a separate tall cone with lava tributaries and lava pools on volcanic ground. Where water meets lava, rock quenches to obsidian. - Rain falls downward from moving cloud decks over wet biomes. - Snowfall comes from cloud decks over the snowy conifer forest. - Blizzard / white windstorm lives on the behind face of the snow mountains (the far alpine side), not as a front-slope decoration facing the forest alone. - Snowy trees carry snow on their canopies, not only on the ground. - Highlands are genuinely elevated relative to plains and grove, with a gentle downslope into the pink forest. ## Biomes ### 1. Snow Mountain Range (far / behind) - Tall alpine peaks -- high relief, not gentle hills - Snow on summits and upper ridges; rock on lower slopes - Meltwater feeding the wet corridor - Active blizzard on the behind mountain face: fast whitish wind sheets (sandstorm motion, white instead of brown), spanning a wide strip of that rear alpine tape ### 2. Snowy Conifer Forest (foothills in front of the peaks) 16 - Elevated shelf / foothill band in front of the snow mountains - Dense pines with snow on canopies as well as ground - Cloud deck directly above the forest that drops snowfall onto it - Distinct from both bare alpine rock and lowland plains ### 3. Temperate Highlands (elevated plain beside the snow forest) - Sit to the side of the snow forest, between mountain influence and the back of the flowering grove -- an elevated open highland plain - Higher than grasslands and grove; rocky hills, scrub, hardy plants - Gentle slope down into the pink flowering grove - Occasional mist and light rain ### 4. Dense Jungle / Rainforest - Thick emerald canopy, layered undergrowth, densely packed trees - Rocky clearings, inland ponds, major waterfall - On the jungle's backside (away from the plains outlet): a backwater / swamp ecosystem -- murky standing pools, mangroves or similar, low mist, stagnant channels (not a desert river) - Persistent jungle rain: rainfall shafts, wet canopy, rising mist after showers ### 5. Central Grasslands (flat river plateau) - Open flat green plain -- clearly not the pink grove - Dense winding river network with many tributaries - Scattered woodland copses and yellow/white wildflowers - Meadow color green/olive, not pink ### 6. Flowering Grove - Moist basin receiving the highland downslope and/or plains moisture - Dense pink flowering canopy and pink-tinted meadow floor - Small streams; visually different from grassland at a glance ### 7. Desert - Mostly flat arid land - Distinct desert mountains / sandstone peaks on that flat floor -- these are NOT the volcano - Dry washes that fade out; no perennial river through desert core or into the volcano - Cacti and life spread across the flats - Desert winds and sandstorm / dust storm ### 8. Volcanic Wasteland - Tall stratovolcano cone with glowing crater -- not a flat puddle - Lava tributaries into lava pools on volcanic ground only - Black basalt, ash plumes, periodic eruptions - Water + lava contact becomes obsidian ## Coastal Ocean Southern and western edges surrounded by turquoise coastal waters: - Shallow lagoons and deeper offshore water - Rocky shoreline, small islands and reefs - Multiple river mouths and a visible braided delta into the ocean - Freshwater mixing into saltwater at the mouths ## Beaches Natural transition from grasslands and forests into beaches: - Bright sand, curved bays, warm-region palms where fitting - Rocky headlands, clear shallow water, coastal vegetation 17 ## Rivers and Hydrology A rich branching system that obeys ecological law: - Mountain melt and highland moisture feed headwaters. - Flow weaves the wet corridor through jungle into the flat grassland plateau, then widens into a coastal delta. - Grove receives a moist feeder from highland slope and/or plains -- not a desert route. - Desert stays dry aside from fading washes; no river into the volcano. - Swamp holds stagnant backwater connected to the jungle wet side. - Show the water cycle: rain and melt feed rivers; heat drives evaporation mist; moisture returns as rain or snow. ## Climate, Clouds, Seasons, and Sun - Rain, snow, mist, sandstorm, ash, and cloud decks arrive in cyclic bursts -- fade in, hold, fade out, quiet gap -- staggered by biome. - Snowfall: cloud deck over the conifer foothill forest. - Blizzard: whitish high-altitude wind sheets on the behind alpine face of the snow mountains. - Desert: winds and sandstorm. Volcano: eruptions, ash, lava pools; water+lava -> obsidian. - Reliable seasonal cycle (spring -> summer -> autumn -> winter) modulating rain/snow intensity and light tint. - Visible square sun on a cyclic day / night pattern: bright noon, dimmer dusk and night (the void stays black; the land darkens). ## UI (required for easy navigation) Keep the HUD minimal and readable over the black void: - Biome legend: color swatch + biome name for every major region (mountains, snow forest, highlands, jungle, swamp, grasslands, flowering grove, desert, volcano, coast / ocean). Optionally note each biome's weather flavor (rain, blizzard, sandstorm, etc.). - Click (or tap) a legend entry to fly / animate the camera to that biome; include a "whole island" overview entry that returns to a comfortable high orbit. - Season readout: show the current season name on screen and update it as the seasonal cycle advances. - Brief control hint in the legend footer (e.g. orbit / zoom / pan). ## Cliffs and Underside - Vertical rock walls around the perimeter - Tall stone foundation matching heightened elevations above - Numerous waterfalls; floating rock fragments beneath - Blue glowing mineral seams in places ## Color Palette RegionColors ------------ ----------------------------- OceanTurquoise, deep blue, cyan BeachesWhite, cream, tan JungleEmerald, dark green, mist grey SwampMurky olive, dark water PlainsGreen, olive GrovePink blossoms, soft meadow Highlands Grey-green scrub Mountains White summits, blue-grey rock ConifersDark pine, snow-capped canopy DesertGold, ochre, dust haze VolcanoBlack, orange, crimson, ash RiversCyan (murkier in swamp) WeatherRain blue, snow white, sand tan, ash grey ## Lighting 18 - Square sun drives a full day -> night cycle; seasonal tint modulates warmth - Nights are clearly darker than noon (ambient + sun intensity drop) - Local weather tinting (cooler in storms, ember glow near volcano) - Crisp terrain shadows when weather is clear - High visibility of the whole island from a comfortable orbit by day - Black void background with no stars or atmosphere dome ## Visual Composition A Minecraft-style voxel floating continent: far snow mountains with a behind-face blizzard; elevated snow-forest foothills under snowfall clouds; highlands on an elevated plain sloping into a pink grove; a large dense jungle with a backwater swamp on its far side; a flat grassland river plateau; arid desert with its own sandstone mountains separated from a tall erupting volcano; wet-corridor rivers to a coastal delta; cyclic weather; square sun with day/night and seasons; clickable biome legend and season HUD. The ocean borders the land before plunging over cliff edges into the void. Success is ecological clarity, vivid living systems, easy navigation, and a coherent blocky voxel look -- invent your own scale; make it as strong or stronger as this description allows. B System and capture details Models under test. Each world was generated by the model named below, queried through its developer’s official API with reasoning enabled where the model supports it, at temperature 1.0 and no output-token cap. A truncated response was continued, up to three turns in total, and the repair loop (Section 3) ran for at most five rounds. One world per model is graded. Figure 11 shows the whole system, generation to score, on the reference world. modeldeveloperlabel in this papergenerated Claude Fable 5.1Anthropic claude-fable-5-12026-09-02 GPT-6 AstraOpenAI gpt-6-astra2026-09-08 Kimi K3Moonshot AI kimi-k-32026-07-20 Grok 4.7xAI grok-4-72026-09-22 Gemini 3.1 ProGoogle DeepMind gemini-3-1-pro2026-07-20 Table 8: Models under test. All were accessed through their developers’ official APIs. Prompt voxel island, 10 biomes, cycles Generate one completion, resumed if cut off Debug node --check + browser console Fix small edits, re-parsed errors retry ≤5 rounds inputs/claude-fable-5-1/ world.html 550 lines, 45 KB; never edited by grading clean, or budget spent Capture headless browser, seeded RNG, the world’s own clock set from outside 102 frames: times, orbits, seasons, biomes, motion bursts WC001voxel36/40 WC002placement15/20 WC003contents76.29/100 WC004physics72.5/100 WC005cycles11/20 total210.79/ 280 source text generation grading Figure 11: The full system on the reference world. Generation (left) and grading (right) share nothing but the world file. Scores are the reference world’s real per-test results. 19 mountains 2 steps blind: mountains3 forest 3 steps blind: forest3 highlands 3 steps blind: highlands3 jungle 3 steps blind: jungle3 swamp 6 steps blind: swamp3 grove 2 steps blind: grove3 grassland 4 steps blind: grassland3 delta 2 steps blind: delta3 desert 2 steps blind: desert3 volcano 2 steps blind: volcano3 Figure 12: Every close view of the reference world: the frame each navigator episode saved (interface hidden), the episode’s step count, and the blind identification answer. All ten were confirmed. C Scoring details Per test. WC003 and WC004 follow Eqs. 2–4. WC005 applies Eq. 2 to each of its items, except weather items, which are scored from code alone. The other two tests are WC001 = ∑︁ 푖 푝 푖 휋 푖 [quote verified 푖 ] + 20· 푛 clean 푛 judged , where휋 푖 is the source judge’s probability for item푖(its points total 20) and the second term is the share of captured views in which the visual bug hunt found no defect, and WC002 = 10 ∑︁ 푏=1 [covered 푏 ]+[covered 푏 ∧ rules 푏 ] , one point for a biome whose layout code is found and verified, and one more if its placement rules pass. The total is Í 5 푘=1 WC 푘 ≤ 280. Two rules act on the delta alone. In WC003 its score is zero when the frames show the ocean running out over the void with no blocks beneath. WC004 drops its delta points when WC001’s bug hunt sees water over the void in at least two wide views; the gate fires on one reported world, kimi-k-3. Pixel measures. Table 9 defines the measurements that decide instead of the visual judge wherever a property is measurable. measuredefinitionpass condition brightness 퐿mean grey level of non-void pixels (grey> 12), left 20% of the frame (the legend) cropped day = arg max 푡 퐿(푡); a day/night cycle exists if 퐿 day − 퐿 night ≥ 4 chromaticity (푟, 푏): red and blue shares of land pixelsshift = max(|Δ푟|,|Δ푏|) ≥ 0.015 (seasons),≥ 0.03 (twilight) light change mean per-channel difference from the day frame ≥ 8 grey levels motion share푓of pixels whose grey level changes by more than 24 within a burst 푓 ≥ 0.001 Table 9: Pixel measures and their thresholds, as implemented in the capture and grading code. D Score breakdown The leaderboard (Table 6) gives one number per test. This appendix splits each test into the parts it is made of, for every model. Colours follow Figure 8. 20 WC001 WC002 WC003WC004 WC005 claude-fable-5-1 WC001 WC002 WC003WC004 WC005 gpt-6-astra WC001 WC002 WC003WC004 WC005 kimi-k-3 WC001 WC002 WC003WC004 WC005 grok-4-7 WC001 WC002 WC003WC004 WC005 gemini-3-1-pro axes: WC001 (top) clockwise to WC005, each as % of the test’s maximum; rings at 50% and 100%;five-model mean Figure 13: Score profile per model: each axis is one test (WC001 at the top, clockwise to WC005) as a share of its maximum; the dashed polygon is the five-model mean. Profile across tests. Figure 13 draws each model’s five test scores, each as a share of the test’s maximum, against the five-model mean. Models differ most on WC003 (a 32-point spread in percentage terms between the best and worst model) and least on WC001 (8 points): every model builds a recognisable voxel island, and they separate on what the island contains and how it is laid out. We show one radar per model rather than all five overlaid because, overlaid, two models that tie on two tests hide each other. WC001 and WC002 by part. WC001 is two checks of 20 points each (Figure 14, left): the source judge scores 17–20 and the visual bug hunt 16–20 across models, so the voxel test is close to saturated. WC002 earns one point per biome covered and one per biome whose placement rules pass (right). Coverage is near complete (9–10 of 10 biomes), while placement rules range from 4 to 8 of 10: nearly every biome is built, but not where the prompt says. The one uncovered biome isgemini-3-1-pro’s grove: its layout quote did not verify against the source, and no blind-confirmed frame shows it. source judgevisual bug hunt 0 5 10 15 20 points (of 20) WC001 voxel island biomes coveredplacement rules met 0 2 4 6 8 10 points (of 10) WC002 coverage & placement claude-fable-5-1gpt-6-astrakimi-k-3 grok-4-7 gemini-3-1-pro Figure 14: WC001 and WC002 split into their parts. WC001: source judge and visual bug hunt, 20 points each. WC002: one point per covered biome and one per biome whose placement rules pass, 10 each. 21 WC001 in one world. Figure 15 shows WC001’s two halves on the reference world. The visual bug hunt looks at each captured view on its own and flags floating blocks, hollow masses or non-voxel shapes; it flagged 3 of 15 views (the overview, the left view and a close view of the delta), for 16 of 20; the figure shows the first two. The flags on the two wide views charge the world for things the prompt asks for. The prompt requires cloud decks, ash plumes and “floating rock fragments beneath” the island, and this world builds all three, the fragments as 70 deliberately detached cubes. The overview is flagged for “cubic cloud-like blocks” above the peaks and, in most of its votes, for detached blocks under the island. The left view is flagged for eruption ash and, again, for blocks under the island. The top view is marked clean, so the same world passes or fails by camera angle. The third flag, flat slabs in the air near the volcano in the delta close view, we did not trace. The bug hunt cannot tell a required floating feature from a defect, so it can cost a faithful world points. Water over the void is counted once per world, and only when at least two views show it. The source judge (Table 10) gives each of its items a probability and a quote; an item earns푝 푖 휋 푖 only if the quote verifies against the source, so it cannot talk a world into points. Here every item verifies, for 20 of 20. ofloating_blocks overview Several small cubic cloud-like blocks or snow chunks float directly above the snowy mountains. ofloating_blocks direction_left Eruption ash and detached floating rock fragments float directly above the volcano crater with visible gaps. nno defect direction_top Figure 15: WC001 visual bug hunt, reference world: two of the three flagged views (the third, a delta close view, is not shown) and one clean view, with the judge’s own words for each flag. Both flagged views show things the prompt requires: cloud decks, ash, and floating rock fragments beneath the island. itempoints prob. verified earned quote (start) cube_primitive11.00✓1 const box=new THREE.BoxGeometry(1,1,1); bulk_placement11.00✓1 function buildInstanced(list,material) const m= ... discrete_grid11.00✓1 const idx=(x,z)=>(x-GX0)+(z-GZ0)*GW, stacked_columns11.00✓1 B(x,bt,g-capT,z,stoneColor(biome[i],x,z)); B(x,g ... cube_terrain11.00✓1 B(x,bt,g-capT,z,stoneColor(biome[i],x,z)); grid_aligned11.00✓1 const x=cellX(i),z=cellZ(i),g=ground[i],bt=Math. ... unit_voxels11.00✓1 const box=new THREE.BoxGeometry(1,1,1); contained_water21.00✓2 const waterMesh=buildInstanced(waters,new THREE. ... water_physics31.00✓3 if(d<=w) water[i]=water[i]>-999?Math.min(water[ ... water_bed51.00✓5 ground[i]=Math.round(sb); water[i]=8; type[i]=sb ... grounded_props31.00✓3 B(x,g,g+H,z,rnd()<0.5?0x5a3a20:0x6b4426); Table 10: WC001 source judge, reference world: each item’s points, the judge’s probability, whether its quote is text the source contains, and the points earned. WC003 and WC004 by biome. Each biome is worth 10 points in each of the two large tests (Figure 16). In WC003 the lowest five-model average is grove (4.7 of 10) and the highest grassland (9.0); in WC004 the lowest is jungle (4.3) and the highest grassland (8.7). The per-biome bars show what the averages hide: in several biomes one model scores near the maximum while another scores near zero. WC004 look and motion; WC005 by item. WC004 mixes items that are judged on appearance alone (65 points per world) with items that also carry a motion part (35 points). Over all five worlds, models earn 70% of the look-only points but 47% of the points on items with a motion part (Figure 17, left): things are more often built than made to move. WC005 has 10 items (right). In every world, 2 of them earn full credit and 0 earn nothing; the rest split the models. The three items with the least credit across models aremoon, dusk_dawn_tint, sky_or_fog_day_cycle. 22 0 5 10 points (of 10) WC003 contents, per biome mountainsforesthighlandsjungleswampgrovegrasslanddeltadesertvolcano 0 5 10 points (of 10) WC004 physics, per biome claude-fable-5-1gpt-6-astrakimi-k-3 grok-4-7 gemini-3-1-pro Figure 16: WC003 (contents) and WC004 (physics) per biome, 10 points per biome in each test, after the island gate (which zeroes kimi-k-3’s WC004 delta). A missing bar is a score of zero. look-only items items with a motion part 0% 25% 50% 75% 100% % of the part’s points WC004 look vs. motion WC005 cycles, per item sun_orbit night_dimming light_follows_sun season_cycle cloud_drift_wrap season_world_tint season_modulates_weather moon dusk_dawn_tint sky_or_fog_day_cycle claude-fable-5-1gpt-6-astrakimi-k-3 grok-4-7 gemini-3-1-pro Figure 17: Left: WC004 split into look-only items and items with a motion part, as a share of each part’s points. Right: WC005 per item and model: filled = full credit, half-filled = half, ring = none. Rows are ordered by total credit across models. E Design history: judges that failed The judge in Section 4 was not designed up front. Each of its checks replaced a simpler judge that was built first, run on real worlds, and found to fail in a specific, measurable way. Table 11 lists them. Two patterns repeat. Judges that match the source text (keywords, patterns) reward what a program names, so they are easy to game and brittle under harmless rewrites. Judges that trust a model’s answer (a quote, a sighting, an “it changed”) accept fluent claims that are not true. Every replacement therefore anchors the claim to something the model under test cannot fake: the source text itself, or measured pixels. 23 simpler judgehow it failedreplaced by keyword biome coverageevery scored model got 10/10; a one-line fake, const pine=1,...,lava=9, also got 10/10 a biome counts only if its placement code is quoted and found in the source regex voxel check (83 patterns) 43 patterns matched exactly one model’s file; renaming variables without changing behaviour moved scores by up to 14/38 code judge giving a probability and a verbatim quote per item unchecked code quotesa plausible but invented scene.add(new THREE.Mesh(palmGeo, ...)) scored a quote counts only if it is text the source contains strict quote check alone real code joined onto one line or shortened with “. . . ” was rejected: sun orbit, season cycle and moon lost (14→ 8) repair that picks source line numbers and rebuilds the quote from the file (8 → 13) asking the VLM “does this show the swamp?” said yes on all 30 frames across three worlds, including a volcano saved as the swamp blind identification of each frame before any item is judged VLM judging lighting change called frames “identical” whose mean brightness was 11.9 / 35.0 / 11.6 / 11.5 change measured from pixels: brightness, frame difference, chromaticity trusting the clock patch one world’s noon landed on the wrong requested time the brightest of four renders is taken as day Table 11: Simpler judges built first, how each failed on real worlds, and what replaced it. F Judge models and settings The main text says “the judge” for readability. Several models fill distinct roles, and every role is set in one configuration file, so a run can be repeated with the same judges or a whole judge set swapped (Table 12). rolewhat it doesmodel visionitem visibility in frames; blind biome identification gemini-3.5-flash-lite code probe quotes the code that builds each item (WC003–WC005) gemini-3.1-flash-lite text judgeWC001 source judge, clock patch, evidence agent gemini-3.1-flash-lite navigatordrives the camera to each biome gemma-4-26b-a4b-it extractWC002 biome slicing; layout reading from the whole source gemma-4-31b-it quote repair picks line numbers for a badly copied quote gemma-4-26b-a4b-it Table 12: Judge roles. A request still failing after three retries on a server error falls back alonggemma-4-31b-it→ gemma-4-26b-a4b-it→ gemini-3.1-flash-lite; a call larger than a model’s per-minute token cap is sent to the first fallback whose cap fits. WC002’s classification and layout reading both send the whole source, which for every reported world but gemini-3-1-pro’s is larger than the Gemma caps, so those calls land on the last. Settings. Frames are sent to the vision judge at 896 px wide and to the navigator at 640 px. The navigator has 8 actions per biome; the evidence agent has 10 tool calls per search. Output is capped at 16,384 tokens per call, and each request times out after 300 s. Shared family with a model under test. All six judge roles are Gemini or Gemma models, the same developer as one of the five models under test,gemini-3-1-pro. Judges are known to favour outputs from their own family [9]. The one comparison we have does not show favouritism:gemini-3-1-proranks last overall (149.00 of 280) and last on WC003 and WC004, the tests that rely most on the vision and code-probe judges. That is a single model, and a low score does not rule out a smaller bias. We did not run a cross-family judge to measure the effect directly. 24

Instant research discovery