Big models on real hardware. Playable worlds, cloud vs local comparisons, and the evidence behind what actually works.
Explore the experiments
Astra vs DeepSeek V4.1 Cloud vs Local. Watch the comparison, then try the originals.
A 512×512-subdivision mesh, click-generated ripples, interacting waves, real-time lighting. One HTML file.
*“Worked for 30s” is UI-reported, not independently timed. Local generated extensive reasoning before writing the HTML and includes Hermes agent/tool overhead. These are different workflows—not a controlled speed benchmark.
Cloud: thinking enabled; 28,479 completion tokens, including 23,977 reasoning tokens; 1.38s to the first model delta. About 280 completion tokens/s after the first delta, including reasoning. Served API model ID: deepseek-flash.
Local: DeepSeek V4.1 Flash; FP8 configuration with FP4 experts; TP4 across four DGX Sparks. A 300,000-token window was configured—not 300,000 tokens used. Approximately 47 tokens/s in a live sample, not a full-run average. No verified task-level token total is reported here.
Packaging: Astra embeds Three.js (~688 KB HTML); cloud (~13.3 KB) and local (~15 KB) load the library externally. File sizes are not directly comparable. Code generation happens on the model servers; the water renders in your browser.
The video combines supplied recordings and a fresh cloud capture. It is a visual comparison, not a synchronized runtime benchmark.
Not concept art. Actual model-written code you can open, orbit, play, and question.
26 selected experiments
Kindling vLLM NVFP4, Mia vLLM EXL3, TensorFold EXL3 and TensorFold MLX 4-bit on the same two Sparks: speed table, exactness, island and volcano. Kindling wins crowds and long prompts; TensorFold MLX wins single-user speed with byte-identical output.
Originals untouched · repairs disclosed per arm · one run per stack per prompt
TensorFold 0.6.1 vs vLLM 0.30.0 vs SGLang 0.5.21, same weights and drafter. TensorFold went 62 → 372 tok/s at 16 at once and is the only engine with byte-identical output under load.
Measured numbers only · original 0.3.6.3 chart kept with the correction
The same pagoda brief the local models get, built by Claude Sonnet 5.5 as a multi-agent workflow: three competing builders, two merges, a polish pass and three adversarial review rounds. ~260K voxel instances, ~37 draw calls, 70+ FPS on an M4 Max.
Uploaded unchanged · agentic multi-pass build, not a one-shot generation
Same checkpoint, same brief, Mac Studio vs DGX Spark, thinking off and on, two seeds. Eight different games; the Spark's two long thinking runs (36–41 min) built the only real rivers.
Originals untouched · 1 arm broken, repair abandoned · capture-side RNG pin disclosed
One switch. Thinking on: 144.6K tokens, 88 minutes, a seven-tier pagoda with lanterns and pines, two one-line repairs. Thinking off: 8.6K tokens, 2.7 minutes, a white box that runs clean.
Originals untouched · 2 disclosed edits on the thinking-on arm · capture-side RNG pin disclosed
Grok 4.7 writes the whole scene in its reasoning and ships nothing over the API. GLM and MiMo both land it first try on two Sparks; MiMo in half the time.
Grok 4.7 arm from grok.com chat, untimed · MiMo capped first run discarded · no repairs
Day/night, tooltip, regenerate. GLM, MiMo and DeepSeek ship it clean; DeepSeek in half GLM's tokens. Qwen is fastest and ships a parse error.
Identical scripted inputs · thinking on, uncapped · Qwen shown after 2 disclosed fixes
A long WebGPU + XPBD brief. DeepSeek renders a holey wedge after one sign fix; Qwen3.8-27B never initialises. Two thinking-on runs burned their whole budget and wrote no HTML. GLM and Qwen-Next pending.
Originals untouched · one disclosed edit each · thinking-on burns listed in evidence.json
Both models delivered first try. GLM's pond is clearly the better one; MiMo finished in half the time.
Identical scripted inputs · thinking on, uncapped · no repairs
An underspecified Three.js brief finally separates the models. Astra runs away with it; GLM proves two Sparks can hang; DeepSeek forgets the ocean.
Cloud arms untimed · DeepSeek TP2 not TP4 · GLM node crash disclosed
Same prompt, ultra reasoning, two Spark pairs — and two nearly identical games. A lesson in what a classic-game prompt actually tests.
Identical scripted inputs · different worlds · no repairs
One water prompt through ChatGPT, the official API and a local Hermes agent. Watch the outputs and inspect the timing caveats.
Different workflows · not a controlled speed benchmark
From the first spark to wind-driven blowout. Three local deployments tackle the same fire simulation, with repairs disclosed.
Repaired variants labeled · 120K request budget
A dense district-planned garden meets a compact colorful pagoda. Two model families, two engines, one creative brief.
Different engines · requested effort is not equivalent policy
The original, the repaired scene and the exact diff. A reminder that generating HTML is not the same as shipping a working world.
Nex: two missing declarations fixed · original downloadable
Two floating islands and an AEON arm that failed to finish a usable artifact. Model failures and harness failures stay separate.
Mac repair disclosed · failed AEON attempts documented
Explore each model’s pagoda garden, from koi ponds to drifting petals. Switch models and orbit the actual outputs.
Includes disclosed display repairs and failed runs
Clock towers, airships and islands above the clouds. One prompt, very different interpretations of a floating world.
Originals and display-repaired variants identified in archive
A midnight run through Tokyo. Drive, drift, switch the camera, and put a model-generated game through its paces.
Paint the landscape, send water downhill, and watch a small world respond. An interactive terraforming diorama.
Two-Spark Flash meets four-Spark Full on the same solar-system brief. Explore both interactive orreries.
Flash display repair disclosed · original preserved
Two long-reasoning models turn a shared brief into interactive galaxies. Inspect the details beyond the hero shot.
Paper folds, transforms and takes flight. A test of geometry, motion and whether the ambitious brief survives rendering.
A quiet museum object becomes a miniature supercell. Compare the generated scenes and watch the recordings.
Flash-Next, Max at 1-bit and a smaller 27B model tackle the same garden. Different deployment sizes, visible trade-offs.
Flash-Next render fix disclosed on experiment page
A useful counterexample: the 1-bit version never became a working game, even after follow-up attempts. The broken output stays public.
Includes failed output and follow-up attempts
Looking for an older output? Browse the full published HTML archive, including benchmark visuals, variants, and failures.
Open full archiveReproduce the setups: DGX Spark and Apple Silicon, quantization, native MTP and DFlash. Configurations, upstream credits and measured limitations included.
Measured acceleration and long-input retrieval, with output-quality caveats. The drafter carries a noncommercial license; this is not production certification.
02The local serving recipe behind recent long-thinking visuals. Configuration, runtime evidence and practical deployment limits—not a context-free speed claim.
03A vision-native model on a development serving stack. Includes a sustained pagoda generation and the setup needed to reproduce it.
04A short-prompt acceleration study with a 4-bit/8-bit follow-up. Serialization and correctness caveats matter as much as the measured throughput.
05Two pinned recipe revisions, full transcripts and a template-regression investigation. Environment and grader differences are disclosed in the evidence pack.
06A native C + Metal deployment, pagoda artifact and repair record. Configured context capacity is separate from filled-context performance.
07Explore precision, engine and speculative-decoding trade-offs. These are deployment comparisons, not a claim that only quantization changed.
08The portable harness for preserving model responses, comparing generated HTML and keeping repairs separate from the original output.
Our open-source benchmark for local, agentic workloads: executable code, tool use, multi-turn workflows and rendered visual outputs.
76 scenarios across 12 domains. The uncapped release preserves responses, grading reasons and run provenance. This link is pinned to the published release, not a moving development branch.
The published v6.8.0 comparison: Qwen3.8-Flash-Next and GLM-5.3-Flash, with two repeats across the full suite. Read the per-repeat results, transcripts and exclusions together.
Published one-, two- and four-Spark cohorts, with their original methodology labels and generated artifacts. Keep these historical scores separate from v6.8.0 results.
Published scope: v6.8.0 at 125ba16. An operator benchmark for specific model-and-recipe combinations—not a universal model ranking. Browse the release’s serving recipes ↗
Local hardware for ownership and experimentation. Cloud models for comparison. No single machine tells the whole story.
Model serving, multi-node inference, quantization and speculative decoding. One-, two-, and four-node experiments.
Persistent agents, orchestration, browser validation and media production, plus a separate history of MLX and llama.cpp experiments.
Verified backups, checkpoint transfers and preserved evidence. Active Spark serving uses local NVMe—not network storage.
Hardware overview, not a live availability dashboard. Runtime, precision, context and topology belong to each individual experiment.
A beautiful output is only half the story. The useful part is knowing what generated it, what broke, and what changed before you saw it.
Model, quant, engine, hardware, context, thinking policy and tool harness. Tokens per second without a denominator is not enough.
Preserve model output. Label display fixes and follow-up attempts. A gallery thumbnail is not proof of an untouched first attempt.
Open the page, exercise the controls, inspect the motion. A passing syntax check is not a working simulation.
Different harnesses, timing boundaries or thinking settings get a caveat—not a manufactured speedup. Failed runs are results too.
Automated twice-daily Visual Arena runs, a unified controller with node leasing, and a repeatable Quant Factory remain roadmap work—not claimed production capabilities. This page features published experiments and sourced research, not completion promises.
The worlds are playable. The research is public. Bring your own skepticism.
