Reorder the Opus (4.8>4.7>4.6>4.5), MiMo (keep the two variants
adjacent) and GLM (5.2>5.1>5 Turbo) entries so the registry sorts by
version high-to-low instead of the previous scrambled order, adjusting
each model JSON's `order` and regenerating the README registry tables.
Also revise the model-vs-harness explainer: reasoning effort is a
capability dial of the model itself (same weights at high vs low effort
reason like different models), so it belongs to the "model" dimension
alongside the weights rather than being framed as a side parameter of
the agent.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename all output artifacts to their canonical case-id names
(mythos-craft.html / pelican-cycling.svg / skeleton-watch.html, with
-2 suffixes for variants) and drop the now-redundant "file" overrides
from the corresponding models/*.json runs.
Register new Claude Code runs for Opus 4.6 and 4.7 (new model JSONs),
add the GPT-5.5 Pro skeleton-watch run, and regenerate the README
registry tables. Remove stray .DS_Store files from outputs/.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step 3.7 Flash (StepFun) via Claude Code · High, one-shot. The mythos-craft
Write tool call failed during generation, so the model's output was saved into
the HTML by hand (code unmodified) — documented in the run note. Registry
tables regenerated; validation passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registers MiMo v2.5 Pro UltraSpeed (Xiaomi, Web · Default) with two one-shot variants each for mythos-craft and pelican-cycling. Verified: SVGs render, both Three.js voxel games run, schema validates, README tables synced (zh+en).
Zhipu AI's GLM-5.2 listed alongside GLM-5.1 as awaiting a contribution (no
runs yet). Added to both README pending lines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #9 added models/kimi-k2-7-code.json but not the README snapshot rows; sync
both zh + en tables (Moonshot AI · Kimi Code · Thinking · 01).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One-shot, unmodified outputs for Qwen3.7-Max (Alibaba) run in Qoder, default
thinking; contributor Nagi-ovo.
- outputs/qwen3-7-max/mythos-craft.html — Three.js voxel world (chunked
terrain, first-person controls, break/place, fantasy structures).
- outputs/qwen3-7-max/pelican-cycling.svg — hand-authored seaside SVG, pelican
on a bicycle with a side-profile study inset.
Filenames normalized to the canonical case ids and the folder renamed to match
the model id (qwen3-7-max). Moved out of Pending in both README registries.
Validates: Data OK, 25 models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two one-shot runs generated in the Vibe web app (Mistral's renamed
Le Chat, Thinking enabled), contributed by Nagi-ovo.
mythos-craft is a fixed build (1 line changed): the world synchronously
built the full RENDER_DISTANCE=4 ring (9x9 = 81 chunks) on load, each
chunk doing an O(voxels x structures) linear scan over mythStructures
for all 16x16x128 voxels, so the main thread froze before the first
frame and the page never opened. RENDER_DISTANCE 4 -> 1 (a 3x3 = 9-chunk
first-screen load) lets it open at 60fps; terrain/visuals/gameplay are
the model's original output, otherwise unmodified. pelican-cycling is
unmodified. Both documented in the run notes; READMEs updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cursor's Composer "Fast" is a serving tier (lower latency, higher cost, same
model intelligence) — not a reasoning-effort level. Drop the Fast/effort framing
entirely: effort is Default, notes just state a plain one-shot Composer 2.5 run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Cursor "Fast" run was registered as label "Composer 2.5 Fast" with effort
"Max" — contradictory, and it baked the speed mode into the model name. Per the
model x harness x effort convention (cf. the Grok Build TUI entry), the label is
just "Composer 2.5" and Fast is the effort, so it sits next to the Cursor · Max
entry as a thinking-budget comparison. Fixed the run notes to match, and synced
both README registry tables (which had also missed the Cursor · High Claude
Fable 5 entry from the prior merge).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the site change — the community leaderboard now unlocks after 3 votes
in any mix of cases, not 3 in each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A blockquote at the top of the contributing section, addressed to coding
agents: states the unit (Agent = model × harness) and the id convention,
names the two current cases with cases.json as the verbatim-prompt source,
and explains how to find open slots from cases.json + models/*.json (and the
Registry "Runs" column) so it stays correct as contributions land.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A missing effort reads as Max by the repo's "effort defaults to max"
convention; this run was the harness default, so set it explicitly to
"Default" and surface it in both README registry rows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lead the model-vs-harness section with the equation Model + Harness =
Agent (English equation + Chinese gloss): a model is just weights; wrapped
in a harness it becomes an agent that does the work, and that agent is the
unit this bench evaluates. Effort is reframed as a parameter of the agent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New Agent composer-2-5-grok-build (Composer 2.5 model in the Grok Build TUI
harness, contributor Nagi-ovo), two runs: mythos-craft + pelican-cycling.
mythos-craft fix (1 line): the WOOD block defines only top+side faces (no
bottom), but the atlas builder requests top/side/bottom for every block.
Fetching WOOD's bottom read t.bottom.image off undefined and threw a
TypeError at the top level of initTextures(), aborting init before the
"enter world" click listener attached — so clicking the start button did
nothing. getFaceImage now falls back to an existing face when one is
missing (bottom→top, the correct ring end-grain for a log). Verified in
headless Chrome: click now fires requestPointerLock and the overlay hides.
Artifacts renamed to the case-id convention; README registry rows synced
(Composer 2.5 now 3 runs across Cursor + Grok Build TUI).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One row per model with its harness x effort combinations listed
together; generated from models/*.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The showcase app and the arena backend now live in the private
nagi-bench-site repo and deploy to Cloudflare Pages, fetching this
repo's models/, outputs/ and cases.json at build time. CI here is a
standalone data validation (same rules as before, no build), plus an
optional repository_dispatch that triggers an instant site rebuild when
SITE_DISPATCH_TOKEN is configured — otherwise the site's 6-hour cron
picks new data up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turnstile tokens are single-use — the old implementation memoized one
token forever, so every batch after the first would fail verification.
Each sync now runs a fresh invisible challenge, and vote syncing is
debounced 2.5s so rapid ballots coalesce into one batch + one challenge.
OG count 16 -> 17 ran.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every (voter, case, pair) now gets exactly one vote: the client sampler
only draws matchups the browser hasn't voted on (with a graduation state
once all are spent, counter shows voted/total pairs), and the worker
normalizes pair order on insert under a unique (ip, case, pair) index so
side-swapping can't double-count. Daily IP cap drops 300 -> 200 since a
voter graduates at ~171 pairs today.
Matchmaking is now active rather than uniform: unvoted pairs are drawn
with weight closeness/(1+battles) — close ratings and under-played
matchups surface more, unrated combos get placement matches first — with
a 25% pure-exploration floor. Ratings come from the community pool (60s
cached) and fall back to local votes. Rankings stay Bradley-Terry.
Also registers the empty outputs/claude-opus-4-8/ folder as a pending
registry entry.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each case now defaults to Arena mode: two anonymous artifacts (identities
hidden, footers suppressed) battle via A / B / Neither, reveal after the
vote, with the same clip-reveal motion language as the rest of the site.
Demo Show replaces the per-combination tab strip with one tab per model;
multi-harness models get a harness switcher and a side-by-side compare,
so the lineup no longer bloats as harnesses multiply.
Votes live in lib/arena.ts (localStorage, custom event fan-out) and rank
via Bradley-Terry fits on pairwise wins, LMSYS Chatbot Arena style, with
'neither' surfaced as a both-bad rate. The #board section stays locked
until 3 votes are cast in each case (blind first, board after), then
shows your votes plus the ranking. When VITE_ARENA_API is configured the
store syncs to the zero-dependency Cloudflare Worker + D1 backend in
worker/ (Turnstile verification, per-IP daily caps, idempotent retries);
deploy steps live in worker/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per PR #2's description those two pelicans were generated in Google AI
Studio, not AntiGravity — they move to gemini-3-1-pro-aistudio and
gemini-3-5-flash-aistudio entries (harness Google AI Studio, High).
XA2333's unknown-environment runs stay where they were, disclaimed in
their notes. Adds a bilingual PR template asking for harness/effort/
one-shot info up front, and registers the new Kimi K2.6 mythos-craft
artifact (16 combinations ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kimi K2.6 (Kimi Code, local) and MiniMax M3 (MiniMax Code web, Thinking
mode) graduate from pending to ran — 14 combinations ran. Harness icon
rules added for Kimi/MiniMax; READMEs and OG count updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The minecraft.html artifact is Fable 5 Max run inside Claude Code — a
different harness from the existing Claude Web App entry, so per the
model x harness x effort architecture it becomes a new combination
(claude-fable-5-code) rather than a variant. The validator now also
rejects loose files under outputs/<id>/ that no run declares, which is
how this artifact briefly slipped into the repo unregistered. OG count
refreshed (11 -> 12 ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There is no "GPT-5.5 Codex" model — Codex is the harness. The entry is
GPT-5.5 run in Codex CLI at xhigh effort, which the harness and effort
fields already record; READMEs and code comments updated to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Bump body text across the registry and case sections (notes, prompts,
tabs, table cells) one size up for readability; SVG viewer no longer
uses the full-bleed breakout and caps at a smaller height (HTML keeps
the big frame).
- Brand icons for model, vendor and harness via @lobehub/icons-static-svg
(zero-dep static assets; deliberately not the @lobehub/icons React
package, whose peers pull in antd). Mono SVGs are currentColor so they
follow the theme. Mapping lives in site/src/lib/brands.tsx, derived
from the strings contributors already fill in models/*.json.
- Case pages now show Harness (with icon) and Effort chips next to the
contributor chip; registry harness column is labeled 运行环境（Harness）.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Runs gain a contributor field (GitHub username); the site renders an
avatar chip linked to their profile next to each run's notes, and CI
validates the field. All existing runs are attributed.
- SVG variants of a case are prefetched during idle time, so switching
model tabs swaps images without a network hiccup.
- Xiaomi MiMo v2.5 Pro (Claude Code, max effort) integrated from PR #4
with both case runs and provenance notes; README registry tables
updated.
Co-authored-by: DuJiabao <Jiabao.du@Yunzhisun.com>
Each entry is now explicitly a model x harness x effort combination:
models/*.json gains harness and effort fields, GPT-5.5 xhigh is
relabeled GPT-5.5 Codex (the harness distinguishes it from GPT-5.5
Pro), and the scrolling marquee is replaced by a readable registry
table (model / vendor / harness / effort / runs, pending rows dimmed)
with an explainer on the model-vs-harness relationship.
Top bar controls enlarged: segmented zh/EN switch and bordered 36px
icon buttons. README rewritten Chinese-first with a hand-maintained
registry snapshot table; README.en.md mirrors it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
