claude-fable-5-code -> claude-fable-5-cc, claude-opus-4-8 ->
claude-opus-4-8-cc (the entry id doubles as the outputs/ folder name).
The Opus mythos-craft artifact is a newer build of the same one-shot
provenance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The showcase app and the arena backend now live in the private
nagi-bench-site repo and deploy to Cloudflare Pages, fetching this
repo's models/, outputs/ and cases.json at build time. CI here is a
standalone data validation (same rules as before, no build), plus an
optional repository_dispatch that triggers an instant site rebuild when
SITE_DISPATCH_TOKEN is configured — otherwise the site's 6-hour cron
picks new data up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cases.json is data (bilingual case definitions, maintainer-owned), so it
moves alongside models/ and outputs/; the site build will consume it
from here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turnstile tokens are single-use — the old implementation memoized one
token forever, so every batch after the first would fail verification.
Each sync now runs a fresh invisible challenge, and vote syncing is
debounced 2.5s so rapid ballots coalesce into one batch + one challenge.
OG count 16 -> 17 ran.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The artifact landed unregistered which would fail CI's loose-file check;
note marks harness/effort as to-be-confirmed. worker/.wrangler/ cache is
local tooling and leaves the repo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every (voter, case, pair) now gets exactly one vote: the client sampler
only draws matchups the browser hasn't voted on (with a graduation state
once all are spent, counter shows voted/total pairs), and the worker
normalizes pair order on insert under a unique (ip, case, pair) index so
side-swapping can't double-count. Daily IP cap drops 300 -> 200 since a
voter graduates at ~171 pairs today.
Matchmaking is now active rather than uniform: unvoted pairs are drawn
with weight closeness/(1+battles) — close ratings and under-played
matchups surface more, unrated combos get placement matches first — with
a 25% pure-exploration floor. Ratings come from the community pool (60s
cached) and fall back to local votes. Rankings stay Bradley-Terry.
Also registers the empty outputs/claude-opus-4-8/ folder as a pending
registry entry.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each case now defaults to Arena mode: two anonymous artifacts (identities
hidden, footers suppressed) battle via A / B / Neither, reveal after the
vote, with the same clip-reveal motion language as the rest of the site.
Demo Show replaces the per-combination tab strip with one tab per model;
multi-harness models get a harness switcher and a side-by-side compare,
so the lineup no longer bloats as harnesses multiply.
Votes live in lib/arena.ts (localStorage, custom event fan-out) and rank
via Bradley-Terry fits on pairwise wins, LMSYS Chatbot Arena style, with
'neither' surfaced as a both-bad rate. The #board section stays locked
until 3 votes are cast in each case (blind first, board after), then
shows your votes plus the ranking. When VITE_ARENA_API is configured the
store syncs to the zero-dependency Cloudflare Worker + D1 backend in
worker/ (Turnstile verification, per-IP daily caps, idempotent retries);
deploy steps live in worker/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per PR #2's description those two pelicans were generated in Google AI
Studio, not AntiGravity — they move to gemini-3-1-pro-aistudio and
gemini-3-5-flash-aistudio entries (harness Google AI Studio, High).
XA2333's unknown-environment runs stay where they were, disclaimed in
their notes. Adds a bilingual PR template asking for harness/effort/
one-shot info up front, and registers the new Kimi K2.6 mythos-craft
artifact (16 combinations ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kimi K2.6 (Kimi Code, local) and MiniMax M3 (MiniMax Code web, Thinking
mode) graduate from pending to ran — 14 combinations ran. Harness icon
rules added for Kimi/MiniMax; READMEs and OG count updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The minecraft.html artifact is Fable 5 Max run inside Claude Code — a
different harness from the existing Claude Web App entry, so per the
model x harness x effort architecture it becomes a new combination
(claude-fable-5-code) rather than a variant. The validator now also
rejects loose files under outputs/<id>/ that no run declares, which is
how this artifact briefly slipped into the repo unregistered. OG count
refreshed (11 -> 12 ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There is no "GPT-5.5 Codex" model — Codex is the harness. The entry is
GPT-5.5 run in Codex CLI at xhigh effort, which the harness and effort
fields already record; READMEs and code comments updated to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Bump body text across the registry and case sections (notes, prompts,
tabs, table cells) one size up for readability; SVG viewer no longer
uses the full-bleed breakout and caps at a smaller height (HTML keeps
the big frame).
- Brand icons for model, vendor and harness via @lobehub/icons-static-svg
(zero-dep static assets; deliberately not the @lobehub/icons React
package, whose peers pull in antd). Mono SVGs are currentColor so they
follow the theme. Mapping lives in site/src/lib/brands.tsx, derived
from the strings contributors already fill in models/*.json.
- Case pages now show Harness (with icon) and Effort chips next to the
contributor chip; registry harness column is labeled 运行环境（Harness）.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Runs gain a contributor field (GitHub username); the site renders an
avatar chip linked to their profile next to each run's notes, and CI
validates the field. All existing runs are attributed.
- SVG variants of a case are prefetched during idle time, so switching
model tabs swaps images without a network hiccup.
- Xiaomi MiMo v2.5 Pro (Claude Code, max effort) integrated from PR #4
with both case runs and provenance notes; README registry tables
updated.
Co-authored-by: DuJiabao <Jiabao.du@Yunzhisun.com>
Each entry is now explicitly a model x harness x effort combination:
models/*.json gains harness and effort fields, GPT-5.5 xhigh is
relabeled GPT-5.5 Codex (the harness distinguishes it from GPT-5.5
Pro), and the scrolling marquee is replaced by a readable registry
table (model / vendor / harness / effort / runs, pending rows dimmed)
with an explainer on the model-vs-harness relationship.
Top bar controls enlarged: segmented zh/EN switch and bordered 36px
icon buttons. README rewritten Chinese-first with a hand-maintained
registry snapshot table; README.en.md mirrors it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A combination can now ship multiple versions of the same case:
runs.<case-id> accepts an array, each extra variant naming its own
file; the viewer grows a RUN 01/02 switcher and validation enforces
distinct files plus a provenance note per variant.
Integrated content, attributed to their authors:
- Gemini 3.1 Pro + 3.5 Flash pelican runs from both PRs, shown as two
variants each (PR #2's Google AI Studio runs first, PR #3's second)
- Doubao Seed 2.0 Pro / Mini pelican runs (folders renamed to the
dash-only convention)
- GPT-5.5 Codex pelican run
Co-authored-by: lpluma <liuyz2023@mails.jlu.edu.cn>
Co-authored-by: XA2333 <97206278+XA2333@users.noreply.github.com>
Contributors no longer touch site code. Model registration moves from
cases.ts literals to repo-root models/<model-id>.json files (filename =
model id = outputs/ folder); cases move to cases.json; cases.ts now just
derives MODELS/RUNS via import.meta.glob. Parallel PRs stop conflicting
because each model owns its own file.
scripts/validate-data.ts runs inside `bun run build` and a new PR CI
workflow: dash-only ids, required bilingual provenance note per run,
declared artifacts must exist, no orphan outputs/ dirs.
Also: run notes render whitespace-pre-wrap (multi-line notes, idea from
PR #2), README contribution guide rewritten around the JSON template,
CLAUDE.md updated, OG card refreshed for the DeepSeek merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Site:
- Cursor follower fades out over iframes (it froze on top of running
artifacts since iframes swallow pointer events) and when the pointer
leaves the window
- HTML viewer: full-bleed width on xl screens (up to 90rem), height
capped to 100dvh-10rem so the frame plus its action row always fits
the viewport (big iframes ate wheel events and trapped page scroll),
fullscreen button (overlay + footer action), and Run now auto-aligns
the frame below the fixed header
Artifacts:
- grok-build: W/S were inverted against the forward vector (W walked
backward); model label renamed to Grok Build
- composer-2-5: added G-key cycle teleport to its three generated
landmarks (elven tower, dragon temple, sky ruins) with a start-screen
hint - they were previously only reachable on foot; ground landmarks
arrive 10 blocks out facing the structure, sky ruins land on the deck
Run notes updated accordingly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- gemini-3-5-flash: ATLAS_COORDS and the canvas tiles share a top-origin
row convention, but default flipY=true mirrored sampling (row r read
row 3-r): grass tops became purple crystal, sides hit the alpha-holed
portal/leaves tiles (the "transparent blocks"). flipY=false, 1 line.
- gpt-5-5-xhigh: movement Z component sign-flipped against the -Z
camera convention, mirroring all motion across the X axis (A/D
swapped facing +-X). Negated Z, 1 line.
Run notes now carry full provenance: GPT-5.5 Pro Extended Pro, Codex
CLI (no skills) for xhigh, AntiGravity high for both Geminis, and the
local Grok Build TUI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Surgical fixes (originals preserved in history), each documented in the
per-model run notes as: where the bug is, pre-fix behavior, root cause,
with the shown build being the fixed one:
- grok-build: 0xAETHER invalid hex literal killed the script at parse
time (pure black screen); atlas flipY mismatch made near terrain
sample undrawn black texels. 2 lines.
- composer-2-5: horizontal movement multiplied dt twice, walking at
~1/60 design speed and making biomes effectively unreachable. 1 line.
- gpt-5-5-pro: 8x2 atlas rows swapped by default flipY, grass became
obsidian and leaves/sand sampled undrawn slots. 1 line.
Gemini 3.5 Flash added as-is (color/transparency analysis next).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Folders renamed to the outputs/<model-id>/<case-id>.html convention
(grok-build per request); model ids normalized to dash-only so they
double as folder names. Gemini 3.1 Pro, Grok and Composer 2.5 flip to
ran status with tabs wired up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Vite base '/' and root favicon for the custom domain
- Artifact paths normalized to outputs/<model-id>/<case-id>.<ext>
(pelican SVG renamed); RUNS now derives the path by convention,
with an optional file override
- Deep links: #<case-id> anchors a case, #<case-id>:<model-id> also
preselects the model tab; tab clicks replaceState the hash so the
current view is always shareable
- README documents the URL scheme and the legacy github.io 301s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-shot LLM eval cases with a bilingual (zh/EN) showcase site:
Vite + React + Tailwind v4 + GSAP, built with bun, deployed to
GitHub Pages. Ships two Claude Fable 5 Max cases (Mythos Craft
voxel game, pelican-by-the-sea SVG) with placeholder slots for
GPT-5.1, Gemini 3 Pro, DeepSeek V3.2 and Qwen3 Max.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
