- AGENTS.md: public contributor/agent guide (two-file rule, model-centric
ids, one-entry-per-harness, bilingual notes, screenshot proof)
- PR template: require a generation proof screenshot; drop the manual
update-registry step; add id-naming and no-README-edit checks
- CI: PRs only run validate-data; pushes to main auto-regenerate and commit
the README registry (contents:write on that job, [skip ci], no loop)
- README/update-registry: reflect auto-synced registry; fix stale 'two cases'
-> three (mythos-craft, pelican-cycling, skeleton-watch); link AGENTS.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The showcase app and the arena backend now live in the private
nagi-bench-site repo and deploy to Cloudflare Pages, fetching this
repo's models/, outputs/ and cases.json at build time. CI here is a
standalone data validation (same rules as before, no build), plus an
optional repository_dispatch that triggers an instant site rebuild when
SITE_DISPATCH_TOKEN is configured — otherwise the site's 6-hour cron
picks new data up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each case now defaults to Arena mode: two anonymous artifacts (identities
hidden, footers suppressed) battle via A / B / Neither, reveal after the
vote, with the same clip-reveal motion language as the rest of the site.
Demo Show replaces the per-combination tab strip with one tab per model;
multi-harness models get a harness switcher and a side-by-side compare,
so the lineup no longer bloats as harnesses multiply.
Votes live in lib/arena.ts (localStorage, custom event fan-out) and rank
via Bradley-Terry fits on pairwise wins, LMSYS Chatbot Arena style, with
'neither' surfaced as a both-bad rate. The #board section stays locked
until 3 votes are cast in each case (blind first, board after), then
shows your votes plus the ranking. When VITE_ARENA_API is configured the
store syncs to the zero-dependency Cloudflare Worker + D1 backend in
worker/ (Turnstile verification, per-IP daily caps, idempotent retries);
deploy steps live in worker/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per PR #2's description those two pelicans were generated in Google AI
Studio, not AntiGravity — they move to gemini-3-1-pro-aistudio and
gemini-3-5-flash-aistudio entries (harness Google AI Studio, High).
XA2333's unknown-environment runs stay where they were, disclaimed in
their notes. Adds a bilingual PR template asking for harness/effort/
one-shot info up front, and registers the new Kimi K2.6 mythos-craft
artifact (16 combinations ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributors no longer touch site code. Model registration moves from
cases.ts literals to repo-root models/<model-id>.json files (filename =
model id = outputs/ folder); cases move to cases.json; cases.ts now just
derives MODELS/RUNS via import.meta.glob. Parallel PRs stop conflicting
because each model owns its own file.
scripts/validate-data.ts runs inside `bun run build` and a new PR CI
workflow: dash-only ids, required bilingual provenance note per run,
declared artifacts must exist, no orphan outputs/ dirs.
Also: run notes render whitespace-pre-wrap (multi-line notes, idea from
PR #2), README contribution guide rewritten around the JSON template,
CLAUDE.md updated, OG card refreshed for the DeepSeek merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-shot LLM eval cases with a bilingual (zh/EN) showcase site:
Vite + React + Tailwind v4 + GSAP, built with bun, deployed to
GitHub Pages. Ships two Claude Fable 5 Max cases (Mythos Craft
voxel game, pelican-by-the-sea SVG) with placeholder slots for
GPT-5.1, Gemini 3 Pro, DeepSeek V3.2 and Qwen3 Max.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
