Commit Graph
124 Commits
Author SHA1 Message Date
Jesse Zhang ce9fd8bafe turf-war prompt: opponents are other Agents (model x harness), not 'AI models' 2026-07-14 03:19:40 +01:00
Jesse Zhang 43a70bd651 turf-war prompt: maintainer-filled [This run] line (model x harness x effort, target filename) 2026-07-14 03:09:38 +01:00
Jesse Zhang 9cd85bad21 turf-war prompt: deliver a file (save turf-war.js in file-capable harnesses, print content otherwise) 2026-07-14 03:07:30 +01:00
Jesse Zhang 5c2a55bbb2 turf-war prompt: agents name themselves (fun name, <=5 chars zh / one short word en) 2026-07-14 03:00:28 +01:00
Jesse Zhang 379ab28279 turf-war prompt: forbid web search / skills / external tools / test runs (world-knowledge only) 2026-07-14 02:55:04 +01:00
Jesse Zhang 6dccc35cd8 Turf War is maintainer-run: no artifact PRs, policies stay private (anti-scouting); README case list updated to five cases 2026-07-14 02:50:27 +01:00
Jesse Zhang 50fb9978dc Add case 05 Turf War (kind: agent) + agent-case contribution guide
One-shot JS policy agents battle head-to-head on a 40x40 grid; ranked by
objective wins (Bradley-Terry), refereed server-side. Inspired by Jiayi
Weng's Learning Beyond Gradients.
2026-07-14 02:24:07 +01:00
github-actions[bot] 1142b7dfde chore: sync README registry [skip ci] 2026-07-13 23:08:59 +00:00
Jesse ZhangandClaude Opus 4.8 e3a60756f9 chore: regroup registry order by vendor
GLM-5.1 -> 120 (beside GLM-5.2/Turbo), DeepSeek V4 Flash -> 90.5 (beside V4-Pro), Nex-AGI -> 65 (out of the xAI block).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 00:08:28 +01:00
Jesse ZhangandClaude Opus 4.8 cbfce3b7cc feat: add cs-dust2 React case with two model takes + react kind support
New case cs-dust2 (kind:react): a from-scratch React + three.js 5v5 Dust2 FPS.
Two takes registered: Claude Fable 5 / Claude Code / High (with a hand fix for a
64Hz fixed-step firing bug on high-refresh displays) and Claude Opus 4.8 /
Claude Code / Max (dust2-arena, one-shot). React cases commit source only; the
single file is built from source at deploy time. validate-data.ts now accepts a
React project directory as the declared artifact. AGENTS.md rewritten
English-only with React-case, bare-harness (no skills/plugins) and
artifact-anonymity rules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 00:08:28 +01:00
github-actions[bot] c4cc293748 chore: sync README registry [skip ci] 2026-07-13 13:04:58 +00:00
_‭⁧HachiMahiro哈基寻⁧‭ e166e31d92 feat(gem3.1proais):add Gemini 3.1 Pro AiStudio High skeleton-watch benchmark (#33)
* Add files via upload

* Update contributor information in JSON file

* Change contributor and update notes in JSON file

Updated contributor name and modified notes for clarity.
2026-07-13 14:04:30 +01:00
Chang ZhouandAv1dity 7475999bff feat: Add DeepSeek V4 Flash with Claude Code Max run mythos-craft (#34)
* feat: Add DeepSeek V4 Flash Claude Code Max run mythos-craft

* feat: Remove emoji from UI-bound artifact text

* feat: Remove emoji completely

---------

Co-authored-by: Av1dity <av1dity@users.noreply.github.com>
2026-07-13 14:04:27 +01:00
github-actions[bot] a1802f0baf chore: sync README registry [skip ci] 2026-07-09 23:38:19 +00:00
Jesse ZhangandClaude Opus 4.8 5d52843621 fix: order GPT-5.5 Pro below GPT-5.6 (newest-first within OpenAI)
GPT-5.5 Pro sat at order 20, above the GPT-5.6 entries, breaking the
newest-first grouping used elsewhere. Move it to 29.8 so the OpenAI block
reads 5.6 -> 5.5 -> 5.4 -> 5.3 -> 5.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 00:37:57 +01:00
github-actions[bot] 64c033c5a7 chore: sync README registry [skip ci] 2026-07-09 22:09:38 +00:00
Jesse ZhangandClaude Opus 4.8 b8ed871bc0 docs: document multi-take variant arrays in AGENTS.md
Explain when runs.<case> becomes an array (same model x harness x effort,
several takes), the distinct-descriptive-file rule, the index.html trap,
and per-variant contributor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:09:15 +01:00
Jesse ZhangandClaude Opus 4.8 64609bd6c8 feat: add GPT-5.6-Terra and alternate takes for GPT-5.6-Sol
New model gpt-5-6-terra (Codex CLI, ultra): mythos-craft, pelican-cycling,
skeleton-watch (one-shot, unmodified).

gpt-5-6-sol-ultra gains alternate one-shot variants for existing cases:
mythos-craft 'Block Era', two more pelican SVGs, and a cleanly completed
skeleton-watch 'Calibre 01' kept alongside the interruption-recovered
AURELIUS S-01. Extra variants attributed to Nagi-ovo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:09:15 +01:00
github-actions[bot] 97e9a860a1 chore: sync README registry [skip ci] 2026-07-09 19:24:14 +00:00
mintcolorandDuJiabao e14d6cd59f feat: add Grok 4.5 benchmark results (#31)
Co-authored-by: DuJiabao <Jiabao.du@Yunzhisun.com>
2026-07-09 20:23:47 +01:00
ZidongChen25 807bdac433 feat: add GPT-5.6-Sol benchmark results (#32) 2026-07-09 20:23:36 +01:00
github-actions[bot] 2192d05ac8 chore: sync README registry [skip ci] 2026-07-06 23:26:03 +00:00
lupan-luand卢盼 ffdd06626f feat: add skeleton-watch and pelican-cycling for Gemini 3.1 Pro × Gemini Web × Deep Think (#30)
Co-authored-by: 卢盼 <lupan@lupandeMac-Studio.local>
2026-07-07 00:25:40 +01:00
github-actions[bot] ada382b2dc chore: sync README registry [skip ci] 2026-07-03 21:43:24 +00:00
lupan-luand卢盼 47a02cb481 feat: add Gemini 3.1 Pro × Gemini Web × Deep Think for mythos-craft (#29)
Co-authored-by: 卢盼 <lupan@lupandeMac-Studio.local>
2026-07-03 22:43:04 +01:00
github-actions[bot] 6c87c23963 chore: sync README registry [skip ci] 2026-07-02 16:15:51 +00:00
BaiLin-William 1a6b884f75 contrib: MiMo V2.5 (High) all cases
Adds MiMo V2.5 (High) MiMo Code artifacts for mythos-craft, pelican-cycling, and skeleton-watch. Maintainer-adjusted skeleton-watch visible calibre label to avoid blind Arena identity leakage.
2026-07-02 17:15:27 +01:00
Jesse Zhang acb27270f0 docs: scope Codex review for benchmark artifacts 2026-07-02 17:08:18 +01:00
github-actions[bot] c726210500 chore: sync README registry [skip ci] 2026-07-02 01:00:01 +00:00
Jesse Zhang 5823fed8e1 Add Claude Fable 5 Claude Code xhigh skeleton-watch run 2026-07-02 01:59:34 +01:00
Jesse ZhangandClaude Opus 4.8 4d466f74cb docs: add default-harness (no extra skills) self-check to PR template
Mirror the new contribution rule as a checklist item so contributors confirm
they ran the harness as it ships.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:31:27 +01:00
Jesse ZhangandClaude Opus 4.8 308c93ac81 docs: reject runs with extra skills outright (addr Codex #25 P2)
A `note` disclosure does not separate a modified-harness run on the board:
leaderboard identity is only model × harness × effort, so a skill-boosted run
would still land in an Agent row it is explicitly not comparable with. Drop the
"disclose in note" escape hatch and make it a flat non-acceptance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:25:08 +01:00
Jesse ZhangandClaude Opus 4.8 5cd30bac2b docs: require running the harness as it ships (no extra skills)
Address fairness concern from #24: the same harness can load completely
different skills / plugins / MCP / custom system prompts, which shifts the
output as much as swapping the harness and breaks Agent comparability.

- README.md / README.en.md: add a bullet to the Model + Harness section
  stating the harness is evaluated in its default, out-of-the-box form.
- AGENTS.md: add a "Run the harness as it ships" rule — no extra skills
  beyond what the harness bundles; disclose any deviation in `note`.

Enforcement stays honor-system + the PR generation screenshot.

Closes #24

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:18:08 +01:00
Jesse ZhangandClaude Opus 4.8 eaf2737ecc ci: verify data to site auto-deploy dispatch
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 12:47:59 +01:00
6bcj fa56bfe524 Add DeepSeek V4 Flash (Reasonix · Max) skeleton-watch run (#21)
One-shot 3D skeleton mechanical watch (Three.js). Reasonix harness confirmed real (reasonix.io, DeepSeek-native terminal coding agent). Entry id normalized to deepseek-v4-flash-reasonix by maintainer; artifact clean, no external calls beyond CDN.
2026-06-21 12:31:36 +01:00
Jesse ZhangandClaude Opus 4.8 1605c5e20b ci: trigger run to verify registry auto-sync workflow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 11:32:30 +01:00
Jesse ZhangandClaude Opus 4.8 30983f0e7d Add AGENTS.md + tighten contribution flow
- AGENTS.md: public contributor/agent guide (two-file rule, model-centric
  ids, one-entry-per-harness, bilingual notes, screenshot proof)
- PR template: require a generation proof screenshot; drop the manual
  update-registry step; add id-naming and no-README-edit checks
- CI: PRs only run validate-data; pushes to main auto-regenerate and commit
  the README registry (contents:write on that job, [skip ci], no loop)
- README/update-registry: reflect auto-synced registry; fix stale 'two cases'
  -> three (mythos-craft, pelican-cycling, skeleton-watch); link AGENTS.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 11:29:23 +01:00
YinLing 2fb8499fa8 Add GPT-5.4 (Codex CLI · xhigh) skeleton-watch run (#22)
Adds the skeleton-watch artifact + run registration for the existing GPT-5.4 / Codex CLI · xhigh agent. Validated: both CI checks pass locally; artifact renders clean, no external calls beyond CDN three.js.
2026-06-21 11:19:43 +01:00
OHCJ099 75b18a76e4 Add skeleton-watch runs: DeepSeek-V4-Pro, Gemini 3.1/3.5, GLM-5.2, Kimi K2.7, MiMo (#23)
Seven skeleton-watch runs across existing models plus the new MiMo Code harness. Validated: validate-data + update-registry --check both pass; artifacts render clean (real Chrome), no external calls beyond standard CDNs, no duplication.
2026-06-21 11:19:25 +01:00
Jesse ZhangandClaude Opus 4.8 c9cf217db3 Update registry: GLM-5.1 now ran (ZCode · Max, pelican-cycling)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:17:15 +01:00
Dmt3tian 06b9415b8d Add GLM-5.1 (ZCode · Max) pelican-cycling SVG
One-shot standalone SVG of a pelican cycling by the sea from GLM-5.1 (ZCode web, Max). Contributed by @Dmt3tianOVO.
2026-06-20 09:16:43 +01:00
Jesse ZhangandClaude Opus 4.8 8fb741ceec Add GPT-5.5 (Codex CLI xhigh) skeleton-watch run
One-shot skeleton-watch generated via codex exec with model gpt-5.5 at
xhigh effort. The prompt was prefixed with an instruction forbidding the
model from reading or referencing any other file in the repo, so it works
purely from its own knowledge. Unmodified original output; renders clean
(no console errors), registered in the model JSON.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 01:20:28 +01:00
Jesse ZhangandClaude Opus 4.8 50a2607db3 Disable GPT-5.5 Pro skeleton-watch bloom entirely for eye safety
The tamed bloom (0.28/0.45/0.9) was still blinding when zoomed in: the
intensity-96 key spotlight's near-white specular on the polished plate
kept blooming into an over-exposed haze over the whole movement. Lowering
threshold/strength was not enough, so disable the bloom (strength 0).
The movement is crisp and the lume still reads via its emissive material.
Done out of concern for photosensitive viewers; note updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 01:09:47 +01:00
Jesse ZhangandClaude Opus 4.8 16f528892c Tame GPT-5.5 Pro skeleton-watch blinding bloom glow
UnrealBloomPass used threshold 0.18 / strength 0.72, so with ACES tone
mapping and several strong lights nearly every highlight and lume surface
bloomed into one over-exposed white blob that washed out the whole dial.
Tame it to strength 0.28 / radius 0.45 / threshold 0.9 so the glow only
haloes the lume markers and the watch is legible again. Documented in the
run note (now a 2-line fix).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 01:01:18 +01:00
Jesse ZhangandClaude Opus 4.8 2cbe8b457f Document Opus 4.7 skeleton-watch exploded-view bug in run note
Clicking Exploded View barely moves anything: animate() does
const t = clock.getElapsedTime() then const dt = clock.getDelta().
getElapsedTime() already consumes the frame delta and resets oldTime,
so the following getDelta() returns ~0 and dt is ~0 every frame, leaving
the explode lerp (and crystal offset) effectively frozen. Gear rotations
use t directly so the movement still runs. Bug documented, not fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 00:54:59 +01:00
Jesse ZhangandClaude Opus 4.8 6b5c5aba25 Fix GPT-5.5 Pro skeleton-watch mainspring crash (fractional tube segments)
createSpiralTube passed Math.max(96, turns * 38) as TubeGeometry's
tubularSegments, yielding fractional counts (5.8 -> 220.4, 6.4 -> 243.2).
TubeGeometry's trailing generateSegment indexes the Frenet-frame arrays
with that count, hits undefined and reads .x, throwing a TypeError that
aborts init and leaves the page stuck on the loading overlay forever.
Round the count to an integer; documented in the run note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 00:50:11 +01:00
Jesse ZhangandClaude Opus 4.8 7eaac09dc0 Sort models by version desc and clarify effort belongs to the model (#18)
Reorder the Opus (4.8>4.7>4.6>4.5), MiMo (keep the two variants
adjacent) and GLM (5.2>5.1>5 Turbo) entries so the registry sorts by
version high-to-low instead of the previous scrambled order, adjusting
each model JSON's `order` and regenerating the README registry tables.

Also revise the model-vs-harness explainer: reasoning effort is a
capability dial of the model itself (same weights at high vs low effort
reason like different models), so it belongs to the "model" dimension
alongside the weights rather than being framed as a side parameter of
the agent.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 23:48:13 +01:00
Jesse ZhangandClaude Opus 4.8 3d5bf6967c Normalize artifact filenames and register new runs (#17)
Rename all output artifacts to their canonical case-id names
(mythos-craft.html / pelican-cycling.svg / skeleton-watch.html, with
-2 suffixes for variants) and drop the now-redundant "file" overrides
from the corresponding models/*.json runs.

Register new Claude Code runs for Opus 4.6 and 4.7 (new model JSONs),
add the GPT-5.5 Pro skeleton-watch run, and regenerate the README
registry tables. Remove stray .DS_Store files from outputs/.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 23:36:38 +01:00
Jesse ZhangandClaude Opus 4.8 dd67d6fda5 Add Step 3.7 Flash runs (mythos-craft, pelican-cycling)
Step 3.7 Flash (StepFun) via Claude Code · High, one-shot. The mythos-craft
Write tool call failed during generation, so the model's output was saved into
the HTML by hand (code unmodified) — documented in the run note. Registry
tables regenerated; validation passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 22:03:52 +01:00
OHCJ099 13da28bd8e Add MiMo v2.5 Pro UltraSpeed runs (mythos-craft, pelican-cycling)
Registers MiMo v2.5 Pro UltraSpeed (Xiaomi, Web · Default) with two one-shot variants each for mythos-craft and pelican-cycling. Verified: SVGs render, both Three.js voxel games run, schema validates, README tables synced (zh+en).
2026-06-16 20:20:34 +01:00