A `note` disclosure does not separate a modified-harness run on the board:
leaderboard identity is only model × harness × effort, so a skill-boosted run
would still land in an Agent row it is explicitly not comparable with. Drop the
"disclose in note" escape hatch and make it a flat non-acceptance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address fairness concern from #24: the same harness can load completely
different skills / plugins / MCP / custom system prompts, which shifts the
output as much as swapping the harness and breaks Agent comparability.
- README.md / README.en.md: add a bullet to the Model + Harness section
stating the harness is evaluated in its default, out-of-the-box form.
- AGENTS.md: add a "Run the harness as it ships" rule — no extra skills
beyond what the harness bundles; disclose any deviation in `note`.
Enforcement stays honor-system + the PR generation screenshot.
Closes#24
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One-shot 3D skeleton mechanical watch (Three.js). Reasonix harness confirmed real (reasonix.io, DeepSeek-native terminal coding agent). Entry id normalized to deepseek-v4-flash-reasonix by maintainer; artifact clean, no external calls beyond CDN.
- AGENTS.md: public contributor/agent guide (two-file rule, model-centric
ids, one-entry-per-harness, bilingual notes, screenshot proof)
- PR template: require a generation proof screenshot; drop the manual
update-registry step; add id-naming and no-README-edit checks
- CI: PRs only run validate-data; pushes to main auto-regenerate and commit
the README registry (contents:write on that job, [skip ci], no loop)
- README/update-registry: reflect auto-synced registry; fix stale 'two cases'
-> three (mythos-craft, pelican-cycling, skeleton-watch); link AGENTS.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the skeleton-watch artifact + run registration for the existing GPT-5.4 / Codex CLI · xhigh agent. Validated: both CI checks pass locally; artifact renders clean, no external calls beyond CDN three.js.
Seven skeleton-watch runs across existing models plus the new MiMo Code harness. Validated: validate-data + update-registry --check both pass; artifacts render clean (real Chrome), no external calls beyond standard CDNs, no duplication.
Reorder the Opus (4.8>4.7>4.6>4.5), MiMo (keep the two variants
adjacent) and GLM (5.2>5.1>5 Turbo) entries so the registry sorts by
version high-to-low instead of the previous scrambled order, adjusting
each model JSON's `order` and regenerating the README registry tables.
Also revise the model-vs-harness explainer: reasoning effort is a
capability dial of the model itself (same weights at high vs low effort
reason like different models), so it belongs to the "model" dimension
alongside the weights rather than being framed as a side parameter of
the agent.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename all output artifacts to their canonical case-id names
(mythos-craft.html / pelican-cycling.svg / skeleton-watch.html, with
-2 suffixes for variants) and drop the now-redundant "file" overrides
from the corresponding models/*.json runs.
Register new Claude Code runs for Opus 4.6 and 4.7 (new model JSONs),
add the GPT-5.5 Pro skeleton-watch run, and regenerate the README
registry tables. Remove stray .DS_Store files from outputs/.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step 3.7 Flash (StepFun) via Claude Code · High, one-shot. The mythos-craft
Write tool call failed during generation, so the model's output was saved into
the HTML by hand (code unmodified) — documented in the run note. Registry
tables regenerated; validation passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registers MiMo v2.5 Pro UltraSpeed (Xiaomi, Web · Default) with two one-shot variants each for mythos-craft and pelican-cycling. Verified: SVGs render, both Three.js voxel games run, schema validates, README tables synced (zh+en).
Zhipu AI's GLM-5.2 listed alongside GLM-5.1 as awaiting a contribution (no
runs yet). Added to both README pending lines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #9 added models/kimi-k2-7-code.json but not the README snapshot rows; sync
both zh + en tables (Moonshot AI · Kimi Code · Thinking · 01).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One-shot, unmodified outputs for Qwen3.7-Max (Alibaba) run in Qoder, default
thinking; contributor Nagi-ovo.
- outputs/qwen3-7-max/mythos-craft.html — Three.js voxel world (chunked
terrain, first-person controls, break/place, fantasy structures).
- outputs/qwen3-7-max/pelican-cycling.svg — hand-authored seaside SVG, pelican
on a bicycle with a side-profile study inset.
Filenames normalized to the canonical case ids and the folder renamed to match
the model id (qwen3-7-max). Moved out of Pending in both README registries.
Validates: Data OK, 25 models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two one-shot runs generated in the Vibe web app (Mistral's renamed
Le Chat, Thinking enabled), contributed by Nagi-ovo.
mythos-craft is a fixed build (1 line changed): the world synchronously
built the full RENDER_DISTANCE=4 ring (9x9 = 81 chunks) on load, each
chunk doing an O(voxels x structures) linear scan over mythStructures
for all 16x16x128 voxels, so the main thread froze before the first
frame and the page never opened. RENDER_DISTANCE 4 -> 1 (a 3x3 = 9-chunk
first-screen load) lets it open at 60fps; terrain/visuals/gameplay are
the model's original output, otherwise unmodified. pelican-cycling is
unmodified. Both documented in the run notes; READMEs updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cursor's Composer "Fast" is a serving tier (lower latency, higher cost, same
model intelligence) — not a reasoning-effort level. Drop the Fast/effort framing
entirely: effort is Default, notes just state a plain one-shot Composer 2.5 run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Cursor "Fast" run was registered as label "Composer 2.5 Fast" with effort
"Max" — contradictory, and it baked the speed mode into the model name. Per the
model x harness x effort convention (cf. the Grok Build TUI entry), the label is
just "Composer 2.5" and Fast is the effort, so it sits next to the Cursor · Max
entry as a thinking-budget comparison. Fixed the run notes to match, and synced
both README registry tables (which had also missed the Cursor · High Claude
Fable 5 entry from the prior merge).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the site change — the community leaderboard now unlocks after 3 votes
in any mix of cases, not 3 in each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A blockquote at the top of the contributing section, addressed to coding
agents: states the unit (Agent = model × harness) and the id convention,
names the two current cases with cases.json as the verbatim-prompt source,
and explains how to find open slots from cases.json + models/*.json (and the
Registry "Runs" column) so it stays correct as contributions land.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A missing effort reads as Max by the repo's "effort defaults to max"
convention; this run was the harness default, so set it explicitly to
"Default" and surface it in both README registry rows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions[bot]