diff --git a/AGENTS.md b/AGENTS.md index 3bec96c..41f1ea0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -130,26 +130,25 @@ there is no per-project dependency setup: file** that the normal viewer serves verbatim. The anonymity rule above still applies: no model / vendor / harness identity anywhere in the code. -## Agent cases — a single JS policy file +## Agent cases — maintainer-run only Some cases (marked `kind: "agent"` in `cases.json`, currently `turf-war`) ask -the model to write a game-playing agent instead of a page. For these your -artifact is **one plain JavaScript file** at -`outputs//.js`: +the model to write a game-playing agent that battles other models' agents, +ranked by real wins instead of votes. These cases are **not open for artifact +PRs**, by design: -- Feed the case prompt to the model **verbatim, one-shot** — no test runs, no - iteration, no extra strategy hints of any kind. Extra context beyond the - canonical prompt invalidates the run. -- Save the model's output **as-is**. The file must define `createAgent` at top - level (the interface is fully specified in the prompt). -- The site runs the file inside an instrumented sandbox with a deterministic - compute budget; `Math.random`, `Date`, network and imports are blocked. CI - smoke-runs one match against a built-in baseline to check API compliance and - budget — it does not judge strength. -- Battles are refereed server-side and ranked by real wins (Bradley-Terry), so - there is nothing to screenshot: the CI check is the proof for these cases. -- The anonymity rule applies as usual: no model / vendor / harness identity in - the code or comments. +- Policy files are deliberately kept off the public repo — if submissions were + public, later entrants could read earlier policies and prompt their model to + counter them, which breaks the one-shot fairness of the case. +- All runs are produced by the maintainer, feeding every model the identical + canonical prompt one-shot (no test runs, no iteration, no extra hints). + Battles are refereed server-side inside an instrumented sandbox with a + deterministic compute budget; the browser only replays action logs. +- The registration in `models/.json` (a `runs.` entry with + the usual bilingual provenance note) is the only public trace; there is no + `outputs/` artifact for these cases. +- Want a model or harness combo included? Open an issue and the maintainer + will run it. ## Review guidelines diff --git a/README.en.md b/README.en.md index 423c0d1..a2ab725 100644 --- a/README.en.md +++ b/README.en.md @@ -102,7 +102,7 @@ bun scripts/update-registry.ts # optional: preview the Registry tables locally; > **If you are an AI agent (Claude Code / Codex / Cursor, etc.), this part is for you:** > > - **The unit you contribute is one Agent = a model × the harness you run in.** The `` (lowercase letters / digits / dashes) encodes that pairing — e.g. `gpt-5-5-pro`, `claude-fable-5-cc` (cc = Claude Code). The same model in a different harness is a different Agent with a different id. The `` is the vote and leaderboard identity; do not rename it just to reorganize files. -> - **There are three cases right now**, defined in [`cases.json`](./cases.json): `mythos-craft` (HTML, a playable voxel world), `pelican-cycling` (SVG, a pelican cycling by the sea), and `skeleton-watch` (HTML, a skeleton mechanical watch). The prompt lives in `cases.json` — **use it verbatim, do not paraphrase**. +> - **There are five cases right now**, defined in [`cases.json`](./cases.json): `mythos-craft` (HTML, a playable voxel world), `pelican-cycling` (SVG, a pelican cycling by the sea), `skeleton-watch` (HTML, a skeleton mechanical watch), `cs-dust2` (React, a multi-file project), and `turf-war` (JS agents battling head-to-head, ranked by objective wins). The prompt lives in `cases.json` — **use it verbatim, do not paraphrase**. Note: `turf-war` does **not** accept artifact PRs — to keep earlier policies unscoutable, every run is executed and submitted personally by the maintainer Nagi ([@Nagi-ovo](https://github.com/Nagi-ovo)); open an issue to request a model. > - **To find what's missing**: read `cases.json` (all cases) and `models/*.json` (what each Agent has run) — a `` absent from an Agent's json is an open slot; any row in the script-generated Registry table above whose "Runs" count is below the number of cases is an Agent missing a case; a brand-new Agent can fill them. > - **Then follow the two-file flow below**, write a bilingual `note` (how the artifact was produced — one-shot or fixed), and run `bun scripts/validate-data.ts` until it passes before opening a PR. diff --git a/README.md b/README.md index 0609cbd..98be4d9 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ bun scripts/update-registry.ts # 可选：本地预览 Registry 表；合并到 > **如果你是一个 AI Agent（Claude Code / Codex / Cursor 等），这一段是给你看的：** > > - **你贡献的单位是一个 Agent = 模型 × 你所在的 Harness。** ``（小写字母 / 数字 / 连字符）就编码这个组合，如 `gpt-5-5-pro`、`claude-fable-5-cc`（cc = Claude Code）。同一模型换个 Harness 就是另一个 Agent、另一个 id。`` 是投票与榜单身份，不要为了整理文件夹而改它。 -> - **当前有三个案例**，定义在 [`cases.json`](./cases.json)：`mythos-craft`（HTML，可玩体素世界）、`pelican-cycling`（SVG，海边骑车的鹈鹕）与 `skeleton-watch`（HTML，镂空机械表）。提示词以 `cases.json` 为准，**逐字使用、不要改写**。 +> - **当前有五个案例**，定义在 [`cases.json`](./cases.json)：`mythos-craft`（HTML，可玩体素世界）、`pelican-cycling`（SVG，海边骑车的鹈鹕）、`skeleton-watch`（HTML，镂空机械表）、`cs-dust2`（React，多文件项目）与 `turf-war`（JS Agent 同场对战，客观胜负排名）。提示词以 `cases.json` 为准，**逐字使用、不要改写**。注意：`turf-war` **不收产物 PR**——为防止后来者读取并针对先前策略，该案例的全部运行由维护者 Nagi（[@Nagi-ovo](https://github.com/Nagi-ovo)）个人执行提交，想加模型请开 issue。 > - **想知道还缺哪些**：读 `cases.json`（全部案例）与 `models/*.json`（每个 Agent 跑过的案例）——某个 Agent 的 json 里缺某个 `` 就是一个空位；上面由脚本生成的 Registry 表里，「产出」数小于案例总数的行，就是还缺案例的 Agent；全新 Agent 则三个案例都可补。 > - **然后照下面两文件流程做**，写好双语 `note`（说明产出怎么来的、是否一次生成、是否有修复），提交前用 `bun scripts/validate-data.ts` 自检通过再发 PR。