Add case 05 Turf War (kind: agent) + agent-case contribution guide
One-shot JS policy agents battle head-to-head on a 40x40 grid; ranked by objective wins (Bradley-Terry), refereed server-side. Inspired by Jiayi Weng's Learning Beyond Gradients.
This commit is contained in:
1 parent
1142b7dfde
commit
50fb9978dc
2 files changed
+49
-1
No files matched your search
@@ -130,6 +130,27 @@ there is no per-project dependency setup:
|
||||
file** that the normal viewer serves verbatim. The anonymity rule above still
|
||||
applies: no model / vendor / harness identity anywhere in the code.
|
||||
|
||||
## Agent cases — a single JS policy file
|
||||
|
||||
Some cases (marked `kind: "agent"` in `cases.json`, currently `turf-war`) ask
|
||||
the model to write a game-playing agent instead of a page. For these your
|
||||
artifact is **one plain JavaScript file** at
|
||||
`outputs/<artifactDir>/<case-id>.js`:
|
||||
|
||||
- Feed the case prompt to the model **verbatim, one-shot** — no test runs, no
|
||||
iteration, no extra strategy hints of any kind. Extra context beyond the
|
||||
canonical prompt invalidates the run.
|
||||
- Save the model's output **as-is**. The file must define `createAgent` at top
|
||||
level (the interface is fully specified in the prompt).
|
||||
- The site runs the file inside an instrumented sandbox with a deterministic
|
||||
compute budget; `Math.random`, `Date`, network and imports are blocked. CI
|
||||
smoke-runs one match against a built-in baseline to check API compliance and
|
||||
budget — it does not judge strength.
|
||||
- Battles are refereed server-side and ranked by real wins (Bradley-Terry), so
|
||||
there is nothing to screenshot: the CI check is the proof for these cases.
|
||||
- The anonymity rule applies as usual: no model / vendor / harness identity in
|
||||
the code or comments.
|
||||
|
||||
## Review guidelines
|
||||
|
||||
NAGI BENCH stores raw benchmark artifacts. Reviewers, including Codex Review,
|
||||
|
||||
Reference in new issue
Block a user