# AGENTS.md

Guidance for humans and coding agents contributing to **NAGI BENCH** data
(https://bench.nagi.fun/). Read this before opening a PR. 中文说明见每节。

## What this repo is / 这个仓库是什么

The **public data repo**: community-contributed *one-shot* LLM eval artifacts.
You add a model's raw output for a case; the site renders it verbatim. This repo
holds **data only** — no site code, no build of the page you see.

公开**数据仓库**：社区贡献的「一次生成」LLM 测评产物。你提交某模型在某个 case 上的
原始输出，站点会原样渲染。这里只有**数据**，没有站点代码。

## A contribution is exactly two files / 一次贡献就两个文件

1. **The artifact** — the raw model output, dropped at
   `outputs/<artifactDir>/<case-id>.<ext>` (e.g. a single-file `.html` or `.svg`).
2. **The registration** — `models/<model-id>.json` declaring it.

Do **not** touch anything else. In particular:

- **Do not edit `README.md` / `README.en.md`.** The Registry tables are
  generated from `models/*.json` and synced automatically after merge. Hand
  edits are overwritten. （**不要改 README**，注册表是自动生成并在合并后自动同步的。）
- **Do not edit `cases.json`.** Cases are maintainer-owned.（case 由维护者管理。）

## `models/<model-id>.json` schema

```jsonc
{
  "label": "DeepSeek V4 Flash",   // human-facing model name (required)
  "vendor": "DeepSeek",            // who makes the model (required)
  "harness": "Reasonix",           // the tool/agent/UI you ran it in (required)
  "effort": "Max",                 // thinking budget / effort level (required)
  "artifactDir": "deepseek-v4-flash/reasonix-max", // optional; defaults to <model-id>
  "order": 50,                      // optional display sort
  "runs": {
    "skeleton-watch": {
      "note": {                     // REQUIRED, bilingual, zh first, no emojis
        "zh": "在 Reasonix 中以 DeepSeek V4 Flash（Max）一次生成，未修改。",
        "en": "One-shot by DeepSeek V4 Flash (Max effort) in Reasonix, unmodified."
      },
      "contributor": "your-github-username"
    }
  }
}
```

### Naming rules / 命名规则

- **The filename IS the entry id** — lowercase letters, digits and dashes only
  (`deepseek-v4-flash-reasonix`). It is the **stable voting key** and the default
  `outputs/` folder, so name it after the **model**, not the harness. Bad:
  `reasonix-reasonix-max`. Good: `deepseek-v4-flash-reasonix`.
  （文件名即条目 id，是稳定投票主键——按**模型**命名，别按 harness 命名。）
- `artifactDir` may be one or two dash-case segments
  (`deepseek-v4-flash/reasonix-max`); set it when several entries share a model
  folder. Leave it out to default to the id.

### One entry per (model × harness × effort) / 一个组合一个条目

The same model under a different **harness** or **effort** is a **separate**
`models/*.json` file, not an extra variant inside an existing one. "Codex",
"Cursor", "Claude Code", "AntiGravity" name the **harness**, never the model.
（同一模型换 harness 或 effort = **新建** JSON，不要塞进现有条目。）

### The `note` is required and bilingual / note 必须双语

Every run needs `note.zh` + `note.en` documenting provenance: which harness,
which effort, one-shot or fixed. If you applied any fix, say where the bug was
and what you changed. **No emojis anywhere** (hard rule). zh first.

### Run the harness as it ships / 用 harness 开箱即用的默认形态

Use the harness in its **default, out-of-the-box** configuration: no extra
skills, plugins, MCP servers or custom system prompts beyond what it bundles.
These shift the output as much as swapping the harness does, so loading them
makes the run no longer comparable to others under the same harness. If you used
any, it is a different setup — say so explicitly in `note`, otherwise it does
not belong on the board.
（用 harness 的**默认形态**：不要额外加载 harness 自带以外的 skill / 插件 /
MCP / 自定义系统提示——它们对产出的影响不亚于换 harness，会让这条 run 失去与
同一 harness 下其它条目的可比性。确实用了就在 `note` 里写清楚，否则不该上榜。）

## Before you open the PR / 提交前

```bash
bun scripts/validate-data.ts   # the exact schema check CI runs
```

This enforces: dash-only ids, required bilingual `note`, declared artifacts
exist and are non-empty, no orphan files under `outputs/`. Green here = green CI.

You do **not** need to run `update-registry.ts` — README is synced for you after
merge.（不需要手动跑 update-registry，README 合并后自动同步。）

### Attach a screenshot / 附一张截图

The PR template asks for a **screenshot of the generation** — your harness/IDE
showing the model name + effort while producing this artifact. It is the proof
that the declared `harness` / `model` / `effort` are real. Paste it into the PR
body (GitHub hosts it); **do not commit image files** into the repo.
（PR 里贴一张**生成过程截图**作为「确实用了这套 harness + 模型 + effort」的凭证，
直接贴进 PR 正文，不要把图片提交进仓库。）

## Conventions / 约定

- No emojis in any data, note, or UI-bound text.
- `outputs/` files are served verbatim in a sandboxed iframe. Keep them
  self-contained; external resources should be well-known CDNs only.