prebuttal 0.1.0__tar.gz

prebuttal-0.1.0/CHANGELOG.md

@@ -0,0 +1,183 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+While pre-1.0: **patch** releases (0.1.x) are fixes and small additions; **minor**
+releases (0.2, 0.3, …) are substantial new features.
+
+## [Unreleased]
+
+## [0.1.0] — 2026-10-25
+
+First public release.
+
+### Security
+
+- Prompt-injection hardening — the paper is untrusted input. `agent-claude`
+  now runs `claude -p --tools "" --strict-mcp-config` (no tools, no MCP),
+  `agent-codex` runs `codex exec --sandbox read-only --ignore-user-config
+  --ephemeral` (no user MCP servers, no session files holding the paper),
+  and both start in an empty temporary directory instead of your project. Every prompt fences the
+  paper in `<submission>…</submission>` as data and asks reviewers to report
+  embedded instructions as a weakness. An outdated CLI that rejects the new
+  flags fails fast with an "update the CLI" message. See SECURITY.md.
+- Reports, JSON, share pages and run history no longer record the paper's
+  absolute local path (it leaked user names when shared); local sources are
+  shown relative to the working directory, else `~`, else by name.
+- A `prebuttal.toml` merely found in the current directory can no longer run
+  its `[agents.*]` commands silently (it could, e.g. inside someone else's
+  paper repo): the commands are shown and need a yes; non-interactive runs
+  refuse unless the file is passed with `--config`.
+- `~/.prebuttal` (run history with paper text, `keys.env`) is user-only
+  (0700); it used to be world-readable on shared machines.
+
+### Added
+
+- Release workflow: pushing a `v*` tag builds, tests and publishes to PyPI
+  via Trusted Publishing (no API token stored anywhere).
+- Python 3.14 and 3.15 in the CI matrix and classifiers; a CI job tests the
+  oldest supported dependency versions.
+- Synthetic weak-paper example (`examples/weak-paper/`, fictional, deliberately
+  flawed) with a real 1/10 run, published on the site; real-paper sample pages
+  carry an "unofficial, not endorsed by the authors" banner with CC BY 4.0
+  attribution.
+- GitHub Pages site (`site/`): landing page + curated sample reports (the
+  Mamba and DPO full runs, regenerated from run history by
+  `scripts/build_site.py`), deployed by the standard `pages` workflow —
+  enable once in repo Settings → Pages → Source: GitHub Actions. The web
+  quick-scan (drag-drop PDF + BYOK key, browser-only) is a v0.2+ roadmap
+  item; hosted/public review services are explicitly out of scope.
+- Privacy & data-custody disclaimer (README "Honesty & policy" + SECURITY.md):
+  manuscript text sent to a backend is governed solely by that provider's
+  terms; Prebuttal is an integration layer, not a data custodian. Local-only
+  paths (`ollama`, `relay`) and what the tool itself stores (local disk only)
+  are spelled out.
+- `api-openrouter` backend: one OPENROUTER_API_KEY, hundreds of models,
+  OpenAI-compatible wire format (attribution headers sent; JSON mode not
+  forced since many OpenRouter models reject it). Default is a `:free` model
+  (google/gemma-4-31b-it:free, non-reasoning — reasoning models on the free
+  tier can spend the whole token budget on hidden thinking and return no
+  content; that case now fails fast with an actionable error instead of
+  retrying) so the zero-cost story holds; `:free`
+  ids show as $0 in `--dry-run`. `prebuttal key set openrouter` stores the
+  key. `prebuttal models` scans OpenRouter's public catalog with no key at
+  all — free ids first with live per-1M prices, then curated notables — and
+  the reviewer wizard accepts typing any custom model id. Free-tier
+  resilience: `:free` reviewers dispatch staggered a few seconds apart and
+  saturation-style throttling (HTTP 429) backs off on a long curve
+  (15/30/60s per call) since shared free pools stay closed for minutes.
+- Interactive plan confirmation for auto mode: on a terminal, `prebuttal run`
+  shows the full review plan (personas, backends, models, cost) before calling
+  anything — `[Enter]` runs, `[e]` opens a numbered-menu wizard (persona →
+  backend → live model list from `prebuttal models`), `[n]` aborts.
+  Non-interactive contexts (CI, pipes) print the plan and proceed; `-y` /
+  `--yes` silences; fully-explicit configs never prompt. `--dry-run` plans now
+  show personas too, and backends registered via `register_backend` are
+  eligible for auto assignment.
+- `PREBUTTAL_BACKEND` environment variable: force one backend for every
+  reviewer and the AC (e.g. `relay` for agent-subagent reviewing);
+  precedence `--backend` > `PREBUTTAL_BACKEND` > `prebuttal.toml`.
+- Inside a coding agent (Claude Code, Gemini CLI), `prebuttal run` and
+  `prebuttal backends` print a hint pointing at relay mode.
+- `prebuttal models` — live model discovery per backend: scans the Gemini /
+  OpenAI / Anthropic model-list endpoints and Ollama's installed models
+  (non-chat ids filtered out), shows the claude CLI aliases, marks the current
+  default with `*`, and prints a copy-paste `model = "..."` toml example.
+- `prebuttal key set/status/unset` — quick API-key setup: hidden-input prompt,
+  stored in `~/.prebuttal/keys.env` (user-only, env vars take precedence);
+  `prebuttal backends` and runs pick stored keys up automatically.
+- Type checking: mypy (CI-enforced) and `py.typed` marker for downstream users.
+- Regression fixture: a real pdflatex-compiled PDF locks ingestion behavior
+  (wrapped titles, separate number/title heading lines, kerned decimals).
+- README reference: environment variables, exit codes, skill installation.
+- OpenAI/Anthropic request-shape tests and the Windows `cmd /c` agent-CLI
+  fallback test; calibration parser unit tests.
+- `prebuttal run` — review a paper from a PDF file, a LaTeX project directory,
+  or an arXiv link/id, against the real review form of NeurIPS, ICLR, ICML or
+  ACL Rolling Review (official scales and anchors; forms are plain-text and
+  community-editable).
+- Three independent-context reviewers + an area chair, fanned out in parallel;
+  score aggregation with disagreement (σ, range) and per-review backend/model
+  transparency in every report.
+- Ingestion pipeline: LaTeX parsing with cell-accurate `tabular` extraction
+  (`\multicolumn` expansion, `\label`/`\ref` resolution, appendix numbering,
+  `.bbl` references), PDF parsing with ruled-table detection and
+  caption-block fallback, arXiv download with metadata title.
+- Reviewer pack with budget policy: abstract and all tables are never
+  truncated; optional LLM compression (`compress = true`) for elided sections.
+- Provider layer with startup auto-detection: `agent-claude` (`claude -p`),
+  `agent-codex` (`codex exec`), `api-gemini` / `api-openai` / `api-anthropic`,
+  local `ollama`, the file-protocol `relay` backend (human / host-agent
+  subagents / external systems as responders), and custom `[agents.<name>]`
+  CLI backends.
+- Outputs: venue-format Markdown report, machine-readable `--json`,
+  self-contained `--share` HTML page, `--dry-run` cost/backend matrix.
+- Personas: `senior_kind`, `adversarial`, `detail_obsessive`, `area_chair`,
+  and the `paper_roast` easter egg.
+- `prebuttal.toml` configuration (per-reviewer persona × backend × model ×
+  temperature × seed), `prebuttal init`, `prebuttal backends`.
+- Agent skill (`skills/prebuttal/SKILL.md`) including relay mode.
+- Calibration pipeline (`scripts/calibration/`): predicted vs true OpenReview
+  ratings, scatter plot + MAE.
+- Run history: every run archived under `~/.prebuttal` (reports, JSON, resolved
+  config; best-effort, `--no-save` to opt out) with `prebuttal runs` / `show` /
+  `diff` — score deltas and weakness movement (gone / persists / new) across
+  revisions.
+
+### Changed
+
+- **PDF parsing moved from PyMuPDF (AGPL-3.0) to pdfminer.six + pdfplumber
+  (MIT)**, so installing the MIT-licensed package no longer pulls in a
+  copyleft dependency. Booktabs-style tables (horizontal rules only) get a
+  dedicated detector; cell-accurate numbers, two-column layout and the real
+  LaTeX regression fixture are covered by tests.
+- Wording: scores are no longer described as "calibrated" — the calibration
+  pipeline ships, published results are pending.
+- README installs with plain `pip install prebuttal` (pipx / `uv tool` as
+  the isolated alternative); share pages and the site say the same.
+- Classifier `Development Status :: 3 - Alpha` (was Beta) to match pre-release.
+- Machine-specific scripts moved to the gitignored `scripts/dev/`.
+- Default models brought up to the current mid-tier generation (verified
+  against live model listings, 2026-10): `api-gemini` → **gemini-3.5-flash-lite**
+  (latest flash-lite; cheapest long-context tier for the free-quota path),
+  `api-openai` → **gpt-5.6-terra** (the GPT-5.6 balanced tier),
+  `api-anthropic` → **claude-sonnet-5-5**. Old ids still work when pinned
+  explicitly; pricing table extended accordingly. `prebuttal models` filters
+  more non-chat noise (gemma / lyria / omni / robotics / nano-banana).
+
+### Fixed
+
+- Windows: redirected output (`> log.txt`, pipes, CI) no longer crashes with
+  `UnicodeEncodeError`; stdout/stderr are switched to UTF-8.
+- The sdist now carries `tests/conftest.py`, the PDF fixture and the
+  calibration script, so the test suite runs from the source tarball.
+- `--out` / `--json` in a directory that doesn't exist yet no longer crashes
+  after the reviews ran; parent dirs are created, and the run is archived
+  before the outputs are written so a write error can't lose a paid review.
+- arXiv downloads are streamed and capped at 100 MB.
+- The relay spool directory writes its own `.gitignore` (it holds paper text).
+- Per-reviewer `model` now actually reaches agent-CLI backends
+  (`claude -p --model X`, `codex exec --model X`); previously it was recorded
+  in reports but never passed to the CLI.
+- Auto mode no longer assigns reviewers to `relay` (which waited up to 30 min
+  for a hand-written answer): relay is now explicit-only (`--backend relay`),
+  and with no automatic backend `prebuttal run` exits 3 with guidance.
+- Agent CLIs are launched via the path `prebuttal backends` detected, so
+  npm-installed `claude.cmd` / `codex.cmd` work on Windows.
+- API keys loaded from `~/.prebuttal/keys.env` are no longer passed to agent
+  CLIs (a stored `ANTHROPIC_API_KEY` switched `claude -p` from subscription
+  auth to API billing).
+- LaTeX: `table*` / `figure*` were captured twice (duplicate tables, shifted
+  numbers, wrong `\ref`); `tabular*` leaked its column spec into the first
+  cell; nested `\input` now resolves from the root directory like LaTeX, and
+  unresolvable inputs produce an ingestion warning instead of vanishing.
+- PDF ingestion on real LaTeX output: wrapped titles now join into one title,
+  section numbers separated from their titles merge back (side-by-side or
+  stacked), and table cells are rebuilt from word geometry so kerned decimals
+  survive (`47.9` no longer reads `479 .`; negative deltas keep their sign).
+
+[Unreleased]: https://github.com/GuoYuKai-SaMuEl/prebuttal/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/GuoYuKai-SaMuEl/prebuttal/releases/tag/v0.1.0

prebuttal-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Prebuttal contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

prebuttal-0.1.0/MANIFEST.in

@@ -0,0 +1,6 @@
+# sdist extras: everything the test suite needs, so packagers (conda-forge,
+# distros) can run `pytest` from the source tarball. The wheel is unaffected.
+include CHANGELOG.md SECURITY.md
+include tests/conftest.py
+recursive-include tests/fixtures *.pdf
+include scripts/calibration/calibrate.py

prebuttal-0.1.0/PKG-INFO

@@ -0,0 +1,407 @@
+Metadata-Version: 2.4
+Name: prebuttal
+Version: 0.1.0
+Summary: Pre-submission review stress test for ML papers: three independent AI reviewers + an area chair, real conference review forms, measured disagreement.
+Author: Prebuttal contributors
+Maintainer: GuoYuKai-SaMuEl
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/GuoYuKai-SaMuEl/prebuttal
+Project-URL: Issues, https://github.com/GuoYuKai-SaMuEl/prebuttal/issues
+Project-URL: Changelog, https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/CHANGELOG.md
+Keywords: peer-review,neurips,iclr,icml,acl,reviewer2,llm,arxiv,papers
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Markup :: LaTeX
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pdfplumber>=0.11.0
+Requires-Dist: pdfminer.six>=20231228
+Requires-Dist: httpx>=0.27
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: reportlab>=4; extra == "dev"
+Provides-Extra: calibration
+Requires-Dist: openreview>=1.42; extra == "calibration"
+Requires-Dist: matplotlib>=3.8; extra == "calibration"
+Dynamic: license-file
+
+# Prebuttal
+
+> **Answer Reviewer 2 before Reviewer 2 exists.**
+
+[![CI](https://github.com/GuoYuKai-SaMuEl/prebuttal/actions/workflows/ci.yml/badge.svg)](https://github.com/GuoYuKai-SaMuEl/prebuttal/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/LICENSE)
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/pyproject.toml)
+[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-261230.svg)](https://github.com/astral-sh/ruff)
+
+[![View a sample report](https://img.shields.io/badge/%E2%96%B6_View_sample-report-2ea44f?style=for-the-badge)](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/examples/sample-report.md)
+[![Share page](https://img.shields.io/badge/Share_page-5087d2?style=for-the-badge)](https://guoyukai-samuel.github.io/prebuttal/reports/sample.html)
+[![Website](https://img.shields.io/badge/Website-5087d2?style=for-the-badge&logo=github&logoColor=white)](https://guoyukai-samuel.github.io/prebuttal/)
+
+**Purpose-built for ML conferences: real NeurIPS / ICLR / ICML / ARR review-form
+output, three reviewers running in fully independent contexts (optionally
+cross-model), and a built-in pipeline to measure its scores against real
+OpenReview ratings.**
+
+Prebuttal is a pre-submission **stress test** for your paper: it sends your
+submission through three isolated AI reviewers with different temperaments
+plus an area chair, and hands you a conference-format review report — scores,
+disagreement, weaknesses with citations to your own tables — *before* the real
+Reviewers 1–3 see it. See a synthetic sample:
+[report](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/examples/sample-report.md) ·
+[share page](https://guoyukai-samuel.github.io/prebuttal/reports/sample.html) — or what a deliberately flawed
+(fictional) paper gets: [1/10 report](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/examples/weak-paper-report.md).
+
+---
+
+## Why not just paste the paper into ChatGPT?
+
+That is the main competitor. It loses, four ways:
+
+| Paste into ChatGPT | Prebuttal |
+|---|---|
+| One conversation role-playing three reviewers → **persona collapse** into a single bland compromise opinion | Three reviewers in **fully independent contexts** (separate calls/processes), genuinely different verdicts + measured disagreement |
+| Long papers get skimmed; tables are flattened into prose | **Ingestion pipeline**: cell-by-cell table extraction (LaTeX tabular parsing, PDF table detection, caption fallback) — reviewers can point at *Table 3, column 4* |
+| Output is free-form chat text | **Real conference review forms**: NeurIPS 1–6, ICLR 1–10, ICML 1–10, ARR Soundness/Excitement/Overall — with official scale anchors |
+| Scores are vibes | Fixed seeds, recorded backends/models per reviewer, and a **calibration script** to measure scores against real OpenReview ratings (published results pending) |
+
+## Quickstart
+
+```bash
+pip install prebuttal
+
+prebuttal run paper.pdf --venue iclr
+prebuttal run https://arxiv.org/abs/2510.12345 --venue neurips
+prebuttal run ./latex-src/ --venue acl
+```
+
+Prefer an isolated install for command-line tools? `pipx install prebuttal`
+or `uv tool install prebuttal` work the same way (or `uvx prebuttal ...` to
+run without installing). Requires Python 3.11+.
+
+The first run costs **zero setup** if you already have a subscription CLI or
+Ollama — Prebuttal auto-detects the cheapest path at startup and prints what it
+chose (`prebuttal backends` to inspect):
+
+| Backend | Auth | Notes |
+|---|---|---|
+| `agent-claude` (`claude -p`) | your Claude subscription, **zero API key** | preferred reviewer |
+| `agent-codex` (`codex exec`) | your Codex subscription, **zero API key** | gives cross-model reviews together with claude; runs locked down, ignoring `~/.codex/config.toml` (see [SECURITY.md](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/SECURITY.md)) |
+| `api-gemini` | GEMINI_API_KEY | **free tier** — the cheapest "first key" path |
+| `api-openai` / `api-anthropic` | OPENAI_API_KEY / ANTHROPIC_API_KEY | BYO key |
+| `api-openrouter` | OPENROUTER_API_KEY | one key, hundreds of models — **`:free` ids cost nothing** (default is one); prices shown live in `prebuttal models` |
+| `ollama` | none | local & private — *unpublished papers never leave your machine* (with the default localhost host) |
+| `relay` | none | file protocol: prompts spool to disk and **you, your agent's subagents, or any system** write the answers — zero key, explicit only (`--backend relay`; never picked by auto) |
+
+**Unpublished manuscript?** The text is sent to the backend's provider, and
+what happens to it there is that provider's privacy policy, not ours — see
+[Privacy & data custody](#privacy--data-custody). For local-only review use
+`ollama` or `relay`.
+
+> ⚠️ **Free tiers are usually paid for with your data.** Free API tiers (the
+> Gemini free tier, OpenRouter `:free` models and their upstream providers)
+> may let the provider log, review or train on what you send. They are fine
+> for published papers and trying the tool out — for an unpublished
+> manuscript, check the provider's current terms, use a paid tier, or stay
+> local.
+
+Useful flags:
+
+```bash
+prebuttal run paper.pdf --venue neurips --dry-run   # cost + plan preview, calls nothing
+prebuttal run paper.pdf --venue iclr --json out.json --share   # machine output + share page
+prebuttal run paper.pdf --seed 42                   # reproducible
+prebuttal run paper.pdf -y                          # skip the plan confirmation
+```
+
+In auto mode on a terminal, Prebuttal shows the review plan before calling
+anything — press Enter to run, `e` to re-pick each reviewer's persona, backend
+and model via numbered menus, or `n` to abort. CI and pipes get the plan
+printed and proceed automatically; fully-explicit configs never prompt.
+
+`prebuttal models` lists every backend's actually-available models (live API
+scan + installed Ollama models + claude aliases) with the current default
+marked — pick one and set `model = "..."` per reviewer in prebuttal.toml.
+Built-in defaults are mid-tier models current at release time and change
+between versions — providers retire model ids quickly, so if a default stops
+working, upgrade prebuttal or pin a model yourself.
+
+## The reviewers
+
+| id | Persona | Behavior |
+|---|---|---|
+| R1 | `senior_kind` | senior, constructive; core contribution + main concerns, grounded |
+| R2 | `adversarial` | *Reviewer 2 in the flesh*: motivation, baselines, novelty, overclaiming |
+| R3 | `detail_obsessive` | reads every table cell; statistics, ablations, reproducibility |
+| AC | `area_chair` | meta-review, adjudicates disagreement, estimates the final rating |
+
+Every persona enforces the same protocol: **cite the evidence** (section,
+table, number) or the point doesn't count; use the full scale; JSON-only
+output. There is also an easter-egg roast persona (`persona = "paper_roast"`)
+for when you need the truth *fast*.
+
+Personas live in [`prebuttal/personas/`](https://github.com/GuoYuKai-SaMuEl/prebuttal/tree/main/prebuttal/personas) and venue forms in
+[`prebuttal/venues/forms/`](https://github.com/GuoYuKai-SaMuEl/prebuttal/tree/main/prebuttal/venues/forms) — plain text files, PRs
+welcome (scale wording drifts every year; keep it current).
+
+## Configuration
+
+`prebuttal init` writes a commented `prebuttal.toml`. Per-reviewer backends,
+models and temperatures:
+
+```toml
+[default]
+venue = "iclr"
+seed = 42
+
+[[reviewer]]
+id = "R1"
+persona = "senior_kind"
+backend = "agent-claude"      # claude -p, subscription auth
+temperature = 0.4
+
+[[reviewer]]
+id = "R2"
+persona = "adversarial"
+backend = "agent-codex"       # codex exec, subscription auth
+model = "gpt-5.1"             # optional: forwarded as --model (agent CLIs too)
+temperature = 0.7
+
+[[reviewer]]
+id = "R3"
+persona = "detail_obsessive"
+backend = "api-gemini"        # free tier; default model gemini-3.5-flash-lite
+temperature = 0.5
+
+[ac]
+persona = "area_chair"
+backend = "agent-claude"
+```
+
+## Calibration (honesty, not prophecy)
+
+Scores are meaningless unless measured. The calibration pipeline runs Prebuttal
+on papers that **already have real OpenReview reviews** and plots predicted vs
+true mean ratings:
+
+The script lives in the repository (it is not part of the pip package), so
+run it from a clone:
+
+```bash
+git clone https://github.com/GuoYuKai-SaMuEl/prebuttal && cd prebuttal
+pip install -e ".[calibration]"
+python scripts/calibration/calibrate.py \
+    --venue-id ICLR.cc/2024/Conference --venue iclr -n 20 --out-dir calib_out
+```
+
+Output: scatter plot + MAE + correlation. **No calibration results are
+published yet** — until they are, treat every score as uncalibrated. Once
+published, the claim they support is narrow
+and deliberate: *this tool was measured* — unlike role-play prompts. It is
+**not** a claim that it predicts your score; real reviewers disagree with each
+other by more than this tool's error. Disagreement (σ) is reported with every
+aggregate for exactly that reason.
+
+## Run history & revision diffing
+
+Every run is archived automatically (best-effort, never blocks a review) under
+`~/.prebuttal/` — reports, JSON, and the resolved config (backends, models,
+seeds) for exact reproduction:
+
+```bash
+prebuttal runs                       # list archived runs
+prebuttal show <run-id-prefix>       # re-read a report (--json for the data)
+prebuttal diff <old> <new>           # ratings A→B + weakness movement
+```
+
+`diff` is the revision loop: fix what the reviewers flagged, re-run with the
+same `--seed`, and see which weaknesses went away, persisted, or appeared.
+Basic usage is unchanged — archiving is silent, `--no-save` opts out.
+
+## Skill package
+
+[`skills/prebuttal/SKILL.md`](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/skills/prebuttal/SKILL.md) teaches any agent to
+drive the CLI and interpret the JSON — including relay mode (the reviewers are
+the agent's own subagents) and the `runs`/`diff` revision loop. The skill file
+is explicit about the one rule that matters: the agent must **run the CLI**,
+never role-play the reviewers itself.
+
+When the CLI detects it is running inside a coding agent (currently Claude
+Code and Gemini CLI), `prebuttal run` and `prebuttal backends` print a one-line
+hint pointing at relay mode. An agent environment that should always use its
+own subagents can pin it once with `PREBUTTAL_BACKEND=relay`. Relay is never
+chosen automatically: the CLI has to run in the background while the agent
+answers the requests, or both sides wait on each other.
+
+Install it into your agent's skills directory — for Claude Code:
+
+```bash
+mkdir -p ~/.claude/skills/prebuttal && curl -fsSL -o ~/.claude/skills/prebuttal/SKILL.md \
+  https://raw.githubusercontent.com/GuoYuKai-SaMuEl/prebuttal/main/skills/prebuttal/SKILL.md
+# or, from a clone: cp -r skills/prebuttal ~/.claude/skills/
+```
+
+Other agents (ZCode, Codex, …): copy or symlink `skills/prebuttal/` wherever
+your agent discovers skills (see its docs), or reference the file from your
+`AGENTS.md` / system prompt.
+
+## Reference
+
+### Environment variables
+
+**API keys** — two ways, pick either:
+
+```bash
+prebuttal key set gemini     # quick setup: hidden prompt, stored for next time
+# or the classic way: set GEMINI_API_KEY / OPENAI_API_KEY / ANTHROPIC_API_KEY yourself
+```
+
+Stored keys live in `~/.prebuttal/keys.env` (user-only permissions, like
+`~/.aws/credentials`); real environment variables always take precedence, so
+CI setups are unaffected. `prebuttal key status` shows what's set (masked),
+`prebuttal key unset <name>` removes one.
+
+| Variable | Purpose |
+|---|---|
+| `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) / `OPENROUTER_API_KEY` | enable the matching API backend |
+| `OPENAI_BASE_URL` | point `api-openai` at an OpenAI-compatible proxy |
+| `PREBUTTAL_OPENAI_MODEL` / `PREBUTTAL_ANTHROPIC_MODEL` / `PREBUTTAL_GEMINI_MODEL` / `PREBUTTAL_OPENROUTER_MODEL` | override each backend's default model |
+| `PREBUTTAL_OLLAMA_HOST` (or `OLLAMA_HOST`) | Ollama base URL (default `http://localhost:11434`) |
+| `PREBUTTAL_BACKEND` | force one backend for every reviewer + the AC (e.g. `relay`); `--backend` still wins, `prebuttal.toml` loses |
+| `PREBUTTAL_HOME` | run-history and key storage location (default `~/.prebuttal`) |
+
+API keys are only read from the environment or `keys.env` — never written to
+reports, JSON, or share pages.
+
+### Exit codes
+
+| Code | Meaning |
+|---|---|
+| 0 | success |
+| 2 | bad usage / config error / paper not found / unknown run id |
+| 3 | no review backend available |
+| 4 | partial failure (some reviewers or the AC failed; report still written) |
+| 5 | every reviewer failed |
+| 130 | interrupted (Ctrl+C) |
+
+## Comparison
+
+What Prebuttal provides, next to the common alternatives (by category —
+check any specific tool's own docs for what it supports):
+
+| Approach | Form | Venue forms | Calibration | Cross-model | Tables in-pipeline |
+|---|---|---|---|---|---|
+| **Prebuttal** | CLI + skill | NeurIPS / ICLR / ICML / ARR (editable plain text) | OpenReview script (scatter + MAE; results pending) | claude × codex × API × ollama | cell-by-cell |
+| generic "review my paper" agent skills / prompts | agent skill | varies | rarely | single model | via the host model |
+| multi-agent peer-review research frameworks | research code | study-specific | study-specific | sometimes | rarely |
+| paste into a chatbot | chat | ✗ | ✗ | ✗ | ✗ |
+
+## How it works
+
+```
+PDF / LaTeX dir / arXiv link
+   └─ ingestion: sections, cell-accurate tables, figure captions, refs
+        └─ reviewer pack (budget-aware; tables are never truncated)
+             ├─ R1 senior_kind ──┐ three independent contexts,
+             ├─ R2 adversarial ──┤ parallel fan-out, venue form + JSON schema
+             ├─ R3 detail_obsessive ─┤
+             └─ AC area chair ◄──┘ meta-review + estimated rating + disagreement
+                  └─ Markdown report · --json · --share page
+```
+
+Orchestration lives in the CLI, not the host agent — the same pipeline runs
+from a terminal, from inside any coding agent, or in CI.
+
+## Roadmap & versioning
+
+Versioning policy while pre-1.0: **patch** releases (0.1.1, 0.1.2, …) are
+fixes and small additions; **minor** releases (0.2, 0.3, …) are substantial
+new features; 1.0 only once the interface is battle-tested.
+
+Tentative — priorities shift with usage and feedback. Short-term the focus
+is **hardening the core on real papers**, not adding features:
+
+1. **v0.1 (current, pre-release)** — core: ingestion pipeline, real venue
+   forms, 3 independent reviewers + AC, every backend (agent CLIs / APIs /
+   Ollama / relay), reports + JSON + share page, calibration script
+2. **Now (v0.1.x)** — polish from real-usage findings: table & ingestion
+   edge cases (complex PDF tables), calibration scatter plot + honest
+   per-model calibration notes, provider-reliability UX (free-pool reality,
+   retry transparency), relay UX
+3. **Next (v0.2+)** — more venues, community personas, OpenReview
+   submission integration, and a browser-only web quick-scan (GitHub Pages:
+   drag-drop a PDF + your own OpenRouter/Gemini key — no server, we stay a
+   non-custodian); then the rebuttal workbench (reviews →
+   point-by-point response letter) and "CI for papers" (GitHub Action:
+   \ref/\label checks, citation sanity)
+4. **Eventually** — the calibration script becomes a public benchmark /
+   leaderboard: tools depreciate, benchmarks appreciate.
+
+## Develop
+
+```bash
+git clone https://github.com/GuoYuKai-SaMuEl/prebuttal && cd prebuttal
+python -m venv .venv && .venv/Scripts/pip install -e ".[dev]"   # Windows
+.venv/bin/pip install -e ".[dev]"                               # macOS/Linux
+pytest -q
+```
+
+Contributions welcome — the easiest ones are venue forms, personas, and
+backends; see [CONTRIBUTING.md](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/CONTRIBUTING.md) for the three-step guide.
+Also read [CHANGELOG.md](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/CHANGELOG.md), [SECURITY.md](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/SECURITY.md), and our
+[Code of Conduct](https://github.com/GuoYuKai-SaMuEl/prebuttal/blob/main/CODE_OF_CONDUCT.md).
+
+## Honesty & policy
+
+- Prebuttal **labels every review** with the backend and model that produced it.
+- It reports score **disagreement**, always. High σ is information.
+- It is a stress test, **not** a predictor — the report footer says so, every time.
+- Mind conference policies: with [NeurIPS 2026 running an official AI-assisted
+  reviewing experiment](https://neurips.cc/Conferences/2026/ai-reviewing-experiment),
+  the etiquette is clear — AI to *prepare* your own paper is yours to decide;
+  AI to *review* others' papers must follow each venue's disclosure rules.
+
+### Privacy & data custody
+
+- **Your paper's text goes to whatever backend you configure.** With API or
+  agent-CLI backends the manuscript is transmitted to that provider — and
+  OpenRouter further routes each request to a third-party upstream provider.
+  What happens to the text there (retention, training use, human review) is
+  governed **solely by each provider's terms and privacy policy**, not by
+  Prebuttal.
+- **Prebuttal is an integration layer, not a data custodian.** This project
+  operates no models and runs no servers; we never see your paper. For
+  unpublished results, check the backend provider's data-use terms before
+  sending — provider-side handling of your data is the provider's
+  responsibility, not this project's.
+- **Free tiers deserve extra care**: free API tiers (e.g. Gemini's free
+  tier, OpenRouter `:free` models) may allow logging or training on inputs;
+  paid tiers usually don't. Read the terms before sending unpublished work.
+- **Local-only is a first-class path**: `ollama` never sends text off your
+  machine (as long as `PREBUTTAL_OLLAMA_HOST` / `OLLAMA_HOST` point at
+  localhost — a remote Ollama server receives the paper like any API), and
+  `relay` leaves who answers entirely to you.
+- **What Prebuttal itself stores stays on your disk**: run history (pack
+  text, raw outputs, resolved config) under `~/.prebuttal/runs/` and keys in
+  `~/.prebuttal/keys.env` (user-only). No telemetry. Note reports and share
+  pages contain excerpts of your paper — sharing them publishes those
+  excerpts.
+
+## License & trademarks
+
+Prebuttal is an independent project, not affiliated with or endorsed by
+NeurIPS, ICLR, ICML, ACL/ARR or any model provider. Venue names are used only
+to describe which review form a report follows; the forms are paraphrased
+summaries of the public reviewer guidelines.
+
+MIT