champollion-mcp-server 0.1.0

package/LICENSE

@@ -0,0 +1,133 @@
+# PolyForm Noncommercial License 1.0.0
+
+<https://polyformproject.org/licenses/noncommercial/1.0.0>
+
+## Acceptance
+
+In order to get any license under these terms, you must agree
+to them as both strict obligations and conditions to all
+your licenses.
+
+## Copyright License
+
+The licensor grants you a copyright license for the
+software to do everything you might do with the software
+that would otherwise infringe the licensor's copyright
+in it for any permitted purpose.  However, you may
+only distribute the software according to [Distribution
+License](#distribution-license) and make changes or new works
+based on the software according to [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Distribution License
+
+The licensor grants you an additional copyright license
+to distribute copies of the software.  Your license
+to distribute covers distributing the software with
+changes and new works permitted by [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of
+the software from you also gets a copy of these terms or the
+URL for them above, as well as copies of any plain-text lines
+beginning with `Required Notice:` that the licensor provided
+with the software.  For example:
+
+> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
+
+## Changes and New Works License
+
+The licensor grants you an additional copyright license to
+make changes and new works based on the software for any
+permitted purpose.
+
+## Patent License
+
+The licensor grants you a patent license for the software that
+covers patent claims the licensor can license, or becomes able
+to license, that you would infringe by using the software.
+
+## Noncommercial Purposes
+
+Any noncommercial purpose is a permitted purpose.
+
+## Personal Uses
+
+Personal use for research, experiment, and testing for
+the benefit of public knowledge, personal study, private
+entertainment, hobby projects, amateur pursuits, or religious
+observance, without any anticipated commercial application,
+is use for a permitted purpose.
+
+## Noncommercial Organizations
+
+Use by any charitable organization, educational institution,
+public research organization, public safety or health
+organization, environmental protection organization,
+or government institution is use for a permitted purpose
+regardless of the source of funding or obligations resulting
+from the funding.
+
+## Fair Use
+
+You may have "fair use" rights for the software under the
+law. These terms do not limit them.
+
+## No Other Rights
+
+These terms do not allow you to sublicense or transfer any of
+your licenses to anyone else, or prevent the licensor from
+granting licenses to anyone else.  These terms do not imply
+any other licenses.
+
+## Patent Defense
+
+If you make any written claim that the software infringes or
+contributes to infringement of any patent, your patent license
+for the software granted under these terms ends immediately. If
+your company makes such a claim, your patent license ends
+immediately for work on behalf of your company.
+
+## Violations
+
+The first time you are notified in writing that you have
+violated any of these terms, or done anything with the software
+not covered by your licenses, your licenses can nonetheless
+continue if you come into full compliance with these terms,
+and take practical steps to correct past violations, within
+32 days of receiving notice.  Otherwise, all your licenses
+end immediately.
+
+## No Liability
+
+***As far as the law allows, the software comes as is, without
+any warranty or condition, and the licensor will not be liable
+to you for any damages arising out of these terms or the use
+or nature of the software, under any kind of legal claim.***
+
+## Definitions
+
+The **licensor** is the individual or entity offering these
+terms, and the **software** is the software the licensor makes
+available under these terms.
+
+**You** refers to the individual or entity agreeing to these
+terms.
+
+**Your company** is any legal entity, sole proprietorship,
+or other kind of organization that you work for, plus all
+organizations that have control over, are under the control of,
+or are under common control with that organization.  **Control**
+means ownership of substantially all the assets of an entity,
+or the power to direct its management and policies by vote,
+contract, or otherwise.  Control can be direct or indirect.
+
+**Your licenses** are all the licenses granted to you for the
+software under these terms.
+
+**Use** means anything you do with the software requiring one
+of your licenses.
+
+Required Notice: Copyright Curtis Forbes — Champollion (https://champollion.dev)

package/README.md

@@ -0,0 +1,245 @@
+# champollion-mcp-server
+
+MCP (Model Context Protocol) server for Champollion. Lets AI agents browse the public benchmark queue, search language metadata, and run `mt-eval` benchmarks — all through natural conversation.
+
+> **Champollion** is infrastructure for trustworthy machine translation across every language — source-available and free for noncommercial use (the evaluation harness and shared registries are open source) — the test sets and the map that show who can translate what, how good each method is, and where the gaps are. Public benchmarks on open data rank every method (human and machine); sovereign benchmarks are secret community-owned test sets we never see. The infrastructure is source-available and singly stewarded; the test sets and the methods for a community's language belong to that community — built with communities, never scraped from them. This server is the agent-facing door into that network ([champollion.dev/docs/network](https://champollion.dev/docs/network/)). This server itself is PolyForm Noncommercial 1.0.0 (see [LICENSE](LICENSE)).
+
+## What it does
+
+When connected to an agent (Claude Code, Antigravity, Cursor, etc.), the server exposes tools, resources, and prompts:
+
+### Tools
+
+| Tool | Type | Description |
+|---|---|---|
+| `list_queue` | Read-only | Browse open benchmark items, filter by language/model/budget |
+| `get_queue_item` | Read-only | Get full details for a specific queue item |
+| `estimate_cost` | Read-only | Estimate cost for a set of benchmark runs |
+| `search_languages` | Read-only | Search language cards by name, code, family, or region |
+| `get_project_info` | Read-only | Get a Champollion project overview |
+| `get_results` | Read-only | Read scored runs from the public leaderboard (closes the run → see-impact loop) |
+| `get_run_card` | Read-only | Get one run card (scores + method/config metadata) by id |
+| `get_metric_reliability` | Read-only | Which metric to TRUST for a target language — correlations with WMT human judgments, per language family ([methodology](https://champollion.dev/docs/network/specifications/metric-reliability)) |
+| `get_training_guardrails` | Read-only | How to train an NMT model without fooling yourself — the guardrail rules (group-disjoint splits, dev-fence, leak audits, preregistration, …) extracted from real measured failures, each naming its enforcing tool in `forge/` (nmt-forge) |
+| `translate` | Action | Translate texts through champollion's tested pipeline — engine choice, register conditioning, persistent Translation Memory (repeats are free), deterministic quality gate. Spends API tokens only on cache misses |
+| `run_benchmark` | Action | Start benchmarks via the mt-eval harness — launches in the background and returns a job id immediately |
+| `get_run_status` | Read-only | Poll a benchmark job by id until it completes (the run continues past the client's 60s timeout) |
+
+#### Training tools (nmt-forge)
+
+These wrap the [nmt-forge](https://github.com/gamedaysuits/Champollion) training suite. forge is part of the Champollion monorepo (not on PyPI) — clone the repo and set `CHAMPOLLION_FORGE_DIR` to its `forge/` directory; scoring additionally needs the eval harness (`pip install mt-eval`). Without forge present these tools return an actionable error rather than crashing.
+
+| Tool | Type | Description |
+|---|---|---|
+| `forge_status` | Read-only | Where am I in an nmt-forge project and what do I run next — call first and after every step |
+| `forge_preflight` | Read-only | Will this command refuse? Renders every gate it will hit (✓/✗ with the fix for each ✗) |
+| `forge_discover` | Read-only | What a language HAS — reads the SSOT language card (scripts, analyzers, dictionaries, corpora) |
+| `forge_init` | Action | Scaffold a forge project from a language card: workspace + starter config + NEXT_STEPS brief |
+| `forge_split` | Action | Carve a parallel corpus into GROUP-DISJOINT train/dev/test (shared-source/target pairs stay together) |
+| `forge_leak_audit` | Read-only | Screen a corpus against every registered eval set BEFORE training (exact/near-dupe detection) |
+| `forge_register_eval` | Action | Register an eval file in the workspace with a role: dev (fenced selection) or test (prereg-gated) |
+| `forge_prereg` | Action | Preregister falsifiable predictions for a test/sealed set BEFORE scoring it |
+| `forge_evaluate` | Action | Close the loop: decode the config's battery with the selected checkpoint, score via the mt-eval harness (forge implements zero metrics itself) |
+| `forge_lint` | Read-only | Diagnose a battery manifest: weak registers and the likeliest cause given co-occurring signals |
+| `forge_report` | Read-only | Re-render the plain-language training report (with the Diagnosis section) from a manifest |
+
+### Resources (read-only data)
+
+| Resource | URI | Description |
+|---|---|---|
+| Contributing guide | `champollion://contributing-guide` | CONTRIBUTING.md — how to help with the project |
+| Queue schema | `champollion://queue-schema` | Field definitions for every queue.json item |
+| Network data map | `champollion://network-data` | Which machine artifact answers which question (queue, mesh, registry, coverage) |
+
+### Prompts (conversation starters)
+
+| Prompt | Arguments | Description |
+|---|---|---|
+| `contribute_compute` | `budget?`, `language?` | "I want to help — what would $X buy?" |
+| `compete_for_prize` | `language?` | "I want to build a competitive method — any prizes?" |
+| `explore_language` | `language` | "Tell me about [language] in Champollion" |
+
+## Quick start
+
+**From the published package** (no clone needed):
+
+```bash
+npx champollion-mcp-server    # start on stdio (for agent connection)
+
+```
+
+**From source:**
+
+```bash
+cd mcp-server
+npm install
+npm test          # run unit tests
+npm start         # start on stdio (for agent connection)
+```
+
+## Connect to your agent
+
+### Claude Code / Antigravity
+
+Add to your MCP configuration — published package:
+
+```json
+{
+  "mcpServers": {
+    "champollion": {
+      "command": "npx",
+      "args": ["-y", "champollion-mcp-server"]
+    }
+  }
+}
+```
+
+Or from a local checkout:
+
+```json
+{
+  "mcpServers": {
+    "champollion": {
+      "command": "node",
+      "args": ["/path/to/Champollion/mcp-server/bin/server.js"]
+    }
+  }
+}
+```
+
+### Cursor
+
+Add to `.cursor/mcp.json` (same two options):
+
+```json
+{
+  "mcpServers": {
+    "champollion": {
+      "command": "npx",
+      "args": ["-y", "champollion-mcp-server"]
+    }
+  }
+}
+```
+
+## What a conversation looks like
+
+Once connected, you can talk to your agent naturally:
+
+> **You:** "I want to help with Champollion — can you devote $10 in API credits to it?"
+>
+> **Agent** uses `get_project_info` → learns about the project
+>
+> **Agent** uses `list_queue` with `budget: 10` → sees what's available
+>
+> **Agent:** "I found a few thousand open benchmark items. Your $10 could fund dozens of runs. Any preference on languages?"
+>
+> **You:** "West African languages"
+>
+> **Agent** uses `list_queue` with `language: "african"` → filters results
+>
+> **Agent** uses `estimate_cost` → calculates the plan
+>
+> **Agent:** "I found 18 items for Yoruba, Hausa, Igbo, Zulu, Xhosa, and Luganda. Total: ~$1.64. Ready to run?"
+>
+> **You:** "Go for it"
+>
+> **Agent** uses `run_benchmark` with `budget: 10` → gets a **job id** back immediately (the run continues in the background)
+>
+> **Agent** polls `get_run_status` with that job id until it reports `COMPLETED`, then uses `get_results` to show what was scored
+
+## Testing
+
+```bash
+npm test
+```
+
+Tests use mock data and don't make network calls. To test the server interactively:
+
+```bash
+npx @modelcontextprotocol/inspector node bin/server.js
+```
+
+## Architecture
+
+```
+mcp-server/
+├── bin/server.js              Entry point (stdio transport)
+├── instructions.md            Agent behavioral guide (loaded at connect time)
+├── src/
+│   ├── index.js               Server setup + tool/resource/prompt registration
+│   └── tools/
+│       ├── queue.js            Queue fetch, filter, cost estimation
+│       ├── languages.js        Language card index + search
+│       ├── results.js          Public leaderboard reads (scored run_cards)
+│       ├── reliability.js      Metric-reliability lookups (which metric to trust)
+│       ├── training.js         Training guardrails (get_training_guardrails)
+│       ├── translate.js        Champollion translate pipeline wrapper
+│       └── harness.js          mt-eval CLI wrapper
+├── test/
+│   ├── tools.test.js          Unit tests (node --test) + SSOT shared vectors
+│   ├── harness.test.js        Unit tests for run_benchmark + the async job model
+│   ├── results.test.js        Unit tests for the leaderboard read tools
+│   ├── queue-fetch.test.js    Unit tests for queue fetching/caching
+│   ├── reliability.test.js    Unit tests for metric-reliability lookups
+│   ├── training.test.js       Unit tests for the training-guardrails tool
+│   └── translate.test.js      Unit tests for the translate tool
+├── package.json
+└── README.md
+```
+
+## Protocol version — and the 2026-07-28 stateless spec
+
+**Transport: stdio only.** One server process per agent, launched by the client.
+
+MCP's [2026-07-28 revision](https://blog.modelcontextprotocol.io/posts/2026-07-28/)
+made the protocol **stateless by default** — the largest change since
+authorization. It retires the `initialize`/`initialized` handshake and the
+`Mcp-Session-Id` header, requires `Mcp-Method`/`Mcp-Name` HTTP headers so
+gateways can route without parsing bodies, replaces held-open bidirectional
+streams with Multi Round-Trip Requests, and deprecates Roots, Sampling, Logging
+and the legacy HTTP+SSE transport (twelve-month support window).
+
+**Where this server stands (checked 2026-08-01):**
+
+| | Status |
+|---|---|
+| Deprecated capabilities (Roots / Sampling / Logging) | **None used.** |
+| Legacy HTTP+SSE transport | **Not used** — stdio only. |
+| `Mcp-Session-Id`, header routing, MRTR | **Not applicable** to stdio. |
+| Application state across calls | **Already uses the prescribed pattern** — see below. |
+| SDK support for `2026-07-28` | **Not yet available.** |
+
+The new spec's guidance for cross-call state is to "mint an explicit handle from
+a tool and have the model pass it back as an argument" rather than lean on
+transport sessions. `run_benchmark` already works exactly that way: it returns a
+job id, and the agent passes that id to `get_run_status`. No transport-level
+session is ever involved.
+
+**One assumption to know about.** The job registry in `src/tools/harness.js` is
+in-memory and assumes a single server process for the agent's lifetime. That
+holds for stdio. It would **not** hold behind a stateless HTTP deployment with
+more than one process, where a poll could land on a process that never started
+the job. Anyone adding an HTTP transport must move that registry to shared
+storage first.
+
+**Why we have not upgraded.** The published TypeScript SDK does not speak the
+new revision yet: `@modelcontextprotocol/sdk@1.30.0` is the only dist-tag on
+npm and its `LATEST_PROTOCOL_VERSION` is `2025-11-25`. The dependency floor here
+was raised to `^1.30.0` (from a stale `^1.12.1`, eighteen releases behind) so
+installs resolve current. Re-check when a `2026-07-28`-capable SDK publishes;
+the migration should be small given the table above.
+
+## Data sources
+
+The champollion.dev homepage map is an idealization of this data — agents
+should read the sources, not the picture (the `champollion://network-data`
+resource carries the full endpoint table).
+
+- **Queue**: Fetched from `champollion.dev/queue.json` (tens of MB — it grows with coverage; cached 5 min in memory). Small slice: `champollion.dev/queue-preview.json`. For the live open-item count, call `get_project_info`
+- **Mesh**: `champollion.dev/mesh.json` — the measured/registered pair network behind the homepage map
+- **Corpus registry**: `champollion.dev/registry.json` — every registered eval corpus with license lane, attribution, checksum
+- **Provider coverage**: `shared/catalogue/method-coverage.json` — each provider's published language list, cited + as-of + `tier` (the data behind the map's covered/uncovered split). The map's green has two tiers by exact ISO-639-3 code: bright = a deployed service lists it (Google/Microsoft/DeepL/LibreTranslate); dim = only an open research model lists it (NLLB/OPUS/M2M-100/MADLAD-400 — a model-card code, not a usable service)
+- **Languages**: Loaded from `cli/shared/language-cards/` on startup (falls back to built-in index of 40 languages if the directory isn't accessible)
+- **Results**: Read from the public Supabase leaderboard (`run_cards`) — the same anon read path the champollion.dev leaderboard uses. Scored aggregates and run-card metadata only; per-entry test sentences are never read. Override the project with `CHAMPOLLION_SUPABASE_URL` / `CHAMPOLLION_SUPABASE_ANON_KEY`.
+- **Harness**: Shells out to `mt-eval` CLI (must be installed separately)

package/bin/server.js

@@ -0,0 +1,19 @@
+#!/usr/bin/env node
+/**
+ * Champollion MCP Server — entry point.
+ *
+ * Starts the MCP server on stdio transport. Designed to be launched by
+ * an AI agent's MCP client (Claude Code, Antigravity, Cursor, etc.)
+ * or run directly for testing:
+ *
+ *   node bin/server.js
+ *
+ * The server exposes read-only tools for exploring the Champollion
+ * benchmark queue and language metadata, plus an action tool for
+ * running benchmarks (which requires user confirmation in the agent).
+ */
+
+import { createServer } from '../src/index.js';
+
+const server = await createServer();
+await server.start();

package/instructions.md

@@ -0,0 +1,234 @@
+Here are guidelines for using the Champollion MCP server effectively:
+
+## Orientation
+
+Start with `get_project_info` to understand what Champollion is and how contributions work. This returns a project overview, current queue statistics, and setup instructions.
+
+## Common Workflows
+
+### "I want to help" / Contributing Compute
+
+1. Call `get_project_info` to understand the project
+2. Ask the user about their budget and any language preferences
+3. Call `search_languages` if they mention a language by name — this resolves to ISO codes
+4. Call `estimate_cost` with their budget to show exactly what they'd fund
+5. Present the estimate and **get explicit confirmation** before proceeding
+6. Only then call `run_benchmark` with the agreed parameters (and `confirm: true`). This returns **immediately** with a **job id** — the benchmark runs in the background (see "Running is asynchronous" below)
+7. Poll `get_run_status` with that job id every ~15-30s until it reports `COMPLETED` or `FAILED`
+8. Once it completes, call `get_results` (filtered to the pair/model they ran) so they can see what they scored on the public leaderboard — this closes the loop
+
+#### Running is asynchronous (important)
+
+A real benchmark runs a corpus through a live model and takes **minutes**, which is longer than the default 60-second request timeout most MCP clients (Claude Code, Cursor) enforce. So `run_benchmark` does **not** wait for the run to finish — it launches the run in the background and returns a `job id` right away. Treat that `STARTED` response as success, **not** completion.
+
+- After `run_benchmark` returns, call `get_run_status { "job_id": "run-N" }`. Each poll returns instantly: `RUNNING` (keep polling), `COMPLETED` (output is in the response), `FAILED`, or `ERROR`.
+- Do **not** re-call `run_benchmark` because nothing "came back" — that would start a **second** run and spend tokens twice. The first call already started it; poll `get_run_status` instead.
+- Jobs live in the server process's memory, so a job id is only pollable from the same session. Call `get_run_status` with no `job_id` to list every job started this session.
+
+The estimate you show in step 4 is what executes: `run_benchmark` runs budget/top items in **deterministic top-of-queue order** (it passes `--no-spread`), so the selection matches the `estimate_cost` / `list_queue` preview item-for-item. (One caveat for honesty: `estimate_cost` samples up to ~500 matching items, so for a very large budget that funds more than that, treat its count/total as a lower bound.) A live run additionally skips any (corpus, model, condition) combo already on the leaderboard, so the executed set can be a subset of the preview — never a different, unseen set.
+
+To spend tokens for **scoring/validation without writing to the leaderboard**, pass `publish: false` to `run_benchmark` (budget/top mode). A single `item_id` run is always scored locally and is never auto-published — publish it afterward with `mt-eval publish`, or use budget/top mode to auto-publish.
+
+### "What's been scored?" / Seeing results
+
+The public leaderboard is the read side of the loop: it shows scored runs (composite, chrF++, BLEU, COMET) with trust level and attribution.
+
+1. Call `get_results` — filter by `source_language` / `target_language` / `model`, and `sort` by the metric of interest
+2. Call `get_run_card` with a result's `id` for the full scores + method/config/provenance metadata
+3. An empty result is normal on a fresh board — it means no one has benchmarked that slice yet, which is exactly where contributing compute has the most impact
+
+Results are scored aggregates and run-card metadata only — never raw test sentences (those stay in the license-gated entries table).
+
+### "Which score should I believe?" / Metric trust
+
+Before comparing scores for a low-resource target language, call
+`get_metric_reliability` with the target language (code or family name). It
+returns how well each metric (BLEU, chrF, chrF++, COMET, MetricX) tracked
+human judgment for that language family in the WMT meta-evaluations — for
+some morphologically rich languages BLEU barely correlates with humans while
+COMET does, and for others the learned metric is the unreliable one. If the
+answer is UNMEASURED, say so to the user rather than treating any metric as
+validated. This evidence is research-lane only (upstream data license under
+review) — never cite it in commercial claims.
+
+### "I'm training a model" / Training hygiene
+
+Before you (or the user) split a corpus, generate synthetic data, or report
+training results, call `get_training_guardrails`. It returns the rules
+Champollion extracted from real, measured failures — group-disjoint splits
+(row-level random splits leak on drill-heavy corpora), the dev-fence
+(checkpoint selection must never see the test set), leak audits, coverage
+checklists, per-kind sampling caps, bootstrap CIs on every number, and
+preregistration before test scoring — each with the mistake it kills and
+the enforcing tool (the monorepo `forge/` package, nmt-forge). Two
+non-negotiables to relay verbatim: datasets marked `do_not_train` or
+quarantined in the registry NEVER enter training mixes, and test sets are
+REAL DATA ONLY.
+
+For the human driving you, two public docs teach this end to end — share
+them: the vocabulary, from zero background
+(https://champollion.dev/docs/network/context/mt-training-concepts), and the
+step-by-step, agent-forward walkthrough — discover a language's data →
+synthesize → split safely → train → evaluate honestly → submit
+(https://champollion.dev/docs/network/tutorials/train-your-own-model). The
+`get_training_guardrails` answer also lists these URLs at the end.
+
+The guardrails are not just rules — they are TOOLS. The `forge_*` family
+drives the nmt-forge training suite directly (a repo checkout with `forge/`
+installed is required; each tool says so when it isn't):
+- `forge_status` — is forge available here, and what state is the run in
+- `forge_preflight` — the go/no-go checklist for a planned training run
+- `forge_discover` — what data exists for a language (registry + cards)
+- `forge_init` / `forge_split` — start a fenced run; group-disjoint splits
+- `forge_leak_audit` — prove the split leaks nothing before training
+- `forge_register_eval` / `forge_prereg` — preregister before test scoring
+- `forge_evaluate` — score through the harness (forge implements NO metrics)
+- `forge_lint` / `forge_report` — hygiene checks + the honest run report
+Typical order: status → discover → preflight → init → split → leak_audit →
+prereg → (train outside MCP) → evaluate → report.
+
+### "Translate this" / Using Champollion as your translation engine
+
+When the user needs actual translation (not benchmarking), call `translate`
+instead of improvising your own translation prompt. You get champollion's
+tested pipeline: engine choice, language-card register conditioning, a
+persistent Translation Memory, and a deterministic quality gate — plus a
+per-call report of what was cached, what was validated, and what it cost.
+
+1. Call `translate` with `texts`, `source_language`, `target_language`. The
+   default engine is `llm` (OpenRouter); pass `method` to use a key the user
+   has (openai, anthropic, gemini, deepl, google-translate, …)
+2. Repeated or unchanged texts are served from the Translation Memory at
+   zero token cost — re-calling with overlapping texts is cheap by design,
+   so prefer several small calls over one giant one
+3. A text that fails the quality gate comes back as an explicit FAILED entry
+   with the reason — never silently return it to the user as a translation;
+   retry with a different method/model or surface the failure
+4. For a language the models barely know, check `get_metric_reliability`
+   and `search_languages` first, and consider telling the user about the
+   coaching lane (see the prize workflow) — that is how translation for
+   their language actually gets better
+
+### "What languages need help?"
+
+1. Call `list_queue` with a generous limit to see what's available
+2. Look for languages with the highest ECV (Expected Chain Value) — these have the most impact per dollar
+3. Use `search_languages` to find context: family, speakers, region, endonym
+
+### "Tell me about [language]"
+
+1. Call `search_languages` with the language name or code
+2. Call `list_queue` filtering by that language to see pending benchmarks
+3. Call `estimate_cost` for that language's items to give a cost picture
+
+### "I want to compete for a prize" / The solvable project
+
+Low-resource MT is an open, WINNABLE problem, and this network is built so a
+person who speaks the language + an agent that iterates diligently is a
+serious entry. The Arena supports sponsored prize pools for translation
+breakthroughs — check the [prize spec](https://champollion.dev/docs/network/specifications/prizes)
+for current status (prizes may or may not be active at any given time). The
+path, concretely:
+
+1. **Orient** — `get_project_info`, then `search_languages` for the user's
+   language (family, speakers, what exists). Ask what language(s) they speak.
+2. **Know the measuring stick** — `get_metric_reliability` for the target:
+   which metric actually tracks human judgment for that family. If it says
+   UNMEASURED, the honest framing is "we'll compare relatively on the public
+   dev sets, and native-speaker judgment (yours!) is the real signal."
+3. **Find the baseline to beat** — `get_results` for the pair; an empty
+   board means the FIRST decent method sets the mark. `mt-eval recommend`
+   (or the CLI) shows what published evidence exists.
+4. **Build in the low-compute lane** — coaching data: grammar rules,
+   dictionary entries, style notes injected into the prompt. This is where
+   language knowledge beats GPU budgets. Tutorial:
+   https://champollion.dev/docs/tutorials/build-a-plugin — iterate: edit
+   coaching → `run_benchmark` on the pair (dev corpus) → `get_results` →
+   repeat. Each iteration costs cents, and the harness caches everything it
+   has already translated (re-runs only pay for what changed).
+5. **Check it's real** — the significance spec
+   (https://champollion.dev/docs/network/specifications/significance): a
+   +0.5 chrF++ bump on 400 sentences is probably noise; the run cards carry
+   confidence intervals. Never claim a win the CIs don't support.
+6. **Anti-gaming architecture** (explain this — it's why a win means
+   something): final evaluation runs against **secret community-owned test
+   corpora** (nobody trains on what nobody sees); methods must be
+   **reproducible** (re-run by the organizer node, scores must match);
+   **native-speaker validation** outranks every automatic metric.
+7. Approach options beyond coaching: FST morphological validation (hardest
+   to hallucinate), dictionary-augmented generation, fine-tuning (needs
+   compute), hybrids (LLM → validate → retry). Method interface spec:
+   https://champollion.dev/docs/network/specifications/methods
+8. **Submitting to a secret set** (when a sovereign contest exists). After the
+   participant clears the public qualifier and has a published hypotheses run,
+   they propose their method against the organizer's sealed corpus via the CLI
+   (this is a human-authorized, custodian-gated flow — there is no MCP tool for
+   it, by design). Two lanes, and the CLI/organizer pick by the submission:
+   - **Lane A — declarative model (preferred for standard NMT):**
+     `mt-eval contest submit-model` — submit safetensors weights + a
+     declarative tokenizer + a config for a whitelisted architecture. No
+     Dockerfile, no code; the organizer runs the weights in its own trusted
+     engine, so the submission is validated code-free. Tell users to export
+     weights as `safetensors` (never a pickle `.bin`/`.pt`).
+   - **Lane B — runnable bundle (for code methods):**
+     `mt-eval contest submit-method` — a Dockerfile + entrypoint the organizer
+     runs in a `--network=none` sandbox.
+   Full runbook (both lanes, what's live vs. in development):
+   https://champollion.dev/docs/network/sovereignty/run-a-sovereign-contest
+
+If there's no active prize for their language, the loop above still stands —
+runs publish to the public leaderboard with attribution, and a standing
+better-than-baseline method is exactly what gets a language ready for a
+sponsored pool.
+
+### "Can I evaluate a local / self-hosted model?" (no API key)
+
+Yes — the harness runs open neural-MT models on the user's own hardware, no
+cloud key needed: **NLLB-200**, **OPUS-MT** (Helsinki-NLP), **MADLAD-400**, or
+any converted **CTranslate2** model. This is where low-resource coverage the
+cloud engines don't serve actually lives.
+
+This is a **harness-CLI capability, not an MCP tool** — `run_benchmark` (and its
+`buildRunArgv`) drive the public queue with *remote* model slugs, so there is no
+MCP verb that loads local weights. Direct the user to run it themselves:
+
+```bash
+pip install 'mt-eval[local-models]'      # or 'mt-eval[ctranslate2]'
+mt-eval run --method local-model \
+  --model facebook/nllb-200-distilled-600M \
+  --dataset flores-eng-fra
+```
+
+`--model` takes a Hugging Face id, a local `from_pretrained()` directory, or a
+CTranslate2 model directory (auto-detected). Language codes come from the
+language card — a language the model doesn't serve (NLLB has no Plains Cree,
+`crk`) fails honestly rather than emitting a guessed code. Results score and
+publish like any other run. Full how-to:
+https://champollion.dev/docs/network/getting-started/contributing-compute
+
+## Important Rules
+
+- **Never call `run_benchmark` without user confirmation.** This spends real money (API credits).
+- **`run_benchmark` is asynchronous — poll, don't re-run.** A confirmed run returns a `job id` immediately and keeps running in the background. Poll `get_run_status` with that id until it reports `COMPLETED`/`FAILED`. Never call `run_benchmark` again just because the first call returned before the run finished — that double-spends.
+- **Always call `estimate_cost` before suggesting a benchmark run.** Show the user what they'll spend.
+- **Trust the queue ranking.** Items are ordered by ECV — the expected improvement in translation quality per dollar. Don't re-sort or second-guess the ranking.
+- **Budget mode skips, it doesn't stop.** If an item exceeds the remaining budget, the system skips it and continues to cheaper items further down the queue. This is by design — it maximizes what gets done within a budget.
+- **Items without cost estimates are skipped in budget mode.** Unknown cost ≠ free.
+- **The preview is what runs.** `run_benchmark` executes in deterministic top-of-queue order (`--no-spread`), so what `estimate_cost`/`list_queue` showed is what spends. Don't assume a different set ran.
+- **Publishing writes to a public, production leaderboard.** Budget/top runs auto-publish each result by default. For a scoring/validation run with no leaderboard write, pass `publish: false`.
+- **Use `translate` for translation; don't improvise.** The tool's Translation Memory makes repeats free and its quality gate rejects garbage deterministically — a hand-rolled prompt has neither. Never present a gate-FAILED text as a translation.
+- **Translation ≠ evidence.** `translate` output is production translation; quality claims about methods and models come only from benchmark runs and the leaderboard.
+
+## Data Sources
+
+The champollion.dev homepage map is an idealization — read the data, not
+the picture. The full endpoint table lives in the
+`champollion://network-data` resource.
+
+- **Queue**: served LIVE from the public database (read-only `queue_top` RPC) by default, with https://champollion.dev/queue.json as the fallback when the DB is unreachable (cached 5 minutes); small preview at https://champollion.dev/queue-preview.json
+- **Mesh**: https://champollion.dev/mesh.json — the measured/registered pair network behind the map
+- **Corpus registry**: https://champollion.dev/registry.json — every registered eval corpus with license lane + attribution
+- **Provider coverage**: `shared/catalogue/method-coverage.json` (repo) — each provider's published language list, cited + as-of + `tier`. The map's green is two tiers by exact ISO-639-3 code: bright = a deployed service lists it (Google/Microsoft/DeepL/LibreTranslate); dim = only an open research model lists it (NLLB/OPUS/M2M-100/MADLAD-400 — a model-card code, not a usable service). "Covered" is a published-list claim, never a quality claim.
+- **Languages**: Loaded from local language card JSON files (7,900+ languages)
+- **Scored runs**: public `run_cards` PostgREST (read-only RLS; aggregates only) — prefer the `get_results` / `get_run_card` tools
+- **Queue ranking**: map-value survey ordering (default) + ECV — see https://champollion.dev/docs/network/specifications/queue-construction