agentera 0.0.0 → 3.0.0-dev.0

package/bundle/references/adapters/runtime-feature-parity.md

@@ -0,0 +1,189 @@
+# Runtime feature parity reference
+
+Tracks release-relevant runtime behavior for the portable agentera suite.
+
+This reference distinguishes implemented behavior from host support. A runtime
+may expose an event while agentera still lacks a shipped adapter path for it.
+
+## Summary
+
+| Runtime | Skill loading | Session preload | Artifact validation | Session bookmark |
+|---------|---------------|-----------------|---------------------|------------------|
+| Claude Code | Full: marketplace plugin and native skill paths load `skills/<name>/SKILL.md` | Active via `SessionStart` in `hooks/hooks.json` | Advisory after mutation via `PostToolUse` for `Edit` or `Write` | Active via `Stop` |
+| OpenCode | Full: native `skill` tool loads `.opencode`, `.claude`, and `.agents` skill paths | Deferred for session start: `session.created` is observable, but no model-context injection path is verified. Active for compaction through bounded `experimental.session.compacting` context from `agentera hej --format json`. | Conditional hard gate for reconstructable `write` and `edit` candidates via `tool.execute.before`; `tool.execute.after` remains advisory | Active via generic `event` hook on `session.idle` |
+| Copilot CLI | Full for portable skills through plugin or skill-folder install paths | Active via `sessionStart` | Conditional hard gate via `preToolUse` when `toolArgs` include path plus candidate content or exact replacement evidence | Active via `sessionEnd` |
+| Codex CLI | Full for portable skills through plugin install, `.agents/skills`, and `$skill` invocation | Not wired by the shipped hook config | Advisory `apply_patch` path validation through shipped PreToolUse and PostToolUse hooks; final patch content is not reconstructed | Not wired by the shipped hook config |
+| Cursor IDE | Full for portable skills through local plugin (`~/.cursor/plugins/local/agentera` via `.cursor-plugin/plugin.json`), repo-native surfaces, and upgrade-installed `.cursor/` targets | Active via `sessionStart` env export plus optional `additional_context` digest; plugin-root fallback when `AGENTERA_HOME` env and project walk-up fail (`hooks/cursor_session_start.py`) | Conditional hard gate for reconstructable `Write` and `Edit` candidates via `preToolUse`; verified after live preToolUse Write smoke (2026-05-24) | Active via `sessionEnd` |
+| Cursor Agent CLI | Full when workspace surfaces are installed; degraded when launched outside a Cursor project | Degraded relative to IDE sessionStart env export | Degraded hook parity; follows IDE smoke evidence only | Degraded relative to IDE `sessionEnd` wiring |
+
+## Profilera session corpus
+
+| Runtime | Local session mining | Evidence |
+| ------- | -------------------- | -------- |
+| Claude Code | Yes: `~/.claude/projects/**/*.jsonl` | `scripts/extract_corpus.py` |
+| Codex CLI | Yes: `~/.codex/sessions/**/*.jsonl` | `scripts/extract_corpus.py` |
+| OpenCode | Yes: `opencode.db` SQLite stores | `scripts/extract_corpus.py`, `references/adapters/opencode.md` |
+| Copilot CLI | Yes: `session-store.db` SQLite stores | `scripts/extract_corpus.py` |
+| Cursor IDE | Yes: `~/.cursor/projects/*/agent-transcripts/*/*.jsonl` | `scripts/extract_corpus.py`, `references/adapters/cursor.md` |
+| Cursor Agent CLI | Yes: gap-fill from `~/.config/cursor/chats/<md5(project)>/<session>/store.db` when no IDE JSONL exists | `scripts/extract_corpus.py`, `references/adapters/cursor.md` |
+
+## Bare `hej` routing
+
+| Runtime | Bare text `hej` behavior | Evidence |
+|---------|--------------------------|----------|
+| OpenCode | Deterministic exact-match adapter route through `chat.message`; only a complete lowercase text message `hej` is rewritten to load `agentera` and run the `agentera hej` dashboard path, accepting OpenCode's CLI-added single trailing newline as a transport artifact. | `.opencode/plugins/agentera.js`, `scripts/smoke_opencode_bootstrap.mjs`, OpenCode `packages/plugin/src/index.ts` Hooks interface |
+| Claude Code | Metadata/context only; `UserPromptSubmit` can observe or add context but is not a verified prompt rewrite router. | `skills/agentera/SKILL.md`, marketplace metadata |
+| Copilot CLI | Metadata/context only; skills, prompts, hooks, and plugins expose Agentera but do not guarantee pre-model bare-prompt routing. | `plugin.json`, `.github/plugin/plugin.json`, `.github/hooks` |
+| Codex CLI | Metadata/context only; `$agentera` is explicit and the legacy `$hej` bridge is not implicitly invocable. | `.codex-plugin/plugin.json`, `agents/openai.yaml` |
+| Cursor IDE | Metadata/context only; `beforeSubmitPrompt` is supported by the host but Agentera v1 does not rewrite bare `hej`. | `.cursor-plugin/plugin.json`, `.cursor/agents/*.md` |
+| Cursor Agent CLI | Metadata/context only like Claude, Copilot, and Codex. | `references/adapters/cursor.md`, eval runner metadata |
+
+## Artifact validation
+
+| Runtime | Blocking surface | Implemented gate | Evidence-insufficient paths | Verification surface |
+|---------|------------------|------------------|-----------------------------|----------------------|
+| Claude Code | None in shipped config; validation runs after `Edit` or `Write` | No pre-write hard gate is claimed | Any invalid artifact can already be written before the warning appears | `hooks/hooks.json`, `hooks/validate_artifact.py` |
+| OpenCode | `tool.execute.before` can throw before mutation | Invalid reconstructable artifact `write` and `edit` candidates are blocked | Sparse payloads and `apply_patch` `patchText` without reconstructed full content are allowed | `.opencode/plugins/agentera.js`, `scripts/smoke_opencode_bootstrap.mjs` |
+| Copilot CLI | `preToolUse` returns `permissionDecision: deny` | Invalid reconstructable artifact candidates are denied | Malformed, sparse, or non-reconstructable `toolArgs` are allowed | `.github/hooks/preToolUse.json`, `hooks/validate_artifact.py`, `tests/test_validate_artifact.py` |
+| Codex CLI | `codex_hooks` can run before and after `apply_patch` | No content hard gate is claimed; the copied user hook config parses touched paths and validates existing files; optional plugin-bundled hooks require `[features].plugin_hooks = true` plus `/hooks` review | Add-file targets and final post-patch candidate content are not reconstructed by the adapter | `~/.codex/hooks.json` generated by upgrade, `.codex-plugin/plugin.json` `hooks`, `hooks/codex-plugin-hooks.json`, `hooks/validate_artifact.py`, live apply_patch hook firing smoke |
+| Cursor IDE | `preToolUse` returns `permission: deny` when wired | Invalid reconstructable artifact candidates are denied; live preToolUse Write smoke passed 2026-05-24 | Malformed, sparse, or non-reconstructable tool_input payloads are allowed | `.cursor/hooks.json`, `hooks/cursor_pre_tool_use.py`, `hooks/validate_artifact.py`, `.agentera/smoke-cursor-pretooluse-evidence.txt` |
+| Cursor Agent CLI | None claimed for standalone CLI | No IDE-equivalent hard gate is claimed | CLI may run without project hook wiring | `scripts/eval_skills.py --runtime cursor-agent` |
+
+Docs may claim functional hard-gate parity only for closeable paths that are
+implemented and verified. Today that means OpenCode, Copilot, and Cursor IDE
+reconstructable artifact candidates. Claude Code and Codex remain active validation
+surfaces, but neither shipped configuration blocks every invalid artifact candidate
+before mutation.
+
+## Lifecycle notes
+
+| Runtime | Runtime reason for degraded or blocked capability |
+|---------|---------------------------------------------------|
+| OpenCode preload | The `event` hook observes `session.created`, but no supported adapter path injects text into model context. |
+| OpenCode compaction context | `experimental.session.compacting` appends bounded Agentera state from `agentera hej --format json`; the plugin does not read raw `.agentera` artifacts for compaction. |
+| OpenCode `apply_patch` hard gate | The adapter receives `patchText` without reconstructing full candidate content. It allows that path rather than guessing. |
+| Copilot sparse edits | Copilot `preToolUse` stdin may omit full content or unique old/new replacement evidence. The hook allows those payloads. |
+| Codex preload/bookmarks | `codex_hooks` supports lifecycle events, but Agentera ships only `apply_patch` PreToolUse/PostToolUse wiring for copied user hooks and optional plugin-bundled hooks. |
+| Codex artifact hard gate | The adapter parses patch headers for touched paths, but it does not reconstruct final candidate content for blocking validation. |
+| Codex plugin hook trust | Plugin-bundled hooks require `[features].plugin_hooks = true` and deliberate `/hooks` review; copied `~/.codex/hooks.json` remains the default reliable install path with generated `[hooks.state]` trust hashes. |
+| Cursor cloud agents | Cloud agents are unsupported in v1; repo hooks and managed agents target local IDE sessions only. |
+| Cursor CLI hook parity | `cursor-agent` print mode is eval-covered but hook/session env parity is degraded relative to IDE wiring. |
+| Cursor hard-gate release gate | Live preToolUse Write smoke passed 2026-05-24; release tagging and publication stay blocked pending broader release closeout. |
+
+## Subagent Dispatch
+
+| Runtime | Dispatch surface | Descriptor source | Tool Access | Verification surface |
+|---------|------------------|-------------------|-------------|----------------------|
+| Claude Code | Native Task/subagent surface | Host-managed; no Agentera descriptor files shipped for this phase | None (no descriptors) | RuntimeAdapter registry |
+| OpenCode | `@<capability>` descriptors under `~/.config/opencode/agents` | `.opencode/agents/*.md`, bootstrapped by `.opencode/plugins/agentera.js` | Per-agent `permission` frontmatter | `scripts/smoke_opencode_bootstrap.mjs`, `agentera validate descriptors` |
+| Copilot CLI | User-driven host action such as `/fleet` when available | Host-managed; no Agentera descriptor files shipped for this phase | N/A (no descriptors) | RuntimeAdapter registry |
+| Codex CLI | Native agent descriptors under `~/.codex/agents` or project `.codex/agents` with bounded `[agents]` settings | `skills/agentera/agents/*.toml`, installed by `scripts/setup_codex.py` and `agentera upgrade` | Global sandbox policy (no per-agent) | `agentera validate descriptors`, `tests/test_setup_codex.py`, `tests/test_upgrade_cli.py` |
+| Cursor IDE | Cursor agent picker / @-mention for managed capability descriptors | `.cursor/agents/*.md`, via local plugin or `agentera upgrade --runtime cursor` | Global full access (no per-agent) | `references/adapters/cursor.md`, `scripts/validate_lifecycle_adapters.py`, `tests/test_upgrade_cli.py` |
+| Cursor Agent CLI | Host-managed `cursor-agent -p` print mode | Workspace `.cursor/agents/*.md` when present; no separate CLI descriptor install | Global full access (no per-agent) | `scripts/eval_skills.py --runtime cursor-agent`, `tests/test_eval_skills.py` |
+
+Agentera v2 does not write legacy `[agents.<name>]` Codex config blocks. Capability dispatch must use runtime-native subagent descriptors or host Task surfaces, not unsupported `agentera <capability>` CLI commands.
+
+## Copilot install notes
+
+Recommended marketplace install:
+
+```bash
+copilot plugin marketplace add jgabor/agentera
+copilot plugin install <skill>@agentera
+```
+
+Umbrella install:
+
+```bash
+copilot plugin install jgabor/agentera
+```
+
+The marketplace install path is verified working. Granular installs avoid
+umbrella discovery bug `github/copilot-cli#2390`.
+
+Granular installs provide core `SKILL.md` behavior. App-home tools such as
+doctor, installer, validators, and shared setup helpers require the managed
+Agentera app or a local clone with the shared `scripts/` directory.
+
+Deprecated fallback: `copilot plugin install OWNER/REPO`, Git URLs, and local
+paths still work, but Copilot warns they are deprecated.
+
+## Cursor install notes
+
+**Local plugin (no Marketplace listing required)**
+
+```bash
+git clone https://github.com/jgabor/agentera.git ~/.cursor/plugins/local/agentera
+# or: ln -s /path/to/agentera ~/.cursor/plugins/local/agentera
+```
+
+Restart Cursor or run **Developer: Reload Window**. The plugin root must contain
+`.cursor-plugin/plugin.json`. Agentera is not published to the Cursor Marketplace
+yet.
+
+The plugin loads skills, managed capability agents, and hooks. When you open a
+project that is not an Agentera install root, `sessionStart` exports
+`AGENTERA_HOME` from the plugin checkout (including a plugin-root fallback when
+env and project walk-up do not resolve a managed root).
+
+**Portable skill plus project upgrade**
+
+Install the bundled skill, then install managed project surfaces:
+
+```bash
+npx skills add jgabor/agentera -g -a cursor --skill agentera -y
+uv run scripts/agentera upgrade --runtime cursor --dry-run
+uv run scripts/agentera upgrade --runtime cursor --yes
+uv run scripts/agentera doctor --runtime cursor
+```
+
+Use the plugin path for a user-global install. Use upgrade when you need
+project-committed `.cursor/hooks.json` and `.cursor/agents/` copies. Both paths can
+be combined.
+
+Repo-native dogfood in this repository uses committed `.cursor/hooks.json` and
+`.cursor/agents/*.md`. Other projects install managed surfaces with the upgrade
+commands above.
+
+Cloud agents are unsupported in v1. Conditional hard-gate validation for IDE
+reconstructable Write and Edit candidates is verified after live preToolUse Write
+smoke (2026-05-24); release tagging and publication remain blocked until explicitly
+approved.
+
+Eval coverage for automation uses:
+
+```bash
+uv run scripts/eval_skills.py --runtime cursor-agent --dry-run
+```
+
+## Source of truth
+
+Runtime adapter facts are owned by the RuntimeAdapter registry at
+`references/adapters/runtime-adapter-registry.yaml` and loaded through
+`scripts/runtime_adapter_registry.py`. This reference may describe registry
+claims, but changes to runtime identity, lifecycle events, artifact-validation
+support, subagent dispatch, config targets, diagnostics, or documentation claims must be validated
+against the registry rather than duplicated here as an independent table.
+
+App-home classification is not runtime-specific. `scripts/install_root.py` is
+the shared Module for `AGENTERA_HOME`, the normal user data root, managed app,
+out-of-date app, unknown directories, and diagnostic semantics. Package metadata
+registry work stays outside both the RuntimeAdapter registry and this shared
+classification Module.
+
+| Surface | Path |
+|---------|------|
+| Shared artifact validator | `hooks/validate_artifact.py` |
+| Claude Code hook registry | `hooks/hooks.json` |
+| OpenCode plugin | `.opencode/plugins/agentera.js` |
+| OpenCode agent descriptors | `.opencode/agents/*.md` |
+| Copilot pre-write hook | `.github/hooks/preToolUse.json` |
+| Codex copied user hook config | `~/.codex/hooks.json` generated from `hooks/codex-hooks.json` with a resolved validator command |
+| Codex plugin hook config | `hooks/codex-plugin-hooks.json` via `.codex-plugin/plugin.json` `hooks` |
+| Codex agent descriptors | `skills/agentera/agents/*.toml` |
+| Cursor hook registry | `.cursor/hooks.json` |
+| Cursor agent descriptors | `.cursor/agents/*.md` |
+| Cursor plugin manifest | `.cursor-plugin/plugin.json` |
+| RuntimeAdapter registry | `references/adapters/runtime-adapter-registry.yaml` |
+| RuntimeAdapter registry loader | `scripts/runtime_adapter_registry.py` |
+| Lifecycle metadata validator | `scripts/validate_lifecycle_adapters.py` |

package/bundle/references/analysis/benchmark.md

@@ -0,0 +1,267 @@
+# Agentera benchmarks
+
+This document indexes benchmark surfaces, execution policy, retained outputs,
+and interpretation rules. Use it when running Agentera benchmarks or adding new
+ones.
+
+Scope: maintainer-run benchmarks only. Normal verification still lives in tests,
+contract validators, and runtime smoke checks.
+
+## Authority order
+
+| Authority | Owns |
+| --- | --- |
+| `references/analysis/startup-measurement-contract.yaml` | Startup state-access metric contract, benchmark privacy boundary, retained fields, and storage shape. |
+| `scripts/startup_analysis_contract.py` | Startup analyzer implementation, report generation, aggregate row construction, and benchmark persistence. |
+| `magefile.go` | Manual benchmark command surface and non-interactive approval gate. |
+| `tests/test_startup_analysis_contract.py` | Fixture-backed consent, persistence, privacy, and no-repo-output checks. |
+| `references/analysis/benchmark.md` | Human runbook, interpretation guide, and future benchmark documentation pattern. |
+
+## Benchmark surfaces
+
+| Surface | Command | Purpose | CI policy |
+| --- | --- | --- | --- |
+| Startup state benchmark | `mage bench:startupState` | Measures how often Agentera CLI state reads are followed by raw artifact access during startup/state gathering. | Manual only; forbidden in normal CI. |
+
+The startup benchmark is an optimization signal for Decision 51 and Decision 52.
+It does not implement a startup state envelope or change runtime behavior.
+
+## Startup State Benchmark
+
+Run the benchmark with no extra setup:
+
+```bash
+mage bench:startupState
+```
+
+The default run uses documented runtime-store defaults, writes retained results
+under the default Agentera benchmark directory, and records unavailable stores as
+bounded degradation evidence. No environment variables are required.
+
+To use different runtime history sources, set each runtime label and concrete
+store path explicitly:
+
+```bash
+AGENTERA_BENCH_RUNTIME_STORES="opencode=/absolute/path/to/opencode.db" mage bench:startupState
+```
+
+Customize a run when needed:
+
+```bash
+AGENTERA_BENCH_RUNTIME_STORES="opencode=/absolute/path/to/opencode.db" \
+AGENTERA_BENCH_SALT="$(openssl rand -hex 32)" \
+AGENTERA_BENCH_PROJECT_ROOTS="/absolute/project/root" \
+AGENTERA_BENCH_OUTPUT_DIR="/absolute/benchmark/output" \
+mage bench:startupState
+```
+
+Inputs:
+
+| Input | Meaning |
+| --- | --- |
+| `AGENTERA_BENCH_RUNTIME_STORES` | Optional. Comma-separated `runtime=/absolute/path` overrides. Use this when the documented runtime-store defaults are not the stores you want measured. A runtime label without its path is not enough. |
+| `AGENTERA_BENCH_SALT` | Optional. Local redaction salt for transient `latest-report.*` pseudonyms. If omitted, Mage generates one. `runs.jsonl` does not retain salts or generated salted hashes, so aggregate history does not require a stable or shared salt. |
+| `AGENTERA_BENCH_PROJECT_ROOTS` | Optional. Absolute project roots for corpus extraction. Use this when the current directory is not the project root you want measured. |
+| `AGENTERA_BENCH_OUTPUT_DIR` | Optional. Absolute override for the durable benchmark directory. Use only when the default user-local directory is not desired. |
+
+Supported runtime labels are owned by the analyzer and Mage wrapper. Invalid
+labels or relative paths fail before runtime history is read.
+
+### Runtime Extraction Contract
+
+The extraction matrix is defined in
+`references/analysis/startup-measurement-contract.yaml` under
+`runtime_extraction_contract`. It lists, for each supported runtime, the accepted
+input schema classes, normalized record fields, status mapping, and redaction
+rules.
+
+Status interpretation is intentionally split:
+
+| Outcome | Status / reason | Meaning |
+| --- | --- | --- |
+| Schema divergence | `degraded` / `schema_divergent` | Candidate runtime storage was found, but the adapter hit schema errors. Treat this as extraction failure evidence. |
+| No matching records | `sparse` / `no_matching_records` | Candidate storage was readable and schema-compatible, but no supported records were extracted. Treat this as sparse coverage evidence. |
+| Successful zero-record window | `ok` / `records_extracted` with `record_count: 0` and `error_count: 0` | Extraction succeeded, but the incremental benchmark window has no records after the previous watermark. Treat this as compatible successful behavior. |
+
+Current known runtime caveats are extraction caveats, not CLI behavior evidence:
+`claude-code` is degraded by `schema_divergent` with 4836 candidates, 0 records,
+and 2 errors; `github-copilot` is degraded by `schema_divergent` with 1
+candidate, 0 records, and 1 error; `opencode` currently contributes records;
+`codex` can validly report `ok` with zero records and zero errors for an empty
+incremental window.
+
+Runtime-store runs are incremental by default. The first run for a runtime scope
+measures all records after the v2.3.0 boundary. Later runs for the same
+`runtime_scope` read the previous `benchmark_watermark_at` from `runs.jsonl` and
+measure only records with timestamps strictly after that watermark. This keeps
+new aggregate rows focused on work since the previous successful benchmark run.
+
+To start a fresh series, use a different `AGENTERA_BENCH_OUTPUT_DIR` or archive
+the existing `runs.jsonl`. Late-arriving records with timestamps at or before the
+stored watermark are treated as already covered.
+
+## How To Reduce Raw Startup Reads
+
+Use the startup benchmark as a CLI completeness profiler. The benchmark does not
+measure wall-clock startup speed. It answers whether agents fetch Agentera state
+through the CLI and then still fall back to raw artifact reads, greps, or globs
+during startup/state gathering.
+
+Follow this loop:
+
+1. Run `mage bench:startupState` and open the retained latest report from the
+   `benchmark.directory` path printed on stdout.
+2. Check evidence quality before planning product changes. `total_state_sequences`
+   must be non-zero, important runtime rows should not be degraded, and
+   `confidence_caveats` plus `degradation_reason_counts` must be understood. If
+   the report has zero state-gathering sequences, treat it as no evidence rather
+   than evidence that the CLI is complete.
+3. Rank the post-CLI raw artifact reads. The best next CLI field is usually the
+   highest repeated artifact in `redundant_raw_artifact_access_counts`, followed
+   by `raw_artifact_access_after_cli_counts` when the read is not redundant yet
+   but still happens after a successful CLI state call.
+4. Map each repeated raw read to the CLI state owner. Prefer enriching existing
+   routine commands or the `hej` composite startup result. Do not add Decision 43
+   slash-route aliases as CLI commands.
+5. Make CLI completeness explicit. A startup-capable response should say whether
+   it is complete for the requested capability, whether raw artifact reads are
+   required, what state families are included, what is missing, and which CLI
+   fallback command should be tried before raw file access.
+6. Update guidance and tests so agents trust complete CLI output and use raw
+   reads only as a fallback. Rerun the benchmark on representative sessions and
+   compare the new rates with prior `runs.jsonl` rows.
+
+Use these fields to decide what to change:
+
+| Field | How to use it |
+| --- | --- |
+| `runtime_coverage` | Verify which runtime stores contributed records, which were degraded, and why. Fix extraction or gather more evidence before product work when key stores are degraded. |
+| `total_state_sequences` | Confirm the run actually observed startup/state-gathering sequences. A zero value blocks CLI-completeness conclusions. |
+| `cli_state_command_counts` | Shows which Agentera state commands anchored the measured sequences. These commands are the first candidates for richer structured output. |
+| `raw_artifact_access_after_cli_counts` | Shows which canonical artifacts agents still read after CLI state. These are candidate missing CLI fields or summaries. |
+| `redundant_raw_artifact_access_counts` | Shows post-CLI raw reads that overlap state already covered by the CLI. These are the highest-priority avoidable reads. |
+| `per_capability_state_counts` | Shows whether the gap is broad startup behavior or narrow capability-specific startup context. |
+| `capability_prose_read_counts` | Shows capability prose reads during startup. Use this to decide whether routing/context guidance, not artifact state, is the repeated lookup. |
+| `startup_recommendation` | Records whether the measured evidence supports closing, targeted guidance, or a broader startup state envelope. |
+
+### Token Impact Estimates
+
+Token-impact fields are approximate, privacy-safe aggregate estimates. The
+contract-owned estimator version is `approx_bytes_div_4_v1`: the analyzer may
+observe content byte counts transiently, group them by canonical artifact label,
+and estimate tokens as bytes divided by 4. Retained outputs must not include raw
+paths, transcript text, raw tool arguments, private salts, or generated salted
+hashes.
+
+Retained latest reports and new history rows may include:
+
+| Field | Meaning |
+| --- | --- |
+| `token_estimator_version` | Estimator identity, currently `approx_bytes_div_4_v1`. |
+| `estimated_raw_after_cli_tokens` | Aggregate estimated tokens for raw artifact reads after CLI state calls. |
+| `estimated_redundant_raw_tokens` | Aggregate estimated tokens for raw artifact reads that overlap CLI-covered state. |
+| `estimated_raw_after_cli_tokens_by_artifact` | Canonical-label breakdown such as `PLAN.md`; never raw paths. |
+| `estimated_redundant_raw_tokens_by_artifact` | Canonical-label breakdown for redundant raw reads. |
+| `estimated_tokens_saved_vs_previous` | Previous comparable row's redundant-token estimate minus the current row's estimate, or `null`. |
+| `estimated_tokens_saved_vs_previous_null_reason` | Concrete reason when savings are `null`, such as `previous_missing_token_estimates`. |
+
+Rows are comparable only when contract version, benchmark mode, runtime scope,
+estimator version, and token-field availability match. Otherwise savings stay
+`null` with a contract-listed reason.
+
+Stop conditions are as important as action triggers. If a run has zero
+state-gathering sequences, sparse records, or degraded schema extraction, improve
+benchmark coverage first. If only one capability repeatedly reads one artifact,
+prefer targeted CLI output or guidance over a broad startup envelope. If multiple
+capabilities repeatedly read several redundant artifacts after CLI state, plan a
+capability-ready startup state envelope or equivalent composite output.
+
+## Retention Policy
+
+Default durable storage is `${AGENTERA_HOME}/benchmarks/startup-state/`.
+
+Retained outputs are limited to:
+
+| File | Retention role |
+| --- | --- |
+| `runs.jsonl` | Append-only aggregate benchmark history and previous-run watermark source. |
+| `latest-report.json` | Latest redacted structured report. |
+| `latest-report.md` | Latest redacted human-readable report. |
+
+Temporary corpus files, intermediates, and per-run detailed reports are not
+durable benchmark history. Failed report generation must not append `runs.jsonl`
+or replace previous latest reports.
+
+Aggregate history must not retain raw transcripts, raw corpus files, raw
+intermediates, raw store paths, raw session ids, private salts, or generated
+salted hashes. Benchmark metrics are user-local, uncommitted, unshipped, and not
+part of normal CI.
+
+The command prints the retained benchmark directory in `benchmark.directory`.
+Review the latest result from that directory, or from the default location under
+your resolved `AGENTERA_HOME` (see `agentera doctor --json` when unset):
+
+```bash
+BENCH_DIR="${AGENTERA_HOME}/benchmarks/startup-state"
+
+less "$BENCH_DIR/latest-report.md"
+python3 -m json.tool "$BENCH_DIR/latest-report.json"
+tail -n 5 "$BENCH_DIR/runs.jsonl"
+```
+
+Set `AGENTERA_HOME` first, or resolve the platform app home from
+`agentera doctor --json` / `agentera upgrade --dry-run` when unset.
+The stdout `reports` filenames are analyzer report names from Mage's temporary
+work directory. The durable operator-facing files are the `benchmark` paths:
+`runs.jsonl`, `latest-report.json`, and `latest-report.md`.
+
+## Interpretation
+
+Interpret the benchmark as a startup state-access optimization signal, not as a
+general runtime performance benchmark.
+
+Default runs demonstrate whether new local Agentera startup/state-gathering
+behavior since the previous successful run repeatedly falls back from CLI state
+reads to raw artifact access. Stores that do not exist or cannot be read are
+bounded degradation evidence, not successful behavior evidence.
+
+Watermark fields:
+
+| Field | Meaning |
+| --- | --- |
+| `benchmark_mode` | `since_previous_benchmark` for Mage runs that use retained history as the previous-run boundary. |
+| `benchmark_previous_watermark_at` | The previous successful watermark for the same `runtime_scope`, or `null` on the first run. |
+| `benchmark_window_started_after` | The exclusive lower timestamp bound used for this run. |
+| `benchmark_watermark_at` | The latest record timestamp covered by this run. The next run for the same `runtime_scope` starts after this value. |
+
+Primary rates:
+
+| Latest report field | History row field | Meaning |
+| --- | --- | --- |
+| `raw_after_cli_sequence_rate` | `raw_after_cli_rate` | Share of startup state-gathering sequences where any raw Agentera artifact access follows CLI state. |
+| `redundant_raw_sequence_rate` | `redundant_raw_access_rate` | Share of startup state-gathering sequences where raw access overlaps state already covered by the CLI. |
+
+High rates support follow-up work such as a CLI startup state envelope. Low or
+zero rates can close the measurement loop without implementation if the corpus is
+representative and degradation counts are bounded.
+
+Always review runtime coverage and degradation counts before treating a trend as
+actionable. Missing, locked, sparse, or unreadable stores are bounded degradation
+evidence, not product behavior evidence.
+
+## Adding Benchmarks
+
+New Agentera benchmarks should follow the same shape:
+
+| Requirement | Rule |
+| --- | --- |
+| Contract first | Define metric, privacy boundary, storage, retained fields, and failure behavior before implementation. |
+| No-prerequisite target | Every `mage bench:*` target must run with no environment variables. The default mode should use documented local defaults and report unavailable inputs as bounded degradation evidence. |
+| Manual unless proven safe | Do not add runtime-history or environment-sensitive benchmarks to normal CI. |
+| Explicit local approval | Require concrete paths or resources, not broad consent flags. |
+| User-local outputs | Keep generated benchmark history outside the repository by default. |
+| Bounded retention | Retain aggregate history and latest redacted reports only. |
+| Fixture verification | Test consent refusal, successful synthetic runs, degradation cases, privacy exclusions, and no repository-local outputs. |
+
+If a future benchmark needs different retention or CI behavior, document the
+reason in its contract and update this file in the same change.