loki-mode 9.8.0 → 9.12.0

package/docs/ARCHITECTURE-OVERVIEW.md

@@ -67,8 +67,10 @@ What you get:
   the changed files (secret-leak axis) - v8.0.0, `SKILL.md`.
 
 Provider-agnostic (stable since v5.0.0): Claude (Tier 1, full), Cline (Tier 2),
-Codex and Aider (Tier 3, degraded, sequential). Gemini deprecated v7.5.18. See
-`skills/providers.md`.
+Codex and Aider (Tier 3, degraded, sequential), opencode (sequential). With
+`LOKI_PROVIDER` unset, auto-detection picks the first installed provider in that
+order (`providers/loader.sh:191`); an explicit choice always wins. Gemini was
+removed as a provider in v7.5.18. See `skills/providers.md`.
 
 ---
 
@@ -168,7 +170,7 @@ in `autonomy/completion-council.sh`; proof generation in
 sequenceDiagram
     participant Runner as run_autonomous (run.sh)
     participant Prompt as build_prompt
-    participant Provider as Provider (Claude/Codex/Cline/Aider)
+    participant Provider as Provider (Claude/Cline/Codex/Aider/opencode)
     participant Verify as Gates + council
     participant Evidence as Evidence gate
     participant Council as completion council

package/docs/CAPABILITY-BACKLOG.md

@@ -0,0 +1,53 @@
+# Capability Backlog
+
+Generated 2026-08-03. Companion to `docs/COMPETITIVE-SCORECARD.md`.
+
+Gaps are referenced by their number in that file's section 4 ("What we do NOT
+know") and are not restated here. Ordering is by pillars unblocked per unit of
+free work, where "free" means no paid model calls and no third-party installs.
+
+Pillar numbers refer to the seven-pillar system in the scorecard's "How to read
+this document" section.
+
+## Priority order
+
+| # | Item | Closes | Pillars | Free? | Measurement that closes it |
+|---|---|---|---|---|---|
+| 1 | Verification tax instrumentation | Gap 13 | 5, 7 | Yes | SHIPPED 2026-08-03, partial. `tools/verification-tax.py` + 14 tests. First reading: 19.0s wall, outcome changed 1/1. Remaining: token cost, and hit rate over real history |
+| 2 | Confidence calibration audit | Gap 14 | 3, 5 | Yes | Reliability diagram of self-reported confidence against observed outcome, over existing run history |
+| 3 | Provider parity measurement | Pillar 7 cell | 7 | No (paid) | Identical task across `--provider` values, scored on completion quality not flag presence |
+| 4 | T0-T1 tier harness | Gaps 2, 11 | 1, 6 | No (paid) | Completion quality and intervention rate on single-file and multi-file tasks |
+| 5 | T2-T4 architecture tiers | Gaps 2, 12 | 2, 3 | No (paid) | Same dimensions on migration, performance, and distributed-system tasks |
+| 6 | T5 research tier | Gap 11 | 4 | No (paid) | Novelty and usefulness of a research result, needs a domain judge |
+
+Items 3-6 require spend and are therefore not startable under the current
+constraint. They are listed so the ordering is visible, not because they are
+actionable now.
+
+## Why item 1 is the slice
+
+It is the only candidate that clears all four constraints: free, measurement
+rather than new verification machinery, testable with one runnable check, and it
+closes a numbered gap.
+
+It is also a prerequisite rather than an end in itself. The mandated trajectory
+for verification is toward near-zero overhead, applied selectively where
+calibrated confidence is low. Neither "shrink the tax" nor "bypass it
+selectively" can be validated as an improvement without a baseline for what the
+tax currently costs and how often it earns its keep. Item 2 depends on item 1's
+outcome-change data for the same reason.
+
+Explicit non-goal: this must not become another gate, another artifact format, or
+another blocking check. If the implementation starts adding verification
+machinery rather than measuring the machinery that exists, it has failed its own
+filter and should be stopped.
+
+## Deliberately not on this list
+
+- Any new gate, reviewer, council, or evidence format. The founder constraint is
+  to avoid building more verification machinery unless it directly unlocks
+  broader capability. Items 1 and 2 measure existing machinery so it can be made
+  cheaper or skipped; neither adds any.
+- Marketing-facing comparison content. The scorecard is an evidence ledger and
+  two thirds of it is UNKNOWN; it is not ready to be a positioning asset and
+  should not be turned into one.

package/docs/COMPARISON.md

@@ -212,7 +212,7 @@
 | **npm/wk** | 6.1K | 21.4K | N/A | N/A | N/A | N/A | N/A |
 | **Agents** | 41 roles in 8 domains | 11 agents | Fresh per task | 108 agents | Swarm-based | 32 agents | N/A |
 | **Skills** | Progressive disclosure | 6 slash commands | N/A | 129 skills | N/A | 35 skills | Memory focus |
-| **Multi-Provider** | Yes (Claude/Codex/Gemini) | 3 CLIs (separate) | No | No | No | No | No |
+| **Multi-Provider** | Yes (Claude/Cline/Codex/Aider/opencode) | 3 CLIs (separate) | No | No | No | No | No |
 | **Memory System** | 3-tier (episodic/semantic/procedural) | None | N/A | N/A | Hybrid | N/A | SQLite+FTS5 |
 | **Quality Gates** | 8 gates + Completion Council | User verify only | Two-Stage Review | N/A | Consensus | Tiered | N/A |
 | **Context Mgmt** | Standard | Fresh per task (core innovation) | Fresh per task | N/A | N/A | N/A | Progressive |
@@ -237,7 +237,7 @@ These are patterns from competing projects that are **practically and scientific
 
 | Strength | Details | Competitors Lacking This |
 |----------|---------|-------------------------|
-| **Multi-Provider Support** | Only skill supporting Claude, Codex, and Gemini with graceful degradation | All 8 competitors are Claude-only |
+| **Multi-Provider Support** | Supports Claude, Cline, Codex, Aider, and opencode with graceful degradation | The 8 competitors compared here are Claude-only |
 | **RARV Cycle** | Reason-Act-Reflect-Verify is more rigorous than Plan-Execute | Most use simple Plan-Execute |
 | **8-Gate Quality System** | Static analysis + test suite (pass/fail) + 3 blind reviewers with severity blocking + devil's advocate + mock-integrity + test-mutation + documentation coverage + Magic Modules debate (backward-compat is a conditional healing auditor) + Phase 1 closure | Superpowers has 2-stage, others have less |
 | **Constitutional AI Integration** | Principles-based self-critique from Anthropic research | None have this |

package/docs/COMPETITIVE-ANALYSIS.md

@@ -24,7 +24,7 @@ GSD is the closest competitor -- a context engineering system that spawns fresh
 | Memory system | Episodic/semantic/procedural + vector search | None | Loki |
 | Context management | Standard | Fresh subagent contexts per task (core innovation) | GSD |
 | Time to value | Learn architecture, understand CLI flags | `npx get-shit-done-cc` and go | GSD |
-| Multi-provider | Orchestrates across Claude/Codex/Gemini with degraded mode | Installs separately to each | Loki |
+| Multi-provider | Orchestrates across Claude/Cline/Codex/Aider/opencode with degraded mode | Installs separately to each | Loki |
 | Real-world proof | No public case studies | Testimonials, 1.1K forks, 75 PRs | GSD |
 
 **Key takeaway:** GSD proved the market wants sharp blades, not Swiss Army knives. Loki Mode's Completion Council and quality gates are genuinely unique IP that no competitor has.

package/docs/COMPETITIVE-SCORECARD.md

@@ -0,0 +1,422 @@
+# Competitive Scorecard
+
+Generated 2026-08-03 against Loki Mode v9.8.1.
+
+Every cell in this document is one of three things and nothing else:
+
+- `[measured]` - reproduced on this machine with a command in the "How to reproduce"
+  section, whose output is pasted here or in that section.
+- `[sourced: URL]` - taken from a public page that was actually fetched on
+  2026-08-03. Vendor marketing claims are labelled `vendor-claimed` and are not
+  treated as fact.
+- `UNKNOWN` - not measured and not sourced. This is a legitimate and common answer
+  in this document.
+
+There is deliberately no fourth category in the axis table. Unmarked guesses do
+not appear.
+
+One exception, quarantined: section 2b is a **planned** benchmark design. Nothing
+in it has been run, none of it is evidence, and it is excluded from the cell
+counts below. It is separated precisely so it can never be mistaken for a result.
+
+**Cell census** (the 48 axis-table cells in section 2, being 6 axes x 8 products):
+**10 measured, 5 sourced, 33 UNKNOWN**. Two thirds of this document is an
+admission of ignorance, which is the honest state of the evidence.
+
+This file is distinct from `docs/COMPETITIVE-ANALYSIS.md`, which is a narrative
+positioning document. This one is an evidence ledger and makes no claim it cannot
+back with a pasted command output or a fetched URL. Where the two disagree, this
+file is the one with the receipts; no attempt is made here to reconcile them.
+
+## How to read this document (scope limits)
+
+Two limits on what these measurements mean. Both narrow the document; neither
+softens a single cell.
+
+**Verification is one sub-axis, not the thesis.** The evidence-document
+capability measured below is adaptive scaffolding: useful at today's model
+capability, expected to become progressively cheaper, more selective, and ideally
+invisible as frontier models get better. The correct trajectory for it is toward
+near-zero overhead, applied where calibrated confidence is low and bypassed where
+it is high - not toward a permanent tollbooth on every build. Pillar 7 sharpens
+this: an orchestration layer is supposed to benefit automatically from model
+improvement rather than be displaced by it, so a verification tax that stays
+fixed while models improve converts from an asset into precisely the drag that
+displaces us. A scorecard row that looks like a win today is therefore a snapshot
+of a gap we intend to make matter less. Do not read the observability row as the
+product's reason to exist, and do not let a competitor closing that gap read as a
+strategic loss. In the structure below, verification appears as one sub-axis of
+observability and as one scored dimension of the proposed benchmark - nowhere as
+the organising claim.
+
+**These six axes do not measure the durable ambition.** The stated ambition is
+frontier software-and-research execution at arbitrary complexity, framed as a
+compound capability system of six pillars. The table below barely touches any of
+them:
+
+| Pillar | Covered by this table? |
+|---|---|
+| 1. Outcome intelligence - ambiguous intent to excellent products/research, not merely passing tests | No. Nothing here measures outcome quality. |
+| 2. Architecture depth - massive repos, distributed systems, migrations, performance, security, UX, data, infra, long-horizon evolution | No. Deepest probe was a 5-file diff. |
+| 3. Adaptive autonomy - dynamic planning/replanning, hierarchical agents, memory, simulation, tool/model routing, recovery, learning from outcomes | No. Not probed at all. |
+| 4. Scientific research - literature/web/code/data synthesis, hypothesis generation, experiment design, reproducibility, statistical rigor | No. Not probed at all. |
+| 5. Economics - best quality-adjusted completion per dollar and wall-clock, budget-aware routing, minimal coordination/verification tax | Barely. One prompt-size ablation, no quality-adjusted or wall-clock figure. |
+| 6. Developer experience - fastest intent to trustworthy shipped value, excellent observability/control, easy onboarding, graceful human collaboration | Partially, and only as affordances. The onboarding and observability rows list which commands exist; no one was timed, and "graceful human collaboration" is untested. |
+| 7. Extensibility/independence - model-agnostic orchestration that benefits automatically from exponential model improvement rather than being displaced by it | Partially, and the flag overstates it. `loki start --provider claude\|codex\|cline\|aider` exists `[measured]` from `loki --help`, but whether quality holds across providers is unmeasured. Concretely: this machine has `codex-cli 0.146.0` `[measured]`, while the repo still classifies Codex as Tier 3 on v0.98-era flag assumptions (CLAUDE.md). A provider flag that exists against stale assumptions is a capability claim, not an independence result - the same conflation this document polices elsewhere. |
+
+The axes in this document are process-and-plumbing axes, and passing all of them
+could coexist with mediocre outcomes. In particular the "quality of completion"
+row measures whether a verdict was produced by an independent runner - it does
+not measure whether the resulting software was any good. Note also that pillar 5
+names the verification tax as a cost to minimise, which is the same direction as
+the scaffolding point above: the observability row measures a capability whose
+own overhead is a tax the roadmap intends to shrink. Treat any gap in this table
+as necessary, not sufficient, evidence. The benchmarks that would test the real
+ambition do not exist yet; the gaps are enumerated in section 4.
+
+## 1. Honest summary
+
+Thirty-three of the 48 cells in this table are UNKNOWN, and that is the headline.
+Three of the six axes (reliability, time-to-first-success, quality of completion)
+are UNKNOWN for every product including Loki Mode, because we have run no
+head-to-head build benchmark and any number there would be invented. None of the
+seven capability pillars that constitute the actual ambition - outcome
+intelligence, architecture depth, adaptive autonomy, scientific research,
+economics, developer experience, extensibility - is measured here at all; pillars
+3 and 4 are entirely unprobed. Every difference this document did find is a
+difference in **CLI affordances**, not in outcomes: what a tool's `--help`
+advertises is not what it delivers. Within that narrow frame, the one capability
+no competitor matched is a persisted, schema-versioned evidence document with a
+freshness re-check (`loki verify --json` emitted a `schema_version: "1.0"` doc
+with a per-gate `runner` field and exit code 2; `--check-fresh` then correctly
+returned STALE once the tree moved). That is one sub-axis of observability, it is
+adaptive scaffolding rather than an identity, and its value is expected to decline
+as models improve - see "How to read this document" above. It is emphatically not
+a claim to be the only tool with review or machine-readable output; the opposite
+is true and competitors are close. Claude Code ships `ultrareview --json`, a
+cloud-hosted multi-agent review printing a raw `bugs.json` payload; Codex ships a
+first-class `codex review` subcommand plus `exec --json` and `--output-schema`;
+cursor-agent has `--output-format json|stream-json`; OpenCode has `run --format
+json`, `export`, and a working `stats`. Structured output is table stakes in 2026.
+On cost we can cite exactly one in-repo measurement (the `LOKI_SIMPLE=1` prompt
+ablation, -78% prompt size, ~1562 tokens/iteration) whose own README entry says
+its effect on build speed and quality is not yet measured - a prompt-size number,
+nothing about value delivered. Devin, Replit and Emergent were **not tested**:
+Devin does publish a local CLI at cli.devin.ai which we deliberately did not
+install, and Replit and Emergent appear to be web-only; all three of their rows
+are sourced or UNKNOWN, never measured.
+
+## 2. Axis table
+
+Column key: the first five columns are CLIs installed and version-verified on this
+machine. The last three ship no CLI we tested; their cells are sourced or UNKNOWN.
+
+Verified versions `[measured]`, via `<cmd> --version`:
+`claude 2.1.220 (Claude Code)` / `codex-cli 0.146.0` / `cursor-agent
+2026.05.24-dda726e` / `opencode 1.18.9` / `Loki Mode v9.8.1`. (`aider 0.86.2` is
+installed and probed; see footnote 1 rather than a column.)
+
+### reliability
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| UNKNOWN. No head-to-head reliability benchmark was run. What is `[measured]` is only that a deterministic gate run emits a per-gate pass/fail with a named runner (`tests: fail (jest)`, `static_analysis: pass (syntax)`, `dependency_audit: fail`) - the presence of gates is not a reliability measurement. | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN. Vendor references "SWE 1.7, our latest model" with no supporting benchmark data on the pricing page `[sourced: https://devin.ai/pricing]` | UNKNOWN. Vendor states the agent "is probabilistic - meaning it may occasionally make mistakes" `[sourced: https://replit.com/pricing]`; this is a disclaimer, not a metric | UNKNOWN `[sourced: https://app.emergent.sh/]` - no reliability or benchmark claim on the landing page |
+
+### time-to-first-success
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| UNKNOWN. Not measured. No build was run for this document. | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN | UNKNOWN |
+
+This entire row is UNKNOWN by construction: measuring it requires running paid
+model builds on a common task across eight products, which was out of scope and
+would have cost money. Do not fill this row from vendor demos.
+
+### quality of completion
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| UNKNOWN as an outcome. `[measured]`: a verdict is computed only from executed gate rows, and `--explain` prints "pass = independent evidence (a real runner/scanner exit), never a self-assessment". Also `[measured]`: `llm_review.status = "skipped"` with reason "deterministic-only MVP ... blind council are deferred to Phase 2" - so the LLM review half of our own quality story is **not shipped** in `loki verify` today. | UNKNOWN as an outcome. `[measured]` from `claude ultrareview --help`: a cloud-hosted multi-agent code review exists with `--json` emitting a raw `bugs.json` payload. Not executed (paid). | UNKNOWN as an outcome. `[measured]` from `codex review --help`: a dedicated non-interactive review subcommand exists with `--uncommitted`, `--base <BRANCH>`, `--commit <SHA>`. Not executed (paid). | UNKNOWN | UNKNOWN | UNKNOWN. `[sourced: https://docs.devin.ai/]` - docs emphasise making "tasks easy to verify - e.g. checking that CI passes", described as manual verification, not a structured output format | UNKNOWN | UNKNOWN |
+
+Note the discipline here: the existence of gates, councils or review subcommands
+is a capability, not a quality measurement. Nobody in this row has a measured
+quality-of-completion number, us included.
+
+### cost
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| Partially `[measured]`, in-repo: `LOKI_SIMPLE=1` strips the coaching half of the system prompt, measured at **-78% prompt size, ~1562 tokens per iteration**, on both the bash and Bun routes (`README.md:757`). The same README entry states plainly that "Whether it changes build speed or quality is NOT yet measured". So this is a prompt-size reduction, not a demonstrated cost-per-delivered-outcome win. Total cost per build: UNKNOWN. `loki report cost` exists `[measured]` from `loki help report`. | UNKNOWN (BYO model spend, not measured) | UNKNOWN (BYO model spend, not measured) | UNKNOWN | `[measured]` `opencode stats` runs locally and prints an OVERVIEW plus COST & TOKENS panel (this machine: 142 sessions, 3,441 messages, Avg Tokens/Session 1.4M, Total Cost $0.00 - the $0.00 reflects local config, not a claim about OpenCode's price) | `[sourced: https://devin.ai/pricing]` Free $0; Pro $20/mo; Max $200/mo; Team $80/mo base + $40/mo per seat; Enterprise custom. Cost per message varies by model and task complexity; overage at API rates | `[sourced: https://replit.com/pricing]` Starter free with daily agent credits; Core $20/mo annually with $25 monthly credits, up to 2 parallel agents; Pro $95/mo annually with $100 monthly credits, up to 10 parallel agents; Enterprise custom | UNKNOWN `[sourced: https://app.emergent.sh/]` - a Pricing link exists in nav but no figures on the landing page |
+
+Cross-product cost comparison is deliberately **not** computed. The units are not
+commensurable (subscription credits vs BYO token spend) and we have no
+cost-per-completed-task figure for any product including our own.
+
+### observability
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| `[measured]` `loki verify --json` writes a persisted `evidence.json` with top-level keys `schema_version, verdict, exit_code, subject, produced_by, deterministic_gates, llm_review, findings, suppressed, scope`; verdict BLOCKED, exit_code 2. `--explain` prints a per-gate table with a RUNNER and REPRO column. `--check-fresh` re-reads a prior evidence doc and returned `VERDICT: STALE (worktree modified since 1c80c85ff059)`. `loki status --json`, `loki doctor --json`, `loki report cost/metrics/export`, and a dashboard server also exist `[measured]` from `loki --help`. | `[measured]` `--output-format text\|json\|stream-json` (with `--print`), `--json-schema` for structured output, `--include-partial-messages`; `ultrareview --json` emits raw `bugs.json`. No persisted on-disk evidence document or freshness re-check was found in `--help`. | `[measured]` `codex exec --json` prints events as JSONL; `--output-schema <FILE>` takes a JSON Schema for the final response shape; `-o/--output-last-message <FILE>` writes the last message to a file. `codex review --help` lists **no** `--json` and no evidence-file flag (grep for json/output flags on that subcommand returned 0 matches). `codex doctor` exists. | `[measured]` `--output-format text\|json\|stream-json` with `--print`, plus `--stream-partial-output`. No evidence artifact or freshness check found in `--help`. | `[measured]` `opencode run --format json` (raw JSON events), `opencode export [sessionID]` exports session data as JSON, `opencode stats` prints usage/cost, `opencode serve` headless server. Session export is not a verdict artifact. | UNKNOWN. `[sourced: https://docs.devin.ai/]` - a Devin API is referenced; docs list no machine-readable verdict or evidence artifact | UNKNOWN | UNKNOWN |
+
+This is the row where the honest differentiator lives, and it is narrow. Four of
+five installed competitors emit structured JSON. What was not found in any
+competitor's help output is a **persisted verdict document plus a staleness
+re-check** (`--check-fresh`). "Not found in help output" is weaker than "does not
+exist" - see section 4.
+
+### onboarding
+
+| Loki Mode | Claude Code | Codex CLI | cursor-agent | OpenCode | Devin | Replit | Emergent |
+|---|---|---|---|---|---|---|---|
+| `[measured]` from `loki --help`: a dedicated first-run path is printed at the top of help - `loki welcome`, `loki doctor`, `loki quickstart`, `loki start ./prd.md`, plus `loki next` ("Run the right next step for you"). Whether this reduces real time-to-first-build: UNKNOWN, not user-tested. | `[measured]` `claude doctor` subcommand; interactive session is the default with no subcommand. | `[measured]` `codex doctor` ("Diagnose local Codex installation, config, auth, and runtime health"); interactive by default; `codex login`. | `[measured]` interactive by default; `install-shell-integration` subcommand. | `[measured]` TUI is the default command; `opencode providers/auth` for credentials; `opencode upgrade`. | `[sourced: https://docs.devin.ai/]` a local CLI ("Devin for Terminal") installs via `curl -fsSL https://cli.devin.ai/install.sh \| bash`, with `/handoff` to escalate to cloud Devin. **We did not install or run it.** | `[sourced: https://replit.com/pricing]` web platform; no local CLI mentioned on the pricing page | `[sourced: https://app.emergent.sh/]` web-based, "Build production-ready apps through conversation"; no CLI mentioned on the landing page |
+
+Footnote 1 - aider 0.86.2 is installed and was probed but is not given a column.
+`[measured]`: grepping `aider --help` for verify/json/report/audit/attest returned
+only `--verify-ssl/--no-verify-ssl` and `--git-commit-verify`, both unrelated to
+output verification. No verdict artifact, no JSON output surface found.
+
+## 2b. Proposed benchmark: complexity tiers (PLANNED - nothing here has been run)
+
+Everything in this section is a **planned experiment**, not a result. No tier has
+been executed, no score exists, and no cell below should ever be cited as
+evidence. It is here so the gaps in section 4 have a concrete shape.
+
+The six axes in section 2 measure plumbing. To measure the seven pillars, tasks
+have to span real complexity and be scored on outcome, not on gate output.
+
+**Tiers** (each a real task, run identically across tools):
+
+| Tier | Task shape | Which pillars it exercises |
+|---|---|---|
+| T0 | Single-file bug fix with a failing test | 6 |
+| T1 | Multi-file feature in an unfamiliar mid-size repo | 1, 6 |
+| T2 | Cross-cutting migration (framework or API version) over a large repo | 2, 3 |
+| T3 | Performance or security investigation with no known answer | 2, 3 |
+| T4 | Distributed-system change spanning services, with a rollback path | 2, 3 |
+| T5 | Open research task: literature plus code plus data synthesis, hypothesis, experiment, reproducible result | 4 |
+
+**Scored dimensions** (per tier, per tool). Note verification is one row here,
+not the frame:
+
+| Dimension | Definition | Why it is not a gate metric |
+|---|---|---|
+| Completion quality | Does the result actually solve the stated problem, judged against the intent | Passing tests is the floor, not the score |
+| Intervention rate | Human corrections needed per task | Directly measures autonomy (pillar 3) |
+| Wall-clock time | Intent to shipped value | Pillar 5, 6 |
+| Cost | Total spend, quality-adjusted | Pillar 5; unadjusted cost is meaningless |
+| Recovery | Behaviour after an induced failure | Pillar 3; needs deliberate fault injection |
+| Maintainability | Quality of what is left behind | Long-horizon evolution, pillar 2 |
+| Novelty/usefulness | For T5, is the finding actually new and useful | Pillar 4; needs a domain judge |
+| User value | Would the requester ship it | The only dimension that ends in a human |
+| Verification tax | Time and tokens spent verifying, and how often it changed an outcome | Pillar 5. A *cost* row. Should trend down over time |
+
+The verification-tax row is the one that keeps scaffolding honest: it measures
+overhead and the hit rate that justifies it. If overhead rises while the hit rate
+falls, the scaffolding is being over-applied and should be selectively bypassed.
+
+### 2b-FROZEN. Design seal for T1 and T2 (2026-08-03)
+
+T1 and T2 are FROZEN as designs. Nothing has been run, no money has been
+spent, and this seal exists so that stays true until a founder decision says
+otherwise.
+
+**Held-out discipline, and why it is written down BEFORE any run.** The value
+of a benchmark is destroyed by the ordinary act of debugging against it. Once a
+task has been attempted, the harness tuned, and the task attempted again, the
+score measures fit to that task rather than capability. So:
+
+1. **The T1 and T2 task instances are not selected yet, and must not be
+   selected by whoever tunes the harness.** Selection and tuning by the same
+   party is how a held-out set stops being held out.
+2. **One scored attempt per tool per task.** A retry after seeing the result is
+   a different experiment and must be reported as such, never averaged in.
+3. **The scoring rubric is frozen before the first run.** A dimension added
+   after seeing results is a dimension chosen because of them.
+4. **Task text is never committed to this repo.** A task in the repo is a task
+   in the training and context of the thing being measured.
+
+**What would make a T1/T2 result quotable**, stated now so it cannot be
+loosened later to fit an outcome:
+
+- n >= 3 independent instances per tier, scored blind to which tool produced
+  the artifact where the dimension allows it.
+- Every competitor run under the same instance, same day, same rubric.
+- Cost and wall clock read from each tool's own artifacts, never from a
+  supervising process.
+- Any dimension that could not be scored reads UNKNOWN. A partial rubric is
+  reported as partial rather than averaged over the dimensions that happened to
+  work.
+
+**Explicitly NOT frozen and NOT designed:** T3, T4 and T5. T5 in particular
+needs a domain judge for novelty, and no such judge is identified. Listing them
+in the tier table is a sketch, not a design.
+
+**Cost, stated because it is the reason this is frozen rather than run.** Each
+tier instance is a real build across eight tools. At n>=3 that is dozens of paid
+runs, and the T2 shape (cross-cutting migration over a large repo) is the
+expensive end. No run happens without an explicit founder spend decision.
+
+**Status separation.** Required reading before quoting anything from this file:
+
+| Category | What is in it |
+|---|---|
+| **Measured capability** | Only section 2, and only cells tagged `[measured]` or `[sourced]`. These are CLI affordances and one prompt-size ablation. No outcome quality anywhere. |
+| **Planned experiments** | All of section 2b. Tiers T0-T5 and every scored dimension. Zero runs to date. |
+| **Aspirational** | The seven pillars as a whole. They are the stated ambition and are not claims of current capability. Pillars 3 and 4 are entirely unprobed. |
+
+## 3. How to reproduce
+
+Every command below was actually run on 2026-08-03 in the repo working tree at
+`/Users/lokesh/git/lokimode-anthropic/.claude/worktrees/pre-push-scoped-pytest`.
+None of them makes a paid model call.
+
+Versions:
+
+```bash
+for c in claude codex cursor-agent opencode aider loki; do
+  printf "=== %s ===\n" "$c"; command -v "$c"; "$c" --version 2>&1 | head -3
+done
+```
+
+Capability probes (help text only - no model calls, no spend):
+
+```bash
+loki --help
+loki help verify
+loki help report
+claude --help | grep -iE "output-format|json"
+claude ultrareview --help
+codex --help
+codex review --help
+codex exec --help | grep -iE "json|output-schema|output"
+cursor-agent --help
+opencode --help
+opencode run --help
+aider --help | grep -iE "verify|json|report|audit|attest"
+```
+
+The evidence-document claim, which is the only differentiator asserted here:
+
+```bash
+# Deterministic-only: loki verify's help states there is NO LLM code review in
+# this slice, so this run is free.
+loki verify HEAD~1 --out /tmp/loki-verify-sc2 --explain
+
+# Persisted, schema-versioned artifact:
+python3 -c "import json;d=json.load(open('/tmp/loki-verify-sc2/evidence.json'));print(list(d.keys()))"
+# -> ['schema_version','verdict','exit_code','subject','produced_by',
+#     'deterministic_gates','llm_review','findings','suppressed','scope']
+
+# Freshness re-check against the current tree:
+loki verify --check-fresh --out /tmp/loki-verify-sc2
+# -> VERDICT: STALE (worktree modified since 1c80c85ff059) -- re-run loki verify
+```
+
+Observed `--explain` output from that run, verbatim:
+
+```
+  GATE               STATUS       RUNNER         REPRO  EVIDENCE
+  ----               ------       ------         -----  --------
+  build              skipped      -              true   no detectable build command
+  tests              fail         jest           true   tests failed (rc=1)
+  static_analysis    pass         syntax         true   4 file(s) checked, no syntax errors
+  nomock             skipped      -              true   no scannable UI/data-render files in diff
+  secret_scan        pass         regex-fallback true   no secrets matched
+  dependency_audit   fail         npm-audit      true   0 critical, 4 high CVEs
+```
+
+SCOPE NOTE on that audit row, because both numbers are real and they mean
+very different things to a release decision:
+
+```
+npm audit                -> 7 vulnerabilities (1 low, 2 moderate, 4 high)
+npm audit --production   -> 2 moderate, 0 high
+
+The four highs (brace-expansion, form-data, js-yaml, ws) do NOT appear in the
+production dependency tree; they arrive through dev tooling. The shipped
+package's own exposure is 2 moderate, both reached via
+@modelcontextprotocol/sdk -> @hono/node-server, one of which is a
+Windows-only path traversal in serve-static.
+
+Both are stated rather than one being chosen. "4 high CVEs" unqualified
+overstates what someone installing the package is exposed to; "2 moderate"
+alone understates what our own gate sees and blocks on.
+  spec_drift         skipped      loki-spec      true   no spec lock (.loki/spec/spec.lock)
+
+VERDICT: BLOCKED
+```
+
+Note this is a BLOCKED verdict on our own tree, reported unchanged. A scorecard
+whose own tool passes cleanly on the first try is a scorecard to distrust.
+
+Cost citation (in-repo, cite - do not re-derive): `README.md:757`, the
+`LOKI_SIMPLE` row. Re-run the ablation on your own workload with
+`benchmarks/run-prompt-ablation.sh` before drawing any conclusion from it.
+
+Cleanup after reproducing:
+
+```bash
+rm -rf /tmp/loki-verify-scorecard /tmp/loki-verify-sc2 /tmp/loki-verify-scorecard.err
+```
+
+## 4. What we do NOT know
+
+This section is mandatory and is not empty.
+
+1. **We did not execute any competitor's review or build command.** Every
+   competitor capability above is read from `--help` output only. Running
+   `claude ultrareview`, `codex review`, `cursor-agent -p`, or an OpenCode build
+   would each make a paid model call, which was out of scope. "Not listed in
+   `--help`" is not proof of absence - a capability may exist undocumented, in a
+   config file, or behind a subcommand we did not enumerate.
+2. **No head-to-head build benchmark exists.** Reliability, time-to-first-success
+   and quality-of-completion are UNKNOWN for all eight products. We have never
+   run the same task through even two of these tools and compared outcomes.
+3. **We have no cost-per-completed-task figure for any product, including Loki.**
+   The `LOKI_SIMPLE` number is a prompt-size measurement only, and its effect on
+   speed and quality is explicitly unmeasured per its own README entry.
+4. **Devin was not tested despite having a local CLI.** `docs.devin.ai` documents
+   `curl -fsSL https://cli.devin.ai/install.sh | bash`. We chose not to install
+   third-party software, so every Devin cell is sourced or UNKNOWN. An earlier
+   read of the pricing page suggested "no local CLI"; the docs page corrected
+   that. Treat any Devin claim here as provisional.
+5. **Replit and Emergent were assessed from marketing pages only** (a pricing page
+   and a landing page). Neither page discusses verification. Absence of a claim on
+   a marketing page is not evidence the capability is absent from the product.
+6. **Our own LLM review stage is not shipped in `loki verify`.** The evidence doc
+   records `llm_review.status = "skipped"`, reason "deterministic-only MVP ...
+   blind council are deferred to Phase 2". The 8-gate build loop is a separate
+   path from `loki verify` and was not exercised for this document.
+7. **The freshness/evidence differentiator is help-text-derived for competitors.**
+   We verified our own `--check-fresh` by running it. We did not exhaustively
+   search competitor filesystems, config schemas, or APIs for an equivalent.
+8. **Onboarding is unmeasured.** No user was timed through first build on any
+   product. The onboarding row records which affordances exist, not whether they
+   work.
+9. **`opencode stats` showed Total Cost $0.00 on this machine**, which reflects
+   local provider configuration and tells us nothing about OpenCode's economics.
+10. **Vendor benchmark claims were not independently verified.** Devin's "SWE 1.7"
+    reference appears with no supporting data on the page fetched.
+11. **We have no measurement of outcome intelligence for anyone, including us.**
+    Nothing here tests whether ambiguous intent becomes an excellent product or
+    a real research result, as opposed to a change that passes its gates. Gate
+    pass rates and verdict artifacts are explicitly not a proxy for this. No
+    benchmark for it currently exists in this repo.
+12. **We have no measurement of architecture depth for anyone, including us.**
+    Nothing here tests reasoning across a massive repository, a distributed
+    system, a migration, or a performance investigation. Every probe in this
+    document ran against a single repo's help text or a 5-file diff, which is
+    the shallowest possible case.
+13. **The overhead of verification itself is only partially measured.**
+    PARTIALLY CLOSED 2026-08-03. `tools/verification-tax.py` now computes wall
+    time and outcome-change rate from evidence documents. First real reading, on
+    the `loki verify HEAD~1` run cited in section 3: **19.0s, outcome changed 1
+    of 1 runs**. That is a single-run sample and a single axis - it is a
+    baseline, not a trend. Still unmeasured: **token** cost of verification (only
+    wall time is captured), and the hit rate over a meaningful history rather
+    than one run. Reproduce with
+    `python3 tools/verification-tax.py <log.jsonl>`.
+14. **Confidence calibration is unmeasured.** Selectively bypassing verification
+    where confidence is high presupposes that the confidence signal is
+    calibrated. We have not measured whether ours is, so we cannot currently
+    say where bypassing would be safe.

package/docs/DASHBOARD-9.12-EVIDENCE.md

@@ -0,0 +1,97 @@
+# Dashboard 9.12: what was measured before any code was written
+
+Every number here was measured on 2026-08-03 against the working tree at
+`76e42773`. Commands are included so each can be re-run rather than trusted.
+
+## The premise, corrected
+
+An earlier pass in this session measured "is the dashboard rendering fabricated
+data" and answered mostly no: 38 of 43 components call a real API, and only two
+files contain `Math.random`. That measurement was sound and it answered the
+wrong question.
+
+The founder directive does not ask whether data is fabricated. It asks whether
+**every metric shows source, freshness, unknown, and error state**. That is a
+different property, and it is where the work is.
+
+## The finding
+
+The Python readers were built with strict envelope discipline: an empty result
+carries a `reason`, an unmeasured number reads `None` rather than `0`, and
+`source` names the files consulted. The UI discards most of that.
+
+| Envelope field | Components consuming it | What its absence means |
+|---|---|---|
+| `freshness_s` | **0 of 43** | A 40-minute-old poll renders identically to a fresh one |
+| `measured` | **0 of 43** | A real cost observation is indistinguishable from an absent one |
+| `reason` | 6 of 43 | "no runs" and "could not read runs" render the same in 37 components |
+| `source` | 10 of 43 | Most rows cannot be audited back to the file they came from |
+
+```bash
+cd dashboard-ui
+grep -l "freshness_s" components/*.js core/*.js | wc -l   # 0
+grep -l "measured"    components/*.js core/*.js | wc -l   # 0
+```
+
+### Absent rendered as zero
+
+```bash
+grep -o '|| 0' components/*.js | wc -l    # 116
+grep -l '|| 0' components/*.js | wc -l    # 23 of 43 components
+grep -o '?? 0' components/*.js | wc -l    # 8
+```
+
+116 occurrences across 23 components. This is the exact violation
+`record_is_measured()` exists to prevent, expressed in the UI layer: an absent
+measurement rendered as the number zero. From an operator's chair it is
+indistinguishable from fabricated data, because a fabricated zero and an
+unmeasured zero look the same.
+
+### No staleness signal exists
+
+`stale` appears 112 times but collapses to a SINGLE distinct usage:
+
+```bash
+grep -h -o ".\{45\}stale.\{25\}" components/*.js | sort -u
+# >${st.status === 'running' || st.status === 'stale' ? ...
+```
+
+It is a server-supplied status enum value, never a client-side computation
+over data age. There are no `isStale`, `staleness`, or `stale_s` identifiers.
+Nothing in the UI derives freshness from a timestamp.
+
+## The one genuine fabrication
+
+`components/loki-session-timeline.js:_buildPhasesFromStatus` synthesizes a
+timeline and renders it as history. Its own comment says so:
+
+```js
+// Simulate a multi-phase timeline based on iteration count
+const phaseOrder = ['planning', 'building', 'testing', 'reviewing'];
+const duration = segmentDuration * (0.8 + Math.random() * 0.4);
+```
+
+Phase NAMES come from a fixed rotation and durations are randomized. Neither is
+measured. An operator reads plausible phase boundaries for work that never
+happened at those times.
+
+The other `Math.random` (`loki-log-stream.js:267`) is benign: a unique key for
+a real log entry.
+
+**Fix direction** follows `version_is_ahead`: not a better estimate, an explicit
+"phase history not recorded". Whether real phase transitions can be read from
+`trust-events.jsonl` must be checked against that file rather than inferred
+from the component.
+
+## What this means for scope
+
+The redesign is not a rewrite of 31,924 lines of working component code. It is
+making the UI carry the honesty contract the backend already has:
+
+1. Every metric renders `unknown` rather than `0` when unmeasured.
+2. Every panel shows data age, and marks itself stale past a threshold.
+3. Every empty state states its `reason` and names its `source`.
+4. Nothing is synthesized and presented as measured.
+
+Items 1-3 are the founder's four required states (source, freshness, unknown,
+error) applied to surfaces that already fetch real data. Item 4 is one bug.