create-pmos 0.4.0 → 0.5.0

package/README.md

@@ -2,9 +2,10 @@
 
 **A product manager's operating system you can adopt in five minutes — no coding expertise
 required.** One command puts a disciplined, eval-driven PM workflow inside your own repo:
-initiative → PRD → eval rubric → agreed contract → build → three gates. Your AI coding agent
-(e.g. Claude Code) does the building. You do the judging. Everything stays in your folder — no
-database, no account, no network calls, no telemetry.
+initiative → PRD → eval rubric → agreed contract → build → three gates. **Any capable coding
+agent** does the building — Claude Code, Codex/ChatGPT, Cursor, Gemini, Copilot; PMOS is
+provider- and model-agnostic. You do the judging. Everything stays in your folder — no database,
+no account, no network calls, no telemetry.
 
 ## Whose task is what
 
@@ -43,7 +44,7 @@ What you'll see:
 
 Next steps in /home/you/my-product:
   1. cp pmos/templates/pmos-gate.yml .github/workflows/pmos-gate.yml   (only if on GitHub)
-  2. Add to the repo's CLAUDE.md: "This product is PMOS-managed (file mode) — read pmos/AGENTS.md before any work."
+  2. Wire PMOS into your agent's instruction file (CLAUDE.md, AGENTS.md, .cursorrules, GEMINI.md, or copilot-instructions): "This product is PMOS-managed (file mode) — read pmos/AGENTS.md before any work."
   3. Commit the pmos/ folder. Then follow pmos/AGENTS.md for every run.
 ```
 
@@ -249,6 +250,36 @@ Refreshes the skills, concepts, and scripts only. Your records — `state/`, `pl
 `okf/product/` — are yours, and an update never touches them. `pmos/KIT-VERSION` always says
 exactly what you're running.
 
+## What's new in 0.5.0
+
+0.4.0 opened the front of the loop. 0.5.0 makes explicit what was always the intent: **the kit is
+provider- and model-agnostic**, and the harness kernel now says so in its own words.
+
+- **Any capable coding agent runs the loop — not just Claude Code.** The runtime amendment
+  (D09/D34, ratified 2026-07-23) generalizes the evaluator, ingest, and design skills from
+  "runs in Claude Code" to "runs in the interactive agent runtime — any capable coding agent;
+  PMOS-self happens to use Claude Code, but the runtime is not Claude-specific." Point Codex,
+  Cursor, Gemini, or Copilot at `pmos/AGENTS.md` and the same protocol holds.
+
+  > Works the same in any of them: open your product folder in the agent, then
+  > "Read pmos/AGENTS.md and follow it." The gates, records, and metrics don't care which model
+  > wrote the code.
+
+- **Per-model calibration.** When you log a run or an eval you can now name the model that did the
+  work (`model`) or the judging (`evaluator_model`), so the evaluator's calibration is read *per
+  model* instead of silently pooling providers — `scripts/metrics.py` warns when labeled evals span
+  more than one evaluator model, because a trust score one model earned isn't inherited by another.
+  The design surface is documented as pluggable too (any surface that can take a brief and export a
+  component library), not tied to one vendor.
+
+- **A stated security posture that travels with the kit** (`pmos/SECURITY.md`), a `frontier.sh`
+  metrics helper, a file-mode `ticket-template.md`, and a sharper `SKILL-FORMAT.md` — all now part
+  of the vendored kernel.
+
+**Upgrade note:** `npx create-pmos@latest --update` brings all of this in and never touches your
+`state/`, `planning/`, or `okf/product/` records. Purely additive — no change to how the gate
+behaves on your existing initiatives.
+
 ## What's new in 0.4.0
 
 0.3.0 sharpened the judging. 0.4.0 adds the **front of the loop** — deciding *what* to build and

package/bin/create-pmos.js

@@ -21,8 +21,10 @@ const PKG = require(path.join(__dirname, '..', 'package.json'));
 const KERNEL = [
   'AGENTS.md',
   'README.md',
+  'SECURITY.md',
   'KIT-VERSION',
   'skills',
+  'skills-native',
   'okf/core',
   'scripts',
   'templates',
@@ -59,7 +61,8 @@ function vendor(targetRoot) {
 Next steps in ${targetRoot}:
   1. If the repo is not git yet:  git init
   2. If it lives on GitHub:       cp pmos/templates/pmos-gate.yml .github/workflows/pmos-gate.yml
-  3. Add to the repo's CLAUDE.md: "This product is PMOS-managed (file mode) — read pmos/AGENTS.md before any work."
+  3. Wire PMOS into your agent's instruction file (CLAUDE.md, AGENTS.md, .cursorrules, GEMINI.md, or
+     .github/copilot-instructions.md): "This product is PMOS-managed (file mode) — read pmos/AGENTS.md before any work."
   4. Commit the pmos/ folder. Then follow pmos/AGENTS.md for every run.`);
 }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-pmos",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Scaffold the PMOS portable kit — run the PMOS product-management workflow (initiative → lane-fit spec → contract → build → three gates, with work-type lanes for features, research, maintenance, incidents, and content) entirely inside your own repo, with all state as git-versioned files. No database, no network, no telemetry.",
   "license": "MIT",
   "bin": {

package/payload/AGENTS.md

@@ -24,8 +24,8 @@ yourself wanting an MCP tool, use the state adapter below instead.
 | `get_skill` | read `pmos/skills/<name>.skill` |
 | `get_okf_concepts` | read `pmos/okf/…` (index-first, then the concept body — progressive disclosure) |
 | `get_eval_rubric` | read `pmos/planning/evals/<id>-eval-rubric.md` |
-| `log_agent_run` | append one JSON line to `pmos/state/runs.jsonl` (`{run_id, initiative_id, pre_run_contract, self_eval, human_corrections, date}`) |
-| `save_eval` | append one JSON line to `pmos/state/evals.jsonl` (`{run_id, initiative_id, rubric_path, score, verdict, dimensions, date}`) |
+| `log_agent_run` | append one JSON line to `pmos/state/runs.jsonl` (`{run_id, initiative_id, pre_run_contract, self_eval, human_corrections, date, model}`) — record `model` = the model that produced the run so metrics stay honest across model switches |
+| `save_eval` | append one JSON line to `pmos/state/evals.jsonl` (`{run_id, initiative_id, rubric_path, score, verdict, dimensions, date, evaluator_model}`) — record `evaluator_model` = the model that rendered the verdict; κ is calibrated **per evaluator model** (a model swap does not inherit a prior κ) |
 | `record_acceptance` | append one JSON line to `pmos/state/acceptances.jsonl` (`{run_id, initiative_id, accepted, pm_label, note, date}`) — **written only on the PM's explicit word** |
 | `record_outcome_check` | append one JSON line to `pmos/state/outcome_checks.jsonl` (`{initiative_id, disposition, evidence, date}`) — the D57 post-acceptance outcome verdict (`validated | refuted | inconclusive`; evidence required for the first two). **PM's word only — never auto-dispose from metrics**; a re-judgment is a new line |
 | `save_okf_doc` | write the doc under `pmos/okf/product/…` marked as a DRAFT; the PM approves it via PR review (the file-mode form of draft→approved) |
@@ -93,6 +93,22 @@ honest.
   not install the pmos gate, or mirror the initiative file locally** — a vendored sibling gate
   fails closed on a missing local record by design (S5 Review Gate finding).
 
+## Decomposing an initiative into tickets (file mode has no board)
+
+When an initiative is bigger than one run, break it into **tickets** — the plain-file equivalent of
+the hosted product's SDLC board. One file per ticket in `pmos/state/tickets/NNN-slug.md`, numbered
+in **dependency order**, from [templates/ticket-template.md](templates/ticket-template.md):
+
+- Frontmatter: `id` (the NNN), `title`, `status` (`todo | doing | done`), and `blocked_by` — an
+  explicit list of the ticket ids that must be `done` first. Blocking edges are **text**, not a
+  hidden graph.
+- **Work the frontier.** `pmos/scripts/frontier.sh` lists exactly the tickets that are not done and
+  whose every blocker is done — the workable set. Start only from there. If it reports the frontier
+  is *blocked*, there is a cycle or an undone blocker to resolve before more work is startable.
+- Each ticket names **one deliverable**, behaviorally (interfaces/types/contracts, never file paths
+  or line numbers — they stale while the ticket waits). Tickets decompose an initiative; they do not
+  replace it — the initiative record, PRD, rubric, and the three gates are unchanged.
+
 ## New agent paths — the lethal-trifecta audit
 
 Before creating any new agent path or widening a tool grant, verify it does **not** hold all three
@@ -117,6 +133,15 @@ deployment until this repo has accumulated its own PM-vs-evaluator record (appen
 `pmos/scripts/metrics.py` computes the north star (human corrections per accepted run), pass rates,
 and the Goodhart tripwire from `pmos/state/`.
 
+**Model-awareness (any provider).** PMOS is provider/model-agnostic — any capable coding agent can
+run this loop — but calibration is a property of the **evaluator model**, not of "the Review Gate"
+abstractly. Record the model on every run/eval (the `model` / `evaluator_model` fields above); a
+model swap **forks** the calibration record — you do not inherit a κ another model earned, and
+`metrics.py` warns when labeled evals pool more than one evaluator model. The high-judgment roles
+(the adversarial evaluator, `explain-run`) assume a **frontier-tier reasoner**: a weaker model
+degrades *silently* to leniency and grep-clean while its output still looks well-formed — the
+per-model κ is what makes that degradation visible.
+
 ## PM protocol (weekly)
 
 Scan `discoveries.md`; a friction theme seen twice with no resolving decision must be promoted to a

package/payload/KIT-VERSION

@@ -1,3 +1,3 @@
-kernel-source: pmos@1687768
-assembled: 2026-07-21
+kernel-source: pmos@917d29e
+assembled: 2026-07-23
 mode: vendor

package/payload/README.md

@@ -13,16 +13,24 @@ personally-owned PMOS backend).
 - `AGENTS-HUMAN.md` — the same loop for a **human** builder (D58): plain terms + the
   `scripts/pmos-log.sh` helper that appends validated state records in one command. Mixed
   human+agent teams share the same state.
+- `SECURITY.md` — the kit's **security posture**: skills are executable instructions, what the kit
+  does structurally to be safe to install, and what stays your responsibility. Read it before you
+  install skills into your agent.
 - `skills/` — the PMOS skill kernel (prd, eval-rubric, okr, evaluator, …). Single-sourced from the
   PMOS repo; where a skill names a hosted tool, apply the AGENTS.md state adapter.
+- `skills-native/` — the same skills in **Agent-Skills standard** form (`<name>/SKILL.md`
+  directories), generated from `skills/` so a Claude Code / Codex / skills.sh harness can load them.
+  The flat `skills/` files stay the source of truth; this is the portable mirror.
 - `okf/core/` — kernel concepts (the sdlc loop, the three-tier gate, watermelon flags, …).
   `okf/product/` — YOUR product's knowledge; drafted by ingest, approved via PR review.
 - `planning/` — okrs, PRDs, rubrics (`agent-run-eval-rubric.md` is the per-initiative template),
   `eval-calibration.md` (append every PM override of an evaluator verdict).
 - `state/` — the operational record, **append-only**: `initiatives/<id>.md`, `runs.jsonl`,
-  `evals.jsonl`, `acceptances.jsonl`, `discoveries.md`.
+  `evals.jsonl`, `acceptances.jsonl`, `discoveries.md`. `tickets/NNN-slug.md` decompose an
+  initiative into ordered, dependency-numbered work (file-mode's board — see AGENTS.md).
 - `scripts/` — `gate-initiative.sh` (the fail-closed CI gate), `metrics.py` (north star +
-  calibration + Goodhart tripwire), `kit-update.sh` (kernel refresh).
+  calibration + Goodhart tripwire), `kit-update.sh` (kernel refresh), `frontier.sh` (lists the
+  workable tickets — those not done whose blockers are all done).
 - `templates/pmos-gate.yml` — CI drop-in; copy to `.github/workflows/`.
 - `KIT-VERSION` — which PMOS kernel this was assembled from.
 

package/payload/SECURITY.md

@@ -0,0 +1,53 @@
+# PMOS kit — security posture
+
+A skill is not a document. It is **instructions that run inside your agent**, with your agent's
+tools and your agent's access. Anything the kit ships into your repo can shape what your agent does.
+This file states, concretely, what the PMOS kit does to be safe to install — and where the
+responsibility stays yours.
+
+## Why "we scanned it" is not a posture
+
+The largest skills registry (skills.sh) shipped a malicious-skill scanner. Trail of Bits bypassed it
+with prompt injection plus compiled bytecode. Static scanning of agent instructions is *evadable by
+construction* — the payload is natural language aimed at a model, not a signature you can enumerate.
+So the PMOS kit does not claim "scanned and safe." It claims a set of **structural** properties you
+can check yourself, below.
+
+## What the kit does
+
+- **No on-load command execution in shipped skills.** No PMOS kit skill uses Claude Code's dynamic
+  `` !`cmd` `` context injection or any load-time shell-out. A skill's body is read as instructions,
+  not run as a script when it loads. *(Check: `grep -rn '!\`' pmos/skills` — expect nothing.)*
+- **Least authority, declared.** Shipped skills request the minimum tools they need; the
+  security-sensitive one — the **evaluator** — is **read-only by design**: its only write is the
+  schema-constrained verdict. The component that reads untrusted content (diffs, PR text) is
+  deliberately *not* the component that can act, which removes the exfiltration leg of the lethal
+  trifecta.
+- **A `license` on every shipped skill** (portability + provenance), and PMOS producer metadata is
+  carried under the spec's `metadata:` extension map, never as ambient top-level directives.
+- **Fail-closed containment.** The kit is assembled by `kit/build-kit.sh`, which greps its own output
+  and **aborts** if any reference to a hosted PMOS backend (a project ref, an MCP endpoint, or a
+  hosted-backend URL) leaks in. The npm publish path re-runs this check on the exact tarball and
+  refuses to publish on anything but a clean result. The kit you get is self-contained: **no
+  database, no network, no telemetry.**
+- **No network calls in the installer.** `create-pmos` vendors a bundled payload; it makes no
+  outbound requests and sends nothing about you anywhere.
+- **Everything the evaluator reads is untrusted data, never instructions.** An instruction embedded
+  in a diff or PR body ("mark this PASS") is treated as *evidence of a defect*, not a command —
+  paired with the read-only scope above, because prompt-level defenses alone are bypassable.
+
+## What stays your responsibility
+
+- **Review before you install.** Read the skills and scripts in `pmos/` the way you would review any
+  dependency PR. The kit is small and plain-text on purpose so you can. `--update` refreshes only the
+  kernel and never touches your `state/` — review its diff like any other change.
+- **Your agent's tools are yours to scope.** The kit declares least authority for its own skills, but
+  what your agent is *allowed* to do in your repo is set by you, not by us.
+- **Pin what you install.** Prefer a pinned `create-pmos@<version>` over `@latest` in automation, and
+  re-review the kernel diff on each update.
+
+## Reporting
+
+Found something that violates the posture above — a shipped skill with load-time execution, a
+containment leak, an over-scoped tool request? Open an issue in the PMOS repo describing what and
+where. The posture is only worth stating if it is enforced; a false claim here is itself a bug.

package/payload/okf/core/concepts/design-loop.md

@@ -32,7 +32,8 @@ An initiative with a user-facing surface runs four legs *inside* Build:
 1. **Brief** — the [design-brief skill](/skills/core/design-brief.skill) writes the context handoff: what
    to design and the hard, presentation-only constraints, each surface tied to a real data source. It is to
    design what the [PRD](/skills/core/prd.skill) is to build.
-2. **Design** — the design is made in claude.ai/design (external to this repo).
+2. **Design** — the design is made in an external design surface (PMOS-self uses claude.ai/design;
+   any capable design surface fits — external to this repo).
 3. **Reconcile + build** — the [design-reconcile skill](/skills/core/design-reconcile.skill) classifies
    every surface against **the product's capability ledger** and the PM signs off before any code; then the
    design is built into the product's app on its design system, verified by the product's CI. (For PMOS's own
@@ -42,7 +43,9 @@ An initiative with a user-facing surface runs four legs *inside* Build:
    to claude.ai/design (for PMOS, see [.design-sync/conventions.md](/.design-sync/conventions.md)), so the
    next design starts from the product's actual parts.
 
-Legs 2 and 4 are external tools PMOS *references*; legs 1 and 3 are the PMOS skills that make the loop
+The design surface (legs 2 and 4) is **pluggable and provider-agnostic** — PMOS-self uses
+claude.ai/design, but the loop is not tied to it; any capable design surface that can consume a brief
+and export a component library fits (harness-model-agnosticism). Legs 2 and 4 are external tools PMOS *references*; legs 1 and 3 are the PMOS skills that make the loop
 repeatable.
 
 ## Presentation-only is the load-bearing constraint
@@ -119,7 +122,7 @@ PMOS's own instance: the `web/` app, the [D38](/okf/products/pmos/adr/d38-design
   [eval-driven](/okf/core/concepts/eval-driven-pm.md) style (the fidelity rubric before the build).
 - Reconciliation is [watermelon-flag](/okf/core/concepts/watermelon-flag.md) applied to UI.
 - PMOS stays a [control plane](/okf/core/concepts/control-plane.md): it specs and evaluates design; the design
-  itself is made in claude.ai/design and built by a build agent.
+  itself is made in an external design surface (e.g. claude.ai/design) and built by a build agent.
 - Formalized by [D48](/okf/products/pmos/adr/d48-design-loop.md); made product-scoped by
   [D50](/okf/products/pmos/adr/d50-design-loop-product-scoped.md); composes with the design *system* (D38,
   PMOS's own instance).

package/payload/okf/core/templates/design-brief-template.md

@@ -1,7 +1,7 @@
 ---
 type: Template
 title: Design Brief Template
-description: The scaffold for a design brief instance — the context handoff from Claude Code to claude.ai/design. Copy into /planning/design/ and fill; presentation-only.
+description: The scaffold for a design brief instance — the context handoff from the interactive agent to an external design surface (e.g. claude.ai/design). Copy into /planning/design/ and fill; presentation-only.
 tags: [template, design, brief, planning]
 timestamp: 2026-07-01
 ---

package/payload/scripts/frontier.sh

@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# frontier.sh — list the workable tickets in a file-mode PMOS kit (E9).
+#
+# "Work the frontier": a ticket is workable when it is NOT done and EVERY ticket it is blocked_by is
+# done. This is the plain-file equivalent of a board's dependency structure — one file per ticket in
+# state/tickets/NNN-slug.md, numbered in dependency order, blocking edges as text.
+#
+#   scripts/frontier.sh              # from the kit root; lists workable tickets
+#   scripts/frontier.sh state/tickets   # explicit tickets dir
+#
+# Dependency-free and portable: POSIX-ish bash, no associative arrays (runs on bash 3.2, e.g. stock
+# macOS). No network, no board. Reads only the YAML frontmatter fields id / status / blocked_by.
+set -euo pipefail
+
+DIR="${1:-state/tickets}"
+[ -d "$DIR" ] || { echo "frontier: no tickets dir at '$DIR' (nothing to do)"; exit 0; }
+
+# --- field readers (frontmatter only: stop at the closing --- fence) ----------------------------
+fm() { awk 'NR>1 && /^---[[:space:]]*$/{exit} {print}' "$1"; }   # lines between the opening and closing ---
+field() { printf '%s\n' "$1" | sed -n "s/^$2:[[:space:]]*//p" | head -1 | sed 's/^["'\'']//;s/["'\'']$//;s/[[:space:]]*$//'; }
+# blocked_by is an inline list "[\"002\", \"005\"]"; extract the bare ids (digits) in order.
+blockers() { printf '%s\n' "$1" | sed -n 's/^blocked_by:[[:space:]]*//p' | head -1 | grep -oE '[0-9]+' || true; }
+# derive an id: explicit `id:` field, else the leading NNN of the filename.
+idof() { local i; i="$(field "$1" id)"; [ -n "$i" ] && { printf '%s' "$i"; return; }; basename "$2" | grep -oE '^[0-9]+' || basename "$2" .md; }
+
+shopt -s nullglob
+files=("$DIR"/*.md)
+[ ${#files[@]} -eq 0 ] && { echo "frontier: no tickets in '$DIR' (nothing to do)"; exit 0; }
+
+# --- pass 1: collect the done ids (newline-delimited; no associative arrays for bash-3.2 portability)
+DONE_IDS=""
+OPEN=0
+for f in "${files[@]}"; do
+  meta="$(fm "$f")"; id="$(idof "$meta" "$f")"; st="$(field "$meta" status)"
+  if [ "$st" = "done" ]; then DONE_IDS="$DONE_IDS$id
+"; else OPEN=$((OPEN+1)); fi
+done
+is_done() { printf '%s' "$DONE_IDS" | grep -qx "$1"; }
+
+# --- pass 2: a ticket is on the frontier when not done and every blocker is done -----------------
+found=0
+for f in "${files[@]}"; do
+  meta="$(fm "$f")"; id="$(idof "$meta" "$f")"; st="$(field "$meta" status)"; st="${st:-todo}"
+  [ "$st" = "done" ] && continue
+  workable=1
+  while IFS= read -r b; do
+    [ -z "$b" ] && continue
+    is_done "$b" || workable=0
+  done < <(blockers "$meta")
+  if [ "$workable" -eq 1 ]; then
+    printf '  ▸ %-5s %-8s %s\n' "$id" "[$st]" "$(field "$meta" title)"
+    found=$((found+1))
+  fi
+done
+
+if [ "$found" -eq 0 ]; then
+  # distinguish "all done" from "everything is blocked" — a stuck frontier is a real signal.
+  if [ "$OPEN" -eq 0 ]; then echo "frontier: all tickets done ✓"
+  else echo "frontier: $OPEN open ticket(s), NONE workable — the frontier is blocked (check for a cycle or an undone blocker)."; fi
+fi

package/payload/scripts/metrics.py

@@ -7,7 +7,9 @@ Reads pmos/state/{runs,evals,acceptances}.jsonl and prints:
   B2  throughput  — accepted runs (guards against winning B1 by doing less)
   --  eval pass rate — share of evals with verdict PASS / PASS-WITH-FINDINGS
   B4  calibration — evaluator verdict vs PM label, joined by run_id: raw agreement, TPR, TNR,
-                    Cohen's kappa (the D40 classifier method, computed locally)
+                    Cohen's kappa (the D40 classifier method, computed locally). κ is broken out
+                    PER evaluator_model, with a mixed-model warning — a κ is a property of the
+                    evaluator model and is not inherited across a model swap (harness-model-agnosticism)
   --  Goodhart tripwire — fires when eval_pass_rate >= 0.8 while avg corrections >= 1.5 (the
                     pass-happy-judge signal; anti-optimism: read the red first)
 
@@ -47,25 +49,33 @@ n_pass = sum(1 for e in evals if str(e.get("verdict", "")).upper() in passing)
 pass_rate = (n_pass / len(evals)) if evals else None
 
 # --- B4: verdict-vs-PM-label confusion matrix, joined by run_id (D40) ---
+# Calibration is a property of the EVALUATOR MODEL (harness-model-agnosticism): a κ earned by one
+# model is not inherited by another. Compute the matrix OVERALL and PER evaluator_model, and warn
+# when one aggregate κ silently pools verdicts from more than one model.
 label_by_run = {a.get("run_id"): str(a.get("pm_label", "")).lower() for a in acc if a.get("pm_label")}
-tp = tn = fp = fn = 0
-for e in evals:
-    lab = label_by_run.get(e.get("run_id"))
-    if lab not in ("pass", "fail"):
-        continue
-    pred_pass = str(e.get("verdict", "")).upper() in passing
-    if pred_pass and lab == "pass":
-        tp += 1
-    elif pred_pass and lab == "fail":
-        fp += 1
-    elif not pred_pass and lab == "fail":
-        tn += 1
-    else:
-        fn += 1
-n = tp + tn + fp + fn
 
 
-def kappa():
+def confusion(rows):
+    """(tp, tn, fp, fn) over evals joined to a pass/fail PM label by run_id."""
+    tp = tn = fp = fn = 0
+    for e in rows:
+        lab = label_by_run.get(e.get("run_id"))
+        if lab not in ("pass", "fail"):
+            continue
+        pred_pass = str(e.get("verdict", "")).upper() in passing
+        if pred_pass and lab == "pass":
+            tp += 1
+        elif pred_pass and lab == "fail":
+            fp += 1
+        elif not pred_pass and lab == "fail":
+            tn += 1
+        else:
+            fn += 1
+    return tp, tn, fp, fn
+
+
+def kappa_of(tp, tn, fp, fn):
+    n = tp + tn + fp + fn
     if n == 0:
         return None
     po = (tp + tn) / n
@@ -73,6 +83,14 @@ def kappa():
     return None if pe == 1 else (po - pe) / (1 - pe)
 
 
+tp, tn, fp, fn = confusion(evals)
+n = tp + tn + fp + fn
+
+
+def kappa():
+    return kappa_of(tp, tn, fp, fn)
+
+
 avg_corr = (sum(corrections_by_run.values()) / len(corrections_by_run)) if corrections_by_run else None
 tripwire = pass_rate is not None and avg_corr is not None and pass_rate >= 0.8 and avg_corr >= 1.5
 
@@ -98,6 +116,28 @@ else:
           f"TNR {'n/a — no PM fails seen (leniency blind spot)' if tnr is None else f'{tnr:.0%}'}, "
           f"kappa {'undefined' if k is None else f'{k:.2f}'} (target >= 0.60 — verdict ADVISORY below it, D40)")
 
+# --- per-evaluator-model calibration (harness-model-agnosticism) -------------
+# κ is a property of (evaluator model × rubric); a model swap does NOT inherit a prior κ. When the
+# labeled evals span >1 evaluator model, the aggregate above is a pool, not a gate — break it out.
+# When no eval records its model, κ silently pools across any swaps: nudge to record provenance.
+labeled_evals = [e for e in evals if label_by_run.get(e.get("run_id")) in ("pass", "fail")]
+by_model = {}
+for e in labeled_evals:
+    by_model.setdefault(e.get("evaluator_model") or "unknown", []).append(e)
+if len(by_model) > 1:
+    print(f"\n🔴 MIXED-MODEL CALIBRATION — labeled evals span {len(by_model)} evaluator models; the "
+          "aggregate κ above POOLS them and is not any single evaluator's calibration (κ is per "
+          "evaluator model, D40). Per model:")
+    for m in sorted(by_model):
+        c = confusion(by_model[m])
+        km = kappa_of(*c)
+        print(f"   - {m}: n={sum(c)}, kappa {'undefined' if km is None else f'{km:.2f}'} "
+              f"(tp{c[0]} tn{c[1]} fp{c[2]} fn{c[3]})")
+elif by_model and set(by_model) == {"unknown"}:
+    print("\n⚠ evaluator_model unrecorded on every labeled eval — κ cannot be attributed to a model "
+          "and silently pools across any model swaps. Pass the evaluator model as the last arg to "
+          "`pmos-log.sh eval` (harness-model-agnosticism).")
+
 # --- D57 outcome checks (accepted-but-unvalidated; due prints red) -----------
 # Declarations live in initiative frontmatter (outcome_metric / outcome_horizon); dispositions are
 # appended to state/outcome_checks.jsonl ({initiative_id, disposition, evidence, date}). The latest

package/payload/scripts/pmos-log.sh

@@ -6,8 +6,8 @@
 # script only ever `>>`s; a correction is a NEW record. No dependencies beyond POSIX + python3
 # (used solely for safe JSON string escaping).
 #
-#   pmos-log.sh run      <run_id> <initiative_id> [human_corrections]
-#   pmos-log.sh eval     <run_id> <initiative_id> <score 0..1> <PASS|PASS-WITH-FINDINGS|FAIL> <rubric_path>
+#   pmos-log.sh run      <run_id> <initiative_id> [human_corrections] [model]
+#   pmos-log.sh eval     <run_id> <initiative_id> <score 0..1> <PASS|PASS-WITH-FINDINGS|FAIL> <rubric_path> [evaluator_model]
 #   pmos-log.sh accept   <run_id> <initiative_id> <pass|fail> [note]
 #   pmos-log.sh outcome  <initiative_id> <validated|refuted|inconclusive> [evidence]
 #   pmos-log.sh friction <note...>
@@ -33,20 +33,30 @@ append() { # append <file> <json-line>
 cmd="${1:-}"; shift || true
 case "$cmd" in
   run)
-    [ $# -ge 2 ] || fail "usage: run <run_id> <initiative_id> [human_corrections]"
+    [ $# -ge 2 ] || fail "usage: run <run_id> <initiative_id> [human_corrections] [model]"
     nonempty run_id "$1"; nonempty initiative_id "$2"
     corr="${3:-0}"
     printf '%s' "$corr" | grep -qE '^[0-9]+$' || fail "human_corrections must be a non-negative integer, got '$corr'."
-    append runs.jsonl "{\"run_id\":$(js "$1"),\"initiative_id\":$(js "$2"),\"human_corrections\":$corr,\"date\":\"$today\"}"
+    # optional builder-model provenance (which model produced the run). Omitted when absent — the
+    # metric layer treats a missing model as 'unknown'. if-form, not `&&`: an empty $4 under set -e
+    # would abort. Keeps calibration honest across model switches (harness-model-agnosticism).
+    model_field=""
+    if [ -n "${4:-}" ]; then model_field=",\"model\":$(js "$4")"; fi
+    append runs.jsonl "{\"run_id\":$(js "$1"),\"initiative_id\":$(js "$2"),\"human_corrections\":$corr,\"date\":\"$today\"$model_field}"
     ;;
   eval)
-    [ $# -ge 5 ] || fail "usage: eval <run_id> <initiative_id> <score 0..1> <PASS|PASS-WITH-FINDINGS|FAIL> <rubric_path>"
+    [ $# -ge 5 ] || fail "usage: eval <run_id> <initiative_id> <score 0..1> <PASS|PASS-WITH-FINDINGS|FAIL> <rubric_path> [evaluator_model]"
     nonempty run_id "$1"; nonempty initiative_id "$2"
     printf '%s' "$3" | grep -qE '^(0?\.[0-9]+|0|1(\.0+)?)$' || fail "score must be 0..1 written as 0, 1, N.N or .N — got '$3'."
     case "$4" in PASS|PASS-WITH-FINDINGS|FAIL) ;; *) fail "verdict must be PASS | PASS-WITH-FINDINGS | FAIL, got '$4'." ;; esac
     score="$3"; case "$score" in .*) score="0$score" ;; esac  # ".5" -> "0.5": bare .5 is not valid JSON
     [ -f "$KIT/$5" ] || [ -f "$5" ] || fail "rubric_path '$5' not found (relative to the kit or the repo)."
-    append evals.jsonl "{\"run_id\":$(js "$1"),\"initiative_id\":$(js "$2"),\"score\":$score,\"verdict\":$(js "$4"),\"rubric_path\":$(js "$5"),\"date\":\"$today\"}"
+    # optional evaluator-model provenance (which model rendered the verdict). Calibration (κ) is a
+    # property of the evaluator MODEL, so metrics.py segments κ by it; a missing value counts as
+    # 'unknown'. if-form, not `&&` (empty $6 under set -e aborts).
+    emodel_field=""
+    if [ -n "${6:-}" ]; then emodel_field=",\"evaluator_model\":$(js "$6")"; fi
+    append evals.jsonl "{\"run_id\":$(js "$1"),\"initiative_id\":$(js "$2"),\"score\":$score,\"verdict\":$(js "$4"),\"rubric_path\":$(js "$5"),\"date\":\"$today\"$emodel_field}"
     ;;
   accept)
     [ $# -ge 3 ] || fail "usage: accept <run_id> <initiative_id> <pass|fail> [note]"

package/payload/skills/SKILL-FORMAT.md

@@ -36,13 +36,36 @@ Rules:
   (specified, non-executable) → graduates to active (key removed) by a PM decision in `DECISIONS.md`.
 - **Optional Phase-B registry metadata (D49)** — a skill may also carry `version`, `owner`, `risk`,
   `category`, and `scope`. These are **optional**, producer-defined registry fields that
-  [`gen_skills_seed.sh`](/supabase/seed/gen_skills_seed.sh) sources into the `skills` table so the skill
+  the resync generator [`gen_okf_mirror_seed.py`](/supabase/gen_okf_mirror_seed.py) sources into the `skills` table so the skill
   library can group/version/scope skills. A skill with only `name` + `description` is still fully
   conformant (e.g. `build-skills/evaluator.skill`); carry the registry keys on durable `skills/core/`
   entries for consistency with their siblings.
+- `lifecycle` — **optional**, in the same D49 registry slot. Values: `active` (the default when the
+  key is absent) | `deprecated`. A deprecated skill **stays on disk and stays in the registry** —
+  `get_skill` returns it in full with a `deprecated_notice` so a retrieving agent is *told* not to
+  start new work on it. Retirement is a state you can read; deletion is a separate act (see the
+  invariant below). Distinct from `maturity_mode`, which is about a skill not yet executable —
+  `lifecycle` is about one on its way out.
 - **No other L1 keys.** Beyond the required `name` + `description`, the optional `maturity_mode`, and the
   optional Phase-B registry keys above, procedural detail does not belong in L1 — it lives in the body.
 
+### The disk ⇄ registry invariant (enforced in CI)
+
+**Every `.skill` on disk is seeded, and every seeded skill exists on disk — exactly.** The `skills`
+table is a *mirror*, and `get_skill` serves it to agents in other people's repos, so a mirror that
+has quietly drifted is a wrong answer delivered confidently.
+
+The `REGISTRY` check in [`check_okf_drift.py`](/supabase/check_okf_drift.py) fails the build on:
+adding or renaming a `.skill` without regenerating the seed; a seeded skill whose file is gone; and
+any registry-metadata value that disagrees with the file's own frontmatter. That last one is not
+theoretical — a generator whose INSERT omitted the Phase-B columns silently nulled
+`version/owner/risk/category/scope` for **all 21 skills for about three weeks** in 2026-07, and was
+caught only by an incidental version pin in an RLS test. So:
+
+> **When you add, rename, delete, or edit the frontmatter of a `.skill`, regenerate the seed in the
+> same PR** ([okf-sync playbook](/okf/core/playbooks/okf-sync.md)) — and sweep `supabase/tests/` for
+> pins on that skill's registry metadata before you do (the prd-grilling F1 rule).
+
 ### Level 2 — SKILL.md body (loaded when relevant)
 
 The body below the frontmatter holds the **full procedural detail**: steps, heuristics, required
@@ -143,15 +166,49 @@ cheap:
 - Level-3 sub-files **≅** the standard's bundled files.
 
 (One nuance for the follow-on below: PMOS's *optional* D49 registry keys —
-`version`/`owner`/`risk`/`category`/`scope` — are top-level custom keys; the standard would expect
+`version`/`owner`/`risk`/`category`/`scope`/`lifecycle` — are top-level custom keys; the standard would expect
 producer-defined extensions under a `metadata:` map. The **core** shape is lossless; relocating the
 registry keys belongs to the native-format packaging follow-on, not this flat form.)
 
-**Named follow-on (not built here):** shipping the **kit** in native Agent-Skills directory format so
-a stranger's Claude Code / Codex harness auto-discovers PMOS skills. It touches `build-kit.sh`,
-`get_skill`, `gen_skills_seed.sh`, and every skill path, so it is its own initiative — revisit when
-native-discovery demand in kit repos is proven. (This mirrors the mattpocock/skills plugin decision:
-keep the editable flat-file form, add a native-package path only when the demand is real.)
+### Kit native-format packaging (E6 — shipped)
+
+The **kit** now ships every skill in native Agent-Skills directory form *as well as* the flat file —
+a lossless portability layer, generated **from** the flat `.skill` at build time (never
+hand-authored, so it cannot drift from its source). For each shipped skill, `kit/build-kit.sh` emits
+`skills-native/<name>/SKILL.md` where:
+
+- the directory name **equals** the frontmatter `name` (the standard's hard rule), and required
+  `name` + `description` are present;
+- PMOS's D49 registry keys (`version`/`owner`/`risk`/`category`/`scope`/`lifecycle`) are relocated
+  **under a `metadata:` map with a `pmos-` prefix** — the standard's producer-extension slot — never
+  as top-level custom keys, so a conformant runtime ignores them cleanly;
+- a **fail-closed conformance self-check** in `build-kit.sh` aborts the build on any violation
+  (name≠dirname, missing required field, or a top-level custom key), the same posture as the
+  containment check.
+
+The flat `.skill` files remain the file-mode source of truth (the AGENTS.md state adapter references
+`pmos/skills/<name>.skill`); the native tree is the ecosystem-portable mirror. **Still a named
+follow-on (not built here):** wiring `create-pmos` to install the native tree into a runtime's
+auto-discovery locations (`.claude/skills/`, `.agents/skills/`). E6 delivers the conformant
+*packaging*; the *install location* is a separate change, revisited when native-discovery demand in
+kit repos is proven.
+
+### Per-skill eval harness (E8 — convention shipped, benchmark deferred)
+
+A skill may carry an **opt-in** eval file at `skills/evals/<name>.evals.json` (mapping to the
+standard's `<name>/evals/evals.json` in native packaging), adopting the skill-creator harness shape:
+
+- **`description_triggers`** — `should_fire` / `should_not_fire` prompt lists that test whether the
+  L1 description fires on the right requests and stays silent on the wrong ones (trigger-accuracy).
+- **`cases`** — each a `{ name, prompt, expect, with_without }`, where `expect` is an **outcome
+  property** a grader checks (never a script) and `with_without` toggles whether the case is meant
+  for with-vs-without-the-skill benchmarking.
+
+`scripts/check_skill_evals.py` (run in the `okf-drift` job) validates the **shape** of any present
+eval file and that it names a real skill. It **never runs a benchmark** — actually scoring
+with-vs-without needs labeled outcomes + compute and is **deferred** (there is no negative class
+yet); running models in CI would make the gate nondeterministic. The convention and a worked example
+(`skills/evals/prd.evals.json`) ship now; the verdict "this skill measurably helps" does not.
 
 ## File naming
 
@@ -170,8 +227,8 @@ keep the editable flat-file form, add a native-package path only when the demand
 A skill is well-formed when:
 
 1. Level 1 frontmatter has `name` + `description` (both required), plus optionally `maturity_mode` (on
-   non-active skills) and the Phase-B registry keys `version`/`owner`/`risk`/`category`/`scope` (D49);
-   no other L1 keys.
+   non-active skills) and the Phase-B registry keys `version`/`owner`/`risk`/`category`/`scope`/
+   `lifecycle` (D49); no other L1 keys.
 2. `name` matches the filename stem.
 3. `description` states both what the skill does and when to use it.
 4. The body (Level 2) carries the full procedure and is self-sufficient for the common case.