yadflow 3.17.2 → 3.18.0

package/CHANGELOG.md

@@ -1,3 +1,15 @@
+# [3.18.0](https://github.com/abdelrahmannasr/yadflow/compare/v3.17.3...v3.18.0) (2026-09-05)
+
+
+### Features
+
+* **cli:** add yad migrate with preview, backup and report ([39dde87](https://github.com/abdelrahmannasr/yadflow/commit/39dde87b9f38936b05b6cd6243a67b9a2aea7fb9))
+* **doctor:** report shape drift against the engine ([f2dc377](https://github.com/abdelrahmannasr/yadflow/commit/f2dc377815a1135b4015cff9f46f3384dc08ee56))
+* **release:** publish majors to a next channel and warn before upgrading ([70d2334](https://github.com/abdelrahmannasr/yadflow/commit/70d23347021a3635e8d71f6465d516bd6e72322d))
+* **state:** stamp schemaVersion 1 on every engine-written file ([872dde6](https://github.com/abdelrahmannasr/yadflow/commit/872dde61975b942e95670c60f329621802256ab8)), closes [#163](https://github.com/abdelrahmannasr/yadflow/issues/163)
+
+## [3.17.3](https://github.com/abdelrahmannasr/yadflow/compare/v3.17.2...v3.17.3) (2026-09-03)
+
 ## [3.17.2](https://github.com/abdelrahmannasr/yadflow/compare/v3.17.1...v3.17.2) (2026-09-03)
 
 ## [3.17.1](https://github.com/abdelrahmannasr/yadflow/compare/v3.17.0...v3.17.1) (2026-09-02)

package/README.md

@@ -181,5 +181,6 @@ workflow-hygiene flags — derived read-only, so an EM can see how the team actu
 end-to-end harness on both). On **Windows use [WSL](https://learn.microsoft.com/windows/wsl/)** — native
 PowerShell is not yet supported. Requires **Node.js ≥ 18**.
 
-**Releases** are automated via [semantic-release](https://semantic-release.gitbook.io/) on merge to
-`main` (Conventional Commits → npm, with provenance). See [RELEASING.md](RELEASING.md).
+**Releases** are a human decision. Merging to `main` never publishes; a person fast-forwards the
+`release` branch, and [semantic-release](https://semantic-release.gitbook.io/) publishes from there
+(Conventional Commits → npm, with provenance). See [RELEASING.md](RELEASING.md).

package/bin/yad.mjs

@@ -16,6 +16,7 @@ import { runRepo } from '../cli/repo.mjs';
 import { runRoster } from '../cli/roster.mjs';
 import { runDocs } from '../cli/docs.mjs';
 import { runDoctor } from '../cli/doctor.mjs';
+import { runMigrate } from '../cli/migrate.mjs';
 import { runNext } from '../cli/next.mjs';
 import { runSkip } from '../cli/skip.mjs';
 import { syncStatuses } from '../cli/artifact-status.mjs';
@@ -47,6 +48,11 @@ ${c.bold('Setup & maintenance')}
                        'yad check --fix --push'; --allow-branch permits a non-default branch
   yad doctor [--json]  Environment + state health: tools/auth, config files,
                        repo paths, epic ledgers (exit 1 on any failure)
+  yad migrate [--apply] [--json]   Move this project's state files onto the shape
+                       this yadflow expects. Prints what WOULD change and writes
+                       nothing until --apply, which copies each file it rewrites to
+                       <file>.yad-orig first. Safe to run twice — the second run
+                       reports there is nothing to do
   yad sync-status [epic]   Update artifact frontmatter status (draft/in-review/approved)
                        from .sdlc/state.json — all epics if omitted (--dry-run to preview)
   yad report [-m <text>]   File a bug in the yadflow repo with auto-scrubbed diagnostics
@@ -218,6 +224,10 @@ function parseArgs(argv) {
     else if (a === '--repos') o.repos = true;
     else if (a === '--wire') o.wire = true;
     else if (a === '--dry-run') o.dryRun = true;
+    else if (a === '--apply') o.apply = true;
+    // The roadmap and the release check spell the default `--preview`. Accepting it means a script can
+    // say what it means rather than relying on the absence of a flag.
+    else if (a === '--preview') o.apply = false;
     else if (a === '--json') o.json = true;
     else if (a === '-h' || a === '--help') o.help = true;
     else if (a === '-v' || a === '--version') o.version = true;
@@ -260,6 +270,9 @@ async function main() {
     case 'doctor':
       await runDoctor(o.dir, { json: o.json });
       break;
+    case 'migrate':
+      await runMigrate(o.dir, { apply: o.apply, json: o.json });
+      break;
     // Harness-invoked, not typed by a human: a tool-call payload arrives on stdin and the exit code
     // is the verdict (0 allow, 2 deny). See cli/hook.mjs for the contract.
     case 'hook': {

package/cli/doctor.mjs

@@ -1,12 +1,14 @@
 // `yad doctor` — environment + state health, the complement of `yad check` (file drift).
-// Three sections: environment (tools on PATH, auth), project state (config files parse and
-// point at real repos), epics (each ledger loads). Pure reporting: exit 1 on any FAIL,
-// 0 with warnings. `--json` emits the checks for CI / bug reports.
+// Five sections: environment (tools on PATH, auth), project state (config files parse and point at
+// real repos), shape (what schemaVersion the files are on vs the engine), epics (each ledger loads),
+// and threads (feature-thread lineage). Pure reporting: exit 1 on any FAIL, 0 with warnings.
+// `--json` emits the checks for CI / bug reports.
 import path from 'node:path';
 import fs from 'node:fs';
 import { c, log, ok, info, warn, fail, hand, run, has, exists, readJSON, readJSONStrict } from './lib.mjs';
 import { VERSION, PROJECT_FILES, DESIGN_TOOLS, TESTING_TOOLS, LEARNING_TOOLS, HOOK_SETTINGS, HOOK_TOOL_MATCHER, isBridgeHub } from './manifest.mjs';
 import { mergeHookSettings, hookMatcherFires, ideTargetsFor } from './plan.mjs';
+import { planMigration } from './migrate.mjs';
 import { loadLedger, epicRoot, isValidEpicId, epicLineage, resolveThread, stateInvariants, contractSurfaceHash, artifactHash } from './epic-state.mjs';
 import { loadDebt } from './thread.mjs';
 import { gitHead, insideWorkspace } from './setup.mjs';
@@ -32,9 +34,11 @@ const underProjectRoot = (root, p) => {
 // corruption, and must not be reassured away as an expected sibling.
 const isRegistrableSibling = (root, rpath) => insideWorkspace(root, rpath);
 
-// Each check: { id, section, status: 'ok'|'warn'|'fail', message, hint? }
-function check(checks, id, section, status, message, hint = '') {
-  checks.push({ id, section, status, message, ...(hint ? { hint } : {}) });
+// Each check: { id, section, status: 'ok'|'warn'|'fail', message, hint?, …extra }
+// `extra` carries structured detail for the `--json` consumer that would be unreadable in the prose
+// line — e.g. the per-file shape table behind a one-sentence drift summary.
+function check(checks, id, section, status, message, hint = '', extra = null) {
+  checks.push({ id, section, status, message, ...(hint ? { hint } : {}), ...(extra || {}) });
 }
 
 export function envChecks(checks) {
@@ -507,6 +511,111 @@ export function epicChecks(checks, root) {
   }
 }
 
+// ---- file shape (schemaVersion) -------------------------------------------------------------
+// What shape this project's files are in, against the shape this engine writes. The stamp itself is
+// silent by design (cli/lib.mjs), and `yad migrate` only speaks when you run it — so without this
+// section a project could sit a shape behind, or a shape ahead, with nothing ever saying so. Rule 6:
+// the engine never goes quiet about what is unprotected.
+//
+// The reading comes from `planMigration`, the same function `yad migrate` previews with, so doctor and
+// migrate can never disagree about what state a project is in or what would fix it.
+//
+// Three outcomes, and the middle one is the whole point:
+//   ok    every file is on the engine's shape
+//   warn  a file is BEHIND — `yad migrate` walks it forward, and the message says so
+//   fail  a file is AHEAD — written by a newer yadflow than this one; migrating would downgrade it,
+//         so the fix is to upgrade the CLI, not to touch the file
+const scopeOf = (rel) => {
+  const parts = rel.split(path.sep);
+  return parts[0] === 'epics' && parts.length > 1 ? parts[1] : null;
+};
+
+function shapeCheckFor(checks, id, label, rows, engine) {
+  // A file that does not parse has no shape to compare. It still gets said out loud here, because
+  // nothing else in doctor reads these files — a corrupt change.json or build-log shard would
+  // otherwise pass a clean health check while `yad migrate` refuses to touch the project over it.
+  const unreadable = rows.filter((r) => r.from === null);
+  if (unreadable.length) {
+    check(checks, `${id}:unreadable`, 'shape', 'fail',
+      `${label}: ${unreadable.length} state file(s) do not parse — ${unreadable.map((r) => r.file).join(', ')}`,
+      'restore them from git — a broken state file blocks `yad migrate` and cannot be read by the gate');
+  }
+
+  const readable = rows.filter((r) => r.from !== null);
+  if (!readable.length) return;
+  const ahead = readable.filter((r) => r.action === 'ahead');
+  // A file behind the engine on a VERIFIED hub is real drift, but `yad migrate` deliberately refuses
+  // to touch it — CI is its only writer. Pointing at migrate there would send someone to a command
+  // that changes nothing while the warning never clears, so those are counted and named separately.
+  const behind = readable.filter((r) => r.from < engine && r.action !== 'ci-owned');
+  const behindCi = readable.filter((r) => r.from < engine && r.action === 'ci-owned');
+  // An object with no key yet is shape 1 by rule 1 — correct, not drifted. Read from the bytes
+  // (`stamped`), not from `action`: that fires on any byte difference, a re-indent included.
+  const unstamped = readable.filter((r) => !r.stamped).length;
+  const shapes = [...new Set(readable.map((r) => r.from))].sort((a, b) => a - b);
+  const on = shapes.length === 1 ? `shape ${shapes[0]}` : `shapes ${shapes.join(' and ')}`;
+  const detail = { shape: { engine, files: readable.map((r) => ({ file: r.file, shape: r.from, stamped: !!r.stamped })) } };
+
+  if (ahead.length) {
+    check(checks, id, 'shape', 'fail',
+      `${label} is on ${on}, the engine is on shape ${engine} — ${ahead.length} file(s) are newer than this yadflow`,
+      'upgrade yadflow (`npm i -g yadflow@latest`) — migrating would move those files BACKWARD and lose what the newer version wrote',
+      detail);
+    return;
+  }
+  if (behind.length || behindCi.length) {
+    const parts = [];
+    if (behind.length) parts.push(`${behind.length} file(s) are behind`);
+    if (behindCi.length) parts.push(`${behindCi.length} are CI-owned and behind`);
+    check(checks, id, 'shape', 'warn',
+      `${label} is on ${on}, the engine is on shape ${engine} — ${parts.join(', ')}`,
+      behind.length
+        ? 'run `yad migrate` to see what would change, then `yad migrate --apply` (each file is backed up first)'
+        : 'nothing to run — in verified mode CI owns these files and moves them on its next gate sync',
+      detail);
+    return;
+  }
+  // The suggestion lives in the MESSAGE, not the hint: runDoctor prints hints only for warn/fail, so a
+  // hint on a passing check would reach `--json` and never the person reading the terminal.
+  check(checks, id, 'shape', 'ok',
+    `${label} is on shape ${engine}, the engine is on shape ${engine}`
+      + (unstamped ? ` (${unstamped} file(s) do not record it yet — counted as shape 1; \`yad migrate --apply\` writes it in)` : ''),
+    '',
+    detail);
+}
+
+// `plan` is injectable for the same reason `yad migrate` takes an injectable migration list: while the
+// engine is on shape 1 nothing can be BEHIND it, so the warn branch — the one this section exists for —
+// is unreachable from a real project until the first real shape change lands. Tests supply a plan that
+// reaches it, which is how the drift report is proven before there is any drift to report.
+export function shapeChecks(checks, root, { plan: injected = null } = {}) {
+  if (!injected && !exists(path.join(root, PROJECT_FILES.hubConfig)) && !exists(path.join(root, PROJECT_FILES.version))) return;
+  let plan = injected;
+  if (!plan) {
+    try {
+      plan = planMigration(root);
+    } catch (e) {
+      // Say so rather than returning quietly. The failure modes here do not overlap with the other
+      // sections — an unreadable shard DIRECTORY, say, throws while `loadLedger` never looks at it —
+      // so a silent return would drop this whole section and let doctor print "all clear" over it.
+      check(checks, 'shape', 'shape', 'warn',
+        `could not read this project's file shapes: ${e.message}`,
+        'fix the path in the message, then re-run — until then neither doctor nor `yad migrate` can tell you what shape this project is on');
+      return;
+    }
+  }
+  const { rows, engine } = plan;
+  // A list ledger has no key to read and never will (rule 1's second half) — reporting it as a file
+  // that "does not record its shape" would be a permanent nag about something that is already correct.
+  const relevant = rows.filter((r) => r.action !== 'list');
+
+  shapeCheckFor(checks, 'shape', 'this project', relevant.filter((r) => scopeOf(r.file) === null), engine);
+  const epics = [...new Set(relevant.map((r) => scopeOf(r.file)).filter(Boolean))].sort();
+  for (const e of epics) {
+    shapeCheckFor(checks, `shape:${e}`, e, relevant.filter((r) => scopeOf(r.file) === e), engine);
+  }
+}
+
 // Phase 6 — feature-thread integrity. A change-epic must thread to a real parent and its denormalized
 // `thread` cache must equal the computed root; an open hotfix reconcile-debt is a warn (the next change
 // on that thread is blocked at the gate until it is paid). Pure reporting, like the other sections.
@@ -535,13 +644,15 @@ export function threadChecks(checks, root) {
   }
 }
 
-// Run every check section and return the diagnostic object without printing. This is the shared
-// core of `runDoctor` — the reporter (cli/report.mjs) consumes it to derive a *scrubbed* safe
-// subset (never the raw checks, which carry names + paths). Same shape `--json` prints.
+// Run every check section and return the diagnostic object without printing. The shared core of
+// `runDoctor`, and the same shape `--json` prints. Checks carry names and paths, so anything that
+// leaves the machine must scrub them — `yad report` does NOT consume this; it builds its own
+// allowlisted subset (cli/report.mjs `sanitizeContext`).
 export function collectDoctor(root) {
   const checks = [];
   envChecks(checks);
   projectChecks(checks, root);
+  shapeChecks(checks, root);
   epicChecks(checks, root);
   threadChecks(checks, root);
   const failed = checks.filter((x) => x.status === 'fail');

package/cli/gate.mjs

@@ -111,7 +111,7 @@ export function loadHub(root) {
 // Solo mode (a lone developer): waive the approval requirement — on GitHub you cannot approve your own
 // PR, so an approval gate would deadlock. The review PR/MR and its merge stay (CI runs on the PR; the
 // merge advances the step). Recorded per-project in hub.json by `yad setup`.
-const isSolo = (hub) => !!(hub && (hub.solo === true || hub.review_gate?.solo === true));
+export const isSolo = (hub) => !!(hub && (hub.solo === true || hub.review_gate?.solo === true));
 
 // Bridge mode: CI is the sole ledger writer, so `gate open`/`sync` stay hands-off. The predicate is
 // defined once in manifest.mjs (`isBridgeHub`) and shared with plan.mjs's wiring and the ledger
@@ -121,7 +121,7 @@ const isBridge = isBridgeHub;
 // requireEngagement (config `hub.review.requireEngagement`): when on, the predicate counts only
 // approvals carrying a verified engagement signal. Soft-off by default — a bare approve still counts
 // but is recorded `engagement: none` and draws the friendly nudge.
-const requireEngagement = (hub) => !!(hub && (hub.review?.requireEngagement === true));
+export const requireEngagement = (hub) => !!(hub && (hub.review?.requireEngagement === true));
 
 // Re-add this step's bridge approvals from the current platform state (drop+re-add => dismissals and
 // revocations vanish idempotently; manual approvals are never touched). Preserve the artifactHash a

package/cli/ledger.mjs

@@ -89,12 +89,21 @@ export const buildShardName = (e) => `${safe(e.story)}-${safe(e.task)}-${safe(e.
 
 // Read every shard object under `dir` (each file = ONE entry object). Sorted for determinism; a
 // corrupt/non-object shard is skipped (these ledgers are advisory evidence, never fatal).
+//
+// `schemaVersion` is dropped here on purpose. It describes the shape of a FILE, and a shard file has
+// one — but `fold()` copies these objects into the folded log's `ships`/`runs` array, where they stop
+// being files and become records inside one. Carrying the stamp across that boundary would bake a
+// per-record version into an append-only ledger for good: after SCHEMA_VERSION moves to 2, a
+// build-log.json stamped 2 would hold entries stamped 1 from shards folded today. The folded file
+// states its own shape; its rows do not have one.
 function readShardDir(dir) {
   if (!fs.existsSync(dir)) return [];
   const out = [];
   for (const name of fs.readdirSync(dir).filter((n) => n.endsWith('.json')).sort()) {
     const obj = readJSON(path.join(dir, name), null);
-    if (obj && typeof obj === 'object' && !Array.isArray(obj)) out.push({ name, obj });
+    if (!obj || typeof obj !== 'object' || Array.isArray(obj)) continue;
+    delete obj.schemaVersion;
+    out.push({ name, obj });
   }
   return out;
 }

package/cli/lib.mjs

@@ -1,6 +1,7 @@
 // Shared helpers for the `yad` CLI. Node >=18 built-ins only — no dependencies.
 import { createHash } from 'node:crypto';
 import { err } from './errors.mjs';
+import { SCHEMA_VERSION } from './manifest.mjs';
 import { spawnSync } from 'node:child_process';
 import * as readline from 'node:readline/promises';
 import { stdin as input, stdout as output } from 'node:process';
@@ -100,10 +101,50 @@ export function dirMatches(src, dest) {
   return files.every((rel) => sameContent(path.join(src, rel), path.join(dest, rel)));
 }
 
+// ---- file shape (schemaVersion) -----------------------------------------
+// Rule 1 of the change-safety rules (docs/roadmap-idea-1.md, Part 2): every file the engine writes
+// states its shape, and a file with no version counts as 1.
+//
+// Both halves live here, in the one JSON reader/writer pair, rather than at the ~30 call sites. Two
+// writers that disagreed about whether — or where — a file carries the stamp would flip its bytes back
+// and forth on every sync, which is the ledger-churn failure issue #163 exists to prevent.
+//
+// Two conditions gate it, and both are load-bearing:
+//
+//   * the path runs through a `.sdlc` directory. writeJSON is also how the CLI writes
+//     `.claude/settings.json` (cli/plan.mjs) and the per-user update cache (cli/update-notice.mjs).
+//     Stamping a file the engine does not own would be a bug, not a feature.
+//   * the value is a plain object. Four ledger kinds — approvals, comments, hub-prs and
+//     reconcile-debt — are top-level JSON arrays, which cannot carry a key. Rule 1's second half
+//     already covers them: no version means version 1. Wrapping them in an object would be a shape
+//     change, and shape changes wait for v4 (rule 7).
+// Matched against the file's own directory and its parent, NOT any ancestor. Every kind the engine
+// writes sits either directly in a `.sdlc/` (state.json, hub.json, …) or one level down in a shard
+// folder (build-log/, trust-log/, build-state/). Matching any ancestor instead would stamp every JSON
+// file in a project that merely happened to live somewhere under a directory called `.sdlc` — the
+// user's own .claude/settings.json included, which is exactly what this must never touch. A new kind
+// nested deeper than that has to be added here on purpose.
+const isPlainObject = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
+const underSdlcDir = (p) => {
+  const dir = path.dirname(path.resolve(p));
+  return path.basename(dir) === '.sdlc' || path.basename(path.dirname(dir)) === '.sdlc';
+};
+const carriesShape = (p, v) => isPlainObject(v) && underSdlcDir(p);
+
+// Always FIRST in the serialized object. A stamp that moved around between writers would change the
+// bytes without changing the meaning. An existing version is preserved, never forced back to 1, so a
+// file already on a newer shape survives being read and written by this release.
+const withShape = (p, v) =>
+  (carriesShape(p, v) ? { schemaVersion: v.schemaVersion ?? SCHEMA_VERSION, ...v } : v);
+
 export function readJSON(p, def = null) {
   try {
-    return JSON.parse(fs.readFileSync(p, 'utf8'));
+    // "Read old, write new" (rule 2): an unstamped file reads back as shape 1, so a caller never has
+    // to ask whether the file it just loaded predates the stamp.
+    return withShape(p, JSON.parse(fs.readFileSync(p, 'utf8')));
   } catch {
+    // The caller's own default is returned untouched: it is not a file, and stamping it would invent
+    // a shape for something that was never read from disk.
     return def;
   }
 }
@@ -113,7 +154,7 @@ export function readJSON(p, def = null) {
 export function readJSONStrict(p, def = null) {
   if (!fs.existsSync(p)) return def;
   try {
-    return JSON.parse(fs.readFileSync(p, 'utf8'));
+    return withShape(p, JSON.parse(fs.readFileSync(p, 'utf8')));
   } catch (e) {
     throw err('YAD-STATE-001', `corrupt JSON in ${p}: ${e.message}`, 'fix the file or restore it from git — never delete a ledger blindly');
   }
@@ -122,7 +163,7 @@ export function readJSONStrict(p, def = null) {
 // then rename over the target. A killed process can never leave a truncated ledger
 // file, and a failed rename never leaves a stray .tmp for `git add -A` to pick up.
 export function writeJSON(p, obj) {
-  const data = JSON.stringify(obj, null, 2) + '\n';
+  const data = JSON.stringify(withShape(p, obj), null, 2) + '\n';
   // Byte-identical content is not a write. The ledger writers are unconditional — they re-serialize
   // whether or not anything changed — so this keeps an unchanged sync from touching the file at all
   // (no mtime churn, nothing for a watcher or a `git add -A` to notice). A backstop, not the fix: the

package/cli/manifest.mjs

@@ -147,6 +147,22 @@ export const TESTING_PRIMARY = 'playwright';
 export const LEARNING_TOOLS = ['deeptutor'];
 export const LEARNING_PRIMARY = 'deeptutor';
 
+// The shape (schema version) every file the engine writes declares, as `"schemaVersion": 1`.
+//
+// Rule 1 of the change-safety rules (docs/roadmap-idea-1.md, Part 2): every file states its shape,
+// and a file with no version counts as 1.
+//
+// Today the stamp is written and read back, and nothing yet acts on it: raising this number would move
+// what new files say without upgrading existing ones. The two halves that make it usable are the next
+// tasks on the roadmap — `yad migrate` (E14), which moves a project from one shape to the next, and a
+// `yad doctor` report (E16) for a project whose files disagree with the engine. Do NOT raise this
+// number before both exist.
+//
+// Deliberately NOT the same thing as `VERSION` above. That is which release of the CLI you are
+// running and moves on every publish; this is what the files on disk look like and moves only when
+// their shape actually changes.
+export const SCHEMA_VERSION = 1;
+
 // Project-level files setup produces (used by `check` to spot missing setup).
 export const PROJECT_FILES = {
   reposRegistry: '.sdlc/repos.json',

package/cli/migrate.mjs

@@ -0,0 +1,317 @@
+// `yad migrate` — the one command that moves a project's files from one shape to the next.
+//
+// Rule 4 of the change-safety rules (docs/roadmap-idea-1.md, Part 2): one real upgrade command —
+// preview, back up, rewrite, report, safe to run twice.
+//
+// The shape of a file is its `schemaVersion` (cli/lib.mjs stamps it; manifest.mjs holds the engine's
+// current number). This command exists so that raising that number is survivable: every project can be
+// walked forward, in the open, with a copy of anything it rewrites left beside the original.
+//
+// Two things it deliberately is NOT:
+//
+//   * not automatic. Nothing here runs as a side effect of another command. An upgrade that happens
+//     without being asked for is how a tool loses the trust this one is selling.
+//   * not destructive by default. Running `yad migrate` prints what WOULD change and touches nothing.
+//     Only `--apply` writes, and only after copying each file it rewrites to `<file>.yad-orig` — the
+//     same backup mechanism `yad check --fix` already uses.
+import fs from 'node:fs';
+import path from 'node:path';
+
+import { c, exists, fail, hand, info, log, ok, readJSON, warn, writeJSON } from './lib.mjs';
+import { BACKUP_SUFFIX, epicFiles, isBridgeHub, MANAGED_LEDGER, PROJECT_FILES, SCHEMA_VERSION, VERSION } from './manifest.mjs';
+import { backupPathFor } from './plan.mjs';
+import { isValidEpicId } from './epic-state.mjs';
+
+// ---- the migration list --------------------------------------------------------------------
+// Ordered steps, each moving a file from one shape to the next. A step is applied to a file only when
+// the file's current shape equals its `from`; the list is walked ONCE per file, in order, so a step
+// can never be applied twice and a same-version step cannot spin.
+//
+// The first entry is 1 → 1 on purpose. It changes no field. It exists so the machinery — preview,
+// backup, rewrite, report, re-run — is exercised and tested by real use before there is any real shape
+// change to trust it with. When a genuine 1 → 2 lands, it appends here and this one stays.
+export const MIGRATIONS = [
+  {
+    from: 1,
+    to: 1,
+    title: 'baseline — every file states its shape',
+    apply: (obj) => obj,
+  },
+];
+
+// ---- reading a file's shape ---------------------------------------------------------------
+// Deliberately NOT readJSON: that reports an unstamped file as shape 1 (rule 2, "read old"), which is
+// the right answer for every other caller and the wrong one here. Migrate is the one place that has to
+// see the bytes as they are, so it can tell a file that already carries the key from one that does not.
+function readRaw(file) {
+  try {
+    return { ok: true, value: JSON.parse(fs.readFileSync(file, 'utf8')) };
+  } catch (e) {
+    return { ok: false, error: e.message };
+  }
+}
+
+const isPlainObject = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
+
+// A file's shape as recorded ON DISK. An object with no key is shape 1 by rule 1; so is an array,
+// which cannot carry a key at all.
+const shapeOf = (v) => (isPlainObject(v) && Number.isInteger(v.schemaVersion) ? v.schemaVersion : 1);
+
+// Walk the migration list once. Returns the migrated object, the shape it ended on, and which steps ran.
+function applyMigrations(obj, migrations) {
+  let out = obj;
+  let version = shapeOf(obj);
+  const applied = [];
+  for (const m of migrations) {
+    if (version !== m.from) continue;
+    out = m.apply({ ...out });
+    version = m.to;
+    applied.push(m.title);
+  }
+  return { obj: out, version, applied };
+}
+
+// ---- what the engine would write -----------------------------------------------------------
+// The single source of truth for "does this file need writing" is the bytes writeJSON would produce.
+// Comparing those against the bytes on disk means preview can never disagree with apply: whatever the
+// preview says would change is exactly what a write would change.
+const serialize = (obj) => JSON.stringify(obj, null, 2) + '\n';
+
+// The version the migration ENDED on wins, so the old key is removed before the new one is written.
+// Spreading the object over the stamp instead would let a file's existing `schemaVersion` shadow it:
+// the content would migrate while the number stayed put, so the file would be migrated again on every
+// later run — each one overwriting its own .yad-orig with already-migrated bytes until the original
+// was gone. (This is the one place that must NOT behave like lib.mjs's writeJSON, which preserves a
+// file's existing version on purpose. Here, changing it is the entire point.)
+const stamped = (obj, version) => {
+  const rest = { ...obj };
+  delete rest.schemaVersion;
+  return { schemaVersion: version, ...rest };
+};
+
+// ---- keeping the backups out of the commit ---------------------------------------------------
+// The `.yad-orig` copies land beside the files they back up, which for a ledger means inside the
+// tracked `epics/<epic>/.sdlc/` tree. Two commands stage that whole directory with `git add -A` —
+// `gate sync`'s merge-phase advance and `yad tidy up` — so telling the user "do not commit these"
+// would be advice they cannot act on: the next gate advance would sweep them into a `chore(gate)`
+// commit and push it to the default branch on its own.
+//
+// So ignore them instead, idempotently, the way `yad setup` already ignores the repomix packs. Only on
+// `--apply`, and only when a backup is actually about to be written — a preview still touches nothing.
+export const BACKUP_IGNORE_GLOB = `*${BACKUP_SUFFIX}`;
+export const BACKUP_IGNORE_BLOCK = [
+  '# Pre-migration copies written by `yad migrate --apply`. They are your safety net on disk,',
+  '# not history — the migrated file is what gets committed.',
+  BACKUP_IGNORE_GLOB,
+];
+
+export function ensureBackupsIgnored(root) {
+  const gi = path.join(root, '.gitignore');
+  const lines = exists(gi) ? fs.readFileSync(gi, 'utf8').split('\n') : [];
+  if (lines.some((l) => l.trim() === BACKUP_IGNORE_GLOB)) return false;
+  const body = lines.join('\n').replace(/\n*$/, '');
+  const prefix = body ? `${body}\n\n` : '';
+  fs.writeFileSync(gi, `${prefix}${BACKUP_IGNORE_BLOCK.join('\n')}\n`);
+  return true;
+}
+
+// ---- the file set --------------------------------------------------------------------------
+// Every JSON file this project's engine owns: the product-level files (including the hub's own
+// provenance ledger and each epic's docs-build cache, neither of which is in PROJECT_FILES/epicFiles),
+// then each epic's ledger and its three shard folders.
+//
+// A CONNECTED REPO's own `.sdlc/managed.json` is deliberately not here. It belongs to that repo, is
+// rewritten wholesale by that repo's `yad check --fix`, and migrating it from the hub would reach
+// across a boundary the rest of the CLI respects. The hub's copy is a different file and is included.
+function shardFiles(dir) {
+  if (!exists(dir)) return [];
+  return fs.readdirSync(dir).filter((n) => n.endsWith('.json')).sort().map((n) => path.join(dir, n));
+}
+
+export function projectJsonFiles(root) {
+  const files = [];
+  for (const rel of Object.values(PROJECT_FILES)) files.push(path.join(root, rel));
+  // The hub's own provenance record (cli/plan.mjs) — a stamped object under .sdlc/ like any other.
+  files.push(path.join(root, MANAGED_LEDGER));
+
+  const epicsDir = path.join(root, 'epics');
+  if (exists(epicsDir)) {
+    for (const epic of fs.readdirSync(epicsDir).sort()) {
+      if (!isValidEpicId(epic)) continue;
+      const epicDir = path.join(epicsDir, epic);
+      if (!fs.statSync(epicDir).isDirectory()) continue;
+      const f = epicFiles(epicDir);
+      files.push(f.state, f.approvals, f.comments, f.hubPrs, f.contractLock,
+        f.buildLog, f.trustLog, f.change, f.reconcileDebt);
+      files.push(...shardFiles(f.buildLogDir), ...shardFiles(f.trustLogDir), ...shardFiles(f.buildStateDir));
+      // The docs-build cache (cli/docs.mjs) lives in the same directory and is written by the engine,
+      // so it moves shape with everything else rather than being quietly left behind.
+      files.push(path.join(epicDir, '.sdlc', 'docs-build.json'));
+    }
+  }
+  return files.filter((f) => exists(f));
+}
+
+// ---- the plan ------------------------------------------------------------------------------
+// One row per file. `action` is what would happen, and it is the same value whether this runs as a
+// preview or as an apply:
+//
+//   stamp             an object with no schemaVersion — gets one, no field changes
+//   migrate           a real shape change, one or more steps applied
+//   unchanged         already on the engine's shape, bytes identical
+//   list              a top-level JSON array: shape 1 by rule 1, and it cannot carry a key
+//   ahead             the file's shape is NEWER than this engine — never touched, always reported
+//   ci-owned          a verified (bridge) hub's ledger file: CI is its only writer
+//   unreadable        does not parse — reported, never rewritten
+//
+// Each row also carries `stamped`: whether the file literally holds a `schemaVersion` key. That is a
+// fact about the bytes, read the same way for every branch, so a caller never has to infer it from
+// `action` — which would be wrong twice over: `stamp` fires whenever the serialized bytes differ for
+// ANY reason (a hand re-indent, say), and `ci-owned`/`ahead`/`list` short-circuit before the byte
+// comparison happens at all.
+export function planMigration(root, { migrations = MIGRATIONS } = {}) {
+  const hub = readJSON(path.join(root, PROJECT_FILES.hubConfig), null);
+  const bridge = isBridgeHub(hub);
+  // On a verified hub the ledger guard refuses a human commit to these, so rewriting them locally
+  // would produce a change that cannot be committed. Of the four the guard names, only state.json is
+  // an object; the rest are arrays and would be skipped anyway.
+  const ciOwned = new Set(['state.json', 'approvals.json', 'comments.json', 'hub-prs.json']);
+
+  const rows = [];
+  for (const file of projectJsonFiles(root)) {
+    const rel = path.relative(root, file);
+    const raw = readRaw(file);
+    if (!raw.ok) {
+      rows.push({ file: rel, from: null, to: null, action: 'unreadable', changes: false, stamped: false, detail: raw.error });
+      continue;
+    }
+    const from = shapeOf(raw.value);
+    // Read from the bytes, before any branch: a list can never carry the key, and every other kind
+    // either does or does not, whoever owns the file.
+    const isStamped = isPlainObject(raw.value) && Number.isInteger(raw.value.schemaVersion);
+    if (!isPlainObject(raw.value)) {
+      rows.push({ file: rel, from, to: from, action: 'list', changes: false, stamped: false });
+      continue;
+    }
+    if (from > SCHEMA_VERSION) {
+      rows.push({ file: rel, from, to: from, action: 'ahead', changes: false, stamped: isStamped });
+      continue;
+    }
+    if (bridge && ciOwned.has(path.basename(file))) {
+      rows.push({ file: rel, from, to: from, action: 'ci-owned', changes: false, stamped: isStamped });
+      continue;
+    }
+    const { obj, version, applied } = applyMigrations(raw.value, migrations);
+    const next = serialize(stamped(obj, version));
+    const current = fs.readFileSync(file, 'utf8');
+    const changes = next !== current;
+    const action = version !== from ? 'migrate' : (changes ? 'stamp' : 'unchanged');
+    // `steps` lists the migrations that actually moved the file's shape. The baseline 1 → 1 runs on
+    // every file by design and moves nothing, so naming it on every row would be noise reported as work.
+    rows.push({ file: rel, from, to: version, action, changes, stamped: isStamped, ...(version !== from ? { steps: applied } : {}) });
+  }
+  return { engine: SCHEMA_VERSION, bridge, rows };
+}
+
+// ---- the report ----------------------------------------------------------------------------
+const ACTION_NOTE = {
+  stamp: 'record its shape',
+  migrate: 'move to a new shape',
+  unchanged: 'already current',
+  list: 'a list — counts as shape 1',
+  ahead: 'newer than this engine',
+  'ci-owned': 'CI writes it — will be stamped by the next gate sync',
+  unreadable: 'does not parse — fix it by hand',
+};
+
+function printRows(rows) {
+  const width = Math.max(...rows.map((r) => r.file.length), 4);
+  for (const r of rows) {
+    const shape = r.from === null ? '?' : (r.from === r.to ? `shape ${r.from}` : `shape ${r.from} → ${r.to}`);
+    const line = `  ${r.file.padEnd(width)}  ${shape.padEnd(14)} ${c.dim(ACTION_NOTE[r.action] ?? r.action)}`;
+    if (r.action === 'unreadable' || r.action === 'ahead') fail(line.trim());
+    else log(line);
+  }
+}
+
+// ---- the command ---------------------------------------------------------------------------
+// `.sdlc/cli-version.json` is migrated as an ordinary row, like every other project file — it is in
+// PROJECT_FILES, so it is previewed, backed up and stamped along with the rest. Nothing here writes it
+// a second time to record that a migration happened: a separate side-write would sit outside the plan,
+// which means no backup, no row in the report, and a preview that under-reports what an apply does. It
+// would also be free to overwrite the one file the same run had just declared corrupt or newer than
+// this engine. What shape a file is in is recorded in that file, which is the whole point of rule 1.
+export async function runMigrate(root, { apply = false, json = false } = {}, { migrations = MIGRATIONS } = {}) {
+  if (!exists(path.join(root, PROJECT_FILES.version)) && !exists(path.join(root, PROJECT_FILES.hubConfig))) {
+    const message = 'no yad project here (.sdlc/ not initialised)';
+    if (json) { log(JSON.stringify({ version: VERSION, ok: false, error: message }, null, 2)); }
+    else { fail(message); hand('run `yad setup` to start one'); }
+    process.exitCode = 1;
+    return { ok: false };
+  }
+
+  const plan = planMigration(root, { migrations });
+  const pending = plan.rows.filter((r) => r.changes);
+  const blocked = plan.rows.filter((r) => r.action === 'ahead' || r.action === 'unreadable');
+  const written = [];
+
+  let ignored = false;
+  if (apply) {
+    // Before the first backup exists, not after — otherwise a gate advance racing this run could stage
+    // one. A no-op when the line is already there.
+    if (pending.length) ignored = ensureBackupsIgnored(root);
+    for (const row of pending) {
+      const file = path.join(root, row.file);
+      const raw = readRaw(file);
+      if (!raw.ok) continue; // re-read defensively; an unreadable file is never in `pending` anyway
+      // Back up first, always. Unlike the wiring copies in plan.mjs — which skip the backup when the
+      // file's bytes are provably ours — a ledger has no provenance record, so there is nothing to
+      // prove and the copy is unconditional.
+      fs.copyFileSync(file, backupPathFor(file));
+      const { obj, version } = applyMigrations(raw.value, migrations);
+      writeJSON(file, stamped(obj, version));
+      written.push(row.file);
+    }
+  }
+
+  if (json) {
+    log(JSON.stringify({
+      version: VERSION,
+      ok: blocked.length === 0,
+      engine: plan.engine,
+      applied: apply,
+      bridge: plan.bridge,
+      changed: apply ? written : pending.map((r) => r.file),
+      ...(ignored ? { gitignored: BACKUP_IGNORE_GLOB } : {}),
+      rows: plan.rows,
+    }, null, 2));
+  } else {
+    log(c.bold(`\nyad migrate  ${c.dim(`shape ${plan.engine}`)}`));
+    log(c.dim(`target: ${root}\n`));
+    printRows(plan.rows);
+    log('');
+    if (!pending.length) {
+      ok(`nothing to do — this project is already on shape ${plan.engine}`);
+    } else if (apply) {
+      ok(`${written.length} file(s) updated — a copy of each is beside it as <file>${BACKUP_SUFFIX}`);
+      info('re-run `yad migrate` to confirm there is nothing left to do');
+      // These land inside the tracked .sdlc/ tree, so a `git add -A` would sweep them into the commit
+      // alongside the migration itself. Say so rather than editing a .gitignore the project owns.
+      info(`the ${BACKUP_SUFFIX} copies are yours to keep or delete${ignored ? ` — .gitignore now excludes ${BACKUP_IGNORE_GLOB}, so they stay out of the ledger commit` : ''}`);
+    } else {
+      info(`${pending.length} file(s) would change — nothing has been written`);
+      hand('run `yad migrate --apply` to make the change (each file is backed up first)');
+    }
+    if (plan.bridge && plan.rows.some((r) => r.action === 'ci-owned')) {
+      info('this project is in verified mode: CI owns some ledger files and stamps them on its next gate sync');
+    }
+    for (const r of blocked) {
+      if (r.action === 'ahead') hand(`${r.file} was written by a newer yadflow — upgrade with \`npm i -g ${'yadflow'}\` rather than migrating`);
+      if (r.action === 'unreadable') hand(`${r.file} does not parse — restore it from git before migrating`);
+    }
+    if (blocked.length) warn('some files were left untouched — see above');
+  }
+
+  if (blocked.length) process.exitCode = 1;
+  return { ok: blocked.length === 0, rows: plan.rows, written };
+}

package/cli/update-notice.mjs

@@ -124,18 +124,48 @@ export function shouldSuppress({ env = process.env, pkgRoot = PKG_ROOT } = {}) {
 // ---- banner -------------------------------------------------------------
 // `yad update` is the necessary second half: upgrading the global CLI leaves this project's installed
 // yad-* skills stamped at the old version in .sdlc/cli-version.json, which `yad doctor` then flags.
+// A MAJOR jump is the only upgrade that may change the shape of the files in a project (rule 7:
+// file-shape changes wait for a major, and ship with their migration guide). Everything else is
+// additive by policy, so the plain banner is right for it.
+//
+// This is the last moment the engine can speak before someone types the upgrade command, so it says
+// the one thing that makes a major safe: look at what it would change to YOUR project first.
+//
+// It has to be `npx yadflow@<new> migrate`, NOT the installed `yad migrate`. A migration list ships
+// inside the engine that introduces it (cli/migrate.mjs), so the copy already installed knows only its
+// own steps — on 3.x that is the 1 -> 1 baseline, which reports "nothing would change" for every
+// project. Advising the installed binary would hand the reader false reassurance about exactly the
+// upgrade this warning exists for. `npx` runs the NEW engine against the current project without
+// installing anything, and a preview writes nothing either way.
+const crossesMajor = (current, latest) => {
+  const cur = parseVersion(current);
+  const l = parseVersion(latest);
+  return !!cur && !!l && l.major > cur.major;
+};
+
 export function formatBanner(current, latest) {
   // Normalize so a `v`-prefixed input can never produce `.../releases/tag/vv3.11.0`. Callers only
   // reach here after isNewer(), so parseVersion has already accepted both — the ?? is belt and braces.
   const v = normalizeVersion(latest) ?? latest;
   const url = `https://github.com/${UPSTREAM_REPO}/releases/tag/v${v}`;
-  return [
+  const lines = [
     '',
     `  ${c.yellow('!')} ${c.bold(`${PKG_NAME} update available`)} — ${c.dim(current)} → ${c.green(v)}`,
     `    ${c.dim('Changelog:')}  ${url}`,
+  ];
+  if (crossesMajor(current, v)) {
+    lines.push(
+      `    ${c.yellow('Major:')}      ${c.bold('this may change the shape of your project files.')}`,
+      `                ${c.dim('Preview it first, with the new engine — this writes nothing:')}`,
+      `                ${c.cyan(`npx ${PKG_NAME}@${v} migrate`)}`,
+      `                ${c.dim('Then upgrade, and run')} ${c.cyan('yad migrate --apply')} ${c.dim('(backs up every file it rewrites)')}`,
+    );
+  }
+  lines.push(
     `    ${c.dim('Update:')}     ${c.cyan(`npm install ${PKG_NAME} -g`)}`,
     `    ${c.dim('Then:')}       ${c.cyan('yad update')}   ${c.dim("(re-sync this project's yad-* skills)")}`,
-  ].join('\n');
+  );
+  return lines.join('\n');
 }
 
 // ---- orchestrator -------------------------------------------------------

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "yadflow",
-  "version": "3.17.2",
+  "version": "3.18.0",
   "description": "Yadflow — the gated, team, multi-repo SDLC: author → review → build with a PR-driven review gate and a zero-dependency `yad` CLI (setup, gate, commit, open-pr, ship, repo, thread, reconcile). A BMAD module + 38 yad-* skills.",
   "type": "module",
   "author": "AbdelRahman Nasr",
@@ -22,6 +22,9 @@
     "!cli/test.mjs",
     "!cli/test-checks.mjs",
     "!cli/test-threads.mjs",
+    "!cli/test-golden.mjs",
+    "!cli/test-migrate.mjs",
+    "!cli/fixtures",
     "skills/",
     "README.md",
     "LICENSE",
@@ -37,9 +40,10 @@
   "scripts": {
     "yad": "node bin/yad.mjs",
     "lint": "eslint cli bin",
-    "test": "node --test cli/test.mjs cli/test-checks.mjs cli/test-threads.mjs",
+    "test": "node --test cli/test.mjs cli/test-checks.mjs cli/test-threads.mjs cli/test-golden.mjs cli/test-migrate.mjs",
     "test:e2e": "bash test/e2e/run.sh",
-    "coverage": "node --test --experimental-test-coverage --test-coverage-exclude='cli/test*.mjs' --test-coverage-lines=70 --test-coverage-branches=70 cli/test.mjs cli/test-checks.mjs cli/test-threads.mjs",
+    "release-check": "bash scripts/release-check.sh",
+    "coverage": "node --test --experimental-test-coverage --test-coverage-exclude='cli/test*.mjs' --test-coverage-lines=70 --test-coverage-branches=70 cli/test.mjs cli/test-checks.mjs cli/test-threads.mjs cli/test-golden.mjs cli/test-migrate.mjs",
     "diagrams": "npx -y @mermaid-js/mermaid-cli -i docs/diagrams/sdlc-overview.mmd -o docs/diagrams/sdlc-overview.svg -b transparent && npx -y @mermaid-js/mermaid-cli -i docs/diagrams/review-loop.mmd -o docs/diagrams/review-loop.svg -b transparent",
     "prepublishOnly": "npm test"
   },

package/skills/yad-analysis/SKILL.md

@@ -108,6 +108,7 @@ authoring step **locked**. Use this exact shape (see `references/state-schema.md
 
 ```json
 {
+  "schemaVersion": 1,
   "epicId": "EP-<slug>",
   "createdAt": "<YYYY-MM-DD>",
   "currentStep": "analysis-review",

package/skills/yad-change/references/triage.md

@@ -28,6 +28,7 @@ re-authors stories+test-cases:
 
 ```json
 {
+  "schemaVersion": 1,
   "epicId": "EP-<slug>", "createdAt": "<today>", "currentStep": "stories",
   "steps": [
     { "id": "epic",                "type": "author",         "artifact": "epic.md",         "assistance": "review", "automation": "human_approve", "locked": true, "status": "done", "inherited": true, "inheritedFrom": "EP-<genesis>", "boundHash": "sha256:…", "risk_tags": [] },

package/skills/yad-epic/SKILL.md

@@ -128,6 +128,7 @@ steps **locked**. Use this exact shape (see `references/state-schema.md`):
 
 ```json
 {
+  "schemaVersion": 1,
   "epicId": "EP-<slug>",
   "createdAt": "<YYYY-MM-DD>",
   "currentStep": "epic-review",

package/skills/yad-epic/references/state-schema.md

@@ -3,6 +3,39 @@
 All SDLC state lives in plain files under `epics/EP-<slug>/.sdlc/` (build plan §1: "All state lives
 in files on disk. Nothing hidden."). No database, no browser storage.
 
+## Every file states its shape — `schemaVersion`
+
+Each JSON **object** the CLI writes under a `.sdlc/` directory carries `"schemaVersion": 1` as its
+first key. It says what shape the file is in, so a future release can recognise an older file and
+upgrade it instead of guessing.
+
+**When you author one of these files by hand, include the key**, exactly as the examples below show.
+Several kinds here — `state.json` on the seeding path, `change.json`, `contract-lock.json`,
+`build-state/<story-id>.json`, `design-links.json`, `test-links.json` — are written by skills, not by
+the CLI. `state.json` is the one both write, so omitting the key there makes the file flip between two
+byte forms: the skill writes it without, the next `yad gate sync` adds it back. Write it and there is
+nothing to flip.
+
+```json
+{
+  "schemaVersion": 1,
+  "epicId": "EP-checkout"
+}
+```
+
+Three rules go with it, and they are permanent (`docs/roadmap-idea-1.md`, Part 2):
+
+1. **A file with no version counts as version 1.** Nothing has to be rewritten to be readable. Files
+   written before the stamp existed are read as shape 1, and get the key the next time the engine
+   writes them.
+2. **The four list files never carry it.** `approvals.json`, `comments.json`, `hub-prs.json` and
+   `reconcile-debt.json` are JSON arrays at the top level, and an array cannot hold a key. Rule 1
+   covers them: no version means version 1.
+3. **`schemaVersion` is not the CLI version.** `.sdlc/cli-version.json` records which release of the
+   `yad` CLI set the project up, and changes on every release. `schemaVersion` describes the file's
+   shape and changes only when that shape really changes — which is rare, and always paired with a
+   `yad migrate` step that moves existing projects onto it.
+
 ## `state.json`
 The per-epic state machine.
 

package/skills/yad-stub/SKILL.md

@@ -93,6 +93,7 @@ into normal authoring with zero re-seeding.
 
 ```json
 {
+  "schemaVersion": 1,
   "epicId": "EP-<slug>",
   "createdAt": "<YYYY-MM-DD>",
   "kind": "stub",