yadflow 3.15.5 → 3.16.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +5 -0
- package/bin/yad.mjs +13 -5
- package/cli/checkpoint.mjs +59 -2
- package/cli/errors.mjs +1 -0
- package/cli/ledger.mjs +121 -35
- package/cli/manifest.mjs +11 -0
- package/cli/plan.mjs +85 -5
- package/cli/reconcile.mjs +40 -11
- package/cli/setup.mjs +18 -3
- package/cli/usage.mjs +6 -0
- package/package.json +1 -1
- package/skills/yad-checks/references/check-gates.md +12 -0
- package/skills/yad-engineer-review/references/ship-and-record.md +30 -2
- package/skills/yad-pr-template/SKILL.md +12 -0
- package/skills/yad-pr-template/templates/checks/pr-template.sh +17 -0
- package/skills/yad-pr-template/templates/gitlab/merge_request_templates/Default.md +4 -0
- package/skills/yad-pr-template/templates/hub/gitlab/merge_request_templates/Default.md +12 -9
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,25 @@
|
|
|
1
|
+
## [3.16.1](https://github.com/abdelrahmannasr/yadflow/compare/v3.16.0...v3.16.1) (2026-08-12)
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
### Bug Fixes
|
|
5
|
+
|
|
6
|
+
* **checkpoint:** guard --retro-ship per repo so a multi-repo story can be fully recorded ([d6d2fae](https://github.com/abdelrahmannasr/yadflow/commit/d6d2faeea91c0d59a00a532c0605f76c5b77eace)), closes [#166](https://github.com/abdelrahmannasr/yadflow/issues/166)
|
|
7
|
+
* **checkpoint:** validate the retro-ship repo instead of relying on the duplicate guard ([f1e085e](https://github.com/abdelrahmannasr/yadflow/commit/f1e085e89e4eb10fbf853647ab10c1912237daf8)), closes [#166](https://github.com/abdelrahmannasr/yadflow/issues/166) [#166](https://github.com/abdelrahmannasr/yadflow/issues/166) [#166](https://github.com/abdelrahmannasr/yadflow/issues/166)
|
|
8
|
+
* **ledger:** hold an exclusive lock across a ledger read-modify-write ([45b849a](https://github.com/abdelrahmannasr/yadflow/commit/45b849a7755a5bc58bc646e627218170a54b31bc)), closes [#166](https://github.com/abdelrahmannasr/yadflow/issues/166)
|
|
9
|
+
|
|
10
|
+
# [3.16.0](https://github.com/abdelrahmannasr/yadflow/compare/v3.15.5...v3.16.0) (2026-08-12)
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
### Bug Fixes
|
|
14
|
+
|
|
15
|
+
* **pr-template:** name GitLab's 2700-character description truncation ([475e2b7](https://github.com/abdelrahmannasr/yadflow/commit/475e2b7db8f0d24909ce5c874122f18dc6508985)), closes [#164](https://github.com/abdelrahmannasr/yadflow/issues/164)
|
|
16
|
+
* **update:** reject an unusable provenance record instead of ignoring it ([5107381](https://github.com/abdelrahmannasr/yadflow/commit/5107381ae2c08b261fddda76433cfc2a3fe3ea46)), closes [#188](https://github.com/abdelrahmannasr/yadflow/issues/188) [#164](https://github.com/abdelrahmannasr/yadflow/issues/164)
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
### Features
|
|
20
|
+
|
|
21
|
+
* **update:** never silently overwrite a locally modified managed file ([28d6ee4](https://github.com/abdelrahmannasr/yadflow/commit/28d6ee4c1af250415c3ad50d999a263ebe91cd83)), closes [#164](https://github.com/abdelrahmannasr/yadflow/issues/164)
|
|
22
|
+
|
|
1
23
|
## [3.15.5](https://github.com/abdelrahmannasr/yadflow/compare/v3.15.4...v3.15.5) (2026-08-12)
|
|
2
24
|
|
|
3
25
|
|
package/README.md
CHANGED
|
@@ -65,6 +65,11 @@ Every step stops at a gate until a human approves. New here? **Walk it lesson-by
|
|
|
65
65
|
|
|
66
66
|
Running `yad` tells you when a new release is out — upgrade with `npm install yadflow -g`, then
|
|
67
67
|
`yad update` to re-sync this project's skills. See [staying up to date](docs/CLI.md#staying-up-to-date).
|
|
68
|
+
An update rewrites the files yad manages (gate scripts, CI, PR/MR templates) — but not one **you**
|
|
69
|
+
edited: yad records the sha of every file it writes, so an edit to one is reported as `modified` and
|
|
70
|
+
left alone. A file it has no record of (an install predating that record) is still replaced, but only
|
|
71
|
+
after saving a `.yad-orig` backup
|
|
72
|
+
([managed files](docs/CLI.md#managed-files-what-yad-owns-and-what-you-edited)).
|
|
68
73
|
|
|
69
74
|
## What `npx yadflow setup` installs
|
|
70
75
|
|
package/bin/yad.mjs
CHANGED
|
@@ -30,11 +30,15 @@ ${c.bold('Setup & maintenance')}
|
|
|
30
30
|
yad setup Guided first-run setup (profile interview, install, connect & wire repos)
|
|
31
31
|
profile flags: --solo | --team <n>, --greenfield | --brownfield,
|
|
32
32
|
--monorepo | --separate, --tools (configure design/testing/learning now)
|
|
33
|
-
yad check Report what is missing / drifted / stale / legacy (read-only)
|
|
33
|
+
yad check Report what is missing / drifted / modified / stale / legacy (read-only)
|
|
34
34
|
yad check --fix Reconcile: fill what is missing, update what changed
|
|
35
35
|
yad update Apply drift only (alias for: check --fix --scope=changed);
|
|
36
36
|
installs newly-added skills, updates changed skills + gate scripts,
|
|
37
|
-
and migrates pre-2.0 sdlc-* installs to the yad-* names
|
|
37
|
+
and migrates pre-2.0 sdlc-* installs to the yad-* names.
|
|
38
|
+
A managed file whose edit yad can prove (its recorded sha) is
|
|
39
|
+
reported 'modified' and left alone; --overwrite-local replaces
|
|
40
|
+
it. Anything else it cannot account for is replaced only after
|
|
41
|
+
a <file>.yad-orig backup
|
|
38
42
|
yad update --push Also commit each repo's applied changes and push them straight to the
|
|
39
43
|
default branch of the hub + every connected repo (a chore(yad-update)
|
|
40
44
|
commit; no PR — the push-on-main yad-update-guard runs verified-commits
|
|
@@ -111,7 +115,8 @@ ${c.bold('Build helpers')}
|
|
|
111
115
|
yad checkpoint --retro-ship <epic>/<story> --repo <r>
|
|
112
116
|
Record a retroactive build-log ship for a PRE-TRACKING story
|
|
113
117
|
(merged before ledger tracking), then carry its status: shipped
|
|
114
|
-
flip in the same commit (--merge-commit <sha>, --task <t> opt.)
|
|
118
|
+
flip in the same commit (--merge-commit <sha>, --task <t> opt.);
|
|
119
|
+
one repo per run — re-run per --repo for a multi-repo story
|
|
115
120
|
yad tidy up [<epic>] [--push] Fold FINISHED back-half shards (a shipped story's
|
|
116
121
|
trust-log/build-log entries) back into the single folded
|
|
117
122
|
ledger, as one chore(hub) commit — the manual "pack it up"
|
|
@@ -159,6 +164,8 @@ ${c.bold('Options')}
|
|
|
159
164
|
--no-push gate ci: commit the ledger but do not push
|
|
160
165
|
--push check --fix / update: commit + push applied changes to the default branch
|
|
161
166
|
--allow-branch check --fix --push / update --push / repo refresh --push: allow committing on a non-default branch
|
|
167
|
+
--overwrite-local check --fix / update: replace managed files reported as 'modified'
|
|
168
|
+
(a <file>.yad-orig backup is written first)
|
|
162
169
|
-h, --help Show this help
|
|
163
170
|
-v, --version Print version
|
|
164
171
|
|
|
@@ -178,6 +185,7 @@ function parseArgs(argv) {
|
|
|
178
185
|
else if (a === '--no-push') o.noPush = true;
|
|
179
186
|
else if (a === '--push') o.push = true;
|
|
180
187
|
else if (a === '--allow-branch') o.allowBranch = true;
|
|
188
|
+
else if (a === '--overwrite-local') o.overwriteLocal = true;
|
|
181
189
|
else if (a === '--merged') o.merged = true;
|
|
182
190
|
else if (a === '--overview') o.overview = true;
|
|
183
191
|
// `--check` is a bare boolean for `docs sync --check`, but takes a value for
|
|
@@ -232,10 +240,10 @@ async function main() {
|
|
|
232
240
|
});
|
|
233
241
|
break;
|
|
234
242
|
case 'check':
|
|
235
|
-
await reconcile(o.dir, { fix: o.fix, scope: o.scope, force: o.force, push: o.push, allowBranch: o.allowBranch, today });
|
|
243
|
+
await reconcile(o.dir, { fix: o.fix, scope: o.scope, force: o.force, push: o.push, allowBranch: o.allowBranch, overwriteLocal: o.overwriteLocal, today });
|
|
236
244
|
break;
|
|
237
245
|
case 'update':
|
|
238
|
-
await reconcile(o.dir, { fix: true, scope: 'changed', force: o.force, push: o.push, allowBranch: o.allowBranch, today });
|
|
246
|
+
await reconcile(o.dir, { fix: true, scope: 'changed', force: o.force, push: o.push, allowBranch: o.allowBranch, overwriteLocal: o.overwriteLocal, today });
|
|
239
247
|
break;
|
|
240
248
|
case 'doctor':
|
|
241
249
|
await runDoctor(o.dir, { json: o.json });
|
package/cli/checkpoint.mjs
CHANGED
|
@@ -22,7 +22,7 @@
|
|
|
22
22
|
// marker would strand the PR's required checks.
|
|
23
23
|
import fs from 'node:fs';
|
|
24
24
|
import path from 'node:path';
|
|
25
|
-
import { c, log, ok, info, fail, hand, exists, pushWithRebase } from './lib.mjs';
|
|
25
|
+
import { c, log, ok, info, fail, hand, exists, readJSON, pushWithRebase } from './lib.mjs';
|
|
26
26
|
import { PROJECT_FILES } from './manifest.mjs';
|
|
27
27
|
import { loadHub } from './gate.mjs';
|
|
28
28
|
import { resolveCommitterLogin } from './platform.mjs';
|
|
@@ -171,6 +171,26 @@ export function stagedStoryIsStatusOnly(git, file) {
|
|
|
171
171
|
// (with a printed reason) aborts the commit; `file` is the shard just written, so a dry run can delete it
|
|
172
172
|
// and leave no side effect. Does NOT author the story frontmatter — it only supplies the missing
|
|
173
173
|
// evidence, and the human must have ALREADY flipped `status:` to a back-half value in the working tree.
|
|
174
|
+
//
|
|
175
|
+
// ONE repo per run (#166). A story that shipped in several repos is backfilled by re-running with each
|
|
176
|
+
// `--repo`; the second run finds the flip already committed, so it lands only the new ship shard.
|
|
177
|
+
//
|
|
178
|
+
// The repo names a retro ship MAY carry — the story's own `repos:` frontmatter (its statement of where
|
|
179
|
+
// it was implemented), else the hub's connected-repo registry as the project-wide fallback. Used to
|
|
180
|
+
// reject a typo'd/mis-cased/invented `--repo` (#166 review): once the duplicate guard is per repo, a
|
|
181
|
+
// wrong name no longer collides with anything, so nothing else would stop it from committing a
|
|
182
|
+
// `retroactive: true` ship for a repo that never existed. Returns `{ names: [], source: 'none' }` when
|
|
183
|
+
// neither declares anything — a legacy story with no metadata is still backfillable, never blocked on
|
|
184
|
+
// a missing list.
|
|
185
|
+
export function retroShipRepos(root, storyFile) {
|
|
186
|
+
const declared = readFrontmatter(storyFile).repos;
|
|
187
|
+
const story = (Array.isArray(declared) ? declared : declared ? [declared] : []).map(String).filter(Boolean);
|
|
188
|
+
if (story.length) return { names: story, source: 'story' };
|
|
189
|
+
const reg = readJSON(path.join(root, PROJECT_FILES.reposRegistry), { repos: [] });
|
|
190
|
+
const names = (Array.isArray(reg?.repos) ? reg.repos : []).map((r) => r?.name).filter(Boolean);
|
|
191
|
+
return { names, source: names.length ? 'registry' : 'none' };
|
|
192
|
+
}
|
|
193
|
+
|
|
174
194
|
export function recordRetroShip(root, { epic, story, repo, task, mergeCommit, today }) {
|
|
175
195
|
if (!epic || !story) { fail('--retro-ship needs <epic>/<story> (e.g. --retro-ship EP-foo/EP-foo-S01)'); return { ok: false }; }
|
|
176
196
|
if (!repo) { fail('--retro-ship needs --repo <name> (the repo the story shipped in)'); return { ok: false }; }
|
|
@@ -189,14 +209,51 @@ export function recordRetroShip(root, { epic, story, repo, task, mergeCommit, to
|
|
|
189
209
|
return { ok: false };
|
|
190
210
|
}
|
|
191
211
|
|
|
212
|
+
// A ship is permanent audit evidence, so the repo it names must be one the story could have shipped
|
|
213
|
+
// in — an unrecognized `--repo` is a typo, not a discovery. Skipped only when nothing declares any
|
|
214
|
+
// repo (`source: 'none'`), so a legacy story with no metadata is never blocked.
|
|
215
|
+
const { names, source } = retroShipRepos(root, storyFile);
|
|
216
|
+
if (names.length && !names.includes(repo)) {
|
|
217
|
+
fail(`${repo} is not a repo ${source === 'story' ? `${story} declares` : 'connected to this hub'} — a retroactive ship must name a real repo, never invent one`);
|
|
218
|
+
hand(`known: ${names.join(', ')} (names are case-sensitive)`);
|
|
219
|
+
return { ok: false };
|
|
220
|
+
}
|
|
221
|
+
// Which of the story's OWN declared repos still lack evidence — read BEFORE the write so both the
|
|
222
|
+
// refusal and the success path can report honestly how much of a multi-repo backfill is left. Only
|
|
223
|
+
// the story's own list is used: the hub registry lists every connected repo, which says nothing
|
|
224
|
+
// about where THIS story shipped.
|
|
225
|
+
const remaining = () => {
|
|
226
|
+
if (source !== 'story') return [];
|
|
227
|
+
let recorded;
|
|
228
|
+
try { recorded = new Set(readShips(epicDir).filter((s) => s.story === story).map((s) => s.repo)); }
|
|
229
|
+
catch { return []; } // corrupt build-log — the write below reports it; don't guess at progress
|
|
230
|
+
return names.filter((n) => !recorded.has(n) && n !== repo);
|
|
231
|
+
};
|
|
232
|
+
const left = remaining();
|
|
233
|
+
|
|
192
234
|
let res;
|
|
193
235
|
try { res = writeRetroShip(epicDir, { story, repo, task, mergeCommit, shippedAt: today }); }
|
|
194
236
|
catch (e) { fail(`could not record retroactive ship — ${e.message}`); return { ok: false }; }
|
|
195
237
|
if (!res.written) {
|
|
196
|
-
|
|
238
|
+
if (res.reason === 'collision') {
|
|
239
|
+
// Distinct names, ONE shard file (`buildShardName` sanitizes each component) — recording this one
|
|
240
|
+
// would overwrite the other repo's ship record, so it is refused rather than silently clobbered.
|
|
241
|
+
fail(`${repo} cannot be recorded: it shares a build-log shard name with ${res.repo ? `the already-recorded ${res.repo}` : `an existing shard (${path.basename(res.file)})`}`);
|
|
242
|
+
hand('recording it would overwrite that ship record — rename one of the repos in the registry, or record this ship through the normal ship/checkpoint flow');
|
|
243
|
+
return { ok: false };
|
|
244
|
+
}
|
|
245
|
+
// Per REPO, not per story (#166) — so the message names the repo. The follow-up hint names the
|
|
246
|
+
// story's OWN still-unrecorded repos; with none left there is nothing to re-run, and inventing a
|
|
247
|
+
// `--repo <other>` at that point would fabricate a ship for a repo the story never shipped in.
|
|
248
|
+
fail(`${story} already has a build-log ship in ${repo} — it is not pre-tracking there; use the normal ship/checkpoint flow`);
|
|
249
|
+
if (left.length) hand(`still unrecorded for ${story}: ${left.join(', ')} — re-run with \`--repo <name>\` for each`);
|
|
197
250
|
return { ok: false };
|
|
198
251
|
}
|
|
199
252
|
ok(`recorded retroactive ship for ${story} (${repo})${mergeCommit ? ` @ ${mergeCommit}` : ''}`);
|
|
253
|
+
// A multi-repo story is only half-reconciled until every declared repo has evidence, and nothing
|
|
254
|
+
// downstream reports the gap (the story already reads `shipped`) — so say it here, while the
|
|
255
|
+
// operator is running the backfill.
|
|
256
|
+
if (left.length) hand(`${story} declares ${names.length} repos — still unrecorded: ${left.join(', ')}; re-run with \`--repo <name>\` for each`);
|
|
200
257
|
return { ok: true, file: res.file };
|
|
201
258
|
}
|
|
202
259
|
|
package/cli/errors.mjs
CHANGED
|
@@ -21,6 +21,7 @@ export const CODES = {
|
|
|
21
21
|
'YAD-STATE-003': 'a registered repo path is missing or not a git repository',
|
|
22
22
|
'YAD-STATE-004': 'an epic step cannot be skipped / un-skipped in its current state',
|
|
23
23
|
'YAD-STATE-005': 'an authoring step is stranded behind its completed review gate',
|
|
24
|
+
'YAD-STATE-006': 'a back-half ledger is locked by another yad process that is writing it',
|
|
24
25
|
'YAD-CFG-001': 'hub.json names an unknown platform (expected github, gitlab, or null)',
|
|
25
26
|
'YAD-CFG-002': 'design.json names an unknown design tool (expected one of config.yaml design.tools, or none)',
|
|
26
27
|
'YAD-CFG-003': 'testing.json names an unknown testing tool (expected one of config.yaml testing.tools, or none)',
|
package/cli/ledger.mjs
CHANGED
|
@@ -9,10 +9,69 @@
|
|
|
9
9
|
//
|
|
10
10
|
// A legacy epic that only has the folded file still reads correctly (no shards to union) → zero
|
|
11
11
|
// migration; new writes simply go to shards.
|
|
12
|
+
//
|
|
13
|
+
// Shards remove the conflict BETWEEN entries; they do not make a single read-modify-write atomic, so
|
|
14
|
+
// every writer that decides something from the ledger before writing takes an exclusive lock first
|
|
15
|
+
// (see "ledger locking" below).
|
|
12
16
|
import fs from 'node:fs';
|
|
13
17
|
import path from 'node:path';
|
|
14
18
|
import { readJSON, readJSONStrict, writeJSON } from './lib.mjs';
|
|
15
19
|
import { epicFiles } from './manifest.mjs';
|
|
20
|
+
import { err } from './errors.mjs';
|
|
21
|
+
|
|
22
|
+
// ---- ledger locking -------------------------------------------------------------------------------
|
|
23
|
+
// Shards make concurrent writers conflict-free ACROSS entries, but a writer that READS THEN WRITES is
|
|
24
|
+
// only safe if nothing slips in between. `writeRetroShip` decides "no ship for this (story, repo) yet"
|
|
25
|
+
// and then writes; `updateShip` finds a ship and then rewrites it; a fold reads shards, merges them,
|
|
26
|
+
// then deletes them. Two `--retro-ship` runs with different `--task` values both read "no ship", both
|
|
27
|
+
// write, and the duplicate guard is defeated — two shards for one (story, repo). So every
|
|
28
|
+
// read-modify-write on a ledger holds an exclusive lock for its whole span.
|
|
29
|
+
//
|
|
30
|
+
// `fs.mkdirSync` is the primitive: directory creation is atomic on POSIX and Windows, so exactly one
|
|
31
|
+
// caller wins and the rest get EEXIST. It needs no fd bookkeeping, releases with one `rmdir`, and an
|
|
32
|
+
// EMPTY DIRECTORY IS INVISIBLE TO GIT — a lock can never be committed, and `readShardDir` only reads
|
|
33
|
+
// `*.json` files, so it is never mistaken for an entry either.
|
|
34
|
+
//
|
|
35
|
+
// The lock is per LEDGER, not per (story, repo). A per-identity lock keyed on the raw identity would
|
|
36
|
+
// let the one pair that most needs serializing race: `api.v2` and `api_v2` are DIFFERENT identities
|
|
37
|
+
// that share ONE shard filename (`safe()` maps both to `api_v2`), so they would take different locks
|
|
38
|
+
// and then collide on the same file. A ledger-wide lock cannot be wrong that way, and what it
|
|
39
|
+
// serializes is a handful of local file operations.
|
|
40
|
+
const LOCK_STALE_MS = 30_000; // a lock older than this belonged to a process that died holding it
|
|
41
|
+
const LOCK_WAIT_MS = 50;
|
|
42
|
+
const LOCK_RETRIES = 100; // ≈5s — long enough for any real ledger write, short enough to report
|
|
43
|
+
|
|
44
|
+
// Block the thread without spinning. `Atomics.wait` on the main thread is allowed in Node (only
|
|
45
|
+
// browsers forbid it), and these ledger writers are synchronous by design.
|
|
46
|
+
const sleep = (ms) => { Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms); };
|
|
47
|
+
|
|
48
|
+
// Run `fn` while holding `lockPath` exclusively; always releases, even when `fn` throws. Throws
|
|
49
|
+
// YAD-STATE-006 when the lock is held by a live process for the whole retry window — better to report
|
|
50
|
+
// than to write over another writer's work.
|
|
51
|
+
export function withLedgerLock(lockPath, fn, { retries = LOCK_RETRIES, waitMs = LOCK_WAIT_MS } = {}) {
|
|
52
|
+
fs.mkdirSync(path.dirname(lockPath), { recursive: true });
|
|
53
|
+
for (let i = 0; ; i++) {
|
|
54
|
+
try {
|
|
55
|
+
fs.mkdirSync(lockPath);
|
|
56
|
+
break;
|
|
57
|
+
} catch (e) {
|
|
58
|
+
if (e.code !== 'EEXIST') throw e;
|
|
59
|
+
// A holder that died leaves its lock behind forever; reclaim one that is provably too old.
|
|
60
|
+
let age;
|
|
61
|
+
try { age = Date.now() - fs.statSync(lockPath).mtimeMs; } catch { continue; } // vanished → retry
|
|
62
|
+
if (age > LOCK_STALE_MS) { try { fs.rmdirSync(lockPath); } catch { /* someone else won */ } continue; }
|
|
63
|
+
if (i >= retries) {
|
|
64
|
+
throw err('YAD-STATE-006', `another process is writing ${path.basename(lockPath, '.lock')} in ${path.basename(path.dirname(path.dirname(lockPath)))}`,
|
|
65
|
+
'wait for the other yad command to finish and re-run; if nothing else is running, delete the stale .lock directory the message names');
|
|
66
|
+
}
|
|
67
|
+
sleep(waitMs);
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
try { return fn(); } finally { try { fs.rmdirSync(lockPath); } catch { /* already reclaimed */ } }
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
const buildLogLock = (epicDir) => `${epicFiles(epicDir).buildLog}.lock`;
|
|
74
|
+
const trustLogLock = (epicDir) => `${epicFiles(epicDir).trustLog}.lock`;
|
|
16
75
|
|
|
17
76
|
// ---- shard filenames — the ONE source of truth for the naming convention -------------------------
|
|
18
77
|
// story ids already contain hyphens; the filename is just a unique handle (the entry inside carries
|
|
@@ -86,46 +145,68 @@ export function readShips(epicDir) {
|
|
|
86
145
|
// ship: `task` defaults to the sentinel `retro`, `mergeCommit` is written only when the caller supplies
|
|
87
146
|
// it (never invented), and `shippedAt` is the backfill date (the `retroactive` flag marks it as such).
|
|
88
147
|
//
|
|
89
|
-
// Guard: refuse when the story
|
|
90
|
-
// normal ship/checkpoint flow applies; a retro record would only muddy the ledger.
|
|
91
|
-
//
|
|
92
|
-
|
|
148
|
+
// Guard: refuse when the story already has a build-log ship IN THIS REPO — then it isn't pre-tracking
|
|
149
|
+
// there and the normal ship/checkpoint flow applies; a retro record would only muddy the ledger. The
|
|
150
|
+
// key is (story, repo), NOT story alone (#166): a story tagged with several repos ships once per repo,
|
|
151
|
+
// so a story-only guard let the FIRST backfill lock out every other repo and a multi-repo pre-tracking
|
|
152
|
+
// story could never be fully recorded. Each repo is recorded by its own run — the shards are distinct
|
|
153
|
+
// by construction (`buildShardName` keys on story+task+repo). Not keyed on `task` too: that would let
|
|
154
|
+
// repeated `--task T0x` pile several retro shards into one repo, the very muddying this guard prevents.
|
|
155
|
+
//
|
|
156
|
+
// Two names can differ yet share ONE shard file: `buildShardName` sanitizes each component through
|
|
157
|
+
// `safe()`, so `api.v2` and `api_v2` both become `api_v2` while the (story, repo) key sees them as
|
|
158
|
+
// distinct. Writing the second would silently overwrite the first repo's ship — destroying evidence in
|
|
159
|
+
// an append-only ledger, and (via the --dry-run cleanup) deleting an already-committed shard. So a
|
|
160
|
+
// name that COLLIDES with an existing ship's shard name is refused as `reason: 'collision'`, and the
|
|
161
|
+
// target file is never overwritten even when `readShips` cannot see it (a corrupt shard is skipped by
|
|
162
|
+
// `readShardDir`). Returns { written: false, reason } when refused, else { written: true, file, ship }.
|
|
163
|
+
export function writeRetroShip(epicDir, { story, repo, task = 'retro', mergeCommit, shippedAt }, lockOpts = {}) {
|
|
93
164
|
if (!story) throw new Error('writeRetroShip: story is required');
|
|
94
165
|
if (!repo) throw new Error('writeRetroShip: repo is required');
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
166
|
+
// Both guards and the write are ONE critical section: read outside the lock and a concurrent run
|
|
167
|
+
// (notably a different `--task` for the same (story, repo)) can write between the check and ours.
|
|
168
|
+
return withLedgerLock(buildLogLock(epicDir), () => {
|
|
169
|
+
const ships = readShips(epicDir).filter((s) => s.story === story);
|
|
170
|
+
if (ships.some((s) => s.repo === repo)) return { written: false, reason: 'exists' };
|
|
171
|
+
const clash = ships.find((s) => safe(s.repo) === safe(repo));
|
|
172
|
+
if (clash) return { written: false, reason: 'collision', repo: clash.repo };
|
|
173
|
+
const t = task || 'retro';
|
|
174
|
+
const ship = { story, task: t, repo, retroactive: true, note: 'pre-tracking backfill' };
|
|
175
|
+
if (mergeCommit) ship.mergeCommit = mergeCommit;
|
|
176
|
+
if (shippedAt) ship.shippedAt = shippedAt;
|
|
177
|
+
const f = epicFiles(epicDir);
|
|
178
|
+
const file = path.join(f.buildLogDir, buildShardName({ story, task: t, repo }));
|
|
179
|
+
if (fs.existsSync(file)) return { written: false, reason: 'collision', file };
|
|
180
|
+
writeJSON(file, ship);
|
|
181
|
+
return { written: true, file, ship };
|
|
182
|
+
}, lockOpts);
|
|
106
183
|
}
|
|
107
184
|
|
|
108
185
|
// Find the ship matching `match(ship)` across loose shards (authoritative until folded) then the
|
|
109
186
|
// folded file, apply `update(ship)`, and write back ONLY the file that holds it. Returns
|
|
110
|
-
// { found, where, file, ship }; found:false writes nothing (the caller warns).
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
187
|
+
// { found, where, file, ship }; found:false writes nothing (the caller warns). The find and the
|
|
188
|
+
// write-back are one locked span: a fold running between them would move the ship into the folded
|
|
189
|
+
// file and delete the shard this call is about to rewrite, resurrecting the deleted shard.
|
|
190
|
+
export function updateShip(epicDir, match, update, lockOpts = {}) {
|
|
191
|
+
return withLedgerLock(buildLogLock(epicDir), () => {
|
|
192
|
+
const f = epicFiles(epicDir);
|
|
193
|
+
for (const { name, obj } of readShardDir(f.buildLogDir)) {
|
|
194
|
+
if (match(obj)) {
|
|
195
|
+
update(obj);
|
|
196
|
+
const file = path.join(f.buildLogDir, name);
|
|
197
|
+
writeJSON(file, obj);
|
|
198
|
+
return { found: true, where: 'shard', file, ship: obj };
|
|
199
|
+
}
|
|
119
200
|
}
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
201
|
+
const foldedObj = readJSONStrict(f.buildLog, null);
|
|
202
|
+
const ship = Array.isArray(foldedObj?.ships) ? foldedObj.ships.find(match) : null;
|
|
203
|
+
if (ship) {
|
|
204
|
+
update(ship);
|
|
205
|
+
writeJSON(f.buildLog, foldedObj);
|
|
206
|
+
return { found: true, where: 'folded', file: f.buildLog, ship };
|
|
207
|
+
}
|
|
208
|
+
return { found: false };
|
|
209
|
+
}, lockOpts);
|
|
129
210
|
}
|
|
130
211
|
|
|
131
212
|
// ---- folding (used by `yad tidy up`) -------------------------------------------------------------
|
|
@@ -170,11 +251,16 @@ function fold(epicDir, { foldedPath, dir, arr, isTrust }, pick, { dryRun = false
|
|
|
170
251
|
return { folded: toFold.length, remaining: shards.length - toFold.length, deleted };
|
|
171
252
|
}
|
|
172
253
|
|
|
254
|
+
// Both folds are read-fold-delete, so they hold their ledger's lock for the whole span — otherwise a
|
|
255
|
+
// ship written (or stamped by `updateShip`) after the read but before the delete is folded away
|
|
256
|
+
// without its change, or deleted without ever being folded.
|
|
173
257
|
export function foldTrust(epicDir, pick, opts = {}) {
|
|
174
258
|
const f = epicFiles(epicDir);
|
|
175
|
-
return
|
|
259
|
+
return withLedgerLock(trustLogLock(epicDir), () =>
|
|
260
|
+
fold(epicDir, { foldedPath: f.trustLog, dir: f.trustLogDir, arr: 'runs', isTrust: true }, pick, opts), opts);
|
|
176
261
|
}
|
|
177
262
|
export function foldBuild(epicDir, pick, opts = {}) {
|
|
178
263
|
const f = epicFiles(epicDir);
|
|
179
|
-
return
|
|
264
|
+
return withLedgerLock(buildLogLock(epicDir), () =>
|
|
265
|
+
fold(epicDir, { foldedPath: f.buildLog, dir: f.buildLogDir, arr: 'ships', isTrust: false }, pick, opts), opts);
|
|
180
266
|
}
|
package/cli/manifest.mjs
CHANGED
|
@@ -230,6 +230,17 @@ export const REPO_WIRING = {
|
|
|
230
230
|
],
|
|
231
231
|
};
|
|
232
232
|
|
|
233
|
+
// Provenance of the wired files above, per repo root. Wiring files are OURS to rewrite, but "the
|
|
234
|
+
// on-disk copy differs from the shipped template" cannot tell a STALE copy (overwrite it) from one a
|
|
235
|
+
// team deliberately CUSTOMIZED (ask first) — so every write records the sha256 of what it wrote here.
|
|
236
|
+
// On the next update, on-disk == recorded proves the copy is untouched since we wrote it; anything
|
|
237
|
+
// else is a local edit, reported as `modified` and left alone (#164). Committed, so the record
|
|
238
|
+
// travels with the repo instead of living in one person's clone.
|
|
239
|
+
export const MANAGED_LEDGER = '.sdlc/managed.json';
|
|
240
|
+
// Suffix for the copy written beside a managed file before its content is replaced without that
|
|
241
|
+
// proof — the local edit is always recoverable from the working tree, not only from git history.
|
|
242
|
+
export const BACKUP_SUFFIX = '.yad-orig';
|
|
243
|
+
|
|
233
244
|
export const wiringFor = (platform) => [
|
|
234
245
|
...REPO_WIRING.common,
|
|
235
246
|
...(REPO_WIRING[platform] || []),
|
package/cli/plan.mjs
CHANGED
|
@@ -3,12 +3,13 @@
|
|
|
3
3
|
// setup (apply all), update (apply changed), and check (report; fix non-ok) share it.
|
|
4
4
|
import fs from 'node:fs';
|
|
5
5
|
import path from 'node:path';
|
|
6
|
+
import { err } from './errors.mjs';
|
|
6
7
|
import {
|
|
7
|
-
asset, exists, copyDir, copyFile, dirMatches, sameContent, readJSON,
|
|
8
|
+
asset, exists, copyDir, copyFile, dirMatches, sameContent, readJSON, readJSONStrict, writeJSON, fileSha,
|
|
8
9
|
} from './lib.mjs';
|
|
9
10
|
import {
|
|
10
|
-
SKILLS, IDE_TARGETS, IDE_OPENCODE_DIR, MODULE_FILES, wiringFor, HUB_WIRING, PROJECT_FILES,
|
|
11
|
-
LEGACY_SKILLS, REMOVED_SKILLS, LEGACY_MARKER, LEGACY_REPO_FILES, LEGACY_HUB_FILES,
|
|
11
|
+
VERSION, SKILLS, IDE_TARGETS, IDE_OPENCODE_DIR, MODULE_FILES, wiringFor, HUB_WIRING, PROJECT_FILES,
|
|
12
|
+
LEGACY_SKILLS, REMOVED_SKILLS, LEGACY_MARKER, LEGACY_REPO_FILES, LEGACY_HUB_FILES, MANAGED_LEDGER, BACKUP_SUFFIX,
|
|
12
13
|
} from './manifest.mjs';
|
|
13
14
|
|
|
14
15
|
// A git pathspec (forward slashes, relative to a repo root) for `dest` under `root`. Actions carry
|
|
@@ -36,6 +37,83 @@ const dirAction = (scope, item, src, dest, { root } = {}) => ({
|
|
|
36
37
|
apply: () => copyDir(src, dest),
|
|
37
38
|
});
|
|
38
39
|
|
|
40
|
+
// ---- managed-file provenance (#164) --------------------------------------------------------
|
|
41
|
+
// Read one repo root's ledger of "files yad wrote, and the sha it wrote". Strict, like every other
|
|
42
|
+
// ledger read: only an ABSENT ledger means "no record" ({}). One that exists but does not parse — or
|
|
43
|
+
// parses into something that is not a `files` map — must throw. Defaulting either to {} would
|
|
44
|
+
// silently downgrade every locally-modified file to an unrecorded one and re-open, one backup short,
|
|
45
|
+
// the silent clobber this record exists to prevent.
|
|
46
|
+
const isMap = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
|
|
47
|
+
export function readManagedLedger(root) {
|
|
48
|
+
const file = path.join(root, MANAGED_LEDGER);
|
|
49
|
+
const rec = readJSONStrict(file, null);
|
|
50
|
+
if (rec === null && !exists(file)) return {};
|
|
51
|
+
if (!isMap(rec) || !isMap(rec.files)) {
|
|
52
|
+
// Parses, but is not a record — YAD-STATE-002 (wrong shape), not -001 (does not parse).
|
|
53
|
+
throw err('YAD-STATE-002', `unreadable provenance record in ${file}: expected an object with a "files" map`,
|
|
54
|
+
'restore it from git — or delete it to start over, which costs the record (the next update then backs up every managed file it replaces)');
|
|
55
|
+
}
|
|
56
|
+
return rec.files;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
// Where the pre-overwrite copy of `dest` goes.
|
|
60
|
+
export const backupPathFor = (dest) => `${dest}${BACKUP_SUFFIX}`;
|
|
61
|
+
|
|
62
|
+
// A wired file (gate script, CI fragment, PR/MR template) — a fileAction plus provenance:
|
|
63
|
+
// 'ok' bytes are the shipped template
|
|
64
|
+
// 'missing' not installed
|
|
65
|
+
// 'outdated' differs, and the recorded sha proves WE wrote what is there (a stale copy) — or the
|
|
66
|
+
// file predates the ledger (no record at all), in which case nothing is proven and
|
|
67
|
+
// apply() saves a .yad-orig copy before replacing it
|
|
68
|
+
// 'modified' differs, and the recorded sha says someone edited our copy — never overwritten by a
|
|
69
|
+
// plain update; `--overwrite-local` replaces it (after a .yad-orig backup)
|
|
70
|
+
// apply() backs up whenever provenance is not proven, so no unproven content is ever discarded.
|
|
71
|
+
const wiredFileAction = (scope, item, src, dest, { root, exec = false, ledger = {} } = {}) => {
|
|
72
|
+
const base = fileAction(scope, item, src, dest, { root, exec });
|
|
73
|
+
const managed = { src, dest, root };
|
|
74
|
+
if (base.status !== 'outdated') return { ...base, managed };
|
|
75
|
+
const recorded = ledger[rel(root, dest)];
|
|
76
|
+
const ours = !!recorded && recorded === fileSha(dest);
|
|
77
|
+
const backup = ours ? null : backupPathFor(dest);
|
|
78
|
+
return {
|
|
79
|
+
...base,
|
|
80
|
+
// No record at all is a pre-ledger install, not evidence of an edit: keep the routine upgrade
|
|
81
|
+
// working (still 'outdated'), but never discard content we cannot prove we wrote — hence backup.
|
|
82
|
+
status: ours || !recorded ? 'outdated' : 'modified',
|
|
83
|
+
managed,
|
|
84
|
+
backup,
|
|
85
|
+
apply: () => {
|
|
86
|
+
if (backup) fs.copyFileSync(dest, backup);
|
|
87
|
+
copyFile(src, dest, { exec });
|
|
88
|
+
},
|
|
89
|
+
};
|
|
90
|
+
};
|
|
91
|
+
|
|
92
|
+
// Persist the provenance of every managed file whose on-disk bytes ARE the shipped template — the
|
|
93
|
+
// ones just applied AND the ones already correct. Seeding the already-correct ones is what migrates
|
|
94
|
+
// an install made before this ledger existed: from then on, an edit to any of them is detectable.
|
|
95
|
+
// A file we skipped as `modified` is deliberately NOT recorded — it is the team's copy, not ours.
|
|
96
|
+
// Keys are sorted so two repos' updates produce mergeable, byte-stable ledgers.
|
|
97
|
+
// Returns the roots written, so the caller can stage them alongside what they describe.
|
|
98
|
+
export function recordManagedWrites(actions = []) {
|
|
99
|
+
const byRoot = new Map();
|
|
100
|
+
for (const a of actions) {
|
|
101
|
+
const m = a?.managed;
|
|
102
|
+
if (!m || !m.root) continue;
|
|
103
|
+
if (!sameContent(m.src, m.dest)) continue;
|
|
104
|
+
if (!byRoot.has(m.root)) byRoot.set(m.root, {});
|
|
105
|
+
byRoot.get(m.root)[rel(m.root, m.dest)] = fileSha(m.dest);
|
|
106
|
+
}
|
|
107
|
+
const roots = [];
|
|
108
|
+
for (const [root, written] of byRoot) {
|
|
109
|
+
const files = { ...readManagedLedger(root), ...written };
|
|
110
|
+
const sorted = Object.fromEntries(Object.keys(files).sort().map((k) => [k, files[k]]));
|
|
111
|
+
writeJSON(path.join(root, MANAGED_LEDGER), { version: VERSION, files: sorted });
|
|
112
|
+
roots.push(root);
|
|
113
|
+
}
|
|
114
|
+
return roots;
|
|
115
|
+
}
|
|
116
|
+
|
|
39
117
|
// Persisted state gets one deliberately narrow compatibility repair. Explicit setup/planner input
|
|
40
118
|
// does not: a caller typo is an error, while the known v3.11.1 `.cluade` stamp is safely migrated.
|
|
41
119
|
const PERSISTED_IDE_ALIASES = new Map([['.cluade', '.claude']]);
|
|
@@ -371,8 +449,9 @@ export function legacyHubActions(root) {
|
|
|
371
449
|
// Per-repo wiring (gate scripts, CI, PR template).
|
|
372
450
|
export function repoActions(root, repo) {
|
|
373
451
|
const repoRoot = path.resolve(root, repo.path);
|
|
452
|
+
const ledger = readManagedLedger(repoRoot);
|
|
374
453
|
return wiringFor(repo.platform).map((w) =>
|
|
375
|
-
|
|
454
|
+
wiredFileAction(repo.name, w.dest, asset(w.src), path.join(repoRoot, w.dest), { root: repoRoot, exec: !!w.exec, ledger }),
|
|
376
455
|
);
|
|
377
456
|
}
|
|
378
457
|
|
|
@@ -383,8 +462,9 @@ export function hubActions(root) {
|
|
|
383
462
|
// `bridge_enabled` is the canonical flag (the documented hub-config schema); older setup versions
|
|
384
463
|
// wrote `bridge` — accept an explicit true in either spelling, wire nothing otherwise.
|
|
385
464
|
if (!hub?.platform || !(hub.bridge_enabled === true || hub.bridge === true)) return [];
|
|
465
|
+
const ledger = readManagedLedger(root);
|
|
386
466
|
return [...HUB_WIRING.common, ...(HUB_WIRING[hub.platform] || [])].map((w) =>
|
|
387
|
-
|
|
467
|
+
wiredFileAction('hub', w.dest, asset(w.src), path.join(root, w.dest), { root, exec: !!w.exec, ledger }),
|
|
388
468
|
);
|
|
389
469
|
}
|
|
390
470
|
|
package/cli/reconcile.mjs
CHANGED
|
@@ -10,18 +10,18 @@ import {
|
|
|
10
10
|
const readFileSafe = (p) => { try { return fs.readFileSync(p, 'utf8'); } catch { return ''; } };
|
|
11
11
|
|
|
12
12
|
import { preflightGuardReadiness } from './hubcommit.mjs';
|
|
13
|
-
import { VERSION, PROJECT_FILES } from './manifest.mjs';
|
|
13
|
+
import { VERSION, PROJECT_FILES, MANAGED_LEDGER, BACKUP_SUFFIX } from './manifest.mjs';
|
|
14
14
|
import {
|
|
15
15
|
moduleActions, repoActions, hubActions, authorsActions,
|
|
16
16
|
legacyModuleActions, removedModuleActions, legacyRepoActions, legacyHubActions,
|
|
17
|
-
ideTargetStateFor,
|
|
17
|
+
ideTargetStateFor, recordManagedWrites,
|
|
18
18
|
} from './plan.mjs';
|
|
19
19
|
import { gitHead, packRepo } from './setup.mjs';
|
|
20
|
-
import { groupByRoot, commitUpdates } from './update-commit.mjs';
|
|
20
|
+
import { groupByRoot, commitUpdates, repoLabel } from './update-commit.mjs';
|
|
21
21
|
|
|
22
|
-
const MARK = { missing: c.red('missing'), new: c.cyan('new'), outdated: c.yellow('outdated'), stale: c.yellow('stale'), legacy: c.yellow('legacy'), removed: c.yellow('removed'), ok: c.green('ok') };
|
|
22
|
+
const MARK = { missing: c.red('missing'), new: c.cyan('new'), outdated: c.yellow('outdated'), modified: c.cyan('modified'), stale: c.yellow('stale'), legacy: c.yellow('legacy'), removed: c.yellow('removed'), ok: c.green('ok') };
|
|
23
23
|
|
|
24
|
-
export async function reconcile(root, { fix = false, scope = 'all', force = false, push = false, allowBranch = false } = {}) {
|
|
24
|
+
export async function reconcile(root, { fix = false, scope = 'all', force = false, push = false, allowBranch = false, overwriteLocal = false } = {}) {
|
|
25
25
|
log(c.bold(`\nSDLC reconcile ${c.dim('v' + VERSION)}`));
|
|
26
26
|
log(c.dim(`target: ${root}\n`));
|
|
27
27
|
|
|
@@ -91,7 +91,7 @@ export async function reconcile(root, { fix = false, scope = 'all', force = fals
|
|
|
91
91
|
if (!byScope.has(a.scope)) byScope.set(a.scope, []);
|
|
92
92
|
byScope.get(a.scope).push(a);
|
|
93
93
|
}
|
|
94
|
-
const counts = { missing: 0, new: 0, outdated: 0, stale: 0, legacy: 0, removed: 0, ok: 0 };
|
|
94
|
+
const counts = { missing: 0, new: 0, outdated: 0, modified: 0, stale: 0, legacy: 0, removed: 0, ok: 0 };
|
|
95
95
|
for (const [scopeName, items] of byScope) {
|
|
96
96
|
const notOk = items.filter((i) => i.status !== 'ok');
|
|
97
97
|
items.forEach((i) => counts[i.status]++);
|
|
@@ -125,16 +125,29 @@ export async function reconcile(root, { fix = false, scope = 'all', force = fals
|
|
|
125
125
|
warn('existing .cluade path was left untouched; review its contents and remove it manually');
|
|
126
126
|
}
|
|
127
127
|
|
|
128
|
+
// A managed file the team edited is NEVER rewritten by a plain update — that silent clobber is what
|
|
129
|
+
// #164 reported. It is reported on every run (honest drift) until either the edit is dropped or
|
|
130
|
+
// `--overwrite-local` replaces it, which still saves the previous content beside it.
|
|
131
|
+
const modified = actions.filter((a) => a.status === 'modified');
|
|
132
|
+
for (const m of modified) {
|
|
133
|
+
warn(`${m.scope}/${m.item} is locally modified — it matches neither the shipped template nor the copy yad wrote`);
|
|
134
|
+
}
|
|
135
|
+
if (modified.length && !overwriteLocal) {
|
|
136
|
+
hand(`keep the edits (reported as \`modified\` on every check), or replace them with \`yad update --overwrite-local\` — each previous version is saved beside the file as <file>${BACKUP_SUFFIX}`);
|
|
137
|
+
}
|
|
138
|
+
|
|
128
139
|
const fixable = actions.filter((a) =>
|
|
129
|
-
a.status !== 'ok'
|
|
140
|
+
a.status !== 'ok'
|
|
141
|
+
&& (a.status !== 'modified' || overwriteLocal)
|
|
142
|
+
&& (scope === 'all' ? true : a.status !== 'missing'),
|
|
130
143
|
);
|
|
131
144
|
log('');
|
|
132
|
-
log(c.dim(`summary: ${counts.missing} missing, ${counts.new} new, ${counts.outdated} outdated, ${counts.stale} stale, ${counts.legacy} legacy, ${counts.removed} removed, ${counts.ok} ok`));
|
|
145
|
+
log(c.dim(`summary: ${counts.missing} missing, ${counts.new} new, ${counts.outdated} outdated, ${counts.modified} modified, ${counts.stale} stale, ${counts.legacy} legacy, ${counts.removed} removed, ${counts.ok} ok`));
|
|
133
146
|
|
|
134
147
|
if (!fix) {
|
|
135
148
|
if (push) warn('--push has no effect without --fix (there is nothing applied to commit).');
|
|
136
149
|
if (fixable.length || gaps.length) hand('run `yad check --fix` to reconcile (or `yad setup` for missing one-time setup).');
|
|
137
|
-
return { counts, gaps, applied: 0 };
|
|
150
|
+
return { counts, gaps, applied: 0, modified: modified.length };
|
|
138
151
|
}
|
|
139
152
|
|
|
140
153
|
// --- apply --- (collect the applied actions so --push can stage each repo's exact allowlist) ---
|
|
@@ -145,16 +158,32 @@ export async function reconcile(root, { fix = false, scope = 'all', force = fals
|
|
|
145
158
|
a.apply();
|
|
146
159
|
applied++;
|
|
147
160
|
appliedActions.push(a);
|
|
148
|
-
|
|
161
|
+
// A backup means the replaced content was not provably ours (a pre-ledger install, or an edit
|
|
162
|
+
// --overwrite-local was told to discard). Never report that as an ordinary template adoption.
|
|
163
|
+
info(`${a.status} → fixed: ${a.scope}/${a.item}${a.backup ? ` ${c.yellow(`(previous content saved to ${path.basename(a.backup)})`)}` : ''}`);
|
|
149
164
|
}
|
|
150
165
|
if (force) {
|
|
166
|
+
// --force re-copies what is already correct; it deliberately does NOT reach a `modified` file —
|
|
167
|
+
// only --overwrite-local discards a local edit, and only after backing it up.
|
|
151
168
|
for (const a of actions.filter((a) => a.status === 'ok')) { a.apply(); appliedActions.push(a); }
|
|
152
169
|
}
|
|
153
170
|
// Refresh the version stamp and persist only the canonical targets used to build actions. This also
|
|
154
171
|
// completes legacy/corrupt target migration even when no skill content itself needed an update.
|
|
155
172
|
writeCanonicalStamp();
|
|
156
173
|
appliedActions.push({ scope: 'hub', item: PROJECT_FILES.version, status: 'stamp', root, paths: [PROJECT_FILES.version] });
|
|
174
|
+
// Record what we wrote (and what was already correct) so the NEXT update can tell a stale managed
|
|
175
|
+
// file from an edited one. Seeding the already-correct files is what migrates a pre-ledger install.
|
|
176
|
+
// A file left as `modified` records nothing — it differs from the template by definition.
|
|
177
|
+
for (const ledgerRoot of recordManagedWrites(actions)) {
|
|
178
|
+
appliedActions.push({
|
|
179
|
+
scope: repoLabel(root, ledgerRoot), item: MANAGED_LEDGER, status: 'stamp',
|
|
180
|
+
root: ledgerRoot, paths: [MANAGED_LEDGER],
|
|
181
|
+
});
|
|
182
|
+
}
|
|
157
183
|
applied ? ok(`reconciled ${applied} item(s)`) : info('nothing to fix');
|
|
184
|
+
if (modified.length && !overwriteLocal) {
|
|
185
|
+
warn(`${modified.length} locally modified file(s) left untouched — this update did not reach them`);
|
|
186
|
+
}
|
|
158
187
|
if (gaps.length) hand('one-time setup still missing — run `yad setup`.');
|
|
159
188
|
|
|
160
189
|
// --- publish: commit each repo's applied changes and push directly to its default branch ---
|
|
@@ -182,5 +211,5 @@ export async function reconcile(root, { fix = false, scope = 'all', force = fals
|
|
|
182
211
|
},
|
|
183
212
|
});
|
|
184
213
|
}
|
|
185
|
-
return { counts, gaps, applied };
|
|
214
|
+
return { counts, gaps, applied, modified: modified.length };
|
|
186
215
|
}
|
package/cli/setup.mjs
CHANGED
|
@@ -10,7 +10,7 @@ import { VERSION, IDE_TARGETS, PROJECT_FILES, DESIGN_TOOLS, DESIGN_PRIMARY, TEST
|
|
|
10
10
|
import {
|
|
11
11
|
moduleActions, repoActions, hubActions, authorsActions,
|
|
12
12
|
legacyModuleActions, removedModuleActions, legacyRepoActions, legacyHubActions,
|
|
13
|
-
safeIdeTargetsFor, detectedIdeTargetStateFor,
|
|
13
|
+
safeIdeTargetsFor, detectedIdeTargetStateFor, recordManagedWrites,
|
|
14
14
|
} from './plan.mjs';
|
|
15
15
|
import { validateLogin, rolesForScope } from './platform.mjs';
|
|
16
16
|
|
|
@@ -348,10 +348,16 @@ export function registerLearning(root, { tool, kb = null, today = null } = {}) {
|
|
|
348
348
|
function applyActions(actions, { force = false } = {}) {
|
|
349
349
|
let changed = 0;
|
|
350
350
|
for (const a of actions) {
|
|
351
|
+
// A managed file the team edited is left alone here too — setup re-runs with force:true, so
|
|
352
|
+
// without this the wizard would be a second silent-clobber path for the same edits (#164).
|
|
353
|
+
if (a.status === 'modified') {
|
|
354
|
+
warn(`kept locally modified ${a.scope}/${a.item} — replace it with \`yad update --overwrite-local\``);
|
|
355
|
+
continue;
|
|
356
|
+
}
|
|
351
357
|
if (a.status === 'ok' && !force) continue;
|
|
352
358
|
a.apply();
|
|
353
359
|
changed++;
|
|
354
|
-
info(`${a.status === 'missing' ? 'installed' : 'updated'} ${a.scope}/${a.item}`);
|
|
360
|
+
info(`${a.status === 'missing' ? 'installed' : 'updated'} ${a.scope}/${a.item}${a.backup ? ` (previous content saved to ${path.basename(a.backup)})` : ''}`);
|
|
355
361
|
}
|
|
356
362
|
if (!changed) info('already up to date');
|
|
357
363
|
return changed;
|
|
@@ -708,9 +714,14 @@ export async function runSetup(root, opts = {}) {
|
|
|
708
714
|
S('Wire connected repos + the hub (CI gates, PR template, gate-sync)');
|
|
709
715
|
guide(['Installs the CI safety gates, PR/MR template, and gate-sync — automatic, no input needed.']);
|
|
710
716
|
if (registry.repos.length === 0) info('no repos to wire');
|
|
717
|
+
// Every managed file this step writes is recorded (sha per repo root) so a LATER `yad update` can
|
|
718
|
+
// tell a stale copy from one the team edited, instead of silently rewriting both (#164).
|
|
719
|
+
const wired = [];
|
|
711
720
|
for (const repo of registry.repos) {
|
|
712
721
|
log(` ${c.bold(repo.name)} ${c.dim(`(${repo.platform})`)}`);
|
|
713
|
-
|
|
722
|
+
const repoWiring = repoActions(root, repo);
|
|
723
|
+
applyActions(repoWiring, { force: true });
|
|
724
|
+
wired.push(...repoWiring);
|
|
714
725
|
// Migrate pre-2.0 wired CI (marker-owned sdlc-*.yml -> yad-*.yml); a user-authored
|
|
715
726
|
// same-named file is never touched.
|
|
716
727
|
applyActions(legacyRepoActions(root, repo), { force: true });
|
|
@@ -720,8 +731,12 @@ export async function runSetup(root, opts = {}) {
|
|
|
720
731
|
if (hubWiring.length) {
|
|
721
732
|
log(` ${c.bold('hub')} ${c.dim('(gate-sync + verified-commits CI)')}`);
|
|
722
733
|
applyActions(hubWiring, { force: true });
|
|
734
|
+
wired.push(...hubWiring);
|
|
723
735
|
}
|
|
724
736
|
applyActions(legacyHubActions(root), { force: true });
|
|
737
|
+
// After every write to a managed path has landed (including the legacy renames), so the recorded
|
|
738
|
+
// sha is the file's final state.
|
|
739
|
+
recordManagedWrites(wired);
|
|
725
740
|
// author allowlists for the verified-commits gate (hub + every repo), from the roster emails
|
|
726
741
|
applyActions(authorsActions(root, registry.repos), { force: true });
|
|
727
742
|
|
package/cli/usage.mjs
CHANGED
|
@@ -243,11 +243,17 @@ function memberFlags(m) {
|
|
|
243
243
|
|
|
244
244
|
// Team-level hygiene, keyed by epic/story — a ship with no recorded engineer review is a process gap,
|
|
245
245
|
// not attributable to one person, so it lives here rather than in a member's flag list.
|
|
246
|
+
//
|
|
247
|
+
// A `retroactive: true` ship is EXCLUDED: it is a `yad checkpoint --retro-ship` reconciliation of a
|
|
248
|
+
// story that shipped before the ledger existed, so it never had a tracked PR to review — counting it
|
|
249
|
+
// as a missing review reports a gap the team could not have filled. It also scales with repo count
|
|
250
|
+
// (one backfill shard per repo since #166), which would swamp the real gaps with reconciliation noise.
|
|
246
251
|
export function shipHygiene(root, { since, until } = {}) {
|
|
247
252
|
const items = [];
|
|
248
253
|
for (const epic of listEpics(root)) {
|
|
249
254
|
for (const s of readShips(path.join(root, 'epics', epic))) {
|
|
250
255
|
if (!inWindow(s.shippedAt, since, until)) continue;
|
|
256
|
+
if (s.retroactive) continue;
|
|
251
257
|
if (!Array.isArray(s.engineer_review) || s.engineer_review.length === 0) {
|
|
252
258
|
items.push({ epic, story: s.story || null, task: s.task || null, repo: s.repo || null, shippedAt: s.shippedAt || null });
|
|
253
259
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "yadflow",
|
|
3
|
-
"version": "3.
|
|
3
|
+
"version": "3.16.1",
|
|
4
4
|
"description": "Yadflow — the gated, team, multi-repo SDLC: author → review → build with a PR-driven review gate and a zero-dependency `yad` CLI (setup, gate, commit, open-pr, ship, repo, thread, reconcile). A BMAD module + 38 yad-* skills.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"author": "AbdelRahman Nasr",
|
|
@@ -203,6 +203,18 @@ catches a free-form description that bypassed it:
|
|
|
203
203
|
(`epics/**`, detected from the CI-supplied `--changed <file>` list) **FAILS** — artifact changes
|
|
204
204
|
must go through a `review/EP-*` PR.
|
|
205
205
|
|
|
206
|
+
**GitLab truncates the description this gate reads.** `$CI_MERGE_REQUEST_DESCRIPTION` stops at **2700
|
|
207
|
+
characters**, so a long but perfectly valid MR can lose a required section *before the gate sees it* —
|
|
208
|
+
the author then reads "does not use the template" while looking at a description that visibly contains
|
|
209
|
+
it (#164). GitHub is unaffected (`github.event.pull_request.body` is not truncated). Two mitigations,
|
|
210
|
+
both shipped:
|
|
211
|
+
|
|
212
|
+
- the GitLab MR templates (`yad-pr-template` `templates/gitlab/…` and `templates/hub/gitlab/…`) carry
|
|
213
|
+
the constraint as a comment and keep every required section early, so a truncated body still passes;
|
|
214
|
+
- when a required section is missing **and** the body it read is ≥ 2700 characters, the gate prints a
|
|
215
|
+
`NOTE` naming the truncation and the fix — reorder the required sections above the cutoff and push
|
|
216
|
+
the long narrative to the end. Never delete a section: reordering is always allowed.
|
|
217
|
+
|
|
206
218
|
## 8. Phase 6 — feature-thread gates (`lineage-check.sh`, `epic-open.sh`, `reconcile-debt-check.sh`)
|
|
207
219
|
|
|
208
220
|
After the contract locks and code ships, a change must not mutate a locked artifact — it becomes a new
|
|
@@ -81,11 +81,39 @@ yad checkpoint --retro-ship <epic>/<story> --repo <r> [--task <t>] [--merge-comm
|
|
|
81
81
|
It writes ONE minimal ship shard marked `retroactive: true` (`task` defaults to the sentinel `retro`;
|
|
82
82
|
`mergeCommit` is written only if you pass `--merge-commit`; `shippedAt` is the backfill date), then runs
|
|
83
83
|
the normal checkpoint so the story's already-made `status:` flip rides along in the **same** commit. It
|
|
84
|
-
refuses when the story already has a
|
|
85
|
-
does **not** author the story frontmatter — and to keep evidence and the flip atomic (the no-drift
|
|
84
|
+
refuses when the story already has a ship **in that repo** (then it isn't pre-tracking there — use the
|
|
85
|
+
normal flow). It does **not** author the story frontmatter — and to keep evidence and the flip atomic (the no-drift
|
|
86
86
|
invariant), it **refuses** unless you have already set `status: shipped` in `stories/<story>.md`, so a
|
|
87
87
|
ship shard is never committed while the artifact still says `approved`.
|
|
88
88
|
|
|
89
|
+
**One repo per run (#166).** A ship is recorded per `(story, task, repo)`, so a story that shipped in
|
|
90
|
+
several repos needs one retroactive shard **per repo** — the guard is keyed on `(story, repo)`, not on
|
|
91
|
+
the story alone, so recording the first repo never locks out the rest. Re-run once per repo:
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
yad checkpoint --retro-ship EP-foo/EP-foo-S01 --repo web --push
|
|
95
|
+
yad checkpoint --retro-ship EP-foo/EP-foo-S01 --repo api --push
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The `status:` flip rides the **first** commit (it only needs one ship to be carried); each later run
|
|
99
|
+
lands only its own ship shard, so the story ends up with complete per-repo evidence. After each run the
|
|
100
|
+
command names the declared repos that still have none, so a half-finished backfill is visible instead of
|
|
101
|
+
looking complete.
|
|
102
|
+
|
|
103
|
+
A ship is permanent audit evidence, so `--repo` is checked before anything is written: it must be a repo
|
|
104
|
+
the story's `repos:` frontmatter declares (or, for a legacy story that declares none, one connected in
|
|
105
|
+
`.sdlc/repos.json`). A typo'd, mis-cased or invented name is **refused** — the per-repo guard means it
|
|
106
|
+
would otherwise collide with nothing and quietly record a ship for a repo that never existed. A name
|
|
107
|
+
that would share a shard **filename** with an already-recorded repo (shard names sanitize everything
|
|
108
|
+
outside `[A-Za-z0-9_-]` to `_`, so `api.v2` and `api_v2` are one file) is refused for the same reason:
|
|
109
|
+
recording it would overwrite the other repo's ship record in an append-only ledger.
|
|
110
|
+
|
|
111
|
+
Those checks decide from the ledger and then write it, so each one runs under an exclusive **lock** on
|
|
112
|
+
the epic's `build-log` — as does `yad review reconcile`'s stamp and `yad tidy up`'s fold. Two commands
|
|
113
|
+
writing the same ledger at once would otherwise both read "no ship yet" and both write. The lock is an
|
|
114
|
+
empty directory (`build-log.json.lock`), so git never sees it; one held by a process that died is
|
|
115
|
+
reclaimed after 30s, and a live one reports `YAD-STATE-006` rather than writing over the other's work.
|
|
116
|
+
|
|
89
117
|
**Engagement (the Review Companion).** Each `engineer_review` entry carries `engagement: verified | none`
|
|
90
118
|
— `verified` when the engineer reviewed through the [companion](../../yad-review-companion/SKILL.md)
|
|
91
119
|
(`yad review trailer/context/nudge`, a real trailer/cards/chat session over the diff), `none` for a bare
|
|
@@ -26,6 +26,18 @@ touched domain). This step **never auto-advances**; it sets up the template and
|
|
|
26
26
|
`templates/hub/gitlab/merge_request_templates/Default.md` →
|
|
27
27
|
`{project-root}/.gitlab/merge_request_templates/Default.md`. The hub body carries no `Task:` trailer
|
|
28
28
|
(hub PRs change artifacts, not code); its routing helper is `yad-hub-bridge`'s `hub-route.sh`.
|
|
29
|
+
- **GitLab reads a truncated description.** The `pr-template` gate is fed
|
|
30
|
+
`$CI_MERGE_REQUEST_DESCRIPTION`, which GitLab cuts at **2700 characters** — a required section below
|
|
31
|
+
that cutoff is invisible to the gate even though the MR shows it, and the failure reads "does not use
|
|
32
|
+
the template" (#164). Both GitLab templates say so in a comment and keep `## Summary` /
|
|
33
|
+
`## Impact & Risk` / `## Checklist` (hub: `## Artifact under review` / `## Impact & Risk (front-half)`
|
|
34
|
+
/ `## Checklist`) early, so a truncated body still passes. Long narrative goes **after** them.
|
|
35
|
+
Sections may be reordered freely; deleting one fails the gate. GitHub is unaffected.
|
|
36
|
+
- **Installed templates are yad-managed.** `yad update` rewrites them on upgrade. An edit yad can
|
|
37
|
+
prove — the file's sha differs from the one it recorded when it wrote the template — is reported as
|
|
38
|
+
`modified` and left alone; a copy it has no record of is replaced after a `.yad-orig` backup (see
|
|
39
|
+
`docs/CLI.md` → *Managed files*). Either way, put knowledge that must survive an upgrade in an ADR
|
|
40
|
+
under `docs/`, not in the template.
|
|
29
41
|
- The Impact & Risk block reuses the conventions of earlier steps: the `Task: <story>-<task>` trailer
|
|
30
42
|
(`yad-implement`), the contract surface (`yad-architecture` / contract-check), and the
|
|
31
43
|
domain-owner escalation (`yad-review-gate`).
|
|
@@ -45,6 +45,18 @@ if [ -z "$BODY" ] || [ ! -f "$BODY" ]; then
|
|
|
45
45
|
exit 1
|
|
46
46
|
fi
|
|
47
47
|
|
|
48
|
+
# GitLab TRUNCATES $CI_MERGE_REQUEST_DESCRIPTION at 2700 characters, so a long-but-valid description
|
|
49
|
+
# can lose a required section before this gate ever reads it — the author then sees "does not use the
|
|
50
|
+
# template" while looking at an MR that visibly contains it (#164). Measure the RAW body now (the
|
|
51
|
+
# trailer strip below shortens it) so a failure at that boundary can say so. GitHub bodies are not
|
|
52
|
+
# truncated, so the note is advisory and only ever printed alongside a real failure.
|
|
53
|
+
GITLAB_DESC_LIMIT=2700
|
|
54
|
+
# CHARACTERS, not bytes — GitLab counts characters, and a description full of multibyte punctuation
|
|
55
|
+
# (an em-dash costs 3 bytes) would hit 2700 bytes long before it could ever be truncated. `wc -m`
|
|
56
|
+
# would need a UTF-8 locale we cannot assume across CI images, so count UTF-8 code points the
|
|
57
|
+
# locale-independent way: every byte that is NOT a continuation byte (0x80-0xBF) starts one.
|
|
58
|
+
RAW_CHARS="$(LC_ALL=C tr -d '\200-\277' < "$BODY" | wc -c | tr -d '[:space:]')"
|
|
59
|
+
|
|
48
60
|
# The Review Companion injects a `<!-- yad:trailer --> … <!-- /yad:trailer -->` briefing block (and
|
|
49
61
|
# `<!-- yad:noblock -->` notes) into the description. Strip those before the template check so the
|
|
50
62
|
# AI-generated prose can never hide a required section heading or be mistaken for the `Risk level:`
|
|
@@ -110,5 +122,10 @@ else
|
|
|
110
122
|
check_code_body
|
|
111
123
|
fi
|
|
112
124
|
|
|
125
|
+
if [ "$rc" != 0 ] && [ "$RAW_CHARS" -ge "$GITLAB_DESC_LIMIT" ]; then
|
|
126
|
+
echo "NOTE [pr-template]: the description this gate read is ${RAW_CHARS} characters. On GitLab the gate reads \$CI_MERGE_REQUEST_DESCRIPTION, which is TRUNCATED at ${GITLAB_DESC_LIMIT} — a section below that cutoff is invisible here even though the MR shows it."
|
|
127
|
+
echo "NOTE [pr-template]: if the missing section IS in your description, move the required sections above the cutoff (reorder, never delete) and push the long narrative to the end."
|
|
128
|
+
fi
|
|
129
|
+
|
|
113
130
|
[ "$rc" = 0 ] && echo "PASS [pr-template]: body uses the ${KIND} template (required sections present)."
|
|
114
131
|
exit "$rc"
|
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
<!-- SDLC MR template (Phase 3 build plan §D). One atomic task per MR. -->
|
|
2
|
+
<!-- GITLAB 2700-CHARACTER LIMIT: the yad-pr-template gate reads $CI_MERGE_REQUEST_DESCRIPTION, which
|
|
3
|
+
GitLab truncates at 2700 characters — a heading past the cutoff reads as missing even though you
|
|
4
|
+
can see it here. Keep Summary, Impact & Risk (with its filled risk-level line) and Checklist
|
|
5
|
+
within the first 2700 characters; put long narrative below them. Reorder, never delete. -->
|
|
2
6
|
|
|
3
7
|
## Summary
|
|
4
8
|
<!-- What this MR does, in one or two sentences. -->
|
|
@@ -2,6 +2,9 @@
|
|
|
2
2
|
<!-- This MR is a REVIEW VEHICLE on the product hub, not a code merge. The file gate (yad-review-gate)
|
|
3
3
|
advances the step; do NOT rely on merging this MR to advance. Reviewers approve/comment here, then a
|
|
4
4
|
`yad-review-gate action: sync` pulls that into the file ledger. -->
|
|
5
|
+
<!-- GITLAB 2700-CHARACTER LIMIT: the yad-pr-template gate reads $CI_MERGE_REQUEST_DESCRIPTION, which
|
|
6
|
+
GitLab truncates at 2700 characters — a heading past the cutoff reads as missing. The required
|
|
7
|
+
sections come first for that reason; keep long narrative last. Reorder, never delete. -->
|
|
5
8
|
|
|
6
9
|
## Artifact under review
|
|
7
10
|
- Epic: `EP-<slug>`
|
|
@@ -9,14 +12,20 @@
|
|
|
9
12
|
- Gate step: `<epic-review | architecture-review | ui-design-review | stories-review>`
|
|
10
13
|
- Owner: `<epic.md owner>`
|
|
11
14
|
|
|
12
|
-
## What changed
|
|
13
|
-
<!-- One or two sentences on what this artifact says / what changed since the last review round. -->
|
|
14
|
-
|
|
15
15
|
## Impact & Risk (front-half)
|
|
16
16
|
- **Domains / repos touched:** <epic.repos, e.g. backend, mobile>
|
|
17
17
|
- **Risk tags:** <none | contract | auth | payments> <!-- contract/auth/payments => escalates to domain owners -->
|
|
18
18
|
- **Contract surface:** <n/a | locked @ sha256:…> <!-- architecture only; a re-lock invalidates prior approvals -->
|
|
19
19
|
|
|
20
|
+
## Checklist
|
|
21
|
+
- [ ] `owner` set in the artifact frontmatter (inherited from `epic.md`)
|
|
22
|
+
- [ ] Contract re-locked (`.sdlc/contract-lock.json`) if the surface changed (architecture only)
|
|
23
|
+
- [ ] Risk tags reflect the real surface touched (contract/auth/payments escalate)
|
|
24
|
+
- [ ] No secrets or tokens in the artifact or this description
|
|
25
|
+
|
|
26
|
+
## What changed
|
|
27
|
+
<!-- One or two sentences on what this artifact says / what changed since the last review round. -->
|
|
28
|
+
|
|
20
29
|
## Required approvals (yad-review-gate rule)
|
|
21
30
|
- Base: **owner + 1 reviewer**.
|
|
22
31
|
- Escalated (risk tag set, or a stories MR): **plus one domain-owner per touched repo** — see the
|
|
@@ -28,10 +37,4 @@
|
|
|
28
37
|
- **Comment** to record review comments (synced into `reviews/<artifact>--<date>--comments.md`).
|
|
29
38
|
- **Do NOT merge to advance** — `yad-review-gate action: sync` + `action: advance` move the step.
|
|
30
39
|
|
|
31
|
-
## Checklist
|
|
32
|
-
- [ ] `owner` set in the artifact frontmatter (inherited from `epic.md`)
|
|
33
|
-
- [ ] Contract re-locked (`.sdlc/contract-lock.json`) if the surface changed (architecture only)
|
|
34
|
-
- [ ] Risk tags reflect the real surface touched (contract/auth/payments escalate)
|
|
35
|
-
- [ ] No secrets or tokens in the artifact or this description
|
|
36
|
-
|
|
37
40
|
/assign me
|