its-magic 0.1.3-1 → 0.1.3-2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/installer.ps1 +8 -0
  2. package/installer.py +8 -0
  3. package/installer.sh +1 -0
  4. package/package.json +1 -1
  5. package/template/.cursor/commands/auto.md +90 -0
  6. package/template/.cursor/commands/execute.md +26 -0
  7. package/template/.cursor/commands/refresh-context.md +32 -0
  8. package/template/.cursor/commands/sovereign-critic.md +104 -0
  9. package/template/.cursor/rules/sovereign-role-manifest.mdc.example +27 -0
  10. package/template/.cursor/scratchpad.md +141 -0
  11. package/template/.cursor/sovereign-role-manifest.yaml.example +53 -0
  12. package/template/decisions/DEC-0104.md +279 -0
  13. package/template/decisions/DEC-0105.md +231 -0
  14. package/template/decisions/DEC-0107.md +246 -0
  15. package/template/docs/engineering/architecture.md +78 -0
  16. package/template/docs/engineering/auto-orchestration-reference.md +7 -0
  17. package/template/docs/engineering/context/installer-owned-paths.manifest +8 -0
  18. package/template/docs/engineering/reason_codes.md +411 -0
  19. package/template/docs/engineering/runbook.md +922 -0
  20. package/template/docs/engineering/sovereign-memory/.gitkeep +0 -0
  21. package/template/docs/engineering/sovereign-memory/retrospectives/.gitkeep +0 -0
  22. package/template/handoffs/sovereign_decisions/.gitkeep +0 -0
  23. package/template/handoffs/sovereign_deferrals/.gitkeep +0 -0
  24. package/template/handoffs/sovereign_role_reviews.jsonl +0 -0
  25. package/template/scripts/__pycache__/decision_ledger_lib.cpython-312.pyc +0 -0
  26. package/template/scripts/check_intake_template_parity.py +119 -0
  27. package/template/scripts/decision_ledger_lib.py +732 -0
  28. package/template/scripts/ledger_validate.py +153 -0
  29. package/template/scripts/parallel_dev_arbiter.py +923 -0
  30. package/template/scripts/release_trigger_adapters.py +843 -0
  31. package/template/scripts/self_healing_deploy_lib.py +463 -0
  32. package/template/scripts/self_healing_deploy_validate.py +78 -0
  33. package/template/scripts/sovereign_convergence_lib.py +994 -0
  34. package/template/scripts/sovereign_convergence_validate.py +206 -0
  35. package/template/scripts/sovereign_critic_lib.py +629 -0
  36. package/template/scripts/sovereign_critic_validate.py +131 -0
  37. package/template/scripts/sovereign_loop_lib.py +828 -0
  38. package/template/scripts/sovereign_loop_validate.py +122 -0
  39. package/template/scripts/sovereign_memory_lib.py +869 -0
  40. package/template/scripts/sovereign_memory_validate.py +153 -0
  41. package/template/scripts/sovereign_role_manifest_lib.py +547 -0
  42. package/template/scripts/sovereign_role_manifest_validate.py +105 -0
  43. package/template/tests/us0108_contract_test.py +207 -0
  44. package/template/tests/us0109_contract_test.py +209 -0
  45. package/template/tests/us0109_us0110_compose_test.py +34 -0
  46. package/template/tests/us0111_contract_test.py +345 -0
@@ -0,0 +1,279 @@
1
+ # DEC-0104: Cross-Model Adversarial Critic (US-0104)
2
+
3
+ - **Date**: 2026-06-28
4
+ - **Status**: Accepted
5
+ - **Story**: `US-0104`
6
+ - **Composes on**: **US-0048** (per-phase isolation evidence), **US-0069** (phase role enforcement), **US-0023** (fresh-context semantics), **US-0110** (convergence conjunct 3) — **do not amend**; US-0104 **populates** `handoffs/sovereign_critic_findings.jsonl` and sets **`cross_model_reviewed`** on **US-0103** ledger entries
7
+ - **Related**: `R-0092` (research anchor), **US-0101** / **US-0102** (model resolution — read-only consumer), **US-0103** (ledger `cross_model_reviewed` field), **US-0108** (anti-slop score consumer), **US-0045** (status authority)
8
+ - **Supersedes**: none
9
+ - **Default-off**: `CROSS_MODEL_REVIEW=0` — zero overhead when off
10
+
11
+ ## Context
12
+
13
+ Sovereign-loop quality gate (US-0103..US-0110). After each producer phase completes, an optional critic subagent using a **different model** evaluates phase artifacts through three fixed lenses (Challenger / Architect / Subtractor). Findings reconcile via parallel-jury agreement; anti-slop scoring triggers bounded producer rework. **US-0048 / US-0069 / US-0023 / US-0110 are UNCHANGED** — US-0104 writes findings JSONL and optional ledger patches only.
14
+
15
+ Research basis: `R-0092` Q1–Q7 closed.
16
+
17
+ ## Decision
18
+
19
+ ### §1 — Scratchpad keys (AC-1)
20
+
21
+ | Key | Values | Default | Notes |
22
+ |-----|--------|---------|-------|
23
+ | **`CROSS_MODEL_REVIEW`** | **`0`** \| **`1`** | **`0`** | When `0`, no critic spawn, no findings writes, no anti-slop gate — **zero overhead**. |
24
+ | **`CROSS_MODEL_ANTISLOP_THRESHOLD`** | int **0–10** | **`6`** | Aggregate below threshold → rework loop. |
25
+ | **`CROSS_MODEL_REWORK_MAX`** | int ≥ **0** | **`2`** | Max producer re-spawns per `(orchestrator_run_id, phase_id)` before decision gate. |
26
+
27
+ Published in **`.cursor/scratchpad.md`** (+ **`template/.cursor/scratchpad.md`** byte-parity per **US-0017**).
28
+
29
+ **Rejected**: always-on critic (**rejected** — latency/token cost; default-off discipline).
30
+ **Rejected**: per-lens threshold keys (**rejected** — single aggregate `min(lens_scores)` sufficient for v1; **US-0108** depends on stable aggregate).
31
+
32
+ ### §2 — Findings JSONL schema v1 (AC-4, AC-5)
33
+
34
+ **Canonical path**: **`handoffs/sovereign_critic_findings.jsonl`** (repo root under `handoffs/`; matches **US-0110** `sovereign_convergence_lib.CRITIC_PATH`). Append-only; one file per repo (not per run).
35
+
36
+ | Field | Type | Required | Notes |
37
+ |-------|------|----------|-------|
38
+ | **`ts`** | ISO 8601 UTC string | yes | `YYYY-MM-DDTHH:MM:SS.mmmZ` |
39
+ | **`orchestrator_run_id`** | non-empty string | yes | Partition key for rework counter |
40
+ | **`phase_id`** | string | yes | **DEC-0086** canonical phase id |
41
+ | **`role`** | string | yes | Producer logical role (`po`, `tech-lead`, `dev`, `qa`, …) |
42
+ | **`producer_model_id`** | non-empty string | yes | Resolved slug or stable alias (`fast`, `inherit`, vendor slug) |
43
+ | **`critic_model_id`** | non-empty string | yes | Same vocabulary as producer |
44
+ | **`lens`** | enum | yes | `challenger` \| `architect` \| `subtractor` |
45
+ | **`finding_id`** | UUIDv4 | yes | Globally unique per finding line |
46
+ | **`severity`** | enum | yes | `low` \| `medium` \| `high` \| `critical` |
47
+ | **`confidence`** | enum | yes | `high` \| `medium` \| `low` — set by reconciliation |
48
+ | **`anti_slop_score`** | int **0–10** | yes | Per-lens score after rubric validation |
49
+ | **`finding_text`** | non-empty string | yes | Operator-auditable prose |
50
+ | **`status`** | enum | yes | `open` \| `resolved` \| `waived` |
51
+ | **`blocking`** | bool | yes | `true` when finding blocks convergence / rework exit |
52
+ | **`degraded_mode`** | bool | yes | `true` when single-model-multi-lens fallback active |
53
+
54
+ **Optional v1 fields** (allowed, not required): **`issue_key`** (normalized dedup key), **`single_finder`** (bool), **`rework_generation`** (int ≥ 0).
55
+
56
+ **Invariant**: `schema_version: 1` implicit via field inventory. Future bumps → `schema_version: 2`.
57
+
58
+ ### §3 — Three lenses + parallel jury (AC-3)
59
+
60
+ | Lens | Evaluation focus |
61
+ |------|------------------|
62
+ | **`challenger`** | Edge cases, race conditions, failure modes, boundary violations |
63
+ | **`architect`** | Coupling, layering, boundaries, dependency direction |
64
+ | **`subtractor`** | Over-engineering, premature abstraction, YAGNI, scope creep |
65
+
66
+ v1 runs **all three lenses** per critic invocation. Lens prompt templates live in **`.cursor/commands/sovereign-critic.md`** + runbook — no runtime lens invention.
67
+
68
+ **Reconciliation rule** (`reconcile_findings`):
69
+ 1. Compute **`issue_key`** per raw finding via **`compute_issue_key(finding_text)`** (§4).
70
+ 2. Group by `issue_key` across lenses.
71
+ 3. **≥2 lenses** share `issue_key` → `confidence=high`, `single_finder=false`.
72
+ 4. **Exactly 1 lens** → `confidence=medium`, `single_finder=true`.
73
+ 5. Emit merged finding rows with reconciled confidence; preserve per-lens `anti_slop_score`.
74
+
75
+ ### §4 — Issue normalization key v1 (R-0092 Q2)
76
+
77
+ Deterministic **`compute_issue_key(finding_text) -> str`**:
78
+
79
+ 1. Lowercase `finding_text`.
80
+ 2. Remove punctuation (`string.punctuation`).
81
+ 3. Collapse whitespace to single spaces; strip.
82
+ 4. Truncate to **80** characters on word boundary.
83
+ 5. SHA-256 hex digest; return first **16** hex chars prefixed `ik_`.
84
+
85
+ **Rejected**: semantic embedding cluster (**rejected** — non-deterministic, no stdlib embedding in v1).
86
+ **Rejected**: exact string match only (**rejected** — too brittle across lens phrasing).
87
+
88
+ ### §5 — `select_critic_model` algorithm (AC-2, AC-7)
89
+
90
+ `select_critic_model(producer_model_id, scratchpad, phase_id) -> SelectCriticResult`:
91
+
92
+ 1. If `producer_model_id` empty: resolve via **`model_tier_lib.resolve_model_for_phase(phase_id, scratchpad)`** → slug or alias.
93
+ 2. Map producer to tier:
94
+ - alias `fast` → **`cheap`**
95
+ - alias `inherit` or empty/omitted → **`balanced`**
96
+ - explicit slug: lookup catalog tier when `MODEL_RESOLVE=local_catalog`; else infer **`strong`** for non-alias slugs.
97
+ 3. Pick critic tier (opposition table):
98
+ | Producer tier | Critic tier |
99
+ |---------------|-------------|
100
+ | `strong` | `cheap` |
101
+ | `balanced` | `cheap` |
102
+ | `cheap` | `strong` |
103
+ 4. Resolve critic slug via **`model_tier_lib`** tier chain for synthetic phase **`sovereign-critic`** (fixed **`cheap`** tier in `DEFAULT_PHASE_TIER_MATRIX` override) **OR** direct tier→slug from catalog.
104
+ 5. Normalize comparison: aliases `fast`≠`inherit`≠vendor slug; if normalized critic slug **equals** producer slug → **`degraded=True`**, reason **`CROSS_MODEL_DEGRADED_MODE`**.
105
+ 6. Catalog miss / only one slug → **`degraded=True`**, reason **`CROSS_MODEL_CRITIC_MODEL_UNAVAILABLE`** or **`CROSS_MODEL_MODEL_COLLISION`**.
106
+
107
+ **R-0088 limitations** (documented, not blocked): Cursor Task tool `model` param may override subagent frontmatter; BYOK keys not inherited by subagents; `inherit` unreliable on some billing plans. Framework records **`degraded_mode=true`** and continues single-model-multi-lens — not a hard stop.
108
+
109
+ **Degraded fallback (L8)**: three sequential fresh subagent spawns (same `model_id`, different lens prompts); all findings record `degraded_mode=true`.
110
+
111
+ ### §6 — Anti-slop rubric + rework (AC-6)
112
+
113
+ **Per-lens deterministic checklist** (agent reports hits; lib validates floor):
114
+
115
+ | Lens | Checks (2.5 pts each, max 10) |
116
+ |------|-------------------------------|
117
+ | challenger | edge case cited; failure mode named; concurrency/race considered; input boundary tested |
118
+ | architect | coupling named; layer boundary stated; dependency direction explicit; interface contract mentioned |
119
+ | subtractor | unnecessary abstraction flagged; YAGNI applied; premature generalization challenged; scope creep identified |
120
+
121
+ `score_lens_antislop(lens, checklist_hits: dict[str, bool]) -> int` — count `True` × 2.5, round to int, clamp 0–10. Agent-reported score must be ≤ rubric ceiling or lib clamps down.
122
+
123
+ **Aggregate**: **`compute_anti_slop_aggregate(lens_scores) -> min(lens_scores)`** — stable for **US-0108**.
124
+
125
+ **Rework orchestration** (`/auto` post-phase hook):
126
+ 1. After `/sovereign-critic`, compute aggregate.
127
+ 2. If aggregate **< `CROSS_MODEL_ANTISLOP_THRESHOLD`** and `blocking` findings exist:
128
+ - Increment `rework_generation` for `(orchestrator_run_id, phase_id)`.
129
+ - If `rework_generation < CROSS_MODEL_REWORK_MAX`: re-spawn producer phase (fresh context per **US-0023**); pass critic summary as read-only input → reason **`CROSS_MODEL_ANTISLOP_FAIL`** (rework, not terminal).
130
+ - Else → **`CROSS_MODEL_REWORK_CAP_EXHAUSTED`** decision gate (operator waive or abort).
131
+
132
+ **`dev_to_qa.md` evidence tuple extension** (additive when `CROSS_MODEL_REVIEW=1`):
133
+
134
+ ```json
135
+ {
136
+ "critic_evidence": {
137
+ "producer_model_id": "...",
138
+ "critic_model_id": "...",
139
+ "anti_slop_aggregate": 7,
140
+ "rework_generation": 0,
141
+ "degraded_mode": false,
142
+ "findings_path": "handoffs/sovereign_critic_findings.jsonl"
143
+ }
144
+ }
145
+ ```
146
+
147
+ ### §7 — Isolation evidence `model_id` v2 extension (AC-4)
148
+
149
+ **Additive** field on **US-0048** evidence tuple (base five fields unchanged):
150
+
151
+ | Field | When required |
152
+ |-------|---------------|
153
+ | **`model_id`** | Required on **both** producer and critic isolation entries when `CROSS_MODEL_REVIEW=1`; omitted when `0` |
154
+
155
+ Fail-closed **`ISOLATION_EVIDENCE_MODEL_ID_MISSING`** when critic enabled and `model_id` absent on any in-scope isolation row for the phase pair.
156
+
157
+ Regression: **`test_us0104_us0048_compose_no_base_schema_change`** — US-0048 base schema literals unchanged.
158
+
159
+ ### §8 — Helper library contract (AC-5)
160
+
161
+ **Script**: **`scripts/sovereign_critic_lib.py`** (+ **`template/scripts/`** mirror).
162
+
163
+ | Function | Purpose |
164
+ |----------|---------|
165
+ | **`is_cross_model_review_enabled(scratchpad) -> bool`** | `CROSS_MODEL_REVIEW=1` |
166
+ | **`select_critic_model(producer_model_id, scratchpad, phase_id) -> SelectCriticResult`** | §5 algorithm |
167
+ | **`compute_issue_key(finding_text) -> str`** | §4 normalization |
168
+ | **`reconcile_findings(raw_findings) -> ReconciliationResult`** | §3 jury merge |
169
+ | **`compute_anti_slop_aggregate(lens_scores) -> int`** | `min(lens_scores)` |
170
+ | **`score_lens_antislop(lens, checklist_hits) -> int`** | §6 rubric |
171
+ | **`append_finding(path, entry) -> (bool, reason_code\|None)`** | Append-only JSONL + fsync |
172
+ | **`read_open_blocking(repo) -> list[dict]`** | Open `blocking=true` findings |
173
+ | **`resolve_finding(path, finding_id, status) -> bool`** | Patch status to `resolved`/`waived` |
174
+ | **`build_qa_cross_reviewer_block(repo) -> dict`** | `cross_reviewer_findings` for `qa-findings.md` |
175
+ | **`patch_ledger_cross_model_reviewed(...)`** | **US-0103** `build_new_entry(..., cross_model_reviewed=True)` when ledger enabled |
176
+ | **`schema_check(entry) -> (bool, error\|None)`** | 15-field v1 validation |
177
+ | **`self_test() -> bool`** | `[SOVEREIGN_CRITIC_SELF_TEST_OK]` |
178
+
179
+ Research stub ships schemas + self-test; full spawn orchestration deferred to execute tranche C.
180
+
181
+ ### §9 — Validator CLI contract (AC-8)
182
+
183
+ **Script**: **`scripts/sovereign_critic_validate.py`** (+ template mirror).
184
+
185
+ | CLI flag | Purpose |
186
+ |----------|---------|
187
+ | **`--file <path>`** | Validate single JSONL file |
188
+ | **`--repo <root>`** | Validate `handoffs/sovereign_critic_findings.jsonl` if present |
189
+ | **`--self-test`** | Library contract self-test |
190
+ | **`--enforce`** | Non-zero exit on fail-closed code |
191
+ | **`--open-blocking`** | List open blocking findings (stdout JSON) |
192
+
193
+ Output on success: **`[SOVEREIGN_CRITIC_VALIDATION_OK]`**.
194
+
195
+ ### §10 — `/sovereign-critic` command (AC-2)
196
+
197
+ **Path**: **`.cursor/commands/sovereign-critic.md`** (+ template mirror).
198
+
199
+ Invoked by **`/auto`** orchestrator **after** producer phase when `CROSS_MODEL_REVIEW=1`. Spawns fresh critic subagent per **US-0048** / **US-0023**. Inputs: `phase_id`, `role`, `evidence_ref`, `producer_model_id`, artifact digest. Outputs: append findings JSONL + optional `cross_reviewer_findings` in sprint `qa-findings.md`.
200
+
201
+ ### §11 — Reason code inventory (AC-8)
202
+
203
+ | Code | Blocking? | Surface |
204
+ |------|-----------|---------|
205
+ | **`CROSS_MODEL_REVIEW_DISABLED`** | no (info) | scratchpad gate off |
206
+ | **`CROSS_MODEL_CRITIC_SPAWN_FAILED`** | yes | orchestrator hook |
207
+ | **`CROSS_MODEL_MODEL_COLLISION`** | no → degraded | same slug resolved |
208
+ | **`CROSS_MODEL_ANTISLOP_FAIL`** | yes (rework) | score below threshold |
209
+ | **`CROSS_MODEL_REWORK_CAP_EXHAUSTED`** | yes (gate) | rework max hit |
210
+ | **`CROSS_MODEL_FINDINGS_INVALID`** | yes | schema validation |
211
+ | **`CROSS_MODEL_RECONCILE_FAILED`** | yes | jury merge error |
212
+ | **`CROSS_MODEL_DEGRADED_MODE`** | no (info) | single-model fallback |
213
+ | **`CROSS_MODEL_CRITIC_MODEL_UNAVAILABLE`** | yes → degraded | catalog miss |
214
+ | **`ISOLATION_EVIDENCE_MODEL_ID_MISSING`** | yes | critic enabled, evidence incomplete |
215
+
216
+ ### §12 — Contract tests + parity (AC-8)
217
+
218
+ Eight **`test_us0104_*`** markers in **`tests/us0104_contract_test.py`**:
219
+
220
+ 1. `test_us0104_scratchpad_keys_literals`
221
+ 2. `test_us0104_sovereign_critic_command_literals`
222
+ 3. `test_us0104_three_lens_enum_contract`
223
+ 4. `test_us0104_findings_jsonl_schema_contract`
224
+ 5. `test_us0104_reconciliation_agreement_branches`
225
+ 6. `test_us0104_model_id_isolation_evidence_extension`
226
+ 7. `test_us0104_antislop_rework_cap_literals`
227
+ 8. `test_us0104_degraded_fallback_zero_overhead`
228
+
229
+ Compose regression guards:
230
+ - `test_us0104_us0048_compose_no_base_schema_change`
231
+ - `test_us0104_us0110_critic_path_unchanged`
232
+
233
+ **`SOVEREIGN_CRITIC_PAIRS`** for `check_intake_template_parity.py --scope=sovereign-critic`:
234
+ - `scripts/sovereign_critic_lib.py` ↔ `template/scripts/sovereign_critic_lib.py`
235
+ - `scripts/sovereign_critic_validate.py` ↔ `template/scripts/sovereign_critic_validate.py`
236
+ - `.cursor/commands/sovereign-critic.md` ↔ `template/.cursor/commands/sovereign-critic.md`
237
+ - `.cursor/scratchpad.md` ↔ `template/.cursor/scratchpad.md` (`CROSS_MODEL_*` block)
238
+ - `decisions/DEC-0104.md` ↔ `template/decisions/DEC-0104.md`
239
+
240
+ ### §13 — Composition rules
241
+
242
+ | Story | Compose rule |
243
+ |-------|--------------|
244
+ | **US-0048** | Base isolation tuple unchanged; `model_id` additive v2 only. |
245
+ | **US-0069** | Phase→role matrix unchanged; critic spawned by orchestrator command, not new canonical phase role. |
246
+ | **US-0023** | Fresh-context semantics unchanged; rework re-spawn uses fresh marker. |
247
+ | **US-0110** | `CRITIC_PATH` / conjunct-3 degrade matrix unchanged; US-0104 populates artifact. |
248
+ | **US-0103** | `cross_model_reviewed` field already in 12-field v1; no ledger schema change. |
249
+ | **US-0101** / **US-0102** | Model resolution read-only via `model_tier_lib`; no precedence chain amendment. |
250
+ | **US-0108** | Consumes stable `min(lens_scores)` aggregate — do not change formula without coordinated DEC. |
251
+ | **US-0045** | US-0104 remains **OPEN** until release closure. |
252
+
253
+ ### §14 — Risks
254
+
255
+ | Risk | Mitigation |
256
+ |------|------------|
257
+ | **R1** Model routing unreliable (**R-0088**) | Deterministic degraded fallback §5 |
258
+ | **R2** Phase latency / token cost | Default-off `CROSS_MODEL_REVIEW=0` |
259
+ | **R3** Rework oscillation | `CROSS_MODEL_REWORK_MAX` + decision gate §6 |
260
+ | **R4** Anti-slop subjectivity | Deterministic checklist rubric §6 |
261
+ | **R5** Jury dedup drift | Stable `issue_key` algorithm §4 |
262
+ | **R6** **US-0108** aggregate coupling | Frozen `min(lens_scores)` formula |
263
+
264
+ ### §15 — Implementation tranche order
265
+
266
+ 1. **Tranche A** — scratchpad keys + reason codes + `DEC-0104` template mirror
267
+ 2. **Tranche B** — **`scripts/sovereign_critic_lib.py`** full API + self-test
268
+ 3. **Tranche C** — **`scripts/sovereign_critic_validate.py`** + `/sovereign-critic` command
269
+ 4. **Tranche D** — `/auto` post-phase hook + rework loop + `dev_to_qa.md` tuple
270
+ 5. **Tranche E** — isolation `model_id` v2 + ledger hook + eight `test_us0104_*` + parity + architecture `# US-0104`
271
+
272
+ ## References
273
+
274
+ - `docs/engineering/research.md` — **`R-0092`**
275
+ - `docs/product/backlog.md` — **`## US-0104`**
276
+ - `handoffs/intake_evidence/intake-sovereign-20260627-01.json`
277
+ - `scripts/sovereign_convergence_lib.py` — **`CRITIC_PATH`**
278
+ - `scripts/decision_ledger_lib.py` — **`cross_model_reviewed`**
279
+ - `scripts/model_tier_lib.py` — model resolution (read-only)
@@ -0,0 +1,231 @@
1
+ # DEC-0105: Sovereign Memory (US-0105)
2
+
3
+ - **Date**: 2026-06-29
4
+ - **Status**: Accepted
5
+ - **Story**: `US-0105`
6
+ - **Composes on**: **US-0029** (external research — do not amend), **US-0080** / **DEC-0062** (token-cost — lib-side digest only), **US-0103** (per-run ledger — optional promotion), **US-0072** / **DEC-0054** (triad — sovereign-memory explicitly excluded), **US-0096** (lean memory layers — distinct cross-run substrate)
7
+ - **Related**: `R-0093` (research anchor), **US-0107** (drain-generate consumer — read API stability), **US-0110** (convergence — orthogonal), **US-0045** (status authority), **US-0017** (template parity)
8
+ - **Supersedes**: none
9
+ - **Default-off**: `SOVEREIGN_MEMORY=0` — zero overhead when off
10
+
11
+ ## Context
12
+
13
+ Project-level institutional memory for the sovereign-loop batch. Cross-run learnings (decisions, mistakes, patterns, plan drift) persist in bounded JSONL artifacts under `docs/engineering/sovereign-memory/` and inject into phase spawns as a char-capped digest. **US-0029** / **US-0080** / **US-0103** / **US-0072** / **US-0096 are UNCHANGED** — US-0105 adds a new directory surface, scratchpad keys, lib API, and compose hooks only.
14
+
15
+ Research basis: `R-0093` Q1–Q7 closed.
16
+
17
+ ## Decision
18
+
19
+ ### §1 — Scratchpad keys (AC-1)
20
+
21
+ | Key | Values | Default | Notes |
22
+ |-----|--------|---------|-------|
23
+ | **`SOVEREIGN_MEMORY`** | **`0`** \| **`1`** | **`0`** | When `0`, no JSONL writes, no injection reads, no digest assembly — **zero overhead**. |
24
+ | **`SOVEREIGN_MEMORY_TOP_N`** | int ≥ **0** | **`5`** | Global recent pool size (all four JSONL families). |
25
+ | **`SOVEREIGN_MEMORY_TOP_K`** | int ≥ **0** | **`3`** | High-impact pool size (`patterns` + `mistakes` only). |
26
+ | **`SOVEREIGN_MEMORY_MAX_CHARS`** | int ≥ **0** | **`2048`** | Hard cap on assembled `digest_text`. |
27
+ | **`SOVEREIGN_MEMORY_JSONL_MAX_LINES`** | int ≥ **1** | **`500`** | Active JSONL line cap before archive rollover. |
28
+
29
+ Published in **`.cursor/scratchpad.md`** (+ **`template/.cursor/scratchpad.md`** byte-parity per **US-0017**).
30
+
31
+ **Lib constant** (not scratchpad): **`SOVEREIGN_MEMORY_READ_TAIL`** default **500** — max tail lines read per file during digest assembly.
32
+
33
+ **Rejected**: always-on memory (**rejected** — token/latency cost; default-off discipline).
34
+ **Rejected**: empty tracked JSONL seed files (**rejected** — create-on-first-write; ship `.gitkeep` only per **R-0093** Q1).
35
+ **Rejected**: per-file round-robin injection (**rejected** — global time/score merge simpler and more predictable).
36
+ **Rejected**: external memory package dependency (**rejected** — stdlib-only; **R-0093** eml-memory evaluation).
37
+
38
+ ### §2 — Directory surface + bootstrap (AC-2)
39
+
40
+ **Active path**: **`docs/engineering/sovereign-memory/`** (+ **`template/docs/engineering/sovereign-memory/`** mirror).
41
+
42
+ | Artifact | Path | Write owner | Injected v1? |
43
+ |----------|------|-------------|--------------|
44
+ | Decisions log | `decisions-log.jsonl` | Curator promotion; explicit phase append | yes |
45
+ | Mistakes | `mistakes.jsonl` | Orchestrator mistake hooks | yes (top-K) |
46
+ | Patterns | `patterns.jsonl` | Phase roles; curator consolidation | yes (top-K) |
47
+ | Plan drift | `plan-drift-register.jsonl` | Plan-fidelity / scope hooks | yes (top-N) |
48
+ | Retrospectives | `retrospectives/<sprint_id>.md` | Curator `/refresh-context` | **no** (human audit) |
49
+
50
+ **Archive path**: **`docs/engineering/sovereign-memory-archive/<basename>-<YYYYMMDDTHHMMSSZ>.jsonl`** — distinct from **US-0072** triad archives.
51
+
52
+ **Bootstrap**: **create-on-first-write** for JSONL; ship **`.gitkeep`** + **`retrospectives/.gitkeep`** only — no empty tracked JSONL seeds.
53
+
54
+ ### §3 — JSONL v1 schemas (AC-2)
55
+
56
+ Four append-only families sharing base fields:
57
+
58
+ | Field | Type | Required | Notes |
59
+ |-------|------|----------|-------|
60
+ | **`schema_version`** | int | yes | **`1`** |
61
+ | **`ts`** | ISO 8601 UTC | yes | sortable |
62
+ | **`entry_id`** | UUIDv4 | yes | globally unique |
63
+ | **`source_orchestrator_run_id`** | string | no | provenance |
64
+ | **`source_story_id`** | string | no | e.g. `US-0105` |
65
+ | **`phase_id`** | string | no | **DEC-0086** canonical id |
66
+ | **`impact_score`** | int **0..100** | yes | high-impact pool sort key |
67
+ | **`text`** | non-empty string | yes | max **2000** chars |
68
+ | **`tags`** | string[] | yes | max **10** × **40** chars each |
69
+ | **`status`** | enum | yes | `active` \| `superseded` \| `archived` (default **`active`**) |
70
+ | **`provenance_ref`** | string | no | e.g. `R-0093`, `ledger:<decision_id>` |
71
+
72
+ **Family extensions**:
73
+
74
+ - **`decisions-log.jsonl`**: `decision_key` (SHA-256 prefix **16** hex), `decision_text`, `rationale` (max **2000** chars).
75
+ - **`mistakes.jsonl`**: `mistake_tag` ∈ {`fix_failed`, `revert_applied`, `plan_fidelity_violation`, `test_regression`, `scope_creep`}, `failure_reason_code`.
76
+ - **`patterns.jsonl`**: `pattern_id` (slug max **64**), `applies_to` (string[]).
77
+ - **`plan-drift-register.jsonl`**: `drift_type` ∈ {`ac_drop`, `ac_reorder`, `scope_add`, `plan_change`, `acceptance_drift`}, `from_artifact`, `to_artifact`; optional `ledger_decision_id`.
78
+
79
+ **Secret scan**: reject append when `text` / `rationale` / `decision_text` match **`SOVEREIGN_MEMORY_SECRET_DETECTED`** heuristics (mirror **US-0103** `LEDGER_SECRET_DETECTED`: `api_key=`, PEM blocks, `sk-`, `ghp_` prefixes).
80
+
81
+ **Validator**: **`scripts/sovereign_memory_validate.py`** (+ template mirror) — `--file`, `--repo`, `--family {decisions|mistakes|patterns|plan-drift|all}`, `--self-test`, `--enforce`. Exit **0/1/2** mirrors **DEC-0103** / **DEC-0104**. Success token: **`[SOVEREIGN_MEMORY_VALIDATION_OK]`**.
82
+
83
+ ### §4 — Helper library API (AC-3, AC-5, AC-6)
84
+
85
+ **Script**: **`scripts/sovereign_memory_lib.py`** (+ template mirror).
86
+
87
+ | Function | Purpose |
88
+ |----------|---------|
89
+ | `is_sovereign_memory_enabled(scratchpad) -> bool` | `SOVEREIGN_MEMORY=1` gate |
90
+ | `resolve_memory_dir(repo_root) -> Path` | `docs/engineering/sovereign-memory/` |
91
+ | `resolve_jsonl_path(family, repo_root) -> Path` | Per-family JSONL path |
92
+ | `read_entries(family, *, tail_n, active_only=True) -> (list, reason\|None)` | Bounded tail read; **`SOVEREIGN_MEMORY_READ_BOUND`** when truncated |
93
+ | `schema_check(entry, family) -> (bool, error\|None)` | Per-family v1 + secret scan |
94
+ | `compute_decision_key(decision_text) -> str` | Normalized SHA-256 prefix |
95
+ | `dedupe_decision(text, existing_keys) -> (key, is_duplicate)` | Pre-append dedup probe |
96
+ | `append_decision` / `append_mistake` / `append_pattern` / `append_drift` | Append-only + fsync + pre-append rollover hook |
97
+ | `build_injection_digest(repo, scratchpad) -> InjectionDigest` | §5 bounded digest |
98
+ | `promote_from_ledger(run_id, *, decision_types) -> (entry_ids, reason\|None)` | Refresh-context when **`AI_DECISION_LEDGER=1`** |
99
+ | `write_retrospective(sprint_id, body) -> (path\|None, reason\|None)` | Curator markdown under **`retrospectives/`** |
100
+ | `maybe_archive_jsonl(family) -> (ok, reason\|None)` | Line-cap rollover (§7) |
101
+ | `scan_secrets(text) -> ReasonCode\|None` | Pre-append secret guard |
102
+ | `self_test() -> bool` | **`[SOVEREIGN_MEMORY_SELF_TEST_OK]`** |
103
+
104
+ Research stub ships schemas + injection + self-test; append/rollover/promotion bodies finalized at execute.
105
+
106
+ ### §5 — Injection merge algorithm (AC-3, AC-4)
107
+
108
+ `build_injection_digest(repo, scratchpad) -> InjectionDigest` with `{digest_text, entry_ids[], char_count}`:
109
+
110
+ 1. **`SOVEREIGN_MEMORY=0`** → empty digest (`digest_text=""`, `entry_ids=[]`, `char_count=0`) — zero overhead.
111
+ 2. **Empty corpus** → placeholder **`(no sovereign memory entries)`**; non-fatal.
112
+ 3. **Read bound**: at most **`SOVEREIGN_MEMORY_READ_TAIL`** (default **500**) lines per file tail; emit **`SOVEREIGN_MEMORY_READ_BOUND`** as warning only.
113
+ 4. **Top-N recent**: merge all four families' `status=active` entries; sort **`ts` desc**, tie-break **`entry_id` asc**; take first **N** unique `entry_id`s.
114
+ 5. **Top-K high-impact**: from **`patterns`** + **`mistakes`** only; sort **`impact_score` desc**, tie-break **`ts` desc**, then **`entry_id` asc**; skip ids already in recent pool; take **K** more.
115
+ 6. **Digest layout**: `## Recent learnings` then `## High-impact patterns`; each line: `- [family] entry_id: text` (per-line text truncated to **160** chars before global cap).
116
+ 7. **Global cap**: hard truncate assembled digest to **`SOVEREIGN_MEMORY_MAX_CHARS`** (suffix **`...`** if truncated).
117
+ 8. **Retrospectives**: **not injected v1**.
118
+
119
+ **Phase spawn hook** (AC-4): when `SOVEREIGN_MEMORY=1`, spawn assembler appends read-only **`sovereign_memory_digest`** block after phase-context narrow-read, before role instructions. **US-0023** fresh-context boundary unchanged — digest is bounded additive input.
120
+
121
+ ### §6 — Mistake-tagging hooks (AC-6)
122
+
123
+ All hooks **no-op when `SOVEREIGN_MEMORY=0`**. Writes call `append_mistake()` with closed **`mistake_tag`** + orchestrator **`failure_reason_code`**.
124
+
125
+ | Trigger | `mistake_tag` | `failure_reason_code` | Wiring surface |
126
+ |---------|---------------|----------------------|----------------|
127
+ | Execute auto-loop exhausts fix attempts | `fix_failed` | `FIX_FAILED` | **`/auto`** post-execute when **`AUTO_IMPLEMENTATION_LOOP=1`** |
128
+ | Git revert / rollback in execute findings | `revert_applied` | `REVERT_APPLIED` | **`/execute`** rollback sub-step |
129
+ | Plan-fidelity hard stop | `plan_fidelity_violation` | `PLAN_FIDELITY_VIOLATION` | Compose **`decision_ledger_lib.classify_deviation`** — optional import |
130
+ | QA regression after green execute | `test_regression` | `TEST_REGRESSION` | **`/qa`** fail-closed (optional v1.1 defer allowed) |
131
+ | Extended-mode scope add without override | `scope_creep` | `PLAN_FIDELITY_SCOPE_GATE` | Same fidelity hook family |
132
+
133
+ **Dedup** (AC-6): `append_decision()` computes `decision_key = sha256(normalize(decision_text))[:16]`; duplicate → **`SOVEREIGN_MEMORY_DECISION_DUPLICATE`** (non-fatal skip).
134
+
135
+ ### §7 — JSONL rollover vs US-0072 (AC-2)
136
+
137
+ - **Not triad**: sovereign-memory JSONL **excluded** from **US-0072** / **DEC-0054** triad hot surfaces.
138
+ - **Rollover trigger**: on mutating append when active file line count **>** `SOVEREIGN_MEMORY_JSONL_MAX_LINES` → atomically move to archive path, create fresh empty file.
139
+ - **Fail-closed**: rollover I/O failure → **`SOVEREIGN_MEMORY_ARCHIVE_REQUIRED`** (block append).
140
+ - **Idempotency**: second rollover in same boundary must not duplicate archive packs.
141
+ - **Contrast**: triad rollover compacts hot phase context; sovereign-memory archive compacts cold institutional learnings.
142
+
143
+ ### §8 — Curator retrospective + ledger promotion (AC-5)
144
+
145
+ At **`/refresh-context`** after release segment close:
146
+
147
+ 1. `write_retrospective(sprint_id, body)` → `retrospectives/<sprint_id>.md` with `{sprint_id, story_ids[], release_ref, summary, learnings[], promoted_entry_ids[]}`.
148
+ 2. When both **`SOVEREIGN_MEMORY=1`** and **`AI_DECISION_LEDGER=1`**: `promote_from_ledger(run_id, decision_types=...)` copies subset to `decisions-log.jsonl` with `provenance_ref=ledger:<decision_id>`.
149
+ 3. When ledger off or filter empty → **`SOVEREIGN_MEMORY_PROMOTION_SKIPPED`** (info).
150
+
151
+ **US-0103 compose**: ledger remains per-run audit; promotion **reads** ledger context but **does not** mutate ledger schema.
152
+
153
+ ### §9 — Reason code inventory (AC-8)
154
+
155
+ | Code | Blocking? | Surface |
156
+ |------|-----------|---------|
157
+ | **`SOVEREIGN_MEMORY_DISABLED`** | no (info) | scratchpad gate off |
158
+ | **`SOVEREIGN_MEMORY_SCHEMA_INVALID`** | yes | validator / append |
159
+ | **`SOVEREIGN_MEMORY_APPEND_FAILED`** | yes | I/O / permission |
160
+ | **`SOVEREIGN_MEMORY_DECISION_DUPLICATE`** | no (skip append) | dedup on `decision_key` |
161
+ | **`SOVEREIGN_MEMORY_SECRET_DETECTED`** | yes | pre-append scan |
162
+ | **`SOVEREIGN_MEMORY_ARCHIVE_REQUIRED`** | yes | rollover fail-closed |
163
+ | **`SOVEREIGN_MEMORY_READ_BOUND`** | no (warning) | tail read truncated |
164
+ | **`SOVEREIGN_MEMORY_PROMOTION_SKIPPED`** | no (info) | ledger off or filter empty |
165
+
166
+ ### §10 — Contract tests + parity (AC-7, AC-8)
167
+
168
+ Eight **`test_us0105_*`** markers in **`tests/us0105_contract_test.py`**:
169
+
170
+ 1. `test_us0105_scratchpad_keys_literals`
171
+ 2. `test_us0105_sovereign_memory_directory_contract`
172
+ 3. `test_us0105_jsonl_schema_contract`
173
+ 4. `test_us0105_injection_digest_char_cap`
174
+ 5. `test_us0105_decision_dedup_branch`
175
+ 6. `test_us0105_mistake_tagging_literals`
176
+ 7. `test_us0105_zero_overhead_default`
177
+ 8. `test_us0105_compose_guards`
178
+
179
+ Compose regression guards (AC-8):
180
+
181
+ - `test_us0105_us0029_compose_no_research_schema_change`
182
+ - `test_us0105_us0080_injection_respects_char_cap`
183
+
184
+ **`SOVEREIGN_MEMORY_PAIRS`** for `check_intake_template_parity.py --scope=sovereign-memory`:
185
+
186
+ - `scripts/sovereign_memory_lib.py` ↔ `template/scripts/sovereign_memory_lib.py`
187
+ - `scripts/sovereign_memory_validate.py` ↔ `template/scripts/sovereign_memory_validate.py`
188
+ - `.cursor/scratchpad.md` ↔ `template/.cursor/scratchpad.md` (`SOVEREIGN_MEMORY_*` block)
189
+ - `docs/engineering/sovereign-memory/.gitkeep` ↔ `template/docs/engineering/sovereign-memory/.gitkeep`
190
+ - `decisions/DEC-0105.md` ↔ `template/decisions/DEC-0105.md`
191
+
192
+ ### §11 — Composition rules
193
+
194
+ | Story | Compose rule |
195
+ |-------|--------------|
196
+ | **US-0029** | `research.md` schema unchanged; `provenance_ref=R-xxxx` only in memory entries. |
197
+ | **US-0080** / **DEC-0062** | Lib-side digest truncation; **`TOKEN_PROFILE`** / slim-command manifests unchanged. |
198
+ | **US-0103** | Per-run ledger unchanged; optional `promote_from_ledger()` at refresh-context only. |
199
+ | **US-0072** / **DEC-0054** | Triad hot surfaces unchanged; sovereign-memory archive is separate path family. |
200
+ | **US-0096** | `pack.json` / `active-context.md` distinct; sovereign memory is cross-run project layer. |
201
+ | **US-0107** | Read API (`read_entries`, `build_injection_digest`) stable via `schema_version`. |
202
+ | **US-0023** | Fresh-context semantics unchanged; digest is read-only additive spawn input. |
203
+ | **US-0045** | US-0105 remains **OPEN** until release closure. |
204
+
205
+ ### §12 — Risks
206
+
207
+ | Risk | Mitigation |
208
+ |------|------------|
209
+ | **R1** Token bloat | Char cap + tail read + lib-side digest §5 |
210
+ | **R2** Research vs learnings overlap | `provenance_ref` only; no **`research.md`** schema change |
211
+ | **R3** Ledger vs decisions-log confusion | Distinct paths/schemas + runbook operator table |
212
+ | **R4** Stale injection | `status` supersession enum on all entries |
213
+ | **R5** Secret leakage | `SOVEREIGN_MEMORY_SECRET_DETECTED` scan on free-text fields |
214
+ | **R6** **US-0107** read API coupling | Stable lib surface; version via `schema_version` |
215
+
216
+ ### §13 — Implementation tranche order
217
+
218
+ 1. **Tranche A** — scratchpad keys + reason codes + `DEC-0105` template mirror + directory `.gitkeep`
219
+ 2. **Tranche B** — **`sovereign_memory_lib.py`** read/injection core + self-test
220
+ 3. **Tranche C** — append/dedup/rollover/promotion bodies + **`sovereign_memory_validate.py`**
221
+ 4. **Tranche D** — phase spawn digest hook + mistake-tagging orchestrator wiring
222
+ 5. **Tranche E** — curator retrospective + eight `test_us0105_*` + parity + runbook + architecture `# US-0105`
223
+
224
+ ## References
225
+
226
+ - `docs/engineering/research.md` — **`R-0093`**
227
+ - `docs/product/backlog.md` — **`## US-0105`**
228
+ - `handoffs/intake_evidence/intake-sovereign-20260627-01.json`
229
+ - `scripts/sovereign_memory_lib.py` — research stub + self-test
230
+ - `scripts/decision_ledger_lib.py` — promotion compose (read-only)
231
+ - `handoffs/archive/po-to-tl-pack-20260628-f.md` — discovery L1–L12