pi-aia-asf 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,79 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.9.0] - 2026-09-27
11
+
12
+ ### Added
13
+
14
+ - (describe changes for 0.9.0)
15
+
16
+
17
+ ### Added
18
+
19
+ - **Indexed reference structure (P-index)** — SKILL.md now carries a reference
20
+ index table (file → covers → read when) so the agent finds the right
21
+ section on demand; each reference is a small single-concern doc with a
22
+ "Read when / Skip when" header. Default context cost is unchanged (SKILL.md
23
+ only); large work reads at most the 3–4 references its phase needs.
24
+ - **`references/06f-stability.md`** — stability patterns (timeouts, bounded
25
+ retries, circuit breaker, bulkheads, bounded result sets, fail fast,
26
+ graceful shutdown) + error-handling design rules + observability DoD for
27
+ service-like deliverables (health endpoint, structured logs, correlation
28
+ IDs, runbook) (P06, P08, P07).
29
+ - **`references/06g-test-design.md`** — risk-based test derivation
30
+ (likelihood × impact, technique selection table) + determinism rules
31
+ (P10, P11).
32
+ - **`references/06h-security.md`** — one-page threat model (trust boundaries,
33
+ STRIDE, registry), ASVS 5.0-based DoD with chapter/level selection,
34
+ supply-chain & secrets gates, and the agent/tool-surface chapter (P15,
35
+ P16, P17, P18).
36
+
37
+ ### Changed
38
+
39
+ - **05-plan.md** — PLAN.md template extended: Decisions (ADRs, P01), Quality
40
+ requirements (ISO 25010 scenarios, P02), Architecture evaluation (ATAM-lite,
41
+ P03), optional Views (P04).
42
+ - **04-adversarial.md** — added security lens (threat model) and architecture
43
+ lens (ATAM-lite) applied on every review (P15, P03).
44
+ - **06c-code-quality.md** — deep-module principle (interface simpler than
45
+ implementation, P05), code smells in 5 groups (P09), characterization
46
+ baseline for refactoring untested code (P13).
47
+ - **06b-testing-qa.md** — Rule 16 determinism (flaky = defect, P11), Rule 17
48
+ delivery report fields incl. not-tested register (P14), Rule 18
49
+ escaped-defect feedback loop (P21); DoD extended with delivery-report,
50
+ stability and security lines.
51
+ - **06e-code-health.md** — report-only depth metric (P05) + supply-chain
52
+ (npm audit/osv-scanner) and secrets (gitleaks) gates with config (P17).
53
+ - **07-release.md** — verify-first now re-runs the dependency/security check
54
+ before every release (P17 RV loop).
55
+ - **06-implementation.md** — drift self-correction: named anti-patterns,
56
+ stop → revert → self-correct → continue, escalate only on genuine
57
+ conflicts (P20).
58
+ - **06d-delegation.md** — Part 3: independent reviewer gate for large work
59
+ (fresh-context hostile reviewer, blockers vs suggestions, autonomous
60
+ triage, human only on genuine conflicts) (P19).
61
+ - **SKILL.md** — ASF self-modification policy: facts may be updated in the
62
+ same change; standards change only via the review process (P22).
63
+ - **test/test-improvements.mjs** — new suite (84 checks) verifying the
64
+ indexed structure and that each proposal landed in its designated file.
65
+ - **Configurable release policy (P23)** — `.asf-release.json` at the project
66
+ root (optional): `when` (user-stated conditional rules: "when X is true,
67
+ publish automatically; when Y is true, do this…") and `how` (user-stated
68
+ instructions with all technical details that the ASF drives with judgment,
69
+ not a deterministic command list). Default with no file = current stance
70
+ (always send for final review); user definition takes precedence.
71
+ **Test on publish / in production, if applicable — always** (package →
72
+ clean-room registry install + observable end state; deployed app → prod
73
+ smoke test). The ASF's own release procedure is codified via the same
74
+ mechanism (repo `.asf-release.json`).
75
+
76
+ ## [0.8.1] - 2026-09-25
77
+
78
+ ### Added
79
+
80
+ - (describe changes for 0.8.1)
81
+
82
+
10
83
  ## [0.8.0] - 2026-09-12
11
84
 
12
85
  ### Added
package/README.md CHANGED
@@ -57,7 +57,7 @@ Then `/reload`.
57
57
 
58
58
  ## How it works
59
59
 
60
- - **Skill** (`skills/aia-asf/SKILL.md`) — the workflow itself, with per-phase reference guides in `references/`.
60
+ - **Skill** (`skills/aia-asf/SKILL.md`) — the workflow itself, with per-phase reference guides in `references/`. The skill uses **progressive disclosure**: only SKILL.md (the phase flow + gates + a reference index) loads by default; each reference is a small single-concern doc the agent reads on demand when its phase needs it (the index's "Read when" column tells it which). Small work reads at most the testing + code-quality standards; large work reads the ones its phase calls for.
61
61
  - **Extension** (`index.ts`) — `/asf` commands, per-project phase state (`~/.pi/agent/skills/aia-asf/projects/<project>/state.json`), dependency checks.
62
62
  - **Specs shared with pi-vigilant** — ASF drives `capture_spec` during intake; pi-vigilant re-verifies every spec at task end and blocks "done" while MUST specs are open. One spec file, two systems.
63
63
 
@@ -86,6 +86,28 @@ Flags: `--diff` (compare vs committed `.asf-code-health-baseline.json`),
86
86
  must be project devDependencies (eslint, eslint-plugin-sonarjs, jscpd, madge,
87
87
  dependency-cruiser); the gate never installs anything.
88
88
 
89
+ ## Release policy (configurable)
90
+
91
+ Publishing is the user's decision **by default** — the factory prepares the
92
+ release and gets explicit approval. A project can define its own release
93
+ policy in `.asf-release.json` at the project root (optional):
94
+
95
+ ```json
96
+ {
97
+ "when": "when the full suite is green and the bump is patch/minor → publish automatically; when the bump is major → send for final review; when the suite is not green → fix first, never publish",
98
+ "how": "To publish: connect with npm using the credentials in ~/.npmrc (2FA token; verify with `npm whoami`), sync git (`git pull --rebase`), run `npm run release <level>`, then verify the published version from the registry in a clean install..."
99
+ }
100
+ ```
101
+
102
+ - **`when`** — user-stated conditional rules the ASF evaluates against the
103
+ current state (suite green, bump scope, credentials) and acts on.
104
+ - **`how`** — user-stated instructions with all technical details; the ASF
105
+ drives the release with judgment, not as a deterministic command list.
106
+ - No file → default: always send for final review; standard workflow in
107
+ `references/07-release.md`.
108
+ - **Always test on publish / in production, if applicable** — verify the
109
+ shipped artifact from the registry (clean-room install) or smoke-test prod.
110
+
89
111
  ## Hygiene rules enforced
90
112
 
91
113
  - Test-first; only green commits
package/index.ts CHANGED
@@ -52,6 +52,7 @@ const QA_CHECKLIST: Array<{ key: string; label: string }> = [
52
52
  { key: "browser", label: "Web surfaces exercised through a real browser (n/a if none)" },
53
53
  { key: "specs", label: "Every MUST spec 'met' with concrete evidence" },
54
54
  { key: "trace", label: "Spec-to-code traceability: every met spec has outcome → codePath → test" },
55
+ { key: "code-health", label: "Code Health Gate passed (or enabled:false / gate.mode:warn with the report shown)" },
55
56
  { key: "surface", label: "Every delivered feature is consumed by a surface (UI or API) — nothing dead" },
56
57
  { key: "e2e", label: "Feature specs have an end-to-end behavioral test through the real entry point" },
57
58
  { key: "honest", label: "Skipped/inconclusive checks reported explicitly" },
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-aia-asf",
3
- "version": "0.8.0",
3
+ "version": "0.9.0",
4
4
  "description": "Ai Applied Agentic Software Factory — codifies the full software development flow: intake, research, spec capture, adversarial analysis, planning with approval gates, test-first implementation, and release. Requires pi-vigilant, pi-smart-web-search, pi-smart-fetch, and pi-aia-browser.",
5
5
  "keywords": [
6
6
  "pi-package",
@@ -119,6 +119,13 @@ Turn the intake answers + research into the authoritative spec set.
119
119
 
120
120
  > **Small work:** skip the formal gate. Do a quick mental pass over edge cases and failure modes while implementing; if something real surfaces, fix it or capture a spec. No user checkpoint.
121
121
 
122
+ > **Large work:** read `references/04-adversarial.md` for the full checklist
123
+ > (per-spec, architecture, security, code-quality, traceability lenses). For
124
+ > security, also read `references/06h-security.md` — write the one-page threat
125
+ > model (trust boundaries + STRIDE + registry) as part of the plan. For
126
+ > stability, ask the `references/06f-stability.md` questions about every
127
+ > external call in the design.
128
+
122
129
  Challenge the plan like a hostile reviewer before committing to it. For each spec and the overall design, ask and resolve:
123
130
 
124
131
  - **Edge cases** — empty input, zero users, max load, missing data, concurrency
@@ -147,7 +154,11 @@ Write `PLAN.md` in the project root (repo root, or cwd if no repo). Structure:
147
154
  ## Goal
148
155
  ## Context
149
156
  ## Approach (from research, cited)
157
+ ## Decisions (ADRs — one per significant decision)
158
+ ## Quality requirements (ISO 25010 scenarios)
150
159
  ## Architecture / Design
160
+ ## Architecture evaluation (ATAM-lite)
161
+ ## Views (context / runtime / deployment — optional)
151
162
  ## Milestones (M1..Mn with exit criteria)
152
163
  ## Task list (per milestone, checkboxes)
153
164
  ## Dependencies (with licenses)
@@ -155,6 +166,9 @@ Write `PLAN.md` in the project root (repo root, or cwd if no repo). Structure:
155
166
  ## Definition of done (tests, deploy, release)
156
167
  ```
157
168
 
169
+ The full template with examples is in `references/05-plan.md` — read it before
170
+ writing the plan.
171
+
158
172
  Keep the plan **implementation-ready**: any competent engineer (or agent) can execute the task list without re-deriving decisions.
159
173
 
160
174
  **Gate 5 — MANDATORY user approval (large only)**: present the plan and ask explicitly:
@@ -195,6 +209,11 @@ Keep the plan **implementation-ready**: any competent engineer (or agent) can ex
195
209
  > 4 concurrent writers left only 1 of 4 edits, silently, all exiting 0 — and **exit code 0
196
210
  > does not mean success**, so always validate the returned output against what you asked for.
197
211
 
212
+ > **Read on demand (only if they apply):** `references/06f-stability.md` if the
213
+ > deliverable makes external calls; `references/06g-test-design.md` for the
214
+ > test plan or non-trivial tests; `references/06h-security.md` if the
215
+ > deliverable handles untrusted input, secrets, or agent-like tool surfaces.
216
+
198
217
  Execute the task list milestone by milestone. Discipline rules:
199
218
 
200
219
  1. **Test-first**: write/update tests before or with implementation; run them; only commit green.
@@ -218,8 +237,17 @@ Run the **Definition of Done checklist** in `references/06b-testing-qa.md` (Rule
218
237
 
219
238
  Also check the **modularity DoD** from `references/06c-code-quality.md` (Phase 7 section): no duplicated shared logic, no hardcoded config values, every module tested standalone with the same calls it gets in the host, architecture writeup exists, existing functionality still green. **Documents too (06c Rule 9)**: no document so large that editing it is fragile — if a doc you worked on started causing edit failures/truncation, it was refactored (split by topic, parent kept as index, cross-references updated).
220
239
 
240
+ Also check the **stability DoD** (`references/06f-stability.md`): if the deliverable makes external calls, it has timeouts and bounded retries (idempotent only). And the **security DoD** (`references/06h-security.md`): no secrets in the repo, no untrusted input reaches a shell/query/path unvalidated, no unbounded resource use on user-controlled input; for agent-like tool surfaces, every tool is least-privilege with an abuse-case test.
241
+
221
242
  **Large work:** run `/asf verify` — it mechanically validates the **spec-to-code traceability matrix** (M1): every `met` spec must carry `trace` (outcome → codePath → testFile + assertion), testFile must exist, assertion must appear in it. FAIL rows block delivery. **Verify ingested specs from external planning docs too** — the doc's ✅ markers are claims, not evidence.
222
243
 
244
+ **Independent reviewer gate (large work, P19):** before delivery, run the
245
+ independent review (fresh-context hostile reviewer, blockers vs suggestions;
246
+ see `references/06d-delegation.md` Part 3). Triage autonomously — fix safe
247
+ blockers, apply safe suggestions, escalate to the user **only** on genuine
248
+ conflicts (conflicting requirements, product-level tradeoffs, scope
249
+ conflicts). Record the review + triage in the delivery report.
250
+
223
251
  **Code Health Gate (large work, Gate 8):** run `/asf health` — it measures convolution objectively at function / module / architecture level (see `references/06e-code-health.md`) and fails the gate when thresholds are crossed. The gate is **on by default** with conservative thresholds; configure via `.asf-code-health.json` at the project root. **Fix the cause, not the threshold** (06e Rule 2). Small work shows the report but never blocks.
224
252
 
225
253
  1. Run the full test suite (all of it, not a subset); fix failures; re-run until green.
@@ -231,7 +259,7 @@ Also check the **modularity DoD** from `references/06c-code-quality.md` (Phase 7
231
259
  3. Run `get_task_specs` and verify **every spec** with `update_spec_status` + concrete evidence (test output, build result, code inspection). Unverifiable → `partial` + ask the user. Never self-certify.
232
260
  4. **Report honestly**: never claim a check you didn't run; state explicitly anything
233
261
  skipped or inconclusive, and distinguish "tests pass" from "works for the user".
234
- 3. If the project is a library/package that the user publishes (npm, GitHub release): **offer** to run the release (see `references/07-release.md`): version bump, CHANGELOG, git tag, push. **Publishing is always the user's decision** — never publish without explicit approval. Optionally offer to set up a CI/CD pipeline for publishing.
262
+ 3. If the project is a library/package that the user publishes (npm, GitHub release): **check the release policy** — read `.asf-release.json` at the project root if present (see `references/07-release.md`). It defines **when** (user-stated conditional rules: "when X is true, publish automatically; when Y is true, do this…") and **how** (instructions the ASF drives with judgment — credentials, git sync, publish steps). If the policy authorizes automatic publishing for the current state, drive the release per `how`; otherwise **offer** to run the release and get explicit approval (the default stance: publishing is the user's decision). **Always test on publish / in production, if applicable** — verify the shipped artifact from the registry / smoke-test prod, never assume it works. Optionally offer to set up a CI/CD pipeline for publishing.
235
263
  4. Present a completion summary: what was built, specs met, tests passing, how to use it.
236
264
 
237
265
  ---
@@ -245,7 +273,8 @@ Also check the **modularity DoD** from `references/06c-code-quality.md` (Phase 7
245
273
  - ❌ Verifying a UI only via API/curl — browser testing is mandatory
246
274
  - ❌ Vague commits or CHANGELOG placeholders
247
275
  - ❌ Declaring done while specs are still `open`
248
- - ❌ Publishing anything without the user's explicit go-ahead
276
+ - ❌ Publishing anything beyond the configured release policy — no `.asf-release.json` means the default: always get the user's explicit go-ahead
277
+ - ❌ Publishing without testing the shipped artifact on publish / in production (if applicable)
249
278
  - ❌ Shipping a package without inspecting the packaged file list (`npm pack`)
250
279
  - ❌ Treating "no error" as "it worked" — malformed config is skipped **silently**
251
280
  - ❌ Verifying against a cached/stale install, or with an old duplicate still present
@@ -272,16 +301,50 @@ Also check the **modularity DoD** from `references/06c-code-quality.md` (Phase 7
272
301
  - ❌ Trusting unit tests as proof of wiring — assert the operator-facing outcome end-to-end
273
302
  - ❌ Trusting an external plan's ✅ (IMPROVEMENT-PLAN / delivery log) — ingest its items as specs and verify them
274
303
  - ❌ Implementing a spec literally when it creates product tension — challenge it and resolve with the user
275
-
276
- ## References
277
-
278
- - `references/01-intake.md` — question bank and probing techniques (incl. external planning docs, M6)
279
- - `references/02-research.md` — research playbook with search templates
280
- - `references/04-adversarial.md` — adversarial checklist per area
281
- - `references/05-plan.md` — PLAN.md template with examples (incl. spec-to-code traceability matrix)
282
- - `references/06-implementation.md` — coding discipline details (incl. M4 challenge designs, M5 trace before claiming)
283
- - `references/06b-testing-qa.md` — **mandatory testing & QA standard** (14 rules + definition of done)
284
- - `references/06c-code-quality.md` — **mandatory modularity & maintainability standard** (9 rules, SSOT, testable-standalone, single escalation path, documents-are-code)
285
- - `references/06e-code-health.md` — **Code Health Gate** (objective convolution measurement; Gate 8 for large work)
286
- - `references/06d-delegation.md` — **intercom & subagents**: when to message another session, when to spawn isolated workers, the no-mutual-dependencies rule, and why exit codes lie
287
- - `references/07-release.md` — release workflow (versioning, CHANGELOG, tags, npm, CI/CD)
304
+ - ❌ Shallow pass-through modules that add no abstraction; a new layer that hides nothing (06c P05)
305
+ - ❌ Refactoring untested code without a characterization baseline (06c P13)
306
+ - ❌ Fixing a smell instance by instance instead of the design (06c P09)
307
+ - ❌ Drift: scope creep, approach drift, gate drift — stop, revert, self-correct (06-implementation P20)
308
+ - ❌ Shipping a delivery summary without the report fields: spec status + evidence, test results, skipped checks, not-tested register (06b Rule 17)
309
+ - ❌ Letting an escaped defect teach nothing — ask which gate should have caught it and close the gap (06b Rule 18)
310
+ - ❌ Self-modifying the ASF's own rules during a task run — standards change only via the review process (P22)
311
+
312
+ ## Reference index (read on demand — never load all)
313
+
314
+ Each reference is a small, single-concern doc. **Read only the ones your
315
+ current phase needs** — the "Read when" column is the index. Small work
316
+ reads at most 06b + 06c; large work reads the ones its phase calls for.
317
+
318
+ | File | Covers | Read when |
319
+ |---|---|---|
320
+ | `01-intake.md` | question bank, probing, external planning docs (M6) | Phase 1 (large) |
321
+ | `02-research.md` | research playbook, search templates | Phase 2 (large) |
322
+ | `04-adversarial.md` | adversarial checklist (spec/architecture/security lenses) | Phase 4 (large) |
323
+ | `05-plan.md` | PLAN.md template + ADRs + quality scenarios + ATAM-lite + views | Phase 5 (large) |
324
+ | `06-implementation.md` | coding discipline, M4/M5, drift self-correction (P20) | Phase 6 |
325
+ | `06b-testing-qa.md` | **mandatory** testing & QA standard (18 rules + DoD) | before writing tests (always) |
326
+ | `06c-code-quality.md` | **mandatory** modularity standard (9 rules, deep modules, smells) | before structuring code |
327
+ | `06d-delegation.md` | intercom & subagents + independent reviewer gate (P19) | before delegating; Phase 7 review (large) |
328
+ | `06e-code-health.md` | Code Health Gate (+ depth, supply-chain, secrets metrics) | Gate 8 (large) |
329
+ | `06f-stability.md` | stability patterns + error handling + observability (P06/P07/P08) | **if the deliverable makes external calls or is a service** |
330
+ | `06g-test-design.md` | risk-based test derivation + determinism | Phase 6 large (test plan) / non-trivial tests |
331
+ | `06h-security.md` | threat model, ASVS DoD, supply chain, agent tools | Phase 4/5 large; small = one line |
332
+ | `07-release.md` | release workflow + **configurable release policy** (`.asf-release.json`: when/how) + test-on-publish | before publishing |
333
+
334
+ ## ASF self-modification policy (P22)
335
+
336
+ The ASF's own rules, gates, and standards are **normative content** — they
337
+ must never be self-modified during a task run (no "while I'm here, let me
338
+ improve the rule").
339
+
340
+ - **Facts may be updated in the same change**: package versions, external
341
+ tool behavior, SOTA findings, URLs — these are non-normative and stay
342
+ current.
343
+ - **Standards change only through the review process**: propose it in
344
+ `docs/research/improvement-proposals.md`, review it, get approval, then
345
+ implement (the same flow that produced this version).
346
+ - **Escalate, don't edit**: if a task reveals the ASF's own rules are wrong
347
+ or missing, that is a proposal + a user decision, not a silent edit.
348
+ - **Rule 9 applies to the ASF too**: if SKILL.md or a reference outgrows
349
+ editability, split it by topic and keep the parent as an index — via the
350
+ review process.
@@ -8,6 +8,22 @@ Challenge every spec and design decision like a hostile reviewer. For each item,
8
8
 
9
9
  ## Per-spec questions
10
10
 
11
+ > **Security lens (always applied):** for every design under review, also ask
12
+ > the `references/06h-security.md` questions — where are the **trust
13
+ > boundaries** (where does untrusted input cross into trusted code)? What does
14
+ > STRIDE say about each (Spoofing, Tampering, Repudiation, Information
15
+ > disclosure, Denial of service, Elevation of privilege)? Is there a **threat
16
+ > registry** (threat → likelihood × impact → mitigation → status)? Any
17
+ > High×High threat unmitigated and unrecorded blocks delivery. **Large work
18
+ > writes the one-page threat model** (context view + STRIDE matrix + registry)
19
+ > as part of the plan (see `references/06h-security.md`).
20
+
21
+ > **Architecture lens (always applied):** for every architecture under review,
22
+ > ask the ATAM-lite questions (`references/05-plan.md`) — which quality goals
23
+ > matter most? Which 3–6 scenarios stress the architecture? Which decisions
24
+ > are **sensitivity/tradeoff points** (load-bearing for one goal, harmful to
25
+ > another)? What are the architectural risks and mitigations?
26
+
11
27
  > **Code-quality lens (always applied):** for every module/design under review,
12
28
  > also ask the `references/06c-code-quality.md` questions — is shared logic
13
29
  > duplicated anywhere? Is the single escalation path identifiable? Is the module
@@ -20,6 +20,57 @@ Existing system, repo layout, relevant prior work. Links to research sources.
20
20
  ## Approach
21
21
  Decided approach with rationale. Cite the research (package names, URLs).
22
22
 
23
+ ## Decisions (P01 — ADRs, one per significant decision)
24
+
25
+ For every decision with alternatives (stack, architecture, protocol, library):
26
+
27
+ ```markdown
28
+ ### <Decision title>
29
+ - **Decision:** <what we chose>
30
+ - **Options considered:** <2+ real options, not strawmen>
31
+ - **Rationale:** <why this one — cite the research>
32
+ - **Consequence:** <what this costs / what it enables>
33
+ ```
34
+
35
+ Five lines max per decision. Significant decisions live in the plan; if a
36
+ project accumulates many, move them to `docs/decisions/NNNN-<slug>.md` and
37
+ keep one line each in the plan. A decision with only one option is not a
38
+ decision — skip it.
39
+
40
+ ## Quality requirements (P02 — ISO 25010 scenarios)
41
+
42
+ Only the characteristics this work actually touches (cap ~8). For each:
43
+
44
+ | Characteristic | Scenario (stimulus → response → measure) | Acceptance |
45
+ |---|---|---|
46
+ | e.g. Performance | 1000 concurrent users submit a report → system responds | p95 < 2s |
47
+ | e.g. Reliability | DB restarts mid-ingest → no data loss | 0 lost records |
48
+ | e.g. Security | unauthenticated request → rejected | 401 |
49
+
50
+ Each scenario is testable: stimulus, response, measure. If you cannot write
51
+ the measure, the requirement is not real yet.
52
+
53
+ ## Architecture evaluation (P03 — ATAM-lite)
54
+
55
+ One page max. Before committing to the architecture:
56
+
57
+ 1. **Quality goals** — from the Quality requirements table (the 3–6 that
58
+ matter most).
59
+ 2. **Key scenarios** — the 3–6 scenarios that stress the architecture most
60
+ (highest risk × impact from Phase 4).
61
+ 3. **Sensitivity/tradeoff points** — which decisions are load-bearing for
62
+ which quality goal, and where two goals pull against each other (e.g.
63
+ performance vs security).
64
+ 4. **Architectural risks** — each with mitigation, or an explicit accepted
65
+ risk with rationale.
66
+
67
+ ## Views (P04 — optional, only when they earn their place)
68
+
69
+ Add a view only if it communicates something the prose does not: **context**
70
+ (external systems/interfaces), **runtime** (key flows/state), **deployment**
71
+ (environments/topology). Each view ≤15 lines (ASCII or mermaid). No view for
72
+ its own sake — the Architecture/Design prose is the primary artifact.
73
+
23
74
  ## Architecture / Design
24
75
  - Modules and their responsibilities (small, single-purpose — see `references/06c-code-quality.md`)
25
76
  - Data model / schema (if any)
@@ -5,6 +5,11 @@
5
5
  > functionality in exactly one implementation (SSOT, single escalation path),
6
6
  > never hardcode what config should drive, and keep every module testable
7
7
  > standalone outside the host.
8
+ >
9
+ > **Read on demand:** `references/06f-stability.md` when the deliverable makes
10
+ > external calls; `references/06g-test-design.md` when writing the test plan or
11
+ > non-trivial tests; `references/06h-security.md` when the deliverable handles
12
+ > untrusted input/secrets/agent tools.
8
13
 
9
14
  ## Test-first
10
15
 
@@ -83,3 +88,39 @@ Before marking anything ✅ (a spec, a milestone, a delivery-log item):
83
88
  A delivery log saying "IMP-006 delivered ✅" proves nothing. The code is the
84
89
  evidence. External planning docs (IMPROVEMENT-PLAN.md, PLAN.md, delivery logs)
85
90
  are inputs to spec capture — their ✅ markers are claims, never ground truth.
91
+
92
+ ## Drift self-correction (P20) — stop, revert, self-correct, continue
93
+
94
+ **Drift** = the implementation is moving away from the approved plan/specs
95
+ (scope creep, a different approach sneaking in, a shortcut that changes
96
+ behavior). It is the most common silent failure of a long implementation.
97
+
98
+ **Named drift anti-patterns** (if you catch yourself doing any of these, you
99
+ are drifting):
100
+
101
+ - **Scope creep** — adding features/edge cases the plan did not call for
102
+ ("while I'm here...").
103
+ - **Approach drift** — quietly switching from the approved approach to a
104
+ different one because it "feels simpler".
105
+ - **Spec drift** — implementing a *sensible* variant of a spec without
106
+ resolving the tension (M4 says: resolve it with the user first).
107
+ - **Gate drift** — skipping a gate "just this once" (tests, adversarial,
108
+ browser verification).
109
+ - **Quality drift** — accepting a shortcut that changes behavior or leaves
110
+ dead machinery, then marking it delivered (M5).
111
+
112
+ **Self-correction loop (autonomous, no user needed):**
113
+
114
+ 1. **Detect** — compare what you are about to do against the plan/specs. If
115
+ it is not in the plan, it is drift.
116
+ 2. **Stop** — do not continue the drift; do not "finish it first".
117
+ 3. **Revert** — undo the drifted change (git checkout / undo the edit). The
118
+ revert is cheap; the drift is not.
119
+ 4. **Self-correct** — re-read the plan/spec, implement the approved version.
120
+ 5. **Continue** — resume the task list.
121
+
122
+ **Escalate to the user only when the plan/requirements themselves are the
123
+ problem** (a genuine conflict, a product-level tradeoff, a scope change).
124
+ Then: stop, state the conflict plainly, propose the resolution, get the
125
+ decision — do not silently pick a side. Everything else, self-correct and
126
+ continue; the human decides only on genuine conflicts.
@@ -4,6 +4,10 @@ These rules are distilled from real failures in production sessions (pi-vigilant
4
4
  pi-aia-asf, pi-aia-browser, conversense, betamaxx). Each rule exists because
5
5
  skipping it **shipped a broken artifact**. They are not optional.
6
6
 
7
+ > **Test design:** for *which* tests to write (risk-based derivation) and how
8
+ > to keep them reliable (determinism), see `references/06g-test-design.md` —
9
+ > read it in Phase 6 large work or when the logic under test is non-trivial.
10
+
7
11
  ---
8
12
 
9
13
  ## Rule 1 — Test the ARTIFACT you ship, not the source you wrote
@@ -143,11 +147,15 @@ never arrives, the wiring is dead — that is a failed test.
143
147
  - [ ] Web surfaces exercised through a real browser
144
148
  - [ ] Every MUST spec `met` with concrete evidence (`update_spec_status`)
145
149
  - [ ] **Spec-to-code traceability: every `met` spec carries `trace` (outcome → codePath → testFile + assertion); `/asf verify` mechanically validates it (large work)**
150
+ - [ ] **Code Health Gate passed (or `enabled: false` / `gate.mode: warn` with the report shown)**
146
151
  - [ ] **Consumed by a surface: every delivered feature's output is visible in the product (UI or API) — nothing ships as dead machinery**
147
152
  - [ ] **E2E behavioral test: every feature spec has a test through the real entry point asserting the operator-facing outcome**
148
153
  - [ ] Unverifiable specs → `partial` + asked the user (never self-certified)
149
154
  - [ ] **Every long-running command ran under an explicit timeout with a stated expected duration (no unbounded waits)**
150
155
  - [ ] **No wait exceeded the 30 min (1800s) absolute ceiling — or it was explicitly waived for that single run, with a reason and a finite bound**
156
+ - [ ] **Delivery report complete (P14): every spec with status + evidence, every test run with result, every skipped/inconclusive check listed with reason, and the not-tested register (what was deliberately not tested and why)**
157
+ - [ ] **Stability line (06f): if the deliverable makes external calls, it has timeouts and bounded retries (idempotent only)**
158
+ - [ ] **Security line (06h): no secrets in the repo; no untrusted input reaches a shell/query/path unvalidated; no unbounded resource use on user-controlled input**
151
159
 
152
160
  ## Rule 11 — Report honestly
153
161
 
@@ -268,3 +276,52 @@ and only for a **single run**:
268
276
 
269
277
  **Applies at both scales.** Small work is not exempt: a hang wastes the same
270
278
  time regardless of how the task was classified.
279
+
280
+ ## Rule 16 — Determinism: a flaky test is a defect
281
+
282
+ A test that fails intermittently is a **defect in the test**, not bad luck.
283
+
284
+ - Never "re-run into green" — that is hiding a broken test (Rule 11).
285
+ - Fix it immediately (inject the clock, seed randomness, own the data, poll
286
+ instead of sleep, no order dependence, no real network in unit tests) — the
287
+ full rules are in `references/06g-test-design.md`.
288
+ - If a test cannot be made deterministic, **delete it and say so** in the
289
+ delivery report (Rule 11) — a flaky test is worse than no test because it
290
+ erodes trust in the whole suite.
291
+
292
+ ## Rule 17 — Delivery report (P14): the fields that make it verifiable
293
+
294
+ The completion summary is not a narrative — it is a **report with fields** a
295
+ reviewer can check:
296
+
297
+ - **Specs:** each spec → status (`met`/`partial`/`not-met`) → evidence
298
+ (test name + output, build result, code inspection).
299
+ - **Tests:** each suite → command → result (pass/fail/skip) → count. State
300
+ which suites you ran and which you did not.
301
+ - **Skipped/inconclusive:** every check not run or inconclusive, with the
302
+ reason (Rule 11).
303
+ - **Not-tested register:** what was deliberately not tested and why (e.g.
304
+ "payment webhook: no sandbox credentials — covered by contract test only").
305
+ An honest register beats a silent gap.
306
+ - **Artifact:** what was shipped (version, file list) and how it was verified
307
+ clean-room (Rule 5).
308
+
309
+ ## Rule 18 — Escaped-defect feedback loop (P21)
310
+
311
+ When a defect escapes to the user (a bug report, a failed integration, a
312
+ production incident), the fix is not the end — the **loop** is:
313
+
314
+ 1. Fix it (with a regression test, Rule 6).
315
+ 2. **Ask: which gate should have caught this?** Answer honestly — the
316
+ missing test technique (06g), the missing check (06b Rule 10), the
317
+ missing adversarial question (04-adversarial), the missing stability
318
+ pattern (06f), the missing security check (06h).
319
+ 3. **Close the gap** — add the check/rule/test so the same class of defect
320
+ cannot escape again. If the gap is in the ASF itself, that is a proposal
321
+ (docs/research/improvement-proposals.md), not a silent edit (P22).
322
+ 4. Record the loop in the delivery report (Rule 17) — what escaped, what
323
+ caught it, what changed.
324
+
325
+ > **Real failure:** a package shipped with a broken `files` allowlist; the
326
+ > fix landed but the *gap* ("no test inspects the packaged artifact") was not
327
+ > closed until Rule 1 was added. The loop is what prevents the next one.
@@ -29,6 +29,36 @@ existing functionality. Each rule below carries the lesson.
29
29
  - you cannot explain what the file does in one sentence
30
30
  - you are about to debug the same area a second time
31
31
 
32
+ ### Deep modules (P05) — interface simpler than implementation
33
+
34
+ > *"A module is deep if its interface is much simpler than its
35
+ > implementation. Shallow modules are a design flaw."* — Ousterhout, *A
36
+ > Philosophy of Software Design*
37
+
38
+ - **Depth is the goal, not size.** A deep module hides a lot behind a small
39
+ interface (e.g. a storage layer with one `get(key)`). A shallow module
40
+ exposes nearly as much interface as it implements (e.g. a "wrapper" that
41
+ just forwards a call with no added value).
42
+ - **Shallow pass-through modules are defects.** A function that adds nothing
43
+ (forwards args, renames, calls one thing) should be inlined — it costs
44
+ context without hiding anything. Exception: a seam that exists for
45
+ testability (06c Rule 4) or a boundary that hides a *real* variation.
46
+ - **New layer = new abstraction.** Do not add a layer unless it hides
47
+ something (a variation, a dependency, a policy). A layer that only passes
48
+ through is ceremony.
49
+ - **Minimize voodoo constants (Rule 3 counterweight).** Config drives real
50
+ variation; but a constant that only this module uses and never changes
51
+ belongs in the module as a named constant, not in a config file. The
52
+ question is *who varies it* — if nobody, it is not config.
53
+ - **Anti-patterns (P05):** temporal decomposition (functions named after
54
+ steps that share state and cannot be understood alone); back-door leakage
55
+ (a module reaches into another's internals instead of its interface);
56
+ information leakage (a caller knows more about a module's internals than
57
+ its interface should reveal).
58
+
59
+ > **Depth is measured, not judged** — see `references/06e-code-health.md`
60
+ > (report-only depth metric, no gate).
61
+
32
62
  ## Rule 2 — One implementation for shared functionality (single escalation path)
33
63
 
34
64
  > *"Make a separate module for ... communication and use IT instead of having
@@ -141,6 +171,24 @@ existing functionality. Each rule below carries the lesson.
141
171
  - This is the SSOT/refactor safety net: refactoring restructures *structure*,
142
172
  never *behavior*.
143
173
 
174
+ ### Characterization baseline (P13) — when refactoring code without tests
175
+
176
+ If you must refactor code that has **no test coverage**, do not guess what it
177
+ should do — **capture what it does** first:
178
+
179
+ 1. Write a **characterization test**: feed representative inputs, record the
180
+ actual outputs (including edge cases and error paths).
181
+ 2. The test asserts **current behavior** (even if it looks wrong) — it is a
182
+ safety net, not a spec.
183
+ 3. Refactor; the characterization test must stay green (Rule 8).
184
+ 4. Then (and only then) write the *spec* tests for the behavior you actually
185
+ want, and change the behavior deliberately with its own test.
186
+
187
+ > **Real failure:** a refactor of untested code "fixed" behavior the author
188
+ > believed was wrong — silently changing semantics and breaking a downstream
189
+ > consumer that depended on it. Characterization first, then change
190
+ > deliberately.
191
+
144
192
  ## Rule 9 — Documents are code: refactor them when they outgrow editability
145
193
 
146
194
  > *"Whenever documents you work on become so big that they start causing
@@ -175,18 +223,40 @@ fact of life. The same triggers that say "extract a module" for code say
175
223
 
176
224
  ---
177
225
 
226
+ ## Code smells — the 5 groups (P09)
227
+
228
+ Smells are *signals*, not verdicts — each one has a standard fix. Check for
229
+ them during Phase 4 (adversarial) and Phase 6 (implementation), and when the
230
+ Code Health Gate flags a metric (06e).
231
+
232
+ | Group | Smells | Standard fix |
233
+ |---|---|---|
234
+ | **Bloaters** | long method, large class, long parameter list, data clumps | Extract Method/Class, Introduce Parameter Object |
235
+ | **Object-orientation abusers** | switch on type, temporary field, refused bequest | Replace Conditional with Polymorphism, Extract Class |
236
+ | **Change preventers** | divergent change, shotgun surgery, parallel inheritance | Extract Class, Move Method, consolidate the variation |
237
+ | **Dispensables** | comments as excuses, duplicate code, dead code, speculative generality | Remove Duplication, Delete Dead Code, YAGNI |
238
+ | **Couplers** | feature envy, inappropriate intimacy, message chains, middle man | Move Method, Hide Delegate, inline the middle man |
239
+
240
+ **Rule of thumb:** a smell that appears once is a code-style question; a
241
+ smell that appears three times is a design problem — fix the design, not the
242
+ instances.
243
+
178
244
  ## Where this applies in ASF
179
245
 
180
246
  - **Phase 4 (adversarial)**: challenge the design — is there duplication?
181
247
  Where is the single source of truth? Is the module testable outside the
182
- host? What breaks if a config value changes?
248
+ host? What breaks if a config value changes? Are there **smells** (P09)?
249
+ Is the module **deep** (P05) or a shallow pass-through? Is any module
250
+ **temporally decomposed / leaking information** (P05 anti-patterns)?
183
251
  - **Phase 5 (PLAN.md)**: the Architecture/Design section must name the modules,
184
252
  their boundaries, the one-way dependencies, where shared truth lives, and
185
- how each module is tested standalone.
253
+ how each module is tested standalone. The quality requirements table (P02)
254
+ and ATAM-lite evaluation (P03) live here too.
186
255
  - **Phase 6 (implementation)**: apply Rules 1–8 as you build; extract modules
187
256
  when triggers fire; write the architecture doc alongside the code. Apply
188
257
  Rule 9 to the documents you write: SKILL.md, references, PLAN.md, README —
189
- split them when they outgrow editability.
258
+ split them when they outgrow editability. **Characterize before refactoring
259
+ untested code (P13)**; check for smells (P09) as you go.
190
260
  - **Phase 7 (verification)**: the DoD checklist includes: no duplicated shared
191
261
  logic (Rule 2), no hardcoded config values (Rule 3), every module tested
192
262
  standalone with the same calls (Rule 4), architecture doc written (Rule 6),
@@ -203,6 +273,12 @@ fact of life. The same triggers that say "extract a module" for code say
203
273
  - ❌ Refactoring "for fun" without the testability/debugging payoff
204
274
  - ❌ Shipping a module that cannot run outside the host
205
275
  - ❌ Skipping the architecture writeup ("the code is self-documenting")
276
+ - ❌ Shallow pass-through modules that add no abstraction (P05)
277
+ - ❌ A new layer that hides nothing (P05)
278
+ - ❌ Temporal decomposition — functions named after steps, sharing state (P05)
279
+ - ❌ Back-door leakage — reaching into another module's internals (P05)
280
+ - ❌ Refactoring untested code without a characterization baseline (P13)
281
+ - ❌ Fixing a smell instance by instance instead of the design (P09)
206
282
  - ❌ Letting a document grow until edits start breaking instead of splitting it
207
283
  (Rule 9)
208
284
  - ❌ A SKILL.md that is a wall of prose instead of a map to focused references
@@ -134,3 +134,55 @@ Delegation changes **who does the work**, never **what must be proven**:
134
134
  - The QA standard (06b) applies to delegated work identically. "A subagent did it" is not evidence.
135
135
  - Subagent findings enter the record as **claims** until verified against code or tests.
136
136
  - For LARGE work, the approval gate is unaffected: no subagent may start implementation before plan approval.
137
+
138
+ ---
139
+
140
+ ## Part 3 — Independent reviewer gate (P19, large work)
141
+
142
+ Before delivery of **large** work, run an **independent review**: a second
143
+ agent with fresh context, no exposure to your reasoning, asked to attack the
144
+ result. It is the strongest anti-rubber-stamp check available (Part 2:
145
+ fresh-perspective review) and it reuses the subagent machinery above.
146
+
147
+ ### How
148
+
149
+ Spawn a subagent (Part 2, direct subprocess or extension) with a read-only
150
+ tool set and this adversarial prompt shape:
151
+
152
+ ```
153
+ You are a hostile reviewer of a delivered change. Review ONLY the diff and
154
+ its tests. Classify every finding as:
155
+ - BLOCKER: the change is wrong, breaks existing behavior, or violates a
156
+ stated spec/gate — must be fixed before delivery.
157
+ - SUGGESTION: improvement that does not block delivery.
158
+ - QUESTION: something you cannot determine from the diff.
159
+ Return: findings list, nothing else. Do not fix anything.
160
+ ```
161
+
162
+ Give it: the diff (or the files changed), the spec list, and the test
163
+ results. Nothing else — no narrative, no context, no "here's what I did".
164
+
165
+ ### Triage (autonomous — human only on genuine conflicts)
166
+
167
+ - **BLOCKER that is safe to fix** (a wrong value, a missing test, a broken
168
+ gate) → **fix it yourself**, add a regression test, re-run the suite. No
169
+ user checkpoint.
170
+ - **SUGGESTION that is safe and clearly better** → apply it. If it is a
171
+ taste call, note it and move on.
172
+ - **QUESTION / ambiguous finding** → investigate; if it resolves, resolve it.
173
+ - **Escalate to the user ONLY on genuine conflicts** — conflicting
174
+ requirements, product-level tradeoffs, scope conflicts, or a finding that
175
+ contradicts an approved decision. State the conflict plainly and propose a
176
+ resolution; never silently pick a side.
177
+
178
+ ### Rules
179
+
180
+ 1. **Fresh context is the point** — the reviewer must not have seen your
181
+ reasoning; do not pre-answer its questions.
182
+ 2. **Read-only tools** — the reviewer never edits; it reports, you fix.
183
+ 3. **Its findings are claims** — verify each blocker against the code/test
184
+ before acting (M5). A reviewer can be wrong too.
185
+ 4. **Record it** — the review + triage outcome goes in the delivery report
186
+ (06b Rule 17): what was found, what was fixed, what was escalated.
187
+ 5. **Small work: none.** The reviewer gate is large-work only; small work
188
+ relies on the standard gates (06b DoD).
@@ -24,6 +24,21 @@ Three levels, each with objective thresholds (all configurable):
24
24
  | module | duplication | jscpd | 5% (min 5 lines / 50 tokens) |
25
25
  | module | circular imports | madge `--circular` | any cycle = fail |
26
26
  | architecture | dependency rules | dependency-cruiser | opt-in (off by default) |
27
+ | design | module depth (interface vs implementation) | report-only heuristic | report-only (never gates) |
28
+ | supply chain | dependency vulnerabilities | npm audit / osv-scanner | warn (high/critical fails) |
29
+ | secrets | secrets in repo | gitleaks | warn (any finding fails) |
30
+
31
+ **Depth metric (P05, report-only):** a heuristic that flags **shallow
32
+ pass-through modules** — modules whose interface is nearly as large as their
33
+ implementation (e.g. a wrapper that forwards every call with no added
34
+ abstraction). It is **never a gate**: depth is a judgment call (06c Rule 1
35
+ Deep modules), and the heuristic only *surfaces candidates* for review.
36
+
37
+ **Supply chain & secrets (P17):** `npm audit`/`osv-scanner` and `gitleaks`
38
+ run as part of the gate. They are **warn by default** (report, don't block)
39
+ for small work; for large work a high/critical vulnerability in a runtime
40
+ dependency or any secret finding blocks delivery. See
41
+ `references/06h-security.md` for the full rules.
27
42
 
28
43
  Metrics for languages the project does not use are silently skipped. Metrics
29
44
  whose tool is not installed are skipped (or fail the gate when
@@ -68,6 +83,13 @@ switched on/off independently.
68
83
  "architecture": {
69
84
  "dependencyRules": { "enabled": false, "config": ".asf-code-health.rules.mjs" }
70
85
  },
86
+ "supplyChain": {
87
+ "npmAudit": { "enabled": true, "failOn": "high" },
88
+ "osvScanner": { "enabled": false }
89
+ },
90
+ "secrets": {
91
+ "gitleaks": { "enabled": true }
92
+ },
71
93
  "ignore": ["**/node_modules/**", "**/dist/**", "**/test/**", "**/*.test.*"]
72
94
  }
73
95
  ```
@@ -0,0 +1,97 @@
1
+ # 06f — Stability & Error Handling
2
+
3
+ > **Read when:** the deliverable makes external calls (network, processes,
4
+ > shared resources), is a service/daemon, or has retryable operations — large
5
+ > work.
6
+ > **Skip when:** small work; pure computation with no external calls; one-off
7
+ > scripts with no retryable surface.
8
+ >
9
+ > **Scale note:** these are *design rules for the deliverable*, not ceremony.
10
+ > If the deliverable has no external calls, the whole file is irrelevant — do
11
+ > not read it.
12
+
13
+ Source: Release It! (Nygard) stability patterns; Ousterhout error-handling
14
+ philosophy; OWASP A10:2025 (Mishandling of Exceptional Conditions).
15
+
16
+ ---
17
+
18
+ ## 1. Stability patterns (apply when the deliverable makes external calls)
19
+
20
+ Every external call is an integration point — the most common source of
21
+ production failure. The patterns below are the standard set; apply the ones
22
+ that fit, skip the rest.
23
+
24
+ | Pattern | One-line rule |
25
+ |---|---|
26
+ | **Timeouts** | Every external call has an explicit timeout: connection, read, and total (a call that hangs is worse than a call that fails). |
27
+ | **Bounded retries** | Retries are bounded (finite count), exponential backoff + jitter, and only for **idempotent** operations. Never retry a non-idempotent write blindly. |
28
+ | **Circuit breaker** | At integration points: closed → open after repeated failures → half-open probe → closed. Prevents hammering a dead dependency. |
29
+ | **Bulkheads** | Shared resources (connection pools, thread pools, rate limits) are partitioned so one consumer's exhaustion cannot starve others. |
30
+ | **Bounded result sets** | Any query/list/stream is bounded (limit, pagination, max size). Unbounded result sets are a stability antipattern. |
31
+ | **Fail fast** | Validate inputs at the boundary and fail immediately with a clear error — do not fail halfway through a multi-step operation. |
32
+ | **Graceful shutdown** | On shutdown: stop accepting new work, let in-flight work finish or time out, flush state, release resources. |
33
+
34
+ **DoD line (large work):** *"if the deliverable makes external calls, it has
35
+ timeouts and bounded retries (idempotent only)."*
36
+
37
+ ## 2. Error-handling design rules
38
+
39
+ - **Define errors out of existence** — the best error handling removes the
40
+ error condition (validate at the boundary, provide defaults, make the
41
+ impossible state unrepresentable). Only handle what cannot be prevented.
42
+ - **Handle at the boundary** — catch at the edge (HTTP handler, CLI entry,
43
+ event consumer), convert to the appropriate response; do not scatter
44
+ try/catch through core logic.
45
+ - **Generic to the user, detailed to the log** — user-facing errors never leak
46
+ internals (stack traces, SQL, paths); logs carry the full detail. (OWASP
47
+ A10:2025.)
48
+ - **Idempotency is a design property** — anything retryable (webhooks, jobs,
49
+ writes) must be idempotent (idempotency keys, unique constraints, natural
50
+ keys). Retries are only safe on idempotent operations.
51
+ - **Never use exceptions for control flow** — expected outcomes are values
52
+ (results, optionals, error unions); exceptions are for exceptional
53
+ conditions only.
54
+ - **Fail fast** — a broken precondition is a bug; surface it immediately
55
+ rather than degrading silently.
56
+
57
+ ## 3. Observability — service-like deliverables (P07, large work)
58
+
59
+ If the deliverable is a service/daemon (long-running, serves requests), the
60
+ DoD includes:
61
+
62
+ - **Health/readiness endpoint** — `/health` (or equivalent) that reports
63
+ liveness and readiness; a service that cannot report its own state is
64
+ undebuggable in production.
65
+ - **Structured logs to stdout** — machine-parseable (JSON lines), not
66
+ scattered `console.log`; logs are the primary debugging surface.
67
+ - **Correlation IDs** — one ID per request/job flows through every log line,
68
+ so a single user action is traceable across modules.
69
+ - **Metrics** — where a metrics system exists (Prometheus, etc.): the
70
+ counters/gauges that answer "is it working?" (request rate, error rate,
71
+ latency). No metrics system → skip, and say so.
72
+ - **Runbook in README** — start, stop, health check, known issues. An
73
+ operator should be able to run and diagnose it from the README alone.
74
+
75
+ **DoD line (large work, service-like):** *"health endpoint, structured logs,
76
+ correlation IDs, and a runbook exist."*
77
+
78
+ ## 4. Where this applies in ASF
79
+
80
+ - **Phase 4 (adversarial):** for each external call in the design, ask — what
81
+ is the timeout? Is the retry bounded and idempotent? What happens when the
82
+ dependency is down (circuit breaker/bulkhead)? What is the max result size?
83
+ - **Phase 5 (PLAN.md):** the Architecture/Design section names the stability
84
+ patterns chosen for each integration point.
85
+ - **Phase 6 (implementation):** apply the patterns as you build; never ship an
86
+ unbounded wait, retry, or result set.
87
+ - **Phase 7 (verification):** DoD includes the stability line above.
88
+
89
+ ## Anti-patterns
90
+
91
+ - ❌ Unbounded retries on non-idempotent writes
92
+ - ❌ No timeout on an external call ("it always returns fast")
93
+ - ❌ Retrying a POST that creates a resource on every attempt
94
+ - ❌ One connection pool shared by all consumers with no bulkhead
95
+ - ❌ Loading an entire table into memory because "it's small"
96
+ - ❌ Leaking stack traces to end users
97
+ - ❌ try/catch around every line of core logic
@@ -0,0 +1,88 @@
1
+ # 06g — Test Design: Risk-Based Derivation & Determinism
2
+
3
+ > **Read when:** Phase 6 large work (writing the test plan) or when the logic
4
+ > under test is non-trivial (branching, state, boundaries, external
5
+ > interactions).
6
+ > **Skip when:** small work; simple linear code where the standard test-first
7
+ > rules in 06b suffice.
8
+ >
9
+ > **Scale note:** this file is about *which tests to write and how to keep
10
+ > them reliable* — it never adds ceremony; it replaces guesswork.
11
+
12
+ Source: ISTQB test design techniques; Kaner's risk-based testing; the
13
+ determinism rules distilled from flaky-test failures.
14
+
15
+ ---
16
+
17
+ ## 1. Risk-based test derivation
18
+
19
+ Test effort follows **risk = likelihood × impact**, not coverage for its own
20
+ sake. Derive the test list from the spec's risk profile:
21
+
22
+ 1. **List the risks** (from Phase 4 adversarial analysis): what can break,
23
+ how likely, how bad.
24
+ 2. **Score each** (High/Medium/Low on both axes).
25
+ 3. **Test High × High first** — those get the most thorough technique set.
26
+ 4. **Low × Low** gets a smoke check or nothing — do not gold-plate.
27
+
28
+ **Technique selection** (use when the shape matches):
29
+
30
+ | Technique | Use when | Example |
31
+ |---|---|---|
32
+ | Equivalence partitioning | Inputs fall into classes that behave the same | age ranges, status enums |
33
+ | Boundary value analysis | Classes have edges | min/max/just-below/just-above |
34
+ | Decision table | Multiple conditions combine into outcomes | discount rules, permission matrix |
35
+ | State transition | Behavior depends on state | order lifecycle, connection states |
36
+ | All-pairs | Many parameters, few interactions | config combos, browser × OS |
37
+ | Scenario/use-case | User journeys | end-to-end flows |
38
+ | Error guessing | Known weak spots | empty input, unicode, huge payloads, concurrency |
39
+ | Branch/decision coverage | High-risk logic | the gate conditions themselves |
40
+ | MC/DC | Safety-critical conditions | anything that can cause data loss |
41
+
42
+ **DoD line (large work):** *"the test list is derived from the risk profile —
43
+ High×High risks have the strongest tests; the derivation is stated in the
44
+ plan."*
45
+
46
+ ## 2. Determinism rules (flaky tests are defects)
47
+
48
+ A flaky test is a **defect in the test**, not an inconvenience. Fix it
49
+ immediately; never "re-run into green" (06b Rule 11).
50
+
51
+ 1. **Inject the clock** — no `Date.now()`/`new Date()` in logic under test;
52
+ pass time in (or a clock interface).
53
+ 2. **Seed randomness** — RNGs take an explicit seed; no `Math.random()` in
54
+ tested logic.
55
+ 3. **Own the data** — tests create their own fixtures/DB state; never depend
56
+ on pre-existing or shared data.
57
+ 4. **Poll, don't sleep** — wait for a condition with a bounded poll (with
58
+ timeout), never a fixed `sleep` that races.
59
+ 5. **No order dependence** — each test passes alone and in any order; no
60
+ shared mutable global state between tests.
61
+ 6. **No real network in unit tests** — external calls are faked at the seam
62
+ (06c Rule 4); integration tests are explicit and bounded.
63
+ 7. **Bound every wait** — every wait/poll carries a hard timeout (06b Rule
64
+ 15); a test that can hang is a broken test.
65
+ 8. **Time zones / locales** — pin the environment (UTC, fixed locale) where
66
+ behavior depends on it.
67
+ 9. **Parallelism is opt-in** — tests that share resources run serially; only
68
+ independent tests run in parallel.
69
+
70
+ **DoD line:** *"the suite is deterministic: no flaky test is tolerated; a
71
+ flaky test is fixed or deleted, never retried into green."*
72
+
73
+ ## 3. Where this applies in ASF
74
+
75
+ - **Phase 5 (PLAN.md):** the test plan states the risk-derived test list
76
+ (which risks, which techniques, which tests).
77
+ - **Phase 6 (implementation):** write tests with the technique that matches
78
+ the shape; keep them deterministic per the rules above.
79
+ - **Phase 7 (verification):** a flaky test blocks delivery (it is a defect).
80
+
81
+ ## Anti-patterns
82
+
83
+ - ❌ Testing everything "for coverage" instead of what can actually break
84
+ - ❌ `sleep(5000)` instead of polling for the condition
85
+ - ❌ Tests that depend on the order they run in
86
+ - ❌ Real network calls inside unit tests
87
+ - ❌ Re-running a flaky test until it passes
88
+ - ❌ Tests that pass only on the developer's machine (locale/timezone)
@@ -0,0 +1,134 @@
1
+ # 06h — Security: Threat Model, ASVS DoD, Supply Chain, Agent Tools
2
+
3
+ > **Read when:** Phase 4/5 for **large work**; and for any deliverable that
4
+ > handles untrusted input, secrets, or exposes an interface.
5
+ > **Skip when:** small work; deliverables with no untrusted input, no secrets,
6
+ > no network exposure — the one-line DoD below is enough.
7
+ >
8
+ > **Scale note:** security effort is proportional to the attack surface. A CLI
9
+ > that formats local files needs the one-liner; a service that accepts
10
+ > untrusted input needs the full checklist. **Never skip the one-liner.**
11
+
12
+ Sources: OWASP Threat Modeling, STRIDE, ASVS 5.0, OWASP Top 10 2025,
13
+ SLSA, gitleaks/OSV best practices.
14
+
15
+ ---
16
+
17
+ ## 0. One-line DoD (applies to ALL work, both scales)
18
+
19
+ > **"No secrets in the repo; no untrusted input reaches a shell/query/path
20
+ > unvalidated; no unbounded resource use on user-controlled input."**
21
+
22
+ ## 1. Threat model as code (large work, Phase 4/5)
23
+
24
+ A threat model is a **document in the repo**, not a meeting. Keep it to one
25
+ page:
26
+
27
+ 1. **Context view** — draw the system boundary and **trust boundaries**
28
+ (where untrusted input crosses into trusted code: user input, webhooks,
29
+ file uploads, subprocess input, agent tool input).
30
+ 2. **STRIDE matrix** — for each trust boundary, walk STRIDE and record what
31
+ applies (Spoofing, Tampering, Repudiation, Information disclosure, Denial
32
+ of service, Elevation of privilege). Most rows will be N/A — that is fine.
33
+ 3. **Threat registry** — each identified threat: id, description, affected
34
+ boundary, likelihood, impact, mitigation, status (mitigated / accepted /
35
+ open). Open high-risk threats block delivery unless explicitly accepted by
36
+ the user.
37
+ 4. **Gate** — no High-likelihood × High-impact threat is unmitigated and
38
+ unrecorded. Accepted risks are recorded with rationale (Phase 4 resolution
39
+ rules).
40
+
41
+ ## 2. ASVS-based Definition of Done (large work)
42
+
43
+ Use OWASP ASVS 5.0 as the checklist source. **Select the chapters that apply
44
+ to the deliverable** — do not run the whole list:
45
+
46
+ | Deliverable type | Applicable chapters (ASVS 5.0) |
47
+ |---|---|
48
+ | Web/API service | V1 (architecture), V2 (auth), V3 (session), V4 (access control), V5 (input validation), V6 (output encoding), V7 (crypto), V8 (errors), V14 (config), V15 (API) |
49
+ | CLI / local tool | V1, V5, V6, V8, V14 (no session/auth chapters) |
50
+ | Library/package | V1, V5, V6, V8, V14 (the library's own surface) |
51
+ | Agent/tool surface | V1, V4, V5, V6, V8 + the agent chapter below |
52
+
53
+ **Level selection:** L1 (baseline) for internal tools; L2 for anything
54
+ internet-facing or holding user data; L3 only for high-value targets. Default
55
+ is L2 for internet-facing, L1 otherwise. Record the chosen level in the plan.
56
+
57
+ **DoD line (large work):** *"the applicable ASVS chapter checks pass at the
58
+ chosen level; the level and chapter selection are recorded in the plan."*
59
+
60
+ ## 3. Supply chain & secrets (P17)
61
+
62
+ ### Dependency decision (Phase 2 research, one line)
63
+
64
+ For every dependency, record in the plan: license, maintenance status, size,
65
+ security history, and the alternative considered. A dependency with no
66
+ maintainer or a risky license needs a recorded decision.
67
+
68
+ ### Automated gates (large work, via `/asf health`)
69
+
70
+ | Gate | Tool | Default | Fails when |
71
+ |---|---|---|---|
72
+ | Dependency vulnerabilities | `npm audit` / `osv-scanner` | warn | known vuln in a runtime dep (high/critical) |
73
+ | Secrets in repo | `gitleaks` | warn | any secret pattern in tracked files |
74
+
75
+ - **Lockfile is committed** — reproducibility is a security property.
76
+ - **RV loop (release):** re-run the dependency check **before every release**
77
+ (07-release.md step 1) — a vulnerability discovered after the last run must
78
+ not ship.
79
+ - **Never invent credentials** — no hardcoded tokens, no `.env` in the repo,
80
+ no secrets in commit messages or CHANGELOGs.
81
+
82
+ ## 4. Agent/tool-surface chapter (P18 — only when applicable)
83
+
84
+ Applies **only** when the deliverable has agent-like tool surfaces (an LLM
85
+ agent with tools, a plugin system, an extension that exposes capabilities to
86
+ a model). It is a chapter inside this file — not a separate reference — and
87
+ it is **strictly proportional**: a small agent surface gets the one-liner, a
88
+ full agent platform gets the full list.
89
+
90
+ **One-liner (small surface):** *"agent tools are least-privilege: no tool can
91
+ do more than the user asked for; tool output is validated before use."*
92
+
93
+ **Full list (large surface):**
94
+
95
+ 1. **Least privilege** — each tool gets the minimum scope; no tool with
96
+ blanket shell/DB/network access unless the user explicitly granted it.
97
+ 2. **Human-in-the-loop** — destructive/irreversible actions (publish, delete,
98
+ pay, deploy) require explicit user confirmation; the agent never
99
+ self-approves.
100
+ 3. **Output validation** — tool results are validated before being used as
101
+ input to another tool or rendered to the user (a tool that returns
102
+ attacker-controlled strings must not inject into a shell or HTML).
103
+ 4. **Prompt-injection surface** — treat tool results and fetched content as
104
+ **untrusted input** (they can carry instructions); the agent must not
105
+ follow instructions found in data.
106
+ 5. **Abuse-case test** — one test that proves the agent cannot escalate
107
+ beyond its granted scope (e.g. asks for a tool it does not have, or
108
+ injects instructions via tool output).
109
+ 6. **Auditability** — tool calls are logged with inputs (redacted) so misuse
110
+ is traceable.
111
+
112
+ **DoD line (agent surfaces):** *"every agent tool is least-privilege, and
113
+ there is an abuse-case test proving the boundary holds."*
114
+
115
+ ## Where this applies in ASF
116
+
117
+ - **Phase 2 (research):** dependency decision line (license, maintenance,
118
+ security history, alternative).
119
+ - **Phase 4 (adversarial):** threat model (trust boundaries, STRIDE,
120
+ registry) for large work; the security lens questions already in
121
+ 04-adversarial.md.
122
+ - **Phase 5 (PLAN.md):** threat model + ASVS level/chapter selection recorded.
123
+ - **Phase 6 (implementation):** secrets hygiene; least-privilege agent tools.
124
+ - **Phase 7 (verification):** ASVS DoD + gates + abuse-case test.
125
+
126
+ ## Anti-patterns
127
+
128
+ - ❌ Secrets in the repo (`.env`, tokens, keys in code or commits)
129
+ - ❌ Unvalidated input reaching a shell/query/path
130
+ - ❌ "It's internal, security doesn't apply"
131
+ - ❌ An agent tool with blanket shell/DB access
132
+ - ❌ Following instructions found in fetched content or tool output
133
+ - ❌ Shipping a release without re-running the dependency check
134
+ - ❌ A threat model that exists only in someone's head
@@ -1,6 +1,34 @@
1
1
  # Phase 7 — Release Workflow
2
2
 
3
- Publishing is **always the user's decision**. The factory prepares everything; the user gives the final go.
3
+ ## Release policy (user-configurable)
4
+
5
+ **Default stance:** publishing is the user's decision — the factory prepares
6
+ everything and gets explicit approval before publishing. **A project may
7
+ define its own release policy** in `.asf-release.json` at the project root
8
+ (optional). The user definition takes precedence; the default is used only
9
+ in the absence of a user definition.
10
+
11
+ ```json
12
+ {
13
+ "when": "when the full suite is green and the bump is patch/minor → publish automatically; when the bump is major → send for final review; when the suite is not green → fix first, never publish",
14
+ "how": "To publish: connect with npm using the credentials in ~/.npmrc (2FA token; verify with `npm whoami`), sync git (`git pull --rebase`), run `npm run release <level>`, then verify the published version from the registry in a clean install..."
15
+ }
16
+ ```
17
+
18
+ - **`when`** — the user's **conditional rules**, in their own words: *"when X
19
+ is true, publish automatically; when Y is true, do this…"*. The ASF
20
+ evaluates them against the current state (suite green, bump scope, breaking
21
+ change, credentials available) and acts: publish automatically, send for
22
+ final review, or do something else (e.g. fix first, skip).
23
+ - **`how`** — the user's **instructions** for driving the release, with all
24
+ technical details: credentials, git sync, publish steps, verification.
25
+ It is **not a deterministic command list** — the ASF drives the release
26
+ with judgment: it checks prerequisites, runs the verification gates,
27
+ handles failures, and applies the ASF's own discipline (06b rules) while
28
+ executing. A user `how` may add project-specific steps; it may not remove
29
+ the mandatory verification below.
30
+ - **No file → default:** `when` = "always send for final review before
31
+ publishing"; `how` = the standard workflow below.
4
32
 
5
33
  ## When release applies
6
34
 
@@ -16,6 +44,10 @@ Publishing is **always the user's decision**. The factory prepares everything; t
16
44
  - test suite green
17
45
  - all MUST specs `met` in pi-vigilant
18
46
  - every long-running verification command ran under an explicit timeout with a stated expected duration (06b Rule 15) — no unbounded waits
47
+ - **re-run the dependency/security check (P17 RV loop):** `npm audit`
48
+ (or osv-scanner) + secrets scan — a vulnerability discovered after the
49
+ last run must not ship. The check is re-run **every release**, not
50
+ just when dependencies change.
19
51
  2. **Version bump** — semantic versioning:
20
52
  - `patch` (0.1.0 → 0.1.1): bug fixes
21
53
  - `minor` (0.1.0 → 0.2.0): new features, backward compatible
@@ -23,9 +55,14 @@ Publishing is **always the user's decision**. The factory prepares everything; t
23
55
  3. **CHANGELOG** — move `[Unreleased]` entries into the new version section; describe each change specifically.
24
56
  4. **Tag** — `git tag vX.Y.Z`
25
57
  5. **Push** — `git push --follow-tags`
26
- 6. **Publish** — only with explicit user approval:
27
- - npm: `npm publish` (ensure 2FA token, `npm login` state)
28
- - GitHub release: via web or `gh release create` if installed
58
+ 6. **Publish** — per the release policy:
59
+ - default: **only with explicit user approval** (npm: `npm publish`, ensure 2FA token + `npm login` state; GitHub release: via web or `gh release create`)
60
+ - a `.asf-release.json` `when` rule may authorize automatic publishing for the stated conditions
61
+ 7. **Test on publish / in production — ALWAYS, if applicable** (mandatory, not optional):
62
+ - **Package:** install it **clean-room from the registry** (fresh dir, caches cleared, explicit version — 06b Rule 5), confirm the installed version is the one just published, and confirm the **observable end state** works (the tool/skill/extension loads and runs; any web surface exercised through `pi-aia-browser`).
63
+ - **Deployed app:** smoke-test in **production** (health endpoint, key user journey) — a deploy that breaks prod is not a successful release.
64
+ - **Not applicable** (internal-only experiment, no runtime surface): say so explicitly in the report (06b Rule 11).
65
+ - If the published artifact is broken: **fix forward** (new patch), never pretend the release succeeded.
29
66
 
30
67
  ## CI/CD offer
31
68
 
@@ -37,7 +74,8 @@ After a successful release (or for a new project), **offer** to set up a CI/CD p
37
74
 
38
75
  ## Hygiene rules
39
76
 
40
- - No auto-publish — always explicit user approval
77
+ - No auto-publish beyond the configured policy — default is explicit user approval
41
78
  - Describe changes specifically in CHANGELOG (no placeholders)
42
79
  - The version in package.json, the git tag, and the npm release must match
43
80
  - If publishing fails partway, fix forward; never publish a second version with the same number
81
+ - **Test on publish / in production, if applicable — always** (step 7)