@multileaf/ai-workflow 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/README.md +12 -1
- package/dist/blocking-reasons.d.ts +67 -0
- package/dist/blocking-reasons.js +199 -0
- package/dist/command-evidence.d.ts +32 -0
- package/dist/command-evidence.js +61 -0
- package/dist/human-approval.d.ts +19 -2
- package/dist/human-approval.js +82 -9
- package/dist/recommendations.js +16 -1
- package/dist/risk-assess.d.ts +50 -0
- package/dist/risk-assess.js +273 -0
- package/dist/workflow.js +280 -30
- package/docs/README.md +2 -0
- package/docs/architecture.md +50 -0
- package/docs/backlog.md +13 -0
- package/docs/continuation-inventory.md +24 -0
- package/docs/decision-model-evaluation.md +4 -0
- package/docs/opportunities/OPP-001-proportional-ceremony-light-lane.md +62 -0
- package/docs/opportunities/OPP-002-risk-assess-command.md +70 -0
- package/docs/opportunities/OPP-003-adversarial-review-package.md +63 -0
- package/docs/opportunities/OPP-004-exact-continuation-commands.md +59 -0
- package/docs/opportunities/OPP-005-friction-benchmark.md +62 -0
- package/docs/opportunities/OPP-006-emission-usability-test-policy.md +57 -0
- package/docs/opportunities/OPP-007-decision-findings-triage-scope-watch.md +54 -0
- package/docs/opportunities/OPP-008-embedding-covering-tests-judge-context.md +52 -0
- package/docs/opportunities/README.md +33 -0
- package/docs/product-requirements.md +3 -0
- package/docs/product-vision.md +1 -0
- package/docs/quality.md +10 -0
- package/docs/release-process.md +9 -1
- package/docs/semantic-mapping.md +4 -0
- package/package.json +1 -1
- package/resources/package.yaml +3 -0
- package/resources/skills/code-review.md +2 -0
- package/resources/skills/review-judge-a.md +24 -0
- package/resources/skills/review-judge-b.md +24 -0
- package/resources/skills/review-refuter.md +28 -0
- package/resources/skills/ticket.md +2 -0
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# OPP-001: Proportional ceremony — light lane for small, understood work
|
|
2
|
+
|
|
3
|
+
- Status: Implemented (2026-10-30, plan Wave 4)
|
|
4
|
+
- Priority: P2
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
Every AI Workflow task, regardless of size or risk, traverses the same staged pipeline (brainstorm → specification → ADR → plan → implementation → verification → review → traceability), each stage producing durable artifacts under `.aiw/generated/`. For small, well-understood changes this generates artifacts nobody reads, and users bypass the workflow manually — which then produces no evidence at all. Gentle-AI retired SDD for exactly this reason; the market signal is that uniform ceremony causes abandonment in daily use.
|
|
11
|
+
|
|
12
|
+
## Source concept (ODD)
|
|
13
|
+
|
|
14
|
+
ODD classifies task size with the **resume test**: work is _small_ when it is understood, risk is contained, and the work could be resumed from the original request plus `git diff`. Small work creates **no durable task artifacts**; substantial work creates exactly one feature document before the first source write. ODD also holds that findings never silently expand authorization, and scope growth is an explicit event.
|
|
15
|
+
|
|
16
|
+
References:
|
|
17
|
+
|
|
18
|
+
- Gentle-AI `docs/usage.md`, section "Organic Driven Development (ODD)" — protocol steps 4 (Classify) and 5 (Track before the first write)
|
|
19
|
+
- Gentle-AI `docs/intended-usage.md` — "Small, understood changes need no persistent task artifact"
|
|
20
|
+
|
|
21
|
+
## Proposal
|
|
22
|
+
|
|
23
|
+
Add a **light lane** alongside the existing full SDD lane:
|
|
24
|
+
|
|
25
|
+
1. New flag `aiw task start --lane=light` (full lane remains the default).
|
|
26
|
+
2. Entry criteria, evaluated and recorded at start — all must hold:
|
|
27
|
+
- the change is understood well enough to resume from the request plus `git diff` (the ODD resume test, stated verbatim in the command output so the human makes the call);
|
|
28
|
+
- expected blast radius is contained (no new requirement IDs, no architecture decision needed);
|
|
29
|
+
- TDD applies or an exception is recorded with reason.
|
|
30
|
+
3. Light-lane behavior:
|
|
31
|
+
- No durable spec/plan/report artifacts are scaffolded.
|
|
32
|
+
- Evidence is still required: task validation commands run and completion evidence is recorded in `.aiw/checkpoints/` (compact form, linked to the ticket if present).
|
|
33
|
+
- The lane decision and its entry criteria are persisted, so audits can see why the light lane was used.
|
|
34
|
+
4. Escalation: if scope grows beyond the entry criteria (new requirement, ADR needed, risk found), the task must escalate to the full lane; the escalation is an explicit recorded event, never a silent expansion. Escalation preserves already-recorded evidence.
|
|
35
|
+
|
|
36
|
+
## Affected areas
|
|
37
|
+
|
|
38
|
+
- `src/workflow.ts`, `src/types.ts` (lane state, escalation event)
|
|
39
|
+
- CLI: task/workflow commands (`src/cli.ts`)
|
|
40
|
+
- Stage skills and templates rendered through adapters (resources that describe the SDD sequence)
|
|
41
|
+
- `docs/README.md` (main README workflow section), `docs/architecture.md` (approval contract description)
|
|
42
|
+
- Checkpoint/evidence recording (`src/evidence.ts`, `src/human-approval.ts` interactions)
|
|
43
|
+
|
|
44
|
+
## Acceptance criteria
|
|
45
|
+
|
|
46
|
+
- Given a task started with `--lane=light`, when no scope growth occurs, then completion requires no `specification.md`/`implementation-plan.md` artifacts and records validation evidence plus the lane rationale in checkpoints.
|
|
47
|
+
- Given a light-lane task, when scope growth is detected (or declared), then escalation to the full lane is required before further stage commands run, and prior evidence is preserved and linked.
|
|
48
|
+
- Given the full lane (default), when a task starts, then behavior is unchanged from today.
|
|
49
|
+
- Given a light-lane completion, when `.aiw/approvals.yml` is inspected, then the ledger shows the light-lane rationale, not a missing approval.
|
|
50
|
+
- Non-interactive runs without an explicit lane default to the full lane.
|
|
51
|
+
|
|
52
|
+
## Non-goals
|
|
53
|
+
|
|
54
|
+
- No global "workflow off" switch (see ADR-001 Decision B).
|
|
55
|
+
- No change to full-lane stage order, gates, or approval semantics.
|
|
56
|
+
- No automatic lane detection that overrides the human's classification; automation may only suggest.
|
|
57
|
+
|
|
58
|
+
## Open questions
|
|
59
|
+
|
|
60
|
+
- Should the light lane allow skipping the brainstorm/spec stages only, or also merge implementation + verification evidence into a single checkpoint artifact?
|
|
61
|
+
- Should escalation be self-declared by the agent, human-confirmed, or both (recommend both: agent proposes, human confirms)?
|
|
62
|
+
- Does the lane need representation in `traceability` output (e.g., a light-lane marker on the ticket)?
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# OPP-002: `aiw risk assess` — content-based risk classification with an exact next command
|
|
2
|
+
|
|
3
|
+
- Status: Implemented (2026-10-30, plan Wave 2); optional decision-model advisory refinement remains a follow-up
|
|
4
|
+
- Priority: P2
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
AI Workflow applies the same gate and review depth to every change. A two-line edit to authentication handling and a thousand-line documentation addition produce identical ceremony. There is no command that answers, before deciding how much process a change needs: _what is the risk of this change, why, and what should I run next?_
|
|
11
|
+
|
|
12
|
+
## Source concept (RDD)
|
|
13
|
+
|
|
14
|
+
RDD's `gentle-ai review assess` is a **read-only** risk classification that:
|
|
15
|
+
|
|
16
|
+
- builds the same candidate a review would, without creating any authority or mutating state;
|
|
17
|
+
- classifies risk by **what changed**, never by size — a README edit is `passive` (zero reviewers), two lines of login code is `high` (four reviewers);
|
|
18
|
+
- reports a typed tier (`passive` / `medium` / `high`), machine-readable reasons with paths, and a **`next_transition.command`**: the exact, literally runnable next command, so callers never reconstruct invocations from prose;
|
|
19
|
+
- supports agent escalation: an agent may raise (never lower) risk citing one of six high-risk items — 1 data/irreversible effects, 2 security, 3 contracts others consume, 4 concurrency, 5 delivery/environment, 6 no test would catch a regression.
|
|
20
|
+
|
|
21
|
+
References:
|
|
22
|
+
|
|
23
|
+
- Gentle-AI `docs/review-integration.md`, section "Read-only risk assessment (`gentle-ai review assess`)" — includes the `gentle-ai.review-assessment/v1` JSON envelope example
|
|
24
|
+
- Gentle-AI `docs/architecture/organic-rdd.md` — risk tier → lens selection (low: structural readback, 0 lenses; medium: 1 lens; high: canonical 4R)
|
|
25
|
+
|
|
26
|
+
## Proposal
|
|
27
|
+
|
|
28
|
+
Add `aiw risk assess`:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
aiw risk assess [--ticket=PROJ-142] [--base-ref <ref>] [--json]
|
|
32
|
+
aiw risk assess --escalate-item <1-6> --escalate-reason "<one line>"
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
1. **Inputs**: working tree changes by default; or `--base-ref` for a committed base-to-HEAD comparison (mirroring RDD's `--base-ref --committed-only`).
|
|
36
|
+
2. **Classification layers** (deterministic first, model optional):
|
|
37
|
+
- deterministic path/content rules for the six high-risk categories (auth/security surfaces, exported contracts, concurrency primitives, migrations/irreversible data ops, delivery/environment scripts, code with no covering test);
|
|
38
|
+
- the existing `decision` module (advisory) may refine reasons when enabled — output stays advisory and never changes evidence states or bypasses approval, consistent with the decision-model contract.
|
|
39
|
+
3. **Output tiers**: `passive` (documentation-only or no risk category hit), `medium`, `high`, with typed reasons `[{"code": "...", "path": "..."}]` and, with `--json`, a schema-versioned envelope (e.g., `aiw.risk-assessment/v1`).
|
|
40
|
+
4. **Exact continuation**: when the tier warrants more process, the output includes `next_command`: the literal runnable `aiw` command (e.g., start the full lane, or open a gate). Follows OPP-004's rule: a next command is printed only when running it resolves the decision at hand.
|
|
41
|
+
5. **Escalation**: `--escalate-item`/`--escalate-reason` raise `passive`/`medium` to `high`, never lower a tier; both flags together, reason ≤ 500 bytes, one line.
|
|
42
|
+
6. **Read-only guarantee**: no writes to `.aiw/`, no approval or gate state changes, works offline with no model configured.
|
|
43
|
+
|
|
44
|
+
## Affected areas
|
|
45
|
+
|
|
46
|
+
- New command module + tests (pattern: `src/decision.ts`, `src/evidence.ts`)
|
|
47
|
+
- `src/quality-gates.ts` / `src/human-gates.test.ts` conventions for envelope schemas
|
|
48
|
+
- `src/scanner.ts` reuse for change/path detection (ignore-aware)
|
|
49
|
+
- CLI wiring (`src/cli.ts`), docs (`docs/README.md`, `docs/decision-model-evaluation.md` cross-link)
|
|
50
|
+
|
|
51
|
+
## Acceptance criteria
|
|
52
|
+
|
|
53
|
+
- Given a documentation-only diff, when `aiw risk assess --json` runs, then tier is `passive` and no reviewer escalation is suggested.
|
|
54
|
+
- Given a diff touching an authentication module, when assessed, then tier is `high` with a reason naming the path and the security category.
|
|
55
|
+
- Given any invocation, when it completes, then no file under `.aiw/` changed (read-only proof in tests).
|
|
56
|
+
- Given `review_due`-style output, when a next step is warranted, then `next_command` executes successfully as printed (tested by executing it).
|
|
57
|
+
- Given `--escalate-item 2 --escalate-reason "..."`, when the base tier is `medium`, then output tier is `high`; escalation never lowers a tier.
|
|
58
|
+
- Given no decision model configured, when assessing, then classification is fully deterministic and offline.
|
|
59
|
+
|
|
60
|
+
## Non-goals
|
|
61
|
+
|
|
62
|
+
- No automatic gate skipping or approval bypass based on tier (tiers inform humans and agents; they do not decide).
|
|
63
|
+
- No review lifecycle/authority machinery (RDD's lineages and burns are out of scope — see ADR-001 Decision B).
|
|
64
|
+
- No model-only classification: deterministic rules are the floor; the model layer is optional refinement.
|
|
65
|
+
|
|
66
|
+
## Open questions
|
|
67
|
+
|
|
68
|
+
- Should `--ticket` bind the assessment into the checkpoint ledger for audit (write-only log, no approval state)?
|
|
69
|
+
- Which concrete path rules seed the six categories for this repository's own stack (self-hosting dogfood)?
|
|
70
|
+
- Should the light lane (OPP-001) consume `risk assess` output as an entry-criteria signal (advisory only)?
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# OPP-003: Adversarial review package — blind judges and proof-carrying findings
|
|
2
|
+
|
|
3
|
+
- Status: Implemented (2026-10-30, plan Wave 6): skills shipped in the self-hosting package with content-rule tests; runtime permission wiring rides the existing package permission model
|
|
4
|
+
- Priority: P3
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
AI Workflow's review stage uses the `code-review` skill when installed, but findings are accepted as written: nothing in the workflow attempts to _refute_ severe claims, no second independent reviewer cross-checks the first, and findings do not have to carry reproducing evidence. The result is both false positives (blocking work on an unreal defect) and unearned authority (a plausible-sounding severe finding with no proof).
|
|
11
|
+
|
|
12
|
+
## Source concept (RDD / Judgment Day)
|
|
13
|
+
|
|
14
|
+
Two Gentle-AI mechanisms address exactly this:
|
|
15
|
+
|
|
16
|
+
1. **Refuter**: every severe (BLOCKER/CRITICAL) finding with candidate causality reaches an adversarial pass that may confirm or **drop** the claim. On runtimes that support isolation (e.g., Codex), the refuter may run one reproducing command per claim inside a disposable scratch directory (fresh `mktemp -d`, no network, sandboxed, destroyed afterwards); the finding must cite the command and its observed output. A refuted finding becomes an `advisory`; corroborated and inconclusive findings go forward. Non-severe findings never reach the refuter.
|
|
17
|
+
2. **Judgment Day blind judges**: review by two independent judge agents (`jd-judge-a`, `jd-judge-b`) plus a fix agent (`jd-fix-agent`), so no single reviewer's output is authoritative.
|
|
18
|
+
|
|
19
|
+
References:
|
|
20
|
+
|
|
21
|
+
- Gentle-AI `docs/review-integration.md`, section "Refuter probe" (isolation matrix per runtime, proof requirements, `probe unavailable on <runtime>` honesty marker)
|
|
22
|
+
- Gentle-AI `docs/kiro.md`, section "ODD Execution Model" — Judgment Day adversarial review agents
|
|
23
|
+
|
|
24
|
+
## Proposal
|
|
25
|
+
|
|
26
|
+
Ship an **optional versioned resource package** `adversarial-review` through the existing package model and registry:
|
|
27
|
+
|
|
28
|
+
1. Package contents:
|
|
29
|
+
- `review-judge-a` and `review-judge-b` agent/skill definitions: two independent reviewers of the same diff, each reporting findings with severity and required evidence fields; neither sees the other's output (blind).
|
|
30
|
+
- `review-refuter` skill: takes severe findings and attempts to refute each — by reading cited code, and, where the host runtime supports safe isolation, by running the cited reproducing command in a disposable directory; marks each finding `corroborated` / `refuted` / `inconclusive` with proof references.
|
|
31
|
+
- Rules rendered into the workflow's review stage: a severe finding without a reproducing command or file:line evidence degrades to `advisory`; only `corroborated` severe findings can block.
|
|
32
|
+
2. Integration points:
|
|
33
|
+
- Declared in the package manifest with permissions (needs command execution for the refuter — flagged as a sensitive capability through the existing package permission/policy model).
|
|
34
|
+
- The review stage template/skill references the package when installed; the existing rule applies that a human must still accept the review outcome.
|
|
35
|
+
3. Honesty markers: when the host runtime cannot isolate command execution, the refuter marks results accordingly (the RDD pattern: state that no command ran, rather than implying verification).
|
|
36
|
+
|
|
37
|
+
## Affected areas
|
|
38
|
+
|
|
39
|
+
- `resources/` (new package), `resources/package.yaml` conventions
|
|
40
|
+
- `src/package-audit.ts` / `src/policy-gate.ts` (refuter execution is a sensitive capability needing explicit permission)
|
|
41
|
+
- Review-stage skill templates and `docs/README.md` review section (currently: "Code review is performed with the `code-review` skill when installed")
|
|
42
|
+
- Registry listing (`src/registry.ts`)
|
|
43
|
+
|
|
44
|
+
## Acceptance criteria
|
|
45
|
+
|
|
46
|
+
- Given the package installed, when the review stage runs, then two blind judges receive the same diff independently and produce separately attributable findings.
|
|
47
|
+
- Given a severe finding without a reproducing command or file:line evidence, when the refuter processes it, then it is downgraded to `advisory` with the reason recorded.
|
|
48
|
+
- Given a severe finding whose cited command fails to reproduce the defect, when the refuter runs it (isolated runtime), then the finding is marked `refuted` and does not block.
|
|
49
|
+
- Given a runtime without isolation support, when the refuter runs, then results carry an explicit "no command executed" marker rather than silent omission.
|
|
50
|
+
- Given the package's execution permission, when it is not granted, then the refuter degrades to evidence-reading mode and the outcome states so.
|
|
51
|
+
- Human acceptance of the review outcome remains required (package never auto-accepts).
|
|
52
|
+
|
|
53
|
+
## Non-goals
|
|
54
|
+
|
|
55
|
+
- No new CLI gate command for review (the existing contract stays: code review via skill, human accepts outcome).
|
|
56
|
+
- No frozen synthetic trees or review authority machinery (ADR-001 Decision B); the package operates on the working diff/cited files.
|
|
57
|
+
- No requirement that both judges agree: disagreement is reported, not averaged away.
|
|
58
|
+
|
|
59
|
+
## Open questions
|
|
60
|
+
|
|
61
|
+
- Should judge outputs be recorded under `.aiw/generated/artifacts/` as versioned review evidence for the traceability graph?
|
|
62
|
+
- Which target adapters can safely host isolated command execution for the refuter (Codex sandbox vs Claude Code vs local-only)?
|
|
63
|
+
- Package name and whether `code-review` (existing) becomes judge-a or stays separate.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# OPP-004: Exact continuation commands — every block names a runnable way out
|
|
2
|
+
|
|
3
|
+
- Status: Partially implemented (2026-10-30, plan Wave 1): workflow, gate, and approval paths emit typed reason codes, continuations, and `--json` envelopes; sync/apply conflicts and doctor diagnostics remain prose-only follow-up
|
|
4
|
+
- Priority: P1 (small scope, high leverage)
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
When an `aiw` command blocks (gate incomplete, stage prerequisite unresolved, conflict, stale artifact), the output explains the problem but does not always provide the exact command that resolves it. Users and agents then reconstruct invocations from prose — which is where drift, dead ends, and wrong flags are born. There is no inventory guaranteeing that every blocking path has a tested, runnable continuation.
|
|
11
|
+
|
|
12
|
+
## Source concept (RDD)
|
|
13
|
+
|
|
14
|
+
RDD's CLI is governed by one rule:
|
|
15
|
+
|
|
16
|
+
> **"A message may name a command only if running that command resolves the block."**
|
|
17
|
+
|
|
18
|
+
Mechanisms behind it:
|
|
19
|
+
|
|
20
|
+
- Every blocking `stop` carries a typed **reason code** and, when a continuation exists, the **exact command** (provider-issued `next_transition.command`), with fixed argument order, runnable verbatim.
|
|
21
|
+
- A complete, published **continuation inventory**: a table mapping every reason code to its continuation or an explicit `Terminal` marker when no in-lineage continuation exists — terminal states say so instead of implying an escape route that isn't there.
|
|
22
|
+
- The rule was learned the hard way: an escalated review told operators to run `review.status`, which _looked_ like a command but was an internal routing name. And a fixed message named a continuation that, when finally executed, turned out not to resolve anything. "Naming a command that runs and does not help is more expensive than naming nothing, because now the person trusts it."
|
|
23
|
+
|
|
24
|
+
References:
|
|
25
|
+
|
|
26
|
+
- Gentle-AI `docs/review-integration.md` — "Continue after a stop reason code" table (complete continuation inventory)
|
|
27
|
+
- Gentle-AI `docs/architecture/the-organic-rdd-story.md` — "the rule that ran over everything else: if you tell someone what to do, make sure it works"
|
|
28
|
+
|
|
29
|
+
## Proposal
|
|
30
|
+
|
|
31
|
+
1. **Typed reason codes**: every blocking exit in the CLI (gates, stage commands, conflicts, doctor diagnostics) emits a stable reason code (e.g., `gate_incomplete`, `stage_prerequisite_unresolved`, `artifact_conflict`, `approval_stale`).
|
|
32
|
+
2. **Exact continuation**: when a continuation exists, the same output includes the literal runnable `aiw` command with all flags resolved (ticket, stage, path). When no continuation exists, the output says so explicitly (`terminal`) and names the human decision required.
|
|
33
|
+
3. **Continuation inventory doc**: a generated or maintained table mapping every reason code → continuation / terminal, published under `docs/` (contract for users, agents, and OPP-005's benchmark).
|
|
34
|
+
4. **Machine-readable form**: `--json` outputs carry `reason_code` and `next_command` fields so agents can chain without parsing prose (aligns with OPP-002's `next_command`).
|
|
35
|
+
|
|
36
|
+
## Affected areas
|
|
37
|
+
|
|
38
|
+
- `src/cli.ts` (error/blocking output paths), `src/quality-gates.ts`, `src/human-gates.test.ts` conventions
|
|
39
|
+
- `src/human-approval.ts` (stale approval blocks), migration/conflict reporting paths (`src/migration.ts`, `src/files.ts`)
|
|
40
|
+
- New reason-code registry module with the inventory table
|
|
41
|
+
- OPP-006 test policy is the enforcement mechanism for this opportunity
|
|
42
|
+
|
|
43
|
+
## Acceptance criteria
|
|
44
|
+
|
|
45
|
+
- Given any blocking exit, when its output is parsed, then it contains a stable reason code and either an exact `next_command` or an explicit terminal statement.
|
|
46
|
+
- Given an emitted `next_command`, when executed in the state that produced the block, then it resolves or advances that block (verified by test, per OPP-006).
|
|
47
|
+
- Given the continuation inventory, when a new reason code is added without a mapped continuation, then CI fails (inventory completeness check).
|
|
48
|
+
- Given `--json`, when a block occurs, then `reason_code` and `next_command` are machine-readable fields.
|
|
49
|
+
- No output names a command that exists but does not resolve the block it was printed for (regression-tested via OPP-006).
|
|
50
|
+
|
|
51
|
+
## Non-goals
|
|
52
|
+
|
|
53
|
+
- No interactive wizard behavior: the command prints the way out; it does not run it automatically.
|
|
54
|
+
- No changes to what blocks — only to what each block tells you.
|
|
55
|
+
|
|
56
|
+
## Open questions
|
|
57
|
+
|
|
58
|
+
- Should the reason-code registry live as code (single source, docs generated from it) or as docs with a CI check?
|
|
59
|
+
- Do adapter-rendered skills need the same guarantee in their prompts (the generated skills also tell agents what to run next)?
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# OPP-005: Friction benchmark — measure how users get stuck
|
|
2
|
+
|
|
3
|
+
- Status: Implemented (2026-10-30, plan Wave 5). Baseline: in-band=5, out-of-band=5, dead-end=0; out-of-band blocks (install-first, no-workflow, git-required, unknown-stage usage, approve-not-pending) are the typing backlog this benchmark exists to shrink
|
|
4
|
+
- Priority: P3
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
AI Workflow measures token efficiency (`src/context-benchmark.ts`, `docs/token-efficiency.md`) but not **friction**: how often the workflow stops a user, and — more importantly — _how_ it stops them. Without that measurement, ceremony changes (like OPP-001) can't be evaluated, and blocking regressions (a gate that strands the user with no way out) are only discovered by users.
|
|
11
|
+
|
|
12
|
+
## Source concept (RDD)
|
|
13
|
+
|
|
14
|
+
Gentle-AI ships a reproducible benchmark that classifies every block into:
|
|
15
|
+
|
|
16
|
+
- **In band** — it stops you and tells you what to run;
|
|
17
|
+
- **Out of band** — it stops you and tells you nothing;
|
|
18
|
+
- **Dead end** — it stops you and there is nothing you can do.
|
|
19
|
+
|
|
20
|
+
Key design points worth copying:
|
|
21
|
+
|
|
22
|
+
- It measures friction, not speed ("speed depends on the provider and the day; friction is yours").
|
|
23
|
+
- It drives the **real packaged binary**, not a mock ("a number I publish and you cannot reproduce is a claim about my honesty; a number you can run yourself is evidence").
|
|
24
|
+
- The journey corpus includes adversarial states, not just the happy guide paths: bare repositories, half-finished migrations, empty files, mode flips mid-operation.
|
|
25
|
+
- Its own history shows why: first measurement was "six blocks, every one out of band"; after fixes, blocks stayed at the same _count_ but all moved in band, with zero dead ends — the tool didn't stop users less, it started telling them how to get out.
|
|
26
|
+
- The instrument must itself be suspect: one round measured a stale binary after a silent build failure; another counted the kill switch _working_ as tool-caused friction. Rule: point the instrument at itself first.
|
|
27
|
+
|
|
28
|
+
References:
|
|
29
|
+
|
|
30
|
+
- Gentle-AI `docs/architecture/the-organic-rdd-story.md`, section "The benchmark" and "The loop"
|
|
31
|
+
|
|
32
|
+
## Proposal
|
|
33
|
+
|
|
34
|
+
1. **Journey corpus**: scripted end-to-end journeys against the packaged CLI (`node dist/cli.js`), starting with the happy paths (install → scan → recommend → workflow start → task → gates → complete) and, critically, adversarial states: missing profile, stale approval, conflicting root context file, interrupted migration, empty repository, pre-existing `.aiw` state, gate failure mid-workflow.
|
|
35
|
+
2. **Block classification**: every non-success exit is classified in-band / out-of-band / dead-end by the harness, using OPP-004's reason codes and `next_command` as the in-band contract (a block with a `next_command` that resolves it, when executed, is in-band; a block with no continuation information is out-of-band; a block whose named continuation does not resolve it is a dead end).
|
|
36
|
+
3. **Reporting**: per-journey and aggregate counts; stored as a benchmark artifact so runs are comparable over time. CI budget: **zero dead ends allowed**; out-of-band count may not increase without an explicit waiver.
|
|
37
|
+
4. **Reproducibility**: the harness runs the same way locally and in CI against the packaged output, no internal APIs.
|
|
38
|
+
|
|
39
|
+
## Affected areas
|
|
40
|
+
|
|
41
|
+
- New `scripts/bench`-style harness (pattern: `src/context-benchmark.ts`, `scripts/*.node-test.mjs`)
|
|
42
|
+
- Depends on OPP-004 reason codes for classification (sequence OPP-004 → OPP-005)
|
|
43
|
+
- CI workflow wiring (`.github/`)
|
|
44
|
+
- `docs/quality.md` (completion criteria reference the friction budget)
|
|
45
|
+
|
|
46
|
+
## Acceptance criteria
|
|
47
|
+
|
|
48
|
+
- Given the journey corpus, when the benchmark runs against the packaged CLI, then every non-success exit is classified and counted.
|
|
49
|
+
- Given a block whose printed continuation resolves it when executed, when classified, then it counts as in-band (the harness actually executes continuations).
|
|
50
|
+
- Given a dead end, when detected, then CI fails with the journey name, reason code, and reproduction steps.
|
|
51
|
+
- Given two runs on unchanged code, when compared, then results are deterministic (no flaky classification).
|
|
52
|
+
- Given the happy-path journeys only, when run today, then the result is recorded as the baseline (expected: some out-of-band blocks; dead ends zero or explicitly waived).
|
|
53
|
+
|
|
54
|
+
## Non-goals
|
|
55
|
+
|
|
56
|
+
- No speed/latency benchmarking (explicitly excluded by the source concept; token efficiency is already covered).
|
|
57
|
+
- No telemetry dependency: the benchmark is local and reproducible; it does not phone home.
|
|
58
|
+
|
|
59
|
+
## Open questions
|
|
60
|
+
|
|
61
|
+
- Corpus growth policy: should each new OPP/blocking feature be required to add its adversarial journeys (ratchet)?
|
|
62
|
+
- Where do results live — `.aiw/` local state, CI artifacts, or a versioned `docs/benchmarks/` record?
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# OPP-006: Emission ≠ usability — test that named commands actually work
|
|
2
|
+
|
|
3
|
+
- Status: Implemented (2026-10-30, plan Wave 3)
|
|
4
|
+
- Priority: P2
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
AI Workflow's test suite is strong on behavior and contracts, but there is no explicit policy category for the failure mode where a user-facing message _names a command_ and that command is wrong, incomplete, or does not resolve the situation. A test can verify a message is emitted with exact text — all green — while the command it names has never been executed by anyone. That is a dead end shipped with green coverage.
|
|
11
|
+
|
|
12
|
+
## Source concept (RDD)
|
|
13
|
+
|
|
14
|
+
The founding anecdote of Gentle-AI's audit story:
|
|
15
|
+
|
|
16
|
+
> There was a message telling you "to get out of this, run this command". There were tests. They verified the message was emitted, with its exact text. All green. **Nobody had ever run the command the message named.** When I ran it, it did not work. I had been sending people into a dead end, with green test coverage, for months.
|
|
17
|
+
|
|
18
|
+
From this, the governing rule: a message may name a command only if running that command resolves the block. Related lessons from the same document that this policy should encode:
|
|
19
|
+
|
|
20
|
+
- "A guard nobody calls is not a guard" — a correct, well-documented check wired to nothing reads like coverage and is more dangerous than missing code.
|
|
21
|
+
- Guide steps written without running them failed three times when testers followed them; the rule that came out: _before naming a continuation, execute it_.
|
|
22
|
+
- A changelog overclaiming a fix ("fixed the tool so it stopped _printing_ the broken form, not the parser so it would accept it") cost a user real time.
|
|
23
|
+
|
|
24
|
+
References:
|
|
25
|
+
|
|
26
|
+
- Gentle-AI `docs/architecture/the-organic-rdd-story.md` — "The audits: the ones that did not" and "The mistakes I made"
|
|
27
|
+
|
|
28
|
+
## Proposal
|
|
29
|
+
|
|
30
|
+
1. **Policy** (added to `docs/quality.md`, "Completion validation"): every user-facing message, gate output, skill instruction, or generated prompt that names an `aiw` command must be covered by a test that **executes that command** in the state the message describes and asserts it resolves or advances that state. Emission assertions alone are insufficient for messages that name commands.
|
|
31
|
+
2. **Extraction helper**: a small test utility that scans emitted messages (and optionally generated skill artifacts) for `aiw ...` invocations and executes them against fixture repositories, so coverage is systematic rather than hand-picked. Feeds naturally from OPP-004's `next_command`/reason-code contract.
|
|
32
|
+
3. **Guidance for generated artifacts**: the same rule applies to skills and rules rendered through adapters — if a generated skill tells the agent to run a command, a test executes that rendered command in a fixture. (The generated `ai-init` skill is the highest-value first target.)
|
|
33
|
+
4. **Changelog/review discipline** (soft rule, docs-only): a fix is described as fixing the _parser_, not the _printing_ — i.e., the claim is bound to the behavior tested, not the symptom observed.
|
|
34
|
+
|
|
35
|
+
## Affected areas
|
|
36
|
+
|
|
37
|
+
- `docs/quality.md` (new policy section)
|
|
38
|
+
- New test helper + fixtures (pattern: `src/fixtures/`, existing e2e tests for install/init/migration)
|
|
39
|
+
- Existing message-emitting modules get incremental coverage as they are touched (no big-bang retrofit required)
|
|
40
|
+
- Enforces OPP-004; the two share the reason-code/`next_command` contract
|
|
41
|
+
|
|
42
|
+
## Acceptance criteria
|
|
43
|
+
|
|
44
|
+
- Given a user-facing output that names an `aiw` command, when the policy applies, then a test executes that command in the described state and asserts progress (resolve or advance).
|
|
45
|
+
- Given the extraction helper, when run over CLI outputs in fixtures, then every discovered `aiw` invocation either has executing coverage or is an allowlisted non-actionable mention.
|
|
46
|
+
- Given the generated `ai-init` skill for each adapter, when rendered in tests, then every `aiw` command it instructs has been executed at least once in the e2e suite.
|
|
47
|
+
- Given `docs/quality.md`, when a reviewer applies completion validation, then emission-only tests are explicitly insufficient for command-naming messages.
|
|
48
|
+
|
|
49
|
+
## Non-goals
|
|
50
|
+
|
|
51
|
+
- No retroactive mandate to fix every existing message in one pass: the policy applies to new and touched code (ratchet).
|
|
52
|
+
- No enforcement inside the product at runtime — this is a development-time test policy, not a CLI feature.
|
|
53
|
+
|
|
54
|
+
## Open questions
|
|
55
|
+
|
|
56
|
+
- Allowlist format for legitimate _informational_ command mentions (e.g., docs examples that are not continuations)?
|
|
57
|
+
- Should the extraction helper live as a vitest utility or a `scripts/` check runnable in CI independent of unit tests?
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# OPP-007: Decision model — findings-evidence triage and lane scope-watch
|
|
2
|
+
|
|
3
|
+
- Status: Proposed
|
|
4
|
+
- Priority: P3
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A (model-integration addendum)
|
|
7
|
+
- Companion: [OPP-008](OPP-008-embedding-covering-tests-judge-context.md)
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
Two implemented flows have deterministic guards with semantic blind spots:
|
|
12
|
+
|
|
13
|
+
1. **Adversarial review (OPP-003)**: a severe finding can be structurally well-formed (has `file:line`, has a command) while the cited evidence does not actually support the claim. The refuter catches this only for findings it processes — and a weak first filter passes noise downstream.
|
|
14
|
+
2. **Light lane (OPP-001)**: lane abuse (substantial work routed as light) is currently caught by deterministic mismatch signals only (`risk assess` tier vs lane). What deterministic rules cannot see: 80 lines spread across three subsystems _is_ substantial even when every path rule stays quiet.
|
|
15
|
+
|
|
16
|
+
Meanwhile the existing `aiw decision triage` keeps serving full-lane artifacts but is not connected to either gap — and its cost profile is already proportional: light-lane tasks produce no artifacts, so there is nothing to triage and the model spend is zero by construction.
|
|
17
|
+
|
|
18
|
+
## Source concept
|
|
19
|
+
|
|
20
|
+
- Spec NFR-2/NFR-3 (odd-rdd-proportionality spec): deterministic floor first; models advisory-only, never change evidence states, gates, or approvals.
|
|
21
|
+
- `buildEvidenceTriageRequest` (`src/decision.ts`) already asks "does this artifact include concrete verification evidence for every stated requirement" — the same pattern aimed at judge findings.
|
|
22
|
+
- `src/model-routing.ts`: advisory per-category routing with named evidence — the natural assignment mechanism (categories such as `review.findings-triage`, `workflow.lane-watch`), no new machinery.
|
|
23
|
+
- ADR-001 principle: ceremony scales with consequence — model spend scales the same way (where ceremony survived, value density is highest; where it shrank, cost is already zero).
|
|
24
|
+
|
|
25
|
+
## Proposal
|
|
26
|
+
|
|
27
|
+
1. **Findings triage** (pairs with OPP-003, prerequisite for the diversity-judge idea): a deterministic shape check always runs (severity present; proof references or `file:line` non-empty) — pure parse. When the decision model is enabled, each severe finding additionally gets one triage question ("does the cited evidence support the claim?") recorded as an advisory `evidence_supported: yes/no/unclear` flag next to the finding. The refuter still owns corroboration/refutation; degradation to advisory stays governed by OPP-003's rules; the flag never blocks by itself.
|
|
28
|
+
2. **Lane scope-watch** (pairs with OPP-001): on light-lane `task start` when `risk assess` reports `high`, and on light-lane completion, a mismatch notice is recorded. The deterministic part (tier vs lane mismatch) runs always; the model adds the semantic dimension (multi-subsystem footprint) only when enabled. Recorded as an advisory note in the checkpoint/ledger; never blocks, never changes the lane.
|
|
29
|
+
3. **Routing and settings**: new `routing.yml` categories with evidence links; everything gated by the existing `decision` settings section; unconfigured means zero behavior change and zero network (NFR-2).
|
|
30
|
+
|
|
31
|
+
## Affected areas
|
|
32
|
+
|
|
33
|
+
- `src/decision.ts` (new request builders: findings triage, lane watch)
|
|
34
|
+
- `src/workflow.ts` (light-lane complete/escalate advisory notice), `src/risk-assess.ts` (mismatch signal hook)
|
|
35
|
+
- OPP-003 integration point (review-refuter skill contract note; runtime wiring lands with OPP-003's runtime phase)
|
|
36
|
+
- `src/model-routing.ts` categories; `docs/decision-model-evaluation.md`
|
|
37
|
+
|
|
38
|
+
## Acceptance criteria
|
|
39
|
+
|
|
40
|
+
- Given the decision model unconfigured, when review or lane flows run, then behavior is byte-identical to today and no network request is made.
|
|
41
|
+
- Given a severe finding whose cited `file:line` does not support the claim, when triage runs enabled, then the finding carries the advisory flag and the refuter still decides corroboration.
|
|
42
|
+
- Given a light-lane start or completion with a `high` deterministic risk signal, then a mismatch note is recorded even without any model.
|
|
43
|
+
- Given any model output, then no evidence state, gate outcome, or approval changes (NFR-3); human acceptance is unchanged.
|
|
44
|
+
- Given `routing.yml`, then the new categories carry evidence links; unevidenced rules are labeled `unevidenced-rule`.
|
|
45
|
+
|
|
46
|
+
## Non-goals
|
|
47
|
+
|
|
48
|
+
- The decision model never judges finding correctness (that is the refuter on the host agent's model), never lowers risk tiers, never chooses lanes, never blocks, never replaces human acceptance.
|
|
49
|
+
- No new settings sections beyond the existing `decision` gate; no per-finding model calls without a batched alternative evaluated first.
|
|
50
|
+
|
|
51
|
+
## Open questions
|
|
52
|
+
|
|
53
|
+
- Per-finding triage (n calls, stronger isolation) vs one batched request (cheaper, weaker)? Measure on the self-hosting package's own review artifacts.
|
|
54
|
+
- Does scope-watch belong only in `workflow complete` output, or also in `task status`?
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# OPP-008: Embedding retrieval — covering-test detection and judge context scoping
|
|
2
|
+
|
|
3
|
+
- Status: Proposed
|
|
4
|
+
- Priority: P3
|
|
5
|
+
- Added: 2026-10-30
|
|
6
|
+
- Decision: [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md), Decision A (model-integration addendum)
|
|
7
|
+
- Companion: [OPP-007](OPP-007-decision-findings-triage-scope-watch.md)
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
Three implemented flows use weak approximations where the existing semantic index would materially sharpen results, and one new flow spends tokens where retrieval would save them:
|
|
12
|
+
|
|
13
|
+
1. **`untested_regression` in `risk assess` (OPP-002)** uses a sibling-file heuristic (`src/x.ts` → `src/x.test.ts`). A test exercising the changed module from a differently named file (`integration.spec.ts` via the public API) is invisible → false `high` flags.
|
|
14
|
+
2. **Adversarial judges (OPP-003)** receive the raw diff; related unchanged code (callers, consumed contracts) is missed or fetched ad hoc by the frontier model — the expensive way.
|
|
15
|
+
3. The semantic index (SEM-001…006) already exists and proved the pattern (SEM-006: scoped context reduced tokens without quality regression), but nothing in the new flows consumes it.
|
|
16
|
+
|
|
17
|
+
## Source concept
|
|
18
|
+
|
|
19
|
+
- ODD token economics: ceremony shrank, so spend shifted from writing/reading artifacts to **reviewing changes** — retrieval should target the new spend.
|
|
20
|
+
- Spec NFR-2: deterministic floor first — the sibling heuristic remains the exact no-model behavior; retrieval only _prevents false positives_ (retrieval absence never implies low risk: no covering test found → the flag stands).
|
|
21
|
+
- SEM contract: retrieval results are aids, never confirmed facts; stale index fails safe toward `high`.
|
|
22
|
+
|
|
23
|
+
## Proposal
|
|
24
|
+
|
|
25
|
+
1. **Covering-test retrieval** (pairs with OPP-002): when semantic mapping is enabled and an index exists, for each high-risk-category source path run one scoped semantic query ("tests exercising this module/symbol"). Top-k test-chunk hits count as covering evidence; none found → `untested_regression` stands. Retrieval-aided verdicts are labeled in reasons (`covering: semantic-retrieval`) so audits can distinguish heuristic from retrieval. Unconfigured: sibling heuristic exactly as today.
|
|
26
|
+
2. **Judge context scoping** (pairs with OPP-003): when rendering judge prompts with an index available, include the diff plus top-k retrieved chunks for changed symbols (callers, types, contracts) under a bounded token budget, labeled as context aids. Unconfigured: judges get the raw diff as today.
|
|
27
|
+
3. Both flow through the existing `semantic` settings gate and the chunk index built by `aiw scan` — no new settings, no reindexing inside risk assess (freshness stays `scan`'s job; a stale index can only miss, which fails safe toward `high`).
|
|
28
|
+
|
|
29
|
+
## Affected areas
|
|
30
|
+
|
|
31
|
+
- `src/risk-assess.ts` (covering-test probe gains an optional retrieval provider; deterministic fallback unchanged)
|
|
32
|
+
- `src/semantic.ts` (query reuse; symbol-aware scoped query helper)
|
|
33
|
+
- OPP-003 judge skill rendering (`resources/skills/review-judge-a.md`, `review-judge-b.md`: context sections are retrieval aids, never facts)
|
|
34
|
+
- `docs/semantic-mapping.md`; cross-links from [OPP-002](OPP-002-risk-assess-command.md) and [OPP-003](OPP-003-adversarial-review-package.md)
|
|
35
|
+
|
|
36
|
+
## Acceptance criteria
|
|
37
|
+
|
|
38
|
+
- Given no semantic config, when risk assess or judge prompts render, then behavior is byte-identical to today and offline.
|
|
39
|
+
- Given an index and a test exercising the changed module from a differently named file, when risk assess runs, then `untested_regression` is not flagged and the reason records `covering: semantic-retrieval`.
|
|
40
|
+
- Given retrieval finds no covering test, then `untested_regression` stands.
|
|
41
|
+
- Given judge prompt rendering with an index, then prompts include bounded retrieved chunks labeled as context aids within existing token ceilings.
|
|
42
|
+
- Given the embedding model unavailable at runtime, then flows degrade to deterministic behavior per the SEM contract.
|
|
43
|
+
|
|
44
|
+
## Non-goals
|
|
45
|
+
|
|
46
|
+
- No model-driven risk _tier_ changes: retrieval only prevents false `untested` flags; tiers stay rule-based.
|
|
47
|
+
- No new providers or settings sections; no reindexing inside risk assess.
|
|
48
|
+
|
|
49
|
+
## Open questions
|
|
50
|
+
|
|
51
|
+
- Symbol extraction for queries: deterministic import/export parsing vs path-semantics-only queries?
|
|
52
|
+
- Judge prompt budget: how to split the token ceiling between the diff and retrieved chunks?
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Workflow Proportionality Opportunities
|
|
2
|
+
|
|
3
|
+
Improvement opportunities derived from the analysis of Gentle-AI's ODD/RDD methodology (Organic Driven Development and Receipt-Driven Development), recorded in [ADR-001](../../.context/adrs/ADR-001-odd-rdd-proportionality-scope.md). Each document is self-contained: problem, source concept with references, proposal, affected areas, acceptance criteria, non-goals, and open questions — so any single opportunity can be picked up without reconstructing the original analysis.
|
|
4
|
+
|
|
5
|
+
The guiding principle: **ceremony should scale with the consequence of the change, not be constant.**
|
|
6
|
+
|
|
7
|
+
| ID | Opportunity | Priority | Depends on |
|
|
8
|
+
| ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -------- | ------------------------------------- |
|
|
9
|
+
| [OPP-001](OPP-001-proportional-ceremony-light-lane.md) | Proportional ceremony — light lane for small, understood work | P2 | OPP-002 (advisory signal, optional) |
|
|
10
|
+
| [OPP-002](OPP-002-risk-assess-command.md) | `aiw risk assess` — content-based risk classification with exact next command | P2 | — |
|
|
11
|
+
| [OPP-003](OPP-003-adversarial-review-package.md) | Adversarial review package — blind judges and proof-carrying findings | P3 | — |
|
|
12
|
+
| [OPP-004](OPP-004-exact-continuation-commands.md) | Exact continuation commands — every block names a runnable way out | P1 | — |
|
|
13
|
+
| [OPP-005](OPP-005-friction-benchmark.md) | Friction benchmark — measure how users get stuck (in-band / out-of-band / dead-end) | P3 | OPP-004 |
|
|
14
|
+
| [OPP-006](OPP-006-emission-usability-test-policy.md) | Emission ≠ usability — test that named commands actually work | P2 | OPP-004 (shared contract) |
|
|
15
|
+
| [007-decision-findings-triage-scope-watch](OPP-007-decision-findings-triage-scope-watch.md) | Decision model — findings-evidence triage and lane scope-watch (advisory-only) | P3 | OPP-001, OPP-003 (integration points) |
|
|
16
|
+
| [008-embedding-covering-tests-judge-context](OPP-008-embedding-covering-tests-judge-context.md) | Embedding retrieval — covering-test detection and judge context scoping | P3 | OPP-002, OPP-003 (integration points) |
|
|
17
|
+
|
|
18
|
+
Suggested starting order: **OPP-004 → OPP-002 → OPP-006 → OPP-001 → OPP-005 → OPP-003** (smallest/least invasive first; OPP-004's reason codes feed OPP-005 and OPP-006).
|
|
19
|
+
|
|
20
|
+
OPP-007 and OPP-008 are **model-integration addenda**: deliberately left for after field exposure — their priority should be set by real usage pain, not speculation. All model features respect the spec's NFR contracts: deterministic floor first, models advisory-only, unconfigured = byte-identical offline behavior.
|
|
21
|
+
|
|
22
|
+
## Planning
|
|
23
|
+
|
|
24
|
+
The implementation is planned as six waves in the project's own SDD format:
|
|
25
|
+
|
|
26
|
+
- Specification: [`.aiw/generated/specs/odd-rdd-proportionality.md`](../../.aiw/generated/specs/odd-rdd-proportionality.md) (requirements REQ-OPP-001…006 with Given/When/Then criteria)
|
|
27
|
+
- Implementation plan: [`.aiw/generated/plans/odd-rdd-proportionality.md`](../../.aiw/generated/plans/odd-rdd-proportionality.md) (TASK-001…010 with code, tests, validation, and evidence per task)
|
|
28
|
+
|
|
29
|
+
## Source material
|
|
30
|
+
|
|
31
|
+
- Gentle-AI repository: `Gentleman-Programming/gentle-ai` (v4, ODD mandatory protocol)
|
|
32
|
+
- Key upstream docs: `docs/usage.md` (ODD protocol), `docs/review-integration.md` (RDD contract, `review assess`, stop reason codes, refuter probe), `docs/architecture/organic-rdd.md` (atomic review lifecycle), `docs/architecture/the-organic-rdd-story.md` (audit lessons, benchmark), `docs/review-authority-threat-model.md`
|
|
33
|
+
- What was deliberately **not** adopted and why: see ADR-001, Decision B
|
|
@@ -31,6 +31,9 @@
|
|
|
31
31
|
- The product must support brainstorming, specification, design, planning, implementation, and verification stages.
|
|
32
32
|
- Requirements, decisions, tasks, code changes, tests, and evidence must be traceable.
|
|
33
33
|
- A task must not be considered complete without configured validation evidence.
|
|
34
|
+
- Ceremony must scale with the consequence of the change: `aiw risk assess` classifies changes by content (read-only, deterministic); small, understood work may run in a human-chosen light lane that records evidence and rationale instead of durable artifacts, and escalates explicitly on scope growth.
|
|
35
|
+
- Every blocking CLI exit must name a stable reason code and an exact runnable continuation, or state explicitly that no automatic continuation exists.
|
|
36
|
+
- When the adversarial review skills are installed, review findings must carry proof: severe findings without a reproducing command or self-proving citations degrade to advisory, and only corroborated severe findings may block.
|
|
34
37
|
|
|
35
38
|
### Portability
|
|
36
39
|
|
package/docs/product-vision.md
CHANGED
|
@@ -17,6 +17,7 @@ AI coding tools often depend on implicit prompts, fragmented configuration, repe
|
|
|
17
17
|
5. Project artifacts are English by default.
|
|
18
18
|
6. Context is loaded just in time to control token cost.
|
|
19
19
|
7. Unsafe capabilities require explicit permission.
|
|
20
|
+
8. Ceremony scales with the consequence of the change — a documentation edit gets zero ceremony; two lines of authentication code get full review.
|
|
20
21
|
|
|
21
22
|
## Product outcomes
|
|
22
23
|
|
package/docs/quality.md
CHANGED
|
@@ -19,6 +19,16 @@ Red → Green → Refactor → Verify
|
|
|
19
19
|
|
|
20
20
|
No feature is complete without tests for its happy path, invalid input, conflicts, and safe failure behavior.
|
|
21
21
|
|
|
22
|
+
## Emission is not usability
|
|
23
|
+
|
|
24
|
+
A test that verifies a message was emitted does not verify it is usable. Any user-facing message, gate output, skill instruction, or generated prompt that names an `aiw` command must be covered by a test that executes that command in the state the message describes and asserts it resolves or advances that state. Emission-only assertions are insufficient for messages that name commands.
|
|
25
|
+
|
|
26
|
+
`src/command-evidence.ts` extracts command mentions from shipped artifacts; a ratchet test in `src/command-evidence.test.ts` executes every command named by the generated skills and the READMEs and fails when a named command is not real. Blocking continuations follow the stronger rule from [OPP-004](opportunities/OPP-004-exact-continuation-commands.md): a message may name a command only when running that command resolves the block it was printed for — verified by executing the printed continuation.
|
|
27
|
+
|
|
28
|
+
## Friction budget
|
|
29
|
+
|
|
30
|
+
`scripts/friction-bench.mjs` drives packaged-CLI journeys (happy paths and adversarial states) and classifies every non-success exit as in-band (the printed continuation, executed, resolves or advances the block), out-of-band (guidance without the typed reason contract), or dead end (the printed continuation fails and names no way out). The budget is **zero dead ends**; CI fails on any. Out-of-band is the honest baseline to shrink over time. Runs are deterministic; the release gate re-verifies the budget and determinism via `scripts/friction-bench.node-test.mjs`.
|
|
31
|
+
|
|
22
32
|
## Completion validation
|
|
23
33
|
|
|
24
34
|
A feature is complete only when all affected layers are validated: domain, module integration, CLI or adapter integration, generated artifacts, end-to-end behavior, documentation, traceability, and security. Domain tests alone are insufficient for changes that affect external behavior or generated project state.
|
package/docs/release-process.md
CHANGED
|
@@ -2,6 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
AI Workflow releases are reproducible, evidence-backed, and published only from a clean protected branch.
|
|
4
4
|
|
|
5
|
+
## How a release ships
|
|
6
|
+
|
|
7
|
+
1. A contributor opens a pull request with the changes, the version bump in `package.json`, and the dated `CHANGELOG.md` entry, and requests review.
|
|
8
|
+
2. A maintainer reviews and merges the pull request into `main`.
|
|
9
|
+
3. The `Auto-tag releases` workflow notices that the merged version has no `v<version>` tag yet, creates the tag on the merge commit, and the `Release` workflow takes over: it validates the tagged source and publishes with npm trusted publishing and provenance.
|
|
10
|
+
|
|
11
|
+
No manual tagging and no environment approvals: merging an reviewed pull request with a version bump is the entire release action. Merges that do not bump the version are ignored by the auto-tag workflow.
|
|
12
|
+
|
|
5
13
|
## Preconditions
|
|
6
14
|
|
|
7
15
|
- The release version follows Semantic Versioning and matches the intended Git tag.
|
|
@@ -24,7 +32,7 @@ Install the exact candidate tarball in a new temporary project and exercise at l
|
|
|
24
32
|
|
|
25
33
|
## Publication
|
|
26
34
|
|
|
27
|
-
After human review of the
|
|
35
|
+
After human review of the pull request, the auto-tag workflow creates the version tag on the merge commit (the manual equivalent, if ever needed, is `git tag v<version> && git push origin v<version>`). The release workflow validates the exact tagged source and publishes with npm trusted publishing and provenance:
|
|
28
36
|
|
|
29
37
|
```bash
|
|
30
38
|
git tag -s v0.1.0 -m "AI Workflow v0.1.0"
|
package/docs/semantic-mapping.md
CHANGED
|
@@ -66,3 +66,7 @@ When semantic mapping is enabled and a valid index built with the active model e
|
|
|
66
66
|
3. Everything else is excluded, so measured token usage decreases while inferred-fact evidence still points at real scoped files.
|
|
67
67
|
|
|
68
68
|
Without semantic mapping (or when the index is missing, built with another model, or the embedding model is unavailable), the interpreter keeps the previous deterministic alphabetical scoping and scans make no network request for scoping purposes.
|
|
69
|
+
|
|
70
|
+
## Planned integration (OPP-008)
|
|
71
|
+
|
|
72
|
+
The next planned retrieval consumers follow the same contracts: covering-test detection for `aiw risk assess` (a semantic hit prevents a false `untested_regression` flag; absence fails safe toward `high`) and judge context scoping for the adversarial review skills (bounded retrieved chunks labeled as context aids). Deterministic fallbacks are unchanged; unconfigured behavior is byte-identical. See [OPP-008](opportunities/OPP-008-embedding-covering-tests-judge-context.md).
|