@antoneeo/mkt-agentic-sdlc-skill 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +48 -0
- package/README.md +35 -1
- package/gemini-extension.json +6 -6
- package/package.json +51 -50
- package/skills/mkt-agentic-sdlc/ENFORCEMENT.md +10 -1
- package/skills/mkt-agentic-sdlc/SKILL.md +221 -240
- package/skills/mkt-agentic-sdlc/dispatch.md +39 -1
- package/skills/mkt-agentic-sdlc/hybrid.md +47 -0
- package/skills/mkt-agentic-sdlc/review.md +80 -3
- package/skills/mkt-agentic-sdlc/scripts/mkt_check.py +2 -2
- package/skills/mkt-agentic-sdlc/scripts/sdlc_core.py +239 -1
- package/skills/mkt-agentic-sdlc/templates.md +15 -8
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# The Hybrid Seam — coexistence with devPNT (marketing lens)
|
|
2
|
+
|
|
3
|
+
**For whom**: the agent working a marketing engagement where the `devpnt_*` tools
|
|
4
|
+
are available and point at this project.
|
|
5
|
+
**Answers**: "which mode am I in, and who owns what when both authorities are live".
|
|
6
|
+
**Does not answer**: the process itself — the engagement triage, the SOSTAC
|
|
7
|
+
phases, the gates and the evidence discipline are `SKILL.md`'s, identical in both
|
|
8
|
+
modes.
|
|
9
|
+
|
|
10
|
+
Moved out of `SKILL.md` by F-052 so a Standalone session stops paying for a seam
|
|
11
|
+
it cannot reach. **The content below is unchanged**: a relocation, not a rewrite.
|
|
12
|
+
|
|
13
|
+
<!-- moved-block-sha256: b9b19d58a104e858f38460065160a61e64a6cd6f64f92c1bbf47fbf534d25ef6 -->
|
|
14
|
+
### Hybrid in symbiosis with devPNT
|
|
15
|
+
|
|
16
|
+
Use this mode when the `devpnt_*` tools are available and point at the current project.
|
|
17
|
+
|
|
18
|
+
Authoritative hierarchy:
|
|
19
|
+
1. **devPNT M-VISION**: strategic beacon of the plan cycle. Before strategy or tactics, read it and verify benefits, success signals, scope-in and non-goals.
|
|
20
|
+
2. **devPNT Master Plan**: the marketing roadmap; milestones are plan cycles (a quarter, a launch, a market entry).
|
|
21
|
+
3. **devPNT Action Plan**: the nine phases of the active engagement as tactical nodes.
|
|
22
|
+
4. **devPNT governed artifacts**: the marketing artifact set (table below).
|
|
23
|
+
5. **Local `mkt_docs/`**: readable context, Standalone fallback, evidence ledger home, shadow/mirror when useful.
|
|
24
|
+
|
|
25
|
+
### Ownership matrix (the Hybrid seam)
|
|
26
|
+
|
|
27
|
+
The skill owns the **process** (triage, phases, gates, lifecycle); devPNT owns the **machinery** (governed storage, versioned proposals, review wiring). The marketing artifacts occupy the same governance slots the software artifacts occupy in the sibling skill:
|
|
28
|
+
|
|
29
|
+
| Artifact | Standalone master | Hybrid master (devPNT slot) | Mirror rule |
|
|
30
|
+
|---|---|---|---|
|
|
31
|
+
| Marketing vision | `vision/MKT_VISION.md` | M-VISION (`milestone_vision_<slug>`) | filesystem copy is a shadow; DB wins |
|
|
32
|
+
| ICP & Personas | `strategy/ICP_PERSONAS.md` | `mkt_icp_personas` (D-UC slot) | shadow `SHADOW_[doc_key]_vX.Y.md` |
|
|
33
|
+
| Threat map | `strategy/THREAT_MAP.md` | `mkt_threat_map` (P-TM slot) | shadow |
|
|
34
|
+
| Objectives | `strategy/OBJECTIVES.md` | `mkt_objectives` | shadow |
|
|
35
|
+
| Strategy | `strategy/STRATEGY.md` | `mkt_strategy` (E-ISP slot) | shadow, exported BEFORE tactics work |
|
|
36
|
+
| Tactical plan | `tactics/TACTICAL_PLAN.md` | `mkt_tactical_plan` (E-TDD slot) | shadow, exported BEFORE action phase |
|
|
37
|
+
| Measurement plan | `tactics/MEASUREMENT_PLAN.md` | `mkt_measurement_plan` (E-TP slot) | shadow |
|
|
38
|
+
| Evidence ledger | `research/evidence_ledger.md` | `research/evidence_ledger.md` — **filesystem-first even in Hybrid** | validator needs it on disk; devPNT may reference, never copies |
|
|
39
|
+
| Final plan + one-pager | `deliverables/` | assembled from ACCEPTED artifact versions | PDF via `devpnt_generate_document_pdf` when available |
|
|
40
|
+
| Handoff | `audit/handoff.md` | `audit/handoff.md` | always filesystem |
|
|
41
|
+
|
|
42
|
+
Hybrid rules:
|
|
43
|
+
- devPNT is the governed source for plans and strategy artifacts; do not create a second truth in `mkt_docs/`.
|
|
44
|
+
- The skill stays autonomous: if devPNT is not there, switch to Standalone without losing capability.
|
|
45
|
+
- If the user request, the local vision and the M-VISION diverge, stop and make the conflict explicit.
|
|
46
|
+
- Never auto-accept devPNT proposals: present the preview and wait for explicit confirmation.
|
|
47
|
+
- Where the local devPNT protocol imposes stricter gates (vision creation/amendment gates, independent review gates), follow them: they are the same discipline this skill encodes.
|
|
@@ -82,15 +82,92 @@ its outcome. The trigger term "permission-gated" scopes the STOP; the row word
|
|
|
82
82
|
|
|
83
83
|
Use a different model from the author's where the client allows it.
|
|
84
84
|
|
|
85
|
+
**The capability floor — which tier may run this role.** Independence says the
|
|
86
|
+
reviewer must not be the author; it says nothing about whether the reviewer *can
|
|
87
|
+
do the job*. A gate run below the capability its judgement needs produces the
|
|
88
|
+
myopia the gate exists to catch, **while reporting as an independent review** —
|
|
89
|
+
worse than no gate, because it certifies. So each role carries a minimum tier.
|
|
90
|
+
Tiers are **client-relative capability levels, never provider names** (a name
|
|
91
|
+
rots at the next model release; the tier→model binding lives in the client's own
|
|
92
|
+
agent definition, never in this file):
|
|
93
|
+
|
|
94
|
+
| Role | Floor | Why there |
|
|
95
|
+
|---|---|---|
|
|
96
|
+
| **authoring a governed artifact** (Vision, the analysis/design, use cases, the Interface Contract, threat model — `dispatch.md`'s never-list owns the enumeration) | **deep** | the purest case of the rule below: the output is judgement, nothing scores it, and `dispatch.md`'s never-list forbids moving it elsewhere — so the session's own capability IS this work's capability |
|
|
97
|
+
| design review, closure review, any verdict on judgement | **deep** | the output is a judgement whose wrongness no check can see |
|
|
98
|
+
| a light conformance or consistency pass (schema, coverage, citation resolution) | **light** | the criterion is stated and mechanical |
|
|
99
|
+
| implementer dispatch, probe execution, anchor resolution, index regeneration | **economy** | a threshold signal catches a wrong cheap answer |
|
|
100
|
+
|
|
101
|
+
The rule underneath the table: **a tier may be lowered only where a threshold
|
|
102
|
+
signal exists** — `task.verify`, a test, a diff against a spec, an assertion
|
|
103
|
+
harness — because that is what makes a wrong cheap answer visible instead of
|
|
104
|
+
plausible. Where the output is judgement and nothing scores it, there is no
|
|
105
|
+
signal to lower against, and the floor is the whole policy.
|
|
106
|
+
|
|
107
|
+
**A session's floor is the highest floor among the roles it performs itself.** The
|
|
108
|
+
table above binds reviewers and dispatched subagents, chosen per call — but the
|
|
109
|
+
session doing the authoring was chosen by the USER before the agent existed, and
|
|
110
|
+
an agent cannot re-tier itself mid-session. So the rule that reaches it is not
|
|
111
|
+
routing, it is **disclosure**: where a session can determine that it is below the
|
|
112
|
+
floor for a role it performs itself, it says so **once per session, at the first
|
|
113
|
+
such role** — not once per unit — and the disclosure is repeated in the review
|
|
114
|
+
row's existing `reviewer` cell (Hybrid: `notes`), which is what makes the pair
|
|
115
|
+
below visible without adding a column.
|
|
116
|
+
|
|
117
|
+
**Say plainly how thin this reach is.** A session knows its model NAME; tiers here
|
|
118
|
+
are deliberately client-relative and this file maps no name to a tier, so the
|
|
119
|
+
antecedent is satisfiable only where a session can recognize its own tier — which
|
|
120
|
+
is also the session least likely to run the check. Where a session **cannot
|
|
121
|
+
determine its own tier**, it owes nothing: the rule binds what an agent can know
|
|
122
|
+
and say, never a capability it cannot acquire. **The reliably working half of this
|
|
123
|
+
rule is the one addressed to the user, not to the agent** — which is why the
|
|
124
|
+
package READMEs carry the tier-selection guidance, and why that is not a footnote.
|
|
125
|
+
|
|
126
|
+
**Never both below the floor on the same unit — except when nothing better
|
|
127
|
+
exists.** Where the authoring session ran below floor, the independent review's
|
|
128
|
+
floor becomes non-negotiable and the same unit should not ALSO run a below-floor
|
|
129
|
+
review. **The exception is the case *When independence and capability conflict*
|
|
130
|
+
governs below**: where no independent rung at or above the floor exists at all,
|
|
131
|
+
that rule wins, the review still runs, BOTH disclosures are recorded, and the
|
|
132
|
+
result is not a violation but the worst admissible state — honestly labelled.
|
|
133
|
+
Forbidding it outright would push the agent toward the abstention that rule
|
|
134
|
+
explicitly rejects.
|
|
135
|
+
|
|
136
|
+
**When independence and capability conflict, independence wins.** A client whose
|
|
137
|
+
only independent rung is bound below the floor (a single-model client, or a
|
|
138
|
+
subagent facility whose agent definition fixes a cheap model) still runs the
|
|
139
|
+
gate: a below-floor *independent* reviewer catches the class the author is
|
|
140
|
+
structurally blind to, while a self-pass catches none of it. What the floor then
|
|
141
|
+
requires is **disclosure, not abstention** — the row records the tier that
|
|
142
|
+
actually ran (`below floor: <reason>`), and that record IS the remedy. Where a
|
|
143
|
+
higher-capability path exists but is permission-gated, nothing new applies: *The
|
|
144
|
+
gated rung* above already governs, one question per gate per intact context.
|
|
145
|
+
Where the client exposes no capability choice at all, the row says
|
|
146
|
+
`single (client exposes no choice)` and the floor is satisfied by construction —
|
|
147
|
+
it is a routing rule, and a routing rule with one route is not a failure.
|
|
148
|
+
|
|
85
149
|
**Rounds are capped at 3.** FAIL → revise → re-review. If findings still stand
|
|
86
150
|
after the third, stop and surface them to the user with the artifact — a gate that
|
|
87
151
|
can block forever gets removed. **Log one row per completed review, PASS or FAIL**
|
|
88
152
|
— a FAIL surfaced to the user is the highest-value outcome the gate produces, and
|
|
89
153
|
logging only passes would erase exactly that evidence. The row goes in
|
|
90
154
|
`ai_docs/audit/reviews/REVIEW_LOG.md` (create it if absent — `templates.md`):
|
|
91
|
-
`| date | doc_key | tier | reviewer | findings_raised | findings_real | verdict | revise_rounds |`,
|
|
92
|
-
with `tier` = `design` or `closure` in Standalone.
|
|
93
|
-
|
|
155
|
+
`| date | doc_key | tier | model | reviewer | findings_raised | findings_real | verdict | revise_rounds |`,
|
|
156
|
+
with `tier` = `design` or `closure` in Standalone. **`model` records the capability
|
|
157
|
+
tier that actually ran** — `deep`, `light`, `economy`, `single (client exposes no
|
|
158
|
+
choice)`, or `below floor: <reason>` — never a provider name in doctrine (a project
|
|
159
|
+
may of course write its own model names in its own log). Without it the floor above
|
|
160
|
+
is a belief: the row would say a gate ran and never whether it could do its job, and
|
|
161
|
+
the correlation with `revise_rounds` and `findings_real` — the only evidence the
|
|
162
|
+
policy pays — could not be drawn.
|
|
163
|
+
|
|
164
|
+
**One schema for both modes**, meaning one required CORE plus each mode's own
|
|
165
|
+
**mode-specific** realization columns. The core is what both modes actually share: `date`, `doc_key`,
|
|
166
|
+
`tier`, `model`, `findings_raised`, `findings_real`, `verdict`, `revise_rounds`.
|
|
167
|
+
Around it, Standalone adds `reviewer` and Hybrid adds `instrument` and `notes` —
|
|
168
|
+
a devPNT row carries no `reviewer` column at all, so "one identical column list"
|
|
169
|
+
was never true of the two modes. This costs nothing for the reason `templates.md`
|
|
170
|
+
§`ai_docs/audit/reviews/REVIEW_LOG.md` owns and states.
|
|
94
171
|
The log is how the gate's value is measured over time; skipping it makes the gate
|
|
95
172
|
unfalsifiable, the same defect as an unnamed EXISTS or a faked router verdict.
|
|
96
173
|
|
|
@@ -18,7 +18,7 @@ From the overlay (unchanged behaviour, unchanged exit codes):
|
|
|
18
18
|
index [--root R] (re)generate INDEX.md
|
|
19
19
|
|
|
20
20
|
From the shared core (new to this distribution):
|
|
21
|
-
stale / mark / gate / orient / plan / migrate -- the spine commands, identical everywhere
|
|
21
|
+
stale / mark / gate / orient / benefit / plan / migrate -- the spine commands, identical everywhere
|
|
22
22
|
|
|
23
23
|
Both files must sit in the same directory. Copying only this one fails at import,
|
|
24
24
|
loudly, which is the intended failure.
|
|
@@ -60,7 +60,7 @@ sdlc_core.set_profile(
|
|
|
60
60
|
skill_name="mkt-agentic-sdlc",
|
|
61
61
|
unit_noun="engagement",
|
|
62
62
|
support_files=("templates.md", "frameworks.md", "research.md", "elicitation.md",
|
|
63
|
-
"review.md", "routing.md", "guides.md", "vision.md", "dispatch.md",
|
|
63
|
+
"review.md", "routing.md", "guides.md", "vision.md", "hybrid.md", "dispatch.md",
|
|
64
64
|
"ENFORCEMENT.md"),
|
|
65
65
|
capabilities=(
|
|
66
66
|
# spine
|
|
@@ -1320,7 +1320,7 @@ def cmd_validate(root, strict=False, hybrid=False):
|
|
|
1320
1320
|
# comment-stripped, anchored: a '<!-- TODO: the ## Capability Ledger -->'
|
|
1321
1321
|
# must not read as the section being present
|
|
1322
1322
|
# Hybrid: the design lives in devPNT and its §4.5 gate owns this slot
|
|
1323
|
-
# (
|
|
1323
|
+
# (the Hybrid ownership matrix: "run ONE of them, never both"), and its log
|
|
1324
1324
|
# rows are keyed on e_isp_/e_tdd_ doc_keys, not on this filename -- so
|
|
1325
1325
|
# firing here would be a permanent, unfixable false positive.
|
|
1326
1326
|
if not hybrid and design_review_due(meta) and not review_logged(root, p.name):
|
|
@@ -2052,6 +2052,237 @@ def cmd_plan(root, args):
|
|
|
2052
2052
|
return 0
|
|
2053
2053
|
|
|
2054
2054
|
|
|
2055
|
+
# --- F-050: the benefit report -------------------------------------------
|
|
2056
|
+
# The process's central claim is that it makes divergence visible BEFORE
|
|
2057
|
+
# implementation. That claim lived in prose for four months while the evidence
|
|
2058
|
+
# for it sat unread in REVIEW_LOG. This reads it.
|
|
2059
|
+
#
|
|
2060
|
+
# Columns are resolved BY HEADER NAME, never by width. `templates.md` states
|
|
2061
|
+
# that contract ("extra or reordered columns are fine -- but the header must
|
|
2062
|
+
# say `tier`") and three widths are legal today: the original 8, Standalone's 9
|
|
2063
|
+
# (F-048 added `model`), and Hybrid's 10 (`instrument` + `notes`, and no
|
|
2064
|
+
# `reviewer` at all). A width whitelist reads none of the last, and silently
|
|
2065
|
+
# reports a devPNT-governed project as having no data at all.
|
|
2066
|
+
#
|
|
2067
|
+
# Two honesty rules, both learned from this unit's own review: whatever cannot
|
|
2068
|
+
# be read is COUNTED and NAMED, at the row level AND at the cell level. A ratio
|
|
2069
|
+
# over a silently shrunken denominator is worse than no ratio.
|
|
2070
|
+
REVIEW_CORE = ("date", "doc_key", "tier", "findings_real", "verdict")
|
|
2071
|
+
REVIEW_POSITIONAL = {
|
|
2072
|
+
8: ("date", "doc_key", "tier", "reviewer", "findings_raised",
|
|
2073
|
+
"findings_real", "verdict", "revise_rounds"),
|
|
2074
|
+
9: ("date", "doc_key", "tier", "model", "reviewer", "findings_raised",
|
|
2075
|
+
"findings_real", "verdict", "revise_rounds"),
|
|
2076
|
+
10: ("date", "doc_key", "tier", "model", "instrument", "findings_raised",
|
|
2077
|
+
"findings_real", "verdict", "revise_rounds", "notes"),
|
|
2078
|
+
}
|
|
2079
|
+
|
|
2080
|
+
|
|
2081
|
+
def _table_cells(line):
|
|
2082
|
+
"""Cells of a Markdown table row, or None when the line is not one.
|
|
2083
|
+
Separator rows (---, :---:, any alignment colons) are not data."""
|
|
2084
|
+
s = line.strip()
|
|
2085
|
+
if not s.startswith("|"):
|
|
2086
|
+
return None
|
|
2087
|
+
if set(s) <= set("|-: "):
|
|
2088
|
+
return None
|
|
2089
|
+
return [c.strip() for c in s.strip("|").split("|")]
|
|
2090
|
+
|
|
2091
|
+
|
|
2092
|
+
def parse_review_log(text):
|
|
2093
|
+
"""-> (rows, unparsed). Columns come from the header when there is one.
|
|
2094
|
+
|
|
2095
|
+
Falls back to the positional maps only for a table with no header, so a
|
|
2096
|
+
hand-kept log still reads. A row whose cell count does not match the
|
|
2097
|
+
header's is unparsed and named -- never dropped."""
|
|
2098
|
+
rows, unparsed, names = [], [], None
|
|
2099
|
+
for n, line in enumerate(text.splitlines(), 1):
|
|
2100
|
+
cells = _table_cells(line)
|
|
2101
|
+
if cells is None:
|
|
2102
|
+
continue
|
|
2103
|
+
lowered = [c.lower() for c in cells]
|
|
2104
|
+
if names is None and "tier" in lowered and "verdict" in lowered:
|
|
2105
|
+
names = lowered # the header IS the column map
|
|
2106
|
+
continue
|
|
2107
|
+
# The header supplies the map -- that is what makes reordered and extra
|
|
2108
|
+
# columns readable. But a log legitimately carries MIXED widths (F-048
|
|
2109
|
+
# widened the schema and did not rewrite history), so a row that does
|
|
2110
|
+
# not match the header is read by the positional map for its own width
|
|
2111
|
+
# before it is given up on. Only a width no map knows is unparsed.
|
|
2112
|
+
if names is not None and len(cells) == len(names):
|
|
2113
|
+
row = dict(zip(names, cells))
|
|
2114
|
+
else:
|
|
2115
|
+
positional = REVIEW_POSITIONAL.get(len(cells))
|
|
2116
|
+
if positional is None:
|
|
2117
|
+
unparsed.append((n, "%d cells: %s" % (
|
|
2118
|
+
len(cells),
|
|
2119
|
+
"header declares %d and no positional map fits"
|
|
2120
|
+
% len(names) if names else "no header, and no map fits")))
|
|
2121
|
+
continue
|
|
2122
|
+
row = dict(zip(positional, cells))
|
|
2123
|
+
missing = [c for c in REVIEW_CORE if c not in row]
|
|
2124
|
+
if missing:
|
|
2125
|
+
unparsed.append((n, "no %s column" % "/".join(missing)))
|
|
2126
|
+
continue
|
|
2127
|
+
if not re.match(r"^\d{4}-\d{2}-\d{2}$", row["date"]):
|
|
2128
|
+
unparsed.append((n, "date cell %r is not a date" % row["date"][:24]))
|
|
2129
|
+
continue
|
|
2130
|
+
rows.append(row)
|
|
2131
|
+
return rows, unparsed
|
|
2132
|
+
|
|
2133
|
+
|
|
2134
|
+
def review_findings(value):
|
|
2135
|
+
"""The count in a `findings_real` cell, or None when it states no number.
|
|
2136
|
+
|
|
2137
|
+
None is NOT zero. `all` and `VOID -- instrument error` appear in real logs,
|
|
2138
|
+
and counting them as zero findings silently deflates the one ratio this
|
|
2139
|
+
report exists to state. Anchored at the START of the cell so a prose cell
|
|
2140
|
+
like `R1 4 BLOCK + 8 WARN` does not yield a confident 4."""
|
|
2141
|
+
m = re.match(r"^(\d+)\b", (value or "").strip())
|
|
2142
|
+
return int(m.group(1)) if m else None
|
|
2143
|
+
|
|
2144
|
+
|
|
2145
|
+
def _is_fail(verdict):
|
|
2146
|
+
"""FAIL as a WORD, so `PASS (no failures)` is not counted as a failure."""
|
|
2147
|
+
return re.search(r"\bFAIL\b", (verdict or "").upper()) is not None
|
|
2148
|
+
|
|
2149
|
+
|
|
2150
|
+
def _is_pass(verdict):
|
|
2151
|
+
return re.search(r"\bPASS\b", (verdict or "").upper()) is not None
|
|
2152
|
+
|
|
2153
|
+
|
|
2154
|
+
def _print_read_cost():
|
|
2155
|
+
"""The cost side, printed with the benefit and NEVER without it: a catch
|
|
2156
|
+
rate with no cost term justifies unbounded doctrine, and a cost term with
|
|
2157
|
+
no catch rate justifies deleting the process. When it cannot be measured
|
|
2158
|
+
the line still prints, saying so -- silence here IS the catch rate quoted
|
|
2159
|
+
alone, which is the thing this report exists to prevent."""
|
|
2160
|
+
skill_dir = Path(__file__).resolve().parent.parent
|
|
2161
|
+
contract = skill_dir / "SKILL.md"
|
|
2162
|
+
print()
|
|
2163
|
+
if not contract.is_file():
|
|
2164
|
+
print("Read cost: unavailable -- SKILL.md not found beside this "
|
|
2165
|
+
"validator (%s)" % skill_dir)
|
|
2166
|
+
return
|
|
2167
|
+
files = [p for p in sorted(skill_dir.glob("*.md")) if p.name != "SKILL.md"]
|
|
2168
|
+
support = sum(p.stat().st_size for p in files)
|
|
2169
|
+
head = contract.stat().st_size
|
|
2170
|
+
# The TOTAL, and the files itemised. Printing only the two terms lets the
|
|
2171
|
+
# one that fell be quoted while the one that grew stays lumped -- which is
|
|
2172
|
+
# exactly how a -7.8% headline coexisted with a read surface that grew.
|
|
2173
|
+
print("Read cost of %s: SKILL.md %d bytes (every session) + %d bytes "
|
|
2174
|
+
"across %d support files (read on trigger) = %d bytes total"
|
|
2175
|
+
% (skill_dir.name, head, support, len(files), head + support))
|
|
2176
|
+
for p in files:
|
|
2177
|
+
print(" %-24s %8d" % (p.name, p.stat().st_size))
|
|
2178
|
+
|
|
2179
|
+
|
|
2180
|
+
def cmd_benefit(root):
|
|
2181
|
+
"""Report what the process caught, beside what it costs to read.
|
|
2182
|
+
|
|
2183
|
+
ALWAYS exits 0 and emits no warning or advisory. This number is the
|
|
2184
|
+
criterion future units are judged by; a measurement that can fail a build
|
|
2185
|
+
becomes a target, and a targeted measurement stops measuring."""
|
|
2186
|
+
log = root / review_log_rel()
|
|
2187
|
+
print("=== benefit: what the process caught, and what it costs to read ===")
|
|
2188
|
+
if not log.is_file():
|
|
2189
|
+
print("[info] no %s yet: nothing to report. The number appears once "
|
|
2190
|
+
"reviews are logged." % review_log_rel())
|
|
2191
|
+
_print_read_cost()
|
|
2192
|
+
return 0
|
|
2193
|
+
rows, unparsed = parse_review_log(read_text(log))
|
|
2194
|
+
# `tier` carries the MOMENT in Standalone (design / closure) and the
|
|
2195
|
+
# reviewer WEIGHT in Hybrid (deep / light / code / guide / vision). A
|
|
2196
|
+
# Hybrid row is a real review whose moment this column does not state, so
|
|
2197
|
+
# it is counted and named, never folded into a moment by guesswork.
|
|
2198
|
+
moments = {"design": [], "closure": []}
|
|
2199
|
+
unstated = {}
|
|
2200
|
+
unstated_found = unstated_code = unstated_uncounted = 0
|
|
2201
|
+
for r in rows:
|
|
2202
|
+
tier = r["tier"].lower()
|
|
2203
|
+
if tier.startswith("design"):
|
|
2204
|
+
moments["design"].append(r)
|
|
2205
|
+
elif tier.startswith("closure"):
|
|
2206
|
+
moments["closure"].append(r)
|
|
2207
|
+
else:
|
|
2208
|
+
unstated[tier] = unstated.get(tier, 0) + 1
|
|
2209
|
+
n = review_findings(r["findings_real"])
|
|
2210
|
+
if n is None:
|
|
2211
|
+
unstated_uncounted += 1
|
|
2212
|
+
else:
|
|
2213
|
+
unstated_found += n
|
|
2214
|
+
if tier.startswith("code"):
|
|
2215
|
+
unstated_code += n
|
|
2216
|
+
classified = moments["design"] + moments["closure"]
|
|
2217
|
+
print("rows parsed: %d rows unparsed: %d%s"
|
|
2218
|
+
% (len(rows), len(unparsed),
|
|
2219
|
+
(" -> " + "; ".join("line %d: %s" % u for u in unparsed[:5]))
|
|
2220
|
+
if unparsed else ""))
|
|
2221
|
+
print("reviews with a stated moment: %d moment not stated by `tier`: %d%s"
|
|
2222
|
+
% (len(classified), sum(unstated.values()),
|
|
2223
|
+
(" (" + ", ".join("%s x%d" % (t, n) for t, n
|
|
2224
|
+
in sorted(unstated.items())) + ")")
|
|
2225
|
+
if unstated else ""))
|
|
2226
|
+
if not classified:
|
|
2227
|
+
_print_read_cost()
|
|
2228
|
+
return 0
|
|
2229
|
+
dates = sorted(r["date"] for r in classified)
|
|
2230
|
+
print("span of the reviews counted below: %s -> %s" % (dates[0], dates[-1]))
|
|
2231
|
+
|
|
2232
|
+
print()
|
|
2233
|
+
print("%-9s %6s %10s %11s %10s" % ("moment", "rows", "findings",
|
|
2234
|
+
"FAIL verdict", "per review"))
|
|
2235
|
+
totals, uncounted = {}, 0
|
|
2236
|
+
for m in ("design", "closure"):
|
|
2237
|
+
rs = moments[m]
|
|
2238
|
+
counts = [review_findings(r["findings_real"]) for r in rs]
|
|
2239
|
+
uncounted += sum(1 for c in counts if c is None)
|
|
2240
|
+
found = sum(c for c in counts if c is not None)
|
|
2241
|
+
failed = sum(1 for r in rs if _is_fail(r["verdict"]))
|
|
2242
|
+
totals[m] = found
|
|
2243
|
+
print("%-9s %6d %10d %11d %10.1f"
|
|
2244
|
+
% (m, len(rs), found, failed, (found / len(rs)) if rs else 0.0))
|
|
2245
|
+
total = sum(totals.values())
|
|
2246
|
+
if uncounted:
|
|
2247
|
+
print("findings cells stating no number: %d (counted as unknown, "
|
|
2248
|
+
"never as zero)" % uncounted)
|
|
2249
|
+
if total:
|
|
2250
|
+
print()
|
|
2251
|
+
print("Caught BEFORE the code existed: %d / %d findings = %.0f%%"
|
|
2252
|
+
% (totals["design"], total, 100.0 * totals["design"] / total))
|
|
2253
|
+
# The residue, stated as a BOUND rather than as a direction. The
|
|
2254
|
+
# sentence this replaces claimed the share was "a floor, not a
|
|
2255
|
+
# ceiling" because `code` rows are post-implementation -- which is
|
|
2256
|
+
# backwards, since folding them LOWERS the design share. But the fact
|
|
2257
|
+
# was right and only the direction wrong, so `code` findings sit on
|
|
2258
|
+
# the closure side of the upper bound instead of being discarded with
|
|
2259
|
+
# it. No direction is printed at all: the unstated set pushes both
|
|
2260
|
+
# ways, and only an interval is true of all of it.
|
|
2261
|
+
if unstated_found:
|
|
2262
|
+
denom = total + unstated_found
|
|
2263
|
+
print("Bounded by the %d findings in rows whose moment `tier` "
|
|
2264
|
+
"does not state: %.0f%%-%.0f%% (`code` rows are "
|
|
2265
|
+
"post-implementation by definition, so their findings sit "
|
|
2266
|
+
"on the closure side of the upper bound)"
|
|
2267
|
+
% (unstated_found, 100.0 * totals["design"] / denom,
|
|
2268
|
+
100.0 * (totals["design"] + unstated_found - unstated_code)
|
|
2269
|
+
/ denom))
|
|
2270
|
+
if unstated_uncounted:
|
|
2271
|
+
print("%d unstated rows state no findings count (`all`, `VOID`): "
|
|
2272
|
+
"neither zero nor countable, so they lie outside both "
|
|
2273
|
+
"bounds" % unstated_uncounted)
|
|
2274
|
+
failed_all = sum(1 for r in classified if _is_fail(r["verdict"]))
|
|
2275
|
+
inconclusive = sum(1 for r in classified if not _is_fail(r["verdict"])
|
|
2276
|
+
and not _is_pass(r["verdict"]))
|
|
2277
|
+
print("Reviews whose verdict contains FAIL: %d / %d = %.0f%%%s"
|
|
2278
|
+
% (failed_all, len(classified), 100.0 * failed_all / len(classified),
|
|
2279
|
+
(" (+%d inconclusive)" % inconclusive) if inconclusive else ""))
|
|
2280
|
+
print("(both figures are over the %d reviews whose moment `tier` states)"
|
|
2281
|
+
% len(classified))
|
|
2282
|
+
_print_read_cost()
|
|
2283
|
+
return 0
|
|
2284
|
+
|
|
2285
|
+
|
|
2055
2286
|
def cmd_orient(args):
|
|
2056
2287
|
"""SessionStart hook: emit a bounded, repo-sourced ai_docs/ orientation to
|
|
2057
2288
|
stdout and ALWAYS return 0 (fail-open, P-TM T8) -- a session hook must never
|
|
@@ -2315,6 +2546,11 @@ def main(argv=None):
|
|
|
2315
2546
|
gp.add_argument("--file", help="file path to evaluate (alternative to --hook)")
|
|
2316
2547
|
gp.add_argument("--protected", default="", help="protected prefixes separated by ';' (e.g. \"src/auth;src/crypto\")")
|
|
2317
2548
|
|
|
2549
|
+
sub.add_parser("benefit", parents=[common],
|
|
2550
|
+
help="report what the review gates caught (and what the "
|
|
2551
|
+
"doctrine costs to read) -- a report, never a gate: "
|
|
2552
|
+
"always exits 0")
|
|
2553
|
+
|
|
2318
2554
|
sub.add_parser("orient", parents=[common, hybrid_opt],
|
|
2319
2555
|
help="SessionStart hook: emit docs-root orientation to stdout (fail-open, zero-execution)")
|
|
2320
2556
|
|
|
@@ -2374,6 +2610,8 @@ def main(argv=None):
|
|
|
2374
2610
|
return cmd_stale(root, hybrid=args.hybrid)
|
|
2375
2611
|
if args.cmd == "mark":
|
|
2376
2612
|
return cmd_mark(root, args.paths)
|
|
2613
|
+
if args.cmd == "benefit":
|
|
2614
|
+
return cmd_benefit(root)
|
|
2377
2615
|
if args.cmd == "migrate":
|
|
2378
2616
|
return cmd_migrate(root, args)
|
|
2379
2617
|
if args.cmd == "plan":
|
|
@@ -496,16 +496,18 @@ Date: 2026-06-11 (UTC)
|
|
|
496
496
|
|
|
497
497
|
One row per completed review (`review.md` §When a review is due). Append-only; it
|
|
498
498
|
is the record that the gate ran and what it was worth. **One schema for both
|
|
499
|
-
modes** —
|
|
500
|
-
|
|
499
|
+
modes** — one required CORE (`date`, `doc_key`, `tier`, `model`, `findings_raised`,
|
|
500
|
+
`findings_real`, `verdict`, `revise_rounds`) plus each mode's own realization
|
|
501
|
+
columns: Standalone adds `reviewer`, a Hybrid devPNT row adds `instrument` and
|
|
502
|
+
`notes` and carries no `reviewer` at all. Both modes write this same file.
|
|
501
503
|
|
|
502
504
|
```markdown
|
|
503
505
|
# Independent Review Log
|
|
504
506
|
|
|
505
|
-
| date | doc_key | tier | reviewer | findings_raised | findings_real | verdict | revise_rounds |
|
|
506
|
-
|
|
507
|
-
| 2026-06-11 | ANALYSIS_login_sso.md | design | subagent (
|
|
508
|
-
| 2026-06-12 | diff feature/sso-login | closure | self-pass (declared; absent) | 2 | 2 | PASS with findings → corrections re-reviewed, PASS | 2 |
|
|
507
|
+
| date | doc_key | tier | model | reviewer | findings_raised | findings_real | verdict | revise_rounds |
|
|
508
|
+
|---|---|---|---|---|---|---|---|---|
|
|
509
|
+
| 2026-06-11 | ANALYSIS_login_sso.md | design | deep | subagent (fresh ctx) | 4 | 3 | FAIL → PASS | 2 |
|
|
510
|
+
| 2026-06-12 | diff feature/sso-login | closure | deep | self-pass (declared; absent) | 2 | 2 | PASS with findings → corrections re-reviewed, PASS | 2 |
|
|
509
511
|
|
|
510
512
|
## Notes
|
|
511
513
|
<!-- One short paragraph per review that found something worth remembering: what
|
|
@@ -517,9 +519,14 @@ adds values to the existing columns rather than a second table.
|
|
|
517
519
|
|
|
518
520
|
`tier` is the moment plus, in Hybrid, the reviewer weight: `design`, `design (late)`
|
|
519
521
|
and `closure` (Standalone); `deep`, `light`, `code`, `guide`, `vision` (devPNT gates
|
|
520
|
-
and the Vision blind check).
|
|
522
|
+
and the Vision blind check). `model` is a different question about the same review: `tier` says WHICH GATE ran,
|
|
523
|
+
`model` says WHAT CAPABILITY ran it (`deep`, `light`, `economy`,
|
|
524
|
+
`single (client exposes no choice)`, `below floor: <reason>` — the floor and its
|
|
525
|
+
arbitration are `review.md` §The capability floor). The two columns share the words
|
|
526
|
+
`deep`/`light` and mean different things by them, which is why they are read by
|
|
527
|
+
header and never by position. The validator reads this column by its header name, so
|
|
521
528
|
extra or reordered columns are fine — but the header must say `tier`. `reviewer`
|
|
522
|
-
records the realization actually used — fresh subagent, one-shot client run, or a
|
|
529
|
+
also carries the AUTHORING session's own below-floor disclosure where one is owed (the log has no column for the authoring tier, so `review.md` §The capability floor routes it here; Hybrid rows use `notes`). Otherwise it records the realization actually used — fresh subagent, one-shot client run, or a
|
|
523
530
|
**declared** self-pass — and, for any rung below rung 1, WHY the rung(s) above did
|
|
524
531
|
not run, in the ladder's reason words (`review.md`): `absent` — the client has no
|
|
525
532
|
such facility (a claim about the client, never about a policy); `gated, declined` —
|