@chrono-meta/fh-gate 1.4.94 → 1.4.96
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +2 -2
- package/AGENTS.md +18 -0
- package/CHEATSHEET.md +1 -1
- package/CLAUDE.md +1 -1
- package/knowledge/shared/harness-core/capability_composition_contract.md +25 -4
- package/knowledge/shared/harness-core/fh_detail_protocols.md +12 -0
- package/knowledge/shared/harness-core/ship_readiness_gate.md +7 -4
- package/knowledge/shared/learnings/subagent_invocations_log.yaml +67 -1
- package/package.json +7 -1
- package/plugins/fh-commons/.claude-plugin/plugin.json +1 -1
- package/plugins/fh-commons/agents/quench-challenger.md +49 -23
- package/plugins/fh-commons/skills/convergence-loop/SKILL.md +14 -0
- package/plugins/fh-commons/skills/deliberation/SKILL.md +14 -0
- package/plugins/fh-commons/skills/mcp-circuit-breaker/SKILL.md +40 -14
- package/plugins/fh-commons/skills/token-budget-gate/SKILL.md +31 -12
- package/plugins/fh-meta/.claude-plugin/plugin.json +1 -1
- package/plugins/fh-meta/CHANGELOG.md +36 -0
- package/plugins/fh-meta/agents/beginner.md +4 -1
- package/plugins/fh-meta/agents/challenger.md +7 -1
- package/plugins/fh-meta/agents/expert.md +1 -1
- package/plugins/fh-meta/agents/fact-checker.md +7 -1
- package/plugins/fh-meta/agents/hub-persona-auditor.md +2 -1
- package/plugins/fh-meta/agents/main-player.md +4 -1
- package/plugins/fh-meta/agents/persona-innovator.md +10 -2
- package/plugins/fh-meta/skills/agent-composer/SKILL.md +2 -2
- package/plugins/fh-meta/skills/apex-review/SKILL.md +5 -0
- package/plugins/fh-meta/skills/asset-placement-gate/SKILL.md +38 -8
- package/plugins/fh-meta/skills/auto-decorrelation/SKILL.md +16 -2
- package/plugins/fh-meta/skills/context-doctor/SKILL_detail.md +45 -10
- package/plugins/fh-meta/skills/corpus-grounding-expander/SKILL.md +14 -5
- package/plugins/fh-meta/skills/cross-ecosystem-synergy-detection/SKILL.md +93 -30
- package/plugins/fh-meta/skills/deep-clarify/SKILL.md +28 -9
- package/plugins/fh-meta/skills/fh/SKILL.md +4 -0
- package/plugins/fh-meta/skills/field-harvest/SKILL.md +34 -11
- package/plugins/fh-meta/skills/frontier-digest/SKILL.md +64 -8
- package/plugins/fh-meta/skills/frontier-digest/SKILL_detail.md +20 -7
- package/plugins/fh-meta/skills/goal-quench/SKILL.md +48 -15
- package/plugins/fh-meta/skills/goal-quench/SKILL_detail.md +58 -11
- package/plugins/fh-meta/skills/harness-doctor/SKILL_detail.md +109 -33
- package/plugins/fh-meta/skills/harvest-loop/SKILL.md +6 -1
- package/plugins/fh-meta/skills/hub-cc-pr-reviewer/SKILL.md +126 -17
- package/plugins/fh-meta/skills/install-doctor/SKILL.md +50 -14
- package/plugins/fh-meta/skills/install-wizard/SKILL.md +26 -7
- package/plugins/fh-meta/skills/install-wizard/SKILL_detail.md +68 -21
- package/plugins/fh-meta/skills/marketplace-gate/SKILL.md +59 -31
- package/plugins/fh-meta/skills/memory-hygiene/SKILL.md +64 -17
- package/plugins/fh-meta/skills/meta-prompt-builder/SKILL.md +38 -4
- package/plugins/fh-meta/skills/persona-roster-expander/SKILL.md +15 -7
- package/plugins/fh-meta/skills/pipeline-conductor/SKILL.md +13 -3
- package/plugins/fh-meta/skills/plugin-recommender/SKILL.md +39 -11
- package/plugins/fh-meta/skills/plugin-recommender/SKILL_detail.md +24 -7
- package/plugins/fh-meta/skills/prompt-regression/SKILL.md +54 -11
- package/plugins/fh-meta/skills/public-surface-audit/SKILL.md +56 -22
- package/plugins/fh-meta/skills/public-surface-audit/SKILL_detail.md +50 -11
- package/plugins/fh-meta/skills/return-path-gate/SKILL.md +36 -7
- package/plugins/fh-meta/skills/salience-splitter/SKILL.md +120 -7
- package/plugins/fh-meta/skills/salience-splitter/SKILL_detail.md +46 -13
- package/plugins/fh-meta/skills/sim-conductor/SKILL_detail.md +28 -3
- package/plugins/fh-meta/skills/steel-quench/SKILL.md +3 -1
- package/plugins/fh-meta/skills/verify-bidirectional/SKILL.md +72 -14
- package/scripts/capability_registry_check.sh +280 -0
- package/scripts/count_check.sh +47 -1
- package/scripts/degrade_direction_scan.sh +276 -6
- package/scripts/degrade_probe_capability.sh +105 -0
- package/scripts/package_coverage_check.sh +8 -0
- package/scripts/psa_probe_capability.sh +78 -0
- package/scripts/psa_scan_lib.sh +53 -1
- package/scripts/public_surface_scan_files.sh +8 -0
- package/scripts/relay_channel.sh +6 -1
- package/scripts/selfcheck.sh +15 -0
- package/scripts/test_capability_entrypoint_shipping.sh +132 -0
- package/scripts/test_count_check_readme_format_lanes.sh +75 -0
- package/scripts/test_degrade_scan_shell_probes.sh +415 -0
- package/scripts/validate_yaml.sh +146 -0
- package/templates/degrade_direction_scan.sh +276 -6
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: plugin-recommender
|
|
3
3
|
description: Given a task description, searches internal and external open-source ecosystems (including Codex marketplace and Claude Code marketplace) to find and recommend suitable plugins with installation guidance. Recommendation is quality-validation based (marketplace-listed + performance-validated), not source-origin based. Activates on "recommend a plugin", "what tool should I use?", "is there a plugin for this?", "recommend a tool". Also checks for duplicate installations.
|
|
4
4
|
user-invocable: true
|
|
5
|
-
allowed-tools: ["
|
|
5
|
+
allowed-tools: ["Read", "Grep", "Bash", "WebSearch", "WebFetch"]
|
|
6
6
|
model: sonnet
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -59,7 +59,9 @@ Tier is **independent of platform origin** (Anthropic / OpenAI / community). A w
|
|
|
59
59
|
|
|
60
60
|
2. **[Priority 1.5] Internal GHE Sister Assets (partially completed work — Tier 2)**: Check if user's task domain already exists in internal sister asset clusters. Prioritize direct use or adoption of sister assets if the user's task falls within these cluster domains.
|
|
61
61
|
|
|
62
|
-
3. **[Priority 2
|
|
62
|
+
3. **[Priority 2] Organization's Internal GHE (Tier 1–4)**: Search internal GHE with keywords like `claude-plugin`, `gemini-plugin` + user keywords / API search. Replace with your organization's internal GHE orgs.
|
|
63
|
+
|
|
64
|
+
4. **[Priority 2.5] Project Reference/Contribution Path**:
|
|
63
65
|
|
|
64
66
|
For cases where referencing or contributing to the project itself is more appropriate than installing a plugin. Provide guidance when there's intent for medium-term contribution rather than immediate use.
|
|
65
67
|
|
|
@@ -71,9 +73,7 @@ Tier is **independent of platform origin** (Anthropic / OpenAI / community). A w
|
|
|
71
73
|
- `plugin-recommender` [Priority 2.5]: When no immediately usable plugin is available → guide to project-level reference/contribution path (alternative to installing)
|
|
72
74
|
- `cross-ecosystem-synergy-detection`: Discover hidden synergies among already-installed skills (post-install utilization optimization)
|
|
73
75
|
|
|
74
|
-
Activation condition: Automatically entered when Step 2 [Priority 1]~[Priority 2] search yields no suitable plugin.
|
|
75
|
-
|
|
76
|
-
4. **[Priority 2] Organization's Internal GHE (Tier 1–4)**: Search internal GHE with keywords like `claude-plugin`, `gemini-plugin` + user keywords / API search. Replace with your organization's internal GHE orgs.
|
|
76
|
+
Activation condition: Automatically entered when Step 2 [Priority 1]~[Priority 2] search yields no suitable plugin. **This block therefore runs only after [Priority 2] has actually been executed** — the list above is in execution order (1 → 1.5 → 2 → 2.5 → 3), and the priority labels are names, not the run sequence.
|
|
77
77
|
|
|
78
78
|
5. **[Priority 3] External Open-Source Ecosystem**: WebSearch / WebFetch — "best github actions for X", "claude plugin for Y", etc. Simplification guard: defer external install if internal assets suffice.
|
|
79
79
|
|
|
@@ -91,16 +91,18 @@ When queried for a specific capability (e.g., "adversarial reviewer for bash cod
|
|
|
91
91
|
0. **Platform built-ins (Tier 0)** — does a built-in skill/command already cover the capability? Check the live session's available-skills list before any plugin search. A built-in that covers ~80% beats installing a plugin for the rest
|
|
92
92
|
1. **Installed locally** — `.claude/agents/`, `plugins/` in current cwd
|
|
93
93
|
2. **FH native skills** — always-loaded knowledge in `plugins/fh-meta/` and `plugins/fh-commons/`
|
|
94
|
-
3. **Claude Code marketplace** — `claude
|
|
95
|
-
4. **Codex marketplace** — `npx @openai/codex list
|
|
94
|
+
3. **Claude Code marketplace** — `claude plugin list --available --json` (marketplace plugins; `--available` **requires** `--json`) + `claude plugin marketplace list` to see which marketplaces are even configured. There is **no** CLI keyword-search subcommand — filter the JSON yourself, and fall back to the known CC registry (see verified targets above) for anything not in a configured marketplace
|
|
95
|
+
4. **Codex marketplace** — `npx --yes @openai/codex plugin list --available --json` (optionally `-m <marketplace>`) + `npx --yes @openai/codex plugin marketplace list`. Same limitation: no keyword search, filter the JSON
|
|
96
96
|
5. **npm ecosystem** — `@chrono-meta/`, `@anthropic/`, and other known-quality scoped packages
|
|
97
97
|
|
|
98
|
+
⚠️ **A failed or empty discovery lane is NOT "no candidates".** Both CLIs above list only *configured* marketplaces, and a non-zero exit / empty array means the lane did not answer — not that nothing exists. Report each lane's state explicitly (`EXECUTED` / `EMPTY` / `FAILED: <stderr>`) and never render a `FAILED` lane as a zero result; a lane that could not run must be re-run or replaced by the web-search fallback (Priority 3) before you tell the user nothing was found.
|
|
99
|
+
|
|
98
100
|
**Discovery priority**: built-in (Tier 0) > installed > FH native > Tier 1 (any platform) > Tier 2 > Tier 3 > Tier 4
|
|
99
101
|
**Tier 0 guard**: FH native wins over a built-in only when the FH skill adds governance the built-in lacks (e.g. `/goal` → `goal-quench` adds budget+quality gates; code diff review stays with built-in `/code-review`, FH-asset coherence with `hub-cc-pr-reviewer`)
|
|
100
102
|
|
|
101
103
|
**When sim-conductor chains here for persona discovery**: apply the same platform-aware search scoped to persona/simulation/review capability tags. Return discovered agents with their Tier rating so sim-conductor can decide whether to install or use a built-in brief.
|
|
102
104
|
|
|
103
|
-
For discovery bash commands (`claude
|
|
105
|
+
For discovery bash commands (`claude plugin list --available --json`, `npx --yes @openai/codex plugin list --available --json`, npm scoped search), see `SKILL_detail.md §Discovery-Bash`.
|
|
104
106
|
|
|
105
107
|
### Step 2.6: Quality Validation Signals
|
|
106
108
|
|
|
@@ -149,7 +151,8 @@ When user selects desired plugin from recommendation list, help with installatio
|
|
|
149
151
|
## Constraints
|
|
150
152
|
|
|
151
153
|
- **Recommendations, not guarantees**: Does not guarantee plugin performance, stability, or security.
|
|
152
|
-
- **Inbound supply-chain risk
|
|
154
|
+
- **Inbound supply-chain risk — EVERY tier, not just 3/4**: public skill registries have shipped malicious skills at scale (pointers: [HN 47370624](https://news.ycombinator.com/item?id=47370624) — 824 malicious skills reported on ClawHub; [HN 48678603](https://news.ycombinator.com/item?id=48678603) — Snyk ToxicSkills study. Secondary sources — verify before quoting figures). ⚠️ **The channel in both incidents was the marketplace listing itself — i.e. Tier 1/2.** An earlier version of this bullet scoped the warning to Tier 3/4, which inverted its own evidence: it warned loudest exactly where vetting exists and stayed silent where the cited attacks landed. Flag the risk before recommending **any** candidate; a high Tier means *listed and maintained*, never *audited for intent*.
|
|
155
|
+
- **Install string must be resolved, not relayed**: the `<name>` in an install command frequently comes from a web-search result. Before surfacing it, confirm the name resolves to the intended repository (owner/repo matches the source you are citing) — a plausible near-name is the cheapest form of this attack, and installation is irreversible on the machine that runs it.
|
|
153
156
|
- **User consent required**: Does not auto-install without explicit consent.
|
|
154
157
|
- **Search scope limitations**: Only searches within configured search space.
|
|
155
158
|
|
|
@@ -213,9 +216,34 @@ sim-conductor needs persona X (no installed/built-in match)
|
|
|
213
216
|
|
|
214
217
|
```
|
|
215
218
|
All Steps 0~5 completed
|
|
216
|
-
|
|
217
|
-
|
|
219
|
+
— mandatory-pass: each step produced its stated output, or is
|
|
220
|
+
explicitly marked N/A with the reason
|
|
221
|
+
|
|
222
|
+
+ Recommendation list table output (top 2~3 items, Tier + Platform + synergy
|
|
223
|
+
grade included)
|
|
224
|
+
— mandatory-pass: the table exists with all three columns populated
|
|
225
|
+
|
|
226
|
+
+ Every discovery lane reported with an explicit state
|
|
227
|
+
(EXECUTED / EMPTY / FAILED: <stderr>)
|
|
228
|
+
— measured: count lanes attempted vs lanes reporting a state; the two
|
|
229
|
+
numbers must match. A FAILED lane rendered as "no candidates" is a FAIL
|
|
230
|
+
of this condition, not a pass — the CLI lanes have no keyword search and
|
|
231
|
+
see only configured marketplaces, so an empty result is routinely a
|
|
232
|
+
non-answer rather than an absence (`not found` != `0`)
|
|
233
|
+
|
|
234
|
+
+ Install completed after user selection (or install skipped / 5-B migration
|
|
235
|
+
path guided)
|
|
236
|
+
— mandatory-pass: explicit user consent recorded before any install ran
|
|
237
|
+
|
|
218
238
|
+ Duplicate detection results reported
|
|
239
|
+
— mandatory-pass: `claude plugin list` output consulted in this run
|
|
240
|
+
|
|
241
|
+
+ Every install string surfaced to the user resolves to the repository it is
|
|
242
|
+
cited from (§Constraints — resolved, not relayed)
|
|
243
|
+
— judged; adversarial pairing: in the same run, resolve one candidate name
|
|
244
|
+
you already know is correct AND check one near-name variant. If the
|
|
245
|
+
procedure cannot separate that pair, the resolution check is
|
|
246
|
+
UNCALIBRATED and no install string may be presented as verified
|
|
219
247
|
```
|
|
220
248
|
|
|
221
249
|
## Failure Response
|
|
@@ -57,16 +57,24 @@ gh auth status # default host (github.com) only
|
|
|
57
57
|
# Unauthenticated → guide github.com PAT generation above
|
|
58
58
|
```
|
|
59
59
|
|
|
60
|
-
**Claude Code marketplace search
|
|
60
|
+
**Claude Code marketplace discovery** (there is **no** `search` subcommand — `claude mcp search` does not
|
|
61
|
+
exist and exits 1 with `unknown command 'search'`; list, then filter yourself):
|
|
61
62
|
```bash
|
|
62
|
-
claude
|
|
63
|
+
claude plugin marketplace list --json # which marketplaces are configured at all
|
|
64
|
+
claude plugin list --available --json # installed + available marketplace plugins (--available REQUIRES --json)
|
|
63
65
|
```
|
|
64
66
|
|
|
65
|
-
**Codex marketplace
|
|
67
|
+
**Codex marketplace discovery** (`list-agents` does not exist either — the real noun is `plugin`):
|
|
66
68
|
```bash
|
|
67
|
-
npx @openai/codex list
|
|
69
|
+
npx --yes @openai/codex plugin marketplace list
|
|
70
|
+
npx --yes @openai/codex plugin list --available --json # add -m <marketplace> to scope
|
|
68
71
|
```
|
|
69
72
|
|
|
73
|
+
> **Lane-state reporting (mandatory).** Neither CLI supports a keyword query, and both see only
|
|
74
|
+
> *configured* marketplaces. Record each lane as `EXECUTED` / `EMPTY` / `FAILED: <stderr>` and carry
|
|
75
|
+
> that state into the recommendation. **A `FAILED` lane must never be rendered as "no candidates"** —
|
|
76
|
+
> re-run it or fall back to Priority 3 web search before reporting an empty result.
|
|
77
|
+
|
|
70
78
|
**npm ecosystem search (scoped packages):**
|
|
71
79
|
```bash
|
|
72
80
|
npm search @chrono-meta [keyword]
|
|
@@ -113,11 +121,20 @@ If either duplicate condition met → skip install → report "Already active"
|
|
|
113
121
|
#### 5-1 through 5-3. Install Steps
|
|
114
122
|
|
|
115
123
|
1. Confirm intent: "Would you like to install the `[plugin-name]` plugin?"
|
|
116
|
-
2. On agreement
|
|
124
|
+
2. On agreement — **the two commands take different arguments**: `marketplace add` takes a *source*
|
|
125
|
+
(`<owner/repo>`, a URL, or a local path), never a plugin name; only `install` takes the plugin name.
|
|
117
126
|
```bash
|
|
118
|
-
claude plugin marketplace add [
|
|
119
|
-
|
|
127
|
+
# Usage: claude plugin marketplace add [options] SOURCE
|
|
128
|
+
# SOURCE = owner/repo, a URL, or a local path — NEVER a plugin name
|
|
129
|
+
SOURCE="owner/repo"
|
|
130
|
+
PLUGIN="plugin-name" # optionally "plugin-name@marketplace" to disambiguate
|
|
131
|
+
|
|
132
|
+
claude plugin marketplace add "$SOURCE"
|
|
133
|
+
# Usage: claude plugin install|i [options] PLUGIN
|
|
134
|
+
claude plugin install "$PLUGIN"
|
|
120
135
|
```
|
|
136
|
+
Skip the `marketplace add` line when the plugin already resolves from a configured marketplace
|
|
137
|
+
(check `claude plugin marketplace list`).
|
|
121
138
|
3. Post-install initial configuration guidance:
|
|
122
139
|
- **API token input**: Guide token generation path for external service APIs (Jira/Confluence/Slack — specify each service's token page URL + env var or plugin config storage location)
|
|
123
140
|
- **MCP connection**: If plugin uses MCP server, guide auto-update of `.mcp.json` or `claude mcp add` command
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: prompt-regression
|
|
3
|
-
description:
|
|
3
|
+
description: Statically checks harness assets against a known-answer probe set after rule/skill changes — inspects the changed source for the trigger phrases, chain links and gate conditions each probe expects, and reports PASS/FAIL/SKIP per probe. Source inspection only; it does not run live sessions, so it catches assets that no longer SAY the right thing, not models that stop DOING it. Triggers on "prompt regression", "did my changes break anything", "regression check", "test harness changes".
|
|
4
4
|
user-invocable: true
|
|
5
|
-
allowed-tools: ["Read", "Bash", "Glob", "Grep"]
|
|
5
|
+
allowed-tools: ["Read", "Write", "Bash", "Glob", "Grep"]
|
|
6
6
|
model: sonnet
|
|
7
7
|
complexity_routing:
|
|
8
8
|
base: sonnet
|
|
@@ -58,8 +58,15 @@ ls .claude/regression/probes.md 2>/dev/null || echo "NO_CUSTOM_PROBES"
|
|
|
58
58
|
```
|
|
59
59
|
|
|
60
60
|
**If custom probes exist**: load and use them. The hub repo ships its golden probe set
|
|
61
|
-
(known-answer offline eval,
|
|
62
|
-
|
|
61
|
+
(known-answer offline eval, **33 probes** with check classes — 27 mandatory-pass · 1 measured ·
|
|
62
|
+
5 judged, of which 2 rows are inert deletion anchors, so live coverage is 31) at exactly this path
|
|
63
|
+
— when present it is canonical and supersedes the default matrix below.
|
|
64
|
+
|
|
65
|
+
> Count from the file, not from this line: `grep -cE '^\| *`[A-Z][A-Z0-9-]*-[0-9]+` *\|'
|
|
66
|
+
> .claude/regression/probes.md`. Both this number and the tally inside `probes.md` read `32` until
|
|
67
|
+
> 2026-08-12, when the actual count was 33 — a probe had been added to section A without touching
|
|
68
|
+
> either summary. A count duplicated in two files rots independently; if the two disagree, the file
|
|
69
|
+
> wins and both get corrected.
|
|
63
70
|
|
|
64
71
|
**If no custom probes** (e.g. Mode C install without the hub repo): use the default
|
|
65
72
|
probe matrix below.
|
|
@@ -93,7 +100,23 @@ If change scope is `CLAUDE.md` core (10+ lines changed): run **full suite** (all
|
|
|
93
100
|
|
|
94
101
|
---
|
|
95
102
|
|
|
96
|
-
### Step 4.
|
|
103
|
+
### Step 4. Evaluate Affected Probes (static source inspection)
|
|
104
|
+
|
|
105
|
+
**What this step is, stated plainly.** Every check below reads the changed *source* and asks whether
|
|
106
|
+
the text a probe expects is still there. No live session is started, no model is prompted, no output
|
|
107
|
+
is compared against a recorded transcript. The description said "running standard prompt probes …
|
|
108
|
+
comparing outputs against saved baselines", which reads as live execution; the honest label is a
|
|
109
|
+
**known-answer static check**.
|
|
110
|
+
|
|
111
|
+
**What it therefore cannot catch** — say this in the report, do not leave it implied:
|
|
112
|
+
- a rule that is still present but has stopped *firing* (salience loss, ordering, competition
|
|
113
|
+
from another rule)
|
|
114
|
+
- a trigger phrase present in the file but shadowed by a higher-priority route
|
|
115
|
+
- any behavior change that leaves the source text identical
|
|
116
|
+
|
|
117
|
+
Those need a dispatched blind sim at the target tier (`sim-conductor`, or the target-tier sim gate in
|
|
118
|
+
`.claude/rules/fh_4axis_gate.md`). A green report here means *the assets still say the right thing*.
|
|
119
|
+
Treating it as behavioral evidence is the misread this section exists to prevent.
|
|
97
120
|
|
|
98
121
|
For each affected probe, evaluate:
|
|
99
122
|
|
|
@@ -150,24 +173,44 @@ If all probes pass:
|
|
|
150
173
|
|
|
151
174
|
After a deliberate behavior change (not a regression — an intentional improvement), update the baseline:
|
|
152
175
|
|
|
176
|
+
Prompt user first: *"Probe `G-GATE-03` now expects the new gate format. Update baseline? (y/n)"*
|
|
177
|
+
Only update on explicit `y` — never auto-update.
|
|
178
|
+
|
|
179
|
+
On `y`, actually perform the write. The previous version of this step was three lines of which two
|
|
180
|
+
were comments — it created the directory and stopped, so "baseline updated" was reported by a step
|
|
181
|
+
that had written nothing.
|
|
182
|
+
|
|
153
183
|
```bash
|
|
154
|
-
# Baseline stored as markdown in .claude/regression/
|
|
155
184
|
mkdir -p .claude/regression
|
|
156
|
-
# Write updated probe expectations
|
|
157
185
|
```
|
|
158
186
|
|
|
159
|
-
|
|
187
|
+
Then **use the `Write` tool** on `.claude/regression/probes.md` (create it from the SKILL.md default
|
|
188
|
+
matrix if absent) to apply the approved edit. Each approved change edits the probe's row in place —
|
|
189
|
+
Expected Behavior, Scope, and Class — and:
|
|
160
190
|
|
|
161
|
-
|
|
191
|
+
- update the `**Count**:` tally line **in the same edit** if a row was added or removed, including
|
|
192
|
+
the per-section breakdown and the class counts (a stale tally is exactly how the `32`/`33`
|
|
193
|
+
mismatch was introduced)
|
|
194
|
+
- append a one-line note under `**Baseline**:` recording the date and the reason for the change
|
|
195
|
+
- never rewrite rows the user did not approve
|
|
196
|
+
|
|
197
|
+
Confirm afterwards by re-reading the file and reporting the row's new content plus the recount —
|
|
198
|
+
`grep -cE '^\| *`[A-Z][A-Z0-9-]*-[0-9]+` *\|' .claude/regression/probes.md`. Do not report a
|
|
199
|
+
baseline update whose result you have not read back.
|
|
162
200
|
|
|
163
201
|
---
|
|
164
202
|
|
|
165
203
|
## Done When
|
|
166
204
|
|
|
167
|
-
- All affected probes are evaluated (PASS / FAIL / SKIP)
|
|
205
|
+
- All affected probes are evaluated (PASS / FAIL / SKIP) and the three counts sum to the number of
|
|
206
|
+
probes selected in Step 3 — class: measured (`PASS + FAIL + SKIP == selected`)
|
|
168
207
|
- Regression report is output with clear PASS/FAIL verdict — class: mandatory-pass
|
|
208
|
+
- The report states its own scope: **static source inspection, no live session run** — a green
|
|
209
|
+
verdict is never presented as behavioral evidence — class: mandatory-pass
|
|
169
210
|
- If FAIL: specific file + line fix is recommended — class: judged, paired with verify-bidirectional (the fix recommendation is re-checked, not trusted as-is)
|
|
170
|
-
- Baseline updated only on explicit user approval
|
|
211
|
+
- Baseline updated only on explicit user approval, and the written file is **read back** and its
|
|
212
|
+
probe count reported — class: mandatory-pass (HITL). Approval alone does not close this: a step
|
|
213
|
+
that prompts, gets `y`, and writes nothing satisfied the old wording.
|
|
171
214
|
|
|
172
215
|
---
|
|
173
216
|
|
|
@@ -38,7 +38,9 @@ deliberately excluded — `CLAUDE.local.md`, the companion store, and local sess
|
|
|
38
38
|
*correct* home for operator-private tokens, so finding them there is not a leak.
|
|
39
39
|
|
|
40
40
|
```bash
|
|
41
|
-
|
|
41
|
+
ARGS="${ARGUMENTS//--json/}" # strip flags first — "--json" alone must not become the path
|
|
42
|
+
REPO_PATH="${ARGS#--target }"
|
|
43
|
+
REPO_PATH="$(printf '%s' "$REPO_PATH" | xargs)"
|
|
42
44
|
REPO_PATH="${REPO_PATH:-$(pwd)}"
|
|
43
45
|
git -C "$REPO_PATH" rev-parse --is-inside-work-tree >/dev/null 2>&1 \
|
|
44
46
|
|| { echo "Not a git repo — public-surface-audit scans git-tracked files only. Aborting."; exit 1; }
|
|
@@ -50,12 +52,18 @@ git -C "$REPO_PATH" ls-files | wc -l | xargs echo "Tracked files:"
|
|
|
50
52
|
|
|
51
53
|
## Step 1. Pattern List (configurable)
|
|
52
54
|
|
|
53
|
-
The
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
55
|
+
The pattern source is **two layers, matching the mechanical scan layer** (`scripts/psa_scan_lib.sh`,
|
|
56
|
+
shared with the pre-commit/pre-push/pre-publish hooks — this skill must render the same verdict the
|
|
57
|
+
hooks would, or the same repo state gets two answers):
|
|
58
|
+
|
|
59
|
+
1. **`.claude/rules/.public-surface-patterns.defaults`** — committed, universal placeholder-grade
|
|
60
|
+
patterns (safe to publish; carries no operator literal).
|
|
61
|
+
2. **`.claude/rules/.public-surface-patterns`** — **gitignored, operator literals** (or `PSA_PATTERNS`
|
|
62
|
+
env override). Your real username and employer name live *only* here — hardcoding them in a tracked
|
|
63
|
+
file would leak exactly what this skill hunts. One `severity<TAB>regex` per line.
|
|
64
|
+
|
|
65
|
+
The scan reads the **union of both layers**. This SKILL.md carries only placeholders; the skill
|
|
66
|
+
dogfoods its own rule.
|
|
59
67
|
|
|
60
68
|
| # | Token class | Severity | Placeholder (real value goes in the gitignored source) | Why private |
|
|
61
69
|
|:-:|---|:-:|---|---|
|
|
@@ -71,12 +79,16 @@ its own rule.
|
|
|
71
79
|
- **LOW** — companion-store / private-wiring name. Methodology should be model-agnostic; naming a private
|
|
72
80
|
store is drift, not a confidentiality breach.
|
|
73
81
|
|
|
74
|
-
> **Setup**: put your real values in the gitignored
|
|
75
|
-
>
|
|
76
|
-
>
|
|
77
|
-
>
|
|
78
|
-
>
|
|
79
|
-
>
|
|
82
|
+
> **Setup**: put your real values in the gitignored override (one `severity<TAB>regex` per line).
|
|
83
|
+
> Degrade states, by layer — never collapse them:
|
|
84
|
+
> - **Neither layer present** → **NOT CONFIGURED** — *not* CLEAN. A missing pattern source must never
|
|
85
|
+
> masquerade as a clean bill of health ("nothing scanned" misread as "nothing leaked").
|
|
86
|
+
> - **Defaults only** (override absent) → scan runs, but the verdict carries the label
|
|
87
|
+
> `operator literals NOT CONFIGURED — universal patterns only; identity/company classes UNSCANNED`.
|
|
88
|
+
> This mirrors what the hooks scan in the same state, instead of reporting NOT CONFIGURED while the
|
|
89
|
+
> hook path quietly scans defaults (two verdicts for one repo state — the divergence this line closes).
|
|
90
|
+
> - To declare "I genuinely have no private tokens", create the override **empty** — an empty file is
|
|
91
|
+
> an explicit declaration, an absent file is unconfigured.
|
|
80
92
|
|
|
81
93
|
---
|
|
82
94
|
|
|
@@ -86,6 +98,11 @@ Some tracked files legitimately reference otherwise-private tokens — the scan
|
|
|
86
98
|
leaks. Maintain an allowlist of `file path :: token` pairs. A match is suppressed only when **both**
|
|
87
99
|
the file and the token are on the allowlist row.
|
|
88
100
|
|
|
101
|
+
**Where the filled allowlist lives**: a populated allowlist row *names private tokens*, so it belongs
|
|
102
|
+
in a **gitignored** source beside the pattern override (`.claude/rules/.public-surface-allowlist`, or a
|
|
103
|
+
section of `CLAUDE.local.md`) — never in a tracked file. The table below shows generic shapes only; a
|
|
104
|
+
tracked copy of your real allowlist would itself be the leak this skill scans for.
|
|
105
|
+
|
|
89
106
|
| Tracked file | Allowed tokens | Reason |
|
|
90
107
|
|---|---|---|
|
|
91
108
|
| `.gitignore` | companion-store name, sync-script name | Must name what it ignores |
|
|
@@ -172,11 +189,17 @@ class of leak). Diagnostic-only: this step never writes — it reports, the oper
|
|
|
172
189
|
Report per-hit `file:line → matched token [class]` grouped by severity, then the overall verdict.
|
|
173
190
|
**Verdict thresholds** (behavioral — these define the gate):
|
|
174
191
|
|
|
175
|
-
- ⚪ **NOT CONFIGURED** — pattern
|
|
176
|
-
- 🟢 **CLEAN** — pattern source present (incl. empty), 0 HIGH + 0 MED + 0 LOW (after allowlist)
|
|
192
|
+
- ⚪ **NOT CONFIGURED** — no pattern layer present (nothing scanned — NOT a clean result; set up first)
|
|
193
|
+
- 🟢 **CLEAN** — pattern source present (incl. empty override), 0 HIGH + 0 MED + 0 LOW (after allowlist)
|
|
177
194
|
- 🟡 **REVIEW** — 0 HIGH + 0 MED, LOW-only (drift, not a breach)
|
|
178
195
|
- 🔴 **LEAK** — 1+ HIGH or 1+ MED (block publish / fix before commit)
|
|
179
196
|
|
|
197
|
+
**Every verdict line carries a `coverage:` qualifier** — `full` (both layers loaded) or
|
|
198
|
+
`defaults-only` (override absent: operator identity/company classes UNSCANNED). The qualifier exists
|
|
199
|
+
because the enum alone cannot preserve the defaults-only state: `CLEAN (coverage: defaults-only)` on
|
|
200
|
+
an operator-configured checkout is missing evidence, not a pass — publish-gating callers there must
|
|
201
|
+
require `coverage: full`.
|
|
202
|
+
|
|
180
203
|
> **Detail**: See `SKILL_detail.md §Report-Template` — the full human-report ASCII layout (severity
|
|
181
204
|
> buckets, allowlist-suppressed line) — read when formatting the Step 4 report.
|
|
182
205
|
|
|
@@ -207,7 +230,9 @@ enforceable (FH's "enforcement is a hook, not a prompt" principle). Imported fro
|
|
|
207
230
|
authoritative — never gate on `counts` alone**: a counts-only check (`HIGH==0 && MED==0`) misreads
|
|
208
231
|
`NOT_CONFIGURED` (which also has zero counts) as a pass. A caller blocks when `verdict` is `LEAK` **or**
|
|
209
232
|
`NOT_CONFIGURED` — an unconfigured scan is not a pass (the same silent-failure guard as the human path:
|
|
210
|
-
absence ≠ CLEAN).
|
|
233
|
+
absence ≠ CLEAN). The JSON carries the same `coverage` field as Step 4; on an operator-configured
|
|
234
|
+
checkout a publish-gating caller additionally blocks on `coverage: "defaults-only"` (identity/company
|
|
235
|
+
classes unscanned there — zero counts prove nothing about the classes that never ran).
|
|
211
236
|
|
|
212
237
|
---
|
|
213
238
|
|
|
@@ -236,15 +261,24 @@ Usable standalone — no hub clone required.
|
|
|
236
261
|
## Done When
|
|
237
262
|
|
|
238
263
|
```
|
|
239
|
-
Step 1 pattern
|
|
240
|
-
|
|
241
|
-
+ Step
|
|
242
|
-
+ Step
|
|
243
|
-
|
|
264
|
+
Step 1 pattern source resolved (both layers checked; absent/partial
|
|
265
|
+
states rendered as their labeled degrade state, never as CLEAN) — mandatory-pass
|
|
266
|
+
+ Step 2 allowlist applied (from a gitignored source) — mandatory-pass
|
|
267
|
+
+ Step 3 scan run for every pattern over git ls-files (tracked only —
|
|
268
|
+
gitignored excluded), malformed pattern rows counted and reported,
|
|
269
|
+
never silently dropped — measured (hit + malformed counts)
|
|
270
|
+
+ Step 3b/3c run (FP-hygiene on matched tokens; ignore-verification
|
|
271
|
+
for expected-private paths) or explicitly skipped with reason — mandatory-pass
|
|
272
|
+
+ Step 4 report output: per-hit file:line + token + severity, plus
|
|
273
|
+
overall verdict — mandatory-pass
|
|
274
|
+
+ "public-surface-audit Complete" declaration output — mandatory-pass
|
|
244
275
|
```
|
|
245
276
|
|
|
246
277
|
Verdict: **CLEAN** (0 tokens after allowlist) | **REVIEW** (LOW-only — drift, prescriptions noted) |
|
|
247
|
-
**LEAK** (1+ HIGH or 1+ MED — block publish, prescriptions attached)
|
|
278
|
+
**LEAK** (1+ HIGH or 1+ MED — block publish, prescriptions attached) | **NOT_CONFIGURED** (no pattern
|
|
279
|
+
source — nothing scanned, not a pass; same 4-value enum as Step 5 `--json`. An earlier version of this
|
|
280
|
+
enum had 3 values here, so a session following only Done When would force absent-config into CLEAN/
|
|
281
|
+
REVIEW/LEAK — re-opening the exact silent-pass the body closes).
|
|
248
282
|
|
|
249
283
|
---
|
|
250
284
|
|
|
@@ -11,19 +11,58 @@ cd "$REPO_PATH" || exit 1
|
|
|
11
11
|
# Build the tracked-file list once.
|
|
12
12
|
git ls-files > /tmp/_psa_tracked.txt
|
|
13
13
|
|
|
14
|
-
#
|
|
15
|
-
|
|
16
|
-
#
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
14
|
+
# ── Single-source preference: when the shared library exists, use it and write NO second loop ──
|
|
15
|
+
# scripts/psa_scan_lib.sh owns loading + row validation + exemptions for the hook layer; a hand-rolled
|
|
16
|
+
# copy here is a second normalizer with its own leniency (the divergence class this rewrite removed).
|
|
17
|
+
if [ -r "scripts/psa_scan_lib.sh" ]; then
|
|
18
|
+
. scripts/psa_scan_lib.sh
|
|
19
|
+
psa_load ".claude/rules/.public-surface-patterns.defaults" \
|
|
20
|
+
"${PSA_PATTERNS:-.claude/rules/.public-surface-patterns}"
|
|
21
|
+
{ [ "$PSA_DEFAULTS_OK" -eq 1 ] || [ "$PSA_OVERRIDE_PRESENT" -eq 1 ]; } \
|
|
22
|
+
|| { echo "⚪ NOT CONFIGURED: neither pattern layer present. Not scanning."; exit 2; }
|
|
23
|
+
[ "$PSA_BAD_ROWS" -gt 0 ] \
|
|
24
|
+
&& { echo "HARNESS_ERROR: $PSA_BAD_ROWS unusable pattern row(s) — verdict cannot be CLEAN"; exit 10; }
|
|
25
|
+
# Feed every tracked file as path<TAB>line, the stream psa_scan_tagged consumes. Sourcing the lib
|
|
26
|
+
# without these calls is a no-op scan — measured on this repo (PSA_STREAM stayed unset), so the
|
|
27
|
+
# calls are spelled out here rather than pointed at.
|
|
28
|
+
while IFS= read -r f; do
|
|
29
|
+
awk -v p="$f" '{printf "%s\t%s\n", p, $0}' "$f" 2>/dev/null
|
|
30
|
+
done < /tmp/_psa_tracked.txt | psa_scan_tagged
|
|
31
|
+
[ "$PSA_OVERRIDE_PRESENT" -eq 1 ] \
|
|
32
|
+
|| echo "coverage: defaults-only — operator literals NOT CONFIGURED (identity/company classes UNSCANNED)"
|
|
33
|
+
else
|
|
34
|
+
# ── Standalone fallback (no hub scripts in this repo) — validated loop, malformed rows COUNTED ──
|
|
35
|
+
PSA_DEFAULTS=".claude/rules/.public-surface-patterns.defaults"
|
|
36
|
+
PATTERN_SRC="${PSA_PATTERNS:-.claude/rules/.public-surface-patterns}"
|
|
37
|
+
SRC_LIST=""
|
|
38
|
+
[ -e "$PSA_DEFAULTS" ] && SRC_LIST="$PSA_DEFAULTS"
|
|
39
|
+
[ -e "$PATTERN_SRC" ] && SRC_LIST="$SRC_LIST $PATTERN_SRC"
|
|
40
|
+
[ -n "$SRC_LIST" ] || { echo "⚪ NOT CONFIGURED: no pattern source (neither defaults nor override). Not scanning."; exit 2; }
|
|
41
|
+
|
|
42
|
+
MALFORMED=0
|
|
43
|
+
cat $SRC_LIST > /tmp/_psa_rows.txt # no pipe into the loop — a piped while runs in a subshell and loses MALFORMED
|
|
44
|
+
# `|| [ -n "$severity" ]` keeps a final row that lacks a trailing newline — `read` alone drops it silently.
|
|
45
|
+
while IFS=$'\t' read -r severity regex || [ -n "$severity" ]; do
|
|
46
|
+
case "$severity" in ''|'#'*) continue ;; esac # blank / comment rows
|
|
47
|
+
if [ -z "$regex" ]; then # no tab separator → malformed, VISIBLE
|
|
48
|
+
MALFORMED=$((MALFORMED+1)); echo "MALFORMED ROW (no <TAB>): $severity" >&2; continue
|
|
49
|
+
fi
|
|
50
|
+
printf 'x\n' | grep -qE "$regex" 2>/dev/null
|
|
51
|
+
rc=$? # plain rc capture — `if !` would negate $?
|
|
52
|
+
if [ "$rc" -ge 2 ]; then # grep rc≥2 = invalid regex, not "no match"
|
|
53
|
+
MALFORMED=$((MALFORMED+1)); echo "MALFORMED ROW (bad regex): $regex" >&2; continue
|
|
54
|
+
fi
|
|
55
|
+
grep -nIE "$regex" $(cat /tmp/_psa_tracked.txt) | sed "s/^/[$severity] /"
|
|
56
|
+
done < /tmp/_psa_rows.txt
|
|
57
|
+
# Malformed rows poison the verdict: part of the pattern file never scanned → CLEAN is unprovable.
|
|
58
|
+
[ "$MALFORMED" -gt 0 ] && { echo "HARNESS_ERROR: $MALFORMED malformed pattern row(s) — verdict cannot be CLEAN"; exit 10; }
|
|
59
|
+
fi
|
|
25
60
|
```
|
|
26
61
|
|
|
62
|
+
**Why the fallback validates instead of skipping**: the previous loop dropped a malformed row with a
|
|
63
|
+
bare `continue` and discarded grep's stderr — a broken pattern file scanned "clean" by silently not
|
|
64
|
+
scanning. `not found ≠ 0`: a row that never ran is not a row with zero hits.
|
|
65
|
+
|
|
27
66
|
For each pattern, run `grep -nIE "<regex>" $(git ls-files)`:
|
|
28
67
|
- `-n` → line numbers (required for `file:line` output)
|
|
29
68
|
- `-I` → skip binary files
|
|
@@ -66,7 +66,27 @@ find plugins/ -name "SKILL.md" | sort
|
|
|
66
66
|
find plugins/ -path "*[name]/SKILL.md"
|
|
67
67
|
```
|
|
68
68
|
|
|
69
|
-
If 0 SKILL.md files are in scope,
|
|
69
|
+
If 0 SKILL.md files are in scope, **distinguish the two causes before stopping** — a failed scope
|
|
70
|
+
command and a genuinely empty scope both render as zero lines, and only one of them is skippable:
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
git rev-parse --git-dir >/dev/null 2>&1 \
|
|
74
|
+
|| { echo "HARNESS_ERROR: git unavailable — scope could NOT be measured (not an empty scope)"; exit 10; }
|
|
75
|
+
|
|
76
|
+
# Repo-liveness alone is NOT enough: a broken ref (e.g. missing origin/main in --pr mode) fails the
|
|
77
|
+
# scope command with rc=1 and empty output while rev-parse stays green. Capture the scope command's
|
|
78
|
+
# OWN exit status — before the grep, which legitimately returns 1 on no-match:
|
|
79
|
+
scope_out=$(git diff --name-only origin/main...HEAD 2>&1); scope_rc=$?
|
|
80
|
+
[ "$scope_rc" -ne 0 ] \
|
|
81
|
+
&& { echo "HARNESS_ERROR: scope command failed (rc=$scope_rc): $scope_out"; exit 10; }
|
|
82
|
+
printf '%s\n' "$scope_out" | grep "SKILL\.md"
|
|
83
|
+
# (same rc-capture shape applies to the default and --all scope commands)
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
With both controls green and genuinely 0 files in scope, output:
|
|
87
|
+
"No SKILL.md files in scope — audit skipped (control: git rev-parse OK)." and stop.
|
|
88
|
+
A HARNESS_ERROR is a distinct non-pass — it must never be reported as "audit skipped", because this
|
|
89
|
+
skill is pipeline-conductor's Step 0.5 pre-flight and a tooling-down skip would render as a pass there.
|
|
70
90
|
|
|
71
91
|
---
|
|
72
92
|
|
|
@@ -221,11 +241,19 @@ Next actions:
|
|
|
221
241
|
## Done When
|
|
222
242
|
|
|
223
243
|
```
|
|
224
|
-
Step 0 scope determined;
|
|
225
|
-
|
|
226
|
-
+ Step
|
|
227
|
-
|
|
228
|
-
+ Step
|
|
244
|
+
Step 0 scope determined; a failed scope command reported as
|
|
245
|
+
HARNESS_ERROR, never as empty scope — mandatory-pass
|
|
246
|
+
+ Step 1 two-pass extraction complete (Pass A caller-wait +
|
|
247
|
+
Pass B callee-output evaluated separately) — mandatory-pass
|
|
248
|
+
+ Step 2 every (caller → callee) pair classified CLOSED or
|
|
249
|
+
OPEN with two-dimension severity — judged (adversarial pairing:
|
|
250
|
+
before ANY chain is reported OPEN, the ground-truth CLOSED
|
|
251
|
+
pair from §Operating Notes must classify CLOSED in the same
|
|
252
|
+
run — an instrument that cannot separate the known pair
|
|
253
|
+
reports nothing)
|
|
254
|
+
+ Step 3 fix prescriptions output for each OPEN chain
|
|
255
|
+
(core FH skills: proposal-only) — mandatory-pass
|
|
256
|
+
+ Step 4 summary report with CLOSED/OPEN counts output — measured (counts)
|
|
229
257
|
```
|
|
230
258
|
|
|
231
259
|
> Fix prescriptions are text output only (Write not in allowed-tools). Prescription application is manual and out of scope for this skill. Verification of applied fixes requires re-running `/return-path-gate --skill [name]`.
|
|
@@ -254,4 +282,5 @@ Verdict: PASS (0 HIGH severity OPEN chains) | CONDITIONAL_PASS (MEDIUM/LOW sever
|
|
|
254
282
|
- **CONDITIONAL_PASS gate is the highest-risk gap**: A chain where CONDITIONAL_PASS conditions are listed by the callee but the caller has no enforcement path is OPEN even if other verdict paths fold correctly.
|
|
255
283
|
- **Scope default is narrow by design**: Default captures modified + newly added files (not just staged changes). Use `--pr` for PR-relative mode in worktrees, `--all` for periodic sweeps.
|
|
256
284
|
- **Core FH skill prescriptions are proposals only**: Edits to harvest-loop, steel-quench, sim-conductor, and other core skills require deliberate review — output proposal notes, not standard prescriptions.
|
|
257
|
-
- **Reference pattern**: `knowledge/shared/harness-core/return_path_gate.md` defines the canonical closed-loop structure and verified instances (apex-review → sim-conductor, agent-composer ↔ deliberation). These are the ground-truth CLOSED examples for calibrating classification.
|
|
285
|
+
- **Reference pattern**: `knowledge/shared/harness-core/return_path_gate.md` defines the canonical closed-loop structure and verified instances (apex-review → sim-conductor, agent-composer ↔ deliberation). These are the ground-truth CLOSED examples for calibrating classification — and that calibration is a Done When condition (Step 2's adversarial pairing), not an optional note.
|
|
286
|
+
- **The Pass A signal list is a closed English keyword list — know its degrade direction**: synonyms ("blocks until", "halts on") and non-English documentation will read as false OPEN. That direction is by design (over-report, never silent-close), but treat an OPEN verdict on unconventional wording as a candidate for human reading before prescribing a fix.
|