session-orchestrator 4.2.0 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/session-start/SKILL.md +1 -1
- package/.agents/skills/ux-grill/SKILL.md +22 -0
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +3 -2
- package/.codex-plugin/plugin.json +1 -1
- package/.codex-plugin/skills/session-start/SKILL.md +1 -1
- package/.codex-plugin/skills/ux-grill/SKILL.md +21 -0
- package/.codex-plugin/skills/ux-grill/agents/openai.yaml +5 -0
- package/.cursor/commands/ux-grill.md +14 -0
- package/.cursor/skills/session-start/SKILL.md +1 -1
- package/.cursor/skills/ux-grill/SKILL.md +13 -0
- package/.cursor-plugin/plugin.json +1 -1
- package/AGENTS.md +2 -1
- package/CHANGELOG.md +94 -1
- package/README.md +98 -86
- package/agents/dialectic-deriver.md +11 -0
- package/commands/ux-grill.md +51 -0
- package/docs/USER-GUIDE.md +2 -2
- package/docs/codex-setup.md +8 -0
- package/docs/components.md +7 -7
- package/docs/events-schema.md +9 -5
- package/docs/github-mirror-protection.md +61 -20
- package/docs/migration-v5.md +62 -0
- package/docs/scope-collision-guard.md +14 -0
- package/hooks/_lib/hook-import-set.json +44 -2
- package/hooks/_lib/lock-bootstrap.mjs +84 -1
- package/hooks/_lib/vcs-create-matcher.mjs +190 -3
- package/hooks/enforce-scope.mjs +201 -0
- package/hooks/hooks-codex.json +1 -1
- package/hooks/hooks-cursor.json +5 -0
- package/hooks/hooks.json +7 -2
- package/hooks/on-session-start.mjs +171 -49
- package/hooks/post-bash-issue-budget-refund.mjs +375 -0
- package/hooks/pre-auq-clarity.mjs +70 -18
- package/hooks/pre-bash-issue-budget.mjs +51 -4
- package/package.json +2 -1
- package/pi/prompts/ux-grill.md +12 -0
- package/scripts/ci/assert-vitest-green.mjs +4 -2
- package/scripts/dialectic-deriver.mjs +32 -8
- package/scripts/emit-session.mjs +72 -1
- package/scripts/lib/agent-status.mjs +441 -9
- package/scripts/lib/auq/schema.mjs +10 -3
- package/scripts/lib/ci-status-banner.mjs +29 -6
- package/scripts/lib/claude-md-budget-lint.mjs +52 -2
- package/scripts/lib/config.mjs +12 -1
- package/scripts/lib/eval/engine.mjs +7 -1
- package/scripts/lib/file-lock.mjs +114 -13
- package/scripts/lib/git-porcelain.mjs +113 -0
- package/scripts/lib/instruction-budget-guard.mjs +415 -47
- package/scripts/lib/io.mjs +29 -4
- package/scripts/lib/issue-budget.mjs +336 -6
- package/scripts/lib/learnings/sizing-subject.mjs +44 -0
- package/scripts/lib/locks/staging-fence-lock.mjs +19 -38
- package/scripts/lib/locks/state-md-lock.mjs +19 -41
- package/scripts/lib/maintenance-due-banner.mjs +11 -1
- package/scripts/lib/peer-cards/merger.mjs +143 -0
- package/scripts/lib/pre-dispatch-check.mjs +20 -14
- package/scripts/lib/project-hygiene.mjs +81 -30
- package/scripts/lib/quality-gate.mjs +14 -65
- package/scripts/lib/reconcile/engine.mjs +19 -1
- package/scripts/lib/reconcile/writer.mjs +278 -11
- package/scripts/lib/scope-echo.mjs +346 -0
- package/scripts/lib/session-lock.mjs +62 -2
- package/scripts/lib/session-record-repair.mjs +91 -0
- package/scripts/lib/session-schema/filters.mjs +26 -1
- package/scripts/lib/session-start-probes.mjs +419 -53
- package/scripts/lib/test-runner/artifact-paths.mjs +30 -5
- package/scripts/lib/test-runner/issue-reconcile.mjs +45 -8
- package/scripts/lib/tmux-layout/layouts.mjs +62 -4
- package/scripts/lib/ux-grill/collect.mjs +1163 -0
- package/scripts/lib/ux-grill/compare.mjs +285 -0
- package/scripts/lib/ux-grill/manifest.mjs +618 -0
- package/scripts/lib/ux-grill/measures.mjs +431 -0
- package/scripts/lib/ux-grill/paths.mjs +224 -0
- package/scripts/lib/ux-grill/pencil-coverage.mjs +284 -0
- package/scripts/lib/ux-grill/reconcile.mjs +344 -0
- package/scripts/lib/ux-grill/run-record.mjs +316 -0
- package/scripts/lib/ux-grill/schema.mjs +321 -0
- package/scripts/lib/validate/check-untracked-test-deps.mjs +33 -19
- package/scripts/lib/validate/check-unwired-features.mjs +48 -20
- package/scripts/lib/vault-status/board-lock.mjs +18 -0
- package/scripts/lib/vault-status/board-writer.mjs +8 -0
- package/scripts/mcp-server.sh +16 -1
- package/scripts/release.mjs +7 -2
- package/skills/bootstrap/SKILL.md +12 -209
- package/skills/bootstrap/references/bootstrap-ecosystem-health-flow.md +48 -0
- package/skills/bootstrap/references/bootstrap-refresh-lock-flow.md +37 -0
- package/skills/bootstrap/references/bootstrap-retroactive-flow.md +108 -0
- package/skills/bootstrap/references/bootstrap-rules-fetch-bridge.md +64 -0
- package/skills/claude-md-drift-check/SKILL.md +9 -2
- package/skills/claude-md-drift-check/checker.mjs +213 -21
- package/skills/discovery/SKILL.md +6 -173
- package/skills/discovery/probes/vault-staleness.mjs +35 -5
- package/skills/discovery/probes-docs.md +8 -4
- package/skills/discovery/probes-supply-chain.md +4 -2
- package/skills/discovery/probes-ui.md +7 -3
- package/skills/discovery/probes-vault.md +12 -4
- package/skills/discovery/references/discovery-interactive-triage.md +139 -0
- package/skills/discovery/references/discovery-triage-state.md +54 -0
- package/skills/eval/rubric-v1.md +13 -0
- package/skills/evolve/SKILL.md +2 -458
- package/skills/evolve/references/evolve-analyze-mode.md +360 -0
- package/skills/evolve/references/evolve-dialectic-mode.md +139 -0
- package/skills/plan/mode-retro.md +4 -3
- package/skills/reconcile/SKILL.md +10 -0
- package/skills/session-end/drift-operations.md +20 -5
- package/skills/session-end/metrics-collection.md +1 -0
- package/skills/session-end/phase-3-6-tail.md +4 -2
- package/skills/session-end/references/phase-2-quality-gate.md +3 -3
- package/skills/session-end/references/phase-5-issue-cleanup.md +6 -1
- package/skills/session-end/session-metrics-write.md +2 -0
- package/skills/session-plan/SKILL.md +2 -144
- package/skills/session-plan/references/session-plan-task-classification.md +152 -0
- package/skills/session-start/SKILL.md +24 -6
- package/skills/session-start/references/operations-contract.md +114 -0
- package/skills/session-start/references/phase-4-ssot-environment-check.md +22 -20
- package/skills/session-start/soul.md +2 -2
- package/skills/test-runner/SKILL.md +1 -1
- package/skills/tmux-layout/SKILL.md +3 -1
- package/skills/ux-grill/SKILL.md +211 -0
- package/skills/ux-grill/rubric-v2.md +201 -0
- package/skills/ux-grill/soul.md +76 -0
- package/skills/wave-executor/SKILL.md +3 -128
- package/skills/wave-executor/references/wave-executor-quality-gate.md +61 -0
- package/skills/wave-executor/references/wave-executor-state-init.md +86 -0
- package/skills/wave-executor/references/wave-loop-dispatch.md +8 -0
- package/skills/wave-executor/references/wave-loop-review.md +18 -5
- package/templates/_shared/ux-manifest.template.md +149 -0
- package/templates/_shared/journey-manifest.md +0 -114
|
@@ -294,137 +294,11 @@ Present both as structured data in your final output. Do not proceed to Phase 5.
|
|
|
294
294
|
|
|
295
295
|
## Phase 5: Interactive Triage (Standalone Mode Only)
|
|
296
296
|
|
|
297
|
-
|
|
297
|
+
Loads persistent triage state, partitions findings into auto-deferred/high-confidence buckets, and walks the user through Critical/High findings individually and Medium/Low findings in a batch before Phase 6 issue creation.
|
|
298
298
|
|
|
299
|
-
|
|
299
|
+
See [references/discovery-interactive-triage.md](references/discovery-interactive-triage.md).
|
|
300
300
|
|
|
301
|
-
|
|
302
|
-
2. Call `filterFindings({ findings: verifiedFindings, stateMap })` to partition findings into three buckets:
|
|
303
|
-
- `toShow` — state is `open`, `reopened`, or **no prior state entry** (new findings — present for user triage)
|
|
304
|
-
- `suppressed` — state is `dismissed` or `accepted-as-known` (skip silently)
|
|
305
|
-
- `tracked` — state is `promoted-to-#NNN` (issue already filed; show as informational)
|
|
306
|
-
|
|
307
|
-
3. Emit a one-line state banner before the summary table:
|
|
308
|
-
```
|
|
309
|
-
Triage state: [N suppressed] suppressed (dismissed/accepted-as-known), [N tracked] tracked in existing issues. Presenting [N toShow] findings.
|
|
310
|
-
```
|
|
311
|
-
Omit the banner entirely if all three counts are zero (first run).
|
|
312
|
-
|
|
313
|
-
4. Render `tracked` findings as informational lines in the summary — NOT as interactive triage items:
|
|
314
|
-
```
|
|
315
|
-
[INFO] Finding "<title>" (<file_path>) is tracked in #<issue_id> — not re-triaged.
|
|
316
|
-
```
|
|
317
|
-
|
|
318
|
-
5. Continue Phase 5 triage using only `toShow` findings. The `suppressed` bucket requires no user interaction.
|
|
319
|
-
|
|
320
|
-
6. After the user completes triage (Steps 1-4 below), append state changes to `.orchestrator/metrics/discovery-triage.jsonl` via `appendTriageEntry()` from `triage-state.mjs`:
|
|
321
|
-
- User selects "Create issue" → append `{ fingerprint, state: 'promoted-to-#<issue_id>', issue_id: <N>, timestamp, session_id }`
|
|
322
|
-
- User selects "Dismiss -- intentional" or "Dismiss -- false positive" → append `{ fingerprint, state: 'dismissed', user_decision: '<reason>', timestamp, session_id }`
|
|
323
|
-
- User selects "Accept all" for batch → append one `{ fingerprint, state: 'open', ... }` entry per finding (so they re-appear next run if not yet promoted)
|
|
324
|
-
|
|
325
|
-
### 5.1 Auto-Defer Low-Confidence Findings
|
|
326
|
-
|
|
327
|
-
Before presenting findings for triage, separate by confidence threshold:
|
|
328
|
-
|
|
329
|
-
1. Findings with confidence >= threshold → present for interactive triage (below)
|
|
330
|
-
2. Findings with confidence < threshold → auto-defer with summary:
|
|
331
|
-
"Auto-deferred [N] low-confidence findings (score < [threshold]). Review with `/discovery --include-deferred`."
|
|
332
|
-
3. List auto-deferred findings in a collapsed section (not interactive — informational only)
|
|
333
|
-
|
|
334
|
-
### 5.1 Present High-Confidence Findings
|
|
335
|
-
|
|
336
|
-
Present findings using AskUserQuestion -- NEVER plain text options. On Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists.
|
|
337
|
-
|
|
338
|
-
Include confidence scores in the presentation:
|
|
339
|
-
```
|
|
340
|
-
[CRITICAL] (confidence: 85) hardcoded-values: API key found in src/config.ts:42
|
|
341
|
-
[HIGH] (confidence: 72) security-basics: eval() usage in src/utils/parser.ts:18
|
|
342
|
-
[MEDIUM] (confidence: 61) orphaned-annotations: TODO without issue in src/lib/auth.ts:55
|
|
343
|
-
```
|
|
344
|
-
|
|
345
|
-
### Step 1: Summary
|
|
346
|
-
|
|
347
|
-
Present a findings overview table:
|
|
348
|
-
|
|
349
|
-
```
|
|
350
|
-
## Discovery Results
|
|
351
|
-
|
|
352
|
-
Probes run: [N] | Findings verified: [N] | False positives discarded: [N]
|
|
353
|
-
|
|
354
|
-
| Category | Critical | High | Medium | Low | Total |
|
|
355
|
-
|----------|----------|------|--------|-----|-------|
|
|
356
|
-
| Code | ... | ... | ... | ... | ... |
|
|
357
|
-
| Infra | ... | ... | ... | ... | ... |
|
|
358
|
-
| UI | ... | ... | ... | ... | ... |
|
|
359
|
-
| Arch | ... | ... | ... | ... | ... |
|
|
360
|
-
| Session | ... | ... | ... | ... | ... |
|
|
361
|
-
| Audit | ... | ... | ... | ... | ... |
|
|
362
|
-
| Vault | ... | ... | ... | ... | ... |
|
|
363
|
-
| Feature | ... | ... | ... | ... | ... |
|
|
364
|
-
```
|
|
365
|
-
|
|
366
|
-
### Step 2: Critical + High Findings -- Review Individually
|
|
367
|
-
|
|
368
|
-
For each Critical or High finding, use AskUserQuestion (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
369
|
-
|
|
370
|
-
```
|
|
371
|
-
AskUserQuestion({
|
|
372
|
-
questions: [{
|
|
373
|
-
question: "<severity> finding in <file_path> — what should happen with it?",
|
|
374
|
-
header: "Finding",
|
|
375
|
-
options: [
|
|
376
|
-
{ label: "Create issue (<severity>)", description: "Files it as priority::<severity>, so it is tracked outside this session. The code below is copied into the issue body.",
|
|
377
|
-
preview: "<finding title>\n\n<file_path>:<line_number>\n```\n<matched_text with +/-3 lines context>\n```\n\n<description>\n\nRecommended fix: <recommended_fix>" },
|
|
378
|
-
{ label: "Adjust priority", description: "Same issue, a priority you pick — this question then comes back with the new label." },
|
|
379
|
-
{ label: "Dismiss -- intentional", description: "The code is deliberate. Nothing is filed, and the finding stays only in this run's report." },
|
|
380
|
-
{ label: "Dismiss -- false positive", description: "The probe misread the code. Nothing is filed; worth reporting if the same probe misfires again." }
|
|
381
|
-
],
|
|
382
|
-
multiSelect: false
|
|
383
|
-
}]
|
|
384
|
-
})
|
|
385
|
-
```
|
|
386
|
-
|
|
387
|
-
If user selects "Adjust priority", ask which priority with another AskUserQuestion. On Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists.
|
|
388
|
-
|
|
389
|
-
### Step 3: Medium + Low Findings -- Review Batched
|
|
390
|
-
|
|
391
|
-
Group remaining findings by category. For each category with medium/low findings (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
392
|
-
|
|
393
|
-
```
|
|
394
|
-
AskUserQuestion({
|
|
395
|
-
questions: [{
|
|
396
|
-
question: "Create issues for all [N] medium/low findings in [category]?",
|
|
397
|
-
header: "Findings",
|
|
398
|
-
options: [
|
|
399
|
-
{ label: "Accept all (Recommended)", description: "Medium and low findings are cheap to file and cheap to close. Cost: [N] issues, roughly one second apart.",
|
|
400
|
-
preview: "1. [title] -- [file_path]:[line] ([severity])\n2. [title] -- [file_path]:[line] ([severity])\n..." },
|
|
401
|
-
{ label: "Review individually", description: "One question per finding, same options as the critical ones. Cost: [N] more prompts." },
|
|
402
|
-
{ label: "Dismiss all", description: "Nothing is filed for this category. The findings stay in this run's report only." }
|
|
403
|
-
],
|
|
404
|
-
multiSelect: false
|
|
405
|
-
}]
|
|
406
|
-
})
|
|
407
|
-
```
|
|
408
|
-
|
|
409
|
-
If "Review individually" selected, walk through each like Step 2.
|
|
410
|
-
|
|
411
|
-
### Step 4: Batch Confirmation
|
|
412
|
-
|
|
413
|
-
Before creating any issues (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
414
|
-
|
|
415
|
-
```
|
|
416
|
-
AskUserQuestion({
|
|
417
|
-
questions: [{
|
|
418
|
-
question: "Ready to create [N] issues?\n\n- [X] critical\n- [Y] high\n- [Z] medium\n- [W] low",
|
|
419
|
-
header: "Confirm",
|
|
420
|
-
options: [
|
|
421
|
-
{ label: "Create all [N] issues", description: "Proceed with issue creation" },
|
|
422
|
-
{ label: "Review list first", description: "Show full list before creating" },
|
|
423
|
-
{ label: "Cancel", description: "Do not create any issues" }
|
|
424
|
-
]
|
|
425
|
-
}]
|
|
426
|
-
})
|
|
427
|
-
```
|
|
301
|
+
**Read WHEN:** running in standalone mode (not embedded mode — see Phase 4.6 Embedded Mode Exit above), after Phase 4 scoring completes.
|
|
428
302
|
|
|
429
303
|
## Phase 6: Issue Creation & Report
|
|
430
304
|
|
|
@@ -515,52 +389,11 @@ After Phase 6 (Issue Creation) completes, prepare discovery statistics for sessi
|
|
|
515
389
|
|
|
516
390
|
## Discovery Triage State (#419)
|
|
517
391
|
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
### State File
|
|
521
|
-
|
|
522
|
-
**Location:** `.orchestrator/metrics/discovery-triage.jsonl` (gitignored via `.orchestrator/metrics/*.jsonl` pattern — machine-local, never committed)
|
|
523
|
-
|
|
524
|
-
**Format:** One JSON object per line:
|
|
525
|
-
```json
|
|
526
|
-
{"fingerprint":"aabb1122ccdd3344","state":"dismissed","user_decision":"intentional — debug log","timestamp":"2026-05-17T10:00:00.000Z","session_id":"deep-2"}
|
|
527
|
-
{"fingerprint":"eeff5566aabb7788","state":"promoted-to-#119","issue_id":119,"timestamp":"2026-05-17T10:01:00.000Z","session_id":"deep-2"}
|
|
528
|
-
```
|
|
529
|
-
|
|
530
|
-
### Fingerprint
|
|
531
|
-
|
|
532
|
-
`computeFingerprint({probe, file, severity, ruleId})` → 16-char hex (sha256 prefix).
|
|
533
|
-
|
|
534
|
-
`line_number` is **intentionally excluded** — it drifts on refactoring without the underlying issue changing. A finding is considered "the same" as long as the probe, file path, severity, and ruleId match.
|
|
535
|
-
|
|
536
|
-
### State Enum
|
|
537
|
-
|
|
538
|
-
| State | Meaning |
|
|
539
|
-
|---|---|
|
|
540
|
-
| `open` | Actively needs triage or was explicitly marked for re-review |
|
|
541
|
-
| `dismissed` | User dismissed as intentional or false positive — suppressed on future runs |
|
|
542
|
-
| `accepted-as-known` | Known issue, accepted without creating a VCS issue — suppressed on future runs |
|
|
543
|
-
| `reopened` | Previously suppressed but re-surfaced by user decision — shown again |
|
|
544
|
-
| `promoted-to-#NNN` | VCS issue created; shown informational ("tracked in #NNN") on future runs |
|
|
545
|
-
|
|
546
|
-
### Re-run Semantics
|
|
547
|
-
|
|
548
|
-
On each `/discovery` run, Phase 5 loads the state file and partitions findings before presenting them:
|
|
549
|
-
|
|
550
|
-
- **New findings** (no fingerprint entry) → always shown
|
|
551
|
-
- **`open` or `reopened`** → shown for triage
|
|
552
|
-
- **`dismissed` or `accepted-as-known`** → suppressed (silent — no user interaction needed)
|
|
553
|
-
- **`promoted-to-#NNN`** → informational line only ("tracked in #NNN")
|
|
554
|
-
|
|
555
|
-
A suppressed finding re-appears only if its fingerprint changes — i.e., the probe, file path, severity, or ruleId changes. No TTL on dismissed state.
|
|
392
|
+
The JSONL schema (`.orchestrator/metrics/discovery-triage.jsonl`), fingerprint derivation, state enum (`open`/`dismissed`/`accepted-as-known`/`reopened`), re-run semantics, and the `scripts/lib/discovery/triage-state.mjs` module API that Phase 5 above reads and writes.
|
|
556
393
|
|
|
557
|
-
|
|
394
|
+
See [references/discovery-triage-state.md](references/discovery-triage-state.md).
|
|
558
395
|
|
|
559
|
-
|
|
560
|
-
- `computeFingerprint({probe, file, severity, ruleId}): string`
|
|
561
|
-
- `loadTriageState(stateFilePath?): Promise<Map<fingerprint, entry>>`
|
|
562
|
-
- `appendTriageEntry(stateFilePath, entry): Promise<void>`
|
|
563
|
-
- `filterFindings({findings, stateMap}): {toShow, suppressed, tracked}`
|
|
396
|
+
**Read WHEN:** implementing or debugging Phase 5 triage persistence, or investigating why a finding was suppressed/re-shown across runs.
|
|
564
397
|
|
|
565
398
|
## Anti-Patterns
|
|
566
399
|
|
|
@@ -21,6 +21,15 @@
|
|
|
21
21
|
* reported "stale", 26 of them >7d, with a demonstrably healthy sync chain —
|
|
22
22
|
* because the clock, not the repo, was the denominator.
|
|
23
23
|
*
|
|
24
|
+
* Passive projects (GitLab #1238 point 2): a `01-projects/<slug>/` carrying a
|
|
25
|
+
* `_passive.md` marker is one nobody syncs on purpose (vault convention
|
|
26
|
+
* `_meta/conventions.md`; the same marker is already honoured by
|
|
27
|
+
* `scripts/lib/context-coverage-banner.mjs` via its `COVERAGE_FILES` check).
|
|
28
|
+
* Such a folder is skipped BEFORE any staleness comparison and counted in
|
|
29
|
+
* `metrics.passive_skipped`, so the skip is visible rather than silent — a probe
|
|
30
|
+
* that quietly ignored folders would be indistinguishable from one that found
|
|
31
|
+
* them healthy.
|
|
32
|
+
*
|
|
24
33
|
* Fallback: an overview WITHOUT `lastCommit` carries no repo-activity signal at
|
|
25
34
|
* all, so the wall-clock comparison is the only thing left. It is retained for
|
|
26
35
|
* that case only, marked `basis: 'probe-runtime'` in the evidence and carried at
|
|
@@ -67,6 +76,12 @@ function parseFrontmatter(text) {
|
|
|
67
76
|
// Helpers
|
|
68
77
|
// ---------------------------------------------------------------------------
|
|
69
78
|
|
|
79
|
+
/**
|
|
80
|
+
* Marker file that declares a `01-projects/<slug>/` deliberately un-synced.
|
|
81
|
+
* Same filename `scripts/lib/context-coverage-banner.mjs` accepts as coverage.
|
|
82
|
+
*/
|
|
83
|
+
const PASSIVE_MARKER = '_passive.md';
|
|
84
|
+
|
|
70
85
|
const HOURS_24 = 24 * 60 * 60 * 1000;
|
|
71
86
|
const HOURS_168 = 7 * 24 * 60 * 60 * 1000; // 7 days
|
|
72
87
|
|
|
@@ -89,7 +104,7 @@ export async function runProbe(projectRoot, config) {
|
|
|
89
104
|
|
|
90
105
|
const makeSkip = (skipped_reason) => ({
|
|
91
106
|
findings: [],
|
|
92
|
-
metrics: { scanned_projects: 0, stale_count: 0, errors: 0 },
|
|
107
|
+
metrics: { scanned_projects: 0, stale_count: 0, errors: 0, passive_skipped: 0 },
|
|
93
108
|
duration_ms: Math.round(Date.now() - start),
|
|
94
109
|
skipped_reason,
|
|
95
110
|
});
|
|
@@ -114,7 +129,7 @@ export async function runProbe(projectRoot, config) {
|
|
|
114
129
|
// --- Scan loop ---
|
|
115
130
|
|
|
116
131
|
const findings = [];
|
|
117
|
-
const metrics = { scanned_projects: 0, stale_count: 0, errors: 0 };
|
|
132
|
+
const metrics = { scanned_projects: 0, stale_count: 0, errors: 0, passive_skipped: 0 };
|
|
118
133
|
const now = Date.now();
|
|
119
134
|
|
|
120
135
|
let entries;
|
|
@@ -129,7 +144,7 @@ export async function runProbe(projectRoot, config) {
|
|
|
129
144
|
description: err.message,
|
|
130
145
|
evidence: {},
|
|
131
146
|
}],
|
|
132
|
-
metrics: { scanned_projects: 0, stale_count: 0, errors: 1 },
|
|
147
|
+
metrics: { scanned_projects: 0, stale_count: 0, errors: 1, passive_skipped: 0 },
|
|
133
148
|
duration_ms: Math.round(Date.now() - start),
|
|
134
149
|
error: err.message,
|
|
135
150
|
};
|
|
@@ -138,7 +153,21 @@ export async function runProbe(projectRoot, config) {
|
|
|
138
153
|
for (const entry of entries) {
|
|
139
154
|
if (!entry.isDirectory()) continue;
|
|
140
155
|
|
|
141
|
-
const
|
|
156
|
+
const projectDir = join(projectsDir, entry.name);
|
|
157
|
+
|
|
158
|
+
// #1238 point 2 — a `_passive.md` marker means "nobody syncs this on
|
|
159
|
+
// purpose". Skipped before any comparison, but COUNTED so the skip shows
|
|
160
|
+
// up in the metrics instead of looking like a healthy project. Checked
|
|
161
|
+
// BEFORE the `_overview.md` existence test: a passive folder that carries
|
|
162
|
+
// no overview is still a deliberate skip, and the earlier ordering dropped
|
|
163
|
+
// it into the silent non-project branch, under-reporting the metric. The
|
|
164
|
+
// finding population is unchanged either way — neither branch can emit one.
|
|
165
|
+
if (existsSync(join(projectDir, PASSIVE_MARKER))) {
|
|
166
|
+
metrics.passive_skipped++;
|
|
167
|
+
continue;
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
const overviewPath = join(projectDir, '_overview.md');
|
|
142
171
|
if (!existsSync(overviewPath)) continue; // non-project dir, skip silently
|
|
143
172
|
|
|
144
173
|
metrics.scanned_projects++;
|
|
@@ -254,6 +283,7 @@ export async function runProbe(projectRoot, config) {
|
|
|
254
283
|
scanned_projects: metrics.scanned_projects,
|
|
255
284
|
stale_count: metrics.stale_count,
|
|
256
285
|
errors: metrics.errors,
|
|
286
|
+
passive_skipped: metrics.passive_skipped,
|
|
257
287
|
duration_ms,
|
|
258
288
|
findings: findings.map(f => ({
|
|
259
289
|
slug: f.evidence.slug,
|
|
@@ -286,7 +316,7 @@ export async function runProbe(projectRoot, config) {
|
|
|
286
316
|
description: err.message,
|
|
287
317
|
evidence: {},
|
|
288
318
|
}],
|
|
289
|
-
metrics: { scanned_projects: 0, stale_count: 0, errors: 1 },
|
|
319
|
+
metrics: { scanned_projects: 0, stale_count: 0, errors: 1, passive_skipped: 0 },
|
|
290
320
|
duration_ms: Math.round(Date.now() - start),
|
|
291
321
|
error: err.message,
|
|
292
322
|
};
|
|
@@ -10,13 +10,15 @@
|
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
12
|
# Step 1: Verify the probe exists; skip if missing
|
|
13
|
-
|
|
13
|
+
# Probes live in the PLUGIN, not the project: ${PLUGIN_ROOT} is resolved per
|
|
14
|
+
# skills/_shared/config-reading.md. The probe still scans process.cwd() (project root).
|
|
15
|
+
test -f "${PLUGIN_ROOT}/skills/discovery/probes/docs-staleness.mjs" || { echo "SKIPPED: docs-staleness -- ${PLUGIN_ROOT}/skills/discovery/probes/docs-staleness.mjs not found (PLUGIN_ROOT='${PLUGIN_ROOT}'; empty = unresolved, see skills/_shared/config-reading.md)"; exit 0; }
|
|
14
16
|
|
|
15
17
|
# Step 2: Run the probe. It reads docs-staleness.thresholds.living from $CONFIG
|
|
16
18
|
# (passed from the discovery skill) and scans docs/*.md (root level) +
|
|
17
19
|
# docs/examples/*.md for filesystem-mtime staleness.
|
|
18
20
|
node --input-type=module -e "
|
|
19
|
-
import {runProbe} from '
|
|
21
|
+
import {runProbe} from '${PLUGIN_ROOT}/skills/discovery/probes/docs-staleness.mjs';
|
|
20
22
|
const cfg = JSON.parse(process.env.SO_CONFIG || '{}');
|
|
21
23
|
const r = await runProbe(process.cwd(), cfg);
|
|
22
24
|
for (const f of r.findings) {
|
|
@@ -49,14 +51,16 @@ if (r.skipped_reason) console.log('SKIPPED:', r.skipped_reason);
|
|
|
49
51
|
|
|
50
52
|
```bash
|
|
51
53
|
# Step 1: Verify the probe exists; skip if missing
|
|
52
|
-
|
|
54
|
+
# Probes live in the PLUGIN, not the project: ${PLUGIN_ROOT} is resolved per
|
|
55
|
+
# skills/_shared/config-reading.md. The probe still scans process.cwd() (project root).
|
|
56
|
+
test -f "${PLUGIN_ROOT}/skills/discovery/probes/ssot-code-diff.mjs" || { echo "SKIPPED: ssot-code-diff -- ${PLUGIN_ROOT}/skills/discovery/probes/ssot-code-diff.mjs not found (PLUGIN_ROOT='${PLUGIN_ROOT}'; empty = unresolved, see skills/_shared/config-reading.md)"; exit 0; }
|
|
53
57
|
|
|
54
58
|
# Step 2: Run the probe. It reads no config keys — it diffs a hand-curated
|
|
55
59
|
# registry of doc "count" claims (blocked-commands.json rules, .claude/rules/
|
|
56
60
|
# file count, skills/ user-facing directory count, commands/ file count)
|
|
57
61
|
# against the live code/filesystem value each claim describes.
|
|
58
62
|
node --input-type=module -e "
|
|
59
|
-
import {runProbe} from '
|
|
63
|
+
import {runProbe} from '${PLUGIN_ROOT}/skills/discovery/probes/ssot-code-diff.mjs';
|
|
60
64
|
const cfg = JSON.parse(process.env.SO_CONFIG || '{}');
|
|
61
65
|
const r = await runProbe(process.cwd(), cfg);
|
|
62
66
|
for (const f of r.findings) {
|
|
@@ -21,10 +21,12 @@ if (!sources.includes('discovery')) { console.log('SKIPPED: supply-chain-slopche
|
|
|
21
21
|
"
|
|
22
22
|
|
|
23
23
|
# Step 2: Run the probe
|
|
24
|
-
|
|
24
|
+
# Probes live in the PLUGIN, not the project: ${PLUGIN_ROOT} is resolved per
|
|
25
|
+
# skills/_shared/config-reading.md. The probe still scans process.cwd() (project root).
|
|
26
|
+
test -f "${PLUGIN_ROOT}/skills/discovery/probes/supply-chain-slopcheck.mjs" || { echo "SKIPPED: supply-chain-slopcheck -- ${PLUGIN_ROOT}/skills/discovery/probes/supply-chain-slopcheck.mjs not found (PLUGIN_ROOT='${PLUGIN_ROOT}'; empty = unresolved, see skills/_shared/config-reading.md)"; exit 0; }
|
|
25
27
|
|
|
26
28
|
node --input-type=module -e "
|
|
27
|
-
import probe from '
|
|
29
|
+
import probe from '${PLUGIN_ROOT}/skills/discovery/probes/supply-chain-slopcheck.mjs';
|
|
28
30
|
const r = await probe({ repoRoot: process.cwd() });
|
|
29
31
|
for (const f of r.findings) {
|
|
30
32
|
console.log('FINDING:', JSON.stringify(f));
|
|
@@ -79,14 +79,18 @@ Value: <dimension if applicable>
|
|
|
79
79
|
|
|
80
80
|
### Probe: design-drift
|
|
81
81
|
|
|
82
|
+
> **Naming collision — unresolved (2026-09-11).** This probe, specified below, is agent-based: Pencil MCP tools, qualitative visual comparison, no fixed detection script. GitLab issue #1300 (`projects-baseline` design-first Pencil-token-contract epic, scope item B3-Wrapper) plans `skills/discovery/probes/design-drift.mjs` — a *second*, mechanical, git-only probe under the identical name `design-drift`, sourced from a staleness script in the sibling `projects-baseline` repo — and explicitly proposes replacing this section (lines 79–107 at the time of writing) with that new spec. Whether the two converge, take distinct names, or one supersedes the other is a naming/design decision for whoever implements #1300; it is not decided here. Resolve the name before writing any code against either contract.
|
|
83
|
+
|
|
82
84
|
**Activation:** Pencil MCP configured in Session Config (`pencil` path provided, e.g. `pencil: designs/app.pen`).
|
|
83
85
|
|
|
84
86
|
**Detection Method:**
|
|
85
87
|
|
|
86
88
|
Use Pencil MCP tools to compare design specifications against implementation:
|
|
87
|
-
1. `
|
|
88
|
-
2. `
|
|
89
|
-
3. `
|
|
89
|
+
1. `mcp__pencil__get_app_state` -- check current design file
|
|
90
|
+
2. `mcp__pencil__execute` with a Get visitor (depth 1 for top-level frames, depth 2 for their children) -- retrieve design node properties (colors, spacing, typography). Read the execute tool's input schema for the exact visitor shape; `.pen` files are encrypted, so never `Read`/`Grep` them.
|
|
91
|
+
3. `mcp__pencil__browser` -- view/capture design frames for visual comparison (no direct screenshot tool exists on the current surface)
|
|
92
|
+
|
|
93
|
+
Tool names have one definition in code: `PENCIL_TOOL_NAMES` in `scripts/lib/ux-grill/pencil-coverage.mjs`. The pre-2026 tool surface it replaced no longer exists (dead names enumerated in that module's JSDoc).
|
|
90
94
|
|
|
91
95
|
Compare against:
|
|
92
96
|
- CSS custom properties / design tokens in codebase
|
|
@@ -10,12 +10,14 @@
|
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
12
|
# Step 1: Verify the probe exists; skip if missing
|
|
13
|
-
|
|
13
|
+
# Probes live in the PLUGIN, not the project: ${PLUGIN_ROOT} is resolved per
|
|
14
|
+
# skills/_shared/config-reading.md. The probe still scans process.cwd() (project root).
|
|
15
|
+
test -f "${PLUGIN_ROOT}/skills/discovery/probes/vault-staleness.mjs" || { echo "SKIPPED: vault-staleness -- ${PLUGIN_ROOT}/skills/discovery/probes/vault-staleness.mjs not found (PLUGIN_ROOT='${PLUGIN_ROOT}'; empty = unresolved, see skills/_shared/config-reading.md)"; exit 0; }
|
|
14
16
|
|
|
15
17
|
# Step 2: Run the probe. It reads vault-integration.vault-dir from $CONFIG
|
|
16
18
|
# (passed from the discovery skill) and scans the vault.
|
|
17
19
|
node --input-type=module -e "
|
|
18
|
-
import {runProbe} from '
|
|
20
|
+
import {runProbe} from '${PLUGIN_ROOT}/skills/discovery/probes/vault-staleness.mjs';
|
|
19
21
|
const cfg = JSON.parse(process.env.SO_CONFIG || '{}');
|
|
20
22
|
const r = await runProbe(process.cwd(), cfg);
|
|
21
23
|
for (const f of r.findings) {
|
|
@@ -30,6 +32,10 @@ if (r.skipped_reason) console.log('SKIPPED:', r.skipped_reason);
|
|
|
30
32
|
|
|
31
33
|
**Default severity:** low (<7d delta), medium (≥7d delta). Missing frontmatter fields → low.
|
|
32
34
|
|
|
35
|
+
**Denominator:** staleness is `lastCommit - lastSync` — both read from the same `_overview.md` frontmatter — never `now - lastSync`. An overview without `lastCommit` falls back to the probe runtime (`basis: 'probe-runtime'` in the evidence, lower confidence).
|
|
36
|
+
|
|
37
|
+
**Passive skip (#1238):** a `01-projects/<slug>/` carrying a `_passive.md` marker is skipped BEFORE any staleness comparison — it is excluded from `scanned_projects` and counted instead in `metrics.passive_skipped` — the SAME key in the in-memory metrics and in the JSONL record — so the skip is visible rather than indistinguishable from a healthy project. The marker is checked before the `_overview.md` existence test, so a passive folder without an overview is counted too rather than falling into the silent non-project branch.
|
|
38
|
+
|
|
33
39
|
---
|
|
34
40
|
|
|
35
41
|
### Probe: vault-narrative-staleness
|
|
@@ -39,10 +45,12 @@ if (r.skipped_reason) console.log('SKIPPED:', r.skipped_reason);
|
|
|
39
45
|
**Detection Method:**
|
|
40
46
|
|
|
41
47
|
```bash
|
|
42
|
-
|
|
48
|
+
# Probes live in the PLUGIN, not the project: ${PLUGIN_ROOT} is resolved per
|
|
49
|
+
# skills/_shared/config-reading.md. The probe still scans process.cwd() (project root).
|
|
50
|
+
test -f "${PLUGIN_ROOT}/skills/discovery/probes/vault-narrative-staleness.mjs" || { echo "SKIPPED: vault-narrative-staleness -- ${PLUGIN_ROOT}/skills/discovery/probes/vault-narrative-staleness.mjs not found (PLUGIN_ROOT='${PLUGIN_ROOT}'; empty = unresolved, see skills/_shared/config-reading.md)"; exit 0; }
|
|
43
51
|
|
|
44
52
|
node --input-type=module -e "
|
|
45
|
-
import {runProbe} from '
|
|
53
|
+
import {runProbe} from '${PLUGIN_ROOT}/skills/discovery/probes/vault-narrative-staleness.mjs';
|
|
46
54
|
const cfg = JSON.parse(process.env.SO_CONFIG || '{}');
|
|
47
55
|
const r = await runProbe(process.cwd(), cfg);
|
|
48
56
|
for (const f of r.findings) {
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# Discovery — Phase 5: Interactive Triage (Standalone Mode Only)
|
|
2
|
+
|
|
3
|
+
> Reference of the `discovery` skill, split out of `SKILL.md` (#1246). Body moved **byte-identical**; only this header is new.
|
|
4
|
+
> **Sibling-file paths inside this body are relative to the parent directory, not to `references/`**: `SKILL.md` → `../SKILL.md`. They were deliberately NOT rewritten, so the moved bytes stay verifiable against the pre-split file.
|
|
5
|
+
|
|
6
|
+
## Phase 5: Interactive Triage (Standalone Mode Only)
|
|
7
|
+
|
|
8
|
+
### 5.0 Load Triage State & Partition Findings
|
|
9
|
+
|
|
10
|
+
Before auto-defer and before presenting any findings for triage, load the persistent discovery triage state and filter findings through it:
|
|
11
|
+
|
|
12
|
+
1. Call `loadTriageState()` from `scripts/lib/discovery/triage-state.mjs` (uses default path `.orchestrator/metrics/discovery-triage.jsonl`). Returns an empty Map if the file does not exist — no error.
|
|
13
|
+
2. Call `filterFindings({ findings: verifiedFindings, stateMap })` to partition findings into three buckets:
|
|
14
|
+
- `toShow` — state is `open`, `reopened`, or **no prior state entry** (new findings — present for user triage)
|
|
15
|
+
- `suppressed` — state is `dismissed` or `accepted-as-known` (skip silently)
|
|
16
|
+
- `tracked` — state is `promoted-to-#NNN` (issue already filed; show as informational)
|
|
17
|
+
|
|
18
|
+
3. Emit a one-line state banner before the summary table:
|
|
19
|
+
```
|
|
20
|
+
Triage state: [N suppressed] suppressed (dismissed/accepted-as-known), [N tracked] tracked in existing issues. Presenting [N toShow] findings.
|
|
21
|
+
```
|
|
22
|
+
Omit the banner entirely if all three counts are zero (first run).
|
|
23
|
+
|
|
24
|
+
4. Render `tracked` findings as informational lines in the summary — NOT as interactive triage items:
|
|
25
|
+
```
|
|
26
|
+
[INFO] Finding "<title>" (<file_path>) is tracked in #<issue_id> — not re-triaged.
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
5. Continue Phase 5 triage using only `toShow` findings. The `suppressed` bucket requires no user interaction.
|
|
30
|
+
|
|
31
|
+
6. After the user completes triage (Steps 1-4 below), append state changes to `.orchestrator/metrics/discovery-triage.jsonl` via `appendTriageEntry()` from `triage-state.mjs`:
|
|
32
|
+
- User selects "Create issue" → append `{ fingerprint, state: 'promoted-to-#<issue_id>', issue_id: <N>, timestamp, session_id }`
|
|
33
|
+
- User selects "Dismiss -- intentional" or "Dismiss -- false positive" → append `{ fingerprint, state: 'dismissed', user_decision: '<reason>', timestamp, session_id }`
|
|
34
|
+
- User selects "Accept all" for batch → append one `{ fingerprint, state: 'open', ... }` entry per finding (so they re-appear next run if not yet promoted)
|
|
35
|
+
|
|
36
|
+
### 5.1 Auto-Defer Low-Confidence Findings
|
|
37
|
+
|
|
38
|
+
Before presenting findings for triage, separate by confidence threshold:
|
|
39
|
+
|
|
40
|
+
1. Findings with confidence >= threshold → present for interactive triage (below)
|
|
41
|
+
2. Findings with confidence < threshold → auto-defer with summary:
|
|
42
|
+
"Auto-deferred [N] low-confidence findings (score < [threshold]). Review with `/discovery --include-deferred`."
|
|
43
|
+
3. List auto-deferred findings in a collapsed section (not interactive — informational only)
|
|
44
|
+
|
|
45
|
+
### 5.1 Present High-Confidence Findings
|
|
46
|
+
|
|
47
|
+
Present findings using AskUserQuestion -- NEVER plain text options. On Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists.
|
|
48
|
+
|
|
49
|
+
Include confidence scores in the presentation:
|
|
50
|
+
```
|
|
51
|
+
[CRITICAL] (confidence: 85) hardcoded-values: API key found in src/config.ts:42
|
|
52
|
+
[HIGH] (confidence: 72) security-basics: eval() usage in src/utils/parser.ts:18
|
|
53
|
+
[MEDIUM] (confidence: 61) orphaned-annotations: TODO without issue in src/lib/auth.ts:55
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Step 1: Summary
|
|
57
|
+
|
|
58
|
+
Present a findings overview table:
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
## Discovery Results
|
|
62
|
+
|
|
63
|
+
Probes run: [N] | Findings verified: [N] | False positives discarded: [N]
|
|
64
|
+
|
|
65
|
+
| Category | Critical | High | Medium | Low | Total |
|
|
66
|
+
|----------|----------|------|--------|-----|-------|
|
|
67
|
+
| Code | ... | ... | ... | ... | ... |
|
|
68
|
+
| Infra | ... | ... | ... | ... | ... |
|
|
69
|
+
| UI | ... | ... | ... | ... | ... |
|
|
70
|
+
| Arch | ... | ... | ... | ... | ... |
|
|
71
|
+
| Session | ... | ... | ... | ... | ... |
|
|
72
|
+
| Audit | ... | ... | ... | ... | ... |
|
|
73
|
+
| Vault | ... | ... | ... | ... | ... |
|
|
74
|
+
| Feature | ... | ... | ... | ... | ... |
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### Step 2: Critical + High Findings -- Review Individually
|
|
78
|
+
|
|
79
|
+
For each Critical or High finding, use AskUserQuestion (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
AskUserQuestion({
|
|
83
|
+
questions: [{
|
|
84
|
+
question: "<severity> finding in <file_path> — what should happen with it?",
|
|
85
|
+
header: "Finding",
|
|
86
|
+
options: [
|
|
87
|
+
{ label: "Create issue (<severity>)", description: "Files it as priority::<severity>, so it is tracked outside this session. The code below is copied into the issue body.",
|
|
88
|
+
preview: "<finding title>\n\n<file_path>:<line_number>\n```\n<matched_text with +/-3 lines context>\n```\n\n<description>\n\nRecommended fix: <recommended_fix>" },
|
|
89
|
+
{ label: "Adjust priority", description: "Same issue, a priority you pick — this question then comes back with the new label." },
|
|
90
|
+
{ label: "Dismiss -- intentional", description: "The code is deliberate. Nothing is filed, and the finding stays only in this run's report." },
|
|
91
|
+
{ label: "Dismiss -- false positive", description: "The probe misread the code. Nothing is filed; worth reporting if the same probe misfires again." }
|
|
92
|
+
],
|
|
93
|
+
multiSelect: false
|
|
94
|
+
}]
|
|
95
|
+
})
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
If user selects "Adjust priority", ask which priority with another AskUserQuestion. On Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists.
|
|
99
|
+
|
|
100
|
+
### Step 3: Medium + Low Findings -- Review Batched
|
|
101
|
+
|
|
102
|
+
Group remaining findings by category. For each category with medium/low findings (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
AskUserQuestion({
|
|
106
|
+
questions: [{
|
|
107
|
+
question: "Create issues for all [N] medium/low findings in [category]?",
|
|
108
|
+
header: "Findings",
|
|
109
|
+
options: [
|
|
110
|
+
{ label: "Accept all (Recommended)", description: "Medium and low findings are cheap to file and cheap to close. Cost: [N] issues, roughly one second apart.",
|
|
111
|
+
preview: "1. [title] -- [file_path]:[line] ([severity])\n2. [title] -- [file_path]:[line] ([severity])\n..." },
|
|
112
|
+
{ label: "Review individually", description: "One question per finding, same options as the critical ones. Cost: [N] more prompts." },
|
|
113
|
+
{ label: "Dismiss all", description: "Nothing is filed for this category. The findings stay in this run's report only." }
|
|
114
|
+
],
|
|
115
|
+
multiSelect: false
|
|
116
|
+
}]
|
|
117
|
+
})
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
If "Review individually" selected, walk through each like Step 2.
|
|
121
|
+
|
|
122
|
+
### Step 4: Batch Confirmation
|
|
123
|
+
|
|
124
|
+
Before creating any issues (on Codex CLI where AskUserQuestion is unavailable, present as numbered Markdown lists):
|
|
125
|
+
|
|
126
|
+
```
|
|
127
|
+
AskUserQuestion({
|
|
128
|
+
questions: [{
|
|
129
|
+
question: "Ready to create [N] issues?\n\n- [X] critical\n- [Y] high\n- [Z] medium\n- [W] low",
|
|
130
|
+
header: "Confirm",
|
|
131
|
+
options: [
|
|
132
|
+
{ label: "Create all [N] issues", description: "Proceed with issue creation" },
|
|
133
|
+
{ label: "Review list first", description: "Show full list before creating" },
|
|
134
|
+
{ label: "Cancel", description: "Do not create any issues" }
|
|
135
|
+
]
|
|
136
|
+
}]
|
|
137
|
+
})
|
|
138
|
+
```
|
|
139
|
+
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Discovery — Discovery Triage State (#419)
|
|
2
|
+
|
|
3
|
+
> Reference of the `discovery` skill, split out of `SKILL.md` (#1246). Body moved **byte-identical**; only this header is new.
|
|
4
|
+
> **Sibling-file paths inside this body are relative to the parent directory, not to `references/`**: `SKILL.md` → `../SKILL.md`. They were deliberately NOT rewritten, so the moved bytes stay verifiable against the pre-split file.
|
|
5
|
+
|
|
6
|
+
## Discovery Triage State (#419)
|
|
7
|
+
|
|
8
|
+
Persistent triage state prevents re-presenting the same finding on every `/discovery` run. State is stored in an append-only JSONL file and keyed by a stable fingerprint.
|
|
9
|
+
|
|
10
|
+
### State File
|
|
11
|
+
|
|
12
|
+
**Location:** `.orchestrator/metrics/discovery-triage.jsonl` (gitignored via `.orchestrator/metrics/*.jsonl` pattern — machine-local, never committed)
|
|
13
|
+
|
|
14
|
+
**Format:** One JSON object per line:
|
|
15
|
+
```json
|
|
16
|
+
{"fingerprint":"aabb1122ccdd3344","state":"dismissed","user_decision":"intentional — debug log","timestamp":"2026-05-17T10:00:00.000Z","session_id":"deep-2"}
|
|
17
|
+
{"fingerprint":"eeff5566aabb7788","state":"promoted-to-#119","issue_id":119,"timestamp":"2026-05-17T10:01:00.000Z","session_id":"deep-2"}
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
### Fingerprint
|
|
21
|
+
|
|
22
|
+
`computeFingerprint({probe, file, severity, ruleId})` → 16-char hex (sha256 prefix).
|
|
23
|
+
|
|
24
|
+
`line_number` is **intentionally excluded** — it drifts on refactoring without the underlying issue changing. A finding is considered "the same" as long as the probe, file path, severity, and ruleId match.
|
|
25
|
+
|
|
26
|
+
### State Enum
|
|
27
|
+
|
|
28
|
+
| State | Meaning |
|
|
29
|
+
|---|---|
|
|
30
|
+
| `open` | Actively needs triage or was explicitly marked for re-review |
|
|
31
|
+
| `dismissed` | User dismissed as intentional or false positive — suppressed on future runs |
|
|
32
|
+
| `accepted-as-known` | Known issue, accepted without creating a VCS issue — suppressed on future runs |
|
|
33
|
+
| `reopened` | Previously suppressed but re-surfaced by user decision — shown again |
|
|
34
|
+
| `promoted-to-#NNN` | VCS issue created; shown informational ("tracked in #NNN") on future runs |
|
|
35
|
+
|
|
36
|
+
### Re-run Semantics
|
|
37
|
+
|
|
38
|
+
On each `/discovery` run, Phase 5 loads the state file and partitions findings before presenting them:
|
|
39
|
+
|
|
40
|
+
- **New findings** (no fingerprint entry) → always shown
|
|
41
|
+
- **`open` or `reopened`** → shown for triage
|
|
42
|
+
- **`dismissed` or `accepted-as-known`** → suppressed (silent — no user interaction needed)
|
|
43
|
+
- **`promoted-to-#NNN`** → informational line only ("tracked in #NNN")
|
|
44
|
+
|
|
45
|
+
A suppressed finding re-appears only if its fingerprint changes — i.e., the probe, file path, severity, or ruleId changes. No TTL on dismissed state.
|
|
46
|
+
|
|
47
|
+
### Module
|
|
48
|
+
|
|
49
|
+
`scripts/lib/discovery/triage-state.mjs` — pure ESM, Node stdlib only. Exports:
|
|
50
|
+
- `computeFingerprint({probe, file, severity, ruleId}): string`
|
|
51
|
+
- `loadTriageState(stateFilePath?): Promise<Map<fingerprint, entry>>`
|
|
52
|
+
- `appendTriageEntry(stateFilePath, entry): Promise<void>`
|
|
53
|
+
- `filterFindings({findings, stateMap}): {toShow, suppressed, tracked}`
|
|
54
|
+
|
package/skills/eval/rubric-v1.md
CHANGED
|
@@ -126,6 +126,19 @@ full-gate quality_gate events + `record.total_waves` / `record.waves`.
|
|
|
126
126
|
| `≥1` full-gate event in window; the **last by timestamp** has `exit_code === 0` | `pass` |
|
|
127
127
|
| `≥1` full-gate event in window; the **last by timestamp** has `exit_code !== 0` | `fail` |
|
|
128
128
|
|
|
129
|
+
Clarification, not a formula change: a record whose waves are **all**
|
|
130
|
+
coordinator-direct `Housekeeping` waves (the session-end writer rule since
|
|
131
|
+
#1321; predicate `isCoordinatorDirectHousekeeping` in
|
|
132
|
+
`scripts/lib/session-schema/filters.mjs`) counts as "no waves ran". The
|
|
133
|
+
decision keys on that wave shape only, never on `session_type`: a housekeeping
|
|
134
|
+
session that ran real waves stays `cannot-determine`. No record written before
|
|
135
|
+
#1321 has that shape. Measured 2026-09-12 on the working copy (the ledger is
|
|
136
|
+
gitignored, so no commit pins it): `jq -s
|
|
137
|
+
'[.[]|select((.waves|type)=="array" and (.waves|length)>0 and
|
|
138
|
+
all(.waves[]; .role=="Housekeeping" and .coordinator_direct==true))]|length'
|
|
139
|
+
.orchestrator/metrics/sessions.jsonl` → `0` of 427 records. So no historical
|
|
140
|
+
verdict changes, and `RUBRIC_VERSION` stays `rubric-v1`.
|
|
141
|
+
|
|
129
142
|
Scorer: `scoreGateHealth`. No `score` field.
|
|
130
143
|
|
|
131
144
|
### 4. `process-safety`
|