@windyroad/itil 0.51.1 → 0.51.2-preview.757
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -304,30 +304,99 @@ From the `**Origin**: inbound-reported (#NN)` field, extract the originating iss
|
|
|
304
304
|
|
|
305
305
|
Reuse Step 3's transition table (filename-suffix vs last-logged Status) — it is direction-agnostic. The inbound leg fires verdict comments for the **fix-released** and **closed** transitions (the verdicts a reporter most wants); Open → Known Error fires an optional progress comment only when the reporter explicitly asked for status. If the suffix matches the last logged inbound entry, exit no-op (`No inbound transition since last update; nothing to post.`).
|
|
306
306
|
|
|
307
|
-
#### I3.
|
|
307
|
+
#### I3. Generate the inbound verdict comment (reporter-facing, LLM-generated — no templates)
|
|
308
|
+
|
|
309
|
+
These comments are **generated from the local ticket's context, not filled from canned templates** (P363 rework, Directive 1). Templates produce cold, patterned prose that reporters recognise as form-letters; instead the agent reads the source-of-truth sections below and GENERATES each comment per the per-transition prompt, then rides the gate chain in [I5](#i5-compose-through-gates--post-same-composition-as-step-5) (cog-a11y → risk → voice-tone).
|
|
310
|
+
|
|
311
|
+
**Source-of-truth sections** (read from the local ticket before generating): `## Description` (the reporter's original symptom report), `## Workaround`, `## Root Cause Analysis`, `## Fix Released` (the release marker, version, commit SHA — the released `@windyroad/<pkg>@<version>` upgrade target comes from here).
|
|
312
|
+
|
|
313
|
+
**No-invention rule (load-bearing, carried from Step 4):** if a cited section is absent or empty, say so honestly ("we don't have a workaround to share yet") — NEVER synthesise technical claims. The gates cannot catch invented facts; the no-invention rule does. This mirrors ADR-062's safe-and-valid acknowledgement shape (Decision Outcome step 6).
|
|
314
|
+
|
|
315
|
+
**Anti-leakage discipline (P229/P350 — REVISED per repo visibility, Directive 3):** these bodies are reporter-facing on our own repo. Determine repo visibility once: `gh repo view --json visibility -q .visibility` → `PUBLIC` / `PRIVATE` / `INTERNAL`.
|
|
316
|
+
|
|
317
|
+
- **PUBLIC** — you MAY refer to a problem / ADR / RFC / JTBD / Story, but ALWAYS by its **title AND as a link**, never a bare ID:
|
|
318
|
+
- Bad (bare ID, opaque): `Tracked locally as P164.`
|
|
319
|
+
- Bad (title only, no link): `Tracked as the octal-eval bug.`
|
|
320
|
+
- Good (title + linked ID): `Tracked as [Latent octal-eval bug in next-ID formula](https://github.com/windyroad/agent-plugins/blob/main/docs/problems/known-error/164-octal-eval-bug-in-next-id-formula-across-ticket-creator-skills-fires-when-ids-cross-099.md) (P164).`
|
|
321
|
+
- Resolve the link: ADR / RFC / JTBD / Story canonical paths live on `main`; for a problem ticket the filename carries the state suffix + slug — resolve the actual filename via `ls docs/<class>/<NNN>-*` (or `gh search code`) before linking. Grounded in [ADR-055](../../../docs/decisions/055-plugin-published-namespace-prefixed-internal-ids.proposed.md)'s permalink-progressive-enhancement shape and P229's jargon-vs-link distinction.
|
|
322
|
+
- **PRIVATE / INTERNAL / indeterminate** — strict ban (the prior P229 rule): omit problem / ADR / RFC / JTBD / Story references entirely. External readers can't reach the URLs, so a link is worse than no reference.
|
|
323
|
+
|
|
324
|
+
**Always banned (regardless of visibility):** classification tokens (`safe-and-valid`, `inbound-reported`, `clear-malicious`, `above-threshold-pushback`), internal step IDs (`Step 4.5e`, `step 6`), agent-internal vocabulary (`framework-resolution boundary`, `mechanical-stage carve-out`), runtime-detail prose (`marker-vs-file deadlock`, `subprocess-boundary contract`), and absolute `docs/problems/...` path strings (always prefer the rendered link). The released `@windyroad/<pkg>@<version>` upgrade target is always permitted.
|
|
325
|
+
|
|
326
|
+
**Open → Known Error generation prompt** (only when the reporter explicitly asked for interim status — otherwise skip O→KE and wait for the fix-released verdict):
|
|
327
|
+
|
|
328
|
+
```text
|
|
329
|
+
GOAL: convey the root cause in plain, reporter-readable terms, AND share the
|
|
330
|
+
WORKAROUND so the reporter is unblocked while the fix is in flight (Directive 2).
|
|
331
|
+
|
|
332
|
+
READ: ## Description (their symptom report), ## Root Cause Analysis, ## Workaround.
|
|
333
|
+
|
|
334
|
+
ROOT CAUSE: explain what's actually going wrong in terms the reporter can act
|
|
335
|
+
on — no internal mechanism jargon.
|
|
336
|
+
|
|
337
|
+
WORKAROUND — share it with correct PROVENANCE (Directive 4):
|
|
338
|
+
1. ## Workaround empty / "(deferred to investigation)": be honest — "we don't
|
|
339
|
+
have a workaround to share yet" — do NOT fabricate one.
|
|
340
|
+
2. Otherwise determine WHO first provided it. Read the originating issue's
|
|
341
|
+
body + comments:
|
|
342
|
+
gh issue view <NN> --repo <OWN_OWNER_REPO> --json comments,body \
|
|
343
|
+
--jq '.body, .comments[].body'
|
|
344
|
+
and semantically match (your judgement, not strict-string) the local
|
|
345
|
+
## Workaround against the issue body and each commenter's prose:
|
|
346
|
+
- MAINTAINER-AUTHORED (no match in issue body or any comment): share the
|
|
347
|
+
workaround in our own voice.
|
|
348
|
+
- REPORTER-PROVIDED (matches the issue body / the reporter's own comment):
|
|
349
|
+
credit the reporter — e.g. "Thanks again — your workaround in the
|
|
350
|
+
original report turned out to be exactly right. To confirm the details:"
|
|
351
|
+
then the exact ## Workaround prose (we've validated it).
|
|
352
|
+
- COMMENTER-PROVIDED (matches a comment by someone other than the
|
|
353
|
+
reporter): credit them by @handle — e.g. "@<handle> — your workaround
|
|
354
|
+
upthread turned out to be the right path. Confirmed the details:" then
|
|
355
|
+
the exact ## Workaround prose.
|
|
356
|
+
- BOTH-SOURCE (reporter proposed X, a commenter refined to X'): credit
|
|
357
|
+
both per their contribution.
|
|
358
|
+
Honour @handles via standard GitHub-flavoured markdown so the credited
|
|
359
|
+
person is notified.
|
|
360
|
+
3. Always CONFIRM THE EXACT workaround details (the validated ## Workaround
|
|
361
|
+
prose) — never paraphrase a confirmed workaround into vagueness.
|
|
362
|
+
|
|
363
|
+
LENGTH: concise — usually 3–6 sentences. Bullet list ONLY for a genuine
|
|
364
|
+
multi-step workaround. Apply the anti-leakage discipline above. Voice/tone per
|
|
365
|
+
docs/VOICE-AND-TONE.md (the voice-tone gate runs anyway).
|
|
366
|
+
```
|
|
308
367
|
|
|
309
|
-
|
|
368
|
+
**Known Error → Verification Pending (fix released) generation prompt:**
|
|
310
369
|
|
|
311
|
-
|
|
370
|
+
```text
|
|
371
|
+
GOAL: tell the reporter the fix shipped, name the upgrade target, invite verify.
|
|
312
372
|
|
|
313
|
-
|
|
314
|
-
Thanks again for the report. The fix has shipped in `@windyroad/<pkg>@<version>`.
|
|
373
|
+
READ: ## Fix Released (release marker, version, commit SHA), ## Workaround.
|
|
315
374
|
|
|
316
|
-
|
|
375
|
+
INCLUDE:
|
|
376
|
+
- What changed, in one reporter-readable sentence (from ## Fix Released) — no
|
|
377
|
+
invented detail.
|
|
378
|
+
- The upgrade target: `@windyroad/<pkg>@<version>` (or later).
|
|
379
|
+
- If ## Workaround had a workaround: note it's no longer needed after upgrade.
|
|
380
|
+
- An invitation to upgrade and confirm, and "reply here if it persists".
|
|
317
381
|
|
|
318
|
-
|
|
382
|
+
LENGTH: concise — usually 3–5 sentences. Apply the anti-leakage discipline
|
|
383
|
+
above. Voice/tone per docs/VOICE-AND-TONE.md.
|
|
319
384
|
```
|
|
320
385
|
|
|
321
|
-
**Verification Pending → Closed
|
|
386
|
+
**Verification Pending → Closed generation prompt:**
|
|
322
387
|
|
|
323
|
-
```
|
|
324
|
-
|
|
325
|
-
```
|
|
388
|
+
```text
|
|
389
|
+
GOAL: close the loop warmly, thank the reporter, give the reopen path.
|
|
326
390
|
|
|
327
|
-
|
|
391
|
+
READ: ## Fix Released (the released version).
|
|
328
392
|
|
|
329
|
-
|
|
330
|
-
|
|
393
|
+
INCLUDE:
|
|
394
|
+
- The fix is confirmed on our side and shipped in `@windyroad/<pkg>@<version>`.
|
|
395
|
+
- A genuine thank-you — their report is what got it fixed.
|
|
396
|
+
- The reopen path: reply or reopen if it resurfaces.
|
|
397
|
+
|
|
398
|
+
LENGTH: concise — usually 2–4 sentences. Apply the anti-leakage discipline
|
|
399
|
+
above. Voice/tone per docs/VOICE-AND-TONE.md.
|
|
331
400
|
```
|
|
332
401
|
|
|
333
402
|
#### I4. Idempotency guard (before posting)
|
|
@@ -343,7 +412,11 @@ The verdict marker is the released `@windyroad/<pkg>@<version>` string (plus the
|
|
|
343
412
|
|
|
344
413
|
#### I5. Compose through gates + post (same composition as Step 5)
|
|
345
414
|
|
|
346
|
-
Route the
|
|
415
|
+
Route the GENERATED comment through the reporter-facing gate chain — **cognitive-accessibility → external-comms (risk) → voice-tone** (Directive 1's "run cog-a11y, risk, voice and tone"). The risk + voice-tone legs are the SAME `wr-risk-scorer:external-comms` + `wr-voice-tone:external-comms` dual gate (AND composition) as Step 5 — no weaker path for inbound.
|
|
416
|
+
|
|
417
|
+
**Cog-a11y gate (when-available, P338-gated — do NOT block):** the cognitive-accessibility evaluator rides FIRST so reporter-facing prose is checked for plain-language / reading-level before the risk + voice-tone legs. `@windyroad/cognitive-a11y` does not exist yet ([P338](../../../docs/problems/open/338-p082-phase-2-cognitive-a11y-evaluator-on-external-comms-surfaces-new-windyroad-cognitive-a11y-plugin.md) Open) — until it lands the chain degrades to the existing external-comms + voice-tone dual gate, exactly as ADR-028's per-evaluator marker scheme handles an uninstalled evaluator (an absent evaluator's gate is simply not registered → the remaining legs' PASS unblocks the retry). The cog-a11y-as-third-external-comms-evaluator declaration is recorded in [ADR-028](../../../docs/decisions/028-voice-tone-gate-external-comms.proposed.md)'s `## Amendments` (the locus ADR-028's own Reassessment Criteria designate for a third evaluator); this leg is the consumer-side wiring note only. **Do NOT block this iteration's inbound dispatch on P338** — ship the wiring, ride the dual gate today.
|
|
418
|
+
|
|
419
|
+
Above-appetite handling is identical to Step 5c (silent risk-reduce + re-score; if still above, save to `## Queued Upstream Update` + queue an `outstanding_questions` entry; the orchestrator continues per P352 — do NOT halt). Within appetite, post on our own repo:
|
|
347
420
|
|
|
348
421
|
```bash
|
|
349
422
|
gh issue comment "${NN}" --repo "${OWN_OWNER_REPO}" --body "${INBOUND_BODY}"
|
|
@@ -448,7 +521,7 @@ Four distinct AFK branches. Per the [ADR-024](../../../docs/decisions/024-cross-
|
|
|
448
521
|
| Above-appetite — silent risk-reduce + re-score within appetite | Re-draft with tighter source-citation + shorter prose; re-invoke `wr-risk-scorer:external-comms`. If within → post per the below-appetite branch. | ADR-024 amendment (P080); ADR-044 framework-resolution boundary; ADR-042 within-axis precedent (open-vocabulary risk-reducing measures) |
|
|
449
522
|
| Above-appetite — silent risk-reduce did not bring within appetite | Save drafted comment to `## Queued Upstream Update` + queue `outstanding_questions` entry (category: `deviation-approval`). Orchestrator continues per P352. | ADR-024 amendment (P080); ADR-013 Rule 6; P352 |
|
|
450
523
|
| Above-appetite commit (Step 7) | Skip the commit, report uncommitted state. | ADR-013 Rule 6 |
|
|
451
|
-
| **Inbound-origin verdict (P363)** | When the ticket carries `**Origin**: inbound-reported (#NN)`, run the [§ Inbound-origin verdict dispatch](#inbound-origin-verdict-dispatch-p363) leg (I1–
|
|
524
|
+
| **Inbound-origin verdict (P363)** | When the ticket carries `**Origin**: inbound-reported (#NN)`, run the [§ Inbound-origin verdict dispatch](#inbound-origin-verdict-dispatch-p363) leg (I1–I7): idempotency-guard, then **GENERATE** the verdict comment from ticket context (no templates; O→KE shares the workaround with provenance-credit), route through the cog-a11y → external-comms → voice-tone gate chain (cog-a11y when-available per P338; dual gate today), post `gh issue comment` on our own repo (and `gh issue close` on Verifying → Closed), back-write a direction-tagged lifecycle log entry. Above-appetite queues per the rows above (does NOT halt). Reporter-facing prose; anti-leakage visibility-gated (PUBLIC → titled+linked refs; PRIVATE/indeterminate → strict ban; classification tokens / step IDs / internal vocab always banned — P229/P350). | ADR-024 amendment (P363); ADR-028 (cog-a11y third evaluator, P338); ADR-055 (linked-title); ADR-076 (Origin field); P352 |
|
|
452
525
|
|
|
453
526
|
The pre-amendment "halt-the-orchestrator on above-appetite" semantics are **superseded** by queue-and-continue per P352 — same shape as the post-P270 initial-filing path.
|
|
454
527
|
|
|
@@ -478,7 +551,8 @@ The skill's no-op exit (Step 1) means firing the trigger unconditionally on ever
|
|
|
478
551
|
- [ADR-075](../../../docs/decisions/075-promptfoo-agent-prose-verdict-eval-harness.proposed.md) Amendment 2026-06-02 — paired promptfoo Tier-A/B eval discharges the R009 prose-floor for SKILL surfaces.
|
|
479
552
|
- [ADR-061](../../../docs/decisions/061-dogfood-graduation-criteria.proposed.md) Rule 4 — evidence-floor; the paired eval ships in the same commit as this SKILL prose for atomic R009 discharge.
|
|
480
553
|
- **P080** — driving problem ticket (No bidirectional update of upstream-reported problems).
|
|
481
|
-
- **P363** — driving problem ticket for the [§ Inbound-origin verdict dispatch](#inbound-origin-verdict-dispatch-p363) leg (inbound-reported tickets never received a fix-released verdict on the originating issue). Fix option (b) — consume the `**Origin**` field — user-ratified 2026-06-22.
|
|
554
|
+
- **P363** — driving problem ticket for the [§ Inbound-origin verdict dispatch](#inbound-origin-verdict-dispatch-p363) leg (inbound-reported tickets never received a fix-released verdict on the originating issue). Fix option (b) — consume the `**Origin**` field — user-ratified 2026-06-22. **Rework 2026-06-23** (four user directives): the inbound verdict is LLM-generated per-context (not templated); O→KE shares the workaround with reporter/commenter provenance-credit; anti-leakage is visibility-gated (PUBLIC → titled+linked refs); the gate chain gains a cog-a11y leg first (P338-gated).
|
|
555
|
+
- [ADR-055](../../../docs/decisions/055-plugin-published-namespace-prefixed-internal-ids.proposed.md) — grounds the PUBLIC-repo titled+linked reference discipline (permalink-progressive-enhancement; cures the bare-ID collision failure mode).
|
|
482
556
|
- [ADR-076](../../../docs/decisions/076-inbound-reported-problems-rank-ahead-via-sort-tier.proposed.md) — owns the `**Origin**: inbound-reported (#NN)` on-ticket field this leg consumes.
|
|
483
557
|
- [ADR-062](../../../docs/decisions/062-inbound-upstream-report-discovery-assessment-pipeline.proposed.md) — inbound intake-time pipeline; its safe-and-valid branch is the Origin-field writer this leg reads at fix-released time.
|
|
484
558
|
- **P079** — sibling problem (inbound-discovery leg); together P079 + P080 close the reporter-loop end-to-end.
|
|
@@ -8,8 +8,10 @@
|
|
|
8
8
|
# packages/itil/skills/update-upstream/eval/promptfooconfig.yaml carries
|
|
9
9
|
# the behavioural Tier-A/B coverage per ADR-075 Amendment 2026-06-02.
|
|
10
10
|
# Provenance: P080 (original bidirectional contract) + P363 / ADR-024
|
|
11
|
-
# amendment 2026-06-22 (inbound-origin verdict dispatch leg
|
|
12
|
-
#
|
|
11
|
+
# amendment 2026-06-22 (inbound-origin verdict dispatch leg) + P363 rework
|
|
12
|
+
# 2026-06-23 (four user directives — LLM-generated verdict, workaround
|
|
13
|
+
# provenance-credit, visibility-gated anti-leakage, cog-a11y gate),
|
|
14
|
+
# behaviourally paired by the inbound eval cases in promptfooconfig.yaml.
|
|
13
15
|
# The lockstep transition-problem/manage-problem grep check is an inherent
|
|
14
16
|
# copy-not-move drift detector (ADR-010/P362) with no behavioural form.)
|
|
15
17
|
#
|
|
@@ -278,6 +280,62 @@ setup() {
|
|
|
278
280
|
[ "$status" -eq 0 ]
|
|
279
281
|
}
|
|
280
282
|
|
|
283
|
+
# ─── P363 rework 2026-06-23 — four user directives ────────────────────────────
|
|
284
|
+
|
|
285
|
+
@test "update-upstream: SKILL.md inbound verdict is LLM-generated per-context, not templated (Directive 1)" {
|
|
286
|
+
# Directive 1 — templates produce cold, patterned prose; the inbound verdict
|
|
287
|
+
# is GENERATED from ticket context via per-transition generation prompts the
|
|
288
|
+
# agent reads at runtime, NOT filled from canned template bodies.
|
|
289
|
+
run grep -iE 'LLM-generated|generated from the local ticket|generation prompt' "$SKILL_MD"
|
|
290
|
+
[ "$status" -eq 0 ]
|
|
291
|
+
run grep -F 'no templates' "$SKILL_MD"
|
|
292
|
+
[ "$status" -eq 0 ]
|
|
293
|
+
}
|
|
294
|
+
|
|
295
|
+
@test "update-upstream: SKILL.md O→KE prompt shares the workaround + scans issue comments for provenance (Directive 2/4)" {
|
|
296
|
+
# Directive 2 — a Known Error comment shares the WORKAROUND, not just root
|
|
297
|
+
# cause. Directive 4 — provenance: read ## Workaround AND scan the originating
|
|
298
|
+
# issue's body+comments (gh issue view --json comments,body) to attribute it.
|
|
299
|
+
run grep -F 'share it with correct PROVENANCE' "$SKILL_MD"
|
|
300
|
+
[ "$status" -eq 0 ]
|
|
301
|
+
run grep -F 'comments,body' "$SKILL_MD"
|
|
302
|
+
[ "$status" -eq 0 ]
|
|
303
|
+
run grep -F '## Workaround' "$SKILL_MD"
|
|
304
|
+
[ "$status" -eq 0 ]
|
|
305
|
+
}
|
|
306
|
+
|
|
307
|
+
@test "update-upstream: SKILL.md O→KE prompt names four provenance branches + credit shape + confirm-exact (Directive 4)" {
|
|
308
|
+
# The prompt must branch on who provided the workaround and credit per branch,
|
|
309
|
+
# always confirming the exact validated details.
|
|
310
|
+
run grep -F 'MAINTAINER-AUTHORED' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
311
|
+
run grep -F 'REPORTER-PROVIDED' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
312
|
+
run grep -F 'COMMENTER-PROVIDED' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
313
|
+
run grep -F 'BOTH-SOURCE' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
314
|
+
run grep -iE 'credit the reporter|credit them by @handle|credit.*both' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
315
|
+
run grep -iE 'CONFIRM THE EXACT|Confirmed the details|confirm the details' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
@test "update-upstream: SKILL.md inbound anti-leakage is visibility-gated — PUBLIC titled+linked, PRIVATE strict ban (Directive 3)" {
|
|
319
|
+
# Directive 3 — for PUBLIC repos refer to problems/ADRs/RFCs by title AND link
|
|
320
|
+
# (never bare ID); for PRIVATE/INTERNAL/indeterminate fall back to strict ban.
|
|
321
|
+
run grep -F 'gh repo view --json visibility' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
322
|
+
run grep -iE 'title AND as a link|titled\+linked' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
323
|
+
run grep -iE 'PRIVATE / INTERNAL / indeterminate|strict ban' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
324
|
+
# classification tokens / step IDs / internal vocab stay banned regardless.
|
|
325
|
+
run grep -iE 'Always banned' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
326
|
+
# grounded in ADR-055's permalink-progressive-enhancement.
|
|
327
|
+
run grep -F 'ADR-055' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
328
|
+
}
|
|
329
|
+
|
|
330
|
+
@test "update-upstream: SKILL.md inbound gate chain rides cog-a11y first (when-available, P338-gated, do-not-block)" {
|
|
331
|
+
# Directive 1's "run cog-a11y, risk, voice and tone" — cog-a11y rides FIRST,
|
|
332
|
+
# gated on P338 (plugin not yet shipped); the chain degrades to the dual gate
|
|
333
|
+
# today and does NOT block on P338.
|
|
334
|
+
run grep -iE 'cog-a11y → risk → voice-tone|cognitive-accessibility → external-comms' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
335
|
+
run grep -F 'P338' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
336
|
+
run grep -iE 'do NOT block|when-available' "$SKILL_MD"; [ "$status" -eq 0 ]
|
|
337
|
+
}
|
|
338
|
+
|
|
281
339
|
@test "transition-problem + manage-problem Step 7 greps match the inbound Origin alternation in LOCKSTEP (P363/P362)" {
|
|
282
340
|
# Both lockstep pre-checks must match ^## Reported Upstream OR
|
|
283
341
|
# ^\*\*Origin\*\*: inbound-reported \(# — drift re-opens the P363 gap on
|