kushi-agents 4.8.3 → 4.9.0

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "kushi-agents",
-  "version": "4.8.3",
-  "description": "Install Kushi — multi-source project evidence agent with snapshot+stream capture across Email, Teams, OneNote, Loop, SharePoint, Meetings, CRM, ADO. WorkIQ-only for M365 sources (Graph / m365_* FORBIDDEN as fallbacks; user-paste is first-class). Host-agnostic.",
+  "version": "4.9.0",
+  "description": "Install Kushi — multi-source project evidence agent with Comprehensive Structured Capture (CSC) into weekly-only files across Email, Teams, OneNote, Loop, SharePoint, Meetings, CRM, ADO. Meetings retain a sibling verbatim/ audit folder. WorkIQ-only for M365 sources (Graph / m365_* FORBIDDEN as fallbacks; user-paste is first-class). Host-agnostic.",
   "type": "module",
   "bin": {
     "kushi-agents": "./bin/cli.mjs"

package/plugin/instructions/comprehensive-structured-capture.instructions.md

@@ -0,0 +1,250 @@
+---
+applyTo: "**"
+description: "Comprehensive Structured Capture (CSC) — replaces verbatim-by-default (v4.9.0). Every per-entity evidence block is a structured set of bulleted sections capturing every material detail, with no prose paragraphs and no omissions. Readable by humans, parseable by build-state, reliably producible by WorkIQ in one call without payload-too-large failures."
+---
+
+# Comprehensive Structured Capture (CSC) — HARD RULE (kushi v4.9.0+)
+
+## Why CSC replaced verbatim-by-default
+
+The old contract (`verbatim-by-default.instructions.md`) required every pull-* skill to capture
+raw verbatim bodies — full email body, full transcript line-by-line, full OneNote page body —
+plus multi-paragraph AI Narrative Summaries. In practice this failed:
+
+- WorkIQ refuses bulk verbatim with `payload-too-large` for OneNote pages.
+- `m365_get_email` tool-errors ~100% of the time; bodies degrade to 200-char search previews.
+- Teams meeting transcripts are often not generated (transcription not enabled).
+- Multi-paragraph prose summaries are easy to fabricate from adjacent evidence — citation
+  discipline degrades.
+
+CSC trades raw verbatim for **structured completeness**: every material fact captured as a
+bullet, organized into named sections, with citations on every bullet. This is what WorkIQ
+returns reliably on the first try, and it is what downstream readers (build-state,
+ask-project, consolidate-evidence) can parse without natural-language understanding.
+
+## What a CSC block looks like (universal shape)
+
+Every per-entity block — per meeting, per email thread, per Teams thread, per OneNote page,
+per SharePoint file, per CRM record, per ADO work item — uses this section order. Sections
+that don't apply for a source are omitted (see per-source overrides below); sections that
+apply but had nothing surfaced MUST be present with `_None surfaced._`.
+
+```markdown
+## <entity-name> {#<entity-anchor>}
+
+- **Source basis**: <one-liner — what WorkIQ call + request-id, what host probe, fidelity>
+- **Coverage window**: <ISO range covered by this block>
+- **Last touched**: <ISO timestamp>
+
+### Participants / Present
+- <name or address> — <role: organizer / attendee / sender / recipient / author / commenter>
+- ...
+
+### Topics Discussed
+- <topic 1>
+  - <sub-point with detail>
+  - <sub-point with detail>
+- <topic 2>
+  - ...
+- ...
+
+### Q & A
+- **Q** (asked by <name>): <question text> — **A** (answered by <name>): <answer text> [source: ... · ts]
+- **Q** (asked by <name>): <question> — **A**: _Left open._ [source: ... · ts]
+- ...
+
+### Who Said What (material statements)
+- <name>: "<material statement / position / pushback>" [source: ... · ts]
+- ...
+- _(Only material statements — not every utterance. Routine acknowledgements omitted.)_
+
+### Decisions
+- **Decided**: <exact wording> — **by**: <name(s)> — **rationale**: <why> — **rejected alternatives**: <what was considered and dropped> [source: ... · ts]
+- ...
+
+### Dates & Numbers Shared
+- <date / number / dollar amount / percentage / count>: <exact value> — **context**: <what it refers to> — **shared by**: <name> [source: ... · ts]
+- ...
+
+### Action Items
+- [ ] <owner> — <action> — **due**: <date or TBD with reason> [source: ... · ts]
+- ...
+
+### Next Steps
+- <forward-looking commitment without hard owner+due> — **expected by**: <approximate date or "next sync">
+- ...
+- _(Distinct from Action Items. "Team will look into X", "we'll circle back", soft signals.)_
+
+### Open Questions / Unresolved
+- <question or unresolved item> — **raised by**: <name> — **why deferred**: <reason> [source: ... · ts]
+- ...
+
+### Risks / Blockers / Dependencies
+- **Risk**: <description> — **owner**: <name or unassigned> — **mitigation**: <plan or _None._> [source: ... · ts]
+- **Blocker**: <description> — **blocking**: <what's blocked> — **owner**: <name> [source: ... · ts]
+- **Dependency**: <description> — **on**: <what we depend on> — **owner**: <name> [source: ... · ts]
+- ...
+
+### Customer Asks (engagement sources only)
+- <ask> — **asked by**: <customer name> — **of**: <Microsoft role/person> — **status**: <open / committed / declined> [source: ... · ts]
+- ...
+
+### Artifacts / Links
+- <file name or link title>: <url> — <type: shared in meeting / attached to email / linked from page / etc.>
+- ...
+
+### Coverage Notes
+- <what was retrievable vs what wasn't>
+- <any caveats: WorkIQ summary-only, transcript-not-generated, body-not-extractable, etc.>
+```
+
+## Hard rules
+
+### 1. Bullets only — no prose paragraphs
+
+Every section content is a bulleted list. A paragraph of running prose anywhere in a CSC block
+is a **defect**. The AI Narrative Summary that the old verbatim doctrine required is removed —
+its purpose (give a reader the whole picture) is satisfied by the union of `Topics Discussed` +
+`Decisions` + `Who Said What` + `Dates & Numbers Shared`.
+
+Bullets may have nested sub-bullets (2–5 lines of detail per top-level bullet is normal).
+Bullets may span multiple lines using markdown soft-wraps. They must not be paragraphs.
+
+### 2. No omission — "and N more" is a defect
+
+If WorkIQ returns `"and 12 more topics..."`, `"key highlights only"`, `"sample of..."`,
+`"10 of 47 shown"`, or any equivalent truncation marker — that is a **defect**. Re-issue with
+the doubled-strict retry from `workiq-only.instructions.md` asking for the full set with no
+truncation. If the second attempt also truncates, write a deferred-retry marker per
+`deferred-retry-on-workiq-fail.instructions.md` and continue.
+
+### 3. Empty sections are explicit
+
+A section that applies to a source but had nothing surfaced MUST be present with
+`_None surfaced._` (or `_None this entity._`). Omitting the section is a defect — the reader
+cannot tell "checked and empty" from "skipped".
+
+### 4. Citation per material bullet
+
+Every bullet that carries a fact (date, number, decision, statement, action, risk, ask) MUST
+carry an inline citation. Format:
+`[source: <alias>/<source>/weekly/YYYY-MM-DD_<source>-csc.md#<entity-anchor> · <iso-ts>]`
+or
+`[source: WorkIQ request-id <guid> · ts]` when citing the WorkIQ probe directly.
+
+Bullets that are organizational ("Participants", "Artifacts / Links") need not be individually
+cited if the Source basis line covers them.
+
+### 5. No fabrication from adjacent evidence
+
+If WorkIQ returned `body-unavailable` or the cascade returned nothing for an entity, the
+correct CSC block is the entity header + `Source basis: unavailable` + `Coverage Notes`
+explaining the failure mode + `_None surfaced._` for every other section. **Do NOT** infer
+what the entity "likely contained" from other emails, chat traffic, file names, or page titles
+— that's fabrication and pollutes the citation chain. Empty is the correct state until the
+entity is actually retrieved.
+
+### 6. Who Said What is for material attribution only
+
+`Who Said What` is NOT a verbatim transcript reproduction. It is the curated list of
+**material** statements — positions taken, pushbacks raised, key data points stated,
+commitments made — with attribution. Routine acknowledgements (`thanks`, `sounds good`, `+1`)
+are omitted. A 50-message thread typically produces 3–10 entries here, not 50.
+
+For meetings, the raw chronological transcript (when retrievable) lives in
+`Evidence/<alias>/meetings/verbatim/<dir>/transcript.txt` per
+`meetings-verbatim-required.instructions.md`. That is the audit artifact; the CSC block in
+the weekly file is the curated, query-able form.
+
+## Per-source section applicability
+
+| Section | email | teams | meetings | onenote | sharepoint | crm | ado |
+|---|---|---|---|---|---|---|---|
+| Participants / Present | ✅ sender+recipients | ✅ chat members | ✅ attendees | ✅ author + editors | ✅ author + editors | ✅ owner + contacts | ✅ assignee + commenters |
+| Topics Discussed | ✅ | ✅ | ✅ | ✅ page content topics | ✅ file content topics | ✅ record story | ✅ WI story |
+| Q & A | ✅ | ✅ | ✅ | ➖ omit | ➖ omit | ➖ omit | ✅ from comments |
+| Who Said What | ✅ | ✅ | ✅ | ➖ omit | ➖ omit | ➖ omit | ✅ from comments |
+| Decisions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Dates & Numbers Shared | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Action Items | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Next Steps | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Open Questions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Risks / Blockers / Dependencies | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Customer Asks | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Artifacts / Links | ✅ attachments | ✅ shared files | ✅ recording+files | ✅ embedded links | ✅ this file | ✅ related records | ✅ related WIs |
+| Coverage Notes | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+
+Sections marked ➖ MUST be omitted for that source (don't write `_None surfaced._` — omit
+entirely).
+
+## Volume guidance
+
+CSC blocks are typically **denser per byte** than the old verbatim shape (no prose padding) but
+LONGER overall (more sections, every fact bulleted). Expect:
+
+- 80–200 lines per substantive meeting
+- 40–120 lines per substantive email thread
+- 30–80 lines per Teams chat thread with activity
+- 50–150 lines per OneNote page with substantive edits
+- 30–100 lines per SharePoint file body
+- 50–200 lines per CRM/ADO record update
+
+A 10-line CSC block for a 90-minute meeting is a defect. So is a 500-line block for a
+3-message email thread — the latter signals padding / fabrication.
+
+## Anti-patterns (defects)
+
+1. **Prose paragraph anywhere in a CSC block.** Re-extract as bullets.
+2. **AI Narrative Summary section.** Removed in v4.9.0 — Topics Discussed + Who Said What
+   carry the load. Re-extract.
+3. **"and N more" / truncation markers.** Doubled-strict retry, then defer.
+4. **Omitted sections that should be `_None surfaced._`.** Add them back.
+5. **Section misorder.** Use the canonical order above (Participants → Topics → Q&A →
+   Who Said What → Decisions → Dates & Numbers → Action Items → Next Steps → Open Questions →
+   Risks → Customer Asks → Artifacts → Coverage Notes).
+6. **Inferred narrative for missing bodies.** When body is unavailable, write
+   `Source basis: unavailable` + `_None surfaced._` for content sections. Don't make it up.
+7. **Verbatim quote reproduction outside meetings/verbatim/.** Quote in `Who Said What`
+   sparingly for material statements only; never copy the full transcript into the weekly
+   file — that lives in `meetings/verbatim/<dir>/transcript.txt`.
+8. **Bullets without citations** on fact-bearing sections.
+9. **`Who Said What` as 50-row utterance log.** Curate to material only.
+
+## Pre-write checklist (every pull-* skill)
+
+Before writing a CSC block, confirm:
+
+- [ ] Section order matches the canonical order
+- [ ] Every applicable section is present (empty → `_None surfaced._`)
+- [ ] Inapplicable sections are omitted entirely (not stubbed)
+- [ ] Every fact-bearing bullet has a citation
+- [ ] No prose paragraphs anywhere
+- [ ] No `and N more` / truncation markers
+- [ ] If body was unavailable, `Source basis: unavailable` + content sections are `_None surfaced._`, not inferred
+- [ ] `Who Said What` curated to material statements only
+- [ ] Entity anchor `{#slug}` is set on the heading for citation targeting
+
+## Self-check enforcement
+
+`plugin/skills/self-check/run.ps1` deep checks D11–D16 (rewritten in v4.9.0):
+- **D11**: every weekly CSC file has at least one entity block.
+- **D12**: every entity block has the canonical section order.
+- **D13**: meetings stream block has a sibling `verbatim/<dir>/` folder (kept from v3.10.0).
+- **D14**: no prose paragraphs in weekly CSC files (heuristic: lines without leading `-`, `*`,
+  `#`, `>`, or empty, longer than 200 chars).
+- **D15**: no `and N more`, `key highlights only`, `sample of`, `N of M shown` markers
+  outside coverage notes.
+- **D16**: `_index/entities.yml` exists for every populated weekly/ folder.
+
+## Apply
+
+Every `pull-<source>` SKILL.md MUST reference this instruction in its front contracts
+blockquote alongside `weekly-csc.instructions.md`. The two are co-equal.
+
+Cross-references:
+- `weekly-csc.instructions.md` — defines the file layout CSC blocks live in.
+- `meetings-verbatim-required.instructions.md` — raw transcript audit folder (meetings-only).
+- `citation-ledger.instructions.md` — citation format.
+- `deferred-retry-on-workiq-fail.instructions.md` — what to do when WorkIQ truncates.
+- `evidence-thoroughness.instructions.md` — rewritten in v4.9.0 to point at this file.
+- `verbatim-by-default.instructions.md` — DEPRECATED in v4.9.0; redirects here.

package/plugin/instructions/evidence-layout-canonical.instructions.md

@@ -30,13 +30,13 @@ For every project under `<engagement-root>/`:
       .settings.yml
       refresh-reports/
       open-questions/
-      ado/        { snapshot/ , stream/ }
-      crm/        { snapshot/ , stream/ }
-      email/      { snapshot/ , stream/ , _index/ , _legacy_*/ }
-      meetings/   { snapshot/ , stream/ , verbatim/ }
-      onenote/    { snapshot/ , stream/ , refresh-reports/ }
-      sharepoint/ { snapshot/ , stream/ }
-      teams/      { snapshot/ , stream/ }
+      ado/        { weekly/ , _index/ }                                     # legacy-read: snapshot/, stream/
+      crm/        { weekly/ , _index/ }                                     # legacy-read: snapshot/, stream/
+      email/      { weekly/ , _index/ , _legacy_*/ }                        # legacy-read: snapshot/, stream/
+      meetings/   { weekly/ , _index/ , verbatim/ }                         # verbatim/ is required (expiring source); legacy-read: snapshot/, stream/
+      onenote/    { weekly/ , _index/ , refresh-reports/ }                  # legacy-read: snapshot/, stream/
+      sharepoint/ { weekly/ , _index/ }                                     # legacy-read: snapshot/, stream/
+      teams/      { weekly/ , _index/ }                                     # legacy-read: snapshot/, stream/
   State/                         # rendered by build-state (read-only outcome)
   Reports/                       # rendered by aggregate / fde-report (read-only)
   integrations.yml               # per-project boundaries (scope-boundaries.instructions.md)
@@ -44,21 +44,27 @@ For every project under `<engagement-root>/`:
 
 `Evidence/` and `<alias>/` are the **only** levels a pull skill may create under `<project>/`. Nothing else.
 
+**v4.9.0 shape change:** pull-* skills now write a single `weekly/` folder per source + a thin
+`_index/entities.yml`, per `weekly-csc.instructions.md`. The old `snapshot/` + `stream/`
+folders are **legacy** — readers may still read them (for graceful migration), but no new
+writes go to them. `meetings/verbatim/<dir>/` is kept (the only expiring source). See
+`weekly-csc.instructions.md` for the writer/reader contracts.
+
 ### Rule 2 — Sibling source-output folders under `<project>/` are FORBIDDEN
 
 Concretely, a pull skill MUST NOT create or write to any of these (representative — list is illustrative, not exhaustive):
 
 | Forbidden path | Canonical replacement |
 |---|---|
-| `<project>/email-context/` | `<project>/Evidence/<alias>/email/snapshot/` + `stream/` |
+| `<project>/email-context/` | `<project>/Evidence/<alias>/email/weekly/` |
 | `<project>/email/` (at project root) | `<project>/Evidence/<alias>/email/` |
-| `<project>/notes/` | `<project>/Evidence/<alias>/onenote/snapshot/pages/` |
-| `<project>/_Weekly Summaries/` | `<project>/Evidence/<alias>/<source>/stream/` |
+| `<project>/notes/` | `<project>/Evidence/<alias>/onenote/weekly/` |
+| `<project>/_Weekly Summaries/` | `<project>/Evidence/<alias>/<source>/weekly/` |
 | `<project>/Meetings/` (at project root) | `<project>/Evidence/<alias>/meetings/` |
 | `<project>/Teams/` (at project root) | `<project>/Evidence/<alias>/teams/` |
 | `<project>/SharePoint/` (at project root) | `<project>/Evidence/<alias>/sharepoint/` |
 | `<project>/CRM/`, `<project>/ADO/` (at project root) | `<project>/Evidence/<alias>/{crm,ado}/` |
-| any `<project>/<source>-context/`, `<project>/<source>-summary/`, etc. | `<project>/Evidence/<alias>/<source>/snapshot/` |
+| any `<project>/<source>-context/`, `<project>/<source>-summary/`, etc. | `<project>/Evidence/<alias>/<source>/weekly/` |
 
 `State/`, `Reports/`, `integrations.yml` are the **only** top-level siblings of `Evidence/` that pull/refresh skills are allowed to leave alone (they are written by `build-state`, `aggregate-project`, and bootstrap respectively — not by pull-* skills).
 
@@ -103,7 +109,9 @@ Together they guarantee that two contributors running the same verb on the same
 
 ## Cross-references
 
-- `snapshot-vs-stream.instructions.md` — the two shapes inside each `<source>/` folder.
+- `weekly-csc.instructions.md` — the v4.9.0 layout inside each `<source>/` folder (weekly/ + _index/).
+- `snapshot-vs-stream.instructions.md` — DEPRECATED v4.9.0; legacy two-folder model.
+- `comprehensive-structured-capture.instructions.md` — the v4.9.0 block shape inside weekly/ files.
 - `scope-boundaries.instructions.md` — what each source is allowed to query (orthogonal: scope vs path).
 - `side-by-side-config.instructions.md` — config files (mutable hints, integrations.yml) live under `<workspace>/.kushi/config/` (v4.4.0+, was `<engagement-root>/.project-evidence/`).
 - `run-reports.instructions.md` — every layout-migration MUST appear in the refresh report.