kushi-agents 3.4.2 → 3.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/.github/copilot-instructions.kushi.md +38 -0
  2. package/README.md +33 -0
  3. package/bin/cli.mjs +2 -0
  4. package/package.json +17 -4
  5. package/plugin/agents/kushi.agent.md +155 -147
  6. package/plugin/instructions/ado-bootstrap-discovery.instructions.md +111 -0
  7. package/plugin/instructions/ado-engagement-tree.instructions.md +73 -0
  8. package/plugin/instructions/answer-from-evidence.instructions.md +1 -1
  9. package/plugin/instructions/auth-and-retry.instructions.md +51 -16
  10. package/plugin/instructions/azure-auth-patterns.instructions.md +13 -6
  11. package/plugin/instructions/bootstrap-status-format.instructions.md +113 -0
  12. package/plugin/instructions/capture-learnings.instructions.md +95 -0
  13. package/plugin/instructions/cleanup-on-resolution.instructions.md +69 -0
  14. package/plugin/instructions/crm-bootstrap-discovery.instructions.md +79 -0
  15. package/plugin/instructions/crm-internal-vs-confirmed.instructions.md +79 -0
  16. package/plugin/instructions/evidence-confidence-ladder.instructions.md +66 -0
  17. package/plugin/instructions/evidence-layout-canonical.instructions.md +115 -0
  18. package/plugin/instructions/evidence-thoroughness.instructions.md +82 -12
  19. package/plugin/instructions/full-view-gate.instructions.md +91 -0
  20. package/plugin/instructions/m365-id-registry.instructions.md +134 -0
  21. package/plugin/instructions/meetings-verbatim-required.instructions.md +176 -0
  22. package/plugin/instructions/run-reports.instructions.md +129 -0
  23. package/plugin/instructions/scope-boundaries.instructions.md +218 -0
  24. package/plugin/instructions/snapshot-vs-stream.instructions.md +2 -0
  25. package/plugin/instructions/update-ledger.instructions.md +132 -0
  26. package/plugin/instructions/verbatim-by-default.instructions.md +73 -0
  27. package/plugin/instructions/workiq-first.instructions.md +15 -31
  28. package/plugin/instructions/workiq-only.instructions.md +193 -0
  29. package/plugin/learnings/README.md +50 -0
  30. package/plugin/learnings/ado.md +45 -0
  31. package/plugin/learnings/crm.md +96 -0
  32. package/plugin/learnings/cross-cutting.md +36 -0
  33. package/plugin/learnings/email.md +33 -0
  34. package/plugin/learnings/meetings.md +30 -0
  35. package/plugin/learnings/misc.md +46 -0
  36. package/plugin/learnings/onenote.md +215 -0
  37. package/plugin/learnings/sharepoint.md +5 -0
  38. package/plugin/learnings/teams.md +5 -0
  39. package/plugin/plugin.json +22 -2
  40. package/plugin/prompts/apply-ado.prompt.md +14 -0
  41. package/plugin/prompts/propose-ado.prompt.md +12 -0
  42. package/plugin/reference-packs/fde/crm-field-manifest.md +165 -0
  43. package/plugin/skills/apply-ado-update/SKILL.md +125 -0
  44. package/plugin/skills/ask-project/SKILL.md +2 -0
  45. package/plugin/skills/bootstrap-project/SKILL.md +81 -3
  46. package/plugin/skills/propose-ado-update/SKILL.md +108 -0
  47. package/plugin/skills/pull-ado/SKILL.md +173 -23
  48. package/plugin/skills/pull-crm/SKILL.md +168 -15
  49. package/plugin/skills/pull-email/SKILL.md +139 -22
  50. package/plugin/skills/pull-meetings/SKILL.md +109 -25
  51. package/plugin/skills/pull-misc/README.md +84 -0
  52. package/plugin/skills/pull-misc/SKILL.md +257 -0
  53. package/plugin/skills/pull-misc/runner.mjs +280 -0
  54. package/plugin/skills/pull-onenote/README.md +90 -0
  55. package/plugin/skills/pull-onenote/SKILL.md +400 -51
  56. package/plugin/skills/pull-onenote/runner.mjs +356 -0
  57. package/plugin/skills/pull-onenote/scripts/recapture-section-url.mjs +295 -0
  58. package/plugin/skills/pull-onenote/write-snapshot.mjs +271 -0
  59. package/plugin/skills/pull-sharepoint/SKILL.md +44 -12
  60. package/plugin/skills/pull-teams/SKILL.md +40 -11
  61. package/plugin/skills/refresh-project/SKILL.md +33 -2
  62. package/plugin/skills/self-check/run.ps1 +186 -4
  63. package/plugin/templates/ado-update/discussion-comment.template.md +26 -0
  64. package/plugin/templates/ado-update/integrations-ado-writes.example.yml +49 -0
  65. package/plugin/templates/ado-update/proposed.template.md +78 -0
  66. package/plugin/templates/init/external-links.template.txt +30 -0
  67. package/plugin/templates/init/project-integrations.template.yml +57 -2
  68. package/plugin/templates/snapshot/meeting-verbatim.template.md +110 -0
  69. package/plugin/templates/snapshot/meetings-series-index.template.md +3 -1
  70. package/plugin/templates/snapshot/onenote-page.template.md +92 -23
  71. package/plugin/templates/weekly/meetings-stream.template.md +11 -6
  72. package/src/copilot-instructions.mjs +80 -0
  73. package/src/main.mjs +18 -1
@@ -1,17 +1,29 @@
1
1
  ---
2
2
  name: "pull-meetings"
3
- version: "2.1.0"
4
- description: "Pull Meetings evidence (snapshot: series-index; stream: per-meeting transcripts + decisions + actions). WorkIQ-first; falls back to chat reconstruction when transcripts don't exist."
3
+ version: "2.2.1"
4
+ description: "Pull Meetings evidence in THREE shapes: snapshot/ series-index, stream/ per-meeting curated blocks, AND verbatim/ raw immutable folder per meeting (meetings expire — verbatim/ MUST contain the full transcript text, not just chat). v2.4.0: WorkIQ-ONLY for transcript capture per workiq-only.instructions.md (m365_get_transcript / Graph REST FORBIDDEN — they have near-100% failure rate in this workspace). Chat is parallel supporting evidence (via m365_list_chat_messages structured dump), NEVER a transcript substitute."
5
5
  ---
6
6
 
7
7
  # Skill: pull-meetings
8
8
 
9
- Pulls **meetings** evidence in two shapes per `snapshot-vs-stream.instructions.md`:
10
9
 
11
- - **snapshot/** series-index.md listing all recurring meeting series for this project (subject, recurrence, organizer, current attendees)
12
- - **stream/**per-meeting blocks: attendees (req/opt/actual), agenda, **chronological transcript walk-through with verbatim quotes + timestamps**, decisions, actions, open questions, artifact links
10
+ > **v3.7.6 + v3.10.0 + v3.11.0 contracts** This skill operates under six HARD-rule doctrines:
11
+ > - `verbatim-by-default.instructions.md`full bodies/notetext/fields by default; no preview-grade pulls accepted.
12
+ > - `meetings-verbatim-required.instructions.md` (v3.10.0) — meetings are an EXPIRING evidence class; every captured meeting MUST also produce a sibling `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` folder with raw chat + transcript + recording URL. Curated snapshot alone is a defect.
13
+ > - **`workiq-only.instructions.md` (v3.11.0)** — transcripts, facilitator notes, calendar discovery, and chat-thread human rendering go through WorkIQ ONLY. `m365_get_transcript`, `m365_get_facilitator_notes`, `m365_list_meetings`, `m365_list_events`, and Graph REST URLs are FORBIDDEN as fallbacks (they fail nearly every call). The canonical WorkIQ prompts are codified in that instruction — do not re-discover them.
14
+ > - `capture-learnings.instructions.md` — every fix/discovery is logged to `plugin/learnings/<source>.md` immediately.
15
+ > - `cleanup-on-resolution.instructions.md` — when a value resolves, all stale `no-match` / `not yet` notes referencing the prior unresolved state must be rewritten in the same turn.
16
+ > - `run-reports.instructions.md` — every refresh writes a per-user report under `Evidence/<alias>/refresh-reports/YYYY-MM-DD-HHMM_refresh.md`.
13
17
 
14
- Auth + retry + error logging per `auth-and-retry.instructions.md`. WorkIQ-first per `workiq-first.instructions.md`. Thoroughness per `evidence-thoroughness.instructions.md`; runtime detector + auto-retry + paste-prompt per `thoroughness-detector.instructions.md`. Citations per `citation-ledger.instructions.md`.
18
+ > **Canonical evidence layout** (HARD, kushi v3.12.1+): all artifacts produced by this skill MUST be written under `<project>/Evidence/<alias>/<source>/{snapshot,stream,...}/` sibling folders under `<project>/` (e.g. `<project>/<source>-context/`, `<project>/<source>/`, `<project>/_Weekly Summaries/`) are FORBIDDEN. See `evidence-layout-canonical.instructions.md`.
19
+
20
+ Pulls **meetings** evidence in three shapes per `snapshot-vs-stream.instructions.md` + `meetings-verbatim-required.instructions.md`:
21
+
22
+ - **snapshot/** — `series-index.md` listing all recurring meeting series for this project (subject, recurrence, organizer, current attendees)
23
+ - **stream/** — per-meeting curated blocks: attendees (req/opt/actual), agenda, **chronological transcript walk-through with verbatim quotes + timestamps**, decisions, actions, open questions, artifact links. Each block MUST cite the matching verbatim/ folder.
24
+ - **verbatim/** (REQUIRED, NEW v2.2.0) — per-meeting subfolder `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` containing the raw immutable capture: `captured-at.txt`, `chat-messages.json`, `chat-messages.md`, `transcript.vtt` or `transcript-source.md`, `recording-url.txt`, `recap-card.md`, `attachments/`, `coverage.md`. See `templates/snapshot/meeting-verbatim.template.md` for the contract.
25
+
26
+ Auth + retry + error logging per `auth-and-retry.instructions.md`. WorkIQ-only per `workiq-only.instructions.md`. Thoroughness per `evidence-thoroughness.instructions.md`; runtime detector + auto-retry + paste-prompt per `thoroughness-detector.instructions.md`. Citations per `citation-ledger.instructions.md`.
15
27
 
16
28
  ## Inputs
17
29
 
@@ -20,39 +32,110 @@ Auth + retry + error logging per `auth-and-retry.instructions.md`. WorkIQ-first
20
32
  - `<window>` — date range. For snapshot: ignored (always full re-fetch). For stream: `(from, to)`.
21
33
  - (read) `<engagement-root>/.project-evidence/m365/m365-mutable.json m365Mutable.knownSections.<project>` — pinned hints (`calendarContext.subjectKeywords`, `calendarContext.knownSeries`).
22
34
 
23
- ## Discovery (snapshot pass)
35
+ ## Discovery (snapshot pass — WorkIQ ONLY per `workiq-only.instructions.md`)
24
36
 
25
- 1. `m365_list_meetings` over the window with `subjectKeywords` filter (post-filter — Graph search is relevance-ranked, not exact).
26
- 2. For each match: capture `id`, `subject`, `start`, `end`, `organizer`, `joinUrl`. Persist to mutable as `calendarContext.knownSeries[]`.
27
- 3. Cross-reference with Teams chat IDs from `pull-teams` (meeting chat IDs of form `19:meeting_…@thread.v2`). Each match enriches with attendee roster + chat ID.
37
+ 1. **WorkIQ meeting discovery** (canonical prompt from `workiq-only.instructions.md` Calendar/online meetings section):
38
+ > `List my Teams meetings between <start> and <end> where the subject contains "<token>". Return subject, date, start time, organizer, joinUrl, and Teams chat id.`
39
+ - Issued ONCE per token in `calendarContext.subjectKeywords`.
40
+ - For each match: capture `subject`, `start`, `end`, `organizer`, `joinUrl`, `chatId`. Persist to mutable as `calendarContext.knownSeries[]`.
41
+ 2. Cross-reference returned `chatId` values with `boundaries.teams.chat_ids[]` — every meeting whose chat-id falls inside the teams boundary is treated as in-scope. This enriches each meeting with the attendee roster the `pull-teams` snapshot already captured.
42
+
43
+ **FORBIDDEN** for discovery: `m365_list_meetings`, `m365_list_events`, Graph REST `/me/calendar/events`, Graph REST `/me/onlineMeetings`. Use WorkIQ.
28
44
 
29
45
  Write `snapshot/series-index.md` listing all matched series. Updated every refresh (organizers/attendees may change).
30
46
 
31
- ## Per-meeting stream pass path cascade
47
+ ## Boundaries (REQUIREDsee `scope-boundaries.instructions.md`)
48
+
49
+ This skill REFUSES to query unless `<engagement-root>/<project>/integrations.yml#boundaries.meetings` is satisfied:
50
+
51
+ - `boundaries.meetings.series_join_urls` — REQUIRED, non-empty (pinned join URLs; no subject-keyword fuzz).
52
+ - `boundaries.meetings.organizer_emails` — optional additional filter.
53
+ - `boundaries.date_window_days` — defaults to 30 if absent.
54
+
55
+ The pre-existing `subjectKeywords` / `knownSeries` discovery loop is now a **bootstrap-time aid only** — it helps populate `boundaries.meetings.series_join_urls` once. At pull time, only meetings whose `joinUrl` is in the boundary list are processed.
56
+
57
+ Refusal message when boundary is missing:
58
+
59
+ ```
60
+ meetings-boundary-missing — add boundaries.meetings.series_join_urls to
61
+ <engagement-root>/<project>/integrations.yml. See doctrine in
62
+ plugin/instructions/scope-boundaries.instructions.md.
63
+ ```
64
+
65
+ ## Per-meeting capture cascade — verbatim/ FIRST, then curated stream
66
+
67
+ For EACH meeting in the window, the cascade has **two halves**: (A) write raw artifacts to `verbatim/<YYYY-MM-DD-HHMM>_<slug>/`, then (B) produce the curated stream block citing those verbatim files. Half A is mandatory before Half B per `meetings-verbatim-required.instructions.md`.
68
+
69
+ ### Half A — verbatim/ capture (REQUIRED, transcript-first, WorkIQ-only per workiq-only.instructions.md)
70
+
71
+ For every meeting, BEFORE writing any curated text:
72
+
73
+ 0. Compute slug + timestamp, create `Evidence/<alias>/meetings/verbatim/<YYYY-MM-DD-HHMM>_<slug>/`, write `captured-at.txt` (started_at, kushi_version).
32
74
 
33
- For EACH meeting in the window, attempt to capture transcript text via this cascade. Apply retry pattern per `auth-and-retry.instructions.md`. Record errors[] entry for each path attempted.
75
+ 1. **Transcript cascade WorkIQ-only, EXHAUSTIVE, in order. Do NOT stop at first weak result; record each path's result in `coverage.md`.** Per `meetings-verbatim-required.instructions.md` "Transcript capture cascade":
76
+ - **a. WorkIQ full-transcript pull (REQUIRED first attempt)** with the canonical prompt from `workiq-only.instructions.md`:
77
+ > `Find the Teams meeting titled "<subject>" that occurred on <YYYY-MM-DD>. Return the full transcript verbatim with speaker labels and timestamps. Do not summarize.`
78
+ - If WorkIQ returns full speaker-turn text (≥ ~10 distinct `Name:` lines) → strip `request-id:` prefix, save body as `transcript.txt` with header (source, query, request-id, fidelity).
79
+ - If WorkIQ returns a summary (paragraphs only, no speaker turns) → run the **doubled-strict retry** from `workiq-only.instructions.md` once. If still summary-only, save as `transcript-source.md` with the `WARNING: NOT a verbatim transcript` header.
80
+ - **b. WorkIQ Copilot recap (SUPPLEMENTARY, always attempt)**:
81
+ > `Get the Copilot meeting recap (decisions, action items, key points) for the Teams meeting "<subject>" on <YYYY-MM-DD>. Return the FULL recap card verbatim. Do not summarize.`
82
+ - Save as `facilitator-notes.md`. Supplementary; does NOT replace the transcript.
83
+ - **c. WorkIQ recording-URL discovery**:
84
+ > `For the meeting "<subject>" on <YYYY-MM-DD>, return the SharePoint Stream recording URL if it exists, and the calendar event body.`
85
+ - Save URL(s) to `recording-url.txt`. If a `.mp4` URL is present, downloading the recording binary is governed by the `pull-meetings` recording-download carve-out (size < 200MB) — this is the ONLY allowed `m365_download_file` call in any kushi pull-* skill, because the recording is binary media and WorkIQ does not download binaries.
86
+ - **d. User-paste fallback (first-class, NOT a degradation)** — when (a) returns empty/`body-unavailable` after doubled-strict retry AND (b) returns nothing usable: prompt the user to paste verbatim transcript or notes. Save to `transcript.txt` with header `User-pasted; WorkIQ returned no transcript on <ISO timestamp>`.
34
87
 
35
- 1. **WorkIQ**: `Get full transcript and Copilot recap for meeting "<subject>" on <date>`.
36
- 2. **`m365_get_transcript`** with the meeting's `joinUrl`.
37
- 3. **Reconstruction from chat** (NOT a failure — first-class fallback). When transcripts simply don't exist (transcription was off, or VTT not yet attached), reconstruct evidence from:
38
- - Meeting chat thread: `m365_list_chat_messages(chatId)` — captured already by `pull-teams`.
39
- - Pre/post-meeting context messages.
40
- - Any Copilot recap card posted to the chat.
41
- Mark the per-meeting block `Source basis: reconstructed from meeting chat (no transcript)`. This is **evidence**, not a gap.
42
- 4. **Ask user**: paste verbatim transcript or notes if all above produced nothing usable.
88
+ **FORBIDDEN** (do NOT attempt; do NOT log as cascade steps in coverage.md): `m365_get_transcript`, `m365_get_facilitator_notes`, `m365_list_meetings`, `m365_list_events`, Graph REST `/me/onlineMeetings/.../transcripts/.../content`. These have a near-100% failure rate in this workspace and pollute the coverage trail. Use WorkIQ steps (a)/(b)/(c).
43
89
 
44
- A meeting block built from chat reconstruction with attendees + key decisions + actions is **valid evidence**. Do NOT mark such a meeting as `failed`. Mark it `partial` only if NO source produced any content (no transcript AND no chat messages).
90
+ 2. **Chat capture (ALWAYS, parallel with step 1 — allowed exception to workiq-only)**: `m365_list_chat_messages(chatId)` → `chat-messages.json` (raw structured-data dump) + `chat-messages.md` (rendered, one heading-block per message). Chat is **supporting evidence**; it is **NEVER** a transcript substitute. If `m365_list_chat_messages` fails, run the WorkIQ chat-thread prompt as fallback (per `workiq-only.instructions.md` Teams chat thread section).
91
+
92
+ 3. Walk chat for attachments → `m365_download_file` each → `attachments/<original-name>` (binary download carve-out, same as recording.mp4).
93
+
94
+ 4. Walk chat for Copilot recap card → `recap-card.md` verbatim.
95
+
96
+ 5. **Classify** the verbatim folder per the doctrine:
97
+ - `transcript-complete` — at least one of `transcript.vtt` / `transcript.txt` / `transcript-source.md` is non-empty.
98
+ - `transcript-missing` — only `chat-messages.*` present; all WorkIQ cascade paths (1a–1d) returned empty.
99
+
100
+ 6. Write `coverage.md` with classification + per-path attempt log (every WorkIQ path attempted, with request-id, even successful ones) + source-basis classification for stream/.
101
+
102
+ 7. Update `captured-at.txt` with `completed_at` + `final_status` (one of `transcript-complete-text` / `transcript-complete-summary-only` / `transcript-missing-chat-only` / `unrecoverable`).
103
+
104
+ If 1a–1d ALL return empty AND step 2 chat is also empty AND user paste is declined → `unrecoverable`. Apply retry pattern per `auth-and-retry.instructions.md`.
105
+
106
+ ### Half B — curated stream/ block (cites verbatim/)
107
+
108
+ Build the per-meeting block in `stream/<week>_meetings-stream.md` using ONLY content already persisted in verbatim/. Every assertion cites a verbatim file. Preferred citation chain:
109
+ - `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript.vtt · <date>]` when transcript-complete-vtt.
110
+ - `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript.txt · <date>]` when transcript-complete-text.
111
+ - `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript-source.md · <date>]` when only Copilot summary (always note the warning).
112
+ - `[source: Evidence/<alias>/meetings/verbatim/<dir>/chat-messages.md · <date>]` only for assertions backed by chat (not transcript content).
113
+
114
+ `Source basis` line in the curated block reflects verbatim classification:
115
+ - `transcript (raw VTT)` — transcript.vtt present
116
+ - `transcript (plain text)` — transcript.txt present, no VTT
117
+ - `transcript (WorkIQ Copilot summary — NOT verbatim)` — only transcript-source.md
118
+ - `❌ no-transcript-recovered-chat-only` — only chat-messages.*
119
+ - `❌ unrecoverable` — only captured-at.txt + coverage.md
120
+
121
+ A meeting with `transcript-missing-chat-only` is valid evidence in a degraded sense — actions/decisions citeable from chat are still useful — but the curated block MUST flag the gap prominently and the run-log MUST record `transcript-unrecoverable` for that meeting.
122
+
123
+ ### Ask-user fallback
124
+
125
+ When the entire cascade (1a–1d + chat) produces nothing usable, ask the user to paste verbatim transcript or notes. Persist any paste to `verbatim/<dir>/transcript.txt` with header `User-pasted; automated paths returned no transcript on <ISO timestamp>`.
45
126
 
46
127
  ## Stream pass
47
128
 
48
- Per `evidence-thoroughness.instructions.md`: every meeting in the window gets a full per-meeting block. **A 30-line meetings file for a week with 2 meetings is a defect — expect 200+ lines.**
129
+ Per `evidence-thoroughness.instructions.md`: every meeting in the window gets a full per-meeting block whose **FIRST sub-section is an AI Narrative Summary (REQUIRED, 5+ paragraphs)** covering the whole meeting end-to-end — context, what was discussed in what order, who took which position, the reasoning, the back-and-forth, soft signals, sentiment, what landed and what stayed open (cited to transcript timestamps). Then attendees → agenda → Detailed Discussion Summary (topic-organized) → chronological transcript walk-through with verbatim quotes → Decisions → Open Questions → Next Steps → Action Items → Risks → Customer Asks → Coverage Notes. **A 30-line meetings file for a week with 2 meetings is a defect — expect 300+ lines (the AI Narrative Summary alone is typically 30-50 lines per meeting).** A per-meeting block missing the AI Narrative Summary as the first sub-section is a defect even if every other section is present.
49
130
 
50
131
  Write to: `<engagement-root>/<project>/Evidence/<alias>/Meetings/stream/<YYYY-MM-DD>_meetings-stream.md` (date = Monday of the ISO week).
51
132
 
52
- Use template: `templates/weekly/meetings-summary.template.md`
133
+ Use template: `templates/weekly/meetings-stream.template.md`
53
134
 
54
135
  If a week file already exists, MERGE (dedupe by event ID, append new events, keep existing).
55
136
 
137
+ **Verbatim sibling required.** Every per-meeting block written here MUST have a matching `Evidence/<alias>/meetings/verbatim/<YYYY-MM-DD-HHMM>_<slug>/` folder produced by Half A. The block's `Source basis` line and `Artifacts` section must cite the verbatim folder's relative path. Self-check D13 enforces this.
138
+
56
139
  ## Mutable hints to upsert (during the run, not at the end)
57
140
 
58
141
  If discovered, immediately write to `m365Mutable.knownSections.<project>.calendarContext` with `discoveredOn` + `confidence`:
@@ -73,5 +156,6 @@ After the pass:
73
156
  ## Stop conditions
74
157
 
75
158
  - Subject keyword resolution returns 0 candidates → ask user once for keywords, persist to mutable, continue.
76
- - A meeting has neither transcript NOR chat messages → write `❌ no source available` block, log error, continue.
77
- - All paths failed for ALL meetings in the window → mark source `failed`, write a single `❌ all paths failed` evidence file with actionable next step.
159
+ - A meeting has neither transcript NOR chat messages → write `verbatim/<dir>/coverage.md` documenting every failed path + write `❌ source-expired-or-unrecoverable` block in stream/, log error, continue.
160
+ - All paths failed for ALL meetings in the window → mark source `failed`, write a single `❌ all paths failed` evidence file with actionable next step. Verbatim/ folders for each meeting still get created with captured-at.txt + coverage.md (the empty-folder audit trail is itself evidence).
161
+ - A per-meeting block lacks a sibling `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` directory → **defect** per `meetings-verbatim-required.instructions.md`. Re-run Half A immediately; do NOT ship the curated block alone.
@@ -0,0 +1,84 @@
1
+ # pull-misc runner
2
+
3
+ Pulls evidence from a user-curated link list (`<project>/external-links.txt`). Handles types that don't fit a dedicated `pull-*` skill: Loop pages, public web pages, learn.microsoft.com / docs sites, GitHub repos, local files. Delegated types (onenote, sharepoint, ado) are recorded as `delegated` and pulled by their dedicated skills.
4
+
5
+ See `SKILL.md` for the full doctrine.
6
+
7
+ ## One-time setup
8
+
9
+ ```pwsh
10
+ # Install deps in the kushi repo
11
+ cd C:\Usha\ISERepos\kushi
12
+ npm install playwright jsdom @mozilla/readability
13
+ npx playwright install chromium
14
+
15
+ # (Loop links only) Seed the Playwright profile by reusing the OneNote profile
16
+ # If you've already done OneNote bootstrap, you're done — Loop reuses it.
17
+ node plugin/skills/pull-onenote/runner.mjs --bootstrap
18
+ ```
19
+
20
+ ## Per-project run
21
+
22
+ ```pwsh
23
+ node plugin/skills/pull-misc/runner.mjs `
24
+ --project "ABN AMRO" `
25
+ --links-file "C:\Users\ushak\OneDrive - Microsoft\ISE\Engagement Assets\ABN AMRO\external-links.txt" `
26
+ --engagement-root "C:\Users\ushak\OneDrive - Microsoft\ISE\Engagement Assets" `
27
+ --headless
28
+ ```
29
+
30
+ ## Output
31
+
32
+ Single JSON object on stdout:
33
+
34
+ ```json
35
+ {
36
+ "project": "ABN AMRO",
37
+ "linksFile": "...",
38
+ "runStatus": "ok | auth-required | partial",
39
+ "counts": {
40
+ "total": 12,
41
+ "captured": 8,
42
+ "placeholder": 2,
43
+ "delegated": 1,
44
+ "auth_required": 0,
45
+ "fetch_failed": 1,
46
+ "skipped_binary": 0
47
+ },
48
+ "links": [
49
+ { "type": "web", "title": "...", "url": "...", "last_status": "captured", "captured_via": "http",
50
+ "http_status": 200, "content_type": "text/html", "etag": "...", "last_modified_http": "...",
51
+ "body": "...", "char_count": 18420, "captured_at": "..." }
52
+ /* ... */
53
+ ]
54
+ }
55
+ ```
56
+
57
+ ## Filtering
58
+
59
+ ```pwsh
60
+ # Only loop links
61
+ node runner.mjs --project ABN ... --types loop
62
+
63
+ # Only specific titles (substring match)
64
+ node runner.mjs --project ABN ... --titles "Workshop,Architecture"
65
+
66
+ # Combine
67
+ node runner.mjs --project ABN ... --types web,loop --titles "Architecture"
68
+ ```
69
+
70
+ ## Scheduled / unattended runs
71
+
72
+ - Browser branch (loop) reuses `~/.copilot/playwright-profile/onenote/`. When MFA fires, runner marks loop links `auth-required` and exits cleanly. User does one interactive bootstrap; next scheduled run silent.
73
+ - HTTP branch needs no auth for public links. For sites behind SSO (auth'd Confluence, internal dashboards) it returns `auth-required` and is currently NOT supported — paste the content into a `file` link as a workaround, OR add a per-site auth handler in a future version.
74
+ - `placeholder` links surface in the run report every refresh until the user fills them in.
75
+
76
+ ## Troubleshooting
77
+
78
+ | Symptom | Cause | Fix |
79
+ |---|---|---|
80
+ | `Cannot find module 'jsdom'` | deps not installed | `npm install jsdom @mozilla/readability` in repo root |
81
+ | All loop links `auth-required` | profile expired | `node plugin/skills/pull-onenote/runner.mjs --bootstrap` |
82
+ | `web` link returns near-empty body | site is SPA / requires JS render | Add it as a `loop` type to use browser path; or paste content as `file` |
83
+ | `confluence` link returns auth-required | private Confluence, no anon access | not supported in v0.1; paste content as `file` |
84
+ | `pdf` link returns "text extraction not yet implemented" | pdfjs not bundled | text extract is a v0.2 item; manually extract for now |
@@ -0,0 +1,257 @@
1
+ ---
2
+ name: "pull-misc"
3
+ version: "2.0.1"
4
+ description: "Pull miscellaneous evidence from a user-curated link list (<project>/external-links.txt). Handles links that don't fit the per-source pull-* skills: Loop pages, public web pages, learn.microsoft.com / docs sites, GitHub repos, local files, anything else linkable. Routes by type: delegated types (onenote, sharepoint, ado) are skipped because dedicated pull-* skills handle them. Per-link retry registry tracks last_status across runs."
5
+ ---
6
+
7
+ # Skill: pull-misc
8
+
9
+ > **v3.9.0 contracts** — This skill operates under six HARD-rule doctrines:
10
+ > - `verbatim-by-default.instructions.md` — full bodies by default; no preview-grade pulls accepted.
11
+ > - `m365-id-registry.instructions.md` — discover-once / consume-deterministically (per-link retry registry under `misc_links[]`).
12
+ > - `capture-learnings.instructions.md` — every fix/discovery is logged to `plugin/learnings/misc.md`.
13
+ > - `cleanup-on-resolution.instructions.md` — when a placeholder URL resolves, all stale unavailable-markers are upgraded in the same turn.
14
+ > - `run-reports.instructions.md` — every refresh writes a per-user report under `Evidence/<alias>/refresh-reports/`.
15
+ > - `thoroughness-detector.instructions.md` — the per-link retry registry IS this skill's thoroughness contract.
16
+
17
+ > **Canonical evidence layout** (HARD, kushi v3.12.1+): all artifacts produced by this skill MUST be written under `<project>/Evidence/<alias>/<source>/{snapshot,stream,...}/` — sibling folders under `<project>/` (e.g. `<project>/<source>-context/`, `<project>/<source>/`, `<project>/_Weekly Summaries/`) are FORBIDDEN. See `evidence-layout-canonical.instructions.md`.
18
+
19
+ Pulls **misc** evidence in two shapes per `snapshot-vs-stream.instructions.md`:
20
+
21
+ - **snapshot/** — full content per link — one file per link with last-fetched + verbatim body
22
+ - **stream/** — change events (HTTP `last-modified` header drift, git commit deltas where applicable, Loop edit signals via WorkIQ when available)
23
+
24
+ ## Tools (in order)
25
+
26
+ 1. **Playwright (browser-scrape, persisted profile)** — for `loop` links. Reuses `~/.copilot/playwright-profile/onenote/` because it's the same M365 cookie scope. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `loop`.
27
+ 2. **HTTP fetch + Readability** — for `web` / `confluence` (anonymous) / `learn` / `docs` / `pdf` links. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `web`.
28
+ 3. **Local file read** — for `file` links pointing at project-relative paths. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `file`.
29
+ 4. **WorkIQ** — used only for stream/edit-event signals on `loop` links (Loop edit events surface in the M365 search index, same as OneNote stream events). NOT used for body retrieval — Loop bodies require the browser path.
30
+ 5. **Delegation** — `onenote`, `sharepoint`, `ado` are NOT pulled here; the runner records `captured_via: delegated` so the registry stays complete and the user can see which links flow through which skill. The actual capture is done by `pull-onenote`, `pull-sharepoint`, `pull-ado` respectively.
31
+
32
+ ## The link list (`<project>/external-links.txt`)
33
+
34
+ This file already exists in most projects (per memory: the External Links Context doctrine). pull-misc preserves the existing format **without breaking changes** and adds `loop` as a recognized type.
35
+
36
+ Format:
37
+
38
+ ```text
39
+ # Comments start with #
40
+ # One link per line:
41
+ <type>|<owner>|<title>|<url-or-path>|<notes>
42
+ ```
43
+
44
+ **Recognized types (v3.9.0):**
45
+
46
+ | Type | Routed to | Captured via |
47
+ |---|---|---|
48
+ | `onenote` | pull-onenote | delegated |
49
+ | `sharepoint` | pull-sharepoint | delegated |
50
+ | `ado` | pull-ado | delegated |
51
+ | `loop` **(new)** | pull-misc | browser (Playwright, OneNote profile) |
52
+ | `web` | pull-misc | http (fetch + readability) |
53
+ | `confluence` | pull-misc | http (anonymous) — auth'd Confluence not yet supported |
54
+ | `learn` | pull-misc | http |
55
+ | `docs` | pull-misc | http |
56
+ | `pdf` | pull-misc | http (binary download + text extract via pdfjs) |
57
+ | `github` | pull-misc | http (raw README + repo metadata) |
58
+ | `file` | pull-misc | file (local read; path is project-relative) |
59
+ | anything else | pull-misc | http (best-effort) |
60
+
61
+ **Placeholder URLs** matching `<PASTE_*_URL>` or `<TODO*>` are SKIPPED with `last_status: placeholder`. They surface in the run report so the user can fill them in.
62
+
63
+ ## Empirical contract
64
+
65
+ These three facts are HARD-rule:
66
+
67
+ 1. **The user's external-links.txt IS the source of truth for what counts as "misc evidence" for that project.** No fuzzy auto-discovery. If a link isn't in the file, it isn't pulled. Discovery is a separate (manual) step where the user paste links in.
68
+ 2. **Web fetches use HTTP first, browser only if HTTP fails or returns interactive shell.** SPAs that don't server-render (Loop, some Confluence, dashboards) drop to browser. Don't run a browser when curl works.
69
+ 3. **Loop pages MUST go through browser.** Same gap as OneNote — Loop has no Graph API for component content. The /loop skill already proved this; pull-misc reuses the same Playwright profile.
70
+
71
+ ## Pre-flight
72
+
73
+ ```pwsh
74
+ # A. Playwright profile (only required if external-links.txt contains loop links)
75
+ $prof = "$env:USERPROFILE\.copilot\playwright-profile\onenote"
76
+ if (-not (Test-Path $prof)) {
77
+ Write-Host "[pull-misc] Playwright profile not yet seeded — loop links will be skipped with last_status: auth-required."
78
+ Write-Host "[pull-misc] Run plugin/skills/pull-onenote/runner.mjs --bootstrap once to seed it."
79
+ }
80
+
81
+ # B. Node deps
82
+ $nodeDeps = @('playwright', 'jsdom', '@mozilla/readability')
83
+ # Runner checks at startup; if missing, exits with "missing-deps" status and lists them.
84
+ ```
85
+
86
+ ## Bootstrap discovery
87
+
88
+ For each project, the bootstrap step:
89
+
90
+ 1. **Locates `<project>/external-links.txt`.** If absent, write a starter template (same format as ABN AMRO has today) and mark `boundaries.misc.externalLinksPath` in `integrations.yml`.
91
+ 2. **Parses the file.** Skip comments, skip blank lines, skip placeholder URLs (`<PASTE_*_URL>`, `<TODO*>`).
92
+ 3. **Initializes `misc_links[]`** in `m365-mutable.json#knownSections.<projectKey>` with one entry per non-placeholder link, all `last_status: not-yet-attempted`.
93
+ 4. **Records** `boundaries.misc.linkCount` and `placeholderCount` to `bootstrap-status.md`.
94
+
95
+ ## Step A — enumerate (every refresh)
96
+
97
+ ```text
98
+ 1. Read external-links.txt; rebuild link inventory (in case the user added/removed links since last run).
99
+ 2. For each existing entry in misc_links[], match by (type, url) tuple — preserve attempts, last_attempt_at, snapshot_path.
100
+ 3. For new entries, create with last_status: not-yet-attempted.
101
+ 4. For deleted entries (in registry but no longer in file), mark last_status: removed and DO NOT delete the snapshot file (audit trail).
102
+ ```
103
+
104
+ ## Step B — fetch (per link)
105
+
106
+ The runner branches per `type`:
107
+
108
+ ### B.1 `loop` — browser
109
+
110
+ ```js
111
+ // pages.goto(loopUrl); detect login redirect → auth-required.
112
+ // Wait for primary canvas selector; for Loop, that's the Fluid container.
113
+ // Read text via document.body.innerText after settle.
114
+ ```
115
+
116
+ If login redirect detected: `last_status: auth-required`, `captured_via: browser`, exit early for this link.
117
+
118
+ ### B.2 `web` / `confluence` / `learn` / `docs` / `github` / unknown — HTTP
119
+
120
+ ```js
121
+ // fetch(url, { headers: { 'User-Agent': 'kushi-pull-misc/0.1' } });
122
+ // If 200 + HTML → run @mozilla/readability for main content extract.
123
+ // If 200 + text/* → use raw body.
124
+ // If 200 + application/pdf → buffer + pdfjs text extract.
125
+ // If 401/403 → last_status: auth-required (cannot resolve unattended for non-MS hosts).
126
+ // If 404 / DNS fail → last_status: fetch-failed with HTTP code.
127
+ // If 5xx → last_status: fetch-failed (transient — retry next refresh).
128
+ ```
129
+
130
+ ### B.3 `file` — local read
131
+
132
+ ```js
133
+ // path is project-relative; resolve under <engagement-root>/<project>/
134
+ // Read text; if binary, mark last_status: skipped-binary (use a different pipeline if needed).
135
+ ```
136
+
137
+ ### B.4 `onenote` / `sharepoint` / `ado` — delegate
138
+
139
+ Skip fetch. Write registry entry with `captured_via: delegated`, `delegated_to: pull-onenote|pull-sharepoint|pull-ado`. Per-skill captures already happen in their own pipelines.
140
+
141
+ ## Per-link retry registry
142
+
143
+ Stored at `m365-mutable.json#knownSections.<projectKey>.misc_links[]`. Schema:
144
+
145
+ ```jsonc
146
+ {
147
+ "type": "loop | web | learn | pdf | github | file | onenote | sharepoint | ado | ...",
148
+ "owner": "<as in external-links.txt>",
149
+ "title": "<as in external-links.txt>",
150
+ "url": "<as in external-links.txt>", // or path for type=file
151
+ "notes": "<as in external-links.txt>",
152
+ "last_status": "captured | placeholder | auth-required | fetch-failed | skipped-binary | removed | not-yet-attempted | delegated",
153
+ "captured_via": "browser | http | file | delegated",
154
+ "delegated_to": "pull-onenote | pull-sharepoint | pull-ado", // only when captured_via=delegated
155
+ "http_status": 200, // only for HTTP
156
+ "content_type": "text/html", // only for HTTP
157
+ "char_count": 12345,
158
+ "attempts": 3,
159
+ "last_attempt_at": "<ISO-8601>",
160
+ "captured_at": "<ISO-8601, only when last_status=captured>",
161
+ "snapshot_path": "Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md",
162
+ "etag": "<HTTP ETag if returned>", // for change detection
163
+ "last_modified_http":"<HTTP Last-Modified if returned>"
164
+ }
165
+ ```
166
+
167
+ ## Snapshot files
168
+
169
+ One file per non-delegated link at:
170
+
171
+ ```
172
+ <project>/Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md
173
+ ```
174
+
175
+ Schema:
176
+
177
+ ```yaml
178
+ ---
179
+ type: web
180
+ owner: "Microsoft"
181
+ title: "Microsoft Fabric Documentation Hub"
182
+ url: "https://learn.microsoft.com/fabric/"
183
+ last_status: captured
184
+ captured_via: http
185
+ http_status: 200
186
+ content_type: text/html
187
+ char_count: 18420
188
+ captured_at: 2026-05-14T04:35:00Z
189
+ schema: v0.1.0-misc
190
+ ---
191
+
192
+ # {title}
193
+
194
+ > Source: {url}
195
+ > Captured via {captured_via} ({captured_at})
196
+ > Owner: {owner}
197
+ > Type: {type}
198
+
199
+ ## AI Narrative Summary
200
+
201
+ {1-2 paragraph summary derived ONLY from the body text below — never fabricate, never extrapolate beyond the captured content}
202
+
203
+ ## Body (verbatim)
204
+
205
+ ```
206
+ {full extracted text}
207
+ ```
208
+ ```
209
+
210
+ ## Step C — stream
211
+
212
+ For supported types, emit edit events:
213
+
214
+ - `web` / `learn` / `docs` / `pdf`: compare current `etag` / `last_modified_http` to registry; if changed since last refresh, emit `{ts, type, title, url, change: 'updated'}`.
215
+ - `loop`: WorkIQ has Loop edit-event signals via search index — query when available. Otherwise infer from body diff.
216
+ - `file`: compare file mtime; emit if changed.
217
+ - `github`: query repo commits since last `captured_at`; emit per-commit events.
218
+ - `onenote` / `sharepoint` / `ado`: NOT emitted here (their respective pull-* skills handle stream events for their content).
219
+
220
+ ## Run report
221
+
222
+ Every refresh writes one block to `Evidence/<alias>/refresh-reports/<date>.md`:
223
+
224
+ ```markdown
225
+ ### pull-misc — <project>
226
+
227
+ - external-links.txt: <N> total links (<P> placeholders, <D> delegated, <M> handled here)
228
+ - captured: <C> / <M>
229
+ - auth-required: <A>
230
+ - fetch-failed: <F>
231
+ - placeholders to fill: <list of titles where url contains <PASTE_*_URL>>
232
+ - removed since last run: <list>
233
+ - new since last run: <list>
234
+ ```
235
+
236
+ ## Anti-patterns (HARD)
237
+
238
+ 1. ❌ Auto-discovering links by crawling project SharePoint or Teams chats. The user's `external-links.txt` IS the boundary.
239
+ 2. ❌ Storing only `url` (not `(type, url)` tuple) as the registry key. Two types may share a URL (e.g. an onenote URL flagged as `web` for ad-hoc fetch test).
240
+ 3. ❌ Silently dropping a link from the registry when it disappears from `external-links.txt`. Mark `removed` and keep the snapshot for audit.
241
+ 4. ❌ Pulling delegated types (onenote / sharepoint / ado). Those have dedicated skills. Double-pulling creates conflicting evidence.
242
+ 5. ❌ Persisting placeholder URLs as `fetch-failed`. They're `placeholder`, surfaced in the run report so the user knows to fill them in.
243
+ 6. ❌ Loop link without Playwright profile silently writes empty snapshot. Must mark `auth-required`.
244
+
245
+ ## Self-check enforcement
246
+
247
+ `plugin/skills/self-check/run.ps1` D-token contract for `pull-misc`: the SKILL.md must contain `external-links`, `misc_links`, `placeholder`, `delegated`. These prove the skill carries the v0.1.0 link-list pipeline + delegation contract.
248
+
249
+ ## Citation
250
+
251
+ Citations from misc evidence MUST follow:
252
+
253
+ ```
254
+ [source: misc/<type>/<safe-title> · YYYY-MM-DD]
255
+ ```
256
+
257
+ Example: `[source: misc/loop/ABN-Core-Team-Sync-2026-03-27 · 2026-05-14]`