kushi-agents 3.4.2 → 3.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/copilot-instructions.kushi.md +38 -0
- package/README.md +33 -0
- package/bin/cli.mjs +2 -0
- package/package.json +17 -4
- package/plugin/agents/kushi.agent.md +155 -147
- package/plugin/instructions/ado-bootstrap-discovery.instructions.md +111 -0
- package/plugin/instructions/ado-engagement-tree.instructions.md +73 -0
- package/plugin/instructions/answer-from-evidence.instructions.md +1 -1
- package/plugin/instructions/auth-and-retry.instructions.md +51 -16
- package/plugin/instructions/azure-auth-patterns.instructions.md +13 -6
- package/plugin/instructions/bootstrap-status-format.instructions.md +113 -0
- package/plugin/instructions/capture-learnings.instructions.md +95 -0
- package/plugin/instructions/cleanup-on-resolution.instructions.md +69 -0
- package/plugin/instructions/crm-bootstrap-discovery.instructions.md +79 -0
- package/plugin/instructions/crm-internal-vs-confirmed.instructions.md +79 -0
- package/plugin/instructions/evidence-confidence-ladder.instructions.md +66 -0
- package/plugin/instructions/evidence-layout-canonical.instructions.md +115 -0
- package/plugin/instructions/evidence-thoroughness.instructions.md +82 -12
- package/plugin/instructions/full-view-gate.instructions.md +91 -0
- package/plugin/instructions/m365-id-registry.instructions.md +134 -0
- package/plugin/instructions/meetings-verbatim-required.instructions.md +176 -0
- package/plugin/instructions/run-reports.instructions.md +129 -0
- package/plugin/instructions/scope-boundaries.instructions.md +218 -0
- package/plugin/instructions/snapshot-vs-stream.instructions.md +2 -0
- package/plugin/instructions/update-ledger.instructions.md +132 -0
- package/plugin/instructions/verbatim-by-default.instructions.md +73 -0
- package/plugin/instructions/workiq-first.instructions.md +15 -31
- package/plugin/instructions/workiq-only.instructions.md +193 -0
- package/plugin/learnings/README.md +50 -0
- package/plugin/learnings/ado.md +45 -0
- package/plugin/learnings/crm.md +96 -0
- package/plugin/learnings/cross-cutting.md +36 -0
- package/plugin/learnings/email.md +33 -0
- package/plugin/learnings/meetings.md +30 -0
- package/plugin/learnings/misc.md +46 -0
- package/plugin/learnings/onenote.md +215 -0
- package/plugin/learnings/sharepoint.md +5 -0
- package/plugin/learnings/teams.md +5 -0
- package/plugin/plugin.json +22 -2
- package/plugin/prompts/apply-ado.prompt.md +14 -0
- package/plugin/prompts/propose-ado.prompt.md +12 -0
- package/plugin/reference-packs/fde/crm-field-manifest.md +165 -0
- package/plugin/skills/apply-ado-update/SKILL.md +125 -0
- package/plugin/skills/ask-project/SKILL.md +2 -0
- package/plugin/skills/bootstrap-project/SKILL.md +81 -3
- package/plugin/skills/propose-ado-update/SKILL.md +108 -0
- package/plugin/skills/pull-ado/SKILL.md +173 -23
- package/plugin/skills/pull-crm/SKILL.md +168 -15
- package/plugin/skills/pull-email/SKILL.md +139 -22
- package/plugin/skills/pull-meetings/SKILL.md +109 -25
- package/plugin/skills/pull-misc/README.md +84 -0
- package/plugin/skills/pull-misc/SKILL.md +257 -0
- package/plugin/skills/pull-misc/runner.mjs +280 -0
- package/plugin/skills/pull-onenote/README.md +90 -0
- package/plugin/skills/pull-onenote/SKILL.md +400 -51
- package/plugin/skills/pull-onenote/runner.mjs +356 -0
- package/plugin/skills/pull-onenote/scripts/recapture-section-url.mjs +295 -0
- package/plugin/skills/pull-onenote/write-snapshot.mjs +271 -0
- package/plugin/skills/pull-sharepoint/SKILL.md +44 -12
- package/plugin/skills/pull-teams/SKILL.md +40 -11
- package/plugin/skills/refresh-project/SKILL.md +33 -2
- package/plugin/skills/self-check/run.ps1 +186 -4
- package/plugin/templates/ado-update/discussion-comment.template.md +26 -0
- package/plugin/templates/ado-update/integrations-ado-writes.example.yml +49 -0
- package/plugin/templates/ado-update/proposed.template.md +78 -0
- package/plugin/templates/init/external-links.template.txt +30 -0
- package/plugin/templates/init/project-integrations.template.yml +57 -2
- package/plugin/templates/snapshot/meeting-verbatim.template.md +110 -0
- package/plugin/templates/snapshot/meetings-series-index.template.md +3 -1
- package/plugin/templates/snapshot/onenote-page.template.md +92 -23
- package/plugin/templates/weekly/meetings-stream.template.md +11 -6
- package/src/copilot-instructions.mjs +80 -0
- package/src/main.mjs +18 -1
|
@@ -1,17 +1,29 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "pull-meetings"
|
|
3
|
-
version: "2.1
|
|
4
|
-
description: "Pull Meetings evidence
|
|
3
|
+
version: "2.2.1"
|
|
4
|
+
description: "Pull Meetings evidence in THREE shapes: snapshot/ series-index, stream/ per-meeting curated blocks, AND verbatim/ raw immutable folder per meeting (meetings expire — verbatim/ MUST contain the full transcript text, not just chat). v2.4.0: WorkIQ-ONLY for transcript capture per workiq-only.instructions.md (m365_get_transcript / Graph REST FORBIDDEN — they have near-100% failure rate in this workspace). Chat is parallel supporting evidence (via m365_list_chat_messages structured dump), NEVER a transcript substitute."
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Skill: pull-meetings
|
|
8
8
|
|
|
9
|
-
Pulls **meetings** evidence in two shapes per `snapshot-vs-stream.instructions.md`:
|
|
10
9
|
|
|
11
|
-
|
|
12
|
-
-
|
|
10
|
+
> **v3.7.6 + v3.10.0 + v3.11.0 contracts** — This skill operates under six HARD-rule doctrines:
|
|
11
|
+
> - `verbatim-by-default.instructions.md` — full bodies/notetext/fields by default; no preview-grade pulls accepted.
|
|
12
|
+
> - `meetings-verbatim-required.instructions.md` (v3.10.0) — meetings are an EXPIRING evidence class; every captured meeting MUST also produce a sibling `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` folder with raw chat + transcript + recording URL. Curated snapshot alone is a defect.
|
|
13
|
+
> - **`workiq-only.instructions.md` (v3.11.0)** — transcripts, facilitator notes, calendar discovery, and chat-thread human rendering go through WorkIQ ONLY. `m365_get_transcript`, `m365_get_facilitator_notes`, `m365_list_meetings`, `m365_list_events`, and Graph REST URLs are FORBIDDEN as fallbacks (they fail nearly every call). The canonical WorkIQ prompts are codified in that instruction — do not re-discover them.
|
|
14
|
+
> - `capture-learnings.instructions.md` — every fix/discovery is logged to `plugin/learnings/<source>.md` immediately.
|
|
15
|
+
> - `cleanup-on-resolution.instructions.md` — when a value resolves, all stale `no-match` / `not yet` notes referencing the prior unresolved state must be rewritten in the same turn.
|
|
16
|
+
> - `run-reports.instructions.md` — every refresh writes a per-user report under `Evidence/<alias>/refresh-reports/YYYY-MM-DD-HHMM_refresh.md`.
|
|
13
17
|
|
|
14
|
-
|
|
18
|
+
> **Canonical evidence layout** (HARD, kushi v3.12.1+): all artifacts produced by this skill MUST be written under `<project>/Evidence/<alias>/<source>/{snapshot,stream,...}/` — sibling folders under `<project>/` (e.g. `<project>/<source>-context/`, `<project>/<source>/`, `<project>/_Weekly Summaries/`) are FORBIDDEN. See `evidence-layout-canonical.instructions.md`.
|
|
19
|
+
|
|
20
|
+
Pulls **meetings** evidence in three shapes per `snapshot-vs-stream.instructions.md` + `meetings-verbatim-required.instructions.md`:
|
|
21
|
+
|
|
22
|
+
- **snapshot/** — `series-index.md` listing all recurring meeting series for this project (subject, recurrence, organizer, current attendees)
|
|
23
|
+
- **stream/** — per-meeting curated blocks: attendees (req/opt/actual), agenda, **chronological transcript walk-through with verbatim quotes + timestamps**, decisions, actions, open questions, artifact links. Each block MUST cite the matching verbatim/ folder.
|
|
24
|
+
- **verbatim/** (REQUIRED, NEW v2.2.0) — per-meeting subfolder `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` containing the raw immutable capture: `captured-at.txt`, `chat-messages.json`, `chat-messages.md`, `transcript.vtt` or `transcript-source.md`, `recording-url.txt`, `recap-card.md`, `attachments/`, `coverage.md`. See `templates/snapshot/meeting-verbatim.template.md` for the contract.
|
|
25
|
+
|
|
26
|
+
Auth + retry + error logging per `auth-and-retry.instructions.md`. WorkIQ-only per `workiq-only.instructions.md`. Thoroughness per `evidence-thoroughness.instructions.md`; runtime detector + auto-retry + paste-prompt per `thoroughness-detector.instructions.md`. Citations per `citation-ledger.instructions.md`.
|
|
15
27
|
|
|
16
28
|
## Inputs
|
|
17
29
|
|
|
@@ -20,39 +32,110 @@ Auth + retry + error logging per `auth-and-retry.instructions.md`. WorkIQ-first
|
|
|
20
32
|
- `<window>` — date range. For snapshot: ignored (always full re-fetch). For stream: `(from, to)`.
|
|
21
33
|
- (read) `<engagement-root>/.project-evidence/m365/m365-mutable.json m365Mutable.knownSections.<project>` — pinned hints (`calendarContext.subjectKeywords`, `calendarContext.knownSeries`).
|
|
22
34
|
|
|
23
|
-
## Discovery (snapshot pass)
|
|
35
|
+
## Discovery (snapshot pass — WorkIQ ONLY per `workiq-only.instructions.md`)
|
|
24
36
|
|
|
25
|
-
1.
|
|
26
|
-
|
|
27
|
-
|
|
37
|
+
1. **WorkIQ meeting discovery** (canonical prompt from `workiq-only.instructions.md` Calendar/online meetings section):
|
|
38
|
+
> `List my Teams meetings between <start> and <end> where the subject contains "<token>". Return subject, date, start time, organizer, joinUrl, and Teams chat id.`
|
|
39
|
+
- Issued ONCE per token in `calendarContext.subjectKeywords`.
|
|
40
|
+
- For each match: capture `subject`, `start`, `end`, `organizer`, `joinUrl`, `chatId`. Persist to mutable as `calendarContext.knownSeries[]`.
|
|
41
|
+
2. Cross-reference returned `chatId` values with `boundaries.teams.chat_ids[]` — every meeting whose chat-id falls inside the teams boundary is treated as in-scope. This enriches each meeting with the attendee roster the `pull-teams` snapshot already captured.
|
|
42
|
+
|
|
43
|
+
**FORBIDDEN** for discovery: `m365_list_meetings`, `m365_list_events`, Graph REST `/me/calendar/events`, Graph REST `/me/onlineMeetings`. Use WorkIQ.
|
|
28
44
|
|
|
29
45
|
Write `snapshot/series-index.md` listing all matched series. Updated every refresh (organizers/attendees may change).
|
|
30
46
|
|
|
31
|
-
##
|
|
47
|
+
## Boundaries (REQUIRED — see `scope-boundaries.instructions.md`)
|
|
48
|
+
|
|
49
|
+
This skill REFUSES to query unless `<engagement-root>/<project>/integrations.yml#boundaries.meetings` is satisfied:
|
|
50
|
+
|
|
51
|
+
- `boundaries.meetings.series_join_urls` — REQUIRED, non-empty (pinned join URLs; no subject-keyword fuzz).
|
|
52
|
+
- `boundaries.meetings.organizer_emails` — optional additional filter.
|
|
53
|
+
- `boundaries.date_window_days` — defaults to 30 if absent.
|
|
54
|
+
|
|
55
|
+
The pre-existing `subjectKeywords` / `knownSeries` discovery loop is now a **bootstrap-time aid only** — it helps populate `boundaries.meetings.series_join_urls` once. At pull time, only meetings whose `joinUrl` is in the boundary list are processed.
|
|
56
|
+
|
|
57
|
+
Refusal message when boundary is missing:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
meetings-boundary-missing — add boundaries.meetings.series_join_urls to
|
|
61
|
+
<engagement-root>/<project>/integrations.yml. See doctrine in
|
|
62
|
+
plugin/instructions/scope-boundaries.instructions.md.
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Per-meeting capture cascade — verbatim/ FIRST, then curated stream
|
|
66
|
+
|
|
67
|
+
For EACH meeting in the window, the cascade has **two halves**: (A) write raw artifacts to `verbatim/<YYYY-MM-DD-HHMM>_<slug>/`, then (B) produce the curated stream block citing those verbatim files. Half A is mandatory before Half B per `meetings-verbatim-required.instructions.md`.
|
|
68
|
+
|
|
69
|
+
### Half A — verbatim/ capture (REQUIRED, transcript-first, WorkIQ-only per workiq-only.instructions.md)
|
|
70
|
+
|
|
71
|
+
For every meeting, BEFORE writing any curated text:
|
|
72
|
+
|
|
73
|
+
0. Compute slug + timestamp, create `Evidence/<alias>/meetings/verbatim/<YYYY-MM-DD-HHMM>_<slug>/`, write `captured-at.txt` (started_at, kushi_version).
|
|
32
74
|
|
|
33
|
-
|
|
75
|
+
1. **Transcript cascade — WorkIQ-only, EXHAUSTIVE, in order. Do NOT stop at first weak result; record each path's result in `coverage.md`.** Per `meetings-verbatim-required.instructions.md` "Transcript capture cascade":
|
|
76
|
+
- **a. WorkIQ full-transcript pull (REQUIRED first attempt)** with the canonical prompt from `workiq-only.instructions.md`:
|
|
77
|
+
> `Find the Teams meeting titled "<subject>" that occurred on <YYYY-MM-DD>. Return the full transcript verbatim with speaker labels and timestamps. Do not summarize.`
|
|
78
|
+
- If WorkIQ returns full speaker-turn text (≥ ~10 distinct `Name:` lines) → strip `request-id:` prefix, save body as `transcript.txt` with header (source, query, request-id, fidelity).
|
|
79
|
+
- If WorkIQ returns a summary (paragraphs only, no speaker turns) → run the **doubled-strict retry** from `workiq-only.instructions.md` once. If still summary-only, save as `transcript-source.md` with the `WARNING: NOT a verbatim transcript` header.
|
|
80
|
+
- **b. WorkIQ Copilot recap (SUPPLEMENTARY, always attempt)**:
|
|
81
|
+
> `Get the Copilot meeting recap (decisions, action items, key points) for the Teams meeting "<subject>" on <YYYY-MM-DD>. Return the FULL recap card verbatim. Do not summarize.`
|
|
82
|
+
- Save as `facilitator-notes.md`. Supplementary; does NOT replace the transcript.
|
|
83
|
+
- **c. WorkIQ recording-URL discovery**:
|
|
84
|
+
> `For the meeting "<subject>" on <YYYY-MM-DD>, return the SharePoint Stream recording URL if it exists, and the calendar event body.`
|
|
85
|
+
- Save URL(s) to `recording-url.txt`. If a `.mp4` URL is present, downloading the recording binary is governed by the `pull-meetings` recording-download carve-out (size < 200MB) — this is the ONLY allowed `m365_download_file` call in any kushi pull-* skill, because the recording is binary media and WorkIQ does not download binaries.
|
|
86
|
+
- **d. User-paste fallback (first-class, NOT a degradation)** — when (a) returns empty/`body-unavailable` after doubled-strict retry AND (b) returns nothing usable: prompt the user to paste verbatim transcript or notes. Save to `transcript.txt` with header `User-pasted; WorkIQ returned no transcript on <ISO timestamp>`.
|
|
34
87
|
|
|
35
|
-
|
|
36
|
-
2. **`m365_get_transcript`** with the meeting's `joinUrl`.
|
|
37
|
-
3. **Reconstruction from chat** (NOT a failure — first-class fallback). When transcripts simply don't exist (transcription was off, or VTT not yet attached), reconstruct evidence from:
|
|
38
|
-
- Meeting chat thread: `m365_list_chat_messages(chatId)` — captured already by `pull-teams`.
|
|
39
|
-
- Pre/post-meeting context messages.
|
|
40
|
-
- Any Copilot recap card posted to the chat.
|
|
41
|
-
Mark the per-meeting block `Source basis: reconstructed from meeting chat (no transcript)`. This is **evidence**, not a gap.
|
|
42
|
-
4. **Ask user**: paste verbatim transcript or notes if all above produced nothing usable.
|
|
88
|
+
**FORBIDDEN** (do NOT attempt; do NOT log as cascade steps in coverage.md): `m365_get_transcript`, `m365_get_facilitator_notes`, `m365_list_meetings`, `m365_list_events`, Graph REST `/me/onlineMeetings/.../transcripts/.../content`. These have a near-100% failure rate in this workspace and pollute the coverage trail. Use WorkIQ steps (a)/(b)/(c).
|
|
43
89
|
|
|
44
|
-
|
|
90
|
+
2. **Chat capture (ALWAYS, parallel with step 1 — allowed exception to workiq-only)**: `m365_list_chat_messages(chatId)` → `chat-messages.json` (raw structured-data dump) + `chat-messages.md` (rendered, one heading-block per message). Chat is **supporting evidence**; it is **NEVER** a transcript substitute. If `m365_list_chat_messages` fails, run the WorkIQ chat-thread prompt as fallback (per `workiq-only.instructions.md` Teams chat thread section).
|
|
91
|
+
|
|
92
|
+
3. Walk chat for attachments → `m365_download_file` each → `attachments/<original-name>` (binary download carve-out, same as recording.mp4).
|
|
93
|
+
|
|
94
|
+
4. Walk chat for Copilot recap card → `recap-card.md` verbatim.
|
|
95
|
+
|
|
96
|
+
5. **Classify** the verbatim folder per the doctrine:
|
|
97
|
+
- `transcript-complete` — at least one of `transcript.vtt` / `transcript.txt` / `transcript-source.md` is non-empty.
|
|
98
|
+
- `transcript-missing` — only `chat-messages.*` present; all WorkIQ cascade paths (1a–1d) returned empty.
|
|
99
|
+
|
|
100
|
+
6. Write `coverage.md` with classification + per-path attempt log (every WorkIQ path attempted, with request-id, even successful ones) + source-basis classification for stream/.
|
|
101
|
+
|
|
102
|
+
7. Update `captured-at.txt` with `completed_at` + `final_status` (one of `transcript-complete-text` / `transcript-complete-summary-only` / `transcript-missing-chat-only` / `unrecoverable`).
|
|
103
|
+
|
|
104
|
+
If 1a–1d ALL return empty AND step 2 chat is also empty AND user paste is declined → `unrecoverable`. Apply retry pattern per `auth-and-retry.instructions.md`.
|
|
105
|
+
|
|
106
|
+
### Half B — curated stream/ block (cites verbatim/)
|
|
107
|
+
|
|
108
|
+
Build the per-meeting block in `stream/<week>_meetings-stream.md` using ONLY content already persisted in verbatim/. Every assertion cites a verbatim file. Preferred citation chain:
|
|
109
|
+
- `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript.vtt · <date>]` when transcript-complete-vtt.
|
|
110
|
+
- `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript.txt · <date>]` when transcript-complete-text.
|
|
111
|
+
- `[source: Evidence/<alias>/meetings/verbatim/<dir>/transcript-source.md · <date>]` when only Copilot summary (always note the warning).
|
|
112
|
+
- `[source: Evidence/<alias>/meetings/verbatim/<dir>/chat-messages.md · <date>]` only for assertions backed by chat (not transcript content).
|
|
113
|
+
|
|
114
|
+
`Source basis` line in the curated block reflects verbatim classification:
|
|
115
|
+
- `transcript (raw VTT)` — transcript.vtt present
|
|
116
|
+
- `transcript (plain text)` — transcript.txt present, no VTT
|
|
117
|
+
- `transcript (WorkIQ Copilot summary — NOT verbatim)` — only transcript-source.md
|
|
118
|
+
- `❌ no-transcript-recovered-chat-only` — only chat-messages.*
|
|
119
|
+
- `❌ unrecoverable` — only captured-at.txt + coverage.md
|
|
120
|
+
|
|
121
|
+
A meeting with `transcript-missing-chat-only` is valid evidence in a degraded sense — actions/decisions citeable from chat are still useful — but the curated block MUST flag the gap prominently and the run-log MUST record `transcript-unrecoverable` for that meeting.
|
|
122
|
+
|
|
123
|
+
### Ask-user fallback
|
|
124
|
+
|
|
125
|
+
When the entire cascade (1a–1d + chat) produces nothing usable, ask the user to paste verbatim transcript or notes. Persist any paste to `verbatim/<dir>/transcript.txt` with header `User-pasted; automated paths returned no transcript on <ISO timestamp>`.
|
|
45
126
|
|
|
46
127
|
## Stream pass
|
|
47
128
|
|
|
48
|
-
Per `evidence-thoroughness.instructions.md`: every meeting in the window gets a full per-meeting block. **A 30-line meetings file for a week with 2 meetings is a defect — expect
|
|
129
|
+
Per `evidence-thoroughness.instructions.md`: every meeting in the window gets a full per-meeting block whose **FIRST sub-section is an AI Narrative Summary (REQUIRED, 5+ paragraphs)** covering the whole meeting end-to-end — context, what was discussed in what order, who took which position, the reasoning, the back-and-forth, soft signals, sentiment, what landed and what stayed open (cited to transcript timestamps). Then attendees → agenda → Detailed Discussion Summary (topic-organized) → chronological transcript walk-through with verbatim quotes → Decisions → Open Questions → Next Steps → Action Items → Risks → Customer Asks → Coverage Notes. **A 30-line meetings file for a week with 2 meetings is a defect — expect 300+ lines (the AI Narrative Summary alone is typically 30-50 lines per meeting).** A per-meeting block missing the AI Narrative Summary as the first sub-section is a defect even if every other section is present.
|
|
49
130
|
|
|
50
131
|
Write to: `<engagement-root>/<project>/Evidence/<alias>/Meetings/stream/<YYYY-MM-DD>_meetings-stream.md` (date = Monday of the ISO week).
|
|
51
132
|
|
|
52
|
-
Use template: `templates/weekly/meetings-
|
|
133
|
+
Use template: `templates/weekly/meetings-stream.template.md`
|
|
53
134
|
|
|
54
135
|
If a week file already exists, MERGE (dedupe by event ID, append new events, keep existing).
|
|
55
136
|
|
|
137
|
+
**Verbatim sibling required.** Every per-meeting block written here MUST have a matching `Evidence/<alias>/meetings/verbatim/<YYYY-MM-DD-HHMM>_<slug>/` folder produced by Half A. The block's `Source basis` line and `Artifacts` section must cite the verbatim folder's relative path. Self-check D13 enforces this.
|
|
138
|
+
|
|
56
139
|
## Mutable hints to upsert (during the run, not at the end)
|
|
57
140
|
|
|
58
141
|
If discovered, immediately write to `m365Mutable.knownSections.<project>.calendarContext` with `discoveredOn` + `confidence`:
|
|
@@ -73,5 +156,6 @@ After the pass:
|
|
|
73
156
|
## Stop conditions
|
|
74
157
|
|
|
75
158
|
- Subject keyword resolution returns 0 candidates → ask user once for keywords, persist to mutable, continue.
|
|
76
|
-
- A meeting has neither transcript NOR chat messages → write `❌
|
|
77
|
-
- All paths failed for ALL meetings in the window → mark source `failed`, write a single `❌ all paths failed` evidence file with actionable next step.
|
|
159
|
+
- A meeting has neither transcript NOR chat messages → write `verbatim/<dir>/coverage.md` documenting every failed path + write `❌ source-expired-or-unrecoverable` block in stream/, log error, continue.
|
|
160
|
+
- All paths failed for ALL meetings in the window → mark source `failed`, write a single `❌ all paths failed` evidence file with actionable next step. Verbatim/ folders for each meeting still get created with captured-at.txt + coverage.md (the empty-folder audit trail is itself evidence).
|
|
161
|
+
- A per-meeting block lacks a sibling `verbatim/<YYYY-MM-DD-HHMM>_<slug>/` directory → **defect** per `meetings-verbatim-required.instructions.md`. Re-run Half A immediately; do NOT ship the curated block alone.
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# pull-misc runner
|
|
2
|
+
|
|
3
|
+
Pulls evidence from a user-curated link list (`<project>/external-links.txt`). Handles types that don't fit a dedicated `pull-*` skill: Loop pages, public web pages, learn.microsoft.com / docs sites, GitHub repos, local files. Delegated types (onenote, sharepoint, ado) are recorded as `delegated` and pulled by their dedicated skills.
|
|
4
|
+
|
|
5
|
+
See `SKILL.md` for the full doctrine.
|
|
6
|
+
|
|
7
|
+
## One-time setup
|
|
8
|
+
|
|
9
|
+
```pwsh
|
|
10
|
+
# Install deps in the kushi repo
|
|
11
|
+
cd C:\Usha\ISERepos\kushi
|
|
12
|
+
npm install playwright jsdom @mozilla/readability
|
|
13
|
+
npx playwright install chromium
|
|
14
|
+
|
|
15
|
+
# (Loop links only) Seed the Playwright profile by reusing the OneNote profile
|
|
16
|
+
# If you've already done OneNote bootstrap, you're done — Loop reuses it.
|
|
17
|
+
node plugin/skills/pull-onenote/runner.mjs --bootstrap
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Per-project run
|
|
21
|
+
|
|
22
|
+
```pwsh
|
|
23
|
+
node plugin/skills/pull-misc/runner.mjs `
|
|
24
|
+
--project "ABN AMRO" `
|
|
25
|
+
--links-file "C:\Users\ushak\OneDrive - Microsoft\ISE\Engagement Assets\ABN AMRO\external-links.txt" `
|
|
26
|
+
--engagement-root "C:\Users\ushak\OneDrive - Microsoft\ISE\Engagement Assets" `
|
|
27
|
+
--headless
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Output
|
|
31
|
+
|
|
32
|
+
Single JSON object on stdout:
|
|
33
|
+
|
|
34
|
+
```json
|
|
35
|
+
{
|
|
36
|
+
"project": "ABN AMRO",
|
|
37
|
+
"linksFile": "...",
|
|
38
|
+
"runStatus": "ok | auth-required | partial",
|
|
39
|
+
"counts": {
|
|
40
|
+
"total": 12,
|
|
41
|
+
"captured": 8,
|
|
42
|
+
"placeholder": 2,
|
|
43
|
+
"delegated": 1,
|
|
44
|
+
"auth_required": 0,
|
|
45
|
+
"fetch_failed": 1,
|
|
46
|
+
"skipped_binary": 0
|
|
47
|
+
},
|
|
48
|
+
"links": [
|
|
49
|
+
{ "type": "web", "title": "...", "url": "...", "last_status": "captured", "captured_via": "http",
|
|
50
|
+
"http_status": 200, "content_type": "text/html", "etag": "...", "last_modified_http": "...",
|
|
51
|
+
"body": "...", "char_count": 18420, "captured_at": "..." }
|
|
52
|
+
/* ... */
|
|
53
|
+
]
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Filtering
|
|
58
|
+
|
|
59
|
+
```pwsh
|
|
60
|
+
# Only loop links
|
|
61
|
+
node runner.mjs --project ABN ... --types loop
|
|
62
|
+
|
|
63
|
+
# Only specific titles (substring match)
|
|
64
|
+
node runner.mjs --project ABN ... --titles "Workshop,Architecture"
|
|
65
|
+
|
|
66
|
+
# Combine
|
|
67
|
+
node runner.mjs --project ABN ... --types web,loop --titles "Architecture"
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Scheduled / unattended runs
|
|
71
|
+
|
|
72
|
+
- Browser branch (loop) reuses `~/.copilot/playwright-profile/onenote/`. When MFA fires, runner marks loop links `auth-required` and exits cleanly. User does one interactive bootstrap; next scheduled run silent.
|
|
73
|
+
- HTTP branch needs no auth for public links. For sites behind SSO (auth'd Confluence, internal dashboards) it returns `auth-required` and is currently NOT supported — paste the content into a `file` link as a workaround, OR add a per-site auth handler in a future version.
|
|
74
|
+
- `placeholder` links surface in the run report every refresh until the user fills them in.
|
|
75
|
+
|
|
76
|
+
## Troubleshooting
|
|
77
|
+
|
|
78
|
+
| Symptom | Cause | Fix |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `Cannot find module 'jsdom'` | deps not installed | `npm install jsdom @mozilla/readability` in repo root |
|
|
81
|
+
| All loop links `auth-required` | profile expired | `node plugin/skills/pull-onenote/runner.mjs --bootstrap` |
|
|
82
|
+
| `web` link returns near-empty body | site is SPA / requires JS render | Add it as a `loop` type to use browser path; or paste content as `file` |
|
|
83
|
+
| `confluence` link returns auth-required | private Confluence, no anon access | not supported in v0.1; paste content as `file` |
|
|
84
|
+
| `pdf` link returns "text extraction not yet implemented" | pdfjs not bundled | text extract is a v0.2 item; manually extract for now |
|
|
@@ -0,0 +1,257 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "pull-misc"
|
|
3
|
+
version: "2.0.1"
|
|
4
|
+
description: "Pull miscellaneous evidence from a user-curated link list (<project>/external-links.txt). Handles links that don't fit the per-source pull-* skills: Loop pages, public web pages, learn.microsoft.com / docs sites, GitHub repos, local files, anything else linkable. Routes by type: delegated types (onenote, sharepoint, ado) are skipped because dedicated pull-* skills handle them. Per-link retry registry tracks last_status across runs."
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Skill: pull-misc
|
|
8
|
+
|
|
9
|
+
> **v3.9.0 contracts** — This skill operates under six HARD-rule doctrines:
|
|
10
|
+
> - `verbatim-by-default.instructions.md` — full bodies by default; no preview-grade pulls accepted.
|
|
11
|
+
> - `m365-id-registry.instructions.md` — discover-once / consume-deterministically (per-link retry registry under `misc_links[]`).
|
|
12
|
+
> - `capture-learnings.instructions.md` — every fix/discovery is logged to `plugin/learnings/misc.md`.
|
|
13
|
+
> - `cleanup-on-resolution.instructions.md` — when a placeholder URL resolves, all stale unavailable-markers are upgraded in the same turn.
|
|
14
|
+
> - `run-reports.instructions.md` — every refresh writes a per-user report under `Evidence/<alias>/refresh-reports/`.
|
|
15
|
+
> - `thoroughness-detector.instructions.md` — the per-link retry registry IS this skill's thoroughness contract.
|
|
16
|
+
|
|
17
|
+
> **Canonical evidence layout** (HARD, kushi v3.12.1+): all artifacts produced by this skill MUST be written under `<project>/Evidence/<alias>/<source>/{snapshot,stream,...}/` — sibling folders under `<project>/` (e.g. `<project>/<source>-context/`, `<project>/<source>/`, `<project>/_Weekly Summaries/`) are FORBIDDEN. See `evidence-layout-canonical.instructions.md`.
|
|
18
|
+
|
|
19
|
+
Pulls **misc** evidence in two shapes per `snapshot-vs-stream.instructions.md`:
|
|
20
|
+
|
|
21
|
+
- **snapshot/** — full content per link — one file per link with last-fetched + verbatim body
|
|
22
|
+
- **stream/** — change events (HTTP `last-modified` header drift, git commit deltas where applicable, Loop edit signals via WorkIQ when available)
|
|
23
|
+
|
|
24
|
+
## Tools (in order)
|
|
25
|
+
|
|
26
|
+
1. **Playwright (browser-scrape, persisted profile)** — for `loop` links. Reuses `~/.copilot/playwright-profile/onenote/` because it's the same M365 cookie scope. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `loop`.
|
|
27
|
+
2. **HTTP fetch + Readability** — for `web` / `confluence` (anonymous) / `learn` / `docs` / `pdf` links. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `web`.
|
|
28
|
+
3. **Local file read** — for `file` links pointing at project-relative paths. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `file`.
|
|
29
|
+
4. **WorkIQ** — used only for stream/edit-event signals on `loop` links (Loop edit events surface in the M365 search index, same as OneNote stream events). NOT used for body retrieval — Loop bodies require the browser path.
|
|
30
|
+
5. **Delegation** — `onenote`, `sharepoint`, `ado` are NOT pulled here; the runner records `captured_via: delegated` so the registry stays complete and the user can see which links flow through which skill. The actual capture is done by `pull-onenote`, `pull-sharepoint`, `pull-ado` respectively.
|
|
31
|
+
|
|
32
|
+
## The link list (`<project>/external-links.txt`)
|
|
33
|
+
|
|
34
|
+
This file already exists in most projects (per memory: the External Links Context doctrine). pull-misc preserves the existing format **without breaking changes** and adds `loop` as a recognized type.
|
|
35
|
+
|
|
36
|
+
Format:
|
|
37
|
+
|
|
38
|
+
```text
|
|
39
|
+
# Comments start with #
|
|
40
|
+
# One link per line:
|
|
41
|
+
<type>|<owner>|<title>|<url-or-path>|<notes>
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Recognized types (v3.9.0):**
|
|
45
|
+
|
|
46
|
+
| Type | Routed to | Captured via |
|
|
47
|
+
|---|---|---|
|
|
48
|
+
| `onenote` | pull-onenote | delegated |
|
|
49
|
+
| `sharepoint` | pull-sharepoint | delegated |
|
|
50
|
+
| `ado` | pull-ado | delegated |
|
|
51
|
+
| `loop` **(new)** | pull-misc | browser (Playwright, OneNote profile) |
|
|
52
|
+
| `web` | pull-misc | http (fetch + readability) |
|
|
53
|
+
| `confluence` | pull-misc | http (anonymous) — auth'd Confluence not yet supported |
|
|
54
|
+
| `learn` | pull-misc | http |
|
|
55
|
+
| `docs` | pull-misc | http |
|
|
56
|
+
| `pdf` | pull-misc | http (binary download + text extract via pdfjs) |
|
|
57
|
+
| `github` | pull-misc | http (raw README + repo metadata) |
|
|
58
|
+
| `file` | pull-misc | file (local read; path is project-relative) |
|
|
59
|
+
| anything else | pull-misc | http (best-effort) |
|
|
60
|
+
|
|
61
|
+
**Placeholder URLs** matching `<PASTE_*_URL>` or `<TODO*>` are SKIPPED with `last_status: placeholder`. They surface in the run report so the user can fill them in.
|
|
62
|
+
|
|
63
|
+
## Empirical contract
|
|
64
|
+
|
|
65
|
+
These three facts are HARD-rule:
|
|
66
|
+
|
|
67
|
+
1. **The user's external-links.txt IS the source of truth for what counts as "misc evidence" for that project.** No fuzzy auto-discovery. If a link isn't in the file, it isn't pulled. Discovery is a separate (manual) step where the user paste links in.
|
|
68
|
+
2. **Web fetches use HTTP first, browser only if HTTP fails or returns interactive shell.** SPAs that don't server-render (Loop, some Confluence, dashboards) drop to browser. Don't run a browser when curl works.
|
|
69
|
+
3. **Loop pages MUST go through browser.** Same gap as OneNote — Loop has no Graph API for component content. The /loop skill already proved this; pull-misc reuses the same Playwright profile.
|
|
70
|
+
|
|
71
|
+
## Pre-flight
|
|
72
|
+
|
|
73
|
+
```pwsh
|
|
74
|
+
# A. Playwright profile (only required if external-links.txt contains loop links)
|
|
75
|
+
$prof = "$env:USERPROFILE\.copilot\playwright-profile\onenote"
|
|
76
|
+
if (-not (Test-Path $prof)) {
|
|
77
|
+
Write-Host "[pull-misc] Playwright profile not yet seeded — loop links will be skipped with last_status: auth-required."
|
|
78
|
+
Write-Host "[pull-misc] Run plugin/skills/pull-onenote/runner.mjs --bootstrap once to seed it."
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
# B. Node deps
|
|
82
|
+
$nodeDeps = @('playwright', 'jsdom', '@mozilla/readability')
|
|
83
|
+
# Runner checks at startup; if missing, exits with "missing-deps" status and lists them.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Bootstrap discovery
|
|
87
|
+
|
|
88
|
+
For each project, the bootstrap step:
|
|
89
|
+
|
|
90
|
+
1. **Locates `<project>/external-links.txt`.** If absent, write a starter template (same format as ABN AMRO has today) and mark `boundaries.misc.externalLinksPath` in `integrations.yml`.
|
|
91
|
+
2. **Parses the file.** Skip comments, skip blank lines, skip placeholder URLs (`<PASTE_*_URL>`, `<TODO*>`).
|
|
92
|
+
3. **Initializes `misc_links[]`** in `m365-mutable.json#knownSections.<projectKey>` with one entry per non-placeholder link, all `last_status: not-yet-attempted`.
|
|
93
|
+
4. **Records** `boundaries.misc.linkCount` and `placeholderCount` to `bootstrap-status.md`.
|
|
94
|
+
|
|
95
|
+
## Step A — enumerate (every refresh)
|
|
96
|
+
|
|
97
|
+
```text
|
|
98
|
+
1. Read external-links.txt; rebuild link inventory (in case the user added/removed links since last run).
|
|
99
|
+
2. For each existing entry in misc_links[], match by (type, url) tuple — preserve attempts, last_attempt_at, snapshot_path.
|
|
100
|
+
3. For new entries, create with last_status: not-yet-attempted.
|
|
101
|
+
4. For deleted entries (in registry but no longer in file), mark last_status: removed and DO NOT delete the snapshot file (audit trail).
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Step B — fetch (per link)
|
|
105
|
+
|
|
106
|
+
The runner branches per `type`:
|
|
107
|
+
|
|
108
|
+
### B.1 `loop` — browser
|
|
109
|
+
|
|
110
|
+
```js
|
|
111
|
+
// pages.goto(loopUrl); detect login redirect → auth-required.
|
|
112
|
+
// Wait for primary canvas selector; for Loop, that's the Fluid container.
|
|
113
|
+
// Read text via document.body.innerText after settle.
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
If login redirect detected: `last_status: auth-required`, `captured_via: browser`, exit early for this link.
|
|
117
|
+
|
|
118
|
+
### B.2 `web` / `confluence` / `learn` / `docs` / `github` / unknown — HTTP
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
// fetch(url, { headers: { 'User-Agent': 'kushi-pull-misc/0.1' } });
|
|
122
|
+
// If 200 + HTML → run @mozilla/readability for main content extract.
|
|
123
|
+
// If 200 + text/* → use raw body.
|
|
124
|
+
// If 200 + application/pdf → buffer + pdfjs text extract.
|
|
125
|
+
// If 401/403 → last_status: auth-required (cannot resolve unattended for non-MS hosts).
|
|
126
|
+
// If 404 / DNS fail → last_status: fetch-failed with HTTP code.
|
|
127
|
+
// If 5xx → last_status: fetch-failed (transient — retry next refresh).
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### B.3 `file` — local read
|
|
131
|
+
|
|
132
|
+
```js
|
|
133
|
+
// path is project-relative; resolve under <engagement-root>/<project>/
|
|
134
|
+
// Read text; if binary, mark last_status: skipped-binary (use a different pipeline if needed).
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### B.4 `onenote` / `sharepoint` / `ado` — delegate
|
|
138
|
+
|
|
139
|
+
Skip fetch. Write registry entry with `captured_via: delegated`, `delegated_to: pull-onenote|pull-sharepoint|pull-ado`. Per-skill captures already happen in their own pipelines.
|
|
140
|
+
|
|
141
|
+
## Per-link retry registry
|
|
142
|
+
|
|
143
|
+
Stored at `m365-mutable.json#knownSections.<projectKey>.misc_links[]`. Schema:
|
|
144
|
+
|
|
145
|
+
```jsonc
|
|
146
|
+
{
|
|
147
|
+
"type": "loop | web | learn | pdf | github | file | onenote | sharepoint | ado | ...",
|
|
148
|
+
"owner": "<as in external-links.txt>",
|
|
149
|
+
"title": "<as in external-links.txt>",
|
|
150
|
+
"url": "<as in external-links.txt>", // or path for type=file
|
|
151
|
+
"notes": "<as in external-links.txt>",
|
|
152
|
+
"last_status": "captured | placeholder | auth-required | fetch-failed | skipped-binary | removed | not-yet-attempted | delegated",
|
|
153
|
+
"captured_via": "browser | http | file | delegated",
|
|
154
|
+
"delegated_to": "pull-onenote | pull-sharepoint | pull-ado", // only when captured_via=delegated
|
|
155
|
+
"http_status": 200, // only for HTTP
|
|
156
|
+
"content_type": "text/html", // only for HTTP
|
|
157
|
+
"char_count": 12345,
|
|
158
|
+
"attempts": 3,
|
|
159
|
+
"last_attempt_at": "<ISO-8601>",
|
|
160
|
+
"captured_at": "<ISO-8601, only when last_status=captured>",
|
|
161
|
+
"snapshot_path": "Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md",
|
|
162
|
+
"etag": "<HTTP ETag if returned>", // for change detection
|
|
163
|
+
"last_modified_http":"<HTTP Last-Modified if returned>"
|
|
164
|
+
}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
## Snapshot files
|
|
168
|
+
|
|
169
|
+
One file per non-delegated link at:
|
|
170
|
+
|
|
171
|
+
```
|
|
172
|
+
<project>/Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
Schema:
|
|
176
|
+
|
|
177
|
+
```yaml
|
|
178
|
+
---
|
|
179
|
+
type: web
|
|
180
|
+
owner: "Microsoft"
|
|
181
|
+
title: "Microsoft Fabric Documentation Hub"
|
|
182
|
+
url: "https://learn.microsoft.com/fabric/"
|
|
183
|
+
last_status: captured
|
|
184
|
+
captured_via: http
|
|
185
|
+
http_status: 200
|
|
186
|
+
content_type: text/html
|
|
187
|
+
char_count: 18420
|
|
188
|
+
captured_at: 2026-05-14T04:35:00Z
|
|
189
|
+
schema: v0.1.0-misc
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
# {title}
|
|
193
|
+
|
|
194
|
+
> Source: {url}
|
|
195
|
+
> Captured via {captured_via} ({captured_at})
|
|
196
|
+
> Owner: {owner}
|
|
197
|
+
> Type: {type}
|
|
198
|
+
|
|
199
|
+
## AI Narrative Summary
|
|
200
|
+
|
|
201
|
+
{1-2 paragraph summary derived ONLY from the body text below — never fabricate, never extrapolate beyond the captured content}
|
|
202
|
+
|
|
203
|
+
## Body (verbatim)
|
|
204
|
+
|
|
205
|
+
```
|
|
206
|
+
{full extracted text}
|
|
207
|
+
```
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
## Step C — stream
|
|
211
|
+
|
|
212
|
+
For supported types, emit edit events:
|
|
213
|
+
|
|
214
|
+
- `web` / `learn` / `docs` / `pdf`: compare current `etag` / `last_modified_http` to registry; if changed since last refresh, emit `{ts, type, title, url, change: 'updated'}`.
|
|
215
|
+
- `loop`: WorkIQ has Loop edit-event signals via search index — query when available. Otherwise infer from body diff.
|
|
216
|
+
- `file`: compare file mtime; emit if changed.
|
|
217
|
+
- `github`: query repo commits since last `captured_at`; emit per-commit events.
|
|
218
|
+
- `onenote` / `sharepoint` / `ado`: NOT emitted here (their respective pull-* skills handle stream events for their content).
|
|
219
|
+
|
|
220
|
+
## Run report
|
|
221
|
+
|
|
222
|
+
Every refresh writes one block to `Evidence/<alias>/refresh-reports/<date>.md`:
|
|
223
|
+
|
|
224
|
+
```markdown
|
|
225
|
+
### pull-misc — <project>
|
|
226
|
+
|
|
227
|
+
- external-links.txt: <N> total links (<P> placeholders, <D> delegated, <M> handled here)
|
|
228
|
+
- captured: <C> / <M>
|
|
229
|
+
- auth-required: <A>
|
|
230
|
+
- fetch-failed: <F>
|
|
231
|
+
- placeholders to fill: <list of titles where url contains <PASTE_*_URL>>
|
|
232
|
+
- removed since last run: <list>
|
|
233
|
+
- new since last run: <list>
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
## Anti-patterns (HARD)
|
|
237
|
+
|
|
238
|
+
1. ❌ Auto-discovering links by crawling project SharePoint or Teams chats. The user's `external-links.txt` IS the boundary.
|
|
239
|
+
2. ❌ Storing only `url` (not `(type, url)` tuple) as the registry key. Two types may share a URL (e.g. an onenote URL flagged as `web` for ad-hoc fetch test).
|
|
240
|
+
3. ❌ Silently dropping a link from the registry when it disappears from `external-links.txt`. Mark `removed` and keep the snapshot for audit.
|
|
241
|
+
4. ❌ Pulling delegated types (onenote / sharepoint / ado). Those have dedicated skills. Double-pulling creates conflicting evidence.
|
|
242
|
+
5. ❌ Persisting placeholder URLs as `fetch-failed`. They're `placeholder`, surfaced in the run report so the user knows to fill them in.
|
|
243
|
+
6. ❌ Loop link without Playwright profile silently writes empty snapshot. Must mark `auth-required`.
|
|
244
|
+
|
|
245
|
+
## Self-check enforcement
|
|
246
|
+
|
|
247
|
+
`plugin/skills/self-check/run.ps1` D-token contract for `pull-misc`: the SKILL.md must contain `external-links`, `misc_links`, `placeholder`, `delegated`. These prove the skill carries the v0.1.0 link-list pipeline + delegation contract.
|
|
248
|
+
|
|
249
|
+
## Citation
|
|
250
|
+
|
|
251
|
+
Citations from misc evidence MUST follow:
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
[source: misc/<type>/<safe-title> · YYYY-MM-DD]
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
Example: `[source: misc/loop/ABN-Core-Team-Sync-2026-03-27 · 2026-05-14]`
|