kushi-agents 3.4.2 → 3.12.1

package/plugin/skills/pull-misc/SKILL.md

@@ -0,0 +1,257 @@
+---
+name: "pull-misc"
+version: "2.0.1"
+description: "Pull miscellaneous evidence from a user-curated link list (<project>/external-links.txt). Handles links that don't fit the per-source pull-* skills: Loop pages, public web pages, learn.microsoft.com / docs sites, GitHub repos, local files, anything else linkable. Routes by type: delegated types (onenote, sharepoint, ado) are skipped because dedicated pull-* skills handle them. Per-link retry registry tracks last_status across runs."
+---
+
+# Skill: pull-misc
+
+> **v3.9.0 contracts** — This skill operates under six HARD-rule doctrines:
+> - `verbatim-by-default.instructions.md` — full bodies by default; no preview-grade pulls accepted.
+> - `m365-id-registry.instructions.md` — discover-once / consume-deterministically (per-link retry registry under `misc_links[]`).
+> - `capture-learnings.instructions.md` — every fix/discovery is logged to `plugin/learnings/misc.md`.
+> - `cleanup-on-resolution.instructions.md` — when a placeholder URL resolves, all stale unavailable-markers are upgraded in the same turn.
+> - `run-reports.instructions.md` — every refresh writes a per-user report under `Evidence/<alias>/refresh-reports/`.
+> - `thoroughness-detector.instructions.md` — the per-link retry registry IS this skill's thoroughness contract.
+
+> **Canonical evidence layout** (HARD, kushi v3.12.1+): all artifacts produced by this skill MUST be written under `<project>/Evidence/<alias>/<source>/{snapshot,stream,...}/` — sibling folders under `<project>/` (e.g. `<project>/<source>-context/`, `<project>/<source>/`, `<project>/_Weekly Summaries/`) are FORBIDDEN. See `evidence-layout-canonical.instructions.md`.
+
+Pulls **misc** evidence in two shapes per `snapshot-vs-stream.instructions.md`:
+
+- **snapshot/** — full content per link — one file per link with last-fetched + verbatim body
+- **stream/** — change events (HTTP `last-modified` header drift, git commit deltas where applicable, Loop edit signals via WorkIQ when available)
+
+## Tools (in order)
+
+1. **Playwright (browser-scrape, persisted profile)** — for `loop` links. Reuses `~/.copilot/playwright-profile/onenote/` because it's the same M365 cookie scope. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `loop`.
+2. **HTTP fetch + Readability** — for `web` / `confluence` (anonymous) / `learn` / `docs` / `pdf` links. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `web`.
+3. **Local file read** — for `file` links pointing at project-relative paths. Implementation: `plugin/skills/pull-misc/runner.mjs` branch `file`.
+4. **WorkIQ** — used only for stream/edit-event signals on `loop` links (Loop edit events surface in the M365 search index, same as OneNote stream events). NOT used for body retrieval — Loop bodies require the browser path.
+5. **Delegation** — `onenote`, `sharepoint`, `ado` are NOT pulled here; the runner records `captured_via: delegated` so the registry stays complete and the user can see which links flow through which skill. The actual capture is done by `pull-onenote`, `pull-sharepoint`, `pull-ado` respectively.
+
+## The link list (`<project>/external-links.txt`)
+
+This file already exists in most projects (per memory: the External Links Context doctrine). pull-misc preserves the existing format **without breaking changes** and adds `loop` as a recognized type.
+
+Format:
+
+```text
+# Comments start with #
+# One link per line:
+<type>|<owner>|<title>|<url-or-path>|<notes>
+```
+
+**Recognized types (v3.9.0):**
+
+| Type | Routed to | Captured via |
+|---|---|---|
+| `onenote` | pull-onenote | delegated |
+| `sharepoint` | pull-sharepoint | delegated |
+| `ado` | pull-ado | delegated |
+| `loop` **(new)** | pull-misc | browser (Playwright, OneNote profile) |
+| `web` | pull-misc | http (fetch + readability) |
+| `confluence` | pull-misc | http (anonymous) — auth'd Confluence not yet supported |
+| `learn` | pull-misc | http |
+| `docs` | pull-misc | http |
+| `pdf` | pull-misc | http (binary download + text extract via pdfjs) |
+| `github` | pull-misc | http (raw README + repo metadata) |
+| `file` | pull-misc | file (local read; path is project-relative) |
+| anything else | pull-misc | http (best-effort) |
+
+**Placeholder URLs** matching `<PASTE_*_URL>` or `<TODO*>` are SKIPPED with `last_status: placeholder`. They surface in the run report so the user can fill them in.
+
+## Empirical contract
+
+These three facts are HARD-rule:
+
+1. **The user's external-links.txt IS the source of truth for what counts as "misc evidence" for that project.** No fuzzy auto-discovery. If a link isn't in the file, it isn't pulled. Discovery is a separate (manual) step where the user paste links in.
+2. **Web fetches use HTTP first, browser only if HTTP fails or returns interactive shell.** SPAs that don't server-render (Loop, some Confluence, dashboards) drop to browser. Don't run a browser when curl works.
+3. **Loop pages MUST go through browser.** Same gap as OneNote — Loop has no Graph API for component content. The /loop skill already proved this; pull-misc reuses the same Playwright profile.
+
+## Pre-flight
+
+```pwsh
+# A. Playwright profile (only required if external-links.txt contains loop links)
+$prof = "$env:USERPROFILE\.copilot\playwright-profile\onenote"
+if (-not (Test-Path $prof)) {
+  Write-Host "[pull-misc] Playwright profile not yet seeded — loop links will be skipped with last_status: auth-required."
+  Write-Host "[pull-misc] Run plugin/skills/pull-onenote/runner.mjs --bootstrap once to seed it."
+}
+
+# B. Node deps
+$nodeDeps = @('playwright', 'jsdom', '@mozilla/readability')
+# Runner checks at startup; if missing, exits with "missing-deps" status and lists them.
+```
+
+## Bootstrap discovery
+
+For each project, the bootstrap step:
+
+1. **Locates `<project>/external-links.txt`.** If absent, write a starter template (same format as ABN AMRO has today) and mark `boundaries.misc.externalLinksPath` in `integrations.yml`.
+2. **Parses the file.** Skip comments, skip blank lines, skip placeholder URLs (`<PASTE_*_URL>`, `<TODO*>`).
+3. **Initializes `misc_links[]`** in `m365-mutable.json#knownSections.<projectKey>` with one entry per non-placeholder link, all `last_status: not-yet-attempted`.
+4. **Records** `boundaries.misc.linkCount` and `placeholderCount` to `bootstrap-status.md`.
+
+## Step A — enumerate (every refresh)
+
+```text
+1. Read external-links.txt; rebuild link inventory (in case the user added/removed links since last run).
+2. For each existing entry in misc_links[], match by (type, url) tuple — preserve attempts, last_attempt_at, snapshot_path.
+3. For new entries, create with last_status: not-yet-attempted.
+4. For deleted entries (in registry but no longer in file), mark last_status: removed and DO NOT delete the snapshot file (audit trail).
+```
+
+## Step B — fetch (per link)
+
+The runner branches per `type`:
+
+### B.1 `loop` — browser
+
+```js
+// pages.goto(loopUrl); detect login redirect → auth-required.
+// Wait for primary canvas selector; for Loop, that's the Fluid container.
+// Read text via document.body.innerText after settle.
+```
+
+If login redirect detected: `last_status: auth-required`, `captured_via: browser`, exit early for this link.
+
+### B.2 `web` / `confluence` / `learn` / `docs` / `github` / unknown — HTTP
+
+```js
+// fetch(url, { headers: { 'User-Agent': 'kushi-pull-misc/0.1' } });
+// If 200 + HTML → run @mozilla/readability for main content extract.
+// If 200 + text/* → use raw body.
+// If 200 + application/pdf → buffer + pdfjs text extract.
+// If 401/403 → last_status: auth-required (cannot resolve unattended for non-MS hosts).
+// If 404 / DNS fail → last_status: fetch-failed with HTTP code.
+// If 5xx → last_status: fetch-failed (transient — retry next refresh).
+```
+
+### B.3 `file` — local read
+
+```js
+// path is project-relative; resolve under <engagement-root>/<project>/
+// Read text; if binary, mark last_status: skipped-binary (use a different pipeline if needed).
+```
+
+### B.4 `onenote` / `sharepoint` / `ado` — delegate
+
+Skip fetch. Write registry entry with `captured_via: delegated`, `delegated_to: pull-onenote|pull-sharepoint|pull-ado`. Per-skill captures already happen in their own pipelines.
+
+## Per-link retry registry
+
+Stored at `m365-mutable.json#knownSections.<projectKey>.misc_links[]`. Schema:
+
+```jsonc
+{
+  "type":             "loop | web | learn | pdf | github | file | onenote | sharepoint | ado | ...",
+  "owner":            "<as in external-links.txt>",
+  "title":            "<as in external-links.txt>",
+  "url":              "<as in external-links.txt>",                  // or path for type=file
+  "notes":            "<as in external-links.txt>",
+  "last_status":      "captured | placeholder | auth-required | fetch-failed | skipped-binary | removed | not-yet-attempted | delegated",
+  "captured_via":     "browser | http | file | delegated",
+  "delegated_to":     "pull-onenote | pull-sharepoint | pull-ado",   // only when captured_via=delegated
+  "http_status":      200,                                            // only for HTTP
+  "content_type":     "text/html",                                    // only for HTTP
+  "char_count":       12345,
+  "attempts":         3,
+  "last_attempt_at":  "<ISO-8601>",
+  "captured_at":      "<ISO-8601, only when last_status=captured>",
+  "snapshot_path":    "Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md",
+  "etag":             "<HTTP ETag if returned>",                      // for change detection
+  "last_modified_http":"<HTTP Last-Modified if returned>"
+}
+```
+
+## Snapshot files
+
+One file per non-delegated link at:
+
+```
+<project>/Evidence/<alias>/misc/snapshot/<safe-type>__<safe-title>.md
+```
+
+Schema:
+
+```yaml
+---
+type: web
+owner: "Microsoft"
+title: "Microsoft Fabric Documentation Hub"
+url: "https://learn.microsoft.com/fabric/"
+last_status: captured
+captured_via: http
+http_status: 200
+content_type: text/html
+char_count: 18420
+captured_at: 2026-05-14T04:35:00Z
+schema: v0.1.0-misc
+---
+
+# {title}
+
+> Source: {url}
+> Captured via {captured_via} ({captured_at})
+> Owner: {owner}
+> Type: {type}
+
+## AI Narrative Summary
+
+{1-2 paragraph summary derived ONLY from the body text below — never fabricate, never extrapolate beyond the captured content}
+
+## Body (verbatim)
+
+```
+{full extracted text}
+```
+```
+
+## Step C — stream
+
+For supported types, emit edit events:
+
+- `web` / `learn` / `docs` / `pdf`: compare current `etag` / `last_modified_http` to registry; if changed since last refresh, emit `{ts, type, title, url, change: 'updated'}`.
+- `loop`: WorkIQ has Loop edit-event signals via search index — query when available. Otherwise infer from body diff.
+- `file`: compare file mtime; emit if changed.
+- `github`: query repo commits since last `captured_at`; emit per-commit events.
+- `onenote` / `sharepoint` / `ado`: NOT emitted here (their respective pull-* skills handle stream events for their content).
+
+## Run report
+
+Every refresh writes one block to `Evidence/<alias>/refresh-reports/<date>.md`:
+
+```markdown
+### pull-misc — <project>
+
+- external-links.txt: <N> total links (<P> placeholders, <D> delegated, <M> handled here)
+- captured: <C> / <M>
+- auth-required: <A>
+- fetch-failed: <F>
+- placeholders to fill: <list of titles where url contains <PASTE_*_URL>>
+- removed since last run: <list>
+- new since last run: <list>
+```
+
+## Anti-patterns (HARD)
+
+1. ❌ Auto-discovering links by crawling project SharePoint or Teams chats. The user's `external-links.txt` IS the boundary.
+2. ❌ Storing only `url` (not `(type, url)` tuple) as the registry key. Two types may share a URL (e.g. an onenote URL flagged as `web` for ad-hoc fetch test).
+3. ❌ Silently dropping a link from the registry when it disappears from `external-links.txt`. Mark `removed` and keep the snapshot for audit.
+4. ❌ Pulling delegated types (onenote / sharepoint / ado). Those have dedicated skills. Double-pulling creates conflicting evidence.
+5. ❌ Persisting placeholder URLs as `fetch-failed`. They're `placeholder`, surfaced in the run report so the user knows to fill them in.
+6. ❌ Loop link without Playwright profile silently writes empty snapshot. Must mark `auth-required`.
+
+## Self-check enforcement
+
+`plugin/skills/self-check/run.ps1` D-token contract for `pull-misc`: the SKILL.md must contain `external-links`, `misc_links`, `placeholder`, `delegated`. These prove the skill carries the v0.1.0 link-list pipeline + delegation contract.
+
+## Citation
+
+Citations from misc evidence MUST follow:
+
+```
+[source: misc/<type>/<safe-title> · YYYY-MM-DD]
+```
+
+Example: `[source: misc/loop/ABN-Core-Team-Sync-2026-03-27 · 2026-05-14]`

package/plugin/skills/pull-misc/runner.mjs

@@ -0,0 +1,280 @@
+#!/usr/bin/env node
+// pull-misc runner — read external-links.txt, classify by type, route to runner branches.
+// Per pull-misc SKILL.md v0.1.0 (kushi v3.9.0).
+//
+// Usage:
+//   node runner.mjs --project HCA --links-file "<path-to-external-links.txt>"
+//   node runner.mjs --project HCA --links-file "..." --headless
+//   node runner.mjs --project HCA --links-file "..." --types "loop,web"           # only fetch some
+//   node runner.mjs --project HCA --links-file "..." --titles "ABN Workshop,..."  # only specific titles
+//
+// Output is a single JSON object on stdout containing:
+//   { project, linksFile, links: [<perLinkRecord>...], runStatus, counts }
+
+import fs from 'node:fs';
+import fsp from 'node:fs/promises';
+import path from 'node:path';
+import os from 'node:os';
+import process from 'node:process';
+
+// ----- arg parse -----
+const args = Object.fromEntries(
+  process.argv.slice(2).reduce((acc, cur, idx, arr) => {
+    if (cur.startsWith('--')) {
+      const key = cur.replace(/^--/, '');
+      const next = arr[idx + 1];
+      acc.push([key, next && !next.startsWith('--') ? next : true]);
+    }
+    return acc;
+  }, [])
+);
+
+const PROJECT = args.project || null;
+const LINKS_FILE = args['links-file'] || null;
+const ENGAGEMENT_ROOT = args['engagement-root'] || null;
+const HEADLESS = !!args.headless;
+const TYPES_FILTER = args.types ? String(args.types).split(',').map(s => s.trim().toLowerCase()) : null;
+const TITLES_FILTER = args.titles ? String(args.titles).split(',').map(s => s.trim()) : null;
+const TIMEOUT_MS = parseInt(args.timeout || '45000', 10);
+const PROFILE_DIR = path.join(os.homedir(), '.copilot', 'playwright-profile', 'onenote');
+
+if (!PROJECT || !LINKS_FILE) {
+  console.error('Usage: --project <name> --links-file <path> [--headless] [--types t1,t2] [--titles "t1,t2"]');
+  process.exit(2);
+}
+
+// ----- constants -----
+const DELEGATED_TYPES = new Set(['onenote', 'sharepoint', 'ado']);
+const BROWSER_TYPES = new Set(['loop']);
+const PLACEHOLDER_RE = /<\s*(PASTE_[A-Z_]*_URL|TODO[^>]*)\s*>/i;
+
+// ----- parse external-links.txt -----
+function parseLinksFile(p) {
+  const txt = fs.readFileSync(p, 'utf-8');
+  const out = [];
+  for (const raw of txt.split(/\r?\n/)) {
+    const line = raw.trim();
+    if (!line || line.startsWith('#')) continue;
+    const parts = line.split('|').map(s => s.trim());
+    if (parts.length < 4) continue; // malformed
+    const [type, owner, title, url, ...noteParts] = parts;
+    out.push({
+      type: type.toLowerCase(),
+      owner,
+      title,
+      url,
+      notes: noteParts.join('|') || ''
+    });
+  }
+  return out;
+}
+
+function safeName(s) {
+  return s.replace(/[\\/:*?"<>|]/g, '-').replace(/\s+/g, '-').slice(0, 120);
+}
+
+function classify(link) {
+  if (PLACEHOLDER_RE.test(link.url)) return { route: 'placeholder' };
+  if (DELEGATED_TYPES.has(link.type)) return { route: 'delegated', delegated_to: `pull-${link.type}` };
+  if (BROWSER_TYPES.has(link.type)) return { route: 'browser' };
+  if (link.type === 'file') return { route: 'file' };
+  return { route: 'http' };
+}
+
+// ----- HTTP fetch branch -----
+async function fetchHttp(url) {
+  try {
+    const res = await fetch(url, {
+      redirect: 'follow',
+      headers: { 'User-Agent': 'kushi-pull-misc/0.1', 'Accept': '*/*' },
+      signal: AbortSignal.timeout(TIMEOUT_MS)
+    });
+    const ct = res.headers.get('content-type') || '';
+    const etag = res.headers.get('etag') || null;
+    const lm = res.headers.get('last-modified') || null;
+
+    if (res.status === 401 || res.status === 403) {
+      return { last_status: 'auth-required', http_status: res.status, content_type: ct };
+    }
+    if (!res.ok) {
+      return { last_status: 'fetch-failed', http_status: res.status, content_type: ct };
+    }
+
+    if (ct.startsWith('application/pdf')) {
+      const buf = Buffer.from(await res.arrayBuffer());
+      // Defer text extract to a separate step (pdfjs not bundled to keep dep weight low).
+      return {
+        last_status: 'captured',
+        http_status: res.status,
+        content_type: ct,
+        etag, last_modified_http: lm,
+        body: `[binary PDF — ${buf.length} bytes; text extraction not yet implemented in v0.1]`,
+        char_count: 0
+      };
+    }
+
+    const body = await res.text();
+
+    // Heuristic: HTML → readability extract; plain → keep raw.
+    let extracted = body;
+    if (ct.includes('html')) {
+      try {
+        const { JSDOM } = await import('jsdom');
+        const { Readability } = await import('@mozilla/readability');
+        const dom = new JSDOM(body, { url });
+        const reader = new Readability(dom.window.document);
+        const article = reader.parse();
+        if (article && article.textContent && article.textContent.length > 100) {
+          extracted = `# ${article.title || ''}\n\n${article.textContent.trim()}`;
+        }
+      } catch (e) {
+        // jsdom or readability not available — fall through to raw
+        extracted = `[readability unavailable: ${e.message}]\n\n${body.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim()}`;
+      }
+    }
+
+    return {
+      last_status: 'captured',
+      http_status: res.status,
+      content_type: ct,
+      etag, last_modified_http: lm,
+      body: extracted,
+      char_count: extracted.length
+    };
+  } catch (e) {
+    return { last_status: 'fetch-failed', error: String(e.message || e) };
+  }
+}
+
+// ----- File read branch -----
+async function readFileBranch(link) {
+  const root = ENGAGEMENT_ROOT || process.cwd();
+  const candidate1 = path.resolve(root, link.url);
+  const candidate2 = path.resolve(root, PROJECT, link.url);
+  const tryPath = fs.existsSync(candidate1) ? candidate1 : (fs.existsSync(candidate2) ? candidate2 : null);
+  if (!tryPath) return { last_status: 'fetch-failed', error: `not found at ${candidate1} or ${candidate2}` };
+  try {
+    const buf = await fsp.readFile(tryPath);
+    if (buf.includes(0)) return { last_status: 'skipped-binary', char_count: buf.length };
+    const txt = buf.toString('utf-8');
+    return { last_status: 'captured', body: txt, char_count: txt.length, resolved_path: tryPath };
+  } catch (e) {
+    return { last_status: 'fetch-failed', error: String(e.message || e) };
+  }
+}
+
+// ----- Browser (Loop) branch -----
+async function fetchBrowser(link, ctx) {
+  const page = await ctx.newPage();
+  try {
+    const resp = await page.goto(link.url, { waitUntil: 'domcontentloaded', timeout: TIMEOUT_MS });
+    if (/login\.microsoftonline\.com|login\.live\.com/.test(page.url())) {
+      return { last_status: 'auth-required', captured_via: 'browser' };
+    }
+    // Wait for Loop content. Loop uses Fluid; the main content settles in a few seconds.
+    await page.waitForTimeout(4000);
+    // Try Loop-specific selectors first, then fallback to body
+    const body = await page.evaluate(() => {
+      const sels = [
+        '[data-loop-canvas]',
+        '.fui-FluentProvider',
+        '[role="main"]',
+        'body'
+      ];
+      for (const s of sels) {
+        const el = document.querySelector(s);
+        if (el && el.innerText && el.innerText.length > 100) return el.innerText;
+      }
+      return document.body ? document.body.innerText : '';
+    });
+    if (!body || body.length < 50) {
+      return { last_status: 'fetch-failed', captured_via: 'browser', error: 'empty or near-empty body', char_count: body?.length || 0 };
+    }
+    return { last_status: 'captured', captured_via: 'browser', body, char_count: body.length };
+  } catch (e) {
+    return { last_status: 'fetch-failed', captured_via: 'browser', error: String(e.message || e) };
+  } finally {
+    await page.close().catch(() => {});
+  }
+}
+
+// ----- main -----
+(async () => {
+  const links = parseLinksFile(LINKS_FILE);
+  const filtered = links.filter(l => {
+    if (TYPES_FILTER && !TYPES_FILTER.includes(l.type)) return false;
+    if (TITLES_FILTER && !TITLES_FILTER.some(t => l.title.includes(t))) return false;
+    return true;
+  });
+
+  const needsBrowser = filtered.some(l => classify(l).route === 'browser');
+  let ctx = null;
+  if (needsBrowser) {
+    if (!fs.existsSync(PROFILE_DIR)) {
+      console.error('[pull-misc] Playwright profile missing — loop links will be marked auth-required.');
+    } else {
+      const { chromium } = await import('playwright');
+      ctx = await chromium.launchPersistentContext(PROFILE_DIR, { headless: HEADLESS, viewport: { width: 1400, height: 900 } });
+    }
+  }
+
+  const results = [];
+  const now = new Date().toISOString();
+
+  for (const link of filtered) {
+    const cls = classify(link);
+    let rec = {
+      type: link.type,
+      owner: link.owner,
+      title: link.title,
+      url: link.url,
+      notes: link.notes,
+      attempts: 1,
+      last_attempt_at: now,
+      snapshot_path: `Evidence/<alias>/misc/snapshot/${safeName(link.type)}__${safeName(link.title)}.md`
+    };
+
+    if (cls.route === 'placeholder') {
+      Object.assign(rec, { last_status: 'placeholder', captured_via: null });
+    } else if (cls.route === 'delegated') {
+      Object.assign(rec, { last_status: 'delegated', captured_via: 'delegated', delegated_to: cls.delegated_to });
+    } else if (cls.route === 'http') {
+      const r = await fetchHttp(link.url);
+      Object.assign(rec, r, { captured_via: 'http' });
+      if (r.last_status === 'captured') rec.captured_at = now;
+    } else if (cls.route === 'file') {
+      const r = await readFileBranch(link);
+      Object.assign(rec, r, { captured_via: 'file' });
+      if (r.last_status === 'captured') rec.captured_at = now;
+    } else if (cls.route === 'browser') {
+      if (!ctx) {
+        Object.assign(rec, { last_status: 'auth-required', captured_via: 'browser', error: 'no playwright profile' });
+      } else {
+        const r = await fetchBrowser(link, ctx);
+        Object.assign(rec, r);
+        if (r.last_status === 'captured') rec.captured_at = now;
+      }
+    }
+    results.push(rec);
+  }
+
+  if (ctx) await ctx.close();
+
+  const counts = {
+    total: results.length,
+    captured: results.filter(r => r.last_status === 'captured').length,
+    placeholder: results.filter(r => r.last_status === 'placeholder').length,
+    delegated: results.filter(r => r.last_status === 'delegated').length,
+    auth_required: results.filter(r => r.last_status === 'auth-required').length,
+    fetch_failed: results.filter(r => r.last_status === 'fetch-failed').length,
+    skipped_binary: results.filter(r => r.last_status === 'skipped-binary').length
+  };
+  const runStatus = counts.auth_required > 0 ? 'auth-required'
+    : counts.fetch_failed > 0 ? 'partial'
+    : 'ok';
+
+  console.log(JSON.stringify({ project: PROJECT, linksFile: LINKS_FILE, runStatus, counts, links: results }, null, 2));
+})().catch(e => {
+  console.error('[pull-misc] runner crashed:', e);
+  process.exit(1);
+});

package/plugin/skills/pull-onenote/README.md

@@ -0,0 +1,90 @@
+# pull-onenote — runner README
+
+## What this is
+
+A Playwright-driven browser scraper for OneNote-for-Web that returns verbatim page bodies for an entire section. Used by the `pull-onenote` skill (v2.6.0+) as the PRIMARY capture path.
+
+## One-time setup
+
+```pwsh
+# 1. Install Node + Playwright if you don't have them
+npm install -g playwright
+npx playwright install chromium
+
+# 2. Seed the persistent browser profile (interactive — one-time)
+node plugin/skills/pull-onenote/runner.mjs --bootstrap
+```
+
+When the bootstrap command opens a Chromium window:
+- Sign in with your work account at `onenote.cloud.microsoft`.
+- Check **Keep me signed in**.
+- Wait for the OneNote home page to load.
+- Close the window.
+
+The profile is persisted at `~/.copilot/playwright-profile/onenote/`. Subsequent runs reuse it.
+
+## Per-section runs
+
+```pwsh
+node plugin/skills/pull-onenote/runner.mjs `
+  --project "HCA" `
+  --section-url "https://microsoft-my.sharepoint-df.com/personal/<upn>/_layouts/15/Doc.aspx?sourcedoc=%7B<notebookSourceDoc>%7D&action=edit&wd=target%28<sectionName>.one%7C<sectionFileId>%2F%29" `
+  --headless
+```
+
+The runner emits a single JSON object on stdout with this shape:
+
+```json
+{
+  "project": "HCA",
+  "sectionUrl": "...",
+  "runStatus": "ok | auth-required | partial",
+  "exitedEarly": false,
+  "authRequiredCount": 0,
+  "pages": [
+    {
+      "title": "4/3 - HCA with Jay and Martin",
+      "pos": 12, "total": 16,
+      "webPageId": "78aac9b5-0629-4daa-ada3-f2436cb2381c",
+      "last_status": "captured | short-suspect | auth-required | browser-error",
+      "captured_via": "browser",
+      "body": "...verbatim text...",
+      "bodyLen": 6904,
+      "capturedAt": "2026-05-14T03:55:00Z"
+    }
+  ]
+}
+```
+
+The skill driver merges this into `m365-mutable.json#knownSections.<projectKey>.one_pages[]` and writes the snapshot pages.
+
+## Filtering
+
+For retry-only runs, pass `--titles` with a comma-separated list:
+
+```pwsh
+node runner.mjs --project HCA --section-url "..." --titles "4/3 - HCA with Jay and Martin,5/7" --headless
+```
+
+## Scheduled / unattended runs
+
+For Monday-9am-style schedules, always pass `--headless`. The runner detects redirects to `login.microsoftonline.com` and exits early with `runStatus: "auth-required"` so the driver can mark all queued pages as `auth-required` and surface in the run report.
+
+There is currently no way to satisfy a Conditional Access / MFA challenge unattended. When that happens (rough estimate: every 1–4 weeks under a Microsoft tenant CA policy), do one interactive `--bootstrap` to refresh the profile and the next scheduled run resumes silently.
+
+## Constructing the section URL
+
+Once a project's `knownSections` entry is populated in `m365-mutable.json`, the URL is:
+
+```
+<one_notebookSpoBaseUrl>/_layouts/15/Doc.aspx?sourcedoc=%7B<one_notebookSourceDoc>%7D&action=edit&wd=target(<one_sectionName>%7C<one_sectionFileId>%2F)
+```
+
+URL-encode the section name; double-encode the pipe (`%7C`) and trailing slash (`%2F`).
+
+## Troubleshooting
+
+- **Picker shown every run.** First-run only, and only if multiple work accounts are listed. After picking once it sticks.
+- **"OneNote canvas frame did not attach."** Almost always auth — check the headless output's `runStatus`.
+- **Bodies look short.** The `last_status: short-suspect` flag is for this. Verify in OneNote desktop; if genuinely sparse, accept; otherwise re-run.
+- **Pages enumerate fine but no `webPageId` returned.** The `wd=target(...)` URL fragment shape may have changed. Update `parseWebPageIdFromUrl` regex in `runner.mjs`.