@gobing-ai/knowledge-kit 0.1.13 → 0.1.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/dist/index.js +796 -176
  3. package/package.json +1 -1
  4. package/plugins/generations/content-gen/dist/index.js +2 -2
  5. package/plugins/generations/core-facts-gen/dist/index.js +2 -2
  6. package/plugins/generations/daily-article-gen/dist/index.js +2 -2
  7. package/plugins/generations/dailynews-gen/dist/index.js +72 -9
  8. package/plugins/generations/dailynews-gen/src/index.ts +23 -1
  9. package/plugins/generations/dailynews-gen/src/script-plan.ts +103 -5
  10. package/plugins/generations/episode-plan-gen/dist/index.js +2 -2
  11. package/plugins/generations/gemini-tts/dist/index.js +2 -2
  12. package/plugins/generations/image-gen/dist/index.js +2 -2
  13. package/plugins/generations/news-report-gen/dist/index.js +2 -2
  14. package/plugins/generations/voice-gen/dist/index.js +2 -2
  15. package/plugins/generations/voxcpm-tts/dist/index.js +2 -2
  16. package/plugins/ingestions/aihot-ingest/dist/index.js +2 -2
  17. package/plugins/ingestions/horizon-ingest/dist/index.js +2 -2
  18. package/plugins/ingestions/last30days-ingest/dist/index.js +2 -2
  19. package/plugins/ingestions/web-search/dist/index.js +2 -2
  20. package/plugins/kk/plugin.json +1 -1
  21. package/plugins/kk/scripts/daily-scheduled-run.ts +53 -5
  22. package/plugins/kk/scripts/daily-stages.ts +476 -12
  23. package/plugins/kk/scripts/kk-workflow-stages.ts +20 -4
  24. package/plugins/kk/skills/content-judge/SKILL.md +5 -4
  25. package/plugins/kk/skills/content-judge/references/rubrics.md +14 -0
  26. package/plugins/kk/skills/news-editorial/SKILL.md +122 -0
  27. package/plugins/kk/skills/news-editorial/references/selection.md +47 -0
  28. package/plugins/kk/skills/news-editorial/references/style-contract.zh.md +99 -0
  29. package/plugins/kk/skills/publish/SKILL.md +3 -3
  30. package/plugins/kk/skills/web-automation/SKILL.md +229 -77
  31. package/plugins/kk/skills/web-automation/references/browser-launch.md +127 -36
  32. package/plugins/kk/workflows/kk-daily-ai-voice2.yaml +4 -6
  33. package/plugins/kk/workflows/kk-daily-ai-voice3.yaml +183 -222
  34. package/plugins/kk/workflows/kk-daily-publish3.yaml +16 -8
  35. package/plugins/publishings/bilibili-pub/dist/index.js +5 -3
  36. package/plugins/publishings/bilibili-pub/dist/profile-check.js +3 -1
  37. package/plugins/publishings/cnblogs-pub/dist/index.js +5 -3
  38. package/plugins/publishings/cnblogs-pub/dist/profile-check.js +3 -1
  39. package/plugins/publishings/csdn-pub/dist/index.js +5 -3
  40. package/plugins/publishings/csdn-pub/dist/profile-check.js +5 -3
  41. package/plugins/publishings/emdash-pub/dist/index.js +2 -2
  42. package/plugins/publishings/infoq-pub/dist/index.js +5 -3
  43. package/plugins/publishings/infoq-pub/dist/profile-check.js +5 -3
  44. package/plugins/publishings/jike-pub/dist/index.js +5 -3
  45. package/plugins/publishings/jike-pub/dist/profile-check.js +5 -3
  46. package/plugins/publishings/juejin-pub/dist/index.js +5 -3
  47. package/plugins/publishings/juejin-pub/dist/profile-check.js +5 -3
  48. package/plugins/publishings/linkedin-pub/dist/index.js +5 -3
  49. package/plugins/publishings/linkedin-pub/dist/profile-check.js +5 -3
  50. package/plugins/publishings/medium-pub/dist/index.js +2 -2
  51. package/plugins/publishings/podcast-pub/dist/index.js +147 -4
  52. package/plugins/publishings/podcast-pub/src/index.ts +71 -2
  53. package/plugins/publishings/podcast-pub/src/map.ts +3 -0
  54. package/plugins/publishings/podcast-pub/src/transcript.ts +195 -0
  55. package/plugins/publishings/qiita-pub/dist/index.js +2 -2
  56. package/plugins/publishings/shipinhao-pub/dist/index.js +5 -3
  57. package/plugins/publishings/shipinhao-pub/dist/profile-check.js +3 -1
  58. package/plugins/publishings/sspai-pub/dist/index.js +5 -3
  59. package/plugins/publishings/sspai-pub/dist/profile-check.js +3 -1
  60. package/plugins/publishings/substack-pub/dist/index.js +5 -3
  61. package/plugins/publishings/substack-pub/dist/profile-check.js +5 -3
  62. package/plugins/publishings/surfdash-pub/dist/index.js +2 -2
  63. package/plugins/publishings/threads-pub/dist/index.js +2 -2
  64. package/plugins/publishings/wechatmp-pub/dist/index.js +52 -33
  65. package/plugins/publishings/wechatmp-pub/dist/profile-check.js +5 -3
  66. package/plugins/publishings/wechatmp-pub/src/flow.ts +70 -33
  67. package/plugins/publishings/weibo-pub/dist/index.js +21 -6
  68. package/plugins/publishings/weibo-pub/dist/profile-check.js +5 -3
  69. package/plugins/publishings/weibo-pub/src/index.ts +31 -7
  70. package/plugins/publishings/x-pub/dist/index.js +5 -3
  71. package/plugins/publishings/x-pub/dist/profile-check.js +5 -3
  72. package/plugins/publishings/xhs-pub/dist/index.js +5 -3
  73. package/plugins/publishings/xhs-pub/dist/profile-check.js +5 -3
  74. package/plugins/publishings/xiaoyuzhou-pub/dist/index.js +5 -3
  75. package/plugins/publishings/xiaoyuzhou-pub/dist/profile-check.js +3 -1
  76. package/plugins/publishings/zenn-pub/dist/index.js +2 -2
  77. package/plugins/kk/scripts/daily-voice-style.zh.md +0 -70
@@ -1,101 +1,253 @@
1
1
  ---
2
2
  name: web-automation
3
3
  description: >-
4
- Discover, enhance, and automate web tasks with Playwright (or agent-browser
5
- over CDP) — "walk this console and record evidence", "figure out this site's
4
+ Discover, enhance, and automate web tasks with Playwright, kk bot, or agent-browser
5
+ over CDP — "walk this console and record evidence", "figure out this site's
6
6
  UI", "automate this web flow", "reuse my cached login on <channel>", "check
7
- whether my session on <channel> is still valid", "fix this channel's
8
- selectors", or when a spur workflow `agent.run` step targets the
9
- web-automation skill. Owns method and evidence; transport stays in kk
10
- plugins; orchestration is a @gobing-ai/ts-dual-workflow-engine
11
- transition-flow (see docs/design/web-automation-engine.md).
12
- requires: [cli:spur, local-fs]
7
+ whether my session on <channel> is still valid", "run a browser collection plan",
8
+ "fix this channel's selectors", or when a spur workflow `agent.run` step targets the
9
+ web-automation skill. Coordinates with the kk bot session lifecycle and collection
10
+ substrate. Owns method and evidence; transport stays in kk plugins; orchestration is
11
+ a @gobing-ai/ts-dual-workflow-engine transition-flow (see docs/design/web-automation-engine.md).
12
+ requires: [cli:kk, cli:spur, local-fs]
13
13
  ---
14
14
 
15
15
  # web-automation — discover, enhance, automate with a real browser
16
16
 
17
17
  ## Purpose
18
18
 
19
- Three operations over one method: drive a **real Chromium on the channel's
20
- cached login profile**, record **evidence** (screenshots, DOM signals, JSON),
19
+ Three operational lenses over one unified browser automation architecture: drive a **real Chromium on the named bot's
20
+ cached login profile**, record **verifiable evidence** (screenshots, DOM signals, JSON responses),
21
21
  and never report an outcome that was not observed.
22
22
 
23
- | Operation | When | Output |
24
- | --- | --- | --- |
25
- | **discover** | new/changed site; "figure out this UI" | evidence dir + drift list (`docs/design/<channel>-ui-discovery.md` convention) |
26
- | **enhance** | selectors/signals drifted; a plugin or workflow fails on a live console | fixed selectors/signals in the owning plugin + updated discovery doc, same commit |
27
- | **automate** | a repeatable web task (login-gated save/draft/fill/read) | workflow-driven run with per-step Result/evidence; draft-safe by default |
23
+ | Operation | When | Tier & Substrate | Output |
24
+ | --- | --- | --- | --- |
25
+ | **discover** | New/changed site; "figure out this UI", inspect selectors/shadow roots | **Tier 3** (`kk bot open` + CDP/Playwright) | Evidence dir (`frames/`, `signals.json`) + drift list (`docs/design/<channel>-ui-discovery.md`) |
26
+ | **enhance** | Selectors drifted; a plugin or collection plan fails on a live console | **Tier 2 / 3** (`kk bot exec`, probe script) | Root-cause fix in plugin + updated discovery doc in same commit |
27
+ | **automate** | Repeatable web task (data collection, form fill, draft save, health audit) | **Tier 1 / 2** (`kk bot exec`, `runBotCollection`) | Bounded Result + captured artifacts (draft-safe by default, fail-closed) |
28
28
 
29
- This skill owns **method**. It does not own transport (that is the
30
- `kind: publish`/`ingestion` plugins) or orchestration (that is the engine —
31
- `spur workflow` runs `@gobing-ai/ts-dual-workflow-engine`; see
32
- `docs/design/web-automation-engine.md` for the layering and the YAML).
29
+ This skill owns **method**. It does not own transport (that belongs to `kind: publish` / `ingestion` plugins) or orchestration
30
+ (that is the engine — `spur workflow` running `@gobing-ai/ts-dual-workflow-engine`; see `docs/design/web-automation-engine.md`).
33
31
 
34
- ## Cached login (`auth: cached` — the default)
32
+ ---
33
+
34
+ ## The 3-Tier Web Automation Model
35
+
36
+ To keep workflows clean, safe, and robust against anti-bot friction, all web automation tasks follow a strict 3-tier hierarchy:
37
+
38
+ ```text
39
+ ┌──────────────────────────────────────────────────────────────────────────────┐
40
+ │ Tier 1: Session & Probe Gateway (kk bot CLI) │
41
+ │ • Profile inventory & expiry estimates: kk bot list │
42
+ │ • Headed login & transactional staging: kk bot login <channel> │
43
+ │ • Single-request authenticated queries: kk bot exec <name> <url> │
44
+ │ • Local credential wipe: kk bot logout <name> │
45
+ └──────────────────────────────────────────────────────────────────────────────┘
46
+ │
47
+ ▼
48
+ ┌──────────────────────────────────────────────────────────────────────────────┐
49
+ │ Tier 2: Declarative Headless Automation (kk bot plan / bot-collection) │
50
+ │ • Canonical CLI runner: kk bot plan <bot> <plan.json> [-o <dir>] [-m <sec>]│
51
+ │ • Bounded multi-step JSON plans (navigate, wait, click, scroll, extract) │
52
+ │ • Declarative network response captures (responses: [{pathPrefix}]) │
53
+ │ • Single profile lease + single headless context launch │
54
+ │ • Fail-closed errors: authentication (auth walls) vs empty vs browser │
55
+ │ • Programmatic runner: runBotCollection in @gobing-ai/publish-harness │
56
+ │ • Low-level probe tool: bun packages/publish-harness/scripts/bot-collection-probe.ts │
57
+ └──────────────────────────────────────────────────────────────────────────────┘
58
+ │
59
+ ▼
60
+ ┌──────────────────────────────────────────────────────────────────────────────┐
61
+ │ Tier 3: Deep Interactive Automation (Playwright / CDP under withProfileLease)│
62
+ │ • Headed visual inspection & interaction: kk bot open <name> [--login] │
63
+ │ • File uploads (setInputFiles) & shadow-root coordinate clicks via CDP │
64
+ │ • Live DOM accessibility dumps & interactive selector reverse-engineering │
65
+ │ • Strict lease enforcement: withProfileLease prevents concurrent corruption │
66
+ └──────────────────────────────────────────────────────────────────────────────┘
67
+ ```
68
+
69
+ ---
70
+
71
+ ## Tier 1: Session & Profile Lifecycle (`kk bot`)
72
+
73
+ All browser sessions live under the fixed default root:
74
+
75
+ ```text
76
+ ~/.config/kk/browser-profiles/<name> # persistent profiles (xhs, infoq, juejin, substack, x, wechatmp, etc.)
77
+ ~/.config/kk/browser-transactions/ # crash-safe staging and atomic promotion for logins
78
+ ~/.config/kk/browser-leases/ # shared lease locks preventing concurrent profile corruption
79
+ ```
80
+
81
+ ### CLI Command Reference
35
82
 
36
- Login state is cached **per channel** as a persistent Playwright profile dir —
37
- cookies, localStorage, and session live on disk outside git:
83
+ | Command | Purpose | Input / Options | Behavior |
84
+ | --- | --- | --- | --- |
85
+ | `kk bot list` | Inventory | None | Prints tab-separated columns: `name`, `cached`, `defaultUrl`, `identity`, `lastLogin`, `expires`. Browser-free. |
86
+ | `kk bot login <channel>` / `<url> --save <name>` | Fresh authentication | Channel name or URL (`--save <name>`) | Opens headed in isolated staging; saves adapter-verified evidence or waits for operator confirmation; promotes atomically. |
87
+ | `kk bot open <name> [--login]` | Inspection | Bot name (`--login` optional) | Opens saved default URL headed until operator closes browser. `--login` re-authenticates first. |
88
+ | `kk bot logout <name>` | Wipe credentials | Bot name | Deletes local profile directory behind shared lease. Idempotent. |
89
+ | `kk bot exec <name> <url> [opts]` | Authenticated HTTP | `-X`, `-H`, `-d`, `-o`, `-i`, `-L`, `-f`, `-m` | One headless persistent launch; sends request with profile's cached cookies; writes body as raw bytes. |
90
+ | `kk bot plan <name> <plan.json> [opts]` | Declarative Collection | `-o, --output <dir>`, `-m, --max-time <sec>` | Headless persistent launch under shared lease; runs declarative plan steps and captures; writes artifacts or prints JSON result. |
38
91
 
92
+ ### Session Preflight & Health Check
93
+
94
+ Before triggering automated runs, perform a fast probe. Age-guessing timestamps is banned — probe, don't predict:
95
+
96
+ ```bash
97
+ # 1. Check local profile inventory
98
+ kk bot list
99
+
100
+ # 2. Lightweight authenticated query probe (returns status + headers)
101
+ kk bot exec <channel> <console-url> -i -m 15 -o /tmp/probe.txt
102
+
103
+ # 3. If expired or unauthenticated (HTTP 401/403 or redirect to login):
104
+ kk bot login <channel>
39
105
  ```
40
- ~/.config/kk/browser-profiles/<channel> # xhs, infoq, juejin, substack, x, wechatmp
106
+
107
+ ---
108
+
109
+ ## Tier 2: Declarative Headless Automation (`bot-collection`)
110
+
111
+ For data collection, scraping, site audits, and structured form workflows, **do not write ad-hoc Playwright scripts**.
112
+ Use the declarative `@gobing-ai/publish-harness/bot-collection` substrate:
113
+
114
+ ```ts
115
+ import { runBotCollection } from '@gobing-ai/publish-harness/bot-collection';
116
+
117
+ const result = await runBotCollection('wechatmp', {
118
+ steps: [
119
+ { id: 'nav-home', kind: 'navigate', url: 'https://mp.weixin.qq.com/' },
120
+ { id: 'wait-dashboard', kind: 'wait', selector: '.weui-desktop-layout', timeoutMs: 15_000 },
121
+ { id: 'extract-stats', kind: 'extract', selector: '.weui-desktop-mass__item', format: 'text' },
122
+ ],
123
+ responses: [
124
+ { id: 'appmsg-list', pathPrefix: '/cgi-bin/appmsg' },
125
+ ],
126
+ timeoutMs: 30_000,
127
+ });
41
128
  ```
42
129
 
43
- | `auth` | Behavior |
44
- | --- | --- |
45
- | `cached` (default) | Preflight the profile; reuse it read-only. Launch with `launchPersistentContext` (see `references/browser-launch.md`). No credentials are typed, read, or stored by any script. |
46
- | `fresh` | Profile missing/expired → **headed** launch, operator completes login by hand (QR/OAuth/password), then the profile is the cache again. Never auto-type credentials; OAuth gates need the stealth args. |
130
+ ### Plan Grammar (`BotCollectionPlan`)
131
+
132
+ A plan is a frozen JSON object with `steps[]`, optional `responses[]` captures, and an optional whole-run `timeoutMs`:
133
+
134
+ - **`navigate`**: `{ id, kind: 'navigate', url, waitUntil?: 'load'|'domcontentloaded'|'networkidle'|'commit' }`
135
+ - **`wait`**: `{ id, kind: 'wait', selector?, timeoutMs? }`
136
+ - **`click`**: `{ id, kind: 'click', selector, timeoutMs? }`
137
+ - **`scroll`**: `{ id, kind: 'scroll', deltaY, selector?, timeoutMs? }`
138
+ - **`extract`**: `{ id, kind: 'extract', selector, format: 'text'|'html'|'count', timeoutMs? }`
139
+ - **`request`**: `{ id, kind: 'request', url, method?: 'GET'|'POST', headers?, data? }`
140
+ - **`responses`**: `[{ id, origin?, pathPrefix, optional?: boolean }]`
47
141
 
48
- Preflight before any drive — cheap and headless:
142
+ ### Fail-Closed Integrity Rules
143
+
144
+ - **`authentication`**: Fails immediately when an explicit 401/403 occurs on a request, navigation, or required response, or when the final navigation URL path matches `/login|signin|sign-in|auth|sso/i`. Action: alert operator to run `kk bot login <channel>`.
145
+ - **`empty`**: Fails immediately when a declared non-optional capture receives no response, or an `extract`/`request` step yields an empty body. Prevents client-side rendered SPA shells from masquerading as a successful quiet run.
146
+ - **`browser`**: Transport faults, timeouts, navigation drops, and DOM exceptions.
147
+
148
+ ### Running Plans via CLI (`kk bot plan`)
149
+
150
+ Execute a collection plan directly from the terminal or a workflow script:
151
+
152
+ ```bash
153
+ # Run plan and stream JSON result to stdout
154
+ kk bot plan <channel> <path/to/plan.json> -m 60
155
+
156
+ # Run plan and write 0600 artifact files + result.json / error.json into output dir
157
+ kk bot plan <channel> <path/to/plan.json> -o <path/to/evidence-dir> -m 60
158
+ ```
159
+
160
+ Exit codes: `0` = success, `1` = operational failure (`authentication`, `empty`, or `browser`), `2` = usage error (invalid JSON, missing profile, invalid timeout).
161
+
162
+ Low-level debugging can also use the repository probe script:
49
163
 
50
164
  ```bash
51
- test -d ~/.config/kk/browser-profiles/<channel> && echo cached || echo fresh-required
165
+ bun packages/publish-harness/scripts/bot-collection-probe.ts \
166
+ --profile <name> \
167
+ --plan <path/to/plan.json> \
168
+ --out <path/to/evidence-dir>
52
169
  ```
53
170
 
54
- A profile dir existing is necessary, not sufficient: before `automate` runs,
55
- do a read-only session probe (open the console URL, check a signed-in-only
56
- selector) and route to `fresh` on failure. Age-guessing timestamps is banned —
57
- probe, don't predict.
58
-
59
- ## Method
60
-
61
- 1. **Launch** — `chromium.launchPersistentContext(profileDir, { headless, args: [CDP port, stealth] })`.
62
- Decision table and the exact snippet: `references/browser-launch.md`.
63
- 2. **Discover** — screenshot first, then read DOM (selectors, shadow roots,
64
- iframes, counts) with small throwaway scripts; save frames + a signals JSON
65
- into the run's evidence dir; write/refresh the channel's
66
- `docs/design/<channel>-ui-discovery.md` (selectors, drift list, date).
67
- 3. **Enhance** — diff observed DOM against the plugin's encoded
68
- selectors/signals; fix the plugin (root cause — one shared helper, not
69
- every caller), re-walk to prove it, update the discovery doc in the same
70
- commit.
71
- 4. **Automate** — encode the flow as engine workflow nodes (preflight →
72
- launch → drive → verify → record); `drive` steps run the measured
73
- selectors and **assert an observed outcome signal** (toast, counter delta,
74
- URL change). An unobserved outcome is reported as failure, never success.
75
- 5. **Confidence-verified verdicts** — every outcome verdict carries
76
- `confidence: HIGH | MEDIUM | LOW` (HIGH = measured signal observed;
77
- MEDIUM = indirect signal, e.g. counter delta without toast; LOW = partial
78
- evidence, one check passed). **Missing or out-of-enum confidence is itself
79
- a failure** — the verify gate blocks it, mirroring the safety-verdict
80
- convention (`kk-daily-ai-voice.md`, 20261003).
81
-
82
- ## Safety rails (non-negotiable)
83
-
84
- - **Never click publish/send/submit/delete** unless the task explicitly
85
- authorizes live publishing. Default target: draft/save/暂存.
86
- - Bot-walled channels (xhs) and fresh logins stay **headed**; headless only
87
- after the session is proven.
88
- - Every reported success cites observed evidence (signal, screenshot, JSON).
89
- No evidence → it is a failure with the reason — and every verdict states
90
- its `HIGH | MEDIUM | LOW` confidence; a verdict without a valid level
91
- fails the gate like a missing verdict.
92
- - Secrets stay out of repo, YAML, and vars. Login state lives only in the
93
- profile dir under `~/.config` (outside git); scripts print paths, never tokens.
94
- - One browser per channel/profile at a time; close contexts you launched.
95
-
96
- ## Engine invocation (summary)
97
-
98
- Orchestration is a `kind: transition-flow` workflow; `startable` nodes let a
99
- run re-enter at `drive` after a manual login without redoing earlier nodes.
100
- Full YAML, node contract, and the option-B (in-process runners) design:
101
- `docs/design/web-automation-engine.md`.
171
+ ---
172
+
173
+ ## Tier 3: Deep Interactive Automation (Playwright / CDP)
174
+
175
+ When an activity requires capabilities outside Tier 2 — specifically:
176
+ 1. **Interactive UI discovery** (navigating unknown layouts, recording screenshots).
177
+ 2. **Shadow-root component piercing** (e.g. XHS creator console shadow boundaries).
178
+ 3. **File uploads** (attaching images/videos via `page.setInputFiles`).
179
+ 4. **Canvas / complex coordinate dragging**.
180
+
181
+ You MUST wrap the run inside the shared profile lease to prevent profile corruption:
182
+
183
+ ```ts
184
+ import { chromium } from 'playwright';
185
+ import { withProfileLease } from '@gobing-ai/publish-harness';
186
+
187
+ const profileName = 'xhs';
188
+ const profileDir = `${process.env.HOME}/.config/kk/browser-profiles/${profileName}`;
189
+ const port = process.env.KK_WEB_CDP ?? '9222';
190
+
191
+ await withProfileLease(profileName, async () => {
192
+ const ctx = await chromium.launchPersistentContext(profileDir, {
193
+ headless: process.env.KK_WEB_HEADLESS === 'true',
194
+ viewport: null,
195
+ args: [
196
+ `--remote-debugging-port=${port}`,
197
+ '--disable-blink-features=AutomationControlled',
198
+ ],
199
+ });
200
+ try {
201
+ const page = ctx.pages()[0] ?? (await ctx.newPage());
202
+ // Drive page actions, upload files, or attach agent-browser via CDP...
203
+ } finally {
204
+ await ctx.close();
205
+ }
206
+ });
207
+ ```
208
+
209
+ ---
210
+
211
+ ## Playbook: Common Automation Scenarios
212
+
213
+ ### Scenario A: Site Discovery & Reverse-Engineering
214
+ 1. Check cached session: `kk bot list`.
215
+ 2. Open target page headed: `kk bot open <channel>`.
216
+ 3. Inspect DOM, capture layout, dump accessibility tree, take screenshots.
217
+ 4. Document selectors, shadow roots, and observed signals into `docs/design/<channel>-ui-discovery.md`.
218
+ 5. Close window; `kk bot open` outputs `<name>\tclosed`.
219
+
220
+ ### Scenario B: Lightweight Authenticated Scraping
221
+ 1. Send direct authenticated request:
222
+ ```bash
223
+ kk bot exec <channel> "https://target.com/api/data" -H "Accept: application/json" -o data.json
224
+ ```
225
+ 2. If the endpoint returns client-rendered HTML shell rather than JSON, escalate to **Scenario C**.
226
+
227
+ ### Scenario C: Declarative Headless Data Collection
228
+ 1. Draft a `plan.json` with required navigation, waits, and response captures.
229
+ 2. Probe run:
230
+ ```bash
231
+ bun packages/publish-harness/scripts/bot-collection-probe.ts --profile <channel> --plan plan.json --out .spur/run/evidence
232
+ ```
233
+ 3. Verify `result.json` shows all artifacts captured with non-empty bodies.
234
+ 4. Wire into ingestion plugin or workflow.
235
+
236
+ ### Scenario D: Broken Selector Repair (`enhance`)
237
+ 1. Replay failing plan or run `kk bot exec`.
238
+ 2. Inspect `error.json` / diagnostics (is it `authentication`, `empty`, or `browser`?).
239
+ 3. If selector changed, launch `kk bot open <channel>`, observe the updated DOM element.
240
+ 4. Update selector in shared helper, re-verify with probe script, commit alongside discovery doc.
241
+
242
+ ---
243
+
244
+ ## Safety Rails (Non-Negotiable)
245
+
246
+ - **Draft-Safe by Default**: Never click publish/send/submit/delete unless the task explicitly authorizes live actions.
247
+ - **Lease Invariant**: Every browser context touching `~/.config/kk/browser-profiles/<name>` must be acquired through `kk bot`, `runBotCollection`, or `withProfileLease`. Direct un-leased `launchPersistentContext` is banned.
248
+ - **Fail Closed on Quiet Failures**: Missing data and auth walls must fail loud (`authentication` or `empty`). Never treat an empty SPA shell or an auth redirect as a normal quiet response.
249
+ - **Headed vs Headless**:
250
+ - Headless: Session probes (`kk bot list`), authenticated queries (`kk bot exec`), declarative collection (`runBotCollection`).
251
+ - Headed: First-time logins (`kk bot login`), interactive discovery (`kk bot open`), and bot-walled platforms (XHS).
252
+ - **Secrets Invariant**: Zero credentials in repo, code, or logs. All state lives exclusively in `~/.config/kk/` outside Git.
253
+ - **Confidence-Verified Verdicts**: Every workflow outcome verdict must declare `confidence: HIGH | MEDIUM | LOW`. Missing or out-of-enum confidence halts execution.
@@ -1,71 +1,160 @@
1
1
  # Browser launch & cached-login recipes
2
2
 
3
- Everything here is measured on the kk channels (E92 discovery, 2026-10).
4
- The launcher is the proven pattern from `.spur/run/discovery/xhs-launch.ts` /
5
- `walk-wechatmp*.ts`; the profile matrix is the live
6
- `~/.config/kk/browser-profiles/` tree.
3
+ Everything here is measured on the kk channels (E92 discovery, A22 named bots, and 0346 collection runner).
4
+ The canonical tool for named session management is `kk bot`; the underlying libraries are `@gobing-ai/publish-harness`
5
+ and `@gobing-ai/publish-harness/bot-collection`.
6
+
7
+ ## Named bot CLI suite (`kk bot`)
8
+
9
+ The profile root `~/.config/kk/browser-profiles/<name>` is managed safely with transactional staging and leases:
10
+
11
+ ```bash
12
+ kk bot list # List cached profiles, last login, and estimated expiry
13
+ kk bot login <channel> # Headed interactive login (delegates to matched channel adapter)
14
+ kk bot login <url> --save <name> # Generic headed login (operator terminal confirmation before save)
15
+ kk bot open <name> [--login] # Open saved page headed until window closes (--login re-authenticates)
16
+ kk bot logout <name> # Delete local profile directory behind the shared lease
17
+ kk bot exec <name> <url> [opts] # Single authenticated HTTP request with cached cookies (curl subset)
18
+ kk bot plan <name> <plan.json> [opts] # Execute declarative collection plan under lease (-o, -m)
19
+ ```
20
+
21
+ ## Rendered collection runner substrate (task 0346)
7
22
 
8
- ## The launcher (persistent context + CDP + stealth)
23
+ For automated structured web data collection without raw browser scripting, use `@gobing-ai/publish-harness/bot-collection`.
24
+ It executes a declarative JSON plan under a single profile lease and a single headless persistent context:
25
+
26
+ ```ts
27
+ import { runBotCollection } from '@gobing-ai/publish-harness/bot-collection';
28
+
29
+ const result = await runBotCollection('xhs', {
30
+ steps: [
31
+ { id: 'home', kind: 'navigate', url: 'https://creator.xiaohongshu.com/' },
32
+ { id: 'wait-nav', kind: 'wait', selector: '.creator-menu' },
33
+ { id: 'click-menu', kind: 'click', selector: '.creator-menu >> text="Publish"' },
34
+ { id: 'extract-data', kind: 'extract', selector: '.data-card', format: 'text' },
35
+ ],
36
+ responses: [
37
+ { id: 'creator-info', pathPrefix: '/api/galaxy/creator/info' },
38
+ ],
39
+ timeoutMs: 45_000,
40
+ });
41
+ ```
42
+
43
+ ### Plan step grammar (`BotCollectionStep`)
44
+
45
+ - `{ id, kind: 'navigate', url, waitUntil?: 'load'|'domcontentloaded'|'networkidle'|'commit' }`
46
+ - `{ id, kind: 'wait', selector?, timeoutMs? }`
47
+ - `{ id, kind: 'click', selector, timeoutMs? }`
48
+ - `{ id, kind: 'scroll', deltaY, selector?, timeoutMs? }`
49
+ - `{ id, kind: 'extract', selector, format: 'text'|'html'|'count', timeoutMs? }`
50
+ - `{ id, kind: 'request', url, method?: 'GET'|'POST', headers?, data? }`
51
+ - Capture: `responses: [{ id, origin?, pathPrefix, optional?: boolean }]`
52
+
53
+ ### Fail-closed error contract
54
+
55
+ - `authentication`: Explicit 401/403 (request, navigation, or required response) or final navigation URL matching `/login|signin|sign-in|auth|sso/i`.
56
+ - `empty`: A declared non-optional capture matched no response, or an extract/request step yielded an empty body. Prevents client-side rendered SPA shells from masquerading as a quiet day.
57
+ - `browser`: Network/transport error, timeout, or uncaught DOM exception.
58
+
59
+ ### CLI Execution (`kk bot plan`) and Probe Script
60
+
61
+ Run collection plans directly with the CLI:
62
+
63
+ ```bash
64
+ # Output JSON result to stdout
65
+ kk bot plan <channel> <plan.json> -m 60
66
+
67
+ # Write individual 0600 artifacts and result.json / error.json into directory
68
+ kk bot plan <channel> <plan.json> -o <output-dir> -m 60
69
+ ```
70
+
71
+ Low-level debugging can also use `bot-collection-probe.ts`:
72
+
73
+ ```bash
74
+ bun packages/publish-harness/scripts/bot-collection-probe.ts \
75
+ --profile <name> \
76
+ --plan <plan.json> \
77
+ --out <output-dir>
78
+ ```
79
+
80
+ Outputs artifact files with 0600 permissions, `result.json` (metadata without bodies), or `error.json` on failure.
81
+
82
+ ## Direct Playwright launcher (persistent context + lease + CDP + stealth)
83
+
84
+ When interactive discovery or custom CDP scripts are needed, always wrap execution with the profile lease:
9
85
 
10
86
  ```ts
11
87
  import { chromium } from 'playwright';
88
+ import { withProfileLease } from '@gobing-ai/publish-harness';
12
89
 
13
- const profileDir = process.env.KK_WEB_PROFILE ?? `${process.env.HOME}/.config/kk/browser-profiles/<channel>`;
90
+ const profileName = process.env.KK_WEB_BOT ?? '<channel>';
91
+ const profileDir = `${process.env.HOME}/.config/kk/browser-profiles/${profileName}`;
14
92
  const port = process.env.KK_WEB_CDP ?? '9222';
15
93
 
16
- const ctx = await chromium.launchPersistentContext(profileDir, {
17
- headless: process.env.KK_WEB_HEADLESS === 'true', // default headed
18
- viewport: null,
19
- args: [
20
- `--remote-debugging-port=${port}`, // agent-browser / external CDP clients attach here
21
- '--disable-blink-features=AutomationControlled', // required at OAuth/bot-wall gates
22
- ],
94
+ await withProfileLease(profileName, async () => {
95
+ const ctx = await chromium.launchPersistentContext(profileDir, {
96
+ headless: process.env.KK_WEB_HEADLESS === 'true', // default headed
97
+ viewport: null,
98
+ args: [
99
+ `--remote-debugging-port=${port}`, // agent-browser / external CDP clients attach here
100
+ '--disable-blink-features=AutomationControlled', // required at OAuth/bot-wall gates
101
+ ],
102
+ });
103
+ const page = ctx.pages()[0] ?? (await ctx.newPage());
104
+ console.log(JSON.stringify({ cdpPort: port, profile: profileDir, url: page.url() }));
105
+ process.on('SIGTERM', async () => { await ctx.close(); process.exit(0); });
106
+ await new Promise(() => {}); // stay up for CDP drivers
23
107
  });
24
- const page = ctx.pages()[0] ?? (await ctx.newPage());
25
- console.log(JSON.stringify({ cdpPort: port, profile: profileDir, url: page.url() }));
26
- process.on('SIGTERM', async () => { await ctx.close(); process.exit(0); });
27
- await new Promise(() => {}); // stay up for CDP drivers
28
108
  ```
29
109
 
30
110
  - The **profile dir is the credential cache** — cookies + storage survive runs;
31
111
  nothing else is persisted, no tokens are printed.
32
112
  - `agent-browser` (or any CDP client) attaches to the printed port; plain
33
113
  Playwright scripts can also import the same process's `page`.
34
- - One context per channel. Two concurrent launches on one profile corrupt it.
114
+ - **Profile lease required**: `withProfileLease` ensures single-process access.
115
+ Two concurrent launches on one profile corrupt it.
35
116
 
36
117
  ## Headless vs headed
37
118
 
38
119
  | Situation | Mode | Why |
39
120
  | --- | --- | --- |
40
- | Session probe / preflight | headless OK | read-only, cheap |
121
+ | Session probe / preflight (`kk bot list`, `check`) | headless OK | read-only, cheap |
122
+ | `kk bot exec` single-request fetch | headless | persistent context attaches cookies automatically |
123
+ | `runBotCollection` automated collection | headless | bounded steps and required response captures |
41
124
  | Automated drive on a proven session | headless OK | measured: infoq/juejin/substack/x/wechatmp drives work headless post-login |
42
- | `auth: fresh` login | **headed** | operator completes QR/OAuth by hand |
43
- | xhs any operation | **headed** | bot-walls headless loads (measured); shadow-root clicks need CDP anyway |
125
+ | `kk bot login` / fresh authentication | **headed** | operator completes QR/OAuth/password by hand |
126
+ | `kk bot open` inspection | **headed** | operator views page and DOM |
127
+ | xhs any publish/drive | **headed** | bot-walls headless loads (measured); shadow-root clicks need CDP anyway |
44
128
  | Any step that might trip a bot wall first time | **headed** | watch it once, record, then automate |
45
129
 
46
130
  ## Profile matrix (as shipped)
47
131
 
48
- | Channel | Profile dir | Notes |
49
- | --- | --- | --- |
50
- | xhs | `browser-profiles/xhs` | headed always; shadow-root controls via CDP box models |
51
- | infoq | `browser-profiles/infoq` | draft save = `/draft/<id>` URL |
52
- | juejin | `browser-profiles/juejin` | draft id from post-save URL |
53
- | substack | `browser-profiles/substack` | robinmin.substack.com; post id in `/publish/post/<id>` |
54
- | x | `browser-profiles/x` | article edit URL carries draft id |
55
- | wechatmp | `browser-profiles/wechatmp` | QR login; draft id from draft/add response + editor URL |
132
+ | Channel | Profile dir | Login command | Notes |
133
+ | --- | --- | --- | --- |
134
+ | xhs | `browser-profiles/xhs` | `kk bot login xhs` | headed always; shadow-root controls via CDP box models |
135
+ | infoq | `browser-profiles/infoq` | `kk bot login infoq` | draft save = `/draft/<id>` URL |
136
+ | juejin | `browser-profiles/juejin` | `kk bot login juejin` | draft id from post-save URL |
137
+ | substack | `browser-profiles/substack` | `kk bot login substack` | robinmin.substack.com; post id in `/publish/post/<id>` |
138
+ | x | `browser-profiles/x` | `kk bot login x` | article edit URL carries draft id |
139
+ | wechatmp | `browser-profiles/wechatmp` | `kk bot login wechatmp` | QR login; draft id from draft/add response + editor URL |
140
+ | weibo | `browser-profiles/weibo` | `kk bot login weibo` | 头条文章 editor |
56
141
 
57
- New channel ⇒ one-time headed `fresh` login on that dir, then it becomes the cache.
142
+ New channel ⇒ one-time headed `kk bot login <url> --save <name>` or `kk bot login <channel>`, then it becomes the cache.
58
143
 
59
144
  ## Session probe (cached-auth validity)
60
145
 
61
- Don't age-guess — probe a signed-in-only selector on the channel's console URL:
146
+ Don't age-guess — probe a signed-in-only selector or run the plugin's probe:
62
147
 
63
- ```ts
64
- const ok = await page.locator('<signed-in-only selector>').first().isVisible({ timeout: 10_000 });
65
- if (!ok) throw new Error('auth=expired: fresh login required');
66
- ```
148
+ ```bash
149
+ # Fast metadata check
150
+ kk bot list
151
+
152
+ # Lightweight authenticated request test
153
+ kk bot exec <name> <console-url>
67
154
 
68
- Per-channel selectors live in the channel's `docs/design/<channel>-ui-discovery.md`.
155
+ # Reopen or re-authenticate
156
+ kk bot open <name> --login
157
+ ```
69
158
 
70
159
  ## storageState vs profile dir
71
160
 
@@ -84,3 +173,5 @@ assumed.
84
173
  timeout (e.g. wechatmp `.weui-desktop-toast` ≤ 15 s) — loud on miss.
85
174
  - Always `ctx.close()` on SIGTERM/exit paths; orphaned Chromiums hold the
86
175
  profile lock and the next run fails confusingly.
176
+ - In collection plans (`runBotCollection`), differentiate `authentication` (action: run `kk bot login`),
177
+ `empty` (action: check site selector/API path drift), and `browser` (action: investigate transport/timeout).
@@ -50,12 +50,9 @@ vars:
50
50
  cover_fallback: ""
51
51
  # 0219 R3: image-gen vendor fallback for the XHS render; "" restores strict fail-fast.
52
52
  xhs_fallback: "true"
53
- # 2026-10-21 operator decision: OFF for now (was on since 2026-09-29). The plugin reported today's
54
- # draft as saved (`草稿箱` 12 -> 13 plus a scoped `保存成功` toast) while the console showed the
55
- # operator no draft, so the channel's success evidence is not trustworthy until that is resolved.
56
- # Flip back to "true" to resume the card plan/render/draft series; `xhs_live` stays a separate
57
- # launch-time switch and nothing else in the run depends on this target.
58
- xhs_enabled: "false"
53
+ # 2026-11-09 operator decision: turned on by default for 上传图文 (cards) and 发播客 (episode note);
54
+ # `xhs_live` stays a separate launch-time switch (draft by default) and nothing else in the run depends on this target.
55
+ xhs_enabled: "true"
59
56
  xhs_cards: "4"
60
57
  xhs_style: "xhs-knowledge"
61
58
  xhs_provider: "minimax"
@@ -391,6 +388,7 @@ nodes:
391
388
  input: &article-write-input |
392
389
  Read ${vars.work_dir}/21-plan/${vars.run_date}_03_plan_plan.json — metadata.docs (Doc[]) is the translated plan; don't edit it.
393
390
  Write vivid simplified-Chinese tech news for 咸话咸说之每日AI乱炖 (column 每日AI乱炖, host Robin). No title line. Open with exactly「这里是由Robin为你带来的咸话咸说之每日AI乱炖」。Never「我是咸话咸说」or an AI-production footer. Each ## section: what happened, why it matters, one concrete detail — a human reporter, not a telegram. Inline [来源:<label>](<sourceUri>). Close by tying the items together.
391
+ 标题纪律(站点 digest 契约,不满足则 refine 拒绝发布):每个 ## 标题必须以主体起头,第一个逗号/冒号之前的那段不超过 14 个汉字(英文专名与数字各算 1 个字位),且是一个名词短语——如「TSMC 把存储搬进逻辑芯片」「OpenAI 新证明」「Qwen3.5 纯汇编推理」。禁止以动词或「上周/一条/再」这类虚词起头的整句式标题;冒号之后可自由展开。
394
392
  选题边界:只写技术内容;某条主线若属政治/外交或贸易管制/军事/公共政策争议/社会公共事件,跳过该条并在文末加一行「> 注:<N> 条条目因选题边界未写入:<id>(<原因>)、<id>(<原因>)」。
395
393
  事实纪律:每个事实取自该条计划内容,不虚构、不引他处;数字与主体不得放大;引语只能归属其原句主语;保留「据报道/称/承诺」等限定词;导语/收束不加计划外程度词;材料不足就少写或不写,不凑数。
396
394
  Narrative register — smooth transitions, no list-like bulleting, no scaffold labels (「可做角度」「背景」「潜在影响」), no hedging (「读者宜另行核实」「仍待验证」). 「」 quotes.