@jackwener/opencli 0.4.2 → 0.4.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/{CLI-CREATOR.md → CLI-EXPLORER.md} +15 -11
- package/CLI-ONESHOT.md +216 -0
- package/LICENSE +28 -0
- package/README.md +114 -63
- package/README.zh-CN.md +115 -63
- package/SKILL.md +25 -6
- package/dist/browser.d.ts +53 -10
- package/dist/browser.js +491 -111
- package/dist/browser.test.d.ts +1 -0
- package/dist/browser.test.js +56 -0
- package/dist/build-manifest.js +4 -0
- package/dist/cli-manifest.json +279 -3
- package/dist/clis/boss/search.js +186 -30
- package/dist/clis/twitter/delete.d.ts +1 -0
- package/dist/clis/twitter/delete.js +73 -0
- package/dist/clis/twitter/followers.d.ts +1 -0
- package/dist/clis/twitter/followers.js +104 -0
- package/dist/clis/twitter/following.d.ts +1 -0
- package/dist/clis/twitter/following.js +90 -0
- package/dist/clis/twitter/like.d.ts +1 -0
- package/dist/clis/twitter/like.js +69 -0
- package/dist/clis/twitter/notifications.d.ts +1 -0
- package/dist/clis/twitter/notifications.js +109 -0
- package/dist/clis/twitter/post.d.ts +1 -0
- package/dist/clis/twitter/post.js +63 -0
- package/dist/clis/twitter/reply.d.ts +1 -0
- package/dist/clis/twitter/reply.js +57 -0
- package/dist/clis/v2ex/daily.d.ts +1 -0
- package/dist/clis/v2ex/daily.js +98 -0
- package/dist/clis/v2ex/me.d.ts +1 -0
- package/dist/clis/v2ex/me.js +99 -0
- package/dist/clis/v2ex/notifications.d.ts +1 -0
- package/dist/clis/v2ex/notifications.js +72 -0
- package/dist/doctor.d.ts +50 -0
- package/dist/doctor.js +372 -0
- package/dist/doctor.test.d.ts +1 -0
- package/dist/doctor.test.js +114 -0
- package/dist/main.js +47 -5
- package/dist/output.test.d.ts +1 -0
- package/dist/output.test.js +20 -0
- package/dist/registry.d.ts +4 -0
- package/dist/registry.js +1 -0
- package/dist/runtime.d.ts +3 -1
- package/dist/runtime.js +2 -2
- package/package.json +2 -2
- package/src/browser.test.ts +77 -0
- package/src/browser.ts +541 -99
- package/src/build-manifest.ts +4 -0
- package/src/clis/boss/search.ts +196 -29
- package/src/clis/twitter/delete.ts +78 -0
- package/src/clis/twitter/followers.ts +119 -0
- package/src/clis/twitter/following.ts +105 -0
- package/src/clis/twitter/like.ts +74 -0
- package/src/clis/twitter/notifications.ts +119 -0
- package/src/clis/twitter/post.ts +68 -0
- package/src/clis/twitter/reply.ts +62 -0
- package/src/clis/v2ex/daily.ts +105 -0
- package/src/clis/v2ex/me.ts +103 -0
- package/src/clis/v2ex/notifications.ts +77 -0
- package/src/doctor.test.ts +133 -0
- package/src/doctor.ts +424 -0
- package/src/main.ts +47 -4
- package/src/output.test.ts +27 -0
- package/src/registry.ts +5 -0
- package/src/runtime.ts +2 -1
|
@@ -1,11 +1,15 @@
|
|
|
1
|
-
# CLI-
|
|
1
|
+
# CLI-EXPLORER — 适配器探索式开发完全指南
|
|
2
2
|
|
|
3
3
|
> 本文档教你(或 AI Agent)如何为 OpenCLI 添加一个新网站的命令。
|
|
4
4
|
> 从零到发布,覆盖 API 发现、方案选择、适配器编写、测试验证全流程。
|
|
5
5
|
|
|
6
|
+
> [!TIP]
|
|
7
|
+
> **只想为一个具体页面快速生成一个命令?** 看 [CLI-ONESHOT.md](./CLI-ONESHOT.md)(~150 行,4 步搞定)。
|
|
8
|
+
> 本文档适合从零探索一个新站点的完整流程。
|
|
9
|
+
|
|
6
10
|
---
|
|
7
11
|
|
|
8
|
-
##
|
|
12
|
+
## AI Agent 开发者必读:用 Playwright MCP Bridge 探索
|
|
9
13
|
|
|
10
14
|
> [!CAUTION]
|
|
11
15
|
> **你(AI Agent)必须通过 Playwright MCP Bridge 打开浏览器去访问目标网站!**
|
|
@@ -38,7 +42,7 @@
|
|
|
38
42
|
| 遇到 HTTP 200 但空数据就放弃 | 检查是否需要 Wbi 签名或 Cookie 鉴权 |
|
|
39
43
|
| 完全依赖 `__INITIAL_STATE__` 拿所有数据 | `__INITIAL_STATE__` 只有首屏数据,深层数据要调 API |
|
|
40
44
|
|
|
41
|
-
###
|
|
45
|
+
### 实战成功案例:5 分钟实现「关注列表」适配器
|
|
42
46
|
|
|
43
47
|
以下是用上述工作流实际发现 Bilibili 关注列表 API 的完整过程:
|
|
44
48
|
|
|
@@ -51,7 +55,7 @@
|
|
|
51
55
|
fetch('/x/relation/followings?vmid=137702077&pn=1&ps=5', {credentials:'include'})
|
|
52
56
|
→ { code: 0, data: { total: 1342, list: [{mid, uname, sign, ...}] } }
|
|
53
57
|
4. 结论:标准 Cookie API,无需 Wbi 签名
|
|
54
|
-
5. 写 following.ts → 一次构建通过
|
|
58
|
+
5. 写 following.ts → 一次构建通过
|
|
55
59
|
```
|
|
56
60
|
|
|
57
61
|
**关键决策点**:
|
|
@@ -174,7 +178,7 @@ opencli cascade https://api.example.com/hot
|
|
|
174
178
|
|
|
175
179
|
## Step 2.5: 准备工作(写代码之前)
|
|
176
180
|
|
|
177
|
-
###
|
|
181
|
+
### 先找模板:从最相似的现有适配器开始
|
|
178
182
|
|
|
179
183
|
**不要从零开始写**。先看看同站点已有哪些适配器:
|
|
180
184
|
|
|
@@ -207,7 +211,7 @@ cat src/clis/<site>/feed.ts # 读最相似的那个
|
|
|
207
211
|
- 含 `/wbi/` 或 `w_rid=` → 必须用 `apiGet(..., { signed: true })`
|
|
208
212
|
- 不含 → 直接用 `fetchJson`
|
|
209
213
|
|
|
210
|
-
>
|
|
214
|
+
> 其他站点(Twitter、小红书等)暂无专用 SDK,直接用 `page.evaluate` + `fetch` 即可。
|
|
211
215
|
|
|
212
216
|
---
|
|
213
217
|
|
|
@@ -252,7 +256,7 @@ func: async (page, kwargs) => {
|
|
|
252
256
|
},
|
|
253
257
|
```
|
|
254
258
|
|
|
255
|
-
>
|
|
259
|
+
> 大多数站点的 `ps` 上限是 20~50。超过会被静默截断或返回错误。
|
|
256
260
|
|
|
257
261
|
### 方式 A: YAML Pipeline(声明式,推荐)
|
|
258
262
|
|
|
@@ -530,13 +534,13 @@ cli({
|
|
|
530
534
|
|
|
531
535
|
> **拦截核心思路**:不自己构造签名,而是利用 `installInterceptor` 劫持网站自己的 `XMLHttpRequest` 和 `fetch`,让网站发请求,我们直接在底层取出解析好的 `response.json()`。
|
|
532
536
|
|
|
533
|
-
>
|
|
537
|
+
> **级联请求**(如 BVID→CID→字幕)的完整模板和要点见下方[进阶模式: 级联请求](#进阶模式-级联请求-cascading-requests)章节。
|
|
534
538
|
|
|
535
539
|
---
|
|
536
540
|
|
|
537
541
|
## Step 4: 测试
|
|
538
542
|
|
|
539
|
-
>
|
|
543
|
+
> **构建通过 ≠ 功能正常**。`npm run build` 只验证 TypeScript / YAML 语法,不验证运行时行为。
|
|
540
544
|
> 每个新命令 **必须实际运行** 并确认输出正确后才算完成。
|
|
541
545
|
|
|
542
546
|
### 必做清单
|
|
@@ -555,7 +559,7 @@ opencli mysite hot --limit 3 -f json # JSON 输出确认字段完整
|
|
|
555
559
|
|
|
556
560
|
### tap 步骤调试(intercept 策略专用)
|
|
557
561
|
|
|
558
|
-
>
|
|
562
|
+
> **不要猜 store name / action name**。先用 evaluate 探索,再写 YAML。
|
|
559
563
|
|
|
560
564
|
#### Step 1: 列出所有 Pinia store
|
|
561
565
|
|
|
@@ -614,7 +618,7 @@ opencli list | grep mysite # 确认注册
|
|
|
614
618
|
git add src/clis/mysite/ && git commit -m "feat(mysite): add hot" && git push
|
|
615
619
|
```
|
|
616
620
|
|
|
617
|
-
>
|
|
621
|
+
> **架构理念**:OpenCLI 内建 **Zero-Dependency jq** 数据流 — 所有解析在 `evaluate` 的原生 JS 内完成,外层 YAML 用 `select`/`map` 提取,无需依赖系统 `jq` 二进制。
|
|
618
622
|
|
|
619
623
|
---
|
|
620
624
|
|
package/CLI-ONESHOT.md
ADDED
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
# CLI-ONESHOT — 单点快速 CLI 生成
|
|
2
|
+
|
|
3
|
+
> 给一个 URL + 一句话描述,4 步生成一个 CLI 命令。
|
|
4
|
+
> 完整探索式开发请看 [CLI-EXPLORER.md](./CLI-EXPLORER.md)。
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## 输入
|
|
9
|
+
|
|
10
|
+
| 项目 | 示例 |
|
|
11
|
+
|------|------|
|
|
12
|
+
| **URL** | `https://x.com/jakevin7/lists` |
|
|
13
|
+
| **Goal** | 获取我的 Twitter Lists |
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 流程
|
|
18
|
+
|
|
19
|
+
### Step 1: 打开页面 + 抓包
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
1. browser_navigate → 打开目标 URL
|
|
23
|
+
2. 等待 3-5 秒(让页面加载完、API 请求触发)
|
|
24
|
+
3. browser_network_requests → 筛选 JSON API
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
**关键**:只关注返回 `application/json` 的请求,忽略静态资源。
|
|
28
|
+
如果没有自动触发 API,手动点击目标按钮/标签再抓一次。
|
|
29
|
+
|
|
30
|
+
### Step 2: 锁定一个接口
|
|
31
|
+
|
|
32
|
+
从抓包结果中找到**那个**目标 API。看这几个字段:
|
|
33
|
+
|
|
34
|
+
| 字段 | 关注什么 |
|
|
35
|
+
|------|----------|
|
|
36
|
+
| URL | API 路径 pattern(如 `/i/api/graphql/xxx/ListsManagePinTimeline`) |
|
|
37
|
+
| Method | GET / POST |
|
|
38
|
+
| Headers | 有 Cookie? Bearer? CSRF? 自定义签名? |
|
|
39
|
+
| Response | 数据在哪个路径(如 `data.list.lists`) |
|
|
40
|
+
|
|
41
|
+
### Step 3: 验证接口能复现
|
|
42
|
+
|
|
43
|
+
在 `browser_evaluate` 中用 `fetch` 复现请求:
|
|
44
|
+
|
|
45
|
+
```javascript
|
|
46
|
+
// Tier 2 (Cookie): 大多数情况
|
|
47
|
+
fetch('/api/endpoint', { credentials: 'include' }).then(r => r.json())
|
|
48
|
+
|
|
49
|
+
// Tier 3 (Header): 如 Twitter 需要额外 header
|
|
50
|
+
const ct0 = document.cookie.match(/ct0=([^;]+)/)?.[1];
|
|
51
|
+
fetch('/api/endpoint', {
|
|
52
|
+
headers: { 'Authorization': 'Bearer ...', 'X-Csrf-Token': ct0 },
|
|
53
|
+
credentials: 'include'
|
|
54
|
+
}).then(r => r.json())
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
如果 fetch 能拿到数据 → 用 YAML 或简单 TS adapter。
|
|
58
|
+
如果 fetch 拿不到(签名/风控)→ 用 intercept 策略。
|
|
59
|
+
|
|
60
|
+
### Step 4: 套模板,生成 adapter
|
|
61
|
+
|
|
62
|
+
根据 Step 3 判定的策略,选一个模板生成文件。
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## 认证速查
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
fetch(url) 直接能拿到? → Tier 1: public (YAML, browser: false)
|
|
70
|
+
fetch(url, {credentials:'include'})? → Tier 2: cookie (YAML)
|
|
71
|
+
加 Bearer/CSRF header 后拿到? → Tier 3: header (TS)
|
|
72
|
+
都不行,但页面自己能请求成功? → Tier 4: intercept (TS, installInterceptor)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## 模板
|
|
78
|
+
|
|
79
|
+
### YAML — Cookie/Public(最简)
|
|
80
|
+
|
|
81
|
+
```yaml
|
|
82
|
+
# src/clis/<site>/<name>.yaml
|
|
83
|
+
site: mysite
|
|
84
|
+
name: mycommand
|
|
85
|
+
description: "一句话描述"
|
|
86
|
+
domain: www.example.com
|
|
87
|
+
strategy: cookie # 或 public (加 browser: false)
|
|
88
|
+
|
|
89
|
+
args:
|
|
90
|
+
limit:
|
|
91
|
+
type: int
|
|
92
|
+
default: 20
|
|
93
|
+
|
|
94
|
+
pipeline:
|
|
95
|
+
- navigate: https://www.example.com/target-page
|
|
96
|
+
|
|
97
|
+
- evaluate: |
|
|
98
|
+
(async () => {
|
|
99
|
+
const res = await fetch('/api/target', { credentials: 'include' });
|
|
100
|
+
const d = await res.json();
|
|
101
|
+
return (d.data?.items || []).map(item => ({
|
|
102
|
+
title: item.title,
|
|
103
|
+
value: item.value,
|
|
104
|
+
}));
|
|
105
|
+
})()
|
|
106
|
+
|
|
107
|
+
- map:
|
|
108
|
+
rank: ${{ index + 1 }}
|
|
109
|
+
title: ${{ item.title }}
|
|
110
|
+
value: ${{ item.value }}
|
|
111
|
+
|
|
112
|
+
- limit: ${{ args.limit }}
|
|
113
|
+
|
|
114
|
+
columns: [rank, title, value]
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### TS — Intercept(抓包模式)
|
|
118
|
+
|
|
119
|
+
```typescript
|
|
120
|
+
// src/clis/<site>/<name>.ts
|
|
121
|
+
import { cli, Strategy } from '../../registry.js';
|
|
122
|
+
|
|
123
|
+
cli({
|
|
124
|
+
site: 'mysite',
|
|
125
|
+
name: 'mycommand',
|
|
126
|
+
description: '一句话描述',
|
|
127
|
+
domain: 'www.example.com',
|
|
128
|
+
strategy: Strategy.INTERCEPT,
|
|
129
|
+
browser: true,
|
|
130
|
+
args: [
|
|
131
|
+
{ name: 'limit', type: 'int', default: 20 },
|
|
132
|
+
],
|
|
133
|
+
columns: ['rank', 'title', 'value'],
|
|
134
|
+
func: async (page, kwargs) => {
|
|
135
|
+
// 1. 导航
|
|
136
|
+
await page.goto('https://www.example.com/target-page');
|
|
137
|
+
await page.wait(3);
|
|
138
|
+
|
|
139
|
+
// 2. 注入拦截器(URL 子串匹配)
|
|
140
|
+
await page.installInterceptor('target-api-keyword');
|
|
141
|
+
|
|
142
|
+
// 3. 触发 API(滚动/点击)
|
|
143
|
+
await page.autoScroll({ times: 2, delayMs: 2000 });
|
|
144
|
+
|
|
145
|
+
// 4. 读取拦截的响应
|
|
146
|
+
const requests = await page.getInterceptedRequests();
|
|
147
|
+
if (!requests?.length) return [];
|
|
148
|
+
|
|
149
|
+
let results: any[] = [];
|
|
150
|
+
for (const req of requests) {
|
|
151
|
+
const items = req.data?.data?.items || [];
|
|
152
|
+
results.push(...items);
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
return results.slice(0, kwargs.limit).map((item, i) => ({
|
|
156
|
+
rank: i + 1,
|
|
157
|
+
title: item.title || '',
|
|
158
|
+
value: item.value || '',
|
|
159
|
+
}));
|
|
160
|
+
},
|
|
161
|
+
});
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### TS — Header(如 Twitter GraphQL)
|
|
165
|
+
|
|
166
|
+
```typescript
|
|
167
|
+
import { cli, Strategy } from '../../registry.js';
|
|
168
|
+
|
|
169
|
+
cli({
|
|
170
|
+
site: 'twitter',
|
|
171
|
+
name: 'mycommand',
|
|
172
|
+
description: '一句话描述',
|
|
173
|
+
domain: 'x.com',
|
|
174
|
+
strategy: Strategy.HEADER,
|
|
175
|
+
browser: true,
|
|
176
|
+
args: [
|
|
177
|
+
{ name: 'limit', type: 'int', default: 20 },
|
|
178
|
+
],
|
|
179
|
+
columns: ['rank', 'name', 'value'],
|
|
180
|
+
func: async (page, kwargs) => {
|
|
181
|
+
await page.goto('https://x.com');
|
|
182
|
+
const data = await page.evaluate(`(async () => {
|
|
183
|
+
const ct0 = document.cookie.match(/ct0=([^;]+)/)?.[1];
|
|
184
|
+
if (!ct0) return { error: 'Not logged in' };
|
|
185
|
+
const bearer = 'AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D...';
|
|
186
|
+
const res = await fetch('/i/api/graphql/QUERY_ID/Endpoint', {
|
|
187
|
+
headers: {
|
|
188
|
+
'Authorization': 'Bearer ' + decodeURIComponent(bearer),
|
|
189
|
+
'X-Csrf-Token': ct0,
|
|
190
|
+
'X-Twitter-Auth-Type': 'OAuth2Session',
|
|
191
|
+
},
|
|
192
|
+
credentials: 'include',
|
|
193
|
+
});
|
|
194
|
+
return res.json();
|
|
195
|
+
})()`);
|
|
196
|
+
// 解析 data...
|
|
197
|
+
return [];
|
|
198
|
+
},
|
|
199
|
+
});
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## 测试(必做)
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
npm run build # 语法检查
|
|
208
|
+
opencli list | grep mysite # 确认注册
|
|
209
|
+
opencli mysite mycommand --limit 3 -v # 实际运行
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
## 就这样,没了
|
|
215
|
+
|
|
216
|
+
写完文件 → build → run → 提交。有问题再看 [CLI-EXPLORER.md](./CLI-EXPLORER.md)。
|
package/LICENSE
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
BSD 3-Clause License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025, jackwener
|
|
4
|
+
|
|
5
|
+
Redistribution and use in source and binary forms, with or without
|
|
6
|
+
modification, are permitted provided that the following conditions are met:
|
|
7
|
+
|
|
8
|
+
1. Redistributions of source code must retain the above copyright notice, this
|
|
9
|
+
list of conditions and the following disclaimer.
|
|
10
|
+
|
|
11
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
12
|
+
this list of conditions and the following disclaimer in the documentation
|
|
13
|
+
and/or other materials provided with the distribution.
|
|
14
|
+
|
|
15
|
+
3. Neither the name of the copyright holder nor the names of its
|
|
16
|
+
contributors may be used to endorse or promote products derived from
|
|
17
|
+
this software without specific prior written permission.
|
|
18
|
+
|
|
19
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
20
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
21
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
22
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
|
23
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
24
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
25
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
26
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
27
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
28
|
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
package/README.md
CHANGED
|
@@ -5,20 +5,84 @@
|
|
|
5
5
|
|
|
6
6
|
[中文文档](./README.zh-CN.md)
|
|
7
7
|
|
|
8
|
-
[](https://www.npmjs.com/package/@jackwener/opencli)
|
|
8
|
+
[](https://www.npmjs.com/package/@jackwener/opencli)
|
|
9
|
+
[](https://nodejs.org)
|
|
10
|
+
[](./LICENSE)
|
|
9
11
|
|
|
10
|
-
A CLI tool that turns **any website** into a command-line interface. **
|
|
12
|
+
A CLI tool that turns **any website** into a command-line interface. **57 commands** across **17 sites** — bilibili, zhihu, xiaohongshu, twitter, reddit, xueqiu, github, v2ex, hackernews, bbc, weibo, boss, yahoo-finance, reuters, smzdm, ctrip, youtube — powered by browser session reuse and AI-native discovery.
|
|
11
13
|
|
|
12
|
-
|
|
14
|
+
---
|
|
13
15
|
|
|
14
|
-
|
|
15
|
-
- 🤖 **AI Agent ready** — `explore` discovers APIs, `synthesize` generates adapters, `cascade` finds auth strategies
|
|
16
|
-
- 🚀 **Dynamic Loader** — Simply drop `.ts` or `.yaml` adapters into the `clis/` folder for auto-registration
|
|
17
|
-
- 📝 **Dual-Engine Architecture**:
|
|
18
|
-
- **YAML Declarative Engine**: Most adapters are minimal ~30 lines of YAML pipeline
|
|
19
|
-
- **Native Browser Injection Engine**: Advanced TypeScript utilities (`installInterceptor`, `autoScroll`) for XHR hijacking, GraphQL unwrapping, and store mutation
|
|
16
|
+
## Table of Contents
|
|
20
17
|
|
|
21
|
-
|
|
18
|
+
- [Highlights](#highlights)
|
|
19
|
+
- [Prerequisites](#prerequisites)
|
|
20
|
+
- [Quick Start](#quick-start)
|
|
21
|
+
- [Built-in Commands](#built-in-commands)
|
|
22
|
+
- [Output Formats](#output-formats)
|
|
23
|
+
- [For AI Agents (Developer Guide)](#for-ai-agents-developer-guide)
|
|
24
|
+
- [Troubleshooting](#troubleshooting)
|
|
25
|
+
- [Releasing New Versions](#releasing-new-versions)
|
|
26
|
+
- [License](#license)
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Highlights
|
|
31
|
+
|
|
32
|
+
- **Account-safe** — Reuses Chrome's logged-in state; your credentials never leave the browser.
|
|
33
|
+
- **AI Agent ready** — `explore` discovers APIs, `synthesize` generates adapters, `cascade` finds auth strategies.
|
|
34
|
+
- **Dynamic Loader** — Simply drop `.ts` or `.yaml` adapters into the `clis/` folder for auto-registration.
|
|
35
|
+
- **Dual-Engine Architecture** — Supports both YAML declarative data pipelines and robust browser runtime typescript injections.
|
|
36
|
+
|
|
37
|
+
## Prerequisites
|
|
38
|
+
|
|
39
|
+
- **Node.js**: >= 18.0.0
|
|
40
|
+
- **Chrome** running **and logged into the target site** (e.g. bilibili.com, zhihu.com, xiaohongshu.com).
|
|
41
|
+
|
|
42
|
+
> **⚠️ Important**: Browser commands reuse your Chrome login session. You must be logged into the target website in Chrome before running commands. If you get empty data or errors, check your login status first.
|
|
43
|
+
|
|
44
|
+
OpenCLI needs a way to communicate with your browser. We highly recommend configuring **both** of the following methods for maximum reliability.
|
|
45
|
+
|
|
46
|
+
### Connection Method A: Playwright MCP Bridge Extension (Primary)
|
|
47
|
+
|
|
48
|
+
1. Install **[Playwright MCP Bridge](https://chromewebstore.google.com/detail/playwright-mcp-bridge/mmlmfjhmonkocbjadbfplnigmagldckm)** extension in Chrome.
|
|
49
|
+
2. Obtain your token by clicking the extension icon in the browser toolbar or from the extension settings page.
|
|
50
|
+
|
|
51
|
+
**You must configure this token in BOTH your MCP configuration AND system environment variables.**
|
|
52
|
+
|
|
53
|
+
First, add it to your MCP client config (e.g. Claude/Cursor):
|
|
54
|
+
|
|
55
|
+
```json
|
|
56
|
+
{
|
|
57
|
+
"mcpServers": {
|
|
58
|
+
"playwright": {
|
|
59
|
+
"command": "npx",
|
|
60
|
+
"args": ["-y", "@playwright/mcp@latest", "--extension"],
|
|
61
|
+
"env": {
|
|
62
|
+
"PLAYWRIGHT_MCP_EXTENSION_TOKEN": "<your-token-here>"
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
And, so that `opencli` commands can use it directly in the terminal, export it in your shell environment (e.g. `~/.zshrc`):
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
export PLAYWRIGHT_MCP_EXTENSION_TOKEN="<your-token-here>"
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Connection Method B: Chrome 144+ Auto-Discovery (Fallback)
|
|
76
|
+
|
|
77
|
+
No extensions needed. Just enable Chrome's built-in remote debugging:
|
|
78
|
+
|
|
79
|
+
1. Open `chrome://inspect#remote-debugging` in Chrome
|
|
80
|
+
2. Check **"Allow remote debugging for this browser instance"**
|
|
81
|
+
3. Set `OPENCLI_USE_CDP=1` before running opencli
|
|
82
|
+
|
|
83
|
+
*You can also manually specify an endpoint via `OPENCLI_CDP_ENDPOINT` env var.*
|
|
84
|
+
|
|
85
|
+
## Quick Start
|
|
22
86
|
|
|
23
87
|
### Install via npm (recommended)
|
|
24
88
|
|
|
@@ -30,63 +94,39 @@ Then use directly:
|
|
|
30
94
|
|
|
31
95
|
```bash
|
|
32
96
|
opencli list # See all commands
|
|
97
|
+
opencli list -f yaml # List commands as YAML
|
|
33
98
|
opencli hackernews top --limit 5 # Public API, no browser
|
|
34
99
|
opencli bilibili hot --limit 5 # Browser command
|
|
35
100
|
opencli zhihu hot -f json # JSON output
|
|
101
|
+
opencli zhihu hot -f yaml # YAML output
|
|
36
102
|
```
|
|
37
103
|
|
|
38
|
-
### Install from source
|
|
104
|
+
### Install from source (for developers)
|
|
39
105
|
|
|
40
106
|
```bash
|
|
41
107
|
git clone git@github.com:jackwener/opencli.git
|
|
42
|
-
cd opencli
|
|
43
|
-
|
|
108
|
+
cd opencli
|
|
109
|
+
npm install
|
|
110
|
+
npm run build
|
|
111
|
+
npm link # Link binary globally
|
|
112
|
+
opencli list # Now you can use it anywhere!
|
|
44
113
|
```
|
|
45
114
|
|
|
46
115
|
### Update
|
|
47
116
|
|
|
48
117
|
```bash
|
|
49
|
-
# npm global
|
|
50
|
-
npm update -g @jackwener/opencli
|
|
51
|
-
|
|
52
|
-
# Or reinstall to latest
|
|
53
118
|
npm install -g @jackwener/opencli@latest
|
|
54
119
|
```
|
|
55
120
|
|
|
56
|
-
##
|
|
57
|
-
|
|
58
|
-
Browser commands need:
|
|
59
|
-
1. **Chrome** running **and logged into the target site** (e.g. bilibili.com, zhihu.com, xiaohongshu.com)
|
|
60
|
-
2. **[Playwright MCP Bridge](https://chromewebstore.google.com/detail/playwright-mcp-bridge/mmlmfjhmonkocbjadbfplnigmagldckm)** extension installed
|
|
61
|
-
3. Configure `PLAYWRIGHT_MCP_EXTENSION_TOKEN` (from the extension settings page) in your MCP config:
|
|
62
|
-
|
|
63
|
-
```json
|
|
64
|
-
{
|
|
65
|
-
"mcpServers": {
|
|
66
|
-
"playwright": {
|
|
67
|
-
"command": "npx",
|
|
68
|
-
"args": ["@playwright/mcp@latest", "--extension"],
|
|
69
|
-
"env": {
|
|
70
|
-
"PLAYWRIGHT_MCP_EXTENSION_TOKEN": "<your-token>"
|
|
71
|
-
}
|
|
72
|
-
}
|
|
73
|
-
}
|
|
74
|
-
}
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
Public API commands (`hackernews`, `github search`, `v2ex`) need no browser at all.
|
|
78
|
-
|
|
79
|
-
> **⚠️ Important**: Browser commands reuse your Chrome login session. You must be logged into the target website in Chrome before running commands. If you get empty data or errors, check your login status first.
|
|
80
|
-
|
|
81
|
-
## 📦 Built-in Commands
|
|
121
|
+
## Built-in Commands
|
|
82
122
|
|
|
83
123
|
| Site | Commands | Mode |
|
|
84
124
|
|------|----------|------|
|
|
85
|
-
| **bilibili** | `hot` `search` `me` `favorite`
|
|
125
|
+
| **bilibili** | `hot` `search` `me` `favorite` ... (11 commands) | 🔐 Browser |
|
|
86
126
|
| **zhihu** | `hot` `search` `question` | 🔐 Browser |
|
|
87
127
|
| **xiaohongshu** | `search` `notifications` `feed` `me` `user` | 🔐 Browser |
|
|
88
128
|
| **xueqiu** | `feed` `hot-stock` `hot` `search` `stock` `watchlist` | 🔐 Browser |
|
|
89
|
-
| **twitter** | `trending` `bookmarks` `profile` `search` `timeline` | 🔐 Browser |
|
|
129
|
+
| **twitter** | `trending` `bookmarks` `profile` `search` `timeline` `following` `followers` `notifications` `post` `reply` `delete` `like` | 🔐 Browser |
|
|
90
130
|
| **reddit** | `hot` `frontpage` `search` `subreddit` | 🔐 Browser |
|
|
91
131
|
| **weibo** | `hot` | 🔐 Browser |
|
|
92
132
|
| **boss** | `search` | 🔐 Browser |
|
|
@@ -100,17 +140,28 @@ Public API commands (`hackernews`, `github search`, `v2ex`) need no browser at a
|
|
|
100
140
|
| **hackernews** | `top` | 🌐 Public |
|
|
101
141
|
| **bbc** | `news` | 🌐 Public |
|
|
102
142
|
|
|
103
|
-
##
|
|
143
|
+
## Output Formats
|
|
144
|
+
|
|
145
|
+
All built-in commands support `--format` / `-f` with `table`, `json`, `yaml`, `md`, and `csv`.
|
|
146
|
+
The `list` command supports the same format options, and keeps `--json` for backward compatibility.
|
|
104
147
|
|
|
105
148
|
```bash
|
|
106
|
-
opencli
|
|
107
|
-
opencli bilibili hot -f
|
|
149
|
+
opencli list -f yaml # Command registry as YAML
|
|
150
|
+
opencli bilibili hot -f table # Default: rich terminal table
|
|
151
|
+
opencli bilibili hot -f json # JSON (pipe to jq or LLMs)
|
|
152
|
+
opencli bilibili hot -f yaml # YAML (human-readable structured output)
|
|
108
153
|
opencli bilibili hot -f md # Markdown
|
|
109
154
|
opencli bilibili hot -f csv # CSV
|
|
110
|
-
opencli bilibili hot -v # Verbose: show pipeline steps
|
|
155
|
+
opencli bilibili hot -v # Verbose: show pipeline debug steps
|
|
111
156
|
```
|
|
112
157
|
|
|
113
|
-
##
|
|
158
|
+
## For AI Agents (Developer Guide)
|
|
159
|
+
|
|
160
|
+
If you are an AI assistant tasked with creating a new command adapter for `opencli`, please follow the AI Agent workflow below:
|
|
161
|
+
|
|
162
|
+
> **Quick mode**: To generate a single command for a specific page URL, see [CLI-ONESHOT.md](./CLI-ONESHOT.md) — just a URL + one-line goal, 4 steps done.
|
|
163
|
+
|
|
164
|
+
> **Full mode**: Before writing any adapter code, read [CLI-EXPLORER.md](./CLI-EXPLORER.md). It contains the complete browser exploration workflow, the 5-tier authentication strategy decision tree, and debugging guide.
|
|
114
165
|
|
|
115
166
|
```bash
|
|
116
167
|
# 1. Deep Explore — discover APIs, infer capabilities, detect framework
|
|
@@ -126,30 +177,30 @@ opencli generate https://example.com --goal "hot"
|
|
|
126
177
|
opencli cascade https://api.example.com/data
|
|
127
178
|
```
|
|
128
179
|
|
|
129
|
-
Explore outputs to `.opencli/explore/<site
|
|
130
|
-
- `manifest.json` — site metadata, framework detection
|
|
131
|
-
- `endpoints.json` — scored API endpoints with response schemas
|
|
132
|
-
- `capabilities.json` — inferred capabilities with confidence scores
|
|
133
|
-
- `auth.json` — authentication strategy recommendations
|
|
180
|
+
Explore outputs to `.opencli/explore/<site>/` (manifest.json, endpoints.json, capabilities.json, auth.json).
|
|
134
181
|
|
|
135
|
-
##
|
|
182
|
+
## Troubleshooting
|
|
136
183
|
|
|
137
|
-
|
|
184
|
+
- **"Failed to connect to Playwright MCP Bridge"**
|
|
185
|
+
- Ensure the Playwright MCP extension is installed and **enabled** in your running Chrome.
|
|
186
|
+
- Restart the Chrome browser if you just installed the extension.
|
|
187
|
+
- **"CDP command failed" or "boss search blocked"**
|
|
188
|
+
- Some sites (like BOSS Zhipin) actively block Chrome DevTools Protocol connections. OpenCLI falls back to cookie extraction, but ensure you didn't force `--chrome-mode` unnecessarily.
|
|
189
|
+
- **Empty data returns or 'Unauthorized' error**
|
|
190
|
+
- Your login session in Chrome might have expired. Open a normal Chrome tab, navigate to the target site, and log in or refresh the page to prove you are human.
|
|
191
|
+
- **Node API errors**
|
|
192
|
+
- Make sure you are using Node.js >= 18. Some dependencies require modern Node APIs.
|
|
138
193
|
|
|
139
194
|
## Releasing New Versions
|
|
140
195
|
|
|
141
196
|
```bash
|
|
142
|
-
# Bump version
|
|
143
197
|
npm version patch # 0.1.0 → 0.1.1
|
|
144
198
|
npm version minor # 0.1.0 → 0.2.0
|
|
145
|
-
npm version major # 0.1.0 → 1.0.0
|
|
146
|
-
|
|
147
|
-
# Push tag to trigger GitHub Actions auto-release
|
|
148
199
|
git push --follow-tags
|
|
149
200
|
```
|
|
150
201
|
|
|
151
202
|
The CI will automatically build, create a GitHub release, and publish to npm.
|
|
152
203
|
|
|
153
|
-
##
|
|
204
|
+
## License
|
|
154
205
|
|
|
155
|
-
|
|
206
|
+
[BSD-3-Clause](./LICENSE)
|