@jackwener/opencli 1.5.8 → 1.6.0

package/skills/opencli-oneshot/SKILL.md

@@ -0,0 +1,222 @@
+---
+name: opencli-oneshot
+description: Use when quickly generating a single OpenCLI command from a specific URL and goal description. 4-step process — open page, capture API, write YAML adapter, test. For full site exploration, use opencli-explorer instead.
+tags: [opencli, adapter, quick-start, yaml, cli, one-shot, automation]
+---
+
+# CLI-ONESHOT — 单点快速 CLI 生成
+
+> 给一个 URL + 一句话描述，4 步生成一个 CLI 命令。
+> 完整探索式开发请看 [CLI-EXPLORER.md](./CLI-EXPLORER.md)。
+
+---
+
+## 输入
+
+| 项目 | 示例 |
+|------|------|
+| **URL** | `https://x.com/jakevin7/lists` |
+| **Goal** | 获取我的 Twitter Lists |
+
+---
+
+## 流程
+
+### Step 1: 打开页面 + 抓包
+
+```
+1. browser_navigate → 打开目标 URL
+2. 等待 3-5 秒（让页面加载完、API 请求触发）
+3. browser_network_requests → 筛选 JSON API
+```
+
+**关键**：只关注返回 `application/json` 的请求，忽略静态资源。
+如果没有自动触发 API，手动点击目标按钮/标签再抓一次。
+
+### Step 2: 锁定一个接口
+
+从抓包结果中找到**那个**目标 API。看这几个字段：
+
+| 字段 | 关注什么 |
+|------|----------|
+| URL | API 路径 pattern（如 `/i/api/graphql/xxx/ListsManagePinTimeline`） |
+| Method | GET / POST |
+| Headers | 有 Cookie? Bearer? CSRF? 自定义签名? |
+| Response | 数据在哪个路径（如 `data.list.lists`） |
+
+### Step 3: 验证接口能复现
+
+在 `browser_evaluate` 中用 `fetch` 复现请求：
+
+```javascript
+// Tier 2 (Cookie): 大多数情况
+fetch('/api/endpoint', { credentials: 'include' }).then(r => r.json())
+
+// Tier 3 (Header): 如 Twitter 需要额外 header
+const ct0 = document.cookie.match(/ct0=([^;]+)/)?.[1];
+fetch('/api/endpoint', {
+  headers: { 'Authorization': 'Bearer ...', 'X-Csrf-Token': ct0 },
+  credentials: 'include'
+}).then(r => r.json())
+```
+
+如果 fetch 能拿到数据 → 用 YAML 或简单 TS adapter。
+如果 fetch 拿不到（签名/风控）→ 用 intercept 策略。
+
+### Step 4: 套模板，生成 adapter
+
+根据 Step 3 判定的策略，选一个模板生成文件。
+
+---
+
+## 认证速查
+
+```
+fetch(url) 直接能拿到？              → Tier 1: public   (YAML, browser: false)
+fetch(url, {credentials:'include'})？ → Tier 2: cookie   (YAML)
+加 Bearer/CSRF header 后拿到？        → Tier 3: header   (TS)
+都不行，但页面自己能请求成功？          → Tier 4: intercept (TS, installInterceptor)
+```
+
+---
+
+## 模板
+
+### YAML — Cookie/Public（最简）
+
+```yaml
+# src/clis/<site>/<name>.yaml
+site: mysite
+name: mycommand
+description: "一句话描述"
+domain: www.example.com
+strategy: cookie          # 或 public (加 browser: false)
+
+args:
+  limit:
+    type: int
+    default: 20
+
+pipeline:
+  - navigate: https://www.example.com/target-page
+
+  - evaluate: |
+      (async () => {
+        const res = await fetch('/api/target', { credentials: 'include' });
+        const d = await res.json();
+        return (d.data?.items || []).map(item => ({
+          title: item.title,
+          value: item.value,
+        }));
+      })()
+
+  - map:
+      rank: ${{ index + 1 }}
+      title: ${{ item.title }}
+      value: ${{ item.value }}
+
+  - limit: ${{ args.limit }}
+
+columns: [rank, title, value]
+```
+
+### TS — Intercept（抓包模式）
+
+```typescript
+// src/clis/<site>/<name>.ts
+import { cli, Strategy } from '../../registry.js';
+
+cli({
+  site: 'mysite',
+  name: 'mycommand',
+  description: '一句话描述',
+  domain: 'www.example.com',
+  strategy: Strategy.INTERCEPT,
+  browser: true,
+  args: [
+    { name: 'limit', type: 'int', default: 20 },
+  ],
+  columns: ['rank', 'title', 'value'],
+  func: async (page, kwargs) => {
+    // 1. 导航
+    await page.goto('https://www.example.com/target-page');
+    await page.wait(3);
+
+    // 2. 注入拦截器（URL 子串匹配）
+    await page.installInterceptor('target-api-keyword');
+
+    // 3. 触发 API（滚动/点击）
+    await page.autoScroll({ times: 2, delayMs: 2000 });
+
+    // 4. 读取拦截的响应
+    const requests = await page.getInterceptedRequests();
+    if (!requests?.length) return [];
+
+    let results: any[] = [];
+    for (const req of requests) {
+      const items = req.data?.data?.items || [];
+      results.push(...items);
+    }
+
+    return results.slice(0, kwargs.limit).map((item, i) => ({
+      rank: i + 1,
+      title: item.title || '',
+      value: item.value || '',
+    }));
+  },
+});
+```
+
+### TS — Header（如 Twitter GraphQL）
+
+```typescript
+import { cli, Strategy } from '../../registry.js';
+
+cli({
+  site: 'twitter',
+  name: 'mycommand',
+  description: '一句话描述',
+  domain: 'x.com',
+  strategy: Strategy.HEADER,
+  browser: true,
+  args: [
+    { name: 'limit', type: 'int', default: 20 },
+  ],
+  columns: ['rank', 'name', 'value'],
+  func: async (page, kwargs) => {
+    await page.goto('https://x.com');
+    const data = await page.evaluate(`(async () => {
+      const ct0 = document.cookie.match(/ct0=([^;]+)/)?.[1];
+      if (!ct0) return { error: 'Not logged in' };
+      const bearer = 'AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D...';
+      const res = await fetch('/i/api/graphql/QUERY_ID/Endpoint', {
+        headers: {
+          'Authorization': 'Bearer ' + decodeURIComponent(bearer),
+          'X-Csrf-Token': ct0,
+          'X-Twitter-Auth-Type': 'OAuth2Session',
+        },
+        credentials: 'include',
+      });
+      return res.json();
+    })()`);
+    // 解析 data...
+    return [];
+  },
+});
+```
+
+---
+
+## 测试（必做）
+
+```bash
+npm run build                              # 语法检查
+opencli list | grep mysite                 # 确认注册
+opencli mysite mycommand --limit 3 -v      # 实际运行
+```
+
+---
+
+## 就这样，没了
+
+写完文件 → build → run → 提交。有问题再看 [CLI-EXPLORER.md](./CLI-EXPLORER.md)。

package/skills/opencli-operate/SKILL.md

@@ -0,0 +1,213 @@
+---
+name: opencli-operate
+description: Make websites accessible for AI agents. Navigate, click, type, extract, wait — using Chrome with existing login sessions. No LLM API key needed.
+allowed-tools: Bash(opencli:*), Read, Edit, Write
+---
+
+# OpenCLI — Make Websites Accessible for AI Agents
+
+Control Chrome step-by-step via CLI. Reuses existing login sessions — no passwords needed.
+
+## Prerequisites
+
+```bash
+opencli doctor    # Verify extension + daemon connectivity
+```
+
+Requires: Chrome running + OpenCLI Browser Bridge extension installed.
+
+## Quickstart for AI Agents (1 step)
+
+Point your AI agent to this file. It contains everything needed to operate browsers.
+
+## Quickstart for Humans (3 steps)
+
+```bash
+npm install -g @jackwener/opencli          # 1. Install
+# Install extension from chrome://extensions  # 2. Load extension
+opencli operate open https://example.com    # 3. Go!
+```
+
+## Core Workflow
+
+1. **Navigate**: `opencli operate open <url>`
+2. **Inspect**: `opencli operate state` → see elements with `[N]` indices
+3. **Interact**: use indices — `click`, `type`, `select`, `keys`
+4. **Wait**: `opencli operate wait selector ".loaded"` or `wait text "Success"`
+5. **Verify**: `opencli operate get title` or `opencli operate screenshot`
+6. **Repeat**: browser stays open between commands
+7. **Save**: write a TS adapter to `~/.opencli/clis/<site>/<command>.ts`
+
+## Commands
+
+### Navigation
+
+```bash
+opencli operate open <url>              # Open URL
+opencli operate back                    # Go back
+opencli operate scroll down             # Scroll (up/down, --amount N)
+opencli operate scroll up --amount 1000
+```
+
+### Inspect
+
+```bash
+opencli operate state                   # Elements with [N] indices
+opencli operate screenshot [path.png]   # Screenshot
+```
+
+### Get (structured data)
+
+```bash
+opencli operate get title               # Page title
+opencli operate get url                 # Current URL
+opencli operate get text <index>        # Element text content
+opencli operate get value <index>       # Input/textarea value
+opencli operate get html                # Full page HTML
+opencli operate get html --selector "h1" # Scoped HTML
+opencli operate get attributes <index>  # Element attributes
+```
+
+### Interact
+
+```bash
+opencli operate click <index>           # Click element [N]
+opencli operate type <index> "text"     # Type into element [N]
+opencli operate select <index> "option" # Select dropdown
+opencli operate keys "Enter"            # Press key (Enter, Escape, Tab, Control+a)
+```
+
+### Wait
+
+```bash
+opencli operate wait selector ".loaded"           # Wait for element
+opencli operate wait selector ".spinner" --timeout 5000  # With timeout
+opencli operate wait text "Success"               # Wait for text
+opencli operate wait time 3                       # Wait N seconds
+```
+
+### Extract
+
+```bash
+opencli operate eval "document.title"
+opencli operate eval "JSON.stringify([...document.querySelectorAll('h2')].map(e => e.textContent))"
+```
+
+### Network (API Discovery)
+
+```bash
+opencli operate network                  # Show captured API requests (auto-captured since open)
+opencli operate network --detail 3       # Show full response body of request #3
+opencli operate network --all            # Include static resources
+```
+
+### Sedimentation (Save as CLI)
+
+```bash
+opencli operate init hn/top              # Generate adapter scaffold
+opencli operate verify hn/top            # Test the adapter
+```
+
+### Session
+
+```bash
+opencli operate close                   # Close automation window
+```
+
+## Example: Extract HN Stories
+
+```bash
+opencli operate open https://news.ycombinator.com
+opencli operate state                   # See [1] a "Story 1", [2] a "Story 2"...
+opencli operate eval "JSON.stringify([...document.querySelectorAll('.titleline a')].slice(0,5).map(a => ({title: a.textContent, url: a.href})))"
+opencli operate close
+```
+
+## Example: Fill a Form
+
+```bash
+opencli operate open https://httpbin.org/forms/post
+opencli operate state                   # See [3] input "Customer Name", [4] input "Telephone"
+opencli operate type 3 "OpenCLI"
+opencli operate type 4 "555-0100"
+opencli operate get value 3             # Verify: "OpenCLI"
+opencli operate close
+```
+
+## Saving as Reusable CLI — Complete Workflow
+
+### Step-by-step sedimentation flow:
+
+```bash
+# 1. Explore the website
+opencli operate open https://news.ycombinator.com
+opencli operate state                          # Understand DOM structure
+
+# 2. Discover APIs (crucial for high-quality adapters)
+opencli operate eval "fetch('/api/...').then(r=>r.json())"   # Trigger API calls
+opencli operate network                        # See captured API requests
+opencli operate network --detail 0             # Inspect response body
+
+# 3. Generate scaffold
+opencli operate init hn/top                    # Creates ~/.opencli/clis/hn/top.ts
+
+# 4. Edit the adapter (fill in func logic)
+# - If API found: use fetch() directly (Strategy.PUBLIC or COOKIE)
+# - If no API: use page.evaluate() for DOM extraction (Strategy.UI)
+
+# 5. Verify
+opencli operate verify hn/top                  # Runs the adapter and shows output
+
+# 6. If verify fails, edit and retry
+# 7. Close when done
+opencli operate close
+```
+
+### Example adapter:
+
+```typescript
+// ~/.opencli/clis/hn/top.ts
+import { cli, Strategy } from '@jackwener/opencli/registry';
+
+cli({
+  site: 'hn',
+  name: 'top',
+  description: 'Top Hacker News stories',
+  domain: 'news.ycombinator.com',
+  strategy: Strategy.PUBLIC,
+  browser: false,
+  args: [{ name: 'limit', type: 'int', default: 5 }],
+  columns: ['rank', 'title', 'score', 'url'],
+  func: async (_page, kwargs) => {
+    const limit = Math.min(Math.max(1, kwargs.limit ?? 5), 50);
+    const resp = await fetch('https://hacker-news.firebaseio.com/v0/topstories.json');
+    const ids = await resp.json();
+    return Promise.all(
+      ids.slice(0, limit).map(async (id: number, i: number) => {
+        const item = await (await fetch(`https://hacker-news.firebaseio.com/v0/item/${id}.json`)).json();
+        return { rank: i + 1, title: item.title, score: item.score, url: item.url ?? '' };
+      })
+    );
+  },
+});
+```
+
+Save to `~/.opencli/clis/<site>/<command>.ts` → immediately available as `opencli <site> <command>`.
+
+### Strategy Guide
+
+| Strategy | When | browser: |
+|----------|------|----------|
+| `Strategy.PUBLIC` | Public API, no auth | `false` |
+| `Strategy.COOKIE` | Needs login cookies | `true` |
+| `Strategy.UI` | Direct DOM interaction | `true` |
+
+**Always prefer API over UI** — if you discovered an API during browsing, use `fetch()` directly.
+
+## Troubleshooting
+
+| Error | Fix |
+|-------|-----|
+| "Browser not connected" | Run `opencli doctor` |
+| "attach failed: chrome-extension://" | Disable 1Password temporarily |
+| Element not found | `opencli operate scroll down` then `opencli operate state` |

package/skills/opencli-usage/SKILL.md

@@ -0,0 +1,152 @@
+---
+name: opencli-usage
+description: "Use when running OpenCLI commands to interact with websites (Bilibili, Twitter, Reddit, Xiaohongshu, etc.), desktop apps (Cursor, Notion), or public APIs (HackerNews, arXiv). Covers installation, command reference, and output formats for 60+ adapters."
+version: 1.6.0
+author: jackwener
+tags: [opencli, cli, browser, web, chrome-extension, cdp, bilibili, twitter, reddit, xiaohongshu, github, youtube, AI, agent, automation]
+---
+
+# OpenCLI Usage Guide
+
+> Make any website or Electron App your CLI. Reuse Chrome login, zero risk, AI-powered discovery.
+
+## Install & Run
+
+```bash
+# npm global install (recommended)
+npm install -g @jackwener/opencli
+opencli <command>
+
+# Or from source
+cd ~/code/opencli && npm install
+npx tsx src/main.ts <command>
+
+# Update to latest
+npm update -g @jackwener/opencli
+```
+
+## Prerequisites
+
+Browser commands require:
+1. Chrome browser running **(logged into target sites)**
+2. **opencli Browser Bridge** Chrome extension installed (load `extension/` as unpacked in `chrome://extensions`)
+3. No further setup needed — the daemon auto-starts on first browser command
+
+> **Note**: You must be logged into the target website in Chrome before running commands. Tabs opened during command execution are auto-closed afterwards.
+
+Public API commands (`hackernews`, `v2ex`) need no browser.
+
+## Quick Lookup by Capability
+
+| Capability | Platforms (partial list) | File |
+|-----------|--------------------------|------|
+| **search** | Bilibili, Twitter, Reddit, Xiaohongshu, Zhihu, YouTube, Google, arXiv, LinkedIn, Pixiv, etc. | browser.md / public-api.md |
+| **hot/trending** | Bilibili, Twitter, Weibo, HackerNews, Reddit, V2EX, Xueqiu, Lobsters, Douban | browser.md / public-api.md |
+| **feed/timeline** | Twitter, Reddit, Xiaohongshu, Xueqiu, Jike, Facebook, Instagram, Medium | browser.md |
+| **user/profile** | Twitter, Reddit, Instagram, TikTok, Facebook, Bilibili, Pixiv | browser.md |
+| **post/create** | Twitter, Jike | browser.md |
+| **AI chat** | Grok, Doubao, Kimi, DeepSeek, Qwen, ChatGPT, Cursor, Codex | browser.md / desktop.md |
+| **finance/stock** | Xueqiu, Yahoo Finance, Barchart, Sina Finance, Bloomberg | browser.md / public-api.md |
+| **web scraping** | `opencli web read --url <url>` — any URL to Markdown | browser.md |
+
+## Command Quick Reference
+
+Usage: `opencli <site> <command> [args] [--limit N] [-f json|yaml|md|csv|table]`
+
+### Browser-based (login required)
+
+| Site | Commands |
+|------|----------|
+| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `user-videos` `subtitle` `dynamic` `ranking` `following` |
+| **zhihu** | `hot` `search` `question` |
+| **xiaohongshu** | `search` `notifications` `feed` `user` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` |
+| **xueqiu** | `hot-stock` `stock` `watchlist` `feed` `hot` `search` `earnings-date` `fund-holdings` `fund-snapshot` |
+| **twitter** | `trending` `bookmarks` `search` `profile` `timeline` `thread` `article` `follow` `unfollow` `bookmark` `unbookmark` `post` `like` `reply` `delete` `block` `unblock` `followers` `following` `notifications` `hide-reply` `download` `accept` `reply-dm` |
+| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` |
+| **youtube** | `search` `video` `transcript` |
+| **facebook** | `feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group` |
+| **instagram** | `explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved` |
+| **tiktok** | `explore` `search` `profile` `user` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `live` `notifications` `friends` |
+| **linkedin** | `search` `timeline` |
+| **medium** | `feed` `search` `user` |
+| **substack** | `feed` `search` `publication` |
+| **sinablog** | `hot` `search` `article` `user` |
+| **weibo** | `hot` |
+| **boss** | `search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats` |
+| **douban** | `search` `top250` `subject` `photos` `download` `marks` `reviews` |
+| **pixiv** | `ranking` `search` `user` `illusts` `detail` `download` |
+| **jike** | `feed` `search` `create` `like` `comment` `repost` `notifications` |
+| **yahoo-finance** | `quote` |
+| **barchart** | `quote` `options` `greeks` `flow` |
+| **sinafinance** | `news` |
+| **reuters** | `search` |
+| **coupang** | `search` `add-to-cart` |
+| **jd** | `item` |
+| **smzdm** | `search` |
+| **ctrip** | `search` |
+| **weread** | `shelf` `search` `book` `highlights` `notes` `ranking` |
+| **chaoxing** | `assignments` `exams` |
+| **jimeng** | `generate` `history` |
+| **yollomi** | `models` `generate` `video` `upload` `remove-bg` `edit` `background` `face-swap` `object-remover` `restore` `try-on` `upscale` |
+| **web** | `read` — any URL to Markdown |
+| **weixin** | `download` — 公众号 article to Markdown |
+| **v2ex** (browser) | `daily` `me` `notifications` |
+| **linux-do** (browser) | `categories` `category` |
+| **bloomberg** (browser) | `news` — full article reader |
+| **grok** | `ask` |
+| **doubao** | `status` `new` `send` `read` `ask` |
+| **kimi** | `status` `new` `ask` |
+| **deepseek** | `status` `new` `ask` |
+| **qwen** | `status` `new` `ask` |
+
+### Desktop (CDP/Electron)
+
+| Site | Commands |
+|------|----------|
+| **gh** | `repo` `pr` `issue` — passthrough to gh CLI |
+| **cursor** | `status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export` |
+| **codex** | `status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `history` `export` |
+| **chatgpt** | `status` `new` `send` `read` `ask` |
+| **chatwise** | `status` `new` `send` `read` `ask` `model` `history` `export` `screenshot` |
+| **notion** | `status` `search` `read` `new` `write` `sidebar` `favorites` `export` |
+| **discord-app** | `status` `send` `read` `channels` `servers` `search` `members` |
+| **doubao-app** | `status` `new` `send` `read` `ask` `screenshot` `dump` |
+| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` |
+
+### Public API (no browser)
+
+| Site | Commands |
+|------|----------|
+| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
+| **v2ex** (public) | `hot` `latest` `topic` `node` `nodes` `member` `user` `replies` |
+| **bbc** | `news` |
+| **lobsters** | `hot` `newest` `active` `tag` |
+| **google** | `news` `search` `suggest` `trends` |
+| **devto** | `top` `tag` `user` |
+| **steam** | `top-sellers` |
+| **apple-podcasts** | `top` `search` `episodes` |
+| **arxiv** | `search` `paper` |
+| **bloomberg** (RSS) | `main` `markets` `tech` `politics` `economics` `opinions` `industries` `businessweek` `feeds` |
+| **dictionary** | `search` `synonyms` `examples` |
+| **hf** | `top` |
+| **stackoverflow** | `hot` `search` `bounties` |
+| **xiaoyuzhou** | `podcast` `podcast-episodes` `episode` |
+| **wikipedia** | `search` `summary` |
+| **producthunt** | `today` `week` `month` `search` |
+
+### Management
+
+```bash
+opencli list [-f json|yaml]     # List all commands
+opencli validate [site]         # Validate adapter definitions
+opencli doctor                  # Diagnose browser bridge
+opencli explore <url>           # AI-powered API discovery
+opencli record <url>            # Record API calls manually
+```
+
+All commands support: `--format` / `-f` with `table` `json` `yaml` `md` `csv`
+
+## Related Skills
+
+- **opencli-explorer** — Full guide for creating new adapters (API discovery, auth strategy, YAML/TS writing)
+- **opencli-oneshot** — Quick 4-step template for adding a single command from a URL