@jackwener/opencli 1.6.1 → 1.6.3

package/docs/advanced/electron.md

@@ -1,125 +0,0 @@
----
-description: How to CLI-ify and automate any Electron Desktop Application via CDP
----
-
-# CLI-ifying Electron Applications (Skill Guide)
-
-Based on the successful automation of **Cursor**, **Codex**, **Antigravity**, **ChatWise**, **Notion**, and **Discord** desktop apps, this guide serves as the standard operating procedure (SOP) for adapting ANY Electron-based application into an OpenCLI adapter.
-
-## Core Concept
-
-Electron apps are essentially local Chromium browser instances. By exposing a debugging port (CDP — Chrome DevTools Protocol) at launch time, we can use the Browser Bridge to pierce through the UI layer, accessing and controlling all underlying state including React/Vue components and Shadow DOM.
-
-> **Note:** Not all desktop apps are Electron. WeChat (native Cocoa) and Feishu/Lark (custom Lark Framework) embed Chromium but do NOT expose CDP. For those apps, use the AppleScript + clipboard approach instead (see [Non-Electron Pattern](#non-electron-pattern-applescript)).
-
-### Launching the Target App
-```bash
-/Applications/AppName.app/Contents/MacOS/AppName --remote-debugging-port=9222
-```
-
-### Verifying Electron
-```bash
-# Check for Electron Framework in the app bundle
-ls /Applications/AppName.app/Contents/Frameworks/Electron\ Framework.framework
-# If this directory exists → Electron → CDP works
-# If not → check for libEGL.dylib (embedded Chromium/CEF, CDP may not work)
-```
-
-## The 5-Command Pattern (CDP / Electron)
-
-Every new Electron adapter should implement these 5 commands in `src/clis/<app_name>/`:
-
-### 1. `status.ts` — Connection Test
-```typescript
-export const statusCommand = cli({
-  site: 'myapp',
-  name: 'status',
-  domain: 'localhost',
-  strategy: Strategy.UI,
-  browser: true,       // Requires CDP connection
-  args: [],
-  columns: ['Status', 'Url', 'Title'],
-  func: async (page: IPage) => {
-    const url = await page.evaluate('window.location.href');
-    const title = await page.evaluate('document.title');
-    return [{ Status: 'Connected', Url: url, Title: title }];
-  },
-});
-```
-
-### 2. `dump.ts` — Reverse Engineering Core
-Modern app DOMs are huge and obfuscated. **Never guess selectors.** Dump first, then extract precise class names with AI or `grep`:
-```typescript
-const dom = await page.evaluate('document.body.innerHTML');
-fs.writeFileSync('/tmp/app-dom.html', dom);
-const snap = await page.snapshot({ interactive: false });
-fs.writeFileSync('/tmp/app-snapshot.json', JSON.stringify(snap, null, 2));
-```
-
-### 3. `send.ts` — Advanced Text Injection
-Electron apps often use complex rich-text editors (Monaco, Lexical, ProseMirror). Setting `.value` directly is ignored by React state.
-
-**Best practice:** Use `document.execCommand('insertText')` to perfectly simulate real user input, fully piercing React state:
-```javascript
-const composer = document.querySelector('[contenteditable="true"]');
-composer.focus();
-document.execCommand('insertText', false, 'Hello');
-```
-Then submit with `await page.pressKey('Enter')`.
-
-### 4. `read.ts` — Context Extraction
-Don't extract the entire page text. Use `dump.ts` output to find the real "conversation container":
-- Look for semantic selectors: `[role="log"]`, `[data-testid="conversation"]`, `[data-content-search-turn-key]`
-- Format output as Markdown — readable by both humans and LLMs
-
-### 5. `new.ts` — Keyboard Shortcuts
-Many GUI actions respond to native shortcuts rather than button clicks:
-```typescript
-const isMac = process.platform === 'darwin';
-await page.pressKey(isMac ? 'Meta+N' : 'Control+N');
-await page.wait(1); // Wait for re-render
-```
-
-## Environment Variable
-```bash
-export OPENCLI_CDP_ENDPOINT="http://127.0.0.1:9222"
-```
-
-## Non-Electron Pattern (AppleScript)
-
-For native macOS apps (WeChat, Feishu) that don't expose CDP:
-```typescript
-export const statusCommand = cli({
-  site: 'myapp',
-  strategy: Strategy.PUBLIC,
-  browser: false,       // No browser needed
-  func: async (page: IPage | null) => {
-    const output = execSync("osascript -e 'application \"MyApp\" is running'", { encoding: 'utf-8' }).trim();
-    return [{ Status: output === 'true' ? 'Running' : 'Stopped' }];
-  },
-});
-```
-
-Core techniques:
-- **status**: `osascript -e 'application "AppName" is running'`
-- **send**: `pbcopy` → activate window → `Cmd+V` → `Enter`
-- **read**: `Cmd+A` → `Cmd+C` → `pbpaste`
-- **search**: Activate → `Cmd+F`/`Cmd+K` → `keystroke "query"`
-
-## Pitfalls & Gotchas
-
-1. **Port conflicts (EADDRINUSE)**: Only one app per port. Use unique ports: Codex=9222, ChatGPT=9224, Cursor=9226, ChatWise=9228, Notion=9230, Discord=9232
-2. **IPage abstraction**: OpenCLI wraps the browser page as `IPage` (`src/types.ts`). Use `page.pressKey()` and `page.evaluate()`, NOT direct DOM APIs
-3. **Timing**: Always add `await page.wait(0.5)` to `1.0` after DOM mutations. Returning too early disconnects prematurely
-4. **AppleScript requires Accessibility**: Terminal app must be granted permission in System Settings → Privacy & Security → Accessibility
-
-## Port Assignment Table
-
-| App | Port | Mode |
-|-----|------|------|
-| Codex | 9222 | CDP |
-| ChatGPT | 9224 | CDP / AppleScript |
-| Cursor | 9226 | CDP |
-| ChatWise | 9228 | CDP |
-| Notion | 9230 | CDP |
-| Discord App | 9232 | CDP |

package/docs/advanced/rate-limiter-plugin.md

@@ -1,99 +0,0 @@
-# Rate Limiter Plugin
-
-An optional plugin that adds a random sleep between browser-based commands to reduce the risk of platform rate-limiting or bot detection.
-
-## Install
-
-```bash
-opencli plugin install github:jackwener/opencli-plugin-rate-limiter
-```
-
-Or copy the example below into `~/.opencli/plugins/rate-limiter/` to use it locally without installing from GitHub.
-
-## What it does
-
-After every command targeting a browser platform (xiaohongshu, weibo, bilibili, douyin, tiktok, …), the plugin sleeps for a random duration — 5–30 seconds by default — before returning control to the caller.
-
-## Configuration
-
-| Variable | Default | Description |
-|---|---|---|
-| `OPENCLI_RATE_MIN` | `5` | Minimum sleep in seconds |
-| `OPENCLI_RATE_MAX` | `30` | Maximum sleep in seconds |
-| `OPENCLI_NO_RATE` | — | Set to `1` to disable entirely (local dev) |
-
-```bash
-# Shorter delays for light scraping
-OPENCLI_RATE_MIN=3 OPENCLI_RATE_MAX=10 opencli xiaohongshu search "AI眼镜"
-
-# Skip delays when iterating locally
-OPENCLI_NO_RATE=1 opencli bilibili comments BV1WtAGzYEBm
-```
-
-## Local installation (without GitHub)
-
-1. Create the plugin directory:
-
-   ```bash
-   mkdir -p ~/.opencli/plugins/rate-limiter
-   ```
-
-2. Create `~/.opencli/plugins/rate-limiter/package.json`:
-
-   ```json
-   { "type": "module" }
-   ```
-
-3. Create `~/.opencli/plugins/rate-limiter/index.js`:
-
-   ```js
-   import { onAfterExecute } from '@jackwener/opencli/hooks'
-
-   const BROWSER_DOMAINS = [
-     'xiaohongshu', 'weibo', 'bilibili', 'douyin', 'tiktok',
-     'instagram', 'twitter', 'youtube', 'zhihu', 'douban',
-     'jike', 'weixin', 'xiaoyuzhou',
-   ]
-
-   onAfterExecute(async (ctx) => {
-     if (process.env.OPENCLI_NO_RATE === '1') return
-
-     const site = ctx.command?.split('/')?.[0] ?? ''
-     if (!BROWSER_DOMAINS.includes(site)) return
-
-     const min = Number(process.env.OPENCLI_RATE_MIN ?? 5)
-     const max = Number(process.env.OPENCLI_RATE_MAX ?? 30)
-     const ms = Math.floor(Math.random() * (max - min + 1) + min) * 1000
-
-     process.stderr.write(`[rate-limiter] ${site}: sleeping ${(ms / 1000).toFixed(0)}s\n`)
-     await new Promise(r => setTimeout(r, ms))
-   })
-   ```
-
-4. Verify it loaded:
-
-   ```bash
-   OPENCLI_NO_RATE=1 opencli xiaohongshu search "test" 2>&1 | grep rate-limiter
-   # → (no output — plugin loaded but rate limit skipped)
-
-   opencli xiaohongshu search "test" 2>&1 | grep rate-limiter
-   # → [rate-limiter] xiaohongshu: sleeping 12s
-   ```
-
-## Writing your own plugin
-
-Plugins are plain JS/TS files in `~/.opencli/plugins/<name>/`. A plugin file must export a hook registration call that matches the pattern `onStartup(`, `onBeforeExecute(`, or `onAfterExecute(` — opencli's discovery engine uses this pattern to identify hook files vs. command files.
-
-```js
-// ~/.opencli/plugins/my-plugin/index.js
-import { onAfterExecute } from '@jackwener/opencli/hooks'
-
-onAfterExecute(async (ctx) => {
-  // ctx.command — e.g. "bilibili/comments"
-  // ctx.args    — coerced command arguments
-  // ctx.error   — set if the command threw
-  console.error(`[my-plugin] finished: ${ctx.command}`)
-})
-```
-
-See [hooks.ts](../../src/hooks.ts) for the full `HookContext` type.

package/docs/advanced/remote-chrome.md

@@ -1,72 +0,0 @@
-# Remote Chrome
-
-Run OpenCLI on a server or headless environment by connecting to a remote Chrome instance.
-
-## Use Cases
-
-- Running CLI commands on a remote server
-- CI/CD automation with headed browser
-- Shared team browser sessions
-
-## Setup
-
-### 1. Start Chrome on the Remote Machine
-
-```bash
-# On the remote machine (or your Mac)
-/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
-  --remote-debugging-port=9222
-```
-
-### 2. SSH Tunnel (If Needed)
-
-If the remote Chrome is on a different machine, create an SSH tunnel:
-
-```bash
-# On your local machine or server
-ssh -L 9222:127.0.0.1:9222 user@remote-host
-```
-
-::: warning
-Use `127.0.0.1` instead of `localhost` in the SSH command to avoid IPv6 resolution issues that can cause timeouts.
-:::
-
-### 3. Configure OpenCLI
-
-```bash
-export OPENCLI_CDP_ENDPOINT="http://127.0.0.1:9222"
-```
-
-### 4. Verify
-
-```bash
-# Test the connection
-curl http://127.0.0.1:9222/json/version
-
-# Run a diagnostic
-opencli doctor
-```
-
-## CI/CD Integration
-
-For CI/CD environments, use a real Chrome instance with `xvfb`:
-
-::: v-pre
-```yaml
-steps:
-  - uses: browser-actions/setup-chrome@latest
-    id: setup-chrome
-  - run: |
-      xvfb-run --auto-servernum \
-        ${{ steps.setup-chrome.outputs.chrome-path }} \
-        --remote-debugging-port=9222 &
-```
-:::
-
-Set the browser executable path:
-::: v-pre
-```yaml
-env:
-  OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
-```
-:::

package/docs/comparison.md

@@ -1,125 +0,0 @@
-# Comparison Guide
-
-OpenCLI occupies a specific niche in the browser automation ecosystem. This guide honestly evaluates where opencli excels, where it's a viable option, and where other tools are a better fit.
-
-## At a Glance
-
-| Tool | Approach | Best for |
-|------|----------|----------|
-| **opencli** | Pre-built adapters (YAML/TS) | Deterministic site commands, broad platform coverage, desktop apps |
-| **Browser-Use** | LLM-driven browser control | General-purpose AI browser automation |
-| **Crawl4AI** | Async web crawler | Large-scale data crawling |
-| **Firecrawl** | Scraping API / self-hosted | Clean markdown extraction, managed or self-hosted infrastructure |
-| **agent-browser** | Browser primitive CLI | Token-efficient AI agent browsing |
-| **Stagehand** | AI browser framework | Developer-friendly browser automation |
-| **Skyvern** | Visual AI automation | Cross-site generalized workflows |
-
-## Scenario Comparison
-
-### 1. Scheduled Batch Data Extraction
-
-> "I want to pull trending posts from Bilibili/Reddit/HackerNews every hour into my pipeline."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | One command, structured JSON output, zero runtime cost. Runs in cron/CI without tokens or API keys. |
-| Crawl4AI | Good | Strong for large-scale crawling, but requires writing extraction logic per site. |
-| Firecrawl | Viable | Managed service with clean output, but costs scale with volume. |
-| Browser-Use / Stagehand | Poor | LLM inference on every run is slow, expensive, and non-deterministic for repeated tasks. |
-
-**Why opencli wins here:** A command like `opencli bilibili hot -f json` returns the same structured schema every time, costs nothing to run, and finishes in seconds. For recurring data extraction from known sites, pre-built adapters beat LLM-driven approaches on cost, speed, and reliability.
-
-### 2. AI Agent Site Operations
-
-> "My AI agent needs to search Twitter, read Reddit threads, or post to Xiaohongshu."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | Structured JSON output, fast deterministic execution, hundreds of commands ready to use. |
-| agent-browser | Good | Token-efficient browser primitives, but requires LLM reasoning for every step. |
-| Browser-Use | Viable | General-purpose, but each operation costs tokens and takes 10-60s. |
-| Stagehand | Viable | Good DX, but same LLM-per-action cost model. |
-
-**Why opencli wins here:** When your agent needs `twitter search "AI news" -f json`, a deterministic command that returns in seconds is strictly better than an LLM clicking through a webpage. The agent saves tokens for reasoning, not navigation.
-
-### 3. Authenticated Operations (Login-Required Sites)
-
-> "I need to access my bookmarks, post content, or interact with sites that require login."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | Reuses your Chrome login session via Browser Bridge. No credentials stored or transmitted. |
-| Browser-Use | Viable | Can use browser profiles, but credential management is manual. |
-| Firecrawl | Poor | Cloud service cannot access your authenticated sessions. |
-| Crawl4AI | Poor | Requires manual cookie/session injection. |
-
-**Why opencli wins here:** The Browser Bridge extension reuses your existing Chrome login state in real-time. You log in once in Chrome, and opencli commands work immediately. No OAuth setup, no API keys, no credential files.
-
-### 4. General Web Browsing & Exploration
-
-> "I need to explore an unknown website, fill forms, or navigate complex multi-step flows."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| Browser-Use | Best | LLM-driven, handles arbitrary websites and flows. |
-| Stagehand | Best | Clean API for `act()`, `extract()`, `observe()` on any page. |
-| agent-browser | Good | Token-efficient primitives for AI agents. |
-| Skyvern | Good | Visual AI that generalizes across sites. |
-| **opencli** | Poor | Only works with sites that have pre-built adapters. Cannot handle arbitrary websites. |
-
-**opencli is not the right tool here.** If you need to explore unknown websites or handle one-off tasks on sites without adapters, use an LLM-driven browser tool. opencli trades generality for determinism and cost.
-
-### 5. Desktop App Control
-
-> "I want to script Cursor, ChatGPT, Notion, or other Electron apps from the terminal."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | 8 desktop adapters via CDP + AppleScript. The only CLI tool with this capability. |
-| All others | N/A | Browser automation tools cannot control desktop applications. |
-
-**This is unique to opencli.** No other tool in this comparison can send a prompt to ChatGPT desktop, extract code from Cursor, or write to Notion pages via CLI.
-
-## Key Trade-offs
-
-### opencli's Strengths
-
-- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times for free.
-- **Deterministic output** — Same command always returns the same schema. Pipeable, scriptable, CI-friendly.
-- **Speed** — Adapter commands return in seconds, not minutes.
-- **Broad platform coverage** — 50+ sites spanning global platforms (Reddit, HackerNews, Twitter, YouTube) and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Douban, Weibo) with adapters that understand local anti-bot patterns.
-- **Desktop app control** — CDP adapters for Cursor, Codex, Notion, ChatGPT, Discord, and more.
-- **Easy to extend** — Drop a `.yaml` or `.ts` adapter into the `clis/` folder for auto-registration. Contributing a new site adapter is straightforward.
-
-### opencli's Limitations
-
-- **Coverage requires adapters** — opencli only works with sites that have pre-built adapters. Adding a new site means writing a YAML or TypeScript adapter.
-- **Adapter maintenance** — When a website updates its DOM or API, the corresponding adapter may need updating. The community maintains these, but breakage is possible.
-- **Not general-purpose** — Cannot handle arbitrary websites. For unknown sites, pair opencli with a general browser tool as a fallback.
-
-## Complementary Usage
-
-opencli works best alongside general-purpose browser tools, not as a replacement:
-
-```
-Has adapter?  ──yes──▶  opencli (fast, free, deterministic)
-     │
-     no
-     │
-     ▼
-One-off task?  ──yes──▶  Browser-Use / Stagehand (LLM-driven)
-     │
-     no
-     │
-     ▼
-Recurring?    ──yes──▶  Write an opencli adapter, then use opencli
-```
-
-## Further Reading
-
-- [Architecture Overview](./developer/architecture.md)
-- [Writing a YAML Adapter](./developer/yaml-adapter.md)
-- [Writing a TypeScript Adapter](./developer/ts-adapter.md)
-- [Testing Guide](./developer/testing.md)
-- [AI Workflow](./developer/ai-workflow.md)
-- [Contributing Guide](./developer/contributing.md)

package/docs/developer/ai-workflow.md

@@ -1,66 +0,0 @@
-# AI Workflow
-
-OpenCLI is designed with AI agents in mind. This guide covers the AI-native discovery and code generation tools.
-
-## Quick Mode (One-Shot)
-
-Generate a single command for a specific page URL — just a URL + one-line goal, 4 steps done:
-
-```bash
-opencli generate https://example.com --goal "trending"
-```
-
-This runs: explore → synthesize → register in one shot.
-
-For the complete one-shot workflow details, see [CLI-ONESHOT.md](https://github.com/jackwener/opencli/blob/main/CLI-ONESHOT.md).
-
-## Full Mode (Explorer Workflow)
-
-### Step 1: Deep Explore
-
-Discover APIs, infer capabilities, and detect framework:
-
-```bash
-opencli explore https://example.com --site mysite
-```
-
-Outputs to `.opencli/explore/<site>/`:
-- `manifest.json` — Site metadata
-- `endpoints.json` — Discovered API endpoints
-- `capabilities.json` — Inferred capabilities
-- `auth.json` — Authentication strategy details
-
-### Step 2: Synthesize
-
-Generate YAML adapters from explore artifacts:
-
-```bash
-opencli synthesize mysite
-```
-
-### Step 3: Strategy Cascade
-
-Auto-probe authentication strategies: `PUBLIC → COOKIE → HEADER`:
-
-```bash
-opencli cascade https://api.example.com/data
-```
-
-### Step 4: Validate & Test
-
-```bash
-opencli validate                           # Validate generated YAML
-opencli <site> <command> --limit 3 -f json # Test the command
-```
-
-## 5-Tier Authentication Strategy
-
-The explorer uses a decision tree to determine the best authentication approach:
-
-1. **PUBLIC** — No auth, direct API call
-2. **COOKIE** — Reuse Chrome session cookies
-3. **HEADER** — Custom auth headers
-4. **BROWSER** — Full browser automation
-5. **CDP** — Chrome DevTools Protocol for Electron apps
-
-For the complete browser exploration workflow and debugging guide, see [CLI-EXPLORER.md](https://github.com/jackwener/opencli/blob/main/CLI-EXPLORER.md).

package/docs/developer/architecture.md

@@ -1,103 +0,0 @@
-# Architecture
-
-OpenCLI is built on a **Dual-Engine Architecture** that supports both declarative YAML pipelines and programmatic TypeScript adapters.
-
-## High-Level Architecture
-
-```
-┌─────────────────────────────────────────────────────┐
-│                     opencli CLI                      │
-│              (Commander.js entry point)               │
-├─────────────────────────────────────────────────────┤
-│                   Engine Layer                        │
-│  ┌──────────────┐  ┌──────────────┐  ┌────────────┐ │
-│  │   Registry   │  │   Dynamic    │  │   Output   │ │
-│  │  (commands)  │  │   Loader     │  │ Formatter  │ │
-│  └──────────────┘  └──────────────┘  └────────────┘ │
-├─────────────────────────────────────────────────────┤
-│                 Adapter Layer                         │
-│  ┌─────────────────┐  ┌──────────────────────────┐  │
-│  │  YAML Pipeline  │  │  TypeScript Adapters     │  │
-│  │  (declarative)  │  │  (browser/desktop/AI)    │  │
-│  └─────────────────┘  └──────────────────────────┘  │
-├─────────────────────────────────────────────────────┤
-│              Connection Layer                         │
-│  ┌─────────────────┐  ┌──────────────────────────┐  │
-│  │ Browser Bridge  │  │  CDP (Chrome DevTools)   │  │
-│  │ (Extension+WS)  │  │  (Electron apps)         │  │
-│  └─────────────────┘  └──────────────────────────┘  │
-└─────────────────────────────────────────────────────┘
-```
-
-## Core Modules
-
-### Registry (`src/registry.ts`)
-Central command registry. All adapters register their commands via the `cli()` function with metadata: site, name, description, domain, strategy, args, columns.
-
-### Discovery (`src/discovery.ts`)
-CLI discovery and manifest loading. Discovers commands from YAML and TypeScript adapter files, parses YAML pipelines, and registers them into the central registry.
-
-### Execution (`src/execution.ts`)
-Command execution: argument validation, lazy loading of adapter modules, and executing the appropriate handler function.
-
-### Commander Adapter (`src/commanderAdapter.ts`)
-Bridges the Registry commands to Commander.js subcommands. Handles positional args, named options, browser session wiring, and output formatting. Isolates all Commander-specific logic so the core is framework-agnostic.
-
-### Browser (`src/browser.ts`)
-Manages connections to Chrome via the Browser Bridge WebSocket daemon. Handles JSON-RPC messaging, tab management, and extension/standalone mode switching.
-
-### Pipeline (`src/pipeline/`)
-The YAML pipeline engine. Processes declarative steps:
-- **fetch** — HTTP requests with cookie/header strategies
-- **map** — Data transformation with template expressions
-- **limit** — Result truncation
-- **filter** — Conditional filtering
-- **download** — Media download support
-
-### Output (`src/output.ts`)
-Unified output formatting: `table`, `json`, `yaml`, `md`, `csv`.
-
-## Authentication Strategies
-
-OpenCLI uses a 3-tier authentication strategy:
-
-| Strategy | How It Works | When to Use |
-|----------|-------------|-------------|
-| `public` | Direct HTTP fetch, no auth | Public APIs (HackerNews, BBC) |
-| `cookie` | Reuse Chrome cookies via Browser Bridge | Logged-in sites (Bilibili, Zhihu) |
-| `header` | Custom auth headers | API-key based services |
-| `intercept` | Network request interception | GraphQL/XHR capture (Twitter) |
-| `ui` | DOM interaction via accessibility snapshot | Desktop apps, write operations |
-
-## Directory Structure
-
-```
-src/
-├── main.ts              # Entry point
-├── cli.ts               # Commander.js CLI setup + built-in commands
-├── commanderAdapter.ts  # Registry → Commander bridge
-├── discovery.ts         # CLI discovery, manifest loading, YAML parsing
-├── execution.ts         # Arg validation, command execution
-├── registry.ts          # Command registry
-├── serialization.ts     # Command serialization helpers
-├── runtime.ts           # Browser session & timeout management
-├── browser/             # Browser Bridge connection
-├── output.ts            # Output formatting
-├── doctor.ts            # Diagnostic tool
-├── pipeline/            # YAML pipeline engine
-│   ├── runner.ts
-│   ├── template.ts
-│   ├── transform.ts
-│   └── steps/
-│       ├── fetch.ts
-│       ├── map.ts
-│       ├── limit.ts
-│       ├── filter.ts
-│       └── download.ts
-└── clis/                # Site adapters
-    ├── twitter/
-    ├── reddit/
-    ├── bilibili/
-    ├── cursor/
-    └── ...
-```