@jackwener/opencli 1.6.2 → 1.6.4

package/docs/comparison.md

@@ -1,125 +0,0 @@
-# Comparison Guide
-
-OpenCLI occupies a specific niche in the browser automation ecosystem. This guide honestly evaluates where opencli excels, where it's a viable option, and where other tools are a better fit.
-
-## At a Glance
-
-| Tool | Approach | Best for |
-|------|----------|----------|
-| **opencli** | Pre-built adapters (YAML/TS) | Deterministic site commands, broad platform coverage, desktop apps |
-| **Browser-Use** | LLM-driven browser control | General-purpose AI browser automation |
-| **Crawl4AI** | Async web crawler | Large-scale data crawling |
-| **Firecrawl** | Scraping API / self-hosted | Clean markdown extraction, managed or self-hosted infrastructure |
-| **agent-browser** | Browser primitive CLI | Token-efficient AI agent browsing |
-| **Stagehand** | AI browser framework | Developer-friendly browser automation |
-| **Skyvern** | Visual AI automation | Cross-site generalized workflows |
-
-## Scenario Comparison
-
-### 1. Scheduled Batch Data Extraction
-
-> "I want to pull trending posts from Bilibili/Reddit/HackerNews every hour into my pipeline."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | One command, structured JSON output, zero runtime cost. Runs in cron/CI without tokens or API keys. |
-| Crawl4AI | Good | Strong for large-scale crawling, but requires writing extraction logic per site. |
-| Firecrawl | Viable | Managed service with clean output, but costs scale with volume. |
-| Browser-Use / Stagehand | Poor | LLM inference on every run is slow, expensive, and non-deterministic for repeated tasks. |
-
-**Why opencli wins here:** A command like `opencli bilibili hot -f json` returns the same structured schema every time, costs nothing to run, and finishes in seconds. For recurring data extraction from known sites, pre-built adapters beat LLM-driven approaches on cost, speed, and reliability.
-
-### 2. AI Agent Site Operations
-
-> "My AI agent needs to search Twitter, read Reddit threads, or post to Xiaohongshu."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | Structured JSON output, fast deterministic execution, hundreds of commands ready to use. |
-| agent-browser | Good | Token-efficient browser primitives, but requires LLM reasoning for every step. |
-| Browser-Use | Viable | General-purpose, but each operation costs tokens and takes 10-60s. |
-| Stagehand | Viable | Good DX, but same LLM-per-action cost model. |
-
-**Why opencli wins here:** When your agent needs `twitter search "AI news" -f json`, a deterministic command that returns in seconds is strictly better than an LLM clicking through a webpage. The agent saves tokens for reasoning, not navigation.
-
-### 3. Authenticated Operations (Login-Required Sites)
-
-> "I need to access my bookmarks, post content, or interact with sites that require login."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | Reuses your Chrome login session via Browser Bridge. No credentials stored or transmitted. |
-| Browser-Use | Viable | Can use browser profiles, but credential management is manual. |
-| Firecrawl | Poor | Cloud service cannot access your authenticated sessions. |
-| Crawl4AI | Poor | Requires manual cookie/session injection. |
-
-**Why opencli wins here:** The Browser Bridge extension reuses your existing Chrome login state in real-time. You log in once in Chrome, and opencli commands work immediately. No OAuth setup, no API keys, no credential files.
-
-### 4. General Web Browsing & Exploration
-
-> "I need to explore an unknown website, fill forms, or navigate complex multi-step flows."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| Browser-Use | Best | LLM-driven, handles arbitrary websites and flows. |
-| Stagehand | Best | Clean API for `act()`, `extract()`, `observe()` on any page. |
-| agent-browser | Good | Token-efficient primitives for AI agents. |
-| Skyvern | Good | Visual AI that generalizes across sites. |
-| **opencli** | Poor | Only works with sites that have pre-built adapters. Cannot handle arbitrary websites. |
-
-**opencli is not the right tool here.** If you need to explore unknown websites or handle one-off tasks on sites without adapters, use an LLM-driven browser tool. opencli trades generality for determinism and cost.
-
-### 5. Desktop App Control
-
-> "I want to script Cursor, ChatGPT, Notion, or other Electron apps from the terminal."
-
-| Tool | Fit | Notes |
-|------|-----|-------|
-| **opencli** | Best | 8 desktop adapters via CDP + AppleScript. The only CLI tool with this capability. |
-| All others | N/A | Browser automation tools cannot control desktop applications. |
-
-**This is unique to opencli.** No other tool in this comparison can send a prompt to ChatGPT desktop, extract code from Cursor, or write to Notion pages via CLI.
-
-## Key Trade-offs
-
-### opencli's Strengths
-
-- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times for free.
-- **Deterministic output** — Same command always returns the same schema. Pipeable, scriptable, CI-friendly.
-- **Speed** — Adapter commands return in seconds, not minutes.
-- **Broad platform coverage** — 73+ sites spanning global platforms (Reddit, HackerNews, Twitter, YouTube) and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Douban, Weibo) with adapters that understand local anti-bot patterns.
-- **Desktop app control** — CDP adapters for Cursor, Codex, Notion, ChatGPT, Discord, and more.
-- **Easy to extend** — Drop a `.yaml` or `.ts` adapter into the `clis/` folder for auto-registration. Contributing a new site adapter is straightforward.
-
-### opencli's Limitations
-
-- **Coverage requires adapters** — opencli only works with sites that have pre-built adapters. Adding a new site means writing a YAML or TypeScript adapter.
-- **Adapter maintenance** — When a website updates its DOM or API, the corresponding adapter may need updating. The community maintains these, but breakage is possible.
-- **Not general-purpose** — Cannot handle arbitrary websites. For unknown sites, pair opencli with a general browser tool as a fallback.
-
-## Complementary Usage
-
-opencli works best alongside general-purpose browser tools, not as a replacement:
-
-```
-Has adapter?  ──yes──▶  opencli (fast, free, deterministic)
-     │
-     no
-     │
-     ▼
-One-off task?  ──yes──▶  Browser-Use / Stagehand (LLM-driven)
-     │
-     no
-     │
-     ▼
-Recurring?    ──yes──▶  Write an opencli adapter, then use opencli
-```
-
-## Further Reading
-
-- [Architecture Overview](./developer/architecture.md)
-- [Writing a YAML Adapter](./developer/yaml-adapter.md)
-- [Writing a TypeScript Adapter](./developer/ts-adapter.md)
-- [Testing Guide](./developer/testing.md)
-- [AI Workflow](./developer/ai-workflow.md)
-- [Contributing Guide](./developer/contributing.md)

package/docs/developer/ai-workflow.md

@@ -1,66 +0,0 @@
-# AI Workflow
-
-OpenCLI is designed with AI agents in mind. This guide covers the AI-native discovery and code generation tools.
-
-## Quick Mode (One-Shot)
-
-Generate a single command for a specific page URL — just a URL + one-line goal, 4 steps done:
-
-```bash
-opencli generate https://example.com --goal "trending"
-```
-
-This runs: explore → synthesize → register in one shot.
-
-For the complete one-shot workflow details, see [opencli-oneshot skill](https://github.com/jackwener/opencli/blob/main/skills/opencli-oneshot/SKILL.md).
-
-## Full Mode (Explorer Workflow)
-
-### Step 1: Deep Explore
-
-Discover APIs, infer capabilities, and detect framework:
-
-```bash
-opencli explore https://example.com --site mysite
-```
-
-Outputs to `.opencli/explore/<site>/`:
-- `manifest.json` — Site metadata
-- `endpoints.json` — Discovered API endpoints
-- `capabilities.json` — Inferred capabilities
-- `auth.json` — Authentication strategy details
-
-### Step 2: Synthesize
-
-Generate YAML adapters from explore artifacts:
-
-```bash
-opencli synthesize mysite
-```
-
-### Step 3: Strategy Cascade
-
-Auto-probe authentication strategies: `PUBLIC → COOKIE → HEADER`:
-
-```bash
-opencli cascade https://api.example.com/data
-```
-
-### Step 4: Validate & Test
-
-```bash
-opencli validate                           # Validate generated YAML
-opencli <site> <command> --limit 3 -f json # Test the command
-```
-
-## 5-Tier Authentication Strategy
-
-The explorer uses a decision tree to determine the best authentication approach:
-
-1. **PUBLIC** — No auth, direct API call
-2. **COOKIE** — Reuse Chrome session cookies
-3. **HEADER** — Custom auth headers
-4. **BROWSER** — Full browser automation
-5. **CDP** — Chrome DevTools Protocol for Electron apps
-
-For the complete browser exploration workflow and debugging guide, see [opencli-explorer skill](https://github.com/jackwener/opencli/blob/main/skills/opencli-explorer/SKILL.md).

package/docs/developer/architecture.md

@@ -1,103 +0,0 @@
-# Architecture
-
-OpenCLI is built on a **Dual-Engine Architecture** that supports both declarative YAML pipelines and programmatic TypeScript adapters.
-
-## High-Level Architecture
-
-```
-┌─────────────────────────────────────────────────────┐
-│                     opencli CLI                      │
-│              (Commander.js entry point)               │
-├─────────────────────────────────────────────────────┤
-│                   Engine Layer                        │
-│  ┌──────────────┐  ┌──────────────┐  ┌────────────┐ │
-│  │   Registry   │  │   Dynamic    │  │   Output   │ │
-│  │  (commands)  │  │   Loader     │  │ Formatter  │ │
-│  └──────────────┘  └──────────────┘  └────────────┘ │
-├─────────────────────────────────────────────────────┤
-│                 Adapter Layer                         │
-│  ┌─────────────────┐  ┌──────────────────────────┐  │
-│  │  YAML Pipeline  │  │  TypeScript Adapters     │  │
-│  │  (declarative)  │  │  (browser/desktop/AI)    │  │
-│  └─────────────────┘  └──────────────────────────┘  │
-├─────────────────────────────────────────────────────┤
-│              Connection Layer                         │
-│  ┌─────────────────┐  ┌──────────────────────────┐  │
-│  │ Browser Bridge  │  │  CDP (Chrome DevTools)   │  │
-│  │ (Extension+WS)  │  │  (Electron apps)         │  │
-│  └─────────────────┘  └──────────────────────────┘  │
-└─────────────────────────────────────────────────────┘
-```
-
-## Core Modules
-
-### Registry (`src/registry.ts`)
-Central command registry. All adapters register their commands via the `cli()` function with metadata: site, name, description, domain, strategy, args, columns.
-
-### Discovery (`src/discovery.ts`)
-CLI discovery and manifest loading. Discovers commands from YAML and TypeScript adapter files, parses YAML pipelines, and registers them into the central registry.
-
-### Execution (`src/execution.ts`)
-Command execution: argument validation, lazy loading of adapter modules, and executing the appropriate handler function.
-
-### Commander Adapter (`src/commanderAdapter.ts`)
-Bridges the Registry commands to Commander.js subcommands. Handles positional args, named options, browser session wiring, and output formatting. Isolates all Commander-specific logic so the core is framework-agnostic.
-
-### Browser (`src/browser.ts`)
-Manages connections to Chrome via the Browser Bridge WebSocket daemon. Handles JSON-RPC messaging, tab management, and extension/standalone mode switching.
-
-### Pipeline (`src/pipeline/`)
-The YAML pipeline engine. Processes declarative steps:
-- **fetch** — HTTP requests with cookie/header strategies
-- **map** — Data transformation with template expressions
-- **limit** — Result truncation
-- **filter** — Conditional filtering
-- **download** — Media download support
-
-### Output (`src/output.ts`)
-Unified output formatting: `table`, `json`, `yaml`, `md`, `csv`.
-
-## Authentication Strategies
-
-OpenCLI uses a 3-tier authentication strategy:
-
-| Strategy | How It Works | When to Use |
-|----------|-------------|-------------|
-| `public` | Direct HTTP fetch, no auth | Public APIs (HackerNews, BBC) |
-| `cookie` | Reuse Chrome cookies via Browser Bridge | Logged-in sites (Bilibili, Zhihu) |
-| `header` | Custom auth headers | API-key based services |
-| `intercept` | Network request interception | GraphQL/XHR capture (Twitter) |
-| `ui` | DOM interaction via accessibility snapshot | Desktop apps, write operations |
-
-## Directory Structure
-
-```
-src/
-├── main.ts              # Entry point
-├── cli.ts               # Commander.js CLI setup + built-in commands
-├── commanderAdapter.ts  # Registry → Commander bridge
-├── discovery.ts         # CLI discovery, manifest loading, YAML parsing
-├── execution.ts         # Arg validation, command execution
-├── registry.ts          # Command registry
-├── serialization.ts     # Command serialization helpers
-├── runtime.ts           # Browser session & timeout management
-├── browser/             # Browser Bridge connection
-├── output.ts            # Output formatting
-├── doctor.ts            # Diagnostic tool
-├── pipeline/            # YAML pipeline engine
-│   ├── runner.ts
-│   ├── template.ts
-│   ├── transform.ts
-│   └── steps/
-│       ├── fetch.ts
-│       ├── map.ts
-│       ├── limit.ts
-│       ├── filter.ts
-│       └── download.ts
-└── clis/                # Site adapters
-    ├── twitter/
-    ├── reddit/
-    ├── bilibili/
-    ├── cursor/
-    └── ...
-```

package/docs/developer/contributing.md

@@ -1,155 +0,0 @@
-# Contributing
-
-Thanks for your interest in contributing to OpenCLI.
-
-## Quick Start
-
-```bash
-# 1. Fork & clone
-git clone git@github.com:<your-username>/opencli.git
-cd opencli
-
-# 2. Install dependencies
-npm install
-
-# 3. Build
-npm run build
-
-# 4. Run a few checks
-npx tsc --noEmit
-npm test
-npm run test:adapter
-
-# 5. Link globally (optional, for testing `opencli` command)
-npm link
-```
-
-## Adding a New Site Adapter
-
-This is the most common type of contribution. Start with YAML when possible, and use TypeScript only when you need browser-side logic or multi-step flows.
-
-Before you start:
-
-- Prefer positional args for the command's primary subject (`search <query>`, `topic <id>`, `download <url>`). Reserve named flags for optional modifiers such as `--limit`, `--sort`, `--lang`, and `--output`.
-- Normalize expected adapter failures to `CliError` subclasses instead of raw `Error` whenever possible. Prefer `AuthRequiredError`, `EmptyResultError`, `CommandExecutionError`, `TimeoutError`, and `ArgumentError` so the top-level CLI can render better messages and hints.
-- If you add a new adapter or make a command newly discoverable, update the matching doc page and the user-facing indexes that expose it.
-
-### YAML Adapter (Recommended for data-fetching commands)
-
-Create a file like `src/clis/<site>/<command>.yaml`:
-
-::: v-pre
-```yaml
-site: mysite
-name: trending
-description: Trending posts on MySite
-domain: www.mysite.com
-strategy: public      # public | cookie | header
-browser: false        # true if browser session is needed
-
-args:
-  limit:
-    type: int
-    default: 20
-    description: Number of items
-
-pipeline:
-  - fetch:
-      url: https://api.mysite.com/trending
-
-  - map:
-      rank: ${{ index + 1 }}
-      title: ${{ item.title }}
-      score: ${{ item.score }}
-      url: ${{ item.url }}
-
-  - limit: ${{ args.limit }}
-
-columns: [rank, title, score, url]
-```
-:::
-
-See [`hackernews/top.yaml`](https://github.com/jackwener/opencli/blob/main/src/clis/hackernews/top.yaml) for a real example.
-
-### TypeScript Adapter (For complex browser interactions)
-
-Create a file like `src/clis/<site>/<command>.ts`:
-
-```typescript
-import { cli, Strategy } from '../../registry.js';
-import { CommandExecutionError, EmptyResultError } from '../../errors.js';
-
-cli({
-  site: 'mysite',
-  name: 'search',
-  description: 'Search MySite',
-  domain: 'www.mysite.com',
-  strategy: Strategy.COOKIE,
-  args: [
-    { name: 'query', positional: true, required: true, help: 'Search query' },
-    { name: 'limit', type: 'int', default: 10, help: 'Max results' },
-  ],
-  columns: ['title', 'url', 'date'],
-
-  func: async (page, kwargs) => {
-    const { query, limit = 10 } = kwargs;
-    // ... browser automation logic
-    if (!Array.isArray(data)) throw new CommandExecutionError('MySite returned an unexpected response');
-    if (!data.length) throw new EmptyResultError('mysite search', 'Try a different keyword');
-    return data.slice(0, Number(limit)).map((item: any) => ({
-      title: item.title,
-      url: item.url,
-      date: item.created_at,
-    }));
-  },
-});
-```
-
-### Validate Your Adapter
-
-```bash
-opencli validate               # Validate YAML syntax and schema
-opencli <site> <command> --limit 3 -f json   # Test your command
-opencli <site> <command> -v    # Verbose mode for debugging
-```
-
-## Code Style
-
-- **TypeScript strict mode** — avoid `any` where possible.
-- **ES Modules** — use `.js` extensions in imports (TypeScript output).
-- **Naming**: `kebab-case` for files, `camelCase` for variables/functions, `PascalCase` for types/classes.
-- **No default exports** — use named exports.
-- **Errors** — throw `CliError` subclasses for expected adapter failures; avoid raw `Error` for normal adapter control flow.
-
-## Commit Convention
-
-We use [Conventional Commits](https://www.conventionalcommits.org/):
-
-```
-feat(twitter): add thread command
-fix(browser): handle CDP timeout gracefully
-docs: update CONTRIBUTING.md
-test(reddit): add e2e test for save command
-chore: bump vitest to v4
-```
-
-## Submitting a Pull Request
-
-1. Create a feature branch: `git checkout -b feat/mysite-trending`
-2. Make your changes and add tests when relevant
-3. Run the checks:
-   ```bash
-   npx tsc --noEmit           # Type check
-   npm test                   # Core unit tests
-   npm run test:adapter       # Focused adapter tests (if adapter logic changed)
-   opencli validate           # YAML validation (if applicable)
-   ```
-4. Commit using conventional commit format
-5. Push and open a PR
-
-If your PR adds a new adapter or changes user-facing commands, also verify:
-
-- Adapter docs exist under `docs/adapters/`
-- `docs/adapters/index.md` is updated for new adapters
-- VitePress sidebar includes the new doc page
-- `README.md` / `README.zh-CN.md` stay aligned when command discoverability changes