publishport-opencli 1.0.0

package/docs/guide/plugins.md

@@ -0,0 +1,226 @@
+# Plugins
+
+OpenCLI supports community-contributed plugins. Install third-party adapters from GitHub, and they're automatically discovered alongside built-in commands.
+
+## Quick Start
+
+```bash
+# Install a plugin
+opencli plugin install github:ByteYue/opencli-plugin-github-trending
+
+# List installed plugins
+opencli plugin list
+
+# Update one plugin
+opencli plugin update github-trending
+
+# Update all installed plugins
+opencli plugin update --all
+
+# Use the plugin (it's just a regular command)
+opencli github-trending repos --limit 10
+
+# Remove a plugin
+opencli plugin uninstall github-trending
+```
+
+## How Plugins Work
+
+Plugins live in `~/.opencli/plugins/<name>/`. Each subdirectory is scanned at startup for `.ts` or `.js` command files — the same formats used by built-in adapters.
+
+### Supported Source Formats
+
+```bash
+# GitHub shorthand
+opencli plugin install github:user/repo
+opencli plugin install github:user/repo/subplugin   # install specific sub-plugin from monorepo
+opencli plugin install https://github.com/user/repo
+
+# Any git-cloneable URL
+opencli plugin install https://gitlab.example.com/team/repo.git
+opencli plugin install ssh://git@gitlab.example.com/team/repo.git
+opencli plugin install git@gitlab.example.com:team/repo.git
+
+# Local plugin (for development)
+opencli plugin install file:///path/to/plugin
+opencli plugin install /path/to/plugin
+```
+
+The repo name prefix `opencli-plugin-` is automatically stripped for the local directory name. For example, `opencli-plugin-hot-digest` becomes `hot-digest`.
+
+## Plugin Manifest (`opencli-plugin.json`)
+
+Plugins can include an `opencli-plugin.json` manifest file at the repo root to declare metadata:
+
+```json
+{
+  "name": "my-plugin",
+  "version": "1.0.0",
+  "opencli": ">=1.0.0",
+  "description": "My awesome plugin"
+}
+```
+
+| Field | Description |
+|-------|-------------|
+| `name` | Plugin name (overrides repo-derived name) |
+| `version` | Semantic version |
+| `opencli` | Required opencli version range (e.g. `>=1.0.0`, `^1.2.0`) |
+| `description` | Human-readable description |
+| `plugins` | Monorepo sub-plugin declarations (see below) |
+
+The manifest is optional — plugins without one continue to work exactly as before.
+
+## Monorepo Plugins
+
+A single repository can contain multiple plugins by declaring a `plugins` field in `opencli-plugin.json`:
+
+```json
+{
+  "version": "1.0.0",
+  "opencli": ">=1.0.0",
+  "description": "My plugin collection",
+  "plugins": {
+    "polymarket": {
+      "path": "packages/polymarket",
+      "description": "Prediction market analysis",
+      "version": "1.2.0"
+    },
+    "defi": {
+      "path": "packages/defi",
+      "description": "DeFi protocol data",
+      "version": "0.8.0",
+      "opencli": ">=1.2.0"
+    },
+    "experimental": {
+      "path": "packages/experimental",
+      "disabled": true
+    }
+  }
+}
+```
+
+### Installing
+
+```bash
+# Install ALL enabled sub-plugins from a monorepo
+opencli plugin install github:user/opencli-plugins
+
+# Install a SPECIFIC sub-plugin
+opencli plugin install github:user/opencli-plugins/polymarket
+```
+
+### How It Works
+
+- The monorepo is cloned once to `~/.opencli/monorepos/<repo>/`
+- Each sub-plugin gets a symlink in `~/.opencli/plugins/<name>/` pointing to its subdirectory
+- Command discovery works transparently — symlinks are scanned just like regular directories
+- Disabled sub-plugins (with `"disabled": true`) are skipped during install
+- Sub-plugins can specify their own `opencli` compatibility range
+
+### Updating
+
+Updating any sub-plugin from a monorepo pulls the entire repo and refreshes all sub-plugins:
+
+```bash
+opencli plugin update polymarket   # updates the monorepo, refreshes all
+```
+
+### Uninstalling
+
+```bash
+opencli plugin uninstall polymarket   # removes just this sub-plugin's symlink
+```
+
+When the last sub-plugin from a monorepo is uninstalled, the monorepo clone is automatically cleaned up.
+
+## Version Tracking
+
+OpenCLI records installed plugin versions in `~/.opencli/plugins.lock.json`. Each entry stores the plugin source, current git commit hash, install time, and last update time. `opencli plugin list` shows the short commit hash when version metadata is available.
+
+## Creating a Plugin
+
+### Creating a TypeScript Plugin
+
+```
+my-plugin/
+├── package.json
+├── my-command.ts
+└── README.md
+```
+
+`package.json`:
+
+```json
+{
+  "name": "opencli-plugin-my-plugin",
+  "version": "0.1.0",
+  "type": "module",
+  "peerDependencies": {
+    "@jackwener/opencli": ">=1.0.0"
+  }
+}
+```
+
+`my-command.ts`:
+
+```typescript
+import { cli, Strategy } from '@jackwener/opencli/registry';
+
+cli({
+  site: 'my-plugin',
+  name: 'my-command',
+  description: 'My custom command',
+  access: 'read', // 'read' | 'write'
+  example: 'opencli my-plugin my-command -f yaml',
+  strategy: Strategy.PUBLIC,
+  browser: false,
+  args: [
+    { name: 'limit', type: 'int', default: 10, help: 'Number of items' },
+  ],
+  columns: ['title', 'score'],
+  func: async (kwargs) => {
+    const res = await fetch('https://api.example.com/data');
+    const data = await res.json();
+    return data.items.slice(0, kwargs.limit).map((item: any, i: number) => ({
+      title: item.title,
+      score: item.score,
+    }));
+  },
+});
+```
+
+### TS Plugin Install Lifecycle
+
+When you run `opencli plugin install`, TS plugins are automatically set up:
+
+1. **Clone** — `git clone --depth 1` from GitHub
+2. **npm install** — Resolves regular dependencies
+3. **Host symlink** — Links the running `@jackwener/opencli` into the plugin's `node_modules/` so `import from '@jackwener/opencli/registry'` always resolves against the host
+4. **Transpile** — Compiles `.ts` → `.js` via `esbuild` (production `node` cannot load `.ts` directly)
+
+On startup, if both `my-command.ts` and `my-command.js` exist, the `.js` version is loaded to avoid duplicate registration.
+
+## Example Plugins
+
+| Repo | Type | Description |
+|------|------|-------------|
+| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | TS | GitHub Trending repositories |
+| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | TS | Multi-platform trending aggregator (zhihu, weibo, bilibili, v2ex, stackoverflow, reddit, linux-do) |
+| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | TS | 稀土掘金 (Juejin) hot articles, categories, and article feed |
+| [opencli-plugin-rubysec](https://github.com/nullptrKey/opencli-plugin-rubysec) | TS | RubySec advisory archive and advisory article reader |
+
+## Troubleshooting
+
+### Command not found after install
+
+Restart opencli (or open a new terminal) — plugins are discovered at startup.
+
+### TS plugin import errors
+
+If you see `Cannot find module '@jackwener/opencli/registry'`, the host symlink may be broken. Reinstall the plugin:
+
+```bash
+opencli plugin uninstall my-plugin
+opencli plugin install github:user/opencli-plugin-my-plugin
+```

package/docs/guide/remote-orchestration.md

@@ -0,0 +1,116 @@
+# Remote Orchestration
+
+Run an OpenCLI command from a remote machine (a CI runner, an agent server, a sandbox) while the **browser session stays on your local laptop**. The remote command sees `localhost:19825` like usual; behind the scenes, traffic is tunneled back to the daemon and Chrome on your machine.
+
+## When you need this
+
+- An autonomous agent (OpenClaw, a CI job, a server-side script) needs to drive a logged-in browser session, but only your local Chrome has the cookies.
+- Target sites do IP-based throttling or risk control — you want web traffic to leave from your home network, not from the agent's data center.
+- The remote machine has no display and no Chrome installed.
+
+## What not to do (and why)
+
+The first instinct is "let the extension connect to a remote daemon" — type a public host into the popup, expose port 19825 with frp, done. **Don't.** The daemon's WebSocket protocol has no built-in authentication. Anything that can reach the port can:
+
+- Read cookies for every site you're logged into
+- Execute arbitrary JavaScript in any of your tabs
+- Take screenshots, send arbitrary HTTP requests, dump page content
+
+Treat the daemon port the way you'd treat your unlocked desktop: never put it on a network you don't fully trust. Native "extension talks to remote daemon" support was [proposed in #636](https://github.com/jackwener/OpenCLI/pull/636) and deferred until daemon authentication exists.
+
+## The pattern: reverse-tunnel a localhost daemon
+
+Keep Chrome, the extension, and the daemon **all on your local machine**. Use a reverse port forward so the remote process can reach your daemon by connecting to its own `localhost:19825`. The daemon itself never leaves localhost.
+
+```
+┌─ Local ─────────────────────────────────┐    ┌─ Remote ──────────┐
+│  Chrome ↔ Extension ↔ Daemon (127.0.0.1) │ ←┐ │  opencli-cli      │
+└──────────────────────────────────────────┘  │ │  (talks to        │
+                                              │ │   localhost:19825)│
+                       reverse tunnel ────────┘ └───────────────────┘
+                       (SSH -R / frpc / VPN)
+```
+
+The remote `opencli` process needs **no flags, no env vars, no extension changes** — it connects to its own loopback, which the tunnel forwards to your laptop.
+
+## Option A — SSH reverse port forward (recommended)
+
+From your local machine, SSH to the remote with `-R` to expose your local daemon to the remote's loopback:
+
+```bash
+ssh -R 19825:127.0.0.1:19825 user@remote-server
+```
+
+While that session is open, anything on the remote connecting to `localhost:19825` is forwarded back to your local daemon. The remote-side workflow is unchanged:
+
+```bash
+# On the remote server
+opencli twitter feed
+opencli browser open https://example.com
+```
+
+::: tip
+Use `127.0.0.1` (not `localhost`) on the `-R` clause to avoid IPv6 resolution stalls.
+:::
+
+For long-lived agent runs, use `autossh` so the tunnel reconnects automatically:
+
+```bash
+autossh -M 0 -N -R 19825:127.0.0.1:19825 user@remote-server
+```
+
+Or as a systemd unit / launchd plist on the local side.
+
+### Why this is safe
+
+- The daemon stays bound to `127.0.0.1` on your machine.
+- The tunnel rides on SSH's authenticated transport — no new auth surface.
+- If the SSH session drops, the tunnel drops; the remote `opencli` simply fails to connect rather than reaching some stale endpoint.
+
+## Option B — frp reverse TCP proxy
+
+If SSH from your local machine to the remote isn't an option (NAT, firewalls), use [frp](https://github.com/fatedier/frp) to expose the local daemon through a public relay. **This is more complex than SSH and has more failure modes — prefer Option A unless you have a hard reason not to.**
+
+1. Run **frps** on a public relay you control.
+2. Run **frpc** on your local machine, exposing daemon port 19825:
+
+   ```toml
+   # ~/frpc.toml on your local machine
+   serverAddr = "<public-relay-ip>"
+   serverPort = 7000
+   auth.method = "token"
+   auth.token  = "<long-random-token>"
+
+   [[proxies]]
+   name = "opencli-daemon"
+   type = "tcp"
+   localIP = "127.0.0.1"
+   localPort = 19825
+   remotePort = 19825
+   ```
+
+3. On the remote server, run a second **frpc** that maps the relay's exposed port back to the remote's `localhost:19825` (a `stcp` visitor or a plain `tcp` client+visitor pair). This way `opencli` on the remote keeps talking to its own loopback.
+
+::: warning
+- Always set a strong `auth.token` on frps and frpc. Without it, anyone who learns the relay address has full control of your browser.
+- Bind the relay's exposed port to a private interface or restrict it with iptables / security groups. A daemon port on the public internet is the same risk as the rejected #636 design.
+- If you find yourself debugging frp auth, revisit Option A — it's cheaper and safer.
+:::
+
+## Verification
+
+After setting up the tunnel, confirm the remote sees the daemon:
+
+```bash
+# On the remote server
+curl -sf http://127.0.0.1:19825/ping && echo "daemon reachable"
+opencli doctor
+```
+
+`opencli doctor` from the remote should report the same extension version your local Chrome is running.
+
+## Caveats
+
+- **Local Chrome must be running** for the duration of the remote command. Closing Chrome closes the extension's WebSocket; remote `opencli` calls will fail with a "no daemon" error until Chrome is reopened.
+- **Tunnel latency adds to every call**. Each browser command crosses the tunnel twice; expect 50–200ms overhead per call on a typical SSH link, more on transcontinental links.
+- **One tunnel per local daemon**. If you start multiple SSH sessions all forwarding 19825, only the first wins; the rest log a "remote port already in use" warning.

package/docs/guide/troubleshooting.md

@@ -0,0 +1,69 @@
+# Troubleshooting
+
+## Common Issues
+
+### "Extension not connected"
+
+- Ensure the opencli Browser Bridge extension is installed and **enabled** in `chrome://extensions`.
+- Run `opencli doctor` to diagnose connectivity.
+
+### Empty data or 'Unauthorized' error
+
+- Your login session in Chrome might have expired. Open a normal Chrome tab, navigate to the target site, and log in or refresh the page.
+- Some sites have geographic restrictions (e.g., Bilibili, Zhihu from outside China).
+
+### Browser command opens the page but still cannot read context
+
+- A healthy Browser Bridge connection does not guarantee that the current page target exposes the data your adapter expects.
+- Some browser adapters are sensitive to the active host or page context.
+- Example: `opencli 1688 item` may fail with `did not expose product context` if the target is too broad.
+- Retry on a real item page, refresh the page in Chrome, and if needed narrow the target, for example:
+
+```bash
+OPENCLI_CDP_TARGET=detail.1688.com opencli 1688 item 841141931191 -f json
+```
+
+### Node API errors
+
+- Make sure you are using **Node.js >= 20**. Run `node --version` to verify.
+
+### Daemon issues
+
+```bash
+# View extension logs
+curl localhost:19825/logs
+
+# Stop the daemon
+opencli daemon stop
+
+# Full diagnostics
+opencli doctor
+```
+
+> The daemon is persistent and stays alive until explicitly stopped (`opencli daemon stop`) or the package is uninstalled.
+
+> When the CLI detects a stale daemon (version mismatch after `npm install -g @jackwener/opencli@latest`), it first asks the daemon to shut down via `/shutdown`, then falls back to `SIGKILL` if the daemon does not release the port within 3 seconds. Manual `opencli daemon stop` is only needed if SIGKILL itself is rejected (cross-user owner / cross-machine PID file).
+
+### Desktop adapter connection issues
+
+For Electron/CDP-based adapters (Cursor, Codex, etc.):
+
+1. Make sure the app is launched with `--remote-debugging-port=XXXX`
+2. Verify the endpoint is set: `echo $OPENCLI_CDP_ENDPOINT`
+3. Test the endpoint: `curl http://127.0.0.1:XXXX/json/version`
+
+### Build errors
+
+```bash
+# Clean rebuild
+rm -rf dist/
+npm run build
+
+# Type check
+npx tsc --noEmit
+```
+
+## Getting Help
+
+- [GitHub Issues](https://github.com/jackwener/opencli/issues) — Bug reports and feature requests
+- Run `opencli doctor` for comprehensive diagnostics

package/package.json

@@ -0,0 +1,116 @@
+{
+  "name": "publishport-opencli",
+  "version": "1.0.0",
+  "publishConfig": {
+    "access": "public",
+    "provenance": false
+  },
+  "description": "Make any website or Electron App your CLI. AI-powered.",
+  "engines": {
+    "node": ">=20.0.0"
+  },
+  "type": "module",
+  "main": "dist/src/main.js",
+  "bin": {
+    "opencli": "dist/src/main.js"
+  },
+  "exports": {
+    ".": "./dist/src/main.js",
+    "./registry": "./dist/src/registry-api.js",
+    "./errors": "./dist/src/errors.js",
+    "./types": "./dist/src/types.js",
+    "./utils": "./dist/src/utils.js",
+    "./logger": "./dist/src/logger.js",
+    "./launcher": "./dist/src/launcher.js",
+    "./browser/cdp": "./dist/src/browser/cdp.js",
+    "./browser/page": "./dist/src/browser/page.js",
+    "./browser/utils": "./dist/src/browser/utils.js",
+    "./download": "./dist/src/download/index.js",
+    "./download/article-download": "./dist/src/download/article-download.js",
+    "./download/media-download": "./dist/src/download/media-download.js",
+    "./download/progress": "./dist/src/download/progress.js",
+    "./pipeline": "./dist/src/pipeline/index.js"
+  },
+  "files": [
+    "dist/src/",
+    "clis/",
+    "docs/guide/",
+    "skills/opencli-*/**",
+    "cli-manifest.json",
+    "scripts/",
+    "README.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "dev": "tsx src/main.ts",
+    "dev:bun": "bun src/main.ts",
+    "release": "npm version patch -m \"chore(release): %s\" && git push origin HEAD && git push origin \"$(git describe --tags --abbrev=0)\"",
+    "build": "npm run clean-dist && npm run copy-yaml && npm run build-manifest",
+    "minify:release": "bun scripts/minify-release.mjs",
+    "prebuild-manifest": "tsc --build && node -e \"require('fs').chmodSync('dist/src/main.js', 0o755)\"",
+    "build-manifest": "tsx src/build-manifest.ts",
+    "clean-dist": "node scripts/clean-dist.cjs",
+    "copy-yaml": "node scripts/copy-yaml.cjs",
+    "start": "node dist/src/main.js",
+    "start:bun": "bun dist/src/main.js",
+    "preuninstall": "node -e \"fetch('http://127.0.0.1:19825/shutdown',{method:'POST',headers:{'X-OpenCLI':'1'},signal:AbortSignal.timeout(3000)}).catch(()=>{})\" || true",
+    "postinstall": "node scripts/postinstall.js || true; node scripts/fetch-adapters.js || true",
+    "typecheck": "tsc --noEmit",
+    "prepare": "[ -d src ] && npm run build || true",
+    "prepublishOnly": "npm run build",
+    "test": "vitest run --project unit --project extension --project adapter",
+    "test:bun": "bun vitest run --project unit --project extension --project adapter",
+    "test:adapter": "vitest run --project adapter",
+    "test:all": "vitest run",
+    "test:e2e": "vitest run --project e2e-fixed-port --project e2e",
+    "advise:listing-id-pairing": "node scripts/check-listing-id-pairing.mjs",
+    "check:silent-column-drop": "node scripts/check-silent-column-drop.mjs",
+    "check:typed-error-lint": "node scripts/check-typed-error-lint.mjs",
+    "docs:dev": "vitepress dev docs",
+    "docs:build": "vitepress build docs",
+    "docs:preview": "vitepress preview docs"
+  },
+  "keywords": [
+    "cli",
+    "browser",
+    "web",
+    "ai"
+  ],
+  "author": "jackwener",
+  "license": "SEE LICENSE IN LICENSE",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/wearzdk/OpenCLI.git"
+  },
+  "dependencies": {
+    "@mozilla/readability": "^0.6.0",
+    "cli-table3": "^0.6.5",
+    "commander": "^14.0.3",
+    "js-yaml": "^4.1.0",
+    "markdown-draft-js": "^2.4.0",
+    "markdown-it": "^14.2.0",
+    "remarkable": "^2.0.1",
+    "turndown": "^7.2.2",
+    "turndown-plugin-gfm": "^1.0.2",
+    "undici": "^6.27.0",
+    "ws": "^8.18.0"
+  },
+  "devDependencies": {
+    "@types/jsdom": "^27.0.0",
+    "@types/js-yaml": "^4.0.9",
+    "@types/node": "^25.5.2",
+    "@types/turndown": "^5.0.6",
+    "@types/ws": "^8.5.13",
+    "jsdom": "^29.0.2",
+    "tsx": "^4.19.3",
+    "typescript": "^6.0.2",
+    "vitepress": "^1.6.4",
+    "vitest": "^4.1.0"
+  },
+  "overrides": {
+    "postcss": "^8.5.10",
+    "vitepress": {
+      "vite": "6.4.2"
+    }
+  }
+}

package/scripts/check-doc-coverage.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+# check-doc-coverage.sh — Verify every adapter in clis/ has a doc page.
+#
+# Exit codes:
+#   0 — all adapters have docs
+#   1 — at least one adapter is missing documentation
+#
+# Usage:
+#   bash scripts/check-doc-coverage.sh          # report only
+#   bash scripts/check-doc-coverage.sh --strict  # exit 1 on missing docs
+
+set -euo pipefail
+
+STRICT=false
+if [[ "${1:-}" == "--strict" ]]; then
+  STRICT=true
+fi
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
+
+SRC_DIR="$ROOT_DIR/clis"
+DOCS_DIR="$ROOT_DIR/docs/adapters"
+
+missing=()
+covered=0
+total=0
+
+for adapter_dir in "$SRC_DIR"/*/; do
+  adapter_name="$(basename "$adapter_dir")"
+  # Skip internal directories (e.g., _shared)
+  [[ "$adapter_name" == _* ]] && continue
+  # Skip directories that only contain utility files (prefixed with _)
+  has_commands=false
+  for f in "$adapter_dir"*; do
+    fname="$(basename "$f")"
+    [[ "$fname" == _* ]] && continue
+    [[ "$fname" == *.test.* ]] && continue
+    has_commands=true
+    break
+  done
+  [[ "$has_commands" == false ]] && continue
+  total=$((total + 1))
+
+  # Check if doc exists in browser/ or desktop/ subdirectories
+  if [[ -f "$DOCS_DIR/browser/$adapter_name.md" ]] || \
+     [[ -f "$DOCS_DIR/desktop/$adapter_name.md" ]]; then
+    covered=$((covered + 1))
+  else
+    # Handle directory name mismatches (e.g., discord-app -> discord)
+    alt_name="${adapter_name%-app}"
+    if [[ "$alt_name" != "$adapter_name" ]] && \
+       { [[ -f "$DOCS_DIR/browser/$alt_name.md" ]] || \
+         [[ -f "$DOCS_DIR/desktop/$alt_name.md" ]]; }; then
+      covered=$((covered + 1))
+    else
+      missing+=("$adapter_name")
+    fi
+  fi
+done
+
+echo "📊 Doc Coverage: $covered/$total adapters documented"
+echo ""
+
+if [[ ${#missing[@]} -gt 0 ]]; then
+  echo "⚠️  Missing docs for ${#missing[@]} adapter(s):"
+  for name in "${missing[@]}"; do
+    echo "   - $name  →  create docs/adapters/browser/$name.md or docs/adapters/desktop/$name.md"
+  done
+  echo ""
+  if $STRICT; then
+    echo "❌ Doc check failed (--strict mode)."
+    exit 1
+  else
+    echo "💡 Run with --strict to fail CI on missing docs."
+    exit 0
+  fi
+else
+  echo "✅ All adapters have documentation."
+  exit 0
+fi