grokbot-control 0.0.0-stage → 0.1.0

package/README.md

@@ -1,3 +1,161 @@
-# Temporary Holding Version
+# grokbot-control
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**人通过 Grok Bot App 驱动 → agent CLI 经 MCP/CLI 驱动同一 Bot。**  
+可编程控制面（原生 Connect RPC），非自建任务系统。
+
+CLI 暂称 **`gbc`**；MCP stdio 入口 **`gbc-mcp`**。
+
+> Live auth / production RPC **未在本轮验证**。默认 `--mock` 仅用于显式仿真（结果带 `simulated: true`），不是在线成功。
+
+## 快速开始
+
+```bash
+npm ci
+npm run build
+npm test
+
+# 显式 mock 仿真（跨进程状态：GBT_MOCK_STATE 或 ~/.gbc/mock-state.json）
+node dist/cli/main.js --mock --json --allow-writes bot create --name demo --description "test"
+node dist/cli/main.js --mock --json bot list
+node dist/cli/main.js --mock --json --allow-writes message send \
+  --agent-id <id> --session-id <sid> --text "hello"
+node dist/cli/main.js --mock --json bot status --agent-id <id>
+node dist/cli/main.js --mock --json --allow-writes sidebar create --name Work
+node dist/cli/main.js --mock --json --allow-writes sidebar move --agent-id <id> --section-id <sid>
+node dist/cli/main.js --mock --json --allow-writes bot delete --agent-id <id> --confirm <id>
+
+# MCP（stdio；stdout 仅协议帧）
+GBT_MOCK=1 node dist/mcp/main.js
+```
+
+Live Connect（真实登录需用户本人在浏览器授权；本仓库测试只用本地 fake 后端）：
+
+```bash
+# 不要设置 GBT_MOCK / GBT_ALLOW_WRITES；默认 backend https://api2.cursor.sh，网站 https://cursor.com
+node dist/cli/main.js --json auth login-begin        # stdout: authorization_url（不含 verifier）
+#   → 用户在浏览器打开该 URL 并登录
+node dist/cli/main.js --json auth complete --timeout 300   # 轮询 GET /auth/poll；成功后凭据写入 ~/.gbc（0600）
+# 或单命令：auth login --timeout 300（URL 走 stderr 的 open_browser 事件，stdout 只放结果）
+node dist/cli/main.js --json auth whoami             # 本地状态，不联网、不打印 token
+node dist/cli/main.js --json auth refresh            # 临期才联网；足够新则 refreshed:false
+node dist/cli/main.js --json auth refresh --force    # 强制刷新（验证/401恢复）
+node dist/cli/main.js --json bot list --limit 20     # 只读
+node dist/cli/main.js --json sidebar list
+node dist/cli/main.js --json bot status --agent-id <agent_id>
+node dist/cli/main.js --json auth logout             # 只删本工具自己的本地凭据
+```
+
+鉴权限制（详见 docs/protocol-map.md「Auth」）：
+- 协议来自 Grok Bot Linux App **静态逆向**，非官方公开 API。`vendor/proto-gen` 现为 **0.79.3**（本仓库从App恢复生成，离线校验）。较早人工登录验证使用旧0.78.4 vendor；0.79.3 的选定正常RPC已进行隔离临时bot验收，范围/skip见 implementation-status。不能由此推定所有RPC或OAuth刷新已验证。
+- 仅支持 Cursor token；xAI identity 登录结果返回 `unsupported`。
+- 刷新需显式配置公开 OAuth client_id：`GBT_OAUTH_CLIENT_ID`（本工具不内置，也不从 JWT 猜测），或登录时已持久化的绑定；旧凭据可用 `GBT_AUTH_CONFIRM_MIGRATE=1` 确认迁移。临期（<5min）/缺非法 exp / `--force` 才刷新；进程内 singleflight + 跨进程锁；`shouldLogout` 不回退已撤销 token。
+- 凭据独立于 App（不读取、不复用 App 的 sand-secrets.json）。logout 只删本地文件，不吊销服务端会话。
+- MCP：`GBT_MCP_READONLY=1 node dist/mcp/main.js` 仅提供 `auth_whoami` + 只读业务工具（不含 login/logout/cancel/refresh 凭据写工具）。非 READONLY 另有 `auth_refresh`（仅非 secret 元数据）。
+
+## Publish / npx
+
+本包可发布到 [npm](https://www.npmjs.com/)（包名 `grokbot-control`）。**未声明 SPDX `license`**：`vendor/proto-gen` 来自逆向恢复的 schema，许可不清，见该目录 `NOTICE.md`；请自行评估分发风险。
+
+### 一次性准备（首发）
+
+1. 在 [npmjs.com](https://www.npmjs.com/) 创建账号并开启 2FA。
+2. 首个版本手动发布一次（包不存在时无法配置 Trusted Publisher）：
+
+   ```bash
+   npm login --auth-type=web --registry=https://registry.npmjs.org
+   npm ci && npm test
+   npm publish --access public --registry=https://registry.npmjs.org
+   ```
+
+3. 在 npmjs.com → `grokbot-control` → **Settings → Trusted Publisher** 选 GitHub Actions，填写：
+   - Organization or user：`younglark`
+   - Repository：`grokbot-control`
+   - Workflow filename：`publish.yml`
+
+   新建配置需在 2 天内完成第一次成功发布，否则过期。配置后可在 **Publishing access** 选择禁止 token 发布。
+
+### 后续发版（GitHub Actions + OIDC）
+
+`.github/workflows/publish.yml` 仅在推送 tag `v*` 时运行（Node 24、npm ≥ 11.5.1、`id-token: write`）：`npm ci` → `npm test` → `npm publish --access public`。无需 npm token 或仓库 secret。
+
+```bash
+# 工作区干净、已通过本地 build/test
+npm version patch   # 或 minor / major；会改 package.json 并打 vX.Y.Z tag
+git push origin main --follow-tags
+```
+
+GitHub Release 不会触发发布，可在 tag 推送后基于该 tag 创建 Release，不会重复 publish。首版 0.1.0 已手动发布，不要再推 `v0.1.0` tag（会因版本已存在而失败）。
+
+### 安装 / 运行
+
+```bash
+npx --yes --package=grokbot-control@latest gbc --help
+npx --yes --package=grokbot-control@latest gbc --mock --json bot list
+npx --yes --package=grokbot-control@latest gbc-mcp
+```
+
+MCP 建议钉版本，例如：
+
+```json
+{
+  "command": "npx",
+  "args": ["--yes", "--package=grokbot-control@0.1.0", "gbc-mcp"]
+}
+```
+
+OAuth 刷新仍需自行配置公开的 `GBT_OAUTH_CLIENT_ID`（本工具不内置 client_id）。
+
+## 文件优先同步（无 SQLite）
+
+CLI：`gbc sync pull --agent-id BOT [--session-id SESSION] --dir /absolute/private/root
+--account-id STABLE_ID --team-id none --ack-unverified-scope`；`gbc sync status|render|attachments --dir /absolute/private/root`。
+pull/render/attachments 明确授权本地写，不需要远端 `--allow-writes`；`--dry-run` 纯本地计划、无 RPC/文件。
+身份是操作者显式声明而非服务端验证，必须核对稳定 ID，不能用邮箱/token 哈希。
+**render 已可直接 rg 已确认 JSON UI entry 的正文**：message、send-message(text/attachment)、notice、
+tool-call 名称/状态/summary、附件文件引用；其余类型/未知 body 仍 opaque，不猜解密。
+正文补读显式开启 `sync pull ... --hydrate missing`（默认 `off`，不默默增加网络请求）。inline 优先；
+omitted 精确重读同版本；blob 仅通过 store presign 读取正文，不是用户附件。原始 wire/protojson 不覆盖，
+补读 bytes/本地 SHA-256/来源/版本上下文另存，JSONL 与离线 render 均可 grep。请求失败/无法解码返回 `partial`（exit 9），已归档行保留。
+默认额外 RPC 32、正文传输/响应总 8MiB、单正文 4MiB，与 pull 总字节/截止时间共享；参数见格式文档。
+静态解码证据和 synthetic genuine-schema fixture 见 [transcript-body-0.79.3.md](docs/protocol/transcript-body-0.79.3.md)。
+例如 `rg '关键词' /absolute/private/root/derived -g '*.md'`；旧 opaque 归档可离线重新 render。
+head刷新与持久化backfill交替，不删缺失行，不承诺完整性；附件单独按需下载；无Watch/`--all`。
+MCP 设 `GBT_SYNC_ROOT` 才提供 `sync_status`，另需 `GBT_SYNC_ALLOW_LOCAL_WRITES=1` 才有
+`sync_pull/sync_render/sync_attachments`（READONLY 排除写工具；scope 在启动配置；attachments 可选远端 file_path 过滤）。
+命令示例、scope 限制、格式与恢复步骤见 [docs/sync-format.md](docs/sync-format.md)。
+
+## 文档
+
+| 文件 | 内容 |
+|---|---|
+| [docs/requirements.md](docs/requirements.md) | 需求 |
+| [docs/protocol-map.md](docs/protocol-map.md) | 覆盖矩阵 / RPC |
+| [docs/protocol/](docs/protocol/README.md) | 协议基线、静态还原方法、App 更新 runbook、兼容矩阵、`tools/protocol` 脚本 |
+| [docs/architecture.md](docs/architecture.md) | 模块 / 错误 / 写安全 |
+| [docs/implementation-plan.md](docs/implementation-plan.md) | 分期 |
+| [docs/acceptance.md](docs/acceptance.md) | 验收 |
+| [docs/review.md](docs/review.md) | 审查 |
+| [docs/references.md](docs/references.md) | 一手来源 |
+| [docs/sync-format.md](docs/sync-format.md) | 文件同步格式、身份限制、预算、崩溃恢复 |
+| [docs/implementation-status.md](docs/implementation-status.md) | 实现状态 |
+| [/workspace/grokbot-control-implementation.md](../grokbot-control-implementation.md) | 本轮实现报告 |
+
+## 退出码
+
+`0` success · `2` usage · `4` auth · `5` not_found · `6` conflict · `7` denied · `8` retryable · `9` failure · `10` unknown · `1` other
+
+## 协议 vendor
+
+正式协议源见 `proto/`；生成客户端见 `vendor/proto-gen/NOTICE.md`（App `0.79.3` commit `4739412`，本仓库从App恢复生成；生成器 Buf Connect Apache-2.0；schema 本身无单独官方许可声明）。
+
+### 同步修订
+支持默认session、generation内持久化backfill与跨调用session轮转；归档是retained observations，不是删除同步/完整镜像。
+附件仅按需App store aliases，用户上传和send-message attachment均支持；每次Resolve+条件GET，流式预算与scope绑定。
+attachments需与pull一致的`--account-id --team-id --ack-unverified-scope`。
+`--page-size 1 --max-pages 1`可做小量分页验收。详细边界见[格式文档](docs/sync-format.md)。
+
+### Native auth lock installation
+跨进程 refresh 使用 `fs-native-extensions@1.5.3` 原生 OS advisory lock（非外部 flock 命令、非 mtime lease）。
+Node20+，Linux/macOS/Windows x64/arm64 npm包预编译；缺binding fail closed。升级前停止全部旧MCP进程，旧Linux flock与新OFD锁不互通。
+Linux x64已实测；macOS/Windows仅源码检查+CI矩阵，未在本机执行。Windows需另核查用户目录ACL；只支持本地文件系统。
+详见 [native lock构建/平台/证据](docs/auth-lock.md)。

package/dist/auth/lock.d.ts

@@ -0,0 +1,14 @@
+/** Backend-neutral contract. Only nonblocking native attempts run on the event loop. */
+export interface AdvisoryLockBinding {
+    tryLock(fd: number): boolean;
+    unlock(fd: number): void;
+}
+export type LockWaitOptions = {
+    waitMs?: number;
+    signal?: AbortSignal;
+};
+/** Permanent independent file. Never truncate, replace or unlink; OS releases on process death.
+ * Cancellation is honored only BEFORE exchange starts, never while fn owns the lock.
+ * Only local filesystems supported; all credential writers must cooperate.
+ */
+export declare function withAdvisoryLock<T>(file: string, fn: () => Promise<T>, opts?: LockWaitOptions): Promise<T>;

package/dist/auth/lock.js

@@ -0,0 +1,71 @@
+import fs from 'node:fs';
+import { createRequire } from 'node:module';
+import { setTimeout as delay } from 'node:timers/promises';
+import { GbcError } from '../errors.js';
+let binding;
+function nativeBinding() {
+    if (binding)
+        return binding;
+    try {
+        if (!['linux', 'darwin', 'win32'].includes(process.platform))
+            throw new Error('unsupported platform');
+        const loaded = createRequire(import.meta.url)('fs-native-extensions');
+        if (typeof loaded.tryLock !== 'function' || typeof loaded.unlock !== 'function')
+            throw new Error('invalid binding');
+        binding = loaded;
+        return loaded;
+    }
+    catch {
+        throw new GbcError('auth_lock_unavailable', 'Native advisory lock unavailable; run npm ci with fs-native-extensions@1.5.3 prebuilds for your OS/architecture. See docs/auth-lock.md for source build instructions. No lease fallback.', { exitCode: 4 });
+    }
+}
+/** Permanent independent file. Never truncate, replace or unlink; OS releases on process death.
+ * Cancellation is honored only BEFORE exchange starts, never while fn owns the lock.
+ * Only local filesystems supported; all credential writers must cooperate.
+ */
+export async function withAdvisoryLock(file, fn, opts = {}) {
+    const native = nativeBinding();
+    const waitMs = opts.waitMs ?? 30000;
+    if (!Number.isFinite(waitMs) || waitMs < 0)
+        throw new GbcError('invalid_argument', 'invalid lock wait deadline');
+    const fd = fs.openSync(file, fs.constants.O_CREAT | fs.constants.O_RDWR | (fs.constants.O_NOFOLLOW ?? 0), 0o600);
+    let held = false;
+    try {
+        const original = fs.fstatSync(fd);
+        if (fs.lstatSync(file).isSymbolicLink() || !original.isFile() || original.nlink !== 1)
+            throw new GbcError('auth_lock_path', 'lock must be an independent regular file');
+        if (process.platform !== 'win32')
+            fs.fchmodSync(fd, 0o600);
+        const deadline = performance.now() + waitMs;
+        for (;;) {
+            if (opts.signal?.aborted)
+                throw new GbcError('auth_lock_canceled', 'lock wait canceled; no exchange started', { exitCode: 8, retryable: true, details: { rpc_dispatched: false } });
+            try {
+                held = native.tryLock(fd);
+            }
+            catch (e) {
+                // libuv Windows ERROR_LOCK_VIOLATION may surface as EBUSY; other failures fail closed.
+                if (!['EAGAIN', 'EBUSY'].includes(e.code ?? ''))
+                    throw new GbcError('auth_lock_unavailable', 'native lock failed; credentials kept', { exitCode: 4 });
+            }
+            if (held) {
+                const current = fs.lstatSync(file);
+                if (current.isSymbolicLink() || original.ino !== current.ino || original.dev !== current.dev)
+                    throw new GbcError('auth_lock_path', 'lock path replaced while waiting; refusing exchange');
+                return await fn();
+            }
+            if (performance.now() >= deadline)
+                throw new GbcError('auth_refresh_lock_timeout', 'lock wait deadline exceeded; owner left intact', { exitCode: 8, retryable: true, details: { rpc_dispatched: false } });
+            await delay(Math.min(25, Math.max(1, deadline - performance.now())));
+        }
+    }
+    finally {
+        try {
+            if (held)
+                native.unlock(fd);
+        }
+        finally {
+            fs.closeSync(fd);
+        }
+    }
+}

package/dist/auth/provider.d.ts

@@ -0,0 +1,89 @@
+import { FileCredentialStore } from "./store.js";
+import type { AuthKind, AuthProvider } from "./types.js";
+/**
+ * Independent Cursor browser login (product-custom challenge + poll; NOT RFC 8628).
+ *
+ * Protocol source: STATIC reverse engineering of Grok Bot Linux 0.79.2 / 0.79.3
+ * (/workspace/grokbot-linux-auth-analysis.md; evidence/main-app.pretty.cjs = A,
+ * main-core.pretty.cjs = C). Not official public API documentation.
+ *  - A:38644-38657 Rdn: verifier = base64url(32 random bytes); challenge =
+ *    base64url(SHA256(verifier string)); uuid = randomUUID();
+ *    `${website}/loginDeepControl?challenge&uuid&mode=login&redirectTarget[&supportsSelectedTeamLogin=true]`
+ *  - A:38720-38816 Aze/Odn: GET `${api}/auth/poll?uuid=..&verifier=..`; 404 = pending;
+ *    max 150 tries, 1s * 1.2^i capped 10s; 3 consecutive non-404 failures stop;
+ *    403 {error:"sign_in_policy_violation"} = policy denial.
+ *  - A:38584-38593 Cdn: success body {accessToken, refreshToken}; A:38605-38622 mze:
+ *    xAI identity body {xaiAccessToken,...} -> UNSUPPORTED here (different refresh path).
+ *  - A:56108-56125: refresh POST new URL("/oauth/token", backend) JSON
+ *    {client_id, grant_type:"refresh_token", refresh_token}; A:54459 response
+ *    {access_token?, refresh_token?, shouldLogout?, error?}; refresh_token defaults to old.
+ *  - C:35811 KLe: refresh 300s before JWT exp.
+ * client_id: the App selects a public client identifier per backend (C:35866-35869).
+ * This tool does NOT embed it; refresh requires explicit GBT_OAUTH_CLIENT_ID config
+ * or a client_id persisted at login / confirmed migration.
+ */
+export declare const DEFAULT_BACKENDS: string[];
+export declare const DEFAULT_WEBSITE = "https://cursor.com";
+export declare const POLICY_VIOLATION = "sign_in_policy_violation";
+export declare const REFRESH_EARLY_MS = 300000;
+export declare const REFRESH_DEADLINE_MS = 20000;
+export declare const PENDING_TTL_MS: number;
+export type AuthHttp = {
+    fetch: typeof fetch;
+};
+export type AuthProviderOptions = {
+    kind: AuthKind;
+    backend: string;
+    /** Browser login site. Default https://cursor.com; custom requires trust. */
+    website?: string;
+    http?: AuthHttp;
+    store?: FileCredentialStore;
+    trustCustomBackend?: boolean;
+    /** Public OAuth client_id for refresh. Default: GBT_OAUTH_CLIENT_ID (no built-in value). */
+    clientId?: string;
+    /** redirectTarget login param. Default GBT_LOGIN_REDIRECT_TARGET || "cli" (library default, A:38644 / A:38845; App uses "sand", A:49834). */
+    redirectTarget?: string;
+    /** Test hooks. */
+    now?: () => number;
+    sleep?: (ms: number, signal?: AbortSignal) => Promise<void>;
+    /** Override refresh HTTP deadline (default 20s). */
+    refreshDeadlineMs?: number;
+};
+export declare function generateLoginChallenge(): {
+    verifier: string;
+    challenge: string;
+    uuid: string;
+};
+export declare function buildLoginUrl(website: string, p: {
+    challenge: string;
+    uuid: string;
+    redirectTarget: string;
+}): string;
+/** JWT payload (unverified decode — only for exp/email display; never trust for auth / client_id). */
+export declare function decodeJwtPayload(token: string): {
+    exp?: number;
+    email?: string;
+    sub?: string;
+    iss?: string;
+} | null;
+export declare function refreshEndpointFor(backend: string): string;
+export type ExpStatus = {
+    ok: true;
+    expires_at_ms: number;
+} | {
+    ok: false;
+    reason: "missing_exp" | "illegal_exp";
+};
+/** Classify stored / JWT exp. Never invents an expiry. */
+export declare function classifyExpiry(expires_at_ms: number | undefined, nowMs: number): {
+    status: ExpStatus;
+    needsRefresh: boolean;
+    stillValid: boolean;
+};
+export declare function createAuthProvider(opts: AuthProviderOptions): AuthProvider;
+/** Injected token provider for tests / mock — never logs the token; never disk-refreshes. */
+export declare function staticTokenProvider(opts: {
+    kind: AuthKind;
+    backend: string;
+    accessToken: string;
+}): AuthProvider;