expandai 0.0.6 → 0.1.0

package/README.md

@@ -20,10 +20,41 @@ npx -y expandai fetch https://example.com
 bunx expandai fetch https://example.com
 ```
 
+### Staging channel
+
+Relevant merges to the Expand monorepo publish an immutable CLI build and move
+the npm `staging` tag. Internal testing can use it without changing `latest`:
+
+```bash
+EXPAND_API_URL="$EXPAND_STAGING_API_URL" \
+EXPAND_API_KEY="$EXPAND_STAGING_API_KEY" \
+npx -y expandai@staging fetch https://example.com
+```
+
+The environment variables are explicit because installing from the `staging`
+tag does not silently change the CLI's default production API.
+
+### Review builds
+
+Deploying a pull request to the Review environment
+(`.github/workflows/deploy-review.yaml`) also packs the CLI, the Node SDK, and
+the Python SDK from that same revision into a `review-packages` workflow
+artifact, and comments the install commands on the PR. Nothing is published to
+npm or PyPI; the versions carry a `review` channel and the source commit so a
+Review build never masquerades as a release:
+
+```bash
+gh run download <run-id> --repo expandai/expand --name review-packages --dir review-packages
+npm install --global ./review-packages/expandai-<version>.tgz
+EXPAND_API_URL=https://api.review.expand.ai \
+EXPAND_APP_URL=https://review.expand.ai \
+expandai fetch https://example.com
+```
+
 ## Standalone binary
 
-A self-contained `expandai` executable is published with each release — no
-Node, Bun, or npm required at runtime. One command detects your platform,
+Each release ships a self-contained `expandai` executable that needs no
+Node, Bun, or npm at runtime. One command detects your platform,
 downloads the right binary from GitHub Releases, and drops it on your
 `PATH`:
 
@@ -39,21 +70,21 @@ expandai whoami
 
 The binary bundles everything `expandai mcp run`, `expandai hook claude-code`,
 and `expandai hook opencode` need at runtime, so the MCP server and webfetch
-hooks work straight from `PATH` — no `node`/`bun` needed in your agent's
-environment.
+hooks work straight from `PATH`. Your agent's environment does not need
+`node` or `bun`.
 
 ## Quick start
 
 ```bash
 expandai init                                             # authorize and configure detected agents
 expandai fetch https://docs.example.com/api               # full markdown
-expandai fetch https://docs.example.com/api --search auth # semantic snippets
+expandai fetch https://docs.example.com/api --highlights auth # semantic snippets
 ```
 
-Human terminal use may omit the harness and scope; init detects supported
-harnesses, asks for scope, completes device browser authorization when
-needed, and configures every detected harness. Pass `--harness` to ensure the
-agent running the command is included even when it has no config directory yet.
+Run by hand, `init` needs no arguments. It detects supported harnesses, asks
+for scope, completes device browser authorization when needed, and configures
+every harness it finds. Pass `--harness` so the agent running the command is
+included even when it has no config directory yet.
 
 Agents and non-interactive VMs use the explicit machine form:
 
@@ -63,8 +94,8 @@ expandai init --harness <current-harness> --scope project --json
 
 If authorization is required, the JSON result contains a browser URL and a
 `userCode`. Put both in front of the user: they open the page, type the code in,
-and approve — nothing comes back to paste. After they approve, run the **same
-command again**; the CLI polls the approval up itself (RFC 8628). Machine mode
+and approve. Nothing comes back to paste. After they approve, run the **same
+command again**; the CLI polls for the approval itself (RFC 8628). Machine mode
 writes exactly one JSON result to stdout or one structured error to stderr, so
 agents never need to answer an interactive CLI prompt.
 
@@ -82,44 +113,45 @@ harness plus every other supported harness detected on the machine. The
 component commands below remain expert tools for migrations and targeted repair:
 
 1. **Hook** (Claude Code, OpenCode). `expandai hook install <host> <scope>`
-   — best invisible upgrade: every native web fetch the agent issues comes
-   back as ExpandAI markdown, no prompt changes required. Also wires the MCP
-   server in the same step. See [Recommended: `hook install`](#recommended-hook-install).
+   upgrades the agent invisibly: every native web fetch it issues comes
+   back as ExpandAI markdown, no prompt changes required. The same step also
+   wires the MCP server. See [Recommended: `hook install`](#recommended-hook-install).
 2. **Skill** (all four harnesses). `expandai skill install <host> <scope>`
-   — lightest-weight: drops a `SKILL.md` that teaches the agent when to
-   reach for `expandai fetch`.
+   is the lightest option. It drops a `SKILL.md` that teaches the agent when
+   to reach for `expandai fetch`.
 3. **MCP server** (all four harnesses). `expandai mcp install <host> <scope>`
-   — exposes the full `fetch` API (snippets, screenshots, structured
+   exposes the full `fetch` API (snippets, screenshots, structured
    extraction) as an MCP-callable tool.
 
-Combinations are fine — installing skill on top of hook, for example, nudges
-the agent to use `mcp__expandai__fetch` for refinement on top of the
-automatic substitution — but any single one is a complete setup.
+Combinations are fine, and any single one is a complete setup. Installing the
+skill on top of the hook, for example, nudges the agent to use
+`mcp__expandai__fetch` for refinement on top of the automatic substitution.
 
 ### Scopes
 
 Every `install` / `uninstall` command takes a `<scope>` argument: `global`
 or `project`.
 
-- `global` — written under your `$HOME`. Applies to every session of the
+- `global` writes under your `$HOME` and applies to every session of the
   harness, regardless of where the agent is launched. Pick this if you want
   ExpandAI on by default everywhere.
-- `project` — written under the current working directory. Applies only
+- `project` writes under the current working directory and applies only
   when the harness is launched from inside that directory (or a
-  subdirectory). Pick this when you want different projects to opt in
+  subdirectory). Pick this when different projects should opt in
   independently, or when the integration should live in version control
   alongside the project. The installed Claude profile uses the stable
   `expandai` command when it is on `PATH` and `npx -y expandai` otherwise;
   restart the harness if it has not inherited the global npm `PATH` yet.
 
-The exact files each scope touches differ per harness and per integration —
-see the path comment next to each `install` line in the sections below.
+The exact files each scope touches differ per harness and per integration.
+See the path comment next to each `install` line in the sections below.
 
 ## Authentication
 
 ```bash
 expandai login    # OAuth device authorization or API key
-expandai whoami   # show current auth state
+expandai whoami   # show account, organization, and authentication
+expandai whoami --format json   # machine-readable identity
 expandai logout   # clear credentials and optionally remove installed integrations
 ```
 
@@ -128,8 +160,30 @@ Credentials are stored at `$XDG_CONFIG_HOME/expandai/config.json` (default
 
 Environment overrides:
 
-- `EXPAND_API_KEY` — overrides the stored auth method.
-- `EXPAND_API_URL` — overrides the API endpoint (defaults to `https://api.expand.ai`).
+- `EXPAND_API_KEY` overrides the stored auth method.
+- `EXPAND_API_URL` overrides the API endpoint (defaults to `https://api.expand.ai`).
+
+
+### Check which account you are using
+
+Run `expandai whoami` to see your name, verified email when available, organization,
+and authentication method. API keys identify the organization. The command never
+prints access tokens, refresh tokens, or API keys.
+
+Use `expandai whoami --format json` in scripts. A successful lookup returns
+`authenticated: true`, optional `user`, `organization`, and `authentication`.
+OAuth authentication includes `expiresAt` after any token refresh.
+
+Without usable credentials, JSON returns `authenticated: false` and `mode: "preview"`,
+or `mode: "unauthenticated"` when `EXPAND_NO_ANONYMOUS` disables preview access.
+Rejected credentials return `mode: "credential-rejected"`; sign in again with `expandai login`.
+If account lookup fails, `mode: "identity-unavailable"` preserves local authentication
+details without claiming the server verified them. Text output retains the local
+status and adds a short identity-unavailable note.
+
+Deploy the API's `GET /v1/account` route before releasing this CLI version. Older
+servers use the same identity-unavailable fallback.
+
 
 ## fetch
 
@@ -140,15 +194,15 @@ JavaScript-heavy SPAs and bot-protected sites that plain `curl` can't read.
 expandai fetch https://example.com                       # markdown (default)
 expandai fetch example.com                               # https:// is added automatically
 expandai fetch https://example.com --format json         # full response with metadata
-expandai fetch https://docs.example.com/api --search "authentication"
+expandai fetch https://docs.example.com/api --highlights "authentication"
 ```
 
 Flags:
 
-- `--format text|json` — output mode (default `text`).
-- `--search <query>` — semantic search. Returns relevant snippets instead of the full page.
-- `--max-results <n>` — snippet count when `--search` is set. Default `5`, max `50`.
-- `--min-score <n>` — minimum relevance score, `0`–`1`. Default `0.6`.
+- `--format text|json` sets the output mode (default `text`).
+- `--highlights <query>` runs Highlights and returns relevant snippets instead of the full page.
+- `--max-results <n>` sets the snippet count when `--highlights` is set. Default `5`, max `50`.
+- `--min-score <n>` sets the minimum relevance score, `0`–`1`. Default `0.6`.
 
 ## Recommended: `hook install`
 
@@ -167,11 +221,11 @@ expandai hook install claude-code global
 expandai hook install opencode    project
 ```
 
-**Scope behavior:** `global` writes to the harness's user settings
+**Scope behavior.** `global` writes to the harness's user settings
 (`~/.claude/settings.json` for Claude Code, `~/.config/opencode/plugins/` for
-OpenCode) — the substitution fires for every session of that harness on the
+OpenCode), so the substitution fires for every session of that harness on the
 machine. `project` writes to per-directory equivalents
-(`<cwd>/.claude/settings.json`, `<cwd>/.opencode/plugins/`) — the
+(`<cwd>/.claude/settings.json`, `<cwd>/.opencode/plugins/`), so the
 substitution only fires when the harness is launched from that directory.
 
 Claude Code hook entries use the portable `expandai` command when it resolves on
@@ -179,22 +233,22 @@ Claude Code hook entries use the portable `expandai` command when it resolves on
 sharing a project-scoped profile, and restart Claude Code if the current session
 has not inherited the updated `PATH`.
 
-### Why both, not just one
+### Why both
 
 - The **runtime hook** makes the agent's native fetch path return ExpandAI
   markdown automatically, so even agents that haven't been told about
   expand.ai get clean rendered output for SPAs and bot-protected pages.
   No prompt changes required.
-- The **MCP server alone** requires the agent to consciously choose
+- The **MCP server alone** requires the agent to choose
   `mcp__expandai__fetch` over its built-in tool, which it usually won't
   unless something nudges it (a skill, a prompt).
-- Together, default web access is upgraded transparently, and explicit
-  refinement (snippets, screenshots, structured extraction) is still one
-  MCP tool call away.
+- Together they upgrade default web access transparently and keep explicit
+  refinement (snippets, screenshots, structured extraction) one MCP tool
+  call away.
 
 ### How the substitution actually works
 
-- **Claude Code** — `expandai hook install claude-code <scope>` writes a
+- **Claude Code.** `expandai hook install claude-code <scope>` writes a
   `PreToolUse` matcher into `settings.json` that scopes to `WebFetch`. When
   the agent invokes `WebFetch`, the runner reads the SDK's hook payload from
   stdin, fetches the URL via expand.ai, and emits
@@ -202,7 +256,7 @@ has not inherited the updated `PATH`.
   blocks the native fetch; `additionalContext` is the documented channel for
   injecting content into the model's next turn, so the markdown still reaches
   the model in place of WebFetch's raw output.
-- **OpenCode** — `expandai hook install opencode <scope>` copies an
+- **OpenCode.** `expandai hook install opencode <scope>` copies an
   `expandai-webfetch.js` plugin into `.opencode/plugins/`. The plugin hooks
   `tool.execute.after` on `webfetch`, shells out to `expandai hook opencode`
   with the URL (retrying through `npx -y expandai` when the binary is not
@@ -227,11 +281,11 @@ expandai skill install opencode    global    # ~/.config/opencode/skills/expanda
 expandai skill install codex       project   # <cwd>/.agents/skills/expandai/
 ```
 
-**Scope behavior:** `global` drops `SKILL.md` into the harness's user-level
+**Scope behavior.** `global` drops `SKILL.md` into the harness's user-level
 skills directory, so the agent auto-discovers it in every session. `project`
 drops it into the cwd-level skills directory, so it's only discovered when
-the harness is launched from that directory — useful for committing the
-skill alongside a project so collaborators pick it up automatically.
+the harness is launched from that directory. That is useful for committing
+the skill alongside a project so collaborators pick it up automatically.
 
 ## MCP server
 
@@ -254,14 +308,14 @@ expandai mcp install codex       global    # ~/.codex/config.toml
 expandai mcp install codex       project   # <cwd>/.codex/config.toml
 ```
 
-**Scope behavior:** `global` writes the `expandai` entry to the harness's
-user-level MCP config — the server is discoverable in every session.
-`project` writes to the per-directory MCP config — the server is only
-discoverable when the harness is launched from that directory, and the
-config file is well-suited for checking into version control.
+**Scope behavior.** `global` writes the `expandai` entry to the harness's
+user-level MCP config, so the server is discoverable in every session.
+`project` writes to the per-directory MCP config, so the server is only
+discoverable when the harness is launched from that directory. That config
+file is a good fit for version control.
 
 Profiles created by `expandai init` invoke the `expandai` command directly when
-it is on `PATH`, and fall back to `npx -y expandai` when it is not — so
+it is on `PATH`, and fall back to `npx -y expandai` when it is not, so
 `npx expandai init` produces a working profile without a global install.
 Re-running init after a global install rewrites the entry to the direct command.
 For the fastest startup, install the package globally and make sure its bin
@@ -282,6 +336,16 @@ expandai uninstall skill                    # all skills across all hosts and sc
 `expandai logout` also offers an interactive multi-select cleanup when run in
 a TTY.
 
+## crawl and sync
+
+`expandai crawl` and `expandai sync` mirror a site into a local Markdown folder.
+
+### Closed one-off crawls
+
+A one-off crawl closes permanently when it finishes, fails, or is canceled. Repeating `expandai sync pull <dir>` drains any remaining events and returns without starting another run. To crawl again, create a new crawl in a new directory; `sync run` reports a closed-crawl error and preserves the existing mirror state. For continuous updates, enable sync when creating the crawl.
+
+Sync initialization and recovery from an expired event checkpoint use a pinned catalog snapshot. After its final page, the mirror resumes the retained event log from the snapshot boundary.
+
 ## License
 
 Apache-2.0. See `LICENSE`.

package/build/dts/Mcp/Server.d.ts

@@ -2,7 +2,7 @@ import { Layer } from 'effect';
 export { CurrentRequestHeaders } from './RequestHeaders.js';
 export declare const ServerInfo: {
     readonly name: "expandai";
-    readonly version: "0.0.6";
+    readonly version: "0.1.0";
 };
 export declare const makeServerLayer: (options?: {
     readonly includeDocs?: boolean;