@anchrd/intel 0.59.0

package/README.md

@@ -0,0 +1,496 @@
+# @anchrd/intel
+
+Intel is a customer-deployed, model-agnostic operating layer for company intelligence and processes. It
+gives people and MCP-capable AI clients the same governed access to three surfaces:
+
+- **Intelligence** — company content as nodes, with sharing, immutable versions, retrieval, citations and a graph
+- **Flows** — versioned, durable processes that reference nodes and Tools
+- **Tools** — MCP capabilities discovered through a Cloudflare MCP Portal
+
+The UI, the HTTP API and the Intel MCP surface call the same application services, so authorization,
+validation and side effects never differ by surface. You deploy it into your own Cloudflare account
+against your own Gate; nothing runs on Anchrd infrastructure.
+
+This package contains the portable server, the Cloudflare adapter, the D1 migrations and the
+customer CLI.
+
+## Requirements
+
+- A [Gate](https://www.npmjs.com/package/@anchrd/gate-sdk) instance — Gate owns identity and
+  capabilities, and is the only required Anchrd dependency
+- A Cloudflare account with Workers, D1, R2, Queues and Workflows (Vectorize, Workers AI and an AI
+  Gateway are optional and only add semantic search and attachment conversion)
+- Node 22 or newer
+
+## Setting Intel up
+
+Nine steps. Step 3 talks only to Gate and can happen at any point; the rest run in this order,
+because each one needs what the one before it produced. **Step 8 is the one people skip**, and
+skipping it produces an installation that looks broken and is not.
+
+```bash
+npm install @anchrd/intel
+```
+
+### 1. Create the Cloudflare resources
+
+Workflows are created by the deploy in step 7; everything else has to exist first.
+
+```bash
+wrangler d1 create intel                  # note the printed database_id
+wrangler r2 bucket create intel-content
+wrangler queues create intel-indexing
+wrangler queues create intel-indexing-dlq # without it an exhausted message is dropped silently
+wrangler vectorize create intel-nodes --dimensions=1024 --metric=cosine   # optional, see Bindings
+```
+
+The Vectorize shape is not a free choice: 1024 dimensions and cosine are what Intel's default
+multilingual Workers AI embedding model (`@cf/baai/bge-m3`) produces. Nothing in the deploy compares
+the two, so an index created with other values fails later, at the first write.
+
+### 2. Write the Worker
+
+The customer edge re-exports the ready-made Cloudflare shell and nothing else. The named Workflow
+class has to travel with it or Wrangler cannot find the entrypoint:
+
+```ts
+// biome-ignore lint/performance/noBarrelFile: Wrangler needs the named Workflow entrypoint.
+export { default, IntelFlowWorkflow } from "@anchrd/intel/cloudflare";
+```
+
+Its Wrangler configuration declares the bindings from the table below, and points
+`migrations_dir` at `.intel/migrations` — the directory step 4 fills. `assets.directory` points at
+`.intel/ui`, which step 5 fills.
+
+### 3. Declare Intel's interfaces in Gate
+
+```bash
+GATE_URL=… GATE_SERVICE_KEY=… npx intel bootstrap
+```
+
+⚠️ **`bootstrap` reads its configuration from the environment of the shell it runs in**, not from
+the Worker's secrets — the two are different places that happen to use the same names. Without both
+values it stops with `Error: GATE_URL and GATE_SERVICE_KEY must be set.` before it does anything.
+
+It declares five interfaces — `intel`, `nodes`, `flows`, `tools`, `mcp` — with the functions listed
+under *Permission, layer one* below. It is idempotent and safe to repeat after an upgrade.
+
+⚠️ **`bootstrap` hands out no grant, and it takes none away as long as the declared list does not
+change.** It creates the permissions an administrator can hand out; it hands out none of them. This
+is step 8, and it is why a fresh installation shows an empty screen to everybody including the
+person who installed it.
+
+⚠️ **But a function that falls out of a still-declared interface takes its grants with it.**
+Declaring is an upsert that REPLACES an interface's function list, and Gate deletes every grant on a
+function the new list no longer names. So an upgrade that retires a function also retires every role
+assignment on it — against a running installation that is a change to who may do what, not a
+read. `bootstrap` reads the catalog before it writes and names each one as it goes:
+
+```text
+Removed flows:approve from Gate; every grant on it was deleted with it.
+Declared 5 Intel interfaces in Gate.
+```
+
+A run that dropped nothing says so instead: `No declared function was dropped, so Gate deleted no
+grant.` And if the catalog cannot be read, `bootstrap` writes nothing at all rather than delete
+grants it would then be unable to name.
+
+⚠️ It never revokes a whole **interface**. One an older version declared stays declared in Gate:
+`bootstrap` writes only the handles it names, so an interface outside that list keeps its functions
+and its grants. That is harmless, and removing one is an act in Gate, by hand.
+
+### 4. Copy the migrations out of `node_modules`
+
+```bash
+npx intel prepare
+```
+
+This writes the versioned SQL to `.intel/migrations`. It only copies — applying them is the next
+step, and nothing does it for you:
+
+```bash
+wrangler d1 migrations apply intel --local    # a local Worker
+wrangler d1 migrations apply intel --remote   # the deployed one
+```
+
+Migration filenames and their order are immutable after release. Re-run `intel prepare` after every
+upgrade of this package, before applying.
+
+### 5. Build the UI
+
+```bash
+npx intel build
+```
+
+`intel build` reads an optional strict `intel.json`, applies the customer branding, and writes the
+static output to `.intel/ui`. See *Branding* below. No customer build output ever lives in
+`node_modules`.
+
+### 6. Configure the Worker
+
+Set the variables and secrets from the two tables below. `GATE_SERVICE_KEY` and
+`INTEL_SESSION_SECRET` are Wrangler secrets (`wrangler secret put …`); the rest are plain vars.
+
+For a local Worker, copy `node_modules/@anchrd/intel/examples/dev.vars.example` to `.dev.vars`.
+
+### 7. Check it, then deploy
+
+```bash
+npx intel doctor
+wrangler deploy
+```
+
+`doctor` checks the same environment `bootstrap` uses, plus the installed packages and migrations
+and the Gate interfaces. It reports every problem at once rather than the first:
+
+```text
+Note: MCP_PORTAL_URL is unset, so the Tools area stays empty.
+FAIL GATE_URL is missing
+FAIL INTEL_URL is missing
+FAIL GATE_SERVICE_KEY is missing
+FAIL TOOL_SOURCE_ORIGINS is missing
+FAIL INTEL_SESSION_SECRET must contain at least 32 bytes
+```
+
+It also compares the Gate interfaces **in both directions**. A function or a whole interface Gate
+carries that this version of Intel no longer declares is reported as a `NOTE`, never as a `FAIL`:
+nothing is broken by it, and a check one knows red stops being read. Each note names the surplus,
+why it went, and what to do about it:
+
+```text
+NOTE Gate carries flows:approve, which Intel does not declare — the approval node is gone (#73).
+NOTE Gate carries the interface knowledge (read, write), which Intel does not declare — the whole interface became `nodes` (#152, after #125).
+NOTE The next `intel bootstrap` removes a surplus function and every grant on it (#474). Take the grant away in Gate first if anybody should keep it.
+NOTE `bootstrap` never touches an interface Intel does not declare, so a surplus one stays until somebody removes it in Gate by hand — which takes its grants with it.
+OK Intel packages, configuration, and Gate interfaces are ready.
+```
+
+⚠️ **`doctor` cannot see the Worker's secrets.** It answers about the shell it runs in, so it says
+nothing about whether the deployed Worker is configured. `GET /health` on the deployed Worker
+answers `{"status":"ok"}`; a Worker missing one of the five required variables answers
+`500 configuration_missing` and names them.
+
+⚠️ **`doctor` does not check that step 4 happened.** It looks for the migrations *installed* in
+`node_modules`, not for the copies in `.intel/migrations`, so it answers `OK` on a project that never
+ran `intel prepare`. `wrangler deploy` does not mind an empty `migrations_dir` either, and the first
+sign is a database with no tables in it. The `.intel/migrations` directory has to be checked by eye.
+
+### 8. Hand out permissions — both layers
+
+This is the step that decides whether anybody sees anything, and it is entirely manual.
+
+1. In **Gate**, grant people the Intel capabilities from *Permission, layer one*. Without
+   `nodes.read` the sidebar answers `403` and says a permission is missing.
+2. In **Intel**, share a node with them. Without a grant the tree is empty — correctly, because
+   nothing has been shared yet.
+
+Both are needed. Neither on its own produces a usable screen, and the symptom of missing either one
+is the same shape: a view with nothing in it. See *Permission has two layers* below.
+
+### 9. Connect the portal, if there is one
+
+Tools are optional. If the installation has a Cloudflare MCP Portal, set `MCP_PORTAL_URL` and add
+its origin to `TOOL_SOURCE_ORIGINS`, then read *Tools come from the portal* below — two settings
+outside this repository decide whether the silent sign-in can work at all.
+
+## Bindings
+
+The reference Wrangler deployment binds all of these. "Required" means a request fails without it,
+not that Wrangler refuses to deploy.
+
+| Binding | Resource | Required | What its absence costs |
+|---|---|---|---|
+| `DB` | D1 | **yes** | Nothing works: D1 owns metadata, relationships, grants, flow and run state, audit |
+| `CONTENT` | R2 | **yes** | Node content and attachments cannot be read or written |
+| `INDEXING` | Queue | **yes** | Every save fails; this is the queue that builds the indexes |
+| `FLOWS` | Workflow `IntelFlowWorkflow` | **yes** | Flows cannot be run |
+| `ASSETS` | static assets from `.intel/ui` | for the UI | The API and MCP surface still answer; the browser application is not served at all |
+| `AI` | Workers AI | no | **Two losses, and only one is obvious.** No semantic search, and no attachment conversion — an uploaded PDF or Word file is stored intact but never becomes searchable text |
+| `SEARCH` | Vectorize index, 1024 dims, cosine | no | Search silently falls back to lexical only. Nothing says so, and half a search looks exactly like a whole one |
+
+⚠️ **Semantic search needs `AI`, `SEARCH` *and* the variable `AI_GATEWAY_ID`.** Any one of the three
+missing leaves search lexical. If you add them to an existing installation, run `intel reindex`
+afterwards — the embeddings for content already stored are built by that pass and by nothing else.
+
+⚠️ **The gateway is a condition, not an option.** Intel makes no Workers AI call without it: no
+gateway name, no semantic index and no attachment conversion. The alternative — calling Workers AI
+directly whenever the variable is absent — is the failure this rule exists against, because a spend
+limit set on the gateway counts only what passes through it. An installation embedding past it works
+perfectly, costs money, and tells nobody.
+
+There is no `AGENT` service binding, and `AI_GATEWAY_ACCOUNT_ID` and the `AI_GATEWAY_READ_TOKEN`
+secret are gone. All three belonged to the agent runtime, which is no longer part of Intel; an
+installation that still carries them can drop them, including
+`wrangler secret delete AI_GATEWAY_READ_TOKEN`. ⚠️ `AI_GATEWAY_ID` is **not** among them: it carries
+a different job now, described in the row below.
+
+## Variables and secrets
+
+Every value below is read in **two different places** depending on the command, and mixing them up
+is the most common setup failure:
+
+- the **Worker's** environment — what the deployed Intel reads on every request;
+- the **shell's** environment — what `intel bootstrap`, `intel doctor` and `intel reindex` read.
+
+`GATE_URL`, `GATE_SERVICE_KEY`, `INTEL_URL`, `INTEL_SESSION_SECRET` and `TOOL_SOURCE_ORIGINS` are
+needed in both. Setting them only as Wrangler secrets leaves the CLI blind, and setting them only in
+a shell leaves the Worker answering `500 configuration_missing`.
+
+⚠️ `MCP_PORTAL_URL` is the eighth value in the same trap, and the one that fails quietly.
+`intel doctor` reads it from the shell as well, so a portal configured only as a Wrangler var makes
+`doctor` report `MCP_PORTAL_URL is unset` about an installation that is configured correctly — and
+skip the origin check it would otherwise have run.
+
+| Name | Where | Required | What it does, and what a wrong value does |
+|---|---|---|---|
+| `GATE_URL` | Worker + shell | **yes** | The Gate this installation authenticates against. Wrong: every request is refused and no login completes |
+| `GATE_SERVICE_KEY` | Worker secret + shell | **yes** | Intel's own service credential at Gate. Wrong: every authorization call fails, so everything answers as unauthenticated. It is a secret and never appears in a response, log or tool result |
+| `INTEL_URL` | Worker + shell | **yes** | Intel's own public base URL. It is also **the OAuth resource identifier**: Intel asks Gate for a token for `<INTEL_URL>/mcp`. ⚠️ A trailing-slash or scheme difference is a *different* resource. ⚠️ **Changing it later is a one-way street**: the resource is bound once on the Gate side and cannot be re-pointed — the way back is deleting the Gate service and setting it up again. Decide the hostname before the first login, not after |
+| `INTEL_SESSION_SECRET` | Worker secret + shell | **yes** | Derives the key for the encrypted `HttpOnly` session cookie and for sealing each person's portal token in D1. Must be at least 32 bytes. ⚠️ **Changing it invalidates every session and makes every stored portal token unreadable** — everybody signs in again, and everybody reconnects to the portal |
+| `TOOL_SOURCE_ORIGINS` | Worker + shell | **yes** | Comma-separated **exact origins** Intel is allowed to fetch tools from. ⚠️ An entry carrying a path or credentials is refused — and refused *per request*, so a bad value deploys cleanly and then answers `500` on everything |
+| `MCP_PORTAL_URL` | Worker + shell (for `doctor`) | no | The portal's Streamable HTTP MCP endpoint. Unset means the Tools area stays empty and Intel says so rather than reporting an error. ⚠️ Its origin must also appear in `TOOL_SOURCE_ORIGINS`; `intel doctor` fails when it does not — but only if it can see the value, so set it in the shell too before believing the check ran |
+| `ALLOW_INSECURE_OAUTH` | Worker | no | `true` permits an `http://` OAuth issuer. **Local development only.** Production issuers must be HTTPS |
+| `FLOW_RUN_STALL_TIMEOUT` | Worker | no | How long a run may stand on one step before it is ended as stalled, e.g. `30 minutes`. Too short is worse than the problem: it ends runs that were merely slow |
+| `AI_GATEWAY_ID` | Worker | for `AI` | The **name** of the AI Gateway every Workers AI call is routed through, as it stands in the address of its dashboard page. ⚠️ Unset means Intel calls Workers AI **not at all** — no semantic index, no attachment conversion — rather than calling it directly. A wrong name fails the embedding and conversion calls loudly, which is the intended direction: a spend limit set on the gateway counts only what reaches it |
+| `INTEL_OPERATOR_TOKEN` | shell | for `reindex` | A short-lived bearer carrying the Gate capability `intel.admin`, used only by `intel reindex`. Not a Worker value and not a stored one |
+
+Gate OAuth uses PKCE and dynamic public-client registration; browser access tokens stay inside an
+encrypted `HttpOnly` cookie and are never exposed to the UI bundle or D1.
+
+## Branding: `intel.json`
+
+Put `intel.json` next to the customer project's `package.json`:
+
+```json
+{
+  "ui": {
+    "theme": "./branding/theme.css",
+    "logo": "./branding/logo.svg",
+    "favicon": "./branding/favicon.svg",
+    "defaultLanguage": "en",
+    "languages": {}
+  }
+}
+```
+
+The theme is plain CSS containing shadcn semantic tokens such as `--primary`, `--background`,
+`--sidebar`, and `--radius`. It is loaded after Intel's defaults, so customer tokens win without a
+component fork. Unknown config fields, missing assets, non-SVG branding, and incomplete language
+catalogs fail the build rather than producing a half-branded application. A ready-to-copy starting
+point is included under `node_modules/@anchrd/intel/examples`.
+
+The file is optional: `intel build` without it produces the default Intel branding.
+
+## Permission has two layers
+
+Access to anything in Intel is the **and** of two independent answers, and neither layer knows about
+the other. This is the single most common source of "it is broken" reports, because a person missing
+either layer sees roughly the same thing: a screen with nothing on it.
+
+### Layer one: the Gate capability
+
+What somebody may do *at all*, granted in Gate. `intel bootstrap` declares these; an administrator
+grants them.
+
+| Interface | Functions |
+|---|---|
+| `intel` | `admin` |
+| `nodes` | `read`, `create`, `write`, `share` |
+| `flows` | `read`, `create`, `write`, `publish`, `run`, `share` |
+| `tools` | `read`, `test`, `execute` |
+| `mcp` | `connect` |
+
+A missing capability answers **`403`**, and the screen says a permission is missing.
+
+⚠️ **`mcp.connect` is the one that gates a whole endpoint rather than an action.** Intel's own MCP
+endpoint (`POST /mcp`, the one an MCP client or a portal connects to) checks it **on every request**
+and answers `403` without it — before any tool runs and before anything is read. Two things follow
+for an operator:
+
+- **Somebody who should use Intel through an MCP client needs `mcp.connect` in Intel**, beside their
+  `nodes.*` capabilities. Without it the client authenticates successfully and every call is refused.
+- **Revoking it takes effect on the next call**, with no new sign-in and nothing to press. If a
+  Cloudflare Access policy in front of the endpoint also compares the `mcp` claim, that layer is the
+  slower of the two — the claim is written once per sign-in and a running session carries it for up
+  to 24 hours, while this check reads the live grant rows every time.
+
+The web UI is unaffected: it speaks the HTTP surface, which is gated by `nodes.*`, `flows.*` and
+`tools.*` as before.
+
+### Layer two: the grant on the node
+
+What somebody may reach *in this tree*, granted in Intel on a node. Four verbs — `read`, `write`,
+`execute`, `share` — each granted on its own; none implies another. Seeing a process is deliberately
+separable from being allowed to start it.
+
+A grant on a **folder is inherited by everything beneath it**, including nodes created after the
+grant was made. That is the usual way to share a whole area.
+
+A missing grant answers **`404`**, never `403`: Intel gives the same answer for "there is no such
+node" and "not for you", so that a refusal never confirms that something exists.
+
+### Reading the empty screen
+
+| What you see | Which layer |
+|---|---|
+| A sentence naming a missing permission | Gate — layer one |
+| An empty tree, no error | Intel — nothing has been shared with this person yet |
+| A node that vanished between two visits | Its grant was revoked; revocation takes effect immediately, on every path, without a new login |
+
+A shared node appears in the recipient's navigation on its own, at the top level, without exposing
+the titles of the folders above it. Nobody has to be sent a link.
+
+### Three things about grants that surprise people
+
+- **A flow is not a node, and is reached by two things at once.** The folder it is filed in passes
+  its grants down, and since #530 a grant may sit on the flow itself; they add up. Move a flow and
+  you have changed the first half of that. A grant on one flow does not reach the flows it calls —
+  the answer names them, and `flow_validate` tells the person about to run what is still missing.
+- **`organization` + `execute` on a folder makes that folder a library**: flows from anywhere in the
+  tree may then call into it. The share dialog warns before the click, because the way back is
+  narrow — revoking is refused with `409 folder_execute_in_use` for as long as one of those callers
+  still calls.
+- **A grant can be complete and a flow still stop.** A run is re-authorized against the person
+  running it, so a flow in a shared folder that reads a document outside it stops for somebody who
+  cannot read that document. The answer names what was out of reach.
+
+## Export and import
+
+`GET /api/v1/nodes/export` writes the whole installation as this caller may read it; the same route
+under a node id exports that subtree. Import is the mirror:
+
+```bash
+curl -X POST https://intel.example.com/api/v1/nodes/import \
+  -H "authorization: Bearer $TOKEN" \
+  -H "content-type: application/zip" \
+  -H "idempotency-key: $(uuidgen)" \
+  --data-binary @bundle.zip
+```
+
+⚠️ **The idempotency key travels as a header.** The body is the zip itself, so there is nowhere else
+to put it. A request without the header is refused.
+
+Only nodes the caller may read enter the bundle; exporting a folder that is out of reach answers
+`404`. A fresh import answers `201`; a repeat of the same idempotency key answers `200` with the
+summary of the earlier import, because this request created nothing.
+
+⚠️ **A bundle carries only the current state of each node. Version history does not survive
+export → import.** A document with two versions comes back with one, and the earlier one is then
+retrievable nowhere. Everything else survives in full — nodes per kind, table content including a
+previous `redefine`, and attachments byte for byte. The loss is accepted and deliberate, but anyone
+moving an installation this way has to know about it **before** they start: after the import there
+is nothing to recover from.
+
+## Tools come from the portal
+
+Intel stores no tool permissions. The catalog is a **live `tools/list`** made with the requesting
+person's own token, never a mirrored table. Two consequences follow, and both are correct behaviour
+rather than bugs:
+
+- **Two people legitimately see different tools**, and the same person can see a different set
+  tomorrow. What is offered is whatever the portal answers for them at that moment.
+- **A tool can disappear between defining a flow and running it.** Published flows pin immutable
+  MCP schema fingerprints, so a changed or vanished tool is reported by name instead of being
+  silently substituted.
+
+The portal endpoint exposes RFC 9728 metadata. The Tools UI follows that metadata, dynamically
+registers with Gate or Cloudflare Access, and completes a separate PKCE flow — silently, with
+`prompt=none`, as soon as a Gate session exists. Nobody is asked to connect anything.
+
+⚠️ **Two settings outside this repository decide whether that silent sign-in can work.** In
+Cloudflare Zero Trust → Access controls → AI controls → your portal → Edit → Advanced settings,
+`Managed OAuth` must be enabled, and an Access policy must carry the people who use Intel. Without
+both, every silent sign-in is refused and the Tools area shows "No access to the company portal" —
+correct behaviour for somebody outside every policy, and a misleading one for a deployment that
+simply never enabled the setting.
+
+⚠️ **Somebody who carries the Gate capability `mcp.connect` at no service at all is not sent into
+that sign-in.** Gate answers that question service-wide, Intel asks it before the first redirect,
+and the person stays here reading a sentence that says whom to ask — rather than landing on the
+portal's own error page, outside Intel, with no way back but the browser's back button
+(anchrd/intel#434). Grant `mcp.connect` alongside the Intel capabilities to everyone who should
+reach Tools.
+
+Two things that guard deliberately does not do. It **names no server**: the answer is one bit, so
+somebody who may reach a different product's MCP server walks on and sees what the portal offers
+them. And it **promises nothing** — this is the part worth knowing before you grant: the portal
+decides on the `mcp` claim frozen into the ID token, and a running Cloudflare Access session carries
+the old claim for up to 24 hours. ⚠️ **Inside that window the old dead end is still open.** Somebody
+who was granted `mcp.connect` a minute ago is not refused here, walks on, and Access then decides on
+the claim state that produced the error page in the first place. The guard closes the trip that is
+certainly pointless; it does not close this one. The way out is the same as before — sign out of
+Access and in again, so a new ID token carries the new claim. (Not measured against a running
+installation; the claim lifetime is the documented one.) Against a Gate too old to answer the
+question at all, nobody is refused here.
+
+The Tools screen distinguishes four situations rather than calling all of them "no access": a
+refusal that really came back, an expired connection, a sign-in that broke, and a sign-in that was
+started and never answered. Only the first has no button, because only there is there nothing a
+second attempt would change.
+
+The only tool secret Intel stores is the resulting per-user access token for its own portal
+endpoint — one per person, never one shared operator token — sealed with a key derived from
+`INTEL_SESSION_SECRET` and kept in the `portal_tokens` table of your D1; provider credentials stay
+with the portal and never reach Intel.
+
+Intel unseals that token just in time and renews it shortly before it expires. **"Expired" above is
+what a failed renewal looks like**: a token that cannot be renewed is dropped rather than kept and
+retried with, the browser signs in again silently, and an MCP client repeats its own authorization.
+Two different causes end up there — a portal that issued no refresh token at all, which is a
+configuration problem, and a refresh that was rejected, which is not — and Intel logs which of the
+two it was, because the person only ever sees "sign in again". The Intel audience token is never forwarded to another OAuth
+resource. Durable Flow state contains no browser, Gate, or provider credentials.
+
+## The `intel` CLI
+
+```text
+intel prepare    Copy versioned D1 migrations to .intel/migrations
+intel bootstrap  Idempotently declare Intel interfaces in Gate
+intel build      Apply intel.json and atomically build the customer UI
+intel doctor     Check packages, environment, migrations, UI, and Gate interfaces
+intel reindex    Request a full authorized search-index rebuild
+```
+
+All five run in the customer project directory — the one holding `package.json`, `intel.json` and
+`node_modules` — and read their configuration from that shell's environment.
+
+`intel reindex` needs `INTEL_URL` and a short-lived `INTEL_OPERATOR_TOKEN`. It asks the running
+installation to rebuild the derived indexes from D1 and R2 and returns as soon as the request is
+accepted; the work happens on the indexing queue. Reach for it after adding `AI`, `SEARCH` and
+`AI_GATEWAY_ID` to an installation that ran without them, and after restoring an emptied Vectorize
+index.
+
+## Writing against the API directly
+
+Everything the browser does is available over HTTP and over the Intel MCP surface, against the same
+application services. The schemas are inside this package under `src/contract` — the runtime-
+validated wire formats the server, the browser application and every MCP tool share. Read
+[`src/contract/README.md`](./src/contract/README.md) first: it carries the six things that reliably
+cost a first-time caller an afternoon.
+
+## What is in this package
+
+One package carries all of Intel, and that is deliberate: an installation pins **one** version
+number instead of three that could drift apart.
+
+| Path | What it is |
+|---|---|
+| `dist/` | the server: portable application services, the HTTP and MCP adapters, the Cloudflare shell behind `@anchrd/intel/cloudflare` |
+| `bin/intel.mjs` | the `intel` CLI — `prepare`, `bootstrap`, `build`, `doctor`, `reindex` |
+| `migrations/` | the D1 migrations, copied out by `intel prepare` |
+| `src/contract/` | the wire schemas, as source, because the browser application compiles them too |
+| `ui/` | the browser application, shipped as **source** so `intel build` applies your own theme, logo and languages — see [`ui/README.md`](./ui/README.md) |
+
+⚠️ **Installing this package pulls the browser build toolchain with it** — Vite, React, Tailwind and
+the document renderers are runtime dependencies here, even if you never run `intel build`. That is
+the knowingly accepted cost of one version number: a headless Intel has no consumer today, and
+splitting one out later is more expensive than not building it now.
+
+⚠️ **`@anchrd/intel-api`, `@anchrd/intel-ui` and `@anchrd/intel-contract` are the predecessors of this
+package.** Once this package is on npm they get an `npm deprecate` pointing here. They are **not**
+deleted, and the versions that are published stay published.
+
+## License
+
+Proprietary. Published publicly for installation convenience; this is not an open-source license and
+grants no right to use, copy, modify or redistribute the software. Contact Anchrd for licensing.

package/bin/intel.mjs

@@ -0,0 +1,134 @@
+#!/usr/bin/env node
+import { spawn } from "node:child_process";
+import { cp, mkdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
+import { dirname, resolve } from "node:path";
+import { createGateClient } from "@anchrd/gate-sdk";
+import { createCli } from "../dist/cli/cli.js";
+
+// Since #827 the surface and the migrations travel inside @anchrd/intel itself, so an installation
+// finds both below one package directory. The second entry of each pair is this repository's own
+// tree, which is what `bun run build` and `bun run dev` walk.
+const uiCandidates = [
+  { dir: "node_modules/@anchrd/intel/ui", outDir: ".intel/ui" },
+  { dir: "packages/intel/ui", outDir: "packages/intel/ui/dist" },
+];
+const migrationCandidates = ["node_modules/@anchrd/intel/migrations", "packages/intel/migrations"];
+
+// npm sets npm_execpath to its own JavaScript entry point. Running that through the current Node
+// binary avoids the .cmd shim entirely, which matters on Windows: Node has refused to spawn a .cmd
+// without a shell since CVE-2024-27980, and cmd.exe expands %NAME% even inside quotes, so a checkout
+// path containing a percent sign would be rewritten.
+function npmScriptPath(command) {
+  if (command !== "npm") return null;
+  const execPath = process.env.npm_execpath;
+  return execPath?.endsWith(".js") ? execPath : null;
+}
+
+// Only reached on the Windows shell fallback, where cmd.exe would otherwise split on whitespace.
+function quoteForCmd(argument) {
+  if (!/[\s"^&|<>()]/.test(argument)) return argument;
+  return `"${argument.replaceAll('"', '""')}"`;
+}
+
+function spawnArguments(command, args) {
+  const script = npmScriptPath(command);
+  if (script) return [process.execPath, [script, ...args]];
+  if (process.platform !== "win32") return [command, args];
+  return [`${command}.cmd`, args.map(quoteForCmd)];
+}
+
+async function isFile(path) {
+  try {
+    return (await stat(path)).isFile();
+  } catch (error) {
+    if (error?.code === "ENOENT" || error?.code === "ENOTDIR") return false;
+    throw error;
+  }
+}
+
+async function isDirectory(path) {
+  try {
+    return (await stat(path)).isDirectory();
+  } catch (error) {
+    if (error?.code === "ENOENT" || error?.code === "ENOTDIR") return false;
+    throw error;
+  }
+}
+
+async function replaceDir(from, to) {
+  await mkdir(dirname(to), { recursive: true });
+  const backup = `${to}.old`;
+  await rm(backup, { recursive: true, force: true });
+  let hasBackup = false;
+  try {
+    await rename(to, backup);
+    hasBackup = true;
+  } catch (error) {
+    if (error?.code !== "ENOENT") throw error;
+  }
+  try {
+    await rename(from, to);
+  } catch (error) {
+    if (hasBackup) await rename(backup, to);
+    throw error;
+  }
+  if (hasBackup) await rm(backup, { recursive: true, force: true });
+}
+
+const cli = createCli({
+  env: process.env,
+  fetch: globalThis.fetch,
+  createGateInterfaces: (url, serviceKey) => createGateClient({ url, serviceKey }).interfaces,
+  readTextFile: async (path) => {
+    try {
+      return await readFile(path, "utf8");
+    } catch (error) {
+      if (error?.code === "ENOENT") return null;
+      throw error;
+    }
+  },
+  writeTextFile: async (path, content) => {
+    await mkdir(dirname(path), { recursive: true });
+    await writeFile(path, content, "utf8");
+  },
+  resolveUi: async () => {
+    for (const candidate of uiCandidates) {
+      if (await isFile(`${candidate.dir}/package.json`)) {
+        return { dir: candidate.dir, outDir: resolve(process.cwd(), candidate.outDir) };
+      }
+    }
+    return null;
+  },
+  resolveMigrations: async () => {
+    for (const dir of migrationCandidates) {
+      if (await isDirectory(dir)) {
+        return { dir, outDir: resolve(process.cwd(), ".intel/migrations") };
+      }
+    }
+    return null;
+  },
+  copyDir: async (from, to) => {
+    await rm(to, { recursive: true, force: true });
+    await mkdir(dirname(to), { recursive: true });
+    await cp(from, to, { recursive: true });
+  },
+  exec: (command, args, { cwd }) =>
+    new Promise((resolveCode, reject) => {
+      const child = spawn(...spawnArguments(command, args), {
+        cwd,
+        stdio: "inherit",
+        shell: process.platform === "win32" && !npmScriptPath(command),
+      });
+      child.on("error", reject);
+      child.on("close", (code) => resolveCode(code ?? 1));
+    }),
+  replaceDir,
+  log: (message) => console.log(message),
+});
+
+try {
+  process.exitCode = await cli.run(process.argv.slice(2));
+} catch (error) {
+  console.error(`Error: ${error instanceof Error ? error.message : String(error)}`);
+  process.exitCode = 1;
+}

package/dist/adapters/cloudflare/cloudflare-flow-workflow.d.ts

@@ -0,0 +1,42 @@
+import type { CloudflareEnv } from "./cloudflare.types.js";
+interface WorkflowEvent<Params> {
+    payload: Readonly<Params>;
+}
+interface WorkflowStep {
+    do<T>(name: string, callback: () => Promise<T>): Promise<T>;
+    waitForEvent(name: string, options: {
+        type: string;
+        timeout?: string | number;
+    }): Promise<unknown>;
+}
+interface FlowWorkflowParams {
+    runId: string;
+}
+interface IntelFlowWorkflowInstance {
+    run(event: Readonly<WorkflowEvent<FlowWorkflowParams>>, step: WorkflowStep): Promise<unknown>;
+}
+/**
+ * What the durable wait actually does, with storage handed in.
+ *
+ * ⚠️ Separated from the class because `WorkflowEntrypoint` is a runtime class that cannot be
+ * constructed outside workerd's workflow context — so as long as this logic lived inside it, none
+ * of it could be tested, including the rule that keeps a caller alive while its child runs. That
+ * rule is the one that decides whether this feature destroys work or not, and it had to be
+ * provable. The shell below is now only bindings and SQL, which is what `CLAUDE.md` asks of it.
+ */
+export declare function driveFlowRun(step: WorkflowStep, deps: {
+    readStatus(): Promise<{
+        status: string;
+        error: string | null;
+    }>;
+    hasRunningChild(): Promise<boolean>;
+    markFailed(error: string): Promise<void>;
+    stallTimeout: string;
+}): Promise<{
+    status: string;
+    error: string | null;
+}>;
+export declare const IntelFlowWorkflow: {
+    new (context: unknown, env: CloudflareEnv): IntelFlowWorkflowInstance;
+};
+export {};