@anchrd/intel-api 0.16.1 → 0.18.0

package/README.md

@@ -22,26 +22,203 @@ customer CLI.
   optional and only add semantic search and attachment conversion)
 - Node 22 or newer
 
-## Install
+## Setting Intel up
+
+Nine steps. Step 3 talks only to Gate and can happen at any point; the rest run in this order,
+because each one needs what the one before it produced. **Step 8 is the one people skip**, and
+skipping it produces an installation that looks broken and is not.
 
 ```bash
 npm install @anchrd/intel-api @anchrd/intel-ui
-npx intel prepare
-npx intel bootstrap
-npx intel build
-npx intel doctor
 ```
 
-Use the Cloudflare export from a customer Worker:
+### 1. Create the Cloudflare resources
+
+Workflows are created by the deploy in step 7; everything else has to exist first.
+
+```bash
+wrangler d1 create intel                  # note the printed database_id
+wrangler r2 bucket create intel-content
+wrangler queues create intel-indexing
+wrangler queues create intel-indexing-dlq # without it an exhausted message is dropped silently
+wrangler vectorize create intel-nodes --dimensions=1024 --metric=cosine   # optional, see Bindings
+```
+
+The Vectorize shape is not a free choice: 1024 dimensions and cosine are what Intel's default
+multilingual Workers AI embedding model (`@cf/baai/bge-m3`) produces. Nothing in the deploy compares
+the two, so an index created with other values fails later, at the first write.
+
+### 2. Write the Worker
+
+The customer edge re-exports the ready-made Cloudflare shell and nothing else. The named Workflow
+class has to travel with it or Wrangler cannot find the entrypoint:
 
 ```ts
 // biome-ignore lint/performance/noBarrelFile: Wrangler needs the named Workflow entrypoint.
 export { default, IntelFlowWorkflow } from "@anchrd/intel-api/cloudflare";
 ```
 
-`intel build` reads an optional strict `intel.json`. Customer theme files override shadcn semantic
-tokens; SVG logo/favicon and complete language catalogs are embedded at build time. Installed builds
-land in `.intel/ui`, and migrations in `.intel/migrations`.
+Its Wrangler configuration declares the bindings from the table below, and points
+`migrations_dir` at `.intel/migrations` — the directory step 4 fills. `assets.directory` points at
+`.intel/ui`, which step 5 fills.
+
+### 3. Declare Intel's interfaces in Gate
+
+```bash
+GATE_URL=… GATE_SERVICE_KEY=… npx intel bootstrap
+```
+
+⚠️ **`bootstrap` reads its configuration from the environment of the shell it runs in**, not from
+the Worker's secrets — the two are different places that happen to use the same names. Without both
+values it stops with `Error: GATE_URL and GATE_SERVICE_KEY must be set.` before it does anything.
+
+It declares five interfaces — `intel`, `nodes`, `flows`, `tools`, `mcp` — with the functions listed
+under *Permission, layer one* below. It is idempotent and safe to repeat after an upgrade.
+
+⚠️ **`bootstrap` declares interfaces and changes no grant.** It creates the permissions an
+administrator can hand out; it hands out none of them. This is step 8, and it is why a fresh
+installation shows an empty screen to everybody including the person who installed it.
+
+⚠️ It also never revokes. An interface an older version declared stays declared in Gate; that is
+harmless, and removing one is an act in Gate, by hand.
+
+### 4. Copy the migrations out of `node_modules`
+
+```bash
+npx intel prepare
+```
+
+This writes the versioned SQL to `.intel/migrations`. It only copies — applying them is the next
+step, and nothing does it for you:
+
+```bash
+wrangler d1 migrations apply intel --local    # a local Worker
+wrangler d1 migrations apply intel --remote   # the deployed one
+```
+
+Migration filenames and their order are immutable after release. Re-run `intel prepare` after every
+upgrade of this package, before applying.
+
+### 5. Build the UI
+
+```bash
+npx intel build
+```
+
+`intel build` reads an optional strict `intel.json`, applies the customer branding, and writes the
+static output to `.intel/ui`. See *Branding* below. No customer build output ever lives in
+`node_modules`.
+
+### 6. Configure the Worker
+
+Set the variables and secrets from the two tables below. `GATE_SERVICE_KEY` and
+`INTEL_SESSION_SECRET` are Wrangler secrets (`wrangler secret put …`); the rest are plain vars.
+
+For a local Worker, copy `node_modules/@anchrd/intel-api/examples/dev.vars.example` to `.dev.vars`.
+
+### 7. Check it, then deploy
+
+```bash
+npx intel doctor
+wrangler deploy
+```
+
+`doctor` checks the same environment `bootstrap` uses, plus the installed packages and migrations
+and the Gate interfaces. It reports every problem at once rather than the first:
+
+```text
+Note: MCP_PORTAL_URL is unset, so the Tools area stays empty.
+FAIL GATE_URL is missing
+FAIL INTEL_URL is missing
+FAIL GATE_SERVICE_KEY is missing
+FAIL TOOL_SOURCE_ORIGINS is missing
+FAIL INTEL_SESSION_SECRET must contain at least 32 bytes
+```
+
+⚠️ **`doctor` cannot see the Worker's secrets.** It answers about the shell it runs in, so it says
+nothing about whether the deployed Worker is configured. `GET /health` on the deployed Worker
+answers `{"status":"ok"}`; a Worker missing one of the five required variables answers
+`500 configuration_missing` and names them.
+
+⚠️ **`doctor` does not check that step 4 happened.** It looks for the migrations *installed* in
+`node_modules`, not for the copies in `.intel/migrations`, so it answers `OK` on a project that never
+ran `intel prepare`. `wrangler deploy` does not mind an empty `migrations_dir` either, and the first
+sign is a database with no tables in it. The `.intel/migrations` directory has to be checked by eye.
+
+### 8. Hand out permissions — both layers
+
+This is the step that decides whether anybody sees anything, and it is entirely manual.
+
+1. In **Gate**, grant people the Intel capabilities from *Permission, layer one*. Without
+   `nodes.read` the sidebar answers `403` and says a permission is missing.
+2. In **Intel**, share a node with them. Without a grant the tree is empty — correctly, because
+   nothing has been shared yet.
+
+Both are needed. Neither on its own produces a usable screen, and the symptom of missing either one
+is the same shape: a view with nothing in it. See *Permission has two layers* below.
+
+### 9. Connect the portal, if there is one
+
+Tools are optional. If the installation has a Cloudflare MCP Portal, set `MCP_PORTAL_URL` and add
+its origin to `TOOL_SOURCE_ORIGINS`, then read *Tools come from the portal* below — two settings
+outside this repository decide whether the silent sign-in can work at all.
+
+## Bindings
+
+The reference Wrangler deployment binds all of these. "Required" means a request fails without it,
+not that Wrangler refuses to deploy.
+
+| Binding | Resource | Required | What its absence costs |
+|---|---|---|---|
+| `DB` | D1 | **yes** | Nothing works: D1 owns metadata, relationships, grants, flow and run state, audit |
+| `CONTENT` | R2 | **yes** | Node content and attachments cannot be read or written |
+| `INDEXING` | Queue | **yes** | Every save fails; this is the queue that builds the indexes |
+| `FLOWS` | Workflow `IntelFlowWorkflow` | **yes** | Flows cannot be run |
+| `ASSETS` | static assets from `.intel/ui` | for the UI | The API and MCP surface still answer; the browser application is not served at all |
+| `AI` | Workers AI | no | **Two losses, and only one is obvious.** No semantic search, and no attachment conversion — an uploaded PDF or Word file is stored intact but never becomes searchable text |
+| `SEARCH` | Vectorize index, 1024 dims, cosine | no | Search silently falls back to lexical only. Nothing says so, and half a search looks exactly like a whole one |
+
+⚠️ **Semantic search needs `AI` *and* `SEARCH`.** Either one alone leaves search lexical. If you
+add them to an existing installation, run `intel reindex` afterwards — the embeddings for content
+already stored are built by that pass and by nothing else.
+
+There is no `AGENT` service binding and no `AI_GATEWAY_*` configuration. Both belonged to the agent
+runtime, which is no longer part of Intel; an installation that still carries them can drop them,
+including `wrangler secret delete AI_GATEWAY_READ_TOKEN`.
+
+## Variables and secrets
+
+Every value below is read in **two different places** depending on the command, and mixing them up
+is the most common setup failure:
+
+- the **Worker's** environment — what the deployed Intel reads on every request;
+- the **shell's** environment — what `intel bootstrap`, `intel doctor` and `intel reindex` read.
+
+`GATE_URL`, `GATE_SERVICE_KEY`, `INTEL_URL`, `INTEL_SESSION_SECRET` and `TOOL_SOURCE_ORIGINS` are
+needed in both. Setting them only as Wrangler secrets leaves the CLI blind, and setting them only in
+a shell leaves the Worker answering `500 configuration_missing`.
+
+⚠️ `MCP_PORTAL_URL` is the eighth value in the same trap, and the one that fails quietly.
+`intel doctor` reads it from the shell as well, so a portal configured only as a Wrangler var makes
+`doctor` report `MCP_PORTAL_URL is unset` about an installation that is configured correctly — and
+skip the origin check it would otherwise have run.
+
+| Name | Where | Required | What it does, and what a wrong value does |
+|---|---|---|---|
+| `GATE_URL` | Worker + shell | **yes** | The Gate this installation authenticates against. Wrong: every request is refused and no login completes |
+| `GATE_SERVICE_KEY` | Worker secret + shell | **yes** | Intel's own service credential at Gate. Wrong: every authorization call fails, so everything answers as unauthenticated. It is a secret and never appears in a response, log or tool result |
+| `INTEL_URL` | Worker + shell | **yes** | Intel's own public base URL. It is also **the OAuth resource identifier**: Intel asks Gate for a token for `<INTEL_URL>/mcp`. ⚠️ A trailing-slash or scheme difference is a *different* resource. ⚠️ **Changing it later is a one-way street**: the resource is bound once on the Gate side and cannot be re-pointed — the way back is deleting the Gate service and setting it up again. Decide the hostname before the first login, not after |
+| `INTEL_SESSION_SECRET` | Worker secret + shell | **yes** | Derives the key for the encrypted `HttpOnly` session cookie and for sealing each person's portal token in D1. Must be at least 32 bytes. ⚠️ **Changing it invalidates every session and makes every stored portal token unreadable** — everybody signs in again, and everybody reconnects to the portal |
+| `TOOL_SOURCE_ORIGINS` | Worker + shell | **yes** | Comma-separated **exact origins** Intel is allowed to fetch tools from. ⚠️ An entry carrying a path or credentials is refused — and refused *per request*, so a bad value deploys cleanly and then answers `500` on everything |
+| `MCP_PORTAL_URL` | Worker + shell (for `doctor`) | no | The portal's Streamable HTTP MCP endpoint. Unset means the Tools area stays empty and Intel says so rather than reporting an error. ⚠️ Its origin must also appear in `TOOL_SOURCE_ORIGINS`; `intel doctor` fails when it does not — but only if it can see the value, so set it in the shell too before believing the check ran |
+| `ALLOW_INSECURE_OAUTH` | Worker | no | `true` permits an `http://` OAuth issuer. **Local development only.** Production issuers must be HTTPS |
+| `FLOW_RUN_STALL_TIMEOUT` | Worker | no | How long a run may stand on one step before it is ended as stalled, e.g. `30 minutes`. Too short is worse than the problem: it ends runs that were merely slow |
+| `INTEL_OPERATOR_TOKEN` | shell | for `reindex` | A short-lived bearer carrying the Gate capability `intel.admin`, used only by `intel reindex`. Not a Worker value and not a stored one |
+
+Gate OAuth uses PKCE and dynamic public-client registration; browser access tokens stay inside an
+encrypted `HttpOnly` cookie and are never exposed to the UI bundle or D1.
+
+## Branding: `intel.json`
 
 Put `intel.json` next to the customer project's `package.json`:
 
@@ -60,16 +237,105 @@ Put `intel.json` next to the customer project's `package.json`:
 The theme is plain CSS containing shadcn semantic tokens such as `--primary`, `--background`,
 `--sidebar`, and `--radius`. It is loaded after Intel's defaults, so customer tokens win without a
 component fork. Unknown config fields, missing assets, non-SVG branding, and incomplete language
-catalogs fail the build. A ready-to-copy starting point is included under
-`node_modules/@anchrd/intel-api/examples`.
+catalogs fail the build rather than producing a half-branded application. A ready-to-copy starting
+point is included under `node_modules/@anchrd/intel-api/examples`.
+
+The file is optional: `intel build` without it produces the default Intel branding.
+
+## Permission has two layers
+
+Access to anything in Intel is the **and** of two independent answers, and neither layer knows about
+the other. This is the single most common source of "it is broken" reports, because a person missing
+either layer sees roughly the same thing: a screen with nothing on it.
+
+### Layer one: the Gate capability
+
+What somebody may do *at all*, granted in Gate. `intel bootstrap` declares these; an administrator
+grants them.
+
+| Interface | Functions |
+|---|---|
+| `intel` | `use`, `admin` |
+| `nodes` | `read`, `create`, `write`, `share` |
+| `flows` | `read`, `create`, `write`, `publish`, `run`, `share` |
+| `tools` | `read`, `test`, `execute`, `admin` |
+| `mcp` | `connect` |
 
-Configure customer-specific `GATE_URL`, `INTEL_URL`, and the comma-separated exact-origin allowlist
-`TOOL_SOURCE_ORIGINS` in the deployment environment. The Worker requires `GATE_SERVICE_KEY` and a
-random `INTEL_SESSION_SECRET` of at least 32 bytes as Wrangler secrets. `ALLOW_INSECURE_OAUTH=true`
-is only supported for explicit local development; production OAuth issuers must use HTTPS. Gate OAuth
-uses PKCE and dynamic public-client registration; browser access tokens stay inside an encrypted
-`HttpOnly` cookie and are never exposed to the UI bundle or D1. Copy
-`node_modules/@anchrd/intel-api/examples/dev.vars.example` to `.dev.vars` for a local customer Worker.
+A missing capability answers **`403`**, and the screen says a permission is missing.
+
+### Layer two: the grant on the node
+
+What somebody may reach *in this tree*, granted in Intel on a node. Four verbs — `read`, `write`,
+`execute`, `share` — each granted on its own; none implies another. Seeing a process is deliberately
+separable from being allowed to start it.
+
+A grant on a **folder is inherited by everything beneath it**, including nodes created after the
+grant was made. That is the usual way to share a whole area.
+
+A missing grant answers **`404`**, never `403`: Intel gives the same answer for "there is no such
+node" and "not for you", so that a refusal never confirms that something exists.
+
+### Reading the empty screen
+
+| What you see | Which layer |
+|---|---|
+| A sentence naming a missing permission | Gate — layer one |
+| An empty tree, no error | Intel — nothing has been shared with this person yet |
+| A node that vanished between two visits | Its grant was revoked; revocation takes effect immediately, on every path, without a new login |
+
+A shared node appears in the recipient's navigation on its own, at the top level, without exposing
+the titles of the folders above it. Nobody has to be sent a link.
+
+### Three things about grants that surprise people
+
+- **A flow is not a node.** It carries no grants of its own; it inherits those of the folder it
+  lives in. Move a flow and you have changed who can reach it.
+- **`organization` + `execute` on a folder makes that folder a library**: flows from anywhere in the
+  tree may then call into it. The share dialog warns before the click, because the way back is
+  narrow — revoking is refused with `409 folder_execute_in_use` for as long as one of those callers
+  still calls.
+- **A grant can be complete and a flow still stop.** A run is re-authorized against the person
+  running it, so a flow in a shared folder that reads a document outside it stops for somebody who
+  cannot read that document. The answer names what was out of reach.
+
+## Export and import
+
+`GET /api/v1/nodes/export` writes the whole installation as this caller may read it; the same route
+under a node id exports that subtree. Import is the mirror:
+
+```bash
+curl -X POST https://intel.example.com/api/v1/nodes/import \
+  -H "authorization: Bearer $TOKEN" \
+  -H "content-type: application/zip" \
+  -H "idempotency-key: $(uuidgen)" \
+  --data-binary @bundle.zip
+```
+
+⚠️ **The idempotency key travels as a header.** The body is the zip itself, so there is nowhere else
+to put it. A request without the header is refused.
+
+Only nodes the caller may read enter the bundle; exporting a folder that is out of reach answers
+`404`. A fresh import answers `201`; a repeat of the same idempotency key answers `200` with the
+summary of the earlier import, because this request created nothing.
+
+⚠️ **A bundle carries only the current state of each node. Version history does not survive
+export → import.** A document with two versions comes back with one, and the earlier one is then
+retrievable nowhere. Everything else survives in full — nodes per kind, table content including a
+previous `redefine`, and attachments byte for byte. The loss is accepted and deliberate, but anyone
+moving an installation this way has to know about it **before** they start: after the import there
+is nothing to recover from.
+
+## Tools come from the portal
+
+Intel stores no tool permissions. The catalog is a **live `tools/list`** made with the requesting
+person's own token, never a mirrored table. Two consequences follow, and both are correct behaviour
+rather than bugs:
+
+- **Two people legitimately see different tools**, and the same person can see a different set
+  tomorrow. What is offered is whatever the portal answers for them at that moment.
+- **A tool can disappear between defining a flow and running it.** Published flows pin immutable
+  MCP schema fingerprints, so a changed or vanished tool is reported by name instead of being
+  silently substituted.
 
 The portal endpoint exposes RFC 9728 metadata. The Tools UI follows that metadata, dynamically
 registers with Gate or Cloudflare Access, and completes a separate PKCE flow — silently, with
@@ -82,106 +348,67 @@ both, every silent sign-in is refused and the Tools area shows "No access to the
 correct behaviour for somebody outside every policy, and a misleading one for a deployment that
 simply never enabled the setting.
 
+⚠️ **Somebody who carries the Gate capability `mcp.connect` at no service at all is not sent into
+that sign-in.** Gate answers that question service-wide, Intel asks it before the first redirect,
+and the person stays here reading a sentence that says whom to ask — rather than landing on the
+portal's own error page, outside Intel, with no way back but the browser's back button
+(anchrd/intel#434). Grant `mcp.connect` alongside the Intel capabilities to everyone who should
+reach Tools.
+
+Two things that guard deliberately does not do. It **names no server**: the answer is one bit, so
+somebody who may reach a different product's MCP server walks on and sees what the portal offers
+them. And it **promises nothing** — this is the part worth knowing before you grant: the portal
+decides on the `mcp` claim frozen into the ID token, and a running Cloudflare Access session carries
+the old claim for up to 24 hours. ⚠️ **Inside that window the old dead end is still open.** Somebody
+who was granted `mcp.connect` a minute ago is not refused here, walks on, and Access then decides on
+the claim state that produced the error page in the first place. The guard closes the trip that is
+certainly pointless; it does not close this one. The way out is the same as before — sign out of
+Access and in again, so a new ID token carries the new claim. (Not measured against a running
+installation; the claim lifetime is the documented one.) Against a Gate too old to answer the
+question at all, nobody is refused here.
+
+The Tools screen distinguishes four situations rather than calling all of them "no access": a
+refusal that really came back, an expired connection, a sign-in that broke, and a sign-in that was
+started and never answered. Only the first has no button, because only there is there nothing a
+second attempt would change.
+
 The only tool secret Intel stores is the resulting per-user access token for its own portal
-endpoint — one per person, never one shared operator token — sealed with a key
-derived from `INTEL_SESSION_SECRET` and kept in the `portal_tokens` table of your D1; provider
-credentials stay with the portal and never reach Intel. The Intel audience token is never forwarded
-to another OAuth resource.
-
-Where agents are deployed, bind the agent runtime Worker as the service `AGENT` — the reference
-deployment does, as `{ "binding": "AGENT", "service": "intel-agent" }`. The browser never talks to
-that Worker: it asks Intel under `/api/v1/agents/:agentId/*`, and Intel forwards the call over the
-binding with the Gate token of the person who asked, after checking `agents/run` and the agent's own
-resource ACL. So the runtime needs no second hostname, no CORS and no token in the browser bundle,
-and the page stays on one origin with one session cookie. The binding is optional: without it those
-routes answer `503 agent_runtime_not_configured` and nothing else changes. Deploy `@anchrd/intel-agent`
-before an Intel deployment that declares it — Wrangler refuses a binding to a service that does not
-exist yet.
-
-### What an agent costs, and what a model costs
-
-Two figures on the agent screen come from Cloudflare rather than from a table in this repository:
-what one agent has actually spent, per run and over the last 7 and 30 days, and what each offered
-model charges per million tokens. Both are read by Intel and never by the browser, because reading
-them needs an account credential and a credential does not belong in a single-page app.
-
-Set `AI_GATEWAY_ACCOUNT_ID` and `AI_GATEWAY_ID` — the same account and gateway the agent runtime
-uses — and add a Wrangler secret:
-
-```sh
-wrangler secret put AI_GATEWAY_READ_TOKEN   # Cloudflare API token: "AI Gateway: Read" + "Workers AI: Read"
+endpoint — one per person, never one shared operator token — sealed with a key derived from
+`INTEL_SESSION_SECRET` and kept in the `portal_tokens` table of your D1; provider credentials stay
+with the portal and never reach Intel.
+
+Intel unseals that token just in time and renews it shortly before it expires. **"Expired" above is
+what a failed renewal looks like**: a token that cannot be renewed is dropped rather than kept and
+retried with, the browser signs in again silently, and an MCP client repeats its own authorization.
+Two different causes end up there — a portal that issued no refresh token at all, which is a
+configuration problem, and a refresh that was rejected, which is not — and Intel logs which of the
+two it was, because the person only ever sees "sign in again". The Intel audience token is never forwarded to another OAuth
+resource. Durable Flow state contains no browser, Gate, or provider credentials.
+
+## The `intel` CLI
+
+```text
+intel prepare    Copy versioned D1 migrations to .intel/migrations
+intel bootstrap  Idempotently declare Intel interfaces in Gate
+intel build      Apply intel.json and atomically build the customer UI
+intel doctor     Check packages, environment, migrations, UI, and Gate interfaces
+intel reindex    Request a full authorized search-index rebuild
 ```
 
-⚠️ **This is not the runtime's `AI_GATEWAY_TOKEN`.** That one carries `AI Gateway: Run` and buys
-inference; this one only reads. Cloudflare cannot scope `AI Gateway: Read` to a single gateway — it
-is account-wide — which is worth knowing when the token is created and is not a reason against it.
-
-All three are optional. Without them the cost view says *"this installation has no AI Gateway read
-token"* and the model figures come off a built-in table that the interface marks as possibly out of
-date. Neither is an error state: an installation that gives Intel no Cloudflare credential is a
-supported installation. What is never done is show a zero — a missing figure and "cost nothing" are
-different answers, and only one of them is about money.
-
-The attribution comes from the agent runtime, which stamps `agentId` and `runId` into
-`cf-aig-metadata` on every model call. Runs made before that stamp existed appear with their token
-counts and without a cost, and say so.
-
-### Creating an agent creates its Gate application
-
-An agent acts as its own machine principal, so creating an agent node also creates the Gate
-Application it runs as, and archiving the node switches that Application off (`disabledAt`) rather
-than deleting it. Restoring the node switches it back on — the same principal, with the same ID and
-the same grants, which is the whole reason it is disabled rather than deleted.
-
-Both calls are made with **the bearer of the person asking**, not with `GATE_SERVICE_KEY`. Creating
-and switching a machine principal is administrative work in Gate: the caller needs the applications
-permission there on top of `knowledge:create` in Intel, and Gate audits the act under their name. A
-caller without it is refused with `agent_application_forbidden`, and a Gate that does not answer
-with `agent_application_unavailable` — in which case **no agent node is created at all**. There is
-deliberately no half-created agent to repair later; the request is simply repeated.
-
-**No response in Intel carries a credential.** The Application key Gate issues once, in plain text,
-is handed to the agent runtime over the `AGENT` service binding inside the same request, and the
-runtime encrypts it into that agent's Durable Object. Intel writes it to no table, no R2 object, no
-audit event, no log and no response body; the browser never sees it, and neither does an MCP client.
-The agent can run the moment it exists — there is no secret to set and no terminal to open.
-
-The handover carries the caller's own bearer plus `AGENT_HANDOVER_SECRET`, the one value Intel and
-the agent runtime share. The runtime cannot check Intel's resource ACLs, so that secret is what says
-the key came through Intel rather than off the open internet; without it on both Workers, creating an
-agent refuses by name.
-
-The handover happens **before** the node row, for the same reason the Gate call does: a runtime that
-will not take the key leaves no agent behind, and the freshly minted Application is switched off
-again. There is deliberately no half-created agent to repair later; the request is simply repeated.
-
-What still stays manual is the reach: grant the application its Gate roles and the resource grants
-on the Intel nodes it works with. That is what makes the agent's reach its own.
-
-`POST /api/v1/nodes/agents/<id>/rotate-key` (and the `agent_rotate_key` MCP tool) replaces the key:
-Intel asks Gate for a new one and hands it to the runtime the same way, the previous key stops
-working at once, and the answer names the principal and the moment rather than the key. It needs
-`knowledge:write` in Intel plus write access to the agent node, and the applications permission in
-Gate. It is the repair for a run that fails with `agent_key_missing`.
-
-**A written schedule is an armed one.** The agent runtime sets its alarm only when something tells
-it to look, so Intel calls `POST /agents/<id>/schedules/sync` over the same service binding after
-every definition write — creating an agent with schedules, and every save. That call goes down the
-handover route with the caller's own bearer beside `AGENT_HANDOVER_SECRET`, because arming an agent
-belongs to editing one and not to driving one. A save whose arming fails keeps the definition it
-wrote and answers `agent_schedules_not_armed`: the version is current, and saving again arms it.
-
-Intel keeps `applicationId` beside the node and returns it on every agent read. It names the
-principal without authenticating it, which is why it may be stored and shown while the key may not.
-An agent whose `applicationId` is `null` has no principal — imported, restored from a bundle, or
-created before this was automatic — and archiving it touches Gate not at all.
-
-The reference Wrangler deployment binds `DB`, `CONTENT`, `INDEXING`, `AI`, `SEARCH`, `FLOWS`, and
-`ASSETS`. Create `SEARCH` as a 1024-dimension cosine Vectorize index for the default multilingual
-Workers AI `@cf/baai/bge-m3` embedding adapter. `FLOWS` targets the exported
-`IntelFlowWorkflow` class. The portal speaks standard Streamable HTTP MCP; Intel caches schemas and
-unseals the caller's portal token from `portal_tokens` just in time, refreshing it shortly before
-use. Durable Flow state contains no browser, Gate, or provider credentials.
+All five run in the customer project directory — the one holding `package.json`, `intel.json` and
+`node_modules` — and read their configuration from that shell's environment.
+
+`intel reindex` needs `INTEL_URL` and a short-lived `INTEL_OPERATOR_TOKEN`. It asks the running
+installation to rebuild the derived indexes from D1 and R2 and returns as soon as the request is
+accepted; the work happens on the indexing queue. Reach for it after adding `AI` and `SEARCH` to an
+installation that ran without them, and after restoring an emptied Vectorize index.
+
+## Writing against the API directly
+
+Everything the browser does is available over HTTP and over the Intel MCP surface, against the same
+application services. If you write your own client, read
+[`@anchrd/intel-contract`](https://www.npmjs.com/package/@anchrd/intel-contract) first — it carries
+the schemas, and the handful of things that reliably cost a first-time caller an afternoon.
 
 ## Related packages
 

package/dist/adapters/db/db-flows.js

@@ -114,10 +114,19 @@ export function createFlowRepository(deps) {
     // An absent `parentId` asks for every visible flow; `null` asks for the root of the shared tree.
     // `IS ?` would collapse the two, so the two cases are separate SQL rather than one binding that
     // silently means both.
+    //
+    // ⚠️ The root is "the top of every region this actor may read", exactly as it is for nodes in
+    // `db.ts` (#429) — a flow whose folder the reader may not open would otherwise sit in a level
+    // nothing can ever open. `flowInSubtree` is what puts such a row in reach at all (its owner, or
+    // `intel/admin`), and this only decides which level it appears in; it can add no flow that
+    // predicate has not already allowed.
     const scopeOf = (parentId) => parentId === undefined
         ? { clause: "", bindings: [] }
         : parentId === null
-            ? { clause: "AND flow.parent_id IS NULL", bindings: [] }
+            ? {
+                clause: "AND (flow.parent_id IS NULL OR flow.parent_id NOT IN (SELECT id FROM allowed))",
+                bindings: [],
+            }
             : { clause: "AND flow.parent_id = ?", bindings: [parentId] };
     /**
      * One statement for the list the sidebar reads and for the bounded read the relation graph makes,

package/dist/adapters/db/db.js

@@ -109,6 +109,28 @@ export function createNodeRepository(deps) {
     const scopeCte = (scopeId) => scopeId === undefined ? "" : `,\n${descendantsCte}`;
     const scopeJoin = (scopeId) => scopeId === undefined ? "" : "\n          JOIN descendants ON descendants.id = n.id";
     const scopeBindings = (scopeId) => (scopeId === undefined ? [] : [scopeId]);
+    /**
+     * Which rows belong to the level being asked for.
+     *
+     * ⚠️ At the ROOT this is not `parent_id IS NULL` but "the top of every region this actor may
+     * read" (#429). A grant on a nested folder has no entry point otherwise: the way to it leads
+     * through a folder the recipient may not see, so the level it would appear in can never be
+     * opened, and the share stays usable only for whoever passes the URL around by hand. It is the
+     * same seed `listVisibleSubtree` already starts a whole-installation export from — written once
+     * more rather than differently.
+     *
+     * ⚠️ It says nothing about the way there. A row carries its own id and its own title, both of
+     * which `getVisible` already answers to this actor; the ancestors it hangs under are never read,
+     * named or counted, so the title of an unreadable parent folder stays where it was.
+     *
+     * ⚠️ Both `?` take the SAME value, and the second is not redundant: SQLite binds by position, so
+     * asking "was the root asked for" needs its own placeholder rather than a second reading of the
+     * first.
+     */
+    const levelPredicate = `(
+      n.parent_id IS ?
+      OR (? IS NULL AND n.parent_id NOT IN (SELECT id FROM allowed))
+    )`;
     /**
      * One statement for the level the tree reads and for the bounded read the relation graph makes,
      * so the visibility predicate cannot drift between them.
@@ -128,7 +150,7 @@ export function createNodeRepository(deps) {
       ) AS has_children
     FROM nodes n
     JOIN allowed ON allowed.id = n.id
-    WHERE n.parent_id IS ? AND (? = 1 OR n.archived_at IS NULL)
+    WHERE ${levelPredicate} AND (? = 1 OR n.archived_at IS NULL)
     ORDER BY CASE n.kind WHEN 'folder' THEN 0 ELSE 1 END, lower(n.title), n.id${bounded ? " LIMIT ?" : ""}`;
     /**
      * Everything archived that this actor may see, wherever it sits (#113).
@@ -150,7 +172,9 @@ export function createNodeRepository(deps) {
         async listVisible(actor, input) {
             const result = await deps.db
                 .prepare(input.archivedOnly ? archivedEverywhere : visibleChildren(false))
-                .bind(...readBindings(actor), ...(input.archivedOnly ? [] : [input.parentId, input.includeArchived ? 1 : 0]))
+                .bind(...readBindings(actor), ...(input.archivedOnly
+                ? []
+                : [input.parentId, input.parentId, input.includeArchived ? 1 : 0]))
                 .all();
             const rows = result.results ?? [];
             return {
@@ -161,7 +185,7 @@ export function createNodeRepository(deps) {
         async listVisibleBounded(actor, input) {
             const result = await deps.db
                 .prepare(visibleChildren(true))
-                .bind(...readBindings(actor), input.parentId, 0, input.limit)
+                .bind(...readBindings(actor), input.parentId, input.parentId, 0, input.limit)
                 .all();
             const rows = result.results ?? [];
             return { items: rows.map(mapNode), total: rows[0]?.total ?? 0 };

package/dist/flows/flows.js

@@ -417,16 +417,31 @@ export function createFlows(deps) {
         // The flow may have been built by someone with wider portal access. Naming the missing tools
         // before the first step beats failing halfway through with a portal error the user cannot
         // place — and the portal is where they can do something about it.
-        const missing = await deps.unavailableTools(actor, graphReferences(version.graph).tools);
-        if (missing.length) {
+        //
+        // ⚠️ ONE entry per missing TOOL, and the unit is the decision (#435). A problem list is a
+        // count, so the number of entries has to be the number of things left to do — and what a
+        // person does about a missing tool is grant it once in the portal, whether one step reaches
+        // for it or six. `graphReferences` already folds the graph down to distinct tool names, which
+        // is why a tool hanging off several steps arrives here once. Deliberately NOT one entry per
+        // step: the same portal click would then be listed several times, and the reader would go
+        // looking for several causes.
+        //
+        // The one consequence to know about: `start` throws the FIRST entry, so it now names the first
+        // missing tool rather than all of them — the same as it has always done for several
+        // unpublished sub-flows below. `validate` is what lists every reason at once, and it does.
+        for (const tool of await deps.unavailableTools(actor, graphReferences(version.graph).tools)) {
             problems.push({
                 status: 403,
                 code: "flow_tools_unavailable",
-                detail: toolStepDetail(missing),
+                detail: toolStepDetail([tool]),
             });
         }
         try {
-            await requireNodeAuthorized(actor, nodeFor(version, first), version.graph);
+            // ⚠️ Tree links only. The loop above has already asked about every tool in the graph — the
+            // first step's included — so the tool half of `requireNodeAuthorized` would answer a second
+            // time about the same names, which is the duplicate #435 reported. Every place that actually
+            // hands a step over still calls the whole of `requireNodeAuthorized`.
+            await requireTreeLinksAuthorized(actor, nodeFor(version, first), version.graph);
         }
         catch (error) {
             if (!(error instanceof IntelError))
@@ -543,26 +558,43 @@ export function createFlows(deps) {
     // `context` edge now, and a run never stands on one — so asking "is this node a tree link"
     // would ask about a node the run can no longer reach, and every check here would silently pass.
     async function requireNodeAuthorized(actor, node, graph) {
-        if (!node)
-            return;
-        const attached = graph.edges
+        await requireTreeLinksAuthorized(actor, node, graph);
+        await requireToolsAuthorized(actor, node, graph);
+    }
+    // The material of one step: what hangs off it on a `context` edge, as nodes.
+    function attachedNodes(node, graph) {
+        return graph.edges
             .filter((edge) => edge.kind === "context" && edge.source === node.id)
             .flatMap((edge) => graph.nodes.filter((candidate) => candidate.id === edge.target));
+    }
+    // ⚠️ The two halves are separate functions only because `collectRunProblems` needs the first one
+    // without the second (#435): every tool of the whole graph is already asked about there, so
+    // asking again for the first step reported the same tool twice. `requireNodeAuthorized` above is
+    // what every execution boundary calls, and it is still both halves — splitting the CHECK would
+    // be the drift this file spends a page warning about; splitting the FUNCTION is not.
+    async function requireTreeLinksAuthorized(actor, node, graph) {
+        if (!node)
+            return;
+        const attached = attachedNodes(node, graph);
         const wanted = [...new Set(treeLinkNodes({ ...graph, nodes: attached }).map(resourceIdOf))];
-        if (wanted.length) {
-            const reachable = await reachableNodes(actor, wanted);
-            if (reachable.length !== wanted.length) {
-                throw new IntelError(403, "flow_node_forbidden", treeLinkStepDetail(node.label, wanted.length - reachable.length));
-            }
+        if (!wanted.length)
+            return;
+        const reachable = await reachableNodes(actor, wanted);
+        if (reachable.length !== wanted.length) {
+            throw new IntelError(403, "flow_node_forbidden", treeLinkStepDetail(node.label, wanted.length - reachable.length));
         }
-        const toolNames = attached
+    }
+    async function requireToolsAuthorized(actor, node, graph) {
+        if (!node)
+            return;
+        const toolNames = attachedNodes(node, graph)
             .filter((candidate) => candidate.kind === "tool")
             .map((candidate) => candidate.configuration.toolName);
-        if (toolNames.length) {
-            const missing = await deps.unavailableTools(actor, [...new Set(toolNames)]);
-            if (missing.length) {
-                throw new IntelError(403, "flow_tools_unavailable", toolStepDetail(missing));
-            }
+        if (!toolNames.length)
+            return;
+        const missing = await deps.unavailableTools(actor, [...new Set(toolNames)]);
+        if (missing.length) {
+            throw new IntelError(403, "flow_tools_unavailable", toolStepDetail(missing));
         }
     }
     /**

package/dist/intel/intel.js

@@ -50,7 +50,32 @@ export function createIntel(deps) {
             if (!authorization) {
                 return context.redirect(`/auth/login?returnTo=${connectReturnTo}`);
             }
-            if (!authorization.can("tools", "read")) {
+            // ⚠️ Two questions, and both are answered BEFORE the first redirect leaves Intel (#434).
+            // `tools:read` is Intel's own door. `mcpPortalRefused()` is Gate's answer to a question Intel
+            // cannot ask itself: does this person carry `mcp.connect` at ANY service? Measured against the
+            // running installation, somebody without it was sent to `mcp.anchrd.sh/servers-callback` and
+            // landed on a white page saying "No allowed servers available, check your Zero trust policies"
+            // — a sentence written for an administrator, on a foreign domain, with not one clickable
+            // element on it.
+            //
+            // ⚠️ The helper, never the raw `mcpConnectAnywhere` beside it. The field has THREE states, and
+            // `undefined` means "this Gate is older than the question" (anchrd/gate#277), not "nowhere".
+            // `if (!authorization.mcpConnectAnywhere)` passes `tsc --noEmit` and `biome check` alike and
+            // would turn every reader of such an installation away without a word — the difference is a
+            // value, not a type, so nothing but this line prevents it (anchrd/gate#288).
+            //
+            // ⚠️ And it is deliberately NOT `can("mcp", "connect")`, which is what this guard was first
+            // built as: Gate trims `rules` to the ASKING service (GATE-64, least privilege), so that asks
+            // `intel.mcp.connect`. The portal is a door onto every server — somebody carrying
+            // `dwh.mcp.connect` reaches it and sees the dwh tools, and that spelling would have locked
+            // them out. The bit knows no service names, on purpose; it is one bit and carries no more.
+            //
+            // ⚠️ It spares the certainly wasted trip and promises no successful one: the portal decides
+            // on the `mcp` claim frozen into the ID token, and a running Access session carries the old
+            // claim for up to 24 hours. Somebody granted a minute ago is NOT refused here, walks on, and
+            // meets Access deciding on the very claim state this ticket is about — inside that window the
+            // old dead end is still open, and this guard does not close it. It closes the certain one.
+            if (!authorization.can("tools", "read") || authorization.mcpPortalRefused()) {
                 return context.redirect("/tools?connectError=permission_required");
             }
             try {

package/dist/nodes/nodes.types.d.ts

@@ -219,6 +219,7 @@ export interface BoundedChildren {
 export interface NodeService {
     list(actor: Actor, input: ListNodesInput): Promise<{
         items: Node[];
+        withChildren: string[];
     }>;
     /**
      * One level of the tree as this actor may see it, bounded: at most `limit` current children, and

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@anchrd/intel-api",
-  "version": "0.16.1",
+  "version": "0.18.0",
   "type": "module",
   "license": "UNLICENSED",
   "repository": {
@@ -42,8 +42,8 @@
     "typecheck": "tsc --noEmit"
   },
   "dependencies": {
-    "@anchrd/gate-sdk": "^0.7.0",
-    "@anchrd/intel-contract": "^0.14.0",
+    "@anchrd/gate-sdk": "^0.15.0",
+    "@anchrd/intel-contract": "^0.15.0",
     "@cfworker/json-schema": "^4.1.1",
     "@modelcontextprotocol/sdk": "^1.30.0",
     "fflate": "^0.8.3",