@zackbart/connecta 0.6.0 → 0.7.0

package/CHANGELOG.md

@@ -2,6 +2,409 @@
 
 All notable changes to this package are documented here.
 
+## 0.7.0 — 2026-07-27
+
+0.7.0 is the surface settlement: one release that finishes moving connecta's
+public shape so that it can stop moving. It arrives from two directions at once.
+A controlled qualification pass against 0.6.1 (issue #86) measured four ways a
+deployment could be worn down — an unbounded cache, an enumerable callback, a
+session leak, and a credential that reports itself usable and then isn't — and
+those are fixed here. Meanwhile the config surface had accumulated eleven flat
+tuning fields, the reference manual had reached 2,600 lines, and the operator UI
+was one page pretending to be three. Those are settled here too, and settling
+them breaks things, which is the argument for doing it in one release instead of
+spreading it across three.
+
+**What breaks.** Three things, and each is a compile error or a loud refusal
+rather than a behavior change you have to notice on your own. Every flat 0.6.x
+tuning option is removed in favour of four grouped ones. A connector that
+implements `finishAuth` without `verifyState` can no longer complete OAuth. And
+`/`, `/credentials`, and `/activity` are now core-owned routes, so a connector
+`handleRequest` that served any of them is shadowed. The first is mechanical:
+`createConnecta` throws once, before it reads anything else, naming every legacy
+path it found and its replacement.
+
+**What a deployment can ignore.** The defaults and runtime behavior of every
+config option are unchanged — only the paths moved. `/ui` bookmarks keep working
+through a permanent redirect, and all 17 documentation anchors still resolve
+even though the manual is now nine files. If you configure connecta with a
+literal object, do not paginate a downstream MCP server, and write your own
+connectors against `remoteMcp`/`api` rather than hand-rolling `finishAuth`, the
+whole migration is a mechanical rewrite of one config block.
+
+### Breaking
+
+- **`ConnectaConfig` tuning is grouped by subsystem for 0.7.0** (issue #28).
+  The flat 0.6.x options are removed rather than deprecated: TypeScript rejects
+  them, and JavaScript callers get one fail-fast error from `createConnecta`
+  listing every legacy own property it finds and its replacement. Supplying
+  both an old and a new path is still an error; there is no precedence rule or
+  compatibility alias. Migrate configuration as follows:
+
+  | 0.6.x | 0.7.0 |
+  |---|---|
+  | `activity` | `activity.store` |
+  | `activityReadGate` | `activity.readGate` |
+  | `activityDeploymentId` | `activity.deploymentId` |
+  | `credentialEncryptionKey` | `credentials.encryptionKey` |
+  | `credentialHealth` | `credentials.health` |
+  | `toolCacheTtlSeconds` | `discovery.catalogTtlSeconds` |
+  | `persistToolCatalog` | `discovery.persistCatalog` |
+  | `toolCatalogStaleSeconds` | `discovery.staleCatalogSeconds` |
+  | `probeTimeoutMs` | `discovery.probeTimeoutMs` |
+  | `defaultToolTimeoutMs` | `calls.defaultTimeoutMs` |
+  | `maxResultBytes` | `calls.maxResultBytes` |
+
+  For example:
+
+  ```ts
+  // 0.6.x
+  createConnecta({
+    connectors,
+    storage,
+    activity,
+    credentialEncryptionKey: env.CONNECTA_CREDENTIAL_KEY,
+    toolCacheTtlSeconds: 300,
+    defaultToolTimeoutMs: 30_000,
+    maxResultBytes: 50_000,
+  });
+
+  // 0.7.0
+  createConnecta({
+    connectors,
+    storage,
+    activity: { store: activity },
+    credentials: { encryptionKey: env.CONNECTA_CREDENTIAL_KEY },
+    discovery: { catalogTtlSeconds: 300 },
+    calls: { defaultTimeoutMs: 30_000, maxResultBytes: 50_000 },
+  });
+  ```
+
+  The defaults and runtime behavior of each option are unchanged. Connector
+  definitions keep their per-connector `maxResultBytes`, and structural seams
+  such as `storage`, `auth`, `connectors`, `toolkits`, and `executor` remain at
+  the top level.
+- **A connector implementing `finishAuth` without `verifyState` can no longer
+  complete OAuth** (issue #62). Such a connector previously exchanged the
+  authorization code with no CSRF guard at all — connecta had no way to
+  establish that it had started the flow being completed. The callback now
+  refuses with the same opaque 400 as every other refusal, exchanges no code,
+  and logs one operator-grade line naming the connector and the missing hook.
+  `verifyState` stays optional in the type system and is required in practice
+  wherever `finishAuth` is present. The shipped `remoteMcp` OAuth provider has
+  always implemented it, so this reaches hand-written connectors only.
+- **`/`, `/credentials`, and `/activity` are now core-owned routes** (issue
+  #57). They previously fell through to connector `handleRequest` and then to a
+  404, so a connector serving any of the three is now shadowed without warning.
+  `GET /` returns the operator shell where 0.6.1 returned 404, and non-GET on
+  those routes and on `/ui` returns 405 rather than falling through.
+
+### Added
+
+- **`Connector.closeScope?(ctx)`** (issue #66) — an optional, best-effort seam
+  for releasing whatever a connector opened for one scope. Called at most once,
+  inside a fixed 100 ms completion window: a hook that is absent, throws, or
+  never settles cannot replace, corrupt, or delay the result of the operation
+  that triggered it. `remoteMcp` implements it, tearing down cached and
+  half-open sessions and fencing late connection races. Connectors that do not
+  implement it are unaffected.
+
+### Changed
+
+- **The operator UI is now a direct-linkable three-page set** (issue #57):
+  Connections at `/`, Credentials at `/credentials`, and Activity at
+  `/activity`. `GET /ui` remains as a permanent `308` compatibility redirect to
+  `/`. The pages use semantic navigation, path-specific titles, direct
+  load/refresh/bookmark and Back/Forward behavior, and the same restrained
+  responsive connector-ledger design. They are three routes over one
+  no-build-step client implementation, not copied applications.
+- **Every canonical operator route serves the same open, data-free shell.** The
+  nonce CSP, framing denial, referrer policy, HSTS behavior, content-type
+  protection, URL gates, and escaping guarantees formerly attached to `/ui`
+  apply equally to `/`, `/credentials`, and `/activity`. Deployment data remains
+  behind the unchanged private APIs: `/ui/data`, `/ui/activity`, and
+  `/ui/credentials/<connectorId>[/test]`.
+- **Vault controls moved out of connector cards and into Credentials.**
+  Connections remains read-only status; Credentials preserves Add/Replace/Test/
+  Remove, masked metadata, stored-shape recovery, and live feedback without ever
+  returning a secret. Its explicit capability states cover eligible Clerk
+  operators, bearer-only sessions, missing vaults, and deployments with no
+  credential slots without broadening the existing Clerk/user-id/toolkit/
+  same-origin mutation boundary. Activity likewise renders an explicit
+  not-configured state when `activity.store.list` is absent. OAuth result pages
+  now return to `/`.
+- **The reference manual is nine documents instead of one** (issue #61).
+  `docs/documentation.md` had reached 2,600 lines; it is now a compatibility
+  index that preserves all 17 `#N-…` anchors, so every existing deep link from
+  the README, the examples, and source comments still resolves. The prose moved
+  to `architecture`, `meta-tools`, `connectors`, `auth`,
+  `storage-and-credentials`, `operations`, `code-mode`, `operator-ui`, and
+  `toolkits`. A new `npm run check:docs` validates local links, fragments,
+  heading slugs, and the legacy-anchor manifest, and runs first in
+  `npm run check`. Contributor-facing only: the published package is unchanged
+  and docs still ship in no tarball.
+
+### Fixed
+
+- **`clerkAuth`'s identity-decision cache is bounded at 1,024 identities per
+  instance** (issue #70). It never evicted: every distinct identity that reached
+  the gate was retained for the lifetime of the isolate. `allowedDomains` made
+  that attacker-reachable on an open-signup Clerk instance, since an
+  authenticated-but-denied identity is still a cache entry — 100,000 denied
+  identities measured at roughly 11.96 MiB retained, with the oldest entry still
+  present. Eviction is LRU, so a small active set stays hot, and the only thing
+  evicting an entry can cause is a fresh check against Clerk: it can never turn
+  a denial into an admission, and it cannot extend an allow past its TTL, since
+  a cache hit re-inserts the same record rather than restamping it. The bound is
+  deliberately fixed rather than an operator knob, and the ~60 s allow / ~30 s
+  deny windows are unchanged.
+- **`/oauth/callback/<id>` no longer lets an unauthenticated caller enumerate
+  configured connector ids** (issue #62). 0.6.1 answered `404 Unknown connector
+  "<id>"` for an id it did not recognise and a distinct 400 for a state
+  mismatch, which made the callback a free directory of every connector a
+  deployment had configured. Unknown ids, non-OAuth connectors, missing or
+  mismatched state, an absent verifier, and a throwing verifier now return one
+  byte-identical 400 — same status, same body, same headers. They also cost the
+  same single storage read, so the clock cannot sort the ids the body refuses to
+  name. That is cost equalisation rather than constant time, and the
+  documentation says so and enumerates what stays distinguishable. The precise
+  diagnosis moved to the operator log, bounded and escaped.
+- **Probe-opened downstream MCP sessions are closed instead of abandoned**
+  (issue #66). Credential-health sweeps, `list_connectors({ probe: true })`, and
+  `/ui/data` each opened downstream sessions and left them for the provider to
+  age out — measured at 200 probes opening 200 sessions and explicitly closing
+  zero, made continuous by the periodic sweep. Closing now ends the session
+  rather than merely dropping the local client: `remoteMcp` issues the
+  specification's `DELETE` carrying `Mcp-Session-Id` *before* the close that
+  would otherwise abort it, feature-detected and best-effort, while a stateless
+  downstream issues none. A downstream that refuses, errors, or never answers is
+  closed anyway. One consequence worth knowing: a probing `list_connectors` now
+  runs on its own scope rather than the request's — it has to, since a request
+  scope cannot be closed — so a request that probes *and* then calls the same
+  connector opens two downstream sessions where it opened one. Per-request
+  `/mcp` scopes are unchanged.
+- **A credential stored under an older declaration no longer reports itself
+  usable and then fails** (issue #69). `/ui`, the credential Test route, and
+  credential health each interpreted the stored shape slightly differently, so a
+  credential could render as configured and testable and then answer 409 with a
+  misleading configure-first message. One pure classifier now answers for all
+  three, and it asks about *containment* rather than equality: a stored set is
+  fine as long as it holds every field the declaration currently names. A
+  missing declared field is drift — which covers every case issue #69 reported,
+  including a rename and a swap between the single-value and named shapes in
+  either direction — and drift is never auto-migrated. Credentials keeps Replace
+  and Remove and hides Test, the test route answers 409 without calling a hook,
+  and health records an explicit error rather than a fabricated verdict, checked
+  before the freshness gate so a stale `ok` cannot mask a redeploy. Leftover
+  keys are explicitly *not* drift: dropping a field from a declaration leaves
+  its secret in the vault, but every accessor keeps returning the right value
+  and every call keeps working, so calling that drift would tell an operator to
+  re-enter a credential that works and that many providers will not reissue in
+  readable form. Credentials prints one non-blocking line naming the leftovers
+  instead. A repeated drift verdict is charged to the freshness budget rather
+  than rewritten on every sweep in every isolate.
+- **A paginated downstream MCP server no longer loses everything after its
+  first `tools/list` page** (issue #77). `tools/list` is cursor-paginated in the
+  MCP specification and the SDK's `Client.listTools()` returns one page,
+  `nextCursor` included, without following it. `remoteMcp()` called it once, so
+  a downstream that paged its catalog was split in half without complaint:
+  page-one tools were counted, searchable, describable and callable, and every
+  tool after them appeared not to exist — missing from `list_connectors` counts,
+  `search_tools`, `describe_tools`, and address resolution alike, with nothing
+  in any log saying a page had been left behind. That is the wrong failure for
+  a product whose premise is progressive discovery over large tool catalogs,
+  since large catalogs are exactly the ones that paginate. A catalog refresh now
+  walks the whole cursor chain on the same request-scoped client before the
+  result is indexed. Cursors are handed back byte-for-byte and never parsed,
+  rewritten, or persisted; an empty-string `nextCursor` is treated as *present*
+  (pagination ends on an absent one, and a truthiness check there would truncate
+  the catalog silently). A downstream that advertises no `nextCursor` is
+  unchanged in both directions: one request, sent with no `cursor` param, and
+  byte-identical tool definitions. Tools are deduplicated by name, first page
+  wins, so an unstable cursor that overlaps pages cannot inflate `toolCount`,
+  double a `search_tools` row, or churn the registry into a persistence write
+  per refresh. A `nextCursor: null` — a common JSON idiom for end-of-pagination
+  that the MCP result schema does not accept — is now reported as a named
+  nonconformance instead of a raw Zod dump. Any page that fails fails the whole
+  refresh rather than publishing a prefix, so the registry keeps serving the
+  last complete catalog through its existing stale fallback.
+
+  The walk is bounded on **tools**, not pages. A page ceiling is the wrong
+  dimension — the server picks the page size, so N pages is N × a number
+  connecta cannot observe, and the common conformant idiom of "advertise a
+  cursor whenever the page came back full, then serve one empty page to
+  terminate" means a well-behaved 10,000-tool server paging at 100 spends 101
+  requests. So a refresh now fails immediately on a cursor handed back twice,
+  fails on two consecutive pages that add nothing while still promising more
+  (one is legal — that empty terminator), caps what it accumulates at
+  `MAX_TOOLS` (100,000, the top of the catalog envelope issue #82 benchmarks),
+  and keeps `MAX_TOOL_PAGES` (raised to 10,000) only as an unreachable runaway
+  backstop. These bounds make an unterminating walk finite and report it as a
+  connector error; they do not cancel one that a probe deadline has already
+  abandoned, because `withTimeout` bounds the caller's wait rather than the
+  work.
+
+- **Downstream output-schema validation now covers every page of a paginated
+  catalog, not just the last one** (issue #77). The MCP SDK's
+  `Client.listTools()` rebuilds its output-schema validators and task-support
+  sets from each page it receives, clearing them first — so walking the cursor
+  chain left the request-scoped client validating the final page alone. A
+  `call_tool` against any earlier-page tool then found no validator and
+  silently skipped both the "declared an `outputSchema` but returned no
+  `structuredContent`" check and the structured-content validation, and lost
+  the required-task guard with them. Enforcement depended on which page a tool
+  happened to land on. A completed walk now re-primes that cache once from the
+  full aggregated catalog; a test asserts the pinned SDK still provides the
+  method it reaches for, so a future bump fails CI rather than quietly
+  restoring the gap.
+
+## 0.6.1 — 2026-07-26
+
+A patch release: three bug fixes and a documentation overhaul. No new
+configuration, no new exported types, no change to the published API surface —
+`ConnectaConfig` and every type are byte-identical to 0.6.0. Nothing here is
+breaking. But two of the fixes *withdraw* behavior a deployment could have been
+relying on by accident, and in both cases withdrawing it is what the fix is, not
+a decision taken alongside it:
+
+- **A `uiAuth.signInUrl` or `signUpUrl` that is not an absolute `https:` URL is
+  now dropped** rather than handed to `Clerk.load` (issue #56). Relative paths,
+  `http:`, protocol-relative, `javascript:` and `data:` values all fall together,
+  under the same gate `frontendApiUrl` has always passed. A rejected value reaches no part of the rendered page: the key is
+  simply absent from `/ui`'s inline `AUTH` object, so Clerk falls back to its own
+  default exactly as it does for an unset field, and `/ui` still renders and
+  still signs operators in. Construction logs one warning naming each dropped
+  field. What belongs in these fields is a hosted Account Portal address
+  (`https://accounts.<domain>`, `https://<slug>.accounts.dev`), so a deployment
+  that noticed this at all was pointing operators at a sign-in page connecta does
+  not host.
+- **The declared credential shape now picks the credential test hook, on every
+  surface** (issue #55). Two of the four shape/hook combinations change behavior.
+  A **single-value `credential` declaring both hooks** now runs
+  `testCredential(value)` with the raw string, where the test route previously
+  preferred `testCredentials` and handed it the vault's reserved `{ value }` map
+  — the single-value hook now receives the string it was written to expect
+  instead of a one-entry map named after a storage detail. A **single-value
+  `credential` declaring only `testCredentials`** is no longer tested or probed
+  at all: /ui renders no Test button, a direct POST answers 400 naming the
+  mismatch, `createConnecta` warns at construction, and the liveness sweep added
+  in 0.6.0 reports it `not_checkable` instead of calling a hook with a shape its
+  connector never declared. Both of those are the fallback order that *was* the
+  bug, so there is no version of this fix that preserves them. Named `fields`
+  with `testCredentials`, or with both hooks, behave exactly as they did.
+
+One textual delta reaches every deployment and no configuration turns it off, so
+anything snapshotting agent-facing error text will diff: `resolveSkill`'s two
+`Available:` branches now say `Available skills:` like the other three (issue
+#50).
+
+The docs are substantially reorganized, and one of the changes ships to npm:
+readers of the package page get a completely different README. See Changed. The
+next intentional breaking release stays reserved for issue #28.
+
+### Changed
+
+- **The README is rewritten around the problem connecta solves** (345 lines to
+  196). It had become a condensed reference manual — roughly 60% code blocks and
+  option tables, every one of them duplicated in `docs/documentation.md` or
+  `examples/` — with a single paragraph explaining why connecta exists. npm
+  readers see only this file, so it was the worst place for the duplication and
+  the best place for the argument. It now runs as prose: the context-window and
+  per-client configuration costs of connecting an agent to N MCP servers, what
+  connecta does about them, why it is shaped the way it is (config as code, a
+  server-side credential vault with liveness checks, fail-closed read-only,
+  toolkits and who is admitted to them, payload-free activity, a read-only
+  operator dashboard), when *not* to use it, one minimal Node quickstart, and
+  links out. Removed from it and unchanged in `docs/` and `examples/`: the
+  nine-row signature-level meta-tool table (the nine names now appear in prose
+  with their purposes), the Worker quickstart, and the toolkits, code-mode,
+  credentials, activity and branding blocks with their option semantics.
+  **Links out of the README are now absolute GitHub URLs**, because `docs/` does
+  not ship in the package and relative links are dead on npmjs.com. This is the
+  only change in the release that reaches npm as content rather than code.
+- **`docs/design.md` is retired into a new `docs/decisions.md`.** design.md was
+  mostly a worse copy of documentation.md, and it had gone stale in at least one
+  load-bearing way: it claimed the Clerk adapter passes `authorizedParties:
+  [connectaOrigin]` to `authenticateRequest`. It does not, and must not — OAuth
+  access tokens may be JWTs with no `azp` claim, and Clerk rejects `azp ===
+  undefined` when that option is set, so passing it would reject every MCP
+  client; the `azp` pin is applied by hand after verification, for session tokens
+  only. decisions.md records that as a **rejected alternative** rather than
+  repeating the wrong version. It answers two questions documentation.md does
+  not: "may I build X?" (non-goals, rejected alternatives) and "must my change
+  preserve Y?" (invariants) — collecting the invariants documentation.md states
+  but never gathers as pointers rather than duplicated prose, alongside the four
+  that lived only in design.md: no runtime admin, nothing request-bound surviving
+  a request, single tenant, and fail-closed read-only. Two facts moved into
+  documentation.md §13 instead, next to the code they explain. Inbound references
+  in the README, documentation.md and the Docker example follow; **any external
+  link to `docs/design.md` is now dead**, and the 0.x CHANGELOG mentions of it
+  stay as historical record. The credential-test invariant added by issue #55
+  also gained the qualifier §17 already carried: a connector whose shape and hook
+  mismatch carries no verdict *from a credential hook*, but is still probed
+  through `status()` if it declares one, since that question never involves the
+  shape.
+- **CLAUDE.md is an agent brief rather than a policy stub.** It now states what
+  gates "done" (`npm run check`, and what that runs), the two-document map
+  (documentation.md as reference manual, decisions.md as non-goals and
+  invariants) with the warning that its section numbers are linked from source
+  comments, the two CI-enforced boundaries that are invisible from inside a
+  single file (import-graph purity, the published surface) as where-new-code-goes
+  guidance, the `WORKERS_SUITES` allowlist trap that silently skips a portable
+  suite left out of it, and the commit, CHANGELOG and release conventions. The
+  roadmap-lives-in-GitHub-issues policy is unchanged, verbatim.
+
+### Fixed
+
+- **One rule decides how a credential is tested, and three copies of it no
+  longer disagree** (issue #55). `/ui` offered a Test button from the mere
+  presence of a hook (`testCredential || testCredentials`); the test route made
+  its own different choice — prefer `testCredentials`, else `testCredential` on
+  the vault's reserved `value` field; and 0.6.0's credential-health prober added
+  a third copy of that same preference order. They disagreed in both mismatch
+  shapes, and the visible bug was the ugliest one: a connector declaring named
+  `credential.fields` with only `testCredential` answered **409 "configure the
+  credential before testing it" on a fully configured credential**, blaming the
+  operator for connecta's own hook selection. `credentialTestRule` in
+  `src/credentials.ts` is now the single source of truth — `buildUiData`'s
+  `testable` flag, the test route's hook selection, the credential-health
+  prober's `testHookFor`/`isCheckableConnector`, and a new construction-time
+  warning all read it, so they cannot drift apart again. **The declared shape
+  picks the hook and the other one is never substituted**: named fields are
+  tested by `testCredentials` with the whole set, a single value by
+  `testCredential` with the raw string. A connector implementing only the
+  mismatched hook is not testable rather than testable-by-accident — no button, a
+  400 sharing its wording with the boot warning, and `not_checkable` to the
+  prober, which carries no verdict rather than an invented one (handing
+  `testCredential` a `value` that named fields never wrote would test the empty
+  string and record a confident `auth_required` about a credential nothing
+  examined). `status()` is unaffected: a mismatched connector that implements it
+  is still probed through it. The two behavior deltas this produces are stated
+  above. Recorded as an invariant in `docs/decisions.md`, and stated in
+  documentation.md §4, §7, §14 and §17.
+- **`uiAuth.signInUrl` and `signUpUrl` are gated like every other URL that
+  reaches the browser** (issue #56). They were the residual — operator config
+  that arrives in the page as a *navigation target* rather than an attribute, so
+  no gate covered them and §14's URL-position invariant had to be read with an
+  exception beside it. Both are serialized into `/ui`'s inline `AUTH` object and
+  handed to `Clerk.load`, which navigates to them when an operator signs in. They
+  now pass the same gate `frontendApiUrl` does, so `isSafeScriptSrcUrl` — named
+  for the one position it used to guard — becomes `isSafeHttpsUrl`, one predicate
+  for all three `uiAuth` URLs. Absolute `https:` only, for the typed fields as
+  for the derived one: the loose carve-outs buy nothing real, since `http:` would
+  carry a sign-in over cleartext and a path relative to this origin is
+  meaningless because connecta hosts no sign-in page of its own. The drop warning
+  decides "did the operator mean to supply this?" through the same `isSetUrlValue`
+  helper `branding` uses, so a blank-string or falsy non-string value cannot warn
+  on one path and stay silent on the other. **§14 now states the invariant over
+  every URL position — attribute and navigation target — with no carve-out left.**
+- **`resolveSkill` enumerates its list under one label** (issue #50). Three
+  branches said `Available skills:` and two said `Available:`, so which one an
+  agent saw depended on the branch it hit rather than on any difference in
+  meaning. Agent-facing error text is interface, and two labels read as two
+  concepts where there is one. Both stragglers now say `Available skills:`, and a
+  test walks all five error branches so reverting any one of them fails.
+
 ## 0.6.0 — 2026-07-26
 
 A feature release that makes two of 0.5.0's mechanisms protective rather than