@tangle-network/agent-integrations 0.25.0 → 0.25.2

package/README.md

@@ -1,23 +1,31 @@
 # @tangle-network/agent-integrations
 
-Vendor-neutral integration contracts for agent apps, sandboxes, and generated
-software that need user-authorized access to external systems.
+Integration infrastructure for agent products, sandbox apps, and generated
+software.
 
-The package standardizes connector catalogs, user connections, scoped sandbox
-capabilities, action invocation, trigger events, provider adapters, and
-first-party connector adapters. Product code can route through hosted gateways,
-custom runtimes, or first-party adapters without changing the agent-facing tool
-contract.
+Use this package when users connect external accounts and agents or apps need
+controlled read/write access to those accounts. It gives products one stable
+contract for connector discovery, OAuth/API-key connections, scoped sandbox
+capabilities, action invocation, workflow triggers, approval, audit,
+healthchecks, and provider/runtime adapters.
+
+The product keeps ownership of UI, tenant policy, persistence, and secret
+storage. `agent-integrations` keeps the runtime contract stable so generated
+apps and agents do not depend on a specific OAuth broker, workflow vendor, or
+provider SDK.
 
 ## Contents
 
 - [What It Provides](#what-it-provides)
 - [Architecture](#architecture)
 - [Install](#install)
+- [Quick Start](#quick-start)
 - [Core Primitives](#core-primitives)
 - [Catalog Registry](#catalog-registry)
+- [Product Adoption](#product-adoption)
 - [Provider Strategy](#provider-strategy)
 - [Executable Coverage](#executable-coverage)
+- [Product Hub Ownership](#product-hub-ownership)
 - [Examples](#examples)
 - [Security Model](#security-model)
 - [Development](#development)
@@ -25,7 +33,7 @@ contract.
 ## What It Provides
 
 - A normalized connector/action/trigger catalog.
-- First-class Tangle integration contracts for every catalog connector.
+- Tangle integration contracts for every catalog connector.
 - User-owned connection records that reference secrets without storing raw
   credentials in public shapes.
 - Short-lived capability tokens for sandbox-safe access to a subset of a user's
@@ -37,13 +45,13 @@ contract.
   runtimes, and internal connector registries.
 - A first-party `ConnectorAdapter` boundary for direct provider execution.
 - A declarative REST adapter factory for promoting REST APIs from reviewed specs.
-- A broad coverage catalog for planning hundreds of integrations without
-  pretending every catalog item is executable.
+- A broad catalog for discovering hundreds of integrations while keeping
+  executable backend state explicit.
 - A canonical registry that deduplicates overlapping catalogs, keeps support
   tiers explicit, and reports auth/category conflicts.
-- App/agent manifests, grants, and sandbox bundles so Builder, generated apps,
-  domain agents, Blueprint Agent, and executor-backed runtimes can reuse the
-  same user-owned connections safely.
+- App/agent manifests, grants, and sandbox bundles so generated apps, domain
+  agents, and executor-backed runtimes can reuse the same user-owned
+  connections safely.
 - Workflow trigger installation and normalized event dispatch for non-agent UI
   automation, sync jobs, webhooks, and product workflows.
 - Approval persistence, audit events, healthchecks, credential resolution,
@@ -57,8 +65,10 @@ contract.
 ## Architecture
 
 ```txt
-connector catalog
+connector contract
   -> user connection
+  -> app/agent manifest
+  -> grant
   -> scoped capability
   -> policy decision
   -> provider/action invocation
@@ -80,6 +90,63 @@ Main boundaries:
 pnpm add @tangle-network/agent-integrations
 ```
 
+## Quick Start
+
+```ts
+import {
+  buildDefaultIntegrationRegistry,
+  buildIntegrationToolCatalog,
+  createIntegrationRuntime,
+  createPlatformIntegrationPolicyPreset,
+  InMemoryConnectionStore,
+  IntegrationHub,
+} from '@tangle-network/agent-integrations'
+
+const registry = buildDefaultIntegrationRegistry({
+  tangleCatalogRuntimeExecutable: false,
+})
+
+const hub = new IntegrationHub({
+  providers: [/* native, gateway, or catalog-runtime providers */],
+  store: productConnectionStore ?? new InMemoryConnectionStore(),
+  capabilitySecret: process.env.INTEGRATION_CAPABILITY_SECRET!,
+  policy: createPlatformIntegrationPolicyPreset(),
+})
+
+const runtime = createIntegrationRuntime({
+  hub,
+  grants: productGrantStore,
+})
+
+const tools = buildIntegrationToolCatalog(registry.connectors)
+```
+
+For a generated app or sandbox:
+
+```ts
+const resolution = await runtime.resolveManifest(manifest, user)
+
+if (resolution.missing.length > 0) {
+  // Show connect UI using IntegrationSpec renderers.
+}
+
+await runtime.createGrants({
+  manifest,
+  owner: user,
+  grantee: { type: 'app', id: manifest.id },
+})
+
+const bundle = await runtime.buildSandboxBundle({
+  manifestId: manifest.id,
+  subject: { type: 'sandbox', id: sandboxId },
+  ttlMs: 15 * 60_000,
+})
+```
+
+Generated code calls your product integration endpoint with the scoped
+capability bundle. It never receives provider refresh tokens, API keys, or raw
+OAuth credentials.
+
 ## Core Primitives
 
 | Primitive | Purpose |
@@ -129,9 +196,9 @@ pnpm add @tangle-network/agent-integrations
 
 ## Catalog Registry
 
-Every catalog connector has a first-class Tangle contract. Native adapters and
-package runtimes are implementation backends behind that contract; product code
-should route through `IntegrationHub` either way.
+Every catalog connector has a Tangle contract. Native adapters, hosted gateways,
+and package runtimes are implementation backends behind that contract; product
+code should route through `IntegrationHub` either way.
 
 Use `buildDefaultIntegrationRegistry()` before creating tool catalogs or
 connection pickers. It produces one canonical connector per integration,
@@ -144,16 +211,16 @@ catalogOnly < setupReady < gatewayExecutable < firstPartyExecutable < sandboxExe
 
 Use `buildDefaultIntegrationRegistry({ tangleCatalogRuntimeExecutable: true })`
 when the Tangle catalog runtime is deployed and should be exposed as executable
-tools. These states describe the backend currently wired into a product, not
-whether the connector has a first-class Tangle contract.
+tools. These states describe the backend currently wired into a product. They
+do not change the connector contract.
 
 See [Catalog Registry](./docs/catalog-registry.md).
 
-## App And Agent Grants
+## Product Adoption
 
 Use `IntegrationManifest` for any app or agent that needs integrations:
-Agent Builder-generated apps, tax/legal/GTM/creative agents, Blueprint Agent
-sandboxes, and executor-backed workflows all use the same shape.
+generated apps, domain agents, sandbox agents, workflow apps, and
+executor-backed runtimes all use the same shape.
 
 ```ts
 const runtime = createIntegrationRuntime({ hub, grants })
@@ -180,7 +247,7 @@ Generated app code should use the tiny client instead of raw provider tokens:
 
 ```ts
 const integrations = createTangleIntegrationsClient({
-  endpoint: 'https://id.tangle.tools',
+  endpoint: 'https://integrations.example.com',
   env: process.env,
 })
 
@@ -204,6 +271,19 @@ That installs provider trigger subscriptions against the user's connection and
 lets the product dispatch normalized events to UI workflows, sync jobs, or
 agent runs.
 
+For a full product checklist, see
+[External Product Integration](./docs/external-product-integration.md).
+
+## Product Hub Ownership
+
+Use a hosted hub when multiple apps intentionally share identity, billing,
+consent, and connection custody. Use a product-owned hub when a standalone
+deployment needs its own customer boundary, OAuth branding, vault, policy, or
+data residency. Both modes use the same package contracts.
+
+See [Product Hub Ownership](./docs/product-hub-ownership.md) for the
+deployment model and launch gates.
+
 ## Provider Strategy
 
 The package deliberately avoids vendor lock-in.

package/docs/adapter-triage.md

@@ -63,7 +63,7 @@ import {
 
 const provider = createTangleCatalogExecutorProvider({
   executeAction: createTangleCatalogHttpExecutor({
-    endpoint: 'https://id.tangle.tools/integration-runtime',
+    endpoint: 'https://integrations.example.com/integration-runtime',
     secret: process.env.TANGLE_CATALOG_RUNTIME_SECRET,
   }),
 })

package/docs/external-product-integration.md

@@ -0,0 +1,203 @@
+# External Product Integration
+
+This guide is for product teams using `@tangle-network/agent-integrations`
+outside this repository. The package gives you stable contracts and runtime
+helpers; your product supplies persistence, secret storage, UI, and deployment
+policy.
+
+## Mental Model
+
+```txt
+user connects account
+  -> product stores IntegrationConnection + secret refs
+  -> app/agent declares IntegrationManifest
+  -> product creates IntegrationGrant
+  -> sandbox/app receives short-lived capability bundle
+  -> sandbox/app invokes product /integrations/invoke endpoint
+  -> IntegrationHub validates capability, policy, approval, idempotency
+  -> provider backend executes action
+  -> product stores audit event and returns normalized result
+```
+
+The invariant: generated apps and agents call the same Tangle integration tools
+no matter whether a connector is backed by a native adapter, hosted gateway, or
+catalog runtime.
+
+## Product-Owned Pieces
+
+You must provide:
+
+- `IntegrationConnection` storage in your database.
+- `IntegrationGrant` storage mapping user-owned connections to apps, agents, or
+  sandboxes.
+- Approval, audit, healthcheck, workflow, and event stores.
+- `IntegrationSecretStore` backed by your vault or KMS.
+- OAuth/API-key connect UI built from `IntegrationSpec` renderers.
+- Tenant policy for which connectors are enabled.
+- Human approval UX for write/destructive actions.
+- A deployed invocation endpoint such as `/v1/integrations/invoke`.
+
+The package provides interfaces and helpers for all of these. It does not store
+your secrets or run your product UI.
+
+## Setup Flow
+
+1. Build the registry.
+
+```ts
+import { buildDefaultIntegrationRegistry } from '@tangle-network/agent-integrations'
+
+const registry = buildDefaultIntegrationRegistry({
+  tangleCatalogRuntimeExecutable: process.env.TANGLE_CATALOG_RUNTIME === '1',
+})
+```
+
+2. Render setup UI from specs.
+
+```ts
+import {
+  listIntegrationSpecs,
+  renderConsoleSteps,
+  validateCredentialSet,
+} from '@tangle-network/agent-integrations'
+
+const spec = listIntegrationSpecs().find((candidate) => candidate.kind === 'google-calendar')
+const steps = renderConsoleSteps(spec!, { host: 'id.example.com' })
+const validation = validateCredentialSet(spec!, submittedCredentials)
+```
+
+3. Store provider credentials in your vault, then persist an
+   `IntegrationConnection` with secret refs, scopes, owner, connector id, and
+   status.
+
+4. Run `runIntegrationHealthchecks()` after setup and on a schedule.
+
+## Runtime Flow
+
+Create one hub per product runtime.
+
+```ts
+import {
+  createConnectorAdapterProvider,
+  createDefaultIntegrationActionGuard,
+  IntegrationHub,
+  createPlatformIntegrationPolicyPreset,
+} from '@tangle-network/agent-integrations'
+
+const hub = new IntegrationHub({
+  providers: [
+    createConnectorAdapterProvider({
+      adapters,
+      resolveDataSource: (connection) => credentialResolver.resolve(connection),
+    }),
+    // createHttpIntegrationProvider(...) or createTangleCatalogExecutorProvider(...)
+  ],
+  store: connections,
+  capabilitySecret: process.env.INTEGRATION_CAPABILITY_SECRET!,
+  policy: createPlatformIntegrationPolicyPreset(),
+  guard: createDefaultIntegrationActionGuard({
+    audit,
+    idempotency,
+  }),
+})
+```
+
+Your `/v1/integrations/invoke` route should:
+
+1. Parse the invocation envelope.
+2. Validate the capability token.
+3. Resolve the user's connection and credentials.
+4. Run policy and approval checks.
+5. Execute through the matching provider.
+6. Store an audit event.
+7. Return a normalized result or `approval_required`.
+
+## Generated Apps And Sandboxes
+
+Generated apps declare needs through `IntegrationManifest`.
+
+```ts
+const manifest = {
+  id: 'calendar-workout-planner',
+  requirements: [
+    {
+      connectorId: 'google-calendar',
+      mode: 'read',
+      reason: 'Find open workout windows from calendar events.',
+      requiredActions: ['events.list'],
+      scopes: ['calendar.read'],
+    },
+  ],
+}
+```
+
+When a user previews or installs the app:
+
+1. Resolve the manifest against the user's existing connections.
+2. Ask the user to connect missing accounts.
+3. Show consent using `renderConsentSummary()`.
+4. Create grants from the user's connections to the app.
+5. Build a sandbox bundle with short-lived capabilities.
+6. Inject the bridge environment into the sandbox.
+
+Generated app code should use `createTangleIntegrationsClient()` or the product
+equivalent. It should not call Google, Slack, GitHub, or any provider directly
+with user credentials.
+
+## Workflows And Triggers
+
+Use `createIntegrationWorkflowRuntime()` for non-agent automations:
+
+- GitHub issue or pull-request sync.
+- Slack message triggers.
+- Calendar event updates.
+- CRM record changes.
+- Webhook-to-sandbox workflows.
+
+Inbound provider webhooks should go through `receiveIntegrationWebhook()` for
+signature verification, provider-event dedupe, and normalized event dispatch.
+
+## Long-Tail Connectors
+
+The registry distinguishes connector contracts from executable backend state.
+Products can expose long-tail connectors only when they have configured a
+backend:
+
+- native adapter
+- hosted integration gateway
+- Tangle catalog runtime
+- product-specific provider
+
+Use `buildDefaultIntegrationRegistry({ tangleCatalogRuntimeExecutable: true })`
+only after the catalog runtime is deployed and audited with
+`auditTangleCatalogRuntimePackages()`.
+
+## Security Requirements
+
+- Never pass provider refresh tokens or API keys into a sandbox.
+- Use short-lived capability tokens scoped to connector, action, subject, and
+  connection.
+- Require approval for writes by default.
+- Deny destructive actions unless the product explicitly enables them.
+- Use idempotency keys for state-changing actions.
+- Store audit events for connect, grant, invoke, approve, revoke, rotate, and
+  webhook flows.
+- Redact provider credentials from logs, traces, errors, and generated app
+  payloads.
+
+## Launch Checklist
+
+- [ ] Connection, grant, approval, audit, healthcheck, workflow, and event stores
+      are backed by production persistence.
+- [ ] Secret refs resolve through vault/KMS and never serialize raw credentials.
+- [ ] OAuth/API-key setup UI renders from `IntegrationSpec`.
+- [ ] Connect, revoke, rotate, approve, and audit-log screens exist.
+- [ ] Generated app manifests feed into `resolveManifest()` and
+      `createGrants()`.
+- [ ] Sandbox launches receive `buildIntegrationBridgeEnvironment()` output.
+- [ ] Sandbox invocations route through your product integration endpoint.
+- [ ] Writes require approval and idempotency.
+- [ ] Webhooks verify signatures and dedupe provider event ids.
+- [ ] Healthchecks run after setup and on a schedule.
+- [ ] Browser E2E covers connect, consent, preview, invoke, approval, revoke,
+      and failure recovery.

package/docs/product-hub-ownership.md

@@ -0,0 +1,126 @@
+# Integration Hub Ownership
+
+Status: deployment guidance
+
+This document describes where integration custody should live when a product
+uses `@tangle-network/agent-integrations`.
+
+## Decision
+
+Use one contract everywhere, not one deployment everywhere.
+
+- Products that share identity, billing, and consent can use a hosted platform
+  hub.
+- Standalone products can run a product-owned hub.
+- Both modes must use `@tangle-network/agent-integrations` contracts:
+  `IntegrationSpec`, `IntegrationManifest`, `IntegrationGrant`, capability
+  bundles, `/v1/integrations/invoke`, approvals, healthchecks, audit, webhooks,
+  and provider/runtime adapters.
+- Product-owned hubs may federate to a hosted platform hub when the deployment
+  wants a shared connection wallet.
+
+The package defines the shared protocol. Each deployment decides where to store
+connections, secrets, grants, approvals, audit records, and workflow state.
+
+## Ownership Modes
+
+| Mode | Use When | Owns OAuth, Vault, Audit | Invocation Path |
+|---|---|---:|---|
+| Hosted platform hub | Multiple apps share account, consent, billing, and connection custody | Platform service | App/sandbox calls platform hub or product proxy with scoped capability |
+| Product-owned hub | Standalone SaaS, private deployment, or customer-owned data boundary | Product app | App/sandbox calls product `/v1/integrations/invoke` |
+| Federated product hub | Product wants local policy with remote connection custody | Product policy + hosted custody | Product issues local grants over remote platform connections |
+
+The important invariant: sandboxes and generated apps never receive provider
+refresh tokens, API keys, or raw OAuth credentials in any mode.
+
+## Where Duplication Is Bad
+
+Fully duplicating hubs is bad when apps are intentionally part of the same
+account system:
+
+- Users reconnect Gmail, Slack, GitHub, and Drive in every app.
+- OAuth app verification, scopes, redirect configuration, and consent review
+  are repeated.
+- Revocation, audit, healthchecks, and billing attribution fragment.
+- Generated apps cannot inherit a user's already-approved connection.
+
+For those deployments, use a hosted hub as the connection source of truth and
+pass scoped grants down to apps, sandboxes, and agents.
+
+## Where Centralization Is Bad
+
+Forcing every deployment through a hosted platform hub is bad when the product
+needs a separate customer, compliance, or operational boundary:
+
+- The buyer may not want shared identity or billing in their end-user flow.
+- Enterprise deployments may require product-branded OAuth apps, product-owned
+  vaults, customer-managed keys, private networking, or data residency.
+- Product-specific subscriptions and compliance boundaries may not match the
+  platform account model.
+- A central platform outage should not necessarily take down a standalone app.
+
+For those products, run a product-owned hub using the same package contracts.
+Optionally add federation to a hosted platform hub as a deployment choice.
+
+## Execution Checklist
+
+- [x] One stable contract for specs, manifests, grants, capabilities, invocation,
+      approvals, healthchecks, webhooks, audit, and bridge payloads.
+- [x] Long-tail connector contracts and runtime-backed execution path are
+      represented without leaking external catalog names into product UX.
+- [x] External product adoption guide documents product-owned deployment.
+- [x] This ownership decision documents platform versus product-owned custody.
+
+### Hosted Hub Bar
+
+- [ ] The hosted hub has production stores for connections, grants, approvals,
+      audit, healthchecks, workflows, webhook events, and idempotency.
+- [ ] The hosted hub vault/KMS stores raw OAuth/API-key credentials behind
+      secret refs.
+- [ ] The hosted hub exposes connect, callback, revoke, rotate, approve, audit,
+      healthcheck, and `/v1/integrations/invoke`.
+- [ ] Sandbox and generated-app launches receive
+      `buildIntegrationBridgeEnvironment()` output.
+- [ ] Browser E2E covers connect, consent, preview, invoke read, approval write,
+      revoke, expired token recovery, and missing-connection recovery.
+
+### Product-Owned Hub Bar
+
+- [ ] The product chooses local hub or federated platform hub per deployment.
+- [ ] Local hubs use `IntegrationHub`; custom provider switch statements are
+      removed or wrapped as `IntegrationProvider` implementations.
+- [ ] The product stores connections, grants, approvals, audit, healthchecks,
+      workflows, events, and idempotency in its own database.
+- [ ] The product uses its own vault/KMS or explicitly delegates secret custody
+      to a hosted hub.
+- [ ] The product has live OAuth/API-key setup UI generated from
+      `IntegrationSpec`.
+- [ ] The product has browser E2E personas using real product UX and live
+      integration secrets where available.
+
+## Product E2E Gates
+
+Run these before launch for every product adopting integrations:
+
+1. Existing connection: user asks for a task requiring Gmail, Calendar, Slack,
+   GitHub, Drive, Sheets, or CRM data; agent detects the existing connection and
+   uses it without asking for manual copy/paste.
+2. Missing connection: user asks for the same task without a connection; product
+   renders connect/consent in flow, resumes the task after OAuth, and does not
+   lose conversation state.
+3. Generated app preview: generated software declares an
+   `IntegrationManifest`; preview requests user consent, receives scoped
+   capabilities, and reads provider data through the invoke endpoint.
+4. Write approval: generated app or agent proposes a write/send/update action;
+   product requires approval, records audit, and executes once with idempotency.
+5. Revocation: user revokes a connection; existing grants fail closed and the UI
+   explains how to reconnect.
+6. Healthcheck failure: expired or revoked upstream credentials surface in admin
+   UI and task UX without exposing secrets in logs or traces.
+
+## Recommendation
+
+Do not fork integration semantics by product. Make hub custody a deployment
+choice while keeping the protocol stable. Hosted hubs, product-owned hubs, and
+federated hubs should all expose the same manifest, grant, capability, approval,
+audit, webhook, and invocation shapes.

package/docs/provider-decision-matrix.md

@@ -74,9 +74,9 @@ Use gateway-backed coverage for the next large tranche:
 - long-tail CRM/support/helpdesk systems
 - vendor-specific workflow triggers
 
-This gives Agent Builder useful breadth immediately without forcing us to own
-hundreds of OAuth apps, refresh-token edge cases, webhook subscription models,
-rate-limit policies, and provider-specific APIs on day one.
+This gives products useful breadth immediately without forcing every team to
+own hundreds of OAuth apps, refresh-token edge cases, webhook subscription
+models, rate-limit policies, and provider-specific APIs on day one.
 
 ## When To Roll Our Own
 

package/examples/calendar-exercise-app.ts

@@ -12,8 +12,8 @@ const consent = renderConsentSummary(manifest, { appName: 'Exercise Planner' })
 
 console.log(consent.body)
 
-// In production this bundle comes from id.tangle.tools after the user grants
-// the generated app access to their Google Calendar connection.
+// In production this bundle comes from the product integration hub after the
+// user grants the generated app access to their Google Calendar connection.
 const bundle: IntegrationSandboxBundle = {
   manifestId: manifest.id,
   subject: { type: 'sandbox', id: 'sandbox_preview_1' },
@@ -64,7 +64,7 @@ const bundle: IntegrationSandboxBundle = {
 
 const env = buildIntegrationBridgeEnvironment(bundle)
 const client = createTangleIntegrationsClient({
-  endpoint: 'https://id.tangle.tools',
+  endpoint: 'https://integrations.example.com',
   env,
 })
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-integrations",
-  "version": "0.25.0",
+  "version": "0.25.2",
   "description": "Vendor-neutral integration contracts and runtime helpers for sandbox and agent apps.",
   "homepage": "https://github.com/tangle-network/agent-integrations#readme",
   "repository": {
@@ -38,15 +38,6 @@
   "publishConfig": {
     "access": "public"
   },
-  "scripts": {
-    "build": "tsup",
-    "dev": "tsup --watch",
-    "audit:execution": "pnpm build >/dev/null && node scripts/audit-integration-execution.mjs",
-    "prepare": "tsup",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "typecheck": "tsc --noEmit"
-  },
   "devDependencies": {
     "@types/node": "^25.6.0",
     "tsup": "^8.0.0",
@@ -57,5 +48,12 @@
     "node": ">=20"
   },
   "license": "MIT",
-  "packageManager": "pnpm@10.28.0"
-}
+  "scripts": {
+    "build": "tsup",
+    "dev": "tsup --watch",
+    "audit:execution": "pnpm build >/dev/null && node scripts/audit-integration-execution.mjs",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit"
+  }
+}