@hotmeshio/long-tail 0.1.13 → 0.1.14

package/docs/self-test.md

@@ -0,0 +1,106 @@
+# The Self-Test: Long Tail Wraps Its Own API
+
+## The Tool That Wanted to Extend Itself
+
+Long Tail ships with a schema-exchange tool — a baseline capability for exchanging data with any external service under schema enforcement. The tool validates requests before sending and responses after receiving. It doesn't know HTTP from Playwright. It knows endpoints, schemas, and whether the data matched.
+
+This is the story of what happened when someone pointed that tool at Long Tail's own API.
+
+---
+
+## The Starting Point
+
+Long Tail has a REST API. It has endpoints for authentication, for listing registered MCP servers, for querying compiled workflows. Every deployment uses these endpoints — the dashboard calls them, integrations call them, crons call them.
+
+But nobody had ever formalized what those endpoints return. The shapes were implicit — TypeScript interfaces in the codebase, but nothing the runtime could assert against. If a deployment changed a response shape, you found out when the dashboard broke. Or when a customer's integration broke. Or, worst case, you didn't find out at all because the consumer silently swallowed the new shape and produced wrong results downstream.
+
+The schema-exchange tool exists precisely for this problem. It doesn't care whether the endpoint is Epic's FHIR server or Long Tail's own API. The principle is the same: endpoint + schema + validated exchange.
+
+---
+
+## Wrapping the API
+
+An engineer opens Plan Mode in the dashboard and pastes three endpoint specifications:
+
+```
+Long Tail API (base: http://localhost:3000/api)
+
+POST /auth/login
+  Request: { username: string, password: string }
+  Response: { token: string, user: { id: string, external_id: string, display_name: string, roles: [{ role: string, type: string }] } }
+
+GET /mcp/servers (requires Bearer token)
+  Response: { servers: [{ id: string, name: string, description: string, tags: string[], status: string, tool_manifest: [{ name: string, description: string }] }] }
+
+GET /yaml-workflows (requires Bearer token)
+  Response: { workflows: [{ id: string, name: string, app_id: string, status: string, graph_topic: string, tags: string[] }], total: number }
+```
+
+The planner decomposes this into three leaf workflows. The builder discovers the `exchange` tool in the inventory, reads the compile hints, and constructs each workflow as a trigger → exchange → output DAG with embedded schemas.
+
+All three deploy under a server namespace: `longtailapi`.
+
+---
+
+## What the Compiled Tools Do
+
+**`login`** — Takes a username and password. Validates the request body (both strings, both required) against the request schema before sending. Calls POST /auth/login. Validates the response (must have `token` string and `user` object with required fields). Returns the token and user profile.
+
+If someone changes the login response — adds a field, removes `display_name`, changes `roles` from an array to a string — the schema validation catches it. `validated: false`, with a human-readable error explaining exactly what changed.
+
+**`list_servers`** — Takes a bearer token. Calls GET /mcp/servers. Validates the response (must be an object with a `servers` array, each server must have `id`, `name`, `tags`). Returns the server list.
+
+This tool can answer the question: "does the schema-exchange server exist?" If `long-tail-schema-exchange` isn't in the returned list, the tool that's asking the question knows its own infrastructure is broken. The snake eating its own tail.
+
+**`list_workflows`** — Takes a bearer token. Calls GET /yaml-workflows. Validates the response (must be an object with `workflows` array and `total` number). Returns the compiled tool inventory.
+
+This tool can check whether the `longtailapi` tools themselves are deployed and active. It can verify that the very workflows it belongs to are in the list. Self-referential validation — the compiled tool confirms its own existence.
+
+---
+
+## The Composition: Self-Health-Check
+
+The three leaf tools compose into a single workflow: `self_health_check`.
+
+1. **Login** — authenticate with a service account
+2. **List servers** — verify all expected MCP servers are registered
+3. **List workflows** — verify all expected compiled tools are deployed
+
+Each step validates its response schema. If any step fails validation, the workflow knows exactly what changed — not "the API is down" but "the `servers` response is missing the `tags` field on server objects."
+
+Schedule it on cron. Every midnight, the platform checks its own API surface against the schemas it captured when the tools were compiled. Schema drift is caught within 24 hours, automatically, without a human looking at anything.
+
+---
+
+## Why This Matters
+
+This isn't a testing framework. It's the same schema-exchange primitive that wraps Epic's FHIR endpoints or Stripe's payment API or any other external service. The fact that it can wrap Long Tail's own API is a proof point, not a special case.
+
+The proof point is:
+
+1. **Any API surface can be formalized as compiled tools.** Paste the endpoint specs. Get schema-validated, cron-testable tools. No MCP server to hand-write. No integration code. The schema is the integration.
+
+2. **Schema drift detection is automatic.** The response_schema embedded in each compiled tool is the contract. When the contract breaks, the tool reports exactly what changed. Not "500 error" — the actual structural diff.
+
+3. **The tools compose.** Login → use token → check servers → check workflows. Each step is independently testable, independently schedulable, independently versionable. But together they form a health check that validates the entire platform surface.
+
+4. **The platform can extend itself.** The schema-exchange tool is a baseline capability. The compiled API tools are built from it. The health check composes them. Every layer uses the same machinery. New endpoints are absorbed the same way — paste, compile, deploy, schedule.
+
+This is the starting point the Epic story describes: "The engineering team registers a custom MCP server that wraps the FHIR endpoints their referral workflows need." Except here, the engineering team is us, the FHIR endpoints are our own API, and the custom MCP server assembled itself from pasted specs and a schema-exchange primitive.
+
+The SOPs come next. But the plumbing works.
+
+---
+
+## The Integration Test
+
+The file `tests/integration/schema-exchange.test.ts` proves this end-to-end:
+
+1. Calls `exchange` to login (POST /auth/login with request + response schema)
+2. Calls `exchange` to list servers (GET /mcp/servers with response schema)
+3. Asserts the schema-exchange server exists in the response
+4. Calls `exchange` to list workflows (GET /yaml-workflows with response schema)
+5. Deliberately uses a wrong schema to prove drift detection
+6. Deliberately sends a malformed request to prove pre-send rejection
+
+If any assertion fails, the schema-exchange tool isn't doing its job. If they all pass, the platform can wrap any API — including its own.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/long-tail",
-  "version": "0.1.13",
+  "version": "0.1.14",
   "description": "Long Tail Workflows — Durable AI workflows with human-in-the-loop escalation. Powered by PostgreSQL.",
   "main": "./build/index.js",
   "types": "./build/index.d.ts",
@@ -61,12 +61,14 @@
   "dependencies": {
     "@anthropic-ai/sdk": "^0.92.0",
     "@aws-sdk/client-s3": "^3.1017.0",
+    "@aws-sdk/s3-request-presigner": "^3.1045.0",
     "@hotmeshio/hotmesh": "^0.14.7",
     "@modelcontextprotocol/sdk": "^1.27.1",
     "@opentelemetry/exporter-trace-otlp-proto": "^0.215.0",
     "@opentelemetry/resources": "^2.5.1",
     "@opentelemetry/sdk-node": "^0.215.0",
     "@opentelemetry/semantic-conventions": "^1.39.0",
+    "ajv": "^8.20.0",
     "arctic": "^3.7.0",
     "bcryptjs": "^2.4.3",
     "express": "^5.1.0",

package/dashboard/dist/assets/CredentialsPage-DJablIbs.js

@@ -1,2 +0,0 @@
-import{A as v,B as C,j as e,O as c,z as P}from"./index-BYZX9tOb.js";import{a as n}from"./vendor-query-DLp59M9_.js";import{P as D}from"./PageHeader-BcTVF9ul.js";import{T as E}from"./TimeAgo-BZdLdrIh.js";import{C as j}from"./ConfirmDeleteModal-DZMgmlof.js";import{a9 as V,C as y,ac as k,j as A}from"./vendor-icons-T4r2DSPD.js";import"./vendor-react-Co3Y8ikm.js";import"./Modal-BuTvD0pz.js";const d={anthropic:{name:"Anthropic",helpText:'Run "claude setup-token" for an OAuth token, or use an API key from console.anthropic.com.',placeholder:"sk-ant-oat01-... or sk-ant-api03-..."},openai:{name:"OpenAI",helpText:"Create an API key at platform.openai.com/api-keys.",placeholder:"sk-..."}};function z(){var m,p;const{data:r,isLoading:u}=v(),i=C(),[a,l]=n.useState(null),[b,N]=n.useState(!1),[t,g]=n.useState("anthropic"),o=(r==null?void 0:r.connections)??[],x=()=>{a&&i.mutate({provider:a.provider,label:a.label},{onSuccess:()=>l(null)})},h=s=>`/api/auth/oauth/connect/${s}?token=${encodeURIComponent(P()||"")}&returnTo=/credentials`;return e.jsxDEV("div",{children:[e.jsxDEV(D,{title:"Credentials",actions:e.jsxDEV("button",{onClick:()=>N(s=>!s),className:"btn-primary text-xs inline-flex items-center gap-1.5",children:[e.jsxDEV(V,{className:"w-3.5 h-3.5"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:57,columnNumber:13},this),"Add Credential"]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:53,columnNumber:11},this)},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:50,columnNumber:7},this),e.jsxDEV("p",{className:"text-sm text-text-secondary mb-6",children:"Manage credentials used by tools when invoked on your behalf. Each credential is encrypted at rest and resolved automatically during tool execution."},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:63,columnNumber:7},this),b&&e.jsxDEV("div",{className:"mb-6 p-4 bg-surface-raised border border-surface-border rounded-md space-y-3",children:[e.jsxDEV("div",{className:"flex items-center gap-3",children:[e.jsxDEV("label",{className:"text-xs text-text-secondary font-medium",children:"Provider"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:72,columnNumber:13},this),e.jsxDEV("div",{className:"relative",children:[e.jsxDEV("select",{value:t,onChange:s=>g(s.target.value),className:"input text-xs pr-7 appearance-none",children:Object.entries(d).map(([s,f])=>e.jsxDEV("option",{value:s,children:f.name},s,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:80,columnNumber:19},this))},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:74,columnNumber:15},this),e.jsxDEV(y,{className:"absolute right-2 top-1/2 -translate-y-1/2 w-3 h-3 text-text-tertiary pointer-events-none"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:83,columnNumber:15},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:73,columnNumber:13},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:71,columnNumber:11},this),e.jsxDEV("p",{className:"text-xs text-text-tertiary",children:(m=d[t])==null?void 0:m.helpText},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:86,columnNumber:11},this),e.jsxDEV("a",{href:h(t),className:"btn-primary text-xs inline-flex items-center gap-1.5",children:[e.jsxDEV(c,{provider:t,className:"w-3.5 h-3.5"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:93,columnNumber:13},this),"Connect ",(p=d[t])==null?void 0:p.name]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:89,columnNumber:11},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:70,columnNumber:9},this),u?e.jsxDEV("div",{className:"animate-pulse space-y-3",children:[1,2].map(s=>e.jsxDEV("div",{className:"h-16 bg-surface-sunken rounded"},s,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:103,columnNumber:13},this))},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:101,columnNumber:9},this):o.length===0?e.jsxDEV("div",{className:"text-center py-16 text-text-tertiary",children:[e.jsxDEV(k,{className:"w-10 h-10 mx-auto mb-3 opacity-40"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:108,columnNumber:11},this),e.jsxDEV("p",{className:"text-sm",children:"No credentials registered."},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:109,columnNumber:11},this),e.jsxDEV("p",{className:"text-xs mt-1",children:"Add a credential to enable tool authentication."},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:110,columnNumber:11},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:107,columnNumber:9},this):e.jsxDEV("div",{className:"space-y-2",children:o.map(s=>e.jsxDEV("div",{className:"group/row flex items-center gap-4 px-4 py-3 bg-surface-raised border border-surface-border rounded-md",children:[e.jsxDEV(c,{provider:s.provider,className:"w-6 h-6 shrink-0"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:119,columnNumber:15},this),e.jsxDEV("div",{className:"flex-1 min-w-0",children:[e.jsxDEV("div",{className:"flex items-center gap-2",children:[e.jsxDEV("span",{className:"text-sm text-text-primary font-medium capitalize",children:s.provider},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:123,columnNumber:19},this),s.label!=="default"&&e.jsxDEV("span",{className:"text-[10px] px-1.5 py-0.5 bg-surface-sunken rounded text-text-tertiary",children:s.label},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:127,columnNumber:21},this),s.credential_type&&e.jsxDEV("span",{className:"text-[10px] px-1.5 py-0.5 bg-accent-faint rounded text-accent",children:s.credential_type},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:132,columnNumber:21},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:122,columnNumber:17},this),e.jsxDEV("div",{className:"flex items-center gap-3 mt-0.5",children:[s.email&&e.jsxDEV("span",{className:"text-xs text-text-tertiary",children:s.email},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:139,columnNumber:21},this),s.expires_at&&e.jsxDEV("span",{className:"text-xs text-text-tertiary",children:["Expires ",e.jsxDEV(E,{date:s.expires_at},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:143,columnNumber:31},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:142,columnNumber:21},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:137,columnNumber:17},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:121,columnNumber:15},this),e.jsxDEV("button",{onClick:()=>l({provider:s.provider,label:s.label}),className:"opacity-0 group-hover/row:opacity-100 transition-opacity text-text-tertiary hover:text-status-error",title:"Revoke credential",children:e.jsxDEV(A,{className:"w-4 h-4"},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:154,columnNumber:17},this)},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:149,columnNumber:15},this)]},`${s.provider}-${s.label}`,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:115,columnNumber:13},this))},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:113,columnNumber:9},this),e.jsxDEV(j,{open:!!a,onClose:()=>l(null),onConfirm:x,title:"Revoke Credential",description:e.jsxDEV(e.Fragment,{children:["Revoke the"," ",e.jsxDEV("span",{className:"font-medium text-text-primary capitalize",children:a==null?void 0:a.provider},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:169,columnNumber:13},this),(a==null?void 0:a.label)!=="default"&&e.jsxDEV(e.Fragment,{children:[" (",a==null?void 0:a.label,")"]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:173,columnNumber:15},this)," ","credential? Tools will no longer be able to use this credential."]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:167,columnNumber:11},this),isPending:i.isPending,error:i.error},void 0,!1,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:161,columnNumber:7},this)]},void 0,!0,{fileName:"/app/dashboard/src/pages/settings/CredentialsPage.tsx",lineNumber:49,columnNumber:5},this)}export{z as CredentialsPage};
-//# sourceMappingURL=CredentialsPage-DJablIbs.js.map