@hotmeshio/long-tail 0.19.1 → 0.21.0

package/docs/schema-exchange.md

@@ -1,173 +0,0 @@
-# Surfacing Any API
-
-## The Gap Between Plumbing and Domain
-
-Long Tail ships with built-in MCP servers. Browser automation for navigating web interfaces. File storage for persisting artifacts. Vision for analyzing images. Knowledge for accumulating structured state. An admin server for the platform to look inward — list its own workflows, manage its own configuration, query its own execution history.
-
-These are plumbing. They solve cross-cutting problems that every deployment shares. But they don't know anything about your domain. They don't know what Epic's FHIR API returns for a `ServiceRequest` resource. They don't know the shape of Stripe's charge response. They don't know that your internal inventory service returns `{ items: [...], cursor: string }` and paginates with cursor-based tokens.
-
-Traditionally, closing this gap meant writing an MCP server. Define the tools. Map each one to an API call. Handle serialization, error cases, pagination. Register the server. Tag it. Write compile hints so the builder knows how to wire it. A week of engineering per external service.
-
-The schema-exchange primitive eliminates that week.
-
----
-
-## The Principle: Endpoint + Schema
-
-The `exchange` tool doesn't know HTTP from Playwright. It knows three things: where to send data, what the data should look like going out, and what it should look like coming back.
-
-```
-endpoint:        where
-request_schema:  what leaves
-response_schema: what returns
-```
-
-Transport is an implementation detail. Today it's Node.js `fetch`. Tomorrow it could be a browser automation step that fills a form, submits, and scrapes the result. Or a gRPC call. Or a SOAP envelope. The consumer doesn't know and doesn't care.
-
-What the consumer gets is a contract. The request is validated before it leaves the system. The response is validated when it arrives. If either validation fails, the failure is structural and specific — "the response is missing the `resourceType` field" — not a vague 500 error three layers deep.
-
-This is the difference between a fetch call and a schema-driven exchange. A fetch call succeeds or fails. An exchange succeeds, fails, or *detects drift* — the call succeeded but the shape changed. That third state is what makes API integrations maintainable over time.
-
----
-
-## From Spec to Toolset
-
-Open the Pipeline Designer. Choose Build mode. Paste:
-
-```
-Stripe Charges API (base: https://api.stripe.com/v1)
-
-POST /charges
-  Headers: Authorization: Bearer {api_key}
-  Request: { amount: number, currency: string, source: string, description?: string }
-  Response: { id: string, amount: number, currency: string, status: string, created: number }
-
-GET /charges/{id}
-  Headers: Authorization: Bearer {api_key}
-  Response: { id: string, amount: number, currency: string, status: string, created: number }
-```
-
-The planner decomposes this into two leaf tools. The builder discovers the `exchange` tool in its inventory, reads the compile hints, and constructs each workflow:
-
-- **`create_charge`** — trigger accepts `api_key`, `amount`, `currency`, `source`. Worker calls `exchange` with the request schema embedded as a static value. Response schema validates the charge object. If Stripe changes the response shape, `validated` flips to `false` and the error names the exact field that changed.
-
-- **`get_charge`** — trigger accepts `api_key`, `charge_id`. Worker calls `exchange` with the charge ID interpolated into the URL. Response schema validates the same shape.
-
-Both deploy under a server namespace: `stripe`. They're compiled, versioned, invocable. They have typed inputs and outputs. They compose into larger workflows — a refund pipeline, an invoicing process, a reconciliation job.
-
-The engineer didn't write an MCP server. They didn't write any code. They pasted two endpoint descriptions and got two schema-validated tools that will catch API drift the moment Stripe changes a field.
-
----
-
-## Schema Drift: The Silent Killer
-
-APIs change. Fields get renamed. Required properties appear. Types shift from strings to objects. These changes are rarely announced in time, rarely caught by tests that don't run against live endpoints, and rarely surfaced until a downstream consumer produces wrong results.
-
-The response schema embedded in each compiled tool is a contract. It doesn't enforce — it detects. When the contract breaks, the tool reports exactly what changed:
-
-```
-response: (root): must have required property 'source' ({"missingProperty":"source"})
-```
-
-Not "API error." Not "unexpected response." The structural diff, in human-readable text, identifying the exact field.
-
-Schedule the tool on cron against a staging or dev endpoint. Every midnight, it calls the API and validates the response. If the schema matches, nothing happens. If it doesn't, the system knows before production does. The team fixes the tool, updates the schema, redeploys. No customer impact.
-
-This is what makes compiled API tools self-maintaining. The schema is both the integration logic and the regression test. They're the same artifact.
-
----
-
-## The Request Gate
-
-Schema enforcement works in both directions. The request schema validates outbound data before it leaves the system. If a caller passes `{ amount: "fifty" }` to `create_charge`, the exchange tool rejects it immediately:
-
-```
-request: /amount: must be number ({"type":"number"})
-```
-
-The request never hits Stripe. The error is instant, specific, and local. No network round-trip. No API rate limit consumed. No partial state created on the remote service.
-
-This matters in compiled workflows where one tool's output feeds another's input. A data-mapping error in step 3 of a 7-step pipeline is caught at step 3, not when Stripe returns a 400 in step 5. The failure is close to the cause.
-
----
-
-## Growing a Toolset
-
-The Pipeline Designer organizes tools into sets — collections of related tools that share a namespace and evolve together. You start with two Stripe endpoints. Next week you add `list_charges` and `create_refund`. The following month, `get_balance` and `list_payouts`.
-
-Each addition goes through the same flow: paste the spec, the planner builds the tool, it deploys into the existing namespace. The server version increments. All tools in the namespace are redeployed together. Active workflows don't stop — the new version activates atomically.
-
-The set is the workbench. It preserves the full specification history — every endpoint description that was pasted, in order. It's the source of truth for what the toolset covers and how it evolved.
-
-Over time, the set becomes a complete API surface. Not hand-written bindings. Not auto-generated client code that drifts from the spec. Compiled tools with embedded schemas that validate on every call.
-
----
-
-## Composition: Tools Built From Tools
-
-Individual API tools are building blocks. A compiled tool that calls `create_charge` can be composed with one that calls `get_customer` and one that calls `send_receipt`. The composition is itself a compiled tool — a pipeline that executes the three steps in sequence, wiring the charge ID from step 1 to the receipt in step 3.
-
-The schema exchange is invisible at the composition level. The composer sees tools with typed inputs and outputs. It doesn't know — or need to know — that these tools validate their data against schemas before every call. The validation is infrastructure, not application logic.
-
-This is where the Epic story begins. The FHIR endpoints are individual tools — `get_patient`, `search_coverage`, `create_task`. Each validates its request and response. Linda's intake process composes them into a pipeline that encodes her institutional knowledge: which checks to run, in what order, with what branching logic. The individual tools handle the API contract. The composition handles the business process. Neither knows about the other's concerns.
-
----
-
-## What This Replaces
-
-Without schema-driven exchange, wrapping an external API requires:
-
-1. Writing an MCP server (TypeScript, tool definitions, error handling)
-2. Registering it with the platform
-3. Writing compile hints so the builder knows how to use it
-4. Maintaining it when the API changes
-5. Repeating for every external service
-
-With schema-driven exchange, wrapping an external API requires:
-
-1. Pasting the endpoint specs into the Pipeline Designer
-
-The MCP server is the `exchange` tool. The compile hints are on the `exchange` tool. The schema validation is in the `exchange` tool. The only thing that changes per API is the spec the engineer pastes — the endpoints, the schemas, the auth pattern.
-
-This is what makes the Epic story practical. The engineering team doesn't spend a week writing a FHIR MCP server. They paste the endpoint descriptions and get compiled tools that speak FHIR with schema enforcement. The week they would have spent on plumbing is spent with Linda instead, capturing the institutional knowledge that makes the tools valuable.
-
----
-
-## The Reflexive Case
-
-Long Tail wraps its own API using the same primitive. The platform's REST endpoints — login, list servers, list workflows — are compiled tools in the `longtailapi` namespace. They validate their own response shapes. They detect drift in their own API surface.
-
-This isn't a special case. It's a proof point. The same `exchange` tool that wraps Epic's FHIR server wraps Long Tail's own REST API. The same schema enforcement that catches Stripe's field changes catches Long Tail's own field changes. The platform doesn't have a privileged self-knowledge path. It discovers its own capabilities the same way it discovers any external service — through endpoints and schemas.
-
-The admin server (`long-tail-admin`) still exists for operations that require internal access — managing workflows, modifying configuration, operations that go beyond data exchange. But for anything that's "call an endpoint, validate the response, return the data" — the schema exchange primitive is the universal answer. Internal or external. FHIR or REST. Stripe or self.
-
----
-
-## Identity: The Third Pillar
-
-Endpoint and schema handle the data contract. The third pillar — identity — handles who's calling.
-
-The `exchange` tool accepts an optional `credential_provider` field. When set, the tool resolves authentication from the platform's connection store using the calling principal's identity. No token input. No manual header wiring. No `get_access_token` step in the workflow.
-
-```
-credential_provider: "stripe"
-```
-
-That single field means: look up the calling user's Stripe credentials from the encrypted connection store, auto-refresh if expired, inject into the request headers. The credential never appears in the workflow state, the execution trace, or the YAML. It's resolved at the last mile — inside the tool execution, after the HotMesh event log has already been written.
-
-This is how the Epic story works in practice. The engineering team registers their SMART on FHIR credentials as a connection. Every compiled FHIR tool specifies `credential_provider: "epic"`. When the tool runs for Customer A, it gets Customer A's token. When it runs for Customer B, it gets Customer B's. Same compiled tool. Different identities. The routing is invisible.
-
-Three auth patterns, one tool:
-- **credential_provider** — resolve from the connection store. Fresh token, auto-refresh, per-principal. The default for production.
-- **Ephemeral references** — opaque `eph:v1:*` strings in headers, exchanged at the last mile. For workflows that need human-provided credentials with TTL.
-- **Raw headers** — pass `Authorization: Bearer {token}` directly. For testing, one-off calls, or systems where the caller manages tokens externally.
-
----
-
-## What Remains
-
-The schema-exchange tool is plumbing. It doesn't know about healthcare or payments or logistics. It knows about endpoints, schemas, identity, and whether the data matched.
-
-Everything above it — the domain knowledge, the business logic, the institutional expertise — comes from the people who use it. Linda's referral intake rules. Maria's document requirements. The Stripe integration team's charge flow. The ops team's self-monitoring pipeline.
-
-The platform's job is to make that knowledge executable, composable, and self-testing. The schema exchange is the foundation layer — endpoint, schema, identity. The three pillars that make "paste an API spec, get a validated tool with automatic auth" possible. Everything else builds on top of it. The tools are the building. The exchange is the foundation.

package/docs/self-test.md

@@ -1,106 +0,0 @@
-# The Self-Test: Long Tail Wraps Its Own API
-
-## The Tool That Wanted to Extend Itself
-
-Long Tail ships with a schema-exchange tool — a baseline capability for exchanging data with any external service under schema enforcement. The tool validates requests before sending and responses after receiving. It doesn't know HTTP from Playwright. It knows endpoints, schemas, and whether the data matched.
-
-This is the story of what happened when someone pointed that tool at Long Tail's own API.
-
----
-
-## The Starting Point
-
-Long Tail has a REST API. It has endpoints for authentication, for listing registered MCP servers, for querying compiled workflows. Every deployment uses these endpoints — the dashboard calls them, integrations call them, crons call them.
-
-But nobody had ever formalized what those endpoints return. The shapes were implicit — TypeScript interfaces in the codebase, but nothing the runtime could assert against. If a deployment changed a response shape, you found out when the dashboard broke. Or when a customer's integration broke. Or, worst case, you didn't find out at all because the consumer silently swallowed the new shape and produced wrong results downstream.
-
-The schema-exchange tool exists precisely for this problem. It doesn't care whether the endpoint is Epic's FHIR server or Long Tail's own API. The principle is the same: endpoint + schema + validated exchange.
-
----
-
-## Wrapping the API
-
-An engineer opens Plan Mode in the dashboard and pastes three endpoint specifications:
-
-```
-Long Tail API (base: http://localhost:3000/api)
-
-POST /auth/login
-  Request: { username: string, password: string }
-  Response: { token: string, user: { id: string, external_id: string, display_name: string, roles: [{ role: string, type: string }] } }
-
-GET /mcp/servers (requires Bearer token)
-  Response: { servers: [{ id: string, name: string, description: string, tags: string[], status: string, tool_manifest: [{ name: string, description: string }] }] }
-
-GET /yaml-workflows (requires Bearer token)
-  Response: { workflows: [{ id: string, name: string, app_id: string, status: string, graph_topic: string, tags: string[] }], total: number }
-```
-
-The planner decomposes this into three leaf workflows. The builder discovers the `exchange` tool in the inventory, reads the compile hints, and constructs each workflow as a trigger → exchange → output DAG with embedded schemas.
-
-All three deploy under a server namespace: `longtailapi`.
-
----
-
-## What the Compiled Tools Do
-
-**`login`** — Takes a username and password. Validates the request body (both strings, both required) against the request schema before sending. Calls POST /auth/login. Validates the response (must have `token` string and `user` object with required fields). Returns the token and user profile.
-
-If someone changes the login response — adds a field, removes `display_name`, changes `roles` from an array to a string — the schema validation catches it. `validated: false`, with a human-readable error explaining exactly what changed.
-
-**`list_servers`** — Takes a bearer token. Calls GET /mcp/servers. Validates the response (must be an object with a `servers` array, each server must have `id`, `name`, `tags`). Returns the server list.
-
-This tool can answer the question: "does the schema-exchange server exist?" If `long-tail-schema-exchange` isn't in the returned list, the tool that's asking the question knows its own infrastructure is broken. The snake eating its own tail.
-
-**`list_workflows`** — Takes a bearer token. Calls GET /yaml-workflows. Validates the response (must be an object with `workflows` array and `total` number). Returns the compiled tool inventory.
-
-This tool can check whether the `longtailapi` tools themselves are deployed and active. It can verify that the very workflows it belongs to are in the list. Self-referential validation — the compiled tool confirms its own existence.
-
----
-
-## The Composition: Self-Health-Check
-
-The three leaf tools compose into a single workflow: `self_health_check`.
-
-1. **Login** — authenticate with a service account
-2. **List servers** — verify all expected MCP servers are registered
-3. **List workflows** — verify all expected compiled tools are deployed
-
-Each step validates its response schema. If any step fails validation, the workflow knows exactly what changed — not "the API is down" but "the `servers` response is missing the `tags` field on server objects."
-
-Schedule it on cron. Every midnight, the platform checks its own API surface against the schemas it captured when the tools were compiled. Schema drift is caught within 24 hours, automatically, without a human looking at anything.
-
----
-
-## Why This Matters
-
-This isn't a testing framework. It's the same schema-exchange primitive that wraps Epic's FHIR endpoints or Stripe's payment API or any other external service. The fact that it can wrap Long Tail's own API is a proof point, not a special case.
-
-The proof point is:
-
-1. **Any API surface can be formalized as compiled tools.** Paste the endpoint specs. Get schema-validated, cron-testable tools. No MCP server to hand-write. No integration code. The schema is the integration.
-
-2. **Schema drift detection is automatic.** The response_schema embedded in each compiled tool is the contract. When the contract breaks, the tool reports exactly what changed. Not "500 error" — the actual structural diff.
-
-3. **The tools compose.** Login → use token → check servers → check workflows. Each step is independently testable, independently schedulable, independently versionable. But together they form a health check that validates the entire platform surface.
-
-4. **The platform can extend itself.** The schema-exchange tool is a baseline capability. The compiled API tools are built from it. The health check composes them. Every layer uses the same machinery. New endpoints are absorbed the same way — paste, compile, deploy, schedule.
-
-This is the starting point the Epic story describes: "The engineering team registers a custom MCP server that wraps the FHIR endpoints their referral workflows need." Except here, the engineering team is us, the FHIR endpoints are our own API, and the custom MCP server assembled itself from pasted specs and a schema-exchange primitive.
-
-The SOPs come next. But the plumbing works.
-
----
-
-## The Integration Test
-
-The file `tests/integration/schema-exchange.test.ts` proves this end-to-end:
-
-1. Calls `exchange` to login (POST /auth/login with request + response schema)
-2. Calls `exchange` to list servers (GET /mcp/servers with response schema)
-3. Asserts the schema-exchange server exists in the response
-4. Calls `exchange` to list workflows (GET /yaml-workflows with response schema)
-5. Deliberately uses a wrong schema to prove drift detection
-6. Deliberately sends a malformed request to prove pre-send rejection
-
-If any assertion fails, the schema-exchange tool isn't doing its job. If they all pass, the platform can wrap any API — including its own.

package/docs/story.md

@@ -1,184 +0,0 @@
-# Long Tail — The Story
-
-## What It Feels Like
-
-You open the dashboard. There's a quiet hum of activity — processes running, knowledge accumulating, agents doing their work. You didn't write most of this. You described what you needed. The system figured out how.
-
-A notification appears: your retirement account agent found a deadline. It read your Gmail, extracted the action item, organized the context in your knowledge store, and created a task for you with a link, a summary, and a phone number to call. You review, make the call, mark it done. The agent learns that this type of deadline requires a phone call and adjusts.
-
-Nobody wrote that workflow. Nobody will maintain it. It emerged from a conversation you had on a Tuesday afternoon, and it's been running every hour since.
-
----
-
-## Three People, One Platform
-
-### Maya — Product Manager
-
-Maya doesn't write code. She writes requirements, talks to customers, and needs things to happen across systems she doesn't control.
-
-**Her journey:**
-
-She opens the dashboard and types into the prompt bar: *"Check our support inbox every morning for emails mentioning 'cancellation'. Summarize each thread, tag the customer's sentiment, and create a task for the retention team with the full context."*
-
-The system discovers the capabilities it needs — Gmail search, thread summarization, sentiment analysis, knowledge storage, task creation. It runs the sequence dynamically, shows Maya the result. She adjusts: *"Include the customer's subscription tier from our database."* It re-runs, incorporating the schema exchange tool to query the customer API.
-
-Maya clicks **Compile**. The dynamic exploration becomes a deterministic pipeline — no AI in the loop, no token cost, sub-second execution. She schedules it for 7 AM daily. An agent is born.
-
-Two weeks later, the customer API changes a field name. The pipeline fails. The agent escalates to Maya's team. An engineer resolves it, the pipeline recompiles. Maya never knew it happened.
-
-**What Maya sees:** Agents, processes, escalations. She lives in outcomes.
-
-### Dev — Product Engineer
-
-Dev writes code when it makes sense and uses the platform when it's faster. He understands both worlds and moves between them.
-
-**His journey:**
-
-Dev is building an integration with a new vendor. He registers a schema exchange endpoint — paste the API docs, define request/response schemas. The system creates a validated, self-testing MCP tool automatically.
-
-He could write a durable workflow in TypeScript to orchestrate the vendor calls. Instead, he opens the pipeline builder and describes the sequence: authenticate → fetch orders → validate → store in knowledge → escalate anomalies. The builder compiles it to a deterministic YAML DAG.
-
-But the anomaly detection logic is complex. Dev writes that as a proper activity function — tested, type-safe, version-controlled. He registers it alongside the compiled pipeline. The system doesn't care how the activity was authored. It's a pearl on the string either way.
-
-When Maya's agent needs the vendor data, it discovers Dev's tools and uses them. Maya's conversation-compiled pipeline calls Dev's hand-crafted activity. Neither knows about the other. The capability just exists.
-
-**What Dev sees:** Servers & tools, workflows, pipeline designer, execution traces. He lives in build mode but watches the outcomes.
-
-### Sam — Platform Engineer
-
-Sam owns the infrastructure. She thinks in terms of reliability, observability, and cost.
-
-**Her journey:**
-
-Sam looks at the capabilities view — every activity the system can perform, sorted by usage, success rate, and cost. She sees that `gmail_search` is called 3,000 times a day across 12 agents. The dynamic path is used 0.2% of the time. 99.8% deterministic. Cost: near zero.
-
-She checks the knowledge store. Each agent has its own domain — clean isolation, no cross-contamination. The retirement agent has accumulated 847 entries over six weeks. The support monitoring agent has 12,000.
-
-An alert fires: a compiled pipeline is failing at a 15% rate. Sam traces the execution timeline — swimlane view shows the `schema_exchange` step timing out. The vendor's API is degrading. She doesn't fix the code. She adjusts the retry policy and timeout on the workflow config. The pipeline adapts.
-
-A new engineer joins. Sam points them to the boilerplate project: "Clone it, add your activities to `src/activities/`, register your MCP server in `src/mcp-servers/`, declare your workflow config in `start()`. Push. It deploys. The dashboard shows your tools immediately."
-
-**What Sam sees:** Task queues, execution metrics, knowledge usage, DB maintenance, agent health. She lives in the system view.
-
----
-
-## The Surface
-
-### Home: What's Alive
-
-The home page isn't a dashboard of charts. It's a **living view of work in motion**:
-
-- Active processes — grouped by origin, showing progress through multi-step workflows
-- Recent escalations — what needs human attention right now
-- Agent activity — which agents ran, what they found, what they learned
-
-No distinction between durable and compiled. No mention of YAML or TypeScript. Just: work happening, knowledge growing, agents running.
-
-### Agents: Who's Working
-
-Each agent is a card:
-
-```
-┌─────────────────────────────────────────────┐
-│  📧 Retirement Monitor                      │
-│  Checks Gmail hourly for 401k updates       │
-│                                             │
-│  Last run: 12 minutes ago                   │
-│  Knowledge: 847 entries in `retirement`     │
-│  Escalations: 3 pending review              │
-│  Status: ● Active                           │
-│                                             │
-│  [View Knowledge]  [Execution Log]  [Edit]  │
-└─────────────────────────────────────────────┘
-```
-
-Click into an agent and see: its knowledge domain (browsable), its execution history (timeline), its escalation feed, its configuration. Everything an agent knows, everything it's done, everything it's waiting on.
-
-**Creating an agent** is a conversation. The agent builder is the MCP query router with a purpose: instead of executing once, it compiles and schedules. The output is an agent, not a one-shot result.
-
-### Capabilities: What's Possible
-
-A single searchable view of every activity — not grouped by server or workflow, but by **what it does**:
-
-```
-Communication
-  gmail_search · gmail_read · gmail_draft · send_sms · notify
-
-Analysis  
-  analyze_image · describe_image · translate · sentiment
-
-Data
-  store_knowledge · search_knowledge · schema_exchange · http_fetch
-
-Automation
-  escalate_and_wait · create_task · capture_page · run_script
-```
-
-Each capability shows who uses it: which agents, which pipelines, which workflows. Click one and see its schema, its compile hints, its execution history. This is the **institutional memory map** — it tells you what your organization has taught this system to do.
-
-### Orchestrate: How It Works
-
-The orchestration backstage — the same durable workflow, offered two ways:
-
-- **Procedural** — readable, durable TypeScript. Configure it, invoke it, watch its executions. Emulated atop the graph: cheap to maintain, heavier to run.
-- **Graph** — that same workflow as a compiled, deterministic DAG. Same guarantees, roughly 3x faster, less to read. Run or schedule it; the router discovers it and pulls the work when it's healthy to.
-
-Every procedural pattern has a graph equivalent and the reverse, so the choice is readability or speed — never durability, escalation, or transactional safety. Engineers move freely between them: execution traces, durable cycle replay, swimlane timelines. Nothing is hidden. The technical depth is the product.
-
-**Design** sits alongside as an optional add-on. With an Anthropic key, the Designer turns a description — or a recorded run of MCP tools — into a graph flow. Without a key, procedural and graph orchestration stand on their own, no tradeoff.
-
-### Storage: What's Remembered
-
-- **Files** — drag-and-drop, signed URLs, image preview
-- **Knowledge** — domain browser, entry editor, JSON drop
-
-These aren't utilities. They're the **memory system**. When an agent stores extracted email data, it appears here. When a pipeline writes a screenshot, it appears here. The knowledge store is the agent's brain, browsable by anyone.
-
----
-
-## The Event Spine
-
-Everything that happens publishes to NATS. Every workflow start, every activity completion, every escalation claim, every knowledge write, every agent run. The event feed at the bottom of the dashboard is a live window into this stream.
-
-But the real power is **subscription**. An agent can trigger on any event:
-
-- A new escalation is created → the triage agent evaluates whether AI can resolve it
-- A file is uploaded → the image analysis agent processes it
-- Knowledge is updated in domain X → the report generator agent recompiles
-- A pipeline fails → the health monitoring agent escalates
-
-The event spine is NATS. Clean pub/sub. Topic-based routing. No polling. The dashboard's real-time updates come from the same pipe that agents use to trigger. **One event system for humans and machines.**
-
-This is what makes the system feel alive. It's not cron-only (check every hour). It's reactive (respond when something happens). Agents that react are faster, cheaper, and more natural than agents that poll.
-
-And though orchestration *feels* top-down — you invoke a flow, it drives the steps — execution is pull-based underneath. A flexible fleet of routers consumes work as a quorum, adapting their draw to find the healthiest throughput the database can sustain. That's how back-pressure is solved without anyone configuring it. NATS is the in-memory adjunct: it filters millions of events down to the ones that matter, and idempotent request handling guarantees exactly one workflow runs in response to each. Choreography this rich starts to look like orchestration — and orchestration, underneath, is just disciplined choreography.
-
----
-
-## The Transition
-
-The beautiful thing about this system is that it doesn't require a "big bang." It's additive:
-
-**Week 1:** An engineer adds activities and a workflow. Old school. Works great.
-
-**Week 2:** Someone discovers the Designer. Compiles a working run into a graph flow. Faster, no tokens.
-
-**Week 3:** A product manager creates their first agent from a conversation. It runs on a schedule.
-
-**Week 4:** The agent's compiled pipeline uses the engineer's activities alongside AI-compiled ones. No one planned this composition. The capability graph grew organically.
-
-**Month 3:** The system has 40 compiled tools, 8 agents, and 3 active knowledge domains. 94% of executions are deterministic. Cost has dropped. Reliability has increased. The team has institutional memory encoded in executable tools that survive personnel changes.
-
-**Month 6:** A new team member asks: "How do we handle the Aetna prior-auth process?" The answer isn't a wiki page. It's a compiled flow they can run, inspect, and modify. The knowledge is alive.
-
-**The turn:** none of this was built for AI — it was built for a human-only team. But every activity was authored as an MCP tool: schema-backed with Zod, discoverable, composable. Every compiled flow is itself an MCP server. So the same reflexive API the dashboard calls is open to an LLM under the same RBAC — a service account with exactly the access a person has. Claiming and resolving an escalation is just another tool call. The human-only core becomes the seed: humans first, then humans and AI sharing one set of tools, and eventually work that resolves itself. Nothing about the plumbing changes — only who pulls the next task.
-
----
-
-## The Name
-
-Long Tail — because it systematically shortens the tail. Every novel problem that gets solved compiles into a permanent capability. The tail of unsolved problems shrinks. The head of automated, reliable, institutional knowledge grows.
-
-What starts as "I need someone to check my email" becomes an agent that runs 24/7, learns from every interaction, escalates when uncertain, and evolves when the world changes.
-
-That's the story. Activities are permanent. Agents compose them. Humans guide the evolution. The system accumulates.