@hotmeshio/long-tail 0.1.4 → 0.1.6

package/docs/api/exports.md

@@ -0,0 +1,170 @@
+# Exports API
+
+Export endpoints expose workflow state and execution history. The execution history endpoint produces structured, typed events — useful for debugging, auditing, and migrating data. All endpoints require authentication.
+
+Every endpoint resolves the workflow automatically from the `workflowId` — no additional parameters needed.
+
+## Raw workflow state
+
+```
+GET /api/workflow-states/:workflowId
+```
+
+Exports the full workflow state using HotMesh's durable export. The response includes facets (data, state, status, timeline, transitions) that can be filtered.
+
+**Query parameters:**
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `allow` | `string` | no | Comma-separated allowlist of facets: `data`, `state`, `status`, `timeline`, `transitions` |
+| `block` | `string` | no | Comma-separated blocklist of facets |
+| `values` | `string` | no | Set to `false` to omit timeline values |
+
+`allow` and `block` are mutually exclusive. If both are provided, `allow` takes precedence.
+
+**Example request — only data and status:**
+
+```
+GET /api/workflow-states/reviewContent-a1b2c3d4?allow=data,status
+```
+
+**Response 200:** Raw workflow state object from HotMesh. Structure varies by facet selection.
+
+**Response 404:**
+
+```json
+{ "error": "No task found for workflow \"unknown-id\"" }
+```
+
+## Execution history
+
+```
+GET /api/workflow-states/:workflowId/execution
+```
+
+Exports the workflow's execution history as structured, typed events (`workflow_execution_started`, `activity_task_scheduled`, `activity_task_completed`, etc.) with ISO timestamps, durations, and cross-references.
+
+**Query parameters:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `excludeSystem` | `string` | no | `false` | Set to `true` to omit LT interceptor activities (`lt*`) |
+| `omitResults` | `string` | no | `false` | Set to `true` to strip activity result payloads |
+| `mode` | `string` | no | `sparse` | `sparse` (flat events) or `verbose` (includes nested child workflow executions) |
+| `maxDepth` | `integer` | no | `5` | Recursion depth limit for verbose mode |
+
+**Example request — clean execution history without system activities:**
+
+```
+GET /api/workflow-states/reviewContent-a1b2c3d4/execution?excludeSystem=true
+```
+
+**Response 200:**
+
+```json
+{
+  "workflowId": "reviewContent-a1b2c3d4",
+  "workflowName": "reviewContent",
+  "taskQueue": "long-tail",
+  "events": [
+    {
+      "eventId": 1,
+      "eventType": "workflow_execution_started",
+      "timestamp": "2025-01-15T10:00:00.000Z",
+      "details": {
+        "input": {
+          "data": { "contentId": "post-456" },
+          "metadata": {}
+        }
+      }
+    },
+    {
+      "eventId": 2,
+      "eventType": "activity_task_scheduled",
+      "timestamp": "2025-01-15T10:00:00.050Z",
+      "details": {
+        "activityType": "analyzeContent",
+        "taskQueue": "long-tail"
+      }
+    },
+    {
+      "eventId": 3,
+      "eventType": "activity_task_completed",
+      "timestamp": "2025-01-15T10:00:02.300Z",
+      "details": {
+        "scheduledEventId": 2,
+        "duration": "2.250s",
+        "result": { "confidence": 0.92 }
+      }
+    },
+    {
+      "eventId": 4,
+      "eventType": "workflow_execution_completed",
+      "timestamp": "2025-01-15T10:00:02.350Z",
+      "details": {
+        "result": { "approved": true }
+      }
+    }
+  ],
+  "summary": {
+    "totalEvents": 4,
+    "duration": "2.350s",
+    "status": "completed"
+  }
+}
+```
+
+**Verbose mode** includes nested `children` arrays for orchestrator workflows, each containing the child workflow's full event sequence. Use `maxDepth` to limit recursion for deeply nested orchestrations.
+
+## Workflow status
+
+```
+GET /api/workflow-states/:workflowId/status
+```
+
+Returns the numeric status semaphore from HotMesh.
+
+**Response 200:**
+
+```json
+{ "workflow_id": "reviewContent-a1b2c3d4", "status": 0 }
+```
+
+| Value | Meaning |
+|-------|---------|
+| `0` | Complete |
+| Positive | Running |
+| Negative | Interrupted |
+
+## Workflow state
+
+```
+GET /api/workflow-states/:workflowId/state
+```
+
+Returns the current job state of the workflow — the internal HotMesh representation of where the workflow is in its execution.
+
+**Response 200:** HotMesh job state object. Structure depends on the workflow's current execution point.
+
+## Programmatic access
+
+The same data is available programmatically through the Durable client:
+
+```typescript
+const client = new Durable.Client({ connection });
+const handle = await client.workflow.getHandle(
+  taskQueue,
+  workflowName,
+  workflowId,
+);
+
+// Execution history
+const execution = await handle.exportExecution({
+  exclude_system: true,
+});
+
+// Raw state export
+const state = await handle.export({
+  allow: ['data', 'status'],
+});
+```

package/docs/api/maintenance.md

@@ -0,0 +1,93 @@
+# Maintenance API
+
+The maintenance API configures automated database cleanup. Long Tail ships a default schedule that runs nightly at 2 AM. These endpoints let you inspect and replace the configuration at runtime. The PUT endpoint requires admin access.
+
+For background on what maintenance does and how rules work, see [Maintenance](../maintenance.md).
+
+## Get current configuration
+
+```
+GET /api/config/maintenance
+```
+
+**Response 200:**
+
+```json
+{
+  "config": {
+    "schedule": "0 2 * * *",
+    "rules": [
+      { "target": "streams", "olderThan": "7 days", "action": "delete" },
+      { "target": "jobs", "olderThan": "7 days", "action": "delete", "hasEntity": false },
+      { "target": "jobs", "olderThan": "7 days", "action": "prune", "hasEntity": true },
+      { "target": "jobs", "olderThan": "90 days", "action": "delete", "pruned": true }
+    ]
+  },
+  "active": true
+}
+```
+
+The `active` field indicates whether the maintenance cron is currently running.
+
+## Update configuration
+
+```
+PUT /api/config/maintenance
+```
+
+Replaces the maintenance configuration and restarts the cron. The endpoint disconnects the current cron (if running), registers the new config, and starts a new cron. This is atomic from the caller's perspective.
+
+**Requires:** Admin access (`requireAdmin` middleware).
+
+**Request body:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `schedule` | `string` | yes | Cron expression (`0 3 * * *`) or interval (`1 day`) |
+| `rules` | `array` | yes | Ordered list of maintenance rules |
+
+Each rule:
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `target` | `string` | yes | `streams` or `jobs` |
+| `action` | `string` | yes | `delete` or `prune` |
+| `olderThan` | `string` | yes | PostgreSQL interval (e.g., `7 days`, `24 hours`, `90 days`) |
+| `hasEntity` | `boolean` | no | `true` for entity jobs, `false` for transient jobs |
+| `pruned` | `boolean` | no | `true` to target only already-pruned jobs |
+
+**Example request:**
+
+```json
+{
+  "schedule": "0 3 * * *",
+  "rules": [
+    { "target": "streams", "olderThan": "24 hours", "action": "delete" },
+    { "target": "jobs", "olderThan": "14 days", "action": "delete", "hasEntity": false },
+    { "target": "jobs", "olderThan": "14 days", "action": "prune", "hasEntity": true },
+    { "target": "jobs", "olderThan": "180 days", "action": "delete", "pruned": true }
+  ]
+}
+```
+
+**Response 200:**
+
+```json
+{
+  "config": {
+    "schedule": "0 3 * * *",
+    "rules": [ ... ]
+  },
+  "restarted": true
+}
+```
+
+The `restarted` field confirms the cron was restarted with the new configuration.
+
+### What happens on PUT
+
+1. The current cron is cancelled (`maintenanceRegistry.disconnect()`).
+2. The new config is stored (`maintenanceRegistry.register(config)`).
+3. A new cron is started (`maintenanceRegistry.connect()`).
+
+If the cron was not previously running, steps 1 is a no-op. The net effect is that the old schedule is replaced and the new one takes effect immediately.

package/docs/api/mcp-runs.md

@@ -0,0 +1,128 @@
+# MCP Runs API (Process Server Runs)
+
+Query execution runs from MCP process servers. Runs are stored in HotMesh's `{namespace}.jobs` table, one schema per namespace. All endpoints require authentication.
+
+## List runs
+
+```
+GET /api/mcp-runs
+```
+
+**Query parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `app_id` | `string` | **Required.** Namespace to query (e.g., `longtail`, `lt-yaml`) |
+| `entity` | `string` | Filter by tool/entity name (e.g., `rotate_and_verify`) |
+| `search` | `string` | Search by workflow ID substring (case-insensitive) |
+| `status` | `string` | Filter by status: `running`, `completed`, `failed` |
+| `limit` | `integer` | Max results (default: 20, max: 100) |
+| `offset` | `integer` | Pagination offset (default: 0) |
+
+**Response 200:**
+
+```json
+{
+  "jobs": [
+    {
+      "workflow_id": "abc123...",
+      "entity": "rotate_and_verify",
+      "status": "completed",
+      "is_live": false,
+      "created_at": "2025-01-15T10:00:00Z",
+      "updated_at": "2025-01-15T10:00:05Z"
+    }
+  ],
+  "total": 1
+}
+```
+
+**Response 400:** Missing `app_id`.
+
+## List entities (tools)
+
+```
+GET /api/mcp-runs/entities
+```
+
+Returns distinct entity names from job runs, supplemented with `graph_topic` values from active/deployed YAML workflows for the namespace.
+
+**Query parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `app_id` | `string` | **Required.** Namespace to query |
+
+**Response 200:**
+
+```json
+{
+  "entities": ["extract_info", "rotate_and_verify"]
+}
+```
+
+Entities are sorted alphabetically and exclude null/empty values.
+
+**Response 400:** Missing `app_id`.
+
+## Get execution detail
+
+```
+GET /api/mcp-runs/:jobId/execution
+```
+
+Returns full execution detail for a specific run, including inflated activity events, trace IDs, and a summary.
+
+**Query parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `app_id` | `string` | **Required.** Namespace to query |
+
+**Response 200:**
+
+```json
+{
+  "workflow_id": "abc123...",
+  "workflow_type": "rotate_and_verify",
+  "workflow_name": "rotate_and_verify",
+  "task_queue": "lt-yaml",
+  "status": "completed",
+  "start_time": "2025-01-15T10:00:00.000Z",
+  "close_time": "2025-01-15T10:00:05.230Z",
+  "duration_ms": 5230,
+  "trace_id": "a5fb792464c1d6c7e692824f0ceb011d",
+  "result": { "rotated_url": "https://...", "verified": true },
+  "events": [
+    {
+      "event_id": 1,
+      "event_type": "workflow_execution_started",
+      "category": "workflow",
+      "event_time": "2025-01-15T10:00:00.000Z",
+      "duration_ms": null,
+      "is_system": false,
+      "attributes": { "kind": "workflow_execution_started", "workflow_type": "rotate_and_verify" }
+    },
+    {
+      "event_id": 2,
+      "event_type": "activity_task_scheduled",
+      "category": "activity",
+      "event_time": "2025-01-15T10:00:00.100Z",
+      "duration_ms": null,
+      "is_system": false,
+      "attributes": { "kind": "activity_task_scheduled", "activity_type": "rotate_image" }
+    }
+  ],
+  "summary": {
+    "total_events": 5,
+    "activities": { "total": 2, "completed": 2, "failed": 0, "system": 1, "user": 1 },
+    "child_workflows": { "total": 0, "completed": 0, "failed": 0 },
+    "timers": 0,
+    "signals": 0
+  }
+}
+```
+
+**Response 400:** Missing `app_id`.
+
+**Response 404:** Job not found.

package/docs/api/mcp-servers.md

@@ -0,0 +1,253 @@
+# MCP Servers API
+
+Manage MCP server registrations, connections, and tool invocations. All endpoints require authentication.
+
+## List servers
+
+```
+GET /api/mcp/servers
+```
+
+**Query parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `status` | `string` | Filter by status: `registered`, `connected`, `error`, `disconnected` |
+| `search` | `string` | Search server name, description, or tool names (case-insensitive) |
+| `auto_connect` | `boolean` | Filter by auto-connect flag |
+| `tags` | `string` | Comma-separated tags for tag-based filtering |
+| `limit` | `integer` | Max results (default: 50) |
+| `offset` | `integer` | Pagination offset (default: 0) |
+
+**Response 200:**
+
+```json
+{
+  "servers": [
+    {
+      "id": "a1b2c3d4-...",
+      "name": "mcp-vision-longtail",
+      "description": "Vision analysis tools",
+      "transport_type": "stdio",
+      "transport_config": { "command": "node", "args": ["server.js"] },
+      "auto_connect": true,
+      "status": "connected",
+      "tool_manifest": [
+        { "name": "analyze_image", "description": "Analyze an image", "inputSchema": {} }
+      ],
+      "tags": ["vision", "image-analysis"],
+      "compile_hints": "Use for image processing tasks",
+      "credential_providers": ["openai"],
+      "metadata": { "builtin": true },
+      "created_at": "2025-01-01T00:00:00Z",
+      "updated_at": "2025-01-01T00:00:00Z"
+    }
+  ],
+  "total": 1
+}
+```
+
+## Register a server
+
+```
+POST /api/mcp/servers
+```
+
+**Request body:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | `string` | Yes | Unique server name |
+| `description` | `string` | No | Human-readable description |
+| `transport_type` | `string` | Yes | `stdio`, `sse`, or `streamable-http` |
+| `transport_config` | `object` | Yes | Transport-specific config (see below) |
+| `auto_connect` | `boolean` | No | Connect on startup (default: false) |
+| `metadata` | `object` | No | Arbitrary metadata |
+| `tags` | `string[]` | No | Tags for tool discovery (e.g., `["database", "analytics"]`) |
+| `compile_hints` | `string` | No | Guidance for the workflow compiler when using this server's tools |
+| `credential_providers` | `string[]` | No | IAM providers required by tools (e.g., `["github", "slack"]`) |
+
+**Transport config for `stdio`:**
+
+```json
+{ "command": "node", "args": ["server.js"], "env": {} }
+```
+
+**Transport config for `sse`:**
+
+```json
+{ "url": "http://localhost:3001/sse" }
+```
+
+**Transport config for `streamable-http`:**
+
+```json
+{ "url": "http://localhost:3001/mcp" }
+```
+
+**Response 201:** Created server record.
+
+**Response 409:** Server name already exists.
+
+## Get a server
+
+```
+GET /api/mcp/servers/:id
+```
+
+**Response 200:** Server record.
+
+**Response 404:** Server not found.
+
+## Update a server
+
+```
+PUT /api/mcp/servers/:id
+```
+
+**Request body:** Any fields from the registration (partial update), including `tags`, `compile_hints`, and `credential_providers`.
+
+**Response 200:** Updated server record.
+
+**Response 404:** Server not found.
+
+## Delete a server
+
+```
+DELETE /api/mcp/servers/:id
+```
+
+**Response 200:**
+
+```json
+{ "deleted": true }
+```
+
+**Response 404:** Server not found.
+
+## Test connection
+
+```
+POST /api/mcp/servers/test-connection
+```
+
+Test connectivity to an MCP server without persisting a registration. Creates a temporary client, connects, lists tools, then disconnects.
+
+**Request body:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `transport_type` | `string` | Yes | `stdio`, `sse`, or `streamable-http` |
+| `transport_config` | `object` | Yes | Transport-specific config |
+
+**Response 200:**
+
+```json
+{
+  "success": true,
+  "tools": [
+    { "name": "analyze_image", "description": "Analyze an image", "inputSchema": {} }
+  ]
+}
+```
+
+On failure:
+
+```json
+{
+  "success": false,
+  "tools": [],
+  "error": "Connection refused"
+}
+```
+
+## Connect to a server
+
+```
+POST /api/mcp/servers/:id/connect
+```
+
+Establishes a connection to the registered MCP server and caches its tool manifest.
+
+**Response 200:**
+
+```json
+{ "connected": true, "serverId": "a1b2c3d4-..." }
+```
+
+## Disconnect from a server
+
+```
+POST /api/mcp/servers/:id/disconnect
+```
+
+**Response 200:**
+
+```json
+{ "disconnected": true, "serverId": "a1b2c3d4-..." }
+```
+
+## List tools
+
+```
+GET /api/mcp/servers/:id/tools
+```
+
+Returns tools available on a connected MCP server.
+
+**Response 200:**
+
+```json
+{
+  "tools": [
+    {
+      "name": "analyze_image",
+      "description": "Analyze an image using vision models",
+      "inputSchema": {
+        "type": "object",
+        "properties": {
+          "image_url": { "type": "string" }
+        }
+      }
+    }
+  ]
+}
+```
+
+## Call a tool
+
+```
+POST /api/mcp/servers/:id/tools/:toolName/call
+```
+
+Invoke a tool on a connected MCP server.
+
+**Request body:**
+
+```json
+{
+  "arguments": {
+    "image_url": "https://example.com/image.png"
+  }
+}
+```
+
+**Response 200:**
+
+```json
+{
+  "result": {
+    "content": [{ "type": "text", "text": "Analysis result..." }]
+  }
+}
+```
+
+**Response 422** (missing credential):
+
+```json
+{
+  "error": "missing_credential",
+  "provider": "openai",
+  "message": "Credential required for provider: openai"
+}
+```

package/docs/api/namespaces.md

@@ -0,0 +1,48 @@
+# Namespaces API
+
+Namespaces organize MCP process servers. Each namespace maps to a separate PostgreSQL schema used by HotMesh for workflow execution. Namespaces are auto-registered when YAML workflows are deployed. All endpoints require authentication.
+
+## List namespaces
+
+```
+GET /api/namespaces
+```
+
+**Response 200:**
+
+```json
+{
+  "namespaces": [
+    {
+      "id": "a1b2c3d4-...",
+      "name": "lt-yaml",
+      "description": "YAML workflow process server",
+      "schema_name": "lt-yaml",
+      "is_default": false,
+      "metadata": null,
+      "created_at": "2025-01-01T00:00:00Z",
+      "updated_at": "2025-01-01T00:00:00Z"
+    }
+  ]
+}
+```
+
+## Register a namespace
+
+```
+POST /api/namespaces
+```
+
+Creates or upserts a namespace. Typically called automatically during YAML workflow deployment.
+
+**Request body:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | `string` | Yes | Unique namespace name (becomes the HotMesh app ID and schema name) |
+| `description` | `string` | No | Human-readable description |
+| `metadata` | `object` | No | Arbitrary metadata |
+
+**Response 200:** Namespace record (created or existing).
+
+**Response 400:** Missing `name`.