@hotmeshio/long-tail 0.2.10 → 0.3.1

package/docs/api/http/topics.md

@@ -0,0 +1,164 @@
+# Topics API
+
+Browse, register, and manage the topic catalog — a persistent registry of known event topics with descriptions, payload schemas, and subscriber counts.
+
+All endpoints require authentication.
+
+## List topics
+
+```
+GET /api/topics?category=task&search=created&limit=50&offset=0
+```
+
+Returns topics with subscriber counts (computed via JOIN on active subscriptions).
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `category` | `string` | No | Filter by category: `task`, `workflow`, `escalation`, `activity`, `knowledge`, `agent`, `app`, `milestone` |
+| `search` | `string` | No | Search topic name or description (case-insensitive) |
+| `limit` | `number` | No | Max results (default: 50) |
+| `offset` | `number` | No | Pagination offset |
+
+**Response 200:**
+
+```json
+{
+  "topics": [
+    {
+      "topic": "task.created",
+      "description": "A new task has been created and queued for execution.",
+      "category": "task",
+      "payload_schema": {
+        "type": "object",
+        "properties": {
+          "taskId": { "type": "string" },
+          "status": { "type": "string" },
+          "workflowId": { "type": "string" }
+        }
+      },
+      "example_payload": { "taskId": "tsk-001", "status": "pending" },
+      "source": "system",
+      "tags": ["lifecycle", "core"],
+      "subscriber_count": 2,
+      "last_seen_at": "2026-05-20T12:00:00Z",
+      "created_at": "2026-05-20T00:00:00Z",
+      "updated_at": "2026-05-20T00:00:00Z"
+    }
+  ],
+  "total": 22
+}
+```
+
+## Get topic
+
+```
+GET /api/topics/by-name/:topic
+```
+
+Returns a single topic with its full schema, example payload, and a list of agents whose subscription patterns match this topic.
+
+The `:topic` parameter must be URL-encoded (e.g., `task.created` becomes `task.created` — Express auto-decodes).
+
+**Response 200:**
+
+```json
+{
+  "topic": "task.created",
+  "description": "A new task has been created and queued for execution.",
+  "category": "task",
+  "payload_schema": { "..." },
+  "example_payload": { "..." },
+  "source": "system",
+  "tags": ["lifecycle", "core"],
+  "subscribers": [
+    {
+      "id": "uuid",
+      "agent_id": "uuid",
+      "agent_name": "health-monitor",
+      "topic": "task.*",
+      "reaction_type": "durable"
+    }
+  ],
+  "created_at": "2026-05-20T00:00:00Z",
+  "updated_at": "2026-05-20T00:00:00Z"
+}
+```
+
+Subscribers are matched using NATS-style pattern matching — an agent subscribed to `task.*` appears on the `task.created` detail page.
+
+**Response 404:** `{ "error": "Topic not found" }`
+
+## Register topic
+
+```
+POST /api/topics
+```
+
+Manually register a topic in the catalog. Use this to pre-declare topics before first publish.
+
+**Body:**
+
+```json
+{
+  "topic": "app.vendor.orders.created",
+  "description": "Fired when a new order is placed.",
+  "category": "app",
+  "payload_schema": {
+    "type": "object",
+    "properties": {
+      "orderId": { "type": "string" },
+      "total": { "type": "number" }
+    }
+  },
+  "tags": ["orders", "lifecycle"]
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `topic` | `string` | Yes | Unique topic name (dot-delimited) |
+| `category` | `string` | Yes | One of: `task`, `workflow`, `escalation`, `activity`, `knowledge`, `agent`, `app`, `milestone` |
+| `description` | `string` | No | Human-readable description |
+| `payload_schema` | `object` | No | JSON Schema for `event.data` |
+| `example_payload` | `object` | No | Concrete example of `event.data` |
+| `source` | `string` | No | Source identifier (default: `'app'`) |
+| `tags` | `string[]` | No | Filterable tags |
+
+**Response 201:** The created topic.
+
+**Response 409:** `{ "error": "Topic \"app.vendor.orders.created\" already exists" }`
+
+## Update topic
+
+```
+PUT /api/topics/by-name/:topic
+```
+
+Partial update — only include fields to change. System topics (`source: 'system'`) can be updated (description, tags) but not deleted.
+
+**Body:**
+
+```json
+{
+  "description": "Updated description",
+  "tags": ["orders", "lifecycle", "critical"]
+}
+```
+
+**Response 200:** The updated topic.
+
+**Response 404:** `{ "error": "Topic not found" }`
+
+## Delete topic
+
+```
+DELETE /api/topics/by-name/:topic
+```
+
+Permanently removes a topic from the catalog. System topics are protected.
+
+**Response 200:** `{ "deleted": true }`
+
+**Response 403:** `{ "error": "System topics cannot be deleted" }`
+
+**Response 404:** `{ "error": "Topic not found" }`

package/docs/api/sdk/topics.md

@@ -0,0 +1,204 @@
+# TopicService
+
+Topic catalog operations — browse, register, and manage event topic definitions.
+
+Imported as a namespace:
+
+```typescript
+import { TopicService } from '@hotmeshio/long-tail';
+```
+
+## listTopics
+
+List topics with optional filters.
+
+```typescript
+const result = await TopicService.listTopics({
+  category: 'task',
+  search: 'failed',
+  limit: 50,
+  offset: 0,
+});
+// result.topics: TopicCatalogEntry[]
+// result.total: number
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `category` | `string` | No | Filter by category |
+| `search` | `string` | No | Search name or description |
+| `limit` | `number` | No | Max results (default: 50) |
+| `offset` | `number` | No | Pagination offset |
+
+**Returns:** `{ topics: TopicCatalogEntry[], total: number }`
+
+## getTopic
+
+Get a single topic with its subscriber list.
+
+```typescript
+const topic = await TopicService.getTopic('task.created');
+// topic.payload_schema — JSON Schema for event.data
+// topic.subscribers — agents whose patterns match this topic
+```
+
+**Returns:** `TopicCatalogEntry & { subscribers: TopicSubscriber[] }` or `null`
+
+Subscribers are matched using NATS-style pattern matching. An agent subscribed to `task.*` appears in the `task.created` subscriber list.
+
+## createTopic
+
+Register a new topic in the catalog.
+
+```typescript
+const entry = await TopicService.createTopic({
+  topic: 'app.vendor.orders.created',
+  description: 'Fired when a new order is placed.',
+  category: 'app',
+  payload_schema: {
+    type: 'object',
+    properties: {
+      orderId: { type: 'string' },
+      total: { type: 'number' },
+    },
+  },
+  tags: ['orders', 'lifecycle'],
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `topic` | `string` | Yes | Unique topic name |
+| `category` | `string` | Yes | Topic category |
+| `description` | `string` | No | Human-readable description |
+| `payload_schema` | `object` | No | JSON Schema for `event.data` |
+| `example_payload` | `object` | No | Example `event.data` |
+| `source` | `string` | No | Source identifier (default: `'app'`) |
+| `tags` | `string[]` | No | Tags for filtering |
+
+**Returns:** `TopicCatalogEntry`
+
+**Throws:** Postgres unique violation if topic already exists.
+
+## updateTopic
+
+Partial update of a topic's metadata.
+
+```typescript
+const updated = await TopicService.updateTopic('app.vendor.orders.created', {
+  description: 'Updated description',
+  tags: ['orders', 'critical'],
+});
+```
+
+**Returns:** `TopicCatalogEntry | null`
+
+## deleteTopic
+
+Delete a topic. The SQL guards against deleting system topics (`source != 'system'`).
+
+```typescript
+const deleted = await TopicService.deleteTopic('app.vendor.orders.created');
+// deleted: true/false
+```
+
+**Returns:** `boolean`
+
+## seedTopic
+
+Insert-if-absent. Used at startup for first-boot seeding.
+
+```typescript
+const inserted = await TopicService.seedTopic({
+  topic: 'app.orders.created',
+  description: 'New order placed',
+  category: 'app',
+  tags: ['orders'],
+});
+// inserted: true if new, false if already existed
+```
+
+**Returns:** `boolean`
+
+## resetTopic
+
+Upsert from config — overwrites description, category, schema, and tags on every call. Used when `reset: true` is set in static config.
+
+```typescript
+await TopicService.resetTopic({
+  topic: 'app.orders.created',
+  description: 'New order placed (updated)',
+  category: 'app',
+  payload_schema: { type: 'object', properties: { orderId: { type: 'string' } } },
+});
+```
+
+**Returns:** `void`
+
+## upsertTopicOnPublish
+
+Auto-register or update a topic when an event is published. Creates the entry on first publish; updates `last_seen_at` and `example_payload` on subsequent publishes.
+
+```typescript
+await TopicService.upsertTopicOnPublish(
+  'app.vendor.orders.error',
+  { orderId: '123', error: 'timeout' },  // example data
+  'order-processor',                       // source
+);
+```
+
+Called automatically by the `publish_event` MCP tool. You only need this for manual publish paths.
+
+**Returns:** `void`
+
+---
+
+## Types
+
+### TopicCatalogEntry
+
+```typescript
+interface TopicCatalogEntry {
+  topic: string;
+  description?: string;
+  category: string;
+  payload_schema?: Record<string, any>;
+  example_payload?: Record<string, any>;
+  source: string;            // 'system', 'config', 'app', 'mcp-tool'
+  tags: string[];
+  subscriber_count?: number; // populated by listTopics
+  last_seen_at?: string;     // ISO 8601, updated on each publish
+  created_at: string;
+  updated_at: string;
+}
+```
+
+### TopicSubscriber
+
+```typescript
+interface TopicSubscriber {
+  id: string;
+  agent_id: string;
+  agent_name: string;
+  topic: string;             // the subscription pattern
+  reaction_type: string;     // 'durable', 'pipeline', 'mcp_query'
+}
+```
+
+## Seeding helpers
+
+For advanced startup flows, two seeding functions are also exported from the package root:
+
+```typescript
+import { seedSystemTopics, seedConfigTopics } from '@hotmeshio/long-tail';
+
+// Seed the 22 built-in system topics
+await seedSystemTopics();
+
+// Seed user-declared topics (respects reset: true)
+await seedConfigTopics([
+  { topic: 'app.orders.created', description: 'Order placed', reset: true },
+]);
+```
+
+These are called automatically by `start()`. You only need them if you're building a custom startup flow.

package/docs/dashboard.md

@@ -23,6 +23,15 @@ The sidebar organizes pages into five groups.
 | **MCP Pipeline Tools** | `/mcp/workflows` | YAML pipeline tools available to the orchestrator. Shows compiled deterministic workflows. |
 | **Pipeline Executions** | `/mcp/executions` | Execution history for MCP pipelines — both dynamic (agentic) and compiled (deterministic) runs. |
 
+### Work
+
+| Page | Route | Purpose |
+|------|-------|---------|
+| **Recent Activity** | `/` | Live event stream and business process overview. |
+| **Capabilities** | `/capabilities` | Browse MCP servers grouped by capability category. |
+| **Agents** | `/agents` | Autonomous event-driven agents. Configure subscriptions, schedules, and knowledge domains. |
+| **Topics** | `/topics` | Topic catalog — browse all known event topics with descriptions, schemas, and subscriber counts. |
+
 ### Storage
 
 | Page | Route | Purpose |
@@ -283,6 +292,19 @@ Knowledge entries are accessed by workflows via the `get_knowledge` MCP tool. Th
 
 **API:** `GET /api/knowledge` lists entries. `PUT /api/knowledge/:domain/:key` creates or updates. `DELETE /api/knowledge/:domain/:key` removes.
 
+### Topic Catalog
+
+The persistent registry of all known event topics. Browse, search, and inspect what the event bus carries.
+
+- **Topic list** — all registered topics with category pills, descriptions, subscriber counts, and last-seen timestamps. Filter by category (task, workflow, escalation, activity, knowledge, agent, app, milestone).
+- **Topic detail** — click any topic to see its full payload schema (JSON Schema), example payload, tags, and a list of agents whose subscription patterns match the topic.
+- **Subscriber discovery** — the detail page uses NATS-style pattern matching to show all agents that would receive this event. An agent subscribed to `task.*` appears on every `task.created`, `task.failed`, etc. detail page.
+- **Schema preview in subscriptions** — when editing an agent's subscriptions, selecting a topic from the catalog shows its payload schema inline, so you know what `{event.data.*}` fields are available for input mapping.
+
+Topics enter the catalog three ways: system topics are seeded at startup (22 built-in), config topics are declared in `startConfig.topics[]`, and runtime topics are auto-discovered when `publish_event` fires for the first time.
+
+**API:** `GET /api/topics` lists topics. `GET /api/topics/by-name/:topic` returns detail with subscribers. `POST /api/topics` registers a new topic. See [Topics HTTP API](api/http/topics.md) and [TopicService SDK](api/sdk/topics.md).
+
 ### Credentials
 
 Accessible via the user menu (or at `/credentials`). Manage OAuth provider connections and API keys for the current user.

package/docs/events.md

@@ -296,3 +296,10 @@ The activity interceptor inspects every activity result. If the result contains
 ### Delivery Semantics
 
 All three call sites use `publishMilestoneEvent()`, which is fire-and-forget. It returns immediately, never throws, and swallows errors. Events are a non-durable side effect: they are not replayed on workflow recovery. If the process crashes between task completion and event publication, the event is lost. Design downstream consumers accordingly.
+
+## Related
+
+- [Topic Catalog](topics.md) — persistent registry of known topics with schemas and discovery
+- [Agents: Subscriptions](agents.md#subscriptions) — wiring topics to reactive workflows
+- [Topics HTTP API](api/http/topics.md) — REST endpoints for the catalog
+- [Topics SDK](api/sdk/topics.md) — `TopicService` programmatic access

package/docs/sdk.md

@@ -81,6 +81,21 @@ The client mirrors the REST route structure:
 | `lt.namespaces` | `/api/namespaces` | `list`, `register` |
 | `lt.maintenance` | `/api/config/maintenance` | `getConfig`, `updateConfig` |
 
+### Direct imports
+
+Some services are exported as namespaces for direct import rather than through the client:
+
+```typescript
+import { TopicService } from '@hotmeshio/long-tail';
+
+const { topics, total } = await TopicService.listTopics({ category: 'task' });
+const detail = await TopicService.getTopic('task.created');
+```
+
+| Import | Key operations | Docs |
+|--------|---------------|------|
+| `TopicService` | `listTopics`, `getTopic`, `createTopic`, `updateTopic`, `deleteTopic`, `seedTopic`, `resetTopic` | [TopicService](api/sdk/topics.md) |
+
 ## Events
 
 The SDK includes a callback-based event adapter. Subscribe to events directly — no socket.io client, no WebSocket connection.

package/docs/topics.md

@@ -0,0 +1,154 @@
+# Topic Catalog
+
+The topic catalog is a persistent registry of known event topics. Every topic has a name, description, payload schema, and metadata — making the event bus discoverable, documented, and queryable.
+
+Topics are the foundation of agentic automation. Agents subscribe to topics and react when events fire. The catalog tells agents (and humans) what topics exist, what data they carry, and who's listening.
+
+## How topics enter the catalog
+
+Topics arrive through three paths:
+
+### 1. System topics (built-in)
+
+The platform seeds 22 system topics at startup — one for every `LTEventType`. These describe the lifecycle events the platform emits automatically: task, workflow, escalation, activity, knowledge, agent, and milestone events.
+
+System topics are read-only. They can't be deleted or renamed. Their schemas are derived directly from the publish helpers in the source code.
+
+### 2. Config topics (declared in code)
+
+Projects declare topics in `startConfig.topics[]`. This is the primary way applications document what they publish:
+
+```typescript
+import { start } from '@hotmeshio/long-tail';
+
+await start({
+  database: { connectionString: process.env.DATABASE_URL },
+  workers: [{ taskQueue: 'orders', workflow: processOrder }],
+
+  topics: [
+    {
+      topic: 'app.orders.created',
+      description: 'Fired when a new order is placed.',
+      payload_schema: {
+        type: 'object',
+        properties: {
+          orderId: { type: 'string' },
+          total: { type: 'number' },
+          customer: { type: 'string' },
+        },
+      },
+      example_payload: { orderId: 'ord-123', total: 99.99, customer: 'acme' },
+      tags: ['orders', 'lifecycle'],
+    },
+    {
+      topic: 'app.orders.failed',
+      description: 'Fired when order processing fails.',
+      payload_schema: {
+        type: 'object',
+        properties: {
+          orderId: { type: 'string' },
+          error: { type: 'string' },
+        },
+      },
+      tags: ['orders', 'error'],
+      reset: true,  // config is source of truth — overwrite DB on every boot
+    },
+  ],
+});
+```
+
+Config topics are seeded on first boot (insert-if-absent). The database owns the record after that — dashboard edits stick.
+
+### 3. Runtime topics (auto-discovered)
+
+When the `publish_event` MCP tool fires with a topic that isn't in the catalog, the system auto-registers it. The topic appears in the catalog with the payload as an example. This is learn-on-first-use — topics accumulate over time.
+
+## The reset flag
+
+By default, config topics follow insert-if-absent semantics. Once seeded, the database owns the record. Dashboard edits persist across restarts.
+
+Set `reset: true` to make config the source of truth:
+
+```typescript
+{
+  topic: 'app.orders.created',
+  description: 'Updated description from code',
+  reset: true,
+}
+```
+
+With `reset: true`:
+- Every boot overwrites description, schema, tags from the config
+- Dashboard edits are transient — next deploy resets them
+- The topic definition lives in git, reviewed in PRs, enforced by CI/CD
+
+Without `reset`:
+- First boot seeds the record
+- Dashboard edits persist
+- Config changes in code are ignored after first boot
+
+## Topic naming
+
+Topics are dot-delimited strings. The convention:
+
+| Prefix | Who publishes | Example |
+|--------|--------------|---------|
+| `task.*` | Platform (automatic) | `task.created`, `task.failed` |
+| `workflow.*` | Platform (automatic) | `workflow.completed` |
+| `escalation.*` | Platform (automatic) | `escalation.resolved` |
+| `activity.*` | Platform (automatic) | `activity.started` |
+| `knowledge.*` | Platform (automatic) | `knowledge.stored` |
+| `agent.*` | Platform (automatic) | `agent.status_changed` |
+| `milestone` | Platform (automatic) | `milestone` |
+| `app.*` | Application code | `app.vendor.orders.created` |
+
+Application topics follow `app.{namespace}.{entity}.{action}`. The `app.` prefix is auto-added by the `publish_event` MCP tool if omitted.
+
+## Payload schemas
+
+Each topic can carry a JSON Schema describing the `event.data` shape. This serves two purposes:
+
+1. **Documentation** — developers and agents know what fields are available
+2. **Input mapping** — when wiring a subscription, the schema shows what `{event.data.*}` templates resolve to
+
+System topics have schemas derived from the publish helpers. Config topics declare schemas explicitly. Runtime topics learn by example.
+
+## Categories
+
+Every topic belongs to a category: `task`, `workflow`, `escalation`, `activity`, `knowledge`, `agent`, `app`, or `milestone`. Categories are inferred from the first segment of the topic name. The dashboard uses categories for filtering and color-coding.
+
+## Subscriber discovery
+
+The topic detail view shows which agents subscribe. This uses the same NATS-style pattern matching as the event bus — if an agent subscribes to `task.*`, it appears on every `task.created`, `task.failed`, etc. detail page.
+
+## MCP tools
+
+Two MCP tools give agents programmatic access to the catalog:
+
+- **`list_topics`** — browse topics by category or search term
+- **`register_topic`** — declare a topic with schema before first publish
+
+This enables autonomous wiring: an agent can discover available topics, read their schemas, and create subscriptions — all through tool calls.
+
+## LTTopicConfig
+
+The TypeScript interface for static topic declarations:
+
+```typescript
+interface LTTopicConfig {
+  topic: string;
+  description?: string;
+  category?: string;           // defaults to first segment or 'app'
+  payload_schema?: object;     // JSON Schema for event.data
+  example_payload?: object;    // concrete example
+  tags?: string[];
+  reset?: boolean;             // true = config is source of truth
+}
+```
+
+## Related
+
+- [Events](events.md) — transport adapters, event registry, publishing
+- [Agents: Subscriptions](agents.md#subscriptions) — wiring topics to workflows
+- [Topics HTTP API](api/http/topics.md) — REST endpoints
+- [Topics SDK](api/sdk/topics.md) — `TopicService` namespace

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/long-tail",
-  "version": "0.2.10",
+  "version": "0.3.1",
   "description": "Long Tail Workflows — Durable AI workflows with human-in-the-loop escalation. Powered by PostgreSQL.",
   "main": "./build/index.js",
   "types": "./build/index.d.ts",
@@ -70,7 +70,7 @@
     "@anthropic-ai/sdk": "^0.92.0",
     "@aws-sdk/client-s3": "^3.1017.0",
     "@aws-sdk/s3-request-presigner": "^3.1045.0",
-    "@hotmeshio/hotmesh": "^0.16.6",
+    "@hotmeshio/hotmesh": "^0.17.1",
     "@modelcontextprotocol/sdk": "^1.27.1",
     "@opentelemetry/exporter-trace-otlp-proto": "^0.215.0",
     "@opentelemetry/resources": "^2.5.1",

package/dashboard/dist/assets/AdminDashboard-45Ya2uzl.js

@@ -1,2 +0,0 @@
-import{s as n,j as o}from"./index-Da8R5Fn_.js";import{c as m}from"./tasks-BLCzIHol.js";import{f as d}from"./useEventHooks-AwwZnweF.js";import{u}from"./mcp-CUFhWTyT.js";import{P as c}from"./PageHeaderWithStats-cQKaJBAM.js";import"./vendor-query-DLp59M9_.js";import"./vendor-react-Co3Y8ikm.js";import"./vendor-icons-1qQ3NKfa.js";function N(){var e;d();const{data:a}=m({limit:1}),{data:t}=n({status:"pending",limit:1}),{data:s}=u(),i=((e=s==null?void 0:s.servers)==null?void 0:e.filter(l=>l.status==="connected").length)??0,r=s?`${i}/${s.total}`:"—";return o.jsxDEV("div",{children:o.jsxDEV(c,{title:"Admin",stats:[{label:"Tasks",value:(a==null?void 0:a.total)??"—"},{label:"Pending Escalations",value:(t==null?void 0:t.total)??"—",dotClass:"bg-status-pending"},{label:"MCP Servers",value:r},{label:"System",value:"Healthy",dotClass:"bg-status-success"}]},void 0,!1,{fileName:"/app/dashboard/src/pages/admin/AdminDashboard.tsx",lineNumber:19,columnNumber:7},this)},void 0,!1,{fileName:"/app/dashboard/src/pages/admin/AdminDashboard.tsx",lineNumber:18,columnNumber:5},this)}export{N as AdminDashboard};
-//# sourceMappingURL=AdminDashboard-45Ya2uzl.js.map