@hotmeshio/long-tail 0.1.14 → 0.1.16

package/docs/api/http/knowledge.md

@@ -0,0 +1,203 @@
+# Knowledge API
+
+CRUD operations for the knowledge store — a domain-scoped key-value store backed by PostgreSQL JSONB. Each entry has a domain, key, arbitrary JSON data, and optional tags.
+
+All endpoints require authentication.
+
+## List domains
+
+```
+GET /api/knowledge/domains
+```
+
+Returns all knowledge domains with entry counts and last-updated timestamps.
+
+**Response 200:**
+
+```json
+{
+  "domains": [
+    { "domain": "screenshot", "count": 3, "latest": "2026-05-09T22:26:51.616Z" },
+    { "domain": "config", "count": 1, "latest": "2026-05-08T10:00:00.000Z" }
+  ]
+}
+```
+
+## List entries
+
+```
+GET /api/knowledge/entries?domain=screenshot&search=google&tags=web&limit=50&offset=0
+```
+
+Returns entries within a domain, ordered by last updated.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain to query |
+| `search` | `string` | No | Search by key name or tag (ILIKE partial match) |
+| `tags` | `string` | No | Comma-separated tag filter (array overlap) |
+| `limit` | `number` | No | Max results (default: 50, max: 200) |
+| `offset` | `number` | No | Pagination offset |
+
+**Response 200:**
+
+```json
+{
+  "entries": [
+    {
+      "id": "uuid",
+      "domain": "screenshot",
+      "key": "google",
+      "data": { "2026-05-09": "description..." },
+      "tags": ["web"],
+      "created_at": "2026-05-09T22:26:51.616Z",
+      "updated_at": "2026-05-09T22:26:51.616Z"
+    }
+  ],
+  "total": 1
+}
+```
+
+## Get entry
+
+```
+GET /api/knowledge/entry?domain=screenshot&key=google
+```
+
+Returns a single knowledge entry by domain and key. Returns `{ found: false }` if not found.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+
+**Response 200:**
+
+```json
+{
+  "id": "uuid",
+  "domain": "screenshot",
+  "key": "google",
+  "data": { "2026-05-09": "The Google homepage..." },
+  "tags": [],
+  "created_at": "2026-05-09T22:26:51.616Z",
+  "updated_at": "2026-05-09T22:26:51.616Z"
+}
+```
+
+## Store entry
+
+```
+POST /api/knowledge/entry
+```
+
+Create or update a knowledge entry. By default, JSONB data is merged at the top level — new keys are added, existing keys are overwritten, but absent keys are preserved. Set `replace: true` to fully replace the data object (required when removing fields).
+
+**Body:**
+
+```json
+{
+  "domain": "screenshot",
+  "key": "google",
+  "data": { "2026-05-09": "The Google homepage..." },
+  "tags": ["web"],
+  "replace": false
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+| `data` | `object` | Yes | JSONB data (top level must be an object) |
+| `tags` | `string[]` | No | Tags are unioned with existing tags on upsert |
+| `replace` | `boolean` | No | When `true`, replaces data entirely instead of merging |
+
+**Response 200:**
+
+```json
+{
+  "id": "uuid",
+  "domain": "screenshot",
+  "key": "google",
+  "created": false,
+  "updated_at": "2026-05-09T22:26:51.616Z"
+}
+```
+
+## Set field (nested path)
+
+```
+PUT /api/knowledge/field
+```
+
+Set a value at a specific JSONB path without overwriting sibling fields. Use dot-notation for nested paths. Creates the entry if it doesn't exist.
+
+**Body:**
+
+```json
+{
+  "domain": "research",
+  "key": "screenshots",
+  "path": "google.holiday",
+  "value": "Mother's Day doodle with flowers",
+  "tags": ["screenshot"]
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+| `path` | `string` | Yes | Dot-notation path (e.g., `google.holiday`, `config.retries`) |
+| `value` | `any` | Yes | Value to set at that path (any JSON type) |
+| `tags` | `string[]` | No | Tags to union with existing |
+
+**Response 200:**
+
+```json
+{ "id": "uuid", "domain": "research", "key": "screenshots", "created": false, "updated_at": "..." }
+```
+
+## Remove field
+
+```
+DELETE /api/knowledge/field?domain=research&key=screenshots&path=google.legacy_data
+```
+
+Remove a specific field at a JSONB path. The entry survives — only the targeted path is deleted.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+| `path` | `string` | Yes | Dot-notation path to remove |
+
+**Response 200:** `{ "removed": true }`
+
+**Response 404:** `{ "error": "Field not found" }`
+
+## Delete entry
+
+```
+DELETE /api/knowledge/entry?domain=screenshot&key=google
+```
+
+Permanently delete a knowledge entry.
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+
+**Response 200:**
+
+```json
+{ "deleted": true, "domain": "screenshot", "key": "google" }
+```
+
+**Response 404:**
+
+```json
+{ "error": "Entry not found" }
+```

package/docs/api/http/settings.md

@@ -19,8 +19,9 @@ GET /api/settings
     "claimDurations": [15, 30, 60, 120, 480]
   },
   "events": {
-    "transport": "nats",
-    "natsWsUrl": "ws://localhost:8222"
+    "transport": "socketio",
+    "natsWsUrl": null,
+    "natsToken": null
   }
 }
 ```
@@ -29,5 +30,6 @@ GET /api/settings
 |-------|------|-------------|
 | `telemetry.traceUrl` | `string \| null` | Template string where `{traceId}` is replaced with the actual trace ID to build a link to the trace viewer. Returns `null` if no trace URL is configured. |
 | `escalation.claimDurations` | `number[]` | Available claim duration options in minutes. Used by the frontend to populate duration selectors. Configurable via the `LT_CLAIM_DURATION_OPTIONS` environment variable (JSON array). |
-| `events.transport` | `string \| null` | Event transport type (e.g., `"nats"`). `null` if no event transport is configured. |
-| `events.natsWsUrl` | `string \| null` | NATS WebSocket URL for real-time event subscriptions. Only present when NATS is the configured transport. |
+| `events.transport` | `'socketio' \| 'nats' \| 'none'` | Dashboard event transport. Defaults to `socketio`. Reports `nats` only when `EVENT_TRANSPORT=nats` is set and a NATS adapter is registered. |
+| `events.natsWsUrl` | `string \| null` | NATS WebSocket URL for browser connections. Only present when a NATS adapter is registered. Read from `VITE_NATS_WS_URL` or `NATS_WS_URL`. |
+| `events.natsToken` | `string \| null` | NATS auth token for browser connections. Only present when a NATS adapter is registered. Read from `NATS_TOKEN`. |

package/docs/api/sdk/files.md

@@ -0,0 +1,68 @@
+# lt.files
+
+File storage operations — browse, inspect, share, and delete files in the storage backend (S3, MinIO, or local filesystem).
+
+## browse
+
+List files and directories at a given prefix.
+
+```typescript
+const result = await lt.files.browse({
+  prefix: 'screenshots/google/',
+  pageSize: 100,
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `prefix` | `string` | No | Directory prefix to list |
+| `pageSize` | `number` | No | Max results (default: 100) |
+| `continuationToken` | `string` | No | Pagination token |
+
+**Returns:** `LTApiResult<{ files: FileEntry[], directories: string[], nextToken?: string }>`
+
+## getMetadata
+
+Get metadata for a single file. Content type is inferred from the file extension when the storage backend returns a generic type.
+
+```typescript
+const result = await lt.files.getMetadata({ filePath: 'screenshots/page.png' });
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `filePath` | `string` | Yes | Path to the file |
+
+**Returns:** `LTApiResult<{ path, size, modified_at, content_type }>`
+
+## delete
+
+Permanently delete a file from storage.
+
+```typescript
+const result = await lt.files.delete({ filePath: 'screenshots/page.png' });
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `filePath` | `string` | Yes | Path to the file |
+
+**Returns:** `LTApiResult<{ deleted, path }>`
+
+## generateSignedUrl
+
+Generate a time-limited signed URL for sharing.
+
+```typescript
+const result = await lt.files.generateSignedUrl({
+  filePath: 'screenshots/page.png',
+  expiresIn: 3600,
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `filePath` | `string` | Yes | Path to the file |
+| `expiresIn` | `number` | Yes | Expiry in seconds (`3600`, `21600`, `86400`, `604800`, `2592000`) |
+
+**Returns:** `LTApiResult<{ url, expiresAt }>`

package/docs/api/sdk/knowledge.md

@@ -0,0 +1,126 @@
+# lt.knowledge
+
+Knowledge store operations — domain-scoped key-value entries backed by PostgreSQL JSONB.
+
+## listDomains
+
+List all knowledge domains with entry counts.
+
+```typescript
+const result = await lt.knowledge.listDomains();
+```
+
+**Parameters:** None
+
+**Returns:** `LTApiResult<{ domains: Array<{ domain, count, latest }> }>`
+
+## listEntries
+
+List entries within a domain.
+
+```typescript
+const result = await lt.knowledge.listEntries({
+  domain: 'screenshot',
+  search: 'google',
+  tags: ['web'],
+  limit: 50,
+  offset: 0,
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain to query |
+| `search` | `string` | No | Search by key name or tag (partial match) |
+| `tags` | `string[]` | No | Filter by tag overlap |
+| `limit` | `number` | No | Max results (default: 50, max: 200) |
+| `offset` | `number` | No | Pagination offset |
+
+**Returns:** `LTApiResult<{ entries: KnowledgeEntry[], total: number }>`
+
+## getEntry
+
+Get a single entry by domain and key.
+
+```typescript
+const result = await lt.knowledge.getEntry({
+  domain: 'screenshot',
+  key: 'google',
+});
+```
+
+**Returns:** `LTApiResult<KnowledgeEntry>` or `{ found: false }` if not found.
+
+## storeEntry
+
+Create or update a knowledge entry. Data is merged by default; set `replace: true` to fully overwrite.
+
+```typescript
+const result = await lt.knowledge.storeEntry({
+  domain: 'screenshot',
+  key: 'google',
+  data: { '2026-05-09': 'The Google homepage...' },
+  tags: ['web'],
+  replace: false,
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+| `data` | `object` | Yes | JSONB data to store |
+| `tags` | `string[]` | No | Tags (unioned on upsert) |
+| `replace` | `boolean` | No | Full replacement instead of merge |
+
+**Returns:** `LTApiResult<{ id, domain, key, created, updated_at }>`
+
+## setField
+
+Set a value at a specific JSONB path without overwriting sibling fields.
+
+```typescript
+const result = await lt.knowledge.setField({
+  domain: 'research',
+  key: 'screenshots',
+  path: 'google.holiday',
+  value: "Mother's Day doodle with flowers",
+});
+```
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `domain` | `string` | Yes | Domain |
+| `key` | `string` | Yes | Entry key |
+| `path` | `string` | Yes | Dot-notation path |
+| `value` | `any` | Yes | Value to set (any JSON type) |
+| `tags` | `string[]` | No | Tags to union |
+
+**Returns:** `LTApiResult<{ id, domain, key, created, updated_at }>`
+
+## removeField
+
+Remove a specific field at a JSONB path.
+
+```typescript
+const result = await lt.knowledge.removeField({
+  domain: 'research',
+  key: 'screenshots',
+  path: 'google.legacy_data',
+});
+```
+
+**Returns:** `LTApiResult<{ removed }>`
+
+## deleteEntry
+
+Permanently delete a knowledge entry.
+
+```typescript
+const result = await lt.knowledge.deleteEntry({
+  domain: 'screenshot',
+  key: 'google',
+});
+```
+
+**Returns:** `LTApiResult<{ deleted, domain, key }>`

package/docs/api/sdk/settings.md

@@ -18,7 +18,8 @@ const result = await lt.settings.get();
 |-------|------|-------------|
 | `telemetry.traceUrl` | `string \| null` | Trace URL for the telemetry provider |
 | `escalation.claimDurations` | `number[]` | Available claim duration options (minutes) |
-| `events.transport` | `'socketio' \| 'nats' \| 'none'` | Active event transport |
-| `events.natsWsUrl` | `string \| null` | NATS WebSocket URL (only when transport is `nats`) |
+| `events.transport` | `'socketio' \| 'nats' \| 'none'` | Dashboard event transport (default: `socketio`; `nats` when `EVENT_TRANSPORT=nats`) |
+| `events.natsWsUrl` | `string \| null` | NATS WebSocket URL (present when NATS adapter registered) |
+| `events.natsToken` | `string \| null` | NATS auth token (present when NATS adapter registered) |
 
 **Auth:** Not required

package/docs/cli.md

@@ -0,0 +1,171 @@
+# CLI
+
+`ltc` is the command-line interface for Long Tail. It provides terminal access to the same capabilities as the dashboard and SDK — escalations, workflows, knowledge, MCP servers, and the durable-to-YAML compiler.
+
+## Install
+
+```bash
+npm install -g @hotmeshio/long-tail
+```
+
+Or run without installing:
+
+```bash
+npx @hotmeshio/long-tail ltc --help
+```
+
+## Connect
+
+### Interactive login
+
+```bash
+ltc login
+```
+
+Prompts for server URL, username, and password. Stores credentials in `~/.config/longtail/credentials.json` (file permissions `0600`).
+
+### Non-interactive (CI / scripts)
+
+```bash
+ltc login --server http://localhost:3000 --username superadmin --password 'l0ngt@1l'
+```
+
+### Environment variables
+
+```bash
+export LT_SERVER=http://localhost:3000
+export LT_TOKEN=eyJ...
+```
+
+Environment variables take precedence over stored credentials.
+
+### Token lifecycle
+
+- JWT expires in 24 hours
+- The CLI checks expiry before each call and auto-refreshes using stored credentials
+- `ltc logout` clears stored credentials
+
+---
+
+## Commands
+
+### Status
+
+```bash
+ltc status
+```
+
+Shows pending escalations, active workers, pipeline tools, and event transport.
+
+### Escalations
+
+```bash
+ltc esc list                              # All escalations
+ltc esc list --status pending             # Pending only
+ltc esc list --role reviewer --limit 10   # Filter by role
+ltc esc get abc123                        # Detail view
+ltc esc claim abc123                      # Claim for yourself
+ltc esc claim abc123 --duration 60        # Claim for 60 minutes
+ltc esc release abc123                    # Release back to pool
+ltc esc resolve abc123 --data '{"approved": true}'   # Resolve with payload
+```
+
+### Workflows
+
+```bash
+ltc wf list                               # Discovered workflows
+ltc wf invoke reviewContent --data '{"content": "test"}'
+ltc wf status wf-abc123                   # Check status (running/completed)
+ltc wf result wf-abc123                   # Get result payload
+ltc wf terminate wf-abc123                # Terminate a running workflow
+```
+
+### Pipeline Tools (YAML Workflows)
+
+```bash
+ltc pip list                              # All pipeline tools
+ltc pip list --status active              # Active only
+ltc pip get abc123                        # Detail view
+ltc pip deploy abc123                     # Deploy a draft
+ltc pip invoke abc123 --data '{"url": "https://example.com"}' --sync
+ltc pip archive abc123                    # Archive
+```
+
+### Knowledge Store
+
+```bash
+ltc kb domains                            # List domains
+ltc kb list research                      # List entries in domain
+ltc kb list research --search google      # Search by key or tag
+ltc kb get research google                # Full entry with data
+ltc kb set research google holiday "Mother's Day"   # Set nested field
+ltc kb remove research google legacy_data           # Remove a field
+ltc kb delete research google                       # Delete entire entry
+```
+
+### MCP Servers
+
+```bash
+ltc mcp servers                           # List registered servers
+ltc mcp tools SERVER_ID                   # List tools for a server
+```
+
+### Users (admin)
+
+```bash
+ltc users list
+ltc users get USER_ID
+```
+
+### Compiler
+
+```bash
+ltc compile workflows/my-workflow.ts      # Compile durable → YAML
+ltc compile src/workflows/ --dry-run      # Scan without compiling
+ltc compile --model claude-opus-4-6    # Use a specific model
+ltc init                                  # Create .env template
+```
+
+---
+
+## Output Modes
+
+Every list command supports three output modes:
+
+```bash
+ltc esc list                    # Formatted table (default)
+ltc esc list --json             # Raw JSON (for jq, scripting)
+ltc esc list -q                 # IDs only, one per line
+```
+
+Pipe-friendly:
+
+```bash
+ltc esc list --json | jq '.escalations[].id'
+ltc pip list -q | xargs -I {} ltc pip get {}
+```
+
+---
+
+## Configuration
+
+| Variable | Description |
+|---|---|
+| `LT_SERVER` | Server URL (overrides stored credentials) |
+| `LT_TOKEN` | JWT token (overrides stored credentials) |
+| `ANTHROPIC_API_KEY` | Required for `ltc compile` (Claude models) |
+| `OPENAI_API_KEY` | Required for `ltc compile` (OpenAI models) |
+| `LT_LLM_MODEL_PRIMARY` | Model for compilation (default: `claude-sonnet-4-6`) |
+
+Credentials file: `~/.config/longtail/credentials.json`
+
+---
+
+## Aliases
+
+| Full | Short |
+|---|---|
+| `ltc escalations` | `ltc esc` |
+| `ltc workflows` | `ltc wf` |
+| `ltc pipelines` | `ltc pip` |
+| `ltc knowledge` | `ltc kb` |

package/docs/compilation.md

@@ -134,3 +134,19 @@ Every subsequent occurrence is routed automatically. A single LLM call extracts
 Each compiled workflow is itself a discoverable MCP tool. Other workflows and agents can invoke it. Solutions compose. The inventory of deterministic pathways grows with every problem the system solves, and the fraction of requests that require LLM reasoning shrinks.
 
 The dynamic path remains for genuinely new problems. But the long tail gets shorter.
+
+---
+
+## Compilation from Source Code
+
+The Pipeline Designer compiles from execution traces — the dynamic path. But there's a second entry point: compiling directly from source code.
+
+If you've written a durable workflow using the Temporal-like API (`proxyActivities`, `sleep`, `condition`, `startChild`), you can compile it to a YAML DAG without executing it first. The `ltc` CLI reads your TypeScript source, extracts the orchestration structure, and produces an equivalent DAG that runs without replay overhead.
+
+```bash
+ltc compile workflows/assembly-line.ts
+```
+
+The source is the spec. The compiled YAML is the optimized execution. Both live in the repo.
+
+See the [Compiler Guide](compiler.md) for the full walkthrough.