@agimon-ai/log-sink-mcp 0.24.3 → 0.25.0

package/README.md

@@ -7,8 +7,9 @@ AI-powered log analysis MCP server with HTTP ingestion and SQLite storage.
 `log-sink-mcp` provides a complete log analysis solution for AI assistants like Claude. It combines:
 
 - **HTTP Server**: REST endpoint for log ingestion from any application
-- **MCP Server**: 4 application-focused tools for searching/filtering, tracing, and inspecting logs
+- **MCP Server**: 5 application-focused tools for searching/filtering, tracing, and inspecting logs
 - **CLI Analysis Commands**: Local shell commands for deeper error and agent-issue analysis
+- **Pi Extension**: Ships a Pi extension that exports the agent's own session telemetry into the sink
 - **SQLite Database**: Fast local storage with FTS5 full-text search plus cached local embeddings
 - **Real-time Analysis**: Query logs, trace distributed requests, analyze error patterns
 
@@ -17,7 +18,8 @@ Perfect for debugging, monitoring, and understanding application behavior throug
 ## Features
 
 - ✅ **HTTP Log Ingestion**: POST logs from any application via REST API
-- ✅ **4 MCP Tools**: Search/filter, trace, and inspect stats/services
+- ✅ **5 MCP Tools**: Search/filter, trace, agent-issue analysis, and inspect stats/services
+- ✅ **Agent Telemetry**: Pi extension records prompts, turns, tool calls, token usage, and cost
 - ✅ **Hybrid Search**: FTS5-powered search plus optional local semantic retrieval across messages and errors
 - ✅ **Distributed Tracing**: Timeline view for trace IDs and span relationships
 - ✅ **Error Analysis**: Pattern detection and error categorization
@@ -43,7 +45,7 @@ Perfect for debugging, monitoring, and understanding application behavior throug
                                                   ▼
 ┌──────────────┐       MCP Protocol      ┌─────────────────┐
 │   Claude /   │ <─────────────────────> │   MCP Server    │
-│   AI Agent   │    4 App Log Tools      │   (stdio)       │
+│   AI Agent   │    5 App Log Tools      │   (stdio)       │
 └──────────────┘                         └────────┬────────┘
                                                   │
                                                   ▼
@@ -319,10 +321,13 @@ Search and filter log entries. Without `query`/`searchQuery`, it behaves as a st
 - `traceId?`: string - Filter by trace ID
 - `spanId?`: string - Filter by span ID
 - `errorType?`: string - Filter by error type
-- `sessionId?`: string - Filter by agent session ID
-- `workflowRunId?`: string - Filter by workflow run ID
-- `workflowName?`: string - Filter by workflow name
-- `jobName?`: string - Filter by workflow job name
+- `sessionId?` / `parentSessionId?`: string - Filter by agent session ID or its parent
+- `agentName?`: string - Filter by agent name
+- `workflowRunId?` / `workflowRunKey?` / `workflowId?`: string - Filter by workflow run ID, stable run key, or workflow file ID
+- `workflowName?` / `workflowWorkspace?`: string - Filter by workflow name or workspace
+- `jobName?` / `jobId?`: string - Filter by workflow job name or job execution ID
+- `stepName?` / `phase?`: string - Filter by workflow step name or phase
+- `signalType?`: string - `log` or `span`
 - `startTime?`: string - ISO 8601 start time filter
 - `endTime?`: string - ISO 8601 end time filter
 - `limit?`: number - Maximum results (default: 100)
@@ -336,7 +341,30 @@ Find error logs from user-service that mention "database timeout"
 
 ---
 
-### 2. `get_trace_timeline`
+### 2. `analyze_agent_issues`
+
+Detect and aggregate agent-run problems for reviewing agent performance and improving the harness: failed tool calls, API errors, refusals, retries exhausted, rejected tools, and error logs. Unlike error-severity analysis, this understands agent event semantics, so it also catches warning-level returned tool errors and info-level tool results carrying `success=false`.
+
+**Parameters:**
+- `sessionId?` / `parentSessionId?`: string - Scope to one agent launch
+- `agentName?`: string - Scope to one agent identity
+- `service?`: string or string[] - Scope by agent/service name (for example `claude-code`, `codex`)
+- `workflowRunId?` / `workflowRunKey?` / `workflowName?` / `workflowWorkspace?`: string - Scope by workflow
+- `jobName?` / `jobId?` / `stepName?` / `phase?`: string - Scope within a run
+- `startTime?` / `endTime?`: string - ISO 8601 analysis window
+- `limit?`: number - Maximum issue samples returned (default: 100)
+- `scanLimit?`: number - Maximum matching records scanned (default: 50000); the response reports when this truncates the window
+
+**Example:**
+```
+Why did the last agent session keep failing its edits?
+```
+
+**Returns:** Counts by category, tool, and error type, plus sample issues carrying the tool, input, and error context needed to diagnose them.
+
+---
+
+### 3. `get_trace_timeline`
 
 Get chronological timeline for a distributed trace.
 
@@ -352,22 +380,25 @@ Show me the trace timeline for trace ID a1b2c3d4e5f6789012345678901234ab
 
 ---
 
-### 3. `get_log_stats`
+### 4. `get_log_stats`
 
-Get statistics grouped by log level.
+Get aggregate statistics, grouped along whichever dimension you ask for.
 
-**Parameters:** None
+**Parameters:**
+- `groupBy?`: string - `both` (default), `level`, `service`, `agent`, `workflow`, `job`, or `signal`
+- `signalType?`: string - `log` (default), `span`, or `all`
+- `startTime?` / `endTime?`: string - ISO 8601 window
 
 **Example:**
 ```
 Show me log statistics
 ```
 
-**Returns:** Count per log level (debug, info, warn, error), total count.
+**Returns:** Counts per group, plus a total.
 
 ---
 
-### 4. `get_services`
+### 5. `get_services`
 
 List all unique services that have logged.
 
@@ -380,6 +411,44 @@ What services are logging to the database?
 
 **Returns:** Array of service names with log counts.
 
+## Pi extension
+
+The package ships a [Pi](https://github.com/earendil-works/pi) extension, declared in the `pi.extensions` field of `package.json` and loaded when the package is installed as a Pi package. It registers no commands and no tools. It subscribes to Pi's session events and exports them as OTLP logs under the service name `pi`, so the agent's own behaviour lands in the same database as your application's.
+
+| Pi event | Record | Level |
+|---|---|---|
+| `session_start` | `pi.session.started` | info |
+| `before_agent_start` | `pi.user_prompt` | info |
+| `agent_start` | `pi.agent.started` | info |
+| `turn_start` | `pi.turn.started` | debug |
+| `turn_end` | `pi.turn.finished` | info |
+| `tool_execution_start` | `pi.tool_call` | debug |
+| `tool_execution_end` | `pi.tool_result` | info |
+| `after_provider_response` | `pi.api_response`, or `pi.api_error` on status 400 and above | debug / error |
+| `model_select` | `pi.model.selected` | info |
+| `agent_end` | `pi.agent.finished` | info |
+| `agent_settled` | `pi.agent.settled` | debug |
+| `session_shutdown` | `pi.session.finished` | info |
+
+Every record carries the session ID, cwd, mode, thinking level, and selected model. `pi.turn.finished` adds the standard `gen_ai.usage.*` attributes: input, output, cache read, cache write, reasoning, total tokens, and cost. `pi.tool_result` adds the tool name and its duration in milliseconds.
+
+Two things are deliberate about what it does not do:
+
+- **Prompts and tool payloads are omitted by default.** Set `AGENT_OTEL_REDACT=0` to include `user.prompt`, `tool.input`, and `tool.result`.
+- **Telemetry is dropped rather than written to disk when no sink is reachable.** Endpoint discovery health-checks a running server; if there is none, the file backend is shut down instead of quietly filling a log directory. Set `LOG_SINK_PI_FILE_FALLBACK=1` to keep it.
+
+Failures in the extension itself warn to the console and never propagate: telemetry that breaks the agent it is measuring is worse than no telemetry.
+
+Once a session has run, query it like any other service:
+
+```bash
+# What went wrong in this agent session
+log-sink-mcp logs agent-issues --session-id <session-id>
+
+# Token spend and per-tool consumption for the session
+log-sink-mcp logs metrics --service pi --group-by agent
+```
+
 ## Integration Examples
 
 ### Pino Logger
@@ -548,7 +617,7 @@ bun run src/cli.ts mcp-serve --type sse --port 3002
 
 #### `logs` command
 
-Run log analysis capabilities directly from the shell. The deeper `analyze-errors` and `agent-issues` commands are CLI-only so the MCP server stays focused on application log inspection.
+Run log analysis capabilities directly from the shell. Most subcommands mirror an MCP tool; `analyze-errors` and `clear` are CLI-only.
 
 ```bash
 bun run src/cli.ts logs <subcommand> [options]
@@ -560,6 +629,7 @@ Subcommands:
 - `search --mode hybrid`: Default hybrid search across FTS5 and semantic embeddings
 - `trace`: Equivalent to `get_trace_timeline`
 - `analyze-errors`: CLI-only grouped error analysis
+- `agent-issues`: Equivalent to `analyze_agent_issues`
 - `stats`: Equivalent to `get_log_stats`
 - `metrics`: Workflow, token, failure, and per-tool consumption metrics
 - `services`: Equivalent to `get_services`
@@ -603,7 +673,22 @@ bun run src/cli.ts logs services
 
 ### Environment Variables
 
-None required. All configuration via CLI flags.
+None are required: the server and its CLI are configured entirely through flags. These apply to the telemetry side, meaning the Pi extension and any process using `@agimon-ai/log-sink-mcp/telemetry/node`.
+
+| Variable | Effect |
+|---|---|
+| `LOG_SINK_ENDPOINT` | Send to this log-sink instead of discovering a running one |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | Standard OTLP endpoint; the signal-specific `_LOGS_` and `_TRACES_` forms take precedence, and any of them override discovery |
+| `OTEL_EXPORTER_OTLP_HEADERS` | Extra export headers; `_LOGS_` and `_TRACES_` variants apply per signal |
+| `OTEL_EXPORTER_OTLP_TIMEOUT` | Export timeout in milliseconds; `_LOGS_` and `_TRACES_` variants apply per signal |
+| `OTEL_SDK_DISABLED` | Disable telemetry entirely |
+| `AGENT_TELEMETRY_DISABLED=1` | Disable the Pi extension only |
+| `AGENT_OTEL_TRACES` | `1`, `true`, or `yes` to export spans as well as logs |
+| `AGENT_OTEL_REDACT=0` | Include prompts, tool inputs, and tool results |
+| `LOG_SINK_PI_FILE_FALLBACK=1` | Let the Pi extension write to files when no sink is reachable |
+| `LOG_SINK_FILE_MAX_BYTES`, `LOG_SINK_FILE_MAX_FILES` | Rotation limits for the file backend |
+
+Telemetry is tagged for later filtering from the environment it runs in. `AGENT_SESSION_ID` and `PARENT_AGENT_SESSION_ID` scope records to an agent launch, and `WORKFLOW_RUN_ID`, `WORKFLOW_RUN_KEY`, `WORKFLOW_ID`, `WORKFLOW_NAME`, `WORKFLOW_WORKSPACE`, `WORKFLOW_JOB_ID`, `WORKFLOW_JOB_NAME`, `WORKFLOW_STEP_NAME`, `WORKFLOW_PHASE`, and `WORKFLOW_RUNNER` scope them to a workflow run. Each becomes an ingest header and then a queryable field, which is what makes `--workflow-run-id` and `--session-id` work across the CLI and MCP tools.
 
 ## Development
 
@@ -628,8 +713,15 @@ log-sink-mcp/
 │   ├── cli.ts                    # CLI entry point
 │   ├── commands/                 # CLI commands
 │   │   ├── http-serve.ts        # HTTP server command
-│   │   └── mcp-serve.ts         # MCP server command
+│   │   ├── mcp-serve.ts         # MCP server command
+│   │   ├── logs.ts              # Log analysis subcommands
+│   │   └── claude-hook.ts       # Claude Code hook context command
 │   ├── container/               # InversifyJS DI container
+│   ├── extensions/              # Pi extension
+│   │   ├── pi.ts                # Extension entry point
+│   │   └── piTelemetry.ts       # Pi session event to OTLP mapping
+│   ├── telemetry/
+│   │   └── node.ts              # Node OTLP setup, endpoint discovery, file fallback
 │   ├── models/                  # Drizzle schemas
 │   │   └── schema.ts            # Log table schema
 │   ├── server/
@@ -645,6 +737,7 @@ log-sink-mcp/
 │   │   ├── QueryLogsTool.ts     # Compatibility wrapper for filter-only queries
 │   │   ├── GetTraceTimelineTool.ts
 │   │   ├── AnalyzeErrorsTool.ts
+│   │   ├── AnalyzeAgentIssuesTool.ts
 │   │   ├── GetLogStatsTool.ts
 │   │   ├── GetServicesTool.ts
 │   │   └── ClearLogsTool.ts

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@agimon-ai/log-sink-mcp",
   "description": "Log sink MCP server with HTTP ingestion and AI analysis",
-  "version": "0.24.3",
+  "version": "0.25.0",
   "license": "BUSL-1.1",
   "keywords": [
     "mcp",
@@ -43,9 +43,9 @@
     "sqlite-vec": "0.1.9",
     "ulidx": "2.4.1",
     "yaml": "2.8.3",
-    "@agimon-ai/foundation-port-registry": "0.24.3",
-    "@agimon-ai/foundation-validator": "0.21.3",
-    "@agimon-ai/foundation-process-registry": "0.24.3"
+    "@agimon-ai/foundation-validator": "0.22.0",
+    "@agimon-ai/foundation-process-registry": "0.25.0",
+    "@agimon-ai/foundation-port-registry": "0.25.0"
   },
   "peerDependencies": {
     "@earendil-works/pi-coding-agent": "*",