log10x-mcp 1.13.2 → 1.14.1

package/README.md

@@ -1,165 +1,37 @@
 # Log10x MCP Server
 
-Observability memory for your logs, exposed to AI assistants. Ask Claude (or any MCP-compatible AI) *"why did our log costs spike this week?"*, *"triage these 3000 events"*, *"what's causing the payments-svc error spike"*, or *"pull all payment_retry events for acme-corp from Jan 15 through Apr 15"* — and get structured answers backed by stable per-pattern identity, not best-effort clustering.
+Ask your AI assistant about log costs, log patterns, and incidents in plain English, and get structured answers backed by real per-pattern metrics. "Why did our log costs spike this week?", "what's driving payments-svc volume?", "pull the payment_retry events for acme-corp from Jan 15 to Apr 15." The server exposes the 10x platform to Claude and any MCP-compatible assistant as a set of tools.
 
-## What it does
+10x groups your logs by message type. The same lines repeat over and over with only the timestamp or request ID changing, so 10x collapses each flood of near-identical lines into one message type and ranks them by volume and cost. That is how the answers stay exact instead of best-effort clustering, no regex required.
 
-Log10x fingerprints every log line into a stable `templateHash` — a structural identity that stays constant across deploys, restarts, pod names, timestamps, and request IDs. That identity is the key to a per-pattern Prometheus time series (volume + cost) and, optionally, a Bloom-indexed S3 archive of the raw events. This MCP server exposes both surfaces to AI assistants as a set of tools:
+This server is open source under Apache-2.0. A Log10x product license (API key) unlocks the account-scoped tools; without one, the server boots read-only against a public demo dataset so the tools can be explored immediately.
 
-### Cost attribution and daily-habit tools
+## What you can ask
 
-| Tool | Answers | Tier |
-|---|---|---|
-| `log10x_cost_drivers` | "Why did our log costs spike?" — dollar-ranked patterns with before→after deltas, keyed by stable templateHash (Datadog Log Patterns re-cluster per query and can't do this honestly) | Reporter |
-| `log10x_event_lookup` | "What is this single log line?" — cost breakdown + AI classification | Reporter |
-| `log10x_pattern_trend` | "When did this pattern start spiking?" — time series + sparkline | Reporter |
-| `log10x_top_patterns` | "What's expensive right now?" — loudest patterns by current cost | Reporter |
-| `log10x_list_by_label` | "Cost by namespace / severity / tenant?" — group-by ranking | Reporter |
-| `log10x_services` | "What services are we monitoring?" — volume + cost by service | Reporter |
-| `log10x_discover_labels` | "What labels can I filter on?" — label universe for the session | Reporter |
-| `log10x_savings` | "How much are we saving?" — per-app savings with annual projection | Reporter |
-| `log10x_dependency_check` | "Anything depending on this before I drop it?" — SIEM dependency scan | None |
-| `log10x_exclusion_filter` | "How do I drop this in Datadog?" — config snippets for 14 vendors | None |
-
-### Investigation, triage, and archive tools (v1.3)
-
-| Tool | Answers | Tier |
-|---|---|---|
-| `log10x_investigate` | "Why is this spiking?" — single-call investigation: anchor resolution, trajectory shape (acute-spike vs drift), cross-pattern lag correlation, temporal-evidence chain with stat/lag/chain confidence sub-scores, drift cohort analysis, two-stage Retriever fallback, recommended next actions. Returns observations + suggestions, NOT verdicts — output framed as "strongest temporal evidence (lead by lag time, not proven cause)." Tags `threshold_basis: "unvalidated_default"` until the operator points `LOG10X_THRESHOLDS_FILE` at calibrated overrides. Surfaces log-only signals (pool saturation, cache evictions, retry amplification) that APM structurally cannot see. See [`docs/cross-pillar-primitives.md`](docs/cross-pillar-primitives.md) for the agent behavior contract. | Reporter |
-| `log10x_resolve_batch` | "Triage these events" — paste a file / array / text dump of raw log lines and get per-pattern frequency, severity, variable concentration, and next-action suggestions. Runs via the Log10x paste endpoint; works at any tier including CLI-only. | None |
-| `log10x_retriever_query` | "Get me the actual events" — direct retrieval from the Retriever archive by templateHash with JS filter expressions over event payloads. Queries the customer's own S3 via pre-computed Bloom filters. Answers forensic, audit, and out-of-retention retrieval. | Retriever |
-| `log10x_backfill_metric` | "Create a new Datadog metric backfilled with 90 days of history" — pulls historical events from the Retriever, aggregates into a bucketed time series, emits to the destination TSDB with historical timestamps preserved. Datadog + Prometheus remote_write supported today. | Retriever |
-
-### Cross-pillar correlation primitives
-
-Three deterministic primitives that take a log-pattern anchor and customer Prometheus metric candidates, then answer "which metrics moved with this log pattern, when, and by how much." They compose into a cascade narrative: filter → rank → overlay.
-
-| Tool | Answers | Tier |
-|---|---|---|
-| `log10x_metrics_that_moved` | "Of these N customer metrics, which ones moved with my log pattern?" — deterministic phase-gap filter. Returns `moved[]`, `not_moved[]`, `evaluation_failed[]`. | Reporter + customer Prometheus |
-| `log10x_rank_by_shape_similarity` | "Of the metrics that moved, which match the anchor's shape best, and which moved first?" — Pearson + signed lag scan. Returns ranked candidates with `lag_seconds`, `lag_at_bound`, `anchor_phase_aligned`. | Reporter + customer Prometheus |
-| `log10x_metric_overlay` | "Show me the two curves side by side." — aligned timeseries + deterministic peak facts (`peak_offset_seconds`, etc.). No Pearson, no tier — raw data. | Reporter + customer Prometheus |
-
-**Designed for AI callers, not humans.** Every output carries a unified envelope: `status`, `threshold_basis`, `anchor_ref` echo, `query_count`, `total_latency_ms`, `backend_pressure_hint`, and a `human_summary` field the agent can paste to a human. `metric_ref` round-trips identically across the three tools — pass it from one output to the next input verbatim.
-
-**Calibration honesty.** The default thresholds (15% phase gap, ±1800s lag search) are hand-picked from one synthetic chaos test, not calibrated against any real customer backend. Every tool tags `threshold_basis: "unvalidated_default"` until a caller-supplied override moves it to `caller_override`. Agents MUST NOT auto-mitigate based on uncalibrated findings. See [`docs/cross-pillar-primitives.md`](docs/cross-pillar-primitives.md) for the full agent behavior contract and the per-backend calibration playbook.
-
-**Structural guards.** When the anchor doesn't have a real busy/quiet split, the tool refuses with `status: "anchor_no_phase_separation"` rather than producing nonsense. When the candidate set produces zero meaningful signal, the tool returns `status: "no_signal"` so the agent stops searching cleanly. Backend errors surface as a structured `PrimitiveError` envelope (`error_type`, `retryable`, `suggested_backoff_ms`, `hint`), not thrown strings.
-
-All tools query `prometheus.log10x.com` (for Reporter-tier tools) over HTTPS, with the same `X-10X-Auth` header used by the rest of the Log10x stack. No log scanning; sub-second at any scale.
-
-## ROI examples — three real flows
-
-These are real round-trips against the Log10x demo environment, captured during development. Every tool call below is verbatim what the model would produce; outputs are abbreviated for the README.
-
-### 1. "Why is checkout-svc cost up?" (`log10x_cost_drivers`)
-
-**Prompt**: *"Why did checkout cost spike this week?"*
-
-**Tool call**: `log10x_cost_drivers({ service: "cart", timeRange: "7d" })`
-
-**Output** (abbreviated):
-
-```
-cart — $137 → $38K/wk (4 cost drivers)
-
-#1  cart cartstore ValkeyCartStore     $51 → $13K/wk   INFO  13.3B events
-#2  shipping service Post shipping...  $34 → $12K/wk   CRIT  1.6B events
-#3  GetCartAsync called with userId    $34 → $8.7K/wk        8.7B events
-#4  AddItemAsync called with...        $18 → $4.6K/wk        4.2B events
-
-4 drivers = 49% of increase · 2442 other patterns
-
-**Next actions**:
-  - call `log10x_investigate({ starting_point: 'cart_cartstore_ValkeyCartStore' })` to trace the cause of the $13K delta on this pattern.
-  - call `log10x_dependency_check({ pattern: '...' })` before muting or dropping — blast-radius safety.
-```
-
-The next-action hints in the output literally tell the model what to do next. No prompt engineering required.
-
-### 2. "What's broken in payments-svc?" (`log10x_investigate`)
-
-**Prompt**: *"Investigate kafka — there's an alert firing."*
-
-**Tool call**: `log10x_investigate({ starting_point: "kafka", window: "1h" })`
-
-**Output** (abbreviated 8-link causal chain):
-
-```
-## Investigation: kafka, last 1h
-
-**Anchor**: cluster_metadata_Wrote_producer_snapshot... (resolved from service_name)
-**Service**: kafka
-**Inflection**: 2026-04-14T00:19:52Z UTC
-**Shape**: acute spike
-**Reporter tier**: edge
-
-### Most likely root cause
-
-Pattern: cluster_metadata_dir_tmp_kafka_logs_Rolled_new_segment...
-Confidence: 43% (stat:1.00 lag:0.43 chain:1.00)
-Why: peaked 300s before the anchor, magnitude 1.4× baseline.
-
-### Causal chain
-
-1. cluster_metadata_dir_tmp_kafka_logs_Rolled... — peaked T-300s
-2. Successfully_wrote_snapshot_org_apache_kafka...  — peaked T-300s
-3. opentelemetry_javaagent_shaded_instrumentation... — peaked T-300s
-... (8 links total, each with stat × lag × chain confidence sub-scores)
-
-### Suggested verification commands
-
-gh api /repos/<owner>/kafka/commits?since=...&until=...
-kubectl get events -n kafka --since=Xm
-dog metric query "avg:trace.kafka.requests{*} by {resource_name}" --from ...
-```
-
-The full causal chain comes back in one tool call. The model doesn't need to compose. The verification commands are pre-substituted with the inflection timestamp so the user can paste them directly.
-
-### 3. "Triage this Slack paste" (`log10x_resolve_batch`)
-
-**Prompt**: *"My teammate dumped these 12 lines from order-processing-svc into Slack — what's happening?"*
-
-**Tool call**: `log10x_resolve_batch({ source: "text", text: "..." })`
-
-**Output** (abbreviated):
-
-```
-## Batch Triage
-
-12 events, resolved into 3 distinct patterns. Templater wall time: 6.4s. Execution: Log10x paste endpoint.
-
-**Severity mix**: INFO: 7 · ERROR: 4 · WARN: 1
-
-### Top 3 patterns by interestingness
-
-**#1  checkout_svc_tenant_acme_corp_order_status_failed_reason_payment_gateway**  · 4 events (33% of batch) · interestingness 0.47
-severity: ERROR
-
-Variable concentration (top values within this batch):
-  - timestamp · 4 distinct · `1776067923000` 25%, `1776067925000` 25%, `1776067928000` 25%
-  - order · 4 distinct · `12347` 25%, `12349` 25%, `12352` 25%
-
-**Next actions**:
-  - call `log10x_investigate({ starting_point: '...' })` for historical correlation (requires Reporter tier).
-  - call `log10x_retriever_query({ pattern: '...', filters: ["event.order === \"12347\""] })` to retrieve all historical events concentrated on order=12347 (requires Retriever tier).
-  - native Datadog follow-up: `dog log search '@order:"12347"' --from now-24h` — filters to the dominant variable concentration directly in the SIEM.
-```
-
-Every pattern is ranked by an interestingness score (severity-weighted); the dominant variable is identified; ready-to-paste next-action commands are pre-constructed for both Log10x tools and the customer's SIEM. The model just needs to relay the output and ask which path the user wants to take.
+| You say | What happens |
+|---|---|
+| "Where do I start? Help me cut log costs." | A guided menu asks what you want (cut cost, investigate, install) and routes to the right next step, plus what's new and what changed. `log10x_start`, `log10x_whats_new`, `log10x_whats_changing` |
+| "Sign me in." / "Who am I connected as?" | GitHub sign-in mints and stores an API key, and shows the environments your account can reach. `log10x_signin_start`, `log10x_login_status`, `log10x_create_env` |
+| "Show me how to install this on my stack." | Paste-ready setup steps for your own pipeline, fetch-back wiring, and a recommended action per service. `log10x_advise_install`, `log10x_advise_retriever`, `log10x_configure_engine` |
+| "Estimate savings on my data before I deploy." | A no-install savings report from a local log file or your existing log platform. `log10x_poc_from_local`, `log10x_poc_from_siem_submit`, `log10x_poc_from_siem_status` |
+| "How much can I cut, and how much have I cut?" | Projected and realized savings with the per-pattern math behind every number. `log10x_estimate_savings`, `log10x_savings`, `log10x_commitment_report` |
+| "What's driving my cost right now?" | The repeating message types and services driving volume and cost, and what moved week over week. `log10x_top_patterns`, `log10x_whats_changing`, `log10x_services` |
+| "Why did payments-svc spike?" | A single-call investigation: timeline, correlated patterns, and the strongest temporal evidence, with confidence shown so nothing is presented as proven cause. `log10x_investigate`, `log10x_metrics_that_moved`, `log10x_metric_overlay` |
+| "This message type is noise, cut it." | Sample, drop, compact, tier down, or offload a chosen message type, with examples and trend so the action is clear. `log10x_pattern_mitigate`, `log10x_pattern_examples`, `log10x_pattern_trend` |
+| "Get me the actual events I offloaded." | Fetch the exact events back on demand from your own S3. `log10x_retriever_query`, `log10x_offload_add`, `log10x_retriever_register` |
+| "Recognize more of my message types." | Build a custom symbol library from your own logs so the engine finds more savings. `log10x_compile`, `log10x_compile_link`, `log10x_compile_status` |
 
 ## Install
 
-### Engine compatibility
+**Claude Code**
 
-| MCP version | tenx engine required | Notes |
-|---|---|---|
-| 1.8.13+ | **1.0.22+** (recommended) | `log10x_resolve_batch` `privacy_mode: true` invokes `tenx @apps/mcp` (stdin/stdout, no temp files). Requires `apps/mcp` shipped in the engine release — first available in 1.0.22. |
-| 1.8.x | 1.0.21 (with `TENX_CONFIG`+`TENX_MODULES` overrides) | Earlier engines can run the MCP only if the user clones `log-10x/config` + `log-10x/modules` at the apps/mcp branches and points `TENX_CONFIG`/`TENX_MODULES` at them. |
+```bash
+claude mcp add --transport stdio --env LOG10X_API_KEY=your-api-key log10x -- npx -y log10x-mcp
+```
 
-`brew upgrade log10x` pulls the latest released engine. See https://doc.log10x.com/install/ for OS-specific installers.
+Verify with `/mcp`. Omit the `--env` flag to run read-only against the public demo dataset.
 
-### Claude Desktop
+**Claude Desktop**
 
 Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows):
 
@@ -177,210 +49,56 @@ Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS)
 }
 ```
 
-Restart Claude Desktop. The server autodiscovers every env your account can reach (default + any shared with you) — pass `environment: "<nickname>"` on any tool call to switch between them.
-
-If `LOG10X_API_KEY` is omitted, the MCP boots in **demo mode** (read-only). Two flavors:
-
-- **Your own demo data** — if you installed a Log10x engine with an anonymous demo license (the install wizard's `license_source: "demo"`, or you set `LOG10X_LICENSE_JWT=<jwt>`), the MCP queries the data *that engine* wrote, via the `/api/v1/demo/*` routes with the same license. The MCP reuses the demo license it minted (persisted at `~/.log10x/demo-license.json`), so reads and writes share one demo tenant. Demo queries are bounded to the **last 3 hours** and rate limited; `query_ai` is unavailable.
-- **Shared sample data** — with no key and no demo license at all, the MCP falls back to the public Log10x demo env (the same sample data the console "Try Live Demo" shows).
+**Cursor, Windsurf, and other MCP clients**
 
-Run `log10x_signin` and the LLM will walk you through the GitHub Device Flow — it mints a real key, writes it to `~/.log10x/credentials`, and hot-reloads without an MCP-host restart.
+Add an `mcpServers` entry of the same shape: `"command": "npx"`, `"args": ["-y", "log10x-mcp"]`, and `LOG10X_API_KEY` in `env`. Omit the key to run read-only demo mode.
 
-### Claude Code
-
-```bash
-claude mcp add --transport stdio \
-  --env LOG10X_API_KEY=your-api-key \
-  log10x -- npx -y log10x-mcp
-```
+Full setup walkthrough: https://doc.log10x.com/apps/mcp/
 
-Verify with `/mcp`.
+## Credentials
 
-### Cursor / Windsurf / other MCP clients
+Account-scoped tools need a Log10x API key. Two ways to get one:
 
-Same pattern — add an `mcpServers` entry with `"command": "npx"`, `"args": ["-y", "log10x-mcp"]`, and `LOG10X_API_KEY`.
+- **GitHub sign-in:** run `log10x_signin_start`, complete the flow, then `log10x_signin_complete`. The key is stored for you. `log10x_login_status` shows who you're connected as.
+- **Environment variable:** set `LOG10X_API_KEY` directly.
 
-### Help your AI find the tools
-
-Even with the MCP installed, agents sometimes miss it for natural-language questions ("am I logged in?" → `which log10x` shell probe instead of `log10x_login_status`). Project rules nudge the agent toward MCP-first tool selection. Drop a snippet that says **"the log10x MCP IS the surface; call its tools before shelling out"** plus a few trigger-phrase mappings into:
-
-| Client | Rule file |
-|---|---|
-| Cursor | `.cursor/rules/log10x.mdc` (with `alwaysApply: true` frontmatter) |
-| Claude Code | `CLAUDE.md` (workspace root) |
-| Windsurf | `.windsurfrules` (workspace root) |
-| GitHub Copilot Chat | `.github/copilot-instructions.md` |
-| Claude Desktop | (no project-rule mechanism — relies on manifest descriptions) |
-
-Full canonical text and explanation in [doc.log10x.com/install/mcp-server/#help-your-ai-find-the-tools](https://doc.log10x.com/install/mcp-server/#help-your-ai-find-the-tools).
-
-## Get your credentials
-
-Two options:
-
-1. **Sign in via GitHub**: launch the MCP without `LOG10X_API_KEY` set, then ask the LLM to "sign in to Log10x". `log10x_signin` runs the GitHub Device Flow, mints an API key, and saves it to `~/.log10x/credentials` — works across every MCP host on the same machine.
-2. **Console**: log into [console.log10x.com](https://console.log10x.com) → Profile → API Settings → copy your API Key. (Optional: set your **Analyzer Cost** ($/GB for your SIEM) on that page — the server reads it automatically.)
-
-## Multi-environment access
-
-Two patterns, depending on what you need:
-
-**Same account, multiple envs** (e.g., your account owns prod, staging, dev). Nothing special to configure — the MCP autodiscovers all envs your account can reach via `GET /api/v1/user`. Pass `environment: "<nickname>"` on any tool call, or just say "check staging costs" and the LLM routes there. The chosen env sticks for follow-up calls until you switch again.
-
-**Multiple accounts** (e.g., a consultant accessing customer envs). The Log10x backend supports per-env permission sharing (OWNER / WRITE / READ). Have the env owner grant your account READ on their env from the console — it then shows up in your `/api/v1/user` response automatically, no client-side multi-credential setup needed. If you genuinely need parallel access from distinct API keys, register one MCP server per account in your host config with distinct server names.
-
-## Usage
-
-You never call tools directly. Just ask your AI assistant a question in plain English:
-
-| You say | The AI calls |
-|---|---|
-| "Why did our log costs spike this week?" | `log10x_cost_drivers` |
-| "What is this Payment Gateway Timeout pattern?" | `log10x_event_lookup` |
-| "How much are we saving with the pipeline?" | `log10x_savings` |
-| "When did the checkout service start spiking?" | `log10x_pattern_trend` |
-| "What services are we monitoring?" | `log10x_services` |
-| "How do I drop this in Datadog?" | `log10x_exclusion_filter` |
-| "Anything depending on this before I drop it?" | `log10x_dependency_check` |
-
-## Cost driver algorithm
-
-When you ask about cost spikes, the server runs this algorithm:
-
-1. **Query current window** — bytes per pattern for the selected timeframe
-2. **Query baseline** — average of the 3 prior windows of the same size
-3. **Compute delta** — `cost_this_period - cost_baseline` per pattern
-4. **Apply gates** — a pattern is a cost driver when it passes both:
-   - Dollar floor: delta exceeds `$500/period`
-   - Contribution floor: delta is at least `5%` of the total service increase
-5. **Sort by delta** descending
-
-Example output:
-
-```
-cart — $103 → $13K/wk (3 cost drivers)
-
-#1  cart cartstore ValkeyCartStore      $51 → $6.4K/wk   INFO  6.6B events
-#2  GetCartAsync called with userId     $34 → $4.3K/wk         4.2B events
-#3  AddItemAsync called with userId     $18 → $2.2K/wk         2.1B events
-
-3 drivers = 98% of increase · 11 other patterns
-```
+**No key?** The server boots read-only against the public 10x demo dataset (the same sample data the website console shows), so analysis, discovery, and the install advisors all work out of the box. Mutating tools are disabled in this mode so shared demo state stays untouched. Set `LOG10X_DEMO_FALLBACK=off` to boot into pure POC mode (savings estimates on your own files, no account). The mode is fixed at startup; restart the server to re-detect.
 
 ## Environment variables
 
-### Reporter-tier (required for cost, trend, and investigate tools)
-
-| Variable | Required | Description |
-|---|---|---|
-| `LOG10X_API_KEY` | No | Your Log10x API key. Omit to boot in demo mode + use `log10x_signin` to mint one via GitHub. The env list is autodiscovered from `GET /api/v1/user` — no env-id pinning needed. |
-| `LOG10X_LICENSE_JWT` | No | A demo license JWT (from `POST /api/v1/license/demo`). With no API key set, the MCP reads back the data an engine running this same license wrote, via `/api/v1/demo/*` (last 3h only, rate limited). Usually unnecessary — the install wizard persists the minted demo license to `~/.log10x/demo-license.json` and the MCP reuses it automatically. |
-| `LOG10X_API_BASE` | No | API base URL (default: `https://prometheus.log10x.com`) |
-
-### Pasted-batch triage (`log10x_resolve_batch`)
-
-| Variable | Required | Description |
-|---|---|---|
-| `LOG10X_PASTE_URL` | No | Override the Log10x paste endpoint (default: `https://meljpepqpd.execute-api.us-east-1.amazonaws.com/paste`). Body limit 100 KB. |
-
-### Retriever (`log10x_retriever_query`, `log10x_backfill_metric`)
-
-| Variable | Required | Description |
-|---|---|---|
-| `__SAVE_LOG10X_RETRIEVER_URL__` | Yes (Retriever tier) | Base URL of the customer's deployed Retriever query endpoint (e.g., `https://retriever.<your-domain>`). When unset, Retriever-dependent tools return a graceful "not configured" message. |
-| `LOG10X_RETRIEVER_AUTH_HEADER` | No | Override the auth header name (default: `X-10X-Auth`, same as the Prometheus gateway). |
-| `LOG10X_RETRIEVER_AUTH_VALUE` | No | Override the auth header value. Default is `${apiKey}/${envId}` from the active environment. |
-| `__SAVE_LOG10X_RETRIEVER_TARGET__` | No | Override the default target prefix under which retriever writes indexed objects (default: `app`). |
-| `LOG10X_RETRIEVER_INDEX_SUBPATH` | No | Override the index subpath inside the bucket (default: `indexing-results`, matching the engine's indexContainer convention). |
-| `LOG10X_RETRIEVER_POLL_MS` | No | Override the marker-stability poll interval (default: `1500` ms). |
-| `LOG10X_RETRIEVER_TIMEOUT_MS` | No | Override the query timeout (default: `90000` ms). |
+All optional. The common path is just `LOG10X_API_KEY` (or no key at all).
 
-**Demo env retriever LB**: the otel-demo cluster has a pre-provisioned retriever LoadBalancer at `http://a2936089108bb492cb41d18cb5b75f8d-1298006809.us-east-1.elb.amazonaws.com`. Set `__SAVE_LOG10X_RETRIEVER_URL__` to that value for the demo. The demo bucket is `tenx-demo-cloud-retriever-351939435334/indexing-results/`.
-
-**Known engine-side issues (GAPS G12)**: `log10x_retriever_query` has two unresolved engine-side bugs that `log10x_doctor` flags as `retriever_forensic_health` warnings: (1) it may return 0 events on windows where `log10x_pattern_trend` proves events exist, (2) it may crash with `MCP error -32000: Connection closed` when passed a canonical slash-underscore pattern name. Workarounds: use short/free-text pattern names, cross-check any zero result against `log10x_pattern_trend`, prefer `log10x_event_lookup` + `log10x_pattern_trend` for incident reconstruction where approximate timing is acceptable. See `docs/ENGINE_TICKETS.md` for the full engine-team ticket.
-
-### Metric backfill destinations (`log10x_backfill_metric`)
-
-| Variable | Required | Description |
-|---|---|---|
-| `DATADOG_API_KEY` (or `DD_API_KEY`) | Yes (Datadog destination) | Datadog API key used to POST to `/api/v2/series`. |
-| `DATADOG_SITE` | No | Datadog site (default: `datadoghq.com`, override for `datadoghq.eu`, `us5.datadoghq.com`, etc.) |
-| `PROMETHEUS_REMOTE_WRITE_URL` | Yes (Prometheus destination) | URL of a Prometheus `remote_write`-compatible adapter. The MCP posts JSON; the adapter translates to the native protobuf/Snappy wire format. |
-
-The server fetches your analyzer cost ($/GB) from your Console profile at startup and refreshes it hourly. To change it, update the cost in your profile — the server picks up the new value within an hour.
-
-## Spawning sub-agents that use this MCP
-
-If you script Claude Code sub-agents (`Agent` tool, OpenAI Agents SDK, custom orchestrators) to call log10x MCP tools, watch out for **deferred-tool bootstrapping**. The sub-agent's static tool list does not always include `mcp__log10x__*` — those tools are loaded on demand via `ToolSearch` (or the equivalent in your client). A sub-agent that reads its tool list and concludes "no MCP tools available" without first calling `ToolSearch({query: "log10x"})` will refuse the task instead of using the tools.
-
-In testing across 17 sub-agent runs, **prompt framing was the deciding factor**:
-
-| Framing | Bootstrap success |
+| Variable | Purpose |
 |---|---|
-| Action-oriented ("pull the events", "build the slide", "investigate this") | 9/9 |
-| Honesty-oriented ("don't fabricate", "refuse if you can't verify") | 0/5 |
-
-The honesty disposition fires too early — the agent applies "be honest about my limits" before trying to discover deferred tools. The fix is one line at the top of every sub-agent prompt:
-
-```
-TOOL BOOTSTRAP: You have log10x MCP tools available, but they are
-deferred-loaded. Before doing anything else, call
-ToolSearch({query: "log10x", max_results: 20}) to load them.
-After that you'll have mcp__log10x__log10x_* tools.
-```
-
-This raised bootstrap success from ~76% to 100% in our test runs and is harmless when MCP tools are already in scope.
-
-## Development
-
-Build:
-
-```bash
-npm install
-npm run build   # tsc → build/index.js
-```
-
-**Operational gotcha — restart after rebuild.** MCP clients (Claude Desktop, Claude Code, Cursor) typically launch the server as a long-running child process. Node caches loaded modules in memory, so the running process **will not pick up your `npm run build` output until it is killed and respawned**. After rebuilding, find and kill the stale processes:
-
-```bash
-pgrep -fl "log10x-mcp/build/index.js"   # show running servers + start times
-ps -o pid=,lstart= -p $(pgrep -f log10x-mcp/build/index.js)
-pkill -f "log10x-mcp/build/index.js"    # client will respawn on next tool call
-```
-
-If you ship a tool description change or a routing-hint update and the agent's behavior doesn't seem to reflect it, this is the most likely cause. Verify by `grep`-ing the raw tool output (most clients log it) for any string you added — if it's missing, the server is stale.
+| `LOG10X_API_KEY` | Your Log10x API key. Omit to boot read-only on the public demo dataset. |
+| `LOG10X_API_BASE` | API base URL override (default `https://prometheus.log10x.com`). |
+| `LOG10X_ENV_ID` | Pin a specific environment id instead of autodiscovering from your account. |
+| `LOG10X_DEMO_FALLBACK` | Set to `off` to boot into POC mode instead of attaching to the public demo dataset when no key is set. |
+| `LOG10X_MCP_READ_ONLY` | Set to `true` or `1` to force read-only mode (all mutating tools refuse). |
+| `LOG10X_CUSTOMER_METRICS_URL` / `_TYPE` / `_AUTH` | Point at your own metrics backend (Prometheus, Grafana Cloud, Mimir, Thanos, AMP) for BYO-metrics setups. |
+| `DD_API_KEY` / `DD_APP_KEY` / `DD_SITE` | Datadog credentials for backfilling per-pattern metrics into Datadog. |
+| `PROMETHEUS_REMOTE_WRITE_URL` / `PROMETHEUS_URL` | Prometheus endpoints for backfilling and reading per-pattern series. |
+| `LOG10X_RETRIEVER_URL` | Base URL of your deployed Retriever query endpoint, for fetch-back (or set it with `log10x_retriever_register`). |
+| `LOG10X_RETRIEVER_TIMEOUT_MS` / `LOG10X_RETRIEVER_POLL_MS` | Fetch-back query timeout (default 90000) and poll interval (default 1500). |
+| `LOG10X_RETRIEVER_AUTH_HEADER` / `LOG10X_RETRIEVER_AUTH_VALUE` | Override the fetch-back auth header (defaults derive from the active environment). |
+| `LOG10X_OFFLOAD_BUCKET` / `LOG10X_STREAMER_BUCKET` | S3 bucket names the offload tools manage. |
+| `TENX_LICENSE_KEY` / `LOG10X_TENX_PATH` | Engine license key and local engine binary path for local compile operations. |
+| `LOG10X_GH_REPO` / `LOG10X_GITOPS_REPO_PATH` | GitHub repo and local path for GitOps-aware config tools. |
+
+## Connect your own data (optional)
+
+10x keeps logs queryable without paying to index everything. The Receiver ships budget-overflow noise to your own S3, and the Retriever fetches those exact events back on demand when you need them. `log10x_offload_add` and `log10x_retriever_register` wire up the offload-and-fetch path; `log10x_backfill_metric` pushes per-pattern volume and cost series into your existing Datadog or Prometheus so the cost views light up alongside the rest of your dashboards.
 
 ## Security
 
-- All API calls use your personal API key (never exposed in tool output)
-- The server runs locally — no data leaves your machine except Prometheus queries
-- Dependency check scripts run locally with your own SIEM credentials (read-only)
-- No caching of log content — all data comes from pre-aggregated metrics
-
-## Documentation
-
-Full documentation: [doc.log10x.com/manage/mcp-server](https://doc.log10x.com/manage/mcp-server/)
+The server runs locally as a subprocess of your AI client. Only pre-aggregated metric queries (per-pattern volume and cost) leave the machine, never raw log content. Local triage with `log10x_resolve_batch` runs entirely on your machine over stdin and stdout. Connections to your own log platform use read-only credentials. Set `LOG10X_MCP_READ_ONLY=true` to refuse every mutating tool regardless of mode.
 
-## License
+## Open source and license
 
-This package is licensed under the [Apache License 2.0](LICENSE).
+Apache-2.0. Published on npm as `log10x-mcp` (Node 20+). The source is public so the behavior behind every tool can be read directly. Account-scoped tools require a Log10x product license (API key); the public demo dataset needs none.
 
-### Important: Log10x Product License Required
+MCP clients run the server as a long-lived child process. After upgrading the package, restart the client so the new version takes effect.
 
-This package is the open-source MCP server for Log10x. While the MCP itself is open source,
-**using Log10x requires a commercial license**.
-
-| Component | License |
-|-----------|---------|
-| This package (`log10x-mcp`) | Apache 2.0 (open source) |
-| Log10x engine and runtime | Commercial license required |
-
-**What this means:**
-- You can freely use, modify, and distribute this MCP server
-- The Log10x backend services and `tenx` runtime that this MCP talks to require a paid subscription
-- A valid Log10x API key is required to call the account-scoped tools (sign in via `log10x_signin` or set `LOG10X_API_KEY`)
+## Documentation
 
-**Get Started:**
-- [Log10x Pricing](https://log10x.com/pricing)
-- [Documentation](https://doc.log10x.com)
-- [Contact Sales](mailto:sales@log10x.com)
+https://doc.log10x.com/apps/mcp/

package/build/index.js

@@ -149,7 +149,7 @@ function getEnvs() {
  * rewrites it, so ops can see the original text when hunting root causes.
  */
 /**
- * Tools that query the metrics backend (cost_drivers, top_patterns,
+ * Tools that query the metrics backend (top_patterns, whats_changing,
  * etc.). When the MCP is in pure-demo mode (no user configuration,
  * silently landed on the demo backend), these tools short-circuit
  * with a structured `not_configured` response instead of returning
@@ -997,7 +997,7 @@ registerLog10xTool('log10x_top_patterns', topPatternsSchema, (args) => wrap('log
 // Set diff of patterns across a time boundary. Compares pattern presence in
 // two windows and returns new / retired / persistent / re_emerged sets plus
 // co_emergence_clusters (deploy fingerprint via first_seen clustering).
-// Coherent only because log10x pattern_hash is stable across queries;
+// Coherent only because 10x's pattern_hash is stable across queries;
 // competitors that re-cluster per query can't answer this.
 registerLog10xTool('log10x_pattern_diff', patternDiffSchema, (args) => wrap('log10x_pattern_diff', async () => {
     const env = resolveEnv(getEnvs(), args.environment);
@@ -1006,9 +1006,9 @@ registerLog10xTool('log10x_pattern_diff', patternDiffSchema, (args) => wrap('log
 // ── Tool: log10x_whats_changing ──
 //
 // Patterns ranked by delta vs a baseline window (not current cost). Restores
-// the capability of the removed log10x_cost_drivers tool
+// the removed cost-drivers ranking
 // using the modern StructuredOutput envelope. Brand-new patterns (no
-// baseline) are excluded — they go to log10x_whats_new.
+// baseline) are excluded; they go to log10x_whats_new.
 registerLog10xTool('log10x_whats_changing', whatsChangingSchema, (args) => wrap('log10x_whats_changing', async () => {
     const env = resolveEnv(getEnvs(), args.environment);
     return executeWhatsChanging(args, env);
@@ -1406,7 +1406,7 @@ const REGISTERED_TOOLS = [
     { name: 'log10x_dependency_check', intent: 'Scan SIEM + dashboards + alerts for refs to a pattern before muting / deleting it' },
     { name: 'log10x_top_patterns', intent: 'Top N patterns by current cost, with per-row delta vs comparison_window and newly-emerged section' },
     { name: 'log10x_pattern_diff', intent: 'Set diff of patterns across a time boundary — new/retired/persistent/re_emerged + co_emergence_clusters (deploy fingerprint). Coherent across boundaries because pattern_hash is stable across queries.' },
-    { name: 'log10x_whats_changing', intent: 'Patterns ranked by delta vs baseline (growth/shrinkage). Brand-new patterns excluded — see log10x_whats_new for those. Restores the deleted log10x_cost_drivers capability.' },
+    { name: 'log10x_whats_changing', intent: 'Patterns ranked by delta vs baseline (growth/shrinkage). Brand-new patterns excluded, see log10x_whats_new for those.' },
     { name: 'log10x_whats_new', intent: 'Patterns whose first_seen falls inside a recency window. Separate from delta-vs-baseline because new patterns have no baseline.' },
     { name: 'log10x_investigate', intent: 'Single-call root-cause — causal chain for acute spikes or cohort for drift' },
     { name: 'log10x_resolve_batch', intent: 'Pasted-batch triage — per-pattern variable concentration + next actions' },
@@ -1501,7 +1501,7 @@ async function handleCliFlags() {
             'Environment:',
             '  LOG10X_API_KEY            API key from console.log10x.com (or run `log10x_signin_start` then `log10x_signin_complete` to mint one via Auth0 Device Flow)',
             '  LOG10X_API_BASE           Override Prometheus gateway URL',
-            '  __SAVE_LOG10X_RETRIEVER_URL__       Retriever query endpoint (optional)',
+            '  LOG10X_RETRIEVER_URL      Retriever query endpoint (optional)',
             '  LOG10X_TENX_MODE          `local` or `docker` backend for the local engine and compile tools; when unset, auto-detects and prefers `docker`, falling back to a native `tenx` install',
             '  LOG10X_TENX_PATH          Path to local tenx CLI (used when LOG10X_TENX_MODE=local; compile requires the cloud flavor)',
             '  LOG10X_TENX_IMAGE         Docker image when LOG10X_TENX_MODE=docker (default: log10x/pipeline-10x:latest)',