@barivia/barsom-mcp 0.20.4 → 0.22.1

package/README.md

@@ -27,6 +27,17 @@ MCP clients typically run it with **`npx`** (downloads on first use):
 
 **Verify:** `npm view @barivia/barsom-mcp version`
 
+## 2-minute quickstart
+
+1. **Get an API key** — Sign up at [barivia.se/dashboard](https://barivia.se/dashboard) (self-serve) or ask your admin for a `bv_…` key.
+2. **Add the MCP server** — Paste the JSON block above into your client’s MCP config (`mcp.json` in Cursor). Set `BARIVIA_API_KEY` and keep `BARIVIA_API_URL=https://api.barivia.se`.
+3. **Orient** — In chat, invoke **`guide_barsom_workflow`** (no parameters). It returns your plan-scoped tool map and async rules.
+4. **Upload data** — `datasets(action=upload, file_path="your.csv")` → note `dataset_id` (and `stage_job_id` if staging is enqueued; poll until complete).
+5. **Train** — `train(action=map, dataset_id=…, preset=quick)` → returns `job_id`. Poll with `jobs(action=status, job_id=…)` every 60–120s, or use **`training_monitor(job_id=…)`** for live curves.
+6. **Results** — When status is `completed`, `results(action=get, job_id=…)` for metrics + headline figure; **`results_explorer(job_id=…)`** to browse all figures.
+
+**Troubleshooting:** Upload fails with “paths outside the workspace” → set `BARIVIA_WORKSPACE_ROOT` to your project folder. Timeouts on slow calls → raise `BARIVIA_FETCH_TIMEOUT_MS`. Tool “server does not exist” in Cursor → use the resolved server id from the MCP tool list, not necessarily the key in `mcp.json`. Hosted HTTP MCP at `mcp.barivia.se` is paused — use this npm package only.
+
 **Audit logging (0.10.3+):** Each tool invocation emits a structured **`mcp_tool_call`** line on stderr for support correlation — not sent to the Barivia API. Fields: `tool`, `action`, `duration_ms`, `outcome`, **`rid`** (matches API `X-Request-ID`), **`job_id`** / **`dataset_id`** when the action returns or accepts them, scale hints **`grid_x`** / **`grid_y`** on training submits, and **`timeout_hit`** / **`output_truncated`** when proxy fetch/output limits apply. No API keys or file paths. Grep client stderr alongside API/worker logs by `rid`; see `docs/OPERATIONS_MANUAL.md` §5b.
 
 **Hosted HTTP MCP paused (2026-06):** `https://mcp.barivia.se/mcp` returns **503** (`hosted_mcp_paused`). Use this npm package only.
@@ -62,7 +73,7 @@ Legacy `BARSOM_API_KEY` / `BARSOM_API_URL` / `BARSOM_WORKSPACE_ROOT` are also ac
 
 All multi-action tools follow the `datasets` / `jobs` / `results` / `inference` / `account` pattern: a required `action` enum routes to the correct operation.
 
-**Prompts (not tools):** `info` (short orientation; prefer `guide_barsom_workflow` for depth), `prepare_training` (requires `dataset_id`; checklist text from API when online).
+**Bootstrap:** `guide_barsom_workflow` (no params; plan-scoped orientation from the API — call first). **Prompts (not tools):** `info` (short orientation), `prepare_training` (requires `dataset_id`; checklist text from API when online).
 
 ### Flattened catalog (optional grouping in UIs)
 
@@ -70,12 +81,12 @@ All multi-action tools follow the `datasets` / `jobs` / `results` / `inference`
 |--------|--------|
 | Bootstrap | `guide_barsom_workflow`, prompts `info`, `prepare_training` |
 | Data | `datasets` |
+| Train prep | `training_guidance` |
 | Train & poll | `train`, `jobs`, `training_monitor` (optional) |
 | Results | `results`, `results_explorer` (optional) |
 | Inference | `inference` |
 | Account | `account` |
 | Optional UI | `training_monitor`, `results_explorer` |
-| Advanced | `training_guidance` |
 
 Agents should not call **`_fetch_figure`** from chat; it exists for the Results Explorer MCP App host. Use `results(action=get)` or `results_explorer` instead.
 
@@ -162,7 +173,7 @@ All actions use a frozen trained map — no retraining. Derived columns use **`d
 
 | Tool | Role |
 |------|------|
-| `training_monitor` | Live QE + topographic-error curves (ordering/convergence, dual y-axis), FLooP step progress when applicable; optional — `jobs(action=status)` is enough |
+| `training_monitor` | Live QE + **panel** topographic-error curves (ordering/convergence, dual y-axis); **Panel TE** and **Map TE** scalars at kernel end; FLooP step progress when applicable; optional — `jobs(action=status)` is enough |
 | `results_explorer` | Browse metrics and figures after training completes |
 | `_fetch_figure` | **Host / App only** — Results Explorer invokes this for one raster figure; not for agent chat |
 
@@ -180,6 +191,7 @@ The right viewer depends on **(MCP App support)** **and** **(can the human reach
 
 ### Migration notes
 
+- **Fixed-panel live TE (0.20.4):** mid-training TE curves use a fixed evaluation panel (`te_panel_size` on `train`). Monitors show **Panel TE** and **Map TE** separately; curve tail no longer snaps to map TE.
 - **Features (0.20.0, non-breaking):**
   - **FLooP-SIOM maps are now projectable.** `inference(action=project_columns)`, `inference(action=predict)`, and `datasets(action=add_expression, project_onto_job=<floop_job>)` work on FLooP-SIOM (free/chain) maps — values render onto the FLooP Voronoi layout instead of failing. Grid-only ops (`inference(action=transition_flow | impute_column | render_variant)`) return a clear `unsupported_topology_for_inference` message; use a fixed-grid SOM/SIOM for those. No client change required.
   - **`datasets(action=reduce_spectral)` column selectors.** New optional `columns_range` (`[first, last]`, the server expands the inclusive numeric block — preferred for wide spectra over sending thousands of names) and `columns_except` (drop non-frequency columns like id/label/target from the block, or use alone for "all numeric except these"). `columns_except` composes with `columns_block` or `columns_range`; `columns_block` and `columns_range` are mutually exclusive. Existing `columns_block` callers are unaffected.

package/dist/audit.js

@@ -2,6 +2,7 @@
  * MCP tool audit wrapper — tool name, action, latency, outcome; no secrets.
  */
 import { logAudit } from "./logger.js";
+import { runWithTrace } from "./shared.js";
 /** Keys safe to include in audit param summaries (no paths, keys, or bodies). */
 const AUDIT_PARAM_KEYS = new Set([
     "action",
@@ -160,7 +161,9 @@ export function registerAuditedTool(server, name, description, schema, handler)
     server.tool(name, description, schema, (async (args) => {
         const rec = args;
         const action = typeof rec.action === "string" && rec.action.length > 0 ? rec.action : "default";
-        return runMcpToolAudit(name, action, rec, () => handler(args));
+        // One trace per tool invocation: every apiCall inside this handler shares one
+        // trace_id, so the API + job chain reconstruct the whole logical action.
+        return runWithTrace(() => runMcpToolAudit(name, action, rec, () => handler(args)));
     }));
 }
 /** Attach audit flags to a tool result (stripped before returning to MCP client). */

package/dist/index.js

@@ -1,2 +1,2 @@
 #!/usr/bin/env node
-import{McpServer as e}from"@modelcontextprotocol/sdk/server/mcp.js";import{StdioServerTransport as t}from"@modelcontextprotocol/sdk/server/stdio.js";import{z as o}from"zod";import{getUiCapability as r,registerAppResource as s,RESOURCE_MIME_TYPE as n}from"@modelcontextprotocol/ext-apps/server";import{startVizServer as i}from"./viz-server.js";import{API_KEY as a,apiCall as l,apiRawCall as c,loadViewHtml as p,setVizPort as m,setClientSupportsMcpApps as u,CLIENT_VERSION as d}from"./shared.js";import{registerDatasetsTool as f}from"./tools/datasets.js";import{registerTrainTool as g}from"./tools/train.js";import{registerJobsTool as h,JOBS_DESCRIPTION_BASE as _}from"./tools/jobs.js";import{registerResultsTool as b}from"./tools/results.js";import{registerExploreMapTool as w,RESULTS_EXPLORER_URI as y}from"./tools/explore_map.js";import{registerAccountTool as j}from"./tools/account.js";import{registerInferenceTool as v}from"./tools/inference.js";import{registerGuideBarsomTool as x}from"./tools/guide_barsom.js";import{registerTrainingGuidanceTool as P}from"./tools/training_guidance.js";import{registerFeedbackTool as k}from"./tools/feedback.js";import{registerTrainingMonitorTool as I,TRAINING_MONITOR_URI as A}from"./tools/training_monitor.js";import{resolvePrepareTrainingPromptText as M}from"./prepare_training_prompt.js";a||(console.error("Error: BARIVIA_API_KEY not set. Set it in your MCP client config."),process.exit(1));const T=new e({name:"analytics-engine",version:d,instructions:'# Barivia Mapping Analytics Engine\n\nSelf-organizing map (SOM) analytics: project high-dimensional data to a 2D grid for clusters, gradients, and anomalies.\n\n## Workflow (short)\n\nUpload (`datasets(upload)`) → `datasets(preview)` and `datasets(analyze)` before train → submit one of `train(action=map)`, `train(action=siom_map)`, or `train(action=floop_siom)` (only if plan allows FLooP) → poll `jobs(status)` every 10–15s until `completed` → `results(get)` for metrics and figures (there is no separate analyze tool), then `results_explorer(job_id)` to browse the figures interactively. Then `jobs(compare)`, `results(download/recolor)`, or `inference` (predict / compare / project_columns / impute_column / transition_flow) as needed.\n\n**Full detail:** Call `guide_barsom_workflow` for plan-scoped tool map, training modes, async rules, optional MCP App UIs, and step-by-step SOP (from the Barivia API when online).\n\n## Tool taxonomy (one tool per stage — do not mix)\n\n| Stage | Tool(s) | Contract |\n|-------|---------|----------|\n| Data | `datasets` | upload, preview, analyze, list, **get** (status/staging), subset, add_expression, reduce_spectral, delete — ingest + feature engineering, no map |\n| Train submit | `train` | map, siom_map, impute (sparse → map + dense imputed.csv), floop_siom (entitled). Async, returns job_id. Submit only |\n| Lifecycle | `jobs` | status, list, compare (**training runs**), cancel, delete. No training/scoring here |\n| Artifacts | `results` | get (default=combined only; figures="none" for metrics-only sweeps; figures="all" or results_explorer/download for every plot), export, download, recolor (async). Read/render completed jobs |\n| Frozen-map ops | `inference` | predict (regime-aware; "compact"|"annotated"), batch_predict, impute_column (neighbor-pool fill for a non-training column), compare (cohort density-diff), project_columns, transition_flow (time-ordered rows only), report. No weight updates |\n| Account | `account` | status, burst/compute actions, history, add_funds |\n| Bootstrap | `guide_barsom_workflow`, `training_guidance`, `prepare_training` prompt | orientation, parameter hints, narrative checklist (API/tier-scoped) |\n| Explore (UI) | `results_explorer`, `training_monitor` | richer UX over the same API; `jobs(status)`+`results(get)` still suffice headless |\n| Other | `send_feedback` | only after user agrees |\n\n### Avoid these mis-routes\n- `jobs(compare)` = metrics table across **training runs** (pick best in a sweep). `inference(compare)` = density-diff of one map vs a **second dataset** (drift/cohort). Different tools.\n- Post-training map analysis (predict, compare, project_columns, impute_column, **transition_flow**) is all **inference** — not `results`. `results` is presentation/artifacts only (`recolor` stays here: it re-renders, it doesn\'t touch data).\n- Project a quantity onto the map: existing columns → `inference(project_columns)`; a formula → `datasets(add_expression, project_onto_job=<job>)`.\n\n## Async pattern\n\n- **Manual poll:** Training submits return `job_id` immediately — poll `jobs(status)` every 10–15s. **Running is not failed**; large grids or FLooP-SIOM can take many minutes. `max_nodes` (FLooP) is a total node budget, not grid side length.\n- **Often auto-polled:** `inference` actions (incl. `transition_flow`) and `results(recolor)` may wait in-proxy; if you get a `job_id`, poll `jobs(status)` the same way.\n\nCredits: jobs consume compute credits; check `account(status)` before big runs. Per-request HTTP timeout defaults to 60s (`datasets(analyze)` is async + auto-polled, so it is not bound by it); on slow networks raise `BARIVIA_FETCH_TIMEOUT_MS`. Timeouts are NOT auto-retried (avoids duplicating slow/expensive work).\n\n## Constraints\n\n- Prep ladder: `prepare_training` prompt = narrative checklist; `training_guidance` = structured hints (presets, resolved normalization/transforms, grid/epochs). Review these, then submit with `train`. Do not guess tiers or FLooP entitlement.\n- `inference(predict)`: prefer `dataset_id` for batch and for SIOM/irregular maps; single-row `rows` uses a fast path that can fail on some topologies — retry with `dataset_id`. FLooP-SIOM: if predict jobs fail while grid SIOM works, capture errors + `job_id`.\n- Column names are case-sensitive — match `datasets(preview)`.\n- Default training path is numeric/cyclic/temporal; use explicit `categorical_features` for baseline categoricals. `predict` must match the model contract.\n- After `results(recolor)`, `inference(transition_flow)`, or `inference(project_columns)`, use the **new** `job_id` returned for follow-up `results` if applicable.\n- **Large results / MCP timeouts:** For sweeps, `jobs(compare)`, or metrics-only reads use `results(action=get, figures="none")`. Default `get` inlines combined only. For every plot use `results_explorer` or `results(action=download)`, not `figures="all"` in chat unless the user wants all images inline.'});s(T,y,y,{mimeType:n},async()=>{const e=await p("results-explorer");return{contents:[{uri:y,mimeType:n,text:e??"<html><body>Results Explorer view not built yet. Run: npm run build:views</body></html>"}]}}),s(T,A,A,{mimeType:n},async()=>{const e=await p("training-monitor");return{contents:[{uri:A,mimeType:n,text:e??"<html><body>Training Monitor view not built yet.</body></html>"}]}}),x(T),w(T),I(T),f(T),g(T),h(T,_),b(T),j(T),v(T),P(T),k(T),T.prompt("info","Short orientation for the Barivia Mapping MCP. For full plan-scoped workflow, tool map, and SOP, the model should call guide_barsom_workflow. Use when the user asks what this MCP can do or how to get started.",{},()=>({messages:[{role:"user",content:{type:"text",text:["Give a concise, scannable answer (headers + bullets):","","**What it is:** MCP client to the Barivia mapping engine (2D SOM / SIOM / FLooP-SIOM when entitled) over HTTPS.","","**First step:** Call `guide_barsom_workflow` for plan-scoped bootstrap (full tool list, async rules, training modes, optional MCP Apps, SOP).","","**Core path:** `datasets(upload)` → `datasets(preview)` + `datasets(analyze)` → choose training action → poll `jobs(status)` every 10–15s until completed → `results(get)` (all main figures/metrics; no separate analyze tool) → `results_explorer(job_id)` to browse figures.","",'**Key tools:** `datasets` (data; reduce_spectral for spectra/long blocks), `train` (map/siom_map/impute/floop_siom; pass `label` for readable compare rows; preset=quick for a fast first map), `jobs` (status/list/compare/cancel/delete — lifecycle only), `results` (get/download/export/recolor; figures="none" for metrics-only), `inference` (predict; batch_predict; impute_column for topology-neighbor pool fill; compare; project_columns; transition_flow; report), `account` (status/credits/queue).',"","**Prep help:** `prepare_training` prompt (checklist) · `training_guidance` (presets/JSON hints + resolved normalization/params) — review, then submit with `train`.","","**Browse results:** `results_explorer(job_id)` is the preferred way to explore figures after a first `results(get)` glance; `training_monitor` shows live progress. Both optional — `results` + `jobs(status)` suffice headless.","","**After training:** `jobs(compare)` across runs, `results(recolor)`, `inference(project_columns)` for variables not in training, `inference(transition_flow)` only if rows are time-ordered.","","**Rules:** Running ≠ failed. Column names must match `datasets(preview)` exactly. Do not call `_fetch_figure` from chat (host/UI only); use `results(get)` or `results_explorer`.","","Offer `send_feedback` only after asking the user."].join("\n")}}]})),T.prompt("prepare_training","Narrative pre-training checklist (prompt). Use after upload and before train. Content is tier-scoped from the API when online. Prep ladder: this prompt = story checklist; training_guidance tool = JSON presets/parameter hints. Review, then submit with train.",{dataset_id:o.string().describe("Dataset ID to prepare for training")},async({dataset_id:e})=>({messages:[{role:"user",content:{type:"text",text:await M(e)}}]}));const S=new t;(async function(){try{const e=await i(l,c,p);m(e)}catch(e){process.env.BARIVIA_VIZ_PORT&&console.error("Barivia viz server failed to start:",e)}const e=T.server;e.oninitialized=()=>{const t=e.getClientCapabilities(),o=r(t);u(!!o?.mimeTypes?.includes(n))},await T.connect(S)})().catch(console.error);
+import{McpServer as e}from"@modelcontextprotocol/sdk/server/mcp.js";import{StdioServerTransport as t}from"@modelcontextprotocol/sdk/server/stdio.js";import{z as o}from"zod";import{getUiCapability as r,registerAppResource as s,RESOURCE_MIME_TYPE as n}from"@modelcontextprotocol/ext-apps/server";import{startVizServer as i}from"./viz-server.js";import{API_KEY as a,apiCall as l,apiRawCall as c,loadViewHtml as p,setVizPort as m,setClientSupportsMcpApps as u,CLIENT_VERSION as d}from"./shared.js";import{registerDatasetsTool as f}from"./tools/datasets.js";import{registerTrainTool as g}from"./tools/train.js";import{registerJobsTool as h,JOBS_DESCRIPTION_BASE as _}from"./tools/jobs.js";import{registerResultsTool as b}from"./tools/results.js";import{registerExploreMapTool as w,RESULTS_EXPLORER_URI as y}from"./tools/explore_map.js";import{registerAccountTool as j}from"./tools/account.js";import{registerInferenceTool as v}from"./tools/inference.js";import{registerGuideBarsomTool as x}from"./tools/guide_barsom.js";import{registerTrainingGuidanceTool as P}from"./tools/training_guidance.js";import{registerFeedbackTool as k}from"./tools/feedback.js";import{registerTrainingMonitorTool as I,TRAINING_MONITOR_URI as A}from"./tools/training_monitor.js";import{resolvePrepareTrainingPromptText as M}from"./prepare_training_prompt.js";a||(console.error("Error: BARIVIA_API_KEY not set. Set it in your MCP client config."),process.exit(1));const T=new e({name:"analytics-engine",version:d,instructions:'# Barivia Mapping Analytics Engine\n\nSelf-organizing map (SOM) analytics: project high-dimensional data to a 2D grid for clusters, gradients, and anomalies.\n\n## Workflow (short)\n\nUpload (`datasets(upload)`) → `datasets(preview)` and `datasets(analyze)` before train → submit one of `train(action=map)`, `train(action=siom_map)`, or `train(action=floop_siom)` (only if plan allows FLooP) → poll `jobs(status)` every 10–15s until `completed` → `results(get)` for metrics and figures (there is no separate analyze tool), then `results_explorer(job_id)` to browse the figures interactively. Then `jobs(compare)`, `results(download/recolor)`, or `inference` (predict / compare / project_columns / impute_column / transition_flow) as needed.\n\n**Full detail:** Call `guide_barsom_workflow` for plan-scoped tool map, training modes, async rules, optional MCP App UIs, and step-by-step SOP (from the Barivia API when online).\n\n## Tool taxonomy (one tool per stage — do not mix)\n\n| Stage | Tool(s) | Contract |\n|-------|---------|----------|\n| Data | `datasets` | upload, preview, analyze, list, **get** (status/staging), subset, add_expression, reduce_spectral, delete — ingest + feature engineering, no map |\n| Train submit | `train` | map, siom_map, impute (sparse → map + dense imputed.csv), floop_siom (entitled). Async, returns job_id. Submit only |\n| Lifecycle | `jobs` | status, list, compare (**training runs**), cancel, delete. No training/scoring here |\n| Artifacts | `results` | get (default=combined only; figures="none" for metrics-only sweeps; figures="all" or results_explorer/download for every plot), export, download, recolor (async). Read/render completed jobs |\n| Frozen-map ops | `inference` | predict (regime-aware; "compact"|"annotated"), batch_predict, impute_column (neighbor-pool fill for a non-training column), compare (cohort density-diff), project_columns, transition_flow (time-ordered rows only), report. No weight updates |\n| Account | `account` | status, burst/compute actions, history, add_funds |\n| Bootstrap | `guide_barsom_workflow`, `train`, `training_guidance`, `prepare_training` prompt | orientation, submit map/siom/impute/floop, parameter hints, narrative checklist (API/tier-scoped) |\n| Explore (UI) | `results_explorer`, `training_monitor` | richer UX over the same API; `jobs(status)`+`results(get)` still suffice headless |\n| Other | `send_feedback` | only after user agrees |\n\n### Avoid these mis-routes\n- `jobs(compare)` = metrics table across **training runs** (pick best in a sweep). `inference(compare)` = density-diff of one map vs a **second dataset** (drift/cohort). Different tools.\n- Post-training map analysis (predict, compare, project_columns, impute_column, **transition_flow**) is all **inference** — not `results`. `results` is presentation/artifacts only (`recolor` stays here: it re-renders, it doesn\'t touch data).\n- Project a quantity onto the map: existing columns → `inference(project_columns)`; a formula → `datasets(add_expression, project_onto_job=<job>)`.\n\n## Async pattern\n\n- **Manual poll:** Training submits return `job_id` immediately — poll `jobs(status)` every 10–15s. **Running is not failed**; large grids or FLooP-SIOM can take many minutes. `max_nodes` (FLooP) is a total node budget, not grid side length.\n- **Often auto-polled:** `inference` actions (incl. `transition_flow`) and `results(recolor)` may wait in-proxy; if you get a `job_id`, poll `jobs(status)` the same way.\n\nCredits: jobs consume compute credits; check `account(status)` before big runs. Per-request HTTP timeout defaults to 60s (`datasets(analyze)` is async + auto-polled, so it is not bound by it); on slow networks raise `BARIVIA_FETCH_TIMEOUT_MS`. Timeouts are NOT auto-retried (avoids duplicating slow/expensive work).\n\n## Constraints\n\n- Prep ladder: `prepare_training` prompt = narrative checklist; `training_guidance` = structured hints (presets, resolved normalization/transforms, grid/epochs). Review these, then submit with `train`. Do not guess tiers or FLooP entitlement.\n- `inference(predict)`: prefer `dataset_id` for batch and for SIOM/irregular maps; single-row `rows` uses a fast path that can fail on some topologies — retry with `dataset_id`. FLooP-SIOM: if predict jobs fail while grid SIOM works, capture errors + `job_id`.\n- Column names are case-sensitive — match `datasets(preview)`.\n- Default training path is numeric/cyclic/temporal; use explicit `categorical_features` for baseline categoricals. `predict` must match the model contract.\n- After `results(recolor)`, `inference(transition_flow)`, or `inference(project_columns)`, use the **new** `job_id` returned for follow-up `results` if applicable.\n- **Large results / MCP timeouts:** For sweeps, `jobs(compare)`, or metrics-only reads use `results(action=get, figures="none")`. Default `get` inlines combined only. For every plot use `results_explorer` or `results(action=download)`, not `figures="all"` in chat unless the user wants all images inline.'});s(T,y,y,{mimeType:n},async()=>{const e=await p("results-explorer");return{contents:[{uri:y,mimeType:n,text:e??"<html><body>Results Explorer view not built yet. Run: npm run build:views</body></html>"}]}}),s(T,A,A,{mimeType:n},async()=>{const e=await p("training-monitor");return{contents:[{uri:A,mimeType:n,text:e??"<html><body>Training Monitor view not built yet.</body></html>"}]}}),x(T),w(T),I(T),f(T),g(T),h(T,_),b(T),j(T),v(T),P(T),k(T),T.prompt("info","Short orientation for the Barivia Mapping MCP. For full plan-scoped workflow, tool map, and SOP, the model should call guide_barsom_workflow. Use when the user asks what this MCP can do or how to get started.",{},()=>({messages:[{role:"user",content:{type:"text",text:["Give a concise, scannable answer (headers + bullets):","","**What it is:** MCP client to the Barivia mapping engine (2D SOM / SIOM / FLooP-SIOM when entitled) over HTTPS.","","**First step:** Call `guide_barsom_workflow` for plan-scoped bootstrap (full tool list, async rules, training modes, optional MCP Apps, SOP).","","**Core path:** `datasets(upload)` → `datasets(preview)` + `datasets(analyze)` → choose training action → poll `jobs(status)` every 10–15s until completed → `results(get)` (all main figures/metrics; no separate analyze tool) → `results_explorer(job_id)` to browse figures.","",'**Key tools:** `datasets` (data; reduce_spectral for spectra/long blocks), `train` (map/siom_map/impute/floop_siom; pass `label` for readable compare rows; preset=quick for a fast first map), `jobs` (status/list/compare/cancel/delete — lifecycle only), `results` (get/download/export/recolor; figures="none" for metrics-only), `inference` (predict; batch_predict; impute_column for topology-neighbor pool fill; compare; project_columns; transition_flow; report), `account` (status/credits/queue).',"","**Prep help:** `prepare_training` prompt (checklist) · `training_guidance` (presets/JSON hints + resolved normalization/params) — review, then submit with `train`.","","**Browse results:** `results_explorer(job_id)` is the preferred way to explore figures after a first `results(get)` glance; `training_monitor` shows live progress. Both optional — `results` + `jobs(status)` suffice headless.","","**After training:** `jobs(compare)` across runs, `results(recolor)`, `inference(project_columns)` for variables not in training, `inference(transition_flow)` only if rows are time-ordered.","","**Rules:** Running ≠ failed. Column names must match `datasets(preview)` exactly. Do not call `_fetch_figure` from chat (host/UI only); use `results(get)` or `results_explorer`.","","Offer `send_feedback` only after asking the user."].join("\n")}}]})),T.prompt("prepare_training","Narrative pre-training checklist (prompt). Use after upload and before train. Content is tier-scoped from the API when online. Prep ladder: this prompt = story checklist; training_guidance tool = JSON presets/parameter hints. Review, then submit with train.",{dataset_id:o.string().describe("Dataset ID to prepare for training")},async({dataset_id:e})=>({messages:[{role:"user",content:{type:"text",text:await M(e)}}]}));const S=new t;(async function(){try{const e=await i(l,c,p);m(e)}catch(e){process.env.BARIVIA_VIZ_PORT&&console.error("Barivia viz server failed to start:",e)}const e=T.server;e.oninitialized=()=>{const t=e.getClientCapabilities(),o=r(t);u(!!o?.mimeTypes?.includes(n))},await T.connect(S)})().catch(console.error);

package/dist/inference_prepare.js

@@ -1,9 +1,9 @@
-import { pollUntilComplete } from "./shared.js";
+import { pollUntilComplete, POLL_FINALIZE_MAX_MS } from "./shared.js";
 /**
  * When the API enqueues prepare_training_matrix for cross-dataset scoring,
  * poll it to completion before polling the main inference job.
  */
-export async function pollPrepareIfPresent(data, label, timeoutMs = 600_000) {
+export async function pollPrepareIfPresent(data, label, timeoutMs = POLL_FINALIZE_MAX_MS) {
     const prepareJobId = data.prepare_job_id;
     if (!prepareJobId)
         return null;

package/dist/job_monitor.js

@@ -54,6 +54,14 @@ export function snapshotFromJob(data, elapsedSec, note) {
         snap.qe = Math.round(qe * 10_000) / 10_000;
     if (te != null)
         snap.te = Math.round(te * 10_000) / 10_000;
+    const panelTe = num(data, "panel_topographic_error");
+    if (panelTe != null)
+        snap.panel_te = Math.round(panelTe * 10_000) / 10_000;
+    if (data.kernel_complete === true)
+        snap.kernel_complete = true;
+    const failureStage = str(data, "failure_stage");
+    if (failureStage)
+        snap.failure_stage = failureStage;
     const eta = num(data, "training_eta_sec");
     if (eta != null && eta > 0)
         snap.eta_sec = Math.round(eta);
@@ -78,7 +86,11 @@ export function shouldRecordSnapshot(prev, next) {
         return true;
     if (Math.abs(prev.progress_pct - next.progress_pct) >= 1)
         return true;
-    if (prev.qe !== next.qe || prev.te !== next.te)
+    if (prev.qe !== next.qe || prev.te !== next.te || prev.panel_te !== next.panel_te)
+        return true;
+    if (prev.kernel_complete !== next.kernel_complete)
+        return true;
+    if (prev.failure_stage !== next.failure_stage)
         return true;
     return false;
 }
@@ -92,6 +104,12 @@ export function formatSnapshotLine(s) {
         parts.push(`QE ${s.qe.toFixed(4)}`);
     if (s.te != null)
         parts.push(`Epoch TE ${s.te.toFixed(4)}`);
+    if (s.panel_te != null)
+        parts.push(`Panel TE ${s.panel_te.toFixed(4)}`);
+    if (s.kernel_complete)
+        parts.push("kernel complete");
+    if (s.failure_stage)
+        parts.push(`failure_stage ${s.failure_stage}`);
     if (s.eta_sec != null)
         parts.push(`ETA ~${s.eta_sec}s`);
     if (s.ordering_errors_tail?.length) {
@@ -118,6 +136,22 @@ export function formatMonitorText(result, opts) {
     lines.push(result.suggested_next_step);
     return lines.join("\n");
 }
+function failureStageHint(stage, error) {
+    const err = (error ?? "").toLowerCase();
+    if (stage === "preprocessing") {
+        return "Check datasets(action=preview) and prepare job status; missing columns → subset or train(action=impute).";
+    }
+    if (stage === "training") {
+        if (err.includes("memory") || err.includes("readonlymemory")) {
+            return "Reduce grid size or batch_size and retrain.";
+        }
+        return "Review train hyperparameters (sigma_f, learning_rate, batch_size).";
+    }
+    if (stage === "visualization" || stage === "upload" || stage === "metrics") {
+        return "Re-run jobs(action=monitor, wait_finalize=true) or results(action=get) after finalize completes.";
+    }
+    return "Read the error above before retrying.";
+}
 function suggestedNextStep(job_id, data) {
     const status = String(data.status ?? "");
     if (status === "completed") {
@@ -129,7 +163,9 @@ function suggestedNextStep(job_id, data) {
     }
     if (status === "failed") {
         const stage = str(data, "failure_stage");
-        return `Job failed${stage ? ` at ${stage}` : ""}. Read the error above before retrying.`;
+        const err = str(data, "error");
+        const hint = stage ? failureStageHint(stage, err) : "Read the error above before retrying.";
+        return `Job failed${stage ? ` at ${stage}` : ""}. ${hint}`;
     }
     if (status === "cancelled")
         return `Job cancelled. Confirm with jobs(action=status, job_id="${job_id}").`;

package/dist/prepare_training_prompt.js

@@ -1,8 +1,16 @@
 import { apiCall } from "./shared.js";
-const FALLBACK_PREFIX = "Before upload: for train(action=map) ensure training columns have no NaNs; for train(action=impute) missing cells in training columns are OK (plain numeric only). Please run datasets(action=preview, dataset_id=\"{id}\") to inspect columns, then datasets(action=analyze, dataset_id=\"{id}\") to see which columns and temporal periods are most informative. Then choose the training path: train(action=map, dataset_id=\"{id}\", ...) for complete-case data, train(action=impute, dataset_id=\"{id}\", ...) for sparse matrices, train(action=siom_map, dataset_id=\"{id}\", ...) for a fixed-grid SIOM, or train(action=floop_siom, dataset_id=\"{id}\", ...) for FLooP-SIOM (default topology=free / CHL; optional topology=chain).";
+const FALLBACK_LINES = [
+    "Before upload: for train(action=map) ensure training columns have no NaNs; for train(action=impute) missing cells in training columns are OK (plain numeric only).",
+    'Please run datasets(action=preview, dataset_id="{id}") to inspect columns, then datasets(action=analyze, dataset_id="{id}") to see which columns and temporal periods are most informative.',
+    "Choose the training path:",
+    '- train(action=map, dataset_id="{id}", ...) for complete-case data',
+    '- train(action=impute, dataset_id="{id}", ...) for sparse matrices',
+    '- train(action=siom_map, dataset_id="{id}", ...) for a fixed-grid SIOM',
+    '- train(action=floop_siom, dataset_id="{id}", ...) for FLooP-SIOM (default topology=free / CHL; optional topology=chain)',
+];
 /** Used by the `prepare_training` MCP prompt; prefers tier-scoped text from the API when online. */
 export async function resolvePrepareTrainingPromptText(datasetId) {
-    let promptText = FALLBACK_PREFIX.replaceAll("{id}", datasetId);
+    let promptText = FALLBACK_LINES.join("\n").replaceAll("{id}", datasetId);
     try {
         const data = (await apiCall("GET", `/v1/docs/prepare_training?dataset_id=${datasetId}`));
         if (typeof data.prompt === "string" && data.prompt.trim()) {

package/dist/shared.js

@@ -5,12 +5,13 @@ import fs from "node:fs/promises";
 import { createReadStream, createWriteStream } from "node:fs";
 import { createGzip } from "node:zlib";
 import { createHash, randomUUID } from "node:crypto";
+import { AsyncLocalStorage } from "node:async_hooks";
 import { Readable } from "node:stream";
 import { pipeline } from "node:stream/promises";
 import os from "node:os";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
-import { logInfo } from "./logger.js";
+import { logInfo, logWarn } from "./logger.js";
 // ---------------------------------------------------------------------------
 // Config
 // ---------------------------------------------------------------------------
@@ -20,19 +21,50 @@ export const API_URL = process.env.BARIVIA_API_URL ??
 export const API_KEY = process.env.BARIVIA_API_KEY ?? process.env.BARSOM_API_KEY ?? "";
 export const FETCH_TIMEOUT_MS = parseInt(process.env.BARIVIA_FETCH_TIMEOUT_MS ?? "60000", 10);
 export const MAX_RETRIES = 2;
+export const RETRY_BASE_MS = parseInt(process.env.BARIVIA_RETRY_BASE_MS ?? "1000", 10);
 export const RETRYABLE_STATUS = new Set([502, 503, 504]);
+/** Exponential backoff for transient API retries (attempt 0 → base, 1 → 2×, …). */
+export function retryDelayMs(attempt) {
+    return RETRY_BASE_MS * 2 ** attempt;
+}
+function newRequestId() {
+    return randomUUID();
+}
 /**
  * Single source of truth for the proxy version. Sent to the API as
  * X-Barsom-Client-Version so the server can annotate tool guidance with the
  * wrapper version each action requires. Keep in sync with package.json on bump.
  */
-export const CLIENT_VERSION = "0.20.3";
+export const CLIENT_VERSION = "0.22.1";
 /** User-facing links; keep aligned with barivia.se / api.barivia.se. */
 export const PUBLIC_SITE_ORIGIN = "https://barivia.se";
 /** Self-serve account dashboard (manage plan, billing, and API keys). */
 export const PUBLIC_DASHBOARD_URL = `${PUBLIC_SITE_ORIGIN}/dashboard`;
 /** Poll window for datasets(add_expression) / derive jobs (server-side work can exceed 30s). */
 export const POLL_DERIVE_MAX_MS = 120_000;
+/** Poll window for async datasets(analyze) jobs (large tables can exceed derive timeout). */
+export const POLL_ANALYZE_MAX_MS = 300_000; // 5 min
+/** Poll windows for inference/results async jobs (named for grep/tuning). */
+export const POLL_INFERENCE_MAX_MS = 120_000;
+export const POLL_PROJECT_MAX_MS = 90_000;
+export const POLL_RECOLOR_MAX_MS = 60_000;
+export const POLL_FINALIZE_MAX_MS = 600_000;
+export const DPI_MAP = { standard: 1, retina: 2, print: 4 };
+export function resolveOutputDpi(dpi, defaultDpi = "retina") {
+    return DPI_MAP[dpi ?? defaultDpi] ?? DPI_MAP[defaultDpi] ?? 2;
+}
+export function fmtDensityDiffNode(n) {
+    return `  node ${n.node_index ?? "?"} [${(n.coords ?? [0, 0]).map((v) => Number(v).toFixed(1)).join(",")}] Δ=${Number(n.density_diff ?? 0).toFixed(4)}`;
+}
+/** Map a local upload path to the API Content-Type (csv vs tsv; ignores .gz suffix). */
+export function resolveUploadContentType(filePath) {
+    const lower = filePath.toLowerCase();
+    const base = lower.endsWith(".gz") ? lower.slice(0, -3) : lower;
+    return path.extname(base) === ".tsv" ? "text/tab-separated-values" : "text/csv";
+}
+export function suggestDatasetPreview(datasetId) {
+    return `datasets(action=preview, dataset_id=${datasetId})`;
+}
 /** Large CSV uploads may exceed default FETCH_TIMEOUT_MS. */
 export const UPLOAD_DATASET_TIMEOUT_MS = 180_000;
 /** Files at/above this size use the presigned direct-to-R2 streaming upload path. */
@@ -126,7 +158,11 @@ export async function putPresignedStream(url, srcPath, contentType, timeoutMs =
         const resp = await fetch(url, {
             method: "PUT",
             body: webStream,
-            headers: { "Content-Type": contentType, "Content-Length": String(contentLength) },
+            headers: {
+                "Content-Type": contentType,
+                "Content-Length": String(contentLength),
+                "Content-Encoding": "gzip",
+            },
             // Required by Node's fetch to send a streaming request body.
             duplex: "half",
             signal: controller.signal,
@@ -342,14 +378,39 @@ function throwApiError(status, bodyText, requestId) {
 /**
  * @param requestTimeoutMs Optional per-request timeout (default `BARIVIA_FETCH_TIMEOUT_MS`).
  */
+// ---- Distributed-trace context (W3C traceparent) ----
+//
+// One logical user action (a tool call that may make several API requests, e.g.
+// upload + analyze + train + poll) shares ONE trace_id so the API and the whole job
+// chain (stage -> prepare -> train -> finalize) can be reconstructed end-to-end. The
+// trace id is scoped per tool invocation via AsyncLocalStorage (see registerAuditedTool);
+// each API call mints a fresh span id. Falls back to a per-call trace if no scope is set.
+const _traceStore = new AsyncLocalStorage();
+function _newTraceId() {
+    return randomUUID().replace(/-/g, ""); // 32 hex
+}
+function _newSpanId() {
+    return randomUUID().replace(/-/g, "").slice(0, 16); // 16 hex
+}
+/** Run `fn` within a fresh trace scope so all apiCall()s inside share one trace_id. */
+export function runWithTrace(fn) {
+    return _traceStore.run(_newTraceId(), fn);
+}
+function _currentTraceId() {
+    return _traceStore.getStore() ?? _newTraceId();
+}
+function _traceparentHeader() {
+    return `00-${_currentTraceId()}-${_newSpanId()}-01`;
+}
 export async function apiCall(method, path, body, extraHeaders, requestTimeoutMs) {
     const url = `${API_URL}${path}`;
     const contentType = extraHeaders?.["Content-Type"] ?? "application/json";
-    const requestId = Math.random().toString(36).slice(2, 10);
+    const requestId = newRequestId();
     const headers = {
         Authorization: `Bearer ${API_KEY}`,
         "Content-Type": contentType,
         "X-Request-ID": requestId,
+        traceparent: _traceparentHeader(),
         "X-Barsom-Client-Version": CLIENT_VERSION,
         ...extraHeaders,
     };
@@ -377,7 +438,17 @@ export async function apiCall(method, path, body, extraHeaders, requestTimeoutMs
             const text = await resp.text();
             if (!resp.ok) {
                 if (attempt < MAX_RETRIES && isTransientError(null, resp.status)) {
-                    await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt));
+                    const delayMs = retryDelayMs(attempt);
+                    logWarn("API retry", {
+                        rid: requestId,
+                        method,
+                        path,
+                        attempt: attempt + 1,
+                        max_retries: MAX_RETRIES,
+                        delay_ms: delayMs,
+                        error_code: `http_${resp.status}`,
+                    });
+                    await new Promise((r) => setTimeout(r, delayMs));
                     continue;
                 }
                 logInfo("API response", {
@@ -398,7 +469,17 @@ export async function apiCall(method, path, body, extraHeaders, requestTimeoutMs
         catch (err) {
             lastError = err;
             if (attempt < MAX_RETRIES && isTransientError(err)) {
-                await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt));
+                const delayMs = retryDelayMs(attempt);
+                logWarn("API retry", {
+                    rid: requestId,
+                    method,
+                    path,
+                    attempt: attempt + 1,
+                    max_retries: MAX_RETRIES,
+                    delay_ms: delayMs,
+                    error_code: err instanceof Error ? err.name : "fetch_error",
+                });
+                await new Promise((r) => setTimeout(r, delayMs));
                 continue;
             }
             if (err instanceof DOMException &&
@@ -414,7 +495,7 @@ export async function apiCall(method, path, body, extraHeaders, requestTimeoutMs
 /** Fetch raw bytes from the API (for image downloads). */
 export async function apiRawCall(path, requestTimeoutMs) {
     const url = `${API_URL}${path}`;
-    const requestId = Math.random().toString(36).slice(2, 10);
+    const requestId = newRequestId();
     const effectiveTimeout = requestTimeoutMs ?? FETCH_TIMEOUT_MS;
     let lastError;
     for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
@@ -424,11 +505,21 @@ export async function apiRawCall(path, requestTimeoutMs) {
                 headers: {
                     Authorization: `Bearer ${API_KEY}`,
                     "X-Request-ID": requestId,
+                    traceparent: _traceparentHeader(),
                 },
             }, effectiveTimeout);
             if (!resp.ok) {
                 if (attempt < MAX_RETRIES && isTransientError(null, resp.status)) {
-                    await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt));
+                    const delayMs = retryDelayMs(attempt);
+                    logWarn("API retry", {
+                        rid: requestId,
+                        path,
+                        attempt: attempt + 1,
+                        max_retries: MAX_RETRIES,
+                        delay_ms: delayMs,
+                        error_code: `http_${resp.status}`,
+                    });
+                    await new Promise((r) => setTimeout(r, delayMs));
                     continue;
                 }
                 const text = await resp.text();
@@ -443,7 +534,16 @@ export async function apiRawCall(path, requestTimeoutMs) {
         catch (err) {
             lastError = err;
             if (attempt < MAX_RETRIES && isTransientError(err)) {
-                await new Promise((r) => setTimeout(r, 1000 * 2 ** attempt));
+                const delayMs = retryDelayMs(attempt);
+                logWarn("API retry", {
+                    rid: requestId,
+                    path,
+                    attempt: attempt + 1,
+                    max_retries: MAX_RETRIES,
+                    delay_ms: delayMs,
+                    error_code: err instanceof Error ? err.name : "fetch_error",
+                });
+                await new Promise((r) => setTimeout(r, delayMs));
                 continue;
             }
             if (err instanceof DOMException &&
@@ -518,7 +618,13 @@ export async function fetchNormalizationGuideFromApi() {
     }
 }
 /** Fetch training guidance from API (used by training_guidance tool). Domain knowledge returned when asked; valid license required. Hints are filtered by plan (allowed_job_types). */
+const GUIDE_CACHE_TTL_MS = 60_000;
+let _trainingGuidanceCache = null;
+let _workflowGuideCache = null;
 export async function fetchTrainingGuidanceFromApi() {
+    if (_trainingGuidanceCache && Date.now() - _trainingGuidanceCache.at < GUIDE_CACHE_TTL_MS) {
+        return _trainingGuidanceCache.text;
+    }
     try {
         const config = (await apiCall("GET", "/v1/training/config"));
         const scopeSuffix = formatGuidanceScopeSuffix(config);
@@ -531,36 +637,46 @@ export async function fetchTrainingGuidanceFromApi() {
             ? "\n\n--- Normalization guide (MAD vs SEPD, decision ladder) ---\n\n" + normGuide
             : "\n\n(Normalization guide: GET /v1/docs/normalization — MAD vs SEPD decision ladder.)";
         const hints = config?.training_hints;
+        let text;
         if (Array.isArray(hints) && hints.length > 0) {
-            return presetBlock + "Parameter guidance (from API):\n- " + hints.join("\n- ") + scopeSuffix + normBlock;
+            text = presetBlock + "Parameter guidance (from API):\n- " + hints.join("\n- ") + scopeSuffix + normBlock;
         }
-        const pg = config?.parameter_guidance;
-        if (pg && typeof pg === "object") {
-            const lines = Object.entries(pg).map(([k, v]) => `${k}: ${v}`);
-            return presetBlock + "Parameter guidance (from API):\n- " + lines.join("\n- ") + scopeSuffix + normBlock;
+        else {
+            const pg = config?.parameter_guidance;
+            if (pg && typeof pg === "object") {
+                const lines = Object.entries(pg).map(([k, v]) => `${k}: ${v}`);
+                text = presetBlock + "Parameter guidance (from API):\n- " + lines.join("\n- ") + scopeSuffix + normBlock;
+            }
+            else if (presetBlock) {
+                text = presetBlock + "No additional parameter guidance in config." + scopeSuffix + normBlock;
+            }
+            else {
+                text = "No parameter guidance available.";
+            }
         }
-        if (presetBlock)
-            return presetBlock + "No additional parameter guidance in config." + scopeSuffix + normBlock;
+        _trainingGuidanceCache = { text, at: Date.now() };
+        return text;
     }
     catch (e) {
         if (e?.httpStatus === 401 || e?.httpStatus === 403)
             throw e;
         return "Could not fetch parameter guidance. Check API key and connectivity.";
     }
-    return "No parameter guidance available.";
 }
 /**
  * Full MCP workflow + bootstrap text from API (tier-scoped). Null on network/parse failure (caller may show offline stub).
  * Re-throws 401/403 so the MCP host surfaces auth errors.
  */
 export async function fetchWorkflowGuideFromApi() {
+    if (_workflowGuideCache && Date.now() - _workflowGuideCache.at < GUIDE_CACHE_TTL_MS) {
+        return _workflowGuideCache.text;
+    }
     try {
         const data = (await apiCall("GET", "/v1/docs/workflow"));
         const md = data?.workflow_markdown;
-        if (typeof md === "string" && md.trim().length > 0) {
-            return md.trim();
-        }
-        return null;
+        const text = typeof md === "string" && md.trim().length > 0 ? md.trim() : null;
+        _workflowGuideCache = { text, at: Date.now() };
+        return text;
     }
     catch (e) {
         if (e?.httpStatus === 401 || e?.httpStatus === 403)
@@ -624,6 +740,16 @@ export function inlineAttachBytesUsed() {
 export function shouldFetchAllRemainingFigures(figures) {
     return figures === "all" || figures === "images";
 }
+/** Attach result figures with captions; tracks inlined names for dedup. */
+export async function attachResultsImages(content, jobId, jobType, summary, figures, includeIndividual, inlinedImages) {
+    for (const name of getResultsImagesToFetch(jobType, summary, figures, includeIndividual)) {
+        const cap = getCaptionForImage(name);
+        if (cap)
+            content.push({ type: "text", text: cap });
+        await tryAttachImage(content, jobId, name);
+        inlinedImages.add(name);
+    }
+}
 export async function tryAttachImage(content, jobId, filename) {
     // MCP image content only supports raster MIME types (png/jpeg/gif/webp).
     // PDF and SVG must be retrieved out-of-band via get_result_image.

package/dist/tools/datasets.js

@@ -4,7 +4,7 @@ import { gzipSync } from "node:zlib";
 import { createHash } from "node:crypto";
 import { z } from "zod";
 import { registerAuditedTool } from "../audit.js";
-import { apiCall, getWorkspaceRootAsync, resolveFilePathForUpload, textResult, pollUntilComplete, POLL_DERIVE_MAX_MS, UPLOAD_DATASET_TIMEOUT_MS, LARGE_UPLOAD_BYTES, PRESIGNED_PUT_TIMEOUT_MS, POLL_STAGE_MAX_MS, streamFileSha256, putPresignedStream, } from "../shared.js";
+import { apiCall, getWorkspaceRootAsync, resolveFilePathForUpload, textResult, pollUntilComplete, POLL_DERIVE_MAX_MS, POLL_ANALYZE_MAX_MS, UPLOAD_DATASET_TIMEOUT_MS, LARGE_UPLOAD_BYTES, PRESIGNED_PUT_TIMEOUT_MS, POLL_STAGE_MAX_MS, streamFileSha256, putPresignedStream, resolveUploadContentType, suggestDatasetPreview, } from "../shared.js";
 import { GZIP_UPLOAD_HINT } from "../job_status_format.js";
 /**
  * Normalize a nullable string field from the API. Returns "" for absent values,
@@ -269,9 +269,11 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
             if (!name)
                 throw new Error("datasets(upload) requires name");
             let body;
+            let uploadContentType = "text/csv";
             if (file_path) {
                 await apiCall("GET", "/v1/system/info");
                 const resolved = await resolveFilePathForUpload(file_path, server);
+                uploadContentType = resolveUploadContentType(resolved);
                 const lower = resolved.toLowerCase();
                 // Accept pre-gzipped CSV/TSV (.csv.gz / .tsv.gz) so large tables transfer
                 // ~3x smaller. Already-gzipped files are stored as-is (the platform stores
@@ -317,9 +319,9 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                     // a fresh upload_url and no idempotent_replay flag, so we re-PUT below.
                     if (init.idempotent_replay && !init.upload_url) {
                         return textResult({ id: datasetId, status: init.status, idempotent_replay: true,
-                            suggested_next_step: `datasets(action=preview, dataset_id=${datasetId})` });
+                            suggested_next_step: suggestDatasetPreview(datasetId) });
                     }
-                    await putPresignedStream(init.upload_url, resolved, init.content_type ?? "application/octet-stream", PRESIGNED_PUT_TIMEOUT_MS, isGzipInput);
+                    await putPresignedStream(init.upload_url, resolved, init.content_type ?? uploadContentType, PRESIGNED_PUT_TIMEOUT_MS, isGzipInput);
                     const fin = (await apiCall("POST", `/v1/datasets/${datasetId}/finalize`, {}));
                     const jobId = (fin.id ?? fin.job_id);
                     const poll = await pollUntilComplete(jobId, POLL_STAGE_MAX_MS);
@@ -332,8 +334,8 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                         status: ready ? "ready" : "staging",
                         job_id: jobId,
                         suggested_next_step: ready
-                            ? `datasets(action=preview, dataset_id=${datasetId}) to inspect columns before training.`
-                            : `Still staging; poll jobs(action=status, job_id="${jobId}") then datasets(action=preview, dataset_id=${datasetId}).`,
+                            ? `${suggestDatasetPreview(datasetId)} to inspect columns before training.`
+                            : `Still staging; poll jobs(action=status, job_id="${jobId}") then ${suggestDatasetPreview(datasetId)}.`,
                     });
                 }
                 // Small pre-gzipped files: send the bytes as-is (the API decompresses
@@ -342,13 +344,13 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                     const gzBytes = await fs.readFile(resolved);
                     const data = (await apiCall("POST", "/v1/datasets", gzBytes, {
                         "X-Dataset-Name": name,
-                        "Content-Type": "text/csv",
+                        "Content-Type": uploadContentType,
                         "Content-Encoding": "gzip",
                         "Idempotency-Key": createHash("sha256").update(`${name}\n`).update(gzBytes).digest("hex"),
                     }, UPLOAD_DATASET_TIMEOUT_MS));
                     const gid = data.id ?? data.dataset_id;
                     if (gid != null)
-                        data.suggested_next_step = `Suggested next step: datasets(action=preview, dataset_id=${gid}) to inspect columns before training.`;
+                        data.suggested_next_step = `${suggestDatasetPreview(String(gid))} to inspect columns before training.`;
                     return textResult(data);
                 }
                 body = await fs.readFile(resolved, "utf-8");
@@ -364,7 +366,7 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
             const GZIP_THRESHOLD = 1024 * 1024; // 1 MB
             const uploadHeaders = {
                 "X-Dataset-Name": name,
-                "Content-Type": "text/csv",
+                "Content-Type": uploadContentType,
                 // Deterministic key so a timed-out retry of the SAME upload reconciles to
                 // the original dataset server-side instead of creating a duplicate.
                 "Idempotency-Key": createHash("sha256").update(`${name}\n`).update(body).digest("hex"),
@@ -377,7 +379,7 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
             const data = (await apiCall("POST", "/v1/datasets", uploadBody, uploadHeaders, UPLOAD_DATASET_TIMEOUT_MS));
             const id = data.id ?? data.dataset_id;
             if (id != null)
-                data.suggested_next_step = `Suggested next step: datasets(action=preview, dataset_id=${id}) to inspect columns before training.`;
+                data.suggested_next_step = `${suggestDatasetPreview(String(id))} to inspect columns before training.`;
             return textResult(data);
         }
         if (action === "preview") {
@@ -477,7 +479,7 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                 const jobId = (submit.id ?? submit.job_id);
                 if (!jobId)
                     throw new Error("analyze did not return a job id");
-                const poll = await pollUntilComplete(jobId, POLL_DERIVE_MAX_MS);
+                const poll = await pollUntilComplete(jobId, POLL_ANALYZE_MAX_MS);
                 if (poll.status === "failed") {
                     return { content: [{ type: "text", text: `datasets(analyze) job ${jobId} failed: ${poll.error ?? "unknown error"}` }] };
                 }
@@ -492,7 +494,8 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                     data = (await apiCall("GET", `/v1/datasets/${dataset_id}/analyze`));
                 }
                 else {
-                    throw err;
+                    const msg = err instanceof Error ? err.message : String(err);
+                    throw new Error(`datasets(analyze) failed: ${msg}. If the dataset is very large, try datasets(action=subset, sample_n=10000) first.`);
                 }
             }
             return { content: [{ type: "text", text: formatAnalyzeResult(data, dataset_id) }] };