@barivia/barsom-mcp 0.15.0 → 0.15.3

package/README.md

@@ -86,10 +86,11 @@ Call at the **start of mapping work** (or when the user asks what the MCP can do
 
 | Action | Use when |
 |--------|----------|
-| `upload` | Adding a new CSV — returns dataset_id. Accepts `.csv`/`.tsv` **and pre-gzipped `.csv.gz`/`.tsv.gz`** (compress big tables locally to transfer ~3× less). Large files stream directly to object storage; multi-GB uploads are supported. |
+| `upload` | Adding a new CSV — returns dataset_id and **`stage_job_id`** when staging is enqueued. Accepts `.csv`/`.tsv` **and pre-gzipped `.csv.gz`/`.tsv.gz`** (compress big tables locally to transfer ~3× less). Large files stream directly to object storage; multi-GB uploads are supported. |
 | `preview` | Before training — inspect columns, stats, cyclic/datetime hints |
 | `analyze` | Pre-training recommendations (correlation, columns to consider dropping, etc.) |
 | `list` | Finding dataset IDs |
+| `get` | One dataset by id — status, staging fields, `stage_job_id`, ingest errors |
 | `subset` | Creating a filtered/sliced copy (row_range, filter conditions) |
 | `add_expression` | Add a derived column from an expression (formula → new column on the dataset) |
 | `reduce_spectral` | Pre-training reducer for long ordered numeric blocks (spectra, time series, sensor fingerprints, gene panels). Methods: **pca** (top-k principal components), **log_sample** (k columns at log-spaced indices — scattering, audio bands), **uniform_sample** (k columns at evenly-spaced indices — regularly-sampled time series), **stats** (6 fixed per-row summary columns). All produce one feature vector per row; appends derived columns to the dataset. |
@@ -214,7 +215,7 @@ MCP Client (Cursor/Claude) ←stdio→ @barivia/barsom-mcp ←HTTPS→ api.bariv
 | `401` / invalid key | `BARIVIA_API_KEY` in MCP config; check your email or manage your key in the [account dashboard](https://barivia.se/dashboard). Error text includes a **request id** for support. |
 | Request timed out | Raise `BARIVIA_FETCH_TIMEOUT_MS` (e.g. `120000`). Large uploads already use an extended timeout; `datasets(analyze)` is async (auto-polled). Timeouts are not auto-retried, so re-running a timed-out upload is safe — an idempotency key reconciles it to the original dataset instead of duplicating. |
 | `Path must be within the workspace` / upload can’t find file | Set `BARIVIA_WORKSPACE_ROOT` to your project directory, or use an absolute path / `file:///...` URI. |
-| Job stuck “running” | Poll `jobs(action=status)` every 10–15s; large grids or FLooP-SIOM can take several minutes—not an MCP error. |
+| Job stuck “running” | Poll `jobs(action=status)` every 10–15s; large grids or FLooP-SIOM can take several minutes—not an MCP error. Staged datasets enqueue **`prepare_training_matrix`** (or impute prepare) first — train submit returns `prepare_job_id`; MCP auto-polls prepare when present. CFD mesh submit may return `prepare_job_id` for **`cfd_prepare`** (barmesh auto-polls). |
 | `429` | Rate limit—wait and retry. |
 | Malformed MCP / client errors | Ensure nothing writes to **stdout** except MCP JSON-RPC (the proxy logs API traffic to **stderr** only). |
 

package/dist/index.js

@@ -1,2 +1,2 @@
 #!/usr/bin/env node
-import{McpServer as e}from"@modelcontextprotocol/sdk/server/mcp.js";import{StdioServerTransport as t}from"@modelcontextprotocol/sdk/server/stdio.js";import{z as o}from"zod";import{getUiCapability as r,registerAppResource as n,RESOURCE_MIME_TYPE as s}from"@modelcontextprotocol/ext-apps/server";import{startVizServer as a}from"./viz-server.js";import{API_KEY as i,apiCall as l,apiRawCall as p,loadViewHtml as c,setVizPort as m,setClientSupportsMcpApps as d,CLIENT_VERSION as u}from"./shared.js";import{registerDatasetsTool as f}from"./tools/datasets.js";import{registerTrainTool as g}from"./tools/train.js";import{registerJobsTool as _,JOBS_DESCRIPTION_BASE as b}from"./tools/jobs.js";import{registerResultsTool as y}from"./tools/results.js";import{registerExploreMapTool as h,RESULTS_EXPLORER_URI as w}from"./tools/explore_map.js";import{registerAccountTool as j}from"./tools/account.js";import{registerInferenceTool as v}from"./tools/inference.js";import{registerGuideBarsomTool as P}from"./tools/guide_barsom.js";import{registerTrainingGuidanceTool as x}from"./tools/training_guidance.js";import{registerFeedbackTool as I}from"./tools/feedback.js";import{registerTrainingPrepTools as k,TRAINING_PREP_URI as T}from"./tools/training_prep.js";import{registerTrainingMonitorTool as M,TRAINING_MONITOR_URI as O}from"./tools/training_monitor.js";import{resolvePrepareTrainingPromptText as A}from"./prepare_training_prompt.js";i||(console.error("Error: BARIVIA_API_KEY not set. Set it in your MCP client config."),process.exit(1));const S=new e({name:"analytics-engine",version:u,instructions:'# Barivia Mapping Analytics Engine\n\nSelf-organizing map (SOM) analytics: project high-dimensional data to a 2D grid for clusters, gradients, and anomalies.\n\n## Workflow (short)\n\nUpload (`datasets(upload)`) → `datasets(preview)` and `datasets(analyze)` before train → submit one of `train(action=map)`, `train(action=siom_map)`, or `train(action=floop_siom)` (only if plan allows FLooP) → poll `jobs(status)` every 10–15s until `completed` → `results(get)` for metrics and figures (there is no separate analyze tool). Then `jobs(compare)`, `results(download/recolor/transition_flow)`, or `inference` as needed.\n\n**Full detail:** Call `guide_barsom_workflow` for plan-scoped tool map, training modes, async rules, optional MCP App UIs, and step-by-step SOP (from the Barivia API when online).\n\n## Tool map (compact)\n\n| Area | Tool | Notes |\n|------|------|--------|\n| Data | `datasets` | upload, preview, analyze, list, subset, add_expression, reduce_spectral (pca/log_sample/uniform_sample/stats for long ordered numeric blocks), delete |\n| Train | `train` | map, siom_map, impute (sparse data: map + dense imputed.csv), floop_siom (entitled), baseline_study — all async, returns job_id |\n| Jobs | `jobs` | status, list, compare, cancel, delete (lifecycle). jobs(train_*/run_baseline_study/batch_predict) still work as deprecated forwarders → use train / inference |\n| Results | `results` | get (figures="none" for metrics-only), export, download, recolor (async), transition_flow (async; time-ordered rows only) |\n| Inference | `inference` | predict (regime-aware; output="compact"|"annotated"), batch_predict (fan out many predict jobs), impute_column (neighbor-pool fill for a non-training column), compare, project_columns, report |\n| Account | `account` | status, burst/compute actions, history, add_funds |\n| Bootstrap | `guide_barsom_workflow` | orientation + SOP |\n| Parameters | `training_guidance` | presets and field hints (API-scoped) |\n| Prep | `prepare_training` prompt, `training_prep` + `submit_prepared_training` | checklist / interactive UI |\n| Explore | `results_explorer`, `training_monitor` | optional MCP Apps; `jobs(status)` and `results(get)` suffice without them |\n| Other | `send_feedback` | only after user agrees |\n\n## Async pattern\n\n- **Manual poll:** Training submits return `job_id` immediately — poll `jobs(status)` every 10–15s. **Running is not failed**; large grids or FLooP-SIOM can take many minutes. `max_nodes` (FLooP) is a total node budget, not grid side length.\n- **Often auto-polled:** `inference` actions, `results(recolor)`, `results(transition_flow)` may wait in-proxy; if you get a `job_id`, poll `jobs(status)` the same way.\n\nCredits: jobs consume compute credits; check `account(status)` before big runs. Per-request HTTP timeout defaults to 60s (`datasets(analyze)` is async + auto-polled, so it is not bound by it); on slow networks raise `BARIVIA_FETCH_TIMEOUT_MS`. Timeouts are NOT auto-retried (avoids duplicating slow/expensive work).\n\n## Constraints\n\n- Prep ladder: `prepare_training` prompt = narrative checklist; `training_guidance` = structured hints; `training_prep` = UI + guarded submit. Do not guess tiers or FLooP entitlement.\n- `inference(predict)`: prefer `dataset_id` for batch and for SIOM/irregular maps; single-row `rows` uses a fast path that can fail on some topologies — retry with `dataset_id`. FLooP-SIOM: if predict jobs fail while grid SIOM works, capture errors + `job_id`.\n- Column names are case-sensitive — match `datasets(preview)`.\n- Default training path is numeric/cyclic/temporal; use explicit `categorical_features` for baseline categoricals. `predict` must match the model contract.\n- After `recolor`, `transition_flow`, or `project_columns`, use the **new** `job_id` returned for follow-up `results` if applicable.'});n(S,w,w,{mimeType:s},async()=>{const e=await c("results-explorer");return{contents:[{uri:w,mimeType:s,text:e??"<html><body>Results Explorer view not built yet. Run: npm run build:views</body></html>"}]}}),n(S,T,T,{mimeType:s},async()=>{const e=await c("training-prep");return{contents:[{uri:T,mimeType:s,text:e??"<html><body>Training Preparation view not built yet.</body></html>"}]}}),n(S,O,O,{mimeType:s},async()=>{const e=await c("training-monitor");return{contents:[{uri:O,mimeType:s,text:e??"<html><body>Training Monitor view not built yet.</body></html>"}]}}),P(S),h(S),k(S),M(S),f(S),g(S),_(S,b),y(S),j(S),v(S),x(S),I(S),S.prompt("info","Short orientation for the Barivia Mapping MCP. For full plan-scoped workflow, tool map, and SOP, the model should call guide_barsom_workflow. Use when the user asks what this MCP can do or how to get started.",{},()=>({messages:[{role:"user",content:{type:"text",text:["Give a concise, scannable answer (headers + bullets):","","**What it is:** MCP client to the Barivia mapping engine (2D SOM / SIOM / FLooP-SIOM when entitled) over HTTPS.","","**First step:** Call `guide_barsom_workflow` for plan-scoped bootstrap (full tool list, async rules, training modes, optional MCP Apps, SOP).","","**Core path:** `datasets(upload)` → `datasets(preview)` + `datasets(analyze)` → choose training action → poll `jobs(status)` every 10–15s until completed → `results(get)` (all main figures/metrics; no separate analyze tool).","",'**Key tools:** `datasets` (data; reduce_spectral for spectra/long blocks), `train` (map/siom_map/impute/floop_siom/baseline_study; pass `label` for readable compare rows), `jobs` (status/list/compare/cancel/delete), `results` (get/download/export/recolor/transition_flow; figures="none" for metrics-only), `inference` (predict; batch_predict; impute_column for topology-neighbor pool fill; compare; project_columns; report), `account` (status/credits/queue).',"","**Prep help:** `prepare_training` prompt (checklist) · `training_guidance` (presets/JSON hints) · `training_prep` + `submit_prepared_training` (interactive UI).","","**Optional UI:** `results_explorer`, `training_monitor` — nice for browsing; not required if you use `results` + `jobs(status)`.","","**After training:** `jobs(compare)` across runs, `results(recolor)`, `inference(project_columns)` for variables not in training, `transition_flow` only if rows are time-ordered.","","**Rules:** Running ≠ failed. Column names must match `datasets(preview)` exactly. Do not call `_fetch_figure` from chat (host/UI only); use `results(get)` or `results_explorer`.","","Offer `send_feedback` only after asking the user."].join("\n")}}]})),S.prompt("prepare_training","Narrative pre-training checklist (prompt). Use after upload and before train. Content is tier-scoped from the API when online. Prep ladder: this prompt = story checklist; training_guidance tool = JSON presets/parameter hints; training_prep tool = interactive UI + submit_prepared_training.",{dataset_id:o.string().describe("Dataset ID to prepare for training")},async({dataset_id:e})=>({messages:[{role:"user",content:{type:"text",text:await A(e)}}]}));const C=new t;(async function(){try{const e=await a(l,p,c);m(e)}catch(e){process.env.BARIVIA_VIZ_PORT&&console.error("Barivia viz server failed to start:",e)}const e=S.server;e.oninitialized=()=>{const t=e.getClientCapabilities(),o=r(t);d(!!o?.mimeTypes?.includes(s))},await S.connect(C)})().catch(console.error);
+import{McpServer as e}from"@modelcontextprotocol/sdk/server/mcp.js";import{StdioServerTransport as t}from"@modelcontextprotocol/sdk/server/stdio.js";import{z as o}from"zod";import{getUiCapability as r,registerAppResource as n,RESOURCE_MIME_TYPE as s}from"@modelcontextprotocol/ext-apps/server";import{startVizServer as a}from"./viz-server.js";import{API_KEY as i,apiCall as l,apiRawCall as p,loadViewHtml as c,setVizPort as m,setClientSupportsMcpApps as d,CLIENT_VERSION as u}from"./shared.js";import{registerDatasetsTool as f}from"./tools/datasets.js";import{registerTrainTool as g}from"./tools/train.js";import{registerJobsTool as _,JOBS_DESCRIPTION_BASE as b}from"./tools/jobs.js";import{registerResultsTool as y}from"./tools/results.js";import{registerExploreMapTool as h,RESULTS_EXPLORER_URI as w}from"./tools/explore_map.js";import{registerAccountTool as j}from"./tools/account.js";import{registerInferenceTool as v}from"./tools/inference.js";import{registerGuideBarsomTool as P}from"./tools/guide_barsom.js";import{registerTrainingGuidanceTool as x}from"./tools/training_guidance.js";import{registerFeedbackTool as I}from"./tools/feedback.js";import{registerTrainingPrepTools as k,TRAINING_PREP_URI as T}from"./tools/training_prep.js";import{registerTrainingMonitorTool as M,TRAINING_MONITOR_URI as O}from"./tools/training_monitor.js";import{resolvePrepareTrainingPromptText as A}from"./prepare_training_prompt.js";i||(console.error("Error: BARIVIA_API_KEY not set. Set it in your MCP client config."),process.exit(1));const S=new e({name:"analytics-engine",version:u,instructions:'# Barivia Mapping Analytics Engine\n\nSelf-organizing map (SOM) analytics: project high-dimensional data to a 2D grid for clusters, gradients, and anomalies.\n\n## Workflow (short)\n\nUpload (`datasets(upload)`) → `datasets(preview)` and `datasets(analyze)` before train → submit one of `train(action=map)`, `train(action=siom_map)`, or `train(action=floop_siom)` (only if plan allows FLooP) → poll `jobs(status)` every 10–15s until `completed` → `results(get)` for metrics and figures (there is no separate analyze tool). Then `jobs(compare)`, `results(download/recolor/transition_flow)`, or `inference` as needed.\n\n**Full detail:** Call `guide_barsom_workflow` for plan-scoped tool map, training modes, async rules, optional MCP App UIs, and step-by-step SOP (from the Barivia API when online).\n\n## Tool map (compact)\n\n| Area | Tool | Notes |\n|------|------|--------|\n| Data | `datasets` | upload, preview, analyze, list, **get** (status/staging), subset, add_expression, reduce_spectral, delete |\n| Train | `train` | map, siom_map, impute (sparse data: map + dense imputed.csv), floop_siom (entitled), baseline_study — all async, returns job_id |\n| Jobs | `jobs` | status, list, compare, cancel, delete (lifecycle). jobs(train_*/run_baseline_study/batch_predict) still work as deprecated forwarders → use train / inference |\n| Results | `results` | get (figures="none" for metrics-only), export, download, recolor (async), transition_flow (async; time-ordered rows only) |\n| Inference | `inference` | predict (regime-aware; output="compact"|"annotated"), batch_predict (fan out many predict jobs), impute_column (neighbor-pool fill for a non-training column), compare, project_columns, report |\n| Account | `account` | status, burst/compute actions, history, add_funds |\n| Bootstrap | `guide_barsom_workflow` | orientation + SOP |\n| Parameters | `training_guidance` | presets and field hints (API-scoped) |\n| Prep | `prepare_training` prompt, `training_prep` + `submit_prepared_training` | checklist / interactive UI |\n| Explore | `results_explorer`, `training_monitor` | optional MCP Apps; `jobs(status)` and `results(get)` suffice without them |\n| Other | `send_feedback` | only after user agrees |\n\n## Async pattern\n\n- **Manual poll:** Training submits return `job_id` immediately — poll `jobs(status)` every 10–15s. **Running is not failed**; large grids or FLooP-SIOM can take many minutes. `max_nodes` (FLooP) is a total node budget, not grid side length.\n- **Often auto-polled:** `inference` actions, `results(recolor)`, `results(transition_flow)` may wait in-proxy; if you get a `job_id`, poll `jobs(status)` the same way.\n\nCredits: jobs consume compute credits; check `account(status)` before big runs. Per-request HTTP timeout defaults to 60s (`datasets(analyze)` is async + auto-polled, so it is not bound by it); on slow networks raise `BARIVIA_FETCH_TIMEOUT_MS`. Timeouts are NOT auto-retried (avoids duplicating slow/expensive work).\n\n## Constraints\n\n- Prep ladder: `prepare_training` prompt = narrative checklist; `training_guidance` = structured hints; `training_prep` = UI + guarded submit. Do not guess tiers or FLooP entitlement.\n- `inference(predict)`: prefer `dataset_id` for batch and for SIOM/irregular maps; single-row `rows` uses a fast path that can fail on some topologies — retry with `dataset_id`. FLooP-SIOM: if predict jobs fail while grid SIOM works, capture errors + `job_id`.\n- Column names are case-sensitive — match `datasets(preview)`.\n- Default training path is numeric/cyclic/temporal; use explicit `categorical_features` for baseline categoricals. `predict` must match the model contract.\n- After `recolor`, `transition_flow`, or `project_columns`, use the **new** `job_id` returned for follow-up `results` if applicable.'});n(S,w,w,{mimeType:s},async()=>{const e=await c("results-explorer");return{contents:[{uri:w,mimeType:s,text:e??"<html><body>Results Explorer view not built yet. Run: npm run build:views</body></html>"}]}}),n(S,T,T,{mimeType:s},async()=>{const e=await c("training-prep");return{contents:[{uri:T,mimeType:s,text:e??"<html><body>Training Preparation view not built yet.</body></html>"}]}}),n(S,O,O,{mimeType:s},async()=>{const e=await c("training-monitor");return{contents:[{uri:O,mimeType:s,text:e??"<html><body>Training Monitor view not built yet.</body></html>"}]}}),P(S),h(S),k(S),M(S),f(S),g(S),_(S,b),y(S),j(S),v(S),x(S),I(S),S.prompt("info","Short orientation for the Barivia Mapping MCP. For full plan-scoped workflow, tool map, and SOP, the model should call guide_barsom_workflow. Use when the user asks what this MCP can do or how to get started.",{},()=>({messages:[{role:"user",content:{type:"text",text:["Give a concise, scannable answer (headers + bullets):","","**What it is:** MCP client to the Barivia mapping engine (2D SOM / SIOM / FLooP-SIOM when entitled) over HTTPS.","","**First step:** Call `guide_barsom_workflow` for plan-scoped bootstrap (full tool list, async rules, training modes, optional MCP Apps, SOP).","","**Core path:** `datasets(upload)` → `datasets(preview)` + `datasets(analyze)` → choose training action → poll `jobs(status)` every 10–15s until completed → `results(get)` (all main figures/metrics; no separate analyze tool).","",'**Key tools:** `datasets` (data; reduce_spectral for spectra/long blocks), `train` (map/siom_map/impute/floop_siom/baseline_study; pass `label` for readable compare rows), `jobs` (status/list/compare/cancel/delete), `results` (get/download/export/recolor/transition_flow; figures="none" for metrics-only), `inference` (predict; batch_predict; impute_column for topology-neighbor pool fill; compare; project_columns; report), `account` (status/credits/queue).',"","**Prep help:** `prepare_training` prompt (checklist) · `training_guidance` (presets/JSON hints) · `training_prep` + `submit_prepared_training` (interactive UI).","","**Optional UI:** `results_explorer`, `training_monitor` — nice for browsing; not required if you use `results` + `jobs(status)`.","","**After training:** `jobs(compare)` across runs, `results(recolor)`, `inference(project_columns)` for variables not in training, `transition_flow` only if rows are time-ordered.","","**Rules:** Running ≠ failed. Column names must match `datasets(preview)` exactly. Do not call `_fetch_figure` from chat (host/UI only); use `results(get)` or `results_explorer`.","","Offer `send_feedback` only after asking the user."].join("\n")}}]})),S.prompt("prepare_training","Narrative pre-training checklist (prompt). Use after upload and before train. Content is tier-scoped from the API when online. Prep ladder: this prompt = story checklist; training_guidance tool = JSON presets/parameter hints; training_prep tool = interactive UI + submit_prepared_training.",{dataset_id:o.string().describe("Dataset ID to prepare for training")},async({dataset_id:e})=>({messages:[{role:"user",content:{type:"text",text:await A(e)}}]}));const C=new t;(async function(){try{const e=await a(l,p,c);m(e)}catch(e){process.env.BARIVIA_VIZ_PORT&&console.error("Barivia viz server failed to start:",e)}const e=S.server;e.oninitialized=()=>{const t=e.getClientCapabilities(),o=r(t);d(!!o?.mimeTypes?.includes(s))},await S.connect(C)})().catch(console.error);

package/dist/inference_prepare.js

@@ -0,0 +1,18 @@
+import { pollUntilComplete } from "./shared.js";
+/**
+ * When the API enqueues prepare_training_matrix for cross-dataset scoring,
+ * poll it to completion before polling the main inference job.
+ */
+export async function pollPrepareIfPresent(data, label, timeoutMs = 600_000) {
+    const prepareJobId = data.prepare_job_id;
+    if (!prepareJobId)
+        return null;
+    const poll = await pollUntilComplete(prepareJobId, timeoutMs);
+    if (poll.status === "failed") {
+        throw new Error(`${label}: prepare_training_matrix job ${prepareJobId} failed: ${poll.error ?? "unknown error"}`);
+    }
+    if (poll.status !== "completed") {
+        throw new Error(`${label}: prepare_training_matrix job ${prepareJobId} did not complete (status=${poll.status})`);
+    }
+    return prepareJobId;
+}

package/dist/job_status_format.js

@@ -0,0 +1,75 @@
+/** Format jobs(status) text from GET /v1/jobs/:id payload. Exported for tests. */
+export function formatJobStatusText(job_id, data) {
+    const status = String(data.status ?? "unknown");
+    const progress = (data.progress ?? 0) * 100;
+    const label = data.label != null && data.label !== "" ? String(data.label) : null;
+    const jobDesc = label ? `Job ${label} (id: ${job_id})` : `Job ${job_id}`;
+    const parts = [`${jobDesc}: ${status} (${progress.toFixed(1)}%)`];
+    const attempt = data.attempt != null ? Number(data.attempt) : null;
+    if (attempt != null && attempt > 1) {
+        parts.push(`attempt ${attempt} (job was requeued after worker timeout — not stuck from scratch)`);
+    }
+    const phase = data.progress_phase != null && String(data.progress_phase) !== ""
+        ? String(data.progress_phase)
+        : null;
+    if (status === "running") {
+        if (phase) {
+            parts.push(`phase: ${phase}`);
+        }
+        else {
+            parts.push(`phase: preprocessing (not set yet — often download/parse before first progress tick)`);
+        }
+    }
+    const startedAt = data.started_at != null ? String(data.started_at) : null;
+    if (startedAt) {
+        parts.push(`started_at: ${startedAt}`);
+        const startedMs = Date.parse(startedAt);
+        if (!Number.isNaN(startedMs)) {
+            const elapsedSec = Math.max(0, Math.round((Date.now() - startedMs) / 1000));
+            parts.push(`elapsed: ${elapsedSec}s`);
+        }
+    }
+    const datasetRows = data.dataset_rows != null ? Number(data.dataset_rows) : null;
+    const datasetName = data.dataset_name != null ? String(data.dataset_name) : null;
+    if (datasetRows != null && datasetRows > 0) {
+        const nameBit = datasetName ? ` (${datasetName})` : "";
+        parts.push(`dataset: ${datasetRows.toLocaleString()} rows${nameBit}`);
+    }
+    if (status === "running" &&
+        progress < 5 &&
+        datasetRows != null &&
+        datasetRows >= 1_000_000) {
+        parts.push("Advisory: still in early preprocessing (download/parse/normalize). Progress may stay near 0% for a long time on large tables; epoch counter starts after ingest.");
+    }
+    else if (status === "running" && progress < 5 && !phase) {
+        parts.push("Advisory: still in early preprocessing — epoch counter starts after ingest completes.");
+    }
+    if (status === "completed") {
+        const finalizeId = data.finalize_job_id != null && String(data.finalize_job_id) !== ""
+            ? String(data.finalize_job_id)
+            : null;
+        if (finalizeId) {
+            parts.push(`Training kernel done; finalize_training still running (finalize_job_id=${finalizeId}). Poll jobs(action=status) again to auto-wait, or poll that job directly before results(get).`);
+        }
+        else {
+            parts.push(`Results ready. Use results(action=get, job_id="${job_id}") to retrieve.`);
+        }
+    }
+    else if (status === "failed") {
+        const failureStage = data.failure_stage != null && String(data.failure_stage) !== ""
+            ? ` [${data.failure_stage}]`
+            : "";
+        parts.push(`Error${failureStage}: ${data.error ?? "unknown"}`);
+    }
+    else if (status === "cancelled") {
+        parts.push(`Cancelled. Poll again to confirm; dataset and partial worker state are unchanged.`);
+    }
+    return parts.join(" | ");
+}
+export function formatJobCancelText(job_id, data) {
+    const status = String(data.status ?? "cancelled");
+    const ts = new Date().toISOString();
+    return (`Cancel accepted at ${ts} for job ${job_id}: status=${status}. ` +
+        `Poll jobs(action=status, job_id="${job_id}") to confirm; training stops between phases (may take up to ~30s).`);
+}
+export const GZIP_UPLOAD_HINT = "Try gzip: save as .csv.gz (often 2–3× smaller). The upload limit applies to the compressed file size on presigned/large uploads.";

package/dist/shared.js

@@ -26,7 +26,7 @@ export const RETRYABLE_STATUS = new Set([502, 503, 504]);
  * X-Barsom-Client-Version so the server can annotate tool guidance with the
  * wrapper version each action requires. Keep in sync with package.json on bump.
  */
-export const CLIENT_VERSION = "0.15.0";
+export const CLIENT_VERSION = "0.15.3";
 /** User-facing links; keep aligned with barivia.se / api.barivia.se. */
 export const PUBLIC_SITE_ORIGIN = "https://barivia.se";
 /** Self-serve account dashboard (manage plan, billing, and API keys). */

package/dist/tools/datasets.js

@@ -5,6 +5,7 @@ import { createHash } from "node:crypto";
 import { z } from "zod";
 import { registerAuditedTool } from "../audit.js";
 import { apiCall, getWorkspaceRootAsync, resolveFilePathForUpload, textResult, pollUntilComplete, POLL_DERIVE_MAX_MS, UPLOAD_DATASET_TIMEOUT_MS, LARGE_UPLOAD_BYTES, PRESIGNED_PUT_TIMEOUT_MS, POLL_STAGE_MAX_MS, streamFileSha256, putPresignedStream, } from "../shared.js";
+import { GZIP_UPLOAD_HINT } from "../job_status_format.js";
 /**
  * Format a dataset analyze result (from either the async job summary or the
  * legacy sync GET response) into the human-readable text block.
@@ -95,18 +96,21 @@ function formatAnalyzeResult(data, dataset_id) {
 export function registerDatasetsTool(server) {
     registerAuditedTool(server, "datasets", `Manage datasets: upload, preview, list, subset, add_expression, or delete.
 
+Formats: plain CSV/TSV or gzip (.csv.gz / .tsv.gz). For files above ~100 MB, prefer .csv.gz (often 2–3× smaller); large uploads use presigned direct-to-storage PUT and accept gzip bodies.
+
 | Action | Use when |
 |--------|----------|
-| upload | You have a CSV file to add — do this first |
+| upload | You have a CSV or .csv.gz file to add — do this first |
 | preview | Before train(action=map) — always preview an unfamiliar dataset to spot cyclics, nulls, column types |
 | analyze | Pre-training column analysis: correlation matrix, periodicity ranking, column recommendations (train / drop / project later). Run after preview, before train_map. |
 | list | Finding dataset IDs for train_map, preview, or subset — see all available datasets |
+| get | Fetch one dataset by id — status, staging fields, ingest_error (use after upload or when staging is slow) |
 | subset | Creating a filtered/sliced view without re-uploading the full CSV |
 | add_expression | Add a computed column from an expression (dataset_id + name + expression). With project_onto_job=<training_job_id>, instead projects the computed expression onto that map as a component plane in one step (no column added). |
 | reduce_spectral | Collapse a long ordered numeric block (e.g. 1000-point spectrum, time series, sensor fingerprint) into a small per-row feature set so the SOM can train on signal-dense features. Methods: pca, log_sample, uniform_sample, stats. |
 | delete | Cleaning up after experiments or freeing the dataset slot |
 
-action=upload: PREFER file_path — server reads from workspace root (token-efficient; no file content in context). Use csv_data only for small inline pastes (e.g. <10KB). Returns dataset ID. Then use datasets(action=preview) before train(action=map).
+action=upload: PREFER file_path — server reads from workspace root (token-efficient; no file content in context). Accepts .csv, .tsv, .csv.gz, .tsv.gz. Use csv_data only for small inline pastes (e.g. <10KB). Returns dataset ID. Then use datasets(action=preview) before train(action=map). If plain CSV exceeds the 5 GB upload cap, gzip it first (.csv.gz).
 
 Step 0 (before upload): (1) CSV with header; one row per observation. (2) Columns you will use for training must be numeric (or datetime if you will use temporal extraction). (3) Complete-case path (train action=map): no NaNs in training columns. Sparse path (train action=impute): missing cells in training columns are OK — the job trains missSOM and returns dense imputed.csv; plain numeric columns only. (4) Categoricals: encode (e.g. one-hot, label) or exclude — do not use raw categorical columns as training features; preview shows which columns are non-numeric. (5) Optional: if you plan to use transition_flow, sort rows chronologically before upload.
 Step 1 (after upload): always datasets(action=preview) to verify column types and spot cyclics/datetime, then choose train_map (complete-case) or train_impute (sparse) before submit.
@@ -114,6 +118,7 @@ Step 1 (after upload): always datasets(action=preview) to verify column types an
 action=preview: Show columns, stats, sample rows, cyclic/datetime detections. ALWAYS preview before train(action=map) on an unfamiliar dataset.
 action=analyze: Pre-training analysis on numeric columns — Pearson correlation, autocorrelation periodicity scores, and column recommendations (train, consider_dropping, project_later, low_variance). Use to choose which columns to include in training and which to project onto the map after training.
 action=list: List all datasets belonging to the organisation with id, name, rows, cols, created_at (use created_at or id to distinguish datasets with the same name).
+action=get: Fetch one dataset by dataset_id — returns status, staged_prefix, staged_version, ingest_error, and stage_job_id when staging is pending.
 action=subset: Create a new dataset from a subset of an existing one. Requires name and at least one of row_range, filters, or sample_n.
   - row_range: [start, end] 1-based inclusive (e.g. [1, 2000] for first 2000 rows)
   - filters: array of conditions, ALL must match (AND logic). Each: { column, op, value }.
@@ -134,12 +139,12 @@ BEST FOR: Tabular numeric data. CSV with header required.
 NOT FOR: Real-time data streams or binary files — upload a snapshot CSV instead.
 ESCALATION: If upload fails with column errors, open the file locally and verify the header row. If preview shows nulls in training columns: use train(action=impute) for sparse data, or clean/subset for train(action=map).`, {
         action: z
-            .enum(["upload", "preview", "analyze", "list", "subset", "delete", "add_expression", "reduce_spectral"])
-            .describe("upload: add CSV; preview: inspect columns/stats; analyze: pre-training correlation and periodicity; list: see all datasets; subset: create filtered subset; delete: remove dataset; add_expression: add derived column from expression (or, with project_onto_job, project the expression onto a trained map as a component plane); reduce_spectral: collapse a long ordered numeric block (e.g. spectrum, time series) into a small per-row feature set via PCA / log_sample / uniform_sample / stats"),
+            .enum(["upload", "preview", "analyze", "list", "get", "subset", "delete", "add_expression", "reduce_spectral"])
+            .describe("upload: add CSV or .csv.gz; preview: inspect columns/stats; analyze: pre-training correlation and periodicity; list: see all datasets; get: fetch one dataset metadata (status/staging); subset: create filtered subset; delete: remove dataset; add_expression: add derived column from expression (or, with project_onto_job, project the expression onto a trained map as a component plane); reduce_spectral: collapse a long ordered numeric block (e.g. spectrum, time series) into a small per-row feature set via PCA / log_sample / uniform_sample / stats"),
         name: z.string().optional().describe("Dataset name (required for action=upload and subset)"),
-        file_path: z.string().optional().describe("Path to local CSV (PREFERRED): absolute path, file:// URI, or relative to workspace root. Token-efficient; server reads file."),
+        file_path: z.string().optional().describe("Path to local CSV or .csv.gz (PREFERRED): absolute path, file:// URI, or relative to workspace root. Token-efficient; server reads file. Use .csv.gz for large scientific tables."),
         csv_data: z.string().optional().describe("Inline CSV string for small pastes only (<10KB). Avoid for large files — use file_path instead to avoid token explosion."),
-        dataset_id: z.string().optional().describe("Dataset ID (required for preview, subset, and delete)"),
+        dataset_id: z.string().optional().describe("Dataset ID (required for preview, get, subset, and delete)"),
         n_rows: z.number().int().optional().default(5).describe("Sample rows to return (preview only)"),
         row_range: z
             .tuple([z.number().int(), z.number().int()])
@@ -243,13 +248,24 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
                         `Set BARIVIA_WORKSPACE_ROOT in your MCP config env if needed (current: ${await getWorkspaceRootAsync(server)}).`);
                 }
                 if (stat.size > HARD_MAX_BYTES) {
-                    throw new Error(`File too large (${(stat.size / 1024 / 1024 / 1024).toFixed(2)} GB). Maximum upload size is 5 GB.`);
+                    const gzipHint = isGzipInput ? "" : ` ${GZIP_UPLOAD_HINT}`;
+                    throw new Error(`File too large (${(stat.size / 1024 / 1024 / 1024).toFixed(2)} GB). Maximum upload size is 5 GB.${gzipHint}`);
                 }
                 // Large files: stream gzip directly to R2 (presigned PUT), then stage async.
                 // Never reads the file into memory or sends it through the API/Cloudflare.
                 if (stat.size >= LARGE_UPLOAD_BYTES) {
                     const idem = await streamFileSha256(resolved);
-                    const init = (await apiCall("POST", "/v1/datasets/upload-url", { name, size_bytes: stat.size }, { "Idempotency-Key": idem }));
+                    let init;
+                    try {
+                        init = (await apiCall("POST", "/v1/datasets/upload-url", { name, size_bytes: stat.size }, { "Idempotency-Key": idem }));
+                    }
+                    catch (e) {
+                        const msg = e instanceof Error ? e.message : String(e);
+                        if (msg.includes("dataset_too_large") && !isGzipInput) {
+                            throw new Error(`${msg} ${GZIP_UPLOAD_HINT}`);
+                        }
+                        throw e;
+                    }
                     const datasetId = (init.dataset_id ?? init.id);
                     // Only short-circuit when the prior upload actually completed. A stale
                     // `pending_upload` row (a previous PUT that never landed) comes back with
@@ -584,9 +600,43 @@ ESCALATION: If upload fails with column errors, open the file locally and verify
             return textResult(data);
         }
         if (action === "list") {
-            const data = await apiCall("GET", "/v1/datasets");
+            const data = (await apiCall("GET", "/v1/datasets"));
+            if (Array.isArray(data)) {
+                const lines = data.map((ds) => {
+                    const id = String(ds.id ?? "");
+                    const name = String(ds.name ?? "");
+                    const rows = ds.rows != null ? Number(ds.rows) : "?";
+                    const cols = ds.cols != null ? Number(ds.cols) : "?";
+                    const st = ds.status != null ? String(ds.status) : "ready";
+                    const statusBit = st !== "ready" ? ` | status=${st}` : "";
+                    const err = ds.ingest_error != null && String(ds.ingest_error) !== ""
+                        ? ` | ingest_error=${String(ds.ingest_error)}`
+                        : "";
+                    return `${name} (${id}) — ${rows}×${cols}${statusBit}${err}`;
+                });
+                return { content: [{ type: "text", text: lines.length > 0 ? lines.join("\n") : "No datasets." }] };
+            }
             return textResult(data);
         }
+        if (action === "get") {
+            if (!dataset_id)
+                throw new Error("datasets(get) requires dataset_id");
+            const ds = (await apiCall("GET", `/v1/datasets/${dataset_id}`));
+            const lines = [
+                `Dataset: ${ds.name ?? "?"} (${ds.id ?? dataset_id})`,
+                `Status: ${ds.status ?? "ready"}`,
+                `Rows × cols: ${ds.rows ?? "?"} × ${ds.cols ?? "?"}`,
+                ds.size_bytes != null ? `Size: ${Number(ds.size_bytes).toLocaleString()} bytes` : "",
+                ds.staged_prefix != null ? `Staged prefix: ${String(ds.staged_prefix)}` : "",
+                ds.staged_version != null ? `Staged version: ${String(ds.staged_version)}` : "",
+                ds.stage_job_id != null ? `Stage job: ${String(ds.stage_job_id)} (poll jobs(action=status))` : "",
+                ds.ingest_error != null && String(ds.ingest_error) !== ""
+                    ? `Ingest error: ${String(ds.ingest_error)}`
+                    : "",
+                ds.created_at != null ? `Created: ${String(ds.created_at)}` : "",
+            ].filter(Boolean);
+            return { content: [{ type: "text", text: lines.join("\n") }] };
+        }
         if (action === "delete") {
             if (!dataset_id)
                 throw new Error("datasets(delete) requires dataset_id");

package/dist/tools/inference.js

@@ -1,6 +1,7 @@
 import { z } from "zod";
 import { registerAuditedTool } from "../audit.js";
 import { apiCall, pollUntilComplete, tryAttachImage } from "../shared.js";
+import { pollPrepareIfPresent } from "../inference_prepare.js";
 const PREDICT_PREVIEW_ROW_CAP = 10;
 /** One line per scored row when worker embedded `predictions_preview` (n_rows ≤ cap). Exported for tests. */
 export function formatPredictPreviewLines(summary) {
@@ -153,6 +154,7 @@ action=report: Returns a report manifest for the given job_id (job must be compl
                 body.rows = rows;
             const data = (await apiCall("POST", `/v1/results/${job_id}/predict`, body));
             const predictJobId = data.id;
+            const prepareId = await pollPrepareIfPresent(data, "inference(predict)");
             // annotated runs over the full dataset; allow up to 120s like compact
             const poll = await pollUntilComplete(predictJobId, 120_000);
             if (poll.status === "completed") {
@@ -180,6 +182,7 @@ action=report: Returns a report manifest for the given job_id (job must be compl
                         : `Output: predictions.csv (row_id, bmu_x, bmu_y, bmu_node_index, cluster_id, quantization_error, potential_anomaly). Summary includes mean_qe, max_qe, qe_p95. Clusters: ${Object.keys(summary.cluster_counts ?? {}).length}.`);
                 return { content: [{ type: "text", text: [
                                 headerLine,
+                                prepareId ? `(prepare_training_matrix ${prepareId} completed before scoring)` : "",
                                 `Regime: ${regime}${regime === "training" ? " (scored against the training set)" : ""} | Style: ${effectiveStyle}`,
                                 `Rows scored: ${summary.n_rows ?? "?"}`,
                                 metricsLine,
@@ -271,6 +274,7 @@ action=report: Returns a report manifest for the given job_id (job must be compl
                 body.colormap = colormap;
             const data = (await apiCall("POST", `/v1/results/${job_id}/project_columns`, body));
             const projJobId = data.id;
+            const prepareId = await pollPrepareIfPresent(data, "inference(project_columns)");
             const poll = await pollUntilComplete(projJobId, 90_000);
             if (poll.status === "completed") {
                 const results = (await apiCall("GET", `/v1/results/${projJobId}`));
@@ -280,6 +284,7 @@ action=report: Returns a report manifest for the given job_id (job must be compl
                 const fmtStat = (v) => v !== null && v !== undefined ? Number(v).toFixed(3) : "—";
                 const lines = [
                     `Project columns complete — job: ${projJobId}`,
+                    prepareId ? `(prepare_training_matrix ${prepareId} completed before projection)` : "",
                     `Columns: ${summary.columns?.join(", ") ?? columns.join(", ")} | Samples: ${summary.n_samples ?? "?"}`,
                     ...Object.entries(columnStats).map(([col, s]) => {
                         const parts = [`${col}: nodes=${s.n_nodes_with_data ?? "?"}`];

package/dist/tools/jobs.js

@@ -1,6 +1,8 @@
 import { z } from "zod";
 import { registerAuditedTool } from "../audit.js";
 import { apiCall, textResult } from "../shared.js";
+import { formatJobCancelText, formatJobStatusText } from "../job_status_format.js";
+import { pollFinalizeIfPresent, refreshJobAfterFinalize } from "../train_finalize.js";
 import { runTrain } from "./train.js";
 import { runBatchPredict } from "./inference.js";
 import { TRAINING_INPUT_SCHEMA } from "./training_core.js";
@@ -195,23 +197,20 @@ export function registerJobsTool(server, description) {
         if (action === "status") {
             if (!job_id)
                 throw new Error("jobs(status) requires job_id");
-            const data = (await apiCall("GET", `/v1/jobs/${job_id}`));
-            const status = data.status;
-            const progress = (data.progress ?? 0) * 100;
-            const label = data.label != null && data.label !== "" ? String(data.label) : null;
-            const jobDesc = label ? `Job ${label} (id: ${job_id})` : `Job ${job_id}`;
-            let text = `${jobDesc}: ${status} (${progress.toFixed(1)}%)`;
-            const phase = data.progress_phase != null && data.progress_phase !== "" ? String(data.progress_phase) : null;
-            if (phase && status === "running") {
-                text += ` — phase: ${phase}`;
+            let data = (await apiCall("GET", `/v1/jobs/${job_id}`));
+            const status = String(data.status ?? "");
+            if (status === "completed" && data.finalize_job_id) {
+                const { note } = await pollFinalizeIfPresent(job_id, data);
+                data = await refreshJobAfterFinalize(job_id);
+                const text = formatJobStatusText(job_id, data);
+                return {
+                    content: [{
+                            type: "text",
+                            text: note ? `${text}\n${note}` : text,
+                        }],
+                };
             }
-            if (status === "completed") {
-                text += ` | Results ready. Use results(action=get, job_id="${job_id}") to retrieve.`;
-            }
-            else if (status === "failed") {
-                text += ` | Error: ${data.error ?? "unknown"}`;
-            }
-            return { content: [{ type: "text", text }] };
+            return { content: [{ type: "text", text: formatJobStatusText(job_id, data) }] };
         }
         if (action === "list") {
             const listPath = dataset_id ? `/v1/jobs?dataset_id=${dataset_id}` : "/v1/jobs";
@@ -238,8 +237,8 @@ export function registerJobsTool(server, description) {
         if (action === "cancel") {
             if (!job_id)
                 throw new Error("jobs(cancel) requires job_id");
-            const data = await apiCall("POST", `/v1/jobs/${job_id}/cancel`);
-            return textResult(data);
+            const data = (await apiCall("POST", `/v1/jobs/${job_id}/cancel`));
+            return { content: [{ type: "text", text: formatJobCancelText(job_id, data) }] };
         }
         if (action === "delete") {
             if (!job_id)

package/dist/tools/results.js

@@ -30,9 +30,9 @@ action=export: Structured data exports. Use export_type= to choose what to expor
   - export_type=nodes: per-node hit count + feature stats. Profile clusters and operating modes.
 
 action=download: Save figures to disk. Use so user can open, share, or version files locally.
-  - folder: e.g. "." or "./results". Interpreted relative to the client's current working directory (or workspace). Files are written into a per-job subfolder named job_type + label (or job_id) under this folder, so downloading several jobs (even with the same label across types) never overwrites shared filenames like summary.json.
+  - folder: e.g. "." or "./results". Relative to the client's working directory (or MCP workspace). Each job lands in a dedicated subfolder under folder: {job_type}_{label} when label was set at train time, otherwise {job_type}_{job_id_prefix} — so parallel downloads never overwrite summary.json or figure names.
   - figures: "all" (default) or array of filenames.
-  - include_json: also save summary.json.
+  - include_json: when true, also saves summary.json (and adds it to the download list if figures omitted).
 
 action=recolor: Change colormap or output format — no retraining. Returns a new job_id; auto-polls 60s.
   AFTER: use results(action=get, job_id=NEW_JOB_ID).

package/dist/tools/train.js

@@ -1,6 +1,7 @@
 import { z } from "zod";
 import { registerAuditedTool } from "../audit.js";
 import { apiCall, textResult } from "../shared.js";
+import { buildTrainSubmitExtras } from "../train_submit_message.js";
 import { TRAINING_INPUT_SCHEMA, buildTrainMapParams, fetchTrainingPresets, } from "./training_core.js";
 export const TRAIN_DESCRIPTION = `Submit a self-organizing map training job. Returns a job_id; poll with jobs(action=status) and then results(action=get). All actions are async.
 
@@ -124,29 +125,41 @@ export async function runTrain(action, args) {
             : action === "impute" ? "variant=train_impute"
                 : "variant=som";
         data.effective_params = `${variantPrefix}, ${paramSummary}`;
+        const hasTransforms = action !== "impute" &&
+            transforms != null &&
+            typeof transforms === "object" &&
+            Object.keys(transforms).length > 0;
+        const extras = buildTrainSubmitExtras({
+            newJobId,
+            totalRows,
+            hasTransforms,
+            prepareJobId: data.prepare_job_id ?? null,
+            variantPrefix,
+            paramSummary,
+            impute: action === "impute",
+        });
         try {
             const sys = (await apiCall("GET", "/v1/system/info"));
             const pending = Number(sys.status?.pending_jobs ?? sys.pending_jobs ?? 0);
             const totalEta = Number(sys.training_time_estimates_seconds?.total ??
                 (sys.gpu_available ? 45 : 120));
             const waitMinutes = Math.round((pending * totalEta) / 60);
-            let msg = `Job submitted (${variantPrefix}, ${paramSummary}). `;
+            let msg = extras.message;
             if (waitMinutes > 1)
-                msg += `You are #${pending + 1} in queue. Estimated wait: ~${waitMinutes} min. `;
-            msg += `Poll with jobs(action=status, job_id="${newJobId}"). When status is completed, use results(action=get, job_id="${newJobId}") to view the map${action === "impute" ? " and imputed.csv" : ""} and metrics.`;
-            msg += ` Optional: training_monitor(job_id="${newJobId}") for charts—not required.`;
+                msg = `You are #${pending + 1} in queue. Estimated wait: ~${waitMinutes} min. ${msg}`;
             if (effectiveGrid && totalRows > 0 && effectiveGrid[0] * effectiveGrid[1] > totalRows * 0.75) {
                 msg += ` Note: Grid may be large for ${totalRows} rows (consider grid=auto for fewer dead nodes).`;
             }
             data.message = msg;
+            data.suggested_next_step = extras.suggested_next_step;
         }
         catch {
-            let msg = `Job submitted (${variantPrefix}, ${paramSummary}). Poll with jobs(action=status, job_id="${newJobId}"). When status is completed, use results(action=get, job_id="${newJobId}") to view the map${action === "impute" ? " and imputed.csv" : ""} and metrics.`;
-            msg += ` Optional: training_monitor(job_id="${newJobId}") for charts—not required.`;
+            let msg = extras.message;
             if (effectiveGrid && totalRows > 0 && effectiveGrid[0] * effectiveGrid[1] > totalRows * 0.75) {
                 msg += ` Note: Grid may be large for ${totalRows} rows (consider grid=auto for fewer dead nodes).`;
             }
             data.message = msg;
+            data.suggested_next_step = extras.suggested_next_step;
         }
         return textResult(data);
     }
@@ -229,6 +242,13 @@ export async function runTrain(action, args) {
             submitBody.label = label;
         const data = (await apiCall("POST", "/v1/jobs", submitBody));
         const newJobId = data.id;
+        let totalRows = 0;
+        try {
+            const preview = (await apiCall("GET", `/v1/datasets/${dataset_id}/preview?n_rows=1`));
+            totalRows = Number(preview?.total_rows ?? 0);
+        }
+        catch { /* ignore */ }
+        const hasTransforms = transforms != null && typeof transforms === "object" && Object.keys(transforms).length > 0;
         const maxNodeSummary = max_nodes === undefined ? "max_nodes=auto(~2*sqrt(n_samples))" : `max_nodes=${max_nodes}`;
         const paramSummary = [
             "variant=floop",
@@ -238,10 +258,17 @@ export async function runTrain(action, args) {
             gamma !== undefined ? `gamma=${gamma}` : "",
         ].filter(Boolean).join(", ");
         data.effective_params = paramSummary;
-        data.message =
-            `Job submitted (${paramSummary}). Poll with jobs(action=status, job_id="${newJobId}"). ` +
-                `When status is completed, use results(action=get, job_id="${newJobId}") to view FLooP-SIOM figures and metrics. ` +
-                `Optional: training_monitor(job_id="${newJobId}") for charts—not required.`;
+        const extras = buildTrainSubmitExtras({
+            newJobId,
+            totalRows,
+            hasTransforms,
+            prepareJobId: data.prepare_job_id ?? null,
+            variantPrefix: "variant=floop",
+            paramSummary,
+            resultsHint: "view FLooP-SIOM figures and metrics",
+        });
+        data.message = extras.message;
+        data.suggested_next_step = extras.suggested_next_step;
         return textResult(data);
     }
     throw new Error("Invalid train action");

package/dist/train_finalize.js

@@ -0,0 +1,25 @@
+import { apiCall, pollUntilComplete } from "./shared.js";
+/**
+ * When train compute finishes, finalize_training may still be rendering figures on worker-io.
+ * Poll it before fetching results.
+ */
+export async function pollFinalizeIfPresent(jobId, data, timeoutMs = 600_000) {
+    const finalizeJobId = data.finalize_job_id;
+    if (!finalizeJobId)
+        return { finalizeJobId: null, note: null };
+    const poll = await pollUntilComplete(finalizeJobId, timeoutMs);
+    if (poll.status === "failed") {
+        throw new Error(`Training job ${jobId}: finalize_training ${finalizeJobId} failed: ${poll.error ?? "unknown error"}`);
+    }
+    if (poll.status !== "completed") {
+        throw new Error(`Training job ${jobId}: finalize_training ${finalizeJobId} did not complete (status=${poll.status})`);
+    }
+    return {
+        finalizeJobId,
+        note: `Finalize job ${finalizeJobId} completed (figures and summary uploaded).`,
+    };
+}
+/** Re-fetch GET /v1/jobs/:id after finalize so callers see cleared finalize_job_id. */
+export async function refreshJobAfterFinalize(jobId) {
+    return (await apiCall("GET", `/v1/jobs/${jobId}`));
+}

package/dist/train_submit_message.js

@@ -0,0 +1,52 @@
+import { getVizPort } from "./shared.js";
+export function buildTrainSubmitExtras(opts) {
+    const { newJobId, totalRows, hasTransforms, prepareJobId, variantPrefix, paramSummary, impute, resultsHint } = opts;
+    const monitorUrl = getVizPort() > 0
+        ? `http://localhost:${getVizPort()}/viz/training-monitor?mode=standalone&job_id=${encodeURIComponent(newJobId)}`
+        : null;
+    let msg = `Job submitted (${variantPrefix}, ${paramSummary}). `;
+    if (prepareJobId) {
+        msg += `Preprocessing job prepare_training_matrix (${prepareJobId}) runs on worker-io first; train job ${newJobId} starts after it completes. Poll jobs(action=status, job_id="${prepareJobId}") until completed, then poll the train job. `;
+    }
+    msg += `Recommended: training_monitor(job_id="${newJobId}")`;
+    if (monitorUrl)
+        msg += ` or open ${monitorUrl}`;
+    msg += ` while polling jobs(action=status, job_id="${newJobId}") every 60–120s for large jobs. `;
+    msg += `When status is completed, check jobs(action=status) for finalize_job_id — if present, poll that finalize_training job before results(action=get). `;
+    msg += `Then use results(action=get, job_id="${newJobId}")`;
+    if (resultsHint) {
+        msg += ` to ${resultsHint}`;
+    }
+    else if (impute) {
+        msg += ` for the map and imputed.csv`;
+    }
+    else {
+        msg += ` to view the map and metrics`;
+    }
+    msg += `.`;
+    if (totalRows >= 1_000_000 || hasTransforms) {
+        msg += ` Large-table note: progress may stay low during staging preprocess before epochs start.`;
+        if (hasTransforms && totalRows >= 1_000_000) {
+            if (prepareJobId) {
+                msg += ` Transforms run in prepare_training_matrix on worker-io (full dataset, no CSV re-download on GPU).`;
+            }
+            else {
+                msg += ` Transforms require prepare_training_matrix on worker-io for staged datasets; the API enqueues it automatically when needed.`;
+            }
+        }
+    }
+    const pollSteps = prepareJobId
+        ? [
+            `1) jobs(action=status, job_id="${prepareJobId}") until prepare completes`,
+            `2) jobs(action=status, job_id="${newJobId}") every 60–120s until completed`,
+            `3) if finalize_job_id is set, poll it until completed`,
+            `4) results(action=download, job_id="${newJobId}", folder=".", include_json=true)`,
+        ]
+        : [
+            `1) training_monitor(job_id="${newJobId}")`,
+            `2) jobs(action=status, job_id="${newJobId}") every 60–120s until completed`,
+            `3) if finalize_job_id is set, poll it until completed`,
+            `4) results(action=download, job_id="${newJobId}", folder=".", include_json=true)`,
+        ];
+    return { message: msg, suggested_next_step: pollSteps.join(" → ") };
+}