runbios-mcp 0.2.21 → 0.2.22-dev.310

package/dist/server.js

@@ -829,7 +829,7 @@ pipeline is deleted. Every other source is switched on with loop_set_capture.
 
 1. loop_set_capture / loop_list_capture_settings - Turn recording on or off for a source, choose how long conversations are kept, and the tags stamped on everything it records (a tag is how live traffic reaches a pipeline). A pipeline's own tag is already on; any other source records nothing until this is done.
 2. loop_capture_trace / loop_import_rows - Record one exchange as it happens (what the model was asked, what it answered, any tool calls), whoever served the model — Run BiOS, another provider, or your own servers. Send it to a pipeline's own_tag as the source, add labels to feed other pipelines too, and give feedback on the trace_id it returns. loop_import_rows brings in data that already exists instead: an export, a spreadsheet of past answers, preference pairs. Imported rows arrive as conversations NEEDING REVIEW, never as finished training data.
-3. loop_list_traces / loop_get_trace / loop_delete_trace - Read what has been recorded; pipeline=<id> lists one pipeline's samples. Credentials and personal details are stripped before storage, never after.
+3. loop_list_traces / loop_get_trace / loop_delete_trace - Read what has been recorded; pipeline=<id> lists one pipeline's samples. Deleting a conversation a pipeline's live version learned from rebuilds that pipeline's model from the base without it (the answer's rebuilds names each). Credentials and personal details are stripped before storage, never after.
 4. loop_add_signal / loop_list_signals - Feedback: accepted (good), rejected (bad), or edited with the corrected answer. Append-only. A correction is the most valuable feedback there is.
 5. loop_label_trace / loop_remove_label / loop_list_labels / loop_get_trace_pipelines - Tags. A pipeline trains on every conversation carrying any of its tags, and a conversation can carry several and feed several pipelines; loop_get_trace_pipelines says which pipelines one conversation feeds.
 6. loop_stats / loop_get_metrics_history - How much is recorded and reviewed, and how the attempts and the feedback moved over time.
@@ -844,7 +844,7 @@ pipeline is deleted. Every other source is switched on with loop_set_capture.
 15. loop_promote_training_run / loop_reject_training_run / loop_rollback_training_run / loop_cancel_training_run - The decision. Promotion makes an attempt the live version and changes what real customers get; it is undoable for 30 days.
 16. set_inference_alias / list_inference_aliases / delete_inference_alias - The re-pointable public handles a promotion writes: a name customers call that can be moved to different weights without them changing anything.
 
-END TO END: capture what a model was asked and answered (or import it), say whether each answer was good, bad or corrected, and tag it; create ONE pipeline per use case. When enough new samples carry its tags to meet the platform's fixed minimum (loop_list_models floors; the pipeline's minimums say how many are still needed) and its schedule is due, if it has one, the platform trains an attempt, compares it with the live version on held-back samples it never trains on and on the pipeline's benchmarks, and promotes it only when it is better by the promotion policy — so each version beats the one before it. Everything before the pipeline costs nothing; saving a pipeline is where money starts, which is why it is spend consent. Two fields carry the state a user actually asks about and BOTH must be read back in plain words rather than summarised as 'fine': status says whether the platform is working or the user has to act, and next_reason says what a quiet pipeline is waiting for, because a pipeline waiting for samples looks identical to a broken one.`;
+END TO END: capture what a model was asked and answered (or import it), say whether each answer was good, bad or corrected, and tag it; create ONE pipeline per use case. When enough new samples carry its tags to meet the platform's fixed minimum (loop_list_models floors; the pipeline's minimums say how many are still needed) and its schedule is due, if it has one, the platform trains an attempt, compares it with the live version on held-back samples it never trains on and on the pipeline's benchmarks, and promotes it by the promotion policy: when it is better, or when a member approves it. A rebuild after a member deleted a conversation the live version learned goes live unless it is clearly worse, under every policy but manual, where it waits for a member like any attempt; so a version need not have beaten the one before it. Everything before the pipeline costs nothing; saving a pipeline is where money starts, which is why it is spend consent. Two fields carry the state a user actually asks about and BOTH must be read back in plain words rather than summarised as 'fine': status says whether the platform is working or the user has to act, and next_reason says what a quiet pipeline is waiting for, because a pipeline waiting for samples looks identical to a broken one.`;
     const server = {
         tool(name, description, paramsSchema, cb) {
             if (allowedTools && !allowedTools.has(name))
@@ -1034,7 +1034,7 @@ Run BiOS provides serverless inference, dedicated GPU deployments, datasets, and
 5. **Configure hyperparameters** — learning rate, batch size, epochs, LoRA rank, etc. Smart defaults provided.
 6. **Choose a GPU** — A100 80GB, H100, A6000 etc. Platform recommends based on model size.
 7. **Launch and monitor** — create_training_job only after preflight and explicit spend approval. Poll get_training_status; read metrics, evals and logs instead of inferring progress from elapsed time. Stop/resume only when available actions permit it.
-8. **Select a checkpoint** — get_training_checkpoints; use a verified/succeeded checkpoint. A running job or incomplete checkpoint is not deployable.
+8. **Select a checkpoint** — get_training_checkpoints; use a verified/succeeded checkpoint. A running job or incomplete checkpoint is not deployable. For a job trained with start_from, deploy its checkpoint whose format is merged: its own adapters are refused with 400 CHAINED_ADAPTER_NOT_DEPLOYABLE.
 9. **Deploy the trained model** — preflight_inference with source_type=checkpoint, source_job_id and source_checkpoint_id, approve GPU/cost, then create_inference. Serving mode and base-model lineage are derived; do not guess them.
 10. **Wait, infer, observe** — poll get_inference_status until running, call chat_with_inference, inspect metrics/notifications, then stop or delete when explicitly requested.
 11. **Clean up safely** — delete_training_checkpoint or delete_dataset only after confirming it is no longer needed. Deleting source data/checkpoints is irreversible and is never implied by stopping training or deleting a deployment.
@@ -1328,7 +1328,7 @@ Capacity errors include bookable alternatives. Never silently substitute a GPU o
 2. Choose a catalog base model; call get_training_capabilities, get_model_config, get_recommended_gpu/recommend_training_config, then preflight_training_job.
 3. State method, adapter, GPU/count, dataset sampling, maximum hourly price and estimated bound; obtain explicit spend approval.
 4. create_training_job. Poll get_training_status and read metrics, evals and logs. Do not translate elapsed time into progress or success.
-5. After success, call get_training_checkpoints and select a verified checkpoint.
+5. After success, call get_training_checkpoints and select a verified checkpoint. For a job trained with start_from, select its checkpoint whose format is merged: its own adapters are refused with 400 CHAINED_ADAPTER_NOT_DEPLOYABLE.
 6. Call preflight_inference with source_type=checkpoint, source_job_id and source_checkpoint_id. Serving mode/base lineage are derived from the checkpoint; do not invent them.
 7. Obtain deployment spend approval, create_inference, poll until running, then chat_with_inference.
 8. Observe metrics/notifications and clean up only on explicit request. Deleting the deployment does not delete the checkpoint or dataset; each destructive operation is separate and irreversible.`,
@@ -1674,15 +1674,19 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         });
         return json(withNextOffset(data, "models", pageSize, skip));
     });
-    server.tool("list_inference_models", "List models this credential can invoke through /v1/models: serverless pool models with serverless scope and this workspace's dedicated deployments with deployments:read or deployments:write. An unreachable source is named in usf_unreachable_sources; never infer it has no models. list_models is the separate trainable-model registry.", {
-        query: z.string().min(1).max(200).optional().describe("Case-insensitive match on the model id."),
+    server.tool("list_inference_models", "List models this credential can invoke through /v1/models: serverless pool models with serverless scope and this workspace's dedicated deployments with deployments:read or deployments:write. Serverless ids are maker/name (anthropic/claude-opus-5) and each carries maker {id, name}: who made it, never who serves it; prefer same-maker models when suggesting an alternative. An unreachable source is named in usf_unreachable_sources; never infer it has no models. list_models is the separate trainable-model registry.", {
+        query: z.string().min(1).max(200).optional().describe("Case-insensitive match on the model id, display name, or maker (e.g. anthropic, Moonshot AI)."),
         limit: z.number().int().min(1).max(200).optional().describe("Models per page (default 50, maximum 200)."),
         offset: z.number().int().min(0).optional().describe("Models to skip; use next_offset from the previous page."),
     }, async ({ query, limit, offset }) => {
         const roster = safeModelDiscoveryPayload(await client.inferenceModels());
         const all = Array.isArray(roster.data) ? roster.data : [];
         const needle = query?.toLowerCase();
-        const matching = needle ? all.filter((row) => String(row.id ?? "").toLowerCase().includes(needle)) : all;
+        const haystack = (row) => {
+            const maker = (row.maker ?? {});
+            return [row.id, row.display_name, maker.id, maker.name].map((v) => String(v ?? "")).join(" ").toLowerCase();
+        };
+        const matching = needle ? all.filter((row) => haystack(row).includes(needle)) : all;
         const pageSize = limit ?? 50;
         const skip = offset ?? 0;
         const data = matching.slice(skip, skip + pageSize);
@@ -2723,7 +2727,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
     /* ══════════════════════════════════════════════════════════════════════════ */
     /*  TOOL: get_training_checkpoints                                          */
     /* ══════════════════════════════════════════════════════════════════════════ */
-    server.tool("get_training_checkpoints", "List recorded checkpoints for a training job. Returns checkpoint identity, step/epoch, size and final/best flags. Use get_checkpoint_download_manifest for file URLs; this list does not fabricate loss, eval metrics, or download URLs.", {
+    server.tool("get_training_checkpoints", "List recorded checkpoints for a training job. Returns checkpoint identity, step/epoch, size, final/best flags and format: adapter (LoRA weights), full (a full model) or merged (the standalone model a job trained with start_from exported: the base with every earlier version and its own adapter merged into one; its step is the adapter step it was merged from). A start_from job's own adapters cannot be deployed: preflight_inference and create_inference answer 400 CHAINED_ADAPTER_NOT_DEPLOYABLE, so deploy its merged checkpoint instead. Use get_checkpoint_download_manifest for file URLs; this list does not fabricate loss, eval metrics, or download URLs.", {
         job_id: z.string().describe("Training job ID"),
     }, async ({ job_id }) => {
         validateId(job_id, "job_id");
@@ -3018,7 +3022,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
     });
     server.tool("loop_list_traces", "List recorded conversations (samples) for this workspace, newest first. To read a pipeline's data by feedback, pass pipeline with signal ('good', 'bad' or 'corrected': the verdict that decides the sample) or unsignalled true (what still needs review). A bad sample is never trained on. from/to narrow by when a conversation was recorded. A conversation kept in the workspace's own storage carries payload_stored_externally; when its text could not be read just now it also carries payload_unavailable, with messages null and completion empty. That conversation is NOT empty: say its text could not be read right now, do not review it, and read it again later.", {
         source: z.string().optional().describe("Only conversations from this source"),
-        model: z.string().optional().describe("Only what THIS model answered, e.g. 'kimi-k3'. The filter distillation is made of — every other filter selects by what was asked or what somebody said about the answer, never by who produced it."),
+        model: z.string().optional().describe("Only what THIS model answered, e.g. 'moonshotai/kimi-k3'. The filter distillation is made of — every other filter selects by what was asked or what somebody said about the answer, never by who produced it."),
         origin: z.enum(["captured", "imported"]).optional().describe("captured = the workspace's own model behaviour; imported = a file somebody brought in. Pooling them makes 'train only on what we actually served' unanswerable."),
         conversation_id: z.string().optional(),
         pipeline: z.string().optional().describe("Only the conversations this pipeline's tags select (its id, from loop_list_pipelines): its samples, reviewed or not."),
@@ -3061,7 +3065,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         const data = await client.api(`/api/loop/traces/${encodeURIComponent(trace_id)}`);
         return json(data);
     });
-    server.tool("loop_delete_trace", "Delete one recorded conversation, with its feedback and tags, for good. It stops counting toward every pipeline that selected it. A version already trained on it keeps what it learned. Confirm with the user first: this cannot be undone.", { trace_id: z.string() }, async ({ trace_id }) => json(await client.api(`/api/loop/traces/${encodeURIComponent(trace_id)}`, { method: "DELETE" })));
+    server.tool("loop_delete_trace", "Delete one recorded conversation, with its feedback and tags, for good. It stops counting toward every pipeline that selected it. Where the live version of a pipeline learned from it, that pipeline rebuilds its model from the base on every remaining sample, without it: the answer's rebuilds lists each such pipeline ({pipeline_id, pipeline_name}; empty when none) -- read them back, because a rebuild is a new attempt paid from the wallet. A switched-off pipeline is listed too: its rebuild starts when it is switched back on. Until the rebuild goes live the live version still holds what it learned from it; once it does, no version that learned it can be rolled back to, and their stored weights are deleted. Any other version already trained on it keeps what it learned, and a conversation that expires under the retention setting rebuilds nothing. Confirm with the user first: this cannot be undone.", { trace_id: z.string() }, async ({ trace_id }) => json(await client.api(`/api/loop/traces/${encodeURIComponent(trace_id)}`, { method: "DELETE" })));
     server.tool("loop_add_signal", "Record a verdict on a conversation: was the answer right? Feedback is APPEND-ONLY — a second verdict does not replace the first, because two reviewers disagreeing is information worth keeping.\n\nWHAT EACH VERDICT DOES (every pipeline fine-tunes on examples):\n• accepted (a thumbs up) — the answer was good. It becomes an example as it stands.\n• rejected (a thumbs down) — it was wrong and you have nothing better. It is LEFT OUT of training and nothing is learned from it; prefer edited whenever you know the better answer.\n• edited — the model was WRONG and here is what it should have said. The most valuable single action: your answer is trained on in place of the model's. Nothing is learned from the answer it replaced.\n• gold — the reference answer for this question, recorded whatever the model happened to say. It trains exactly like edited; it only makes no claim that the model was wrong.", {
         trace_id: z.string(),
         verdict: z.enum(["accepted", "rejected", "edited", "gold"]).describe("accepted = thumbs up; rejected = thumbs down with nothing better to offer; edited = the model was wrong and you wrote the better answer; gold = the reference answer for this question regardless of what the model said"),
@@ -3155,7 +3159,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
             }).nullable().optional().describe("Also wait for this time. Training starts only when BOTH the platform's fixed sample minimum and the schedule are met. null trains as soon as the minimum is met."),
         }).strict().optional().describe("When to train, beyond the platform's fixed sample minimum: only a schedule. There is no min_samples: the minimums are fixed (loop_list_models floors) and cannot be set."),
         promotion: z.object({
-            policy: z.enum(["holdout", "all", "primary", "k_of_n", "weighted", "manual"]).describe("holdout (default): better on the held-back samples. all / primary / k_of_n / weighted: by the pipeline's benchmarks. manual: never promoted without a person."),
+            policy: z.enum(["holdout", "all", "primary", "k_of_n", "weighted", "manual"]).describe("holdout (default): better on the held-back samples. all / primary / k_of_n / weighted: by the pipeline's benchmarks. manual: never promoted without a person, a rebuild after a deleted conversation included. Under every other policy a rebuild after a deleted conversation goes live unless it is clearly worse than the live version."),
             k: z.number().int().min(1).optional().describe("For k_of_n only: how many measurements must improve"),
         }).optional(),
         // STRICT, and never null: the share is the member's, the floor of 100 and
@@ -3167,8 +3171,8 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         max_versions: z.number().int().min(1).max(100).nullable().optional().describe("How many versions it may make, 1 to 100; null (the default) is no limit. A version is an attempt that became the live model; attempts that lost are not counted."),
         enabled: z.boolean().optional().describe("false saves it switched off"),
     };
-    server.tool("loop_list_models", "The base models a pipeline can train — the ONLY models loop_create_pipeline accepts — recommended first, each with its name, author, family and size, and floors, the platform's fixed sample minimums (read-only; no pipeline sets them): first_version_samples (what a new pipeline's first attempt needs at the default held-back share, the held-back samples included), new_samples and new_samples_percent (each later attempt waits for new_samples new samples, or new_samples_percent percent of the samples the last trained attempt trained on when that is more), and held_back (at least min_rows samples are held back, or the share a pipeline chooses from default_percent to max_percent if that is more; each attempt is judged on at most scored_rows of them, so a share that holds back more only keeps samples out of training). Offer the user a choice from this list; never type a model id from memory. 503 MODELS_UNAVAILABLE: the list could not be read just then; try again in a minute.", {}, async () => json(await client.api("/api/loop/models")));
-    server.tool("loop_create_pipeline", "Create a pipeline: one use case, trained again whenever enough new samples carrying its tags exist. SPEND CONSENT: saving it authorises the platform to train attempts on the workspace's wallet with nobody present — whenever train_when is met it trains on GPUs the platform chooses, compares the attempt with the live version, and promotes it by the promotion policy. Only the GPU time actually used is charged. Say that in plain words and WAIT FOR AN EXPLICIT YES before calling. Take model.id from loop_list_models, never from memory. Nothing is saved when it is refused. 409 PIPELINE_NAME_TAKEN or OWN_TAG_TAKEN: choose another name. 422 NO_GPU_FITS, MODEL_NOT_TRAINABLE or PROMOTION_*: the message names the fix. 422 NO_JUDGE_MODEL: no listed model can score the attempts; tell the user rather than retrying. 422 MODEL_REVISION_UNAVAILABLE: the model could not be pinned because the training service did not answer; try again shortly. 422 METHOD_NOT_ENABLED or CEILING_TOO_LOW: the platform cannot train this model now and nothing in the request changes that; tell the user. 409 TRAINING_NOT_ENABLED: training is in beta and the organisation is not enabled yet — tell the user the platform team enables it, and do not retry. 503 MODELS_UNAVAILABLE, GPU_OPTIONS_UNAVAILABLE or AGENT_COULD_NOT_START: a service the save needs did not answer just then; try again in a minute.", pipelineInputShape, async (input) => json(await client.api("/api/loop/pipelines", { method: "POST", body: input })));
+    server.tool("loop_list_models", "The base models a pipeline can train — the ONLY models loop_create_pipeline accepts — recommended first, each with its name, author, family and size, and floors, the platform's fixed sample minimums (read-only; no pipeline sets them): first_version_samples (what a new pipeline's first attempt needs at the default held-back share, the held-back samples included), new_samples (each later attempt waits for new_samples new samples, 1,000, counted since the last trained attempt, however many samples that attempt trained on; where a pipeline builds on its live version -- its chain is set -- that attempt trains a new adapter on top of the live version, on what it has not learned yet plus a replay of what it already learned (about half of what it trains on, picked at random, so it is not forgotten), and a winning version is merged in and served as one model), and held_back (at least min_rows samples are held back, or the share a pipeline chooses from default_percent to max_percent if that is more; each attempt is judged on at most scored_rows of them, so a share that holds back more only keeps samples out of training). Offer the user a choice from this list; never type a model id from memory. 503 MODELS_UNAVAILABLE: the list could not be read just then; try again in a minute.", {}, async () => json(await client.api("/api/loop/models")));
+    server.tool("loop_create_pipeline", "Create a pipeline: one use case, trained again whenever enough new samples carrying its tags exist (1,000 new ones for each attempt after the first; where the pipeline builds on its live version, each attempt trains a new adapter on top of it, on what it has not learned plus a random replay of what it learned, about half of what it trains on, and a winner is merged in). SPEND CONSENT: saving it authorises the platform to train attempts on the workspace's wallet with nobody present — whenever train_when is met it trains on GPUs the platform chooses, compares the attempt with the live version, and promotes it by the promotion policy. Only the GPU time actually used is charged. Say that in plain words and WAIT FOR AN EXPLICIT YES before calling. Take model.id from loop_list_models, never from memory. Nothing is saved when it is refused. 409 PIPELINE_NAME_TAKEN or OWN_TAG_TAKEN: choose another name. 422 NO_GPU_FITS, MODEL_NOT_TRAINABLE or PROMOTION_*: the message names the fix. 422 NO_JUDGE_MODEL: no listed model can score the attempts; tell the user rather than retrying. 422 MODEL_REVISION_UNAVAILABLE: the model could not be pinned because the training service did not answer; try again shortly. 422 METHOD_NOT_ENABLED or CEILING_TOO_LOW: the platform cannot train this model now and nothing in the request changes that; tell the user. 409 TRAINING_NOT_ENABLED: training is in beta and the organisation is not enabled yet — tell the user the platform team enables it, and do not retry. 503 MODELS_UNAVAILABLE, GPU_OPTIONS_UNAVAILABLE or AGENT_COULD_NOT_START: a service the save needs did not answer just then; try again in a minute.", pipelineInputShape, async (input) => json(await client.api("/api/loop/pipelines", { method: "POST", body: input })));
     server.tool("loop_update_pipeline", "Edit a pipeline: its name, use case, model, challenger, tags, when to train, held-back share, promotion policy, version limit, or switch it off and on. Send ONLY what changes: an absent key leaves it alone, and null clears max_versions (no limit), challenger_model (no challenger) or train_when.schedule (train as soon as the minimum is met). Saving is the authorization, as on a create: a change that makes attempts cost more (a bigger model, switching it back on) is SPEND CONSENT, so say so first. A larger held-back share is not SPEND CONSENT (nothing is asked again and the limits do not move), and what it changes depends on how many are held back: while fewer than floors.held_back.scored_rows (1,500) are, each attempt can score more of them, paid from the wallet; once that many are, a larger share scores nothing more and costs nothing more, and only keeps more new samples out of training, for good; and a share that still comes to the floor of floors.held_back.min_rows (100) changes nothing. Say whichever applies, never that a raise always costs more. Pass expected_revision (the pipeline's revision as you read it) to be refused REVISION_MISMATCH rather than overwrite someone else's edit; REVISION_MISMATCH also comes back, sent or not, when the pipeline changed while the edit was being saved (a version went live, which can change its models): nothing was saved, so read it again and redo the edit. A model change while an attempt is running is refused RUN_ACTIVE. 409 NOT_A_SIMPLE_PIPELINE: this pipeline was created before pipelines had tags, a use case or a challenger, so those three cannot be edited; everything else can. Every other refusal is loop_create_pipeline's, with the same meaning.", {
         pipeline_id: z.string().describe("The pipeline's id (rule_id), from loop_list_pipelines"),
         ...pipelineInputShape,
@@ -3178,7 +3182,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         body,
     })));
     server.tool("loop_delete_pipeline", "Delete a pipeline for good. An attempt still running is cancelled; the deployment its live version answers from is stopped and deleted, so the live version stops answering and is no longer billed, unless another pipeline or model name of yours answers from it too. If one does, that deployment keeps running and billed until you stop it on Deployments, and its trained checkpoint is kept (and billed as storage) until that deployment is deleted. Its history is deleted in the background after the call returns: the checkpoints its attempts trained, the sets they trained on, and every conversation of their comparison and benchmark reports. Its samples and tags are kept. Recording from its own tag's source stops: an app still sending there gets captured:false. THIS CANNOT BE UNDONE and the name is spent: the response's retained_name can never be used by another pipeline in this workspace. To stop it for now, loop_update_pipeline with enabled false. Confirm with the user first, and say that the live version stops answering and the history is deleted.", { pipeline_id: z.string() }, async ({ pipeline_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}`, { method: "DELETE" })));
-    server.tool("loop_run_pipeline", "Train now: start an attempt without waiting for the schedule. SPEND CONSENT: it trains on real GPUs charged to the wallet, so say so and get a yes. It never skips the minimum: 422 NOT_ENOUGH_SAMPLES carries have and need — read both back. 409 RUN_ACTIVE: an attempt is already running. An attempt waiting for GPUs to compare (active_run.state waiting_gpus) is the one exception: Train now books its two comparison machines again at once and answers 202; nothing is trained again. 409 VERSION_LIMIT_REACHED: raise or remove max_versions. 409 RULE_PAUSED: it is switched off or paused (paused_reason says why). 409 PIPELINE_NEEDS_SAVE: its settings changed without a pipeline save; save it with loop_update_pipeline, then train again. 409 MONTHLY_LIMIT_REACHED: the platform's runaway guard for this month; read resumes_at back. 409 SCORING_CAPPED: the comparison's key is at its monthly cap, so what it trained could not be scored; read resumes_at back (the pipeline's scoring_capped_until says the same before you try). 409 LIVE_VERSION_STOPPED: the deployment the live version answers from is stopped, paused or deleted, so an attempt could not be compared against it; the user resumes it in Deployments or rolls the version back. 503 AGENT_COULD_NOT_START or 409 AGENT_OFF: the key every attempt is scored with could not be renewed just then, so nothing was started; train now again in a minute (saving the pipeline does not help). Do not retry any of the others unchanged.", { pipeline_id: z.string() }, async ({ pipeline_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/run`, {
+    server.tool("loop_run_pipeline", "Train now: start an attempt without waiting for the schedule. SPEND CONSENT: it trains on real GPUs charged to the wallet, so say so and get a yes. It never skips the minimum: 422 NOT_ENOUGH_SAMPLES carries have and need — read both back. 409 RUN_ACTIVE: an attempt is already running. An attempt waiting for GPUs to compare (active_run.state waiting_gpus) is the one exception: Train now books its two comparison machines again at once and answers 202; nothing is trained again. With a rebuild on its way (chain.rebuild: a member deleted a conversation the live version learned from), Train now starts the rebuild: a retrain from the base on every remaining sample, with no wait for new ones, at the version limit too (each rebuild that goes live is a version, so the pipeline can end past its limit). 409 VERSION_LIMIT_REACHED: raise or remove max_versions. 409 RULE_PAUSED: it is switched off or paused (paused_reason says why). 409 PIPELINE_NEEDS_SAVE: its settings changed without a pipeline save; save it with loop_update_pipeline, then train again. 409 MONTHLY_LIMIT_REACHED: the platform's runaway guard for this month; read resumes_at back. 409 SCORING_CAPPED: the comparison's key is at its monthly cap, so what it trained could not be scored; read resumes_at back (the pipeline's scoring_capped_until says the same before you try). 409 LIVE_VERSION_STOPPED: the deployment the live version answers from is stopped, paused or deleted, so an attempt could not be compared against it; the user resumes it in Deployments or rolls the version back. 503 AGENT_COULD_NOT_START or 409 AGENT_OFF: the key every attempt is scored with could not be renewed just then, so nothing was started; train now again in a minute (saving the pipeline does not help). Do not retry any of the others unchanged.", { pipeline_id: z.string() }, async ({ pipeline_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/run`, {
         method: "POST",
         body: {},
     })));
@@ -3205,11 +3209,11 @@ create (1-5 choices), and the queue itself stays opt-in.`,
     }, async ({ rule_id, state, limit, offset }) => json(await client.api("/api/loop/training-runs", {
         params: { rule_id, state, limit: limit?.toString(), offset: offset?.toString() },
     })));
-    server.tool("loop_get_training_run", "One run in full: the base model it trained (base_model_id, model_family) and the recipe, the rows it trained on and held back, its GPU time and what it was billed (billed_candidate_up_to true means billed_candidate_cents is 'up to': a comparison machine was charged in it, counted from its first boot while the platform bills from its first answer, and at the hourly cap when no price was reported -- say 'up to', never 'spent'), the timeline of every state it passed through and why, links to the job, the candidate and the comparison, benchmark_runs (every benchmark it was replayed on; each run_id is the replay's id for loop_get_benchmark_run and loop_list_benchmark_run_items), promotion (outcome as the pipeline's attempts list says it; decision, where rejected means a member chose not to publish it even when the verdict says better, auto_rejected that the pipeline's rule turned it down on its own, and expired that nobody decided before the review deadline -- an expired attempt whose verdict is better was better, never say the rule refused it; and the promotion rule's reason and every measurement it read, each better, not_better, worse or not_measured with its reason -- null and empty when the rule recorded nothing), verdict_checks (every bar the held-back comparison had to clear -- conversations compared, failures, a rubric set, mean change, win rate, and the chance a model no better would win this often -- each with needed, got and met; null until the comparison is finished, when no thresholds were recorded, and for a report decided under an earlier rule that these bars would not explain: when the user asks why an attempt did not go live, read the bars that are not met back in these words), verdict_checks_withheld (decided_under_earlier_rule when a finished report's bars are withheld because an earlier version of the rule decided it, null otherwise: then say the comparison was decided under an earlier version of the rule, so its bars are not listed -- its verdict stands as recorded; never say they were not recorded, and never work them out from today's rule), cost (training_cents, comparison_cents, scoring_cents, benchmark_cents and total_cents, the attempt's cost_cents on the pipeline; up_to true means comparison_cents and the total are 'up to', the most the comparison machines can have cost), and available_actions — which says what THIS reader may actually do next. Use available_actions rather than guessing from the state: a run can look promotable and not be, because the consent moved or the candidate is already gone.", { run_id: z.string() }, async ({ run_id }) => json(await client.api(`/api/loop/training-runs/${encodeURIComponent(run_id)}`)));
+    server.tool("loop_get_training_run", "One run in full: the base model it trained (base_model_id, model_family) and the recipe, how the platform trained it (training_summary, the service's own sentence -- the steps it took, the passes planned over its samples and the checkpoint it kept; read it back as written, and trained_steps is the steps the trainer counted, never an estimate; both null until its job ended, and on an attempt from before), the rows it trained on and held back (lineage says what it was built on: trained_from versions -- a new adapter on top of built_on's versions merged into the base -- or base, with from_base_reason; and train_rows = new_samples + carried_samples, from attempts that did not go live, + replayed_samples, drawn from what those versions learned (the counts are null on a run from before the chain, or of a pipeline that does not build on its live version: say only train_rows then); error_code CHAIN_MOVED means it was built on a version that was rolled back while it trained, so it was set aside with no comparison paid and its samples go to the next attempt, and NO_MERGED_EXPORT that its training finished without the merged model it is served as), its GPU time and what it was billed (billed_candidate_up_to true means billed_candidate_cents is 'up to': a comparison machine was charged in it, counted from its first boot while the platform bills from its first answer, and at the hourly cap when no price was reported -- say 'up to', never 'spent'), the timeline of every state it passed through and why, links to the job, the candidate and the comparison, benchmark_runs (every benchmark it was replayed on; each run_id is the replay's id for loop_get_benchmark_run and loop_list_benchmark_run_items), promotion (outcome as the pipeline's attempts list says it; decision, where rejected means a member chose not to publish it even when the verdict says better, auto_rejected that the pipeline's rule turned it down on its own, and expired that nobody decided before the review deadline -- an expired attempt whose verdict is better was better, never say the rule refused it; and the promotion rule's reason and every measurement it read, each better, not_better, worse or not_measured with its reason -- null and empty when the rule recorded nothing), verdict_checks (every bar the held-back comparison had to clear -- conversations compared, failures, a rubric set, mean change, win rate, and the chance a model no better would win this often -- each with needed, got and met; null until the comparison is finished, when no thresholds were recorded, and for a report decided under an earlier rule that these bars would not explain: when the user asks why an attempt did not go live, read the bars that are not met back in these words), verdict_checks_withheld (decided_under_earlier_rule when a finished report's bars are withheld because an earlier version of the rule decided it, null otherwise: then say the comparison was decided under an earlier version of the rule, so its bars are not listed -- its verdict stands as recorded; never say they were not recorded, and never work them out from today's rule), cost (training_cents, comparison_cents, scoring_cents, benchmark_cents and total_cents, the attempt's cost_cents on the pipeline; up_to true means comparison_cents and the total are 'up to', the most the comparison machines can have cost), and available_actions — which says what THIS reader may actually do next. Use available_actions rather than guessing from the state: a run can look promotable and not be, because the consent moved or the candidate is already gone.", { run_id: z.string() }, async ({ run_id }) => json(await client.api(`/api/loop/training-runs/${encodeURIComponent(run_id)}`)));
     server.tool("loop_list_pipelines", "Every pipeline in the workspace: the series of versions it has produced (versions are attempts that were put live; attempts[] lists every attempt, winner or not, each with its outcome, trigger and decision -- rejected means a member chose not to publish it, where outcome says stopped, and expired that nobody decided before the review deadline; auto_rejected that the pipeline's rule turned it down on its own; an expired attempt whose verdict is better was better: never say the rule refused it -- and train_rows and holdout_rows, the samples it trained on and held back; each attempt is judged on at most scored_rows (1,500) of the held-back samples, the same ones every attempt), which one serves the app today (live_version, the same as champion_version, null when what serves is not one of its versions) and whether its deployment still answers (live_deployment.status: serving answers; starting is on its way up; stopped, paused_for_funds or deleted answers nothing and is not billed; paused_by_platform was paused because billing could not be reached and answers nothing; failed answers nothing; live_deployment null means no check is recorded yet or what serves is not a deployment -- never say a live version with any status but serving is answering the app), how each version did against the champion of its day and on the standing benchmark, versions_made against max_versions, the run in flight and which version it will become, and status. Read-only. When the user asks 'is this getting better' or 'what is it doing', answer from this, and read status back honestly: needs_review and needs_funds are waiting on THEM (a decision, a top-up), paused names why in paused_reason (consent_invalid means its settings changed since it was last saved: saving it again starts it), complete means the version limit is reached, and running is the platform working -- including an attempt whose active_run.state is waiting_gpus: it trained, but no GPUs were free to run the new model and the one it is compared with side by side, so both machines were stopped; each try can bill the machine that got a GPU for up to 20 minutes while it waits for the other, nothing bills between tries, it keeps what it trained, tries again after 30 minutes, then 1, 2 and 4 hours, and ends after four tries or 24 hours (its last_reason says when; a last_reason opening 'waiting for room in this workspace to compare' means the workspace is at its limit of running deployments, and stopping a deployment lets it run), and Train now tries again at once. cost_up_to true on an attempt, active_run or a version means its cost_cents is 'up to', the most it can have cost: a comparison machine was charged in it, counted from its first boot while the platform bills it from its first answer (and at the hourly cap when no price was reported), so say 'up to', never 'spent'. next_reason is the pipeline's own sentence about what it is waiting for. scoring_capped_until, when set, is when the workspace's scoring key comes off its monthly cap: until then nothing trains and Train now answers SCORING_CAPPED, so say so instead of offering Train now. month_spent_cents against monthly_ceiling_cents is what decides whether the next run may start; month_spent_cents is what runs started this UTC month have cost at most (a run still in progress counts at its ceilings), so say 'up to', never 'spent'. A pipeline trains only on new samples: it never retrains the conversations an attempt already had with a recipe changed. The list is a page of 100, newest first: total is how many pipelines the workspace has and truncated is true when more exist after this page. When truncated, say it is the newest of total, and pass offset (this page's offset plus its length) for the next page.", { offset: z.number().int().min(0).optional().describe("How many of the newest pipelines to skip") }, async ({ offset }) => json(await client.api("/api/loop/pipelines", {
         params: { offset: offset ? offset.toString() : undefined },
     })));
-    server.tool("loop_get_pipeline", "One pipeline in full, by its id: every version with its verdict, the decision a person made, its win rate, its pass rates on the held-back samples (candidate_pass_rate and incumbent_pass_rate: the share each side passed, score 0.70 or more) and benchmark score, what it cost and why it was made (trigger, in the same words as attempts[]), plus the run in flight, and what this month's runs have cost at most (month_spent_cents, an upper bound, not an exact spend) against the monthly limit. Read-only: promote, reject and roll back stay on the run tools, and its settings on loop_update_pipeline. minimums says how many samples it has against what the next attempt needs. scoring_capped_until, when set, is when the workspace's scoring key comes off its monthly cap: until then nothing trains and Train now answers SCORING_CAPPED with that time as resumes_at. Only compare benchmark scores whose benchmark_comparable is true; the others were measured on a different yardstick. versions[] are WINNERS ONLY -- attempts that were put live, numbered v1, v2 ... in the order they went live, each with its attempt, base_model_id and model_family -- and live_version is the one the app's serving name reaches now, with live_deployment.status saying whether its deployment answers (serving answers; starting is on its way up; stopped, paused_for_funds or deleted answers nothing and is not billed; paused_by_platform was paused because billing could not be reached and answers nothing; failed answers nothing; null when no check is recorded yet or what serves is not a deployment); attempts[] is every training run the pipeline made, winner or not, with its outcome (became_version, not_better, inconclusive, stopped, failed, running, waiting_for_review), trigger (data, schedule, manual, or challenger for the second model trained on the same data), decision (what was decided about it once compared, and by whom: promoted, auto_promoted, rejected, auto_rejected, expired or rolled_back; null while nothing has been -- rejected means a member chose not to publish it, where outcome says stopped, so say a member declined it rather than that it stopped; promoted that a member put it live and auto_promoted that the pipeline's rule did; auto_rejected that the pipeline's rule turned it down on its own; expired that nobody decided before the review deadline, so it was retired undecided -- an expired attempt whose verdict is better reads stopped and was better: never say the rule refused it), and train_rows and holdout_rows (the samples its set trained on and held back; null until its set was built). Nothing in flight is a version yet. holdout is the pipeline's held-back share: percent (5 to 20), min_rows (at least that many samples are held back whatever the share), rows (how many it holds back today) and held_before (how many of rows an earlier run of the pipeline held back: they stay held back whatever became of that run and whatever the share says, so a lower share releases none of them; before the first version the first attempt needs held_before plus 1,000 to train on when that is more than the share asks for). Each attempt is judged on at most scored_rows (1,500) of the held-back samples, the same ones every attempt, so rows above that only keeps samples out of training. storage is what the pipeline's history takes up in storage (bytes, objects, and where: platform, your_storage or both): every conversation, answer and scoring reason of its reports, and the training rows its attempts are built from, is kept there and never in the database; it is not billed, counts toward the workspace's storage quota and is deleted with the pipeline. An active_run whose last_reason says it is waiting for storage is holding answers already paid for until storage can keep them; they are kept then without asking the models again. cost_up_to true on attempts[], active_run or versions[] means that cost_cents is 'up to', the most it can have cost: a comparison machine was charged in it, counted from its first boot while the platform bills from its first answer (and at the hourly cap when no price was reported), so say 'up to', never 'spent'. An active_run.state of waiting_gpus means the attempt trained and is waiting for GPUs to run the new model and the one it is compared with side by side: each try can bill the machine that got a GPU for up to 20 minutes while it waits for the other, nothing bills between tries, it keeps what it trained, tries again after 30 minutes, then 1, 2 and 4 hours, and ends after four tries or 24 hours (its last_reason says when; a last_reason opening 'waiting for room in this workspace to compare' means the workspace is at its limit of running deployments, and stopping a deployment lets it run), and Train now tries again at once.", { rule_id: z.string().describe("The pipeline's id, from loop_list_pipelines") }, async ({ rule_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(rule_id)}`)));
+    server.tool("loop_get_pipeline", "One pipeline in full, by its id: every version with its verdict, the decision a person made, its win rate, its pass rates on the held-back samples (candidate_pass_rate and incumbent_pass_rate: the share each side passed, score 0.70 or more) and benchmark score, what it cost and why it was made (trigger, in the same words as attempts[]), plus the run in flight, and what this month's runs have cost at most (month_spent_cents, an upper bound, not an exact spend) against the monthly limit. Read-only: promote, reject and roll back stay on the run tools, and its settings on loop_update_pipeline. minimums says how many samples it has against what the next attempt needs. scoring_capped_until, when set, is when the workspace's scoring key comes off its monthly cap: until then nothing trains and Train now answers SCORING_CAPPED with that time as resumes_at. Only compare benchmark scores whose benchmark_comparable is true; the others were measured on a different yardstick. versions[] are WINNERS ONLY -- attempts that were put live, numbered v1, v2 ... in the order they went live, each with its attempt, base_model_id and model_family -- and live_version is the one the app's serving name reaches now, with live_deployment.status saying whether its deployment answers (serving answers; starting is on its way up; stopped, paused_for_funds or deleted answers nothing and is not billed; paused_by_platform was paused because billing could not be reached and answers nothing; failed answers nothing; null when no check is recorded yet or what serves is not a deployment); attempts[] is every training run the pipeline made, winner or not, with its outcome (became_version, not_better, inconclusive, stopped, failed, running, waiting_for_review), trigger (data, schedule, manual, challenger for the second model -- trained from its own base on all samples, never on this pipeline's versions -- or rebuild, a retrain from the base after a member deleted a conversation the live version learned from), decision (what was decided about it once compared, and by whom: promoted, auto_promoted, rejected, auto_rejected, expired or rolled_back; null while nothing has been -- rejected means a member chose not to publish it, where outcome says stopped, so say a member declined it rather than that it stopped; promoted that a member put it live and auto_promoted that the pipeline's rule did; auto_rejected that the pipeline's rule turned it down on its own; expired that nobody decided before the review deadline, so it was retired undecided -- an expired attempt whose verdict is better reads stopped and was better: never say the rule refused it), and train_rows and holdout_rows (the samples its set trained on and held back; null until its set was built). lineage on each version, attempt and the run in flight says what it was built on: trained_from versions (a new adapter trained on top of the live version: the pinned base with built_on's versions merged in, oldest first) or base (with from_base_reason: first_version, challenger, rebuild -- a deleted conversation when lineage.rebuild is true, or, when it is false, a live version already built on 64 versions, the most one attempt can build on -- model_cannot_build_on or chain_link_unreadable), and what it trained on -- train_rows = new_samples + carried_samples (samples of the attempts in carried_from_attempts, which did not go live, trained again) + replayed_samples (about half: as many as new_samples + carried_samples, or all of learned_before when that is fewer, drawn at random from what the versions it builds on already learned, so they are not forgotten); learned_before is what those versions had learned and was not trained again, except the replayed ones; served_as is adapter or merged (one model); learned_deleted_conversation true means a conversation it learned was deleted and it can no longer be rolled back to. The counts are null on an attempt from before the chain, or of a pipeline that does not build on its live version: say only train_rows then. Say it as 'built on v1 + v2, trained on 4,000 samples: 1,000 new, 1,000 from attempt 3 and 2,000 of what v1 + v2 learned, replayed' -- name the replayed part whenever replayed_samples is above 0, so the parts add up to train_rows. chain, when set, is the live version as the versions merged into it (versions, live one last), learned_samples and not_learned_samples (trained by attempts that did not go live, so trained again next), and rebuild when a deleted conversation the live version learned has a retrain from the base on its way; chain null means the pipeline does not build on its live version and every attempt trains from the base on all samples. Nothing in flight is a version yet. holdout is the pipeline's held-back share: percent (5 to 20), min_rows (at least that many samples are held back whatever the share), rows (how many it holds back today) and held_before (how many of rows an earlier run of the pipeline held back: they stay held back whatever became of that run and whatever the share says, so a lower share releases none of them; before the first version the first attempt needs held_before plus 1,000 to train on when that is more than the share asks for). Each attempt is judged on at most scored_rows (1,500) of the held-back samples, the same ones every attempt, so rows above that only keeps samples out of training. storage is what the pipeline's history takes up in storage (bytes, objects, and where: platform, your_storage or both): every conversation, answer and scoring reason of its reports, and the training rows its attempts are built from, is kept there and never in the database; it is not billed, counts toward the workspace's storage quota and is deleted with the pipeline. Where the pipeline builds on its live version, the merged models its versions answer from (the live one's, and each one a rollback can still put back; an attempt that did not go live keeps its merged model, billed, until it is cleaned up within a day) are billed as model storage, like any trained model, and each version's own adapter is billed as checkpoint storage, like any training checkpoint; its reports, benchmark results and training rows are not billed. An active_run whose last_reason says it is waiting for storage is holding answers already paid for until storage can keep them; they are kept then without asking the models again. cost_up_to true on attempts[], active_run or versions[] means that cost_cents is 'up to', the most it can have cost: a comparison machine was charged in it, counted from its first boot while the platform bills from its first answer (and at the hourly cap when no price was reported), so say 'up to', never 'spent'. An active_run.state of waiting_gpus means the attempt trained and is waiting for GPUs to run the new model and the one it is compared with side by side: each try can bill the machine that got a GPU for up to 20 minutes while it waits for the other, nothing bills between tries, it keeps what it trained, tries again after 30 minutes, then 1, 2 and 4 hours, and ends after four tries or 24 hours (its last_reason says when; a last_reason opening 'waiting for room in this workspace to compare' means the workspace is at its limit of running deployments, and stopping a deployment lets it run), and Train now tries again at once.", { rule_id: z.string().describe("The pipeline's id, from loop_list_pipelines") }, async ({ rule_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(rule_id)}`)));
     server.tool("loop_list_pipeline_benchmarks", "The benchmarks a pipeline is measured on, in priority order (1 is the primary), then the ones it stopped using, and its promotion policy. Every attempt is replayed on each attached benchmark; the policy decides which of those measurements make it a version. Each says how it scores an answer (check: judge, a scoring model reading its rubric, or exact, the answer must match its right answer) and what its pinned set takes up in storage (storage: bytes, objects, where), so no benchmark needs reading one by one to say either.", { pipeline_id: z.string() }, async ({ pipeline_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/benchmarks`)));
     server.tool("loop_attach_pipeline_benchmark", "Measure a pipeline on one more benchmark (from loop_list_benchmarks or loop_create_benchmark), last unless priority is given; priority 1 makes it the primary. Applies from the next attempt; older versions are not re-measured and show not_measured for it. 409 BENCHMARK_ALREADY_ATTACHED or BENCHMARK_RETIRED.", {
         pipeline_id: z.string(),
@@ -3240,7 +3244,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
     }, async ({ pipeline_id, a, b }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/versions/compare`, {
         params: { a: a?.toString(), b: b?.toString() },
     })));
-    server.tool("loop_compare_attempts", "Any two attempts of a pipeline side by side, put live or not -- the one to use for 'why was attempt 8 not better than 7' or a challenger against the attempt whose samples it trained on. a and b are RUN ids (attempts[].run_id from loop_get_pipeline), two different attempts of this pipeline. Each side has its outcome, decision and held-back numbers against what it was compared with on its day, with the share each side passed (score 0.70 or more); never subtract two attempts' win rates, their opponents differ. benchmarks lists every benchmark both replayed, with delta (b minus a) only where comparable, why otherwise. shared_sample counts the held-back conversations both were scored on and how many b scored higher, lower or the same on (by the tie band: 0.02 or more above counts as higher; less than 0.02 apart is about the same): these are two absolute scores, not a reading of one answer against the other. shared_sample.a and .b are each attempt's mean and pass rate over those same conversations -- the like-for-like figures, which may be subtracted; each side's holdout numbers are over its own held-back set, which grows between attempts, so do not subtract those. When shared_sample.comparable is false the two were scored differently (another scoring model, holdout.judge_model, or another rubric) or how one was scored was not recorded (an older attempt that froze no rubric is never taken to match): say why in its words, and never read a direction off them. A conversation whose right answer changed between the two attempts has direction null and says why (answer_changed counts them); so does one where it is not known whether both were scored against the same right answer, because one was compared before that was recorded (answer_unknown counts them; never read those as unchanged or as changed). items is one page of them, numbers and ids only: read an answer with loop_list_evaluation_items on that side's holdout.evaluation_id. filter narrows items and total to one direction, and is refused when the two are not comparable.", {
+    server.tool("loop_compare_attempts", "Any two attempts of a pipeline side by side, put live or not -- the one to use for 'why was attempt 8 not better than 7' or a challenger against the attempt whose samples it trained on. a and b are RUN ids (attempts[].run_id from loop_get_pipeline), two different attempts of this pipeline. Each side has its outcome, decision, its lineage (what it was built on and trained on: 'attempt 3 built on v1, trained on 2,000' beside 'attempt 4 built on v1 + v2, trained on 1,000' is often the difference that matters) and held-back numbers against what it was compared with on its day, with the share each side passed (score 0.70 or more); never subtract two attempts' win rates, their opponents differ. benchmarks lists every benchmark both replayed, with delta (b minus a) only where comparable, why otherwise. shared_sample counts the held-back conversations both were scored on and how many b scored higher, lower or the same on (by the tie band: 0.02 or more above counts as higher; less than 0.02 apart is about the same): these are two absolute scores, not a reading of one answer against the other. shared_sample.a and .b are each attempt's mean and pass rate over those same conversations -- the like-for-like figures, which may be subtracted; each side's holdout numbers are over its own held-back set, which grows between attempts, so do not subtract those. When shared_sample.comparable is false the two were scored differently (another scoring model, holdout.judge_model, or another rubric) or how one was scored was not recorded (an older attempt that froze no rubric is never taken to match): say why in its words, and never read a direction off them. A conversation whose right answer changed between the two attempts has direction null and says why (answer_changed counts them); so does one where it is not known whether both were scored against the same right answer, because one was compared before that was recorded (answer_unknown counts them; never read those as unchanged or as changed). items is one page of them, numbers and ids only: read an answer with loop_list_evaluation_items on that side's holdout.evaluation_id. filter narrows items and total to one direction, and is refused when the two are not comparable.", {
         pipeline_id: z.string(),
         a: z.string().describe("The first attempt's run id"),
         b: z.string().describe("The second attempt's run id; directions say b against a"),
@@ -3265,7 +3269,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         method: "POST",
         body: { reason },
     })));
-    server.tool("loop_rollback_training_run", "Undo a promotion: point the serving name back at what answered before it. Available for 30 days, after which the answer is ROLLBACK_WINDOW_CLOSED and the way back is a new run. 409 RUN_ACTIVE: another attempt is being compared with this version right now; roll back once that comparison finishes, or cancel the attempt first. Tell the user what the name will be serving once this lands — the previous target is named on the run.", {
+    server.tool("loop_rollback_training_run", "Undo a promotion: point the serving name back at what answered before it. Available for 30 days, after which the answer is ROLLBACK_WINDOW_CLOSED and the way back is a new run. 409 RUN_ACTIVE: another attempt is being compared with this version right now, or is being built on top of it (its lineage.built_on names this version); roll back once it finishes, or cancel that attempt with loop_cancel_training_run first. 409 ROLLBACK_DELETED_DATA: the version it would put back learned from a conversation a member deleted since, so it never answers again; a rebuild without it replaces it. Where the pipeline builds on its live version, the next attempt builds on the version put back and trains again on what the rolled-back one learned -- when the version put back is on the pipeline's model now; one on the model before a second model took over cannot be built on, so the next attempt trains the pipeline's model from its base on every sample. Tell the user what the name will be serving once this lands — the previous target is named on the run.", {
         run_id: z.string(),
         reason: z.string().optional().describe("Why. It goes on the run's timeline."),
     }, async ({ run_id, reason }) => json(await client.api(`/api/loop/training-runs/${encodeURIComponent(run_id)}/rollback`, {
@@ -3289,7 +3293,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         has_reference: z.boolean().optional().describe("Only pairs with (true) or without (false) a right answer"),
         status: z.enum(["scored", "failed"]).optional(),
         sort: z.enum(["ordinal", "score_change"]).optional().describe("score_change: the attempt's score less the other's, largest first"),
-        q: z.string().min(2).max(200).optional().describe("Search the conversation, the right answer, both answers and every reason (case-insensitive); searched in the answer says how many pairs were read. A search can be refused 429 RATE_LIMITED while this workspace already has a search running or while the service is running as many searches as it can at once, so a 429 does not say which: never tell the user they already had one running; try again after the seconds its message gives"),
+        q: z.string().min(2).max(200).optional().describe("Search the conversation (not the system prompt every row shares), the right answer, both answers and every reason (case-insensitive); searched in the answer says how many pairs were read. A search can be refused 429 RATE_LIMITED while this workspace already has a search running or while the service is running as many searches as it can at once, so a 429 does not say which: never tell the user they already had one running; try again after the seconds its message gives"),
         limit: z.number().int().min(1).max(200).optional().describe("50 when omitted, at most 200"),
         offset: z.number().int().min(0).optional(),
     }, async ({ evaluation_id, winner, candidate_passed, incumbent_passed, orders, has_reference, status, sort, q, limit, offset }) => json(await client.api(`/api/loop/evaluations/${encodeURIComponent(evaluation_id)}/items`, {
@@ -3335,7 +3339,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         method: "POST",
         body: { reason },
     })));
-    server.tool("loop_get_benchmark_run", "One replay in full: both scores, the difference, the per-dimension breakdown, how many conversations each side passed (score 0.70 or more) and the pass rates, the conversations the attempt won, lost and tied (wins, losses, ties), and what it cost. Read status first — a replay that did not score every pinned conversation publishes no score, and its pass counts, rates and wins are null too. loop_list_benchmark_run_items reads every conversation behind it.", { benchmark_run_id: z.string() }, async ({ benchmark_run_id }) => json(await client.api(`/api/loop/benchmark-runs/${encodeURIComponent(benchmark_run_id)}`)));
+    server.tool("loop_get_benchmark_run", "One replay in full: both scores, the difference, the per-dimension breakdown, how many conversations each side passed (score 0.70 or more) and the pass rates, the conversations the attempt won, lost and tied (wins, losses, ties), learned_by_version (how many of these conversations the version under test learned -- trained on itself or learned by a version it was built on; when it is above 0, say \"N of these conversations were learned by this version\", since its score on them is partly recall; null for a version the merge chain did not build: say nothing then), and what it cost. Read status first — a replay that did not score every pinned conversation publishes no score, and its pass counts, rates and wins are null too. loop_list_benchmark_run_items reads every conversation behind it.", { benchmark_run_id: z.string() }, async ({ benchmark_run_id }) => json(await client.api(`/api/loop/benchmark-runs/${encodeURIComponent(benchmark_run_id)}`)));
     server.tool("loop_list_benchmark_run_items", "The conversations behind one replay, in the benchmark's order: the pinned prompt and its reference (answer, kind and the tool_calls the right answer makes; null when none), what the version serving today answered, what the attempt answered, each with the tool_calls it made, what the judge made of each (judge.overall, judge.dimensions and judge.reason; on an exact benchmark overall and dimensions.match are 1 or 0 and the reason says what matched or what differed) and whether each passed (score 0.70 or more), and which side won it (winner compares the two scores by the tie band promotion uses: 0.02 or more above counts as higher, less than 0.02 apart is a tie, and it is null until both are scored). Filter to winner='serving' to read the conversations the attempt LOST (the items themselves say winner 'incumbent'). A replay that stopped before it began has no conversations: read its status_reason back instead.", {
         benchmark_run_id: z.string().describe("The replay's id: from loop_get_benchmark_history, loop_compare_versions, or a run's benchmark_runs"),
         winner: z.enum(["candidate", "serving", "tie"]).optional().describe("Only conversations with this outcome: candidate (the attempt won), serving (the version serving today won) or tie"),
@@ -3344,7 +3348,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
         has_reference: z.boolean().optional().describe("Only conversations with (true) or without (false) a right answer"),
         status: z.enum(["scored", "failed"]).optional(),
         sort: z.enum(["ordinal", "score_change"]).optional().describe("score_change: the attempt's score less the other's, largest first"),
-        q: z.string().min(2).max(200).optional().describe("Search the conversation, the right answer, both answers and every reason (case-insensitive); searched in the answer says how many were read. A search can be refused 429 RATE_LIMITED while this workspace already has a search running or while the service is running as many searches as it can at once, so a 429 does not say which: never tell the user they already had one running; try again after the seconds its message gives"),
+        q: z.string().min(2).max(200).optional().describe("Search the conversation (not the system prompt every row shares), the right answer, both answers and every reason (case-insensitive); searched in the answer says how many were read. A search can be refused 429 RATE_LIMITED while this workspace already has a search running or while the service is running as many searches as it can at once, so a 429 does not say which: never tell the user they already had one running; try again after the seconds its message gives"),
         limit: z.number().int().min(1).max(200).optional().describe("50 when omitted, at most 200"),
         offset: z.number().int().min(0).optional(),
     }, async ({ benchmark_run_id, winner, candidate_passed, incumbent_passed, has_reference, status, sort, q, limit, offset }) => json(await client.api(`/api/loop/benchmark-runs/${encodeURIComponent(benchmark_run_id)}/items`, {