vectorstep-service-mcp 0.1.0__tar.gz

vectorstep_service_mcp-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Alex Dalton
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vectorstep_service_mcp-0.1.0/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: vectorstep-service-mcp
+Version: 0.1.0
+Summary: MCP server exposing VectorStep's authoring, inspection, and analytics surface to MCP clients (Claude Code/Desktop).
+Author-email: Alex Dalton <alex@vectorstep.io>
+License-Expression: MIT
+Project-URL: Homepage, https://vectorstep.io
+Project-URL: Documentation, https://vectorstep.io/docs/integrations/mcp/
+Project-URL: Changelog, https://vectorstep.io/docs/about/status-and-support/
+Keywords: mcp,vectorstep,ai,agents,orchestration,pipelines
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: mcp==1.28.1
+Requires-Dist: httpx>=0.27
+Requires-Dist: pyyaml>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.24; extra == "dev"
+Dynamic: license-file
+
+# VectorStep Service MCP
+
+An [MCP](https://modelcontextprotocol.io) server that exposes [VectorStep](https://github.com/bantex01/VectorStep) — pipeline authoring, run inspection, and analytics — to MCP clients such as Claude Code and Claude Desktop.
+
+## What it is
+
+A separate, standalone repository and process with no import-level dependency on VectorStep — the two are coupled only over HTTP, so each can be developed, versioned, and deployed independently.
+
+It lets an MCP client create and edit pipelines and step-library definitions (with the same validation VectorStep itself uses), inspect runs and steps, answer operational and quality questions, and trigger runs or submit human feedback. Agents are not authored here — that's the job of the companion `VectorStep-Gateway-MCP`; this server may only read agents, to help author pipelines that reference them.
+
+## Quick start
+
+```bash
+python3 -m venv .venv
+.venv/bin/pip install -e .
+```
+
+CI runs this repo's test suite on every push and pull request
+(`.github/workflows/tests.yml`).
+
+Then register it with your MCP client, pointing at a running VectorStep instance:
+
+```bash
+claude mcp add vectorstep-service \
+  --env VECTORSTEP_BASE_URL=http://127.0.0.1:8000 \
+  -- /absolute/path/to/VectorStep-Service-MCP/.venv/bin/python -m vectorstep_service_mcp
+```
+
+Full install, client config (Claude Code, Claude Desktop, MCP Inspector), and tool inventory: [MCP servers](https://vectorstep.io/docs/integrations/mcp/).
+
+## Documentation
+
+Full docs at [vectorstep.io](https://vectorstep.io/docs/):
+
+| Section | Covers |
+|---|---|
+| [MCP servers](https://vectorstep.io/docs/integrations/mcp/) | Install, client config, tool inventory, `explain` tool, write-path design notes for both MCP servers |
+| [Pipelines](https://vectorstep.io/docs/pipelines/schema/) | The pipeline/step schema this server authors against |
+
+## The ecosystem
+
+| Repo | Role |
+|---|---|
+| **VectorStep** | The orchestration service: webhook intake, pipeline runner, trust gating, UI, analytics |
+| **VectorStep-Gateway** | WebSocket gateway that runs agents: LLM providers, MCP tools, the full agentic loop |
+| **VectorStep-Service-MCP** | MCP server exposing pipeline authoring, run inspection and analytics to Claude Code/Desktop |
+| **VectorStep-Gateway-MCP** | MCP server for authoring and inspecting Gateway agents |
+
+## Licence
+
+VectorStep Service MCP is proprietary software, free to download and use under the terms in [`LICENSE`](LICENSE). The source is not publicly distributed and this repository is private.
+
+Bug reports, questions, and feature requests are welcome at **alex@vectorstep.io**. Code contributions are not accepted — see clause 6 of the licence. For a suspected vulnerability, follow [`SECURITY.md`](SECURITY.md) rather than emailing the address above.

vectorstep_service_mcp-0.1.0/README.md

@@ -0,0 +1,53 @@
+# VectorStep Service MCP
+
+An [MCP](https://modelcontextprotocol.io) server that exposes [VectorStep](https://github.com/bantex01/VectorStep) — pipeline authoring, run inspection, and analytics — to MCP clients such as Claude Code and Claude Desktop.
+
+## What it is
+
+A separate, standalone repository and process with no import-level dependency on VectorStep — the two are coupled only over HTTP, so each can be developed, versioned, and deployed independently.
+
+It lets an MCP client create and edit pipelines and step-library definitions (with the same validation VectorStep itself uses), inspect runs and steps, answer operational and quality questions, and trigger runs or submit human feedback. Agents are not authored here — that's the job of the companion `VectorStep-Gateway-MCP`; this server may only read agents, to help author pipelines that reference them.
+
+## Quick start
+
+```bash
+python3 -m venv .venv
+.venv/bin/pip install -e .
+```
+
+CI runs this repo's test suite on every push and pull request
+(`.github/workflows/tests.yml`).
+
+Then register it with your MCP client, pointing at a running VectorStep instance:
+
+```bash
+claude mcp add vectorstep-service \
+  --env VECTORSTEP_BASE_URL=http://127.0.0.1:8000 \
+  -- /absolute/path/to/VectorStep-Service-MCP/.venv/bin/python -m vectorstep_service_mcp
+```
+
+Full install, client config (Claude Code, Claude Desktop, MCP Inspector), and tool inventory: [MCP servers](https://vectorstep.io/docs/integrations/mcp/).
+
+## Documentation
+
+Full docs at [vectorstep.io](https://vectorstep.io/docs/):
+
+| Section | Covers |
+|---|---|
+| [MCP servers](https://vectorstep.io/docs/integrations/mcp/) | Install, client config, tool inventory, `explain` tool, write-path design notes for both MCP servers |
+| [Pipelines](https://vectorstep.io/docs/pipelines/schema/) | The pipeline/step schema this server authors against |
+
+## The ecosystem
+
+| Repo | Role |
+|---|---|
+| **VectorStep** | The orchestration service: webhook intake, pipeline runner, trust gating, UI, analytics |
+| **VectorStep-Gateway** | WebSocket gateway that runs agents: LLM providers, MCP tools, the full agentic loop |
+| **VectorStep-Service-MCP** | MCP server exposing pipeline authoring, run inspection and analytics to Claude Code/Desktop |
+| **VectorStep-Gateway-MCP** | MCP server for authoring and inspecting Gateway agents |
+
+## Licence
+
+VectorStep Service MCP is proprietary software, free to download and use under the terms in [`LICENSE`](LICENSE). The source is not publicly distributed and this repository is private.
+
+Bug reports, questions, and feature requests are welcome at **alex@vectorstep.io**. Code contributions are not accepted — see clause 6 of the licence. For a suspected vulnerability, follow [`SECURITY.md`](SECURITY.md) rather than emailing the address above.

vectorstep_service_mcp-0.1.0/pyproject.toml

@@ -0,0 +1,55 @@
+[project]
+name = "vectorstep-service-mcp"
+version = "0.1.0"
+description = "MCP server exposing VectorStep's authoring, inspection, and analytics surface to MCP clients (Claude Code/Desktop)."
+readme = "README.md"
+requires-python = ">=3.11"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Alex Dalton", email = "alex@vectorstep.io" }]
+keywords = ["mcp", "vectorstep", "ai", "agents", "orchestration", "pipelines"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: System Administrators",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: System :: Monitoring",
+]
+dependencies = [
+    "mcp==1.28.1",
+    "httpx>=0.27",
+    "pyyaml>=6.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "pytest-asyncio>=0.24",
+]
+
+[project.urls]
+Homepage = "https://vectorstep.io"
+Documentation = "https://vectorstep.io/docs/integrations/mcp/"
+Changelog = "https://vectorstep.io/docs/about/status-and-support/"
+
+[project.scripts]
+vectorstep-service-mcp = "vectorstep_service_mcp.__main__:main"
+
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+vectorstep_service_mcp = ["docs/*.md"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]

vectorstep_service_mcp-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/__main__.py

@@ -0,0 +1,13 @@
+"""Entry point for the VectorStep Service MCP server. Runs over stdio (Claude Code /
+Desktop) — see the README for VECTORSTEP_BASE_URL / VECTORSTEP_WEBHOOK_TOKEN configuration."""
+
+from . import tools  # noqa: F401 — importing registers every @mcp.tool()
+from .server import mcp
+
+
+def main() -> None:
+    mcp.run(transport="stdio")
+
+
+if __name__ == "__main__":
+    main()

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/client.py

@@ -0,0 +1,59 @@
+"""Thin HTTP client for talking to a VectorStep service instance.
+
+This is the ONLY coupling to VectorStep (SPEC-vectorstep-service-mcp.md §2.3): no VectorStep
+code is imported anywhere in this package. Configured from VECTORSTEP_BASE_URL /
+VECTORSTEP_WEBHOOK_TOKEN — see the README.
+"""
+
+import os
+
+import httpx
+
+from .errors import VectorStepAPIError, error_from_response
+
+
+class VectorStepClient:
+    def __init__(
+        self,
+        base_url: str | None = None,
+        token: str | None = None,
+        timeout: float = 30.0,
+        transport: httpx.AsyncBaseTransport | None = None,
+    ):
+        self.base_url = (base_url or os.environ.get("VECTORSTEP_BASE_URL") or "http://127.0.0.1:8000").rstrip("/")
+        self.token = token if token is not None else os.environ.get("VECTORSTEP_WEBHOOK_TOKEN")
+        self.timeout = timeout
+        # Injectable for tests (httpx.MockTransport) — None uses the real network.
+        self.transport = transport
+
+    def _headers(self) -> dict:
+        return {"Authorization": f"Bearer {self.token}"} if self.token else {}
+
+    async def request(self, method: str, path: str, **kwargs) -> dict:
+        url = f"{self.base_url}{path}"
+        try:
+            async with httpx.AsyncClient(timeout=self.timeout, transport=self.transport) as http_client:
+                resp = await http_client.request(method, url, headers=self._headers(), **kwargs)
+        except httpx.RequestError as exc:
+            raise VectorStepAPIError(
+                "network", f"Could not reach VectorStep at {self.base_url}: {exc}",
+            ) from exc
+
+        if resp.status_code >= 400:
+            raise error_from_response(resp)
+
+        if not resp.content:
+            return {}
+        return resp.json()
+
+    async def get(self, path: str, params: dict | None = None) -> dict:
+        return await self.request("GET", path, params=params)
+
+    async def post(self, path: str, json: dict | None = None) -> dict:
+        return await self.request("POST", path, json=json)
+
+    async def put(self, path: str, json: dict | None = None) -> dict:
+        return await self.request("PUT", path, json=json)
+
+    async def delete(self, path: str) -> dict:
+        return await self.request("DELETE", path)

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/docs/confidence_and_trust_vector.md

@@ -0,0 +1,120 @@
+# Confidence and the trust vector
+
+`get_run` returns several confidence-shaped fields per step. They are **not**
+duplicates of the same number — each is a distinct, independent signal, and
+VectorStep combines them by taking the *weakest* one, not an average. This doc
+maps those fields to what they actually mean, so you don't report
+`primary_confidence` as "VectorStep's confidence in this step" when it's really
+just the agent's own unverified self-report.
+
+## The four signals (+ calibration)
+
+| Signal | Field on a step | What it actually measures |
+|---|---|---|
+| **S** — self-report | `primary_confidence` | What the agent said about its own answer. The **least** trustworthy number here — it's the agent grading its own work, no outside check. |
+| **V** — verifier | `verifier_confidence` (see also `verifier_mode`, `verifier_agent`, `verifier_model`) | A second agent's opinion, if a `verifier:` block was configured. `verifier_mode: "critic"` saw the primary's answer and reviewed it (its errors correlate with the primary's own — weaker signal); `"independent"` redid the task blind (uncorrelated — stronger signal). A verifier can only ever *lower* the combined score, never raise it. |
+| **G** — grounding | `grounding_score` (0–1, or `null` if not computed) | Of the primary's specific factual claims, what fraction are actually backed by a real tool-call result in its own trace? This is about evidence, not correctness of reasoning. `null` means grounding wasn't configured for this step at all — that's different from a low score. |
+| **D** — deterministic check | `deterministic_passed` (bool, or `null` if none declared) | A hard, computer-verified fact (a shell command, webhook call, or human approval) — no model involved. If declared and failed, it forces trust to **zero**, no matter how confident everything else was. `null` means no checks were declared, not that one passed. |
+| **Calibration** | inside `trust_report.calibration` | Whether this exact agent/model's *real historical accuracy* at this confidence level (from past human-graded runs) should replace the raw self-report. The one signal that can move the score **up** as well as down. |
+
+To see the full calibration picture for a step — every confidence bin, per
+(agent, model, provider), not just the one bin a single run happened to land
+in — call `get_step_calibration(step_name)` directly rather than piecing it
+together by sampling `get_run` across many runs.
+
+### Calibration buckets are versioned
+
+A calibration bucket is keyed by `(step_name, agent, model, provider,
+prompt_hash, agent_version)`, not just the first four. Editing a step's
+prompt template, or editing a Gateway agent's `agent.yaml`/`soul.md`, starts
+a **new** bucket — outcomes from the old prompt/agent config never count as
+evidence for the new one, and vice versa. This is deliberate: a bucket
+mixing two different prompts isn't a bigger sample, it's a wrong one.
+
+Consequences worth knowing before reporting on calibration data:
+
+- **A small bucket isn't necessarily a new or unused step.** It may just be
+  mid-history under a recently-edited prompt or agent — call
+  `get_step_versions(step_name)` / `get_agent_versions(agent_name)` to see
+  the rest of that history before concluding a step "has no track record."
+- **Reverting a prompt restores its history for free.** Because buckets are
+  keyed by content hash, going back to an exact previous template rejoins
+  that version's original bucket and all its labelled results — nothing is
+  lost.
+- **`trust_report.calibration.bucket_reset`**, when present, tells you
+  explicitly that THIS run's bucket just reset and why:
+  `reason` is `"prompt_changed"`, `"agent_changed"`, or `"both_changed"`;
+  `previous_validated_n` is how many marked results the prior version had.
+  Treat this as the explanation for a step that suddenly stopped being
+  calibrated, not as a sign anything is broken.
+- **`agent_version` changes originate in the Gateway repo, not VectorStep's own
+  YAML.** A pipeline/step YAML file can look completely unchanged while an
+  agent's `soul.md` was edited on the Gateway — if calibration resets for a
+  step you didn't touch, check `get_agent_versions` for the agent it uses
+  before assuming something is wrong on the VectorStep side.
+
+## `effective_confidence` vs. `trust_report.combined_trust`
+
+`effective_confidence` on a step is the self-report *after* the verifier is
+applied (S adjusted by V) — it predates grounding/deterministic/calibration
+in the pipeline's history and is kept for backward compatibility.
+
+`trust_report.combined_trust` (inside the JSON blob in `trust_report`) is the
+**actual final number the step's gate compared against its
+`confidence_threshold`** — after verifier, calibration, grounding, and
+deterministic checks have all been applied in that order. **When you want
+"how much did VectorStep actually trust this step's result," use
+`trust_report.combined_trust`, not `effective_confidence`.**
+
+## Reading `trust_report`
+
+```json
+{
+  "signals": {
+    "S": 0.95, "S_after_V": 0.95,
+    "V": 0.85, "V_mode": "critic", "V_combination_strategy": "veto", "V_veto_floor": 0.6,
+    "G": 0.50, "C": null, "D": true
+  },
+  "combined_trust": 0.50,
+  "grounding": { "...": "per-claim detail" },
+  "deterministic_checks": [{"name": "...", "passed": true, "...": "..."}],
+  "calibration": { "...": "bin/bucket detail, or null if not enforced" },
+  "gate": {
+    "policy": "trust_vector",
+    "confidence_threshold": 0.75,
+    "on_low_confidence": "escalate"
+  }
+}
+```
+
+Order the signals are applied, each stage only able to lower the number
+(calibration is the sole exception):
+
+```
+S → (verifier) → S_after_V → (calibration, if enforced+validated) →
+  → (grounding, if enforced — a ceiling) → (deterministic checks — force to
+  zero on any failure) → combined_trust → compared to confidence_threshold
+```
+
+`trust_report.gate.policy` tells you whether any of this mattered at all:
+`"legacy_confidence"` means the step has none of verifier/grounding/
+deterministic/calibration configured — it's a plain self-report-vs-threshold
+gate, and every signal above `S` will be `null`. `"trust_vector"` means at
+least one of these mechanisms is active.
+
+## A field being `null` is information, not a gap
+
+`verifier_confidence: null` → no verifier was configured for this step, not
+"the verifier failed." `grounding_score: null` → grounding wasn't computed,
+not "zero evidence." `deterministic_passed: null` → no hard checks were
+declared, not "unknown pass/fail." Don't treat `null` as a bad result — it
+means the mechanism was never turned on for this step.
+
+## What "escalated because of low confidence" actually means
+
+If `trust_report.combined_trust` fell below `confidence_threshold`, the step
+took whatever action `gate.on_low_confidence` says (`escalate` / `abort` /
+`proceed`). A step escalating doesn't mean anything went wrong with the
+agent's answer — it means the *evidence for trusting that answer* didn't
+clear the bar the pipeline author set. See `statuses-and-accuracy` for why
+that's often the *correct*, intended outcome rather than a failure.

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/docs/promotion_readiness.md

@@ -0,0 +1,97 @@
+# Promotion readiness — owner-defined criteria, not a hardcoded bar
+
+`get_promotion_readiness(name)` reports whether a `stage: testing` pipeline's
+steps already have the evidence their OWNER requires before flipping
+`stage:` to `production`. `preview_promotion_readiness(name, readiness,
+apply_to)` tests a candidate `readiness:` config against the same evidence
+without writing anything. Both are **strictly advisory** — neither call
+blocks, gates, or changes VectorStep's behaviour in any way. The `stage:` edit
+itself stays a one-line YAML change a human makes and commits, exactly as
+before this feature existed.
+
+## Four independent tiers
+
+A pipeline (or a step within it) can configure any subset of four tiers in
+its `readiness:` YAML block:
+
+- **operational** — pure run-completion counting: N runs that ended in an
+  acceptable status. The cheapest bar, and the only one a non-LLM step
+  (`webhook`/`notify`/`human`/`pipeline` executor) can ever satisfy, since
+  those never write `effective_confidence`. Can only ever report `pass` or
+  `insufficient_data` — twenty completed runs plus five failed ones still
+  satisfies "twenty completed runs," so this tier never reports `fail`.
+- **confidence** — mean self-reported `effective_confidence` over qualifying
+  runs. A weak signal alone (a model can be confidently wrong), useful as an
+  early checkpoint before anyone has marked anything.
+- **accuracy** — judged accuracy (correct=1.0, partial=0.5, incorrect=0.0)
+  over human/deterministic/run-level labels, using the same label-precedence
+  chain as calibration: a human mark beats a failed deterministic check beats
+  an inherited run-level rating.
+- **calibration** — the strongest bar: not just "the output is good" but "the
+  step's confidence NUMBER can be trusted," using the same bucket machinery
+  as `get_step_calibration`.
+
+Every CONFIGURED tier must show `pass` for a step to read `ready`. An
+unconfigured tier is not a failure — it's simply not asked. A pipeline can
+require `operational` + `accuracy` without touching `calibration`, because
+those answer genuinely different questions.
+
+## Verdict vocabulary
+
+Per tier: `not_configured | insufficient_data | pass | fail`.
+
+**`insufficient_data` means "the bar can still be met by accumulating more
+evidence." `fail` means "observed evidence actively contradicts the bar."**
+These are not interchangeable when summarising a step's readiness for a
+user — a step reading `insufficient_data` just needs more runs/marks; a step
+reading `fail` has a real problem right now.
+
+Step-level and pipeline-level roll-ups use the same precedence:
+`fail > insufficient_data > pass > not_configured`, surfacing as
+`not_ready | building | no_data | ready | not_configured`.
+
+## THE trap: n_min is per confidence band, not a total
+
+`calibration.n_min` (default 20) is the number of marked results needed **at
+the same predicted-confidence band**, not a total across all bands. A step
+with 100 marked results spread evenly across 10 confidence bands has only 10
+in each band, and will **not** validate at `n_min: 20` even though 100
+sounds like plenty. Always check the fullest band's own count
+(`bins[i].n` on the relevant combo), never `total_n`, before concluding a
+step is close to validating calibration.
+
+## `require_current_config` and `require_own_evidence`
+
+Most tiers default `require_current_config: true` (except `operational`,
+which defaults `false`) — evidence is filtered to runs matching the
+pipeline's CURRENT prompt template and the step's most recently observed
+`agent_version`. A step whose prompt was just edited can drop straight to
+`insufficient_data` even with a long track record; `current_config` on each
+step's result reports `prompt_hash_matches_history` and an explicit `notes`
+entry naming how many earlier marked results were excluded, so this is never
+a silent zero.
+
+`calibration.require_own_evidence` (default `false`) lets a shared library
+step's PRODUCTION track record from a *different* pipeline count, when the
+agent/model/prompt/agent-version all match exactly — `production_pipelines`
+on the calibration combo names which pipeline(s) contributed that evidence.
+Set it `true` to restrict a step to only this pipeline's own runs.
+
+## Reading a result without asserted criteria
+
+A pipeline with no `readiness:` block anywhere returns
+`criteria_source: "none"` and asserts no verdict — but each step's
+`observed_combos` still carries calibration bins evaluated at
+`bin_width`/`n_min` (the tool's own parameters, default 0.1/20), so real
+signal never disappears behind "no criteria configured." A step's own
+configured calibration tier always uses its own values instead of these
+defaults.
+
+## Preferring the ready-made narrative
+
+Each step's result includes a `narrative`: a numbers-first, plain-English
+walkthrough of every configured tier's verdict. Prefer quoting it directly
+over re-deriving your own summary from the raw tier numbers — it already
+accounts for the per-band n_min trap, provenance breakdowns, and the
+`acceptable_statuses` "adding a status makes the bar LAXER, not stricter"
+gotcha.

vectorstep_service_mcp-0.1.0/src/vectorstep_service_mcp/docs/prompt_versions.md

@@ -0,0 +1,110 @@
+# Prompt and agent versioning
+
+Every calibration bucket VectorStep computes is scoped to the exact configuration
+that produced it — not just `(step_name, agent, model, provider)`, but also
+the step's `prompt_hash` and the Gateway agent's `agent_version`. This doc
+explains why, and how to read the tools that expose it
+(`get_step_versions`, `get_agent_versions`) without misreading a healthy
+reset as a problem.
+
+## The bug this exists to fix
+
+Before this existed, editing a step's prompt template — or editing a
+Gateway agent's `soul.md`/`agent.yaml` — silently kept counting outcomes
+from the OLD configuration as evidence for the NEW one. For a step with
+`calibration: {enforce: true}`, that meant the enforcement gate was making
+real control decisions using a measured accuracy figure that described a
+configuration that no longer existed. Nothing in the data indicated this
+was happening — it was invisible.
+
+Now, `prompt_hash` (a content hash of the step's prompt template) and
+`agent_version` (a content hash of the agent's full config, computed by the
+Gateway) are part of every calibration bucket's key. An edit to either one
+starts a fresh bucket. Old labelled results stop counting toward the new
+configuration.
+
+## A small bucket is not the same as a bad one
+
+The bucket key growing from 4 components to 6 means buckets are smaller and
+take longer to reach the labelled-sample minimum (`n_min`) than before this
+existed. That is the honest cost of measuring the right thing — a bucket
+that blends two different prompts is not a bigger sample, it's a wrong one.
+
+Practically: if `get_step_calibration` shows a step with very few labelled
+samples even though you know it's been running for months, don't conclude
+it's under-used or unlabelled. Call `get_step_versions(step_name)` — it's
+very likely mid-history under a prompt version that was edited recently,
+and the rest of its track record is sitting under a different `prompt_hash`
+in that same response (or under the synthetic legacy entry described next,
+if the step predates this feature entirely).
+
+## The `prompt_hash: null` / `agent_version: null` legacy entry
+
+`get_step_versions`/`get_agent_versions` can each return one entry whose
+`prompt_hash`/`agent_version` is `null`. This is NOT an error and NOT a step
+with no history — it represents every run recorded **before** prompt/agent
+versioning existed, or before an operator ran the backfill script
+(`backfill_prompt_versions.py`) for that step. `NULL` is a real, distinct
+bucket in `get_step_calibration` too (never a wildcard — a `NULL` row never
+pools with a real-hash row), so this entry's `runs_total`/`labelled_n` are
+just as real as any other version's.
+
+What's different about it: `template` (or `soul_md`/`agent_yaml`) is always
+`null` for this entry — no text was ever captured for pre-versioning runs,
+so there's nothing to show. `diff_from_previous` is `null` for it, and also
+`null` for whichever real version comes right after it chronologically
+(nothing to diff against a version with no text). Treat it exactly like a
+`note`d agent snapshot: an honest gap, not evidence anything is broken.
+Running the backfill script resolves it into a real, named version (on the
+operator's assertion that the current on-disk prompt/agent config matches
+what those historical runs actually used) — but until that happens, expect
+to see it on nearly every step/agent that predates this feature.
+
+## Reverting a prompt restores its history — for free
+
+The prompt-version registry is content-addressed: the hash IS the primary
+key. If an operator edits a prompt, doesn't like the result, and reverts it
+back to the exact original text, that revert reproduces the exact same
+hash as before — and automatically rejoins that version's original
+calibration history. No special-casing, no data loss. This is intentional,
+not a quirk to route around.
+
+The corollary: a *reformatting* of a prompt (different whitespace,
+reindentation, an added blank line) genuinely produces a different hash and
+a fresh bucket — normalisation only strips trailing whitespace per line and
+leading/trailing blank lines, nothing else. Blank-line structure and
+indentation can genuinely affect model behaviour, so treating a
+reformatted prompt as unchanged would be the wrong call more often than
+the current conservative default.
+
+## `agent_version` changes happen in a different repo
+
+`prompt_hash` is something VectorStep computes itself, from a step's
+`prompt_template` — the text is always right there in `get_step_versions`'
+response. `agent_version` is different: it's a hash the **Gateway**
+computes over an agent's entire config, including `soul.md`, and VectorStep
+cannot recompute it or predict when it will change. A pipeline/step YAML
+file can be completely untouched while an agent's `soul.md` is edited on
+the Gateway side — and that alone is enough to reset calibration for every
+step using that agent.
+
+If a step's calibration resets and you didn't edit anything in VectorStep:
+1. Check `trust_report.calibration.bucket_reset.reason` on a recent run of
+   that step — `"agent_changed"` or `"both_changed"` points at the Gateway,
+   not the step's own prompt.
+2. Call `get_agent_versions(agent_name)` (bare name, no `gateway:` prefix)
+   to see the agent's version history and what changed.
+3. A version's `note` field (instead of recovered `soul_md`/`agent_yaml`
+   text) means VectorStep couldn't confirm that snapshot at the time — the
+   Gateway was unreachable, or the agent changed again before VectorStep could
+   ask. That's an honest gap in VectorStep's records, not evidence the agent
+   itself has no configuration.
+
+## Quick reference
+
+| Question | Tool |
+|---|---|
+| Is this step's confidence trustworthy right now? | `get_step_calibration(step_name)` |
+| Did a specific prompt edit actually help? | `get_step_versions(step_name)` |
+| What changed in this agent, and which steps did it affect? | `get_agent_versions(agent_name)` |
+| Why did this run's bucket reset? | `get_run(run_id)` → `trust_report.calibration.bucket_reset` |