@malloy-publisher/skills 0.1.51 → 0.1.53

package/package.json

@@ -1,7 +1,7 @@
 {
    "name": "@malloy-publisher/skills",
    "description": "Agent skills for Malloy and the Malloy Publisher",
-   "version": "0.1.51",
+   "version": "0.1.53",
    "license": "MIT",
    "type": "module",
    "main": "dist/index.js",

package/skills/eval-answer/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-answer
 description: 'Score one analytical answer against a verified golden, and score which of the entities the golden depends on retrieval delivered to the answerer. Run the contamination checklist, re-execute the submitted query yourself, then spawn a judge subagent per skill:eval-judge. Append attempt, tool_call, score, and retrieval_score events to the file ledger (reference/ledger-schema.md). Never explain the failure (eval-diagnose) or edit the model (eval-improve). Use when asked whether an answer was correct, to score a run, or to baseline a model.'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # Evaluate One Answer

package/skills/eval-diagnose/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-diagnose
 description: 'Diagnose why a scored answer failed and who owns the fix, then cluster the failures by shared root cause. Walk dataset, agent-call, get_context/model, get_context/retrieval, construction, then model-definition. Append issue events to the file ledger, linked by traceId, one per cluster. Use after eval-answer, when triaging a run, or before changing a model. Does not edit the model (eval-improve).'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # Diagnose One Answer

package/skills/eval-import/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-import
 description: 'Turn a list of questions into an eval set, whatever shape it arrived in: a JSONL a customer sent, a CSV, a spreadsheet export, a markdown doc, an email thread, or a pull from production logs. Classify each item by what came WITH the question (a query, a number, prose criteria, or nothing), write cases.jsonl and set.json per reference/case-format.md, keep the file as it arrived, and seal each question so a later edit is detectable. Never marks a value-bearing golden verified and never invents a question. Use when questions arrive from outside and need to become a set, or before the first run against a set nobody here authored.'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # Import questions into an eval set

package/skills/eval-improve/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-improve
 description: 'Make the smallest safe edit that closes a diagnosed gap the model repo owns: a Malloy model edit, or a package skill under skills/. Every factual claim carries a probe receipt. Use after eval-diagnose, or when asked to fix a model so an agent can discover the right answer. Never accepts its own edit; the acceptance check belongs to eval-loop. Does not decide whether an answer was wrong (eval-answer) or why (eval-diagnose).'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # Improve the Model
@@ -117,12 +117,14 @@ publishing.
 
 Know which copy of the file the server actually reads. Hosts commonly serve a
 copy of the package rather than your working tree, so editing the model repo
-and reloading recompiles the unchanged copy: the reload succeeds, nothing
-changes, and a verification probe quietly tests the old model. Confirm the
-edit reached what is served before you trust a probe, and keep the model repo
-the source of truth that gets committed. On open-source Publisher the served
-copy lives under `publisher_data/<env>/<pkg>/` unless the environment is
-watch-mounted; other hosts distinguish a draft from a published version.
+and reloading changes nothing: the reload succeeds, and a verification probe
+quietly tests the old model. Confirm the edit reached what is served before
+you trust a probe, and keep the model repo the source of truth that gets
+committed. On open-source Publisher, a package outside a watch-mounted
+environment is a published version, which is immutable: a reload answers it
+`mode: unchanged` and never reads your files. Serve the model in watch mode
+(`eval.py serve model --set <set> --watch-env <env> --reinit`, then
+`improve.py --watch-mode`). Other hosts distinguish a draft from a published version.
 
 ## Step 3: One smallest edit
 

package/skills/eval-improve/scripts/improve.py

@@ -39,13 +39,18 @@ events describe diffs that never existed alone.
 
 MAKING AN EDIT VISIBLE TO THE SERVER
 
-Publisher normally serves a snapshot copy under `publisher_data/<env>/<pkg>/`,
-so editing the repo and reloading recompiles the unchanged copy: the reload
-succeeds, nothing changes, and the verification probe quietly tests the old
-model. `--watch-mode` says Publisher was started with `--watch-env`, which
-symlinks local-dir packages in place and makes edits live; then the helper is
-just a reload. Without it the helper syncs first. Either way the agent gets one
-generated script rather than a general shell.
+Publisher serves a package outside a watched environment as a published
+version under `publisher_data/<env>/<pkg>/<version>/`, which is immutable:
+editing the repo and reloading changes nothing, and the verification probe
+quietly tests the old model. `--watch-mode` says Publisher was started with
+`--watch-env` (`eval.py serve model --set <set> --watch-env <env>`), which symlinks
+local-dir packages in place and makes edits live; then the helper is just a
+reload. Without it the helper syncs the edit onto the served copy first, which
+works only on a server with versioning off. Either way a package the server
+serves as a published version is refused before any agent starts (the server
+is asked, in both modes): nothing synced there is served, the sync would
+delete it, and a reload of it answers 200 without reading the edit. Either way
+the agent gets one generated script rather than a general shell.
 """
 from __future__ import annotations
 
@@ -55,6 +60,7 @@ import pathlib
 import subprocess
 import sys
 import time
+import urllib.request
 from typing import Any
 
 sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent.parent
@@ -116,6 +122,37 @@ echo "reloaded; if sourceContentSha did not change, the edit did not reach the s
 SYNC_HELPER = "sync_and_reload.sh"
 
 
+def served_version(rest_url: str | None, environment: str,
+                   package: str) -> str | None:
+    """The version the model server serves this package as, or None.
+
+    The server's own answer: a published version's resource carries its
+    `versionId`, and a watch-mounted or copied package has none. A version
+    is immutable, so neither the sync nor watch mode reaches it. None also
+    when the server cannot be asked; the folder check below still stands.
+    """
+    if not rest_url:
+        return None
+    url = (f"{rest_url.rstrip('/')}/api/v0/environments/{environment}"
+           f"/packages/{package}")
+    try:
+        with urllib.request.urlopen(url, timeout=30) as r:
+            return json.load(r).get("versionId") or None
+    except Exception:
+        return None
+
+
+def served_as_versions(served: pathlib.Path) -> bool:
+    """Whether Publisher serves this package folder as published versions.
+
+    A version lives in `<pkg>/<version>/`, so the folder holds no
+    publisher.json of its own. A copy (versioning off) does, and a watch
+    mount is a symlink to the source.
+    """
+    return (served.is_dir() and not served.is_symlink()
+            and not (served / "publisher.json").exists())
+
+
 def git(args: list[str], cwd: pathlib.Path) -> str:
     return subprocess.run(["git", *args], cwd=cwd, capture_output=True,
                           text=True).stdout
@@ -386,7 +423,9 @@ def main(argv: list[str] | None = None) -> int:
     ap.add_argument("--server-root", type=pathlib.Path, default=None,
                     help="Publisher SERVER_ROOT; only needed without watch "
                          "mode. Each accepted edit is rsynced onto the copy "
-                         "this root serves and reloaded. Default: the one "
+                         "this root serves and reloaded, which works only "
+                         "with versioning off: a root serving published "
+                         "versions is refused. Default: the one "
                          "`serve.py --role model` uses, when it exists")
     ap.add_argument("--watch-mode", action="store_true",
                     help="Publisher was started with --watch-env, so edits are "
@@ -483,6 +522,21 @@ def main(argv: list[str] | None = None) -> int:
         print("warning: no --watch-mode and no --server-root, so the agent "
               "cannot sync its edit into Publisher's snapshot; every probe it "
               "runs will test the unedited model")
+    # Asked in both modes: --watch-mode is the caller's claim, and a server
+    # started with --watch-env over a preserved root still serves a version.
+    version = served_version(a.rest_url, a.environment, a.package)
+    served = (a.server_root / "publisher_data" / a.environment / a.package
+              if a.server_root and not a.watch_mode else None)
+    if version or (served is not None and served_as_versions(served)):
+        what = (f"the model server serves {a.package} as published version "
+                f"{version}" if version else f"{served} holds published versions")
+        why = ("so watch mode is not in effect for it" if a.watch_mode else
+               "no edit synced there is ever served, and the sync would "
+               "delete them")
+        raise SystemExit(
+            f"{what}, which is immutable: {why}. Fix: restart the model "
+            f"server in watch mode, `eval.py serve model --set {a.set_dir} "
+            f"--watch-env {a.environment} --reinit`, then pass --watch-mode")
 
     events = read_jsonl(a.run / "events.jsonl")
     cases = {c["qid"]: c for c in read_jsonl(a.set_dir / "cases.jsonl")}

package/skills/eval-improve/scripts/improve_test.py

@@ -229,6 +229,72 @@ class TruthEnvironment(unittest.TestCase):
 
 
 
+class ServedAsVersions(unittest.TestCase):
+    """The snapshot sync (`rsync --delete` onto the served folder) is safe only
+    over a copy. Over published versions it serves nothing and deletes them,
+    so it is refused."""
+
+    def setUp(self):
+        self.root = pathlib.Path(tempfile.mkdtemp())
+        self.addCleanup(shutil.rmtree, self.root, True)
+
+    def test_a_folder_of_versions_is_versions(self):
+        served = self.root / "pkg"
+        (served / "1.0.0").mkdir(parents=True)
+        (served / "1.0.0" / "publisher.json").write_text("{}")
+        self.assertTrue(improve.served_as_versions(served))
+
+    def test_a_copy_is_not(self):
+        served = self.root / "pkg"
+        served.mkdir()
+        (served / "publisher.json").write_text("{}")
+        self.assertFalse(improve.served_as_versions(served))
+
+    def test_a_watch_mount_is_not(self):
+        source = self.root / "src"
+        source.mkdir()
+        served = self.root / "pkg"
+        served.symlink_to(source, target_is_directory=True)
+        self.assertFalse(improve.served_as_versions(served))
+
+    def test_a_missing_folder_is_not(self):
+        self.assertFalse(improve.served_as_versions(self.root / "nothing"))
+
+
+class ServedVersion(unittest.TestCase):
+    """The server says whether it serves the package as a published version."""
+
+    def serve(self, body: bytes):
+        import http.server
+        import threading
+
+        class H(http.server.BaseHTTPRequestHandler):
+            def do_GET(self):
+                self.send_response(200)
+                self.end_headers()
+                self.wfile.write(body)
+
+            def log_message(self, *a):
+                pass
+
+        srv = http.server.HTTPServer(("127.0.0.1", 0), H)
+        threading.Thread(target=srv.serve_forever, daemon=True).start()
+        self.addCleanup(srv.shutdown)
+        return f"http://127.0.0.1:{srv.server_port}"
+
+    def test_a_version_is_named(self):
+        base = self.serve(b'{"name": "ecommerce", "versionId": "1.0.0"}')
+        self.assertEqual(improve.served_version(base, "e", "ecommerce"), "1.0.0")
+
+    def test_a_watch_mounted_package_has_none(self):
+        base = self.serve(b'{"name": "ecommerce"}')
+        self.assertIsNone(improve.served_version(base, "e", "ecommerce"))
+
+    def test_a_server_that_cannot_be_asked_is_none(self):
+        self.assertIsNone(improve.served_version("http://127.0.0.1:9", "e", "p"))
+        self.assertIsNone(improve.served_version(None, "e", "p"))
+
+
 class ClusterMembers(unittest.TestCase):
     """The improver sees each case's key. A rows golden kept in
     `golden.path` used to reach it as no key at all."""

package/skills/eval-judge/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-judge
 description: 'Decide whether ONE answer matches its golden, and say whether you believe the golden. Read this before emitting any verdict. Covers containment, column pairing, near_match, refusals, and the gold_status judgement. Use when scoring an attempt in an evaluation run; never to conduct a run (eval-loop), diagnose a failure (eval-diagnose) or edit a model (eval-improve).'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # The judge

package/skills/eval-loop/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-loop
 description: 'Conduct a local Publisher evaluation loop in five steps: scrape/run, eval, diagnose, improve, checkpoint. You are the conductor: import cases into the file ledger, spawn a blind answerer, then run eval-answer, eval-diagnose, and eval-improve. Persistence is plain files: the set in the model package''s evals/ directory, runs in the set''s workdir; checkpoints are git commits of the model repo. `eval.py` runs each step from the set''s eval.toml. Use to score a model, diagnose failures, improve behind an acceptance check, or roll back a bad direction.'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # The Evaluation Loop

package/skills/eval-loop/scripts/build_run_package.py

@@ -32,6 +32,7 @@ import pathlib
 import shlex
 import shutil
 import sys
+import time
 import urllib.error
 import urllib.request
 from typing import Any
@@ -555,6 +556,7 @@ def build(run_dirs: list[pathlib.Path], set_dir: pathlib.Path,
     for name in ("publisher.json", "eval_run.malloy"):
         if (TEMPLATE / name).exists():
             shutil.copy(TEMPLATE / name, out / name)
+    stamp_version(out / "publisher.json")
     if (TEMPLATE / "notebooks").exists():
         shutil.copytree(TEMPLATE / "notebooks", out / "notebooks", dirs_exist_ok=True)
     if (TEMPLATE / "public").exists():
@@ -566,6 +568,27 @@ def build(run_dirs: list[pathlib.Path], set_dir: pathlib.Path,
             "steps": len(steps_rows)}
 
 
+def stamp_version(manifest: pathlib.Path) -> None:
+    """Give this build a version of its own, higher than any earlier build's.
+
+    Publisher publishes a package from its location as the version its
+    publisher.json declares, and refuses different content under a version it
+    already holds (409). A version per build lets a rebuilt report replace the
+    one served: the highest version becomes `latest`. The patch is the time in
+    milliseconds, and above any version already in the file, so a second stamp
+    in the same millisecond still rises. Builds from different machines rely
+    on their clocks: one whose clock runs behind publishes a lower version,
+    which does not become `latest`.
+    """
+    if not manifest.exists():
+        return
+    body = json.loads(manifest.read_text())
+    previous = str(body.get("version", "")).rsplit(".", 1)[-1]
+    floor = int(previous) + 1 if previous.isdigit() else 0
+    body["version"] = f"1.0.{max(time.time_ns() // 1_000_000, floor)}"
+    manifest.write_text(json.dumps(body, indent=4) + "\n")
+
+
 def serving_lines(cfg: config.Config, run_dirs: list[pathlib.Path],
                   out: pathlib.Path, on_model_server: bool = False) -> list[str]:
     """The registration command and both URLs, which are in different path spaces.
@@ -605,8 +628,10 @@ def serving_lines(cfg: config.Config, run_dirs: list[pathlib.Path],
 def register(base: str, env: str, out: pathlib.Path) -> str | None:
     """Register the built package on `base`, or return why it could not be.
 
-    A POST for a name the server already has re-copies the package, so a
-    rebuilt report replaces the one it served; no delete is needed first.
+    A POST publishes the build as the version stamp_version gave it, so a
+    rebuilt report becomes the package's `latest` and replaces the one it
+    served; no delete is needed first. (A server with versioning off
+    re-copies the package instead, to the same effect.)
     """
     body = json.dumps({"name": out.name, "location": str(out.resolve())}).encode()
     req = urllib.request.Request(

package/skills/eval-loop/scripts/build_run_package_test.py

@@ -301,6 +301,25 @@ class RefusalsAndServing(unittest.TestCase):
         self.assertEqual(json.loads(body)["location"], str(out.resolve()))
 
 
+class AVersionPerBuild(unittest.TestCase):
+    """A rebuilt report publishes as a new version, which becomes `latest`."""
+
+    def test_each_build_declares_a_higher_version(self):
+        tmp = pathlib.Path(tempfile.mkdtemp())
+        self.addCleanup(shutil.rmtree, tmp, True)
+        manifest = tmp / "publisher.json"
+        manifest.write_text(json.dumps({"name": "eval-run", "version": "1.0.0"}))
+        build_run_package.stamp_version(manifest)
+        first = json.loads(manifest.read_text())
+        build_run_package.stamp_version(manifest)
+        second = json.loads(manifest.read_text())
+        self.assertEqual(first["name"], "eval-run")
+        major_minor = [v["version"].rsplit(".", 1)[0] for v in (first, second)]
+        self.assertEqual(major_minor, ["1.0", "1.0"])
+        patches = [int(v["version"].rsplit(".", 1)[1]) for v in (first, second)]
+        self.assertGreater(patches[1], patches[0])
+
+
 class Register(unittest.TestCase):
     """`package` registers the report on the truth server itself."""
 

package/skills/eval-loop/scripts/run_baseline.py

@@ -138,6 +138,12 @@ TOOL_ARM_TOOLS = ("mcp__publisher__get_skill",)
 # `get_status` and `list_packages` reveal the other packages on the server and
 # are no use to an answerer given its scope in the prompt.
 #
+#   report_api              a second query path, under REST rules rather than
+#                           execute_query's; its answer is not one about the model.
+#   show_report             draws a panel for a person; a headless answerer gets
+#   list_reports            only text, and report titles and descriptions are
+#                           discovery outside get_context, the thing measured.
+#
 # Enumerated, because deny beats allow: naming the server (`mcp__publisher`,
 # or `mcp__publisher__*`) removes ALL of its tools including the ones wanted,
 # verified 2026-09-08.
@@ -146,7 +152,10 @@ ANSWER_DENIED = ("mcp__publisher__compile_model",
                  "mcp__publisher__search_database_schema",
                  "mcp__publisher__reload_package",
                  "mcp__publisher__get_status",
-                 "mcp__publisher__list_packages")
+                 "mcp__publisher__list_packages",
+                 "mcp__publisher__list_reports",
+                 "mcp__publisher__show_report",
+                 "mcp__publisher__report_api")
 
 # Every tool this server exposes, so the two lists above can be checked to
 # PARTITION it rather than merely not overlap. That is the answer to the one
@@ -158,8 +167,9 @@ ANSWER_DENIED = ("mcp__publisher__compile_model",
 # a mismatch here is a decision, not a lint.
 PUBLISHER_MCP_TOOLS = ("compile_model", "execute_query", "get_agent",
                        "get_context", "get_skill", "get_status",
-                       "list_packages", "reload_package",
-                       "search_database_schema", "search_malloy_docs")
+                       "list_packages", "list_reports", "reload_package",
+                       "report_api", "search_database_schema",
+                       "search_malloy_docs", "show_report")
 # The platform target: a hosted MCP server exposing the same two operations
 # under its own names. The CLI addresses a tool as `mcp__<server>__<tool>`, so
 # both halves are configuration -- `--hosted-mcp-server` names the server (which

package/skills/eval-loop/scripts/serve.py

@@ -176,6 +176,22 @@ def init_decision(root: pathlib.Path, reinit: bool,
                    f"publisher.config.json")
 
 
+def watch_note(watch_env: str | None, seed: bool) -> str | None:
+    """The warning for --watch-env on a start that does not seed, or None.
+
+    An environment is mounted in place only when it is first loaded from the
+    config, so on a preserved root a package already published stays a
+    version: the flag sets PUBLISHER_WATCH and changes nothing an edit
+    reaches.
+    """
+    if not watch_env or seed:
+        return None
+    return (f"  ! --watch-env {watch_env} on a preserved server root: a package "
+            f"this root already published stays an immutable version, so an "
+            f"edit never reaches it. Pass --reinit too, unless the root was "
+            f"first served with --watch-env")
+
+
 # One ranking call, unscoped, is what kicks the embedding sync.
 #
 # RANKING, because the marker and the sync both hang off the ranking path: a
@@ -458,6 +474,14 @@ def main(argv: list[str] | None = None) -> int:
                     "With --role, the role's mcp_port in eval.toml, as --port")
     ap.add_argument("--allow-proxy", action="store_true")
     ap.add_argument("--trace-retrieval", action="store_true")
+    ap.add_argument("--watch-env", metavar="ENV",
+                    help="serve ENV in watch mode (PUBLISHER_WATCH): its local "
+                         "packages are mounted in place, so an edit is read on "
+                         "reload, which eval-improve needs. Without it each "
+                         "package is a published version, which no edit "
+                         "reaches. The mount is decided when the environment "
+                         "is first loaded, so add --reinit to switch a server "
+                         "root that is already serving")
     ap.add_argument("--reinit", action="store_true",
                     help="drop the store and re-read publisher.config.json. "
                          "Needed after a config edit -- a new environment or a "
@@ -565,8 +589,13 @@ def main(argv: list[str] | None = None) -> int:
         env["PUBLISHER_ALLOW_PROXY_CONNECTIONS"] = "true"
     if a.trace_retrieval:
         env["PUBLISHER_MCP_TRACE"] = "retrieval"
+    if a.watch_env:
+        env["PUBLISHER_WATCH"] = a.watch_env
     seed, why = init_decision(root, a.reinit, changed)
     print(why)
+    stale_watch = watch_note(a.watch_env, seed)
+    if stale_watch:
+        print(stale_watch)
     note = retrieval_note(env) if a.role != "truth" else None
     if note:
         print(note)

package/skills/eval-loop/scripts/serve_test.py

@@ -470,6 +470,46 @@ class Roles(unittest.TestCase):
                          '{"old": true}\n')
         self.assertFalse((root / "publisher.pid").exists())
 
+    def test_watch_env_reaches_the_server_as_publisher_watch(self):
+        """Watch mode is the one way an edit reaches a served package."""
+        d = a_set(TOML)
+        root = pathlib.Path(tempfile.mkdtemp(prefix="serve-root-"))
+        pub = pathlib.Path(tempfile.mkdtemp(prefix="serve-pub-"))
+        (pub / "dist").mkdir()
+        (pub / "dist" / "server.mjs").write_text("")
+        stuck = [sys.executable, "-c", "import time; time.sleep(60)"]
+        seen = {}
+        real_popen = serve.subprocess.Popen
+
+        def spy(cmd, **kwargs):
+            seen.update(kwargs["env"])
+            return real_popen(cmd, **kwargs)
+
+        with mock.patch.object(serve, "listening", return_value=False), \
+             mock.patch.object(serve, "alive", return_value=False), \
+             mock.patch.object(serve, "server_cmd", return_value=stuck), \
+             mock.patch.object(serve.subprocess, "Popen", side_effect=spy):
+            serve.main(["--role", "model", "--set", str(d),
+                        "--server-root", str(root), "--publisher-dir", str(pub),
+                        "--wait", "1", "--no-warm-retrieval",
+                        "--watch-env", "evals"])
+        self.assertEqual(seen.get("PUBLISHER_WATCH"), "evals")
+
+
+
+class WatchNote(unittest.TestCase):
+    """--watch-env mounts in place only on a start that seeds the store."""
+
+    def test_a_preserved_root_is_warned(self):
+        note = serve.watch_note("evals", seed=False)
+        self.assertIn("--reinit", note)
+        self.assertIn("immutable version", note)
+
+    def test_a_seeding_start_is_not(self):
+        self.assertIsNone(serve.watch_note("evals", seed=True))
+
+    def test_no_watch_env_is_not(self):
+        self.assertIsNone(serve.watch_note(None, seed=False))
 
 
 class WarmByDefault(unittest.TestCase):

package/skills/eval-loop/templates/eval-run-package/README.md

@@ -167,22 +167,20 @@ regardless -- so check the `PUBLISHER_READY` line for the port it actually took.
 Two more Publisher behaviours worth knowing, because both look like your edit did
 nothing.
 
-It serves a **snapshot copy** under `publisher_data/<env>/<pkg>/`. The `location`
-in `publisher.config.json` is a one-time import source, not a live mount, and
-`reload_package` does **not** re-fetch from it -- it answers
-`mode: in-place`, re-reading the snapshot. So the working recipe is sync then
-reload:
-
-```bash
-rsync -a --delete <location>/ "$SERVER_ROOT/publisher_data/<env>/<pkg>/"
-# then reload_package {environmentName, packageName}
-```
-
-Sync the whole package. A new model over stale CSVs fails with a column-not-found
-(`` `attempt_key` not found ``) that reads like a model bug rather than a stale
-copy. Watch `sourceContentSha` across the reload to confirm it saw the change.
-Restarting works too, and is the only option if you would rather not reason about
-which half is stale.
+Unless its environment is watched, it serves a **published version**, under
+`publisher_data/<env>/<pkg>/<version>/`. The `location` in `publisher.config.json`
+is read once, when the version is published, and a version is immutable:
+`reload_package` answers it `mode: unchanged` and never reads your files, and
+neither does a restart. Never copy files into `publisher_data/` to get around
+that. It holds the version folders, and a version's files are not read again.
+
+To iterate, start the server with `--watch-env <env>`. The package is mounted in
+place, and `reload_package` (`mode: in-place`) recompiles what you saved. Watch
+`sourceContentSha` across the reload to confirm it saw the change. To ship an
+edit to a published package, bump `version` in its `publisher.json` and publish
+it again (`POST …/environments/<env>/packages` with its `location`): the new
+version becomes `latest`. `eval.py package` does this for the report: each build
+declares a higher version.
 
 And a package must be declared in `publisher.config.json` with an `environments`
 block -- a bare `{"frozenConfig": false}` starts cleanly and serves nothing, logging

package/skills/eval-report/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: eval-report
 description: 'Write the summary a person reads after an evaluation run: what ran, what it scored, which failures were the MODEL and which were the EVAL itself (out of turns, contaminated, unestablished golden, unreadable judge), what was learned, and what to do next. Every claim links to the artifact behind it. Use at the end of any run, before quoting a number to anyone. Never scores an answer (eval-answer) or decides who owns a failure (eval-diagnose).'
-version: 0.1.51
+version: 0.1.53
 ---
 
 # Report a Run

package/skills/malloy-analysis/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-analysis
 description: "Read before your first get_context call on any business or data question. The required workflow, not background: which source, which rows, how to verify. Skipping it causes confident wrong answers."
-version: 0.1.51
+version: 0.1.53
 ---
 
 <!-- Copyright (c) Credible Data Inc. SPDX-License-Identifier: MIT -->
@@ -83,6 +83,7 @@ Your first result is a draft, not an answer. The difference between a useful ana
 - **Ask "what would make this wrong?"** then run the query that would expose that problem. A plausible-looking wrong answer is the most dangerous kind.
 - **Check the common failure modes:**
   - Fan-out / double-counting: if you joined across grain, compare `count()` with `count(base_key)`, the base source's own key (in Malloy `count(field)` is already the distinct count). A large gap means base rows are repeated and aggregates are inflated. The check reads one way only: `count(joined.field)` is a distinct count at the joined grain, so it is normally much SMALLER than `count()`, which is not missing data. Measure coverage at the base grain: `count() { where: joined.field is null }`.
+  - Grouping by an array: a field with `relationship: "many"` that is not reached through a join (`lines.sku`, `tags.each`) gives one group per distinct element value, and a row lands in every group its array holds, so its groups add up to more than the total, as across a one-to-many join. Do not print a total row across it, and count elements with `tags.count()`, not `count(tags.each)`, which is the distinct count.
   - Broken filters: a quick count confirms a filter narrowed the data as expected. Watch case, spelling and date-format mismatches; a filter that matches nothing still returns a result, just the wrong one. Confirm which dimension the user's word ("brand") maps to when several fields look alike, and that the field means what you assume (gross against net revenue): read its doc. Before a `pick` or `where` that groups values into categories, list the field's distinct values, since an unlisted one falls into `else` unnoticed. Apply the filter the question implies ("last quarter" with no time filter returns all-time data). A view can have a `where:` built in (`recent_orders`): read its code, and query the base source with explicit filters if it conflicts with the ask.
   - Null-driven loss: `count() { where: the_field is null }` shows how many rows a key field drops (`count(the_field)` counts distinct values, so subtracting it from `count()` is not a null count).
   - Parts that do not sum to the whole: if you split a total into categories, confirm they add up. If a doc says the parts are not additive (one event logged on two layers, a non-additive measure), do not print a total row across that dimension, least of all one nobody asked for; give the per-category figures and say why.

package/skills/malloy-analysis-report/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-analysis-report
 description: Combine validated Malloy queries into a report (a linear report, read top to bottom). Use when the user asks to "create a report", "combine these into a report", or wants a persistent multi-query artifact.
-version: 0.1.51
+version: 0.1.53
 ---
 
 <!--

package/skills/malloy-charts/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-charts
 description: Read before choosing a chart or adding a chart annotation. Chart types, tag syntax and scale rules, KPI cards, dashboards, sparklines, and the renderer mistakes that fail silently.
-version: 0.1.51
+version: 0.1.53
 ---
 <!--
 Copyright (c) Credible Data Inc.

package/skills/malloy-classify/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-classify
 description: 'Propose #(classification) annotations for a Malloy model from an access policy''s class scheme, backed by aggregate value evidence from the served data, and show them as a diff before writing anything. Use when the user asks to "classify the model", "tag PHI/PII", "label sensitive fields", or adds fields to a package that a policy governs, and before running malloy-compliance-review on a model nobody has classified.'
-version: 0.1.51
+version: 0.1.53
 ---
 
 <!--

package/skills/malloy-compliance-review/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-compliance-review
 description: 'Check a Malloy package and its controls against a written access policy: turn the policy''s prose into a control register, run compliance.py check against a served package (structure, access as each role, and the values themselves), judge what needs a person, and produce a report and a signable attestation. Use when the user asks whether a model "complies with", "satisfies" or "meets" a policy, HIPAA, SOC 2, PCI or the like, asks for a compliance review or attestation, or wants the review in CI.'
-version: 0.1.51
+version: 0.1.53
 ---
 
 <!--

package/skills/malloy-dashboards/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-dashboards
 description: "Renamed to malloy-reports. A dashboard is now a report with layout=grid: a .malloy file with an ## artifact tag, written at the package root. Load malloy-reports instead when the user asks for a dashboard, a filterable view, or drill-through between views."
-version: 0.1.51
+version: 0.1.53
 ---
 
 <!--

package/skills/malloy-define/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-define
 description: Propose scope, a source plan and field definitions for a Malloy model. Which tables and questions, which sources at what grain, then renames, dimensions and measures, each backed by querying data.
-version: 0.1.51
+version: 0.1.53
 ---
 <!--
 Copyright (c) Credible Data Inc.

package/skills/malloy-define-agent/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: malloy-define-agent
 description: Define a package agent - a named brief, with its own skills, optional schedules and a run budget, declared in a package's publisher.json. Use when the user asks to create, add, change or fix an agent in a package, or wants a recurring or unattended job. Not for adopting an agent that already exists (that is malloy-run-agent), and not for a one-off question, which you just answer.
-version: 0.1.51
+version: 0.1.53
 ---
 <!--
 Copyright (c) Credible Data Inc.