@malloy-publisher/server 0.0.243 → 0.0.245

package/README.md

@@ -2,11 +2,53 @@
 
 The Malloy Publisher Server is an Express.js server that provides an API for managing and accessing Malloy data models, packages, and queries
 
+## Quick start
+
+This section is self-contained: this README ships in the npm tarball, where most of the repository does not, so nothing here needs a file you do not have. Later sections are written for a clone and do point into the repository.
+
+**New workspace?** One command scaffolds a package, the server config, the MCP wiring, and the agent skills:
+
+```bash
+npm create @malloy-publisher/malloy-package@latest sales
+npm start
+```
+
+Run bare like that, the package comes with a small sample dataset, so there is something to query immediately. To start from a file of your own instead, add `-- --data ./orders.csv`, naming a delimited or Parquet/Excel file you actually have (the `--` is required, and the path is relative to where you run the command). Keep the `@latest`: `npm create` resolves through the npx cache, and an unversioned name silently reuses whatever old copy is on the machine.
+
+**Existing directory of models?** Write a `publisher.config.json` beside a package directory that holds a `publisher.json`, then run the server pointing at it:
+
+```bash
+npx @malloy-publisher/server --server_root . --watch-env local
+```
+
+```json
+{
+  "frozenConfig": false,
+  "environments": [
+    {
+      "name": "local",
+      "packages": [{ "name": "sales", "location": "./sales" }],
+      "connections": []
+    }
+  ]
+}
+```
+
+The config is read from `<server_root>/publisher.config.json` by default; `--config <path>` overrides it.
+
+Three facts that are easy to get wrong:
+
+- **A flat-file (CSV/Parquet/XLSX) package needs no `connections` entry at all.** Every loaded package automatically gets its own DuckDB sandbox connection named `duckdb`, which is what `duckdb.table('data/file.csv')` resolves against. That name is reserved: an environment-level connection named `duckdb` fails the whole environment at init (call an env-level DuckDB connection `shared_duckdb` or similar).
+- **A package `location` is local only when it starts with `./`, `../`, `~/`, or `/`.** Write `./sales`, not `sales`: a bare name is read as a remote URI, and the package is skipped with `Invalid package path: "sales". Must be an absolute mounted path or a GCS/S3/GitHub URI.` in `loadErrors`. A `./` or `../` path resolves against the config file's directory; `~/` against your home directory.
+- **Local authoring means `--watch-env <env>`.** Without it the server copies each local package into `publisher_data/` at boot and serves the copy, so edits to your source directory are never read — a reload recompiles the copy and still answers 200. Adding the flag to a later boot does not undo a copy already made: pass `--watch-env <env> --init` once to re-mount (it wipes `publisher_data/` and re-syncs from config), then plain `--watch-env` boots keep watching. `publisher_data/<env>/<pkg>` tells you which you got — a symlink is mounted, a real directory is a copy.
+
+Poll `curl -s http://localhost:4000/api/v0/status` until `operationalState` is `"serving"`, then check `loadErrors` (absent when everything loaded). The MCP endpoint for agents is `http://localhost:4040/mcp`.
+
 ## Configuration
 
-`publisher.config.json` lives in this directory. The repository root also contains a symlink (`/publisher.config.json` → `./packages/server/publisher.config.json`) so that running the server from either location picks up the same config. Edit one and you've edited both.
+Two example configs ship in the npm package, beside this README: [`publisher.config.example.duckdb.json`](./publisher.config.example.duckdb.json) (the GitHub-hosted sample packages, no connection block) and [`publisher.config.example.bigquery.json`](./publisher.config.example.bigquery.json) (adds a BigQuery connection). Copy either one to `publisher.config.json` and point `--server_root` at its directory.
 
-For the BigQuery-enabled variant, see [`publisher.config.example.bigquery.json`](./publisher.config.example.bigquery.json) and the [Quick Start in the repo root README](../../README.md#quick-start).
+In a clone, the live `publisher.config.json` lives in this directory, and the repository root contains a symlink (`/publisher.config.json` → `./packages/server/publisher.config.json`) so that running the server from either location picks up the same config. Edit one and you've edited both.
 
 ### Remaining deprecation warnings
 
@@ -15,11 +57,11 @@ Removing the unused `trino` CLI direct dep (it pulled in `@google-cloud/translat
 - **npm CLI tooling**: `npmlog`, `gauge`, `are-we-there-yet`, `glob@7/8/10`, `rimraf@3`, `tar@6.2.1`, `inflight`, `@npmcli/move-file`, `node-domexception`, `querystring` — pulled in by npm itself and by `node-pre-gyp`/`node-gyp`. Not actionable from this repo.
 - **`uuid@8.x` / `uuid@9.x`**: surfaced across multiple transitives (Malloy, AWS SDKs, others). Resolves when each upstream bumps to `uuid@11`.
 - **`q@1.5.1`**: pulled in via `thrift` → `@databricks/sql` → `@malloydata/db-databricks`. Resolves when Databricks upgrades `@databricks/sql` past the thrift dep, or when we replace the Databricks driver.
-- **`aws-sdk@2.1693.0`**: listed as a direct dep in `packages/server/package.json` but not imported anywhere in source — leftover, candidate for removal in a follow-up PR. The actual consumer is `@aws-sdk/client-s3` v3.
+- **`aws-sdk@2.1693.0`**: no longer a direct dep (removed from `packages/server/package.json`); anything still surfacing it is transitive. The actual S3 consumer is `@aws-sdk/client-s3` v3.
 
 ## K6 Test Presets
 
-The Malloy Publisher Server includes several K6 test presets to help you test its performance and stability.
+The Malloy Publisher Server includes several K6 test presets to help you test its performance and stability. These live in the repository, not the npm package, so run them from a clone.
 
 Below is a list of the available test presets:
 
@@ -124,7 +166,7 @@ For more information on how to configure OpenTelemetry collectors, please refer
 
 ## MCP Prompt Capability
 
-Publisher's MCP interface exposes the bundled agent **skills** (under [`skills/`](../../skills/)) as **LLM-ready prompts**, so hosts that ingest MCP but do not load skill files can pull the same guidance. For authoring or contributing skills, see [`docs/agent-skills/`](../../docs/agent-skills/).
+Publisher's MCP interface exposes the bundled agent **skills** as **LLM-ready prompts**, so hosts that ingest MCP but do not load skill files can pull the same guidance. The skills are published separately as [`@malloy-publisher/skills`](https://www.npmjs.com/package/@malloy-publisher/skills); their source is [`skills/`](https://github.com/malloydata/publisher/tree/main/skills), and authoring guidance is in [`docs/agent-skills/`](https://github.com/malloydata/publisher/tree/main/docs/agent-skills).
 
 List prompts:
 

package/dist/app/api-doc.yaml

@@ -2530,10 +2530,12 @@ components:
           description:
             Configured packages and environments that did not load, and why. A
             load failure is not fatal to the server, so operationalState stays
-            serving and the affected packages are simply absent from
-            environments; this is the only place that difference is reported, so
-            check it before treating a serving server as complete. Present only
-            when something failed to load, so a healthy server omits the field.
+            serving and the affected packages are absent from environments,
+            unless the entry carries stale = true, in which case the package is
+            still listed and still serving its last successful compile. This is
+            the only place either difference is reported, so check it before
+            treating a serving server as complete or current. Present only when
+            something failed, so a healthy server omits the field.
           items:
             type: object
             properties:
@@ -2551,6 +2553,24 @@ components:
               message:
                 type: string
                 description: Why it did not load.
+              stale:
+                type: boolean
+                description:
+                  When true, the package did load earlier and is still serving
+                  that last successful compile; the message is why its most
+                  recent reload (a watch-mode save, malloy_reloadPackage, or
+                  reload=true) failed. The served model is older than the files
+                  on disk. Absent on ordinary load failures. This array is the
+                  only place staleness is reported, because the package's entry
+                  under environments, and its own package resource, both read
+                  SERVING; join on the package name here to tell a current
+                  package from a stale one.
+              failedAt:
+                type: string
+                format: date-time
+                description:
+                  When the failed reload happened. Only present alongside
+                  stale = true.
         frozenConfig:
           type: boolean
           description:
@@ -2753,8 +2773,19 @@ components:
         location:
           type: string
           description:
-            Package location, can be an absolute path or URI (e.g. github, s3,
-            gcs, etc.)
+            Package location. A LOCAL path must start with `./`, `../`, `~/`,
+            or `/`; a `./` or `../` path resolves against the config file's
+            directory, and `~/` against the home directory. Anything else is
+            read as a remote URI (github, s3, gcs, https), so a bare name like
+            `sales` is not local and the package is skipped with
+            "Invalid package path" in the server's loadErrors. At boot a local
+            package is COPIED into publisher_data/ and the copy is served, so
+            edits to the source directory are not read; starting the server
+            with `--watch-env <env>` mounts local packages in place (symlink)
+            and live-reloads them instead. The mount is chosen when the
+            environment is first loaded from config, so adding `--watch-env`
+            to a later boot needs `--init` to re-mount a package already
+            copied.
         scope:
           type: string
           enum:
@@ -2812,9 +2843,55 @@ components:
             findings: a `storage=` source not served from storage, a `#@ persist`
             source that was not recognized as materializable, a persist-target
             collision, and materialization-config problems that degrade
-            attribution. Server-computed and read-only: ignored on create/update
-            requests and only returned in responses. Present only when there are
-            such findings.
+            attribution. Discovery findings: a model listed in `explores` whose
+            export closure surfaces no sources or named queries, so it lists as
+            empty and renders a blank page. Server-computed and read-only:
+            ignored on create/update requests and only returned in responses.
+            Present only when there are such findings.
+
+
+            **Pre-aggregation contributes nothing to this array. A
+            `#@ preaggregate` that cannot be honored is an ERROR, never a
+            warning** — a 400 at publish or PATCH, and a load failure thereafter.
+            Rejected are a `#@ preaggregate` anywhere but on a measure;
+            a `#@ preaggregate` with no `grain=`;
+            a grain naming anything but a dimension the base source itself
+            declares, which rules out an inline time truncation such as
+            `order_time.day`; a measure that cannot be correctly
+            re-aggregated to a coarser grain, whether because its aggregate has no
+            exact merge function (`count(distinct)`, `median`, percentiles, `mode`,
+            and `avg` until it is decomposed) or because it carries its own filter,
+            an ungrouping (`all`, `exclude`), or a required group-by; a measure
+            whose expression cannot be POSITIVELY classified as re-aggregatable,
+            which fails closed rather than being guessed at; a base source with a
+            FAN-OUT join (`join_many` or `join_cross`), which can multiply rows,
+            while a `join_one` is permitted and a measure aggregating through one
+            is served normally; a base source that already fails persist eligibility (unbound
+            parameters, `given` references, `#(authorize)` gates); `sharing=` or
+            `schedule=` at the resource level; and a synthesized rollup name that
+            collides with an existing model object. Each names the measure and the
+            fix.
+
+
+            Refusing rather than silently dropping the measure is the deliberate
+            choice. A dropped measure is not served from the rollup at ANY grain,
+            so the declaration would have no effect whatsoever while still reading
+            as applied — and the build cost of the rollup would still be paid. An
+            author who writes `#@ preaggregate` on a measure that cannot be rolled
+            up has made a modeling mistake, and an error is where a mistake
+            belongs.
+
+
+            The same verdict is enforced at LOAD, not only at publish, so a
+            package whose declarations stopped being honorable does not load at
+            all: it is absent from its environment and reported in
+            `ServerStatus.loadErrors`. Worth knowing because whether a measure can
+            be re-aggregated is derived from the compiled model, so a Malloy
+            version change can in principle reclassify a measure that published
+            cleanly long before. That surfaces as a package that stops loading
+            rather than as a package quietly serving without its rollups — louder,
+            and deliberately so, since the alternative is paying for rollups that
+            answer nothing.
           items:
             type: object
             properties:
@@ -2850,15 +2927,28 @@ components:
           description: >-
             Controls whether the discovery surface is also a query boundary.
             `"declared"` (the default) makes queryable == discoverable: when
-            `explores` is declared, only `explores` model files — and within
-            them only the `export {}` closure — are valid top-level query
-            targets; every other source still compiles, imports, joins, and
-            extends but is not directly queryable (denied with 404). `"all"`
-            decouples them: `explores`/`export {}` gate discovery only and every
-            compiled source stays directly queryable. When `explores` is absent
-            there is no curated surface, so both modes are equivalent
-            (everything queryable). Invalid values fall back to `"declared"`.
-            Identity-based access is a separate concern — see `#(authorize)`.
+            `explores` is declared, only `explores` model files are valid query
+            entry points, and the queryable sources are the union of those
+            files' `export {}` closures — so a source exported by any listed
+            file stays queryable whichever listed model path the request
+            addresses it through. Admission is by declaration, not by name: a
+            request clears only when the model it names resolves the name to the
+            very source a listed file exported, so a same-named source declared
+            in a hidden file is not admitted by the coincidence. Every other
+            source still compiles, imports, joins, and extends but is not
+            directly queryable (denied with 404).
+            The `/compile` endpoint is exempt (compile is the authoring loop;
+            the boundary is discovery curation, not access control — for
+            confidentiality use `#(authorize)`, which gates compile too). A
+            source that is both hidden and `#(authorize)`-gated still answers
+            `/compile` with the generic 404, so the exemption cannot be used to
+            enumerate gated names.
+            `"all"` decouples the axes: `explores`/`export {}` gate discovery
+            only and every compiled source stays directly queryable. When
+            `explores` is absent there is no curated surface, so both modes are
+            equivalent (everything queryable). Invalid values fall back to
+            `"declared"`. Identity-based access is a separate concern — see
+            `#(authorize)`.
         manifestLocation:
           type: ["string", "null"]
           description: |
@@ -3566,7 +3656,10 @@ components:
         statements a query issues, for the backend's own reporting — cost
         attribution, workload classification, tracing. Each backend applies it
         through a per-query mechanism: Snowflake a per-statement `QUERY_TAG`
-        (the bag as JSON, so its properties are queryable in `QUERY_HISTORY`),
+        (the bag as JSON, so its properties are queryable in `QUERY_HISTORY`) —
+        except on a `storage=` build, whose read goes through DuckDB's
+        query-passthrough and so sets the tag for the SESSION, which means the
+        driver's own connection probes carry the build's bag too,
         BigQuery per-job `labels` (queryable in `INFORMATION_SCHEMA.JOBS`), and
         every other backend a leading SQL comment (`-- team="finance"`, visible
         in query history / `pg_stat_activity`). It never affects results, and it
@@ -3645,6 +3738,74 @@ components:
             Which of the two queryRowLimit came from. Only server_default means
             rows were probably left behind; a deliberate limit or top returning
             exactly that many rows is a complete answer.
+        servedFrom:
+          type: ["string", "null"]
+          enum: [storage, live_fallback, null]
+          description: >-
+            How this answer was produced, when the query reached the storage
+            routing decision at all.
+
+            `storage` — served from a materialized table in a `storage=`
+            destination, via the virtual-source transform. The source's own
+            warehouse was not queried.
+
+            `live_fallback` — the query DID route to storage and the store then
+            failed underneath it, and the source's freshness fallback allowed it
+            to degrade to the live warehouse. **The rows are correct and the
+            request succeeded**, which is why this needs its own value: it is
+            indistinguishable from a storage hit at the call site, and treating it
+            as one would report a healthy hit rate for a broken store.
+
+            Null when the query never had a storage binding to consider — every
+            query on a deployment with no `storage=` sources, which is the default.
+
+            A storage-served result is byte-identical to a live one by design, so
+            this field is the only way a caller can tell that materialization did
+            anything, or show a user which of their sources it applies to.
+        executionTimeMs:
+          type: integer
+          description: >-
+            Wall-clock milliseconds from the start of query handling through the
+            end of execution.
+
+            Covers compile, authorize, storage routing, prepare and the warehouse
+            round trip. Narrower than the HTTP round trip — it excludes
+            serialization and transport, so it reads lower than a caller's own
+            stopwatch — and it is the same span the `malloy_model_query_duration`
+            histogram records, so the two are directly comparable. Use it to
+            attribute ONE query, which an aggregate histogram cannot do; use the
+            histogram for rates and percentiles.
+        queryCostBytes:
+          type: ["integer", "null"]
+          description: >-
+            Warehouse bytes this query SCANNED, as reported by the backend that
+            ran it. Not what it is billed — see below.
+
+            **Null is not zero.** It means the backend did not report a figure,
+            and there are two very different reasons for that. Either the backend
+            has no inline equivalent — BigQuery returns `totalBytesProcessed` in
+            the job metadata the driver already fetches, while Snowflake exposes
+            bytes only in `QUERY_HISTORY` keyed by query id (join it later on the
+            `query_id` query-metadata property rather than paying a round trip per
+            query) and Postgres has none at all. Or the query touched no warehouse,
+            which is the case for every `servedFrom: storage` answer. Check
+            `servedFrom` before reading a null as "free".
+
+            Where it is reported it is the executed job's ACTUAL bytes
+            processed, i.e. bytes SCANNED — not an estimate, and not what the
+            query is billed. BigQuery bills a 10MB minimum per query, so a small
+            read is billed well above this figure. Good for comparing queries
+            against each other; not a spend number.
+
+            Named `cost` rather than `scanned` because it carries Malloy's own
+            `runStats.queryCostBytes` through verbatim — the name follows the
+            figure back to its source.
+
+            One asymmetry worth knowing on BigQuery: the connector reports a
+            genuinely-zero scan as absent rather than as 0, so a fully cached
+            query is indistinguishable here from a backend that cannot say. Free
+            and unknown are the same value, and a savings calculation built on
+            this will treat free queries as unmeasured rather than as free.
 
     LogMessage:
       type: object
@@ -3728,7 +3889,14 @@ components:
           description: Resource path to the connection
         name:
           type: string
-          description: Name of the connection
+          description:
+            Name of the connection. The name `duckdb` is RESERVED for the
+            per-package DuckDB sandbox every loaded package gets automatically;
+            declaring an environment-level connection with that name fails the
+            whole environment at init. A flat-file (CSV/Parquet/XLSX) package
+            needs no connection entry at all — its models reach the sandbox as
+            `duckdb.table('data/…')`. Name an environment-level DuckDB
+            connection something else (e.g. `shared_duckdb`).
         type:
           type: string
           description: Type of database connection
@@ -4702,6 +4870,45 @@ components:
           type: string
         dialect:
           type: string
+        origin:
+          type: string
+          enum: [persist, preaggregate]
+          default: persist
+          description: >-
+            Which authoring annotation produced this plan entry. `persist` — the
+            modeler wrote `#@ persist` on the source; every entry on a deployment
+            that uses no pre-aggregation, and the meaning of an absent value.
+            `preaggregate` — the modeler wrote `#@ preaggregate` on measures of
+            `preaggregate.baseSourceName`, and the publisher SYNTHESIZED this
+            rollup source to serve them.
+
+
+            Note that BOTH kinds carry a `#@ persist` annotation by the time they
+            reach this plan, because synthesis emits one (see `annotationFields`).
+            So this reports which surface the AUTHOR used, not which annotation
+            sits on the source — which is the question worth answering, since a
+            `preaggregate` entry's `name` appears in no model file and without
+            this field an operator or orchestrator sees a source nobody wrote with
+            no way to tell it from one they did.
+
+
+            Everything downstream is deliberately indifferent to it: a synthesized
+            entry is built, identified, manifested, shared and garbage-collected
+            exactly like a hand-written one.
+        preaggregate:
+          oneOf:
+            - $ref: "#/components/schemas/PreaggregatePlan"
+            - type: "null"
+          description: >-
+            What the synthesized rollup covers: the base source it was derived
+            from, the one grain it is built at, and the measures served there.
+            Present only when `origin` is `preaggregate`; null otherwise.
+
+
+            Coverage is complete by construction — a measure that cannot be
+            correctly re-aggregated is refused at publish rather than dropped
+            from the rollup — so there is no per-measure omission list to
+            reconcile against the model.
         sourceEntityId:
           type: string
           description: >-
@@ -4740,6 +4947,14 @@ components:
             scope modes. (Per-source `sharing`/`schedule` were retired: scope is
             a single package-level mode and a schedule is package-root-only —
             declaring either on a source is a publish-time manifest error.)
+
+
+            A synthesized rollup (`origin: preaggregate`) declares nothing of its
+            own and gets no freshness policy of its own: it INHERITS its base's,
+            so what is reported here is the base source's effective objective
+            resolved by the rule above. A rollup and its base therefore go stale
+            together, which is what makes "serve the base when the rollup is
+            stale" a safe degradation rather than a change of answer.
         queryMetadata:
           oneOf:
             - $ref: "#/components/schemas/QueryMetadata"
@@ -4794,15 +5009,142 @@ components:
             a dangling or stale name is rejected rather than deferred to a build.
             Neither participates in `sourceEntityId`: adding a watermark to an
             existing source must not re-address its table.
+
+
+            For a synthesized rollup (`origin: preaggregate`) the annotation was
+            generated by the publisher rather than authored, so this reports what
+            synthesis emitted. It is still the annotation that governs the build,
+            which is why it is reported the same way: a rollup is an ordinary
+            persist source whose `#@ persist` nobody typed.
         modelPath:
           type: string
-          description:
+          description: >-
             Package-relative path of the `.malloy` model that declares this
             source (e.g. `order_rollup.malloy`). The source's sourceID embeds an
             absolute `file://` modelURL with no package boundary, so this is the
             only place the relative path is exposed; callers use it to deep-link
             the source back to its model.
 
+
+            A synthesized rollup (`origin: preaggregate`) is declared by no model,
+            so this is the model of its base — the file holding the
+            `#@ preaggregate` annotations it was synthesized from, which is where
+            an author would go to change it.
+
+    PreaggregatePlan:
+      type: object
+      description: >-
+        Provenance for a rollup source synthesized from `#@ preaggregate`
+        annotations. A query never names a rollup, never changes shape because one
+        exists, and gets the same answer whether one is used or not, so this is
+        the only place the set of rollups and the coverage of each is visible at
+        all.
+
+
+        **This object describes ONE rollup table, at ONE grain.** A rollup is a
+        single `GROUP BY`, so a grain is a property of the table, not of the
+        individual measures in it: `grainDimensions` lists the dimensions of that
+        one grain, and `measures` are the measures served at it.
+
+
+        That inverts how the grain was authored, which is worth knowing before
+        reading a plan. The modeler declares per measure — `#@ preaggregate
+        grain="order_day, category"` — and may declare a measure at SEVERAL grains
+        by writing several annotation lines on it, each of which is its own rollup.
+        The publisher then packs every measure declaring the same (grain,
+        connection, base source) into a single synthesized source, because ten
+        measures at one grain should be one table and one CTAS rather than ten. So
+        the plan has one entry per DISTINCT grain declared anywhere on the source,
+        which is neither the number of annotated measures nor the number of
+        annotations: a measure declared at two grains appears in two entries, and
+        two measures sharing a grain appear in one. To recover the per-measure view
+        that was written, iterate `buildPlan.sources`, keep the `origin:
+        preaggregate` entries, and invert each entry's grain-to-measures pairing.
+
+
+        Choosing between two rollups and one at the combined grain is a size
+        judgement, not a correctness one — see `grainDimensions`.
+      required: [baseSourceName, grainDimensions, measures]
+      properties:
+        baseSourceName:
+          type: string
+          description: >-
+            The source whose measures carry the `#@ preaggregate` annotations this
+            rollup was synthesized from. Queries name THIS source; the rollup is
+            selected behind it, or bypassed, per grain coverage.
+        grainDimensions:
+          type: array
+          items:
+            type: string
+          description: >-
+            The dimensions of the ONE grain this rollup is built at — the parsed
+            form of a single `grain=` annotation value, so `grain="order_day,
+            category"` reports as two entries here and not as two grains.
+
+
+            **Every entry is a dimension the base source itself declares**, never
+            a truncation written inline. `grain="order_time.day"` is refused at
+            publish: a stored column of truncated values either shadows the base's
+            own `order_time` and answers untruncated queries from truncated rows,
+            or is named something no query references and serves nothing. So time
+            grains are reached by declaring `dimension: order_day is
+            order_time.day` on the source and naming `order_day` here.
+
+
+            **Sorted, and canonically so.** Synthesis derives the rollup's name
+            and identity from the sorted grain, so two authors who write the same
+            dimensions in different orders get one table rather than two. Compare
+            grains as sets using this list; never by the annotation's original
+            string, which is not canonical.
+
+
+            A query can be answered here only if this grain covers it: every
+            dimension the query groups by is in this list, or is a coarser
+            truncation of one (a `month` of a stored `order_day` is derivable and
+            does route; a `day` of a stored `order_month` is not). Which makes the
+            set of grains ACROSS a package the whole performance story, and the
+            reason adding a rollup per query shape is the documented way to spend
+            more on builds than you save.
+
+
+            Note that covering does not require an exact match: a query grouping
+            by a SUBSET of these dimensions — or by none of them — is served here
+            too, by re-aggregating across the ones it omits. So one rollup at the
+            finest grain an author needs does subsume every coarser one, and a
+            combined grain is often the economical choice.
+
+
+            That subsumption is about coverage, though, not cost, and it is why a
+            measure may still be declared at several grains. A combined grain has
+            roughly the PRODUCT of its dimensions' cardinalities, so
+            `customer_id, order_day` can approach the base table's row count and
+            save almost nothing, while `order_day` alone is tiny. Two rollups then
+            beat one, each query reading the smallest that covers it, at the price
+            of two tables to build and refresh. Correctness is identical either
+            way; the choice is size against build cost.
+        measures:
+          type: array
+          items:
+            type: string
+          description: >-
+            The measures served at this rollup's grain, under their original names
+            — the names a query uses, not the names of the partial-aggregate
+            columns stored to serve them. A query using only these, at a grain
+            this rollup covers, is eligible to be served from its table.
+
+
+            These are the measures whose `#@ preaggregate` named THIS grain. A
+            measure declared at a different grain is not missing: it is in the
+            plan entry for that grain. Do not read this list as the base source's
+            full set of pre-aggregated measures — that is the union of these lists
+            across every entry sharing a `baseSourceName`.
+
+
+            **Every measure that declared this grain is here.** There is no
+            partial rollup and no per-measure omission to check: a measure that
+            cannot be correctly re-aggregated is refused at publish, so a rollup
+            that exists carries everything that asked to be in it.
+
     Realization:
       type: string
       enum: [SNAPSHOT, COPY]
@@ -5046,8 +5388,40 @@ components:
       properties:
         sourceEntityId:
           type: string
+        error:
+          type: [string, "null"]
+          description: >-
+            Why this source failed to materialize, in the words the warehouse or
+            compiler reported. Present only on a source that failed; absent on one
+            that built, whose table the remaining fields describe. A build that
+            loses some of its sources still returns the ones that succeeded, so an
+            entry carrying this is how a consumer tells a failed source from a
+            healthy one.
         sourceName:
           type: string
+          description: >-
+            The persist source this table was built for. Convenience only:
+            `sourceEntityId` is the identity, and this is the human-readable label
+            beside it.
+
+
+            **It may name no source in any model file.** A pre-aggregation rollup
+            is a persist source the publisher synthesized, so its name is
+            generated (see `PersistSourcePlan.origin`). Match on
+            `sourceEntityId`; do not resolve this against a package's models and
+            treat a miss as a corrupt manifest.
+        origin:
+          type: string
+          enum: [persist, preaggregate]
+          default: persist
+          description: >-
+            Mirrors `PersistSourcePlan.origin` for the source this entry was built
+            from: `persist` for a source the modeler wrote `#@ persist` on, which
+            is the meaning of an absent value, and `preaggregate` for a rollup
+            synthesized from `#@ preaggregate` measures. Repeated here because a
+            manifest travels without its build plan, and a consumer holding only
+            the manifest otherwise cannot tell why it holds a table for a source
+            that does not appear in the package.
         materializedTableId:
           type: string
           description: Echoes the caller-assigned id from the BuildInstruction.
@@ -5088,6 +5462,56 @@ components:
           $ref: "#/components/schemas/Realization"
         rowCount:
           type: ["integer", "null"]
+        buildDurationMs:
+          type: ["integer", "null"]
+          description: >-
+            Wall-clock milliseconds spent building this one source's table —
+            measured around the build itself, so it excludes planning, manifest
+            distribution, and anything the orchestrator does around it.
+
+            Null when no build ran, which is the normal case for an incremental
+            source whose refresh SKIPPED because its boundary already covered the
+            requested range. Null rather than 0 on purpose: a zero would average
+            into build-duration series as an implausibly fast build.
+
+            Per SOURCE, where a run-level duration is per instruction. A run that
+            builds five sources reports five of these, and their sum is less than
+            the run's duration by whatever the orchestration cost.
+        queryCostBytes:
+          type: ["integer", "null"]
+          description: >-
+            Warehouse bytes SCANNED by the read that produced this table — the
+            recurring cost of keeping the source materialized, and the debit
+            against whatever materializing it saves on the read side.
+
+            Bytes scanned, never bytes billed. The two differ by up to BigQuery's
+            10MB-per-query floor, and materialization refreshes are mostly small
+            reads, so a field carrying one on some paths and the other elsewhere
+            would be unsummable across a package's sources.
+
+            **Null is not zero**, and which paths report a figure differs.
+
+            A COLOCATED build reports one, taken from the Malloy connection's own
+            statistics.
+
+            A plain `storage=` build reports one when its warehouse read carried
+            query metadata, and null otherwise. That read goes through DuckDB's
+            query-passthrough rather than a Malloy connector, so the figure is
+            read back from the warehouse's own accounting afterwards — which needs
+            an identifier for the job, and obtaining one is part of how a TAGGED
+            read is issued. An untagged read is issued the older way, which yields
+            no such identifier, and reports null. A read whose connection cannot
+            reach its own job records also reports null; that case is counted by
+            `publisher_storage_build_attribution_skipped_total`.
+
+            An incremental delta reports null because its statements do not run
+            through a single call whose result reaches here. A chained `storage=`
+            build reports null because it read its parent's already-materialized
+            table and touched no warehouse at all. A POSTGRES source reports null
+            on every path: the label that makes a read reportable is BigQuery's,
+            and Postgres has no per-statement tag to carry one. And a bag whose
+            every property is unusable as a BigQuery label key renders no label,
+            so such a read is issued the untagged way and reports null too.
         dataAsOf:
           type: string
           format: date-time

package/dist/app/assets/{EnvironmentPage-zu0NeDu-.js → EnvironmentPage-EjXlK1AR.js}

@@ -1 +1 @@
-import{$ as r,H as t,j as e,a1 as i,aa as o}from"./index-B-jw9EkA.js";function m(){const s=r(),{environmentName:n}=t();if(n){const a=i({environmentName:n});return e.jsx(o,{onSelectPackage:s,resourceUri:a})}else return e.jsx("div",{children:e.jsx("h2",{children:"Missing environment name"})})}export{m as default};
+import{$ as r,H as t,j as e,a1 as i,aa as o}from"./index-DJ-3RN5W.js";function m(){const s=r(),{environmentName:n}=t();if(n){const a=i({environmentName:n});return e.jsx(o,{onSelectPackage:s,resourceUri:a})}else return e.jsx("div",{children:e.jsx("h2",{children:"Missing environment name"})})}export{m as default};