agentic-data-contracts 0.62.0__tar.gz → 0.63.0__tar.gz

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/.github/workflows/ci.yml

@@ -32,7 +32,7 @@ jobs:
       contents: read
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           cache-dependency-glob: "pyproject.toml"
@@ -54,7 +54,7 @@ jobs:
         python-version: ["3.12", "3.13"]
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           cache-dependency-glob: "pyproject.toml"
@@ -99,7 +99,7 @@ jobs:
       contents: read
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           cache-dependency-glob: "pyproject.toml"
@@ -168,7 +168,7 @@ jobs:
       contents: read
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           cache-dependency-glob: "pyproject.toml"
@@ -215,7 +215,7 @@ jobs:
         python-version: ["3.12", "3.13"]
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           cache-dependency-glob: "pyproject.toml"
@@ -256,7 +256,7 @@ jobs:
       contents: read
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
       - run: uvx uv-secure uv.lock
 
   publish:
@@ -268,9 +268,9 @@ jobs:
       id-token: write
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
       - run: uv python install 3.13
       - run: uv build --no-sources
-      - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
+      - uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/.gitignore

@@ -32,8 +32,15 @@ docs/paper/pvldb.out
 docs/paper/pvldb.bbl
 docs/paper/pvldb.blg
 
-# Local git worktrees (isolated workspaces for plan execution)
+# Local git worktrees (isolated workspaces for plan execution). The harness
+# puts its own under .claude/worktrees/; ignoring them also keeps ty, which
+# respects .gitignore, from checking a worktree's nested experiments/ copy
+# against the root venv.
 .worktrees/
+.claude/worktrees/
 
 # Subagent-driven-development scratch (ledgers, briefs, review packages)
 .superpowers/
+
+# Local secrets (API tokens)
+.env

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.16.10
+    rev: v0.17.3
     hooks:
       - id: ruff-check
         args: [--fix]
@@ -11,7 +11,7 @@ repos:
   # -- a `language: system` hook calling `ty check` cannot see them, and needs
   # `unresolved-import = "ignore"` to stay quiet.
   - repo: https://github.com/astral-sh/ty-pre-commit
-    rev: v0.0.85
+    rev: v0.0.95
     hooks:
       - id: ty
 

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/CHANGELOG.md

@@ -2,6 +2,37 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.63.0] - 2026-11-07
+
+### Fixed
+
+- **`OssieSource` reads current-spec Ossie documents.** apache/ossie#383 made a document one model, with `name` and `datasets` at the root, and the spec no longer accepts the earlier `semantic_model` list. `OssieSource` read only that list, so a current-spec document loaded as a source with no tables, metrics or relationships, and nothing reported it. The contract's own allowlist and forbidden operations were still enforced; the agent lost the metrics and joins. NVIDIA Auto-Ontology's Ossie export is in the current shape, and its real converter output is now a test fixture. A document in the legacy shape is still read and logs a warning.
+- **An Ossie table schema lists the physical columns its fields read.** A field is a name over an expression, and the field's name was registered as the column. A field `customer_name` over the column `name` registered `customer_name`, so the agent was told to select a column the table does not have, and a computed field such as `net_total` (`subtotal - discount`) was listed as a column too. The column is now the one the expression reads, in plain, backtick or bracket quoting; a computed field, a keyword such as `current_date` and an unparseable expression register none. Relationship endpoints are mapped the same way; a relationship that joins on a computed field is skipped with a warning. Metric expressions still name fields, so the bridge is in `describe_table`: a renamed column's description ends `(Ossie field: customer_name)`, and a table's lists its computed fields as `net_total = subtotal - discount`. Every fixture named its fields after their columns, so nothing caught this until a real export.
+- **A metric written with only `expression.sql` has an expression.** The spec added `sql` as the primary portable expression (apache/ossie#617); such a metric loaded with an empty `sql_expression`. `sql` is read as Ossie SQL and overrides a `dialects` entry for `OSSIE_SQL_2026`. `ANSI_SQL` is still preferred over both. Dialect names, the caller's `dialect=` included, are now compared ignoring case, so the spec's `OSSIE_SQL_2026` and the proposal's `Ossie_SQL_2026` name one dialect.
+- **Vendor blocks on datasets, fields, metrics and relationships are carried.** The schema allows `custom_extensions` on each, and only the model-level list was read; the rest were dropped silently. They are now in `get_extras()["ossie_entity_extensions"]`, keyed by model, entity kind, entity name, then vendor. NVIDIA Auto-Ontology puts each metric's physical SQL there. Our own block is still read only at the model level; one found on an entity is named in a warning.
+
+### Added
+
+- **A BI formula is never handed to the agent as SQL.** When no preferred dialect is present, an Ossie metric fell back to its first declared expression, whatever its language, so a Tableau export's `SUM([Profit])` reached `lookup_metric` as the metric's SQL. The fallback now skips the formula languages in Ossie's dialect registry (`TABLEAU`, `DAX`, `MDX`, `MAQL`, `SIGMA`, `THOUGHTSPOT`, `HOLISTICS_AQL`, `SPARQL`); asking for one with `dialect=` still returns it, and an unregistered dialect is still a fallback. A metric written only in those is kept with an empty `sql_expression` and an `untranslated` reason quoting each formula, as dbt and Cube report a metric they cannot express. A field written only in one registers no column.
+- **Dataset-scoped Ossie metrics** (`datasets[].metrics`, apache/ossie#591), which were not read. Each is named `dataset.metric`, the form the spec references it by. Our vendor block's `metrics` reaches it by that name, and a key in the bare form is named in a warning that suggests the scoped one. Its expression is passed through unchanged, as a model-scoped one's is.
+
+### Changed
+
+- **An Ossie document is refused at load when it would load as less than it declares**, with a `ValueError` naming what is missing, instead of loading empty or partial. That covers a document with neither `datasets` at the root nor a `semantic_model` list, an empty `semantic_model` list, a document with both, a model with no datasets, a dataset, field or metric with no name, and a metric with no expression. A field with no expression, which the spec also requires, is still read, with its name as its column.
+- **A `drill_by` in our Ossie vendor block names a physical column.** It is validated against the table schema, which now holds physical columns, so a `drill_by` written with a renamed field's name (`orders.customer_name` for a field over `cust_nm`) now fails at load with "unknown column". It named a column the table does not have; write the column instead.
+- **Silent drops in an Ossie document are now named in a warning:** a key in our vendor block that matches no metric (its owners, tier and decompositions applied to nothing), a vendor listed twice in one `custom_extensions` list (the later block wins), and a duplicate metric name (lookups return the first).
+
+## [0.62.1] - 2026-10-16
+
+### Fixed
+
+- **`blocked_columns` no longer rejects `COUNT(*)` as `SELECT *`** (#148). On any table with `blocked_columns`, a plain row count failed with "SELECT * may expose blocked columns", because the checker matched a star anywhere in the query and `COUNT(*)` contains one. The same false positive was fixed in `no_select_star` earlier, but that fix did not reach `blocked_columns`. A star as the argument of `COUNT` (`COUNT(*)`, `COUNT(t.*)`, `COUNT(DISTINCT *)`) now passes. Every other star is still rejected: `SELECT *`, `SELECT t.*`, a starred subquery, and a star inside any other function. That last case is why `no_select_star`'s rule was not reused: it allows any star that is not itself a projection, but `row_to_json(t.*)`, `ARRAY_AGG(t.*)` and `STRUCT(*)` return every column's value, blocked ones included.
+- **The `blocked_columns` message no longer says `SELECT *` about a query without one.** It now reads "A `*` outside COUNT(*) may expose blocked columns: …" for every rejected star. It does not quote the query back: the checker has no dialect, and SQL rendered in sqlglot's default one can name a function the query does not contain (Snowflake's `OBJECT_CONSTRUCT(*)` renders as `STAR_MAP(*)`).
+
+### Documentation
+
+- **The README says what query checks do not protect against.** They stop a cooperating agent from breaking a rule by mistake. They are not an access-control boundary: a query written to evade `blocked_columns` can still return a blocked column's values. A column that must never reach a caller needs database column privileges or a view as well.
+
 ## [0.62.0] - 2026-10-12
 
 ### Added

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: agentic-data-contracts
-Version: 0.62.0
+Version: 0.63.0
 Summary: YAML-first, domain-driven data governance for AI agents
 Project-URL: Homepage, https://github.com/flyersworder/agentic-data-contracts
 Project-URL: Repository, https://github.com/flyersworder/agentic-data-contracts
@@ -114,14 +114,17 @@ That table is the claim; [`experiments/dabstep-contract-eval/`](experiments/dabs
 | `manual_prompt` | the vendor manual verbatim in the prompt | 22.9% | 42.9% | 50.3% | 23.8% |
 | `schema_only` | table and column names only | 13.9% | 22.6% | 37.0% | 22.9% |
 
-- **The gain is the content, not the scaffolding.** `contract_hollow` runs identical tooling with the domain knowledge stripped out. That is the only difference between the two arms, and on every model tested it explains more of the contract's lead than the tooling does.
-- **The contract is also the cheapest way to be right.** On every model the contract arm uses the fewest turns and by far the least reasoning; on three of four it also makes the fewest tool calls and is cheapest per correct answer (on Sonnet 5, about half the cost of any other arm). The exception is `gpt-5.6-sol`, where the contract's own text is a real input-token bill and `manual_prompt` edges it. It is the only arm that never had to be forced to answer. It is not thinking harder; it has less to search for.
-- **Governance held.** Across the ungoverned arms the models submitted 166 mutating SQL statements on 26 tasks. Both governed arms submitted zero.
-- **No leakage.** The contract was written from the vendor's manual alone and frozen, digest-pinned, before any benchmark question was read.
+- **The gain is the content, not the scaffolding.** `contract_hollow` runs identical tooling with the domain knowledge stripped out. That is the only difference between the two arms, and on every model tested it explains more of the contract's lead than the tooling does. Repeated three times on two more models, emptying the contract costs 26-50 hard-task points; the tools alone are worth -9 to +15.
+- **Most of the lead over the manual was knowledge the manual lacks.** Follow-up experiments found two facts the vendor manual never states (an empty list means "all", and the fees of overlapping rules add up). Stated as two notes in the prompt, they recover most of the contract's lead; with the facts stated alike, the contract's format adds about 4-6 points on the fee tasks. Write the conventions down; that is where the accuracy is.
+- **Delivery buys cost and an audit trail, not much accuracy.** For one database the contract in a cached prompt is as cheap or cheaper than looking it up. From about twice that size, lookup is cheaper (1.7-2.4x at four databases' worth, 6.4x at 22) and keeps working past the context window, and the lookup log names the definition behind each answer.
+- **Stated rules deterred writes.** Across the ungoverned arms of the single runs the models submitted 166 mutating SQL statements on 26 tasks; the governed arms submitted none. Across every later DABStep run, no run whose prompt stated the forbidden operations corrupted its database copy, whether the rules came through the tools or the contract's text. Serve the agent through a read-only role regardless.
+- **Frozen before scoring.** The contract was written in one interactive session, a model drafting under the author's direction, from the vendor's manual, the dataset README and the data, and frozen, digest-pinned, before any benchmark score existed; the paper gives the full timeline and what it cannot prove.
 - **Externally scored.** The contract arm answered all 450 tasks and was submitted to the [DABStep leaderboard](https://huggingface.co/spaces/adyen/DABstep), where Adyen graded it against the official golds: **51.9% on the hard split** for `glm-5.3-flash`, against a published band of ~20–26% hard for manual-in-prompt baselines running stronger models.
 
 Scope, stated plainly. The effect concentrates in the domain-specific task bucket; the non-fee bucket shows no contract advantage on any model. The table's golds are reconstructed for 401 of 450 tasks, and the leaderboard check shows that scoring runs ~6 points optimistic on hard tasks, one-sidedly; correcting for it moves the arm gaps by under 3 points, in both directions. Only the contract arm was submitted, so the gap above is not graded the same way on both sides. And this is one benchmark at k=1, where a replicate finds 23% of task verdicts flip at temperature 0. Full results, every caveat, and all 6,416 transcripts: **[`FINDINGS.md`](experiments/dabstep-contract-eval/FINDINGS.md)**.
 
+Archived on Zenodo for the paper: the code at release `dabstep-eval-2026-11`, [10.5281/zenodo.24015265](https://doi.org/10.5281/zenodo.24015265), and the transcripts of releases `dabstep-eval-2026-10` and `dabstep-eval-2026-11`, [10.5281/zenodo.24015263](https://doi.org/10.5281/zenodo.24015263). Paper v2 used the `dabstep-eval-2026-10` versions: code [10.5281/zenodo.23598672](https://doi.org/10.5281/zenodo.23598672), transcripts [10.5281/zenodo.23598750](https://doi.org/10.5281/zenodo.23598750).
+
 **Works with:** any Python agent framework — first-class helpers for the [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python), [LangChain](https://github.com/langchain-ai/langchain) / [deepagents](https://github.com/langchain-ai/deepagents), and [Pydantic AI](https://github.com/pydantic/pydantic-ai), plus a framework-free path (the tools are plain async functions). Optionally integrates with [ai-agent-contracts](https://pypi.org/project/ai-agent-contracts/) for formal resource governance.
 
 > **See it running:** [three example agents](#examples) — `revenue_agent` (finance), `growth_agent` (experimentation), `ops_agent` (SRE) — each runs end-to-end in demo mode with no API key.
@@ -701,7 +704,7 @@ Either way, the agent knows that "revenue is recognized at fulfillment, not at b
 
 ### Why progressive disclosure works
 
-This pattern — compact index in the prompt, detailed context on demand — is the same philosophy validated by agent skill systems, MCP tool servers, and RAG architectures. Instead of overloading the agent's context window with everything upfront, you give it just enough to know *where to look*, then let it pull details when needed. The result is better token efficiency, more focused reasoning, and fewer hallucinations from context overload.
+This pattern -- compact index in the prompt, detailed context on demand -- is the same philosophy behind agent skill systems, MCP tool servers, and RAG architectures. Measured on our benchmarks, its payoff is cost and traceability rather than accuracy: a contract several databases large in the prompt lost no accuracy on questions that name their metric, but on-demand lookup kept the cost per question nearly flat as the contract grew to 22 databases, ran where the full contract no longer fit the context window, and logged which definition each answer used.
 
 ## Contract Rules
 
@@ -723,6 +726,8 @@ Each rule carries a `query_check` (pre-execution) or `result_check` (post-execut
 | `require_limit` | Require a LIMIT clause |
 | `max_joins` | Cap the number of JOINs |
 
+Query checks read the SQL text before it runs. They keep a cooperating agent from doing by mistake what a rule forbids, and tell it why, so it can correct the query. They are not an access-control boundary: SQL has more ways to route a value into a result than a static check can recognise, so a query written to evade `blocked_columns` can still return a blocked column's values. If a column must never reach a given caller, enforce that in the database as well, with column privileges or a view that leaves it out, and connect the agent with credentials that have only that access.
+
 **Built-in result checks** (post-execution, validated against query output):
 
 | Check | Description |
@@ -943,15 +948,20 @@ semantic:
     path: "./semantic/model.yml"
 ```
 
-Ossie standardises what a metric *is*; this library enforces what an agent may *do* with it. So the spec is a strict subset of the vocabulary here — an Ossie `Metric` carries only `name`, `expression`, `description`, `datatype`, and `ai_context`.
+Ossie standardises what a metric *is*; this library enforces what an agent may *do* with it. So the spec is a strict subset of the vocabulary here — an Ossie `Metric` carries only `name`, `expression`, `description`, `datatype`, `ai_context`, and `custom_extensions`.
+
+A document is one model, with its `name`, `datasets`, `relationships` and `metrics` at the root, as the current spec defines it. A file in the earlier shape, with models in a `semantic_model` list, is still read and logs a warning saying it is legacy. Nothing in a model is dropped silently: a file with neither shape or a model with no datasets is refused at load, as is a dataset, field or metric with no name or a metric with no expression, rather than loading as less than it declares.
 
 | Ossie | Read as |
 | --- | --- |
-| `datasets[].source` + `fields[]` | Table schemas. A three-part `database.schema.table` is keyed on its trailing `schema.table`; a query-backed dataset registers no table |
-| `metrics[].expression.dialects[]` | `sql_expression`, resolved deterministically — your `dialect=` first, then `ANSI_SQL`, then `Ossie_SQL_2026`, then the first declared entry |
-| `relationships[]` | `Relationship`, with cardinality *derived from both sides* — see below |
+| `datasets[].source` + `fields[]` | Table schemas of *physical* columns. A field's column is the one its expression reads, in any identifier quoting: a field `customer_name` over `name` registers `name`, and a computed field (`subtotal - discount`) registers none. Because metric expressions still name fields, `describe_table` carries the bridge: the column's description ends `(Ossie field: customer_name)`, and the table's lists its computed fields as `net_total = subtotal - discount`. A `drill_by` in our vendor block names the physical column. A three-part `database.schema.table` is keyed on its trailing `schema.table`; a query-backed dataset registers no table |
+| `metrics[].expression` | `sql_expression`, resolved deterministically — your `dialect=` first, then `ANSI_SQL`, then Ossie SQL (`expression.sql`, which overrides a `dialects` entry for `OSSIE_SQL_2026`), then the first declared entry that is SQL. Dialect names are matched ignoring case. A formula language from Ossie's dialect registry (`TABLEAU`, `DAX`, `MDX`, `MAQL`, `SIGMA`, `THOUGHTSPOT`, `HOLISTICS_AQL`, `SPARQL`) is never the fallback, though you get it if you ask for it with `dialect=`. A metric written only in those keeps an empty `sql_expression`, and its `untranslated` reason, which `lookup_metric` shows the agent, quotes the formula |
+| `datasets[].metrics[]` | A metric named `dataset.metric`, as the spec references it |
+| `relationships[]` | `Relationship` between the physical columns its fields read, with cardinality *derived from both sides* — see below |
 | `ai_context` (anywhere) | Carried into `get_extras()["ossie_ai_context"]`, keyed by model then entity kind, never interpreted |
-| `custom_extensions[]` | Ours read as real vocabulary; every other vendor's carried into `get_extras()["ossie_custom_extensions"]`, keyed by model then vendor |
+| `custom_extensions[]` | Ours, at the model level, read as real vocabulary. Every other vendor's carried: model-level blocks into `get_extras()["ossie_custom_extensions"]` (by model, then vendor), and blocks on a dataset, field, metric or relationship into `get_extras()["ossie_entity_extensions"]` (by model, entity kind, entity name, then vendor) |
+
+Metric expressions, of either scope, are passed through unchanged. They name the model's datasets and fields, which may be renamed or computed, not physical columns, so Ossie gives a metric no `source_model`: set one in our vendor block when the expression does read one table as written.
 
 **Cardinality.** Ossie never writes the join type down; it is implied by which endpoints are keys. The spec documents `to` as the one side, but nothing validates that — and trusting it is not free, because `RelationshipChecker` fires its fan-out warning only on `one_to_many`. So both sides are checked:
 
@@ -966,41 +976,41 @@ Keys are optional in Ossie, and their absence is not evidence of fan-out — whe
 
 Composite-key relationships are **skipped with a warning** rather than split into one edge per column pair — our `Relationship` has single-column endpoints, and splitting would assert two joins that are each individually wrong. Declare those in a `YamlSource` overlay.
 
-**Governance fields.** Ownership, review dates, tiers, decompositions, and the metric-impact graph have no home in the Ossie spec, and every `$def` in its JSON Schema sets `additionalProperties: false` — so they cannot be added as extra keys. They ride in the spec's own escape hatch, `custom_extensions`, under this project's vendor name:
+**Governance fields.** Ownership, review dates, tiers, decompositions, and the metric-impact graph have no home in the Ossie spec, and every entity in its JSON Schema (model, dataset, field, metric, relationship) sets `additionalProperties: false` — so they cannot be added as extra keys. They ride in the spec's own escape hatch, `custom_extensions`, under this project's vendor name, in the model-level block. Metrics are keyed by name, a dataset-scoped one as `dataset.metric`:
 
 ```yaml
-    custom_extensions:
-      - vendor_name: AGENTIC_DATA_CONTRACTS
-        data: |
-          {
-            "metrics": {
-              "total_sales": {
-                "business_owner": "revenue-analytics",
-                "operational_owner": "data-platform",
-                "last_reviewed": "2026-05-01",
-                "tier": ["gold"],
-                "domains": ["sales"],
-                "decompositions": [
-                  {"operator": "sum", "operands": ["total_profit", "total_cost"]}
-                ],
-                "drill_by": [
-                  {"dimension": "region", "column": "public.customer.c_region"}
-                ]
-              }
-            },
-            "metric_impacts": [
-              {"from": "total_cost", "to": "total_profit",
-               "direction": "negative", "confidence": "verified",
-               "evidence": "Margin bridge reconciled 2026-Q1."}
+custom_extensions:
+  - vendor_name: AGENTIC_DATA_CONTRACTS
+    data: |
+      {
+        "metrics": {
+          "total_sales": {
+            "business_owner": "revenue-analytics",
+            "operational_owner": "data-platform",
+            "last_reviewed": "2026-05-01",
+            "tier": ["gold"],
+            "domains": ["sales"],
+            "decompositions": [
+              {"operator": "sum", "operands": ["total_profit", "total_cost"]}
+            ],
+            "drill_by": [
+              {"dimension": "region", "column": "public.customer.c_region"}
             ]
           }
+        },
+        "metric_impacts": [
+          {"from": "total_cost", "to": "total_profit",
+           "direction": "negative", "confidence": "verified",
+           "evidence": "Margin bridge reconciled 2026-Q1."}
+        ]
+      }
 ```
 
 Ossie stores extension payloads as a JSON *string*, so `data` is JSON nested inside YAML (a plain YAML mapping is accepted too). Restored decompositions and `drill_by` go through the same validators as `YamlSource`, failing loudly at load, and a bare `"tier": "gold"` is promoted to a list exactly as `YamlSource` does. Another vendor's malformed payload is carried verbatim and logged rather than raised on — a neighbour's typo must not stop your contract from being enforced.
 
 `expected_extras` does not apply to an Ossie source: its extras are synthesized by the parser under fixed keys, not authored as free top-level sections. Declaring it on a non-`yaml` source logs a warning rather than being silently dropped.
 
-> The spec is at `0.2.0.dev0` with no tagged releases yet, so expect churn. `dialect` is deliberately treated as an opaque string and never validated against the enum, since the accepted expression-language proposal adds `Ossie_SQL_2026` and makes it the default.
+> The spec is at `0.2.0.dev0` with no tagged releases yet, so expect churn. `dialect` is deliberately treated as an opaque string and never validated, as the spec itself accepts any non-empty string. Producers that export Ossie, such as NVIDIA Auto-Ontology's Import / Export, are read directly; `tests/fixtures/ossie_flat_nvidia_auto_ontology.yml` is that converter's real output. It is a record of what a producer emits, not a spec-valid example: its metric reads `orders.subtotal`, a column that no field declares, which the spec's name-resolution rules do not allow.
 
 ## Table Relationships
 
@@ -1990,6 +2000,8 @@ DATA_PLUGIN_PATH=/tmp/kwp/data \
 
 **Do I have to use dbt or Cube?** No. Author metrics inline in a `semantic.yml` with `YamlSource`. dbt (`manifest.json`), Cube, and Apache Ossie are supported if you already have them — the agent-facing behavior is identical regardless of source.
 
+**How does this compare to WrenAI?** [WrenAI](https://github.com/Canner/WrenAI) is a governed text-to-SQL *engine*: you model your data in its MDL semantic layer, and its Apache DataFusion-based engine plans each agent query against that model, dry-plan validates it, and executes it across its supported sources. This library has no engine. It checks the SQL the agent writes against a YAML contract, then passes the query to your own database through a `DatabaseAdapter` as written, and it reads metric definitions from the semantic source you already maintain (dbt, Cube, Apache Ossie, or inline YAML). Pick Wren if you want one engine to own both modelling and execution; pick this library if your warehouse and semantic layer stay as they are and you need governance in front of them. One licensing difference: per-caller table and column rules ([`caller_principal`](#per-caller-access-control-optional)) are part of this MIT library, while Wren lists row- and column-level security as a commercial feature.
+
 **Does it work without the Claude Agent SDK?** Yes. The tools are plain async functions usable from LangChain/deepagents, Pydantic AI, or directly; the example agents fall back to a no-SDK demo mode.
 
 **Is it production-ready?** It's pre-1.0 and actively evolving. Every breaking change is documented in [`CHANGELOG.md`](CHANGELOG.md) with migration notes.

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/README.md

@@ -42,14 +42,17 @@ That table is the claim; [`experiments/dabstep-contract-eval/`](experiments/dabs
 | `manual_prompt` | the vendor manual verbatim in the prompt | 22.9% | 42.9% | 50.3% | 23.8% |
 | `schema_only` | table and column names only | 13.9% | 22.6% | 37.0% | 22.9% |
 
-- **The gain is the content, not the scaffolding.** `contract_hollow` runs identical tooling with the domain knowledge stripped out. That is the only difference between the two arms, and on every model tested it explains more of the contract's lead than the tooling does.
-- **The contract is also the cheapest way to be right.** On every model the contract arm uses the fewest turns and by far the least reasoning; on three of four it also makes the fewest tool calls and is cheapest per correct answer (on Sonnet 5, about half the cost of any other arm). The exception is `gpt-5.6-sol`, where the contract's own text is a real input-token bill and `manual_prompt` edges it. It is the only arm that never had to be forced to answer. It is not thinking harder; it has less to search for.
-- **Governance held.** Across the ungoverned arms the models submitted 166 mutating SQL statements on 26 tasks. Both governed arms submitted zero.
-- **No leakage.** The contract was written from the vendor's manual alone and frozen, digest-pinned, before any benchmark question was read.
+- **The gain is the content, not the scaffolding.** `contract_hollow` runs identical tooling with the domain knowledge stripped out. That is the only difference between the two arms, and on every model tested it explains more of the contract's lead than the tooling does. Repeated three times on two more models, emptying the contract costs 26-50 hard-task points; the tools alone are worth -9 to +15.
+- **Most of the lead over the manual was knowledge the manual lacks.** Follow-up experiments found two facts the vendor manual never states (an empty list means "all", and the fees of overlapping rules add up). Stated as two notes in the prompt, they recover most of the contract's lead; with the facts stated alike, the contract's format adds about 4-6 points on the fee tasks. Write the conventions down; that is where the accuracy is.
+- **Delivery buys cost and an audit trail, not much accuracy.** For one database the contract in a cached prompt is as cheap or cheaper than looking it up. From about twice that size, lookup is cheaper (1.7-2.4x at four databases' worth, 6.4x at 22) and keeps working past the context window, and the lookup log names the definition behind each answer.
+- **Stated rules deterred writes.** Across the ungoverned arms of the single runs the models submitted 166 mutating SQL statements on 26 tasks; the governed arms submitted none. Across every later DABStep run, no run whose prompt stated the forbidden operations corrupted its database copy, whether the rules came through the tools or the contract's text. Serve the agent through a read-only role regardless.
+- **Frozen before scoring.** The contract was written in one interactive session, a model drafting under the author's direction, from the vendor's manual, the dataset README and the data, and frozen, digest-pinned, before any benchmark score existed; the paper gives the full timeline and what it cannot prove.
 - **Externally scored.** The contract arm answered all 450 tasks and was submitted to the [DABStep leaderboard](https://huggingface.co/spaces/adyen/DABstep), where Adyen graded it against the official golds: **51.9% on the hard split** for `glm-5.3-flash`, against a published band of ~20–26% hard for manual-in-prompt baselines running stronger models.
 
 Scope, stated plainly. The effect concentrates in the domain-specific task bucket; the non-fee bucket shows no contract advantage on any model. The table's golds are reconstructed for 401 of 450 tasks, and the leaderboard check shows that scoring runs ~6 points optimistic on hard tasks, one-sidedly; correcting for it moves the arm gaps by under 3 points, in both directions. Only the contract arm was submitted, so the gap above is not graded the same way on both sides. And this is one benchmark at k=1, where a replicate finds 23% of task verdicts flip at temperature 0. Full results, every caveat, and all 6,416 transcripts: **[`FINDINGS.md`](experiments/dabstep-contract-eval/FINDINGS.md)**.
 
+Archived on Zenodo for the paper: the code at release `dabstep-eval-2026-11`, [10.5281/zenodo.24015265](https://doi.org/10.5281/zenodo.24015265), and the transcripts of releases `dabstep-eval-2026-10` and `dabstep-eval-2026-11`, [10.5281/zenodo.24015263](https://doi.org/10.5281/zenodo.24015263). Paper v2 used the `dabstep-eval-2026-10` versions: code [10.5281/zenodo.23598672](https://doi.org/10.5281/zenodo.23598672), transcripts [10.5281/zenodo.23598750](https://doi.org/10.5281/zenodo.23598750).
+
 **Works with:** any Python agent framework — first-class helpers for the [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python), [LangChain](https://github.com/langchain-ai/langchain) / [deepagents](https://github.com/langchain-ai/deepagents), and [Pydantic AI](https://github.com/pydantic/pydantic-ai), plus a framework-free path (the tools are plain async functions). Optionally integrates with [ai-agent-contracts](https://pypi.org/project/ai-agent-contracts/) for formal resource governance.
 
 > **See it running:** [three example agents](#examples) — `revenue_agent` (finance), `growth_agent` (experimentation), `ops_agent` (SRE) — each runs end-to-end in demo mode with no API key.
@@ -629,7 +632,7 @@ Either way, the agent knows that "revenue is recognized at fulfillment, not at b
 
 ### Why progressive disclosure works
 
-This pattern — compact index in the prompt, detailed context on demand — is the same philosophy validated by agent skill systems, MCP tool servers, and RAG architectures. Instead of overloading the agent's context window with everything upfront, you give it just enough to know *where to look*, then let it pull details when needed. The result is better token efficiency, more focused reasoning, and fewer hallucinations from context overload.
+This pattern -- compact index in the prompt, detailed context on demand -- is the same philosophy behind agent skill systems, MCP tool servers, and RAG architectures. Measured on our benchmarks, its payoff is cost and traceability rather than accuracy: a contract several databases large in the prompt lost no accuracy on questions that name their metric, but on-demand lookup kept the cost per question nearly flat as the contract grew to 22 databases, ran where the full contract no longer fit the context window, and logged which definition each answer used.
 
 ## Contract Rules
 
@@ -651,6 +654,8 @@ Each rule carries a `query_check` (pre-execution) or `result_check` (post-execut
 | `require_limit` | Require a LIMIT clause |
 | `max_joins` | Cap the number of JOINs |
 
+Query checks read the SQL text before it runs. They keep a cooperating agent from doing by mistake what a rule forbids, and tell it why, so it can correct the query. They are not an access-control boundary: SQL has more ways to route a value into a result than a static check can recognise, so a query written to evade `blocked_columns` can still return a blocked column's values. If a column must never reach a given caller, enforce that in the database as well, with column privileges or a view that leaves it out, and connect the agent with credentials that have only that access.
+
 **Built-in result checks** (post-execution, validated against query output):
 
 | Check | Description |
@@ -871,15 +876,20 @@ semantic:
     path: "./semantic/model.yml"
 ```
 
-Ossie standardises what a metric *is*; this library enforces what an agent may *do* with it. So the spec is a strict subset of the vocabulary here — an Ossie `Metric` carries only `name`, `expression`, `description`, `datatype`, and `ai_context`.
+Ossie standardises what a metric *is*; this library enforces what an agent may *do* with it. So the spec is a strict subset of the vocabulary here — an Ossie `Metric` carries only `name`, `expression`, `description`, `datatype`, `ai_context`, and `custom_extensions`.
+
+A document is one model, with its `name`, `datasets`, `relationships` and `metrics` at the root, as the current spec defines it. A file in the earlier shape, with models in a `semantic_model` list, is still read and logs a warning saying it is legacy. Nothing in a model is dropped silently: a file with neither shape or a model with no datasets is refused at load, as is a dataset, field or metric with no name or a metric with no expression, rather than loading as less than it declares.
 
 | Ossie | Read as |
 | --- | --- |
-| `datasets[].source` + `fields[]` | Table schemas. A three-part `database.schema.table` is keyed on its trailing `schema.table`; a query-backed dataset registers no table |
-| `metrics[].expression.dialects[]` | `sql_expression`, resolved deterministically — your `dialect=` first, then `ANSI_SQL`, then `Ossie_SQL_2026`, then the first declared entry |
-| `relationships[]` | `Relationship`, with cardinality *derived from both sides* — see below |
+| `datasets[].source` + `fields[]` | Table schemas of *physical* columns. A field's column is the one its expression reads, in any identifier quoting: a field `customer_name` over `name` registers `name`, and a computed field (`subtotal - discount`) registers none. Because metric expressions still name fields, `describe_table` carries the bridge: the column's description ends `(Ossie field: customer_name)`, and the table's lists its computed fields as `net_total = subtotal - discount`. A `drill_by` in our vendor block names the physical column. A three-part `database.schema.table` is keyed on its trailing `schema.table`; a query-backed dataset registers no table |
+| `metrics[].expression` | `sql_expression`, resolved deterministically — your `dialect=` first, then `ANSI_SQL`, then Ossie SQL (`expression.sql`, which overrides a `dialects` entry for `OSSIE_SQL_2026`), then the first declared entry that is SQL. Dialect names are matched ignoring case. A formula language from Ossie's dialect registry (`TABLEAU`, `DAX`, `MDX`, `MAQL`, `SIGMA`, `THOUGHTSPOT`, `HOLISTICS_AQL`, `SPARQL`) is never the fallback, though you get it if you ask for it with `dialect=`. A metric written only in those keeps an empty `sql_expression`, and its `untranslated` reason, which `lookup_metric` shows the agent, quotes the formula |
+| `datasets[].metrics[]` | A metric named `dataset.metric`, as the spec references it |
+| `relationships[]` | `Relationship` between the physical columns its fields read, with cardinality *derived from both sides* — see below |
 | `ai_context` (anywhere) | Carried into `get_extras()["ossie_ai_context"]`, keyed by model then entity kind, never interpreted |
-| `custom_extensions[]` | Ours read as real vocabulary; every other vendor's carried into `get_extras()["ossie_custom_extensions"]`, keyed by model then vendor |
+| `custom_extensions[]` | Ours, at the model level, read as real vocabulary. Every other vendor's carried: model-level blocks into `get_extras()["ossie_custom_extensions"]` (by model, then vendor), and blocks on a dataset, field, metric or relationship into `get_extras()["ossie_entity_extensions"]` (by model, entity kind, entity name, then vendor) |
+
+Metric expressions, of either scope, are passed through unchanged. They name the model's datasets and fields, which may be renamed or computed, not physical columns, so Ossie gives a metric no `source_model`: set one in our vendor block when the expression does read one table as written.
 
 **Cardinality.** Ossie never writes the join type down; it is implied by which endpoints are keys. The spec documents `to` as the one side, but nothing validates that — and trusting it is not free, because `RelationshipChecker` fires its fan-out warning only on `one_to_many`. So both sides are checked:
 
@@ -894,41 +904,41 @@ Keys are optional in Ossie, and their absence is not evidence of fan-out — whe
 
 Composite-key relationships are **skipped with a warning** rather than split into one edge per column pair — our `Relationship` has single-column endpoints, and splitting would assert two joins that are each individually wrong. Declare those in a `YamlSource` overlay.
 
-**Governance fields.** Ownership, review dates, tiers, decompositions, and the metric-impact graph have no home in the Ossie spec, and every `$def` in its JSON Schema sets `additionalProperties: false` — so they cannot be added as extra keys. They ride in the spec's own escape hatch, `custom_extensions`, under this project's vendor name:
+**Governance fields.** Ownership, review dates, tiers, decompositions, and the metric-impact graph have no home in the Ossie spec, and every entity in its JSON Schema (model, dataset, field, metric, relationship) sets `additionalProperties: false` — so they cannot be added as extra keys. They ride in the spec's own escape hatch, `custom_extensions`, under this project's vendor name, in the model-level block. Metrics are keyed by name, a dataset-scoped one as `dataset.metric`:
 
 ```yaml
-    custom_extensions:
-      - vendor_name: AGENTIC_DATA_CONTRACTS
-        data: |
-          {
-            "metrics": {
-              "total_sales": {
-                "business_owner": "revenue-analytics",
-                "operational_owner": "data-platform",
-                "last_reviewed": "2026-05-01",
-                "tier": ["gold"],
-                "domains": ["sales"],
-                "decompositions": [
-                  {"operator": "sum", "operands": ["total_profit", "total_cost"]}
-                ],
-                "drill_by": [
-                  {"dimension": "region", "column": "public.customer.c_region"}
-                ]
-              }
-            },
-            "metric_impacts": [
-              {"from": "total_cost", "to": "total_profit",
-               "direction": "negative", "confidence": "verified",
-               "evidence": "Margin bridge reconciled 2026-Q1."}
+custom_extensions:
+  - vendor_name: AGENTIC_DATA_CONTRACTS
+    data: |
+      {
+        "metrics": {
+          "total_sales": {
+            "business_owner": "revenue-analytics",
+            "operational_owner": "data-platform",
+            "last_reviewed": "2026-05-01",
+            "tier": ["gold"],
+            "domains": ["sales"],
+            "decompositions": [
+              {"operator": "sum", "operands": ["total_profit", "total_cost"]}
+            ],
+            "drill_by": [
+              {"dimension": "region", "column": "public.customer.c_region"}
             ]
           }
+        },
+        "metric_impacts": [
+          {"from": "total_cost", "to": "total_profit",
+           "direction": "negative", "confidence": "verified",
+           "evidence": "Margin bridge reconciled 2026-Q1."}
+        ]
+      }
 ```
 
 Ossie stores extension payloads as a JSON *string*, so `data` is JSON nested inside YAML (a plain YAML mapping is accepted too). Restored decompositions and `drill_by` go through the same validators as `YamlSource`, failing loudly at load, and a bare `"tier": "gold"` is promoted to a list exactly as `YamlSource` does. Another vendor's malformed payload is carried verbatim and logged rather than raised on — a neighbour's typo must not stop your contract from being enforced.
 
 `expected_extras` does not apply to an Ossie source: its extras are synthesized by the parser under fixed keys, not authored as free top-level sections. Declaring it on a non-`yaml` source logs a warning rather than being silently dropped.
 
-> The spec is at `0.2.0.dev0` with no tagged releases yet, so expect churn. `dialect` is deliberately treated as an opaque string and never validated against the enum, since the accepted expression-language proposal adds `Ossie_SQL_2026` and makes it the default.
+> The spec is at `0.2.0.dev0` with no tagged releases yet, so expect churn. `dialect` is deliberately treated as an opaque string and never validated, as the spec itself accepts any non-empty string. Producers that export Ossie, such as NVIDIA Auto-Ontology's Import / Export, are read directly; `tests/fixtures/ossie_flat_nvidia_auto_ontology.yml` is that converter's real output. It is a record of what a producer emits, not a spec-valid example: its metric reads `orders.subtotal`, a column that no field declares, which the spec's name-resolution rules do not allow.
 
 ## Table Relationships
 
@@ -1918,6 +1928,8 @@ DATA_PLUGIN_PATH=/tmp/kwp/data \
 
 **Do I have to use dbt or Cube?** No. Author metrics inline in a `semantic.yml` with `YamlSource`. dbt (`manifest.json`), Cube, and Apache Ossie are supported if you already have them — the agent-facing behavior is identical regardless of source.
 
+**How does this compare to WrenAI?** [WrenAI](https://github.com/Canner/WrenAI) is a governed text-to-SQL *engine*: you model your data in its MDL semantic layer, and its Apache DataFusion-based engine plans each agent query against that model, dry-plan validates it, and executes it across its supported sources. This library has no engine. It checks the SQL the agent writes against a YAML contract, then passes the query to your own database through a `DatabaseAdapter` as written, and it reads metric definitions from the semantic source you already maintain (dbt, Cube, Apache Ossie, or inline YAML). Pick Wren if you want one engine to own both modelling and execution; pick this library if your warehouse and semantic layer stay as they are and you need governance in front of them. One licensing difference: per-caller table and column rules ([`caller_principal`](#per-caller-access-control-optional)) are part of this MIT library, while Wren lists row- and column-level security as a commercial feature.
+
 **Does it work without the Claude Agent SDK?** Yes. The tools are plain async functions usable from LangChain/deepagents, Pydantic AI, or directly; the example agents fall back to a no-SDK demo mode.
 
 **Is it production-ready?** It's pre-1.0 and actively evolving. Every breaking change is documented in [`CHANGELOG.md`](CHANGELOG.md) with migration notes.

{agentic_data_contracts-0.62.0 → agentic_data_contracts-0.63.0}/docs/architecture.md

@@ -893,7 +893,7 @@ class SemanticSource(Protocol):
 
 **Cube relationship parsing (v0.18.0+):** `CubeSource.get_relationships()` parses each cube's `joins:` block. The parser builds a `cube_name -> sql_table` map, regexes the single-equality form `{X}.col1 = {Y}.col2` from each join's `sql:` field, and normalises so the `from` side is always the column on the cube declaring the join (independent of which side `{CUBE}` appears on in the SQL). Cube's `relationship` enum (`belongsTo` / `hasOne` / `hasMany` plus `many_to_one` / `one_to_one` / `one_to_many` aliases) maps to canonical `Relationship.type` strings; `meta.relationship_type` overrides. `meta.preferred` and `meta.required_filter` work the same way as `YamlSource` and `DbtSource`. Composite-key joins (multiple `AND`-chained equalities) and joins whose target cube can't be resolved by name are skipped — declare those in contract YAML via `YamlSource` instead.
 
-**Ossie parsing (v0.42.0+):** `OssieSource` reads an [Apache Ossie (incubating)](https://ossie.apache.org/) semantic model — the vendor-neutral spec, formerly Open Semantic Interchange, now governed by the ASF. The spec is a strict subset of this library's vocabulary: an Ossie `Metric` is `name` + multi-dialect `expression` + `description` + `datatype` + `ai_context`, and nothing else. Four decisions define the adapter.
+**Ossie parsing (v0.42.0+):** `OssieSource` reads an [Apache Ossie (incubating)](https://ossie.apache.org/) semantic model — the vendor-neutral spec, formerly Open Semantic Interchange, now governed by the ASF. The spec is a strict subset of this library's vocabulary: an Ossie `Metric` is `name` + `expression` (portable `sql` and/or per-dialect entries) + `description` + `datatype` + `ai_context` + `custom_extensions`, and nothing else. Four decisions define the adapter.
 
 *Table keys drop the database qualifier.* Ossie sources are `database.schema.table`; our keys are two-part because `Relationship` endpoints are `schema.table.column` and `build_relationship_index` recovers the table with a single `rsplit`. A three-part key would make every endpoint four deep and break that contract. Collisions (two databases sharing a schema and table name) are logged. Query-backed datasets register no table at all.
 
@@ -901,11 +901,17 @@ class SemanticSource(Protocol):
 
 *Composite joins are skipped, not split.* Ossie relationships carry parallel `from_columns` / `to_columns` lists; our `Relationship` has single-column endpoints. Emitting one edge per column pair would assert two joins that are each individually wrong, and the join planner walks those edges — so a composite relationship is skipped with a warning naming it, matching the precedent `CubeSource` set for Cube's `AND`-chained joins.
 
-*Governance rides in `custom_extensions`.* Every `$def` in `osi-schema.json` sets `additionalProperties: false` and `version` is a `const`, so ownership, review dates, tiers, `decompositions`, `drill_by`, and the impact graph cannot be added as extra keys. They live under the `AGENTIC_DATA_CONTRACTS` vendor block, whose `data` is a JSON *string* per the spec. Restored `decompositions` / `drill_by` pass through the same `validate_decompositions` / `validate_drill_by` as `YamlSource`. Scalar `tier` / `domains` / `filters` are promoted to one-element lists exactly as `YamlSource` does, since `list("gold")` would otherwise yield `['g', 'o', 'l', 'd']` and then drive tier policy.
+*Governance rides in `custom_extensions`.* Every entity in `ossie-schema.json` (model, dataset, field, metric, relationship) sets `additionalProperties: false` and `version` is a `const`, so ownership, review dates, tiers, `decompositions`, `drill_by`, and the impact graph cannot be added as extra keys. They live under the `AGENTIC_DATA_CONTRACTS` vendor block, whose `data` is a JSON *string* per the spec. Restored `decompositions` / `drill_by` pass through the same `validate_decompositions` / `validate_drill_by` as `YamlSource`. Scalar `tier` / `domains` / `filters` are promoted to one-element lists exactly as `YamlSource` does, since `list("gold")` would otherwise yield `['g', 'o', 'l', 'd']` and then drive tier policy.
 
-Every *other* vendor's block, plus every `ai_context` in the file, is carried through `get_extras()` under `ossie_custom_extensions` / `ossie_ai_context` — the same boundary `YamlSource` draws, where the framework carries extras and renders them on request but never interprets them. Both are keyed by **semantic-model name first**: Ossie's top level is a list and it namespaces entity names per model, so a flat key would silently drop the first of two `customer` datasets or two blocks from the same vendor. Both sections are omitted entirely when empty rather than emitted as `{}`, which would add a noise key to every Ossie contract's canonical bytes and trip a contract declaring `expected_extras` on rehydrate; both are put through the shared `jsonify_extras` (promoted from `yaml_source` alongside `parse_review_date`), so a YAML-native date in an `ai_context` cannot reach `contract_canonical_bytes` unconverted. A foreign vendor's malformed JSON is carried verbatim and logged rather than raised on.
+Every *other* vendor's block, plus every `ai_context` in the file, is carried through `get_extras()` under `ossie_custom_extensions` (model-level blocks), `ossie_entity_extensions` (blocks on a dataset, field, metric or relationship, keyed by entity kind and name) and `ossie_ai_context` — the same boundary `YamlSource` draws, where the framework carries extras and renders them on request but never interprets them. Our own block is read only at the model level, where it is keyed by metric name; one found on an entity is named in a warning and ignored. All three are keyed by **model name first**: a legacy document's top level is a list and Ossie namespaces entity names per model, so a flat key would silently drop the first of two `customer` datasets or two blocks from the same vendor. Both sections are omitted entirely when empty rather than emitted as `{}`, which would add a noise key to every Ossie contract's canonical bytes and trip a contract declaring `expected_extras` on rehydrate; both are put through the shared `jsonify_extras` (promoted from `yaml_source` alongside `parse_review_date`), so a YAML-native date in an `ai_context` cannot reach `contract_canonical_bytes` unconverted. A foreign vendor's malformed JSON is carried verbatim and logged rather than raised on.
 
-The spec is `0.2.0.dev0` with no tagged releases (apache/ossie#102), and the accepted expression-language proposal introduces an `Ossie_SQL_2026` dialect absent from today's enum. `dialect` is therefore treated as an opaque string and never validated against the enum: a closed check would break on the next spec bump for no benefit. Dialect resolution is deterministic, as the spec requires — caller's choice, then `ANSI_SQL`, then `Ossie_SQL_2026`, then the first declared entry, so a model written only in a vendor dialect still yields an expression rather than a silent empty string.
+The spec is `0.2.0.dev0` with no tagged releases (apache/ossie#102), and it changes underneath us. `dialect` is treated as an opaque string and never validated, which the spec now also does: any non-empty string is accepted. Dialect resolution is deterministic, as the spec requires — caller's choice, then `ANSI_SQL`, then Ossie SQL, then the first declared entry, so a model written only in a vendor dialect still yields an expression rather than a silent empty string. `ANSI_SQL` stays ahead of Ossie SQL because the expression goes to an agent that runs it on a warehouse, and ANSI is the declared dialect closest to that; Ossie SQL is the spec's own portable language, which a consumer is expected to translate. The spec allows either ranking, as long as it is deterministic. Ossie SQL is `expression.sql` (apache/ossie#617), which overrides a `dialects` entry for `OSSIE_SQL_2026`. Dialect names are compared case-folded, the caller's included, so the proposal's `Ossie_SQL_2026` spelling and the spec's name one dialect. The last resort, the first declared entry, skips the formula languages in Ossie's dialect registry (`TABLEAU`, `DAX`, `MDX`, `MAQL`, `SIGMA`, `THOUGHTSPOT`, `HOLISTICS_AQL`, `SPARQL`): handed to the agent as SQL, `SUM([Profit])` is a query it would run or try to repair. The list names what is known not to be SQL rather than what is, because a miss is cheaper that way round: an unregistered vendor dialect such as `DENODO` may well be SQL and is still a fallback. A metric written only in formula languages is valid Ossie with no SQL to run, so it is kept the way `DbtSource` and `CubeSource` keep a metric they cannot express (#123): an empty `sql_expression` and an `untranslated` reason quoting each formula with its dialect. A field written only in one registers no column, like a computed field.
+
+**Ossie document shape (v0.63.0+).** apache/ossie#383 made a document one model, with `name` and `datasets` at the root, and the schema now rejects the earlier `semantic_model` list. `OssieSource` read only that list, so a current-spec document, such as NVIDIA Auto-Ontology's export, loaded as a source with no tables, metrics or relationships, and nothing reported it. Both shapes are now read, the legacy one with a warning. Every way of finding no model is an error: a document with neither shape, an empty list, both shapes at once (where reading either would drop the other's models), or a model with no datasets. An empty source enforces nothing the agent can see, and nothing downstream can tell it apart from a model that simply has no metrics. The same rule reaches inside a model: a dataset, field or metric with no name, or a metric with no expression, is refused at load instead of being skipped, and a vendor-block key that matches no metric, a repeated vendor block and a duplicate metric name are each named in a warning.
+
+Dataset-scoped metrics (apache/ossie#591) are named `dataset.metric`, the form the spec references them by, which also keeps them apart from a model-scoped metric of the same bare name. A vendor-block key written in the bare form is named in a warning that suggests the scoped one, since moving a metric onto its dataset renames it. Neither scope gets a `source_model` from Ossie: without a `from_sql`, a `source_model` says the expression reads that table alone, and an Ossie expression names datasets and fields, which may be renamed or computed. Expressions are passed through unchanged; translating them to physical SQL is not done here.
+
+*Fields map to physical columns.* A field is a name over an expression, so the table schema registered for a dataset holds the columns those expressions read: a field `customer_name` over `name` registers `name`, and a computed field registers no column. Relationship endpoints are mapped the same way, and a relationship joining on a computed field is skipped with a warning. The column is read with sqlglot, plain SQL first and then BigQuery and T-SQL for backtick and bracket quoting; whichever first reads the expression as a column names it, so `current_date`, `TRUE` and `NULL` register none, and a tokenizer error means no column rather than a failed load. Metric expressions still name fields, and translating them would mean generating SQL, which a consumer whose dialect sqlglot cannot emit (Denodo's VQL) cannot use. So the field vocabulary is carried to where the agent reads the table instead: a column named apart from its field ends its description `(Ossie field: customer_name)`, and the table's description lists each computed field as `name = expression`. Before v0.63.0 the field name was registered as the column, which no test caught because every fixture named its fields after their columns; NVIDIA Auto-Ontology's export does not.
 
 **Metric-first domain membership (v0.26.0+):** A metric declares the domains it belongs to via `MetricDefinition.domains`, populated from `meta.domains` by every adapter (`YamlSource`, `DbtSource`, `CubeSource`). The contract's `Domain` model carries only catalog metadata (summary, description, owners, `last_reviewed`) and does **not** list its metrics; it sets `extra="forbid"`, so a stale `metrics:` key from a pre-0.26 contract fails loudly at load time instead of being silently dropped. To enumerate a domain's members, callers reverse-look-up with the `metrics_in_domain(metrics, domain_name)` helper in `base.py` (used by `lookup_domain` and the `list_metrics(domain=...)` filter); per-domain `metric_count` in the prompt index and the `lookup_domain` fuzzy fallback is tallied in a single `Counter` pass over the metrics. This makes the metric the single source of truth for membership: the contract and the semantic source cannot disagree, because only one of them states it. (Prior to v0.26.0 the contract also carried a `Domain.metrics` list, reconciled with `metric.domains` by a union shim; that redundancy — and the drift it allowed — has been removed.)