terrine 0.1.0.dev0__tar.gz → 0.1.0.dev2__tar.gz

{terrine-0.1.0.dev0 → terrine-0.1.0.dev2}/.github/workflows/ci.yml

@@ -72,6 +72,10 @@ jobs:
           --splits 6 --group ${{ matrix.group }}
           --splitting-algorithm least_duration
           --durations-path integration/.test_durations
+      # The same known answers compiled through the portable base expression
+      # policy; about twenty seconds beside the native run.
+      - if: matrix.group == 1
+        run: uv run pytest -q integration/conformance --generic-dialect
   docs-and-examples:
     runs-on: ubuntu-latest
     timeout-minutes: 30

{terrine-0.1.0.dev0 → terrine-0.1.0.dev2}/.gitignore

@@ -14,6 +14,7 @@ dist/
 build/
 *.egg-info/
 .superpowers/
+docs/superpowers/
 .worktrees/
 
 # Local mutation campaigns

{terrine-0.1.0.dev0 → terrine-0.1.0.dev2}/AGENTS.md

@@ -94,12 +94,30 @@ Reached schedules and deadlines use notify.* for the current owner, or the autho
   random sample without replacement; the default executes every mutant.
   Campaigns are local only, not part of ordinary checks or CI.
 - `make test-postgres` runs the live PostgreSQL probes (`@pytest.mark.postgres`,
-  deselected by default); they fail, never skip, without
+  deselected by default) and then the conformance known answers with the
+  portable base expression policy forced; they fail, never skip, without
   `TERRINE_TEST_POSTGRES_*` settings. Start a throwaway local server with
   `eval "$(uv run python -m tests.integration.postgres_scratch start)"` and stop it
   with `uv run python -m tests.integration.postgres_scratch stop`. CI runs them
   on every push against a PostgreSQL 16 service container. Each probe owns a
   uniquely named schema; never share a warehouse namespace between tests.
+- `uv run pytest integration/conformance` runs known answers on DuckDB by default,
+  with parameters derived from the verified engine registry. Select the same cases
+  on a configured scratch warehouse with
+  `uv run --extra <engine> pytest integration/conformance -m <engine>` for
+  `postgres`, `bigquery` or `snowflake`; they are also included in the matching
+  `make test-<engine>` target. The module's connection-fixture table must cover
+  every registered engine. Add `--generic-dialect` to compile the same known
+  answers with the portable base expression policy on each real connection
+  (`uv run pytest integration/conformance --generic-dialect`; with
+  `--extra postgres ... -m postgres` on the scratch server). The forced run
+  takes about twenty seconds per backend and is part of the integration and
+  PostgreSQL checks on every push; the portable policy's own numerical and
+  relational oracles live in `integration/backends/test_generic.py` and force
+  it unconditionally, and its real unregistered-backend reads on Ibis SQLite
+  live in `integration/execution/test_generic_provider.py`. Test module
+  basenames under `integration/` must be unique: the tree has no packages, so
+  two `test_x.py` files cannot both be collected.
 - `make test-bigquery` runs the live BigQuery probes (`@pytest.mark.bigquery`,
   deselected by default); they fail, never skip, without
   `TERRINE_TEST_BIGQUERY_PROJECT`, `TERRINE_TEST_BIGQUERY_DATASET` and
@@ -242,16 +260,47 @@ Reached schedules and deadlines use notify.* for the current owner, or the autho
   raw SQL strings or `@ibis.udf.*.builtin` bindings to native functions there;
   ruff (`TID251`) rejects `ibis.udf` and `ibis.to_sql` outside the allowed
   packages.
-- When Ibis lacks or mistranslates an operation on a supported backend, add a
-  dispatched helper in `terrine/backends/` (`dialect`, `numeric`, `timestamps`,
-  `joins`, `extrema`) with a live parity test, not a workaround at each call
-  site. Supporting a new warehouse means implementing those helpers for it.
-  Known Ibis gaps: no ASOF JOIN on BigQuery, NULL-propagating LEAST/GREATEST
-  there, and `interval * column` lowered through `EXTRACT(SECOND ...)` (multiply
-  integer seconds, then `.as_interval("s")`).
+- When Ibis lacks or mistranslates an operation on a supported backend, implement
+  the override in that engine's `terrine/backends/engines/<name>.py` dialect,
+  with a live parity test, not a workaround at each call site. Shared helpers
+  (`numeric`, `timestamps`, `joins`, `extrema`, `sequences`) delegate engine
+  differences to the selected dialect. Native function bindings belong in
+  concrete engine modules. Known Ibis gaps: no ASOF JOIN on BigQuery,
+  NULL-propagating LEAST/GREATEST there, and `interval * column` lowered through
+  `EXTRACT(SECOND ...)` (multiply integer seconds, then `.as_interval("s")`).
 - Raw SQL is acceptable only for: terrine's own DuckDB runtime state tables
   (`terrine/runtime/`), backend-provider catalog introspection and transaction
-  control (`terrine/execution/`), and admitted user SQL relations.
+  control (`terrine/execution/`), admitted user SQL relations, and the existing
+  qualified read-only PostgreSQL identity callback in `terrine/backends/engines/`.
+- `terrine/backends/registry.py` owns verified engine names, capability values,
+  feature boundaries and lazy dialect, identity and execution-provider paths.
+  Add an engine through that registry and one concrete dialect override module,
+  with admission evidence; do not introduce supported-engine lists in consumers.
+  Capability inspection is metadata-only and never runs querying identity admission.
+  `tests/test_engine_dispatch.py` rejects engine-name selection outside those
+  modules; it is a heuristic review aid, not proof of centralized dispatch.
+- Backends have three tiers. Verified engines are the `ENGINES` registrations
+  with native dialect overrides and consistent-view providers. Any other
+  genuine Ibis backend is unverified: it refuses by default and, only after a
+  Model's `allow_unverified=True`, runs read-only statement-scoped reads under
+  the separate `GENERIC` record, whose expression policy is the unmodified
+  base `Dialect` in `terrine/backends/dialect.py` and whose provider is
+  `terrine/execution/_generic.py`. Raw drivers, name-only objects and failed
+  registered identities stay refused with or without the flag, as do every
+  effect (staging, retained capture, publication, reopening) on an unverified
+  backend. Extend the portable tier by completing the base `Dialect` method
+  or adding an operation-specific coded refusal there, never by naming a
+  backend; a base default must not advertise an operation a registered
+  engine's `operations` declaration excludes, so add the engine override that
+  keeps its refusal. The portable policy answers exactly or refuses by code:
+  lazy scalar known-answer witnesses (exact decimals, UTC wall clocks across
+  daylight saving) run once per execution context before source rows are
+  read, and accumulations are withheld by a derived magnitude bound rather
+  than trusted to the backend.
+- The PostgreSQL engine's identity callback owns the existing qualified,
+  read-only `SELECT pg_catalog.version()` admission exception. It checks the
+  Ibis floor before reading identity and refuses before source metadata access;
+  expression policies otherwise issue no connection probes.
 - Warehouse results match the DuckDB reference within a stated tolerance
   (floats: tight, relative to the window's absolute sum, since cancellation
   defeats a purely relative bound; exact for counts and decimal sums).