atif-sql 0.3.0__tar.gz → 0.5.0__tar.gz

{atif_sql-0.3.0 → atif_sql-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: atif-sql
-Version: 0.3.0
+Version: 0.5.0
 Summary: ATIF-native analytics over Claude Code agent trajectories: convert, materialize, and query agent sessions with DuckDB.
 Project-URL: Homepage, https://github.com/laithalsaadoon/atif-sql
 Project-URL: Repository, https://github.com/laithalsaadoon/atif-sql
@@ -21,7 +21,7 @@ Requires-Dist: anthropic>=1.11.0
 Requires-Dist: anyio>=4.11.0
 Requires-Dist: boto3>=1.42.91
 Requires-Dist: cyclopts>=5.1.1
-Requires-Dist: duckdb<2,>=1.5.2
+Requires-Dist: duckdb<3,>=2.0.0
 Requires-Dist: imagesize>=2.0
 Requires-Dist: lancedb<0.40,>=0.30
 Requires-Dist: loguru>=0.7.3
@@ -149,7 +149,7 @@ the only thing documented as installable is the `atif-sql` CLI above.
 | `atif-converter` | Claude Code and Codex CLI transcript → ATIF converters (ours, built on Harbor's ATIF models, vendored) + per-agent fidelity policy (loss accounting per session) |
 | `atif-corpus` | Corpus materialization: discovery, watermarks, quiescence, atomic artifact writes |
 | `atif-duck` | DuckDB views + macros over the materialized corpus (core surface plus the v2 analytics surface) |
-| `atif-models` | Model alias registry + structured-output LLM clients for GPT-6 and Claude 5.5 on Bedrock; no other package hardcodes a model id |
+| `atif-models` | Model alias registry + structured-output LLM clients for GPT-6 and Claude 5.5 on Bedrock; no other package hardcodes an LLM model id |
 | `atif-analytics` | the LLM pipelines `atif-sql analyze` runs: classify, conflicts, friction, and perceived |
 | `atif-embed` | Cohere Embed v4 on Bedrock or EmbeddingGemma 2 on this machine + LanceDB vector store + embedding backfill |
 | `atif-cli` | the composition root, and the source of the `atif-sql` command (see [What you can do](#what-you-can-do)) |
@@ -194,7 +194,7 @@ matching `ATIF_SQL_*` env var still wins.
 ```bash
 atif-sql materialize --agent codex
 atif-sql status --agent codex
-atif-sql query "SELECT agent, count(*) FROM sessions GROUP BY 1"
+atif-sql query --agent codex "SELECT agent, count(*) FROM sessions GROUP BY 1"
 ```
 
 One corpus holds one agent, so a Codex corpus and a Claude Code corpus stay
@@ -269,7 +269,7 @@ vulnerability in a tool that reads local transcripts are in
 corpus is registered (override with `ATIF_SQL_QUERY_MEMORY_LIMIT` and `ATIF_SQL_QUERY_THREADS`),
 a private spill directory outside the corpus that's removed on exit, no extension installs at
 query time (`atif-sql embed --install-extension` is where the lance extension comes from), file
-facing statements (`COPY`, `EXPORT`, `ATTACH`, `INSTALL`, `LOAD`, `PREPARE`, `EXECUTE`) refused
+facing statements (`COPY`, `EXPORT`, `ATTACH`, `DETACH`, `INSTALL`, `LOAD`, `PREPARE`, `EXECUTE`) refused
 before they run, and a refusal to run as root unless `ATIF_SQL_ALLOW_ROOT=1` says so. The details
 and the one accepted disclosure (`duckdb_settings()` lists the granted paths) are in
 [docs/reference/cli.md](docs/reference/cli.md#query).

{atif_sql-0.3.0 → atif_sql-0.5.0}/README.md

@@ -107,7 +107,7 @@ the only thing documented as installable is the `atif-sql` CLI above.
 | `atif-converter` | Claude Code and Codex CLI transcript → ATIF converters (ours, built on Harbor's ATIF models, vendored) + per-agent fidelity policy (loss accounting per session) |
 | `atif-corpus` | Corpus materialization: discovery, watermarks, quiescence, atomic artifact writes |
 | `atif-duck` | DuckDB views + macros over the materialized corpus (core surface plus the v2 analytics surface) |
-| `atif-models` | Model alias registry + structured-output LLM clients for GPT-6 and Claude 5.5 on Bedrock; no other package hardcodes a model id |
+| `atif-models` | Model alias registry + structured-output LLM clients for GPT-6 and Claude 5.5 on Bedrock; no other package hardcodes an LLM model id |
 | `atif-analytics` | the LLM pipelines `atif-sql analyze` runs: classify, conflicts, friction, and perceived |
 | `atif-embed` | Cohere Embed v4 on Bedrock or EmbeddingGemma 2 on this machine + LanceDB vector store + embedding backfill |
 | `atif-cli` | the composition root, and the source of the `atif-sql` command (see [What you can do](#what-you-can-do)) |
@@ -152,7 +152,7 @@ matching `ATIF_SQL_*` env var still wins.
 ```bash
 atif-sql materialize --agent codex
 atif-sql status --agent codex
-atif-sql query "SELECT agent, count(*) FROM sessions GROUP BY 1"
+atif-sql query --agent codex "SELECT agent, count(*) FROM sessions GROUP BY 1"
 ```
 
 One corpus holds one agent, so a Codex corpus and a Claude Code corpus stay
@@ -227,7 +227,7 @@ vulnerability in a tool that reads local transcripts are in
 corpus is registered (override with `ATIF_SQL_QUERY_MEMORY_LIMIT` and `ATIF_SQL_QUERY_THREADS`),
 a private spill directory outside the corpus that's removed on exit, no extension installs at
 query time (`atif-sql embed --install-extension` is where the lance extension comes from), file
-facing statements (`COPY`, `EXPORT`, `ATTACH`, `INSTALL`, `LOAD`, `PREPARE`, `EXECUTE`) refused
+facing statements (`COPY`, `EXPORT`, `ATTACH`, `DETACH`, `INSTALL`, `LOAD`, `PREPARE`, `EXECUTE`) refused
 before they run, and a refusal to run as root unless `ATIF_SQL_ALLOW_ROOT=1` says so. The details
 and the one accepted disclosure (`duckdb_settings()` lists the granted paths) are in
 [docs/reference/cli.md](docs/reference/cli.md#query).

{atif_sql-0.3.0 → atif_sql-0.5.0}/docs/README.md

@@ -6,9 +6,11 @@ this codebase quietly assumes. Every factual claim carries a `path:LOC` citation
 Prose is generated; structure is mechanical. Cross-references are deterministic.
 
 Citations come as full paths or as shorthands, and each one resolves into a
-source file. A validator rejects any citation naming a file that does not exist, a line
-outside that file's range, or a shorthand with no resolvable antecedent, and it exits non-zero on the
-first one, so a citation here resolves or the tree does not ship.
+source file. The docs site build turns each one into a link and fails on the first citation naming
+a file that does not exist at the commit it links to, or a shorthand with no resolvable antecedent.
+`mise run docs:citations` also fails on a line outside its file and, against the base commit, on a
+citation whose cited lines a change moved or removed without editing the citation, so a citation
+here resolves to the code it names or the tree does not ship.
 
 ## Start here
 
@@ -24,8 +26,8 @@ first one, so a citation here resolves or the tree does not ship.
 
 - [System overview](architecture/system-overview.md): the short version, the stack, and one
   diagram of the workspace members.
-- [Module map](architecture/module-map.md): one section per member, ordered by source size, with the
-  files in each that carry the weight.
+- [Module map](architecture/module-map.md): one section per member, with the files in each that
+  carry the weight.
 - [Data flow](architecture/data-flow.md): the flows (`materialize`, `query`, `analyze`) traced end
   to end, each as a sequence diagram.
 
@@ -75,11 +77,11 @@ first one, so a citation here resolves or the tree does not ship.
 
 ## Documents this tree deliberately omits
 
-- **`analysis/ownership.md`**: the history sits under a single bot identity,
-  with no human authors. A per-person ranked table and a bus-factor list would be noise dressed as
+- **`analysis/ownership.md`**: every commit in the history names either the one maintainer or a
+  GitHub bot account. A per-person ranked table and a bus-factor list would be noise dressed as
   analysis.
 - **`reference/rpc-tools.md`**: there is no RPC, MCP, or HTTP surface. `fastapi`, `uvicorn`,
-  `starlette`, and the supabase client stack do appear in the installed dependency closure, arriving
+  `starlette`, and the supabase client stack do appear in the locked dev dependency closure, arriving
   transitively through harbor, and no first-party module imports any of them.
 
 An empty file in either slot would imply absence by omission rather than by silence, which is why

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-analytics/README.md

@@ -6,8 +6,8 @@ The v2 analytics pipelines over the materialized ATIF corpus:
   read the materialized corpus (`trajectory.json` steps + `edges.jsonl`,
   each stored compressed or plain),
   render session transcripts under the documented caps, and classify through
-  `atif_models.LlmStructuredProvider` (GPT-6 or Claude 5.5 structured
-  outputs on bedrock-runtime; sizes classify=small, conflicts=large,
+  `atif_models.domain.ports.LlmStructuredProvider` (GPT-6 or Claude 5.5
+  structured outputs on bedrock-runtime; sizes classify=small, conflicts=large,
   friction=small, perceived=medium). `classify` labels `work_category` and `goal` only.
   `classify` and `conflicts` skip non-interactive sessions (`turn_audit`,
   `one_shot_job`), and `friction` and `perceived` read human turns only.
@@ -25,8 +25,9 @@ The v2 analytics pipelines over the materialized ATIF corpus:
   sharded parquet caches under `<corpus_root>/analytics/<name>/`.
 
 Composition: atif-analytics imports **only** atif-models among the workspace
-packages (import-linter independence contract); atif-cli composes it with
-atif-duck's registered views via the `atif-sql analyze` command.
+packages (import-linter forbidden contract); atif-cli composes it via the
+`atif-sql analyze` command, handing it a session source over atif-duck's lake
+(`atif_cli.lake_sessions`).
 
 All pipelines default to `dry_run=True` and return plan dicts with
 `estimate_cost_tokens` projections (cost guard per CONTRACT-V2).

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-analytics/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "atif-analytics"
-version = "0.3.0"
+version = "0.5.0"
 description = "LLM analytics pipelines over the materialized ATIF corpus: session classification, stance conflicts, user friction, and perceived errors."
 readme = "README.md"
 license = "Apache-2.0"
@@ -20,7 +20,7 @@ dependencies = [
     # a LOCAL source an external installer never sees, so a bare name would ship as
     # `Requires-Dist: atif-models` and resolve to whatever the newest release of
     # that name is. `[tool.commitizen] version_files` rewrites this line on a bump.
-    "atif-models==0.3.0",
+    "atif-models==0.5.0",
     "loguru>=0.7.3",
     "polars>=1.40.0",
     "pyarrow>=23.0.1",

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-cli/README.md

@@ -16,7 +16,7 @@ atif-sql materialize [--force] [--quiesce-seconds N]
                      [--sessions id,id]   # contract-compatible extension:
                                           #   plan only the named sessions
 atif-sql status                        # corpus freshness, read-only
-atif-sql query 'SQL' [--format auto|json|csv]
+atif-sql query 'SQL' [--format auto|table|json|csv]
 atif-sql schema                        # static catalog, no duckdb bind, <50ms
 atif-sql examples                      # tested example queries per view/macro
                                        #   (alias: atif-sql query --examples)
@@ -27,6 +27,11 @@ atif-sql embed --all --no-dry-run      # embed corpus steps into LanceDB
 atif-sql search '<text>'               # embed the text, then semantic kNN
 atif-sql cron install | status         # print the refresh crontab block, or
                                        #   report lane locks + last runs
+atif-sql lake rebuild | verify | status | compact
+                                       # build, check, report on, and compact
+                                       #   the DuckLake every query reads
+atif-sql corpus slim [--no-dry-run]    # convert a corpus to the compressed
+                                       #   layout; a dry run by default
 ```
 
 Conventions: `--format auto` resolves to a table on a TTY and JSON on a pipe;

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-cli/pyproject.toml

@@ -5,7 +5,7 @@
 # import-linter can read what it is allowed to import, and so its tests have a
 # home — none of which needs an index.
 name = "atif-cli"
-version = "0.3.0"
+version = "0.5.0"
 description = "ATIF-native analytics over Claude Code agent trajectories: convert, materialize, and query agent sessions with DuckDB."
 readme = "README.md"
 license = "Apache-2.0"
@@ -29,17 +29,17 @@ dependencies = [
     # namespace to whatever the newest release of that name is, by whoever owns it.
     # One version moves the whole workspace, so `==` is the honest constraint, and
     # `[tool.commitizen] version_files` rewrites every one of these on each bump.
-    "atif-analytics==0.3.0",
-    "atif-converter==0.3.0",
-    "atif-corpus==0.3.0",
-    "atif-duck==0.3.0",
-    "atif-embed==0.3.0",
+    "atif-analytics==0.5.0",
+    "atif-converter==0.5.0",
+    "atif-corpus==0.5.0",
+    "atif-duck==0.5.0",
+    "atif-embed==0.5.0",
     "cyclopts>=5.1.1",
     # Imported directly by duck_errors.py to classify a DuckDB failure into the
     # parse/catalog/runtime exit-code taxonomy. Reaching it transitively through
     # atif-duck would make this package's exit codes depend on a sibling's
     # dependency choice.
-    "duckdb>=1.5.2,<2",
+    "duckdb>=2.0.0,<3",
     "loguru>=0.7.3",
 ]
 

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-cli/src/atif_cli/app.py

@@ -454,6 +454,15 @@ def _refuse_root(command: str, fmt: OutputFormat) -> None:
 #: PREPARE and EXECUTE (a prepared COPY parses as PREPARE). An allowlist, so a
 #: statement kind a future DuckDB adds is refused until someone reads what it
 #: does.
+#:
+#: PRAGMA and MULTI are out because DuckDB 2.0's parser no longer says what
+#: they hold. 1.5 expanded a PRAGMA while parsing, so ``PRAGMA table_info``
+#: came back as SELECT and ``COPY FROM DATABASE`` as COPY_DATABASE; 2.0 hands
+#: back the unexpanded PRAGMA for every pragma and for ``COPY FROM DATABASE``
+#: and ``IMPORT DATABASE`` alike, and one MULTI for a PIVOT that has to build
+#: its column list first, whatever statement wraps it (``COPY (PIVOT ...) TO``
+#: is one MULTI). Their SELECT forms still run: ``pragma_table_info('t')``,
+#: ``DESCRIBE``, ``SHOW TABLES``, and PIVOT with an explicit ``IN`` list.
 _QUERY_STATEMENT_KINDS: frozenset[str] = frozenset(
     {
         "SELECT",
@@ -469,7 +478,6 @@ _QUERY_STATEMENT_KINDS: frozenset[str] = frozenset(
         "DELETE",
         "MERGE_INTO",
         "CALL",
-        "PRAGMA",
         "TRANSACTION",
         "VACUUM",
         "ANALYZE",
@@ -485,10 +493,17 @@ def _refused_statement_kinds(con: Any, sql: str) -> list[str]:
     connection so a PRAGMA that reads a file while parsing meets the same
     allowlist the query would. A parse error propagates as DuckDB's own
     ``ParserException`` (exit 64), exactly as ``execute`` would have raised it.
+
+    A statement whose type the Python binding has no member for is refused as
+    ``UNKNOWN``: DuckDB 2.0's ``StatementType`` raises ``ValueError`` there
+    (``UPDATE EXTENSIONS`` is type 29), where 1.5 answered ``???``.
     """
     refused: list[str] = []
     for statement in con.extract_statements(sql):
-        kind = str(statement.type.name)
+        try:
+            kind = str(statement.type.name)
+        except ValueError:
+            kind = "UNKNOWN"
         if kind not in _QUERY_STATEMENT_KINDS and kind not in refused:
             refused.append(kind)
     return refused
@@ -557,7 +572,7 @@ def _harden_query_connection(
       ``register`` bound this connection to). A file grant lets
       ``read_parquet`` open that exact path. Nothing else under the corpus
       is named, so ``read_text`` of a ``trajectory.json`` is refused. DuckDB
-      1.5.5 has no read-only grant (measured: a ``read_parquet`` relation
+      has no read-only grant, on 1.5.5 or 2.0.0 (measured: a ``read_parquet`` relation
       bound before ``enable_external_access=false`` is refused at query time
       without a grant, and ``duckdb_settings()`` lists no write switch), so
       each grant is read-write and ``COPY ... TO <granted parquet>
@@ -1364,7 +1379,11 @@ def query(
       parquet> (USE_TMP_FILE false)`` used to overwrite one, so every
       statement kind that names a file is refused BEFORE execution, for any
       uid, by DuckDB's own parser: COPY, EXPORT, ATTACH, DETACH, INSTALL,
-      LOAD, and PREPARE/EXECUTE (which could defer one). Exit 70, kind
+      LOAD, and PREPARE/EXECUTE (which could defer one). So are PRAGMA and
+      MULTI, which DuckDB 2.0 no longer expands while parsing (a PRAGMA can be
+      ``COPY FROM DATABASE`` or ``IMPORT DATABASE``, and a MULTI can wrap a
+      COPY): run ``pragma_table_info(...)``, ``DESCRIBE`` or ``SHOW`` instead,
+      and give a PIVOT an explicit ``IN`` list. Exit 70, kind
       ``sandbox_refused``.
     * A 0444 file mode does not bind root, so ``query`` refuses to run as
       uid 0 (exit 77) unless ``ATIF_SQL_ALLOW_ROOT=1`` is set, which logs a
@@ -1405,7 +1424,7 @@ def query(
     * 65  catalog_error unknown view/macro/column (try ``atif-sql schema``)
     * 65  embedding_mismatch the Lance store was written by another provider
     * 70  sandbox_refused a statement kind the sandbox never runs (COPY,
-      EXPORT, ATTACH, INSTALL, LOAD, PREPARE, ...)
+      EXPORT, ATTACH, INSTALL, LOAD, PREPARE, PRAGMA, ...)
     * 70  runtime_error everything else (an unmaterialized corpus, or SQL
       the sandbox refused)
     * 77  root_refused  running as uid 0 without ``ATIF_SQL_ALLOW_ROOT=1``
@@ -1507,7 +1526,10 @@ def query(
                         message=f"the query sandbox does not run {', '.join(refused)} statements",
                         hint="query runs SELECT and in-memory DDL/DML only; COPY, EXPORT, "
                         "ATTACH, DETACH, INSTALL, LOAD, PREPARE and EXECUTE are refused "
-                        "because they reach files or defer a statement past this check",
+                        "because they reach files or defer a statement past this check, and "
+                        "PRAGMA and MULTI because the parser no longer says what they hold "
+                        "(use pragma_table_info(...), DESCRIBE or SHOW for a PRAGMA, and an "
+                        "explicit IN list for a PIVOT)",
                     )
                     emit_error(err, fmt)
                     raise SystemExit(err.exit_code)

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-converter/README.md

@@ -2,8 +2,10 @@
 
 Converts Claude Code session JSONL and Codex rollouts into ATIF trajectories
 (converters ported from harbor and held to parity with harbor 0.24.0, on harbor's ATIF models vendored in
-`atif_converter.domain.atif`), and owns the FIDELITY POLICY: the seven known upstream
-conversion gaps are encoded as `atif_converter.domain.fidelity.FidelityGap`,
+`atif_converter.domain.atif`), and owns the FIDELITY POLICY: the known upstream
+conversion gaps are encoded as `atif_converter.domain.fidelity.FidelityGap` (six
+for Claude Code, plus one retired number) and
+`atif_converter.domain.codex_fidelity.CodexFidelityGap` (six for Codex),
 and every conversion is audited into a `LossReport` (raw-side census vs
 converted output). harbor is a dev dependency only; the tests pin harbor 0.24.0
 behavior and hold the vendored models to it, so they are the drift alarm for a
@@ -14,7 +16,7 @@ harbor bump.
 `LossReport.record_counts` (and therefore `records_total` / `records_dropped`)
 counts every record of every `*.jsonl` discovered under the session's side
 directory, at any depth and regardless of whether a `subagents/` part appears
-in the path. That is the same file set the adapter stages into harbor and the
+in the path. That is the same file set the converter reads and the
 same set `edges.jsonl` is built from, so `records_total` always equals the
 edges line count. A session with a side-file outside `subagents/` reports a
 higher `records_total` than a census scoped to `subagents/` alone would.
@@ -27,7 +29,7 @@ value: `atif_converter.domain.pricing` returns the same floats, bit for bit,
 without litellm being installed. It reads `domain/model_prices.json`, a
 filtered copy of litellm's public `model_prices_and_context_window.json` (MIT;
 the notice and source ref are in its `meta` block), and repeats litellm
-1.103.2's arithmetic in the same float operation order. The table holds the
+1.104.2's arithmetic in the same float operation order. The table holds the
 Claude and OpenAI text models our transcripts name under the `anthropic`,
 `openai`, `bedrock` and `bedrock_converse` providers, and keeps litellm's
 `fallback_generalizations` rules, which route an unmapped `claude-<family>-<n>`

{atif_sql-0.3.0 → atif_sql-0.5.0}/packages/atif-converter/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "atif-converter"
-version = "0.3.0"
+version = "0.5.0"
 description = "Claude Code and Codex transcripts -> ATIF conversion, with the fidelity policy per agent."
 readme = "README.md"
 license = "Apache-2.0"