dirsql 0.3.118__tar.gz → 0.3.120__tar.gz

{dirsql-0.3.118 → dirsql-0.3.120}/Cargo.lock

@@ -500,7 +500,7 @@ dependencies = [
 
 [[package]]
 name = "dirsql-py-ext"
-version = "0.3.118"
+version = "0.3.120"
 dependencies = [
  "dirsql",
  "pyo3",

{dirsql-0.3.118 → dirsql-0.3.120}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dirsql
-Version: 0.3.118
+Version: 0.3.120
 Summary: Ephemeral SQL index over a local directory
 Keywords: sql,filesystem,directory,sqlite,index
 Author: Kevin Scott

{dirsql-0.3.118 → dirsql-0.3.120}/docs/howto/write-a-plugin.md

@@ -217,7 +217,7 @@ Discovery is deliberately narrow. Know exactly who does what:
   your config still takes ordering precedence
   ([composing configs](../reference/config.md#composing-multiple-configs)). When
   you pass no `-c` of your own, the launcher also keeps the
-  shipped starter `files` table (an internal `--include-default`), so plugins
+  shipped starter `records` table (an internal `--include-default`), so plugins
   **add** tables rather than standing alone. Discovery is **pip/uvx only** for now — the `npx`
   launcher does not yet discover — and is switched off per invocation with
   [`--no-plugin` or `DIRSQL_NO_PLUGIN=1`](../reference/cli.md#plugins).

{dirsql-0.3.118/packages/rust → dirsql-0.3.120}/docs/reference/cli.md

@@ -151,6 +151,25 @@ uses**, so behavior is identical to `POST /query` by construction:
   statements; dirsql separately never modifies the files it indexes — see
   [Read-only by design](../explanation#read-only-by-design).
 
+#### `--on-file <command>`
+
+Attach a parser to every [path-table](./path-tables.md#parsing-rows-with-on-file)
+in the query, so each matched file yields the rows the command prints (a JSON
+array of row objects) instead of the stat columns:
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command follows the [`on-file` hook contract](./hooks.md#on-file) — argv
+splitting, `{path}`/`{root}` placeholders, per-file failure isolation, and the
+timeout. The parser's output is the whole schema; the stat columns are not
+reachable on a parsed path-table. `--on-file` may be given **at most once** (a
+repeat is an error pointing at config files) and never touches config-declared
+tables. It is a `query`-only flag — server mode rejects it as an unknown
+argument.
+
 Errors print the same diagnostic the HTTP `{"error": …}` body carries —
 config failures, SQL errors, rejected reads, hook failures, timeouts — to
 stderr, with exit code `1`.
@@ -164,8 +183,11 @@ stderr, with exit code `1`.
 
 ## `dirsql init`
 
-Writes a starter `.dirsql.toml` defining a catch-all `files` table, as a
-scaffold to edit:
+Writes a starter `.dirsql.toml` as a scaffold to edit. It does **not**
+duplicate the zero-config floor (`SELECT * FROM './'` already lists every file
+with no config); instead it shows the **escalation**: one named `[[table]]`
+with a glob, a schema, and a real `on-file` hook that pulls structured rows
+out of your files.
 
 ```bash
 dirsql init
@@ -206,7 +228,7 @@ installed in the same environment as `dirsql` (`pip install …`, or
 loads its fragment — its tables are queryable with zero config edits.
 Installed = active: there is no enable step and no naming convention. The
 fragment is composed *after* your own `-c` configs (so your config takes
-precedence in ordering), and the shipped starter `files` table is preserved.
+precedence in ordering), and the shipped starter `records` table is preserved.
 
 Discovery is **launcher-only** — the standalone `cargo`-installed binary does no
 discovery, and the SDKs never auto-discover (pass a plugin's config explicitly

{dirsql-0.3.118 → dirsql-0.3.120}/docs/reference/path-tables.md

@@ -114,6 +114,39 @@ SELECT path FROM './docs/*.md' WHERE content LIKE '%deprecated%';
 A file that cannot be read, or is not valid UTF-8, yields `NULL` content rather
 than failing the query.
 
+## Parsing rows with `--on-file`
+
+By default a path-table's columns are the stat columns above — one row per
+file. When you want *one row per record inside* each file, attach a parser with
+the `dirsql query` flag [`--on-file`](/reference/cli):
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command runs once per matched file and prints a JSON array of row objects,
+exactly like a declared table's [`on-file` hook](/reference/hooks) — same argv
+splitting, same `{path}`/`{root}` placeholders, same timeout. Its output *is*
+the table:
+
+- **The parser supplies the whole schema.** Columns are inferred from the keys
+  across the emitted rows. The stat columns (`path`, `size`, …) are **not**
+  reachable on a parsed path-table — a parser that wants the path emits it (it
+  has `{path}`). The two modes stay cleanly separate.
+- **Failures are isolated per file.** A file whose parser fails (spawn, non-zero
+  exit, timeout, or no output) or whose output is not a JSON array of rows
+  contributes no rows; a one-line warning naming the file goes to stderr and the
+  scan continues. The schema is inferred from the files that did parse.
+- **The skip rules still apply.** A parsed scan honors the same `node_modules`
+  /`.git`/`ignore` rules a stat scan does (see below).
+
+`--on-file` applies to **every** path-table in the query and may be given **at
+most once**. For different parsers per file set, define named tables in a
+`.dirsql.toml` with their own `on-file` keys and pass it with `-c` — the flag
+never touches config-declared tables. It is a `query`-subcommand flag; server
+mode does not accept it.
+
 ## Freshness and scope
 
 A path-table is scanned when the statement runs, so it always reflects the

{dirsql-0.3.118 → dirsql-0.3.120}/packages/python/Cargo.toml

@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
 # pypi/maturin handler can rewrite it via `write-version` before
 # `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
 # and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
-version = "0.3.118"
+version = "0.3.120"
 # abi3 note: the `abi3-py311` pyo3 feature (below) builds ONE stable-ABI
 # `cp311-abi3` wheel per platform that loads on every CPython >= 3.11
 # (matching `pyproject.toml`'s `requires-python = ">=3.11"`), instead of

{dirsql-0.3.118/packages/rust → dirsql-0.3.120/packages/python}/docs/howto/write-a-plugin.md

@@ -217,7 +217,7 @@ Discovery is deliberately narrow. Know exactly who does what:
   your config still takes ordering precedence
   ([composing configs](../reference/config.md#composing-multiple-configs)). When
   you pass no `-c` of your own, the launcher also keeps the
-  shipped starter `files` table (an internal `--include-default`), so plugins
+  shipped starter `records` table (an internal `--include-default`), so plugins
   **add** tables rather than standing alone. Discovery is **pip/uvx only** for now — the `npx`
   launcher does not yet discover — and is switched off per invocation with
   [`--no-plugin` or `DIRSQL_NO_PLUGIN=1`](../reference/cli.md#plugins).

{dirsql-0.3.118 → dirsql-0.3.120}/packages/python/docs/reference/cli.md

@@ -151,6 +151,25 @@ uses**, so behavior is identical to `POST /query` by construction:
   statements; dirsql separately never modifies the files it indexes — see
   [Read-only by design](../explanation#read-only-by-design).
 
+#### `--on-file <command>`
+
+Attach a parser to every [path-table](./path-tables.md#parsing-rows-with-on-file)
+in the query, so each matched file yields the rows the command prints (a JSON
+array of row objects) instead of the stat columns:
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command follows the [`on-file` hook contract](./hooks.md#on-file) — argv
+splitting, `{path}`/`{root}` placeholders, per-file failure isolation, and the
+timeout. The parser's output is the whole schema; the stat columns are not
+reachable on a parsed path-table. `--on-file` may be given **at most once** (a
+repeat is an error pointing at config files) and never touches config-declared
+tables. It is a `query`-only flag — server mode rejects it as an unknown
+argument.
+
 Errors print the same diagnostic the HTTP `{"error": …}` body carries —
 config failures, SQL errors, rejected reads, hook failures, timeouts — to
 stderr, with exit code `1`.
@@ -164,8 +183,11 @@ stderr, with exit code `1`.
 
 ## `dirsql init`
 
-Writes a starter `.dirsql.toml` defining a catch-all `files` table, as a
-scaffold to edit:
+Writes a starter `.dirsql.toml` as a scaffold to edit. It does **not**
+duplicate the zero-config floor (`SELECT * FROM './'` already lists every file
+with no config); instead it shows the **escalation**: one named `[[table]]`
+with a glob, a schema, and a real `on-file` hook that pulls structured rows
+out of your files.
 
 ```bash
 dirsql init
@@ -206,7 +228,7 @@ installed in the same environment as `dirsql` (`pip install …`, or
 loads its fragment — its tables are queryable with zero config edits.
 Installed = active: there is no enable step and no naming convention. The
 fragment is composed *after* your own `-c` configs (so your config takes
-precedence in ordering), and the shipped starter `files` table is preserved.
+precedence in ordering), and the shipped starter `records` table is preserved.
 
 Discovery is **launcher-only** — the standalone `cargo`-installed binary does no
 discovery, and the SDKs never auto-discover (pass a plugin's config explicitly

{dirsql-0.3.118 → dirsql-0.3.120}/packages/python/docs/reference/path-tables.md

@@ -114,6 +114,39 @@ SELECT path FROM './docs/*.md' WHERE content LIKE '%deprecated%';
 A file that cannot be read, or is not valid UTF-8, yields `NULL` content rather
 than failing the query.
 
+## Parsing rows with `--on-file`
+
+By default a path-table's columns are the stat columns above — one row per
+file. When you want *one row per record inside* each file, attach a parser with
+the `dirsql query` flag [`--on-file`](/reference/cli):
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command runs once per matched file and prints a JSON array of row objects,
+exactly like a declared table's [`on-file` hook](/reference/hooks) — same argv
+splitting, same `{path}`/`{root}` placeholders, same timeout. Its output *is*
+the table:
+
+- **The parser supplies the whole schema.** Columns are inferred from the keys
+  across the emitted rows. The stat columns (`path`, `size`, …) are **not**
+  reachable on a parsed path-table — a parser that wants the path emits it (it
+  has `{path}`). The two modes stay cleanly separate.
+- **Failures are isolated per file.** A file whose parser fails (spawn, non-zero
+  exit, timeout, or no output) or whose output is not a JSON array of rows
+  contributes no rows; a one-line warning naming the file goes to stderr and the
+  scan continues. The schema is inferred from the files that did parse.
+- **The skip rules still apply.** A parsed scan honors the same `node_modules`
+  /`.git`/`ignore` rules a stat scan does (see below).
+
+`--on-file` applies to **every** path-table in the query and may be given **at
+most once**. For different parsers per file set, define named tables in a
+`.dirsql.toml` with their own `on-file` keys and pass it with `-c` — the flag
+never touches config-declared tables. It is a `query`-subcommand flag; server
+mode does not accept it.
+
 ## Freshness and scope
 
 A path-table is scanned when the statement runs, so it always reflects the

dirsql-0.3.120/packages/python/e2e-attestations/feat-631-on-file-flag.json

@@ -0,0 +1,7 @@
+{
+  "command": "UV_NO_SYNC=1 uv run python -m pytest tests/e2e/ -x -q",
+  "ran_at": 1784652812,
+  "exit_code": 0,
+  "commit": "c59b0667084ba1d9542031a60cd55ae0bc3c7db6",
+  "branch": "feat/631-on-file-flag"
+}

dirsql-0.3.120/packages/python/e2e-attestations/feat-637-init-escalation-example.json

@@ -0,0 +1,7 @@
+{
+  "command": "just test-e2e",
+  "ran_at": 1784825108,
+  "exit_code": 0,
+  "commit": "6b2c6696cfbf87c61a1e4d57c5f0acd6c65fb50d",
+  "branch": "feat/637-init-escalation-example"
+}

dirsql-0.3.120/packages/rust/changelog.d/2026-07-21-on-file-flag.md

@@ -0,0 +1,14 @@
+**Added**
+
+`dirsql query "<sql>" --on-file '<command>'` attaches a parser to every
+[path-table](https://thekevinscott.github.io/dirsql/reference/path-tables) in
+the query. With the flag set, a path-table's rows and schema come from the
+command's JSON output (the `on-file` hook contract — argv splitting,
+`{path}`/`{root}` placeholders, timeout) instead of the stat columns. The
+parser's output is the whole schema; stat columns are not reachable on a parsed
+path-table. Failures are isolated per file — a file whose parser fails or whose
+output does not parse contributes no rows and a stderr warning, and the scan
+continues. Parsed scans honor the same `node_modules`/`.git`/`ignore` skip
+rules stat scans do. The flag may be given at most once (a repeat errors,
+pointing at config files), applies only to path-tables (config `[[table]]`
+`on-file` hooks are untouched), and is `query`-only (server mode rejects it).

dirsql-0.3.120/packages/rust/changelog.d/2026-07-23-init-escalation-example.md

@@ -0,0 +1,10 @@
+**Changed** — `dirsql init` now scaffolds an *escalation* example instead of a
+catch-all `files` table. Because any path (`./`, `../`, `/`, `~/`) is already a
+zero-config live view of the filesystem (`SELECT * FROM './'`), duplicating
+that as a starter table taught nothing. The generated `.dirsql.toml` is now a
+single named `[[table]]` (`records`) with a scoped `glob`, a pinned DDL, and a
+real `on-file = "cat {path}"` hook that turns each matched `*.json` file into
+rows — with a commented custom-parser stub and a header comment pointing at the
+zero-config floor. The scaffold works as written: `dirsql init` then
+`dirsql query "SELECT * FROM records" -c .dirsql.toml` returns the parsed rows.
+The `--include-default` launcher path seeds this same table.

{dirsql-0.3.118/packages/python → dirsql-0.3.120/packages/rust}/docs/howto/write-a-plugin.md

@@ -217,7 +217,7 @@ Discovery is deliberately narrow. Know exactly who does what:
   your config still takes ordering precedence
   ([composing configs](../reference/config.md#composing-multiple-configs)). When
   you pass no `-c` of your own, the launcher also keeps the
-  shipped starter `files` table (an internal `--include-default`), so plugins
+  shipped starter `records` table (an internal `--include-default`), so plugins
   **add** tables rather than standing alone. Discovery is **pip/uvx only** for now — the `npx`
   launcher does not yet discover — and is switched off per invocation with
   [`--no-plugin` or `DIRSQL_NO_PLUGIN=1`](../reference/cli.md#plugins).

{dirsql-0.3.118 → dirsql-0.3.120/packages/rust}/docs/reference/cli.md

@@ -151,6 +151,25 @@ uses**, so behavior is identical to `POST /query` by construction:
   statements; dirsql separately never modifies the files it indexes — see
   [Read-only by design](../explanation#read-only-by-design).
 
+#### `--on-file <command>`
+
+Attach a parser to every [path-table](./path-tables.md#parsing-rows-with-on-file)
+in the query, so each matched file yields the rows the command prints (a JSON
+array of row objects) instead of the stat columns:
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command follows the [`on-file` hook contract](./hooks.md#on-file) — argv
+splitting, `{path}`/`{root}` placeholders, per-file failure isolation, and the
+timeout. The parser's output is the whole schema; the stat columns are not
+reachable on a parsed path-table. `--on-file` may be given **at most once** (a
+repeat is an error pointing at config files) and never touches config-declared
+tables. It is a `query`-only flag — server mode rejects it as an unknown
+argument.
+
 Errors print the same diagnostic the HTTP `{"error": …}` body carries —
 config failures, SQL errors, rejected reads, hook failures, timeouts — to
 stderr, with exit code `1`.
@@ -164,8 +183,11 @@ stderr, with exit code `1`.
 
 ## `dirsql init`
 
-Writes a starter `.dirsql.toml` defining a catch-all `files` table, as a
-scaffold to edit:
+Writes a starter `.dirsql.toml` as a scaffold to edit. It does **not**
+duplicate the zero-config floor (`SELECT * FROM './'` already lists every file
+with no config); instead it shows the **escalation**: one named `[[table]]`
+with a glob, a schema, and a real `on-file` hook that pulls structured rows
+out of your files.
 
 ```bash
 dirsql init
@@ -206,7 +228,7 @@ installed in the same environment as `dirsql` (`pip install …`, or
 loads its fragment — its tables are queryable with zero config edits.
 Installed = active: there is no enable step and no naming convention. The
 fragment is composed *after* your own `-c` configs (so your config takes
-precedence in ordering), and the shipped starter `files` table is preserved.
+precedence in ordering), and the shipped starter `records` table is preserved.
 
 Discovery is **launcher-only** — the standalone `cargo`-installed binary does no
 discovery, and the SDKs never auto-discover (pass a plugin's config explicitly

{dirsql-0.3.118 → dirsql-0.3.120}/packages/rust/docs/reference/path-tables.md

@@ -114,6 +114,39 @@ SELECT path FROM './docs/*.md' WHERE content LIKE '%deprecated%';
 A file that cannot be read, or is not valid UTF-8, yields `NULL` content rather
 than failing the query.
 
+## Parsing rows with `--on-file`
+
+By default a path-table's columns are the stat columns above — one row per
+file. When you want *one row per record inside* each file, attach a parser with
+the `dirsql query` flag [`--on-file`](/reference/cli):
+
+```sh
+dirsql query "SELECT title, author FROM './posts/*.md'" \
+  --on-file 'extract.py {path}'
+```
+
+The command runs once per matched file and prints a JSON array of row objects,
+exactly like a declared table's [`on-file` hook](/reference/hooks) — same argv
+splitting, same `{path}`/`{root}` placeholders, same timeout. Its output *is*
+the table:
+
+- **The parser supplies the whole schema.** Columns are inferred from the keys
+  across the emitted rows. The stat columns (`path`, `size`, …) are **not**
+  reachable on a parsed path-table — a parser that wants the path emits it (it
+  has `{path}`). The two modes stay cleanly separate.
+- **Failures are isolated per file.** A file whose parser fails (spawn, non-zero
+  exit, timeout, or no output) or whose output is not a JSON array of rows
+  contributes no rows; a one-line warning naming the file goes to stderr and the
+  scan continues. The schema is inferred from the files that did parse.
+- **The skip rules still apply.** A parsed scan honors the same `node_modules`
+  /`.git`/`ignore` rules a stat scan does (see below).
+
+`--on-file` applies to **every** path-table in the query and may be given **at
+most once**. For different parsers per file set, define named tables in a
+`.dirsql.toml` with their own `on-file` keys and pass it with `-c` — the flag
+never touches config-declared tables. It is a `query`-subcommand flag; server
+mode does not accept it.
+
 ## Freshness and scope
 
 A path-table is scanned when the statement runs, so it always reflects the

dirsql-0.3.120/packages/rust/migrations.d/2026-07-23-init-escalation-example.md

@@ -0,0 +1,64 @@
+### `dirsql init` output rewritten as an escalation example (#637)
+
+#### Summary
+
+`dirsql init` no longer writes a catch-all `files` table over `**/*` with a
+stat-column schema. That table only duplicated the zero-config path-table floor
+(`SELECT * FROM './'`), which already lists every file with `path`, `basename`,
+`dir`, `ext`, `size`, `mtime` and `ctime` and no config at all. The generated
+`.dirsql.toml` is now an **escalation** scaffold: one named `[[table]]`
+(`records`) with a scoped `glob = "**/*.json"`, a pinned
+`ddl = "CREATE TABLE records (id TEXT, name TEXT)"`, and a real
+`on-file = "cat {path}"` hook whose rows come entirely from the hook (dirsql
+injects no columns). The internal `--include-default` launcher path seeds this
+same table's glob and DDL.
+
+The public `init` contract is otherwise unchanged: `--root`, `--output`,
+`--force`, the no-auto-load behavior, and every exit-code / error arm are the
+same. Only the *bytes written* changed.
+
+Carrying a genuine `on-file` hook keeps the shipped asset a valid config both
+today and once hook-less `[[table]]` entries become a load error, so
+`--include-default` (which parses the asset with `.expect(...)`) never panics.
+
+#### Required changes
+
+| Surface | Before | After |
+| ------- | ------ | ----- |
+| Query the init scaffold | `dirsql query "SELECT * FROM files" -c .dirsql.toml` | `dirsql query "SELECT * FROM records" -c .dirsql.toml` |
+| Want the old `files` table | (what `init` emitted) | write it yourself (snippet below) and pass with `-c` |
+
+To restore the previous starter table verbatim, write this `.dirsql.toml` and
+pass it explicitly:
+
+```toml
+[[table]]
+ddl  = "CREATE TABLE files (path TEXT, basename TEXT, dir TEXT, ext TEXT, size INTEGER, mtime INTEGER, ctime INTEGER)"
+glob = "**/*"
+```
+
+Note: this hook-less form loads today but is on track to become a config error;
+prefer the zero-config `SELECT * FROM './'` for filesystem listings.
+
+#### Deprecations removed
+
+_None._
+
+#### Behavior changes without code changes
+
+- `dirsql init` writes different bytes: a `records` escalation example rather
+  than a `files` catch-all table. No flags or exit codes changed.
+- `--include-default` now seeds a `records` table (glob `**/*.json`) instead of
+  a `files` table (glob `**/*`).
+
+#### Verification
+
+```bash
+cd "$(mktemp -d)"
+printf '[{"id":"1","name":"widget"}]' > things.json
+dirsql init
+dirsql query "SELECT id, name FROM records" -c .dirsql.toml
+# [{"id":"1","name":"widget"}]
+dirsql query "SELECT basename FROM './'"    # the zero-config floor still needs no config
+# [{"basename":".dirsql.toml"},{"basename":"things.json"}]
+```