dirsql 0.3.69 → 0.3.71

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,141 @@
1
+ # Command hooks
2
+
3
+ Three [config keys](./config.md) run an external command: `on-file` (per
4
+ `[[table]]`), and the server-wide `pre-query` and `post-query` (under
5
+ `[dirsql]`; CLI server only). All three share one execution contract.
6
+
7
+ ## Execution contract
8
+
9
+ ### argv, not a shell
10
+
11
+ The command string is split into an argv with shell-like quoting: whitespace
12
+ separates arguments, and single or double quotes group them (so
13
+ `sh -c 'grep foo {path} | sort'` keeps the quoted script as a single
14
+ argument). **No shell is invoked** — there is no globbing, piping, `$VAR`
15
+ expansion, or `&&`/`;` chaining. To get shell features, ask for a shell
16
+ explicitly with `sh -c '…'`.
17
+
18
+ A command that is empty, whitespace-only, or has unbalanced quotes is
19
+ invalid (empty/whitespace commands are already rejected at
20
+ [config parse time](./config.md#parse-errors)).
21
+
22
+ ### Placeholders
23
+
24
+ A `{name}` in the command is substituted with its value, in every
25
+ occurrence, within whole argv tokens, in a single left-to-right pass:
26
+
27
+ - A substituted value is always exactly one argv element — a value
28
+ containing spaces, quotes, or shell metacharacters stays a single
29
+ argument. This makes untrusted input (file paths, request bodies)
30
+ injection-safe at the argv level.
31
+ - Substituted values are never re-scanned: a value that itself contains
32
+ `{…}` is inert.
33
+ - An unrecognized `{…}` is left literal.
34
+
35
+ Which placeholders exist depends on the hook (see
36
+ [per-hook contracts](#per-hook-contracts) below).
37
+
38
+ ### Working directory and environment
39
+
40
+ The command runs in the **config file's directory**, so relative paths in
41
+ the command resolve predictably regardless of where `dirsql` was launched.
42
+ It inherits `dirsql`'s environment, so tools like `uvx --with …` / `npx …`
43
+ resolve their dependencies as usual.
44
+
45
+ ### stdout protocol
46
+
47
+ The command's result payload is the **last non-empty line of stdout**,
48
+ trimmed. Any log or chatter lines above it are ignored. A command that
49
+ exits successfully but prints no non-empty line is a failure ("produced no
50
+ output on stdout").
51
+
52
+ stderr is never data — it is captured only to enrich error messages (the
53
+ last 2 000 characters are attached to failures).
54
+
55
+ ::: tip Print single-line output
56
+ Because only the last non-empty line is the payload, multi-line output loses
57
+ everything above the last line. `jq` users: pass `-c` so the JSON is emitted
58
+ compactly on one line.
59
+ :::
60
+
61
+ ### Timeout
62
+
63
+ Every hook run is bounded by a **30-second** default timeout. A run
64
+ exceeding it is killed and treated as a failure. One global config key
65
+ raises (or tightens) the bound for **all** hooks:
66
+
67
+ ```toml
68
+ [dirsql]
69
+ hook-timeout = 300 # positive whole seconds
70
+ ```
71
+
72
+ Zero and negative values are a config error.
73
+
74
+ ### Failure semantics
75
+
76
+ A hook run fails when the command:
77
+
78
+ - cannot be spawned (e.g. the program is not found),
79
+ - exits non-zero (the exit code — or `signal`, if killed by one — and the
80
+ stderr tail are reported),
81
+ - exceeds the timeout (killed; stderr tail reported),
82
+ - exits zero but prints no non-empty stdout line,
83
+ - or (per hook, below) prints output that does not parse as expected.
84
+
85
+ What a failure *means* differs per hook:
86
+
87
+ | Hook | On failure |
88
+ |---|---|
89
+ | `on-file` | **Per-file isolation.** The file contributes no rows; a one-line warning naming the file and error goes to stderr; the scan continues. One bad file never aborts the scan. |
90
+ | `pre-query` | The request returns `500 Internal Server Error` with the command's stderr tail in the JSON `error` body. |
91
+ | `post-query` | The request returns `500 Internal Server Error` with the command's stderr tail (or, for unparseable output, `post-query did not return valid JSON: <err>`) in the JSON `error` body. |
92
+
93
+ ## Per-hook contracts
94
+
95
+ ### `on-file`
96
+
97
+ Runs once per file matched by the table's `glob`, at initial scan and on
98
+ every watched change. The command reads the file itself and prints a JSON
99
+ array of row objects; see [`[[table]]`](./config.md#table) for the
100
+ row-mapping rules.
101
+
102
+ | Placeholder | Value |
103
+ |---|---|
104
+ | `{path}` | The matched file's path **relative to the index root**. Appended automatically as a final argument when the command omits it, so `extract.py` and `extract.py {path}` behave identically. |
105
+ | `{abspath}` | The matched file's absolute path. |
106
+ | `{root}` | The index root directory. |
107
+
108
+ ### `pre-query`
109
+
110
+ Runs once per `POST /query` request, before the query. The raw request body
111
+ goes in; plain-text SQL comes out (the stdout payload line). `dirsql` runs
112
+ that SQL and returns rows as usual. With no `pre-query` key, the body is
113
+ parsed as `{"sql": …}` instead — see [HTTP API](./http-api.md#post-query).
114
+
115
+ | Placeholder | Value |
116
+ |---|---|
117
+ | `{args}` | The raw `POST /query` request body, verbatim, as one argv token. |
118
+
119
+ **The hook owns SQL safety.** The `{args}` substitution keeps the untrusted
120
+ body inert *as an argv token*, but whatever SQL string the hook prints is
121
+ executed as-is. Validate, escape, or parameterize inside the hook.
122
+
123
+ ### `post-query`
124
+
125
+ Runs once per successful `POST /query`, after the query. The result rows
126
+ are serialized to a JSON array and delivered two ways:
127
+
128
+ - **On stdin** — always, unbounded. This is the recommended path.
129
+ - **As `{args}`** — only when the serialized payload is ≤ **96 KiB**. Above
130
+ that, `{args}` is substituted with an **empty string** and a stderr
131
+ warning naming the byte size directs the operator to stdin. The full
132
+ payload is still on stdin — this is a fallback, not truncation.
133
+
134
+ | Placeholder | Value |
135
+ |---|---|
136
+ | `{args}` | The result rows as a JSON array, as one argv token; emptied (with a stderr warning) when the payload exceeds 96 KiB. |
137
+
138
+ The stdout payload line is parsed as JSON and returned verbatim as the
139
+ `200 application/json` response body. A payload that is not valid JSON
140
+ fails the request (`500`). With no `post-query` key, the bare row array is
141
+ returned — see [HTTP API](./http-api.md#post-query).
@@ -0,0 +1,120 @@
1
+ # HTTP API
2
+
3
+ The [`dirsql` server](./cli.md#server-mode) (default `localhost:7117`)
4
+ exposes two endpoints: `POST /query` and `GET /events`.
5
+
6
+ ## `POST /query`
7
+
8
+ Run a read-only SQL query. Request body is JSON:
9
+
10
+ ```json
11
+ {"sql": "SELECT title, author FROM posts WHERE draft = 0"}
12
+ ```
13
+
14
+ The `200` response is a JSON array of row objects keyed by column name:
15
+
16
+ ```json
17
+ [
18
+ {"title": "Hello World", "author": "alice"},
19
+ {"title": "Second Post", "author": "bob"}
20
+ ]
21
+ ```
22
+
23
+ ```bash
24
+ curl -s http://localhost:7117/query \
25
+ -H 'content-type: application/json' \
26
+ -d '{"sql":"SELECT COUNT(*) AS n FROM files"}' \
27
+ | jq
28
+ ```
29
+
30
+ ### Value serialization
31
+
32
+ | SQLite value | JSON |
33
+ |---|---|
34
+ | `NULL` | `null` |
35
+ | `INTEGER` | number |
36
+ | `REAL` | number; `NaN` / `±Infinity` become `null` |
37
+ | `TEXT` | string |
38
+ | `BLOB` | lowercase hex string (e.g. `"deadbeef"`) |
39
+
40
+ Internal tracking columns (`_dirsql_file_path`, `_dirsql_row_index`) are
41
+ excluded from `SELECT *` results.
42
+
43
+ ### Status codes
44
+
45
+ | Status | When |
46
+ |---|---|
47
+ | `200` | Query succeeded. Body: array of row objects (or the [`post-query`](./hooks.md#post-query) hook's JSON). |
48
+ | `400` | Malformed JSON body, missing or empty `sql` field, or a SQL error (syntax error, unknown table, or a statement SQLite classifies as a write — queries are read-only). |
49
+ | `405` | `GET /query`. Plain-text body `method not allowed`. |
50
+ | `408` | The query exceeded the 30-second per-query timeout. |
51
+ | `500` | Internal server fault, or a failed [`pre-query`](./hooks.md#pre-query) / [`post-query`](./hooks.md#post-query) hook. |
52
+ | `503` | The server is in [degraded mode](./cli.md#degraded-mode) (the config file exists but failed to load). |
53
+
54
+ All error responses (except `405`) are `application/json`:
55
+
56
+ ```json
57
+ {"error": "syntax error near \"SLECT\""}
58
+ ```
59
+
60
+ ### Hook interactions
61
+
62
+ - With [`[dirsql].pre-query`](./config.md#dirsql-keys) configured, the
63
+ request body is **not** parsed as `{"sql": …}`; the raw body is passed to
64
+ the hook, which prints the SQL to run. Hook failure → `500`.
65
+ - With [`[dirsql].post-query`](./config.md#dirsql-keys) configured, the
66
+ `200` body is whatever JSON the hook prints instead of the bare row
67
+ array. Hook failure or non-JSON output → `500`.
68
+
69
+ ## `GET /events`
70
+
71
+ Opens a [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)
72
+ stream of row-change events.
73
+
74
+ ```bash
75
+ curl -N http://localhost:7117/events
76
+ ```
77
+
78
+ On open, the server emits a single `ready` frame signalling the
79
+ subscription is attached. Every subsequent frame is named `row`:
80
+
81
+ ```
82
+ event: ready
83
+ data: {}
84
+
85
+ event: row
86
+ data: {"action":"insert","table":"posts","file_path":"posts/hello.json","row":{"title":"Hello World"},"old_row":null}
87
+ ```
88
+
89
+ ### Event payloads
90
+
91
+ The `data:` payload is one JSON object per row-level change:
92
+
93
+ | Field | `insert` | `update` | `delete` | `error` |
94
+ |---|---|---|---|---|
95
+ | `action` | `"insert"` | `"update"` | `"delete"` | `"error"` |
96
+ | `table` | table name | table name | table name | table name, or `null` when the failure isn't tied to one table |
97
+ | `file_path` | path relative to the root | same | same | same |
98
+ | `row` | the new row | the new row | the deleted row | — (absent) |
99
+ | `old_row` | `null` | the previous row | `null` | — (absent) |
100
+ | `error` | — | — | — | message string |
101
+
102
+ Row values serialize as in [`POST /query`](#value-serialization).
103
+
104
+ ### Stream semantics
105
+
106
+ - **Errors do not terminate the stream.** A malformed file produces an
107
+ `error` event; the stream continues.
108
+ - **Slow consumers skip, not crash.** A subscriber that lags behind the
109
+ event buffer silently misses the overflowed events and keeps receiving
110
+ new ones.
111
+ - The server sends periodic SSE keep-alive comments.
112
+ - The stream closes when the server shuts down.
113
+
114
+ ### Status codes
115
+
116
+ | Status | When |
117
+ |---|---|
118
+ | `200` | Stream opened (`text/event-stream`). |
119
+ | `405` | `POST /events`. Plain-text body `method not allowed`. |
120
+ | `503` | The server is in [degraded mode](./cli.md#degraded-mode). JSON `{"error": …}` body. |
@@ -0,0 +1,357 @@
1
+ # SDK
2
+
3
+ `dirsql` ships as a library for Python (PyPI: `dirsql`), TypeScript (npm:
4
+ `dirsql`), and Rust (crates.io: `dirsql`). All three are thin bindings over
5
+ one shared Rust core, so behavior is identical; only the calling conventions
6
+ differ.
7
+
8
+ ## Install
9
+
10
+ ::: code-group
11
+
12
+ ```bash [Python]
13
+ uv add dirsql
14
+ ```
15
+
16
+ ```bash [TypeScript]
17
+ pnpm add dirsql
18
+ ```
19
+
20
+ ```bash [Rust]
21
+ cargo add dirsql
22
+ ```
23
+
24
+ :::
25
+
26
+ ## Import
27
+
28
+ ::: code-group
29
+
30
+ ```python [Python]
31
+ from dirsql import DirSQL, Table, RowEvent
32
+ ```
33
+
34
+ ```typescript [TypeScript]
35
+ import { DirSQL, Table } from "dirsql";
36
+ import type { TableDef, RowEvent, ExtensionSpec, DirSQLOptions } from "dirsql";
37
+ ```
38
+
39
+ ```rust [Rust]
40
+ use dirsql::{DirSQL, AsyncDirSQL, Table, Row, RowEvent, Value, Extension, DirSqlError};
41
+ ```
42
+
43
+ :::
44
+
45
+ ## `DirSQL`
46
+
47
+ ### Constructor
48
+
49
+ ::: code-group
50
+
51
+ ```python [Python]
52
+ DirSQL(
53
+ root: str | None = None,
54
+ *,
55
+ tables: list[Table] | None = None,
56
+ ignore: list[str] | None = None,
57
+ config: str | None = None,
58
+ persist: bool = False,
59
+ persist_path: str | None = None,
60
+ extensions: list[dict] | None = None, # [{ "path": str, "entrypoint"?: str }]
61
+ )
62
+ ```
63
+
64
+ ```typescript [TypeScript]
65
+ new DirSQL(configPath: string)
66
+ // or
67
+ new DirSQL({
68
+ root?: string,
69
+ tables?: TableDef[],
70
+ ignore?: string[],
71
+ config?: string,
72
+ persist?: boolean,
73
+ persistPath?: string,
74
+ extensions?: ExtensionSpec[], // [{ path: string, entrypoint?: string }]
75
+ })
76
+ ```
77
+
78
+ ```rust [Rust]
79
+ DirSQL::builder()
80
+ .root(root) // optional
81
+ .tables(tables) // optional; append one with .table(t)
82
+ .ignore(patterns) // optional
83
+ .config(config_toml_path) // optional
84
+ .persist(true) // optional; default false
85
+ .persist_path(path) // optional
86
+ .extensions(extensions) // optional; append one with .extension(e)
87
+ .poll_interval(duration) // optional; watch-loop cadence, default 200ms
88
+ .build() // -> Result<DirSQL> (synchronous scan)
89
+ // Shortcuts: DirSQL::new(root, tables), DirSQL::with_ignore(root, tables, ignore),
90
+ // DirSQL::from_config(root), DirSQL::from_config_path(path)
91
+ ```
92
+
93
+ :::
94
+
95
+ Creates a SQLite index over a directory. At least one of `root` or `config`
96
+ must be supplied; with neither, construction fails with a "no root
97
+ directory" error.
98
+
99
+ **Parameters:**
100
+
101
+ - `root` — Directory to index. Optional if `config` is supplied.
102
+ - `tables` — Programmatic [`Table`](#table) definitions.
103
+ - `ignore` — Glob patterns matched against root-relative paths; matched
104
+ files are skipped entirely (scan and watch).
105
+ - `config` — Path to a [`.dirsql.toml`](./config.md). Its `[[table]]`
106
+ entries are appended after any programmatic `tables`; its `ignore`
107
+ patterns and `[[dirsql.extension]]` entries are appended likewise; its
108
+ `[dirsql].root` supplies the root when no explicit `root` is passed.
109
+ When both an explicit `root` and a config root are present, the explicit
110
+ value wins and a warning is emitted on stderr.
111
+ - `persist` — Keep the SQLite index on disk between runs (default off:
112
+ ephemeral, rebuilt every startup). The cache lives at
113
+ `<root>/.dirsql/cache.db` by default; on restart, only files whose stat
114
+ changed are re-parsed. `persist = true` in the config also enables it.
115
+ - `persist_path` / `persistPath` — Override the cache location. An
116
+ explicit value wins over the config's `persist_path`. Ignored when
117
+ persistence is off. (Config-file values resolve relative to the config's
118
+ parent directory; constructor values are used as given.)
119
+ - `extensions` — SQLite extensions to load at startup, before any table
120
+ DDL (enable → load → disable, so SQL `load_extension()` is never
121
+ exposed). Each entry pairs a shared-library `path` with an optional
122
+ `entrypoint` init symbol. In Python and TypeScript, `path` may be a bare
123
+ **package name**, resolved from the installed package (`importlib` /
124
+ `node_modules`) — for both the programmatic list and a `config` file's
125
+ `[[dirsql.extension]]` entries. The Rust SDK is file-path-only.
126
+ Programmatic entries load first, then the config's. A relative
127
+ programmatic path is used verbatim (resolved by the OS against the
128
+ process working directory); config-file paths resolve against the config
129
+ file's parent.
130
+
131
+ **Construction is asynchronous in Python and TypeScript.** The constructor
132
+ returns immediately and the initial scan runs in the background; `query`
133
+ and the other methods transparently await [readiness](#ready). In Rust,
134
+ `.build()` scans synchronously; use `.build_async()` → [`AsyncDirSQL`](#asyncdirsql-rust)
135
+ for the non-blocking equivalent.
136
+
137
+ ::: details Rust builder extras
138
+ - `Table` appears in the builder via `.table(t)` (append) or `.tables(v)`
139
+ (replace).
140
+ - `.poll_interval(Duration)` sets the channel-based `watch()` loop's poll
141
+ cadence (default 200 ms): lower values give tighter event latency at
142
+ higher idle CPU.
143
+ - `.suppress_config_extensions(true)` skips loading the config file's own
144
+ `[[dirsql.extension]]` entries. Launcher plumbing (used with
145
+ pre-resolved `.extensions(...)`); not needed in application code.
146
+ - `Extension { path: PathBuf, entrypoint: Option<String> }` is the
147
+ extension spec type.
148
+ :::
149
+
150
+ ### `ready`
151
+
152
+ ::: code-group
153
+
154
+ ```python [Python]
155
+ await db.ready() -> None
156
+ ```
157
+
158
+ ```typescript [TypeScript]
159
+ await db.ready // awaitable property
160
+ ```
161
+
162
+ ```rust [Rust]
163
+ // AsyncDirSQL only; DirSQL::build() is already complete when it returns.
164
+ db.ready().await -> Result<()>
165
+ ```
166
+
167
+ :::
168
+
169
+ Waits for the initial scan to complete, re-raising any construction error.
170
+ Safe to call multiple times. In Python and TypeScript every other method
171
+ awaits readiness internally, so an explicit `ready` is only needed to
172
+ observe construction errors before issuing a query.
173
+
174
+ ### `query`
175
+
176
+ ::: code-group
177
+
178
+ ```python [Python]
179
+ await db.query(sql: str) -> list[dict]
180
+ ```
181
+
182
+ ```typescript [TypeScript]
183
+ await db.query(sql: string) -> Record<string, unknown>[]
184
+ // Runs on the libuv threadpool; the JS event loop stays responsive.
185
+ ```
186
+
187
+ ```rust [Rust]
188
+ db.query(sql: &str) -> Result<Vec<Row>> // Row = HashMap<String, Value>
189
+ ```
190
+
191
+ :::
192
+
193
+ Executes a SQL query and returns rows keyed by column name.
194
+
195
+ - **Read-only.** Each statement is classified by SQLite itself
196
+ (`sqlite3_stmt_readonly`); anything classified as a write — `INSERT`,
197
+ `UPDATE`, `DELETE`, `DROP`, `CREATE`, `ALTER`, `VACUUM`, … — is rejected
198
+ before producing rows. Rust surfaces this as
199
+ `DirSqlError::WriteForbidden`; Python raises a `RuntimeError` and
200
+ TypeScript rejects with an `Error` carrying a "read-only" message.
201
+ - Internal tracking columns (`_dirsql_file_path`, `_dirsql_row_index`) are
202
+ excluded from `SELECT *` results; name them explicitly to see them.
203
+ - SQLite values map back to language types:
204
+
205
+ | SQLite | Python | TypeScript | Rust |
206
+ |---|---|---|---|
207
+ | TEXT | `str` | `string` | `Value::Text` |
208
+ | INTEGER | `int` | `number` | `Value::Integer` |
209
+ | REAL | `float` | `number` | `Value::Real` |
210
+ | BLOB | `bytes` | `Buffer` | `Value::Blob` |
211
+ | NULL | `None` | `null` | `Value::Null` |
212
+
213
+ ### `watch`
214
+
215
+ ::: code-group
216
+
217
+ ```python [Python]
218
+ async for event in db.watch(): # AsyncIterator[RowEvent]
219
+ ...
220
+ ```
221
+
222
+ ```typescript [TypeScript]
223
+ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
224
+ ...
225
+ }
226
+ ```
227
+
228
+ ```rust [Rust]
229
+ use futures::StreamExt;
230
+ let mut stream = db.watch()?; // WatchStream: impl Stream<Item = RowEvent>
231
+ while let Some(event) = stream.next().await { ... }
232
+ ```
233
+
234
+ :::
235
+
236
+ Returns a stream of [`RowEvent`](#rowevent)s reflecting filesystem changes
237
+ under the root. The watcher starts on first iteration (Python/TypeScript)
238
+ or at the `watch()` call (Rust). The stream never terminates on its own;
239
+ stop consuming it to stop.
240
+
241
+ Rust: `watch()` may be called once per instance and is mutually exclusive
242
+ with the polling API below — mixing them returns an error, since both drain
243
+ the same underlying filesystem watcher.
244
+
245
+ ### Low-level watch primitives
246
+
247
+ ::: code-group
248
+
249
+ ```typescript [TypeScript]
250
+ await db.startWatcher() // idempotent; must precede pollEvents
251
+ await db.pollEvents(timeoutMs) -> RowEvent[]
252
+ ```
253
+
254
+ ```rust [Rust]
255
+ db.start_watching() -> Result<()> // idempotent; implied by poll_events/watch
256
+ db.poll_events(timeout: Duration) -> Result<Vec<RowEvent>>
257
+ ```
258
+
259
+ :::
260
+
261
+ `pollEvents` / `poll_events` blocks up to the timeout for the first event,
262
+ then drains everything that arrived, applies it to the database, and
263
+ returns the batch (possibly empty). Safe to call in a loop. TypeScript runs
264
+ the poll on the libuv threadpool. Python does not expose public polling
265
+ primitives — use `watch()`.
266
+
267
+ ## `AsyncDirSQL` (Rust)
268
+
269
+ Rust's non-blocking wrapper: the constructor returns immediately while the
270
+ scan runs on a background thread. Python and TypeScript need no equivalent —
271
+ their `DirSQL` is already async-by-default.
272
+
273
+ ```rust
274
+ let db = DirSQL::builder().root("./data").tables(tables).build_async()?; // -> AsyncDirSQL
275
+ db.ready().await?; // required before use
276
+ let rows = db.query("SELECT ...").await?; // spawn_blocking under the hood
277
+ let stream = db.watch()?; // same WatchStream as DirSQL
278
+ let sync: DirSQL = db.sync()?; // unwrap the inner sync handle
279
+ ```
280
+
281
+ Shortcuts mirror `DirSQL`: `AsyncDirSQL::new`, `with_ignore`,
282
+ `from_config`, `from_config_path`. Unlike the Python/TypeScript SDKs,
283
+ methods called before `ready().await` completes return a "not ready" error
284
+ rather than waiting — an intentional, language-idiomatic difference.
285
+
286
+ ## `Table`
287
+
288
+ ::: code-group
289
+
290
+ ```python [Python]
291
+ Table(*, ddl: str, glob: str, extract: Callable[[str], list[dict]], strict: bool = False)
292
+ ```
293
+
294
+ ```typescript [TypeScript]
295
+ new Table({ ddl, glob, extract, strict? })
296
+ // or a plain object — TableDef and Table are interchangeable:
297
+ { ddl: string, glob: string, extract: (path: string) => Record<string, unknown>[], strict?: boolean }
298
+ ```
299
+
300
+ ```rust [Rust]
301
+ Table::new(ddl, glob, extract) // extract: Fn(&str) -> Vec<Row>, infallible
302
+ Table::try_new(ddl, glob, extract) // extract: Fn(&str) -> Result<Vec<Row>, _>
303
+ Table::strict(ddl, glob, extract) // Table::new with strict = true
304
+ ```
305
+
306
+ :::
307
+
308
+ Maps files to table rows.
309
+
310
+ - `ddl` — A SQLite `CREATE TABLE` statement; the table name is parsed from
311
+ it. Table names must be unique across all tables.
312
+ - `glob` — Glob pattern matched against root-relative paths. May contain
313
+ `{name}` [captures](./columns.md#glob-captures).
314
+ - `extract` — Callback receiving the matched file's full path (the root
315
+ joined with the file's relative path — absolute when `root` is absolute)
316
+ and returning the rows that file contributes. `dirsql` never reads file
317
+ contents itself; a callback that needs the body reads the path. Return
318
+ an empty list to skip a file. [Virtual columns and glob
319
+ captures](./columns.md) are merged onto each returned row; values the
320
+ callback emits win over same-named facts.
321
+ - `strict` — Default off: extra row keys are dropped and missing declared
322
+ columns become `NULL`. When on, any extra or missing key is an error
323
+ (see [`strict`](./config.md#table)).
324
+
325
+ Readable attributes: `ddl`, `glob` (all SDKs), plus `strict`.
326
+
327
+ ## `RowEvent`
328
+
329
+ Emitted by [`watch`](#watch) and the polling primitives; one event per
330
+ row-level change caused by a filesystem event.
331
+
332
+ | Field | Python | Rust | TypeScript |
333
+ |---|---|---|---|
334
+ | Action | `action: str` | enum variant (`Insert` / `Update` / `Delete` / `Error`) | `action: string` |
335
+ | Table | `table: str \| None` | `table: String` (`Option<String>` on `Error`) | `table: string \| null` |
336
+ | New/current row | `row: dict \| None` | `row` / `new_row` on the variant | `row?: Record \| null` |
337
+ | Previous row | `old_row: dict \| None` | `old_row` on `Update` | `oldRow?: Record \| null` |
338
+ | Error message | `error: str \| None` | `error: String` on `Error` | `error?: string \| null` |
339
+ | File path | `file_path: str \| None` | `file_path` (`String`; `PathBuf` on `Error`) | `filePath?: string \| null` |
340
+
341
+ Actions: `insert` (new row; no previous), `update` (row changed in place;
342
+ carries both old and new), `delete` (row removed), `error` (an extraction
343
+ or watch failure — carries `error`, never `row`/`old_row`; `table` is
344
+ `None`/`null` when the failure isn't tied to one table). `file_path` is
345
+ relative to the root. Errors never terminate the stream.
346
+
347
+ ## Value types
348
+
349
+ An `extract` callback (and the SQLite columns it feeds) accepts:
350
+
351
+ | SQLite column | Python value | TypeScript value | Rust `Value` |
352
+ |---|---|---|---|
353
+ | TEXT | `str` | `string` | `Value::Text(String)` |
354
+ | INTEGER | `int`, `bool` | `number`, `boolean` | `Value::Integer(i64)` |
355
+ | REAL | `float` | `number` | `Value::Real(f64)` |
356
+ | BLOB | `bytes` | `Buffer` / `Uint8Array` | `Value::Blob(Vec<u8>)` |
357
+ | NULL | `None` | `null` | `Value::Null` |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dirsql",
3
- "version": "0.3.69",
3
+ "version": "0.3.71",
4
4
  "description": "Ephemeral SQL index over a local directory",
5
5
  "license": "MIT",
6
6
  "repository": "https://github.com/thekevinscott/dirsql",
@@ -230,15 +230,15 @@
230
230
  ]
231
231
  },
232
232
  "optionalDependencies": {
233
- "@dirsql/lib-linux-x64-gnu": "0.3.69",
234
- "@dirsql/lib-linux-arm64-gnu": "0.3.69",
235
- "@dirsql/lib-darwin-x64": "0.3.69",
236
- "@dirsql/lib-darwin-arm64": "0.3.69",
237
- "@dirsql/lib-win32-x64-msvc": "0.3.69",
238
- "@dirsql/cli-linux-x64-gnu": "0.3.69",
239
- "@dirsql/cli-linux-arm64-gnu": "0.3.69",
240
- "@dirsql/cli-darwin-x64": "0.3.69",
241
- "@dirsql/cli-darwin-arm64": "0.3.69",
242
- "@dirsql/cli-win32-x64-msvc": "0.3.69"
233
+ "@dirsql/lib-linux-x64-gnu": "0.3.71",
234
+ "@dirsql/lib-linux-arm64-gnu": "0.3.71",
235
+ "@dirsql/lib-darwin-x64": "0.3.71",
236
+ "@dirsql/lib-darwin-arm64": "0.3.71",
237
+ "@dirsql/lib-win32-x64-msvc": "0.3.71",
238
+ "@dirsql/cli-linux-x64-gnu": "0.3.71",
239
+ "@dirsql/cli-linux-arm64-gnu": "0.3.71",
240
+ "@dirsql/cli-darwin-x64": "0.3.71",
241
+ "@dirsql/cli-darwin-arm64": "0.3.71",
242
+ "@dirsql/cli-win32-x64-msvc": "0.3.71"
243
243
  }
244
244
  }
@@ -1,2 +0,0 @@
1
- export {};
2
- //# sourceMappingURL=index.d.ts.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/cli/interpret/index.ts"],"names":[],"mappings":"AAaA,OAAO,EAAE,CAAC"}
@@ -1,15 +0,0 @@
1
- // Empty package shell — the native-config `interpret` subcommand was removed.
2
- //
3
- // The `interpret` NDJSON helper (`interpret`, `load-app`, `dispatch-extract`,
4
- // `build-tables`, `err-message`, `write-message`) was removed in #321 (#324).
5
- // The CLI now accepts only `.dirsql.toml`; to run user-defined `extract`
6
- // callbacks, use the programmatic SDK (`new DirSQL(...)` with in-process
7
- // closures).
8
- //
9
- // This file carries no logic and no re-exports. It remains only because the
10
- // colocated-test tooling cannot yet express *deleting* an exempt barrel (the
11
- // co-change check flags a deleted source that has no co-deleted colocated
12
- // test, and a retained exempt for a deleted path is rejected as stale). The
13
- // directory is removed once that is resolved.
14
- export {};
15
- //# sourceMappingURL=index.js.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/cli/interpret/index.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,EAAE;AACF,8EAA8E;AAC9E,8EAA8E;AAC9E,yEAAyE;AACzE,yEAAyE;AACzE,aAAa;AACb,EAAE;AACF,4EAA4E;AAC5E,6EAA6E;AAC7E,0EAA0E;AAC1E,4EAA4E;AAC5E,8CAA8C;AAC9C,OAAO,EAAE,CAAC"}