dirsql 0.3.42__tar.gz → 0.3.43__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (119) hide show
  1. {dirsql-0.3.42 → dirsql-0.3.43}/Cargo.lock +1 -1
  2. {dirsql-0.3.42 → dirsql-0.3.43}/PKG-INFO +1 -1
  3. {dirsql-0.3.42/packages/python → dirsql-0.3.43}/docs/AGENTS.md +5 -7
  4. {dirsql-0.3.42 → dirsql-0.3.43}/docs/api/index.md +1 -21
  5. {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/config.md +6 -118
  6. {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/index.md +2 -3
  7. {dirsql-0.3.42/packages/rust → dirsql-0.3.43}/docs/cli/server.md +2 -3
  8. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/Cargo.toml +1 -1
  9. {dirsql-0.3.42 → dirsql-0.3.43/packages/python}/docs/AGENTS.md +5 -7
  10. {dirsql-0.3.42/packages/rust → dirsql-0.3.43/packages/python}/docs/api/index.md +1 -21
  11. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/config.md +6 -118
  12. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/index.md +2 -3
  13. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/server.md +2 -3
  14. {dirsql-0.3.42/packages/python → dirsql-0.3.43/packages/rust}/docs/api/index.md +1 -21
  15. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/config.md +6 -118
  16. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/index.md +2 -3
  17. {dirsql-0.3.42 → dirsql-0.3.43/packages/rust}/docs/cli/server.md +2 -3
  18. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/bin/dirsql.rs +21 -84
  19. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/mod.rs +0 -1
  20. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/lib.rs +2 -99
  21. dirsql-0.3.42/packages/rust/src/cli/native_config.rs +0 -582
  22. {dirsql-0.3.42 → dirsql-0.3.43}/Cargo.toml +0 -0
  23. {dirsql-0.3.42 → dirsql-0.3.43}/README.md +0 -0
  24. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/__init__.py +0 -0
  25. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/_async.py +0 -0
  26. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/_dirsql.pyi +0 -0
  27. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/__init__.py +0 -0
  28. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/binary_path.py +0 -0
  29. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/interpret/__init__.py +0 -0
  30. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/is_windows.py +0 -0
  31. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/main.py +0 -0
  32. {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/py.typed +0 -0
  33. {dirsql-0.3.42 → dirsql-0.3.43}/docs/.claude/CLAUDE.md +0 -0
  34. {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/config.ts +0 -0
  35. {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/theme/index.ts +0 -0
  36. {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/theme/lang.ts +0 -0
  37. {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/http-api.md +0 -0
  38. {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/init.md +0 -0
  39. {dirsql-0.3.42 → dirsql-0.3.43}/docs/getting-started.md +0 -0
  40. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/async.md +0 -0
  41. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/crdt.md +0 -0
  42. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/persistence.md +0 -0
  43. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/querying.md +0 -0
  44. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/tables.md +0 -0
  45. {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/watching.md +0 -0
  46. {dirsql-0.3.42 → dirsql-0.3.43}/docs/index.md +0 -0
  47. {dirsql-0.3.42 → dirsql-0.3.43}/docs/migrations.md +0 -0
  48. {dirsql-0.3.42 → dirsql-0.3.43}/docs/package.json +0 -0
  49. {dirsql-0.3.42 → dirsql-0.3.43}/docs/playwright.config.ts +0 -0
  50. {dirsql-0.3.42 → dirsql-0.3.43}/docs/pnpm-lock.yaml +0 -0
  51. {dirsql-0.3.42 → dirsql-0.3.43}/docs/pnpm-workspace.yaml +0 -0
  52. {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/home.spec.ts +0 -0
  53. {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/language-flag.spec.ts +0 -0
  54. {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/sidebar.spec.ts +0 -0
  55. {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/unit/config.test.ts +0 -0
  56. {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/unit/lang.test.ts +0 -0
  57. {dirsql-0.3.42 → dirsql-0.3.43}/docs/vitest.config.ts +0 -0
  58. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/README.md +0 -0
  59. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/conftest.py +0 -0
  60. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.claude/CLAUDE.md +0 -0
  61. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/config.ts +0 -0
  62. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/theme/index.ts +0 -0
  63. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/theme/lang.ts +0 -0
  64. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/http-api.md +0 -0
  65. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/init.md +0 -0
  66. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/getting-started.md +0 -0
  67. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/async.md +0 -0
  68. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/crdt.md +0 -0
  69. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/persistence.md +0 -0
  70. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/querying.md +0 -0
  71. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/tables.md +0 -0
  72. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/watching.md +0 -0
  73. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/index.md +0 -0
  74. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/migrations.md +0 -0
  75. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/package.json +0 -0
  76. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/playwright.config.ts +0 -0
  77. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/pnpm-lock.yaml +0 -0
  78. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/pnpm-workspace.yaml +0 -0
  79. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/home.spec.ts +0 -0
  80. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/language-flag.spec.ts +0 -0
  81. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/sidebar.spec.ts +0 -0
  82. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/unit/config.test.ts +0 -0
  83. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/unit/lang.test.ts +0 -0
  84. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/vitest.config.ts +0 -0
  85. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/e2e-attestation.json +0 -0
  86. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/src/lib.rs +0 -0
  87. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/__init__.py +0 -0
  88. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/conftest.py +0 -0
  89. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/e2e/__init__.py +0 -0
  90. {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/integration/__init__.py +0 -0
  91. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/Cargo.toml +0 -0
  92. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/README.md +0 -0
  93. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/db_bench.rs +0 -0
  94. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/differ_bench.rs +0 -0
  95. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/matcher_bench.rs +0 -0
  96. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/scanner_bench.rs +0 -0
  97. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/http-api.md +0 -0
  98. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/init.md +0 -0
  99. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/getting-started.md +0 -0
  100. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/async.md +0 -0
  101. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/crdt.md +0 -0
  102. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/persistence.md +0 -0
  103. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/querying.md +0 -0
  104. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/tables.md +0 -0
  105. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/watching.md +0 -0
  106. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/index.md +0 -0
  107. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/migrations.md +0 -0
  108. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/init.rs +0 -0
  109. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/router.rs +0 -0
  110. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/serialize.rs +0 -0
  111. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/server.rs +0 -0
  112. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/config.rs +0 -0
  113. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/db.rs +0 -0
  114. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/differ.rs +0 -0
  115. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/matcher.rs +0 -0
  116. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/persist.rs +0 -0
  117. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/scanner.rs +0 -0
  118. {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/watcher.rs +0 -0
  119. {dirsql-0.3.42 → dirsql-0.3.43}/pyproject.toml +0 -0
@@ -499,7 +499,7 @@ dependencies = [
499
499
 
500
500
  [[package]]
501
501
  name = "dirsql-py-ext"
502
- version = "0.3.42"
502
+ version = "0.3.43"
503
503
  dependencies = [
504
504
  "dirsql",
505
505
  "pyo3",
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dirsql
3
- Version: 0.3.42
3
+ Version: 0.3.43
4
4
  Requires-Dist: pytest>=8 ; extra == 'dev'
5
5
  Requires-Dist: pytest-describe>=2 ; extra == 'dev'
6
6
  Requires-Dist: pytest-asyncio>=0.23 ; extra == 'dev'
@@ -45,19 +45,17 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
45
45
  The **CLI** (`cli/`) is a self-contained section reachable from its own
46
46
  top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
47
47
  `config.ts`). Everything a CLI user needs -- installation, running the server,
48
- `init`, config files (`.dirsql.toml` or native-language), and the HTTP API --
49
- lives under `cli/`. Do not move CLI pages back into `guide/`.
48
+ `init`, the `.dirsql.toml` config file, and the HTTP API -- lives under
49
+ `cli/`. Do not move CLI pages back into `guide/`.
50
50
 
51
51
  ## Conventions
52
52
 
53
53
  - **Lead with the use case.** Open each feature description with *why* a
54
54
  reader would reach for it before *how* it works. Don't frame a feature
55
55
  by what an adjacent feature can't do.
56
- *Don't:* "`.dirsql.toml` can't parse contents you need code in
57
- `extract`. Point `--config` at a Python module..."
58
- *Do:* "Native-language configs let you build tables from the
59
- *contents* of files — frontmatter, JSON values, CSV cells — by
60
- writing a dynamic `extract` callback."
56
+ *Don't:* "Persistence avoids the thing the default mode can't do..."
57
+ *Do:* "Persistence keeps the SQLite index on disk between runs so large
58
+ directories don't re-scan on every startup."
61
59
  - Wrap `dirsql` in backticks in all prose text
62
60
  - Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
63
61
  - Internal links use relative paths (e.g., `./guide/tables.md`)
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
145
145
 
146
146
  Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
147
147
 
148
- #### `serialization`
149
-
150
- ::: code-group
151
-
152
- ```python [Python]
153
- vars(db) -> dict
154
- ```
155
-
156
- ```rust [Rust]
157
- db.config() -> DirSQLConfig
158
- ```
159
-
160
- ```typescript [TypeScript]
161
- JSON.stringify(db) // via db.toJSON()
162
- ```
163
-
164
- :::
165
-
166
- Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
167
-
168
148
  ---
169
149
 
170
150
  ## Table
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
213
193
  - `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
214
194
  - `glob` -- A glob pattern matched against file paths relative to the root directory.
215
195
  - `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
216
- - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error. Surfaced in [serialization](#serialization) above as part of each table's `{ ddl, glob, strict }`.
196
+ - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
217
197
 
218
198
  **Attributes:**
219
199
 
@@ -6,16 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
6
6
 
7
7
  > Online: <https://thekevinscott.github.io/dirsql/cli/config>
8
8
 
9
- `dirsql` is configured with an optional config file; with none, the server
10
- falls back to [zero-config defaults](./server.md#defaults). Choose a format by
11
- what you need:
12
-
13
- - **[TOML](#toml)** — declarative; defines filesystem-fact tables (the path,
14
- glob captures, and stat metadata). Works with any installation.
15
- - **[Python](#python)** and **[JavaScript](#javascript)** — native-language
16
- configs that build tables from the *contents* of files (frontmatter, JSON
17
- values, CSV cells) through a dynamic `extract` callback. CLI-only; only the
18
- launcher matching the file's language can run it.
9
+ `dirsql` is configured with an optional `.dirsql.toml` file; with none, the
10
+ server falls back to [zero-config defaults](./server.md#defaults). A
11
+ [TOML](#toml) config is declarative: it defines filesystem-fact tables (the
12
+ path, glob captures, and stat metadata) and works with any installation.
19
13
 
20
14
  ## TOML
21
15
 
@@ -194,9 +188,8 @@ strict = true
194
188
 
195
189
  Strict mode does **not** apply to auto-injected stat virtuals — those are
196
190
  always filtered to the DDL's declared columns regardless. Strict mode
197
- applies only to keys produced by an extract callback (relevant for the
198
- [Python](#python) / [JavaScript](#javascript) configs below and programmatic
199
- [tables](../guide/tables.md)).
191
+ applies only to keys produced by an extract callback (relevant for
192
+ programmatic [tables](../guide/tables.md)).
200
193
 
201
194
  ### Full Example
202
195
 
@@ -216,108 +209,3 @@ glob = "**/index.md"
216
209
  ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
217
210
  glob = "logs/*.csv"
218
211
  ```
219
-
220
- ## Python
221
-
222
- Reach for a Python config when your columns come from the *contents* of a
223
- file — parsed JSON, frontmatter, CSV cells — rather than from filesystem
224
- facts alone. You write a dynamic `extract` callback in Python, and the file
225
- otherwise looks exactly like the in-process SDK construction (same `DirSQL` /
226
- `Table` API):
227
-
228
- ```bash
229
- dirsql --config dirsql.config.py
230
- ```
231
-
232
- ```python [dirsql.config.py]
233
- import json
234
- from dirsql import DirSQL, Table
235
-
236
- def extract_meta(path):
237
- with open(path) as f:
238
- return [json.load(f)]
239
-
240
- # Python must export a module-level `app`.
241
- app = DirSQL(
242
- root="papers", # optional; defaults to the current directory
243
- tables=[
244
- Table(
245
- ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
246
- glob="**/meta.json",
247
- extract=extract_meta,
248
- ),
249
- ],
250
- )
251
- ```
252
-
253
- `extract` receives the path of each matched file and returns a list of rows
254
- (one dict per row).
255
-
256
- ## JavaScript
257
-
258
- A JavaScript config gives you the same contents-driven `extract` in Node,
259
- in either ES module or CommonJS form:
260
-
261
- ```bash
262
- dirsql --config dirsql.config.mjs
263
- ```
264
-
265
- ::: code-group
266
-
267
- ```javascript [dirsql.config.mjs]
268
- import { readFileSync } from "node:fs";
269
- import { DirSQL } from "dirsql";
270
-
271
- export default new DirSQL({
272
- root: "papers", // optional; defaults to the current directory
273
- tables: [
274
- {
275
- ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
276
- glob: "**/meta.json",
277
- extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
278
- },
279
- ],
280
- });
281
- ```
282
-
283
- ```javascript [dirsql.config.cjs]
284
- const { readFileSync } = require("node:fs");
285
- const { DirSQL } = require("dirsql");
286
-
287
- module.exports = new DirSQL({
288
- root: "papers", // optional; defaults to the current directory
289
- tables: [
290
- {
291
- ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
292
- glob: "**/meta.json",
293
- extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
294
- },
295
- ],
296
- });
297
- ```
298
-
299
- :::
300
-
301
- ## Notes for native-language configs
302
-
303
- These apply to both the Python and JavaScript forms above.
304
-
305
- - **Export the config.** Python exposes a module-level `app = DirSQL(...)`; an
306
- ES module (`.mjs`, or `.js` in an ESM package) uses
307
- `export default new DirSQL(...)`; CommonJS (`.cjs`, or `.js` in a CJS
308
- package) uses `module.exports = new DirSQL(...)`. Only the extension
309
- matters — the file can be named anything; `dirsql.config.{py,mjs,cjs}` is the
310
- suggested convention, not a requirement.
311
- - **`root` defaults to the current directory.** A native-language config with
312
- no `root` indexes the process's current working directory — the directory you
313
- ran `dirsql` from. (This differs from TOML configs, which default the scan
314
- root to the config file's own directory.) Pass `root` explicitly to index
315
- somewhere else.
316
- - **No nested `config=`.** A native-language config builds its `DirSQL` from
317
- `tables` and an optional `root`; it must not itself set `config=` to delegate
318
- to another config file. `dirsql interpret` rejects such a config (a nested
319
- config can't be represented in the handshake and would recurse).
320
- - **Install the launcher on your `PATH`.** To run your `extract`, the server
321
- spawns `dirsql interpret`, so the matching `dirsql` launcher must be installed
322
- and on your `PATH` — a global `pip`/`uv` install for `.py`, or `npm` for
323
- `.mjs` / `.cjs`. Only the launcher matching the file's language can run it.
@@ -17,9 +17,8 @@ Everything you need to run `dirsql` as a CLI lives in this section:
17
17
  - **[Installation](#installation)** — get the `dirsql` binary.
18
18
  - **[Running the Server](./server.md)** — subcommands and flags.
19
19
  - **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
20
- - **[Configuration File](./config.md)** — the `.dirsql.toml` format and the
21
- `.py` / `.js` native-language alternative. Custom tables
22
- are defined through a config file; without one, the server runs in
20
+ - **[Configuration File](./config.md)** — the `.dirsql.toml` format. Custom
21
+ tables are defined through a config file; without one, the server runs in
23
22
  [zero-config mode](./server.md#defaults).
24
23
  - **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
25
24
  endpoints, status codes, and event streaming.
@@ -28,8 +28,7 @@ $ Running at localhost:7117
28
28
 
29
29
  The server reads tables from a [config file](./config.md). By default it
30
30
  looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
31
- `.toml` file or a [native-language config](./config.md)
32
- (`.py` / `.js`).
31
+ `.toml` file.
33
32
 
34
33
  ## Defaults
35
34
 
@@ -53,7 +52,7 @@ A config file will override the default.
53
52
 
54
53
  | Flag | Default | Description |
55
54
  |---|---|---|
56
- | `--config <path>` | `./.dirsql.toml` | Path to the config file (`.toml` or [native-language](./config.md)). The index is rooted at the directory containing this file. |
55
+ | `--config <path>` | `./.dirsql.toml` | Path to the `.toml` [config file](./config.md). The index is rooted at the directory containing this file. |
57
56
  | `--host <addr>` | `localhost` | Bind address |
58
57
  | `--port <n>` | `7117` | TCP port to bind |
59
58
 
@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
4
4
  # pypi/maturin handler can rewrite it via `write-version` before
5
5
  # `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
6
6
  # and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
7
- version = "0.3.42"
7
+ version = "0.3.43"
8
8
  edition.workspace = true
9
9
  publish = false
10
10
  readme = "README.md"
@@ -45,19 +45,17 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
45
45
  The **CLI** (`cli/`) is a self-contained section reachable from its own
46
46
  top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
47
47
  `config.ts`). Everything a CLI user needs -- installation, running the server,
48
- `init`, config files (`.dirsql.toml` or native-language), and the HTTP API --
49
- lives under `cli/`. Do not move CLI pages back into `guide/`.
48
+ `init`, the `.dirsql.toml` config file, and the HTTP API -- lives under
49
+ `cli/`. Do not move CLI pages back into `guide/`.
50
50
 
51
51
  ## Conventions
52
52
 
53
53
  - **Lead with the use case.** Open each feature description with *why* a
54
54
  reader would reach for it before *how* it works. Don't frame a feature
55
55
  by what an adjacent feature can't do.
56
- *Don't:* "`.dirsql.toml` can't parse contents you need code in
57
- `extract`. Point `--config` at a Python module..."
58
- *Do:* "Native-language configs let you build tables from the
59
- *contents* of files — frontmatter, JSON values, CSV cells — by
60
- writing a dynamic `extract` callback."
56
+ *Don't:* "Persistence avoids the thing the default mode can't do..."
57
+ *Do:* "Persistence keeps the SQLite index on disk between runs so large
58
+ directories don't re-scan on every startup."
61
59
  - Wrap `dirsql` in backticks in all prose text
62
60
  - Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
63
61
  - Internal links use relative paths (e.g., `./guide/tables.md`)
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
145
145
 
146
146
  Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
147
147
 
148
- #### `serialization`
149
-
150
- ::: code-group
151
-
152
- ```python [Python]
153
- vars(db) -> dict
154
- ```
155
-
156
- ```rust [Rust]
157
- db.config() -> DirSQLConfig
158
- ```
159
-
160
- ```typescript [TypeScript]
161
- JSON.stringify(db) // via db.toJSON()
162
- ```
163
-
164
- :::
165
-
166
- Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
167
-
168
148
  ---
169
149
 
170
150
  ## Table
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
213
193
  - `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
214
194
  - `glob` -- A glob pattern matched against file paths relative to the root directory.
215
195
  - `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
216
- - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error. Surfaced in [serialization](#serialization) above as part of each table's `{ ddl, glob, strict }`.
196
+ - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
217
197
 
218
198
  **Attributes:**
219
199
 
@@ -6,16 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
6
6
 
7
7
  > Online: <https://thekevinscott.github.io/dirsql/cli/config>
8
8
 
9
- `dirsql` is configured with an optional config file; with none, the server
10
- falls back to [zero-config defaults](./server.md#defaults). Choose a format by
11
- what you need:
12
-
13
- - **[TOML](#toml)** — declarative; defines filesystem-fact tables (the path,
14
- glob captures, and stat metadata). Works with any installation.
15
- - **[Python](#python)** and **[JavaScript](#javascript)** — native-language
16
- configs that build tables from the *contents* of files (frontmatter, JSON
17
- values, CSV cells) through a dynamic `extract` callback. CLI-only; only the
18
- launcher matching the file's language can run it.
9
+ `dirsql` is configured with an optional `.dirsql.toml` file; with none, the
10
+ server falls back to [zero-config defaults](./server.md#defaults). A
11
+ [TOML](#toml) config is declarative: it defines filesystem-fact tables (the
12
+ path, glob captures, and stat metadata) and works with any installation.
19
13
 
20
14
  ## TOML
21
15
 
@@ -194,9 +188,8 @@ strict = true
194
188
 
195
189
  Strict mode does **not** apply to auto-injected stat virtuals — those are
196
190
  always filtered to the DDL's declared columns regardless. Strict mode
197
- applies only to keys produced by an extract callback (relevant for the
198
- [Python](#python) / [JavaScript](#javascript) configs below and programmatic
199
- [tables](../guide/tables.md)).
191
+ applies only to keys produced by an extract callback (relevant for
192
+ programmatic [tables](../guide/tables.md)).
200
193
 
201
194
  ### Full Example
202
195
 
@@ -216,108 +209,3 @@ glob = "**/index.md"
216
209
  ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
217
210
  glob = "logs/*.csv"
218
211
  ```
219
-
220
- ## Python
221
-
222
- Reach for a Python config when your columns come from the *contents* of a
223
- file — parsed JSON, frontmatter, CSV cells — rather than from filesystem
224
- facts alone. You write a dynamic `extract` callback in Python, and the file
225
- otherwise looks exactly like the in-process SDK construction (same `DirSQL` /
226
- `Table` API):
227
-
228
- ```bash
229
- dirsql --config dirsql.config.py
230
- ```
231
-
232
- ```python [dirsql.config.py]
233
- import json
234
- from dirsql import DirSQL, Table
235
-
236
- def extract_meta(path):
237
- with open(path) as f:
238
- return [json.load(f)]
239
-
240
- # Python must export a module-level `app`.
241
- app = DirSQL(
242
- root="papers", # optional; defaults to the current directory
243
- tables=[
244
- Table(
245
- ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
246
- glob="**/meta.json",
247
- extract=extract_meta,
248
- ),
249
- ],
250
- )
251
- ```
252
-
253
- `extract` receives the path of each matched file and returns a list of rows
254
- (one dict per row).
255
-
256
- ## JavaScript
257
-
258
- A JavaScript config gives you the same contents-driven `extract` in Node,
259
- in either ES module or CommonJS form:
260
-
261
- ```bash
262
- dirsql --config dirsql.config.mjs
263
- ```
264
-
265
- ::: code-group
266
-
267
- ```javascript [dirsql.config.mjs]
268
- import { readFileSync } from "node:fs";
269
- import { DirSQL } from "dirsql";
270
-
271
- export default new DirSQL({
272
- root: "papers", // optional; defaults to the current directory
273
- tables: [
274
- {
275
- ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
276
- glob: "**/meta.json",
277
- extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
278
- },
279
- ],
280
- });
281
- ```
282
-
283
- ```javascript [dirsql.config.cjs]
284
- const { readFileSync } = require("node:fs");
285
- const { DirSQL } = require("dirsql");
286
-
287
- module.exports = new DirSQL({
288
- root: "papers", // optional; defaults to the current directory
289
- tables: [
290
- {
291
- ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
292
- glob: "**/meta.json",
293
- extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
294
- },
295
- ],
296
- });
297
- ```
298
-
299
- :::
300
-
301
- ## Notes for native-language configs
302
-
303
- These apply to both the Python and JavaScript forms above.
304
-
305
- - **Export the config.** Python exposes a module-level `app = DirSQL(...)`; an
306
- ES module (`.mjs`, or `.js` in an ESM package) uses
307
- `export default new DirSQL(...)`; CommonJS (`.cjs`, or `.js` in a CJS
308
- package) uses `module.exports = new DirSQL(...)`. Only the extension
309
- matters — the file can be named anything; `dirsql.config.{py,mjs,cjs}` is the
310
- suggested convention, not a requirement.
311
- - **`root` defaults to the current directory.** A native-language config with
312
- no `root` indexes the process's current working directory — the directory you
313
- ran `dirsql` from. (This differs from TOML configs, which default the scan
314
- root to the config file's own directory.) Pass `root` explicitly to index
315
- somewhere else.
316
- - **No nested `config=`.** A native-language config builds its `DirSQL` from
317
- `tables` and an optional `root`; it must not itself set `config=` to delegate
318
- to another config file. `dirsql interpret` rejects such a config (a nested
319
- config can't be represented in the handshake and would recurse).
320
- - **Install the launcher on your `PATH`.** To run your `extract`, the server
321
- spawns `dirsql interpret`, so the matching `dirsql` launcher must be installed
322
- and on your `PATH` — a global `pip`/`uv` install for `.py`, or `npm` for
323
- `.mjs` / `.cjs`. Only the launcher matching the file's language can run it.
@@ -17,9 +17,8 @@ Everything you need to run `dirsql` as a CLI lives in this section:
17
17
  - **[Installation](#installation)** — get the `dirsql` binary.
18
18
  - **[Running the Server](./server.md)** — subcommands and flags.
19
19
  - **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
20
- - **[Configuration File](./config.md)** — the `.dirsql.toml` format and the
21
- `.py` / `.js` native-language alternative. Custom tables
22
- are defined through a config file; without one, the server runs in
20
+ - **[Configuration File](./config.md)** — the `.dirsql.toml` format. Custom
21
+ tables are defined through a config file; without one, the server runs in
23
22
  [zero-config mode](./server.md#defaults).
24
23
  - **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
25
24
  endpoints, status codes, and event streaming.
@@ -28,8 +28,7 @@ $ Running at localhost:7117
28
28
 
29
29
  The server reads tables from a [config file](./config.md). By default it
30
30
  looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
31
- `.toml` file or a [native-language config](./config.md)
32
- (`.py` / `.js`).
31
+ `.toml` file.
33
32
 
34
33
  ## Defaults
35
34
 
@@ -53,7 +52,7 @@ A config file will override the default.
53
52
 
54
53
  | Flag | Default | Description |
55
54
  |---|---|---|
56
- | `--config <path>` | `./.dirsql.toml` | Path to the config file (`.toml` or [native-language](./config.md)). The index is rooted at the directory containing this file. |
55
+ | `--config <path>` | `./.dirsql.toml` | Path to the `.toml` [config file](./config.md). The index is rooted at the directory containing this file. |
57
56
  | `--host <addr>` | `localhost` | Bind address |
58
57
  | `--port <n>` | `7117` | TCP port to bind |
59
58
 
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
145
145
 
146
146
  Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
147
147
 
148
- #### `serialization`
149
-
150
- ::: code-group
151
-
152
- ```python [Python]
153
- vars(db) -> dict
154
- ```
155
-
156
- ```rust [Rust]
157
- db.config() -> DirSQLConfig
158
- ```
159
-
160
- ```typescript [TypeScript]
161
- JSON.stringify(db) // via db.toJSON()
162
- ```
163
-
164
- :::
165
-
166
- Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
167
-
168
148
  ---
169
149
 
170
150
  ## Table
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
213
193
  - `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
214
194
  - `glob` -- A glob pattern matched against file paths relative to the root directory.
215
195
  - `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
216
- - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error. Surfaced in [serialization](#serialization) above as part of each table's `{ ddl, glob, strict }`.
196
+ - `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
217
197
 
218
198
  **Attributes:**
219
199