dirsql 0.3.42__tar.gz → 0.3.43__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dirsql-0.3.42 → dirsql-0.3.43}/Cargo.lock +1 -1
- {dirsql-0.3.42 → dirsql-0.3.43}/PKG-INFO +1 -1
- {dirsql-0.3.42/packages/python → dirsql-0.3.43}/docs/AGENTS.md +5 -7
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/api/index.md +1 -21
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/config.md +6 -118
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/index.md +2 -3
- {dirsql-0.3.42/packages/rust → dirsql-0.3.43}/docs/cli/server.md +2 -3
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/Cargo.toml +1 -1
- {dirsql-0.3.42 → dirsql-0.3.43/packages/python}/docs/AGENTS.md +5 -7
- {dirsql-0.3.42/packages/rust → dirsql-0.3.43/packages/python}/docs/api/index.md +1 -21
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/config.md +6 -118
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/index.md +2 -3
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/server.md +2 -3
- {dirsql-0.3.42/packages/python → dirsql-0.3.43/packages/rust}/docs/api/index.md +1 -21
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/config.md +6 -118
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/index.md +2 -3
- {dirsql-0.3.42 → dirsql-0.3.43/packages/rust}/docs/cli/server.md +2 -3
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/bin/dirsql.rs +21 -84
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/mod.rs +0 -1
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/lib.rs +2 -99
- dirsql-0.3.42/packages/rust/src/cli/native_config.rs +0 -582
- {dirsql-0.3.42 → dirsql-0.3.43}/Cargo.toml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/README.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/_async.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/_dirsql.pyi +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/binary_path.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/interpret/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/is_windows.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/cli/main.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/dirsql/py.typed +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/http-api.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/cli/init.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/getting-started.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/async.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/crdt.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/persistence.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/querying.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/tables.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/guide/watching.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/index.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/migrations.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/package.json +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/playwright.config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/pnpm-lock.yaml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/home.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/language-flag.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/integration/sidebar.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/unit/config.test.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/tests/unit/lang.test.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/docs/vitest.config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/README.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/conftest.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/http-api.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/cli/init.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/getting-started.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/async.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/crdt.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/persistence.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/querying.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/tables.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/guide/watching.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/index.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/migrations.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/package.json +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/playwright.config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/pnpm-lock.yaml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/home.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/language-flag.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/integration/sidebar.spec.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/unit/config.test.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/tests/unit/lang.test.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/docs/vitest.config.ts +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/e2e-attestation.json +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/src/lib.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/conftest.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/e2e/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/python/tests/integration/__init__.py +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/Cargo.toml +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/README.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/db_bench.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/differ_bench.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/matcher_bench.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/benches/scanner_bench.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/http-api.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/cli/init.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/getting-started.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/async.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/crdt.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/persistence.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/querying.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/tables.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/guide/watching.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/index.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/docs/migrations.md +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/init.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/router.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/serialize.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/cli/server.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/config.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/db.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/differ.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/matcher.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/persist.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/scanner.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/packages/rust/src/watcher.rs +0 -0
- {dirsql-0.3.42 → dirsql-0.3.43}/pyproject.toml +0 -0
|
@@ -45,19 +45,17 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
|
|
|
45
45
|
The **CLI** (`cli/`) is a self-contained section reachable from its own
|
|
46
46
|
top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
|
|
47
47
|
`config.ts`). Everything a CLI user needs -- installation, running the server,
|
|
48
|
-
`init`,
|
|
49
|
-
|
|
48
|
+
`init`, the `.dirsql.toml` config file, and the HTTP API -- lives under
|
|
49
|
+
`cli/`. Do not move CLI pages back into `guide/`.
|
|
50
50
|
|
|
51
51
|
## Conventions
|
|
52
52
|
|
|
53
53
|
- **Lead with the use case.** Open each feature description with *why* a
|
|
54
54
|
reader would reach for it before *how* it works. Don't frame a feature
|
|
55
55
|
by what an adjacent feature can't do.
|
|
56
|
-
*Don't:* "
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
*contents* of files — frontmatter, JSON values, CSV cells — by
|
|
60
|
-
writing a dynamic `extract` callback."
|
|
56
|
+
*Don't:* "Persistence avoids the thing the default mode can't do..."
|
|
57
|
+
*Do:* "Persistence keeps the SQLite index on disk between runs so large
|
|
58
|
+
directories don't re-scan on every startup."
|
|
61
59
|
- Wrap `dirsql` in backticks in all prose text
|
|
62
60
|
- Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
|
|
63
61
|
- Internal links use relative paths (e.g., `./guide/tables.md`)
|
|
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
|
|
|
145
145
|
|
|
146
146
|
Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
|
|
147
147
|
|
|
148
|
-
#### `serialization`
|
|
149
|
-
|
|
150
|
-
::: code-group
|
|
151
|
-
|
|
152
|
-
```python [Python]
|
|
153
|
-
vars(db) -> dict
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
```rust [Rust]
|
|
157
|
-
db.config() -> DirSQLConfig
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
```typescript [TypeScript]
|
|
161
|
-
JSON.stringify(db) // via db.toJSON()
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
:::
|
|
165
|
-
|
|
166
|
-
Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
|
|
167
|
-
|
|
168
148
|
---
|
|
169
149
|
|
|
170
150
|
## Table
|
|
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
|
|
|
213
193
|
- `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
|
|
214
194
|
- `glob` -- A glob pattern matched against file paths relative to the root directory.
|
|
215
195
|
- `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
|
|
216
|
-
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
196
|
+
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
217
197
|
|
|
218
198
|
**Attributes:**
|
|
219
199
|
|
|
@@ -6,16 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
|
|
|
6
6
|
|
|
7
7
|
> Online: <https://thekevinscott.github.io/dirsql/cli/config>
|
|
8
8
|
|
|
9
|
-
`dirsql` is configured with an optional
|
|
10
|
-
falls back to [zero-config defaults](./server.md#defaults).
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
- **[TOML](#toml)** — declarative; defines filesystem-fact tables (the path,
|
|
14
|
-
glob captures, and stat metadata). Works with any installation.
|
|
15
|
-
- **[Python](#python)** and **[JavaScript](#javascript)** — native-language
|
|
16
|
-
configs that build tables from the *contents* of files (frontmatter, JSON
|
|
17
|
-
values, CSV cells) through a dynamic `extract` callback. CLI-only; only the
|
|
18
|
-
launcher matching the file's language can run it.
|
|
9
|
+
`dirsql` is configured with an optional `.dirsql.toml` file; with none, the
|
|
10
|
+
server falls back to [zero-config defaults](./server.md#defaults). A
|
|
11
|
+
[TOML](#toml) config is declarative: it defines filesystem-fact tables (the
|
|
12
|
+
path, glob captures, and stat metadata) and works with any installation.
|
|
19
13
|
|
|
20
14
|
## TOML
|
|
21
15
|
|
|
@@ -194,9 +188,8 @@ strict = true
|
|
|
194
188
|
|
|
195
189
|
Strict mode does **not** apply to auto-injected stat virtuals — those are
|
|
196
190
|
always filtered to the DDL's declared columns regardless. Strict mode
|
|
197
|
-
applies only to keys produced by an extract callback (relevant for
|
|
198
|
-
[
|
|
199
|
-
[tables](../guide/tables.md)).
|
|
191
|
+
applies only to keys produced by an extract callback (relevant for
|
|
192
|
+
programmatic [tables](../guide/tables.md)).
|
|
200
193
|
|
|
201
194
|
### Full Example
|
|
202
195
|
|
|
@@ -216,108 +209,3 @@ glob = "**/index.md"
|
|
|
216
209
|
ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
|
|
217
210
|
glob = "logs/*.csv"
|
|
218
211
|
```
|
|
219
|
-
|
|
220
|
-
## Python
|
|
221
|
-
|
|
222
|
-
Reach for a Python config when your columns come from the *contents* of a
|
|
223
|
-
file — parsed JSON, frontmatter, CSV cells — rather than from filesystem
|
|
224
|
-
facts alone. You write a dynamic `extract` callback in Python, and the file
|
|
225
|
-
otherwise looks exactly like the in-process SDK construction (same `DirSQL` /
|
|
226
|
-
`Table` API):
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
dirsql --config dirsql.config.py
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
```python [dirsql.config.py]
|
|
233
|
-
import json
|
|
234
|
-
from dirsql import DirSQL, Table
|
|
235
|
-
|
|
236
|
-
def extract_meta(path):
|
|
237
|
-
with open(path) as f:
|
|
238
|
-
return [json.load(f)]
|
|
239
|
-
|
|
240
|
-
# Python must export a module-level `app`.
|
|
241
|
-
app = DirSQL(
|
|
242
|
-
root="papers", # optional; defaults to the current directory
|
|
243
|
-
tables=[
|
|
244
|
-
Table(
|
|
245
|
-
ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
246
|
-
glob="**/meta.json",
|
|
247
|
-
extract=extract_meta,
|
|
248
|
-
),
|
|
249
|
-
],
|
|
250
|
-
)
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
`extract` receives the path of each matched file and returns a list of rows
|
|
254
|
-
(one dict per row).
|
|
255
|
-
|
|
256
|
-
## JavaScript
|
|
257
|
-
|
|
258
|
-
A JavaScript config gives you the same contents-driven `extract` in Node,
|
|
259
|
-
in either ES module or CommonJS form:
|
|
260
|
-
|
|
261
|
-
```bash
|
|
262
|
-
dirsql --config dirsql.config.mjs
|
|
263
|
-
```
|
|
264
|
-
|
|
265
|
-
::: code-group
|
|
266
|
-
|
|
267
|
-
```javascript [dirsql.config.mjs]
|
|
268
|
-
import { readFileSync } from "node:fs";
|
|
269
|
-
import { DirSQL } from "dirsql";
|
|
270
|
-
|
|
271
|
-
export default new DirSQL({
|
|
272
|
-
root: "papers", // optional; defaults to the current directory
|
|
273
|
-
tables: [
|
|
274
|
-
{
|
|
275
|
-
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
276
|
-
glob: "**/meta.json",
|
|
277
|
-
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
278
|
-
},
|
|
279
|
-
],
|
|
280
|
-
});
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
```javascript [dirsql.config.cjs]
|
|
284
|
-
const { readFileSync } = require("node:fs");
|
|
285
|
-
const { DirSQL } = require("dirsql");
|
|
286
|
-
|
|
287
|
-
module.exports = new DirSQL({
|
|
288
|
-
root: "papers", // optional; defaults to the current directory
|
|
289
|
-
tables: [
|
|
290
|
-
{
|
|
291
|
-
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
292
|
-
glob: "**/meta.json",
|
|
293
|
-
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
294
|
-
},
|
|
295
|
-
],
|
|
296
|
-
});
|
|
297
|
-
```
|
|
298
|
-
|
|
299
|
-
:::
|
|
300
|
-
|
|
301
|
-
## Notes for native-language configs
|
|
302
|
-
|
|
303
|
-
These apply to both the Python and JavaScript forms above.
|
|
304
|
-
|
|
305
|
-
- **Export the config.** Python exposes a module-level `app = DirSQL(...)`; an
|
|
306
|
-
ES module (`.mjs`, or `.js` in an ESM package) uses
|
|
307
|
-
`export default new DirSQL(...)`; CommonJS (`.cjs`, or `.js` in a CJS
|
|
308
|
-
package) uses `module.exports = new DirSQL(...)`. Only the extension
|
|
309
|
-
matters — the file can be named anything; `dirsql.config.{py,mjs,cjs}` is the
|
|
310
|
-
suggested convention, not a requirement.
|
|
311
|
-
- **`root` defaults to the current directory.** A native-language config with
|
|
312
|
-
no `root` indexes the process's current working directory — the directory you
|
|
313
|
-
ran `dirsql` from. (This differs from TOML configs, which default the scan
|
|
314
|
-
root to the config file's own directory.) Pass `root` explicitly to index
|
|
315
|
-
somewhere else.
|
|
316
|
-
- **No nested `config=`.** A native-language config builds its `DirSQL` from
|
|
317
|
-
`tables` and an optional `root`; it must not itself set `config=` to delegate
|
|
318
|
-
to another config file. `dirsql interpret` rejects such a config (a nested
|
|
319
|
-
config can't be represented in the handshake and would recurse).
|
|
320
|
-
- **Install the launcher on your `PATH`.** To run your `extract`, the server
|
|
321
|
-
spawns `dirsql interpret`, so the matching `dirsql` launcher must be installed
|
|
322
|
-
and on your `PATH` — a global `pip`/`uv` install for `.py`, or `npm` for
|
|
323
|
-
`.mjs` / `.cjs`. Only the launcher matching the file's language can run it.
|
|
@@ -17,9 +17,8 @@ Everything you need to run `dirsql` as a CLI lives in this section:
|
|
|
17
17
|
- **[Installation](#installation)** — get the `dirsql` binary.
|
|
18
18
|
- **[Running the Server](./server.md)** — subcommands and flags.
|
|
19
19
|
- **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
|
|
20
|
-
- **[Configuration File](./config.md)** — the `.dirsql.toml` format
|
|
21
|
-
|
|
22
|
-
are defined through a config file; without one, the server runs in
|
|
20
|
+
- **[Configuration File](./config.md)** — the `.dirsql.toml` format. Custom
|
|
21
|
+
tables are defined through a config file; without one, the server runs in
|
|
23
22
|
[zero-config mode](./server.md#defaults).
|
|
24
23
|
- **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
|
|
25
24
|
endpoints, status codes, and event streaming.
|
|
@@ -28,8 +28,7 @@ $ Running at localhost:7117
|
|
|
28
28
|
|
|
29
29
|
The server reads tables from a [config file](./config.md). By default it
|
|
30
30
|
looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
|
|
31
|
-
`.toml` file
|
|
32
|
-
(`.py` / `.js`).
|
|
31
|
+
`.toml` file.
|
|
33
32
|
|
|
34
33
|
## Defaults
|
|
35
34
|
|
|
@@ -53,7 +52,7 @@ A config file will override the default.
|
|
|
53
52
|
|
|
54
53
|
| Flag | Default | Description |
|
|
55
54
|
|---|---|---|
|
|
56
|
-
| `--config <path>` | `./.dirsql.toml` | Path to the
|
|
55
|
+
| `--config <path>` | `./.dirsql.toml` | Path to the `.toml` [config file](./config.md). The index is rooted at the directory containing this file. |
|
|
57
56
|
| `--host <addr>` | `localhost` | Bind address |
|
|
58
57
|
| `--port <n>` | `7117` | TCP port to bind |
|
|
59
58
|
|
|
@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
|
|
|
4
4
|
# pypi/maturin handler can rewrite it via `write-version` before
|
|
5
5
|
# `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
|
|
6
6
|
# and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
|
|
7
|
-
version = "0.3.
|
|
7
|
+
version = "0.3.43"
|
|
8
8
|
edition.workspace = true
|
|
9
9
|
publish = false
|
|
10
10
|
readme = "README.md"
|
|
@@ -45,19 +45,17 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
|
|
|
45
45
|
The **CLI** (`cli/`) is a self-contained section reachable from its own
|
|
46
46
|
top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
|
|
47
47
|
`config.ts`). Everything a CLI user needs -- installation, running the server,
|
|
48
|
-
`init`,
|
|
49
|
-
|
|
48
|
+
`init`, the `.dirsql.toml` config file, and the HTTP API -- lives under
|
|
49
|
+
`cli/`. Do not move CLI pages back into `guide/`.
|
|
50
50
|
|
|
51
51
|
## Conventions
|
|
52
52
|
|
|
53
53
|
- **Lead with the use case.** Open each feature description with *why* a
|
|
54
54
|
reader would reach for it before *how* it works. Don't frame a feature
|
|
55
55
|
by what an adjacent feature can't do.
|
|
56
|
-
*Don't:* "
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
*contents* of files — frontmatter, JSON values, CSV cells — by
|
|
60
|
-
writing a dynamic `extract` callback."
|
|
56
|
+
*Don't:* "Persistence avoids the thing the default mode can't do..."
|
|
57
|
+
*Do:* "Persistence keeps the SQLite index on disk between runs so large
|
|
58
|
+
directories don't re-scan on every startup."
|
|
61
59
|
- Wrap `dirsql` in backticks in all prose text
|
|
62
60
|
- Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
|
|
63
61
|
- Internal links use relative paths (e.g., `./guide/tables.md`)
|
|
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
|
|
|
145
145
|
|
|
146
146
|
Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
|
|
147
147
|
|
|
148
|
-
#### `serialization`
|
|
149
|
-
|
|
150
|
-
::: code-group
|
|
151
|
-
|
|
152
|
-
```python [Python]
|
|
153
|
-
vars(db) -> dict
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
```rust [Rust]
|
|
157
|
-
db.config() -> DirSQLConfig
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
```typescript [TypeScript]
|
|
161
|
-
JSON.stringify(db) // via db.toJSON()
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
:::
|
|
165
|
-
|
|
166
|
-
Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
|
|
167
|
-
|
|
168
148
|
---
|
|
169
149
|
|
|
170
150
|
## Table
|
|
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
|
|
|
213
193
|
- `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
|
|
214
194
|
- `glob` -- A glob pattern matched against file paths relative to the root directory.
|
|
215
195
|
- `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
|
|
216
|
-
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
196
|
+
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
217
197
|
|
|
218
198
|
**Attributes:**
|
|
219
199
|
|
|
@@ -6,16 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
|
|
|
6
6
|
|
|
7
7
|
> Online: <https://thekevinscott.github.io/dirsql/cli/config>
|
|
8
8
|
|
|
9
|
-
`dirsql` is configured with an optional
|
|
10
|
-
falls back to [zero-config defaults](./server.md#defaults).
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
- **[TOML](#toml)** — declarative; defines filesystem-fact tables (the path,
|
|
14
|
-
glob captures, and stat metadata). Works with any installation.
|
|
15
|
-
- **[Python](#python)** and **[JavaScript](#javascript)** — native-language
|
|
16
|
-
configs that build tables from the *contents* of files (frontmatter, JSON
|
|
17
|
-
values, CSV cells) through a dynamic `extract` callback. CLI-only; only the
|
|
18
|
-
launcher matching the file's language can run it.
|
|
9
|
+
`dirsql` is configured with an optional `.dirsql.toml` file; with none, the
|
|
10
|
+
server falls back to [zero-config defaults](./server.md#defaults). A
|
|
11
|
+
[TOML](#toml) config is declarative: it defines filesystem-fact tables (the
|
|
12
|
+
path, glob captures, and stat metadata) and works with any installation.
|
|
19
13
|
|
|
20
14
|
## TOML
|
|
21
15
|
|
|
@@ -194,9 +188,8 @@ strict = true
|
|
|
194
188
|
|
|
195
189
|
Strict mode does **not** apply to auto-injected stat virtuals — those are
|
|
196
190
|
always filtered to the DDL's declared columns regardless. Strict mode
|
|
197
|
-
applies only to keys produced by an extract callback (relevant for
|
|
198
|
-
[
|
|
199
|
-
[tables](../guide/tables.md)).
|
|
191
|
+
applies only to keys produced by an extract callback (relevant for
|
|
192
|
+
programmatic [tables](../guide/tables.md)).
|
|
200
193
|
|
|
201
194
|
### Full Example
|
|
202
195
|
|
|
@@ -216,108 +209,3 @@ glob = "**/index.md"
|
|
|
216
209
|
ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
|
|
217
210
|
glob = "logs/*.csv"
|
|
218
211
|
```
|
|
219
|
-
|
|
220
|
-
## Python
|
|
221
|
-
|
|
222
|
-
Reach for a Python config when your columns come from the *contents* of a
|
|
223
|
-
file — parsed JSON, frontmatter, CSV cells — rather than from filesystem
|
|
224
|
-
facts alone. You write a dynamic `extract` callback in Python, and the file
|
|
225
|
-
otherwise looks exactly like the in-process SDK construction (same `DirSQL` /
|
|
226
|
-
`Table` API):
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
dirsql --config dirsql.config.py
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
```python [dirsql.config.py]
|
|
233
|
-
import json
|
|
234
|
-
from dirsql import DirSQL, Table
|
|
235
|
-
|
|
236
|
-
def extract_meta(path):
|
|
237
|
-
with open(path) as f:
|
|
238
|
-
return [json.load(f)]
|
|
239
|
-
|
|
240
|
-
# Python must export a module-level `app`.
|
|
241
|
-
app = DirSQL(
|
|
242
|
-
root="papers", # optional; defaults to the current directory
|
|
243
|
-
tables=[
|
|
244
|
-
Table(
|
|
245
|
-
ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
246
|
-
glob="**/meta.json",
|
|
247
|
-
extract=extract_meta,
|
|
248
|
-
),
|
|
249
|
-
],
|
|
250
|
-
)
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
`extract` receives the path of each matched file and returns a list of rows
|
|
254
|
-
(one dict per row).
|
|
255
|
-
|
|
256
|
-
## JavaScript
|
|
257
|
-
|
|
258
|
-
A JavaScript config gives you the same contents-driven `extract` in Node,
|
|
259
|
-
in either ES module or CommonJS form:
|
|
260
|
-
|
|
261
|
-
```bash
|
|
262
|
-
dirsql --config dirsql.config.mjs
|
|
263
|
-
```
|
|
264
|
-
|
|
265
|
-
::: code-group
|
|
266
|
-
|
|
267
|
-
```javascript [dirsql.config.mjs]
|
|
268
|
-
import { readFileSync } from "node:fs";
|
|
269
|
-
import { DirSQL } from "dirsql";
|
|
270
|
-
|
|
271
|
-
export default new DirSQL({
|
|
272
|
-
root: "papers", // optional; defaults to the current directory
|
|
273
|
-
tables: [
|
|
274
|
-
{
|
|
275
|
-
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
276
|
-
glob: "**/meta.json",
|
|
277
|
-
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
278
|
-
},
|
|
279
|
-
],
|
|
280
|
-
});
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
```javascript [dirsql.config.cjs]
|
|
284
|
-
const { readFileSync } = require("node:fs");
|
|
285
|
-
const { DirSQL } = require("dirsql");
|
|
286
|
-
|
|
287
|
-
module.exports = new DirSQL({
|
|
288
|
-
root: "papers", // optional; defaults to the current directory
|
|
289
|
-
tables: [
|
|
290
|
-
{
|
|
291
|
-
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
292
|
-
glob: "**/meta.json",
|
|
293
|
-
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
294
|
-
},
|
|
295
|
-
],
|
|
296
|
-
});
|
|
297
|
-
```
|
|
298
|
-
|
|
299
|
-
:::
|
|
300
|
-
|
|
301
|
-
## Notes for native-language configs
|
|
302
|
-
|
|
303
|
-
These apply to both the Python and JavaScript forms above.
|
|
304
|
-
|
|
305
|
-
- **Export the config.** Python exposes a module-level `app = DirSQL(...)`; an
|
|
306
|
-
ES module (`.mjs`, or `.js` in an ESM package) uses
|
|
307
|
-
`export default new DirSQL(...)`; CommonJS (`.cjs`, or `.js` in a CJS
|
|
308
|
-
package) uses `module.exports = new DirSQL(...)`. Only the extension
|
|
309
|
-
matters — the file can be named anything; `dirsql.config.{py,mjs,cjs}` is the
|
|
310
|
-
suggested convention, not a requirement.
|
|
311
|
-
- **`root` defaults to the current directory.** A native-language config with
|
|
312
|
-
no `root` indexes the process's current working directory — the directory you
|
|
313
|
-
ran `dirsql` from. (This differs from TOML configs, which default the scan
|
|
314
|
-
root to the config file's own directory.) Pass `root` explicitly to index
|
|
315
|
-
somewhere else.
|
|
316
|
-
- **No nested `config=`.** A native-language config builds its `DirSQL` from
|
|
317
|
-
`tables` and an optional `root`; it must not itself set `config=` to delegate
|
|
318
|
-
to another config file. `dirsql interpret` rejects such a config (a nested
|
|
319
|
-
config can't be represented in the handshake and would recurse).
|
|
320
|
-
- **Install the launcher on your `PATH`.** To run your `extract`, the server
|
|
321
|
-
spawns `dirsql interpret`, so the matching `dirsql` launcher must be installed
|
|
322
|
-
and on your `PATH` — a global `pip`/`uv` install for `.py`, or `npm` for
|
|
323
|
-
`.mjs` / `.cjs`. Only the launcher matching the file's language can run it.
|
|
@@ -17,9 +17,8 @@ Everything you need to run `dirsql` as a CLI lives in this section:
|
|
|
17
17
|
- **[Installation](#installation)** — get the `dirsql` binary.
|
|
18
18
|
- **[Running the Server](./server.md)** — subcommands and flags.
|
|
19
19
|
- **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
|
|
20
|
-
- **[Configuration File](./config.md)** — the `.dirsql.toml` format
|
|
21
|
-
|
|
22
|
-
are defined through a config file; without one, the server runs in
|
|
20
|
+
- **[Configuration File](./config.md)** — the `.dirsql.toml` format. Custom
|
|
21
|
+
tables are defined through a config file; without one, the server runs in
|
|
23
22
|
[zero-config mode](./server.md#defaults).
|
|
24
23
|
- **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
|
|
25
24
|
endpoints, status codes, and event streaming.
|
|
@@ -28,8 +28,7 @@ $ Running at localhost:7117
|
|
|
28
28
|
|
|
29
29
|
The server reads tables from a [config file](./config.md). By default it
|
|
30
30
|
looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
|
|
31
|
-
`.toml` file
|
|
32
|
-
(`.py` / `.js`).
|
|
31
|
+
`.toml` file.
|
|
33
32
|
|
|
34
33
|
## Defaults
|
|
35
34
|
|
|
@@ -53,7 +52,7 @@ A config file will override the default.
|
|
|
53
52
|
|
|
54
53
|
| Flag | Default | Description |
|
|
55
54
|
|---|---|---|
|
|
56
|
-
| `--config <path>` | `./.dirsql.toml` | Path to the
|
|
55
|
+
| `--config <path>` | `./.dirsql.toml` | Path to the `.toml` [config file](./config.md). The index is rooted at the directory containing this file. |
|
|
57
56
|
| `--host <addr>` | `localhost` | Bind address |
|
|
58
57
|
| `--port <n>` | `7117` | TCP port to bind |
|
|
59
58
|
|
|
@@ -145,26 +145,6 @@ for await (const event of db.watch()) { // AsyncIterable<RowEvent>
|
|
|
145
145
|
|
|
146
146
|
Returns an async iterable of `RowEvent` objects. The file watcher starts automatically on first iteration. The iterator never terminates on its own.
|
|
147
147
|
|
|
148
|
-
#### `serialization`
|
|
149
|
-
|
|
150
|
-
::: code-group
|
|
151
|
-
|
|
152
|
-
```python [Python]
|
|
153
|
-
vars(db) -> dict
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
```rust [Rust]
|
|
157
|
-
db.config() -> DirSQLConfig
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
```typescript [TypeScript]
|
|
161
|
-
JSON.stringify(db) // via db.toJSON()
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
:::
|
|
165
|
-
|
|
166
|
-
Returns the resolved construction state as a JSON-compatible value with fields `root`, `tables`, `ignore`, `persist`, `persist_path` (camelCase `persistPath` in TypeScript). Each table is `{ ddl, glob, strict }`. All three SDK snapshots also include `extensions` -- an array of `{ path, entrypoint }` (empty when none are configured; `entrypoint` is `null` when no override was supplied). Excludes the original `config` path (already merged into `root` / `tables` / `ignore`), per-table `extract`, and per-table `name`. Available immediately after construction in Python and TypeScript; Rust's sync `build()` returns a ready instance.
|
|
167
|
-
|
|
168
148
|
---
|
|
169
149
|
|
|
170
150
|
## Table
|
|
@@ -213,7 +193,7 @@ Defines a mapping from files to SQLite table rows.
|
|
|
213
193
|
- `ddl` -- A `CREATE TABLE` statement. The table name is parsed from this DDL.
|
|
214
194
|
- `glob` -- A glob pattern matched against file paths relative to the root directory.
|
|
215
195
|
- `extract` -- A callable `(path) -> list[dict]`. Receives the path of the matched file -- relative to the scan root, or absolute when `root` is absolute. `dirsql` does not read file contents; a callback that needs the file body reads `path` itself. Returns a list of dicts/maps mapping column names to values. Return an empty list to skip a file.
|
|
216
|
-
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
196
|
+
- `strict` -- Optional (default `False`). Controls row/schema validation. In the default relaxed mode, extra row keys are dropped and missing columns become `NULL`. When `True`, every row key must be a valid column identifier and any extra or missing key raises an error.
|
|
217
197
|
|
|
218
198
|
**Attributes:**
|
|
219
199
|
|