dirsql 0.3.18__tar.gz → 0.3.20__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dirsql-0.3.18 → dirsql-0.3.20}/Cargo.lock +1 -1
- {dirsql-0.3.18 → dirsql-0.3.20}/PKG-INFO +1 -1
- {dirsql-0.3.18/packages/python → dirsql-0.3.20}/docs/AGENTS.md +10 -2
- {dirsql-0.3.18/packages/rust → dirsql-0.3.20}/docs/cli/config.md +76 -7
- {dirsql-0.3.18/packages/rust → dirsql-0.3.20}/docs/cli/index.md +5 -4
- {dirsql-0.3.18/packages/rust → dirsql-0.3.20}/docs/cli/server.md +6 -4
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/package.json +1 -1
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/pnpm-lock.yaml +13 -13
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/Cargo.toml +1 -1
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/conftest.py +1 -1
- {dirsql-0.3.18 → dirsql-0.3.20/packages/python}/docs/AGENTS.md +10 -2
- {dirsql-0.3.18 → dirsql-0.3.20/packages/python}/docs/cli/config.md +76 -7
- {dirsql-0.3.18 → dirsql-0.3.20/packages/python}/docs/cli/index.md +5 -4
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/cli/server.md +6 -4
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/package.json +1 -1
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/pnpm-lock.yaml +13 -13
- {dirsql-0.3.18/packages/python → dirsql-0.3.20/packages/rust}/docs/cli/config.md +76 -7
- {dirsql-0.3.18/packages/python → dirsql-0.3.20/packages/rust}/docs/cli/index.md +5 -4
- {dirsql-0.3.18 → dirsql-0.3.20/packages/rust}/docs/cli/server.md +6 -4
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/mod.rs +10 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/db.rs +254 -11
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/lib.rs +120 -31
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/persist.rs +9 -2
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/async_sdk.rs +4 -1
- dirsql-0.3.20/packages/rust/tests/code_review_findings.rs +288 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/sdk.rs +5 -2
- {dirsql-0.3.18 → dirsql-0.3.20}/Cargo.toml +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/README.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/_async.py +0 -0
- /dirsql-0.3.18/dirsql/test_async.py → /dirsql-0.3.20/dirsql/_async_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/_dirsql.pyi +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/binary_path.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/binary_path_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/dispatch_extract.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/dispatch_extract_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/load_app.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/load_app_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/run.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/run_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/write_message.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/interpret/write_message_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/is_windows.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/is_windows_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/main.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/cli/main_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/py.typed +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/resolve_config.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/dirsql/resolve_config_test.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/.vitepress/config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/api/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/cli/http-api.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/cli/init.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/getting-started.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/async.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/crdt.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/persistence.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/querying.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/tables.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/guide/watching.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/migrations.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/playwright.config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/tests/integration/home.spec.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/tests/integration/language-flag.spec.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/tests/unit/config.test.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/tests/unit/lang.test.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/docs/vitest.config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/README.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/.vitepress/config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/api/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/cli/http-api.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/cli/init.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/getting-started.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/async.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/crdt.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/persistence.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/querying.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/tables.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/guide/watching.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/migrations.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/playwright.config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/tests/integration/home.spec.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/tests/integration/language-flag.spec.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/tests/unit/config.test.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/tests/unit/lang.test.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/docs/vitest.config.ts +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/src/lib.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/conftest.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/e2e/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/data/a/meta.json +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/data/b/meta.json +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/dirsql.config.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/interpret/data/a/meta.json +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/interpret/data/b/meta.json +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/interpret/dirsql.config.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/interpret/dirsql.config_no_app.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__fixtures__/interpret/dirsql.config_raises.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/__init__.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/interpret_subprocess.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_async_dirsql.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_binding.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_dirsql.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_docs_examples.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_docs_gaps.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_from_config.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_interpret.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_native_config.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_persist.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/python/tests/integration/test_serialization.py +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/Cargo.toml +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/README.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/benches/db_bench.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/benches/differ_bench.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/benches/matcher_bench.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/benches/scanner_bench.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/api/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/cli/http-api.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/cli/init.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/getting-started.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/async.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/crdt.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/persistence.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/querying.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/tables.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/guide/watching.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/index.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/docs/migrations.md +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/bin/dirsql.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/init.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/native_config.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/router.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/serialize.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/cli/server.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/config.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/differ.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/matcher.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/scanner.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/src/watcher.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/cli_e2e.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/cli_integration.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/docs_examples.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/docs_gaps.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/from_config.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/init_e2e.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/init_integration.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/persist.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/readonly_query.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/packages/rust/tests/serialization.rs +0 -0
- {dirsql-0.3.18 → dirsql-0.3.20}/pyproject.toml +0 -0
|
@@ -45,11 +45,19 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
|
|
|
45
45
|
The **CLI** (`cli/`) is a self-contained section reachable from its own
|
|
46
46
|
top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
|
|
47
47
|
`config.ts`). Everything a CLI user needs -- installation, running the server,
|
|
48
|
-
`init`,
|
|
49
|
-
`cli/`. Do not move CLI pages back into `guide/`.
|
|
48
|
+
`init`, config files (`.dirsql.toml` or native-language), and the HTTP API --
|
|
49
|
+
lives under `cli/`. Do not move CLI pages back into `guide/`.
|
|
50
50
|
|
|
51
51
|
## Conventions
|
|
52
52
|
|
|
53
|
+
- **Lead with the use case.** Open each feature description with *why* a
|
|
54
|
+
reader would reach for it before *how* it works. Don't frame a feature
|
|
55
|
+
by what an adjacent feature can't do.
|
|
56
|
+
*Don't:* "`.dirsql.toml` can't parse contents — you need code in
|
|
57
|
+
`extract`. Point `--config` at a Python module..."
|
|
58
|
+
*Do:* "Native-language configs let you build tables from the
|
|
59
|
+
*contents* of files — frontmatter, JSON values, CSV cells — by
|
|
60
|
+
writing a dynamic `extract` callback."
|
|
53
61
|
- Wrap `dirsql` in backticks in all prose text
|
|
54
62
|
- Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
|
|
55
63
|
- Internal links use relative paths (e.g., `./guide/tables.md`)
|
|
@@ -6,7 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
|
|
|
6
6
|
|
|
7
7
|
> Online: <https://thekevinscott.github.io/dirsql/cli/config>
|
|
8
8
|
|
|
9
|
-
`dirsql` can be configured with an optional
|
|
9
|
+
`dirsql` can be configured with an optional config file (if omitted, server falls back to [defaults](./server.md#defaults)). Two formats are accepted:
|
|
10
|
+
|
|
11
|
+
- **`dirsql.toml`** — declarative; covers filesystem-fact tables. Works with any installation.
|
|
12
|
+
- **`.py` / `.js`** — native-language; lets you write `extract` callbacks in Python or JavaScript. CLI-only, and only the launcher matching the file's language can run it. See [Native-Language Configs](#native-language-configs).
|
|
10
13
|
|
|
11
14
|
## Basic Example
|
|
12
15
|
|
|
@@ -172,10 +175,76 @@ ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
|
|
|
172
175
|
glob = "logs/*.csv"
|
|
173
176
|
```
|
|
174
177
|
|
|
175
|
-
##
|
|
178
|
+
## Native-Language Configs
|
|
179
|
+
|
|
180
|
+
You can provide a config file in a particular language, allowing you to define a dynamic extract function. This can be useful for building a database based on the _contents_ of a file.
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
dirsql --config dirsql.config.py
|
|
184
|
+
dirsql --config dirsql.config.js
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
The file looks exactly like the in-process SDK construction — same
|
|
188
|
+
`DirSQL` / `Table` API:
|
|
189
|
+
|
|
190
|
+
::: code-group
|
|
191
|
+
|
|
192
|
+
```python [dirsql.config.py]
|
|
193
|
+
import json
|
|
194
|
+
from dirsql import DirSQL, Table
|
|
195
|
+
|
|
196
|
+
def extract_meta(path):
|
|
197
|
+
with open(path) as f:
|
|
198
|
+
return [json.load(f)]
|
|
199
|
+
|
|
200
|
+
# Python must export an `app` variable
|
|
201
|
+
app = DirSQL(
|
|
202
|
+
tables=[
|
|
203
|
+
Table(
|
|
204
|
+
ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
205
|
+
glob="**/meta.json",
|
|
206
|
+
extract=extract_meta,
|
|
207
|
+
),
|
|
208
|
+
],
|
|
209
|
+
)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
```javascript [dirsql.config.mjs]
|
|
213
|
+
import { readFileSync } from "node:fs";
|
|
214
|
+
import { DirSQL } from "dirsql";
|
|
215
|
+
|
|
216
|
+
export default new DirSQL({
|
|
217
|
+
tables: [
|
|
218
|
+
{
|
|
219
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
220
|
+
glob: "**/meta.json",
|
|
221
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
222
|
+
},
|
|
223
|
+
],
|
|
224
|
+
});
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
```javascript [dirsql.config.cjs]
|
|
228
|
+
const { readFileSync } = require("node:fs");
|
|
229
|
+
const { DirSQL } = require("dirsql");
|
|
230
|
+
|
|
231
|
+
module.exports = new DirSQL({
|
|
232
|
+
tables: [
|
|
233
|
+
{
|
|
234
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
235
|
+
glob: "**/meta.json",
|
|
236
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
237
|
+
},
|
|
238
|
+
],
|
|
239
|
+
});
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
:::
|
|
243
|
+
|
|
244
|
+
Only the extension matters — the file can be named anything. `dirsql.config.{py,mjs,cjs}` is the suggested convention but not required.
|
|
245
|
+
|
|
246
|
+
### Module conventions
|
|
176
247
|
|
|
177
|
-
`.
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
bytes in your host language. Glob captures and stat virtuals are still
|
|
181
|
-
auto-injected into rows produced by your extract.
|
|
248
|
+
- **Python (`.py`)** — module-level `app = DirSQL(...)`.
|
|
249
|
+
- **ESM (`.mjs`, or `.js` in an ESM package)** — `export default new DirSQL(...)`.
|
|
250
|
+
- **CommonJS (`.cjs`, or `.js` in a CJS package)** — `module.exports = new DirSQL(...)`.
|
|
@@ -10,16 +10,17 @@ canonical: https://thekevinscott.github.io/dirsql/cli/
|
|
|
10
10
|
the same indexing, querying, and watching functionality as the SDK — no host
|
|
11
11
|
language required. Run it in any directory to query your files over HTTP:
|
|
12
12
|
with no config it serves a default `files` table, and a
|
|
13
|
-
[
|
|
13
|
+
[config file](./config.md) defines custom tables.
|
|
14
14
|
|
|
15
15
|
Everything you need to run `dirsql` as a CLI lives in this section:
|
|
16
16
|
|
|
17
17
|
- **[Installation](#installation)** — get the `dirsql` binary.
|
|
18
18
|
- **[Running the Server](./server.md)** — subcommands and flags.
|
|
19
19
|
- **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
|
|
20
|
-
- **[Configuration File](./config.md)** — the `.dirsql.toml` format
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
- **[Configuration File](./config.md)** — the `.dirsql.toml` format and the
|
|
21
|
+
`.py` / `.js` native-language alternative. Custom tables
|
|
22
|
+
are defined through a config file; without one, the server runs in
|
|
23
|
+
[zero-config mode](./server.md#zero-config-mode).
|
|
23
24
|
- **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
|
|
24
25
|
endpoints, status codes, and event streaming.
|
|
25
26
|
|
|
@@ -26,8 +26,10 @@ dirsql
|
|
|
26
26
|
$ Running at localhost:7117
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
The server reads tables from a [
|
|
30
|
-
|
|
29
|
+
The server reads tables from a [config file](./config.md). By default it
|
|
30
|
+
looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
|
|
31
|
+
`.toml` file or a [native-language config](./config.md#native-language-configs)
|
|
32
|
+
(`.py` / `.js`).
|
|
31
33
|
|
|
32
34
|
## Defaults
|
|
33
35
|
|
|
@@ -45,13 +47,13 @@ curl -s localhost:7117/query -H 'content-type: application/json' \
|
|
|
45
47
|
-d '{"sql":"SELECT _basename, _size FROM files ORDER BY _size DESC LIMIT 5"}'
|
|
46
48
|
```
|
|
47
49
|
|
|
48
|
-
A
|
|
50
|
+
A config file will override the default.
|
|
49
51
|
|
|
50
52
|
## Flags
|
|
51
53
|
|
|
52
54
|
| Flag | Default | Description |
|
|
53
55
|
|---|---|---|
|
|
54
|
-
| `--config <path>` | `./.dirsql.toml` | Path to the config file. The index is rooted at the directory containing this file. |
|
|
56
|
+
| `--config <path>` | `./.dirsql.toml` | Path to the config file (`.toml` or [native-language](./config.md#native-language-configs)). The index is rooted at the directory containing this file. |
|
|
55
57
|
| `--host <addr>` | `localhost` | Bind address |
|
|
56
58
|
| `--port <n>` | `7117` | TCP port to bind |
|
|
57
59
|
|
|
@@ -9,8 +9,8 @@ importers:
|
|
|
9
9
|
.:
|
|
10
10
|
devDependencies:
|
|
11
11
|
'@playwright/test':
|
|
12
|
-
specifier: ^1.
|
|
13
|
-
version: 1.
|
|
12
|
+
specifier: ^1.60.0
|
|
13
|
+
version: 1.60.0
|
|
14
14
|
vitepress:
|
|
15
15
|
specifier: ^1.5.0
|
|
16
16
|
version: 1.6.4(@algolia/client-search@5.50.1)(postcss@8.5.8)(search-insights@2.17.3)
|
|
@@ -283,8 +283,8 @@ packages:
|
|
|
283
283
|
'@jridgewell/sourcemap-codec@1.5.5':
|
|
284
284
|
resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==}
|
|
285
285
|
|
|
286
|
-
'@playwright/test@1.
|
|
287
|
-
resolution: {integrity: sha512-
|
|
286
|
+
'@playwright/test@1.60.0':
|
|
287
|
+
resolution: {integrity: sha512-O71yZIbAh/PxDMNGns37GHBIfrVkEVyn+AXyIa5dOTfb4/xNvRWV+Vv/NMbNCtODB/pO7vLlF2OTmMVLhmr7Ag==}
|
|
288
288
|
engines: {node: '>=18'}
|
|
289
289
|
hasBin: true
|
|
290
290
|
|
|
@@ -774,13 +774,13 @@ packages:
|
|
|
774
774
|
picocolors@1.1.1:
|
|
775
775
|
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
|
|
776
776
|
|
|
777
|
-
playwright-core@1.
|
|
778
|
-
resolution: {integrity: sha512-
|
|
777
|
+
playwright-core@1.60.0:
|
|
778
|
+
resolution: {integrity: sha512-9bW6zvX/m0lEbgTKJ6YppOKx8H3VOPBMOCFh2irXFOT4BbHgrx5hPjwJYLT40Lu+4qtD36qKc/Hn56StUW57IA==}
|
|
779
779
|
engines: {node: '>=18'}
|
|
780
780
|
hasBin: true
|
|
781
781
|
|
|
782
|
-
playwright@1.
|
|
783
|
-
resolution: {integrity: sha512-
|
|
782
|
+
playwright@1.60.0:
|
|
783
|
+
resolution: {integrity: sha512-hheHdokM8cdqCb0lcE3s+zT4t4W+vvjpGxsZlDnikarzx8tSzMebh3UiFtgqwFwnTnjYQcsyMF8ei2mCO/tpeA==}
|
|
784
784
|
engines: {node: '>=18'}
|
|
785
785
|
hasBin: true
|
|
786
786
|
|
|
@@ -1206,9 +1206,9 @@ snapshots:
|
|
|
1206
1206
|
|
|
1207
1207
|
'@jridgewell/sourcemap-codec@1.5.5': {}
|
|
1208
1208
|
|
|
1209
|
-
'@playwright/test@1.
|
|
1209
|
+
'@playwright/test@1.60.0':
|
|
1210
1210
|
dependencies:
|
|
1211
|
-
playwright: 1.
|
|
1211
|
+
playwright: 1.60.0
|
|
1212
1212
|
|
|
1213
1213
|
'@rollup/rollup-android-arm-eabi@4.60.1':
|
|
1214
1214
|
optional: true
|
|
@@ -1686,11 +1686,11 @@ snapshots:
|
|
|
1686
1686
|
|
|
1687
1687
|
picocolors@1.1.1: {}
|
|
1688
1688
|
|
|
1689
|
-
playwright-core@1.
|
|
1689
|
+
playwright-core@1.60.0: {}
|
|
1690
1690
|
|
|
1691
|
-
playwright@1.
|
|
1691
|
+
playwright@1.60.0:
|
|
1692
1692
|
dependencies:
|
|
1693
|
-
playwright-core: 1.
|
|
1693
|
+
playwright-core: 1.60.0
|
|
1694
1694
|
optionalDependencies:
|
|
1695
1695
|
fsevents: 2.3.2
|
|
1696
1696
|
|
|
@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
|
|
|
4
4
|
# pypi/maturin handler can rewrite it via `write-version` before
|
|
5
5
|
# `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
|
|
6
6
|
# and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
|
|
7
|
-
version = "0.3.
|
|
7
|
+
version = "0.3.20"
|
|
8
8
|
edition.workspace = true
|
|
9
9
|
publish = false
|
|
10
10
|
readme = "README.md"
|
|
@@ -8,7 +8,7 @@ imports real types from `dirsql._dirsql`.
|
|
|
8
8
|
|
|
9
9
|
The stub is installed ONLY when the real extension can't be imported.
|
|
10
10
|
When `maturin develop` has been run (as in CI), the real extension wins
|
|
11
|
-
and tests that depend on it — `
|
|
11
|
+
and tests that depend on it — `_async_test.py`, the integration suites —
|
|
12
12
|
see the real bindings.
|
|
13
13
|
"""
|
|
14
14
|
|
|
@@ -45,11 +45,19 @@ The docs follow the [Diataxis](https://diataxis.fr/) framework:
|
|
|
45
45
|
The **CLI** (`cli/`) is a self-contained section reachable from its own
|
|
46
46
|
top-level `CLI` nav tab, with a path-scoped sidebar (`/cli/` key in
|
|
47
47
|
`config.ts`). Everything a CLI user needs -- installation, running the server,
|
|
48
|
-
`init`,
|
|
49
|
-
`cli/`. Do not move CLI pages back into `guide/`.
|
|
48
|
+
`init`, config files (`.dirsql.toml` or native-language), and the HTTP API --
|
|
49
|
+
lives under `cli/`. Do not move CLI pages back into `guide/`.
|
|
50
50
|
|
|
51
51
|
## Conventions
|
|
52
52
|
|
|
53
|
+
- **Lead with the use case.** Open each feature description with *why* a
|
|
54
|
+
reader would reach for it before *how* it works. Don't frame a feature
|
|
55
|
+
by what an adjacent feature can't do.
|
|
56
|
+
*Don't:* "`.dirsql.toml` can't parse contents — you need code in
|
|
57
|
+
`extract`. Point `--config` at a Python module..."
|
|
58
|
+
*Do:* "Native-language configs let you build tables from the
|
|
59
|
+
*contents* of files — frontmatter, JSON values, CSV cells — by
|
|
60
|
+
writing a dynamic `extract` callback."
|
|
53
61
|
- Wrap `dirsql` in backticks in all prose text
|
|
54
62
|
- Use VitePress [code group](https://vitepress.dev/guide/markdown#code-groups) syntax (`::: code-group`) for multi-language examples with `Python`, `Rust`, and `TypeScript` tabs
|
|
55
63
|
- Internal links use relative paths (e.g., `./guide/tables.md`)
|
|
@@ -6,7 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
|
|
|
6
6
|
|
|
7
7
|
> Online: <https://thekevinscott.github.io/dirsql/cli/config>
|
|
8
8
|
|
|
9
|
-
`dirsql` can be configured with an optional
|
|
9
|
+
`dirsql` can be configured with an optional config file (if omitted, server falls back to [defaults](./server.md#defaults)). Two formats are accepted:
|
|
10
|
+
|
|
11
|
+
- **`dirsql.toml`** — declarative; covers filesystem-fact tables. Works with any installation.
|
|
12
|
+
- **`.py` / `.js`** — native-language; lets you write `extract` callbacks in Python or JavaScript. CLI-only, and only the launcher matching the file's language can run it. See [Native-Language Configs](#native-language-configs).
|
|
10
13
|
|
|
11
14
|
## Basic Example
|
|
12
15
|
|
|
@@ -172,10 +175,76 @@ ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
|
|
|
172
175
|
glob = "logs/*.csv"
|
|
173
176
|
```
|
|
174
177
|
|
|
175
|
-
##
|
|
178
|
+
## Native-Language Configs
|
|
179
|
+
|
|
180
|
+
You can provide a config file in a particular language, allowing you to define a dynamic extract function. This can be useful for building a database based on the _contents_ of a file.
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
dirsql --config dirsql.config.py
|
|
184
|
+
dirsql --config dirsql.config.js
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
The file looks exactly like the in-process SDK construction — same
|
|
188
|
+
`DirSQL` / `Table` API:
|
|
189
|
+
|
|
190
|
+
::: code-group
|
|
191
|
+
|
|
192
|
+
```python [dirsql.config.py]
|
|
193
|
+
import json
|
|
194
|
+
from dirsql import DirSQL, Table
|
|
195
|
+
|
|
196
|
+
def extract_meta(path):
|
|
197
|
+
with open(path) as f:
|
|
198
|
+
return [json.load(f)]
|
|
199
|
+
|
|
200
|
+
# Python must export an `app` variable
|
|
201
|
+
app = DirSQL(
|
|
202
|
+
tables=[
|
|
203
|
+
Table(
|
|
204
|
+
ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
205
|
+
glob="**/meta.json",
|
|
206
|
+
extract=extract_meta,
|
|
207
|
+
),
|
|
208
|
+
],
|
|
209
|
+
)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
```javascript [dirsql.config.mjs]
|
|
213
|
+
import { readFileSync } from "node:fs";
|
|
214
|
+
import { DirSQL } from "dirsql";
|
|
215
|
+
|
|
216
|
+
export default new DirSQL({
|
|
217
|
+
tables: [
|
|
218
|
+
{
|
|
219
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
220
|
+
glob: "**/meta.json",
|
|
221
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
222
|
+
},
|
|
223
|
+
],
|
|
224
|
+
});
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
```javascript [dirsql.config.cjs]
|
|
228
|
+
const { readFileSync } = require("node:fs");
|
|
229
|
+
const { DirSQL } = require("dirsql");
|
|
230
|
+
|
|
231
|
+
module.exports = new DirSQL({
|
|
232
|
+
tables: [
|
|
233
|
+
{
|
|
234
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
235
|
+
glob: "**/meta.json",
|
|
236
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
237
|
+
},
|
|
238
|
+
],
|
|
239
|
+
});
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
:::
|
|
243
|
+
|
|
244
|
+
Only the extension matters — the file can be named anything. `dirsql.config.{py,mjs,cjs}` is the suggested convention but not required.
|
|
245
|
+
|
|
246
|
+
### Module conventions
|
|
176
247
|
|
|
177
|
-
`.
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
bytes in your host language. Glob captures and stat virtuals are still
|
|
181
|
-
auto-injected into rows produced by your extract.
|
|
248
|
+
- **Python (`.py`)** — module-level `app = DirSQL(...)`.
|
|
249
|
+
- **ESM (`.mjs`, or `.js` in an ESM package)** — `export default new DirSQL(...)`.
|
|
250
|
+
- **CommonJS (`.cjs`, or `.js` in a CJS package)** — `module.exports = new DirSQL(...)`.
|
|
@@ -10,16 +10,17 @@ canonical: https://thekevinscott.github.io/dirsql/cli/
|
|
|
10
10
|
the same indexing, querying, and watching functionality as the SDK — no host
|
|
11
11
|
language required. Run it in any directory to query your files over HTTP:
|
|
12
12
|
with no config it serves a default `files` table, and a
|
|
13
|
-
[
|
|
13
|
+
[config file](./config.md) defines custom tables.
|
|
14
14
|
|
|
15
15
|
Everything you need to run `dirsql` as a CLI lives in this section:
|
|
16
16
|
|
|
17
17
|
- **[Installation](#installation)** — get the `dirsql` binary.
|
|
18
18
|
- **[Running the Server](./server.md)** — subcommands and flags.
|
|
19
19
|
- **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
|
|
20
|
-
- **[Configuration File](./config.md)** — the `.dirsql.toml` format
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
- **[Configuration File](./config.md)** — the `.dirsql.toml` format and the
|
|
21
|
+
`.py` / `.js` native-language alternative. Custom tables
|
|
22
|
+
are defined through a config file; without one, the server runs in
|
|
23
|
+
[zero-config mode](./server.md#zero-config-mode).
|
|
23
24
|
- **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
|
|
24
25
|
endpoints, status codes, and event streaming.
|
|
25
26
|
|
|
@@ -26,8 +26,10 @@ dirsql
|
|
|
26
26
|
$ Running at localhost:7117
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
The server reads tables from a [
|
|
30
|
-
|
|
29
|
+
The server reads tables from a [config file](./config.md). By default it
|
|
30
|
+
looks for `./.dirsql.toml`; pass `--config <path>` to point at a different
|
|
31
|
+
`.toml` file or a [native-language config](./config.md#native-language-configs)
|
|
32
|
+
(`.py` / `.js`).
|
|
31
33
|
|
|
32
34
|
## Defaults
|
|
33
35
|
|
|
@@ -45,13 +47,13 @@ curl -s localhost:7117/query -H 'content-type: application/json' \
|
|
|
45
47
|
-d '{"sql":"SELECT _basename, _size FROM files ORDER BY _size DESC LIMIT 5"}'
|
|
46
48
|
```
|
|
47
49
|
|
|
48
|
-
A
|
|
50
|
+
A config file will override the default.
|
|
49
51
|
|
|
50
52
|
## Flags
|
|
51
53
|
|
|
52
54
|
| Flag | Default | Description |
|
|
53
55
|
|---|---|---|
|
|
54
|
-
| `--config <path>` | `./.dirsql.toml` | Path to the config file. The index is rooted at the directory containing this file. |
|
|
56
|
+
| `--config <path>` | `./.dirsql.toml` | Path to the config file (`.toml` or [native-language](./config.md#native-language-configs)). The index is rooted at the directory containing this file. |
|
|
55
57
|
| `--host <addr>` | `localhost` | Bind address |
|
|
56
58
|
| `--port <n>` | `7117` | TCP port to bind |
|
|
57
59
|
|
|
@@ -9,8 +9,8 @@ importers:
|
|
|
9
9
|
.:
|
|
10
10
|
devDependencies:
|
|
11
11
|
'@playwright/test':
|
|
12
|
-
specifier: ^1.
|
|
13
|
-
version: 1.
|
|
12
|
+
specifier: ^1.60.0
|
|
13
|
+
version: 1.60.0
|
|
14
14
|
vitepress:
|
|
15
15
|
specifier: ^1.5.0
|
|
16
16
|
version: 1.6.4(@algolia/client-search@5.50.1)(postcss@8.5.8)(search-insights@2.17.3)
|
|
@@ -283,8 +283,8 @@ packages:
|
|
|
283
283
|
'@jridgewell/sourcemap-codec@1.5.5':
|
|
284
284
|
resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==}
|
|
285
285
|
|
|
286
|
-
'@playwright/test@1.
|
|
287
|
-
resolution: {integrity: sha512-
|
|
286
|
+
'@playwright/test@1.60.0':
|
|
287
|
+
resolution: {integrity: sha512-O71yZIbAh/PxDMNGns37GHBIfrVkEVyn+AXyIa5dOTfb4/xNvRWV+Vv/NMbNCtODB/pO7vLlF2OTmMVLhmr7Ag==}
|
|
288
288
|
engines: {node: '>=18'}
|
|
289
289
|
hasBin: true
|
|
290
290
|
|
|
@@ -774,13 +774,13 @@ packages:
|
|
|
774
774
|
picocolors@1.1.1:
|
|
775
775
|
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
|
|
776
776
|
|
|
777
|
-
playwright-core@1.
|
|
778
|
-
resolution: {integrity: sha512-
|
|
777
|
+
playwright-core@1.60.0:
|
|
778
|
+
resolution: {integrity: sha512-9bW6zvX/m0lEbgTKJ6YppOKx8H3VOPBMOCFh2irXFOT4BbHgrx5hPjwJYLT40Lu+4qtD36qKc/Hn56StUW57IA==}
|
|
779
779
|
engines: {node: '>=18'}
|
|
780
780
|
hasBin: true
|
|
781
781
|
|
|
782
|
-
playwright@1.
|
|
783
|
-
resolution: {integrity: sha512-
|
|
782
|
+
playwright@1.60.0:
|
|
783
|
+
resolution: {integrity: sha512-hheHdokM8cdqCb0lcE3s+zT4t4W+vvjpGxsZlDnikarzx8tSzMebh3UiFtgqwFwnTnjYQcsyMF8ei2mCO/tpeA==}
|
|
784
784
|
engines: {node: '>=18'}
|
|
785
785
|
hasBin: true
|
|
786
786
|
|
|
@@ -1206,9 +1206,9 @@ snapshots:
|
|
|
1206
1206
|
|
|
1207
1207
|
'@jridgewell/sourcemap-codec@1.5.5': {}
|
|
1208
1208
|
|
|
1209
|
-
'@playwright/test@1.
|
|
1209
|
+
'@playwright/test@1.60.0':
|
|
1210
1210
|
dependencies:
|
|
1211
|
-
playwright: 1.
|
|
1211
|
+
playwright: 1.60.0
|
|
1212
1212
|
|
|
1213
1213
|
'@rollup/rollup-android-arm-eabi@4.60.1':
|
|
1214
1214
|
optional: true
|
|
@@ -1686,11 +1686,11 @@ snapshots:
|
|
|
1686
1686
|
|
|
1687
1687
|
picocolors@1.1.1: {}
|
|
1688
1688
|
|
|
1689
|
-
playwright-core@1.
|
|
1689
|
+
playwright-core@1.60.0: {}
|
|
1690
1690
|
|
|
1691
|
-
playwright@1.
|
|
1691
|
+
playwright@1.60.0:
|
|
1692
1692
|
dependencies:
|
|
1693
|
-
playwright-core: 1.
|
|
1693
|
+
playwright-core: 1.60.0
|
|
1694
1694
|
optionalDependencies:
|
|
1695
1695
|
fsevents: 2.3.2
|
|
1696
1696
|
|
|
@@ -6,7 +6,10 @@ canonical: https://thekevinscott.github.io/dirsql/cli/config
|
|
|
6
6
|
|
|
7
7
|
> Online: <https://thekevinscott.github.io/dirsql/cli/config>
|
|
8
8
|
|
|
9
|
-
`dirsql` can be configured with an optional
|
|
9
|
+
`dirsql` can be configured with an optional config file (if omitted, server falls back to [defaults](./server.md#defaults)). Two formats are accepted:
|
|
10
|
+
|
|
11
|
+
- **`dirsql.toml`** — declarative; covers filesystem-fact tables. Works with any installation.
|
|
12
|
+
- **`.py` / `.js`** — native-language; lets you write `extract` callbacks in Python or JavaScript. CLI-only, and only the launcher matching the file's language can run it. See [Native-Language Configs](#native-language-configs).
|
|
10
13
|
|
|
11
14
|
## Basic Example
|
|
12
15
|
|
|
@@ -172,10 +175,76 @@ ddl = "CREATE TABLE logs (_path TEXT, _size INTEGER, _mtime INTEGER)"
|
|
|
172
175
|
glob = "logs/*.csv"
|
|
173
176
|
```
|
|
174
177
|
|
|
175
|
-
##
|
|
178
|
+
## Native-Language Configs
|
|
179
|
+
|
|
180
|
+
You can provide a config file in a particular language, allowing you to define a dynamic extract function. This can be useful for building a database based on the _contents_ of a file.
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
dirsql --config dirsql.config.py
|
|
184
|
+
dirsql --config dirsql.config.js
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
The file looks exactly like the in-process SDK construction — same
|
|
188
|
+
`DirSQL` / `Table` API:
|
|
189
|
+
|
|
190
|
+
::: code-group
|
|
191
|
+
|
|
192
|
+
```python [dirsql.config.py]
|
|
193
|
+
import json
|
|
194
|
+
from dirsql import DirSQL, Table
|
|
195
|
+
|
|
196
|
+
def extract_meta(path):
|
|
197
|
+
with open(path) as f:
|
|
198
|
+
return [json.load(f)]
|
|
199
|
+
|
|
200
|
+
# Python must export an `app` variable
|
|
201
|
+
app = DirSQL(
|
|
202
|
+
tables=[
|
|
203
|
+
Table(
|
|
204
|
+
ddl="CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
205
|
+
glob="**/meta.json",
|
|
206
|
+
extract=extract_meta,
|
|
207
|
+
),
|
|
208
|
+
],
|
|
209
|
+
)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
```javascript [dirsql.config.mjs]
|
|
213
|
+
import { readFileSync } from "node:fs";
|
|
214
|
+
import { DirSQL } from "dirsql";
|
|
215
|
+
|
|
216
|
+
export default new DirSQL({
|
|
217
|
+
tables: [
|
|
218
|
+
{
|
|
219
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
220
|
+
glob: "**/meta.json",
|
|
221
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
222
|
+
},
|
|
223
|
+
],
|
|
224
|
+
});
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
```javascript [dirsql.config.cjs]
|
|
228
|
+
const { readFileSync } = require("node:fs");
|
|
229
|
+
const { DirSQL } = require("dirsql");
|
|
230
|
+
|
|
231
|
+
module.exports = new DirSQL({
|
|
232
|
+
tables: [
|
|
233
|
+
{
|
|
234
|
+
ddl: "CREATE TABLE papers (title TEXT, _path TEXT)",
|
|
235
|
+
glob: "**/meta.json",
|
|
236
|
+
extract: (path) => [JSON.parse(readFileSync(path, "utf8"))],
|
|
237
|
+
},
|
|
238
|
+
],
|
|
239
|
+
});
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
:::
|
|
243
|
+
|
|
244
|
+
Only the extension matters — the file can be named anything. `dirsql.config.{py,mjs,cjs}` is the suggested convention but not required.
|
|
245
|
+
|
|
246
|
+
### Module conventions
|
|
176
247
|
|
|
177
|
-
`.
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
bytes in your host language. Glob captures and stat virtuals are still
|
|
181
|
-
auto-injected into rows produced by your extract.
|
|
248
|
+
- **Python (`.py`)** — module-level `app = DirSQL(...)`.
|
|
249
|
+
- **ESM (`.mjs`, or `.js` in an ESM package)** — `export default new DirSQL(...)`.
|
|
250
|
+
- **CommonJS (`.cjs`, or `.js` in a CJS package)** — `module.exports = new DirSQL(...)`.
|
|
@@ -10,16 +10,17 @@ canonical: https://thekevinscott.github.io/dirsql/cli/
|
|
|
10
10
|
the same indexing, querying, and watching functionality as the SDK — no host
|
|
11
11
|
language required. Run it in any directory to query your files over HTTP:
|
|
12
12
|
with no config it serves a default `files` table, and a
|
|
13
|
-
[
|
|
13
|
+
[config file](./config.md) defines custom tables.
|
|
14
14
|
|
|
15
15
|
Everything you need to run `dirsql` as a CLI lives in this section:
|
|
16
16
|
|
|
17
17
|
- **[Installation](#installation)** — get the `dirsql` binary.
|
|
18
18
|
- **[Running the Server](./server.md)** — subcommands and flags.
|
|
19
19
|
- **[Generating a Config (`init`)](./init.md)** — scaffold a `.dirsql.toml`.
|
|
20
|
-
- **[Configuration File](./config.md)** — the `.dirsql.toml` format
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
- **[Configuration File](./config.md)** — the `.dirsql.toml` format and the
|
|
21
|
+
`.py` / `.js` native-language alternative. Custom tables
|
|
22
|
+
are defined through a config file; without one, the server runs in
|
|
23
|
+
[zero-config mode](./server.md#zero-config-mode).
|
|
23
24
|
- **[HTTP API](./http-api.md)** — the `POST /query` and `GET /events`
|
|
24
25
|
endpoints, status codes, and event streaming.
|
|
25
26
|
|