dirsql 0.3.109 → 0.3.110
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/getting-started.md +7 -5
- package/docs/howto/columns-from-paths.md +5 -2
- package/docs/howto/define-tables.md +7 -5
- package/docs/howto/extract-from-contents.md +5 -2
- package/docs/howto/load-extension.md +4 -3
- package/docs/howto/search-by-meaning.md +5 -3
- package/docs/howto/skip-files.md +5 -2
- package/docs/reference/cli.md +33 -18
- package/docs/reference/config.md +7 -5
- package/package.json +11 -11
package/docs/getting-started.md
CHANGED
|
@@ -92,7 +92,7 @@ open a **second terminal** for the next step.
|
|
|
92
92
|
## 3. Query your files
|
|
93
93
|
|
|
94
94
|
You gave `dirsql` no configuration, so it serves a single default table
|
|
95
|
-
named `files` ([
|
|
95
|
+
named `files` ([default mode](./reference/cli.md#default-mode)).
|
|
96
96
|
Ask it how many rows it has:
|
|
97
97
|
|
|
98
98
|
```bash
|
|
@@ -167,16 +167,18 @@ Two keys define the table:
|
|
|
167
167
|
## 5. Restart and query the new shape
|
|
168
168
|
|
|
169
169
|
Config is read at startup, so go back to the **first terminal**, stop the
|
|
170
|
-
server with `Ctrl-C`, and start it again
|
|
170
|
+
server with `Ctrl-C`, and start it again — this time pointing `dirsql` at
|
|
171
|
+
your config with `-c` (`dirsql` does not auto-load a `.dirsql.toml` from the
|
|
172
|
+
current directory; you always pass it explicitly):
|
|
171
173
|
|
|
172
174
|
::: code-group
|
|
173
175
|
|
|
174
176
|
```bash [npm]
|
|
175
|
-
npx dirsql
|
|
177
|
+
npx dirsql -c .dirsql.toml
|
|
176
178
|
```
|
|
177
179
|
|
|
178
180
|
```bash [PyPI]
|
|
179
|
-
uvx dirsql
|
|
181
|
+
uvx dirsql -c .dirsql.toml
|
|
180
182
|
```
|
|
181
183
|
|
|
182
184
|
:::
|
|
@@ -185,7 +187,7 @@ uvx dirsql
|
|
|
185
187
|
Running at localhost:7117
|
|
186
188
|
```
|
|
187
189
|
|
|
188
|
-
This time `dirsql`
|
|
190
|
+
This time `dirsql` loaded your `.dirsql.toml` and served the `notes` table
|
|
189
191
|
you defined instead of the default `files` table. Query it from the second
|
|
190
192
|
terminal:
|
|
191
193
|
|
|
@@ -29,8 +29,11 @@ within one path segment) are in
|
|
|
29
29
|
|
|
30
30
|
## 2. Query the captured columns
|
|
31
31
|
|
|
32
|
+
Pass the config with [`-c`](../reference/cli.md#flags) (`dirsql` does not
|
|
33
|
+
auto-load a `.dirsql.toml` from the current directory):
|
|
34
|
+
|
|
32
35
|
```bash
|
|
33
|
-
dirsql query "SELECT year, month, basename FROM photos ORDER BY year, month"
|
|
36
|
+
dirsql -c ./.dirsql.toml query "SELECT year, month, basename FROM photos ORDER BY year, month"
|
|
34
37
|
```
|
|
35
38
|
|
|
36
39
|
```json
|
|
@@ -40,7 +43,7 @@ dirsql query "SELECT year, month, basename FROM photos ORDER BY year, month"
|
|
|
40
43
|
Captures are real SQL columns, so aggregation works:
|
|
41
44
|
|
|
42
45
|
```bash
|
|
43
|
-
dirsql query "SELECT year, COUNT(*) AS photos FROM photos GROUP BY year"
|
|
46
|
+
dirsql -c ./.dirsql.toml query "SELECT year, COUNT(*) AS photos FROM photos GROUP BY year"
|
|
44
47
|
```
|
|
45
48
|
|
|
46
49
|
```json
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Map a glob of files to a named SQL table so you query exactly the files you
|
|
4
4
|
care about, with exactly the columns you care about — instead of the
|
|
5
|
-
catch-all `files` table that [
|
|
5
|
+
catch-all `files` table that [default mode](../reference/cli.md#default-mode)
|
|
6
6
|
serves.
|
|
7
7
|
|
|
8
8
|
## 1. Create a config next to your files
|
|
@@ -26,10 +26,12 @@ glob = "posts/**/*.md"
|
|
|
26
26
|
|
|
27
27
|
## 2. Query the table
|
|
28
28
|
|
|
29
|
-
|
|
29
|
+
Pass the config with [`-c`](../reference/cli.md#flags) — `dirsql` does not
|
|
30
|
+
auto-load a `.dirsql.toml` from the current directory. Each matched file is
|
|
31
|
+
one row:
|
|
30
32
|
|
|
31
33
|
```bash
|
|
32
|
-
dirsql query "SELECT path, size FROM posts ORDER BY path"
|
|
34
|
+
dirsql -c ./.dirsql.toml query "SELECT path, size FROM posts ORDER BY path"
|
|
33
35
|
```
|
|
34
36
|
|
|
35
37
|
```json
|
|
@@ -37,8 +39,8 @@ dirsql query "SELECT path, size FROM posts ORDER BY path"
|
|
|
37
39
|
```
|
|
38
40
|
|
|
39
41
|
Files that don't match the glob (a `README.txt` next to `posts/`, say) are
|
|
40
|
-
simply not in the table.
|
|
41
|
-
|
|
42
|
+
simply not in the table. Passing a config with `-c` fully replaces the
|
|
43
|
+
default `files` table — only the tables you define are served.
|
|
42
44
|
|
|
43
45
|
## Multiple tables
|
|
44
46
|
|
|
@@ -31,8 +31,11 @@ by every hook.
|
|
|
31
31
|
|
|
32
32
|
## 2. Query the extracted columns
|
|
33
33
|
|
|
34
|
+
Pass the config with [`-c`](../reference/cli.md#flags) (`dirsql` does not
|
|
35
|
+
auto-load a `.dirsql.toml` from the current directory):
|
|
36
|
+
|
|
34
37
|
```bash
|
|
35
|
-
dirsql query "SELECT title, author, year, path FROM books ORDER BY year"
|
|
38
|
+
dirsql -c ./.dirsql.toml query "SELECT title, author, year, path FROM books ORDER BY year"
|
|
36
39
|
```
|
|
37
40
|
|
|
38
41
|
```json
|
|
@@ -57,7 +60,7 @@ on-file = "jq -c -s '.' {path}"
|
|
|
57
60
|
```
|
|
58
61
|
|
|
59
62
|
```bash
|
|
60
|
-
dirsql query "SELECT event, user FROM events"
|
|
63
|
+
dirsql -c ./.dirsql.toml query "SELECT event, user FROM events"
|
|
61
64
|
```
|
|
62
65
|
|
|
63
66
|
```json
|
|
@@ -23,10 +23,11 @@ overrides the init symbol when it doesn't match the filename-derived
|
|
|
23
23
|
default — `sqlite-vec` is exactly such a case
|
|
24
24
|
([reference](../reference/config.md#dirsql-extension)).
|
|
25
25
|
|
|
26
|
-
The extension's functions are callable
|
|
26
|
+
The extension's functions are callable (pass the config with
|
|
27
|
+
[`-c`](../reference/cli.md#flags) so its `[[dirsql.extension]]` entry loads):
|
|
27
28
|
|
|
28
29
|
```bash
|
|
29
|
-
dirsql query "SELECT vec_version() AS vec_version"
|
|
30
|
+
dirsql -c ./.dirsql.toml query "SELECT vec_version() AS vec_version"
|
|
30
31
|
```
|
|
31
32
|
|
|
32
33
|
```json
|
|
@@ -52,7 +53,7 @@ entrypoint = "sqlite3_vec_init"
|
|
|
52
53
|
```
|
|
53
54
|
|
|
54
55
|
```bash
|
|
55
|
-
uvx --with sqlite-vec dirsql
|
|
56
|
+
uvx --with sqlite-vec dirsql -c ./.dirsql.toml
|
|
56
57
|
```
|
|
57
58
|
|
|
58
59
|
**Node (`npx dirsql`, TypeScript SDK).** Use the *npm package name* — but
|
|
@@ -103,10 +103,12 @@ runtime — and the literal-path alternative that works everywhere — are in
|
|
|
103
103
|
|
|
104
104
|
Run with `sqlite-vec` available to the launcher's environment. The initial
|
|
105
105
|
scan runs `embed.py` once per note, then the query argument goes straight to
|
|
106
|
-
`pre-query`, exactly as a `POST /query` body would
|
|
106
|
+
`pre-query`, exactly as a `POST /query` body would. Pass the config with
|
|
107
|
+
[`-c`](../reference/cli.md#flags) — `dirsql` does not auto-load a
|
|
108
|
+
`.dirsql.toml` from the current directory:
|
|
107
109
|
|
|
108
110
|
```bash
|
|
109
|
-
uvx --with sqlite-vec dirsql query '{"q": "how do I cook pasta?"}'
|
|
111
|
+
uvx --with sqlite-vec dirsql -c ./.dirsql.toml query '{"q": "how do I cook pasta?"}'
|
|
110
112
|
```
|
|
111
113
|
|
|
112
114
|
```json
|
|
@@ -114,7 +116,7 @@ uvx --with sqlite-vec dirsql query '{"q": "how do I cook pasta?"}'
|
|
|
114
116
|
```
|
|
115
117
|
|
|
116
118
|
```bash
|
|
117
|
-
uvx --with sqlite-vec dirsql query '{"q": "reviewing code on github"}'
|
|
119
|
+
uvx --with sqlite-vec dirsql -c ./.dirsql.toml query '{"q": "reviewing code on github"}'
|
|
118
120
|
```
|
|
119
121
|
|
|
120
122
|
```json
|
package/docs/howto/skip-files.md
CHANGED
|
@@ -31,8 +31,11 @@ ignored file never reaches any table — even one whose glob would match it.
|
|
|
31
31
|
|
|
32
32
|
## 2. Confirm what made it in
|
|
33
33
|
|
|
34
|
+
Pass the config with [`-c`](../reference/cli.md#flags) (`dirsql` does not
|
|
35
|
+
auto-load a `.dirsql.toml` from the current directory):
|
|
36
|
+
|
|
34
37
|
```bash
|
|
35
|
-
dirsql query "SELECT path FROM notes ORDER BY path"
|
|
38
|
+
dirsql -c ./.dirsql.toml query "SELECT path FROM notes ORDER BY path"
|
|
36
39
|
```
|
|
37
40
|
|
|
38
41
|
```json
|
|
@@ -42,7 +45,7 @@ dirsql query "SELECT path FROM notes ORDER BY path"
|
|
|
42
45
|
## Notes
|
|
43
46
|
|
|
44
47
|
- `ignore` lives in a config file, so it needs one:
|
|
45
|
-
[
|
|
48
|
+
[default mode](../reference/cli.md#default-mode) indexes
|
|
46
49
|
everything with no ignores.
|
|
47
50
|
- The top-level `.dirsql/` directory is always excluded, ignore list or
|
|
48
51
|
not — it is reserved for `dirsql`'s own metadata
|
package/docs/reference/cli.md
CHANGED
|
@@ -45,7 +45,7 @@ requests, closes open `/events` streams, and exits.
|
|
|
45
45
|
|
|
46
46
|
| Flag | Default | Description |
|
|
47
47
|
|---|---|---|
|
|
48
|
-
| `-c, --config <path>` |
|
|
48
|
+
| `-c, --config <path>` | baked-in default | Path to a [config file](./config.md). **Repeatable** (`-c a -c b`): the configs load and merge in argv order — see [Composing multiple configs](./config.md#composing-multiple-configs). The index is always rooted at the **invocation directory** (the current working directory), regardless of where a config lives — so `--config /elsewhere/.dirsql.toml` still indexes the directory you ran `dirsql` from. With none given, the [baked-in default](#default-mode) `files` table is served — a `./.dirsql.toml` on disk is **not** auto-loaded; pass it explicitly. A `-c` naming a file that does not exist is an [error](#degraded-mode), not a silent fallback to the default. |
|
|
49
49
|
| `--host <addr>` | `localhost` | Bind address. |
|
|
50
50
|
| `--port <n>` | `7117` | TCP port to bind. |
|
|
51
51
|
| `--persist [<path>]` | off | Keep the SQLite index on disk between runs so a restart only re-parses files that actually changed. Bare `--persist` caches at `<root>/.dirsql/cache.db`; `--persist <path>` caches at `<path>`. Off by default (the index is ephemeral). Global — also honored by [`dirsql query`](#dirsql-query). See [Keep the index across restarts](../howto/persist.md). |
|
|
@@ -61,11 +61,14 @@ requests, closes open `/events` streams, and exits.
|
|
|
61
61
|
**30-second** timeout each, overridable with the config key
|
|
62
62
|
[`[dirsql].hook-timeout`](./config.md#dirsql-keys).
|
|
63
63
|
|
|
64
|
-
###
|
|
64
|
+
### Default mode
|
|
65
65
|
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
66
|
+
With no `-c/--config`, the server serves the **baked-in default** — the
|
|
67
|
+
shipped config compiled into the binary, indexing the invocation directory
|
|
68
|
+
with a single table named `files`. This is a fixed default, *not* a
|
|
69
|
+
`.dirsql.toml` read from disk: a `./.dirsql.toml` sitting in the current
|
|
70
|
+
directory is **not** auto-loaded (pass it with `-c ./.dirsql.toml` to use
|
|
71
|
+
it). The default `files` table:
|
|
69
72
|
|
|
70
73
|
- Glob: `**/*` — every file under the root, at any depth, no ignores.
|
|
71
74
|
- One row per file, with all seven
|
|
@@ -77,21 +80,25 @@ curl -s localhost:7117/query -H 'content-type: application/json' \
|
|
|
77
80
|
-d '{"sql":"SELECT basename, size FROM files ORDER BY size DESC LIMIT 5"}'
|
|
78
81
|
```
|
|
79
82
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
below).
|
|
83
|
+
Passing a config with `-c` fully overrules this default. A `-c` naming a file
|
|
84
|
+
that does not exist is an error (not a fallback to the default); a config that
|
|
85
|
+
exists but fails to load degrades the server (see below).
|
|
83
86
|
|
|
84
87
|
### Degraded mode
|
|
85
88
|
|
|
86
|
-
When
|
|
87
|
-
|
|
88
|
-
every request to `/query` and `/events` returns
|
|
89
|
-
with a JSON body describing the failure
|
|
89
|
+
When a config passed with `-c` cannot be resolved or loaded — the file does
|
|
90
|
+
not exist, is unreadable, or has invalid TOML / schema errors — the server
|
|
91
|
+
still starts and binds, but every request to `/query` and `/events` returns
|
|
92
|
+
`503 Service Unavailable` with a JSON body describing the failure (the
|
|
93
|
+
diagnostic names the offending path or key):
|
|
90
94
|
|
|
91
95
|
```json
|
|
92
96
|
{"error": "failed to load config: ..."}
|
|
93
97
|
```
|
|
94
98
|
|
|
99
|
+
The one-shot [`dirsql query`](#dirsql-query) surfaces the same failure as a
|
|
100
|
+
non-zero exit with the diagnostic on stderr.
|
|
101
|
+
|
|
95
102
|
### Exit codes
|
|
96
103
|
|
|
97
104
|
| Code | Meaning |
|
|
@@ -104,10 +111,12 @@ with a JSON body describing the failure:
|
|
|
104
111
|
Run a SQL query from the shell:
|
|
105
112
|
|
|
106
113
|
```bash
|
|
114
|
+
# No -c: the baked-in default `files` table.
|
|
107
115
|
dirsql query "SELECT basename, size FROM files ORDER BY size DESC LIMIT 5"
|
|
108
116
|
# [{"basename":"model.bin","size":104857600}, …]
|
|
109
117
|
|
|
110
|
-
|
|
118
|
+
# A config table (`posts`) needs its config passed explicitly.
|
|
119
|
+
dirsql -c ./.dirsql.toml query "SELECT COUNT(*) AS n FROM posts" | jq '.[0].n'
|
|
111
120
|
```
|
|
112
121
|
|
|
113
122
|
The subcommand builds the index, runs the SQL, prints the result rows as a
|
|
@@ -117,9 +126,9 @@ response body), and exits `0`.
|
|
|
117
126
|
`dirsql query` is a thin adapter over the **same query pipeline the server
|
|
118
127
|
uses**, so behavior is identical to `POST /query` by construction:
|
|
119
128
|
|
|
120
|
-
- **Config discovery** honors `--config` (
|
|
121
|
-
[
|
|
122
|
-
|
|
129
|
+
- **Config discovery** honors `--config` (with none given, the
|
|
130
|
+
[baked-in default](#default-mode)), and `--extension` overrides, exactly as
|
|
131
|
+
server mode does.
|
|
123
132
|
- **`--persist [<path>]`** is honored, so a repeated `dirsql query` reuses the
|
|
124
133
|
on-disk cache. Because its value is optional, place a bare `--persist` after
|
|
125
134
|
the SQL (`dirsql query "SELECT …" --persist`) or use the `=` form
|
|
@@ -144,13 +153,19 @@ stderr, with exit code `1`.
|
|
|
144
153
|
|
|
145
154
|
## `dirsql init`
|
|
146
155
|
|
|
147
|
-
Writes a starter `.dirsql.toml
|
|
156
|
+
Writes a starter `.dirsql.toml` — the same table the [baked-in
|
|
157
|
+
default](#default-mode) serves — as a scaffold to edit:
|
|
148
158
|
|
|
149
159
|
```bash
|
|
150
160
|
dirsql init
|
|
151
161
|
```
|
|
152
162
|
|
|
153
|
-
|
|
163
|
+
The output does **not** auto-load. Once you've tweaked it, pass it explicitly
|
|
164
|
+
to run against it:
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
dirsql -c ./.dirsql.toml
|
|
168
|
+
```
|
|
154
169
|
|
|
155
170
|
### Flags
|
|
156
171
|
|
package/docs/reference/config.md
CHANGED
|
@@ -7,9 +7,10 @@ all-defaults one. Unknown keys are a parse error at every level (top level,
|
|
|
7
7
|
`[dirsql]`, `[[table]]`, `[[dirsql.extension]]`) — a typo or a removed key
|
|
8
8
|
fails loudly, naming the offending key, rather than silently no-opping.
|
|
9
9
|
|
|
10
|
-
The [CLI](./cli.md) loads
|
|
11
|
-
|
|
12
|
-
|
|
10
|
+
The [CLI](./cli.md) loads a config only when you pass it with `-c/--config`;
|
|
11
|
+
with none given it serves the [baked-in default](./cli.md#default-mode) (a
|
|
12
|
+
`./.dirsql.toml` on disk is **not** auto-loaded). The [SDKs](./sdk.md) load a
|
|
13
|
+
config via the `config` constructor parameter.
|
|
13
14
|
|
|
14
15
|
**Path resolution.** Relative paths in the config (`[[dirsql.extension]]`
|
|
15
16
|
`path`) resolve against the config file's parent
|
|
@@ -160,8 +161,9 @@ The configs load and merge in **argv order**:
|
|
|
160
161
|
error**, naming the table.
|
|
161
162
|
|
|
162
163
|
The index [root](./cli.md#flags) is the invocation directory regardless of where
|
|
163
|
-
any config lives. With no `-c`,
|
|
164
|
-
exactly as
|
|
164
|
+
any config lives. With no `-c`, the [baked-in default](./cli.md#default-mode) is
|
|
165
|
+
served (no `./.dirsql.toml` auto-discovery); a single `-c` behaves exactly as
|
|
166
|
+
before.
|
|
165
167
|
|
|
166
168
|
## Parse errors
|
|
167
169
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dirsql",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.110",
|
|
4
4
|
"description": "Ephemeral SQL index over a local directory",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": "https://github.com/thekevinscott/dirsql",
|
|
@@ -212,15 +212,15 @@
|
|
|
212
212
|
]
|
|
213
213
|
},
|
|
214
214
|
"optionalDependencies": {
|
|
215
|
-
"@dirsql/lib-linux-x64-gnu": "0.3.
|
|
216
|
-
"@dirsql/lib-linux-arm64-gnu": "0.3.
|
|
217
|
-
"@dirsql/lib-darwin-x64": "0.3.
|
|
218
|
-
"@dirsql/lib-darwin-arm64": "0.3.
|
|
219
|
-
"@dirsql/lib-win32-x64-msvc": "0.3.
|
|
220
|
-
"@dirsql/cli-linux-x64-gnu": "0.3.
|
|
221
|
-
"@dirsql/cli-linux-arm64-gnu": "0.3.
|
|
222
|
-
"@dirsql/cli-darwin-x64": "0.3.
|
|
223
|
-
"@dirsql/cli-darwin-arm64": "0.3.
|
|
224
|
-
"@dirsql/cli-win32-x64-msvc": "0.3.
|
|
215
|
+
"@dirsql/lib-linux-x64-gnu": "0.3.110",
|
|
216
|
+
"@dirsql/lib-linux-arm64-gnu": "0.3.110",
|
|
217
|
+
"@dirsql/lib-darwin-x64": "0.3.110",
|
|
218
|
+
"@dirsql/lib-darwin-arm64": "0.3.110",
|
|
219
|
+
"@dirsql/lib-win32-x64-msvc": "0.3.110",
|
|
220
|
+
"@dirsql/cli-linux-x64-gnu": "0.3.110",
|
|
221
|
+
"@dirsql/cli-linux-arm64-gnu": "0.3.110",
|
|
222
|
+
"@dirsql/cli-darwin-x64": "0.3.110",
|
|
223
|
+
"@dirsql/cli-darwin-arm64": "0.3.110",
|
|
224
|
+
"@dirsql/cli-win32-x64-msvc": "0.3.110"
|
|
225
225
|
}
|
|
226
226
|
}
|