dirsql 0.3.116 → 0.3.118
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/getting-started.md +7 -7
- package/docs/howto/define-tables.md +3 -4
- package/docs/howto/react-to-changes.md +13 -2
- package/docs/howto/skip-files.md +1 -1
- package/docs/howto/write-a-plugin.md +2 -3
- package/docs/reference/cli.md +20 -19
- package/docs/reference/config.md +7 -7
- package/docs/reference/sdk.md +6 -4
- package/package.json +11 -11
package/docs/getting-started.md
CHANGED
|
@@ -88,14 +88,15 @@ open a **second terminal** for the next step.
|
|
|
88
88
|
|
|
89
89
|
## 3. Query your files
|
|
90
90
|
|
|
91
|
-
You gave `dirsql` no configuration, so
|
|
92
|
-
|
|
93
|
-
|
|
91
|
+
You gave `dirsql` no configuration, so no named tables exist. Query the
|
|
92
|
+
filesystem directly with a [path-table](./reference/path-tables.md) — a
|
|
93
|
+
quoted path where a table name goes. `'./'` means everything under the
|
|
94
|
+
index root. Ask it how many files there are:
|
|
94
95
|
|
|
95
96
|
```bash
|
|
96
97
|
curl -s http://localhost:7117/query \
|
|
97
98
|
-H 'content-type: application/json' \
|
|
98
|
-
-d '{"sql":"SELECT COUNT(*) AS files FROM
|
|
99
|
+
-d '{"sql":"SELECT COUNT(*) AS files FROM \'./\'"}'
|
|
99
100
|
```
|
|
100
101
|
|
|
101
102
|
```
|
|
@@ -109,7 +110,7 @@ through `jq` to pretty-print. Now select some columns:
|
|
|
109
110
|
```bash
|
|
110
111
|
curl -s http://localhost:7117/query \
|
|
111
112
|
-H 'content-type: application/json' \
|
|
112
|
-
-d '{"sql":"SELECT path, size FROM
|
|
113
|
+
-d '{"sql":"SELECT path, size FROM \'./\' ORDER BY path"}' \
|
|
113
114
|
| jq
|
|
114
115
|
```
|
|
115
116
|
|
|
@@ -185,8 +186,7 @@ Running at localhost:7117
|
|
|
185
186
|
```
|
|
186
187
|
|
|
187
188
|
This time `dirsql` loaded your `.dirsql.toml` and served the `notes` table
|
|
188
|
-
you defined
|
|
189
|
-
terminal:
|
|
189
|
+
you defined. Query it from the second terminal:
|
|
190
190
|
|
|
191
191
|
```bash
|
|
192
192
|
curl -s http://localhost:7117/query \
|
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
Map a glob of files to a named SQL table so you query exactly the files you
|
|
4
4
|
care about, with exactly the columns you care about — instead of the
|
|
5
|
-
|
|
6
|
-
|
|
5
|
+
ad-hoc [path-tables](../reference/path-tables.md)
|
|
6
|
+
[configless mode](../reference/cli.md#configless-mode) leaves you with.
|
|
7
7
|
|
|
8
8
|
## 1. Create a config next to your files
|
|
9
9
|
|
|
@@ -39,8 +39,7 @@ dirsql query "SELECT path, size FROM posts ORDER BY path" -c ./.dirsql.toml
|
|
|
39
39
|
```
|
|
40
40
|
|
|
41
41
|
Files that don't match the glob (a `README.txt` next to `posts/`, say) are
|
|
42
|
-
simply not in the table.
|
|
43
|
-
default `files` table — only the tables you define are served.
|
|
42
|
+
simply not in the table. Only the tables you define are served.
|
|
44
43
|
|
|
45
44
|
## Multiple tables
|
|
46
45
|
|
|
@@ -5,10 +5,21 @@ change — and [`GET /events`](../reference/http-api.md#get-events) pushes
|
|
|
5
5
|
every row-level change to you as it happens. No polling, no diffing on your
|
|
6
6
|
side.
|
|
7
7
|
|
|
8
|
+
Row events are emitted for **named tables**, so this flow needs a config —
|
|
9
|
+
[path-tables](../reference/path-tables.md) are scanned per query and are not
|
|
10
|
+
watched. Define one next to your files:
|
|
11
|
+
|
|
12
|
+
```toml
|
|
13
|
+
# .dirsql.toml
|
|
14
|
+
[[table]]
|
|
15
|
+
ddl = "CREATE TABLE files (path TEXT, basename TEXT, dir TEXT, ext TEXT, size INTEGER, mtime INTEGER, ctime INTEGER)"
|
|
16
|
+
glob = "**/*"
|
|
17
|
+
```
|
|
18
|
+
|
|
8
19
|
## 1. Open the stream
|
|
9
20
|
|
|
10
|
-
With the server running (`npx dirsql
|
|
11
|
-
another terminal:
|
|
21
|
+
With the server running (`npx dirsql -c ./.dirsql.toml` /
|
|
22
|
+
`uvx dirsql -c ./.dirsql.toml`), subscribe from another terminal:
|
|
12
23
|
|
|
13
24
|
```bash
|
|
14
25
|
curl -N http://localhost:7117/events
|
package/docs/howto/skip-files.md
CHANGED
|
@@ -45,7 +45,7 @@ dirsql query "SELECT path FROM notes ORDER BY path" -c ./.dirsql.toml
|
|
|
45
45
|
## Notes
|
|
46
46
|
|
|
47
47
|
- `ignore` lives in a config file, so it needs one:
|
|
48
|
-
[
|
|
48
|
+
[configless mode](../reference/cli.md#configless-mode) indexes
|
|
49
49
|
everything with no ignores.
|
|
50
50
|
- The top-level `.dirsql/` directory is always excluded, ignore list or
|
|
51
51
|
not — it is reserved for `dirsql`'s own metadata
|
|
@@ -217,9 +217,8 @@ Discovery is deliberately narrow. Know exactly who does what:
|
|
|
217
217
|
your config still takes ordering precedence
|
|
218
218
|
([composing configs](../reference/config.md#composing-multiple-configs)). When
|
|
219
219
|
you pass no `-c` of your own, the launcher also keeps the
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
replacing the default. Discovery is **pip/uvx only** for now — the `npx`
|
|
220
|
+
shipped starter `files` table (an internal `--include-default`), so plugins
|
|
221
|
+
**add** tables rather than standing alone. Discovery is **pip/uvx only** for now — the `npx`
|
|
223
222
|
launcher does not yet discover — and is switched off per invocation with
|
|
224
223
|
[`--no-plugin` or `DIRSQL_NO_PLUGIN=1`](../reference/cli.md#plugins).
|
|
225
224
|
- **The SDK never auto-discovers.** Pass a plugin's config explicitly (the
|
package/docs/reference/cli.md
CHANGED
|
@@ -45,7 +45,7 @@ requests, closes open `/events` streams, and exits.
|
|
|
45
45
|
|
|
46
46
|
| Flag | Default | Description |
|
|
47
47
|
|---|---|---|
|
|
48
|
-
| `-c, --config <path>` |
|
|
48
|
+
| `-c, --config <path>` | none | Path to a [config file](./config.md). **Repeatable** (`-c a -c b`): the configs load and merge in argv order — see [Composing multiple configs](./config.md#composing-multiple-configs). The index is always rooted at the **invocation directory** (the current working directory), regardless of where a config lives — so `--config /elsewhere/.dirsql.toml` still indexes the directory you ran `dirsql` from. With none given, **no named tables are defined** — query the filesystem with a [path-table](./path-tables.md) (`FROM './'`). A `./.dirsql.toml` on disk is **not** auto-loaded; pass it explicitly. A `-c` naming a file that does not exist is an [error](#degraded-mode). |
|
|
49
49
|
| `--host <addr>` | `localhost` | Bind address. |
|
|
50
50
|
| `--port <n>` | `7117` | TCP port to bind. |
|
|
51
51
|
| `--persist [<path>]` | off | Keep the SQLite index on disk between runs so a restart only re-parses files that actually changed. Bare `--persist` caches at `<root>/.dirsql/cache.db`; `--persist <path>` caches at `<path>`. Off by default (the index is ephemeral). Also available on [`dirsql query`](#dirsql-query), passed after the subcommand. See [Keep the index across restarts](../howto/persist.md). |
|
|
@@ -61,25 +61,26 @@ requests, closes open `/events` streams, and exits.
|
|
|
61
61
|
**30-second** timeout each, overridable with the config key
|
|
62
62
|
[`[dirsql].hook-timeout`](./config.md#dirsql-keys).
|
|
63
63
|
|
|
64
|
-
###
|
|
64
|
+
### Configless mode
|
|
65
65
|
|
|
66
|
-
With no `-c/--config`, the server
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
it). The default `files` table:
|
|
66
|
+
With no `-c/--config`, the server indexes the invocation directory but
|
|
67
|
+
defines **no named tables**. Filesystem queries go through
|
|
68
|
+
[path-tables](./path-tables.md): a quoted path in place of a table name,
|
|
69
|
+
scanned live. A `./.dirsql.toml` sitting in the current directory is **not**
|
|
70
|
+
auto-loaded (pass it with `-c ./.dirsql.toml` to use it).
|
|
72
71
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
`size`, `mtime`, `ctime`.
|
|
72
|
+
`'./'` is the whole root — every file at any depth, one row per file, with
|
|
73
|
+
all seven [stat columns](./columns.md): `path`, `basename`, `dir`, `ext`,
|
|
74
|
+
`size`, `mtime`, `ctime`.
|
|
77
75
|
|
|
78
76
|
```bash
|
|
79
77
|
curl -s localhost:7117/query -H 'content-type: application/json' \
|
|
80
|
-
-d '{"sql":"SELECT basename, size FROM
|
|
78
|
+
-d '{"sql":"SELECT basename, size FROM \'./\' ORDER BY size DESC LIMIT 5"}'
|
|
81
79
|
```
|
|
82
80
|
|
|
81
|
+
Earlier versions served an implicit table named `files` here. It is gone; a
|
|
82
|
+
`SELECT ... FROM files` with no config now fails and points at `FROM './'`.
|
|
83
|
+
|
|
83
84
|
Passing a config with `-c` fully overrules this default. A `-c` naming a file
|
|
84
85
|
that does not exist is an error (not a fallback to the default); a config that
|
|
85
86
|
exists but fails to load degrades the server (see below).
|
|
@@ -111,8 +112,8 @@ non-zero exit with the diagnostic on stderr.
|
|
|
111
112
|
Run a SQL query from the shell:
|
|
112
113
|
|
|
113
114
|
```bash
|
|
114
|
-
# No -c: the
|
|
115
|
-
dirsql query "SELECT basename, size FROM
|
|
115
|
+
# No -c: query the filesystem with a path-table.
|
|
116
|
+
dirsql query "SELECT basename, size FROM './' ORDER BY size DESC LIMIT 5"
|
|
116
117
|
# [{"basename":"model.bin","size":104857600}, …]
|
|
117
118
|
|
|
118
119
|
# A config table (`posts`) needs its config passed explicitly, AFTER the subcommand.
|
|
@@ -135,7 +136,7 @@ response body), and exits `0`.
|
|
|
135
136
|
uses**, so behavior is identical to `POST /query` by construction:
|
|
136
137
|
|
|
137
138
|
- **Config discovery** honors `--config` passed after the subcommand (with none
|
|
138
|
-
given,
|
|
139
|
+
given, [no named tables](#configless-mode)), and `--extension` overrides,
|
|
139
140
|
exactly as server mode does.
|
|
140
141
|
- **`--persist [<path>]`** is honored, so a repeated `dirsql query` reuses the
|
|
141
142
|
on-disk cache. Because its value is optional, place a bare `--persist` after
|
|
@@ -163,8 +164,8 @@ stderr, with exit code `1`.
|
|
|
163
164
|
|
|
164
165
|
## `dirsql init`
|
|
165
166
|
|
|
166
|
-
Writes a starter `.dirsql.toml`
|
|
167
|
-
|
|
167
|
+
Writes a starter `.dirsql.toml` defining a catch-all `files` table, as a
|
|
168
|
+
scaffold to edit:
|
|
168
169
|
|
|
169
170
|
```bash
|
|
170
171
|
dirsql init
|
|
@@ -205,7 +206,7 @@ installed in the same environment as `dirsql` (`pip install …`, or
|
|
|
205
206
|
loads its fragment — its tables are queryable with zero config edits.
|
|
206
207
|
Installed = active: there is no enable step and no naming convention. The
|
|
207
208
|
fragment is composed *after* your own `-c` configs (so your config takes
|
|
208
|
-
precedence in ordering), and the
|
|
209
|
+
precedence in ordering), and the shipped starter `files` table is preserved.
|
|
209
210
|
|
|
210
211
|
Discovery is **launcher-only** — the standalone `cargo`-installed binary does no
|
|
211
212
|
discovery, and the SDKs never auto-discover (pass a plugin's config explicitly
|
package/docs/reference/config.md
CHANGED
|
@@ -8,7 +8,7 @@ all-defaults one. Unknown keys are a parse error at every level (top level,
|
|
|
8
8
|
fails loudly, naming the offending key, rather than silently no-opping.
|
|
9
9
|
|
|
10
10
|
The [CLI](./cli.md) loads a config only when you pass it with `-c/--config`;
|
|
11
|
-
with none given
|
|
11
|
+
with none given [no named tables](./cli.md#configless-mode) are defined (a
|
|
12
12
|
`./.dirsql.toml` on disk is **not** auto-loaded). The [SDKs](./sdk.md) load a
|
|
13
13
|
config via the `config` constructor parameter.
|
|
14
14
|
|
|
@@ -110,8 +110,8 @@ callback.
|
|
|
110
110
|
|
|
111
111
|
```toml
|
|
112
112
|
[[table]]
|
|
113
|
-
ddl = "CREATE TABLE comments (
|
|
114
|
-
glob = "_comments
|
|
113
|
+
ddl = "CREATE TABLE comments (path TEXT, basename TEXT, mtime INTEGER)"
|
|
114
|
+
glob = "_comments/*/*.jsonl"
|
|
115
115
|
|
|
116
116
|
[[table]]
|
|
117
117
|
ddl = "CREATE TABLE papers (paper_id TEXT, title TEXT)"
|
|
@@ -161,8 +161,8 @@ The configs load and merge in **argv order**:
|
|
|
161
161
|
error**, naming the table.
|
|
162
162
|
|
|
163
163
|
The index [root](./cli.md#flags) is the invocation directory regardless of where
|
|
164
|
-
any config lives. With no `-c`,
|
|
165
|
-
|
|
164
|
+
any config lives. With no `-c`, [no named tables](./cli.md#configless-mode) are
|
|
165
|
+
defined (no `./.dirsql.toml` auto-discovery); a single `-c` behaves exactly as
|
|
166
166
|
before.
|
|
167
167
|
|
|
168
168
|
## Parse errors
|
|
@@ -193,8 +193,8 @@ path = "sqlite_vec" # Python module name; on Node use the
|
|
|
193
193
|
entrypoint = "sqlite3_vec_init"
|
|
194
194
|
|
|
195
195
|
[[table]]
|
|
196
|
-
ddl = "CREATE TABLE comments (
|
|
197
|
-
glob = "_comments
|
|
196
|
+
ddl = "CREATE TABLE comments (path TEXT, basename TEXT, mtime INTEGER)"
|
|
197
|
+
glob = "_comments/*/*.jsonl"
|
|
198
198
|
|
|
199
199
|
[[table]]
|
|
200
200
|
ddl = "CREATE TABLE documents (path TEXT, basename TEXT, size INTEGER)"
|
package/docs/reference/sdk.md
CHANGED
|
@@ -97,10 +97,12 @@ Creates a SQLite index over a directory. The index root is the explicit
|
|
|
97
97
|
`root` when given, else the **process cwd** — the `config` file's location
|
|
98
98
|
never sets the root.
|
|
99
99
|
|
|
100
|
-
Constructing with **neither a `config` nor programmatic `tables`**
|
|
101
|
-
**
|
|
102
|
-
|
|
103
|
-
|
|
100
|
+
Constructing with **neither a `config` nor programmatic `tables`** defines
|
|
101
|
+
**no named tables** — the same as the CLI with no
|
|
102
|
+
[`-c`](./cli.md#configless-mode). Query the filesystem with a
|
|
103
|
+
[path-table](./path-tables.md) (`SELECT * FROM './'`); a `SELECT ... FROM
|
|
104
|
+
files` in that state fails with a hint pointing at that form. There is no
|
|
105
|
+
implicit `<root>/.dirsql.toml` discovery: to read a
|
|
104
106
|
config on disk, pass its path via `config` (Rust: `.config(path)` /
|
|
105
107
|
`DirSQL::from_config_path(path)`). The root-joining `DirSQL::from_config(root)`
|
|
106
108
|
shortcut was removed in #603 — use
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dirsql",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.118",
|
|
4
4
|
"description": "Ephemeral SQL index over a local directory",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": "https://github.com/thekevinscott/dirsql",
|
|
@@ -212,15 +212,15 @@
|
|
|
212
212
|
]
|
|
213
213
|
},
|
|
214
214
|
"optionalDependencies": {
|
|
215
|
-
"@dirsql/lib-linux-x64-gnu": "0.3.
|
|
216
|
-
"@dirsql/lib-linux-arm64-gnu": "0.3.
|
|
217
|
-
"@dirsql/lib-darwin-x64": "0.3.
|
|
218
|
-
"@dirsql/lib-darwin-arm64": "0.3.
|
|
219
|
-
"@dirsql/lib-win32-x64-msvc": "0.3.
|
|
220
|
-
"@dirsql/cli-linux-x64-gnu": "0.3.
|
|
221
|
-
"@dirsql/cli-linux-arm64-gnu": "0.3.
|
|
222
|
-
"@dirsql/cli-darwin-x64": "0.3.
|
|
223
|
-
"@dirsql/cli-darwin-arm64": "0.3.
|
|
224
|
-
"@dirsql/cli-win32-x64-msvc": "0.3.
|
|
215
|
+
"@dirsql/lib-linux-x64-gnu": "0.3.118",
|
|
216
|
+
"@dirsql/lib-linux-arm64-gnu": "0.3.118",
|
|
217
|
+
"@dirsql/lib-darwin-x64": "0.3.118",
|
|
218
|
+
"@dirsql/lib-darwin-arm64": "0.3.118",
|
|
219
|
+
"@dirsql/lib-win32-x64-msvc": "0.3.118",
|
|
220
|
+
"@dirsql/cli-linux-x64-gnu": "0.3.118",
|
|
221
|
+
"@dirsql/cli-linux-arm64-gnu": "0.3.118",
|
|
222
|
+
"@dirsql/cli-darwin-x64": "0.3.118",
|
|
223
|
+
"@dirsql/cli-darwin-arm64": "0.3.118",
|
|
224
|
+
"@dirsql/cli-win32-x64-msvc": "0.3.118"
|
|
225
225
|
}
|
|
226
226
|
}
|