dirsql 0.4.100 → 0.4.102
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/dirsql.d.ts +1 -1
- package/docs/howto/persist.md +11 -16
- package/docs/howto/query-without-config.md +3 -2
- package/docs/howto/search-indexes.md +1 -1
- package/docs/howto/skip-files.md +0 -3
- package/docs/reference/cli.md +2 -2
- package/docs/reference/config.md +0 -6
- package/docs/reference/glob.md +2 -3
- package/docs/reference/path-tables.md +9 -27
- package/docs/reference/sdk.md +4 -4
- package/package.json +6 -6
package/dist/dirsql.d.ts
CHANGED
|
@@ -43,7 +43,7 @@ export interface DirSQLOptions {
|
|
|
43
43
|
config?: string | string[];
|
|
44
44
|
/**
|
|
45
45
|
* Enable persistent on-disk SQLite cache. When `true`, the database is
|
|
46
|
-
* written to
|
|
46
|
+
* written to the platform cache directory (override via `persistPath`) so
|
|
47
47
|
* subsequent startups only re-parse files that have actually changed.
|
|
48
48
|
*/
|
|
49
49
|
persist?: boolean;
|
package/docs/howto/persist.md
CHANGED
|
@@ -17,9 +17,11 @@ why it is a CLI flag, not a config key.
|
|
|
17
17
|
dirsql --persist
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
That's the whole change. On the next run the cache is written to
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
That's the whole change. On the next run the cache is written to your platform cache directory
|
|
21
|
+
(`$XDG_CACHE_HOME/dirsql/<root hash>/cache.db` on Linux, `~/Library/Caches`
|
|
22
|
+
on macOS, `%LOCALAPPDATA%` on Windows), outside the scanned tree; runs after
|
|
23
|
+
that start from it. To put the cache elsewhere (a CI cache dir, a tmpfs), pass
|
|
24
|
+
a path:
|
|
23
25
|
|
|
24
26
|
```bash
|
|
25
27
|
dirsql --persist /var/cache/dirsql.db
|
|
@@ -30,20 +32,13 @@ put a bare `--persist` after the SQL there so it does not consume the query
|
|
|
30
32
|
argument. Embedding `dirsql`? The SDK constructors expose the same switch —
|
|
31
33
|
see [_Embedding `dirsql`?_](#embedding-dirsql) below.
|
|
32
34
|
|
|
33
|
-
## 2.
|
|
35
|
+
## 2. Nothing to ignore in git
|
|
34
36
|
|
|
35
|
-
The cache
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
.
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
The top-level `.dirsql/` directory is reserved for `dirsql`'s metadata and
|
|
43
|
-
is never scanned as data, so the cache can't index itself
|
|
44
|
-
([config reference](../reference/config.md#dirsql-keys)). While running,
|
|
45
|
-
`dirsql` also creates transient `cache.db-wal` and `cache.db-shm` sidecar
|
|
46
|
-
files next to the cache; the `.dirsql/` ignore already covers all three.
|
|
37
|
+
The default cache lives outside the scanned tree, so there is nothing to add
|
|
38
|
+
to `.gitignore` and the cache can't index itself. With `--persist <path>`, keep
|
|
39
|
+
that path out of the scanned tree too, or ignore it. While running, `dirsql`
|
|
40
|
+
also creates transient `cache.db-wal` and `cache.db-shm` sidecar files next to
|
|
41
|
+
the cache.
|
|
47
42
|
|
|
48
43
|
## What survives, what rebuilds
|
|
49
44
|
|
|
@@ -67,5 +67,6 @@ why indexes belong to only one of them, are in
|
|
|
67
67
|
- Paths outside the root resolve too — `'/var/log/*.log'`, `'../notes'`,
|
|
68
68
|
`'~/notes/*.md'` — reporting absolute paths
|
|
69
69
|
([path-tables reference](../reference/path-tables.md#paths-outside-the-index-root)).
|
|
70
|
-
-
|
|
71
|
-
not drown in machinery
|
|
70
|
+
- `.git/` is skipped by default, and a git repo's `.gitignore` hides what it
|
|
71
|
+
lists, so a bare `'./**'` does not drown in machinery
|
|
72
|
+
([skip rules](../reference/path-tables.md#skip-rules)).
|
|
@@ -224,7 +224,7 @@ Rows are embedded once, at ingest, and cached on disk by content and model
|
|
|
224
224
|
::: warning Editing `ddl` wedges a persisted `vec0` cache
|
|
225
225
|
Under `--persist`, editing any part of a `ddl` batch whose cache holds a
|
|
226
226
|
`vec0` table currently fails with `SQLite error: no such module: vec0`, and
|
|
227
|
-
keeps failing until the cache is deleted (
|
|
227
|
+
keeps failing until the cache is deleted (delete it, or pass a fresh `--persist <path>`). Tracked in
|
|
228
228
|
[#1008](https://github.com/thekevinscott/dirsql/issues/1008); FTS5 is
|
|
229
229
|
unaffected.
|
|
230
230
|
:::
|
package/docs/howto/skip-files.md
CHANGED
|
@@ -51,9 +51,6 @@ dirsql query "SELECT basename FROM notes ORDER BY basename" -c ./.dirsql.toml
|
|
|
51
51
|
list, though its path-tables still apply the built-in
|
|
52
52
|
[skip rules](../reference/path-tables.md#skip-rules) and respect
|
|
53
53
|
`.gitignore`.
|
|
54
|
-
- The top-level `.dirsql/` directory is always excluded, ignore list or
|
|
55
|
-
not — it is reserved for `dirsql`'s own metadata
|
|
56
|
-
([config reference](../reference/config.md#dirsql-keys)).
|
|
57
54
|
- Narrow table globs are the other half of the story: a file matching no
|
|
58
55
|
table's glob contributes no rows either. Use `ignore` for things that
|
|
59
56
|
should never be looked at; use precise globs to shape what each table
|
package/docs/reference/cli.md
CHANGED
|
@@ -248,8 +248,8 @@ Config flags are subcommand-local: pass them after `server`
|
|
|
248
248
|
| `--host <addr>` | `localhost` | Bind address. |
|
|
249
249
|
| `--port <n>` | `7117` | TCP port to bind. |
|
|
250
250
|
| `--cors-origin <origin>` | off | Let browser pages on `<origin>` (an exact origin such as `http://localhost:3202`, or `*` for any) call `/query` and open `/events`. See [Cross-origin requests](#cross-origin-requests). |
|
|
251
|
-
| `--persist [<path>]` | off | Keep the SQLite index on disk between runs so a restart only re-parses files that actually changed. Bare `--persist` caches
|
|
252
|
-
| `--no-ignore` | off | Scan files a `.gitignore` would hide. [Path-tables](./path-tables.md#skip-rules) respect `.gitignore` files by default; this flag restores the full walk. The built-in
|
|
251
|
+
| `--persist [<path>]` | off | Keep the SQLite index on disk between runs so a restart only re-parses files that actually changed. Bare `--persist` caches under the platform cache directory (`$XDG_CACHE_HOME/dirsql/<root hash>/cache.db` on Linux), outside the root; `--persist <path>` caches at `<path>`. Off by default (the index is ephemeral). Also available on [`dirsql query`](#dirsql-query). See [Keep the index across restarts](../howto/persist.md). |
|
|
252
|
+
| `--no-ignore` | off | Scan files a `.gitignore` would hide. [Path-tables](./path-tables.md#skip-rules) respect `.gitignore` files by default; this flag restores the full walk. The built-in `.git` skip and configured `ignore` patterns still apply. Also available on [`dirsql query`](#dirsql-query). |
|
|
253
253
|
| `--extension <path>` | none | Load a SQLite extension by literal path, overriding the config's `[[dirsql.extension]]` entries. Repeatable. Format: `<path>` or `<path>::<entrypoint>`. Internal plumbing for the pip/npm launchers, which resolve package-name extensions and pass the resolved paths here — not intended for direct use. When any `--extension` is present, the config file's own extension entries are not loaded. |
|
|
254
254
|
| `--version` | | Print the version and exit. |
|
|
255
255
|
| `--help` | | Print usage and exit. |
|
package/docs/reference/config.md
CHANGED
|
@@ -34,12 +34,6 @@ A config that still declares the removed `hook-timeout` key fails to load with
|
|
|
34
34
|
an error naming that replacement. [`[[dirsql.function]]`](#dirsql-function)
|
|
35
35
|
worker calls have their own per-call `timeout` key (default 30 seconds).
|
|
36
36
|
|
|
37
|
-
The top-level `.dirsql/` directory under the root is always excluded from
|
|
38
|
-
scanning, whether or not it appears in `ignore` — it is reserved for
|
|
39
|
-
`dirsql`'s own metadata (the persist cache lives there by default). Only the
|
|
40
|
-
top-level `.dirsql/` is reserved; a nested `sub/.dirsql/` is an ordinary
|
|
41
|
-
directory.
|
|
42
|
-
|
|
43
37
|
```toml
|
|
44
38
|
[dirsql]
|
|
45
39
|
ignore = ["node_modules/**", ".git/**"]
|
package/docs/reference/glob.md
CHANGED
|
@@ -70,7 +70,7 @@ A test runs every row against dirsql as a path-table and, with the leading
|
|
|
70
70
|
| --- | --- | --- |
|
|
71
71
|
| `./*.md` | `1.md`, `2.md`, `3.md`, `4.md`, `[a].md`, `[z.md`, `a.md`, `b.md`, `c.md`, `link.md`, `my file.md`, `top.md`, `{q}.md` | |
|
|
72
72
|
| `./**/*.md` | `1.md`, `2.md`, `3.md`, `4.md`, `[a].md`, `[z.md`, `a.md`, `b.md`, `c.md`, `docs/api.md`, `docs/guide.md`, `docs/nested/deep.md`, `link.md`, `linkdir/r.md`, `my file.md`, `real/r.md`, `top.md`, `{q}.md` | |
|
|
73
|
-
| `./**` | `1.md`, `2.md`, `3.md`, `4.md`, `[a].md`, `[z.md`, `a.md`, `b.md`, `c.md`, `docs/api.md`, `docs/guide.md`, `docs/nested/deep.md`, `link.md`, `my file.md`, `real/r.md`, `top.md`, `x.txt`, `{q}.md` | |
|
|
73
|
+
| `./**` | `1.md`, `2.md`, `3.md`, `4.md`, `[a].md`, `[z.md`, `a.md`, `b.md`, `c.md`, `docs/api.md`, `docs/guide.md`, `docs/nested/deep.md`, `link.md`, `my file.md`, `node_modules/pkg/index.js`, `real/r.md`, `top.md`, `x.txt`, `{q}.md` | |
|
|
74
74
|
| `./?.md` | `1.md`, `2.md`, `3.md`, `4.md`, `a.md`, `b.md`, `c.md` | |
|
|
75
75
|
| `./[ab].md` | `a.md`, `b.md` | |
|
|
76
76
|
| `./[!a].md` | `1.md`, `2.md`, `3.md`, `4.md`, `b.md`, `c.md` | |
|
|
@@ -96,7 +96,7 @@ A test runs every row against dirsql as a path-table and, with the leading
|
|
|
96
96
|
| `./*/` | `docs/api.md`, `docs/guide.md`, `linkdir/r.md`, `real/r.md` | directory-name |
|
|
97
97
|
| `./d*` | none | files-only |
|
|
98
98
|
| `./*.log` | none | gitignore |
|
|
99
|
-
| `./**/*.js` |
|
|
99
|
+
| `./**/*.js` | `node_modules/pkg/index.js` | |
|
|
100
100
|
| `./node_modules/*/index.js` | `node_modules/pkg/index.js` | |
|
|
101
101
|
| `./docs/**/*.md` | `docs/api.md`, `docs/guide.md`, `docs/nested/deep.md` | |
|
|
102
102
|
| `./docs/*.md` | `docs/api.md`, `docs/guide.md` | |
|
|
@@ -111,7 +111,6 @@ A test runs every row against dirsql as a path-table and, with the leading
|
|
|
111
111
|
| `gitignore` | Inside a git repo, `.gitignore` files hide matching files and directories. `--no-ignore` turns it off. | A query over a repo should not drown in build output. |
|
|
112
112
|
| `directory-name` | A pattern's last component that names a directory lists the files directly inside it, like `ls docs`. A trailing `/` is `*` appended. | A table is rows of files, and `ls` is the shell habit. |
|
|
113
113
|
| `files-only` | A pattern that matches a directory but no file returns nothing. | Every row is a file. |
|
|
114
|
-
| `node-modules` | A `node_modules` directory is skipped unless the pattern spells its name. | A zero-config `./**` should not walk dependencies. |
|
|
115
114
|
| `absolute` | `/`, `~/` and `../` patterns report absolute `path` values, with `..` folded out textually. | The scanned root is derived from the pattern, so there is no relative base. |
|
|
116
115
|
|
|
117
116
|
`.git` needs no rule of its own: the dotfile rule hides it.
|
|
@@ -198,8 +198,8 @@ over a very large table. Its output *is* the table:
|
|
|
198
198
|
```
|
|
199
199
|
dirsql_parsed: parser produced no rows for `posts/*.md`; cannot infer a schema
|
|
200
200
|
```
|
|
201
|
-
- **The skip rules still apply.** A parsed scan honors the same `
|
|
202
|
-
|
|
201
|
+
- **The skip rules still apply.** A parsed scan honors the same `.git`
|
|
202
|
+
/`ignore` rules a stat scan does (see below).
|
|
203
203
|
- **The parser runs on every start.** A parsed path-table is not kept in the
|
|
204
204
|
[persistent cache](/howto/persist); declared tables are.
|
|
205
205
|
|
|
@@ -228,8 +228,7 @@ The table itself is per-connection: it lives in `temp`, so it cannot leak into
|
|
|
228
228
|
`sqlite_master` or survive a restart. Under `--persist` a *parsed* table's rows
|
|
229
229
|
outlive the connection in the cache (above), but the table is still minted
|
|
230
230
|
fresh each run and the scan still decides what exists. A `.dirsql/` directory
|
|
231
|
-
|
|
232
|
-
everywhere else.
|
|
231
|
+
is an ordinary dot directory.
|
|
233
232
|
|
|
234
233
|
### When to promote to a declared table
|
|
235
234
|
|
|
@@ -251,14 +250,9 @@ choice is stated as one trade in
|
|
|
251
250
|
A path-table scan applies the same [`ignore`](/reference/config) patterns your
|
|
252
251
|
declared tables use — matched against root-relative paths under the same
|
|
253
252
|
[glob rule](/reference/config#glob-rule) as the path itself, `*` one level and
|
|
254
|
-
`**` any depth — plus
|
|
255
|
-
`
|
|
256
|
-
|
|
257
|
-
- `**/node_modules/**`
|
|
258
|
-
- `**/.git/**`
|
|
259
|
-
|
|
260
|
-
Both apply at any depth, so a `node_modules` nested inside a subdirectory is
|
|
261
|
-
skipped just like one at the top.
|
|
253
|
+
`**` any depth — plus one built-in default, `**/.git/**`, which applies at any depth.
|
|
254
|
+
`node_modules` is an ordinary directory: a `.gitignore` that lists it hides it,
|
|
255
|
+
as it hides any other.
|
|
262
256
|
|
|
263
257
|
### `.gitignore`
|
|
264
258
|
|
|
@@ -266,9 +260,7 @@ Path-table scans also respect `.gitignore` files by default, inside a git
|
|
|
266
260
|
repo, the way git, fd and ripgrep do: a `.gitignore` applies below its own
|
|
267
261
|
directory, deeper files override shallower ones, `!pattern` re-includes, and
|
|
268
262
|
an ignored directory is pruned rather than walked. In a typical repo this
|
|
269
|
-
excludes build output, virtualenvs, and caches with zero ceremony.
|
|
270
|
-
built-in defaults above remain as a floor for directories with no
|
|
271
|
-
`.gitignore` at all.
|
|
263
|
+
excludes build output, virtualenvs, and caches with zero ceremony.
|
|
272
264
|
|
|
273
265
|
A `.gitignore` is in force only when a directory holding `.git` encloses it.
|
|
274
266
|
Outside a repo none applies, at the scan's start, above it or below it: a
|
|
@@ -280,7 +272,7 @@ applies, including those above the directory a scan starts in, so
|
|
|
280
272
|
Pass [`--no-ignore`](./cli.md#flags) to restore the full walk — the
|
|
281
273
|
determinism switch for scripted use, since results otherwise depend on
|
|
282
274
|
`.gitignore` state. It disables only the `.gitignore` respect; the built-in
|
|
283
|
-
|
|
275
|
+
default and configured `ignore` patterns still apply.
|
|
284
276
|
|
|
285
277
|
### Naming a skipped directory
|
|
286
278
|
|
|
@@ -288,23 +280,13 @@ Skip rules are judged from the directory the scan starts in, so pointing at a
|
|
|
288
280
|
skipped directory — built-in or gitignored — still scans it:
|
|
289
281
|
|
|
290
282
|
```sql
|
|
291
|
-
SELECT path FROM '
|
|
292
|
-
SELECT path FROM './node_modules/*/package.json'; -- scans it anyway
|
|
283
|
+
SELECT path FROM './.git/*'; -- scans .git anyway
|
|
293
284
|
SELECT path FROM './dist'; -- scans dist/ even when gitignored
|
|
294
285
|
```
|
|
295
286
|
|
|
296
287
|
The `.gitignore` files in force there, including those above it, still
|
|
297
288
|
filter what lies beneath it.
|
|
298
289
|
|
|
299
|
-
A `node_modules` component after a glob names it too, at any depth:
|
|
300
|
-
|
|
301
|
-
```sql
|
|
302
|
-
SELECT path FROM './**/node_modules/*/package.json'; -- every node_modules
|
|
303
|
-
SELECT path FROM './*/node_modules/**'; -- one level down
|
|
304
|
-
```
|
|
305
|
-
|
|
306
|
-
Only the literal name counts; `'./**/*.js'` still skips every `node_modules`.
|
|
307
|
-
|
|
308
290
|
### Hidden files
|
|
309
291
|
|
|
310
292
|
A path component starting with `.` is skipped unless the path spells it, the
|
package/docs/reference/sdk.md
CHANGED
|
@@ -86,7 +86,7 @@ DirSQL::builder()
|
|
|
86
86
|
.config(config_toml_path) // optional; repeatable — call again to
|
|
87
87
|
// merge another config in call order
|
|
88
88
|
.persist(cache_path) // optional; Some(path), or None for the
|
|
89
|
-
//
|
|
89
|
+
// platform cache dir
|
|
90
90
|
.extensions(extensions) // optional; append one with .extension(e)
|
|
91
91
|
.poll_interval(duration) // optional; watch-loop cadence, default 200ms
|
|
92
92
|
.build() // -> Result<DirSQL> (synchronous scan)
|
|
@@ -123,7 +123,7 @@ shortcut was removed in #603 — use
|
|
|
123
123
|
matched files are skipped entirely (scan and watch).
|
|
124
124
|
- `no_ignore` / `noIgnore` — Scan files a `.gitignore` would hide.
|
|
125
125
|
[Path-tables](./path-tables.md#skip-rules) respect `.gitignore` files by
|
|
126
|
-
default; the built-in
|
|
126
|
+
default; the built-in `.git` skip and any `ignore` patterns
|
|
127
127
|
still apply. The CLI's [`--no-ignore`](./cli.md#flags).
|
|
128
128
|
- `config` — Path to a [`.dirsql.toml`](./config.md). Its `[[table]]`
|
|
129
129
|
entries are appended after any programmatic `tables`; its `ignore`
|
|
@@ -139,8 +139,8 @@ shortcut was removed in #603 — use
|
|
|
139
139
|
accumulate; a duplicate table name across configs errors), mirroring the
|
|
140
140
|
CLI's repeatable [`-c/--config`](./cli.md#flags).
|
|
141
141
|
- `persist` — Keep the SQLite index on disk between runs (default off:
|
|
142
|
-
ephemeral, rebuilt every startup). The cache lives
|
|
143
|
-
|
|
142
|
+
ephemeral, rebuilt every startup). The cache lives in the
|
|
143
|
+
platform cache directory by default (outside `<root>`); on restart, only files whose stat
|
|
144
144
|
changed are re-parsed. (The CLI exposes the same switch as the
|
|
145
145
|
[`--persist [PATH]`](./cli.md#dirsql-server) flag; it is not a config key.)
|
|
146
146
|
- `persist_path` / `persistPath` — Override the cache location. Ignored when
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dirsql",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.102",
|
|
4
4
|
"description": "Ephemeral SQL index over a local directory",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": "https://github.com/thekevinscott/dirsql",
|
|
@@ -221,10 +221,10 @@
|
|
|
221
221
|
]
|
|
222
222
|
},
|
|
223
223
|
"optionalDependencies": {
|
|
224
|
-
"@dirsql/lib-linux-x64-gnu": "0.4.
|
|
225
|
-
"@dirsql/lib-linux-arm64-gnu": "0.4.
|
|
226
|
-
"@dirsql/lib-darwin-x64": "0.4.
|
|
227
|
-
"@dirsql/lib-darwin-arm64": "0.4.
|
|
228
|
-
"@dirsql/lib-win32-x64-msvc": "0.4.
|
|
224
|
+
"@dirsql/lib-linux-x64-gnu": "0.4.102",
|
|
225
|
+
"@dirsql/lib-linux-arm64-gnu": "0.4.102",
|
|
226
|
+
"@dirsql/lib-darwin-x64": "0.4.102",
|
|
227
|
+
"@dirsql/lib-darwin-arm64": "0.4.102",
|
|
228
|
+
"@dirsql/lib-win32-x64-msvc": "0.4.102"
|
|
229
229
|
}
|
|
230
230
|
}
|