sensemaking 0.22.1 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +6 -8
  2. package/dist/cjs/chunk/extract.d.cts +3 -3
  3. package/dist/cjs/chunk/extract.d.ts +3 -3
  4. package/dist/cjs/chunk/extract.js +369 -72
  5. package/dist/cjs/chunk/extract.js.map +1 -1
  6. package/dist/cjs/chunk/group.js +2 -2
  7. package/dist/cjs/chunk/group.js.map +1 -1
  8. package/dist/cjs/chunk/parse.js +63 -67
  9. package/dist/cjs/chunk/parse.js.map +1 -1
  10. package/dist/cjs/chunk/parser.d.cts +2 -0
  11. package/dist/cjs/chunk/parser.d.ts +2 -0
  12. package/dist/cjs/chunk/parser.js +40 -0
  13. package/dist/cjs/chunk/parser.js.map +1 -0
  14. package/dist/cjs/chunk/types.d.cts +2 -2
  15. package/dist/cjs/chunk/types.d.ts +2 -2
  16. package/dist/cjs/chunk/version.d.cts +1 -1
  17. package/dist/cjs/chunk/version.d.ts +1 -1
  18. package/dist/cjs/chunk/version.js +1 -1
  19. package/dist/cjs/chunk/version.js.map +1 -1
  20. package/dist/cjs/features/links.js +67 -71
  21. package/dist/cjs/features/links.js.map +1 -1
  22. package/dist/cjs/features/sections.js +1 -1
  23. package/dist/cjs/features/sections.js.map +1 -1
  24. package/dist/cjs/scan/pool.js +1 -1
  25. package/dist/cjs/scan/pool.js.map +1 -1
  26. package/dist/cjs/store/builder.js +2 -3
  27. package/dist/cjs/store/builder.js.map +1 -1
  28. package/dist/cjs/store/duckdb/open.d.cts +1 -1
  29. package/dist/cjs/store/duckdb/open.d.ts +1 -1
  30. package/dist/cjs/store/duckdb/open.js +1 -1
  31. package/dist/cjs/store/duckdb/open.js.map +1 -1
  32. package/dist/cjs/store/sqlite/open.d.cts +1 -1
  33. package/dist/cjs/store/sqlite/open.d.ts +1 -1
  34. package/dist/cjs/store/sqlite/open.js +1 -1
  35. package/dist/cjs/store/sqlite/open.js.map +1 -1
  36. package/dist/cjs/store/turso/connection.d.cts +1 -0
  37. package/dist/cjs/store/turso/connection.d.ts +1 -0
  38. package/dist/cjs/store/turso/connection.js +134 -13
  39. package/dist/cjs/store/turso/connection.js.map +1 -1
  40. package/dist/cjs/store/turso/open.d.cts +1 -1
  41. package/dist/cjs/store/turso/open.d.ts +1 -1
  42. package/dist/cjs/store/turso/open.js +8 -2
  43. package/dist/cjs/store/turso/open.js.map +1 -1
  44. package/dist/cjs/store/turso/store.js +8 -1
  45. package/dist/cjs/store/turso/store.js.map +1 -1
  46. package/dist/cjs/text/strip.js +1 -3
  47. package/dist/cjs/text/strip.js.map +1 -1
  48. package/dist/cjs/workers/parse.js.map +1 -1
  49. package/dist/esm/chunk/extract.d.ts +3 -3
  50. package/dist/esm/chunk/extract.js +276 -63
  51. package/dist/esm/chunk/extract.js.map +1 -1
  52. package/dist/esm/chunk/group.js +2 -2
  53. package/dist/esm/chunk/group.js.map +1 -1
  54. package/dist/esm/chunk/parse.js +65 -64
  55. package/dist/esm/chunk/parse.js.map +1 -1
  56. package/dist/esm/chunk/parser.d.ts +2 -0
  57. package/dist/esm/chunk/parser.js +26 -0
  58. package/dist/esm/chunk/parser.js.map +1 -0
  59. package/dist/esm/chunk/types.d.ts +2 -2
  60. package/dist/esm/chunk/types.js.map +1 -1
  61. package/dist/esm/chunk/version.d.ts +1 -1
  62. package/dist/esm/chunk/version.js +1 -1
  63. package/dist/esm/chunk/version.js.map +1 -1
  64. package/dist/esm/features/links.js +59 -22
  65. package/dist/esm/features/links.js.map +1 -1
  66. package/dist/esm/features/sections.js +1 -1
  67. package/dist/esm/features/sections.js.map +1 -1
  68. package/dist/esm/scan/pool.js +1 -1
  69. package/dist/esm/scan/pool.js.map +1 -1
  70. package/dist/esm/store/builder.js +2 -3
  71. package/dist/esm/store/builder.js.map +1 -1
  72. package/dist/esm/store/duckdb/open.d.ts +1 -1
  73. package/dist/esm/store/duckdb/open.js +1 -1
  74. package/dist/esm/store/duckdb/open.js.map +1 -1
  75. package/dist/esm/store/sqlite/open.d.ts +1 -1
  76. package/dist/esm/store/sqlite/open.js +1 -1
  77. package/dist/esm/store/sqlite/open.js.map +1 -1
  78. package/dist/esm/store/turso/connection.d.ts +1 -0
  79. package/dist/esm/store/turso/connection.js +21 -8
  80. package/dist/esm/store/turso/connection.js.map +1 -1
  81. package/dist/esm/store/turso/open.d.ts +1 -1
  82. package/dist/esm/store/turso/open.js +3 -2
  83. package/dist/esm/store/turso/open.js.map +1 -1
  84. package/dist/esm/store/turso/store.js +2 -0
  85. package/dist/esm/store/turso/store.js.map +1 -1
  86. package/dist/esm/text/strip.js +1 -1
  87. package/dist/esm/text/strip.js.map +1 -1
  88. package/dist/esm/workers/parse.js.map +1 -1
  89. package/package.json +5 -19
  90. package/skills/sense-setup/SKILL.md +1 -1
@@ -10,7 +10,7 @@ Querying an existing tree is the `sense` skill. This one covers making a tree: i
10
10
  ## Setup
11
11
 
12
12
  - `npm install -g sensemaking`, then `sense init` at the tree root writes `sense.config.json`: two presets (`default`, and `large` showing what a big tree tunes) and an `embed` block naming the model. The model fetches once per machine at the first vector search (progress on stderr); `sense download` prefetches it instead where that timing matters (CI, air-gapped setup). Config discovery walks up from cwd; `--config <path>` overrides.
13
- - **Backing store.** The config's `store` key: `sqlite` (default, zero-dependency, Node's built-in SQLite), or the experimental `duckdb` and `turso`, whose engine package the first command that opens such a tree installs on its own (`@duckdb/node-api`, a one-time native download of about 110 MB; `@tursodatabase/database`, much smaller). The same commands and tables run on all three. Two things do not port, and each one decides a tree. **FTS5 syntax:** under `duckdb` and `turso`, `search` text and raw `MATCH` reject FTS5's prefix, boolean, `NEAR`, initial-token and column-filter operators with a named error, and sqlite's FTS5 SQL (`MATCH`, `snippet()`, `bm25()`) does not run, so saved queries written in that syntax are sqlite dialect; a tree whose saved queries or search vocabulary depend on FTS5 operators stays on `sqlite`. **SQL functions:** `has`/`basename`/`segment` are registered on `sqlite` and `duckdb` but not `turso`, whose client cannot register them at all, so a tree whose queries call them stays off `turso`. `sense watch` runs on all three; `duckdb` and `turso` lock their cache file per connection, so a concurrent command waits out the watcher's current cycle instead of failing. Each store keeps its own cache file (`.sense/cache.db`, `.sense/cache.duckdb`, `.sense/cache.turso.db`); switching stores is a rebuild, not a migration.
13
+ - **Backing store.** The config's `store` key: `sqlite` (default, zero-dependency, Node's built-in SQLite), or the experimental `duckdb` and `turso`, whose engine package the first command that opens such a tree installs on its own (`@duckdb/node-api`, a one-time native download of about 110 MB; `@tursodatabase/database`, much smaller). The same commands and tables run on all three. Two things do not port, and each one decides a tree. **FTS5 syntax:** under `duckdb` and `turso`, `search` text and raw `MATCH` reject FTS5's prefix, boolean, `NEAR`, initial-token and column-filter operators with a named error, and sqlite's FTS5 SQL (`MATCH`, `snippet()`, `bm25()`) does not run, so saved queries written in that syntax are sqlite dialect; a tree whose saved queries or search vocabulary depend on FTS5 operators stays on `sqlite`. **SQL functions:** `has`/`basename`/`segment` are registered on `sqlite` and `duckdb` but not `turso`, whose client cannot register them at all, so a tree whose queries call them stays off `turso`. `sense watch` runs on all three; `duckdb` and `turso` lock their cache file per connection, so a concurrent command waits out the watcher's current cycle instead of failing. Each store keeps its own cache file (`.sense/cache.db`, `.sense/cache.duckdb`, `.sense/cache.turso.db`); switching stores is a rebuild, not a migration. Indexing speed is the other axis, and it does not follow from any of the above: sqlite builds a cold index fastest and turso slowest, by a wide margin on a large tree, and no setting closes that gap, since turso's engine costs more per write and more again to maintain each index. What turso buys instead is concurrent writers, non-blocking I/O and encryption, none of which a one-shot command uses, so reach for it for those rather than for speed. Per-store figures: BENCHMARKING.md in the sensemaking repo.
14
14
  - Globs resolve relative to the config file, never the cwd.
15
15
  - `sense status` and `sense map` show each preset's coverage (files matched, embedded count), so what a config actually indexes is always visible in output. A config edit that changes coverage rebuilds the cache and names the preset that caused it on stderr.
16
16