sensemaking 0.15.4 → 0.16.0

package/README.md

@@ -74,12 +74,12 @@ ORDER BY bm25(content, 10.0, 5.0, 1.0) LIMIT 10
 ```json
 {
   "$schema": "https://unpkg.com/sensemaking/schema.json",
-  "version": 4,
+  "version": 5,
   "presets": {
     "default": { "include": ["**/*.md"], "k": 10 },
     "raw":     { "include": ["raw/**/*.md"], "k": 5 }
   },
-  "embed": { "model": "minishlab/potion-retrieval-32M", "type": "static" },
+  "embed": { "model": "minishlab/potion-retrieval-32M", "provider": "static" },
   "queries": {
     "dead-links": { "sql": "SELECT src, target FROM links WHERE dst IS NULL" },
     "by-tag":     { "sql": "SELECT path, title FROM frontmatter WHERE has(tags, ?) ORDER BY path" },
@@ -90,7 +90,7 @@ ORDER BY bm25(content, 10.0, 5.0, 1.0) LIMIT 10
 
 | key | holds |
 |---|---|
-| `presets` | named bundles of `include`/`exclude` globs, `k` (result count), `semantic` (vectors for this scope, on unless `false`), `where` (a standing SQL filter). A file is indexed if any preset includes it, embedded if a model is named and some covering preset has `semantic` on; `status` shows each preset's coverage. |
+| `presets` | named bundles of `include`/`exclude` globs, `k` (result count), `signals` (which engines this scope searches with -- `words`, `links`, `vectors`; every signal whose prerequisites hold, unless the preset lists them exhaustively), `where` (a standing SQL filter). A file is indexed if any preset includes it, embedded if a model is named and some covering preset's `signals` include `vectors`; `status` shows each preset's coverage. |
 | `embed` | the model vectors are built with. Naming one gives the tree vectors; omitting the block means none at all, whatever the presets say. `sense download` fetches it. |
 | `content` | settings for the `content` table. `tokenize` names the FTS5 tokenizer, defaulting to `porter unicode61`: name `trigram` for substring matching inside a Latin word, or `unicode61 tokenchars '-_'` to keep hyphenated terms whole. Languages written without word spaces (Chinese, Japanese, Thai, Khmer, Lao, Burmese) need no decision here: they are indexed per grapheme and searched as ordered phrases, substring semantics, what grep gives, needing no minimum length; naming a tokenizer turns that off, since the tree has then chosen its own scheme. Changing it rebuilds the text index only; vectors, links, and sections are kept. |
 | `queries` | entries runnable as `sense <name>`, each naming the verb it runs: `{ sql }` for SQL (`?` binds positional args) or `{ search }` for a ranked search with its settings baked in, so `sense hot` needs no flags. Running an entry validates it: a typo'd column errors and exits nonzero, and a parameterised entry validates with any argument, since preparing precedes binding. |
@@ -98,10 +98,38 @@ ORDER BY bm25(content, 10.0, 5.0, 1.0) LIMIT 10
 
 Bare commands use the `default` preset; `--preset` names another; flags override single fields. Editing a preset rebuilds the cache and says which preset caused it.
 
-Vectors need a model, and nothing downloads it implicitly: `sense download` fetches it once per machine into `$XDG_CACHE_HOME/sensemaking/models` (or `~/.cache/...`), one directory per model, shared by every tree (124 MB, never in the package). `embed.model` is a Hugging Face id, or a path to a directory holding `model.safetensors` and `tokenizer.json`, which `sense download` leaves to you to populate. A preset that asks for vectors without the model is an error naming `sense download`, rather than a quieter result that would make the same search answer differently before and after; a `semantic: false` preset never asks, so it is unaffected. An optional top-level `"embed": { "model", "type", "url", "key" }` block points at any Model2Vec model, local path, or OpenAI-compatible endpoint (Ollama, LM Studio, hosted). Embedding the notes themselves happens on the first semantic search, with progress.
+Vectors need a model. Naming a Hugging Face id in `embed.model` is consent to fetch it: it downloads lazily, on the first vector search, with progress on stderr, into `~/.sense/models` (huggingface_hub's cache layout, one snapshot directory per resolved revision, shared by every tree, 124 MB, never in the package). `sense download` prefetches the same model ahead of time, so the wait happens on your schedule instead of the first query's; it is idempotent and prints the resolved revision. `embed.model` is a Hugging Face id, or a path to a directory holding `model.safetensors` and `tokenizer.json`, which nothing fetches for you. A preset that asks for vectors when a local model path is missing those files is an error naming the fix, rather than a quieter result that would make the same search answer differently before and after; a preset whose `signals` exclude `vectors` never asks, so it is unaffected. An optional top-level `"embed": { "model", "provider", "url", "key" }` block points at any Model2Vec model, local path, or OpenAI-compatible endpoint (Ollama, LM Studio, hosted). Embedding the notes themselves happens on the first vector search, with progress.
 
 Two custom SQL functions. `has(field, value)`: array membership on JSON-array fields, substring on strings, false on missing keys. `segment(terms)`: rewrites a run of Chinese, Japanese, Thai, Khmer, Lao, or Burmese text into the ordered grapheme phrase the sidecar columns need, leaving everything else unchanged, so it is safe to add to any hand-written `MATCH`. Frontmatter parsing is lenient: syntax errors are per-file warnings, and the values are still indexed, so one bad note never costs you the crawl.
 
+## Providers
+
+`embed.provider` picks the wire protocol; `embed.model` names the model. [INTEGRATIONS.md](INTEGRATIONS.md) is the record of what has actually been run and verified against this codebase -- only that gets named as a recommendation.
+
+**static** -- a local, pure-JS Model2Vec model; no network at query time.
+
+```json
+"embed": { "model": "minishlab/potion-retrieval-32M", "provider": "static" }
+```
+
+`model` is a Hugging Face id, fetched to `~/.sense/models` on first use, or a path to a local directory holding `model.safetensors` and `tokenizer.json`.
+
+**openai** -- any endpoint serving an OpenAI-shaped `POST /embeddings`.
+
+```json
+"embed": { "model": "nomic-embed-text", "provider": "openai", "url": "http://localhost:11434/v1" }
+```
+
+Ollama serves this shape at `http://localhost:11434/v1`. LM Studio serves the same shape at `http://localhost:1234/v1`, with narrower platform support than Ollama: Apple Silicon only on Mac, AVX2 on Windows. Content stays on the machine for either. Any other OpenAI-shaped endpoint, including a hosted one, works the same way through `provider: "openai"` and its own `url`; a hosted endpoint means tree content is sent to that service.
+
+**cohere** -- Cohere's native `/v2/embed`, which expresses the doc/query distinction (`input_type`) the OpenAI shape cannot.
+
+```json
+"embed": { "model": "embed-v4.0", "provider": "cohere", "key": "COHERE_API_KEY" }
+```
+
+`key` names the environment variable holding the API key; the key value itself never goes in the config. Content is sent to Cohere.
+
 ## Scale
 
 Every query starts with a freshness check against the cache in `.sense/`; only changed files are re-parsed. What to expect as a tree grows:
@@ -136,7 +164,7 @@ Two skills: `sense` for querying a tree (what each command is for, FTS5 syntax,
 - **RAG / vector stores:** similarity can't express `WHERE status = 'active'`. Here vectors are one signal inside `search`: same SQLite file, filters compose, every row labels its evidence (`via`), and a preset turns vectors off per layer of the tree. No second store, no daemon, no native builds.
 - **Document-OS apps (Anytype, Logseq, SilverBullet, Capacities):** full applications with their own UI and storage. `sense` is headless: your files stay files, there's no app to run.
 
-Dependencies: [yaml](https://github.com/eemeli/yaml), [remove-markdown](https://github.com/zuchka/remove-markdown), [@huggingface/tokenizers](https://github.com/huggingface/tokenizers.js) (pure JS), and Node's built-in SQLite. No native builds.
+Dependencies: [yaml](https://github.com/eemeli/yaml), [remove-markdown](https://github.com/zuchka/remove-markdown), [@huggingface/tokenizers](https://github.com/huggingface/tokenizers.js), [franc-min](https://github.com/wooorm/franc) (all pure JS), and Node's built-in SQLite. No native builds.
 
 ## License
 

package/dist/cjs/cli/download.js

@@ -8,9 +8,10 @@ Object.defineProperty(exports, "default", {
         return _default;
     }
 });
+var _nodepath = require("node:path");
 var _indexts = require("../config/index.js");
+var _storets = require("../embed/store.js");
 var _errorsts = require("../errors.js");
-var _embedts = require("../features/embed.js");
 var _indexts1 = require("./index.js");
 var _sharedts = require("./shared.js");
 function asyncGeneratorStep(gen, resolve, reject, _next, _throw, key, arg) {
@@ -164,12 +165,11 @@ function _ts_generator(thisArg, body) {
         };
     }
 }
-// The one command that fetches weights. Nothing else touches the network for them, so a
-// query that reads like a query never spends minutes pulling 124 MB. Asks nothing, so there
-// is no prompt for CI to bypass; running it twice is a no-op.
+// A prefetch over the store's own fetch path: a search would fetch the same weights
+// lazily, so this only moves the wait earlier. Idempotent; asks nothing, so CI can run it unattended.
 var download = function download(ctx) {
     return _async_to_generator(function() {
-        var values, cfg, e, dir;
+        var values, cfg, e, at, dir;
         return _ts_generator(this, function(_state) {
             switch(_state.label){
                 case 0:
@@ -177,27 +177,28 @@ var download = function download(ctx) {
                     cfg = ctx.resolveConfig(values.config);
                     e = (0, _indexts.embedConfig)(cfg);
                     if (!e) throw new _errorsts.SenseError('EMBED_DISABLED', 'no preset asks for vectors, so there is no model to download');
-                    if (e.type === 'api') {
-                        console.log('embed type is "api" ('.concat(e.url, "), so there is nothing to download; ").concat(ctx.name, " search reaches it over the network"));
+                    if (e.provider !== 'static') {
+                        console.log('embed provider is "'.concat(e.provider, '" (').concat(e.url, "), so there is nothing to download; ").concat(ctx.name, " search reaches it over the network"));
                         return [
                             2
                         ];
                     }
-                    if ((0, _embedts.modelPresent)(cfg)) {
-                        console.log("".concat(e.model, " is already available in ").concat((0, _embedts.modelDir)(e.model)));
+                    if ((0, _storets.modelPresent)(cfg)) {
+                        at = (0, _storets.isDownloadable)(e.model) ? "@".concat((0, _nodepath.basename)((0, _storets.modelDir)(e.model)), " ") : ' ';
+                        console.log("".concat(e.model).concat(at, "is already available in ").concat((0, _storets.modelDir)(e.model)));
                         return [
                             2
                         ];
                     }
                     return [
                         4,
-                        (0, _embedts.downloadModel)(e.model, function(file, into) {
+                        (0, _storets.downloadModel)(e.model, function(file, into) {
                             return console.error("fetching ".concat(e.model, "/").concat(file, " into ").concat(into));
                         })
                     ];
                 case 1:
                     dir = _state.sent();
-                    console.log("downloaded ".concat(e.model, " into ").concat(dir));
+                    console.log("downloaded ".concat(e.model, "@").concat((0, _nodepath.basename)(dir), " into ").concat(dir));
                     return [
                         2
                     ];

package/dist/cjs/cli/download.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/download.ts"],"sourcesContent":["import { embedConfig } from '../config/index.ts';\nimport { SenseError } from '../errors.ts';\nimport { downloadModel, modelDir, modelPresent } from '../features/embed.ts';\nimport { USAGE } from './index.ts';\nimport { CONFIG, parse } from './shared.ts';\nimport type { Command } from './types.ts';\n\n// The one command that fetches weights. Nothing else touches the network for them, so a\n// query that reads like a query never spends minutes pulling 124 MB. Asks nothing, so there\n// is no prompt for CI to bypass; running it twice is a no-op.\nconst download: Command = async (ctx) => {\n  const { values } = parse(ctx.argv, `usage: ${ctx.name} ${USAGE.download}`, { ...CONFIG });\n  const cfg = ctx.resolveConfig(values.config as string | undefined);\n  const e = embedConfig(cfg);\n  if (!e) throw new SenseError('EMBED_DISABLED', 'no preset asks for vectors, so there is no model to download');\n\n  if (e.type === 'api') {\n    console.log(`embed type is \"api\" (${e.url}), so there is nothing to download; ${ctx.name} search reaches it over the network`);\n    return;\n  }\n  if (modelPresent(cfg)) {\n    console.log(`${e.model} is already available in ${modelDir(e.model)}`);\n    return;\n  }\n  const dir = await downloadModel(e.model, (file, into) => console.error(`fetching ${e.model}/${file} into ${into}`));\n  console.log(`downloaded ${e.model} into ${dir}`);\n};\nexport default download;\n"],"names":["download","ctx","values","cfg","e","dir","parse","argv","USAGE","name","CONFIG","resolveConfig","config","embedConfig","SenseError","type","console","log","url","modelPresent","modelDir","model","downloadModel","file","into","error"],"mappings":";;;;+BA2BA;;;eAAA;;;uBA3B4B;wBACD;uBAC2B;wBAChC;wBACQ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAG9B,wFAAwF;AACxF,4FAA4F;AAC5F,8DAA8D;AAC9D,IAAMA,WAAoB,kBAAOC;;YACvBC,QACFC,KACAC,GAWAC;;;;oBAbEH,SAAWI,IAAAA,eAAK,EAACL,IAAIM,IAAI,EAAE,AAAC,UAAqBC,OAAZP,IAAIQ,IAAI,EAAC,KAAkB,OAAfD,eAAK,CAACR,QAAQ,GAAI,mBAAKU,gBAAM,GAA9ER;oBACFC,MAAMF,IAAIU,aAAa,CAACT,OAAOU,MAAM;oBACrCR,IAAIS,IAAAA,oBAAW,EAACV;oBACtB,IAAI,CAACC,GAAG,MAAM,IAAIU,oBAAU,CAAC,kBAAkB;oBAE/C,IAAIV,EAAEW,IAAI,KAAK,OAAO;wBACpBC,QAAQC,GAAG,CAAC,AAAC,wBAAmEhB,OAA5CG,EAAEc,GAAG,EAAC,wCAA+C,OAATjB,IAAIQ,IAAI,EAAC;wBACzF;;;oBACF;oBACA,IAAIU,IAAAA,qBAAY,EAAChB,MAAM;wBACrBa,QAAQC,GAAG,CAAC,AAAC,GAAqCG,OAAnChB,EAAEiB,KAAK,EAAC,6BAA6C,OAAlBD,IAAAA,iBAAQ,EAAChB,EAAEiB,KAAK;wBAClE;;;oBACF;oBACY;;wBAAMC,IAAAA,sBAAa,EAAClB,EAAEiB,KAAK,EAAE,SAACE,MAAMC;mCAASR,QAAQS,KAAK,CAAC,AAAC,YAAsBF,OAAXnB,EAAEiB,KAAK,EAAC,KAAgBG,OAAbD,MAAK,UAAa,OAALC;;;;oBAArGnB,MAAM;oBACZW,QAAQC,GAAG,CAAC,AAAC,cAA6BZ,OAAhBD,EAAEiB,KAAK,EAAC,UAAY,OAAJhB;;;;;;IAC5C;;IACA,WAAeL"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/download.ts"],"sourcesContent":["import { basename } from 'node:path';\nimport { embedConfig } from '../config/index.ts';\nimport { downloadModel, isDownloadable, modelDir, modelPresent } from '../embed/store.ts';\nimport { SenseError } from '../errors.ts';\nimport { USAGE } from './index.ts';\nimport { CONFIG, parse } from './shared.ts';\nimport type { Command } from './types.ts';\n\n// A prefetch over the store's own fetch path: a search would fetch the same weights\n// lazily, so this only moves the wait earlier. Idempotent; asks nothing, so CI can run it unattended.\nconst download: Command = async (ctx) => {\n  const { values } = parse(ctx.argv, `usage: ${ctx.name} ${USAGE.download}`, { ...CONFIG });\n  const cfg = ctx.resolveConfig(values.config as string | undefined);\n  const e = embedConfig(cfg);\n  if (!e) throw new SenseError('EMBED_DISABLED', 'no preset asks for vectors, so there is no model to download');\n\n  if (e.provider !== 'static') {\n    console.log(`embed provider is \"${e.provider}\" (${e.url}), so there is nothing to download; ${ctx.name} search reaches it over the network`);\n    return;\n  }\n  if (modelPresent(cfg)) {\n    const at = isDownloadable(e.model) ? `@${basename(modelDir(e.model))} ` : ' ';\n    console.log(`${e.model}${at}is already available in ${modelDir(e.model)}`);\n    return;\n  }\n  const dir = await downloadModel(e.model, (file, into) => console.error(`fetching ${e.model}/${file} into ${into}`));\n  console.log(`downloaded ${e.model}@${basename(dir)} into ${dir}`);\n};\nexport default download;\n"],"names":["download","ctx","values","cfg","e","at","dir","parse","argv","USAGE","name","CONFIG","resolveConfig","config","embedConfig","SenseError","provider","console","log","url","modelPresent","isDownloadable","model","basename","modelDir","downloadModel","file","into","error"],"mappings":";;;;+BA4BA;;;eAAA;;;wBA5ByB;uBACG;uBAC0C;wBAC3C;wBACL;wBACQ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAG9B,oFAAoF;AACpF,sGAAsG;AACtG,IAAMA,WAAoB,kBAAOC;;YACvBC,QACFC,KACAC,GAQEC,IAIFC;;;;oBAdEJ,SAAWK,IAAAA,eAAK,EAACN,IAAIO,IAAI,EAAE,AAAC,UAAqBC,OAAZR,IAAIS,IAAI,EAAC,KAAkB,OAAfD,eAAK,CAACT,QAAQ,GAAI,mBAAKW,gBAAM,GAA9ET;oBACFC,MAAMF,IAAIW,aAAa,CAACV,OAAOW,MAAM;oBACrCT,IAAIU,IAAAA,oBAAW,EAACX;oBACtB,IAAI,CAACC,GAAG,MAAM,IAAIW,oBAAU,CAAC,kBAAkB;oBAE/C,IAAIX,EAAEY,QAAQ,KAAK,UAAU;wBAC3BC,QAAQC,GAAG,CAAC,AAAC,sBAAqCd,OAAhBA,EAAEY,QAAQ,EAAC,OAAiDf,OAA5CG,EAAEe,GAAG,EAAC,wCAA+C,OAATlB,IAAIS,IAAI,EAAC;wBACvG;;;oBACF;oBACA,IAAIU,IAAAA,qBAAY,EAACjB,MAAM;wBACfE,KAAKgB,IAAAA,uBAAc,EAACjB,EAAEkB,KAAK,IAAI,AAAC,IAA+B,OAA5BC,IAAAA,kBAAQ,EAACC,IAAAA,iBAAQ,EAACpB,EAAEkB,KAAK,IAAG,OAAK;wBAC1EL,QAAQC,GAAG,CAAC,AAAC,GAAYb,OAAVD,EAAEkB,KAAK,EAAgCE,OAA7BnB,IAAG,4BAA4C,OAAlBmB,IAAAA,iBAAQ,EAACpB,EAAEkB,KAAK;wBACtE;;;oBACF;oBACY;;wBAAMG,IAAAA,sBAAa,EAACrB,EAAEkB,KAAK,EAAE,SAACI,MAAMC;mCAASV,QAAQW,KAAK,CAAC,AAAC,YAAsBF,OAAXtB,EAAEkB,KAAK,EAAC,KAAgBK,OAAbD,MAAK,UAAa,OAALC;;;;oBAArGrB,MAAM;oBACZW,QAAQC,GAAG,CAAC,AAAC,cAAwBK,OAAXnB,EAAEkB,KAAK,EAAC,KAAyBhB,OAAtBiB,IAAAA,kBAAQ,EAACjB,MAAK,UAAY,OAAJA;;;;;;IAC7D;;IACA,WAAeN"}

package/dist/cjs/cli/index.d.cts

@@ -1,6 +1,6 @@
 import type { Command } from './types.js';
 export declare const USAGE: {
-    readonly init: "init";
+    readonly init: "init [--model id-or-path] [--provider static|openai|cohere] [--url url]";
     readonly watch: "watch [--force] [--config path]";
     readonly status: "status [--format table|json] [--config path]";
     readonly download: "download [--config path]";

package/dist/cjs/cli/index.d.ts

@@ -1,6 +1,6 @@
 import type { Command } from './types.js';
 export declare const USAGE: {
-    readonly init: "init";
+    readonly init: "init [--model id-or-path] [--provider static|openai|cohere] [--url url]";
     readonly watch: "watch [--force] [--config path]";
     readonly status: "status [--format table|json] [--config path]";
     readonly download: "download [--config path]";

package/dist/cjs/cli/index.js

@@ -47,7 +47,7 @@ function _interop_require_wildcard(obj, nodeInterop) {
     return newObj;
 }
 var USAGE = {
-    init: 'init',
+    init: 'init [--model id-or-path] [--provider static|openai|cohere] [--url url]',
     watch: 'watch [--force] [--config path]',
     status: 'status [--format table|json] [--config path]',
     download: 'download [--config path]',

package/dist/cjs/cli/index.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/index.ts"],"sourcesContent":["import type { Command } from './types.ts';\n\n// Each command's usage line, minus the leading \"usage: <name> \" -- shared by cli.ts's\n// top-level usage() and the command's own parse() error/--help output, so there is one\n// copy of each line, not two that can drift.\nexport const USAGE = {\n  init: 'init',\n  watch: 'watch [--force] [--config path]',\n  status: 'status [--format table|json] [--config path]',\n  download: 'download [--config path]',\n  sql: 'sql \"<statement>\" [params...] [--preset name] [--format table|json|csv] [--config path]',\n  search: 'search \"<terms>\" [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n  map: 'map [--format table|json] [--config path]',\n  peek: 'peek <path> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--format table|json] [--config path]',\n  path: 'path <a> <b> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--max-depth n] [--format table|json] [--config path]',\n  related: 'related <note> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n} as const;\n\n// Lazy registry: a command's imports load only when it runs, so a heavy dependency in one\n// command never taxes the others (or --version/--help). These names are reserved: a saved\n// entry in config can never shadow them.\nexport const COMMANDS: Record<string, () => Promise<{ default: Command }>> = {\n  init: () => import('./init.ts'),\n  watch: () => import('./watch.ts'),\n  status: () => import('./status.ts'),\n  download: () => import('./download.ts'),\n  sql: () => import('./sql.ts'),\n  search: () => import('./search.ts'),\n  map: () => import('./map.ts'),\n  peek: () => import('./peek.ts'),\n  path: () => import('./path.ts'),\n  related: () => import('./related.ts'),\n};\n\nexport type { Command, Ctx } from './types.ts';\n"],"names":["COMMANDS","USAGE","init","watch","status","download","sql","search","map","peek","path","related"],"mappings":";;;;;;;;;;;QAqBaA;eAAAA;;QAhBAC;eAAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAN,IAAMA,QAAQ;IACnBC,MAAM;IACNC,OAAO;IACPC,QAAQ;IACRC,UAAU;IACVC,KAAK;IACLC,QAAQ;IACRC,KAAK;IACLC,MAAM;IACNC,MAAM;IACNC,SAAS;AACX;AAKO,IAAMX,WAAgE;IAC3EE,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,OAAO,SAAPA;eAAa;2DAAA,QAAO;;;IACpBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,UAAU,SAAVA;eAAgB;2DAAA,QAAO;;;IACvBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,SAAS,SAATA;eAAe;2DAAA,QAAO;;;AACxB"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/index.ts"],"sourcesContent":["import type { Command } from './types.ts';\n\n// Each command's usage line, minus the leading \"usage: <name> \" -- shared by cli.ts's\n// top-level usage() and the command's own parse() error/--help output, so there is one\n// copy of each line, not two that can drift.\nexport const USAGE = {\n  init: 'init [--model id-or-path] [--provider static|openai|cohere] [--url url]',\n  watch: 'watch [--force] [--config path]',\n  status: 'status [--format table|json] [--config path]',\n  download: 'download [--config path]',\n  sql: 'sql \"<statement>\" [params...] [--preset name] [--format table|json|csv] [--config path]',\n  search: 'search \"<terms>\" [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n  map: 'map [--format table|json] [--config path]',\n  peek: 'peek <path> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--format table|json] [--config path]',\n  path: 'path <a> <b> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--max-depth n] [--format table|json] [--config path]',\n  related: 'related <note> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n} as const;\n\n// Lazy registry: a command's imports load only when it runs, so a heavy dependency in one\n// command never taxes the others (or --version/--help). These names are reserved: a saved\n// entry in config can never shadow them.\nexport const COMMANDS: Record<string, () => Promise<{ default: Command }>> = {\n  init: () => import('./init.ts'),\n  watch: () => import('./watch.ts'),\n  status: () => import('./status.ts'),\n  download: () => import('./download.ts'),\n  sql: () => import('./sql.ts'),\n  search: () => import('./search.ts'),\n  map: () => import('./map.ts'),\n  peek: () => import('./peek.ts'),\n  path: () => import('./path.ts'),\n  related: () => import('./related.ts'),\n};\n\nexport type { Command, Ctx } from './types.ts';\n"],"names":["COMMANDS","USAGE","init","watch","status","download","sql","search","map","peek","path","related"],"mappings":";;;;;;;;;;;QAqBaA;eAAAA;;QAhBAC;eAAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAN,IAAMA,QAAQ;IACnBC,MAAM;IACNC,OAAO;IACPC,QAAQ;IACRC,UAAU;IACVC,KAAK;IACLC,QAAQ;IACRC,KAAK;IACLC,MAAM;IACNC,MAAM;IACNC,SAAS;AACX;AAKO,IAAMX,WAAgE;IAC3EE,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,OAAO,SAAPA;eAAa;2DAAA,QAAO;;;IACpBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,UAAU,SAAVA;eAAgB;2DAAA,QAAO;;;IACvBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,SAAS,SAATA;eAAe;2DAAA,QAAO;;;AACxB"}

package/dist/cjs/cli/init.js

@@ -12,13 +12,39 @@ var _indexts = require("../config/index.js");
 var _indexts1 = require("./index.js");
 var _sharedts = require("./shared.js");
 var init = function init(ctx) {
-    (0, _sharedts.parse)(ctx.argv, "usage: ".concat(ctx.name, " ").concat(_indexts1.USAGE.init), {});
-    var configPath = (0, _indexts.initConfig)(process.cwd());
+    var values = (0, _sharedts.parse)(ctx.argv, "usage: ".concat(ctx.name, " ").concat(_indexts1.USAGE.init), {
+        model: {
+            type: 'string'
+        },
+        provider: {
+            type: 'string'
+        },
+        url: {
+            type: 'string'
+        }
+    }).values;
+    var modelFlag = values.model;
+    var providerFlag = values.provider;
+    var model = modelFlag !== null && modelFlag !== void 0 ? modelFlag : _indexts.DEFAULT_EMBED_MODEL;
+    var provider = providerFlag !== null && providerFlag !== void 0 ? providerFlag : 'static';
+    var url = values.url;
+    // An invalid --provider throws here (validateConfig's own shape error), before anything is
+    // written -- one validator, not a second copy of its rules.
+    var configPath = (0, _indexts.initConfig)(process.cwd(), {
+        model: modelFlag,
+        provider: providerFlag,
+        url: url
+    });
     console.log("created ".concat(configPath));
     console.log("query away: ".concat(ctx.name, ' sql "SELECT path FROM frontmatter LIMIT 10"'));
-    // Semantic search needs a 124 MB model that nothing downloads implicitly; name the step
-    // here so the first search meets it as a choice rather than as a degraded result.
-    console.log("for meaning-based search: ".concat(ctx.name, " download (one 124 MB model, once)"));
+    // Naming a model is consent to fetch it, so writing one into the config says so loudly,
+    // with the download consequence and the prefetch command, right where it was just written.
+    if (provider === 'static') {
+        console.log("embed model: ".concat(model, " (static); nothing downloads until the first vector search -- ").concat(ctx.name, " download fetches it now instead"));
+    } else {
+        console.log("embed model: ".concat(model, " (").concat(provider).concat(url ? " at ".concat(url) : '', "); reached over the network at search time, nothing to download"));
+    }
+    console.log('a non-English tree should pick a model for its languages: see the sense-setup skill or INTEGRATIONS.md');
     // The decisions a new tree faces -- features, frontmatter conventions, note size -- are not
     // in this output; name both places that hold them, since an agent may have neither.
     console.log('features and tree design: the sense-setup skill, or schema.json');

package/dist/cjs/cli/init.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/init.ts"],"sourcesContent":["import { initConfig } from '../config/index.ts';\nimport { USAGE } from './index.ts';\nimport { parse } from './shared.ts';\nimport type { Command } from './types.ts';\n\nconst init: Command = (ctx) => {\n  parse(ctx.argv, `usage: ${ctx.name} ${USAGE.init}`, {});\n  const configPath = initConfig(process.cwd());\n  console.log(`created ${configPath}`);\n  console.log(`query away: ${ctx.name} sql \"SELECT path FROM frontmatter LIMIT 10\"`);\n  // Semantic search needs a 124 MB model that nothing downloads implicitly; name the step\n  // here so the first search meets it as a choice rather than as a degraded result.\n  console.log(`for meaning-based search: ${ctx.name} download (one 124 MB model, once)`);\n  // The decisions a new tree faces -- features, frontmatter conventions, note size -- are not\n  // in this output; name both places that hold them, since an agent may have neither.\n  console.log('features and tree design: the sense-setup skill, or schema.json');\n};\nexport default init;\n"],"names":["init","ctx","parse","argv","USAGE","name","configPath","initConfig","process","cwd","console","log"],"mappings":";;;;+BAiBA;;;eAAA;;;uBAjB2B;wBACL;wBACA;AAGtB,IAAMA,OAAgB,cAACC;IACrBC,IAAAA,eAAK,EAACD,IAAIE,IAAI,EAAE,AAAC,UAAqBC,OAAZH,IAAII,IAAI,EAAC,KAAc,OAAXD,eAAK,CAACJ,IAAI,GAAI,CAAC;IACrD,IAAMM,aAAaC,IAAAA,mBAAU,EAACC,QAAQC,GAAG;IACzCC,QAAQC,GAAG,CAAC,AAAC,WAAqB,OAAXL;IACvBI,QAAQC,GAAG,CAAC,AAAC,eAAuB,OAATV,IAAII,IAAI,EAAC;IACpC,wFAAwF;IACxF,kFAAkF;IAClFK,QAAQC,GAAG,CAAC,AAAC,6BAAqC,OAATV,IAAII,IAAI,EAAC;IAClD,4FAA4F;IAC5F,oFAAoF;IACpFK,QAAQC,GAAG,CAAC;AACd;IACA,WAAeX"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/init.ts"],"sourcesContent":["import { DEFAULT_EMBED_MODEL, initConfig } from '../config/index.ts';\nimport { USAGE } from './index.ts';\nimport { parse } from './shared.ts';\nimport type { Command } from './types.ts';\n\nconst init: Command = (ctx) => {\n  const { values } = parse(ctx.argv, `usage: ${ctx.name} ${USAGE.init}`, {\n    model: { type: 'string' },\n    provider: { type: 'string' },\n    url: { type: 'string' },\n  });\n  const modelFlag = values.model as string | undefined;\n  const providerFlag = values.provider as string | undefined;\n  const model = modelFlag ?? DEFAULT_EMBED_MODEL;\n  const provider = providerFlag ?? 'static';\n  const url = values.url as string | undefined;\n  // An invalid --provider throws here (validateConfig's own shape error), before anything is\n  // written -- one validator, not a second copy of its rules.\n  const configPath = initConfig(process.cwd(), { model: modelFlag, provider: providerFlag, url });\n  console.log(`created ${configPath}`);\n  console.log(`query away: ${ctx.name} sql \"SELECT path FROM frontmatter LIMIT 10\"`);\n  // Naming a model is consent to fetch it, so writing one into the config says so loudly,\n  // with the download consequence and the prefetch command, right where it was just written.\n  if (provider === 'static') {\n    console.log(`embed model: ${model} (static); nothing downloads until the first vector search -- ${ctx.name} download fetches it now instead`);\n  } else {\n    console.log(`embed model: ${model} (${provider}${url ? ` at ${url}` : ''}); reached over the network at search time, nothing to download`);\n  }\n  console.log('a non-English tree should pick a model for its languages: see the sense-setup skill or INTEGRATIONS.md');\n  // The decisions a new tree faces -- features, frontmatter conventions, note size -- are not\n  // in this output; name both places that hold them, since an agent may have neither.\n  console.log('features and tree design: the sense-setup skill, or schema.json');\n};\nexport default init;\n"],"names":["init","ctx","values","parse","argv","USAGE","name","model","type","provider","url","modelFlag","providerFlag","DEFAULT_EMBED_MODEL","configPath","initConfig","process","cwd","console","log"],"mappings":";;;;+BAiCA;;;eAAA;;;uBAjCgD;wBAC1B;wBACA;AAGtB,IAAMA,OAAgB,cAACC;IACrB,IAAM,AAAEC,SAAWC,IAAAA,eAAK,EAACF,IAAIG,IAAI,EAAE,AAAC,UAAqBC,OAAZJ,IAAIK,IAAI,EAAC,KAAc,OAAXD,eAAK,CAACL,IAAI,GAAI;QACrEO,OAAO;YAAEC,MAAM;QAAS;QACxBC,UAAU;YAAED,MAAM;QAAS;QAC3BE,KAAK;YAAEF,MAAM;QAAS;IACxB,GAJQN;IAKR,IAAMS,YAAYT,OAAOK,KAAK;IAC9B,IAAMK,eAAeV,OAAOO,QAAQ;IACpC,IAAMF,QAAQI,sBAAAA,uBAAAA,YAAaE,4BAAmB;IAC9C,IAAMJ,WAAWG,yBAAAA,0BAAAA,eAAgB;IACjC,IAAMF,MAAMR,OAAOQ,GAAG;IACtB,2FAA2F;IAC3F,4DAA4D;IAC5D,IAAMI,aAAaC,IAAAA,mBAAU,EAACC,QAAQC,GAAG,IAAI;QAAEV,OAAOI;QAAWF,UAAUG;QAAcF,KAAAA;IAAI;IAC7FQ,QAAQC,GAAG,CAAC,AAAC,WAAqB,OAAXL;IACvBI,QAAQC,GAAG,CAAC,AAAC,eAAuB,OAATlB,IAAIK,IAAI,EAAC;IACpC,wFAAwF;IACxF,2FAA2F;IAC3F,IAAIG,aAAa,UAAU;QACzBS,QAAQC,GAAG,CAAC,AAAC,gBAAqFlB,OAAtEM,OAAM,kEAAyE,OAATN,IAAIK,IAAI,EAAC;IAC7G,OAAO;QACLY,QAAQC,GAAG,CAAC,AAAC,gBAAyBV,OAAVF,OAAM,MAAeG,OAAXD,UAAmC,OAAxBC,MAAM,AAAC,OAAU,OAAJA,OAAQ,IAAG;IAC3E;IACAQ,QAAQC,GAAG,CAAC;IACZ,4FAA4F;IAC5F,oFAAoF;IACpFD,QAAQC,GAAG,CAAC;AACd;IACA,WAAenB"}