sensemaking 0.7.2 → 0.9.0

package/skills/sense/SKILL.md

@@ -5,31 +5,36 @@ description: Query a markdown tree with the sense CLI — filter notes by frontm
 
 # sense
 
-SQL over a markdown tree, kept fresh by a filesystem check on every query. Four tables per file:
-`frontmatter` (one column per key, plus `path`/`_mtime`/`_size`/`_rank`), `content` (FTS5:
-`title`, `summary`, `text`), `links` (`src`, `target`, `dst` — `NULL` dst = dead link),
-`sections` (heading outline with line ranges and token estimates).
+SQL over a markdown tree, kept fresh by a filesystem check on every query. Every file becomes
+rows in `frontmatter` (one column per key, plus `path`/`_mtime`/`_size`/`_rank`), `content`
+(FTS5: `title`, `summary`, `text`), `links` (`src`, `target`, `dst` — `NULL` dst = dead link),
+and `sections` (heading outline with line ranges and token estimates). Features add their own
+storage; `map` and `status` report which are on.
 
 ## What each tool is for
 
 Every result is a reference (path, metadata, excerpt), never file contents; prose enters
-context only when you Read it. Costs: `map` is fixed-size, a `find` row ~30 tokens, `peek`
-~17% of reading the file. Which tool fits is a property of the question:
+context only when you Read it. Costs: `map` is fixed-size, a `search` row is tens of tokens,
+and a `peek` stays flat however large the note is. Which tool fits is a property of the
+question:
 
 - A deterministic, factual answer over known fields — counts, filters, "which notes have
-  X" — is SQL: `sense query`, a named query, or `find --where`. Enumerates every match;
+  X" — is SQL: `sense query`, a named query, or `search --where`. Enumerates every match;
   same result regardless of phrasing.
-- Locating notes by words in their prose is `find` — ranked lexical match. Results shift as
-  phrasing shifts, and bare words AND-join (one absent word = zero rows): write
-  `a OR b OR c` for any-word matching.
-- A conceptual question the notes phrase in different words is `find --semantic` (exists only
-  on trees whose config enables `embed`): adds meaning-based candidates labeled `via: vector`.
-  Conceptual similarity, not typo-tolerance; false positives are expected, labeled, and
-  bounded by `--k`.
+- Locating notes about something is `search` — one text through every engine the scope
+  has: word match (bare words AND-join — one absent word = zero lexical rows; write
+  `a OR b OR c` for any-word), link-graph expansion, and vector similarity, fused into one
+  ranked list. Read `via` per row: `match` rows contained your words; `vector`-only rows
+  did not — they are the "these words aren't in the tree; this is what's near in meaning"
+  signal. Vector rows are conceptual similarity, not typo-tolerance; false positives are
+  expected, labeled, and bounded by `--k`. `--lexical` skips vectors for one command when
+  word-presence is the question.
 - `map` answers "what is this tree" — fields, hub notes, recent changes — when the tree is
   unfamiliar.
 - `peek <path>` prices a file before you pay for it: outline with `[L143-162, ~380t]`
-  ranges, links both ways.
+  ranges, links both ways. Every list shows its first 20 with the true total; the
+  `sections` and `links` tables hold the rest, so a peek costs a few hundred tokens on any
+  note — heading-dense monsters included.
 - When you know the file and need its contents, `Read` it — sense adds nothing there. On
   large files peek's ranges let you read just one section; small files are often cheaper
   whole.
@@ -37,14 +42,15 @@ context only when you Read it. Costs: `map` is fixed-size, a `find` row ~30 toke
 Output defaults to a table, built for humans; `--format json` returns the same rows
 machine-parseable.
 
-## Verbs
+## Commands
 
 ```
-sense find "pricing OR billing OR invoicing" --where "f.status = 'active'" --k 10
+sense search "pricing OR billing OR invoicing" --where "f.status = 'active'" --k 10
+sense search "sourcing quotes" --preset raw     # a named settings bundle from the config
 sense peek notes/pricing-model.md               # a unique basename also works
 sense map
 sense query "<sql>" [params...]                 # ad-hoc SQL; ? binds positional args, count-checked
-sense <name> [params...]                        # named query from sense.config.json
+sense <name> [params...]                        # named query or saved search from sense.config.json
 sense --list | status | rebuild | check
 ```
 
@@ -54,44 +60,52 @@ sense --list | status | rebuild | check
   rules apply to search commands you write into subagent briefs.
 - When a search misses, the recall levers are: OR-in synonyms and concrete instances (the
   index only knows the words in the files — a note about a specific tool rarely names its
-  category), raise `--k` (a row costs ~30 tokens), and on embed-enabled trees `--semantic`
-  (matches meaning where term overlap fails). Each widening adds candidates and dilutes
-  ranking, so the noise trade-off runs both ways.
+  category), raise `--k` (a row costs tens of tokens), and widen the scope (`--preset`, or
+  `--include` for an ad-hoc glob). Vector rows already cover the meaning-over-words gap by
+  default. Each widening adds candidates and dilutes ranking, so the noise trade-off runs
+  both ways.
 - A frontmatter query enumerates its matches deterministically; search ranks by term overlap,
   so results shift as phrasing shifts. Trade-off: a query needs a known field, search doesn't.
-- `find` fuses BM25 with link-graph expansion; the `via` column says what produced each row —
-  `match` (terms hit), `link` (connected to notes that hit), `match+link` (both). With
-  `--semantic`, `vector` joins the composition and rows gain a `lines` column pointing at the
-  best-matching section, a direct `Read` range.
-- `--where` takes any SQL condition against frontmatter alias `f` — not only field equality:
-  `"f.status = 'active' AND has(f.tags, 'x')"`, `"f.path NOT LIKE 'generated/%'"`,
-  `"f.created >= datetime(?)"`. A tree can declare a default scope in `sense.config.json`
-  (`defaults.find.where`); an explicit `--where` replaces it, so `--where "1=1"` searches
-  everything. `sense status` prints the active default.
+- The `via` column says what produced each row — `match` (words hit), `link` (connected to
+  notes that hit), `vector` (near in meaning), and combinations. The `lines` column, when
+  set, points at the section that earned the row — the best-matching chunk on vector rows,
+  the term cluster's section on large lexical notes — and is a direct `Read` range; null
+  means the whole note is the reference.
+- Scope comes from presets: bare `search` uses the config's `default` preset; `--preset
+  <name>` picks another (unknown names error, listing what's declared); `--include <glob>`
+  is an ad-hoc scope that replaces the preset's globs for one command. `--where` takes any
+  SQL condition against frontmatter alias `f` — not only field equality:
+  `"f.status = 'active' AND has(f.tags, 'x')"`, `"datetime(f.created) >= datetime(?)"` —
+  and filters within the scope. `sense status` shows every preset with its coverage.
 - `score` is a rank-fusion value: it ranks rows within one result set and is not comparable
-  across queries, not a relevance magnitude — a perfect lexical hit and a weak vector-only
-  hit can both read ~0.017, because the number encodes how many signals fired and at what
-  rank. With `--semantic`, rows carry `similarity`: the cosine (-1 to 1) of the query
-  against that file's best-matching chunk — the same chunk the `lines` range points at.
-  It orders vector evidence within a result set; its absolute range depends on the corpus.
-  Measured: on thousands of notes, unrelated queries ~0.2 and genuine matches ~0.6; on a
-  few dozen notes the ranges compress and can overlap, because even a nonsense query has a
-  moderately near neighbour somewhere. Compare similarities within a result set rather than
-  against a fixed cutoff carried between trees.
-- Lexical `find` returns 0 rows when nothing matches, so it answers "is this in the tree at
-  all". `--semantic` always returns up to `k` rows — nearest-neighbour search has a nearest
-  neighbour for any input — so absence is a lexical question; `similarity` and the snippet
-  are the evidence for judging whether a vector row is a real hit.
-- `sense check` prepares every saved query (catching syntax and unknown-column errors), runs
-  the ones taking no parameters, and prints row counts: a saved query returning 0 rows looks
-  the same as a true empty result until something distinguishes them. For queries that
-  encode invariants (a dead-link list, an unsupported-claims list — rows are violations),
-  `checks: { "<name>": "empty" }` in the config inverts the meaning: `check` fails when the
-  query returns rows, making it usable as a test suite rather than a linter.
+  across queries, not a relevance magnitude — it encodes how many signals fired and at what
+  rank, so a perfect lexical hit and a weak vector-only hit can read the same number. With
+  vectors active, rows carry `similarity`: the cosine (-1 to 1) of the query against that
+  file's best-matching chunk — the same chunk the `lines` range points at. It orders vector
+  evidence within a result set; the range it spans depends on the corpus and the embedding
+  model, and compresses on small trees, where even a nonsense query has a moderately near
+  neighbour somewhere. Compare similarities within a result set rather than against a fixed
+  cutoff carried between trees.
+- Absence evidence lives in the labels: `search --lexical` (or a semantic-off scope)
+  returns 0 rows when the words are nowhere in the tree. Default `search` always returns
+  up to `k` rows — nearest-neighbour search has a nearest neighbour for any input — so a
+  result of only `via: vector` rows IS the absence signal for the words themselves;
+  `similarity` and the snippet are the evidence for judging whether a vector row is a real
+  conceptual hit.
+- Besides SQL strings, a config entry can save a whole search:
+  `"hot": { "search": "pricing OR billing", "preset": "raw", "k": 20 }` runs as
+  `sense hot` — the scenario's settings ride along with the name, so repeat runs need no
+  flags. An invocation-level `--preset`, `--k`, `--where`, or `--lexical` overrides the
+  saved value; `--list` marks these entries `(search)`.
+- `sense check` prepares every saved query and probes every saved search lexically with
+  k=1, so a typo'd column, stale SQL, bad FTS5 syntax, or unknown preset fails at check
+  time instead of silently mid-task. It reports row counts; whether an empty result is
+  good or bad is the reader's judgment — a dead-link query returning rows means broken
+  citations to fix, and the agent reads that directly.
 
 ## SQL
 
-The verbs are shorthands over the same four tables; anything they don't express, SQL does.
+The commands are shorthands over those tables; anything they don't express, SQL does.
 
 ```
 sense query "SELECT name FROM pragma_table_info('frontmatter')"          # what fields exist
@@ -108,7 +122,11 @@ sense query "SELECT j.value, COUNT(*) n FROM frontmatter, json_each(frontmatter.
   punctuation — bare `customer-facing` errors (`-` reads as a column filter), bare
   apostrophes are syntax errors: write `"customer-facing"`, `"founder's"`.
 - Rank with `ORDER BY bm25(content, 10.0, 5.0, 1.0)` (title > summary > body); excerpt with
-  `snippet(content, -1, '«', '»', '…', 10)`.
+  `snippet(content, -1, '«', '»', '…', 10)`. snippet() re-tokenizes each matched doc and its
+  cost grows superlinearly with doc size — measured ~10 s per query on a tree holding one
+  1 MB note. `search` bounds this itself (docs past 16 KB get an equivalent excerpt another
+  way); in hand-written SQL, guard it: `CASE WHEN length(text) <= 16384 THEN snippet(...)
+  END`, or select `title`/`summary` instead of an excerpt.
 - Select `content.title`/`content.summary` (always exist, empty when absent) rather than
   `f.title`/`f.summary` (discovered columns — error on trees that never declare them).
 - Frontmatter values keep their YAML type: strings are TEXT, whole numbers and booleans are
@@ -136,10 +154,11 @@ Worked traces: [EXAMPLES.md](EXAMPLES.md).
 
 - Missing CLI: `npm install -g sensemaking`. Missing config: `sense init` at the tree root.
   Discovery walks up from cwd; `--config <path>` overrides. Setting up or restructuring a
-  tree (features, frontmatter conventions, note design) is the `sense-setup` skill.
-- `map` and `status` report feature state (`features: links, sections, rank · off: embed
-  (features.embed)`). Invoking a capability whose feature is off is an error naming the
-  config key to enable — nothing silently falls back.
+  tree (presets, frontmatter conventions, note design) is the `sense-setup` skill.
+- `map` and `status` report each preset's coverage (files matched, embedded count) —
+  indexing derives from presets, so the coverage numbers are how you see what a config
+  actually indexes and embeds. A scope with fewer signals just uses fewer (a semantic-off
+  preset searches lexically); a saved search naming an unknown preset errors at `check`.
 - Save a query into `sense.config.json` only when it will be reused; run ad-hoc otherwise.
 - A one-line `summary:` per note is optional and pays twice: it appears in result rows and is a
   weighted search field. Date comparisons work for dates written as ISO 8601 (`2026-08-12`, or

package/skills/sense-setup/EXAMPLES.md

@@ -0,0 +1,121 @@
+# sense-setup: worked configurations
+
+Four tree shapes, each with its config and the commands an agent actually runs. Field
+names and folder names are illustrative — your tree defines its own.
+
+## A. Compiled wiki over immutable sources (the llm-wiki pattern)
+
+`raw/` holds ingested sources — big, never hand-edited. `wiki/` holds agent-compiled
+pages — linked, curated. The human drops sources and asks questions; the agent compiles
+and cites.
+
+```json
+{
+  "version": 3,
+  "presets": {
+    "default": { "include": ["wiki/**/*.md"], "k": 10 },
+    "raw":     { "include": ["raw/**/*.md"], "k": 5, "semantic": false }
+  },
+  "queries": {
+    "uncompiled": "SELECT path, _mtime FROM frontmatter WHERE path LIKE 'raw/%' AND path NOT IN (SELECT dst FROM links WHERE dst IS NOT NULL) ORDER BY _mtime DESC",
+    "stubs":      "SELECT path, _size FROM frontmatter WHERE path LIKE 'wiki/%' AND _size < 500 ORDER BY _size",
+    "dead-links": "SELECT src, target FROM links WHERE dst IS NULL ORDER BY src"
+  }
+}
+```
+
+```
+sense uncompiled                                # compile queue: raw files nothing cites yet
+sense search "how does attention scale"         # wiki only (default preset), vectors on
+sense search "rotary embeddings" --preset raw   # cite from sources; lexical, k=5
+sense dead-links                                # rows are broken citations to fix
+```
+
+What the shape buys: bare search never ranks raw noise above compiled pages; raw pays no
+vector/link cost; the compile queue, stub list, and citation integrity are one saved
+query each. The maintenance loop is `uncompiled` → write the wiki page citing its sources
+→ `dead-links` stays empty.
+
+## B. Nightly agent memory, consolidated (the dreaming pattern)
+
+`memory/` accumulates small notes written at session end, each with `project`,
+`created`, and `kind` (observation / steer / decision) frontmatter. A consolidation agent
+runs periodically: prune, merge, surface contradictions for the human. Retired notes move
+to `archive/` — still queryable, no longer embedded or ranked.
+
+```json
+{
+  "version": 3,
+  "presets": {
+    "default": { "include": ["memory/**/*.md"], "k": 10 },
+    "archive": { "include": ["archive/**/*.md"], "k": 10, "semantic": false }
+  },
+  "queries": {
+    "project":    "SELECT path, kind, created, title FROM frontmatter WHERE project = ? ORDER BY created DESC",
+    "steers":     "SELECT path, created, title FROM frontmatter WHERE kind = 'steer' AND project = ? ORDER BY created",
+    "retirement": "SELECT path, project, created FROM frontmatter WHERE datetime(created) < datetime('now','-90 day') AND path NOT IN (SELECT dst FROM links WHERE dst IS NOT NULL)",
+    "unfiled":    "SELECT path FROM frontmatter WHERE project IS NULL"
+  }
+}
+```
+
+```
+sense project acme-app                          # one project's notes, newest first
+sense search "prefers terse commit messages" --k 5
+  → memory/acme-app/2026-08-02-commits.md   via: match
+  → memory/acme-app/2026-06-11-style.md     via: vector  similarity: 0.71   # near-duplicate → merge candidate
+sense steers acme-app                           # oldest first: does a new steer override an old one?
+sense retirement                                # old + uncited → move to archive/
+sense unfiled                                   # rows are notes missing a project — file them
+```
+
+Presets are structural (live vs archived); per-project filtering is metadata (`project = ?`)
+— one tree serves every project. Semantic search over the memory preset is the
+near-duplicate detector: search a new note's own summary and read `similarity` within the
+results. Whether an old steer was overridden is a question for the human — found by
+search, never decided by it.
+
+## C. Evidence corpus: claims trace to sources
+
+`sources/` (immutable imports), `notes/` (one reading note per source), `reviews/`
+(synthesis whose claims must cite notes). The same shape fits incident reports and
+postmortems, user research and findings, due diligence and memos.
+
+```json
+{
+  "version": 3,
+  "presets": {
+    "default": { "include": ["reviews/**/*.md", "notes/**/*.md"], "k": 10 },
+    "source":  { "include": ["sources/**/*.md"], "k": 5, "semantic": false }
+  },
+  "queries": {
+    "unsupported": "SELECT path, title FROM frontmatter WHERE path LIKE 'reviews/%' AND path NOT IN (SELECT src FROM links WHERE dst LIKE 'notes/%')",
+    "unread":      "SELECT path FROM frontmatter WHERE path LIKE 'sources/%' AND path NOT IN (SELECT dst FROM links WHERE dst IS NOT NULL)",
+    "by-topic":    "SELECT path, title FROM frontmatter WHERE path NOT LIKE 'sources/%' AND has(topics, ?) ORDER BY path"
+  }
+}
+```
+
+```
+sense search "replication failures in priming studies"     # reviews + notes; sources never dilute
+sense search "the claim's exact phrasing" --preset source  # citation pull on demand
+sense unsupported                               # rows are synthesis claims with no note behind them
+sense unread                                    # the reading queue: sources no note cites
+```
+
+## D. A plain vault: zero configuration
+
+Someone else's Obsidian vault, heterogeneous, no structure worth declaring — the
+`sense init` starter untouched. The workflow is discovery:
+
+```
+sense map                                       # fields in use, hub notes, recent changes
+sense search "dataview queries"                 # words + links + meaning, one ranked list
+sense query "SELECT j.value AS tag, COUNT(*) n FROM frontmatter, json_each(frontmatter.tags) j GROUP BY j.value ORDER BY n DESC LIMIT 20"
+sense peek "Plugins/dataview.md"                # outline + links before reading
+```
+
+Presets earn their place only when a tree has parts deserving different treatment; a tree
+that is one kind of thing needs none of the vocabulary above. On a big vault, raise
+`default`'s `k` and read `lines` ranges instead of whole files — or start from the
+starter's `large` preset.

package/skills/sense-setup/SKILL.md

@@ -1,73 +1,91 @@
 ---
 name: sense-setup
-description: Set up the sense CLI on a markdown tree and make the tree-design decisions that shape it — sense init, enabling features (links, sections, rank, embed), and the trade-offs of frontmatter conventions, summaries, folder layout, and note size. Use when creating or restructuring a markdown knowledge base, running sense init, editing sense.config.json, enabling semantic search, or deciding how notes should be written for an agent to query later.
+description: Set up the sense CLI on a markdown tree and make the tree-design decisions that shape it — sense init, presets (which files, which settings, vectors on or off), and the trade-offs of frontmatter conventions, summaries, folder layout, and note size. Use when creating or restructuring a markdown knowledge base, running sense init, editing sense.config.json, configuring search scope or vectors, or deciding how notes should be written for an agent to query later.
 ---
 
 # sense: setup and tree design
 
 Querying an existing tree is the `sense` skill. This one covers making a tree:
-installing, configuring features, and the design decisions a tree owner faces.
+installing, writing presets, and the design decisions a tree owner faces.
+Worked configurations for common tree shapes: [EXAMPLES.md](EXAMPLES.md).
 
 ## Setup
 
 - `npm install -g sensemaking`, then `sense init` at the tree root writes
-  `sense.config.json` (all opt-out features on, `embed` off). Config discovery
-  walks up from cwd; `--config <path>` overrides.
-- `scan.include` globs resolve relative to the config file, never the cwd.
-- `sense map` and `sense status` report feature state
-  (`features: links, sections, rank · off: embed (features.embed)`), so the
-  current config is always discoverable from output.
+  `sense.config.json` — two presets (`default`, and `large` showing what a big
+  vault tunes), everything on. Config discovery walks up from cwd;
+  `--config <path>` overrides.
+- Globs resolve relative to the config file, never the cwd.
+- `sense status` and `sense map` show each preset's coverage (files matched,
+  embedded count), so what a config actually indexes is always visible in
+  output. A config edit that changes coverage rebuilds the cache and names the
+  preset that caused it on stderr.
 
-## Features
+## Presets
 
-| feature | powers | cost when on | config key |
-|---|---|---|---|
-| `links` | backlinks, dead-link queries, `find`'s link expansion, `peek`'s link lists | link re-resolution at reconcile | `features.links` (default on) |
-| `sections` | `peek`'s outline, line-range reads, per-section token estimates | heading extraction at parse | `features.sections` (default on) |
-| `rank` | `map`'s hubs, `_rank` in any ORDER BY | PageRank pass at reconcile; requires `links` | `features.rank` (default on) |
-| `embed` | `find --semantic` (meaning-based expansion) | vectors computed at reconcile; model download to `~/.cache/sensemaking` on first use; ~40 ms model load per semantic invocation | `features.embed` (default off) |
+A preset is a named, self-contained bundle of settings. `default` (required) is
+what bare commands use; every other preset is addressed by name
+(`sense search "..." --preset raw`, or `"preset": "raw"` in a saved search).
+No inheritance: what a preset states is all it does.
 
-- `embed` accepts `true` (built-in static model) or an object: `model` (Hugging
-  Face id or local path), `type` (`static` pure-JS built-in, or `api` for any
-  OpenAI-compatible `/v1/embeddings` endpoint — Ollama, LM Studio, llama.cpp,
-  hosted), `url`, `key` (env var name). A local `model` path is fully offline.
-- The two types differ in what they match, not only in cost: `static` is a
-  context-free distilled model that handles paraphrase and reworded concepts
-  (a query like "delegating without micromanaging" can reach a note that uses
-  neither word); tight near-synonyms and domain jargon ("heart attack" for
-  "myocardial infarction") are where it misses and an `api` transformer model
-  tends to succeed. The gain concentrates where the searcher's vocabulary
-  differs from the notes' — measured in the retrieval eval (BENCHMARKING.md,
-  "Retrieval quality"): on a vocabulary-gap corpus semantic expansion adds
-  recall; where vocabulary overlaps, BM25 plus link expansion already answers
-  most queries and vectors mostly reorder.
-- Enabling `embed` changes no default `find` result — expansion runs only when
-  a query passes `--semantic`. Invoking `--semantic` on a tree without `embed`
-  is an error naming the config key.
-- Toggling any feature rebuilds the cache on the next query (safe, automatic).
-- Disabled features degrade output, visibly: `peek` prints
-  `sections: off (features.sections)` rather than an empty outline.
+| field | means | default |
+|---|---|---|
+| `include` / `exclude` | which files this preset covers (globs) | required |
+| `k` | how many results a search returns | 10 |
+| `semantic` | vectors for this preset's files and searches | on; only ever written as `false` |
+| `where` | a standing SQL filter on frontmatter | none |
+
+**Indexing derives from presets.** A file is indexed if any preset includes it;
+it is embedded if any covering preset has semantic on. Consequences worth
+designing around:
+
+- A layer of the tree covered only by a `semantic: false` preset (raw sources,
+  archives, generated output) is fully searchable lexically and by SQL but
+  costs no vector work — the main scale lever.
+- Files no preset includes are not indexed at all.
+- Presets may overlap; they are views, not partitions.
+- The first semantic search embeds everything covered (progress on stderr;
+  minutes on tens of thousands of notes, seconds on small trees). Vectors use
+  the built-in static model unless a top-level
+  `"embed": { "model", "type": "static"|"api", "url", "key" }` block points at
+  a Model2Vec model, local path, or OpenAI-compatible endpoint. `static`
+  handles paraphrase and reworded concepts; tight domain jargon ("heart
+  attack" for "myocardial infarction") is where an `api` transformer model
+  tends to do better — measured in BENCHMARKING.md, "Retrieval quality".
+- Global `features` (`links`, `sections`, `rank`) still toggle tree-wide;
+  most trees never touch them.
+
+**Large vaults**: everything except the vector build is measured linear to
+100k notes with no tuning (BENCHMARKING.md). The knobs that matter are `k`
+(more, smaller results — rows carry `lines` section ranges, so agents read
+sections, not files) and `semantic: false` on the layers that don't earn
+vectors.
 
 ## Tree design decisions
 
 These belong to the tree's owner. sense works with any of them and reads no
-instruction files of its own; each choice below only changes what queries can
-do, and every consequence is listed so the choice can be made deliberately.
+instruction files of its own; each choice only changes what queries can do.
 
 - **Frontmatter fields.** Columns are discovered per tree — whatever keys notes
   declare become queryable. Consistent fields across notes make SQL filters
-  and named queries possible (`WHERE status = 'active'`). Reserved keys
+  and named queries possible (`WHERE status = 'active'`). SQLite's compiled
+  column limit (2,000; sqlite.org/limits.html) bounds distinct keys per tree —
+  the crawl stops with an error naming the count and the levers. Reserved keys
   (dropped with a warning): `path`, `_mtime`, `_size`, `_rank`, `content`,
   `links`, `sections`. Values keep their YAML type: strings TEXT, whole numbers
   and booleans INTEGER (`true` is 1), fractions REAL, lists and maps JSON text;
   `map` prints the observed type per field.
+- **Presets are path-shaped; frontmatter is state-shaped.** A preset's coverage
+  must be computable from the path alone (it decides indexing, baked into the
+  cache). Volatile state (`status`, `project`, dates) lives in frontmatter and
+  filters at query time (`where`, `has()`, `datetime()`). A state worth
+  different *indexing* (retired memory, superseded sources) is a state worth
+  moving the file — the archive-folder pattern in EXAMPLES.md.
 - **What a note omits is also a filter.** A layer that deliberately carries none
   of the fields the saved views filter on is excluded from all of them without
-  any view naming the layer; only a view filtering solely on a field the layer
-  does share needs an explicit condition (`AND type != 'raw'`). Sparse fields cut
-  both ways: less of the tree filters when you want breadth, and exactly this
-  separation when layers differ in authority — source extractions vs.
-  conclusions, generated output vs. notes.
+  any view naming the layer. Sparse fields cut both ways: less of the tree
+  filters when you want breadth, and exactly this separation when layers
+  differ in authority.
 - **Dates.** `datetime()` comparisons work for dates written as ISO 8601 —
   the only format it parses. A tree that mixes date formats can store them,
   but can't compare them in SQL.
@@ -75,17 +93,18 @@ do, and every consequence is listed so the choice can be made deliberately.
   every result row (often answering a question with no file read) and is a
   weighted search field ranked above body text. The cost is writing and
   maintaining the line as notes change.
-- **Folder shape.** sense is structure-indifferent: globs find the files,
-  paths are queryable text, links resolve by basename at any depth. Folders
-  are for the humans and agents navigating the tree, not for the index —
-  flat and nested trees query identically.
-- **Note size.** Many small notes: precise `find` hits, whole-file reads stay
-  cheap, more links to maintain. Fewer large notes: `sections` and `peek`
-  carry the cost down to line-range reads. Both work; per-section token
-  estimates exist either way.
+- **Folder shape.** Globs find the files, paths are queryable text, links
+  resolve by basename at any depth — but presets make folders meaningful:
+  a folder is the natural unit that gets its own coverage and settings.
+- **Note size.** Many small notes: precise search hits, whole-file reads stay
+  cheap, more links to maintain. Fewer large notes: `sections`, `peek`, and
+  the `lines` column carry the cost down to line-range reads. Both work.
+- **Recurring questions.** Save a scenario an agent will repeat: a SQL string
+  for filters and reports, or a saved search
+  (`"hot": { "search": "...", "preset": "raw", "k": 5 }`). Either runs as
+  `sense <name>`, and `sense check` validates both kinds against the real
+  tree, so a broken saved scenario fails at check time, not mid-task.
 - **Where decisions live.** Choices that should outlive one conversation can
   be recorded in the agent's own instruction or skill files, or in a note in
   the tree itself; a one-off search over an existing corpus needs none of
-  that. Whether to settle a choice with the user or proceed on the corpus as
-  found depends on whether the agent is only querying or also authoring —
-  an authoring agent's choices compound; a querying agent's don't.
+  that.

package/dist/cjs/commands/check.js

@@ -1,143 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", {
-    value: true
-});
-Object.defineProperty(exports, "default", {
-    enumerable: true,
-    get: function() {
-        return _default;
-    }
-});
-var _dbts = require("../db.js");
-var _outputts = require("../output.js");
-var _sharedts = require("./shared.js");
-function _array_like_to_array(arr, len) {
-    if (len == null || len > arr.length) len = arr.length;
-    for(var i = 0, arr2 = new Array(len); i < len; i++)arr2[i] = arr[i];
-    return arr2;
-}
-function _array_with_holes(arr) {
-    if (Array.isArray(arr)) return arr;
-}
-function _iterable_to_array_limit(arr, i) {
-    var _i = arr == null ? null : typeof Symbol !== "undefined" && arr[Symbol.iterator] || arr["@@iterator"];
-    if (_i == null) return;
-    var _arr = [];
-    var _n = true;
-    var _d = false;
-    var _s, _e;
-    try {
-        for(_i = _i.call(arr); !(_n = (_s = _i.next()).done); _n = true){
-            _arr.push(_s.value);
-            if (i && _arr.length === i) break;
-        }
-    } catch (err) {
-        _d = true;
-        _e = err;
-    } finally{
-        try {
-            if (!_n && _i["return"] != null) _i["return"]();
-        } finally{
-            if (_d) throw _e;
-        }
-    }
-    return _arr;
-}
-function _non_iterable_rest() {
-    throw new TypeError("Invalid attempt to destructure non-iterable instance.\\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.");
-}
-function _sliced_to_array(arr, i) {
-    return _array_with_holes(arr) || _iterable_to_array_limit(arr, i) || _unsupported_iterable_to_array(arr, i) || _non_iterable_rest();
-}
-function _unsupported_iterable_to_array(o, minLen) {
-    if (!o) return;
-    if (typeof o === "string") return _array_like_to_array(o, minLen);
-    var n = Object.prototype.toString.call(o).slice(8, -1);
-    if (n === "Object" && o.constructor) n = o.constructor.name;
-    if (n === "Map" || n === "Set") return Array.from(n);
-    if (n === "Arguments" || /^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)) return _array_like_to_array(o, minLen);
-}
-// A saved query that returns zero rows looks identical to a true empty result, so a broken
-// one can sit unnoticed -- one field report had `WHERE flag = 'true'` against a numeric
-// column reading as "nothing tagged yet" for hours. Preparing each query catches syntax and
-// unknown-column errors without executing it; queries that take no parameters are also run
-// for a row count. Parameterised queries are validated but not counted: inventing arguments
-// would report a count for a query nobody ran.
-var check = function check(ctx) {
-    var _cfg_queries;
-    var cfg = ctx.resolveConfig();
-    var _open = (0, _dbts.open)(cfg), db = _open.db, warnings = _open.warnings;
-    (0, _sharedts.printWarnings)(warnings);
-    var rows = [];
-    var failed = 0;
-    var _iteratorNormalCompletion = true, _didIteratorError = false, _iteratorError = undefined;
-    try {
-        for(var _iterator = Object.entries((_cfg_queries = cfg.queries) !== null && _cfg_queries !== void 0 ? _cfg_queries : {})[Symbol.iterator](), _step; !(_iteratorNormalCompletion = (_step = _iterator.next()).done); _iteratorNormalCompletion = true){
-            var _step_value = _sliced_to_array(_step.value, 2), name = _step_value[0], sql = _step_value[1];
-            var _sql_match;
-            var _cfg_checks;
-            var params = ((_sql_match = sql.match(/\?/g)) !== null && _sql_match !== void 0 ? _sql_match : []).length;
-            var expectEmpty = ((_cfg_checks = cfg.checks) === null || _cfg_checks === void 0 ? void 0 : _cfg_checks[name]) === 'empty';
-            try {
-                var statement = db.prepare(sql);
-                if (params > 0) {
-                    rows.push({
-                        query: name,
-                        params: params,
-                        rows: '—',
-                        status: 'ok (not run: needs parameters)'
-                    });
-                    continue;
-                }
-                var count = statement.all().length;
-                if (expectEmpty) {
-                    // An invariant query: rows are violations, so 0 is the pass, anything else fails.
-                    if (count > 0) failed++;
-                    rows.push({
-                        query: name,
-                        params: params,
-                        rows: count,
-                        status: count === 0 ? 'ok (empty, as asserted)' : "FAILED: expected empty, returned ".concat(count, " row(s)")
-                    });
-                } else {
-                    rows.push({
-                        query: name,
-                        params: params,
-                        rows: count,
-                        status: count === 0 ? 'ok, but returns 0 rows' : 'ok'
-                    });
-                }
-            } catch (err) {
-                failed++;
-                rows.push({
-                    query: name,
-                    params: params,
-                    rows: '—',
-                    status: "FAILED: ".concat(err.message)
-                });
-            }
-        }
-    } catch (err) {
-        _didIteratorError = true;
-        _iteratorError = err;
-    } finally{
-        try {
-            if (!_iteratorNormalCompletion && _iterator.return != null) {
-                _iterator.return();
-            }
-        } finally{
-            if (_didIteratorError) {
-                throw _iteratorError;
-            }
-        }
-    }
-    db.close();
-    if (rows.length === 0) {
-        console.log('no saved queries in config');
-        return;
-    }
-    (0, _outputts.printRows)(rows, ctx.format);
-    if (failed > 0) process.exit(1);
-};
-var _default = check;
-/* CJS INTEROP */ if (exports.__esModule && exports.default) { try { Object.defineProperty(exports.default, '__esModule', { value: true }); for (var key in exports) { exports.default[key] = exports[key]; } } catch (_) {}; module.exports = exports.default; }