sensemaking 0.26.0 → 0.27.1

package/skills/sense/references/search.md

@@ -2,6 +2,8 @@
 
 Read this guide when search results will support a factual claim, when absence matters, when results look noisy, or when changing signal weights.
 
+[Sense terminology](terminology.md) defines the result objects. In particular, snippets, sections and embedding chunks are different units.
+
 ## What each signal establishes
 
 `words` ranks literal occurrences after stemming. It is the right signal for identifiers, error strings, names, and quoted phrases. A `match` row is lexical evidence because its `snippets` show the occurrence.
@@ -22,12 +24,26 @@ Weights are corpus and model choices. Compare representative queries before chan
 
 - `via` names the evidence that produced the row.
 - `snippets` contains marked lexical passages. It is empty when no word matched.
-- `lines` names the best section to read. Null means the whole note is the reference.
+- `lines` is the selected section or embedding chunk's line range in the indexed note. Null means the whole note is the reference.
 - `score` orders the fused result. Its scale changes with the participating signals and ranks, so compare rows only inside one result.
 - `similarity` is cosine similarity against the best chunk. Compare it inside one result and one model. Small trees can give unrelated text a moderately close nearest neighbor.
 
 Relay the evidence label when reporting a result. "The note contains these words" and "the note is semantically related" support different claims.
 
+## Finite retrieval and paging
+
+`k` is the maximum number of notes returned by one finite retrieval. Changing `k` can change the candidate pools, link seeds and fused ranking, including the earlier rows. It starts a new retrieval rather than continuing a stable result set. When you need pages, retain one returned array or JSON result and page that locally.
+
+## Explain a ranking
+
+Add `--explain` to a direct `sense search` invocation when you need to inspect how the configured signals contributed. With `--format json`, each returned row adds an `explanation` array containing `signal`, `rank`, `weight` and `contribution`. Entries follow words, links, vectors order and include only signals that contributed to that candidate. Without the flag, ordinary output is unchanged.
+
+Ranks are one-based within each signal's candidate list. A contribution is `weight / (60 + rank - 1)`, using the actual unrounded value accumulated during ranking. The sum is the fused score before its public display rounding. These values are not native BM25 scores and do not make lexical rankings comparable across stores.
+
+An isolated word-match seed can receive link-signal restart mass while its `via` label remains `match`. The explanation reports the numeric contribution; `via` remains the evidence label, not an exhaustive score breakdown. An explanation does not establish that a claim is true or describe why an absent note failed to enter the candidate set.
+
+Table and CSV formats encode the explanation cell as JSON text. Table output follows the existing cell clipping rules; use JSON or CSV for the complete values. Explanation reuses the current search and does not make an additional embedding request. It is not a saved-query configuration field.
+
 ## When a search misses
 
 For word search, try concrete terms that the notes may use, widen the selected preset, or raise `--k`. Search grammar depends on the selected store. Read its guide before adding operators.

package/skills/sense/references/sql.md

@@ -94,7 +94,7 @@ SELECT DISTINCT path FROM hop WHERE d > 0;
 
 Use `sense path` for the shortest chain between two known notes. Use raw recursion when the task needs a set of neighbors to filter or join.
 
-## Dead links
+## Unresolved links
 
 `dst IS NULL` includes links to attachments that sense never indexes, such as images, PDFs, and `.base` files. Exclude the attachment extensions used by the tree before treating the remaining rows as broken links. Trees with dotted markdown titles need an explicit extension list instead of a blanket "contains a dot" filter.
 

package/skills/sense/references/terminology.md

@@ -0,0 +1,32 @@
+# Sense terminology
+
+This is the shared vocabulary for Sense's README, command help, API documentation and skills. Use these names when describing results and output controls.
+
+| Term | Meaning |
+|---|---|
+| Note | One Markdown file. A document is the same unit; SQL and some status fields use that name. |
+| Tree | The directory of notes configured as Sense's root. |
+| Frontmatter | The YAML metadata at the beginning of a note, such as its title, tags or status. |
+| Search result | One matching note, returned with its path, ranking information and any snippets. A result is not a section or a chunk. |
+| Snippet | A short passage selected around matching words, marked with `«»`. A note can return multiple snippets. A snippet need not contain a whole section. |
+| Heading | A Markdown heading such as `## Installation`. Its level is its depth, from 1 to 6. |
+| Section | A heading and the content following it until the next heading of any level, or the end of the note. Text before the first heading is not a section in the `sections` table. A parent section does not include subsequent subsections. |
+| Section description | The heading text, level, start and end lines, and estimated token count returned by `peek`. It does not contain the section's prose. |
+| Outline | The section descriptions in document order. It is a view of the note's structure, not another stored type of content. |
+| Chunk | A span of note content prepared for embedding. Chunking follows document structure and a token budget; a chunk is not necessarily one section or one snippet. |
+| Line range | Inclusive, one-based lines in the indexed source note, such as `L12-28`. Search can point to a section or an embedding chunk. A null range provides no narrower location. |
+| Outbound link | A link from the current note to another indexed note. `peek.outbound` returns distinct destination paths. |
+| Backlink | A link from another indexed note to the current note. `peek.backlinks` returns distinct source paths. |
+| Unresolved link | A written link target that did not resolve to an indexed note. This does not establish that the link is broken: attachments and notes outside the index can also be unresolved. `peek.unresolved` returns written targets. |
+| Scope | The indexed notes selected for an operation by its preset and filters. Scope is not an access-control boundary. |
+| Search signal | One source of ranking evidence: word matches, link relationships or vector similarity. A signal's rank is a candidate's position within that signal, not the final combined position. |
+
+## Reading a section description
+
+The illustrative entry `## Installation [L12-28, ~90t]` describes a section headed “Installation”, starting at line 12 and ending at line 28, with an estimated 90 tokens. `peek` returns that description. Reading the indicated lines retrieves the content. With nested headings, the description ends before the next heading, even when that heading is deeper.
+
+## Naming output controls
+
+Name the object and the unit: a snippet count is how many passages are returned; a snippet character limit controls passage length; a section count is how many section descriptions are returned; a link count controls link entries. A count limit is not pagination. An offset, when supported by an operation, skips entries before returning the requested count. Raw SQL supports `LIMIT` and `OFFSET`; search and peek do not currently expose offset options.
+
+Peek count limits request prefixes of the ordered section and link lists. Each search or peek reads one committed index snapshot; separately issued operations can observe later commits. See [finite retrieval and paging](search.md#finite-retrieval-and-paging) for search budgets and paging retained results.

package/skills/sense-setup/references/store-benchmarks.md

@@ -1,15 +1,15 @@
-<!-- sense-store-benchmark release=0.26.0 -->
+<!-- sense-store-benchmark release=0.27.0 -->
 # Current store benchmark summary
 
-The release assessment generated this file for store selection. Release `0.26.0` passed on 2026-10-01, measured on Apple M1 Pro with Node v26.10.0.
+The release assessment generated this file for store selection. Release `0.27.0` passed on 2026-10-03, measured on Apple M1 Pro with Node v26.10.0.
 
 The timing rows ran on the same 6,566-note tree. They include CLI startup and each store's complete selected path. Ranked candidates and downstream work can differ by store, so these are current operating measurements rather than an isolated database-engine contest.
 
 | Store | Cold index | Warm count | Lexical search | Semantic search | Portable semantic nDCG@10 |
 |---|---|---|---|---|---|
-| sqlite | 2,158 ms | 256 ms | 351 ms | 569 ms | 0.3306 |
-| duckdb | 3,100 ms | 315 ms | 501 ms | 691 ms | 0.3282 |
-| turso | 3,268 ms | 276 ms | 409 ms | 1,066 ms | 0.3299 |
+| sqlite | 2,254 ms | 260 ms | 350 ms | 576 ms | 0.3306 |
+| duckdb | 3,026 ms | 319 ms | 510 ms | 695 ms | 0.3282 |
+| turso | 3,273 ms | 279 ms | 412 ms | 1,081 ms | 0.3299 |
 
 Cold index measures core index creation and a count query, excluding document embeddings. Warm count is a no-change `COUNT(*)` query. Lexical and semantic search are steady-state `sense search` commands. Lower timing is faster. Higher nDCG@10 is better; that quality column uses the same NFCorpus queries, judgments, result count, and model on every store.