sensemaking 0.18.3 → 0.19.0

package/README.md

@@ -32,7 +32,7 @@ Every file becomes rows in these tables, plus whatever an enabled feature adds o
 | table | holds | for |
 |---|---|---|
 | `frontmatter` | one column per key, plus `path`, `_mtime`, `_size`, `_rank`, `_parse_error` | filtering |
-| `content` | `title`, `summary`, `text`, `path`; on the default `sqlite` store an FTS5 index with machine-written `title_seg`/`summary_seg`/`text_seg` sidecars for matching in Chinese, Japanese, Thai, Khmer, Lao, and Burmese, on `duckdb` a plain table that matches those scripts as substrings | search and ranking |
+| `content` | `title`, `summary`, `text`, `path`; on the default `sqlite` store an FTS5 index with machine-written `title_seg`/`summary_seg`/`text_seg` sidecars for matching in Chinese, Japanese, Thai, Khmer, Lao, and Burmese, on `duckdb` a plain table that matches those scripts as substrings, on `turso` a Tantivy index with `_ngram` sidecars serving the same scripts | search and ranking |
 | `links` | `src`, `target` as written, `dst` resolved (`NULL` = dead link, but see the skill: a link to an attachment can never resolve) | graph |
 | `tags` | `path`, `tag`; frontmatter and inline `#tags` merged and deduplicated, nested tags stored full | tag filters |
 | `sections` | heading, `level`, `start_line`, `end_line`, `tokens` estimate | structure |
@@ -66,7 +66,7 @@ ORDER BY bm25(content, 10.0, 5.0, 1.0) LIMIT 10
 | `download` | fetch the embedding model named in the config (once per machine; the first vector search fetches it otherwise) |
 | `watch` | keep the index warm in the background (optional; see [WATCH.md](WATCH.md)) |
 
-`search` runs one text through every engine its scope has: FTS5 word match (BM25-ranked, bare words AND-join, operators are yours on `sqlite`; on `duckdb` the FTS5 operators are a named error, see Config), a personalized-PageRank walk over the link graph, and vector similarity, fused into one list. `via` labels each row's evidence (`match`, `link`, `vector`, combinations); `similarity` is the cosine against the best-matching chunk; `lines` points at the section that earned the row (a direct read range). A `vector`-only row means the search words don't appear in that note; it showed up because the model judged it semantically related. `--preset` picks a named settings bundle from the config, `--where` filters on frontmatter. `--format json` on any reporting command returns structured output, and `--format csv` writes the row-returning commands one row per line, for redirecting a large result to a file instead of into context; `--version` and `--help` do what they say.
+`search` runs one text through every engine its scope has: FTS5 word match (BM25-ranked, bare words AND-join, operators are yours on `sqlite`; on `duckdb` and `turso` the FTS5 operators are a named error, see Config), a personalized-PageRank walk over the link graph, and vector similarity, fused into one list. `via` labels each row's evidence (`match`, `link`, `vector`, combinations); `similarity` is the cosine against the best-matching chunk; `lines` points at the section that earned the row (a direct read range). A `vector`-only row means the search words don't appear in that note; it showed up because the model judged it semantically related. `--preset` picks a named settings bundle from the config, `--where` filters on frontmatter. `--format json` on any reporting command returns structured output, and `--format csv` writes the row-returning commands one row per line, for redirecting a large result to a file instead of into context; `--version` and `--help` do what they say.
 
 ## Config
 
@@ -94,7 +94,7 @@ ORDER BY bm25(content, 10.0, 5.0, 1.0) LIMIT 10
 | `presets` | named bundles of `include`/`exclude` globs, `k` (result count), `signals` (which engines this scope searches with, `words`, `links`, `vectors`; every signal whose prerequisites hold, unless the preset lists them exhaustively), `where` (a standing SQL filter). A file is indexed if any preset includes it, embedded if a model is named and some covering preset's `signals` include `vectors`; `status` shows each preset's coverage. |
 | `embed` | the model vectors are built with. Naming one gives the tree vectors; omitting the block means none at all, whatever the presets say. `sense download` fetches it. |
 | `content` | settings for the `content` table. `tokenize` names the FTS5 tokenizer, defaulting to `porter unicode61`: name `trigram` for substring matching inside a Latin word, or `unicode61 tokenchars '-_'` to keep hyphenated terms whole. Languages written without word spaces (Chinese, Japanese, Thai, Khmer, Lao, Burmese) need no decision here: they are indexed per grapheme and searched as ordered phrases, substring semantics, what grep gives, needing no minimum length; naming a tokenizer turns that off, since the tree has then chosen its own scheme. Changing it rebuilds the text index only; vectors, links, and sections are kept. |
-| `store` | backing store engine: `sqlite` (default, zero-dependency, Node's built-in SQLite) or `duckdb` (experimental). The first command that opens a `duckdb` tree installs `@duckdb/node-api` on its own (a one-time native download, ~110 MB). The same commands and table names run on both; what does not port is FTS5 syntax. Under `duckdb`, `search` text and raw `MATCH` reject FTS5's prefix (`foo*`), boolean (`AND`/`OR`/`NOT`), `NEAR`, initial-token (`^`), and column-filter (`title:foo`) operators with a named error that says how to rephrase or set `store` to `sqlite`; bare words and quoted phrases work on both. Raw `sql` is a per-store dialect: the tables and `has`/`basename`/`segment` are portable, but sqlite's FTS5 `MATCH`/`snippet()`/`bm25()` do not run under `duckdb`, so saved queries written in FTS5 syntax are sqlite dialect. Each store keeps its own cache file (`.sense/cache.db`, `.sense/cache.duckdb`); switching stores is a rebuild, not a migration. |
+| `store` | backing store engine: `sqlite` (default, zero-dependency, Node's built-in SQLite), or the experimental `duckdb` and `turso`. The first command that opens such a tree installs that engine's package on its own (`@duckdb/node-api`, a one-time native download of ~110 MB; `@tursodatabase/database`, much smaller). The same commands and table names run on all three; what does not port is FTS5 syntax. Under `duckdb` and `turso`, `search` text and raw `MATCH` reject FTS5's prefix (`foo*`), boolean (`AND`/`OR`/`NOT`), `NEAR`, initial-token (`^`), and column-filter (`title:foo`) operators with a named error that says how to rephrase or set `store` to `sqlite`; bare words and quoted phrases work on all three. Raw `sql` is a per-store dialect: the tables are portable, but sqlite's FTS5 `MATCH`/`snippet()`/`bm25()` do not run under `duckdb` or `turso`, so saved queries written in FTS5 syntax are sqlite dialect, and `has`/`basename`/`segment` are registered on `sqlite` and `duckdb` but not `turso`, whose client cannot register SQL functions. `sense watch` requires `sqlite`; on the other two it errors at start naming the fix. Each store keeps its own cache file (`.sense/cache.db`, `.sense/cache.duckdb`, `.sense/cache.turso.db`); switching stores is a rebuild, not a migration. |
 | `queries` | entries runnable as `sense <name>`, each naming the verb it runs: `{ sql }` for SQL (`?` binds positional args) or `{ search }` for a ranked search with its settings baked in, so `sense hot` needs no flags. Running an entry validates it: a typo'd column errors and exits nonzero, and a parameterised entry validates with any argument, since preparing precedes binding. |
 | `version` | schema version; older configs auto-migrate on load, noted on stderr. |
 
@@ -166,7 +166,25 @@ Three skills: `sense` for querying a tree (what each command is for, FTS5 syntax
 - **RAG / vector stores:** similarity can't express `WHERE status = 'active'`. Here vectors are one signal inside `search`: same database file, filters compose, every row labels its evidence (`via`), and a preset turns vectors off per layer of the tree. No second store, no daemon, no native builds on the default store.
 - **Document-OS apps (Anytype, Logseq, SilverBullet, Capacities):** full applications with their own UI and storage. `sense` is headless: your files stay files, there's no app to run.
 
-Dependencies: [yaml](https://github.com/eemeli/yaml), [@huggingface/tokenizers](https://github.com/huggingface/tokenizers.js), [franc-min](https://github.com/wooorm/franc), [tinypool](https://github.com/tinylibs/tinypool) (worker pool for parallel parsing on large trees), [mdast-util-from-markdown](https://github.com/syntax-tree/mdast-util-from-markdown), and the five GFM extensions it needs (not the full gfm/tagfilter bundle): [micromark-extension-gfm-autolink-literal](https://github.com/micromark/micromark-extension-gfm-autolink-literal), [micromark-extension-gfm-footnote](https://github.com/micromark/micromark-extension-gfm-footnote), [micromark-extension-gfm-strikethrough](https://github.com/micromark/micromark-extension-gfm-strikethrough), [micromark-extension-gfm-table](https://github.com/micromark/micromark-extension-gfm-table), [micromark-extension-gfm-task-list-item](https://github.com/micromark/micromark-extension-gfm-task-list-item), and their mdast counterparts [mdast-util-gfm-autolink-literal](https://github.com/syntax-tree/mdast-util-gfm-autolink-literal), [mdast-util-gfm-footnote](https://github.com/syntax-tree/mdast-util-gfm-footnote), [mdast-util-gfm-strikethrough](https://github.com/syntax-tree/mdast-util-gfm-strikethrough), [mdast-util-gfm-table](https://github.com/syntax-tree/mdast-util-gfm-table), [mdast-util-gfm-task-list-item](https://github.com/syntax-tree/mdast-util-gfm-task-list-item) (all pure JS), Node's built-in SQLite, and [install-module-linked](https://github.com/kmalakoff/install-module-linked), which installs the optional `duckdb` store's `@duckdb/node-api` native binding on first use instead of shipping it to every install. No native builds by default.
+Dependencies, all pure JS. No native builds by default.
+
+| | |
+|---|---|
+| [yaml](https://github.com/eemeli/yaml) | frontmatter |
+| [mdast-util-from-markdown](https://github.com/syntax-tree/mdast-util-from-markdown) | markdown parsing |
+| [@huggingface/tokenizers](https://github.com/huggingface/tokenizers.js) | chunking |
+| [franc-min](https://github.com/wooorm/franc) | language detection |
+| [tinypool](https://github.com/tinylibs/tinypool) | worker pool for parallel parsing on large trees |
+| [install-module-linked](https://github.com/kmalakoff/install-module-linked) | installs the optional `duckdb` and `turso` bindings on first use, instead of shipping them to every install |
+| Node's built-in SQLite | the default store |
+
+Plus the five GFM extensions `mdast-util-from-markdown` needs and their mdast counterparts, not the full gfm/tagfilter bundle:
+
+- [micromark-extension-gfm-autolink-literal](https://github.com/micromark/micromark-extension-gfm-autolink-literal) / [mdast-util-gfm-autolink-literal](https://github.com/syntax-tree/mdast-util-gfm-autolink-literal)
+- [micromark-extension-gfm-footnote](https://github.com/micromark/micromark-extension-gfm-footnote) / [mdast-util-gfm-footnote](https://github.com/syntax-tree/mdast-util-gfm-footnote)
+- [micromark-extension-gfm-strikethrough](https://github.com/micromark/micromark-extension-gfm-strikethrough) / [mdast-util-gfm-strikethrough](https://github.com/syntax-tree/mdast-util-gfm-strikethrough)
+- [micromark-extension-gfm-table](https://github.com/micromark/micromark-extension-gfm-table) / [mdast-util-gfm-table](https://github.com/syntax-tree/mdast-util-gfm-table)
+- [micromark-extension-gfm-task-list-item](https://github.com/micromark/micromark-extension-gfm-task-list-item) / [mdast-util-gfm-task-list-item](https://github.com/syntax-tree/mdast-util-gfm-task-list-item)
 
 ## License
 

package/dist/cjs/chunk/group.js

@@ -102,7 +102,7 @@ var DEFAULT_TARGET_TOKENS = 500;
 var PGC_GROUP_SIZE = 2;
 var OVERSIZE_TRIGGER_MULTIPLE = 2;
 // D5: segment.ts's unspaced-script set packs close to one token per character; Hangul is
-// spaced but still token-dense, added here as a conservative bound (one Unicode set, one home).
+// spaced but still token-dense and sits in this set as a conservative bound (one Unicode set, one home).
 var DENSE_SCRIPT = new RegExp("[".concat(_segmentts.UNSPACED_SCRIPTS, "\\p{scx=Hangul}]"), 'u');
 function estimateTokens(text) {
     var dense = 0;
@@ -232,12 +232,8 @@ function pack(segments, working) {
     if (current.length > 0) groups.push(current);
     return groups;
 }
-// A single line's text, still over working after rule 5's line-split can't reduce a lone line
-// (the CJK case, one sentence-dense line with no newlines), falls back to Intl.Segmenter sentence
-// boundaries, then word boundaries for any one sentence still over working. Never splits inside a
-// grapheme either way (both are ECMA-402, the same proven component segment.ts uses for
-// graphemes). Mode-agnostic: the caller decides whether `text` is the raw line or its extracted
-// flavor -- this only packs whatever text it is given.
+// Line-split alone can't shrink a lone dense line (the CJK case): falls back to sentence then
+// word boundaries (Intl.Segmenter, the same grapheme-safe engine as segment.ts), mode-agnostic on `text`.
 function splitLineText(text, working) {
     var sentences = segmentsOf(text, SENTENCE_SEGMENTER);
     var out = [];
@@ -287,10 +283,8 @@ var ATOMIC_TYPES = new Set([
     'table',
     'list'
 ]);
-// An atomic block's (code/table/list) piece is always a raw line slice: re-parsing a table's
-// later pieces without their header/delimiter rows demotes them to paragraph text. A non-atomic
-// piece is a raw line slice too when raw mode requested it; extracted mode re-parses and resolves
-// flavor as before.
+// Atomic (code/table/list) pieces are always a raw line slice: re-parsing a table's later pieces
+// without their header/delimiter rows would demote them to paragraph text.
 function piece(pieceLines, startLine, endLine, blockType, textMode) {
     var text = ATOMIC_TYPES.has(blockType) || textMode === 'raw' ? pieceLines.join('\n') : (0, _parsets.parse)(pieceLines.join('\n')).map(function(b) {
         return (0, _extractts.extractText)(b.node);
@@ -301,13 +295,8 @@ function piece(pieceLines, startLine, endLine, blockType, textMode) {
         text: text
     };
 }
-// A piece over working size that came from exactly one source line cannot be reduced by another
-// line-boundary pass (rule 5's gap); it is sliced instead, at sentence then word boundaries, over
-// the mode-resolved text piece() already produced -- so raw mode packs the raw line and extracted
-// mode packs its flavor-resolved text. Extents collapse to that one line for every sub-piece (F5:
-// re-derivation is by index over chunk()'s own deterministic output, not by a unique line range
-// per piece); `final` flags each sub-piece so group() knows not to re-derive its text from that
-// shared extent.
+// A one-line piece over working can't shrink via another line-boundary pass (rule 5's gap), so it
+// splits at sentence/word boundaries instead; `final` stops group() re-deriving its text by extent (F5).
 function finalizePiece(pieceLines, startLine, endLine, working, blockType, textMode) {
     var p = piece(pieceLines, startLine, endLine, blockType, textMode);
     if (pieceLines.length === 1 && estimateTokens(p.text) > working) {
@@ -324,9 +313,8 @@ function finalizePiece(pieceLines, startLine, endLine, working, blockType, textM
         p
     ];
 }
-// A block over 2x working size splits at line boundaries into pieces each <= working size,
-// never mid-line. `seed`: pending tokens (a heading) the first piece must join, checked against
-// the limit too.
+// A block over 2x working size splits at line boundaries into pieces each <= working size, never
+// mid-line. `seed`: pending tokens (e.g. a heading) the first piece must join, checked against the limit.
 function splitOversizeBlock(lines, startLine, endLine, working, blockType, textMode) {
     var seed = arguments.length > 6 && arguments[6] !== void 0 ? arguments[6] : 0;
     var _pieces;
@@ -468,8 +456,7 @@ function group(blocks, body, opts) {
         }
     }
     // 'raw': the chunk's own source lines verbatim, replacing the flavor-resolved join above (D9).
-    // A `final` chunk (a sub-line split) already carries its own slice's raw text -- re-slicing by
-    // extent would return the whole shared source line instead, once per sub-piece.
+    // A `final` chunk already carries its own slice's raw text; re-slicing by extent would return the whole shared line.
     var texted = resolved.text === 'raw' ? chunks.map(function(c) {
         return c.final ? c : _object_spread_props(_object_spread({}, c), {
             text: lines.slice(c.startLine - 1, c.endLine).join('\n').trim()

package/dist/cjs/chunk/group.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/chunk/group.ts"],"sourcesContent":["import { UNSPACED_SCRIPTS } from '../text/segment.ts';\nimport { extractText } from './extract.ts';\nimport { parse } from './parse.ts';\nimport type { Block, BlockType, Chunk, ChunkOptions } from './types.ts';\n\nexport const DEFAULT_TARGET_TOKENS = 500;\nconst PGC_GROUP_SIZE = 2;\nconst OVERSIZE_TRIGGER_MULTIPLE = 2;\n\n// D5: segment.ts's unspaced-script set packs close to one token per character; Hangul is\n// spaced but still token-dense, added here as a conservative bound (one Unicode set, one home).\nconst DENSE_SCRIPT = new RegExp(`[${UNSPACED_SCRIPTS}\\\\p{scx=Hangul}]`, 'u');\n\n// D5's size estimate: dense-script graphemes 1:1, everything else at 4 chars/token.\nexport function estimateTokens(text: string): number {\n  let dense = 0;\n  let other = 0;\n  for (const ch of text) {\n    if (DENSE_SCRIPT.test(ch)) dense++;\n    else other++;\n  }\n  return dense + other / 4;\n}\n\ninterface ResolvedOptions {\n  targetTokens: number;\n  text: 'extracted' | 'raw';\n}\n\nfunction resolveOptions(opts?: ChunkOptions): ResolvedOptions {\n  return {\n    targetTokens: opts?.targetTokens ?? DEFAULT_TARGET_TOKENS,\n    text: opts?.text ?? 'raw',\n  };\n}\n\n// A heading of any depth ends the current scope and starts a new one (D1); the heading block\n// itself is carried into the new scope, where it joins that scope's first group (F7/F10).\nfunction splitScopes(blocks: Block[]): Block[][] {\n  const scopes: Block[][] = [];\n  let current: Block[] = [];\n  for (const block of blocks) {\n    if (block.type === 'heading' && current.length > 0) {\n      scopes.push(current);\n      current = [];\n    }\n    current.push(block);\n  }\n  if (current.length > 0) scopes.push(current);\n  return scopes;\n}\n\ninterface Part {\n  startLine: number;\n  endLine: number;\n  text: string;\n  // true for a sub-line split piece (finalizePiece's splitLineText branch): its text is already\n  // the active mode's final text for its own slice of the line, not the whole line, so group()'s\n  // extent-based raw re-slice (startLine === endLine for every sibling sub-piece) must not touch it.\n  final?: boolean;\n}\n\nfunction finalize(parts: Part[]): (Chunk & { final?: boolean }) | undefined {\n  if (parts.length === 0) return undefined;\n  const first = parts[0];\n  const last = parts[parts.length - 1];\n  return { startLine: first.startLine, endLine: last.endLine, text: parts.map((p) => p.text).join('\\n'), final: parts.some((p) => p.final) };\n}\n\nconst NEWLINE_TOKENS = estimateTokens('\\n');\nconst SENTENCE_SEGMENTER = new Intl.Segmenter(undefined, { granularity: 'sentence' });\nconst WORD_SEGMENTER = new Intl.Segmenter(undefined, { granularity: 'word' });\n\nfunction segmentsOf(text: string, segmenter: Intl.Segmenter): string[] {\n  return Array.from(segmenter.segment(text), (s) => s.segment);\n}\n\n// Greedily packs segments (already contiguous, tiling the source text with no gaps) into groups\n// of at most `working` estimated tokens; a lone segment over `working` still stands alone.\nfunction pack(segments: string[], working: number): string[] {\n  const groups: string[] = [];\n  let current = '';\n  let tokens = 0;\n  for (const segment of segments) {\n    const segmentTokens = estimateTokens(segment);\n    if (current.length > 0 && tokens + segmentTokens > working) {\n      groups.push(current);\n      current = '';\n      tokens = 0;\n    }\n    current += segment;\n    tokens += segmentTokens;\n  }\n  if (current.length > 0) groups.push(current);\n  return groups;\n}\n\n// A single line's text, still over working after rule 5's line-split can't reduce a lone line\n// (the CJK case, one sentence-dense line with no newlines), falls back to Intl.Segmenter sentence\n// boundaries, then word boundaries for any one sentence still over working. Never splits inside a\n// grapheme either way (both are ECMA-402, the same proven component segment.ts uses for\n// graphemes). Mode-agnostic: the caller decides whether `text` is the raw line or its extracted\n// flavor -- this only packs whatever text it is given.\nfunction splitLineText(text: string, working: number): string[] {\n  const sentences = segmentsOf(text, SENTENCE_SEGMENTER);\n  const out: string[] = [];\n  let current = '';\n  let tokens = 0;\n  const flush = () => {\n    if (current.length > 0) {\n      out.push(current);\n      current = '';\n      tokens = 0;\n    }\n  };\n  for (const sentence of sentences) {\n    const sentenceTokens = estimateTokens(sentence);\n    if (sentenceTokens > working) {\n      flush();\n      out.push(...pack(segmentsOf(sentence, WORD_SEGMENTER), working));\n      continue;\n    }\n    if (current.length > 0 && tokens + sentenceTokens > working) flush();\n    current += sentence;\n    tokens += sentenceTokens;\n  }\n  flush();\n  return out;\n}\n\nconst ATOMIC_TYPES: ReadonlySet<BlockType> = new Set(['code', 'table', 'list']);\n\n// An atomic block's (code/table/list) piece is always a raw line slice: re-parsing a table's\n// later pieces without their header/delimiter rows demotes them to paragraph text. A non-atomic\n// piece is a raw line slice too when raw mode requested it; extracted mode re-parses and resolves\n// flavor as before.\nfunction piece(pieceLines: string[], startLine: number, endLine: number, blockType: BlockType, textMode: 'extracted' | 'raw'): Part {\n  const text =\n    ATOMIC_TYPES.has(blockType) || textMode === 'raw'\n      ? pieceLines.join('\\n')\n      : parse(pieceLines.join('\\n'))\n          .map((b) => extractText(b.node))\n          .join('\\n');\n  return { startLine, endLine, text };\n}\n\n// A piece over working size that came from exactly one source line cannot be reduced by another\n// line-boundary pass (rule 5's gap); it is sliced instead, at sentence then word boundaries, over\n// the mode-resolved text piece() already produced -- so raw mode packs the raw line and extracted\n// mode packs its flavor-resolved text. Extents collapse to that one line for every sub-piece (F5:\n// re-derivation is by index over chunk()'s own deterministic output, not by a unique line range\n// per piece); `final` flags each sub-piece so group() knows not to re-derive its text from that\n// shared extent.\nfunction finalizePiece(pieceLines: string[], startLine: number, endLine: number, working: number, blockType: BlockType, textMode: 'extracted' | 'raw'): Part[] {\n  const p = piece(pieceLines, startLine, endLine, blockType, textMode);\n  if (pieceLines.length === 1 && estimateTokens(p.text) > working) {\n    return splitLineText(p.text, working).map((text) => ({ startLine, endLine, text, final: true }));\n  }\n  return [p];\n}\n\n// A block over 2x working size splits at line boundaries into pieces each <= working size,\n// never mid-line. `seed`: pending tokens (a heading) the first piece must join, checked against\n// the limit too.\nfunction splitOversizeBlock(lines: string[], startLine: number, endLine: number, working: number, blockType: BlockType, textMode: 'extracted' | 'raw', seed = 0): Part[] {\n  const pieces: Part[] = [];\n  let pieceLines: string[] = [];\n  let pieceStart = startLine;\n  let tokens = seed;\n  for (let line = startLine; line <= endLine; line++) {\n    const lineText = lines[line - 1];\n    const sep = pieceLines.length > 0 || tokens > 0 ? NEWLINE_TOKENS : 0;\n    const lineTokens = estimateTokens(lineText);\n    if (pieceLines.length > 0 && tokens + sep + lineTokens > working) {\n      pieces.push(...finalizePiece(pieceLines, pieceStart, line - 1, working, blockType, textMode));\n      pieceLines = [];\n      tokens = 0;\n      pieceStart = line;\n      pieceLines.push(lineText);\n      tokens += lineTokens;\n      continue;\n    }\n    pieceLines.push(lineText);\n    tokens += sep + lineTokens;\n  }\n  if (pieceLines.length > 0) pieces.push(...finalizePiece(pieceLines, pieceStart, endLine, working, blockType, textMode));\n  return pieces;\n}\n\n// One heading scope's groups (D1): a heading opens the first group, and an oversize block\n// (rule 5, including an oversize heading) splits into pieces that each close their own group.\nfunction groupScope(scopeBlocks: Block[], lines: string[], resolved: ResolvedOptions): (Chunk & { final?: boolean })[] {\n  const working = resolved.targetTokens;\n  const trigger = working * OVERSIZE_TRIGGER_MULTIPLE;\n  const finished: (Chunk & { final?: boolean })[] = [];\n  let parts: Part[] = [];\n  let paragraphCount = 0;\n  let tokens = 0;\n\n  function close(): void {\n    const group = finalize(parts);\n    if (group) finished.push(group);\n    parts = [];\n    paragraphCount = 0;\n    tokens = 0;\n  }\n\n  // tokens tracks the active text mode's own estimate (a newline between parts costs\n  // NEWLINE_TOKENS too), so packing decisions size the text the chunk will actually ship as.\n  function addPart(text: string, sizeText: string, startLine: number, endLine: number): void {\n    tokens += (parts.length > 0 ? NEWLINE_TOKENS : 0) + estimateTokens(sizeText);\n    parts.push({ startLine, endLine, text });\n  }\n\n  for (const block of scopeBlocks) {\n    const raw = lines.slice(block.startLine - 1, block.endLine).join('\\n');\n    const blockTokens = estimateTokens(raw);\n\n    if (blockTokens > trigger) {\n      const seed = parts.length > 0 ? tokens : 0;\n      for (const p of splitOversizeBlock(lines, block.startLine, block.endLine, working, block.type, resolved.text, seed)) {\n        parts.push(p);\n        close();\n      }\n      continue;\n    }\n\n    const extracted = extractText(block.node);\n    const sizeText = resolved.text === 'raw' ? raw : extracted;\n\n    if (block.type === 'heading') {\n      addPart(extracted, sizeText, block.startLine, block.endLine);\n      continue;\n    }\n\n    // The 2x-working invariant holds even under pgc's paper-faithful 2-paragraph pairing --\n    // close first if the pair about to form would cross it.\n    const pairOversize = parts.length > 0 && tokens + NEWLINE_TOKENS + blockTokens > trigger;\n    if (pairOversize) close();\n\n    addPart(extracted, sizeText, block.startLine, block.endLine);\n    paragraphCount++;\n\n    if (paragraphCount >= PGC_GROUP_SIZE) close();\n  }\n  close();\n\n  return finished;\n}\n\n// Groups already-parsed blocks per opts (D1/D3), against the same body the blocks were parsed\n// from (line lookups for oversize splitting).\nexport function group(blocks: Block[], body: string, opts?: ChunkOptions): Chunk[] {\n  const resolved = resolveOptions(opts);\n  const lines = body.split('\\n');\n  const chunks: (Chunk & { final?: boolean })[] = [];\n  for (const scope of splitScopes(blocks)) chunks.push(...groupScope(scope, lines, resolved));\n  // 'raw': the chunk's own source lines verbatim, replacing the flavor-resolved join above (D9).\n  // A `final` chunk (a sub-line split) already carries its own slice's raw text -- re-slicing by\n  // extent would return the whole shared source line instead, once per sub-piece.\n  const texted =\n    resolved.text === 'raw'\n      ? chunks.map((c) =>\n          c.final\n            ? c\n            : {\n                ...c,\n                text: lines\n                  .slice(c.startLine - 1, c.endLine)\n                  .join('\\n')\n                  .trim(),\n              }\n        )\n      : chunks;\n  // A group can be all-blank (flavor-stripped to nothing, or a raw slice of pure syntax); it never produces a chunk.\n  return texted.filter((c) => c.text.trim().length > 0).map((c) => ({ startLine: c.startLine, endLine: c.endLine, text: c.text }));\n}\n"],"names":["DEFAULT_TARGET_TOKENS","estimateTokens","group","PGC_GROUP_SIZE","OVERSIZE_TRIGGER_MULTIPLE","DENSE_SCRIPT","RegExp","UNSPACED_SCRIPTS","text","dense","other","ch","test","resolveOptions","opts","targetTokens","splitScopes","blocks","scopes","current","block","type","length","push","finalize","parts","undefined","first","last","startLine","endLine","map","p","join","final","some","NEWLINE_TOKENS","SENTENCE_SEGMENTER","Intl","Segmenter","granularity","WORD_SEGMENTER","segmentsOf","segmenter","Array","from","segment","s","pack","segments","working","groups","tokens","segmentTokens","splitLineText","sentences","out","flush","sentence","sentenceTokens","ATOMIC_TYPES","Set","piece","pieceLines","blockType","textMode","has","parse","b","extractText","node","finalizePiece","splitOversizeBlock","lines","seed","pieces","pieceStart","line","lineText","sep","lineTokens","groupScope","scopeBlocks","resolved","trigger","finished","paragraphCount","close","addPart","sizeText","raw","slice","blockTokens","extracted","pairOversize","body","chunks","split","scope","texted","c","trim","filter"],"mappings":";;;;;;;;;;;QAKaA;eAAAA;;QASGC;eAAAA;;QA8OAC;eAAAA;;;yBA5PiB;yBACL;uBACN;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAGf,IAAMF,wBAAwB;AACrC,IAAMG,iBAAiB;AACvB,IAAMC,4BAA4B;AAElC,yFAAyF;AACzF,gGAAgG;AAChG,IAAMC,eAAe,IAAIC,OAAO,AAAC,IAAoB,OAAjBC,2BAAgB,EAAC,qBAAmB;AAGjE,SAASN,eAAeO,IAAY;IACzC,IAAIC,QAAQ;IACZ,IAAIC,QAAQ;QACP,kCAAA,2BAAA;;QAAL,QAAK,YAAYF,yBAAZ,SAAA,6BAAA,QAAA,yBAAA,iCAAkB;YAAlB,IAAMG,KAAN;YACH,IAAIN,aAAaO,IAAI,CAACD,KAAKF;iBACtBC;QACP;;QAHK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAIL,OAAOD,QAAQC,QAAQ;AACzB;AAOA,SAASG,eAAeC,IAAmB;;IACzC,OAAO;QACLC,YAAY,UAAED,iBAAAA,2BAAAA,KAAMC,YAAY,uCAAIf;QACpCQ,IAAI,WAAEM,iBAAAA,2BAAAA,KAAMN,IAAI,yCAAI;IACtB;AACF;AAEA,6FAA6F;AAC7F,0FAA0F;AAC1F,SAASQ,YAAYC,MAAe;IAClC,IAAMC,SAAoB,EAAE;IAC5B,IAAIC,UAAmB,EAAE;QACpB,kCAAA,2BAAA;;QAAL,QAAK,YAAeF,2BAAf,SAAA,6BAAA,QAAA,yBAAA,iCAAuB;YAAvB,IAAMG,QAAN;YACH,IAAIA,MAAMC,IAAI,KAAK,aAAaF,QAAQG,MAAM,GAAG,GAAG;gBAClDJ,OAAOK,IAAI,CAACJ;gBACZA,UAAU,EAAE;YACd;YACAA,QAAQI,IAAI,CAACH;QACf;;QANK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAOL,IAAID,QAAQG,MAAM,GAAG,GAAGJ,OAAOK,IAAI,CAACJ;IACpC,OAAOD;AACT;AAYA,SAASM,SAASC,KAAa;IAC7B,IAAIA,MAAMH,MAAM,KAAK,GAAG,OAAOI;IAC/B,IAAMC,QAAQF,KAAK,CAAC,EAAE;IACtB,IAAMG,OAAOH,KAAK,CAACA,MAAMH,MAAM,GAAG,EAAE;IACpC,OAAO;QAAEO,WAAWF,MAAME,SAAS;QAAEC,SAASF,KAAKE,OAAO;QAAEtB,MAAMiB,MAAMM,GAAG,CAAC,SAACC;mBAAMA,EAAExB,IAAI;WAAEyB,IAAI,CAAC;QAAOC,OAAOT,MAAMU,IAAI,CAAC,SAACH;mBAAMA,EAAEE,KAAK;;IAAE;AAC3I;AAEA,IAAME,iBAAiBnC,eAAe;AACtC,IAAMoC,qBAAqB,IAAIC,KAAKC,SAAS,CAACb,WAAW;IAAEc,aAAa;AAAW;AACnF,IAAMC,iBAAiB,IAAIH,KAAKC,SAAS,CAACb,WAAW;IAAEc,aAAa;AAAO;AAE3E,SAASE,WAAWlC,IAAY,EAAEmC,SAAyB;IACzD,OAAOC,MAAMC,IAAI,CAACF,UAAUG,OAAO,CAACtC,OAAO,SAACuC;eAAMA,EAAED,OAAO;;AAC7D;AAEA,gGAAgG;AAChG,2FAA2F;AAC3F,SAASE,KAAKC,QAAkB,EAAEC,OAAe;IAC/C,IAAMC,SAAmB,EAAE;IAC3B,IAAIhC,UAAU;IACd,IAAIiC,SAAS;QACR,kCAAA,2BAAA;;QAAL,QAAK,YAAiBH,6BAAjB,SAAA,6BAAA,QAAA,yBAAA,iCAA2B;YAA3B,IAAMH,UAAN;YACH,IAAMO,gBAAgBpD,eAAe6C;YACrC,IAAI3B,QAAQG,MAAM,GAAG,KAAK8B,SAASC,gBAAgBH,SAAS;gBAC1DC,OAAO5B,IAAI,CAACJ;gBACZA,UAAU;gBACViC,SAAS;YACX;YACAjC,WAAW2B;YACXM,UAAUC;QACZ;;QATK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAUL,IAAIlC,QAAQG,MAAM,GAAG,GAAG6B,OAAO5B,IAAI,CAACJ;IACpC,OAAOgC;AACT;AAEA,8FAA8F;AAC9F,kGAAkG;AAClG,kGAAkG;AAClG,wFAAwF;AACxF,gGAAgG;AAChG,uDAAuD;AACvD,SAASG,cAAc9C,IAAY,EAAE0C,OAAe;IAClD,IAAMK,YAAYb,WAAWlC,MAAM6B;IACnC,IAAMmB,MAAgB,EAAE;IACxB,IAAIrC,UAAU;IACd,IAAIiC,SAAS;IACb,IAAMK,QAAQ;QACZ,IAAItC,QAAQG,MAAM,GAAG,GAAG;YACtBkC,IAAIjC,IAAI,CAACJ;YACTA,UAAU;YACViC,SAAS;QACX;IACF;QACK,kCAAA,2BAAA;;QAAL,QAAK,YAAkBG,8BAAlB,SAAA,6BAAA,QAAA,yBAAA,iCAA6B;YAA7B,IAAMG,WAAN;YACH,IAAMC,iBAAiB1D,eAAeyD;YACtC,IAAIC,iBAAiBT,SAAS;oBAE5BM;gBADAC;gBACAD,CAAAA,OAAAA,KAAIjC,IAAI,OAARiC,MAAS,qBAAGR,KAAKN,WAAWgB,UAAUjB,iBAAiBS;gBACvD;YACF;YACA,IAAI/B,QAAQG,MAAM,GAAG,KAAK8B,SAASO,iBAAiBT,SAASO;YAC7DtC,WAAWuC;YACXN,UAAUO;QACZ;;QAVK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAWLF;IACA,OAAOD;AACT;AAEA,IAAMI,eAAuC,IAAIC,IAAI;IAAC;IAAQ;IAAS;CAAO;AAE9E,6FAA6F;AAC7F,gGAAgG;AAChG,kGAAkG;AAClG,oBAAoB;AACpB,SAASC,MAAMC,UAAoB,EAAElC,SAAiB,EAAEC,OAAe,EAAEkC,SAAoB,EAAEC,QAA6B;IAC1H,IAAMzD,OACJoD,aAAaM,GAAG,CAACF,cAAcC,aAAa,QACxCF,WAAW9B,IAAI,CAAC,QAChBkC,IAAAA,cAAK,EAACJ,WAAW9B,IAAI,CAAC,OACnBF,GAAG,CAAC,SAACqC;eAAMC,IAAAA,sBAAW,EAACD,EAAEE,IAAI;OAC7BrC,IAAI,CAAC;IACd,OAAO;QAAEJ,WAAAA;QAAWC,SAAAA;QAAStB,MAAAA;IAAK;AACpC;AAEA,gGAAgG;AAChG,kGAAkG;AAClG,kGAAkG;AAClG,kGAAkG;AAClG,gGAAgG;AAChG,gGAAgG;AAChG,iBAAiB;AACjB,SAAS+D,cAAcR,UAAoB,EAAElC,SAAiB,EAAEC,OAAe,EAAEoB,OAAe,EAAEc,SAAoB,EAAEC,QAA6B;IACnJ,IAAMjC,IAAI8B,MAAMC,YAAYlC,WAAWC,SAASkC,WAAWC;IAC3D,IAAIF,WAAWzC,MAAM,KAAK,KAAKrB,eAAe+B,EAAExB,IAAI,IAAI0C,SAAS;QAC/D,OAAOI,cAActB,EAAExB,IAAI,EAAE0C,SAASnB,GAAG,CAAC,SAACvB;mBAAU;gBAAEqB,WAAAA;gBAAWC,SAAAA;gBAAStB,MAAAA;gBAAM0B,OAAO;YAAK;;IAC/F;IACA,OAAO;QAACF;KAAE;AACZ;AAEA,2FAA2F;AAC3F,gGAAgG;AAChG,iBAAiB;AACjB,SAASwC,mBAAmBC,KAAe,EAAE5C,SAAiB,EAAEC,OAAe,EAAEoB,OAAe,EAAEc,SAAoB,EAAEC,QAA6B;QAAES,OAAAA,iEAAO;QAqBjIC;IApB3B,IAAMA,SAAiB,EAAE;IACzB,IAAIZ,aAAuB,EAAE;IAC7B,IAAIa,aAAa/C;IACjB,IAAIuB,SAASsB;IACb,IAAK,IAAIG,OAAOhD,WAAWgD,QAAQ/C,SAAS+C,OAAQ;QAClD,IAAMC,WAAWL,KAAK,CAACI,OAAO,EAAE;QAChC,IAAME,MAAMhB,WAAWzC,MAAM,GAAG,KAAK8B,SAAS,IAAIhB,iBAAiB;QACnE,IAAM4C,aAAa/E,eAAe6E;QAClC,IAAIf,WAAWzC,MAAM,GAAG,KAAK8B,SAAS2B,MAAMC,aAAa9B,SAAS;gBAChEyB;YAAAA,CAAAA,WAAAA,QAAOpD,IAAI,OAAXoD,UAAY,qBAAGJ,cAAcR,YAAYa,YAAYC,OAAO,GAAG3B,SAASc,WAAWC;YACnFF,aAAa,EAAE;YACfX,SAAS;YACTwB,aAAaC;YACbd,WAAWxC,IAAI,CAACuD;YAChB1B,UAAU4B;YACV;QACF;QACAjB,WAAWxC,IAAI,CAACuD;QAChB1B,UAAU2B,MAAMC;IAClB;IACA,IAAIjB,WAAWzC,MAAM,GAAG,GAAGqD,CAAAA,UAAAA,QAAOpD,IAAI,OAAXoD,SAAY,qBAAGJ,cAAcR,YAAYa,YAAY9C,SAASoB,SAASc,WAAWC;IAC7G,OAAOU;AACT;AAEA,0FAA0F;AAC1F,8FAA8F;AAC9F,SAASM,WAAWC,WAAoB,EAAET,KAAe,EAAEU,QAAyB;IAClF,IAAMjC,UAAUiC,SAASpE,YAAY;IACrC,IAAMqE,UAAUlC,UAAU9C;IAC1B,IAAMiF,WAA4C,EAAE;IACpD,IAAI5D,QAAgB,EAAE;IACtB,IAAI6D,iBAAiB;IACrB,IAAIlC,SAAS;IAEb,SAASmC;QACP,IAAMrF,QAAQsB,SAASC;QACvB,IAAIvB,OAAOmF,SAAS9D,IAAI,CAACrB;QACzBuB,QAAQ,EAAE;QACV6D,iBAAiB;QACjBlC,SAAS;IACX;IAEA,mFAAmF;IACnF,2FAA2F;IAC3F,SAASoC,QAAQhF,IAAY,EAAEiF,QAAgB,EAAE5D,SAAiB,EAAEC,OAAe;QACjFsB,UAAU,AAAC3B,CAAAA,MAAMH,MAAM,GAAG,IAAIc,iBAAiB,CAAA,IAAKnC,eAAewF;QACnEhE,MAAMF,IAAI,CAAC;YAAEM,WAAAA;YAAWC,SAAAA;YAAStB,MAAAA;QAAK;IACxC;QAEK,kCAAA,2BAAA;;QAAL,QAAK,YAAe0E,gCAAf,SAAA,6BAAA,QAAA,yBAAA,iCAA4B;YAA5B,IAAM9D,QAAN;YACH,IAAMsE,MAAMjB,MAAMkB,KAAK,CAACvE,MAAMS,SAAS,GAAG,GAAGT,MAAMU,OAAO,EAAEG,IAAI,CAAC;YACjE,IAAM2D,cAAc3F,eAAeyF;YAEnC,IAAIE,cAAcR,SAAS;gBACzB,IAAMV,OAAOjD,MAAMH,MAAM,GAAG,IAAI8B,SAAS;oBACpC,mCAAA,4BAAA;;oBAAL,QAAK,aAAWoB,mBAAmBC,OAAOrD,MAAMS,SAAS,EAAET,MAAMU,OAAO,EAAEoB,SAAS9B,MAAMC,IAAI,EAAE8D,SAAS3E,IAAI,EAAEkE,0BAAzG,UAAA,8BAAA,SAAA,0BAAA,kCAAgH;wBAAhH,IAAM1C,IAAN;wBACHP,MAAMF,IAAI,CAACS;wBACXuD;oBACF;;oBAHK;oBAAA;;;6BAAA,8BAAA;4BAAA;;;4BAAA;kCAAA;;;;gBAIL;YACF;YAEA,IAAMM,YAAYxB,IAAAA,sBAAW,EAACjD,MAAMkD,IAAI;YACxC,IAAMmB,WAAWN,SAAS3E,IAAI,KAAK,QAAQkF,MAAMG;YAEjD,IAAIzE,MAAMC,IAAI,KAAK,WAAW;gBAC5BmE,QAAQK,WAAWJ,UAAUrE,MAAMS,SAAS,EAAET,MAAMU,OAAO;gBAC3D;YACF;YAEA,wFAAwF;YACxF,wDAAwD;YACxD,IAAMgE,eAAerE,MAAMH,MAAM,GAAG,KAAK8B,SAAShB,iBAAiBwD,cAAcR;YACjF,IAAIU,cAAcP;YAElBC,QAAQK,WAAWJ,UAAUrE,MAAMS,SAAS,EAAET,MAAMU,OAAO;YAC3DwD;YAEA,IAAIA,kBAAkBnF,gBAAgBoF;QACxC;;QA9BK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IA+BLA;IAEA,OAAOF;AACT;AAIO,SAASnF,MAAMe,MAAe,EAAE8E,IAAY,EAAEjF,IAAmB;QAI7BkF;IAHzC,IAAMb,WAAWtE,eAAeC;IAChC,IAAM2D,QAAQsB,KAAKE,KAAK,CAAC;IACzB,IAAMD,SAA0C,EAAE;QAC7C,kCAAA,2BAAA;;QAAL,QAAK,YAAehF,YAAYC,4BAA3B,SAAA,6BAAA,QAAA,yBAAA;YAAA,IAAMiF,QAAN;YAAoCF,CAAAA,UAAAA,QAAOzE,IAAI,OAAXyE,SAAY,qBAAGf,WAAWiB,OAAOzB,OAAOU;;;QAA5E;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IACL,+FAA+F;IAC/F,+FAA+F;IAC/F,gFAAgF;IAChF,IAAMgB,SACJhB,SAAS3E,IAAI,KAAK,QACdwF,OAAOjE,GAAG,CAAC,SAACqE;eACVA,EAAElE,KAAK,GACHkE,IACA,wCACKA;YACH5F,MAAMiE,MACHkB,KAAK,CAACS,EAAEvE,SAAS,GAAG,GAAGuE,EAAEtE,OAAO,EAChCG,IAAI,CAAC,MACLoE,IAAI;;SAGfL;IACN,mHAAmH;IACnH,OAAOG,OAAOG,MAAM,CAAC,SAACF;eAAMA,EAAE5F,IAAI,CAAC6F,IAAI,GAAG/E,MAAM,GAAG;OAAGS,GAAG,CAAC,SAACqE;eAAO;YAAEvE,WAAWuE,EAAEvE,SAAS;YAAEC,SAASsE,EAAEtE,OAAO;YAAEtB,MAAM4F,EAAE5F,IAAI;QAAC;;AAC/H"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/chunk/group.ts"],"sourcesContent":["import { UNSPACED_SCRIPTS } from '../text/segment.ts';\nimport { extractText } from './extract.ts';\nimport { parse } from './parse.ts';\nimport type { Block, BlockType, Chunk, ChunkOptions } from './types.ts';\n\nexport const DEFAULT_TARGET_TOKENS = 500;\nconst PGC_GROUP_SIZE = 2;\nconst OVERSIZE_TRIGGER_MULTIPLE = 2;\n\n// D5: segment.ts's unspaced-script set packs close to one token per character; Hangul is\n// spaced but still token-dense and sits in this set as a conservative bound (one Unicode set, one home).\nconst DENSE_SCRIPT = new RegExp(`[${UNSPACED_SCRIPTS}\\\\p{scx=Hangul}]`, 'u');\n\n// D5's size estimate: dense-script graphemes 1:1, everything else at 4 chars/token.\nexport function estimateTokens(text: string): number {\n  let dense = 0;\n  let other = 0;\n  for (const ch of text) {\n    if (DENSE_SCRIPT.test(ch)) dense++;\n    else other++;\n  }\n  return dense + other / 4;\n}\n\ninterface ResolvedOptions {\n  targetTokens: number;\n  text: 'extracted' | 'raw';\n}\n\nfunction resolveOptions(opts?: ChunkOptions): ResolvedOptions {\n  return {\n    targetTokens: opts?.targetTokens ?? DEFAULT_TARGET_TOKENS,\n    text: opts?.text ?? 'raw',\n  };\n}\n\n// A heading of any depth ends the current scope and starts a new one (D1); the heading block\n// itself is carried into the new scope, where it joins that scope's first group (F7/F10).\nfunction splitScopes(blocks: Block[]): Block[][] {\n  const scopes: Block[][] = [];\n  let current: Block[] = [];\n  for (const block of blocks) {\n    if (block.type === 'heading' && current.length > 0) {\n      scopes.push(current);\n      current = [];\n    }\n    current.push(block);\n  }\n  if (current.length > 0) scopes.push(current);\n  return scopes;\n}\n\ninterface Part {\n  startLine: number;\n  endLine: number;\n  text: string;\n  // True for a sub-line split piece: its text is already the final slice, not the whole line --\n  // group()'s extent-based raw re-slice must not touch it (siblings share startLine === endLine).\n  final?: boolean;\n}\n\nfunction finalize(parts: Part[]): (Chunk & { final?: boolean }) | undefined {\n  if (parts.length === 0) return undefined;\n  const first = parts[0];\n  const last = parts[parts.length - 1];\n  return { startLine: first.startLine, endLine: last.endLine, text: parts.map((p) => p.text).join('\\n'), final: parts.some((p) => p.final) };\n}\n\nconst NEWLINE_TOKENS = estimateTokens('\\n');\nconst SENTENCE_SEGMENTER = new Intl.Segmenter(undefined, { granularity: 'sentence' });\nconst WORD_SEGMENTER = new Intl.Segmenter(undefined, { granularity: 'word' });\n\nfunction segmentsOf(text: string, segmenter: Intl.Segmenter): string[] {\n  return Array.from(segmenter.segment(text), (s) => s.segment);\n}\n\n// Greedily packs segments (already contiguous, tiling the source text with no gaps) into groups\n// of at most `working` estimated tokens; a lone segment over `working` still stands alone.\nfunction pack(segments: string[], working: number): string[] {\n  const groups: string[] = [];\n  let current = '';\n  let tokens = 0;\n  for (const segment of segments) {\n    const segmentTokens = estimateTokens(segment);\n    if (current.length > 0 && tokens + segmentTokens > working) {\n      groups.push(current);\n      current = '';\n      tokens = 0;\n    }\n    current += segment;\n    tokens += segmentTokens;\n  }\n  if (current.length > 0) groups.push(current);\n  return groups;\n}\n\n// Line-split alone can't shrink a lone dense line (the CJK case): falls back to sentence then\n// word boundaries (Intl.Segmenter, the same grapheme-safe engine as segment.ts), mode-agnostic on `text`.\nfunction splitLineText(text: string, working: number): string[] {\n  const sentences = segmentsOf(text, SENTENCE_SEGMENTER);\n  const out: string[] = [];\n  let current = '';\n  let tokens = 0;\n  const flush = () => {\n    if (current.length > 0) {\n      out.push(current);\n      current = '';\n      tokens = 0;\n    }\n  };\n  for (const sentence of sentences) {\n    const sentenceTokens = estimateTokens(sentence);\n    if (sentenceTokens > working) {\n      flush();\n      out.push(...pack(segmentsOf(sentence, WORD_SEGMENTER), working));\n      continue;\n    }\n    if (current.length > 0 && tokens + sentenceTokens > working) flush();\n    current += sentence;\n    tokens += sentenceTokens;\n  }\n  flush();\n  return out;\n}\n\nconst ATOMIC_TYPES: ReadonlySet<BlockType> = new Set(['code', 'table', 'list']);\n\n// Atomic (code/table/list) pieces are always a raw line slice: re-parsing a table's later pieces\n// without their header/delimiter rows would demote them to paragraph text.\nfunction piece(pieceLines: string[], startLine: number, endLine: number, blockType: BlockType, textMode: 'extracted' | 'raw'): Part {\n  const text =\n    ATOMIC_TYPES.has(blockType) || textMode === 'raw'\n      ? pieceLines.join('\\n')\n      : parse(pieceLines.join('\\n'))\n          .map((b) => extractText(b.node))\n          .join('\\n');\n  return { startLine, endLine, text };\n}\n\n// A one-line piece over working can't shrink via another line-boundary pass (rule 5's gap), so it\n// splits at sentence/word boundaries instead; `final` stops group() re-deriving its text by extent (F5).\nfunction finalizePiece(pieceLines: string[], startLine: number, endLine: number, working: number, blockType: BlockType, textMode: 'extracted' | 'raw'): Part[] {\n  const p = piece(pieceLines, startLine, endLine, blockType, textMode);\n  if (pieceLines.length === 1 && estimateTokens(p.text) > working) {\n    return splitLineText(p.text, working).map((text) => ({ startLine, endLine, text, final: true }));\n  }\n  return [p];\n}\n\n// A block over 2x working size splits at line boundaries into pieces each <= working size, never\n// mid-line. `seed`: pending tokens (e.g. a heading) the first piece must join, checked against the limit.\nfunction splitOversizeBlock(lines: string[], startLine: number, endLine: number, working: number, blockType: BlockType, textMode: 'extracted' | 'raw', seed = 0): Part[] {\n  const pieces: Part[] = [];\n  let pieceLines: string[] = [];\n  let pieceStart = startLine;\n  let tokens = seed;\n  for (let line = startLine; line <= endLine; line++) {\n    const lineText = lines[line - 1];\n    const sep = pieceLines.length > 0 || tokens > 0 ? NEWLINE_TOKENS : 0;\n    const lineTokens = estimateTokens(lineText);\n    if (pieceLines.length > 0 && tokens + sep + lineTokens > working) {\n      pieces.push(...finalizePiece(pieceLines, pieceStart, line - 1, working, blockType, textMode));\n      pieceLines = [];\n      tokens = 0;\n      pieceStart = line;\n      pieceLines.push(lineText);\n      tokens += lineTokens;\n      continue;\n    }\n    pieceLines.push(lineText);\n    tokens += sep + lineTokens;\n  }\n  if (pieceLines.length > 0) pieces.push(...finalizePiece(pieceLines, pieceStart, endLine, working, blockType, textMode));\n  return pieces;\n}\n\n// One heading scope's groups (D1): a heading opens the first group, and an oversize block\n// (rule 5, including an oversize heading) splits into pieces that each close their own group.\nfunction groupScope(scopeBlocks: Block[], lines: string[], resolved: ResolvedOptions): (Chunk & { final?: boolean })[] {\n  const working = resolved.targetTokens;\n  const trigger = working * OVERSIZE_TRIGGER_MULTIPLE;\n  const finished: (Chunk & { final?: boolean })[] = [];\n  let parts: Part[] = [];\n  let paragraphCount = 0;\n  let tokens = 0;\n\n  function close(): void {\n    const group = finalize(parts);\n    if (group) finished.push(group);\n    parts = [];\n    paragraphCount = 0;\n    tokens = 0;\n  }\n\n  // tokens tracks the active text mode's own estimate (a newline between parts costs\n  // NEWLINE_TOKENS too), so packing decisions size the text the chunk will actually ship as.\n  function addPart(text: string, sizeText: string, startLine: number, endLine: number): void {\n    tokens += (parts.length > 0 ? NEWLINE_TOKENS : 0) + estimateTokens(sizeText);\n    parts.push({ startLine, endLine, text });\n  }\n\n  for (const block of scopeBlocks) {\n    const raw = lines.slice(block.startLine - 1, block.endLine).join('\\n');\n    const blockTokens = estimateTokens(raw);\n\n    if (blockTokens > trigger) {\n      const seed = parts.length > 0 ? tokens : 0;\n      for (const p of splitOversizeBlock(lines, block.startLine, block.endLine, working, block.type, resolved.text, seed)) {\n        parts.push(p);\n        close();\n      }\n      continue;\n    }\n\n    const extracted = extractText(block.node);\n    const sizeText = resolved.text === 'raw' ? raw : extracted;\n\n    if (block.type === 'heading') {\n      addPart(extracted, sizeText, block.startLine, block.endLine);\n      continue;\n    }\n\n    // The 2x-working invariant holds even under pgc's paper-faithful 2-paragraph pairing --\n    // close first if the pair about to form would cross it.\n    const pairOversize = parts.length > 0 && tokens + NEWLINE_TOKENS + blockTokens > trigger;\n    if (pairOversize) close();\n\n    addPart(extracted, sizeText, block.startLine, block.endLine);\n    paragraphCount++;\n\n    if (paragraphCount >= PGC_GROUP_SIZE) close();\n  }\n  close();\n\n  return finished;\n}\n\n// Groups already-parsed blocks per opts (D1/D3), against the same body the blocks were parsed\n// from (line lookups for oversize splitting).\nexport function group(blocks: Block[], body: string, opts?: ChunkOptions): Chunk[] {\n  const resolved = resolveOptions(opts);\n  const lines = body.split('\\n');\n  const chunks: (Chunk & { final?: boolean })[] = [];\n  for (const scope of splitScopes(blocks)) chunks.push(...groupScope(scope, lines, resolved));\n  // 'raw': the chunk's own source lines verbatim, replacing the flavor-resolved join above (D9).\n  // A `final` chunk already carries its own slice's raw text; re-slicing by extent would return the whole shared line.\n  const texted =\n    resolved.text === 'raw'\n      ? chunks.map((c) =>\n          c.final\n            ? c\n            : {\n                ...c,\n                text: lines\n                  .slice(c.startLine - 1, c.endLine)\n                  .join('\\n')\n                  .trim(),\n              }\n        )\n      : chunks;\n  // A group can be all-blank (flavor-stripped to nothing, or a raw slice of pure syntax); it never produces a chunk.\n  return texted.filter((c) => c.text.trim().length > 0).map((c) => ({ startLine: c.startLine, endLine: c.endLine, text: c.text }));\n}\n"],"names":["DEFAULT_TARGET_TOKENS","estimateTokens","group","PGC_GROUP_SIZE","OVERSIZE_TRIGGER_MULTIPLE","DENSE_SCRIPT","RegExp","UNSPACED_SCRIPTS","text","dense","other","ch","test","resolveOptions","opts","targetTokens","splitScopes","blocks","scopes","current","block","type","length","push","finalize","parts","undefined","first","last","startLine","endLine","map","p","join","final","some","NEWLINE_TOKENS","SENTENCE_SEGMENTER","Intl","Segmenter","granularity","WORD_SEGMENTER","segmentsOf","segmenter","Array","from","segment","s","pack","segments","working","groups","tokens","segmentTokens","splitLineText","sentences","out","flush","sentence","sentenceTokens","ATOMIC_TYPES","Set","piece","pieceLines","blockType","textMode","has","parse","b","extractText","node","finalizePiece","splitOversizeBlock","lines","seed","pieces","pieceStart","line","lineText","sep","lineTokens","groupScope","scopeBlocks","resolved","trigger","finished","paragraphCount","close","addPart","sizeText","raw","slice","blockTokens","extracted","pairOversize","body","chunks","split","scope","texted","c","trim","filter"],"mappings":";;;;;;;;;;;QAKaA;eAAAA;;QASGC;eAAAA;;QAiOAC;eAAAA;;;yBA/OiB;yBACL;uBACN;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAGf,IAAMF,wBAAwB;AACrC,IAAMG,iBAAiB;AACvB,IAAMC,4BAA4B;AAElC,yFAAyF;AACzF,yGAAyG;AACzG,IAAMC,eAAe,IAAIC,OAAO,AAAC,IAAoB,OAAjBC,2BAAgB,EAAC,qBAAmB;AAGjE,SAASN,eAAeO,IAAY;IACzC,IAAIC,QAAQ;IACZ,IAAIC,QAAQ;QACP,kCAAA,2BAAA;;QAAL,QAAK,YAAYF,yBAAZ,SAAA,6BAAA,QAAA,yBAAA,iCAAkB;YAAlB,IAAMG,KAAN;YACH,IAAIN,aAAaO,IAAI,CAACD,KAAKF;iBACtBC;QACP;;QAHK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAIL,OAAOD,QAAQC,QAAQ;AACzB;AAOA,SAASG,eAAeC,IAAmB;;IACzC,OAAO;QACLC,YAAY,UAAED,iBAAAA,2BAAAA,KAAMC,YAAY,uCAAIf;QACpCQ,IAAI,WAAEM,iBAAAA,2BAAAA,KAAMN,IAAI,yCAAI;IACtB;AACF;AAEA,6FAA6F;AAC7F,0FAA0F;AAC1F,SAASQ,YAAYC,MAAe;IAClC,IAAMC,SAAoB,EAAE;IAC5B,IAAIC,UAAmB,EAAE;QACpB,kCAAA,2BAAA;;QAAL,QAAK,YAAeF,2BAAf,SAAA,6BAAA,QAAA,yBAAA,iCAAuB;YAAvB,IAAMG,QAAN;YACH,IAAIA,MAAMC,IAAI,KAAK,aAAaF,QAAQG,MAAM,GAAG,GAAG;gBAClDJ,OAAOK,IAAI,CAACJ;gBACZA,UAAU,EAAE;YACd;YACAA,QAAQI,IAAI,CAACH;QACf;;QANK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAOL,IAAID,QAAQG,MAAM,GAAG,GAAGJ,OAAOK,IAAI,CAACJ;IACpC,OAAOD;AACT;AAWA,SAASM,SAASC,KAAa;IAC7B,IAAIA,MAAMH,MAAM,KAAK,GAAG,OAAOI;IAC/B,IAAMC,QAAQF,KAAK,CAAC,EAAE;IACtB,IAAMG,OAAOH,KAAK,CAACA,MAAMH,MAAM,GAAG,EAAE;IACpC,OAAO;QAAEO,WAAWF,MAAME,SAAS;QAAEC,SAASF,KAAKE,OAAO;QAAEtB,MAAMiB,MAAMM,GAAG,CAAC,SAACC;mBAAMA,EAAExB,IAAI;WAAEyB,IAAI,CAAC;QAAOC,OAAOT,MAAMU,IAAI,CAAC,SAACH;mBAAMA,EAAEE,KAAK;;IAAE;AAC3I;AAEA,IAAME,iBAAiBnC,eAAe;AACtC,IAAMoC,qBAAqB,IAAIC,KAAKC,SAAS,CAACb,WAAW;IAAEc,aAAa;AAAW;AACnF,IAAMC,iBAAiB,IAAIH,KAAKC,SAAS,CAACb,WAAW;IAAEc,aAAa;AAAO;AAE3E,SAASE,WAAWlC,IAAY,EAAEmC,SAAyB;IACzD,OAAOC,MAAMC,IAAI,CAACF,UAAUG,OAAO,CAACtC,OAAO,SAACuC;eAAMA,EAAED,OAAO;;AAC7D;AAEA,gGAAgG;AAChG,2FAA2F;AAC3F,SAASE,KAAKC,QAAkB,EAAEC,OAAe;IAC/C,IAAMC,SAAmB,EAAE;IAC3B,IAAIhC,UAAU;IACd,IAAIiC,SAAS;QACR,kCAAA,2BAAA;;QAAL,QAAK,YAAiBH,6BAAjB,SAAA,6BAAA,QAAA,yBAAA,iCAA2B;YAA3B,IAAMH,UAAN;YACH,IAAMO,gBAAgBpD,eAAe6C;YACrC,IAAI3B,QAAQG,MAAM,GAAG,KAAK8B,SAASC,gBAAgBH,SAAS;gBAC1DC,OAAO5B,IAAI,CAACJ;gBACZA,UAAU;gBACViC,SAAS;YACX;YACAjC,WAAW2B;YACXM,UAAUC;QACZ;;QATK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAUL,IAAIlC,QAAQG,MAAM,GAAG,GAAG6B,OAAO5B,IAAI,CAACJ;IACpC,OAAOgC;AACT;AAEA,8FAA8F;AAC9F,0GAA0G;AAC1G,SAASG,cAAc9C,IAAY,EAAE0C,OAAe;IAClD,IAAMK,YAAYb,WAAWlC,MAAM6B;IACnC,IAAMmB,MAAgB,EAAE;IACxB,IAAIrC,UAAU;IACd,IAAIiC,SAAS;IACb,IAAMK,QAAQ;QACZ,IAAItC,QAAQG,MAAM,GAAG,GAAG;YACtBkC,IAAIjC,IAAI,CAACJ;YACTA,UAAU;YACViC,SAAS;QACX;IACF;QACK,kCAAA,2BAAA;;QAAL,QAAK,YAAkBG,8BAAlB,SAAA,6BAAA,QAAA,yBAAA,iCAA6B;YAA7B,IAAMG,WAAN;YACH,IAAMC,iBAAiB1D,eAAeyD;YACtC,IAAIC,iBAAiBT,SAAS;oBAE5BM;gBADAC;gBACAD,CAAAA,OAAAA,KAAIjC,IAAI,OAARiC,MAAS,qBAAGR,KAAKN,WAAWgB,UAAUjB,iBAAiBS;gBACvD;YACF;YACA,IAAI/B,QAAQG,MAAM,GAAG,KAAK8B,SAASO,iBAAiBT,SAASO;YAC7DtC,WAAWuC;YACXN,UAAUO;QACZ;;QAVK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IAWLF;IACA,OAAOD;AACT;AAEA,IAAMI,eAAuC,IAAIC,IAAI;IAAC;IAAQ;IAAS;CAAO;AAE9E,iGAAiG;AACjG,2EAA2E;AAC3E,SAASC,MAAMC,UAAoB,EAAElC,SAAiB,EAAEC,OAAe,EAAEkC,SAAoB,EAAEC,QAA6B;IAC1H,IAAMzD,OACJoD,aAAaM,GAAG,CAACF,cAAcC,aAAa,QACxCF,WAAW9B,IAAI,CAAC,QAChBkC,IAAAA,cAAK,EAACJ,WAAW9B,IAAI,CAAC,OACnBF,GAAG,CAAC,SAACqC;eAAMC,IAAAA,sBAAW,EAACD,EAAEE,IAAI;OAC7BrC,IAAI,CAAC;IACd,OAAO;QAAEJ,WAAAA;QAAWC,SAAAA;QAAStB,MAAAA;IAAK;AACpC;AAEA,kGAAkG;AAClG,yGAAyG;AACzG,SAAS+D,cAAcR,UAAoB,EAAElC,SAAiB,EAAEC,OAAe,EAAEoB,OAAe,EAAEc,SAAoB,EAAEC,QAA6B;IACnJ,IAAMjC,IAAI8B,MAAMC,YAAYlC,WAAWC,SAASkC,WAAWC;IAC3D,IAAIF,WAAWzC,MAAM,KAAK,KAAKrB,eAAe+B,EAAExB,IAAI,IAAI0C,SAAS;QAC/D,OAAOI,cAActB,EAAExB,IAAI,EAAE0C,SAASnB,GAAG,CAAC,SAACvB;mBAAU;gBAAEqB,WAAAA;gBAAWC,SAAAA;gBAAStB,MAAAA;gBAAM0B,OAAO;YAAK;;IAC/F;IACA,OAAO;QAACF;KAAE;AACZ;AAEA,iGAAiG;AACjG,0GAA0G;AAC1G,SAASwC,mBAAmBC,KAAe,EAAE5C,SAAiB,EAAEC,OAAe,EAAEoB,OAAe,EAAEc,SAAoB,EAAEC,QAA6B;QAAES,OAAAA,iEAAO;QAqBjIC;IApB3B,IAAMA,SAAiB,EAAE;IACzB,IAAIZ,aAAuB,EAAE;IAC7B,IAAIa,aAAa/C;IACjB,IAAIuB,SAASsB;IACb,IAAK,IAAIG,OAAOhD,WAAWgD,QAAQ/C,SAAS+C,OAAQ;QAClD,IAAMC,WAAWL,KAAK,CAACI,OAAO,EAAE;QAChC,IAAME,MAAMhB,WAAWzC,MAAM,GAAG,KAAK8B,SAAS,IAAIhB,iBAAiB;QACnE,IAAM4C,aAAa/E,eAAe6E;QAClC,IAAIf,WAAWzC,MAAM,GAAG,KAAK8B,SAAS2B,MAAMC,aAAa9B,SAAS;gBAChEyB;YAAAA,CAAAA,WAAAA,QAAOpD,IAAI,OAAXoD,UAAY,qBAAGJ,cAAcR,YAAYa,YAAYC,OAAO,GAAG3B,SAASc,WAAWC;YACnFF,aAAa,EAAE;YACfX,SAAS;YACTwB,aAAaC;YACbd,WAAWxC,IAAI,CAACuD;YAChB1B,UAAU4B;YACV;QACF;QACAjB,WAAWxC,IAAI,CAACuD;QAChB1B,UAAU2B,MAAMC;IAClB;IACA,IAAIjB,WAAWzC,MAAM,GAAG,GAAGqD,CAAAA,UAAAA,QAAOpD,IAAI,OAAXoD,SAAY,qBAAGJ,cAAcR,YAAYa,YAAY9C,SAASoB,SAASc,WAAWC;IAC7G,OAAOU;AACT;AAEA,0FAA0F;AAC1F,8FAA8F;AAC9F,SAASM,WAAWC,WAAoB,EAAET,KAAe,EAAEU,QAAyB;IAClF,IAAMjC,UAAUiC,SAASpE,YAAY;IACrC,IAAMqE,UAAUlC,UAAU9C;IAC1B,IAAMiF,WAA4C,EAAE;IACpD,IAAI5D,QAAgB,EAAE;IACtB,IAAI6D,iBAAiB;IACrB,IAAIlC,SAAS;IAEb,SAASmC;QACP,IAAMrF,QAAQsB,SAASC;QACvB,IAAIvB,OAAOmF,SAAS9D,IAAI,CAACrB;QACzBuB,QAAQ,EAAE;QACV6D,iBAAiB;QACjBlC,SAAS;IACX;IAEA,mFAAmF;IACnF,2FAA2F;IAC3F,SAASoC,QAAQhF,IAAY,EAAEiF,QAAgB,EAAE5D,SAAiB,EAAEC,OAAe;QACjFsB,UAAU,AAAC3B,CAAAA,MAAMH,MAAM,GAAG,IAAIc,iBAAiB,CAAA,IAAKnC,eAAewF;QACnEhE,MAAMF,IAAI,CAAC;YAAEM,WAAAA;YAAWC,SAAAA;YAAStB,MAAAA;QAAK;IACxC;QAEK,kCAAA,2BAAA;;QAAL,QAAK,YAAe0E,gCAAf,SAAA,6BAAA,QAAA,yBAAA,iCAA4B;YAA5B,IAAM9D,QAAN;YACH,IAAMsE,MAAMjB,MAAMkB,KAAK,CAACvE,MAAMS,SAAS,GAAG,GAAGT,MAAMU,OAAO,EAAEG,IAAI,CAAC;YACjE,IAAM2D,cAAc3F,eAAeyF;YAEnC,IAAIE,cAAcR,SAAS;gBACzB,IAAMV,OAAOjD,MAAMH,MAAM,GAAG,IAAI8B,SAAS;oBACpC,mCAAA,4BAAA;;oBAAL,QAAK,aAAWoB,mBAAmBC,OAAOrD,MAAMS,SAAS,EAAET,MAAMU,OAAO,EAAEoB,SAAS9B,MAAMC,IAAI,EAAE8D,SAAS3E,IAAI,EAAEkE,0BAAzG,UAAA,8BAAA,SAAA,0BAAA,kCAAgH;wBAAhH,IAAM1C,IAAN;wBACHP,MAAMF,IAAI,CAACS;wBACXuD;oBACF;;oBAHK;oBAAA;;;6BAAA,8BAAA;4BAAA;;;4BAAA;kCAAA;;;;gBAIL;YACF;YAEA,IAAMM,YAAYxB,IAAAA,sBAAW,EAACjD,MAAMkD,IAAI;YACxC,IAAMmB,WAAWN,SAAS3E,IAAI,KAAK,QAAQkF,MAAMG;YAEjD,IAAIzE,MAAMC,IAAI,KAAK,WAAW;gBAC5BmE,QAAQK,WAAWJ,UAAUrE,MAAMS,SAAS,EAAET,MAAMU,OAAO;gBAC3D;YACF;YAEA,wFAAwF;YACxF,wDAAwD;YACxD,IAAMgE,eAAerE,MAAMH,MAAM,GAAG,KAAK8B,SAAShB,iBAAiBwD,cAAcR;YACjF,IAAIU,cAAcP;YAElBC,QAAQK,WAAWJ,UAAUrE,MAAMS,SAAS,EAAET,MAAMU,OAAO;YAC3DwD;YAEA,IAAIA,kBAAkBnF,gBAAgBoF;QACxC;;QA9BK;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IA+BLA;IAEA,OAAOF;AACT;AAIO,SAASnF,MAAMe,MAAe,EAAE8E,IAAY,EAAEjF,IAAmB;QAI7BkF;IAHzC,IAAMb,WAAWtE,eAAeC;IAChC,IAAM2D,QAAQsB,KAAKE,KAAK,CAAC;IACzB,IAAMD,SAA0C,EAAE;QAC7C,kCAAA,2BAAA;;QAAL,QAAK,YAAehF,YAAYC,4BAA3B,SAAA,6BAAA,QAAA,yBAAA;YAAA,IAAMiF,QAAN;YAAoCF,CAAAA,UAAAA,QAAOzE,IAAI,OAAXyE,SAAY,qBAAGf,WAAWiB,OAAOzB,OAAOU;;;QAA5E;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;IACL,+FAA+F;IAC/F,qHAAqH;IACrH,IAAMgB,SACJhB,SAAS3E,IAAI,KAAK,QACdwF,OAAOjE,GAAG,CAAC,SAACqE;eACVA,EAAElE,KAAK,GACHkE,IACA,wCACKA;YACH5F,MAAMiE,MACHkB,KAAK,CAACS,EAAEvE,SAAS,GAAG,GAAGuE,EAAEtE,OAAO,EAChCG,IAAI,CAAC,MACLoE,IAAI;;SAGfL;IACN,mHAAmH;IACnH,OAAOG,OAAOG,MAAM,CAAC,SAACF;eAAMA,EAAE5F,IAAI,CAAC6F,IAAI,GAAG/E,MAAM,GAAG;OAAGS,GAAG,CAAC,SAACqE;eAAO;YAAEvE,WAAWuE,EAAEvE,SAAS;YAAEC,SAASsE,EAAEtE,OAAO;YAAEtB,MAAM4F,EAAE5F,IAAI;QAAC;;AAC/H"}

package/dist/cjs/chunk/parse.js

@@ -28,11 +28,8 @@ var BLOCK_TYPES = {
     list: 'list',
     blockquote: 'blockquote'
 };
-// The five extensions extract.ts/group.ts actually consume, imported individually rather than
-// through micromark-extension-gfm/mdast-util-gfm: those bundles also pull in gfm-tagfilter,
-// an HTML-serialization sanitizer this library never uses (no htmlExtensions call anywhere).
-// micromark and mdast-util-from-markdown each already flatten an extensions array themselves,
-// so no combine-extensions helper is needed here.
+// Imported individually, not via micromark-extension-gfm/mdast-util-gfm: those bundles also pull
+// in gfm-tagfilter, an HTML sanitizer this library never uses (no htmlExtensions call anywhere).
 var EXTENSIONS = [
     (0, _micromarkextensiongfmautolinkliteral.gfmAutolinkLiteral)(),
     (0, _micromarkextensiongfmfootnote.gfmFootnote)(),

package/dist/cjs/chunk/parse.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/chunk/parse.ts"],"sourcesContent":["import type { RootContent } from 'mdast';\nimport { fromMarkdown } from 'mdast-util-from-markdown';\nimport { gfmAutolinkLiteralFromMarkdown } from 'mdast-util-gfm-autolink-literal';\nimport { gfmFootnoteFromMarkdown } from 'mdast-util-gfm-footnote';\nimport { gfmStrikethroughFromMarkdown } from 'mdast-util-gfm-strikethrough';\nimport { gfmTableFromMarkdown } from 'mdast-util-gfm-table';\nimport { gfmTaskListItemFromMarkdown } from 'mdast-util-gfm-task-list-item';\nimport { gfmAutolinkLiteral } from 'micromark-extension-gfm-autolink-literal';\nimport { gfmFootnote } from 'micromark-extension-gfm-footnote';\nimport { gfmStrikethrough } from 'micromark-extension-gfm-strikethrough';\nimport { gfmTable } from 'micromark-extension-gfm-table';\nimport { gfmTaskListItem } from 'micromark-extension-gfm-task-list-item';\nimport { extractText } from './extract.ts';\nimport type { Block, BlockType } from './types.ts';\n\nconst BLOCK_TYPES: Partial<Record<RootContent['type'], BlockType>> = {\n  heading: 'heading',\n  paragraph: 'paragraph',\n  code: 'code',\n  table: 'table',\n  list: 'list',\n  blockquote: 'blockquote',\n};\n\n// The five extensions extract.ts/group.ts actually consume, imported individually rather than\n// through micromark-extension-gfm/mdast-util-gfm: those bundles also pull in gfm-tagfilter,\n// an HTML-serialization sanitizer this library never uses (no htmlExtensions call anywhere).\n// micromark and mdast-util-from-markdown each already flatten an extensions array themselves,\n// so no combine-extensions helper is needed here.\nconst EXTENSIONS = [gfmAutolinkLiteral(), gfmFootnote(), gfmStrikethrough(), gfmTable(), gfmTaskListItem()];\nconst MDAST_EXTENSIONS = [gfmAutolinkLiteralFromMarkdown(), gfmFootnoteFromMarkdown(), gfmStrikethroughFromMarkdown(), gfmTableFromMarkdown(), gfmTaskListItemFromMarkdown()];\n\n// Top-level blocks of a markdown body, typed and line-extent bounded. Extents come from\n// mdast's own node.position, never a regex guess at where a block starts or ends; GFM adds\n// tables, task lists, footnotes and strikethrough to the CommonMark set fromMarkdown parses.\nexport function parse(body: string): Block[] {\n  const tree = fromMarkdown(body, { extensions: EXTENSIONS, mdastExtensions: MDAST_EXTENSIONS });\n  return tree.children.map((node) => {\n    const position = node.position;\n    const block: Block = {\n      type: BLOCK_TYPES[node.type] ?? 'other',\n      startLine: position ? position.start.line : 1,\n      endLine: position ? position.end.line : 1,\n      node,\n    };\n    if (node.type === 'heading') {\n      block.depth = node.depth;\n      block.text = extractText(node);\n    }\n    return block;\n  });\n}\n"],"names":["parse","BLOCK_TYPES","heading","paragraph","code","table","list","blockquote","EXTENSIONS","gfmAutolinkLiteral","gfmFootnote","gfmStrikethrough","gfmTable","gfmTaskListItem","MDAST_EXTENSIONS","gfmAutolinkLiteralFromMarkdown","gfmFootnoteFromMarkdown","gfmStrikethroughFromMarkdown","gfmTableFromMarkdown","gfmTaskListItemFromMarkdown","body","tree","fromMarkdown","extensions","mdastExtensions","children","map","node","position","block","type","startLine","start","line","endLine","end","depth","text","extractText"],"mappings":";;;;+BAmCgBA;;;eAAAA;;;qCAlCa;2CACkB;oCACP;yCACK;iCACR;wCACO;oDACT;6CACP;kDACK;0CACR;iDACO;yBACJ;AAG5B,IAAMC,cAA+D;IACnEC,SAAS;IACTC,WAAW;IACXC,MAAM;IACNC,OAAO;IACPC,MAAM;IACNC,YAAY;AACd;AAEA,8FAA8F;AAC9F,4FAA4F;AAC5F,6FAA6F;AAC7F,8FAA8F;AAC9F,kDAAkD;AAClD,IAAMC,aAAa;IAACC,IAAAA,wDAAkB;IAAIC,IAAAA,0CAAW;IAAIC,IAAAA,oDAAgB;IAAIC,IAAAA,oCAAQ;IAAIC,IAAAA,kDAAe;CAAG;AAC3G,IAAMC,mBAAmB;IAACC,IAAAA,2DAA8B;IAAIC,IAAAA,6CAAuB;IAAIC,IAAAA,uDAA4B;IAAIC,IAAAA,uCAAoB;IAAIC,IAAAA,qDAA2B;CAAG;AAKtK,SAASnB,MAAMoB,IAAY;IAChC,IAAMC,OAAOC,IAAAA,mCAAY,EAACF,MAAM;QAAEG,YAAYf;QAAYgB,iBAAiBV;IAAiB;IAC5F,OAAOO,KAAKI,QAAQ,CAACC,GAAG,CAAC,SAACC;YAGhB1B;QAFR,IAAM2B,WAAWD,KAAKC,QAAQ;QAC9B,IAAMC,QAAe;YACnBC,IAAI,GAAE7B,yBAAAA,WAAW,CAAC0B,KAAKG,IAAI,CAAC,cAAtB7B,oCAAAA,yBAA0B;YAChC8B,WAAWH,WAAWA,SAASI,KAAK,CAACC,IAAI,GAAG;YAC5CC,SAASN,WAAWA,SAASO,GAAG,CAACF,IAAI,GAAG;YACxCN,MAAAA;QACF;QACA,IAAIA,KAAKG,IAAI,KAAK,WAAW;YAC3BD,MAAMO,KAAK,GAAGT,KAAKS,KAAK;YACxBP,MAAMQ,IAAI,GAAGC,IAAAA,sBAAW,EAACX;QAC3B;QACA,OAAOE;IACT;AACF"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/chunk/parse.ts"],"sourcesContent":["import type { RootContent } from 'mdast';\nimport { fromMarkdown } from 'mdast-util-from-markdown';\nimport { gfmAutolinkLiteralFromMarkdown } from 'mdast-util-gfm-autolink-literal';\nimport { gfmFootnoteFromMarkdown } from 'mdast-util-gfm-footnote';\nimport { gfmStrikethroughFromMarkdown } from 'mdast-util-gfm-strikethrough';\nimport { gfmTableFromMarkdown } from 'mdast-util-gfm-table';\nimport { gfmTaskListItemFromMarkdown } from 'mdast-util-gfm-task-list-item';\nimport { gfmAutolinkLiteral } from 'micromark-extension-gfm-autolink-literal';\nimport { gfmFootnote } from 'micromark-extension-gfm-footnote';\nimport { gfmStrikethrough } from 'micromark-extension-gfm-strikethrough';\nimport { gfmTable } from 'micromark-extension-gfm-table';\nimport { gfmTaskListItem } from 'micromark-extension-gfm-task-list-item';\nimport { extractText } from './extract.ts';\nimport type { Block, BlockType } from './types.ts';\n\nconst BLOCK_TYPES: Partial<Record<RootContent['type'], BlockType>> = {\n  heading: 'heading',\n  paragraph: 'paragraph',\n  code: 'code',\n  table: 'table',\n  list: 'list',\n  blockquote: 'blockquote',\n};\n\n// Imported individually, not via micromark-extension-gfm/mdast-util-gfm: those bundles also pull\n// in gfm-tagfilter, an HTML sanitizer this library never uses (no htmlExtensions call anywhere).\nconst EXTENSIONS = [gfmAutolinkLiteral(), gfmFootnote(), gfmStrikethrough(), gfmTable(), gfmTaskListItem()];\nconst MDAST_EXTENSIONS = [gfmAutolinkLiteralFromMarkdown(), gfmFootnoteFromMarkdown(), gfmStrikethroughFromMarkdown(), gfmTableFromMarkdown(), gfmTaskListItemFromMarkdown()];\n\n// Top-level blocks of a markdown body, typed and line-extent bounded from mdast's own\n// node.position (never a regex guess). GFM extensions add tables, task lists, footnotes, strikethrough.\nexport function parse(body: string): Block[] {\n  const tree = fromMarkdown(body, { extensions: EXTENSIONS, mdastExtensions: MDAST_EXTENSIONS });\n  return tree.children.map((node) => {\n    const position = node.position;\n    const block: Block = {\n      type: BLOCK_TYPES[node.type] ?? 'other',\n      startLine: position ? position.start.line : 1,\n      endLine: position ? position.end.line : 1,\n      node,\n    };\n    if (node.type === 'heading') {\n      block.depth = node.depth;\n      block.text = extractText(node);\n    }\n    return block;\n  });\n}\n"],"names":["parse","BLOCK_TYPES","heading","paragraph","code","table","list","blockquote","EXTENSIONS","gfmAutolinkLiteral","gfmFootnote","gfmStrikethrough","gfmTable","gfmTaskListItem","MDAST_EXTENSIONS","gfmAutolinkLiteralFromMarkdown","gfmFootnoteFromMarkdown","gfmStrikethroughFromMarkdown","gfmTableFromMarkdown","gfmTaskListItemFromMarkdown","body","tree","fromMarkdown","extensions","mdastExtensions","children","map","node","position","block","type","startLine","start","line","endLine","end","depth","text","extractText"],"mappings":";;;;+BA+BgBA;;;eAAAA;;;qCA9Ba;2CACkB;oCACP;yCACK;iCACR;wCACO;oDACT;6CACP;kDACK;0CACR;iDACO;yBACJ;AAG5B,IAAMC,cAA+D;IACnEC,SAAS;IACTC,WAAW;IACXC,MAAM;IACNC,OAAO;IACPC,MAAM;IACNC,YAAY;AACd;AAEA,iGAAiG;AACjG,iGAAiG;AACjG,IAAMC,aAAa;IAACC,IAAAA,wDAAkB;IAAIC,IAAAA,0CAAW;IAAIC,IAAAA,oDAAgB;IAAIC,IAAAA,oCAAQ;IAAIC,IAAAA,kDAAe;CAAG;AAC3G,IAAMC,mBAAmB;IAACC,IAAAA,2DAA8B;IAAIC,IAAAA,6CAAuB;IAAIC,IAAAA,uDAA4B;IAAIC,IAAAA,uCAAoB;IAAIC,IAAAA,qDAA2B;CAAG;AAItK,SAASnB,MAAMoB,IAAY;IAChC,IAAMC,OAAOC,IAAAA,mCAAY,EAACF,MAAM;QAAEG,YAAYf;QAAYgB,iBAAiBV;IAAiB;IAC5F,OAAOO,KAAKI,QAAQ,CAACC,GAAG,CAAC,SAACC;YAGhB1B;QAFR,IAAM2B,WAAWD,KAAKC,QAAQ;QAC9B,IAAMC,QAAe;YACnBC,IAAI,GAAE7B,yBAAAA,WAAW,CAAC0B,KAAKG,IAAI,CAAC,cAAtB7B,oCAAAA,yBAA0B;YAChC8B,WAAWH,WAAWA,SAASI,KAAK,CAACC,IAAI,GAAG;YAC5CC,SAASN,WAAWA,SAASO,GAAG,CAACF,IAAI,GAAG;YACxCN,MAAAA;QACF;QACA,IAAIA,KAAKG,IAAI,KAAK,WAAW;YAC3BD,MAAMO,KAAK,GAAGT,KAAKS,KAAK;YACxBP,MAAMQ,IAAI,GAAGC,IAAAA,sBAAW,EAACX;QAC3B;QACA,OAAOE;IACT;AACF"}

package/dist/cjs/cli/index.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/index.ts"],"sourcesContent":["import type { Command } from './types.ts';\n\n// Each command's usage line, minus the leading \"usage: <name> \" -- shared by cli.ts's\n// top-level usage() and the command's own parse() error/--help output, so there is one\n// copy of each line, not two that can drift.\nexport const USAGE = {\n  init: 'init [--model id-or-path] [--provider static|openai|cohere] [--url url]',\n  watch: 'watch [--force] [--config path]',\n  status: 'status [--format table|json] [--config path]',\n  download: 'download [--config path]',\n  sql: 'sql \"<statement>\" [params...] [--preset name] [--format table|json|csv] [--config path]',\n  search: 'search \"<terms>\" [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n  map: 'map [--format table|json] [--config path]',\n  peek: 'peek <path> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--format table|json] [--config path]',\n  path: 'path <a> <b> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--max-depth n] [--format table|json] [--config path]',\n  related: 'related <note> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n} as const;\n\n// Lazy registry: a command's imports load only when it runs, so a heavy dependency in one\n// command never taxes the others (or --version/--help). These names are reserved: a saved\n// entry in config can never shadow them.\nexport const COMMANDS: Record<string, () => Promise<{ default: Command }>> = {\n  init: () => import('./init.ts'),\n  watch: () => import('./watch.ts'),\n  status: () => import('./status.ts'),\n  download: () => import('./download.ts'),\n  sql: () => import('./sql.ts'),\n  search: () => import('./search.ts'),\n  map: () => import('./map.ts'),\n  peek: () => import('./peek.ts'),\n  path: () => import('./path.ts'),\n  related: () => import('./related.ts'),\n};\n\nexport type { Command, Ctx } from './types.ts';\n"],"names":["COMMANDS","USAGE","init","watch","status","download","sql","search","map","peek","path","related"],"mappings":";;;;;;;;;;;QAqBaA;eAAAA;;QAhBAC;eAAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAN,IAAMA,QAAQ;IACnBC,MAAM;IACNC,OAAO;IACPC,QAAQ;IACRC,UAAU;IACVC,KAAK;IACLC,QAAQ;IACRC,KAAK;IACLC,MAAM;IACNC,MAAM;IACNC,SAAS;AACX;AAKO,IAAMX,WAAgE;IAC3EE,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,OAAO,SAAPA;eAAa;2DAAA,QAAO;;;IACpBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,UAAU,SAAVA;eAAgB;2DAAA,QAAO;;;IACvBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,SAAS,SAATA;eAAe;2DAAA,QAAO;;;AACxB"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/index.ts"],"sourcesContent":["import type { Command } from './types.ts';\n\n// Each command's usage line, minus the leading \"usage: <name> \": shared by cli.ts's\n// usage() and the command's own parse() error/--help output so there is one copy, not two.\nexport const USAGE = {\n  init: 'init [--model id-or-path] [--provider static|openai|cohere] [--url url]',\n  watch: 'watch [--force] [--config path]',\n  status: 'status [--format table|json] [--config path]',\n  download: 'download [--config path]',\n  sql: 'sql \"<statement>\" [params...] [--preset name] [--format table|json|csv] [--config path]',\n  search: 'search \"<terms>\" [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n  map: 'map [--format table|json] [--config path]',\n  peek: 'peek <path> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--format table|json] [--config path]',\n  path: 'path <a> <b> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--max-depth n] [--format table|json] [--config path]',\n  related: 'related <note> [--preset name] [--include glob ...] [--exclude glob ...] [--no-exclude] [--where \"<sql>\"] [--k n] [--format table|json|csv] [--config path]',\n} as const;\n\n// Lazy registry: a command's imports load only when it runs, so a heavy dependency in one\n// command never taxes the others (or --version/--help). These names are reserved from saved queries.\nexport const COMMANDS: Record<string, () => Promise<{ default: Command }>> = {\n  init: () => import('./init.ts'),\n  watch: () => import('./watch.ts'),\n  status: () => import('./status.ts'),\n  download: () => import('./download.ts'),\n  sql: () => import('./sql.ts'),\n  search: () => import('./search.ts'),\n  map: () => import('./map.ts'),\n  peek: () => import('./peek.ts'),\n  path: () => import('./path.ts'),\n  related: () => import('./related.ts'),\n};\n\nexport type { Command, Ctx } from './types.ts';\n"],"names":["COMMANDS","USAGE","init","watch","status","download","sql","search","map","peek","path","related"],"mappings":";;;;;;;;;;;QAmBaA;eAAAA;;QAfAC;eAAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAN,IAAMA,QAAQ;IACnBC,MAAM;IACNC,OAAO;IACPC,QAAQ;IACRC,UAAU;IACVC,KAAK;IACLC,QAAQ;IACRC,KAAK;IACLC,MAAM;IACNC,MAAM;IACNC,SAAS;AACX;AAIO,IAAMX,WAAgE;IAC3EE,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,OAAO,SAAPA;eAAa;2DAAA,QAAO;;;IACpBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,UAAU,SAAVA;eAAgB;2DAAA,QAAO;;;IACvBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,QAAQ,SAARA;eAAc;2DAAA,QAAO;;;IACrBC,KAAK,SAALA;eAAW;2DAAA,QAAO;;;IAClBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,MAAM,SAANA;eAAY;2DAAA,QAAO;;;IACnBC,SAAS,SAATA;eAAe;2DAAA,QAAO;;;AACxB"}

package/dist/cjs/cli/shared.js

@@ -300,8 +300,8 @@ function scopeOf(values) {
         noExclude: values['no-exclude'] === true
     };
 }
-// An unrecognised value used to fall through to `table`, so a typo looked like it worked.
-// parseArgs is strict about flag names; this is the same strictness for the value.
+// parseArgs is strict about flag names; this applies the same strictness to the value.
+// An unrecognised --format is a usage error, not a silent fallback to `table`.
 function pickFormat(values, allowed) {
     var format = String(values.format);
     if (allowed.includes(format)) return format;
@@ -430,11 +430,8 @@ function withDb(ctx, configPath, fn) {
         });
     })();
 }
-// `--preset` on SQL binds the scope rather than applying it: the statement joins `scope`
-// itself. Filtering behind the query's back is not available -- the obvious version, temp views
-// shadowing the base tables, cannot cover `content`, because FTS5 `MATCH` uses the table name
-// as a hidden column and a view has none. A flag that silently scoped three tables of four
-// would look scoped and not be.
+// `--preset` on SQL binds a temp `scope` table rather than transparently filtering: temp views shadowing the base tables can't cover `content`, since FTS5 MATCH treats the table name as a hidden column.
+// A flag that silently scoped three tables of four would look scoped and not be.
 function bindScope(store, cfg, preset) {
     return _async_to_generator(function() {
         var name, insert;
@@ -552,9 +549,8 @@ function runSql(cfg, sql, params, format, label, preset) {
                     ];
                 case 8:
                     _state.sent();
-                    // A saved query or ad-hoc SQL can carry `content MATCH ?` too, so the same FTS5
-                    // punctuation trap applies -- the bound parameters are the search terms there. SQL
-                    // without MATCH gets its error verbatim; search advice on a plain typo would mislead.
+                    // A saved query or ad-hoc SQL can carry `content MATCH ?` too, so the same FTS5 punctuation trap applies to its bound parameters.
+                    // SQL without MATCH gets its error verbatim; search advice on a plain typo would mislead.
                     if (/\bMATCH\b/i.test(sql)) throw (0, _searcherrorts.searchError)(hinted, params.join(' '));
                     throw hinted;
                 case 9:

package/dist/cjs/cli/shared.js.map

@@ -1 +1 @@
-{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/shared.ts"],"sourcesContent":["import type { ParseArgsOptionsConfig } from 'node:util';\nimport { parseArgs } from 'node:util';\nimport type { ResolvedConfig, SearchOverrides } from '../config/index.ts';\nimport { resolvePreset } from '../config/index.ts';\nimport { columnHint } from '../output/column-hint.ts';\nimport type { Row, RowFormat } from '../output/output.ts';\nimport { printRowStream } from '../output/output.ts';\nimport { searchError } from '../output/search-error.ts';\nimport type { OpenResult } from '../store/index.ts';\nimport { openStore } from '../store/index.ts';\nimport type { Store } from '../store/types.ts';\nimport type { Ctx } from './types.ts';\n\n// Spreadable option fragments -- one flag name keeps one meaning across every command's table.\nexport const FORMAT: ParseArgsOptionsConfig = { format: { type: 'string', default: 'table' } };\nexport const CONFIG: ParseArgsOptionsConfig = { config: { type: 'string' } };\n// The scope vocabulary every scoped command shares: named preset, ad hoc include/exclude\n// globs, a where SQL condition. search adds k on top.\nexport const SCOPE: ParseArgsOptionsConfig = {\n  where: { type: 'string' },\n  preset: { type: 'string' },\n  include: { type: 'string', multiple: true },\n  exclude: { type: 'string', multiple: true },\n  // Widening, which no other flag can express: --include and --exclude each override their\n  // own side of the preset, so neither can drop an exclusion the preset declares.\n  'no-exclude': { type: 'boolean', default: false },\n};\nexport const SEARCH_FLAGS: ParseArgsOptionsConfig = { ...SCOPE, k: { type: 'string' } };\n\ntype Values = Record<string, string | boolean | string[] | undefined>;\n\n// The SCOPE fragment's parsed values as the overrides every scoped command passes down. One\n// place to read them, so a new scope field is added to the fragment and here, not in each\n// command's call.\nexport function scopeOf(values: Values): SearchOverrides {\n  return {\n    preset: values.preset as string | undefined,\n    include: values.include as string[] | undefined,\n    exclude: values.exclude as string[] | undefined,\n    where: values.where as string | undefined,\n    noExclude: values['no-exclude'] === true,\n  };\n}\n\n// An unrecognised value used to fall through to `table`, so a typo looked like it worked.\n// parseArgs is strict about flag names; this is the same strictness for the value.\nfunction pickFormat<T extends string>(values: Values, allowed: readonly T[]): T {\n  const format = String(values.format);\n  if ((allowed as readonly string[]).includes(format)) return format as T;\n  console.error(`unknown --format \"${format}\"; expected ${allowed.join(', ')}`);\n  process.exit(2);\n}\n\nexport function formatOf(values: Values): 'table' | 'json' {\n  return pickFormat(values, ['table', 'json'] as const);\n}\n\n// csv is a row set rendered as rows; map, peek, status, and path render structures instead,\n// and take formatOf.\nexport function rowFormatOf(values: Values): RowFormat {\n  return pickFormat(values, ['table', 'json', 'csv'] as const);\n}\n\n// Per-command parseArgs: strict (a foreign flag exits 2), and every table gets --help for free.\nexport function parse(argv: string[], usage: string, options: ParseArgsOptionsConfig): { values: Values; positionals: string[] } {\n  let values: Values;\n  let positionals: string[];\n  try {\n    ({ values, positionals } = parseArgs({\n      args: argv,\n      options: { ...options, help: { type: 'boolean', default: false, short: 'h' } },\n      strict: true,\n      allowPositionals: true,\n    }));\n  } catch (err) {\n    console.error((err as Error).message);\n    console.error(usage);\n    process.exit(2);\n  }\n  if (values.help) {\n    console.log(usage);\n    process.exit(0);\n  }\n  return { values, positionals };\n}\n\n// --k must be a positive integer: SQLite reads a bound LIMIT of -1 as \"unlimited\" and 0 as\n// \"nothing\", and parseInt would silently truncate \"5.9\" -- all three are caller mistakes\n// worth a usage error, matching the config-level SavedSearch validation.\nexport function parseK(k: string | undefined, usageError: (message: string) => never): number | undefined {\n  if (k === undefined) return undefined;\n  const parsed = Number(k);\n  if (!Number.isInteger(parsed) || parsed <= 0) usageError(`--k expects a positive integer, got \"${k}\"`);\n  return parsed;\n}\n\n// Shared open-query-close envelope for commands that touch the tree.\n\nexport function printWarnings(warnings: string[]): void {\n  for (const w of warnings) console.warn(w);\n}\n\nexport async function withDb(ctx: Ctx, configPath: string | undefined, fn: (store: OpenResult['store'], cfg: ResolvedConfig) => void | Promise<void>): Promise<void> {\n  const cfg = ctx.resolveConfig(configPath);\n  const { store, warnings } = await openStore(cfg);\n  printWarnings(warnings);\n  try {\n    await fn(store, cfg);\n  } finally {\n    await store.close();\n  }\n}\n\n// `--preset` on SQL binds the scope rather than applying it: the statement joins `scope`\n// itself. Filtering behind the query's back is not available -- the obvious version, temp views\n// shadowing the base tables, cannot cover `content`, because FTS5 `MATCH` uses the table name\n// as a hidden column and a view has none. A flag that silently scoped three tables of four\n// would look scoped and not be.\nasync function bindScope(store: Store, cfg: ResolvedConfig, preset: string): Promise<void> {\n  const { name } = resolvePreset(cfg, preset); // unknown names throw, listing what is declared\n  await store.exec('DROP TABLE IF EXISTS temp.scope');\n  await store.exec('CREATE TEMP TABLE scope (\"path\" TEXT PRIMARY KEY)');\n  const insert = await store.prepare('INSERT INTO temp.scope (\"path\") SELECT \"path\" FROM preset_files WHERE preset = ?');\n  await insert.run(name);\n}\n\n// An unbound `?` silently binds NULL, so mismatched param counts fail loudly instead.\nexport async function runSql(cfg: ResolvedConfig, sql: string, params: string[], format: RowFormat, label: string, preset?: string): Promise<void> {\n  const placeholderCount = (sql.match(/\\?/g) ?? []).length;\n  if (params.length !== placeholderCount) {\n    console.error(`${label} expects ${placeholderCount} parameter(s), got ${params.length}`);\n    process.exit(2);\n  }\n  // Naming a preset and never joining it would return the whole index while reading as scoped,\n  // which is the one hazard of binding rather than applying. Refuse instead.\n  if (preset !== undefined && !/\\bscope\\b/i.test(sql)) {\n    console.error(`--preset binds a temporary \"scope\" table of the preset's paths, and ${label} never joins it, so the preset would have no effect`);\n    console.error(`add: JOIN scope ON scope.\"path\" = <table>.\"path\"`);\n    process.exit(2);\n  }\n  const { store, warnings } = await openStore(cfg);\n  printWarnings(warnings);\n  if (preset !== undefined) await bindScope(store, cfg, preset);\n  // Streamed rather than collected: `sql` is the one caller whose result size is the\n  // statement's business, not this package's, so the rows are never held here in bulk.\n  // columns() reads the statement's own metadata, so a 0-row csv still has its header.\n  // A mid-stream SQLite error (e.g. SQLITE_BUSY outlasting busy_timeout) can still leave\n  // partial output; the nonzero exit code is the caller's signal, output completeness is not\n  // guaranteed on failure.\n  try {\n    const statement = await store.raw.prepare(sql);\n    const columns = statement.columns().map((c) => c.name);\n    await printRowStream(statement.iterate(...params) as AsyncIterable<Row>, format, columns);\n  } catch (err) {\n    const hinted = columnHint(await store.docs.columns(), err as Error); // columns read while the store is still open\n    await store.close();\n    // A saved query or ad-hoc SQL can carry `content MATCH ?` too, so the same FTS5\n    // punctuation trap applies -- the bound parameters are the search terms there. SQL\n    // without MATCH gets its error verbatim; search advice on a plain typo would mislead.\n    if (/\\bMATCH\\b/i.test(sql)) throw searchError(hinted, params.join(' '));\n    throw hinted;\n  }\n  await store.close();\n}\n"],"names":["CONFIG","FORMAT","SCOPE","SEARCH_FLAGS","formatOf","parse","parseK","printWarnings","rowFormatOf","runSql","scopeOf","withDb","format","type","default","config","where","preset","include","multiple","exclude","k","values","noExclude","pickFormat","allowed","String","includes","console","error","join","process","exit","argv","usage","options","positionals","parseArgs","args","help","short","strict","allowPositionals","err","message","log","usageError","undefined","parsed","Number","isInteger","warnings","w","warn","ctx","configPath","fn","cfg","store","resolveConfig","openStore","close","bindScope","name","insert","resolvePreset","exec","prepare","run","sql","params","label","placeholderCount","statement","columns","hinted","match","length","test","raw","map","c","printRowStream","iterate","docs","columnHint","searchError"],"mappings":";;;;;;;;;;;QAeaA;eAAAA;;QADAC;eAAAA;;QAIAC;eAAAA;;QASAC;eAAAA;;QA0BGC;eAAAA;;QAWAC;eAAAA;;QAyBAC;eAAAA;;QASAC;eAAAA;;QAvCAC;eAAAA;;QAoEMC;eAAAA;;QA7FNC;eAAAA;;QAoEMC;eAAAA;;;wBArGI;uBAEI;4BACH;wBAEI;6BACH;wBAEF;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAKnB,IAAMV,SAAiC;IAAEW,QAAQ;QAAEC,MAAM;QAAUC,SAAS;IAAQ;AAAE;AACtF,IAAMd,SAAiC;IAAEe,QAAQ;QAAEF,MAAM;IAAS;AAAE;AAGpE,IAAMX,QAAgC;IAC3Cc,OAAO;QAAEH,MAAM;IAAS;IACxBI,QAAQ;QAAEJ,MAAM;IAAS;IACzBK,SAAS;QAAEL,MAAM;QAAUM,UAAU;IAAK;IAC1CC,SAAS;QAAEP,MAAM;QAAUM,UAAU;IAAK;IAC1C,yFAAyF;IACzF,gFAAgF;IAChF,cAAc;QAAEN,MAAM;QAAWC,SAAS;IAAM;AAClD;AACO,IAAMX,eAAuC,wCAAKD;IAAOmB,GAAG;QAAER,MAAM;IAAS;;AAO7E,SAASH,QAAQY,MAAc;IACpC,OAAO;QACLL,QAAQK,OAAOL,MAAM;QACrBC,SAASI,OAAOJ,OAAO;QACvBE,SAASE,OAAOF,OAAO;QACvBJ,OAAOM,OAAON,KAAK;QACnBO,WAAWD,MAAM,CAAC,aAAa,KAAK;IACtC;AACF;AAEA,0FAA0F;AAC1F,mFAAmF;AACnF,SAASE,WAA6BF,MAAc,EAAEG,OAAqB;IACzE,IAAMb,SAASc,OAAOJ,OAAOV,MAAM;IACnC,IAAI,AAACa,QAA8BE,QAAQ,CAACf,SAAS,OAAOA;IAC5DgB,QAAQC,KAAK,CAAC,AAAC,qBAAyCJ,OAArBb,QAAO,gBAAiC,OAAnBa,QAAQK,IAAI,CAAC;IACrEC,QAAQC,IAAI,CAAC;AACf;AAEO,SAAS5B,SAASkB,MAAc;IACrC,OAAOE,WAAWF,QAAQ;QAAC;QAAS;KAAO;AAC7C;AAIO,SAASd,YAAYc,MAAc;IACxC,OAAOE,WAAWF,QAAQ;QAAC;QAAS;QAAQ;KAAM;AACpD;AAGO,SAASjB,MAAM4B,IAAc,EAAEC,KAAa,EAAEC,OAA+B;IAClF,IAAIb;IACJ,IAAIc;IACJ,IAAI;;cACyBC,IAAAA,mBAAS,EAAC;YACnCC,MAAML;YACNE,SAAS,wCAAKA;gBAASI,MAAM;oBAAE1B,MAAM;oBAAWC,SAAS;oBAAO0B,OAAO;gBAAI;;YAC3EC,QAAQ;YACRC,kBAAkB;QACpB,IALGpB,aAAAA,QAAQc,kBAAAA;IAMb,EAAE,OAAOO,KAAK;QACZf,QAAQC,KAAK,CAAC,AAACc,IAAcC,OAAO;QACpChB,QAAQC,KAAK,CAACK;QACdH,QAAQC,IAAI,CAAC;IACf;IACA,IAAIV,OAAOiB,IAAI,EAAE;QACfX,QAAQiB,GAAG,CAACX;QACZH,QAAQC,IAAI,CAAC;IACf;IACA,OAAO;QAAEV,QAAAA;QAAQc,aAAAA;IAAY;AAC/B;AAKO,SAAS9B,OAAOe,CAAqB,EAAEyB,UAAsC;IAClF,IAAIzB,MAAM0B,WAAW,OAAOA;IAC5B,IAAMC,SAASC,OAAO5B;IACtB,IAAI,CAAC4B,OAAOC,SAAS,CAACF,WAAWA,UAAU,GAAGF,WAAW,AAAC,wCAAyC,OAAFzB,GAAE;IACnG,OAAO2B;AACT;AAIO,SAASzC,cAAc4C,QAAkB;QACzC,kCAAA,2BAAA;;QAAL,QAAK,YAAWA,6BAAX,SAAA,6BAAA,QAAA,yBAAA;YAAA,IAAMC,IAAN;YAAqBxB,QAAQyB,IAAI,CAACD;;;QAAlC;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;AACP;AAEO,SAAezC,OAAO2C,GAAQ,EAAEC,UAA8B,EAAEC,EAA6E;;YAC5IC,KACsB,MAApBC,OAAOP;;;;oBADTM,MAAMH,IAAIK,aAAa,CAACJ;oBACF;;wBAAMK,IAAAA,mBAAS,EAACH;;;oBAAhB,OAAA,eAApBC,QAAoB,KAApBA,OAAOP,WAAa,KAAbA;oBACf5C,cAAc4C;;;;;;;;;oBAEZ;;wBAAMK,GAAGE,OAAOD;;;oBAAhB;;;;;;oBAEA;;wBAAMC,MAAMG,KAAK;;;oBAAjB;;;;;;;;;;IAEJ;;AAEA,yFAAyF;AACzF,gGAAgG;AAChG,8FAA8F;AAC9F,2FAA2F;AAC3F,gCAAgC;AAChC,SAAeC,UAAUJ,KAAY,EAAED,GAAmB,EAAExC,MAAc;;YAChE8C,MAGFC;;;;oBAHED,OAASE,IAAAA,sBAAa,EAACR,KAAKxC,QAA5B8C,MAAqC,gDAAgD;oBAC7F;;wBAAML,MAAMQ,IAAI,CAAC;;;oBAAjB;oBACA;;wBAAMR,MAAMQ,IAAI,CAAC;;;oBAAjB;oBACe;;wBAAMR,MAAMS,OAAO,CAAC;;;oBAA7BH,SAAS;oBACf;;wBAAMA,OAAOI,GAAG,CAACL;;;oBAAjB;;;;;;IACF;;AAGO,SAAetD,OAAOgD,GAAmB,EAAEY,GAAW,EAAEC,MAAgB,EAAE1D,MAAiB,EAAE2D,KAAa,EAAEtD,MAAe;;YACtGoD,YAApBG,kBAYsB,MAApBd,OAAOP,UAYQsB,YAFfA,WACAC,SAEC/B,KACDgC;;;;oBA1BFH,mBAAmB,EAACH,aAAAA,IAAIO,KAAK,CAAC,oBAAVP,wBAAAA,iBAAwBQ,MAAM;oBACxD,IAAIP,OAAOO,MAAM,KAAKL,kBAAkB;wBACtC5C,QAAQC,KAAK,CAAC,AAAC,GAAmB2C,OAAjBD,OAAM,aAAiDD,OAAtCE,kBAAiB,uBAAmC,OAAdF,OAAOO,MAAM;wBACrF9C,QAAQC,IAAI,CAAC;oBACf;oBACA,6FAA6F;oBAC7F,2EAA2E;oBAC3E,IAAIf,WAAW8B,aAAa,CAAC,aAAa+B,IAAI,CAACT,MAAM;wBACnDzC,QAAQC,KAAK,CAAC,AAAC,wEAA4E,OAAN0C,OAAM;wBAC3F3C,QAAQC,KAAK,CAAC;wBACdE,QAAQC,IAAI,CAAC;oBACf;oBAC4B;;wBAAM4B,IAAAA,mBAAS,EAACH;;;oBAAhB,OAAA,eAApBC,QAAoB,KAApBA,OAAOP,WAAa,KAAbA;oBACf5C,cAAc4C;yBACVlC,CAAAA,WAAW8B,SAAQ,GAAnB9B;;;;oBAAsB;;wBAAM6C,UAAUJ,OAAOD,KAAKxC;;;oBAA5B;;;;;;;;;oBAQN;;wBAAMyC,MAAMqB,GAAG,CAACZ,OAAO,CAACE;;;oBAApCI,YAAY;oBACZC,UAAUD,UAAUC,OAAO,GAAGM,GAAG,CAAC,SAACC;+BAAMA,EAAElB,IAAI;;oBACrD;;wBAAMmB,IAAAA,wBAAc,EAACT,CAAAA,aAAAA,WAAUU,OAAO,OAAjBV,YAAkB,qBAAGH,UAA+B1D,QAAQ8D;;;oBAAjF;;;;;;oBACO/B;oBACmB;;wBAAMe,MAAM0B,IAAI,CAACV,OAAO;;;oBAA5CC,SAASU,wBAAU;wBAAC;wBAA4B1C;;oBACtD;;wBAAMe,MAAMG,KAAK;;;oBAAjB;oBACA,gFAAgF;oBAChF,mFAAmF;oBACnF,sFAAsF;oBACtF,IAAI,aAAaiB,IAAI,CAACT,MAAM,MAAMiB,IAAAA,0BAAW,EAACX,QAAQL,OAAOxC,IAAI,CAAC;oBAClE,MAAM6C;;oBAER;;wBAAMjB,MAAMG,KAAK;;;oBAAjB;;;;;;IACF"}
+{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/cli/shared.ts"],"sourcesContent":["import type { ParseArgsOptionsConfig } from 'node:util';\nimport { parseArgs } from 'node:util';\nimport type { ResolvedConfig, SearchOverrides } from '../config/index.ts';\nimport { resolvePreset } from '../config/index.ts';\nimport { columnHint } from '../output/column-hint.ts';\nimport type { Row, RowFormat } from '../output/output.ts';\nimport { printRowStream } from '../output/output.ts';\nimport { searchError } from '../output/search-error.ts';\nimport type { OpenResult } from '../store/index.ts';\nimport { openStore } from '../store/index.ts';\nimport type { Store } from '../store/types.ts';\nimport type { Ctx } from './types.ts';\n\n// Spreadable option fragments -- one flag name keeps one meaning across every command's table.\nexport const FORMAT: ParseArgsOptionsConfig = { format: { type: 'string', default: 'table' } };\nexport const CONFIG: ParseArgsOptionsConfig = { config: { type: 'string' } };\n// The scope vocabulary every scoped command shares: named preset, ad hoc include/exclude\n// globs, a where SQL condition. search adds k on top.\nexport const SCOPE: ParseArgsOptionsConfig = {\n  where: { type: 'string' },\n  preset: { type: 'string' },\n  include: { type: 'string', multiple: true },\n  exclude: { type: 'string', multiple: true },\n  // Widening, which no other flag can express: --include and --exclude each override their\n  // own side of the preset, so neither can drop an exclusion the preset declares.\n  'no-exclude': { type: 'boolean', default: false },\n};\nexport const SEARCH_FLAGS: ParseArgsOptionsConfig = { ...SCOPE, k: { type: 'string' } };\n\ntype Values = Record<string, string | boolean | string[] | undefined>;\n\n// The SCOPE fragment's parsed values as the overrides every scoped command passes down.\n// One place to read them: a new scope field is added to the fragment and here, not per-command.\nexport function scopeOf(values: Values): SearchOverrides {\n  return {\n    preset: values.preset as string | undefined,\n    include: values.include as string[] | undefined,\n    exclude: values.exclude as string[] | undefined,\n    where: values.where as string | undefined,\n    noExclude: values['no-exclude'] === true,\n  };\n}\n\n// parseArgs is strict about flag names; this applies the same strictness to the value.\n// An unrecognised --format is a usage error, not a silent fallback to `table`.\nfunction pickFormat<T extends string>(values: Values, allowed: readonly T[]): T {\n  const format = String(values.format);\n  if ((allowed as readonly string[]).includes(format)) return format as T;\n  console.error(`unknown --format \"${format}\"; expected ${allowed.join(', ')}`);\n  process.exit(2);\n}\n\nexport function formatOf(values: Values): 'table' | 'json' {\n  return pickFormat(values, ['table', 'json'] as const);\n}\n\n// csv is a row set rendered as rows; map, peek, status, and path render structures instead,\n// and take formatOf.\nexport function rowFormatOf(values: Values): RowFormat {\n  return pickFormat(values, ['table', 'json', 'csv'] as const);\n}\n\n// Per-command parseArgs: strict (a foreign flag exits 2), and every table gets --help for free.\nexport function parse(argv: string[], usage: string, options: ParseArgsOptionsConfig): { values: Values; positionals: string[] } {\n  let values: Values;\n  let positionals: string[];\n  try {\n    ({ values, positionals } = parseArgs({\n      args: argv,\n      options: { ...options, help: { type: 'boolean', default: false, short: 'h' } },\n      strict: true,\n      allowPositionals: true,\n    }));\n  } catch (err) {\n    console.error((err as Error).message);\n    console.error(usage);\n    process.exit(2);\n  }\n  if (values.help) {\n    console.log(usage);\n    process.exit(0);\n  }\n  return { values, positionals };\n}\n\n// --k must be a positive integer: SQLite reads a bound LIMIT of -1 as \"unlimited\" and 0 as \"nothing\", and parseInt would silently truncate \"5.9\".\n// All three are caller mistakes worth a usage error, matching the config-level SavedSearch validation.\nexport function parseK(k: string | undefined, usageError: (message: string) => never): number | undefined {\n  if (k === undefined) return undefined;\n  const parsed = Number(k);\n  if (!Number.isInteger(parsed) || parsed <= 0) usageError(`--k expects a positive integer, got \"${k}\"`);\n  return parsed;\n}\n\n// Shared open-query-close envelope for commands that touch the tree.\n\nexport function printWarnings(warnings: string[]): void {\n  for (const w of warnings) console.warn(w);\n}\n\nexport async function withDb(ctx: Ctx, configPath: string | undefined, fn: (store: OpenResult['store'], cfg: ResolvedConfig) => void | Promise<void>): Promise<void> {\n  const cfg = ctx.resolveConfig(configPath);\n  const { store, warnings } = await openStore(cfg);\n  printWarnings(warnings);\n  try {\n    await fn(store, cfg);\n  } finally {\n    await store.close();\n  }\n}\n\n// `--preset` on SQL binds a temp `scope` table rather than transparently filtering: temp views shadowing the base tables can't cover `content`, since FTS5 MATCH treats the table name as a hidden column.\n// A flag that silently scoped three tables of four would look scoped and not be.\nasync function bindScope(store: Store, cfg: ResolvedConfig, preset: string): Promise<void> {\n  const { name } = resolvePreset(cfg, preset); // unknown names throw, listing what is declared\n  await store.exec('DROP TABLE IF EXISTS temp.scope');\n  await store.exec('CREATE TEMP TABLE scope (\"path\" TEXT PRIMARY KEY)');\n  const insert = await store.prepare('INSERT INTO temp.scope (\"path\") SELECT \"path\" FROM preset_files WHERE preset = ?');\n  await insert.run(name);\n}\n\n// An unbound `?` silently binds NULL, so mismatched param counts fail loudly instead.\nexport async function runSql(cfg: ResolvedConfig, sql: string, params: string[], format: RowFormat, label: string, preset?: string): Promise<void> {\n  const placeholderCount = (sql.match(/\\?/g) ?? []).length;\n  if (params.length !== placeholderCount) {\n    console.error(`${label} expects ${placeholderCount} parameter(s), got ${params.length}`);\n    process.exit(2);\n  }\n  // Naming a preset and never joining it would return the whole index while reading as scoped,\n  // which is the one hazard of binding rather than applying. Refuse instead.\n  if (preset !== undefined && !/\\bscope\\b/i.test(sql)) {\n    console.error(`--preset binds a temporary \"scope\" table of the preset's paths, and ${label} never joins it, so the preset would have no effect`);\n    console.error(`add: JOIN scope ON scope.\"path\" = <table>.\"path\"`);\n    process.exit(2);\n  }\n  const { store, warnings } = await openStore(cfg);\n  printWarnings(warnings);\n  if (preset !== undefined) await bindScope(store, cfg, preset);\n  // Streamed, not collected: `sql`'s result size is the caller's business, not ours. columns() reads the statement's own metadata, so a 0-row csv gets a header too.\n  // A mid-stream SQLite error (e.g. SQLITE_BUSY outlasting busy_timeout) may leave partial output; the nonzero exit code is the failure signal, not a completeness guarantee.\n  try {\n    const statement = await store.raw.prepare(sql);\n    const columns = statement.columns().map((c) => c.name);\n    await printRowStream(statement.iterate(...params) as AsyncIterable<Row>, format, columns);\n  } catch (err) {\n    const hinted = columnHint(await store.docs.columns(), err as Error); // columns read while the store is still open\n    await store.close();\n    // A saved query or ad-hoc SQL can carry `content MATCH ?` too, so the same FTS5 punctuation trap applies to its bound parameters.\n    // SQL without MATCH gets its error verbatim; search advice on a plain typo would mislead.\n    if (/\\bMATCH\\b/i.test(sql)) throw searchError(hinted, params.join(' '));\n    throw hinted;\n  }\n  await store.close();\n}\n"],"names":["CONFIG","FORMAT","SCOPE","SEARCH_FLAGS","formatOf","parse","parseK","printWarnings","rowFormatOf","runSql","scopeOf","withDb","format","type","default","config","where","preset","include","multiple","exclude","k","values","noExclude","pickFormat","allowed","String","includes","console","error","join","process","exit","argv","usage","options","positionals","parseArgs","args","help","short","strict","allowPositionals","err","message","log","usageError","undefined","parsed","Number","isInteger","warnings","w","warn","ctx","configPath","fn","cfg","store","resolveConfig","openStore","close","bindScope","name","insert","resolvePreset","exec","prepare","run","sql","params","label","placeholderCount","statement","columns","hinted","match","length","test","raw","map","c","printRowStream","iterate","docs","columnHint","searchError"],"mappings":";;;;;;;;;;;QAeaA;eAAAA;;QADAC;eAAAA;;QAIAC;eAAAA;;QASAC;eAAAA;;QAyBGC;eAAAA;;QAWAC;eAAAA;;QAwBAC;eAAAA;;QASAC;eAAAA;;QAtCAC;eAAAA;;QAgEMC;eAAAA;;QAzFNC;eAAAA;;QAmEMC;eAAAA;;;wBAnGI;uBAEI;4BACH;wBAEI;6BACH;wBAEF;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAKnB,IAAMV,SAAiC;IAAEW,QAAQ;QAAEC,MAAM;QAAUC,SAAS;IAAQ;AAAE;AACtF,IAAMd,SAAiC;IAAEe,QAAQ;QAAEF,MAAM;IAAS;AAAE;AAGpE,IAAMX,QAAgC;IAC3Cc,OAAO;QAAEH,MAAM;IAAS;IACxBI,QAAQ;QAAEJ,MAAM;IAAS;IACzBK,SAAS;QAAEL,MAAM;QAAUM,UAAU;IAAK;IAC1CC,SAAS;QAAEP,MAAM;QAAUM,UAAU;IAAK;IAC1C,yFAAyF;IACzF,gFAAgF;IAChF,cAAc;QAAEN,MAAM;QAAWC,SAAS;IAAM;AAClD;AACO,IAAMX,eAAuC,wCAAKD;IAAOmB,GAAG;QAAER,MAAM;IAAS;;AAM7E,SAASH,QAAQY,MAAc;IACpC,OAAO;QACLL,QAAQK,OAAOL,MAAM;QACrBC,SAASI,OAAOJ,OAAO;QACvBE,SAASE,OAAOF,OAAO;QACvBJ,OAAOM,OAAON,KAAK;QACnBO,WAAWD,MAAM,CAAC,aAAa,KAAK;IACtC;AACF;AAEA,uFAAuF;AACvF,+EAA+E;AAC/E,SAASE,WAA6BF,MAAc,EAAEG,OAAqB;IACzE,IAAMb,SAASc,OAAOJ,OAAOV,MAAM;IACnC,IAAI,AAACa,QAA8BE,QAAQ,CAACf,SAAS,OAAOA;IAC5DgB,QAAQC,KAAK,CAAC,AAAC,qBAAyCJ,OAArBb,QAAO,gBAAiC,OAAnBa,QAAQK,IAAI,CAAC;IACrEC,QAAQC,IAAI,CAAC;AACf;AAEO,SAAS5B,SAASkB,MAAc;IACrC,OAAOE,WAAWF,QAAQ;QAAC;QAAS;KAAO;AAC7C;AAIO,SAASd,YAAYc,MAAc;IACxC,OAAOE,WAAWF,QAAQ;QAAC;QAAS;QAAQ;KAAM;AACpD;AAGO,SAASjB,MAAM4B,IAAc,EAAEC,KAAa,EAAEC,OAA+B;IAClF,IAAIb;IACJ,IAAIc;IACJ,IAAI;;cACyBC,IAAAA,mBAAS,EAAC;YACnCC,MAAML;YACNE,SAAS,wCAAKA;gBAASI,MAAM;oBAAE1B,MAAM;oBAAWC,SAAS;oBAAO0B,OAAO;gBAAI;;YAC3EC,QAAQ;YACRC,kBAAkB;QACpB,IALGpB,aAAAA,QAAQc,kBAAAA;IAMb,EAAE,OAAOO,KAAK;QACZf,QAAQC,KAAK,CAAC,AAACc,IAAcC,OAAO;QACpChB,QAAQC,KAAK,CAACK;QACdH,QAAQC,IAAI,CAAC;IACf;IACA,IAAIV,OAAOiB,IAAI,EAAE;QACfX,QAAQiB,GAAG,CAACX;QACZH,QAAQC,IAAI,CAAC;IACf;IACA,OAAO;QAAEV,QAAAA;QAAQc,aAAAA;IAAY;AAC/B;AAIO,SAAS9B,OAAOe,CAAqB,EAAEyB,UAAsC;IAClF,IAAIzB,MAAM0B,WAAW,OAAOA;IAC5B,IAAMC,SAASC,OAAO5B;IACtB,IAAI,CAAC4B,OAAOC,SAAS,CAACF,WAAWA,UAAU,GAAGF,WAAW,AAAC,wCAAyC,OAAFzB,GAAE;IACnG,OAAO2B;AACT;AAIO,SAASzC,cAAc4C,QAAkB;QACzC,kCAAA,2BAAA;;QAAL,QAAK,YAAWA,6BAAX,SAAA,6BAAA,QAAA,yBAAA;YAAA,IAAMC,IAAN;YAAqBxB,QAAQyB,IAAI,CAACD;;;QAAlC;QAAA;;;iBAAA,6BAAA;gBAAA;;;gBAAA;sBAAA;;;;AACP;AAEO,SAAezC,OAAO2C,GAAQ,EAAEC,UAA8B,EAAEC,EAA6E;;YAC5IC,KACsB,MAApBC,OAAOP;;;;oBADTM,MAAMH,IAAIK,aAAa,CAACJ;oBACF;;wBAAMK,IAAAA,mBAAS,EAACH;;;oBAAhB,OAAA,eAApBC,QAAoB,KAApBA,OAAOP,WAAa,KAAbA;oBACf5C,cAAc4C;;;;;;;;;oBAEZ;;wBAAMK,GAAGE,OAAOD;;;oBAAhB;;;;;;oBAEA;;wBAAMC,MAAMG,KAAK;;;oBAAjB;;;;;;;;;;IAEJ;;AAEA,2MAA2M;AAC3M,iFAAiF;AACjF,SAAeC,UAAUJ,KAAY,EAAED,GAAmB,EAAExC,MAAc;;YAChE8C,MAGFC;;;;oBAHED,OAASE,IAAAA,sBAAa,EAACR,KAAKxC,QAA5B8C,MAAqC,gDAAgD;oBAC7F;;wBAAML,MAAMQ,IAAI,CAAC;;;oBAAjB;oBACA;;wBAAMR,MAAMQ,IAAI,CAAC;;;oBAAjB;oBACe;;wBAAMR,MAAMS,OAAO,CAAC;;;oBAA7BH,SAAS;oBACf;;wBAAMA,OAAOI,GAAG,CAACL;;;oBAAjB;;;;;;IACF;;AAGO,SAAetD,OAAOgD,GAAmB,EAAEY,GAAW,EAAEC,MAAgB,EAAE1D,MAAiB,EAAE2D,KAAa,EAAEtD,MAAe;;YACtGoD,YAApBG,kBAYsB,MAApBd,OAAOP,UAQQsB,YAFfA,WACAC,SAEC/B,KACDgC;;;;oBAtBFH,mBAAmB,EAACH,aAAAA,IAAIO,KAAK,CAAC,oBAAVP,wBAAAA,iBAAwBQ,MAAM;oBACxD,IAAIP,OAAOO,MAAM,KAAKL,kBAAkB;wBACtC5C,QAAQC,KAAK,CAAC,AAAC,GAAmB2C,OAAjBD,OAAM,aAAiDD,OAAtCE,kBAAiB,uBAAmC,OAAdF,OAAOO,MAAM;wBACrF9C,QAAQC,IAAI,CAAC;oBACf;oBACA,6FAA6F;oBAC7F,2EAA2E;oBAC3E,IAAIf,WAAW8B,aAAa,CAAC,aAAa+B,IAAI,CAACT,MAAM;wBACnDzC,QAAQC,KAAK,CAAC,AAAC,wEAA4E,OAAN0C,OAAM;wBAC3F3C,QAAQC,KAAK,CAAC;wBACdE,QAAQC,IAAI,CAAC;oBACf;oBAC4B;;wBAAM4B,IAAAA,mBAAS,EAACH;;;oBAAhB,OAAA,eAApBC,QAAoB,KAApBA,OAAOP,WAAa,KAAbA;oBACf5C,cAAc4C;yBACVlC,CAAAA,WAAW8B,SAAQ,GAAnB9B;;;;oBAAsB;;wBAAM6C,UAAUJ,OAAOD,KAAKxC;;;oBAA5B;;;;;;;;;oBAIN;;wBAAMyC,MAAMqB,GAAG,CAACZ,OAAO,CAACE;;;oBAApCI,YAAY;oBACZC,UAAUD,UAAUC,OAAO,GAAGM,GAAG,CAAC,SAACC;+BAAMA,EAAElB,IAAI;;oBACrD;;wBAAMmB,IAAAA,wBAAc,EAACT,CAAAA,aAAAA,WAAUU,OAAO,OAAjBV,YAAkB,qBAAGH,UAA+B1D,QAAQ8D;;;oBAAjF;;;;;;oBACO/B;oBACmB;;wBAAMe,MAAM0B,IAAI,CAACV,OAAO;;;oBAA5CC,SAASU,wBAAU;wBAAC;wBAA4B1C;;oBACtD;;wBAAMe,MAAMG,KAAK;;;oBAAjB;oBACA,kIAAkI;oBAClI,0FAA0F;oBAC1F,IAAI,aAAaiB,IAAI,CAACT,MAAM,MAAMiB,IAAAA,0BAAW,EAACX,QAAQL,OAAOxC,IAAI,CAAC;oBAClE,MAAM6C;;oBAER;;wBAAMjB,MAAMG,KAAK;;;oBAAjB;;;;;;IACF"}

package/dist/cjs/cli/status.js

@@ -223,13 +223,10 @@ function _unsupported_iterable_to_array(o, minLen) {
     if (n === "Map" || n === "Set") return Array.from(n);
     if (n === "Arguments" || /^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)) return _array_like_to_array(o, minLen);
 }
-// status names what you cannot read off the config file: locations sense chose, values it
-// derived, state it stored. Config content is not echoed -- the config line says which file to
-// read, and duplicating its globs here would create two places that disagree after an edit.
+// status names what you cannot read off the config file: locations sense chose, values it derived, state it stored. Config content itself is not echoed, since duplicating its globs here would create two places that disagree after an edit.
 // The exception is the embed block's url and key, whose effect is remote and otherwise invisible.
-// Vectors are computed on the first semantic query, not at reconcile, so a tree can have a
-// semantic-on preset with nothing embedded yet -- reported here because a fast reconcile on
-// such a tree otherwise reads as "the feature isn't on".
+// Vectors are computed on the first semantic query, not at reconcile, so a tree can have a semantic-on preset with nothing embedded yet.
+// Reported here because a fast reconcile on such a tree otherwise reads as "the feature isn't on".
 function vectorState(store) {
     return _async_to_generator(function() {
         var stmt, unused;
@@ -325,9 +322,39 @@ function embedReachable(e) {
         });
     })();
 }
+// Config-declared languages win over the model card, for every provider, as registry.ts does.
+// A local-path model has no card by design, so it is never "unresolved": that means an HF id.
+function languagesInfo(e) {
+    if (e.languages) return {
+        languages: e.languages,
+        state: 'declared',
+        source: 'config'
+    };
+    if (e.provider !== 'static' || !(0, _storets.isDownloadable)(e.model)) return {
+        languages: null,
+        state: null,
+        source: null
+    };
+    var cardLanguages = (0, _storets.readLanguages)(e.model);
+    if (cardLanguages === undefined) return {
+        languages: null,
+        state: 'unresolved',
+        source: null
+    };
+    if (cardLanguages.length === 0) return {
+        languages: null,
+        state: 'none',
+        source: null
+    };
+    return {
+        languages: cardLanguages,
+        state: 'declared',
+        source: 'card'
+    };
+}
 var status = function status(ctx) {
     return _async_to_generator(function() {
-        var _ref, _process_env_e_key, _cfg_migratedFrom, _cfg_unknownKeys, _cfg_queries, values, format, cfg, _ref1, store, dbPath, warnings, features, e, hasModel, reachable, _tmp, vectors, _tmp1, presets, staticLanguages, declaredLanguages, detectedLanguages, heartbeat, cacheSchema, engine, keyEnv, result, _tmp2, migrated, shown, total, breakdown, _iteratorNormalCompletion, _didIteratorError, _iteratorError, _iterator, _step, line, pid, _iteratorNormalCompletion1, _didIteratorError1, _iteratorError1, _iterator1, _step1, _step_value, key, value;
+        var _ref, _process_env_e_key, _cfg_migratedFrom, _cfg_unknownKeys, _cfg_queries, values, format, cfg, _ref1, store, dbPath, warnings, features, e, hasModel, reachable, _tmp, vectors, _tmp1, presets, _ref2, declaredLanguages, languagesState, languagesSource, detectedLanguages, heartbeat, cacheSchema, engine, keyEnv, result, _tmp2, migrated, shown, total, breakdown, _iteratorNormalCompletion, _didIteratorError, _iteratorError, _iterator, _step, line, pid, _iteratorNormalCompletion1, _didIteratorError1, _iteratorError1, _iterator1, _step1, _step_value, key, value;
         return _ts_generator(this, function(_state) {
             switch(_state.label){
                 case 0:
@@ -388,8 +415,11 @@ var status = function status(ctx) {
                     ];
                 case 8:
                     presets = _state.sent();
-                    staticLanguages = e && e.provider === 'static' ? (0, _storets.readLanguages)(e.model) : undefined;
-                    declaredLanguages = staticLanguages && staticLanguages.length > 0 ? staticLanguages : null;
+                    _ref2 = e ? languagesInfo(e) : {
+                        languages: null,
+                        state: null,
+                        source: null
+                    }, declaredLanguages = _ref2.languages, languagesState = _ref2.state, languagesSource = _ref2.source;
                     return [
                         4,
                         (0, _distributionts.languageDistribution)(store)
@@ -449,6 +479,7 @@ var status = function status(ctx) {
                         keyEnv: keyEnv,
                         downloaded: hasModel,
                         languages: declaredLanguages,
+                        languagesState: languagesState,
                         detectedLanguages: detectedLanguages
                     }, vectors !== null && vectors !== void 0 ? vectors : {}) : null, _tmp2.presets = presets, _tmp2.queries = Object.keys((_cfg_queries = cfg.queries) !== null && _cfg_queries !== void 0 ? _cfg_queries : {}).length;
                     return [
@@ -475,7 +506,10 @@ var status = function status(ctx) {
                         else {
                             console.log("embed:    ".concat(e.provider, " ").concat(e.model));
                             if (e.provider === 'static') console.log("          model:   ".concat((0, _storets.modelDir)(e.model), " (").concat(hasModel ? 'present' : "missing, ".concat((0, _storets.isDownloadable)(e.model) ? "fetches automatically at first search, or run `".concat(ctx.name, " download` to prefetch") : "no ".concat(_storets.MODEL_FILENAMES, " in that directory")), ")"));
-                            if (declaredLanguages) console.log("          languages: ".concat(declaredLanguages.join(', '), " (declared by the model card)"));
+                            if (declaredLanguages && languagesSource === 'config') console.log("          languages: ".concat(declaredLanguages.join(', '), " (declared in embed.languages)"));
+                            else if (declaredLanguages && languagesSource === 'card') console.log("          languages: ".concat(declaredLanguages.join(', '), " (declared by the model card)"));
+                            else if (languagesState === 'none') console.log('          languages: none declared by the model card');
+                            else if (languagesState === 'unresolved') console.log("          languages: unresolved (model card unreachable; run `".concat(ctx.name, " download`)"));
                             // Cohere defaults to api.cohere.com with no url in the config; show it anyway so the
                             // reachability probe below has something to name.
                             if (e.url || reachable !== null) {