llmnav 0.6.2 → 0.6.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,32 @@ The npm package follows Semantic Versioning. The `llmnav/N` source protocol is v
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.6.6] — 2026-08-12
10
+
11
+ ### Added
12
+
13
+ * Added a read-only cross-repository conformance matrix with repository-isolated validation, retrieval, audit, repeatability, cache-freshness, and held-language evidence.
14
+
15
+ ### Fixed
16
+
17
+ * Invalidated parsed file-state caches created by older source indexers so upgraded installations rebuild semantic and declaration hashes from source instead of silently reusing incompatible state.
18
+
19
+ ## [0.6.4] — 2026-08-11
20
+
21
+ ### Fixed
22
+
23
+ * Rejected symlink and junction traversal across search caches, incremental state, and public registry reads and writes.
24
+ * Enforced parser card limits before materialization and replaced repeated interval and line-number scans with bounded indexes.
25
+ * Bounded aggregate declaration extraction work per source file, including malformed declarations and whitespace lookahead.
26
+
27
+ ## [0.6.3] — 2026-08-11
28
+
29
+ ### Fixed
30
+
31
+ * Rejected nested symlink and junction traversal before managed cache, state, schema, and prompt-bundle filesystem operations.
32
+ * Replaced per-marker lexical rescans with a bounded single-pass parser classification.
33
+ * Required npm release tags to resolve to commits on the protected `origin/main` lineage.
34
+
9
35
  ## [0.6.2] — 2026-08-11
10
36
 
11
37
  ### Added
package/ROADMAP.md CHANGED
@@ -77,6 +77,12 @@ Implemented:
77
77
 
78
78
  The source grammar and generated formats will be declared stable only after use across multiple TypeScript, Go, Rust, Python, and mixed-language repositories. A 1.0 release requires migration tooling, documented compatibility guarantees, benchmark fixtures with published methodology, sustained Windows and Linux verification, and no unresolved high-severity parser or transaction ambiguity.
79
79
 
80
+ In progress:
81
+
82
+ * A read-only cross-repository conformance matrix measures validation, retrieval, audit, repeatability, and cache freshness without averaging weak repositories away.
83
+ * LLMNav, Workduck, Sairon, and AI BOM Generator currently pass repository-isolated conformance checks across JavaScript, TypeScript, Rust, Go, and Python.
84
+ * The current evidence covers 4 repositories and 5 required languages with no held or failed repository, while 1.0 still requires sustained Windows and Linux verification and published benchmark methodology.
85
+
80
86
  ## Non-goals
81
87
 
82
88
  LLMNav will not become a general documentation generator, a source-of-truth call graph maintained by comments, an autonomous code modification service, a hosted source-ingestion platform, or a mandatory embedding database.
@@ -94,6 +94,26 @@ npx llmnav eval --json
94
94
 
95
95
  Track at least Recall@1, Recall@5, and mean reciprocal rank. Record every acceptable target ID when a task has multiple valid entry points.
96
96
 
97
+ ## Cross-repository conformance matrix
98
+
99
+ Run the checked-in read-only matrix from a workspace that contains the configured sibling repositories:
100
+
101
+ ```sh
102
+ node benchmarks/run-conformance.js
103
+ ```
104
+
105
+ Use `--matrix <path>` to select another checked-in matrix and `--output <path>` to retain a JSON report. The report schema is `benchmarks/conformance-report.schema.json`.
106
+
107
+ Each repository keeps its own validation counts, retrieval thresholds, Recall@1, Recall@5, MRR, audit summary, repeatability result, and generated-cache freshness result. The harness deliberately does not average retrieval scores across repositories. A large or easy dataset must not hide a failing repository.
108
+
109
+ Verdicts have asymmetric meaning:
110
+
111
+ * `pass` requires valid cards, sufficient reviewed queries, repository thresholds, repeatable retrieval, and a source-current generated cache.
112
+ * `fail` requires a reproducible validation, retrieval, repeatability, cache-freshness, or measurement failure.
113
+ * `held` records missing adoption or insufficient reviewed cases without pretending that unmeasured language coverage passed or failed.
114
+
115
+ The default matrix currently measures LLMNav and Workduck. Go and Python remain held until their candidate repositories receive reviewed cards and task-language query datasets. Matrix output is navigation evidence, not an end-to-end claim about model tokens or coding-task success.
116
+
97
117
  ## End-to-end agent metrics
98
118
 
99
119
  A successful deployment should reduce exploration cost without reducing task correctness.
package/docs/spec.md CHANGED
@@ -351,6 +351,10 @@ For repositories with at least 50,000 scanned source bytes, LLMNav comments SHOU
351
351
 
352
352
  These are anti-bloat limits, not targets.
353
353
 
354
+ The reference parser also rejects a single source file larger than 16 MiB and more than 10,000
355
+ LLMNav blocks in one file. These resource budgets keep repository-controlled input from causing
356
+ unbounded parser memory or CPU work; they do not change the `llmnav/1` comment grammar.
357
+
354
358
  ## Canonicalization
355
359
 
356
360
  A canonical formatter MUST refuse to erase malformed lines, unknown fields, overlapping blocks, or duplicate scalar values. Unsafe cards remain unchanged until the checker-reported issue is fixed.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "llmnav",
3
- "version": "0.6.2",
3
+ "version": "0.6.6",
4
4
  "description": "A deterministic semantic navigation layer for LLM coding agents.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -10,7 +10,7 @@ stability=architecture
10
10
  */
11
11
 
12
12
  import path from "node:path";
13
- import { lineAtOffset, normalizeNewlines, sha256 } from "./util.js";
13
+ import { buildLineStarts, lineAtOffsetFromStarts, normalizeNewlines, sha256 } from "./util.js";
14
14
 
15
15
  const DECLARATION_PATTERNS = {
16
16
  javascript: [
@@ -42,8 +42,18 @@ const DECLARATION_PATTERNS = {
42
42
  { kind: "function", pattern: /^(?:public\s+|private\s+|protected\s+|internal\s+|static\s+|final\s+|async\s+)*(?:[A-Za-z_][\w<>?\[\],.]*\s+)+([A-Za-z_][A-Za-z0-9_]*)\s*\(/u },
43
43
  ],
44
44
  };
45
+ const JAVASCRIPT_REGEX_PREFIX_KEYWORDS = new Set([
46
+ "case", "delete", "do", "else", "in", "instanceof", "new", "return", "throw", "typeof", "void", "yield",
47
+ ]);
45
48
 
46
- export function findAttachedDeclaration(source, block, filePath) {
49
+ export function createDeclarationScanContext(source) {
50
+ return {
51
+ lineStarts: buildLineStarts(source),
52
+ remainingScanUnits: Math.max(1, source.length * 2),
53
+ };
54
+ }
55
+
56
+ export function findAttachedDeclaration(source, block, filePath, scanContext = createDeclarationScanContext(source)) {
47
57
  if (block.scope !== "symbol") return null;
48
58
  const window = source.slice(block.end, block.end + 3000);
49
59
  const skipped = skipTrivia(window);
@@ -61,11 +71,12 @@ export function findAttachedDeclaration(source, block, filePath) {
61
71
  const declarationOffset = block.end + skipped;
62
72
  const signature = extractSignature(collapsed);
63
73
  const exported = isExportedDeclaration(language, match[1], signature);
64
- const endOffset = findDeclarationEnd(source, declarationOffset, family);
74
+ const endOffset = findDeclarationEnd(source, declarationOffset, family, scanContext);
75
+ if (endOffset === null) return null;
65
76
  return {
66
77
  symbol: match[1],
67
78
  kind: definition.kind,
68
- line: lineAtOffset(source, declarationOffset),
79
+ line: lineAtOffsetFromStarts(scanContext.lineStarts, declarationOffset),
69
80
  signature,
70
81
  language,
71
82
  exported,
@@ -155,8 +166,8 @@ function extractGoReceiver(candidate) {
155
166
  return (parts.at(-1) ?? "").replace(/^\*+/u, "") || null;
156
167
  }
157
168
 
158
- function findDeclarationEnd(source, start, family) {
159
- if (family === "python") return findPythonDeclarationEnd(source, start);
169
+ function findDeclarationEnd(source, start, family, scanContext) {
170
+ if (family === "python") return findPythonDeclarationEnd(source, start, scanContext);
160
171
  let state = "normal";
161
172
  let escaped = false;
162
173
  let regexCharacterClass = false;
@@ -166,6 +177,7 @@ function findDeclarationEnd(source, start, family) {
166
177
  let openedBody = false;
167
178
 
168
179
  for (let index = start; index < source.length; index += 1) {
180
+ if (!consumeDeclarationScan(scanContext)) return null;
169
181
  const character = source[index];
170
182
  const next = source[index + 1];
171
183
  if (state === "line-comment") {
@@ -230,15 +242,16 @@ function findDeclarationEnd(source, start, family) {
230
242
  } else if (character === ";" && !openedBody && parentheses === 0 && brackets === 0) {
231
243
  return index + 1;
232
244
  } else if (character === "\n" && !openedBody && parentheses === 0 && brackets === 0) {
233
- const current = source.slice(start, index).trimEnd();
234
- const nextCharacter = source.slice(index + 1).match(/^\s*(.)/u)?.[1] ?? "";
235
- if (nextCharacter !== "{" && !/(?:=>|[=|&,([{])$/u.test(current)) return index;
245
+ const continuation = declarationContinuesBefore(source, start, index, scanContext);
246
+ const next = nextSignificantCharacter(source, index + 1, scanContext);
247
+ if (continuation === null || next.exhausted) return null;
248
+ if (next.character !== "{" && !continuation) return index;
236
249
  }
237
250
  }
238
251
  return source.length;
239
252
  }
240
253
 
241
- function findPythonDeclarationEnd(source, start) {
254
+ function findPythonDeclarationEnd(source, start, scanContext) {
242
255
  const declarationLineStart = source.lastIndexOf("\n", start - 1) + 1;
243
256
  const baseIndent = indentationWidth(source.slice(declarationLineStart, start));
244
257
  let cursor = source.indexOf("\n", start);
@@ -248,6 +261,7 @@ function findPythonDeclarationEnd(source, start) {
248
261
  while (cursor < source.length) {
249
262
  const lineEnd = source.indexOf("\n", cursor);
250
263
  const end = lineEnd < 0 ? source.length : lineEnd + 1;
264
+ if (!consumeDeclarationScan(scanContext, Math.max(1, end - cursor))) return null;
251
265
  const line = source.slice(cursor, lineEnd < 0 ? source.length : lineEnd);
252
266
  if (/^\s*(?:#.*)?$/u.test(line)) {
253
267
  cursor = end;
@@ -261,6 +275,15 @@ function findPythonDeclarationEnd(source, start) {
261
275
  return source.length;
262
276
  }
263
277
 
278
+ function consumeDeclarationScan(scanContext, units = 1) {
279
+ if (scanContext.remainingScanUnits < units) {
280
+ scanContext.remainingScanUnits = 0;
281
+ return false;
282
+ }
283
+ scanContext.remainingScanUnits -= units;
284
+ return true;
285
+ }
286
+
264
287
  function indentationWidth(value) {
265
288
  let width = 0;
266
289
  for (const character of value) width += character === "\t" ? 8 - (width % 8) : 1;
@@ -268,11 +291,36 @@ function indentationWidth(value) {
268
291
  }
269
292
 
270
293
  function canStartJavaScriptRegex(source, start, offset) {
271
- const prefix = source.slice(start, offset).trimEnd();
272
- if (!prefix) return true;
273
- const previous = prefix.at(-1);
294
+ let cursor = offset - 1;
295
+ while (cursor >= start && /\s/u.test(source[cursor])) cursor -= 1;
296
+ if (cursor < start) return true;
297
+ const previous = source[cursor];
274
298
  if (/[=(:,!&|?{};\[]/u.test(previous)) return true;
275
- return /(?:^|\W)(?:case|delete|do|else|in|instanceof|new|return|throw|typeof|void|yield)\s*$/u.test(prefix);
299
+ if (!/[A-Za-z]/u.test(previous)) return false;
300
+ const wordEnd = cursor + 1;
301
+ while (cursor >= start && /[A-Za-z]/u.test(source[cursor])) cursor -= 1;
302
+ return JAVASCRIPT_REGEX_PREFIX_KEYWORDS.has(source.slice(cursor + 1, wordEnd));
303
+ }
304
+
305
+ function declarationContinuesBefore(source, start, offset, scanContext) {
306
+ let cursor = offset - 1;
307
+ while (cursor >= start && /\s/u.test(source[cursor])) {
308
+ if (!consumeDeclarationScan(scanContext)) return null;
309
+ cursor -= 1;
310
+ }
311
+ if (cursor < start) return false;
312
+ const character = source[cursor];
313
+ if (/[=|&,([{]/u.test(character)) return true;
314
+ return character === ">" && source[cursor - 1] === "=";
315
+ }
316
+
317
+ function nextSignificantCharacter(source, offset, scanContext) {
318
+ let cursor = offset;
319
+ while (cursor < source.length && /\s/u.test(source[cursor])) {
320
+ if (!consumeDeclarationScan(scanContext)) return { character: "", exhausted: true };
321
+ cursor += 1;
322
+ }
323
+ return { character: source[cursor] ?? "", exhausted: false };
276
324
  }
277
325
 
278
326
  function looksLikeRustCharacterLiteral(source, offset) {
package/src/generator.js CHANGED
@@ -217,7 +217,7 @@ async function generateProjectLocked(root, options) {
217
217
 
218
218
  if (statHints && hintsPath) {
219
219
  try {
220
- await persistStatHints(hintsPath, statHints);
220
+ await persistStatHints(root, hintsPath, statHints);
221
221
  statHintsPersisted = true;
222
222
  } catch (error) {
223
223
  statHintsError = error instanceof Error ? error.message : String(error);
@@ -11,11 +11,12 @@ stability=architecture
11
11
  import { readFile, stat } from "node:fs/promises";
12
12
  import path from "node:path";
13
13
  import { loadConfig } from "./config.js";
14
- import { findAttachedDeclaration, extractImports } from "./declaration.js";
14
+ import { createDeclarationScanContext, findAttachedDeclaration, extractImports } from "./declaration.js";
15
15
  import { collectSourceFiles } from "./files.js";
16
16
  import { parseLlmnavBlocks } from "./parser.js";
17
17
  import { loadRegistry } from "./registry.js";
18
18
  import {
19
+ assertNoSymlinkTraversal,
19
20
  atomicWrite,
20
21
  compareText,
21
22
  readJsonSafe,
@@ -25,15 +26,19 @@ import {
25
26
  } from "./util.js";
26
27
 
27
28
  export const FILE_STATE_SCHEMA_VERSION = 1;
28
- export const SOURCE_INDEXER_VERSION = 4;
29
+ export const SOURCE_INDEXER_VERSION = 6;
29
30
  const STAT_HINTS_SCHEMA_VERSION = 1;
30
31
 
31
32
  export async function scanProjectIncremental(root, options = {}) {
32
33
  const { config, configPath } = await loadConfig(root);
33
34
  const files = await collectSourceFiles(root, config, options.paths ?? []);
34
35
  const cacheDirectory = path.join(root, config.generation.cacheDirectory);
35
- const previousState = options.previousState ?? await readJsonSafe(path.join(cacheDirectory, "file-state.json"), null);
36
+ await assertNoSymlinkTraversal(root, cacheDirectory, config.generation.cacheDirectory);
37
+ const fileStatePath = path.join(cacheDirectory, "file-state.json");
38
+ await assertNoSymlinkTraversal(root, fileStatePath, relativePosix(root, fileStatePath));
39
+ const previousState = options.previousState ?? await readJsonSafe(fileStatePath, null);
36
40
  const hintsPath = path.join(root, ".llmnav", "state", "stat-hints.json");
41
+ await assertNoSymlinkTraversal(root, hintsPath, relativePosix(root, hintsPath));
37
42
  const previousHints = options.useStatHints === false
38
43
  ? null
39
44
  : await readJsonSafe(hintsPath, null);
@@ -133,8 +138,9 @@ export async function scanProjectIncremental(root, options = {}) {
133
138
  export function buildFileStateFromProject(project) {
134
139
  const files = project.fileRecords.map((fileRecord) => {
135
140
  const source = fileRecord.source ?? "";
141
+ const declarationScanContext = createDeclarationScanContext(source);
136
142
  const declarations = fileRecord.blocks.map((block) =>
137
- findAttachedDeclaration(source, block, fileRecord.relativePath),
143
+ findAttachedDeclaration(source, block, fileRecord.relativePath, declarationScanContext),
138
144
  );
139
145
  return normalizeStateFile(
140
146
  {
@@ -157,7 +163,8 @@ export function buildFileStateFromProject(project) {
157
163
  };
158
164
  }
159
165
 
160
- export async function persistStatHints(hintsPath, statHints) {
166
+ export async function persistStatHints(root, hintsPath, statHints) {
167
+ await assertNoSymlinkTraversal(root, hintsPath, relativePosix(root, hintsPath));
161
168
  await atomicWrite(hintsPath, stableStringify(statHints));
162
169
  }
163
170
 
@@ -176,6 +183,7 @@ export function usableFileState(value) {
176
183
 
177
184
  function analyzeFile(relativePath, source) {
178
185
  const blocks = parseLlmnavBlocks(source, relativePath);
186
+ const declarationScanContext = createDeclarationScanContext(source);
179
187
  return {
180
188
  path: relativePath,
181
189
  contentHash: sha256(source),
@@ -183,7 +191,7 @@ function analyzeFile(relativePath, source) {
183
191
  semanticBytes: blocks.reduce((sum, block) => sum + Buffer.byteLength(block.raw), 0),
184
192
  imports: extractImports(source, relativePath),
185
193
  blocks,
186
- declarations: blocks.map((block) => findAttachedDeclaration(source, block, relativePath)),
194
+ declarations: blocks.map((block) => findAttachedDeclaration(source, block, relativePath, declarationScanContext)),
187
195
  };
188
196
  }
189
197
 
@@ -20,9 +20,14 @@ import { assertNoSymlinkTraversal, atomicWrite, readJson, readText, stableString
20
20
  const PACKAGE_ROOT = fileURLToPath(new URL("..", import.meta.url));
21
21
 
22
22
  export async function initializeProject(root, options = {}) {
23
- await assertNoSymlinkTraversal(root, path.join(root, ".llmnav"), ".llmnav");
24
- await mkdir(path.join(root, ".llmnav", "eval"), { recursive: true });
25
- await mkdir(path.join(root, ".llmnav", "schema"), { recursive: true });
23
+ const controlDirectory = path.join(root, ".llmnav");
24
+ const evalDirectory = path.join(controlDirectory, "eval");
25
+ const schemaDirectory = path.join(controlDirectory, "schema");
26
+ await assertNoSymlinkTraversal(root, controlDirectory, ".llmnav");
27
+ await assertNoSymlinkTraversal(root, evalDirectory, ".llmnav/eval");
28
+ await assertNoSymlinkTraversal(root, schemaDirectory, ".llmnav/schema");
29
+ await mkdir(evalDirectory, { recursive: true });
30
+ await mkdir(schemaDirectory, { recursive: true });
26
31
  const repositoryId = await inferRepositoryId(root);
27
32
  const changed = [];
28
33
 
@@ -30,6 +35,7 @@ export async function initializeProject(root, options = {}) {
30
35
  config.repositoryId = repositoryId;
31
36
  config.$schema = "./schema/config.schema.json";
32
37
  await writeIfMissingOrForced(
38
+ root,
33
39
  path.join(root, ".llmnav", "config.json"),
34
40
  stableStringify(config),
35
41
  options.force,
@@ -38,6 +44,7 @@ export async function initializeProject(root, options = {}) {
38
44
  );
39
45
 
40
46
  await writeIfMissingOrForced(
47
+ root,
41
48
  path.join(root, ".llmnav", "lexicon.json"),
42
49
  stableStringify({ version: 1, aliases: {} }),
43
50
  false,
@@ -45,6 +52,7 @@ export async function initializeProject(root, options = {}) {
45
52
  ".llmnav/lexicon.json",
46
53
  );
47
54
  await writeIfMissingOrForced(
55
+ root,
48
56
  path.join(root, ".llmnav", "ids.jsonl"),
49
57
  "",
50
58
  false,
@@ -52,6 +60,7 @@ export async function initializeProject(root, options = {}) {
52
60
  ".llmnav/ids.jsonl",
53
61
  );
54
62
  await writeIfMissingOrForced(
63
+ root,
55
64
  path.join(root, ".llmnav", "order.lock"),
56
65
  "",
57
66
  false,
@@ -59,6 +68,7 @@ export async function initializeProject(root, options = {}) {
59
68
  ".llmnav/order.lock",
60
69
  );
61
70
  await writeIfMissingOrForced(
71
+ root,
62
72
  path.join(root, ".llmnav", "eval", "queries.jsonl"),
63
73
  '# One JSON object per line: {"query":"...","expected":["domain.feature.action"]}\n',
64
74
  false,
@@ -66,6 +76,7 @@ export async function initializeProject(root, options = {}) {
66
76
  ".llmnav/eval/queries.jsonl",
67
77
  );
68
78
  await writeIfMissingOrForced(
79
+ root,
69
80
  path.join(root, ".llmnav", ".gitignore"),
70
81
  "tmp/\nstate/\n.transactions/\ngeneration-transaction.json\ngeneration.lock\ngeneration.lock.release-*\n*.tmp-*\n",
71
82
  options.force,
@@ -75,6 +86,7 @@ export async function initializeProject(root, options = {}) {
75
86
 
76
87
  const schemaSource = path.join(PACKAGE_ROOT, "schema", "config.schema.json");
77
88
  const schemaTarget = path.join(root, ".llmnav", "schema", "config.schema.json");
89
+ await assertNoSymlinkTraversal(root, schemaTarget, ".llmnav/schema/config.schema.json");
78
90
  if (options.force || (await readText(schemaTarget, null)) === null) {
79
91
  await copyFile(schemaSource, schemaTarget);
80
92
  changed.push(".llmnav/schema/config.schema.json");
@@ -106,7 +118,8 @@ async function inferRepositoryId(root) {
106
118
  return normalized || "repository";
107
119
  }
108
120
 
109
- async function writeIfMissingOrForced(filePath, content, force, changed, displayPath) {
121
+ async function writeIfMissingOrForced(root, filePath, content, force, changed, displayPath) {
122
+ await assertNoSymlinkTraversal(root, filePath, displayPath);
110
123
  const existing = await readText(filePath, null);
111
124
  if (existing !== null && !force) return;
112
125
  if (existing === content) return;
@@ -116,6 +129,7 @@ async function writeIfMissingOrForced(filePath, content, force, changed, display
116
129
 
117
130
  async function addPackageScripts(root) {
118
131
  const packagePath = path.join(root, "package.json");
132
+ await assertNoSymlinkTraversal(root, packagePath, "package.json");
119
133
  const text = await readText(packagePath, null);
120
134
  if (text === null) return false;
121
135
  const parsed = JSON.parse(text);
package/src/parser.js CHANGED
@@ -15,7 +15,13 @@ import {
15
15
  SCOPES,
16
16
  SPEC_VERSION,
17
17
  } from "./spec.js";
18
- import { detectNewline, lineAtOffset, normalizeNewlines, splitPipe } from "./util.js";
18
+ import {
19
+ buildLineStarts,
20
+ detectNewline,
21
+ lineAtOffsetFromStarts,
22
+ normalizeNewlines,
23
+ splitPipe,
24
+ } from "./util.js";
19
25
 
20
26
  const BLOCK_PATTERNS = [
21
27
  {
@@ -31,24 +37,33 @@ const BLOCK_PATTERNS = [
31
37
  terminator: "-->",
32
38
  },
33
39
  ];
40
+ const MAX_SOURCE_BYTES = 16 * 1024 * 1024;
41
+ const MAX_BLOCKS_PER_FILE = 10_000;
34
42
 
35
43
  export function parseLlmnavBlocks(source, filePath = "<memory>") {
44
+ if (Buffer.byteLength(source) > MAX_SOURCE_BYTES) {
45
+ throw new Error(`${filePath} exceeds the ${MAX_SOURCE_BYTES}-byte parser byte limit.`);
46
+ }
47
+ const literalMask = buildLiteralMask(source, filePath);
48
+ const lineStarts = buildLineStarts(source);
49
+ const blockBudget = { count: 0 };
36
50
  const blocks = [];
37
51
  for (const definition of BLOCK_PATTERNS) {
38
52
  definition.pattern.lastIndex = 0;
39
53
  for (const match of source.matchAll(definition.pattern)) {
40
54
  const matchedRaw = match[0];
41
55
  const matchStart = match.index ?? 0;
42
- if (isInsideStringLiteral(source, matchStart, filePath)) continue;
56
+ if (literalMask[matchStart] === 1) continue;
43
57
  const lineStart = source.lastIndexOf("\n", matchStart - 1) + 1;
44
58
  const leading = source.slice(lineStart, matchStart);
45
59
  const start = /^\s*$/u.test(leading) ? lineStart : matchStart;
46
60
  const end = matchStart + matchedRaw.length;
47
61
  const indent = start === lineStart ? leading : "";
48
62
  const raw = source.slice(start, end);
63
+ reserveBlock(blockBudget, filePath);
49
64
  blocks.push(
50
65
  createBlock({
51
- source,
66
+ lineStarts,
52
67
  filePath,
53
68
  raw,
54
69
  body: match[2],
@@ -63,8 +78,8 @@ export function parseLlmnavBlocks(source, filePath = "<memory>") {
63
78
  }
64
79
  }
65
80
 
66
- blocks.push(...parseLineBlocks(source, filePath));
67
- blocks.push(...parseUnterminatedBlockComments(source, filePath, blocks));
81
+ blocks.push(...parseLineBlocks(source, filePath, literalMask, lineStarts, blockBudget));
82
+ blocks.push(...parseUnterminatedBlockComments(source, filePath, blocks, literalMask, lineStarts, blockBudget));
68
83
  blocks.sort((left, right) => left.start - right.start);
69
84
 
70
85
  const overlapping = [];
@@ -85,7 +100,7 @@ export function parseLlmnavBlocks(source, filePath = "<memory>") {
85
100
  return blocks;
86
101
  }
87
102
 
88
- function parseLineBlocks(source, filePath) {
103
+ function parseLineBlocks(source, filePath, literalMask, lineStarts, blockBudget) {
89
104
  const blocks = [];
90
105
  const lines = source.split(/(?<=\n)/u);
91
106
  const offsets = [];
@@ -101,7 +116,7 @@ function parseLineBlocks(source, filePath) {
101
116
  if (!header) continue;
102
117
 
103
118
  const start = offsets[index];
104
- if (isInsideStringLiteral(source, start, filePath)) continue;
119
+ if (literalMask[start] === 1) continue;
105
120
  const indent = header[1];
106
121
  const prefix = header[2];
107
122
  const scope = header[3].toLowerCase();
@@ -127,8 +142,9 @@ function parseLineBlocks(source, filePath) {
127
142
  }
128
143
 
129
144
  const raw = source.slice(start, end);
145
+ reserveBlock(blockBudget, filePath);
130
146
  const block = createBlock({
131
- source,
147
+ lineStarts,
132
148
  filePath,
133
149
  raw,
134
150
  body: bodyLines.join("\n"),
@@ -154,24 +170,28 @@ function parseLineBlocks(source, filePath) {
154
170
  return blocks;
155
171
  }
156
172
 
157
- function parseUnterminatedBlockComments(source, filePath, parsedBlocks) {
173
+ function parseUnterminatedBlockComments(source, filePath, parsedBlocks, literalMask, lineStarts, blockBudget) {
158
174
  const blocks = [];
175
+ const parsedIntervals = [...parsedBlocks].sort((left, right) => left.start - right.start);
159
176
  for (const definition of BLOCK_PATTERNS) {
177
+ let intervalIndex = 0;
160
178
  definition.opening.lastIndex = 0;
161
179
  for (const match of source.matchAll(definition.opening)) {
162
180
  const tokenStart = match.index ?? 0;
163
- if (isInsideStringLiteral(source, tokenStart, filePath)) continue;
164
- if (parsedBlocks.some((block) => tokenStart >= block.start && tokenStart < block.end)) continue;
181
+ if (literalMask[tokenStart] === 1) continue;
182
+ while (parsedIntervals[intervalIndex]?.end <= tokenStart) intervalIndex += 1;
183
+ const interval = parsedIntervals[intervalIndex];
184
+ if (interval && tokenStart >= interval.start && tokenStart < interval.end) continue;
165
185
  const bodyStart = tokenStart + match[0].length;
166
- if (source.indexOf(definition.terminator, bodyStart) >= 0) continue;
167
186
 
168
187
  const lineStart = source.lastIndexOf("\n", tokenStart - 1) + 1;
169
188
  const leading = source.slice(lineStart, tokenStart);
170
189
  const start = /^\s*$/u.test(leading) ? lineStart : tokenStart;
171
190
  const indent = start === lineStart ? leading : "";
172
191
  const end = source.length;
192
+ reserveBlock(blockBudget, filePath);
173
193
  const block = createBlock({
174
- source,
194
+ lineStarts,
175
195
  filePath,
176
196
  raw: source.slice(start, end),
177
197
  body: source.slice(bodyStart),
@@ -192,15 +212,17 @@ function parseUnterminatedBlockComments(source, filePath, parsedBlocks) {
192
212
  return blocks;
193
213
  }
194
214
 
195
- function isInsideStringLiteral(source, targetOffset, filePath) {
215
+ function buildLiteralMask(source, filePath) {
196
216
  const extension = filePath.toLowerCase().match(/\.[a-z0-9]+$/u)?.[0] ?? "";
197
217
  const hashComments = [".py", ".rb", ".sh", ".bash", ".zsh"].includes(extension);
198
218
  const dashComments = extension === ".sql";
199
219
  const tripleQuotes = extension === ".py";
200
220
  let state = "normal";
201
221
  let escaped = false;
222
+ const mask = new Uint8Array(source.length);
202
223
 
203
- for (let index = 0; index < targetOffset; index += 1) {
224
+ for (let index = 0; index < source.length; index += 1) {
225
+ mask[index] = state === "normal" ? 0 : 1;
204
226
  const character = source[index];
205
227
  const next = source[index + 1];
206
228
  const nextTwo = source.slice(index, index + 3);
@@ -284,15 +306,22 @@ function isInsideStringLiteral(source, targetOffset, filePath) {
284
306
  }
285
307
  }
286
308
 
287
- return state !== "normal";
309
+ return mask;
288
310
  }
289
311
 
290
312
  function looksLikeRustCharacterLiteral(source, offset) {
291
313
  return /^'(?:\\.|[^'\\\r\n])'/u.test(source.slice(offset));
292
314
  }
293
315
 
294
- function createBlock({ source, filePath, raw, body, scope, style, start, end, indent, prefix }) {
295
- const startLine = lineAtOffset(source, start);
316
+ function reserveBlock(blockBudget, filePath) {
317
+ blockBudget.count += 1;
318
+ if (blockBudget.count > MAX_BLOCKS_PER_FILE) {
319
+ throw new Error(`${filePath} exceeds the ${MAX_BLOCKS_PER_FILE}-block parser limit.`);
320
+ }
321
+ }
322
+
323
+ function createBlock({ lineStarts, filePath, raw, body, scope, style, start, end, indent, prefix }) {
324
+ const startLine = lineAtOffsetFromStarts(lineStarts, start);
296
325
  const bodyStartLine = style === "line" ? startLine + 1 : startLine;
297
326
  const parsed = parseBody(body, bodyStartLine);
298
327
  const card = materializeCard(scope, parsed.entries);
@@ -310,7 +339,7 @@ function createBlock({ source, filePath, raw, body, scope, style, start, end, in
310
339
  start,
311
340
  end,
312
341
  startLine,
313
- endLine: lineAtOffset(source, Math.max(start, end - 1)),
342
+ endLine: lineAtOffsetFromStarts(lineStarts, Math.max(start, end - 1)),
314
343
  newline: detectNewline(raw),
315
344
  };
316
345
  }
package/src/project.js CHANGED
@@ -12,7 +12,7 @@ stability=architecture
12
12
 
13
13
  import { readFile } from "node:fs/promises";
14
14
  import { loadConfig } from "./config.js";
15
- import { findAttachedDeclaration, extractImports } from "./declaration.js";
15
+ import { createDeclarationScanContext, findAttachedDeclaration, extractImports } from "./declaration.js";
16
16
  import { collectSourceFiles } from "./files.js";
17
17
  import { parseLlmnavBlocks } from "./parser.js";
18
18
  import { loadRegistry } from "./registry.js";
@@ -34,6 +34,7 @@ export async function scanProject(root, options = {}) {
34
34
  sourceBytes += Buffer.byteLength(source);
35
35
  semanticBytes += blocks.reduce((sum, block) => sum + Buffer.byteLength(block.raw), 0);
36
36
  const imports = extractImports(source, relativePath);
37
+ const declarationScanContext = createDeclarationScanContext(source);
37
38
  const fileRecord = {
38
39
  absolutePath,
39
40
  relativePath,
@@ -47,7 +48,7 @@ export async function scanProject(root, options = {}) {
47
48
  };
48
49
  fileRecords.push(fileRecord);
49
50
  for (const block of blocks) {
50
- const declaration = findAttachedDeclaration(source, block, relativePath);
51
+ const declaration = findAttachedDeclaration(source, block, relativePath, declarationScanContext);
51
52
  records.push({
52
53
  root,
53
54
  absolutePath,
@@ -12,7 +12,7 @@ stability=architecture
12
12
 
13
13
  import path from "node:path";
14
14
  import { loadConfig } from "./config.js";
15
- import { approximateTokens, compareText, readText, sha256, stableJson, stableStringify, toPosix } from "./util.js";
15
+ import { approximateTokens, assertNoSymlinkTraversal, compareText, readText, sha256, stableJson, stableStringify, toPosix } from "./util.js";
16
16
 
17
17
  export const PROMPT_BUNDLE_SCHEMA_VERSION = 1;
18
18
 
@@ -62,7 +62,11 @@ export function isCompatiblePromptPrefixBundle(bundle, repositoryId = undefined)
62
62
  export async function loadPromptPrefixBundle(root) {
63
63
  const { config } = await loadConfig(root);
64
64
  const relativePath = `${toPosix(config.generation.cacheDirectory).replace(/\/+$/u, "")}/prompt-prefix.json`;
65
- const content = await readText(path.join(root, relativePath), "");
65
+ const bundlePath = path.join(root, relativePath);
66
+ const manifestPath = path.join(root, config.generation.cacheDirectory, "manifest.json");
67
+ await assertNoSymlinkTraversal(root, bundlePath, relativePath);
68
+ await assertNoSymlinkTraversal(root, manifestPath, `${config.generation.cacheDirectory}/manifest.json`);
69
+ const content = await readText(bundlePath, "");
66
70
  if (!content) throw new Error(`Missing generated prompt bundle ${relativePath}.`);
67
71
  let bundle;
68
72
  try {
@@ -73,7 +77,7 @@ export async function loadPromptPrefixBundle(root) {
73
77
  if (!isCompatiblePromptPrefixBundle(bundle, config.repositoryId)) {
74
78
  throw new Error(`Incompatible generated prompt bundle ${relativePath}.`);
75
79
  }
76
- const manifestContent = await readText(path.join(root, config.generation.cacheDirectory, "manifest.json"), "");
80
+ const manifestContent = await readText(manifestPath, "");
77
81
  const manifest = manifestContent ? JSON.parse(manifestContent) : null;
78
82
  if (manifest?.files?.[relativePath] !== sha256(content)) throw new Error(`Prompt bundle hash does not match manifest.json.`);
79
83
  return bundle;
package/src/registry.js CHANGED
@@ -11,13 +11,14 @@ stability=contract
11
11
 
12
12
  import path from "node:path";
13
13
  import { ID_PATTERN } from "./spec.js";
14
- import { atomicWrite, parseJsonLines, readText } from "./util.js";
14
+ import { assertNoSymlinkTraversal, atomicWrite, parseJsonLines, readText } from "./util.js";
15
15
 
16
16
  const REGISTRY_STATES = new Set(["active", "redirect", "replaced", "retired"]);
17
17
  const REGISTRY_KEYS = new Set(["id", "state", "to", "by"]);
18
18
 
19
19
  export async function loadRegistry(root) {
20
20
  const registryPath = path.join(root, ".llmnav", "ids.jsonl");
21
+ await assertNoSymlinkTraversal(root, registryPath, ".llmnav/ids.jsonl");
21
22
  const text = await readText(registryPath, "");
22
23
  const parsed = parseJsonLines(text, registryPath);
23
24
  const byId = new Map();
@@ -72,8 +73,13 @@ export async function loadRegistry(root) {
72
73
  }
73
74
 
74
75
  export async function ensureActiveIds(root, registry, ids) {
76
+ const registryPath = path.join(root, ".llmnav", "ids.jsonl");
77
+ if (path.resolve(registry.registryPath) !== path.resolve(registryPath)) {
78
+ throw new Error("Registry path does not belong to the requested project root.");
79
+ }
80
+ await assertNoSymlinkTraversal(root, registryPath, ".llmnav/ids.jsonl");
75
81
  const { records, changed } = mergeActiveIds(registry, ids);
76
- if (changed) await atomicWrite(registry.registryPath, renderRegistryRecords(records));
82
+ if (changed) await atomicWrite(registryPath, renderRegistryRecords(records));
77
83
  return { records, changed };
78
84
  }
79
85
 
package/src/search.js CHANGED
@@ -14,7 +14,7 @@ import path from "node:path";
14
14
  import { loadConfig } from "./config.js";
15
15
  import { resolveRegistryId, loadRegistry } from "./registry.js";
16
16
  import { renderCompactCard } from "./generator.js";
17
- import { approximateTokens, compareText, readJsonSafe, readText, sha256, toPosix, truncateToTokenBudget } from "./util.js";
17
+ import { assertNoSymlinkTraversal, approximateTokens, compareText, readJsonSafe, readText, sha256, toPosix, truncateToTokenBudget } from "./util.js";
18
18
  import {
19
19
  buildInvertedIndex,
20
20
  isCompatibleSearchIndex,
@@ -33,12 +33,21 @@ export async function loadSearchData(root) {
33
33
  const { config } = await loadConfig(root);
34
34
  await recoverGenerationTransaction(root, { cacheDirectory: config.generation.cacheDirectory });
35
35
  const cacheRoot = path.join(root, config.generation.cacheDirectory);
36
- const index = await readJsonSafe(path.join(cacheRoot, "index.json"), null);
36
+ await assertNoSymlinkTraversal(root, cacheRoot, config.generation.cacheDirectory);
37
+ const indexPath = path.join(cacheRoot, "index.json");
38
+ const manifestPath = path.join(cacheRoot, "manifest.json");
39
+ const graphPath = path.join(cacheRoot, "graph.json");
40
+ const searchPath = path.join(cacheRoot, "search-index.json");
41
+ const lexiconPath = path.join(root, ".llmnav", "lexicon.json");
42
+ for (const managedPath of [indexPath, manifestPath, graphPath, searchPath, lexiconPath]) {
43
+ await assertNoSymlinkTraversal(root, managedPath, toPosix(path.relative(root, managedPath)));
44
+ }
45
+ const index = await readJsonSafe(indexPath, null);
37
46
  if (!index) throw new Error("No generated index found. Run `llmnav generate` first.");
38
- const lexicon = await readJsonSafe(path.join(root, ".llmnav", "lexicon.json"), { version: 1, aliases: {} });
39
- const manifest = await readJsonSafe(path.join(cacheRoot, "manifest.json"), null);
47
+ const lexicon = await readJsonSafe(lexiconPath, { version: 1, aliases: {} });
48
+ const manifest = await readJsonSafe(manifestPath, null);
40
49
  const graphRelative = `${toPosix(config.generation.cacheDirectory).replace(/\/+$/u, "")}/graph.json`;
41
- const graphText = await readText(path.join(cacheRoot, "graph.json"), null);
50
+ const graphText = await readText(graphPath, null);
42
51
  let graph = null;
43
52
  if (graphText !== null) {
44
53
  try {
@@ -54,7 +63,7 @@ export async function loadSearchData(root) {
54
63
  );
55
64
  if (!graphMatches) graph = null;
56
65
  const searchRelative = `${toPosix(config.generation.cacheDirectory).replace(/\/+$/u, "")}/search-index.json`;
57
- const searchText = await readText(path.join(cacheRoot, "search-index.json"), null);
66
+ const searchText = await readText(searchPath, null);
58
67
  let searchIndex = null;
59
68
  if (searchText !== null) {
60
69
  try {
package/src/spec.js CHANGED
@@ -10,7 +10,7 @@ rel=workflow>llmnav.rules.validate
10
10
  stability=contract
11
11
  */
12
12
 
13
- export const PACKAGE_VERSION = "0.6.2";
13
+ export const PACKAGE_VERSION = "0.6.6";
14
14
  export const SPEC_VERSION = "1";
15
15
 
16
16
  export const SCOPES = Object.freeze(["file", "module", "symbol"]);
@@ -47,6 +47,7 @@ export async function acquireGenerationLock(root, options = {}) {
47
47
  const controlDirectory = path.join(root, ".llmnav");
48
48
  const lockPath = path.join(controlDirectory, "generation.lock");
49
49
  await assertNoSymlinkTraversal(root, controlDirectory, ".llmnav");
50
+ await assertNoSymlinkTraversal(root, lockPath, ".llmnav/generation.lock");
50
51
  await mkdir(controlDirectory, { recursive: true });
51
52
  const ownerId = options.ownerId ?? createTransactionId();
52
53
  const timeoutMs = options.timeoutMs ?? DEFAULT_LOCK_TIMEOUT_MS;
@@ -95,6 +96,8 @@ export async function commitGeneratedCache(root, cacheDirectory, artifacts, opti
95
96
  const transactionsDirectory = path.join(controlDirectory, ".transactions");
96
97
  const journalPath = path.join(controlDirectory, "generation-transaction.json");
97
98
  await assertNoSymlinkTraversal(root, controlDirectory, ".llmnav");
99
+ await assertNoSymlinkTraversal(root, transactionsDirectory, ".llmnav/.transactions");
100
+ await assertNoSymlinkTraversal(root, journalPath, ".llmnav/generation-transaction.json");
98
101
  await assertNoSymlinkTraversal(root, cachePath, cacheRelative);
99
102
  const recovery = await recoverGenerationTransaction(root, {
100
103
  cacheDirectory: cacheRelative,
@@ -106,6 +109,7 @@ export async function commitGeneratedCache(root, cacheDirectory, artifacts, opti
106
109
  const transactionPath = path.join(transactionsDirectory, transactionId);
107
110
  const stagePath = path.join(transactionPath, "stage");
108
111
  const backupPath = path.join(transactionPath, "backup");
112
+ await assertNoSymlinkTraversal(root, transactionPath, relativePosix(root, transactionPath));
109
113
  const hadExistingCacheAtStart = await pathExists(cachePath);
110
114
  let controlRecords = [];
111
115
  await mkdir(stagePath, { recursive: true });
@@ -218,6 +222,11 @@ export async function recoverGenerationTransaction(root, options = {}) {
218
222
  const controlDirectory = path.join(root, ".llmnav");
219
223
  await assertNoSymlinkTraversal(root, controlDirectory, ".llmnav");
220
224
  const journalPath = path.join(controlDirectory, "generation-transaction.json");
225
+ await assertNoSymlinkTraversal(root, journalPath, ".llmnav/generation-transaction.json");
226
+ if (options.cacheDirectory) {
227
+ const configuredCacheDirectory = projectRelativePath(options.cacheDirectory, "cacheDirectory");
228
+ await assertNoSymlinkTraversal(root, path.join(root, configuredCacheDirectory), configuredCacheDirectory);
229
+ }
221
230
  const journal = await readJson(journalPath, null);
222
231
  if (!journal) return { recovered: false, action: "none" };
223
232
  validateJournal(root, journal, options.cacheDirectory);
package/src/util.js CHANGED
@@ -77,11 +77,26 @@ export function relativePosix(root, absolutePath) {
77
77
  }
78
78
 
79
79
  export function lineAtOffset(source, offset) {
80
- let line = 1;
81
- for (let index = 0; index < offset; index += 1) {
82
- if (source.charCodeAt(index) === 10) line += 1;
80
+ return lineAtOffsetFromStarts(buildLineStarts(source), offset);
81
+ }
82
+
83
+ export function buildLineStarts(source) {
84
+ const starts = [0];
85
+ for (let index = 0; index < source.length; index += 1) {
86
+ if (source.charCodeAt(index) === 10) starts.push(index + 1);
87
+ }
88
+ return starts;
89
+ }
90
+
91
+ export function lineAtOffsetFromStarts(lineStarts, offset) {
92
+ let low = 0;
93
+ let high = lineStarts.length;
94
+ while (low < high) {
95
+ const middle = Math.floor((low + high) / 2);
96
+ if (lineStarts[middle] <= offset) low = middle + 1;
97
+ else high = middle;
83
98
  }
84
- return line;
99
+ return Math.max(1, low);
85
100
  }
86
101
 
87
102
  export function offsetAtLine(source, targetLine) {