jcodemunch-mcp 1.108.332__tar.gz → 1.108.333__tar.gz

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "jcodemunch",
   "displayName": "jCodeMunch",
-  "version": "1.108.332",
+  "version": "1.108.333",
   "description": "Token-efficient MCP server for source code exploration via tree-sitter AST parsing",
   "author": {
     "name": "J. Gravelle",

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/CHANGELOG.md

@@ -2,6 +2,77 @@
 
 ## [Unreleased]
 
+## [1.108.333] - 2026-10-07 - the parse budget stops a slow parse again on a fresh install
+
+### Fixed
+
+- **`init`'s index line reports what the indexer did (reported by Dave).** After indexing,
+  `jcodemunch-mcp init` printed `indexed <folder> (? files, ? symbols)`. The line read
+  `files_indexed` and `symbols_indexed` from the indexer's result. No run returns the second,
+  and only a run cut short by the file cap returns the first. A full index answers `file_count` and `symbol_count`, a run that
+  re-indexes some files answers `changed`, `new`, `deleted` and `symbol_count`. So the line has
+  read `? symbols` for every index `init` ran since the command arrived (2026-04-03), and
+  `? files` for all but a capped one. It also
+  read `indexed` for a run the indexer had refused, a folder with no source file among them.
+  The line now has one form per answer (`evidence/init_index_line.txt`, the real indexer on a
+  scratch project): `indexed <folder> (2 files, 3 symbols)`,
+  `indexed <folder> (1 changed, 1 new, 1 deleted, 3 symbols)`,
+  `<folder> is up to date (nothing changed since the last index)`, and
+  `indexing failed: No source files found`. A count the result does not carry is left out, and
+  a run the file cap cut short adds `the file cap was reached, 2 of 4 files are in the index` to
+  whichever form it has.
+  Nothing had tested the line: the `init` tests replace the function that writes it. The new
+  tests run the real indexer on a project they own
+  (`tests/test_init_index_summary_reads_the_indexers_result.py`). The index itself was correct.
+  Not changed: the indexer's `warnings` (a file skipped by the parse budget, a provider skipped)
+  are still not printed by `init`. A test that reads the same absent key is LEDGER L-134.
+
+- **Three more ways a macro got the wrong docstring (LEDGER L-129, L-130, L-131).** 1.108.332
+  gave a dbt or Jinja/Twig directive the `{# comment #}` that ends directly above it; its
+  review found three inputs the rule still read wrongly, measured through `parse_file` on
+  1.108.332 and then on the fix (`evidence/l129_l131_before_after.txt`). A comment that TRAILS
+  code on its line was taken by the directive on the next line: after
+  `{% macro a() %}select 1{% endmacro %} {# end of a #}`, `b` read `end of a` and reads nothing.
+  The comment must now be first on its line, with only other comments (or a `--` lead) before it
+  there. A `{#`
+  inside an expression opened a comment: with `{{ '{#' }}` in the macro above, `second` read
+  `' }}{% endmacro %}` and the line after it, and reads `about second`. The file is read as
+  Jinja reads it, `{{ ... }}`, `{% ... %}` and `{# ... #}` left to right, each to its first
+  closer. And a whitespace-control mark was dropped whatever the dialect: `{#~approx 5 rows#}`
+  in a `.sql` read `approx 5 rows` and reads `~approx 5 rows`; `{#+1 to the offset #}` in a
+  `.twig` read `1 to the offset` and reads `+1 to the offset`. `-` is a mark everywhere, `+` in
+  Jinja and dbt, `~` in Twig. Nothing inside `{% raw %}` or `{% verbatim %}` is a delimiter, and
+  a byte-order mark or a zero-width space before a comment is not text, so a file that starts
+  with a BOM keeps its first docstring. ⚠ The docstring changes on unchanged content, so
+  `PARSER_GENERATION` is 11 and an existing index re-parses once at its next index run; no
+  symbol id moves. ⚠ What reads WORSE than on 1.108.332 (LEDGER L-132, same evidence file):
+  a stray `{{` or `{%` outside a raw block that has a later closer is one expression up to that
+  closer, so `-- wrap in {{ to call` above a documented macro, with a `{{ x }}` further down,
+  costs that macro its docstring (`doc` before, nothing now); Jinja itself refuses such a file.
+  A closer inside a string in an expression (`{{ '}}' }}`) still ends the expression there, and
+  when the leftover holds an opener (`{{ '}} {%' }}`) the documented macro below it loses its
+  docstring too.
+  Not run: a dbt, Jinja or Twig corpus.
+
+- **The parse budget stops a slow parse again on a fresh install (reported by Dave).** The
+  per-file budget (`JCODEMUNCH_PARSE_BUDGET_SECONDS`, 1.108.329) stops tree-sitter's C parse
+  through `Parser.timeout_micros`. tree-sitter 0.26.0 removed that attribute, and the
+  dependency read `tree-sitter>=0.25` with no upper bound, so an install that resolved 0.26.0
+  could no longer stop a parse. Its only sign was one line at index time: `the installed
+  tree-sitter has no Parser.timeout_micros`. Measured with this tree's source and a 0.5 s budget
+  on a 405,035-byte Python file (`evidence/tree_sitter_026.txt`): on tree-sitter 0.25.2 the file
+  is stopped after 0.54 s; on 0.26.0 after 7.06 s, when the parse had run to its end. Through
+  `index_folder` the file is named as skipped on both (`parse exceeded the 0.5s budget`), so no
+  wrong symbol was served; the time was spent. CI never saw
+  it: it installs from `uv.lock`, which pins 0.25.2, and the guard that fails on a binding
+  without the attribute runs there. The range is now `>=0.25,<0.26`, so an upgrade moves such an
+  install back to a version that can stop a parse, and
+  `tests/test_tree_sitter_range_is_the_tested_range.py` fails when the declared range admits any
+  release past the locked minor, an exclusion such as `!=0.26.0` included. The warning now names
+  the remedy. Not changed: tree-sitter
+  0.26 is not supported; its replacement for the timeout, a progress callback, crashed the
+  interpreter when it was tried on 0.25.2 (LEDGER L-133).
+
 ## [1.108.332] - 2026-10-07 - a macro's docstring is the comment directly above it
 
 ### Fixed

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/ISSUE-HISTORY.md

@@ -2228,3 +2228,17 @@ The 1.108.331 entry, verbatim, dropped when it was compressed to a `Prior` line:
 The `Tests:` line's 1.108.329 count, verbatim:
 
 - ⚠ Prior (1.108.329): 14726 passed, 25 skipped, **0 failed** (14751 total).
+
+## Current State rotation (2026-10-07, release 1.108.333)
+
+The 1.108.330 entry, verbatim as it stood in `CLAUDE.md`:
+
+- **Prior (1.108.330):** **The parse budget bounds a file's Python-side time** (L-116): `parse_budget.checkpoint()` raises inside `parse_file` once the file's deadline has passed, and every loop on the parse path starts with one (rules: the Env Vars row; residue L-117, L-118). Also a bare Python module name resolves to the script directory beside the importer (#972, @whakomatic; L-120). Full text: `CHANGELOG.md`; verbatim in `ISSUE-HISTORY.md` (2026-10-06, release 1.108.331).
+
+The 1.108.332 entry, verbatim, dropped when it was compressed to a `Prior` line:
+
+- **Version:** 1.108.332 — **A macro's docstring is the comment directly above it** (L-126): `_PrecedingComments` (`parser/sql_preprocessor.py`) gives a dbt or Jinja/Twig directive the `{# #}` that ENDS where the text before it ends, less one whitespace-control mark per side. ⚠ `PARSER_GENERATION` is 10: a stored docstring moved on unchanged content. ⚠ Residue, each owing its own generation: L-129, L-130, L-131. Also three loops on the parse path are linear in the file (L-117; residue L-127, L-128). Forensics: `CHANGELOG.md`, `docs/workflows/LEDGER.md`.
+
+The `Tests:` line's 1.108.330 count, verbatim:
+
+- ⚠ Prior (1.108.330): 14783 passed, 25 skipped, **0 failed** (14808 total).

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: jcodemunch-mcp
-Version: 1.108.332
+Version: 1.108.333
 Summary: Token-efficient MCP server for source code exploration via tree-sitter AST parsing
 Project-URL: Homepage, https://github.com/jgravelle/jcodemunch-mcp
 Project-URL: Repository, https://github.com/jgravelle/jcodemunch-mcp
@@ -27,7 +27,7 @@ Requires-Dist: pathspec>=0.12.0
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: tree-sitter-fsharp==0.3.12
 Requires-Dist: tree-sitter-language-pack<1.0.0,>=0.7.0
-Requires-Dist: tree-sitter>=0.25
+Requires-Dist: tree-sitter<0.26,>=0.25
 Provides-Extra: all
 Requires-Dist: anthropic>=0.40.0; extra == 'all'
 Requires-Dist: anyio>=4.14.2; extra == 'all'
@@ -246,10 +246,10 @@ That's the highlight reel. The complete tour of 90+ tools, the MUNCH compact wir
 <!-- WHATSNEW:START -->
 #### What's new
 
+- **[v1.108.333](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.333)** (2026-10-07) — the parse budget stops a slow parse again on a fresh install
 - **[v1.108.332](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.332)** (2026-10-07) — a macro's docstring is the comment directly above it
 - **[v1.108.331](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.331)** (2026-10-06) — a full re-index keeps the embedding metadata beside its vectors
 - **[v1.108.330](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.330)** (2026-10-06) — the parse budget bounds a file's Python-side time
-- **[v1.108.329](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.329)** (2026-10-05) — the parse budget stops a slow parse
 <!-- WHATSNEW:END -->
 
 ---

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/README.md

@@ -158,10 +158,10 @@ That's the highlight reel. The complete tour of 90+ tools, the MUNCH compact wir
 <!-- WHATSNEW:START -->
 #### What's new
 
+- **[v1.108.333](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.333)** (2026-10-07) — the parse budget stops a slow parse again on a fresh install
 - **[v1.108.332](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.332)** (2026-10-07) — a macro's docstring is the comment directly above it
 - **[v1.108.331](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.331)** (2026-10-06) — a full re-index keeps the embedding metadata beside its vectors
 - **[v1.108.330](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.330)** (2026-10-06) — the parse budget bounds a file's Python-side time
-- **[v1.108.329](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.329)** (2026-10-05) — the parse budget stops a slow parse
 <!-- WHATSNEW:END -->
 
 ---

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/docs/harness/ARCHAEOLOGY.md

@@ -248,6 +248,7 @@ Criterion codes: 1 correctness, 2 token reduction, 3 freshness/caches/absence, 4
 | tests/test_init.py | `init` client detection, `_patch_mcp_config` backup/merge, CLAUDE.md/Cursor/Windsurf installs, hooks install, audit, and the client registry incl. Codex/opencode. | `e176e7a v1.21.26 Cursor/Windsurf rules + --demo`; `8c6f85b v1.108.35 fix CI-only break`; `c0a82da feat(init): register Codex CLI and opencode` | 6 | LOAD-BEARING | 44 tests; the largest init surface; per-client schema shapes are in test_init_client_schemas.py. |
 | tests/test_init_client_schemas.py | Each host gets its own config shape (Codex TOML with resolved binary and never `uvx`, opencode `mcp` array command, VS Code `servers`, Gemini/Cline detection). | `c0a82da feat(init): register Codex CLI and opencode, each on its own schema`; `8c7a6d8 feat(init): register Gemini CLI, Cline and VS Code/Copilot` | 6 | LOAD-BEARING | Docstring: a wrong schema produces no error (silent no-tools); cites CLIENTS.md multi-hour hang. |
 | tests/test_init_hooks_paths.py | Hook commands are written with forward slashes on Windows (bash eats backslashes) and re-running `init` never appends duplicate hooks (subcommand-keyed dedupe). | `105e5a1 fix: Windows hook path bash-escape eating + duplicate hook accumulation (v1.108.7)`; `9b375cd fix(hooks): ten confirmed defects` | 6 | LOAD-BEARING | Patches `_hook_invocation` to return literal `C:\Python314\...` and `C:\Program Files\...` paths (intentional inputs). |
+| tests/test_init_index_summary_reads_the_indexers_result.py | `init`'s index line reports what `index_folder` returned: files and symbols for a full index, changed/new/deleted and symbols for a run that re-indexed some files, `up to date` when nothing changed, `indexing failed` with the indexer's error for a refusal, and how many files the index holds when the file cap cut the walk short; never `?`, and no count the result lacks. | "fix: init's index line reads the keys the indexer returns" (a user report by email, 2026-10-07) | 6 | LOAD-BEARING | Five cases run the REAL indexer in a project and a store the test owns: the line read `symbols_indexed`, which no run returns, and `files_indexed`, which only a capped run returns, and the only tests near it (`test_init_minimal.py`, `test_init_policy_matches_surface.py`) replace `run_index` whole, so a mock would supply whichever keys its author expected. |
 | tests/test_init_minimal.py | `init --minimal` registers the MCP server and touches no other channel (no CLAUDE.md, rules, AGENTS.md, hooks, index, audit). | `aa4cb54 release: v1.108.22 — ... init --minimal ...`; `e3663c5 test: retire the src.jcodemunch_mcp package twin` | 6 | LOAD-BEARING | 3 tests, spies on every helper. |
 | tests/test_init_policy_matches_surface.py | The policy `init` writes names only tools the advertised surface carries — after `load_config`, honouring `tool_surface` and profile — and `--minimal --hooks` installs hooks (#397). | `d7e3b17 release: v1.108.206 - init wrote a policy for a decision it had not made yet`; `054755d (#507)`; `9cf763e refactor: break three import cycles` | 4 | LOAD-BEARING | Header warns patches must target `cli.policy`, not `init` re-exports (monkeypatch trap). |
 | tests/test_install_copilot_hooks.py | `init --copilot-hooks` writes `.github/hooks/hooks.json` with one postToolUse rule keyed by subcommand, idempotently. | `aede3b2 release: v1.81.0 — VS Code extension, Copilot hooks`; `1650270 test(hooks): assert the Copilot rule by subcommand, not by bare-name substring` | 6 | LOAD-BEARING | Asserts outcome not string shape (Windows `.EXE` absolute path). |
@@ -351,9 +352,10 @@ Criterion codes: 1 correctness, 2 token reduction, 3 freshness/caches/absence, 4
 | tests/test_paid_embeddings_optin.py | `_detect_provider` never selects a paid cloud embedding provider from an API key alone -- the explicit `*_EMBED_MODEL` is the opt-in and the local ONNX encoder wins at priority 0. | `2971bb2 test: pin that a bare cloud key cannot select a paid embedding provider`; docstring: suite-parity check for a jdocmunch defect, deliberately NOT a port of its gate | 8 | LOAD-BEARING | Pins a property jcm already had so a "simplification" cannot remove it; env-only, no network. |
 | tests/test_parity_map.py | `get_parity_map` classifies ported/renamed/diverged/added/orphaned symbols across two scopes, computes `parity_pct`, orders the port plan topologically with SCC grouping, errors on same scope/unindexed/empty, and is read-only. | `07bbb0c get_parity_map: correspondence-aware migration parity map (v1.108.111)`; docstring | 1 | LOAD-BEARING | Fixture is a legacy/ vs v2/ tree with commented expected outcomes per symbol. |
 | tests/test_parse_budget_cancels.py | An over-budget tree-sitter parse is stopped at `JCODEMUNCH_PARSE_BUDGET_SECONDS` and the file is named in `warnings`, on `parse_file`, a full index and an incremental re-index, at any file size; a stop is recognised without a clock; another thread's time is not charged to the file; a first index starts no wait thread; the installed binding can still cancel a parse. | LEDGER L-114 (2026-10-04): the thread wait could not return while the parse held the GIL | 5 | LOAD-BEARING | The slow input is L-113's shape; each cancelled case costs its 0.25 s budget. The fake-parser cases pin both directions of the stop rule and the expired-deadline branch (review rounds found mutants that passed an earlier form). `test_a_first_index_leaves_no_thread_behind` predates L-116's fix and still holds; every route is in `tests/test_parse_budget_python_side.py`. `OLD_WAIT_GATE` is the retired wait's 128 KiB threshold, kept as a fixture boundary. The binding canary fails when `Parser.timeout_micros` is gone |
+| tests/test_tree_sitter_range_is_the_tested_range.py | The `tree-sitter` range `pyproject.toml` declares admits the version `uv.lock` pins, admits no version of a later minor or major over a grid of them (so an exclusion such as `!=0.26.0` fails, twelve spellings pinned), refuses 0.26.0 by name, and equals the range the lock records; the requirement is read from the parsed TOML, every extra and dependency group included. | "fix: the tree-sitter range stops at the locked minor, so the parse budget can stop a parse" (a user report by email, 2026-10-07) | 7 | LOAD-BEARING | CI installs from the lock and a user installs from the range, so a guard that runs on the installed binding (`test_parse_budget_cancels.py`) says nothing about what a user gets. This reads the two files as text and compares them. Raising the cap means moving the lock in the same change; do not widen the range to make this pass. |
 | tests/test_parse_budget_python_side.py | A file's Python-side work is stopped at `JCODEMUNCH_PARSE_BUDGET_SECONDS` in the parsing thread and the file is named: the generic walker, dedicated parsers, a regex language, seven walkers on a tree that builds no symbol, and three quadratic loops that are no recursion. Over the source: every `for`/`while` in the six parse-path modules starts with a checkpoint (three named exemptions), no recursion lacks one, every module of `parser/` is classified. Also: a parser that swallows the stop builds nothing more; a parser-side stop raises at the next checkpoint; a file that ends past its deadline with no checkpoint having seen it is named; time the thread did not spend is not charged; a first index and a re-index name the same file the same way; every file is parsed on the calling thread; no clock is read outside a file or with the budget off. | LEDGER L-116 (2026-10-05): the thread wait was wall-clock, skipped one route and left its worker running | 5 | LOAD-BEARING | The clock is the test's: the deadline passes 5.5 s into a 5 s budget when the file's C parse returns, so nothing depends on a slow fixture and a far-over-only stop fails. ⚠ A new loop on the parse path fails the scan until its first statement is `parse_budget.checkpoint()`; do NOT add it to the exemption table to pass (each entry there has its reason beside it). A new module in `parser/` must be put on or off the path. The scans see statements, not a comprehension, a loop's iterable or one slow call (L-118) |
-| tests/test_parse_path_loops_are_linear.py | Three loops on the parse path are linear in the file: attributing calls to their enclosing symbol, the Razor `@code {` brace scan, and the dbt/Jinja directive loop. Each fix returns what the implementation it replaced returns (kept in the file as an oracle) on seeded generated input (since L-126 a directive's docstring is the one exception: the old loop's docstring was that defect, so the field is held to the rule, written the slow way beside the oracle), finishes a file twice as big before the oracle finishes the small one, and where `parse_budget.checkpoint()` sees the loop, passes under 2.6 times the checkpoints for twice the file. The Razor scan searches under 2.6 times the characters for twice the blocks when each opens an unclosed comment; the directive loop copies or searches under 2.6 times the characters for twice the directives; the call sweep raises under 2.6 times the line events for twice the calls. No module of `parser/` counts newlines up to an offset (`_line_numbers` is the one table), and that scan sees both spellings. Also: a call inside two symbols goes to the inner one; a brace in a string or a comment does not close a block; a macro keeps its line, its end and its `--` comment. | "fix: three loops on the parse path take time proportional to the file (LEDGER L-117)" | 5 | LOAD-BEARING | The race is against the oracle on the same machine in the same test, never against a typed number of seconds. A count of checkpoints is blind to work inside one C call (a slice, a `count`, a regex search), which is why the race is there too. The race is blind to a rescan cheaper than the old loop (a copy of the file per directive wins it), which is why the character and line-event counts are there: a new loop on these paths needs a count that can see its work, not only a race. The race is at TWICE the size, not more: coverage's C tracer (the 3.10 and 3.11 jobs) slows the fix and not the oracle, and at eight times the size the fix lost. Before changing a size, read the margins with the tracer on. The oracles are the old code and must not be edited to agree with a change. |
-| tests/test_a_macro_docstring_is_the_comment_above_it.py | A dbt macro, a Jinja template macro and a Twig macro and block each take the `{# comment #}` that ends directly above them as their docstring, through `parse_file`; the dashes of `{#- ... -#}` are not part of it; a comment elsewhere in the file, a stray `#}`, text that only ends like a comment and an unclosed comment are nobody's docstring; of two comments the nearer one is used; a `--` comment nearer than a Jinja one wins; 300 documented macros each keep their own; `PARSER_GENERATION` is at least 10. | "fix: a macro's docstring is the comment directly above it (LEDGER L-126)" | 1 | LOAD-BEARING | The lookup returned the FIRST comment of the file for any directive under a `#}`, so a one-macro fixture cannot fail on it: every case here has a first comment that is NOT the answer. The generation assertion is a floor, not an equality, so the next bump does not edit this file. |
+| tests/test_parse_path_loops_are_linear.py | Three loops on the parse path are linear in the file: attributing calls to their enclosing symbol, the Razor `@code {` brace scan, and the dbt/Jinja directive loop. Each fix returns what the implementation it replaced returns (kept in the file as an oracle) on seeded generated input (since L-126 a directive's docstring is the one exception: the old loop's docstring was that defect, so the field is held to the rule, written the slow way beside the oracle), finishes a file twice as big before the oracle finishes the small one, and where `parse_budget.checkpoint()` sees the loop, passes under 2.6 times the checkpoints for twice the file. The Razor scan searches under 2.6 times the characters for twice the blocks when each opens an unclosed comment; the directive loop copies or searches under 2.6 times the characters, and passes under 2.6 times the checkpoints, for twice the directives, over ten shapes; the call sweep raises under 2.6 times the line events for twice the calls. No module of `parser/` counts newlines up to an offset (`_line_numbers` is the one table), and that scan sees both spellings. Also: a call inside two symbols goes to the inner one; a brace in a string or a comment does not close a block; a macro keeps its line, its end and its `--` comment. | "fix: three loops on the parse path take time proportional to the file (LEDGER L-117)" | 5 | LOAD-BEARING | The race is against the oracle on the same machine in the same test, never against a typed number of seconds. A count of checkpoints is blind to work inside one C call (a slice, a `count`, a regex search), which is why the race is there too. The race is blind to a rescan cheaper than the old loop (a copy of the file per directive wins it), which is why the character and line-event counts are there: a new loop on these paths needs a count that can see its work, not only a race. The race is at TWICE the size, not more: coverage's C tracer (the 3.10 and 3.11 jobs) slows the fix and not the oracle, and at eight times the size the fix lost. Before changing a size, read the margins with the tracer on. The oracles are the old code and must not be edited to agree with a change. |
+| tests/test_a_macro_docstring_is_the_comment_above_it.py | A dbt macro, a Jinja template macro and a Twig macro and block each take the `{# comment #}` that ends directly above them as their docstring, through `parse_file`; the dashes of `{#- ... -#}` are not part of it; a comment that trails code on its line is nobody's, and one after a `--` lead is still the docstring (L-129); a `{#` inside `{{ ... }}` or `{% ... %}` opens no comment (L-130); `+` is a mark in Jinja and dbt and text in Twig, `~` the reverse (L-131); nothing inside `{% raw %}` or `{% verbatim %}` is a delimiter; a file that starts with a byte-order mark keeps its first docstring; one case pins a reading the ledger calls a loss (L-132: a stray `{{` with a later closer), so that a fix has to move it on purpose; a comment elsewhere in the file, a stray `#}`, text that only ends like a comment and an unclosed comment are nobody's docstring; of two comments the nearer one is used; a `--` comment nearer than a Jinja one wins; 300 documented macros each keep their own; `PARSER_GENERATION` is at least 11. | "fix: a macro's docstring is the comment directly above it (LEDGER L-126)" | 1 | LOAD-BEARING | The lookup returned the FIRST comment of the file for any directive under a `#}`, so a one-macro fixture cannot fail on it: every case here has a first comment that is NOT the answer. The generation assertion is a floor, not an equality, so the next bump does not edit this file. |
 | tests/test_parse_cache.py | Content-addressed parse cache is a passthrough when disabled, returns identical symbols on hit, and FIFO-evicts past `DEFAULT_MAX_ROWS`. | `c57b070 release: v1.108.40 -- content-addressed parse cache for shared-host indexing`; `444925e v1.108.41 -- bound the shared parse cache (FIFO eviction)` | 3 | LOAD-BEARING | Env-gated by `JCODEMUNCH_PARSE_CACHE`; compares `dataclasses.asdict` of symbols. |
 | tests/test_parse_warnings.py | `index_folder` reports `missing_extractors` for a language with symbols but no import extractor (Elixir), and `get_dead_code_v2` emits `framework_warning` when no entry points are found. | `b4db152 release: v1.21.19 -- Phase 5 uncertainty disclosure (T15-T18)`; docstring T17/T18 | 9 | LOAD-BEARING | Elixir is the representative gap; if an Elixir import extractor is added this fixture language must change. |
 | tests/test_parser.py | Core `parse_file` extraction for Python plus Lua, JS/TS const constants, and Astro frontmatter/components. | `ac91053 feat: add Lua language support (v1.0.1)`; `c984804 feat: extract JS/TS const declarations as constants`; `d1aa0e5 feat(parser): enhance Astro support ... regression tests` | 1 | LOAD-BEARING | Phase 1 file that accreted language cases; one `read_text` on a fixture. |

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/docs/workflows/LEDGER.md

@@ -176,7 +176,10 @@ with jjg's reason. Keep the row. To promote a row to an issue, set Status to
 | L-125 | 2026-10-06, CI on PR #984 | `tests/test_parse_budget_cancels.py::test_a_first_index_leaves_no_thread_behind` failed once on `full: test (windows-latest, 3.13)` (run 37517459287) and passed when that job was re-run with no change. PR #984 changes one package block of `uv.lock` and no code. The failing assertion is `result.get("symbol_count", 0) >= 2`, and the count read 1. The test indexes two files under a short real-time parse budget, one of them large, so a slow runner that spends the budget on the large file leaves one symbol: that reading fits the count and is not confirmed from the log. Second timing-sensitive test to fail once on a Windows runner that day (L-123 is the first). Not run: the test in a loop on a Windows runner. Second occurrence, 2026-10-07, PR #985: `full: test (ubuntu-latest, 3.10)` (run 37623527802), the same assertion (`assert (True is True and 1 >= 2)`, 1 failed of 14832), passed when that job was re-run with no change; the other seven matrix jobs passed. So it is not a Windows-runner effect. | `tests/test_parse_budget_cancels.py` | flake | low | OPEN |
 | L-126 | 2026-10-06, L-117 fix | A dbt macro (or a Jinja `macro`/`block`) that follows a `{# comment #}` is given the FIRST `{# ... #}` of the file as its docstring, whichever comment it follows. The lookup searches the text before the directive from the start of the file and takes the first match; it checks only that the text ends with `#}`. `evidence/l126_first_comment.txt` on `main`: `{# about first #}` above `first`, `{# about second #}` above `second` gives `second` the docstring `about first`. A macro under `--` lines is right. The L-117 fix keeps this as it was, so it moves no symbol field; fixing it changes docstrings on unchanged files and owes a `PARSER_GENERATION` bump. Not run: how many macros in a real dbt project sit under a second or later Jinja comment. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments`) | defect | high | FIXED 2026-10-07 (`fix/l126-macro-docstring-comment`). `_PrecedingComments` lists the file's comments once, left to right, and a directive takes the one that ENDS where the text before it ends; a `#}` that closes no comment falls through to the `--` lines. `evidence/l126_before_after.txt`, `parse_file`, `main` then the fix: `second` read `about first` and reads `about second`; a `.j2` `button` read `renders a field` and reads `renders a button`. `PARSER_GENERATION` 10: the docstring moves on unchanged content. Guard: `tests/test_a_macro_docstring_is_the_comment_above_it.py`; the L-117 oracle test holds every field but the docstring to the old loop and the docstring to the rule. Review round 1: the dashes of `{#- ... -#}` were served as part of the docstring (`- about m -`, the same on `main` for a file's first comment); they are dropped, in the same generation, with Jinja's `+` and Twig's `~` (review round 2), one mark per side. What the rule still gets wrong: L-129, L-130, L-131. Not run: a dbt or Jinja corpus. |
 | L-127 | 2026-10-06, L-117 fix | Three shapes on the paths L-117 fixed are still superlinear; each time about quadruples when the file doubles, the same on `main` and after the fix (`evidence/l117_before_after.txt`, the residue lines, budget off). (1) Razor `@code {` blocks nested in each other: every block's body is parsed as C#, and an outer body holds the inner ones, so 150 nested blocks took 0.458 s and 300 took 1.846 s. Not valid Razor; a real file has one or two blocks. (2) The directive regex (`_directive_pattern`) on an unclosed `{% macro m(`: each one reads on for its `)`, so 2000 such lines took 0.681 s and 4000 took 2.577 s. One regex call, so the budget cannot stop it part-way: the L-118 class, and not in L-118's list. (3) A directive on a `--` line takes every `--` line above it as its docstring, so N macros each on its own `--` line produce docstrings that total N * N: 500 took 0.124 s and 1000 took 0.432 s. The output is that size by its own definition. | `src/jcodemunch_mcp/parser/extractor.py` (`_parse_razor_symbols`); `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_directive_pattern`, `_PrecedingComments.before`) | defect | low | OPEN |
-| L-128 | 2026-10-07, L-117 review round 3 | The count instruments of `tests/test_parse_path_loops_are_linear.py` wrap the spellings the old loops used and nothing else, so a per-item rescan in another spelling passes all 35 tests. The reviewer put five back into a copy of the fix and each stayed green: `text.rfind(...)`, `x in text`, `text.partition(...)` and a slice of the undecoded bytes per directive, and `list(callable_syms)` per call (one line event, however long the list). None of the five has been in the tree. A list membership test per call is seen by the wall-clock race alone. The durable form is a counted `str` that fails on any method it does not list. Also `monkeypatch.undo()` inside the two counting helpers undoes the autouse `_every_language_enabled` patch for the rest of that test; no caller parses a file after a helper today. | `tests/test_parse_path_loops_are_linear.py` (`_characters_read_by_the_directive_loop`, `_line_events`, `_characters_searched`) | test | low | OPEN |
-| L-129 | 2026-10-07, L-126 review round 1 | A Jinja comment that trails code on its own line is taken as the docstring of the directive on the NEXT line: `{% macro a() %}x{% endmacro %} {# end of a #}` then `{% macro b() %}` gives `b` the docstring `end of a`. The `--` branch requires a comment line to start with `--`; the Jinja branch asks only that the comment END where the text before the directive ends. The same on `main` when that comment is the file's first. `tests/test_a_macro_docstring_is_the_comment_above_it.py` pins the same-line reading (`{# a #} x {# b #}` gives `b`), so a fix inverts that case. ⚠ A fix moves stored docstrings on unchanged content and owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments.before`) | defect | low | OPEN |
-| L-130 | 2026-10-07, L-126 review round 1 | The comment table of `_PrecedingComments` reads `{#` wherever it stands, an expression or a string included: `{% macro first() %}{{ '{#' }}{% endmacro %}`, then `{# about second #}`, then `{% macro second() %}` gives `second` the docstring `' }}{% endmacro %}` + newline + `{# about second` (code in a docstring). `main` gave `about first`: wrong before, wrong differently now. Jinja's own lexer reads `{{ ... }}`, `{% ... %}` and `{# ... #}` left to right, so a `{#` inside an expression opens no comment; `JINJA_PATTERN` in the same module already lexes that way. ⚠ A fix owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments.__init__`) | defect | low | OPEN |
-| L-131 | 2026-10-07, L-126 review round 3 | The whitespace-control marks dropped from a Jinja comment's ends are the same three for every dialect (`-`, `+`, `~`), and `~` is a mark in Twig only, `+` in Jinja only. A comment whose text starts or ends flush against the delimiter with the other dialect's mark loses that character: in a `.sql`, `{#~approx 5 rows#}` reads `approx 5 rows`; a Twig `{#+1 to the offset #}` reads `1 to the offset`. One punctuation character, and only with no space beside the delimiter (`{# ~approx #}` is untouched). The exact form passes the dialect's marks in beside `directive_keywords`. ⚠ A fix owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_WHITESPACE_CONTROL_MARKS`, `_PrecedingComments.before`) | defect | low | OPEN |
+| L-128 | 2026-10-07, L-117 review round 3 | The count instruments of `tests/test_parse_path_loops_are_linear.py` wrap the spellings the old loops used and nothing else, so a per-item rescan in another spelling passes all 35 tests. The reviewer put five back into a copy of the fix and each stayed green: `text.rfind(...)`, `x in text`, `text.partition(...)` and a slice of the undecoded bytes per directive, and `list(callable_syms)` per call (one line event, however long the list). None of the five has been in the tree. A list membership test per call is seen by the wall-clock race alone. The durable form is a counted `str` that fails on any method it does not list. Also `monkeypatch.undo()` inside the two counting helpers undoes the autouse `_every_language_enabled` patch for the rest of that test; no caller parses a file after a helper today. The same holds for the walk L-129 added (`_is_first_on_its_line`): its review put back `text.rfind(...)` with `set(self._line_starts)` per step, and `dict(self._comment_body_start).get(pos)` per step, and all tests stayed green. Round 2 of that review: dropping the cache write in `_first_text_of` passes too; no shape has long leading whitespace before a chain of comments on one line. | `tests/test_parse_path_loops_are_linear.py` (`_characters_read_by_the_directive_loop`, `_line_events`, `_characters_searched`) | test | low | OPEN |
+| L-129 | 2026-10-07, L-126 review round 1 | A Jinja comment that trails code on its own line is taken as the docstring of the directive on the NEXT line: `{% macro a() %}x{% endmacro %} {# end of a #}` then `{% macro b() %}` gives `b` the docstring `end of a`. The `--` branch requires a comment line to start with `--`; the Jinja branch asks only that the comment END where the text before the directive ends. The same on `main` when that comment is the file's first. `tests/test_a_macro_docstring_is_the_comment_above_it.py` pins the same-line reading (`{# a #} x {# b #}` gives `b`), so a fix inverts that case. ⚠ A fix moves stored docstrings on unchanged content and owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments.before`) | defect | low | FIXED 2026-10-07 (`fix/l129-l131-macro-docstring-residue`). The comment must be first on its line, with only other comments before it there (`_PrecedingComments._is_first_on_its_line`). `evidence/l129_l131_before_after.txt`: `b` read `end of a` and reads nothing. The pinned `{# a #} x {# b #}` case is inverted. Review round 1: a byte-order mark or a zero-width space before the comment is not text (a file starting with a BOM lost its first docstring in the first form of this fix). `PARSER_GENERATION` 11, shared with L-130 and L-131. |
+| L-130 | 2026-10-07, L-126 review round 1 | The comment table of `_PrecedingComments` reads `{#` wherever it stands, an expression or a string included: `{% macro first() %}{{ '{#' }}{% endmacro %}`, then `{# about second #}`, then `{% macro second() %}` gives `second` the docstring `' }}{% endmacro %}` + newline + `{# about second` (code in a docstring). `main` gave `about first`: wrong before, wrong differently now. Jinja's own lexer reads `{{ ... }}`, `{% ... %}` and `{# ... #}` left to right, so a `{#` inside an expression opens no comment; `JINJA_PATTERN` in the same module already lexes that way. ⚠ A fix owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments.__init__`) | defect | low | FIXED 2026-10-07 (`fix/l129-l131-macro-docstring-residue`). The comment table reads `{{`, `{%` and `{#` left to right, each to its first closer; a kind with no closer left is text from there on. `evidence/l129_l131_before_after.txt`: `second` reads `about second`. Nothing inside `{% raw %}` or `{% verbatim %}` is a delimiter (review round 1: the first form of this fix lost the docstring below a raw block holding a `{{`). What the lexer reads worse than 1.108.332 did: L-132. `PARSER_GENERATION` 11. |
+| L-131 | 2026-10-07, L-126 review round 3 | The whitespace-control marks dropped from a Jinja comment's ends are the same three for every dialect (`-`, `+`, `~`), and `~` is a mark in Twig only, `+` in Jinja only. A comment whose text starts or ends flush against the delimiter with the other dialect's mark loses that character: in a `.sql`, `{#~approx 5 rows#}` reads `approx 5 rows`; a Twig `{#+1 to the offset #}` reads `1 to the offset`. One punctuation character, and only with no space beside the delimiter (`{# ~approx #}` is untouched). The exact form passes the dialect's marks in beside `directive_keywords`. ⚠ A fix owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_WHITESPACE_CONTROL_MARKS`, `_PrecedingComments.before`) | defect | low | FIXED 2026-10-07 (`fix/l129-l131-macro-docstring-residue`). `extract_dbt_directives` takes `whitespace_marks` (`JINJA_WHITESPACE_MARKS` by default, `TWIG_WHITESPACE_MARKS` from `template_shared` for Twig). `evidence/l129_l131_before_after.txt`: `{#~approx 5 rows#}` in a `.sql` reads `~approx 5 rows`; `{#+1 to the offset #}` in a `.twig` reads `+1 to the offset`. `PARSER_GENERATION` 11. |
+| L-132 | 2026-10-07, L-129/L-130/L-131 review round 1 | Two inputs read worse after L-130's lexer than on 1.108.332 (`evidence/l129_l131_before_after.txt`, the L-132 line; the second from the review's probe). (1) A stray `{{` or `{%` outside a raw block that has a later closer of its kind is read as one expression up to that closer, so every comment between is not a comment: `-- wrap in {{ to call`, then `{# doc #}`, then `{% macro m() %}`, with a `{{ x }}` further down, gave `m` the docstring `doc` and gives it nothing. Jinja refuses such a file, so a rendered dbt project cannot hold one; a `.sql` that is never rendered can. `test_a_stray_opener_with_a_later_closer_is_one_expression_to_that_closer` pins the reading. (2) A closer inside a string in an expression still ends the expression there, and the leftover can pair with a later closer: `{{ '}} {%' }}` above a documented macro costs it its docstring. Reading strings inside an expression needs a state-shared scan to stay linear (the L-117 Razor problem). A third the review found, a comment after a `--` lead (`-- note {# doc #}`), was fixed in the same change: a `--` lead is a comment, so `doc` is still the docstring. Review round 2 added two readings that are odd and not worse than 1.108.332: a `--` that starts a continuation line INSIDE a comment trailing code counts as a `--` lead (`x {# a`, newline, `-- still a #} {# b #}` gives `b`, where the same without the `--` gives nothing); and a line holding only a zero-width space is a blank line to the `--` branch, so `-- a`, that line, `-- b` reads both lines where 1.108.332 read `b`. ⚠ A fix moves stored docstrings and owes its own `PARSER_GENERATION` bump. | `src/jcodemunch_mcp/parser/sql_preprocessor.py` (`_PrecedingComments.__init__`, `before`) | defect | low | OPEN |
+| L-133 | 2026-10-07, a user report by email (Dave) | tree-sitter 0.26 is not supported: it removed `Parser.timeout_micros`, the one mechanism the parse budget has for stopping a C parse, so the dependency is capped at `<0.26` (`evidence/tree_sitter_026.txt`: a 0.5 s budget stopped a 405,035-byte file after 0.54 s on 0.25.2 and after 7.06 s on 0.26.0). To probe 0.26 again: the capped range no longer resolves it, so resolve this project's requirements, set the `tree-sitter` line to the version wanted, and run with `uv run --no-project --with-requirements` and `PYTHONPATH=src` (the evidence file's header). The replacement is `parse(progress_callback=)`; on 0.25.2 it was ignored for a bytestring and killed the interpreter with an access violation with a read callback, on all eight variants probed (`parser/parse_budget.py` module docstring). Not run: the callback on 0.26.0 itself. Until it works, a host that needs tree-sitter 0.26 for another package cannot install both. | `src/jcodemunch_mcp/parser/parse_budget.py` (`bind`, `_BudgetedParser`); `pyproject.toml` | enhancement | medium | OPEN |
+| L-134 | 2026-10-07, a user report by email (Dave), second finding | `tests/test_suggest_queries.py:28` reads `symbols_indexed` off `index_folder`'s result with a default of 0. No run returns that key (the indexer answers `symbol_count`), so the condition is true for any successful run and the `empty index` branch does not depend on the count it names. The branch is also not reached today: the fixture holds only a `README.md`, so the indexer refuses with `No source files found`. Found while fixing the same key in `cli/init.py`. To close: a fixture the indexer accepts with no symbol in it, and `symbol_count`. | `tests/test_suggest_queries.py` (`test_empty_index_returns_error`) | test | low | OPEN |

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "jcodemunch-mcp"
-version = "1.108.332"
+version = "1.108.333"
 description = "Token-efficient MCP server for source code exploration via tree-sitter AST parsing"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -64,7 +64,13 @@ dependencies = [
     # The F# wheel's grammar is ABI 15, which a tree-sitter runtime older than
     # 0.25 cannot load: `Language()` raises and F# indexes as empty. The pack
     # carried that floor only transitively, and its own spec does not promise it.
-    "tree-sitter>=0.25",
+    # CAPPED at the locked minor: the per-file parse budget stops a slow parse
+    # through `Parser.timeout_micros`, and tree-sitter 0.26.0 removed it. With
+    # no cap an install resolved 0.26.0 and lost the stop, while CI, locked to
+    # 0.25.2, saw nothing. Raise the cap only together with `uv.lock`, so
+    # `tests/test_parse_budget_cancels.py` runs on the new version first
+    # (`tests/test_tree_sitter_range_is_the_tested_range.py`).
+    "tree-sitter>=0.25,<0.26",
     "pathspec>=0.12.0",
     "pyyaml>=6.0",
 ]

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/server.json

@@ -3,12 +3,12 @@
   "name": "io.github.jgravelle/jcodemunch-mcp",
   "title": "jCodemunch MCP",
   "description": "Token-efficient code exploration via tree-sitter AST parsing. 70+ languages, 86-99% token savings.",
-  "version": "1.108.332",
+  "version": "1.108.333",
   "packages": [
     {
       "registryType": "pypi",
       "identifier": "jcodemunch-mcp",
-      "version": "1.108.332",
+      "version": "1.108.333",
       "transport": {
         "type": "stdio"
       }

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/src/jcodemunch_mcp/cli/init.py

@@ -1134,14 +1134,45 @@ def run_index(*, dry_run: bool = False) -> str:
 
     try:
         from ..tools.index_folder import index_folder
-        result = index_folder(path=cwd)
-        files = result.get("files_indexed", "?")
-        symbols = result.get("symbols_indexed", "?")
-        return f"  indexed {cwd} ({files} files, {symbols} symbols)"
+        return _index_summary(cwd, index_folder(path=cwd))
     except Exception as e:
         return f"  indexing failed: {e}"
 
 
+def _index_summary(cwd: str, result: dict) -> str:
+    """One line for what `index_folder` answered. It answers in four shapes:
+    a refusal (`success` false, `error`), a full index (`file_count`), a run
+    that re-indexed some files (`changed`/`new`/`deleted`, `symbol_count`) and
+    a run that found nothing changed (the three counts at 0, no symbol count).
+    A count the result does not carry is left out, never printed as `?`.
+    Any successful shape can also say the file cap cut the walk short
+    (`truncated`); the line then says how many files the index holds."""
+    if not result.get("success"):
+        return f"  indexing failed: {result.get('error') or 'the indexer gave no reason'}"
+    parts = []
+    unchanged = False
+    if "file_count" in result:
+        parts.append(f"{result['file_count']} files")
+    elif "changed" in result:
+        touched = [result.get(key, 0) for key in ("changed", "new", "deleted")]
+        unchanged = not any(touched)
+        parts.append(
+            "nothing changed since the last index" if unchanged
+            else "{} changed, {} new, {} deleted".format(*touched)
+        )
+    if result.get("symbol_count") is not None:
+        parts.append(f"{result['symbol_count']} symbols")
+    detail = ", ".join(parts)
+    if result.get("truncated"):
+        capped = "the file cap was reached, {} of {} files are in the index".format(
+            result.get("files_indexed", "some"), result.get("files_discovered", "the")
+        )
+        detail = f"{detail}; {capped}" if detail else capped
+    if unchanged:
+        return f"  {cwd} is up to date ({detail})"
+    return f"  indexed {cwd} ({detail})" if detail else f"  indexed {cwd}"
+
+
 # ---------------------------------------------------------------------------
 # Audit agent config
 # ---------------------------------------------------------------------------

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/src/jcodemunch_mcp/parser/parse_budget.py

@@ -16,9 +16,13 @@ is left of the deadline as its own timeout, and the C parser stops itself.
 in favour of `parse(progress_callback=)`. The callback is not usable: it is
 ignored for a bytestring, and with a read callback it killed the interpreter
 with an access violation on all eight variants probed (tree-sitter 0.25.2,
-Python 3.12.4, Windows). A binding without the attribute leaves only the thread
-wait; `bind()` says so once in the log and
-`tests/test_parse_budget_cancels.py` fails on it.
+Python 3.12.4, Windows). A binding without the attribute cannot stop the C
+parse: the file is named only when the parse returns and the next checkpoint
+fires. `bind()` says so once in the log and
+`tests/test_parse_budget_cancels.py` fails on it. ⚠⚠ tree-sitter 0.26.0 IS
+such a binding, and that test runs on the LOCKED version only, so the
+dependency range is capped at the locked minor (`pyproject.toml`,
+`tests/test_tree_sitter_range_is_the_tested_range.py`).
 
 ⚠ Many dedicated parsers catch `Exception` around their parse and return `[]`
 (a grammar that failed to load is "indexed for text search only"). A cancel
@@ -298,7 +302,8 @@ def bind(parser):
             _warned_no_timeout = True
             logger.warning(
                 "the installed tree-sitter has no Parser.timeout_micros; "
-                "JCODEMUNCH_PARSE_BUDGET_SECONDS cannot stop a slow parse on it"
+                "JCODEMUNCH_PARSE_BUDGET_SECONDS cannot stop a slow parse on it. "
+                'tree-sitter 0.26 removed it: upgrade jcodemunch-mcp, or install "tree-sitter>=0.25,<0.26"'
             )
         return parser
     return _BudgetedParser(parser, scope)

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/src/jcodemunch_mcp/parser/sql_preprocessor.py

@@ -71,9 +71,25 @@ _LINE_BREAK_RE = re.compile(
     + "".join(chr(c) for c in (0x0A, 0x0D, 0x0B, 0x0C, 0x1C, 0x1D, 0x1E, 0x85, 0x2028, 0x2029))
     + "]"
 )
-_NON_SPACE_RE = re.compile(r"\S")
-# what may stand between a comment delimiter and its body: `{#-`, `{#+`, `{#~`
-_WHITESPACE_CONTROL_MARKS = ("-", "+", "~")
+# A byte-order mark and a zero-width space print nothing and are not text: a
+# file that starts with a BOM still starts with its first comment.
+_INVISIBLE = chr(0xFEFF) + chr(0x200B)
+_NON_SPACE_RE = re.compile(r"[^\s" + _INVISIBLE + "]")
+# What may stand between a comment delimiter and its body, per dialect (L-131):
+# `{#-` everywhere, `{#+` in Jinja and dbt, `{#~` in Twig. The other dialect's
+# mark is the comment's own text.
+JINJA_WHITESPACE_MARKS = ("-", "+")
+TWIG_WHITESPACE_MARKS = ("-", "~")
+# `{{`, `{%` and `{#`, and what closes each
+_JINJA_OPENER_RE = re.compile(r"\{[{%#]")
+_JINJA_CLOSER = {"{": "}}", "%": "%}", "#": "#}"}
+# `{% raw %}` (Jinja) and `{% verbatim %}` (Twig): nothing up to the end tag is
+# a delimiter, so a `{{` written there opens no expression.
+_RAW_OPEN_RE = re.compile(r"\{%[-+~]?\s*(raw|verbatim)\s*[-+~]?%\}")
+_RAW_END_RE = {
+    word: re.compile(r"\{%[-+~]?\s*end" + word + r"\s*[-+~]?%\}")
+    for word in ("raw", "verbatim")
+}
 
 
 class _PrecedingComments:
@@ -95,25 +111,81 @@ class _PrecedingComments:
     of the file and checked only that the text before the directive ended in
     ``#}``, so every documented macro after the first carried the FIRST
     comment of the file. A ``#}`` that closes no comment is not one.
+
+    ⚠⚠ Three more rules, each a wrong docstring before it (L-129, L-130,
+    L-131). The comment is FIRST ON ITS LINE, with only other comments before
+    it there (or a ``--`` lead): one that trails code (``{% endmacro %} {# end of a #}``) belongs
+    to that code, not to the directive on the next line. The file is read as
+    Jinja reads it, ``{{ ... }}``, ``{% ... %}`` and ``{# ... #}`` left to
+    right, so a ``{#`` inside an expression or a tag opens no comment, and
+    nothing inside ``{% raw %}`` or ``{% verbatim %}`` opens anything. And a
+    whitespace-control mark is dropped only if it is this dialect's.
     """
 
-    def __init__(self, text: str) -> None:
+    def __init__(self, text: str, whitespace_marks: tuple[str, ...] = JINJA_WHITESPACE_MARKS) -> None:
         self._text = text
+        self._marks = whitespace_marks
         self._line_starts = [0] + [m.end() for m in _LINE_BREAK_RE.finditer(text)]
         self._line_ends = [m.start() for m in _LINE_BREAK_RE.finditer(text)] + [len(text)]
         self._first_text: dict[int, int] = {}  # line index -> its first non-space character
         # where a comment ends -> where its body starts. Read left to right,
-        # a comment running to the first `#}` after its `{#`; when a `{#` has
-        # no `#}`, no later one has either.
+        # each `{{`, `{%` or `{#` running to the first closer of its kind.
+        # When one has no closer, no later one of that kind has either, so
+        # that kind is text from there on (one failed search per kind).
         self._comment_body_start: dict[int, int] = {}
-        opened = text.find("{#")
-        while opened >= 0:
+        unclosed: set[str] = set()  # also a raw word whose end tag is nowhere ahead
+        pos = 0
+        while True:
             parse_budget.checkpoint()
-            closed = text.find("#}", opened + 2)
-            if closed < 0:
+            found = _JINJA_OPENER_RE.search(text, pos)
+            if found is None:
                 break
-            self._comment_body_start[closed + 2] = opened + 2
-            opened = text.find("{#", closed + 2)
+            opened = found.start()
+            kind = text[opened + 1]
+            if kind == "%":
+                raw_tag = _RAW_OPEN_RE.match(text, opened)
+                if raw_tag is not None and raw_tag.group(1) not in unclosed:
+                    ended = _RAW_END_RE[raw_tag.group(1)].search(text, raw_tag.end())
+                    if ended is not None:
+                        pos = ended.end()
+                        continue
+                    unclosed.add(raw_tag.group(1))
+            closed = -1 if kind in unclosed else text.find(_JINJA_CLOSER[kind], opened + 2)
+            if closed < 0:
+                unclosed.add(kind)
+                pos = opened + 1
+                continue
+            if kind == "#":
+                self._comment_body_start[closed + 2] = opened + 2
+            pos = closed + 2
+
+    def _first_text_of(self, idx: int) -> int:
+        """Where line ``idx``'s first visible character is; its end when it has none."""
+        first_text = self._first_text.get(idx)
+        if first_text is None:
+            found = _NON_SPACE_RE.search(self._text, self._line_starts[idx], self._line_ends[idx])
+            first_text = self._first_text[idx] = found.start() if found else self._line_ends[idx]
+        return first_text
+
+    def _is_first_on_its_line(self, opened: int) -> bool:
+        """Whether only space, and other comments, stand between a line start and ``opened``."""
+        text = self._text
+        pos = opened
+        while True:
+            parse_budget.checkpoint()
+            first_text = self._first_text_of(bisect.bisect_right(self._line_starts, pos) - 1)
+            # A line that starts with `--` is a comment up to here, not code:
+            # `-- note {# doc #}` leaves the Jinja comment first of its kind.
+            if first_text >= pos or text.startswith("--", first_text, pos):
+                return True
+            # text stands before `pos` on this line, so this stops at it
+            while text[pos - 1].isspace() or text[pos - 1] in _INVISIBLE:
+                parse_budget.checkpoint()
+                pos -= 1
+            body_start = self._comment_body_start.get(pos)  # a comment ends here
+            if body_start is None:
+                return False
+            pos = body_start - 2
 
     @staticmethod
     def _clean(comment_body: str) -> str:
@@ -144,14 +216,14 @@ class _PrecedingComments:
 
         # Check for {# comment #} immediately before
         body_start = self._comment_body_start.get(end)
-        if body_start is not None:
+        if body_start is not None and self._is_first_on_its_line(body_start - 2):
             body = text[body_start:end - 2]
             # `{#- ... -#}` trims the whitespace around the comment; the
-            # dashes are the delimiter's, not the comment's. So are Jinja's
-            # `+` and Twig's `~`. One mark per side, read before any space.
-            if body[:1] in _WHITESPACE_CONTROL_MARKS:
+            # mark is the delimiter's, not the comment's. One mark per side,
+            # read before any space, and only this dialect's.
+            if body[:1] in self._marks:
                 body = body[1:]
-            if body[-1:] in _WHITESPACE_CONTROL_MARKS:
+            if body[-1:] in self._marks:
                 body = body[:-1]
             return self._clean(body)
 
@@ -161,10 +233,7 @@ class _PrecedingComments:
         line_end = end
         while idx >= 0:
             parse_budget.checkpoint()
-            first_text = self._first_text.get(idx)
-            if first_text is None:
-                found = _NON_SPACE_RE.search(text, self._line_starts[idx], self._line_ends[idx])
-                first_text = self._first_text[idx] = found.start() if found else self._line_ends[idx]
+            first_text = self._first_text_of(idx)
             if text.startswith("--", first_text, line_end):
                 lines.append(text[first_text + 2:line_end].strip())
             elif first_text >= line_end:
@@ -183,6 +252,7 @@ class _PrecedingComments:
 def extract_dbt_directives(
     sql_bytes: bytes,
     directive_keywords: tuple[str, ...] = _DEFAULT_DBT_DIRECTIVES,
+    whitespace_marks: tuple[str, ...] = JINJA_WHITESPACE_MARKS,
 ) -> list[DbtDirective]:
     """Extract Jinja ``{% <kw> name(params) %}`` directive blocks as metadata.
 
@@ -202,7 +272,7 @@ def extract_dbt_directives(
     sql_str = sql_bytes.decode("utf-8", errors="replace")
     directives: list[DbtDirective] = []
     newline_offsets = [m.start() for m in re.finditer("\n", sql_str)]
-    comments = _PrecedingComments(sql_str)
+    comments = _PrecedingComments(sql_str, tuple(whitespace_marks))
     # directive keyword -> (starts, ends) of every end tag of that keyword
     end_tags: dict[str, tuple[list[int], list[int]]] = {}
 

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/src/jcodemunch_mcp/parser/template_shared.py

@@ -33,7 +33,7 @@ from dataclasses import dataclass
 from typing import Callable, Optional
 
 from . import parse_budget
-from .sql_preprocessor import extract_dbt_directives
+from .sql_preprocessor import JINJA_WHITESPACE_MARKS, TWIG_WHITESPACE_MARKS, extract_dbt_directives
 from .symbols import Symbol, compute_content_hash, make_symbol_id
 
 
@@ -53,6 +53,10 @@ _TEMPLATE_DIRECTIVES = ("macro", "block")
 # named, overridable region (constant-like named anchor).
 _DIRECTIVE_KIND = {"macro": "function", "block": "constant"}
 
+# Engine -> the whitespace-control marks of its comment delimiters (L-131);
+# an engine not named here reads as Jinja.
+_WHITESPACE_MARKS = {"twig": TWIG_WHITESPACE_MARKS}
+
 
 def extract_jinja_directives(
     text: str, filename: str, language: str = "jinja"
@@ -68,7 +72,9 @@ def extract_jinja_directives(
     """
     symbols: list[Symbol] = []
     for d in extract_dbt_directives(
-        text.encode("utf-8"), directive_keywords=_TEMPLATE_DIRECTIVES
+        text.encode("utf-8"),
+        directive_keywords=_TEMPLATE_DIRECTIVES,
+        whitespace_marks=_WHITESPACE_MARKS.get(language, JINJA_WHITESPACE_MARKS),
     ):
         parse_budget.checkpoint()
         kind = _DIRECTIVE_KIND[d.directive]

{jcodemunch_mcp-1.108.332 → jcodemunch_mcp-1.108.333}/src/jcodemunch_mcp/storage/index_store.py

@@ -586,7 +586,13 @@ INDEX_VERSION = 17
 #   it as its docstring. Every such directive after the first documented one
 #   carried the FIRST comment of the file. ⚠⚠ No symbol id moves; only
 #   `docstring` (and a summary built from it) changes, on UNCHANGED content.
-PARSER_GENERATION = 10
+#
+# 11 (L-129, L-130, L-131): three more rules for that docstring. A comment
+#   that trails code on its line is not the next directive's; a `{#` inside
+#   `{{ ... }}` or `{% ... %}` opens no comment; and a whitespace-control
+#   mark is dropped only for its own dialect (`+` Jinja and dbt, `~` Twig).
+#   ⚠⚠ No symbol id moves; only `docstring` changes, on UNCHANGED content.
+PARSER_GENERATION = 11
 
 
 @dataclass(frozen=True)