@khanhicetea/pi-better-tool 0.2.1 → 0.2.3

package/README.md

@@ -1,20 +1,10 @@
 # pi-better-tool
 
-Better built-in tools for the [pi coding agent](https://github.com/earendil-works/pi-mono) — starting with an `edit` override that turns failed edits into recoverable ones.
+Context-aware tools for the [Pi coding agent](https://github.com/earendil-works/pi-mono):
 
-## Why
-
-The built-in `edit` tool requires `edits[].oldText` to match **exactly and uniquely**. When it doesn't, the tool fails with a bare error:
-
-```
-Could not find edits[1] in /code/app.go. The oldText must match exactly including all whitespace and newlines.
-```
-
-That failure wastes a whole loop: the model has to `read` the file again, guess a larger context, and retry — sometimes failing again. `pi-better-tool` resolves only ambiguity supported by verified evidence and otherwise returns **recovery context** so the next call succeeds without re-reading:
-
-- **Ambiguous literal match after a bounded read** — when exactly one occurrence was fully visible in the latest successful, byte-verified `read` of the same file, edit selects that occurrence. Its success message reports the selected/read ranges and up to four remaining occurrences with effective prefix/suffix context and retryable snippets.
-- **Other ambiguous matches (2+ occurrences)** — bounded occurrence line numbers plus the **minimum prefix/suffix context** that makes the first few occurrences unique, rendered as ready-to-use `oldText` snippets when they fit safely.
-- **Text not found** — the closest matching region (fuzzy line similarity), a line-by-line comparison against your `oldText`, the exact file bytes to retry with, and likely causes (tabs vs spaces, indentation, case).
+- **`edit`** replaces the built-in edit tool. Failures explain what failed, what was not written, and what to do next.
+- **`read_symbol`** is a new tool. Read a whole function, method, class, or type by containing line or exact name. With only a path, get a symbol outline.
+- **`read` stays unchanged** for ordinary text, images, and explicit line ranges.
 
 ## Install
 
@@ -22,100 +12,157 @@ That failure wastes a whole loop: the model has to `read` the file again, guess
 pi install npm:@khanhicetea/pi-better-tool
 ```
 
-For local development from this monorepo:
+From this monorepo:
 
 ```bash
 pi install /absolute/path/to/pi-kit/packages/pi-better-tool
 ```
 
-It is also registered in the root `package.json` under `pi.extensions`.
+The root `package.json` also registers the extension. After local changes, use `/reload` in Pi to load the new tool definition. After upgrading to a release that adds grammar packages, restart Pi once: ast-grep’s native language registry cannot be extended by `/reload` in an already-running process.
 
-## Example: ambiguous oldText
+## Read the symbol, not guessed line windows
 
-```text
-Found 2 occurrences of the text in dup.go. The text must be unique. Please provide more context to make it unique.
+After grep identifies a location:
 
-Occurrences:
-  1. lines 2-3
-  2. lines 6-7
+```json
+{"path":"src/server.ts","line":142}
+```
 
-Retry with a disambiguated oldText: pick ONE occurrence below and reuse its snippet exactly. Each snippet already includes the minimum surrounding context that makes it unique:
+Call `read_symbol` with an exact name instead:
 
-Occurrence 1 (lines 2-3) — minimum context: 0 lines before, 1 line after:
+```json
+{"path":"src/server.ts","symbol":"Server.handleRequest"}
 ```
-	log()
-}
 
-func second() {
-```
+Get the enclosing class or function:
 
-Occurrence 2 (lines 6-7) — minimum context: 0 lines before, 0 lines after:
-```
-	log()
-}
+```json
+{"path":"src/server.ts","line":142,"parent":1}
 ```
 
-Tip: use the snippet byte-for-byte as the new oldText, and make newText the snippet with your change applied (the snippet may span whole lines).
+Discover names and ranges without reading every body:
 
-No changes were written — the file was not modified.
+```json
+{"path":"src/server.ts"}
 ```
 
-## Example: text not found
+### Arguments
 
-```text
-Could not find the exact text in tabs.go. The old text must match exactly including all whitespace and newlines.
+| Argument | Meaning |
+| --- | --- |
+| `path` | Local relative/absolute path; supports `@path`, `~/path`, and file URLs. |
+| `line` | 1-based file line. Select the innermost declaration containing it. |
+| `column` | Optional 1-based UTF-16 column with `line`, to distinguish same-line symbols. |
+| `symbol` | Exact, case-sensitive name or qualified name such as `Server.run`. Combine with `line` for duplicate names. |
+| `parent` | Move outward through enclosing declarations; default 0, maximum 20. |
+| `context` | Extra whole lines before/after the declaration; default 0, maximum 20. |
+| `offset` | 1-based position **within the selection**, not a file line. For an outline, the entry position. |
+| `limit` | Maximum source lines (default 1000) or outline entries (default 50); maximum 1800. |
 
-Closest match in the file: lines 3-5 (~91% line similarity).
-Differences vs your oldText (2 of 3 compared lines match):
-  file line 4 differs from your oldText line 2:
-    file:     →tab→fmt.Println("hi")
-    oldText:      fmt.Println("hi")
+Omit both `line` and `symbol` for an outline. Named selection never silently chooses the first duplicate. Line selection never silently chooses between same-line siblings. Candidate lists include concrete calls with names and positions.
 
-Exact file content at lines 3-5 — retry using this text as oldText (then apply your change to newText):
-```
-func main() {
-	fmt.Println("hi")
+### Languages and boundaries
+
+Syntax parsing uses Tree-sitter through `@ast-grep/napi`, not indentation or brace-counting guesses:
+
+- JavaScript, JSX, TypeScript, TSX, and their module extensions
+- Bash-compatible shell scripts (`.sh`, `.bash`, `.zsh`, and related extensions)
+- C and C++ (including common header and CUDA/Arduino extensions)
+- C# and Java
+- Kotlin
+- PHP
+- Python and `.pyi`
+- Ruby (including `Gemfile` and `Rakefile`)
+- Rust
+- Swift
+
+The reader handles named declarations, nested functions, JS/TS arrow functions and methods, Python decorators, Rust attributes, and containing classes/types. It includes export/declaration wrappers when applicable. It returns **whole source lines**, so a line can also contain adjacent code. It does not resolve imports, references, overload implementations, macros, or runtime bindings. Leading standalone comments are not automatically attached to a declaration.
+
+Incomplete syntax, unsupported languages, unavailable parsers, and locations without declarations do not produce guessed symbol boundaries. Failures give a bounded source preview or candidate list and concrete next-call arguments. Missing paths include a bounded list of nearby files when the parent directory is accessible.
+
+### Output and pagination
+
+Results identify the selected symbol, enclosing names, full declaration range, displayed file range, and a SHA-256 source snapshot. Source appears in an unnumbered fenced block, so line-number prefixes cannot accidentally enter `oldText`.
+
+Large selections return a whole-line page, explicitly marked **partial**, with the complete next `read_symbol` arguments. Follow that continuation instead of calculating file offsets. Each call reads a fresh snapshot; do not combine pages whose snapshot hashes differ.
+
+Limits:
+
+- Source analysis: 2 MiB UTF-8, 100,000 syntax nodes, 10,000 declarations, 1,024 characters per qualified symbol name.
+- Complete read output: 48 KiB / 1,950 lines, including metadata and fences.
+- No clipped copyable source lines or broken fences. A line too large to display gets preview/read guidance instead.
+- Invalid UTF-8, NUL-containing input, and non-regular files are rejected.
+
+Parser packages are runtime dependencies. Common platforms use prebuilt native binaries. If a grammar is unavailable on a platform, the tool gives read guidance; it never runs repository code, installs a compiler, or builds a grammar during a tool call. The edit tool can still work without loading symbol parsers.
+
+## Recover from an edit failure in the next call
+
+The `edit` input remains:
+
+```json
+{
+  "path": "src/server.ts",
+  "edits": [
+    { "oldText": "exact current text", "newText": "replacement text" }
+  ]
 }
 ```
 
-Possible cause:
-- whitespace mismatch: the text matches when ALL whitespace is removed — check tabs vs spaces and indentation width
+All entries match the **original file**, not the output of earlier entries. Matching/overlap failures apply none of the batch. Fix the reported entries and resubmit the **complete batch**.
+
+| Failure | Returned context |
+| --- | --- |
+| Repeated `oldText` | Occurrence ranges, bounded unique anchor expansions, and concrete context-read calls. |
+| Text not found | Closest-region comparison with original whitespace, likely causes, and exact retry text only when unique and clearly better than competing candidates. |
+| Overlapping entries | Both ranges plus a unique merged source anchor when it fits. Apply both intended changes to one `newText`. |
+| Empty anchor | Explain insertion anchoring and give a context-read call. |
+| No change | Explain that the output is identical; do not repeat the same call. |
+| Replacement already appears | Report its locations as a clue, not proof that the intended change is complete. |
 
-No changes were written — the file was not modified.
+A failed batch starts with, for example:
+
+```text
+[edit failure: not-found]
+Batch status: 0/2 replacements written. Fix edits[1] and resubmit the complete batch against the original file; no earlier replacement was applied.
 ```
 
-## Behavior
+Only snippets explicitly presented as retryable may be copied directly into `oldText`. Low-confidence, competing, stale, oversized, or omitted candidates require verification. Suggested `read_symbol` calls get the enclosing source declaration without another search for its boundary. Similarity scores are heuristics, not probabilities.
 
-Normal unique-match semantics are **identical** to the built-in `edit` tool. The one intentional extension is conservative read-based selection for repeated literal text:
+### Verified read evidence
 
-- same schema (`path` + `edits[{oldText,newText}]`), including the compatibility shim for models that send `edits` as a JSON string, a single edit object, or legacy top-level `oldText`/`newText`
-- same matching engine ported from pi's `edit-diff.ts`: exact match first, fuzzy fallback (trailing whitespace, smart quotes, dashes, unicode spaces), uniqueness checked in fuzzy-normalized space, all edits matched against the original content, overlap/empty/no-change detection
-- same BOM and CRLF handling
-- same success result shape (`details.diff` / `details.patch` / `details.firstChangedLine`), and no custom renderers — the built-in diff renderer is inherited
-- read-based selection fails closed unless the newest same-file read is still in active context, its stored output exactly matches the current bytes and built-in read formatting, and exactly one complete literal occurrence lies inside the visible range; fuzzy/Unicode-equivalent ambiguity, same-line ambiguity, stale reads, and broad reads containing multiple occurrences still refuse
+For repeated **literal** text, edit can select an occurrence only when exactly one tracked occurrence is fully contained in the newest verified same-file read result in Pi's active, compaction-aware **stored session context**.
 
-Failure behavior is the other difference: errors carry the recovery context described above, and nothing is written on failure (edits remain atomic).
+Both readers participate:
 
-Diagnostics degrade gracefully: files over ~2 MB skip the analysis and return the plain built-in-style error; repeated blocks that cannot be disambiguated within 12 context lines get a guidance note instead of snippets. Complete diagnostic output is bounded below pi's 50 KB / 2,000-line tool-output limit. Oversized exact snippets are omitted with a line range instead of being presented as copyable text, and low-confidence or non-unique closest matches require verification before retrying.
+- Built-in `read`: reproduce its current LF-normalized output and truncation format.
+- `read_symbol`: regenerate the result from the original arguments and current source. Snapshot, selection, envelope, and displayed source must all agree. Result `details` alone are never trusted.
 
-## Development
+Only displayed source is evidence. Outline entries, unseen parts of partial symbols, and omitted final newline separators are not. The newest same-file failed, missing, malformed, or stale result blocks fallback to an older read. Retained-tail compaction messages are supported when the host exposes them. Canonical paths support symlink aliases.
 
-```bash
-npm run check     # typecheck + tests
-npm test          # vitest only
-```
+This verifies stored context, not the final provider payload: other extensions can remove messages or rewrite requests. Built-in BOM-bearing read output remains conservatively rejected; `read_symbol` deliberately strips a UTF-8 BOM and normalizes CRLF before parsing and snapshotting. Fuzzy-equivalent ambiguity and highly repetitive files fail closed.
+
+## Edit safety and compatibility
+
+The matching engine preserves exact-first/fuzzy-fallback behavior, fuzzy-space uniqueness, original-file batch matching, overlap/no-change checks, CRLF restoration, and UTF-8 BOM preservation. Success details keep Pi's `diff`, `patch`, and `firstChangedLine` shape; the built-in edit renderer is inherited.
 
-## Publishing
+Intentional safeguards include 1–100 replacements per call, rejection of empty/fuzzy-empty anchors, and rejection of invalid UTF-8 or NUL-containing files. Self-overlapping strings retain Pi's non-overlapping occurrence-counting policy. The pure compatibility shim accepts array, JSON-string, single-object, and legacy top-level edit arguments.
+
+Both tools use the **local filesystem**. They do not inherit an SSH, container, or sandbox backend. File mutations use Pi's shared mutation queue. Immediately before writing, edit rechecks file identity and content to catch many external changes.
+
+The final write remains an **in-place overwrite**, not a cross-process lock, race-free compare-and-swap, or crash-safe atomic transaction. Another process can change a file after the check. A rejected write may leave the file unchanged, partially written, or fully written: inspect it before retrying. A resolved write is the commit boundary; later cancellation cannot roll it back. In-place writes preserve existing symlink/hard-link semantics.
+
+Edit diagnostics stay below Pi's 50 KiB / 2,000-line limits. Snippets are omitted atomically when needed, and similarity work is bounded. Renderer `details.diff` and `details.patch` remain complete rather than being blindly truncated.
+
+## Development
 
-From the repository root, verify the package and inspect its tarball before publishing:
+Tested against `@earendil-works/pi-coding-agent` 0.82.1. Pi/typebox peer dependencies remain wildcard ranges, as Pi's packaging guidance requires. Re-run the read-format, session-evidence, and renderer compatibility tests on host upgrades.
 
 ```bash
-npm run check --workspace=@khanhicetea/pi-better-tool
-npm pack --dry-run --workspace=@khanhicetea/pi-better-tool
-npm publish --workspace=@khanhicetea/pi-better-tool
+npm run check --workspace @khanhicetea/pi-better-tool
+npm pack --dry-run --workspace @khanhicetea/pi-better-tool
 ```
 
-The package is configured for public publishing under the `@khanhicetea` scope. npm authentication is required.
+Tests cover existing edit behavior, recovery round-trips, syntax boundaries across languages, nested/duplicate symbols, pagination, encoding/size failures, cancellation, and read-to-edit evidence.
 
 ## License
 

package/package.json

@@ -1,7 +1,7 @@
 {
 	"name": "@khanhicetea/pi-better-tool",
-	"version": "0.2.1",
-	"description": "Better built-in tools for pi: an edit tool override that returns recovery context (closest match + disambiguation snippets) instead of bare failures",
+	"version": "0.2.3",
+	"description": "Context-aware tools for pi: safe edit recovery and syntax-aware read_symbol for whole functions, methods, and classes by line or name",
 	"type": "module",
 	"license": "MIT",
 	"repository": {
@@ -13,8 +13,20 @@
 	"bugs": {
 		"url": "https://github.com/khanhicetea/pi-kit/issues"
 	},
-	"keywords": ["pi-package", "pi-extension", "edit", "tools"],
-	"files": ["src", "README.md", "LICENSE"],
+	"keywords": [
+		"pi-package",
+		"pi-extension",
+		"edit",
+		"read",
+		"symbols",
+		"tree-sitter",
+		"tools"
+	],
+	"files": [
+		"src",
+		"README.md",
+		"LICENSE"
+	],
 	"scripts": {
 		"test": "vitest run",
 		"typecheck": "tsc --noEmit",
@@ -33,12 +45,29 @@
 		"vitest": "^3.2.0"
 	},
 	"pi": {
-		"extensions": ["./src/index.ts"]
+		"extensions": [
+			"./src/index.ts"
+		]
 	},
 	"engines": {
 		"node": ">=20"
 	},
 	"publishConfig": {
 		"access": "public"
+	},
+	"dependencies": {
+		"@ast-grep/lang-bash": "0.0.8",
+		"@ast-grep/lang-c": "0.0.6",
+		"@ast-grep/lang-cpp": "0.0.6",
+		"@ast-grep/lang-csharp": "0.0.6",
+		"@ast-grep/lang-go": "0.0.6",
+		"@ast-grep/lang-java": "0.0.7",
+		"@ast-grep/lang-kotlin": "0.0.7",
+		"@ast-grep/lang-php": "0.0.7",
+		"@ast-grep/lang-python": "0.0.6",
+		"@ast-grep/lang-ruby": "0.0.7",
+		"@ast-grep/lang-rust": "0.0.7",
+		"@ast-grep/lang-swift": "0.0.8",
+		"@ast-grep/napi": "0.45.3"
 	}
 }

package/src/apply.ts

@@ -1,12 +1,12 @@
 /**
  * Edit matching and application engine.
  *
- * A port of pi's built-in `applyEditsToNormalizedContent` with one key
- * difference: instead of throwing opaque errors, `analyzeEdits` returns a
- * structured failure describing *why* an edit failed, which the diagnostics
- * layer turns into actionable recovery context for the model.
+ * A port of pi's built-in `applyEditsToNormalizedContent`. Instead of throwing
+ * opaque errors, `analyzeEdits` returns a structured failure describing *why*
+ * an edit failed. It also rejects fuzzy-normalized-empty needles and can accept
+ * a conservatively verified literal selection as intentional safety extensions.
  *
- * Matching semantics are kept identical to the built-in tool:
+ * Otherwise matching semantics follow the built-in tool:
  * - exact match first, then fuzzy-normalized fallback (trailing whitespace,
  *   smart quotes, dashes, unicode spaces)
  * - uniqueness is always checked in fully fuzzy-normalized space
@@ -18,10 +18,11 @@ import {
 	countFuzzyOccurrences,
 	findAllOccurrences,
 	getLineSpans,
+	getLogicalLineSpans,
 	lineAt,
 	normalizeForFuzzyMatch,
 	normalizeToLF,
-	splitLinesWithEndings,
+	splitLogicalLinesWithEndings,
 	type LineSpan,
 } from "./text.ts";
 
@@ -42,7 +43,9 @@ export type EditFailure =
 	| {
 			kind: "ambiguous";
 			editIndex: number;
-			/** Offsets of every occurrence, enumerated in fully fuzzy-normalized space. */
+			/** Total non-overlapping occurrences in fully fuzzy-normalized space. */
+			occurrenceCount: number;
+			/** A bounded prefix of occurrence offsets for diagnostics/selection. */
 			occurrenceOffsets: number[];
 	  }
 	| {
@@ -85,6 +88,9 @@ export interface AnalyzeOptions {
 	ambiguousSelections?: ReadonlyMap<number, number>;
 }
 
+/** Avoid materializing unbounded offset arrays for highly repetitive files. */
+const MAX_TRACKED_OCCURRENCE_OFFSETS = 256;
+
 export function normalizeEdits(edits: EditOp[]): EditOp[] {
 	return edits.map((edit) => ({
 		oldText: normalizeToLF(edit.oldText),
@@ -159,7 +165,11 @@ export function analyzeEdits(normalizedContent: string, rawEdits: EditOp[], opti
 		const occurrences = countFuzzyOccurrences(fuzzyBase, edit.oldText);
 		let selectedMatch = matchResult;
 		if (occurrences > 1) {
-			const occurrenceOffsets = findAllOccurrences(fuzzyBase, normalizeForFuzzyMatch(edit.oldText));
+			const occurrenceOffsets = findAllOccurrences(
+				fuzzyBase,
+				normalizeForFuzzyMatch(edit.oldText),
+				MAX_TRACKED_OCCURRENCE_OFFSETS,
+			);
 			const selectedOffset = options.ambiguousSelections?.get(i);
 			if (
 				selectedOffset === undefined ||
@@ -167,7 +177,7 @@ export function analyzeEdits(normalizedContent: string, rawEdits: EditOp[], opti
 			) {
 				return {
 					ok: false,
-					failure: { kind: "ambiguous", editIndex: i, occurrenceOffsets },
+					failure: { kind: "ambiguous", editIndex: i, occurrenceCount: occurrences, occurrenceOffsets },
 				};
 			}
 			selectedMatch = {
@@ -257,10 +267,10 @@ function applyReplacementsPreservingUnchangedLines(
 	baseContent: string,
 	replacements: Replacement[],
 ): string {
-	const originalLines = splitLinesWithEndings(originalContent);
-	const baseLines = getLineSpans(baseContent);
+	const originalLines = splitLogicalLinesWithEndings(originalContent);
+	const baseLines = getLogicalLineSpans(baseContent);
 	if (originalLines.length !== baseLines.length) {
-		throw new Error("Cannot preserve unchanged lines because the base content has a different line count.");
+		throw new Error("Cannot preserve unchanged lines because the base content has a different logical line count.");
 	}
 
 	const groups: Array<InternalLineWindow & { replacements: Replacement[] }> = [];