codetool-explore 0.5.0__tar.gz → 0.7.0__tar.gz

codetool_explore-0.7.0/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+.pytest_cache/
+*.py[cod]
+rust/target/
+benchmark-corpus/
+benchmarks/corpus/
+reports/
+research/
+.tmp*/
+.ruff_cache/

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codetool-explore
-Version: 0.5.0
+Version: 0.7.0
 Summary: Fast, dependency-free workspace search, read, and list exploration for coding-agent tools with Rust backend
 Project-URL: Homepage, https://github.com/pbi-agent/codetool-explore
 Project-URL: Repository, https://github.com/pbi-agent/codetool-explore
@@ -109,16 +109,21 @@ path-only rows under `target="content_or_path"` are returned without
 line/snippet fields.
 
 `target="read"` treats `pattern` as one known file path, resolves relative paths
-under a single `root`, and returns plain text with no line-number prefixes.
-Use `start_line` and `limit` to cap the returned line range; if more lines
-remain, text output starts with `-- more: cursor=N`. CSV files are read as
-ordinary text. Binary-looking, missing, unreadable, or directory paths fail with
-controlled `ExploreError` subclasses.
+under each root, and returns plain text with no line-number prefixes. When more
+than one file is read, each file is prefixed by a compact `path:` header. Use
+`start_line` and `limit` to cap the returned line range; if more lines remain,
+text output starts with `-- more: cursor=N`. CSV files are read as ordinary
+text. Binary-looking, missing, unreadable, or directory paths fail with
+controlled `ExploreError` subclasses. If `root` is already a file,
+`target="read"` reads that file directly, so accidental search-like `pattern`
+values from tool calls still produce the requested line range.
 
 `target="list"` treats `pattern` as one file/directory path and returns one
-directory level. Text output uses the same compact tree display as raw search
-output when that saves tokens. Directories end with `/`; file paths are returned
-as one entry. It honors `glob`, `exclude`, ignore files, `limit`, and `cursor`.
+directory level under each root. Text output uses the same compact tree display
+as raw search output when that saves tokens, appending each root's listing in
+root order. Directories end with `/`; file paths are returned as one entry. It
+honors multi-root common-base paths, `glob`, `exclude`, ignore files, `limit`,
+and `cursor`.
 Read/list use the pure-Python stdlib implementation even when `backend="auto"`
 or `"rust"` is requested.
 
@@ -139,9 +144,10 @@ treated as multiple roots when that exact path does not exist and every split
 token is an existing file or directory. Existing paths with spaces still take
 priority; quote individual spaced paths if combining them in one string.
 
-File roots search only that file and report paths relative to the file's parent
-directory. Multi-root searches report paths relative to the roots' common base,
-so sibling roots keep prefixes such as `src/...` and `tests/...`; this also lets
+File roots search/read only that file and report paths relative to the file's
+parent directory; listing a file path returns one file entry. Multi-root
+searches/reads/listings report paths relative to the roots' common base, so
+sibling roots keep prefixes such as `src/...` and `tests/...`; this also lets
 `exclude=["src/generated/**"]` target one root.
 
 Controlled failures raise `ExploreError` subclasses:
@@ -159,15 +165,16 @@ codetool-explore "service" . --target path --literal
 codetool-explore "User(Service|Repository)" --root src --mode snippets --raw
 codetool-explore "search_workspace" --root src --root webapp --root tests --literal
 codetool-explore --read README.md --start-line 20 --limit 40
-codetool-explore --list src --glob "*.py"
+codetool-explore --read settings.py --root src --root tests
+codetool-explore --list . --root src --root tests --glob "*.py"
 ```
 
 The CLI defaults to compact JSON for search, plain text for `--read`, and
 tree-compressed text for `--list`.
 Use `--format text` or `--raw` for raw search text; no search matches print
-`No Match`. Repeat `--root` for multiple search roots; read/list accept a single
-root only. A single quoted space-delimited `--root` is accepted as a compatibility
-fallback when it can be split into existing roots.
+`No Match`. Repeat `--root` for multiple search/read/list roots. A single
+quoted space-delimited `--root` is accepted as a compatibility fallback when it
+can be split into existing roots.
 
 ## Install
 
@@ -217,13 +224,11 @@ Token counts use `tiktoken` when available. The table compares output across 7 R
 
 | Output | Tokens | Bytes | Chart |
 | --- | ---: | ---: | --- |
-| `explore(..., result_format="text")` | 11,008 | 34.3 KB | ██░░░░░░░░░░░░░░░░ |
+| `codetool-explore` | 11,008 | 34.3 KB | ██░░░░░░░░░░░░░░░░ |
 | `rtk grep` stdout | 19,646 | 60.1 KB | ███░░░░░░░░░░░░░░░ |
-| default `explore(...)` | 38,393 | 125.3 KB | █████░░░░░░░░░░░░░ |
-| `explore(..., result_format="full")` | 39,027 | 134.7 KB | █████░░░░░░░░░░░░░ |
 | `rg` stdout | 129,775 | 402.4 KB | ██████████████████ |
 
-Default structured output is 7.03% smaller than the full structured shape. Raw text omits backend/totals metadata, includes only a cursor hint when truncated, and prints `No Match` for empty pages. Raw text is 0.56× the `rtk grep` token count in this run.
+`codetool-explore` is raw text from `explore(..., result_format="text")`; it omits backend/totals metadata, includes only a cursor hint when truncated, and prints `No Match` for empty pages. It is 0.56× the `rtk grep` token count in this run.
 
 Source: `reports/rtk_vs_codetool_output_lengths.json`.
 

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/README.md

@@ -79,16 +79,21 @@ path-only rows under `target="content_or_path"` are returned without
 line/snippet fields.
 
 `target="read"` treats `pattern` as one known file path, resolves relative paths
-under a single `root`, and returns plain text with no line-number prefixes.
-Use `start_line` and `limit` to cap the returned line range; if more lines
-remain, text output starts with `-- more: cursor=N`. CSV files are read as
-ordinary text. Binary-looking, missing, unreadable, or directory paths fail with
-controlled `ExploreError` subclasses.
+under each root, and returns plain text with no line-number prefixes. When more
+than one file is read, each file is prefixed by a compact `path:` header. Use
+`start_line` and `limit` to cap the returned line range; if more lines remain,
+text output starts with `-- more: cursor=N`. CSV files are read as ordinary
+text. Binary-looking, missing, unreadable, or directory paths fail with
+controlled `ExploreError` subclasses. If `root` is already a file,
+`target="read"` reads that file directly, so accidental search-like `pattern`
+values from tool calls still produce the requested line range.
 
 `target="list"` treats `pattern` as one file/directory path and returns one
-directory level. Text output uses the same compact tree display as raw search
-output when that saves tokens. Directories end with `/`; file paths are returned
-as one entry. It honors `glob`, `exclude`, ignore files, `limit`, and `cursor`.
+directory level under each root. Text output uses the same compact tree display
+as raw search output when that saves tokens, appending each root's listing in
+root order. Directories end with `/`; file paths are returned as one entry. It
+honors multi-root common-base paths, `glob`, `exclude`, ignore files, `limit`,
+and `cursor`.
 Read/list use the pure-Python stdlib implementation even when `backend="auto"`
 or `"rust"` is requested.
 
@@ -109,9 +114,10 @@ treated as multiple roots when that exact path does not exist and every split
 token is an existing file or directory. Existing paths with spaces still take
 priority; quote individual spaced paths if combining them in one string.
 
-File roots search only that file and report paths relative to the file's parent
-directory. Multi-root searches report paths relative to the roots' common base,
-so sibling roots keep prefixes such as `src/...` and `tests/...`; this also lets
+File roots search/read only that file and report paths relative to the file's
+parent directory; listing a file path returns one file entry. Multi-root
+searches/reads/listings report paths relative to the roots' common base, so
+sibling roots keep prefixes such as `src/...` and `tests/...`; this also lets
 `exclude=["src/generated/**"]` target one root.
 
 Controlled failures raise `ExploreError` subclasses:
@@ -129,15 +135,16 @@ codetool-explore "service" . --target path --literal
 codetool-explore "User(Service|Repository)" --root src --mode snippets --raw
 codetool-explore "search_workspace" --root src --root webapp --root tests --literal
 codetool-explore --read README.md --start-line 20 --limit 40
-codetool-explore --list src --glob "*.py"
+codetool-explore --read settings.py --root src --root tests
+codetool-explore --list . --root src --root tests --glob "*.py"
 ```
 
 The CLI defaults to compact JSON for search, plain text for `--read`, and
 tree-compressed text for `--list`.
 Use `--format text` or `--raw` for raw search text; no search matches print
-`No Match`. Repeat `--root` for multiple search roots; read/list accept a single
-root only. A single quoted space-delimited `--root` is accepted as a compatibility
-fallback when it can be split into existing roots.
+`No Match`. Repeat `--root` for multiple search/read/list roots. A single
+quoted space-delimited `--root` is accepted as a compatibility fallback when it
+can be split into existing roots.
 
 ## Install
 
@@ -187,13 +194,11 @@ Token counts use `tiktoken` when available. The table compares output across 7 R
 
 | Output | Tokens | Bytes | Chart |
 | --- | ---: | ---: | --- |
-| `explore(..., result_format="text")` | 11,008 | 34.3 KB | ██░░░░░░░░░░░░░░░░ |
+| `codetool-explore` | 11,008 | 34.3 KB | ██░░░░░░░░░░░░░░░░ |
 | `rtk grep` stdout | 19,646 | 60.1 KB | ███░░░░░░░░░░░░░░░ |
-| default `explore(...)` | 38,393 | 125.3 KB | █████░░░░░░░░░░░░░ |
-| `explore(..., result_format="full")` | 39,027 | 134.7 KB | █████░░░░░░░░░░░░░ |
 | `rg` stdout | 129,775 | 402.4 KB | ██████████████████ |
 
-Default structured output is 7.03% smaller than the full structured shape. Raw text omits backend/totals metadata, includes only a cursor hint when truncated, and prints `No Match` for empty pages. Raw text is 0.56× the `rtk grep` token count in this run.
+`codetool-explore` is raw text from `explore(..., result_format="text")`; it omits backend/totals metadata, includes only a cursor hint when truncated, and prints `No Match` for empty pages. It is 0.56× the `rtk grep` token count in this run.
 
 Source: `reports/rtk_vs_codetool_output_lengths.json`.
 

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "codetool-explore"
-version = "0.5.0"
+version = "0.7.0"
 description = "Fast, dependency-free workspace search, read, and list exploration for coding-agent tools with Rust backend"
 readme = "README.md"
 authors = [

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/scripts/update_readme_benchmarks.py

@@ -175,37 +175,30 @@ def render_tokens(data: dict[str, Any], source: str) -> list[str]:
     if "summary" not in data:
         aggregate = compression_totals(data)
         totals = aggregate["totals"]
-        full = int(totals["full_bytes"])
-        compressed = int(totals["compressed_bytes"])
         raw = int(totals["raw_bytes"])
         rtk = int(totals["rtk_bytes"])
         rg = int(totals["rg_bytes"])
-        grep = int(totals["grep_bytes"])
-        full_tokens = int(totals["full_tokens"])
-        compressed_tokens = int(totals["compressed_tokens"])
         raw_tokens = int(totals["raw_tokens"])
         rtk_tokens = int(totals["rtk_tokens"])
         rg_tokens = int(totals["rg_tokens"])
-        grep_tokens = int(totals["grep_tokens"])
         savings = aggregate["savings"]
-        if not full or not compressed:
+        if not raw:
             raise SystemExit(f"{source} does not contain token-compression data")
-        rows = [
-            ('`explore(..., result_format="full")`', full_tokens, full),
-            ("default `explore(...)`", compressed_tokens, compressed),
-        ]
+        rows = []
         if raw:
-            rows.append(('`explore(..., result_format="text")`', raw_tokens, raw))
+            rows.append(("`codetool-explore`", raw_tokens, raw))
         if rtk:
             rows.append(("`rtk grep` stdout", rtk_tokens, rtk))
         if rg:
             rows.append(("`rg` stdout", rg_tokens, rg))
-        if grep:
-            rows.append(("GNU `grep` stdout", grep_tokens, grep))
         max_tokens = max(tokens for _, tokens, _ in rows)
-        percent = round(((full - compressed) / full) * 100, 2)
-        average_percent = round(mean(savings), 2) if savings else percent
         samples = aggregate["samples"] or len(savings)
+        raw_ratio = round(raw_tokens / rtk_tokens, 2) if rtk_tokens else None
+        raw_note = (
+            f" It is {raw_ratio}× the `rtk grep` token count in this run."
+            if raw_ratio is not None
+            else ""
+        )
         lines = [
             "### Token compression",
             "",
@@ -224,9 +217,9 @@ def render_tokens(data: dict[str, Any], source: str) -> list[str]:
         lines += [
             "",
             (
-                f"Default structured output is {percent}% smaller in bytes total and "
-                f"{average_percent}% smaller on average per sample than the full shape. "
-                "Raw text omits metadata and is optimized for minimum tokens."
+                "`codetool-explore` is raw text from "
+                '`explore(..., result_format="text")`; it omits metadata and '
+                "is optimized for minimum tokens." + raw_note
             ),
             "",
             f"Source: `{source}`.",
@@ -235,33 +228,20 @@ def render_tokens(data: dict[str, Any], source: str) -> list[str]:
 
     summary = data["summary"]
     totals = summary["total_bytes"]
-    compressed = int(totals["codetool_compressed_json"])
-    full = int(totals["codetool_full_json"])
     raw = int(totals.get("codetool_raw_text", 0))
     rtk = int(totals.get("rtk_grep_stdout", 0))
     rg = int(totals.get("rg_stdout", 0))
-    grep = int(totals.get("grep_stdout", 0))
     token_totals = summary.get("total_tokens", {})
-    full_tokens = int(token_totals.get("codetool_full_json", round(full / 4)))
-    compressed_tokens = int(
-        token_totals.get("codetool_compressed_json", round(compressed / 4))
-    )
     raw_tokens = int(token_totals.get("codetool_raw_text", round(raw / 4)))
     rtk_tokens = int(token_totals.get("rtk_grep_stdout", round(rtk / 4)))
     rg_tokens = int(token_totals.get("rg_stdout", round(rg / 4)))
-    grep_tokens = int(token_totals.get("grep_stdout", round(grep / 4)))
-    rows = [
-        ('`explore(..., result_format="full")`', full_tokens, full),
-        ("default `explore(...)`", compressed_tokens, compressed),
-    ]
+    rows = []
     if raw:
-        rows.append(('`explore(..., result_format="text")`', raw_tokens, raw))
+        rows.append(("`codetool-explore`", raw_tokens, raw))
     if rtk:
         rows.append(("`rtk grep` stdout", rtk_tokens, rtk))
     if rg:
         rows.append(("`rg` stdout", rg_tokens, rg))
-    if grep:
-        rows.append(("GNU `grep` stdout", grep_tokens, grep))
     max_tokens = max(tokens for _, tokens, _ in rows)
     comparison = summary["total_comparison"]
     lines = [
@@ -281,15 +261,15 @@ def render_tokens(data: dict[str, Any], source: str) -> list[str]:
         )
     raw_ratio = comparison.get("codetool_raw_over_rtk_tokens_ratio")
     raw_note = (
-        f" Raw text is {raw_ratio}× the `rtk grep` token count in this run."
+        f" It is {raw_ratio}× the `rtk grep` token count in this run."
         if raw_ratio is not None
         else ""
     )
     lines += [
         "",
         (
-            f"Default structured output is {comparison['codetool_compressed_vs_full_percent_smaller']}% "
-            "smaller than the full structured shape. Raw text omits backend/totals "
+            "`codetool-explore` is raw text from "
+            '`explore(..., result_format="text")`; it omits backend/totals '
             "metadata, includes only a cursor hint when truncated, and prints "
             "`No Match` for empty pages." + raw_note
         ),

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/src/codetool_explore/api.py

@@ -107,12 +107,13 @@ def explore(
     paths without opening files, and ``"content_or_path"`` returns files
     matching either.
     ``"read"`` treats ``pattern`` as one file path and returns a controlled
-    line range. ``"list"`` treats ``pattern`` as one file/directory path and
-    returns a one-level listing.
+    line range for each root. ``"list"`` treats ``pattern`` as one
+    file/directory path and returns a one-level listing for each root.
     Patterns are interpreted as regex by default; pass ``regex=False`` for exact
     literal search. ``root`` may be one directory/file path or a non-empty list
-    of directory/file paths. Multi-root searches report paths relative to the
-    roots' common base, so sibling roots keep prefixes such as ``src/...``.
+    of directory/file paths. Multi-root searches/reads/listings report paths
+    relative to the roots' common base, so sibling roots keep prefixes such as
+    ``src/...``.
     To tolerate common JSON/tool-call mistakes, a whitespace-separated root
     string is treated as multiple roots only when that exact path does not
     exist and every split token is an existing file or directory.
@@ -122,8 +123,8 @@ def explore(
     compatibility fallback.
 
     Search results are returned in a compact structured shape by default; read
-    results default to plain text and list results default to tree-compressed
-    text. Pass
+    results default to plain text (with path headers only when multiple files
+    are read) and list results default to tree-compressed text. Pass
     ``result_format="full"`` to receive the pre-compression backend result
     dictionary unchanged. Pass ``result_format="text"`` (or ``"raw"``) for an
     RTK-inspired plain-text rendering optimized for token compression.

{codetool_explore-0.5.0 → codetool_explore-0.7.0}/src/codetool_explore/cli.py

@@ -26,12 +26,12 @@ def build_parser() -> argparse.ArgumentParser:
     action_group.add_argument(
         "--read",
         dest="read_path",
-        help="read one known file path with controlled line output",
+        help="read one known file path under each root with controlled line output",
     )
     action_group.add_argument(
         "--list",
         dest="list_path",
-        help="list one directory level or one file path",
+        help="list one directory level or one file path under each root",
     )
     parser.add_argument(
         "--root",