diffctx 1.12.1__tar.gz → 1.12.3__tar.gz

{diffctx-1.12.1 → diffctx-1.12.3}/CHANGELOG.md

@@ -9,6 +9,157 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **Scoop install on Windows.** This repository is now itself the bucket:
+  `scoop bucket add diffctx https://github.com/nikolay-e/diffctx` then
+  `scoop install diffctx/diffctx`. The manifest moved from `packaging/scoop/`
+  to `bucket/diffctx.json`, where `scoop bucket add` reads it, so CD pushing
+  the regenerated manifest to `main` is the publication step. Previously the
+  manifest was correct but unreachable — no bucket existed, so
+  `scoop search diffctx` found nothing.
+
+- `--with-raw-diff` bundles git's raw unified diff ahead of the selected
+  context in every Python-CLI format (md/yaml/json/txt) and in the Python API
+  (`build_diff_context(..., with_raw_diff=True)`). Additive only — selection is
+  byte-identical with and without it — and not charged to `--budget`; the
+  stderr token summary reports the real output size and breaks out the patch's
+  share. Lock-file (#112), ignored and secret-like sections stay omitted. Not
+  in the native binary or the MCP server yet (#150).
+- Token counting is documented explicitly: every count and `--budget` use
+  tiktoken `o200k_base` (GPT-4o family) and are approximate for other model
+  families — `docs/product/token-budget.md`, plus a `Token counting` section in
+  `--help` (#150).
+- A GitHub Action (`action.yml`) runs diffctx as a CI step and exposes the
+  context file, its exact token count and an `empty` flag as step outputs, so a
+  downstream job can feed an LLM without re-tokenizing
+  (`docs/product/github-action.md`, #145).
+- The Claude Code plugin declares its MCP server self-bootstrapping:
+  `uvx --from 'diffctx[mcp]' diffctx-mcp`. Installing the plugin no longer
+  assumes `pip install 'diffctx[mcp]'` has already happened — `uv` fetches it
+  on first use. (The entry point rather than the `diffctx mcp` subcommand: the
+  subcommand only exists from this release on, so on any older version the
+  bare command would try to map a directory named `mcp`.)
+- `diffctx mcp` starts the MCP server, alongside the existing `diffctx-mcp`
+  console script. MCP registries publish a *package* name, and clients that
+  derive the executable from it run `diffctx` — which started a tree-mapping
+  run and wrote 31 MB of the working directory into the protocol transport
+  instead of speaking MCP. Both spellings now reach the same server.
+
+### Security
+
+- **A diff range could re-enable repository-configured diff commands.** The
+  range validator allowed a leading `-`, so `--ext-diff` or `--textconv` passed
+  as a "range" landed in argv *after* the `--no-ext-diff --no-textconv` that
+  every diff invocation sets — undoing them and letting `.gitattributes` run
+  external commands. Neither side of a range may now begin with a dash, and
+  single revisions are additionally rejected for whitespace or control
+  characters (a newline split one `cat-file --batch` request into two). Refs
+  starting with a dash are unaddressable on a git command line, so nothing
+  legitimate is lost.
+- MCP tool calls had no wall-clock deadline — the CLI's watchdog covered only
+  the CLI, leaving parse, fragmentation and scoring unbounded on the MCP path.
+  All three tools now share the CLI's 300 s default.
+- `SECURITY.md` documents prompt injection via repository content honestly: it
+  is inherent to moving repository text into a model's context, diffctx does
+  not detect or neutralize it, and output must be treated as untrusted.
+
+### Fixed
+
+- **A changed file could vanish from the diff entirely when `.gitignore`
+  excluded its parent directory.** `check-ignore --no-index` — required for
+  `.diffctx/ignore` to apply to tracked files — also revives git's rule that a
+  file cannot be re-included once a parent directory is excluded. pandoc
+  excludes every dotted root entry with `/*.*` and re-includes `!.github/**`;
+  git keeps the tracked workflow file, diffctx dropped it, and a one-file
+  commit rendered as an empty selection with exit 4. An exclusion inherited
+  from an excluded ancestor no longer counts; a pattern matching the path
+  itself still excludes it, so `.diffctx/ignore` and per-directory
+  `.gitignore` rules are unaffected (#153).
+
+- **The container images ran as root.** The published 1.12.1 image had no
+  `USER` directive, so every `docker run` executed as uid 0 while the README
+  promised an unprivileged user. The image now runs as uid 10001.
+- **The MCP server advertised the SDK's version as its own.** `FastMCP` takes
+  no version argument, so `initialize` reported the installed `mcp` package
+  version (e.g. `1.28.1`) as the diffctx server version — a number that drifts
+  with every SDK bump and never matched the shipped package. Clients now see
+  the real version.
+- **The native binary hard-capped every run at 4096 tokens.** `--budget`
+  carried a fixed clap default instead of leaving the budget unset, so the
+  auto-sizing the Python CLI has always used never ran on the binary shipped
+  via crates.io, npm, the container images and the release archives — the
+  same command returned a truncated selection there (14-file
+  self-eat range: 15 fragments across 10 files, against 172 across 44). The
+  binary now defaults to auto sizing, and `--budget -1` means unlimited as it
+  does in Python.
+- The token cache grew without bound (a machine that had analyzed a few dozen
+  repositories reached 817k files / 4.0 GB). It is now capped at 512 MB,
+  overridable with `DIFFCTX_TOKEN_CACHE_MAX_BYTES` (`0` disables eviction);
+  each run trims one of the 256 shards back under its share of the cap,
+  oldest entries first (#122).
+- The native binary exited `0` on a diff that produced no semantic context
+  (clean tree, binary-only, everything over the size cap), so callers on the
+  binary channels could not tell an empty selection from a successful one. It
+  now mirrors the Python CLI: the `no semantic context` warning plus its hint
+  on stderr and exit code `4`.
+- The native binary printed no token summary, although the README documents
+  one for every run. It now writes `N tokens (o200k_base), SIZE` to stderr,
+  silenced by the new `-q/--quiet`.
+- `diffctx -v` was a usage error on the native binary while it printed the
+  version on the Python CLI. Both short forms (`-v`, `-V`) now work.
+- **A diff could lose its change signal entirely.** When the only fragment
+  covering a hunk was a whole-file chunk — flat data files, languages without a
+  tree-sitter parser, parse degradation — that chunk had no cheap variant to
+  fall back on, so an oversized core was skipped and the output carried *zero*
+  `role: "changed"` fragments while rendering the unchanged rest of the file as
+  context. Such cores now fall back to an excerpt: the changed lines plus three
+  lines of context, cut from the chunk itself and rendered as `kind: excerpt`,
+  `role: "changed"`. On the elasticsearch `muted-tests.yml` repro the output
+  goes from 121 fragments and no change signal to 118 fragments carrying the
+  appended entry, 5,337 tokens instead of 5,970 (#103). The whole-file
+  fragmentation of flat lists that the same repro shows is the separate
+  granularity issue (#105) and is unchanged.
+- A diff whose selection came back empty rendered as a bare `name`/`type` stub
+  in every format: the writer gated *all* diff metadata on there being
+  fragments, so the commit message and the list of changed files — the only
+  actionable facts about such a run — were dropped. They are now always
+  written.
+
+### Removed
+
+- **AUR support.** `diffctx-bin` had never been submitted (the AUR RPC reported
+  `resultcount: 0`), the publishing job gated on a credential this repository
+  does not hold, and every release regenerated a PKGBUILD nobody could install.
+  Arch users are served by `pipx install diffctx` and `cargo install diffctx`.
+
+### Changed
+
+- **Lock files no longer render their hunks in diff mode.** A dependency bump
+  is signal, the checksum churn carrying it is not: `diffctx . --diff` used to
+  spend 12 KB of a 48 KB output on a `Cargo.lock` chunk. The touched lock files
+  are now listed under `lockfile_changes:` (paths only, like `deleted_files:`),
+  mirroring the tree-mode ignore policy while keeping the fact that they
+  changed — on the reported range the output drops from 12,947 to 7,888 tokens
+  (#112). Recognized: `Cargo.lock`, `package-lock.json`, `npm-shrinkwrap.json`,
+  `yarn.lock`, `pnpm-lock.yaml`, `bun.lock(b)`, `deno.lock`, `Pipfile.lock`,
+  `poetry.lock`, `uv.lock`, `pdm.lock`, `composer.lock`, `Gemfile.lock`,
+  `flake.lock`, `go.sum`, `mix.lock`, `packages.lock.json`, `gradle.lockfile`,
+  `Package.resolved`, `cabal.project.freeze`. `--full` still renders their
+  content — it is the escape hatch that promises every fragment of the changed
+  files.
+- Native binary CLI parity: `-f` is accepted as the short form of `--format`,
+  bare `--diff` means the working tree against `HEAD`, `--scoring` advertises
+  its accepted values, and every option carries help text.
+- The standalone binary is covered by its own integration suite
+  (`crates/diffctx-native/tests/native_cli.rs`, run in CI): exit codes, both
+  version flags,
+  the token summary, `--quiet`, format validation and budget handling are now
+  contract-tested against real git repositories. Every parity defect above
+  shipped because nothing exercised the clap parser.
+
+## [1.12.1] - 2026-07-23
+
+### Added
+
 - Standalone native binaries (linux x86_64/aarch64, macOS arm64, Windows
   x64) are built and attached to every GitHub release. The README had
   promised them since 1.9.x; releases only ever carried wheels and an sdist.
@@ -19,10 +170,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Container image `ghcr.io/nikolay-e/diffctx` (linux amd64/arm64), built from
   the release tag and smoke-tested against a real repository before the tag
   moves: `docker run --rm -v "$PWD:/repo" ghcr.io/nikolay-e/diffctx . --diff HEAD~1`.
-- Packaging manifests generated from the release checksums: Scoop
-  (`packaging/scoop/diffctx.json`), AUR (`packaging/aur/`) and an npm wrapper
-  (`packaging/npm/`) that downloads the platform binary and verifies its
-  SHA-256 against the published checksum.
+  Mirrored to Docker Hub as `nikolajer/diffctx`.
+- npm wrapper (`packaging/npm/`) that downloads the platform binary and
+  verifies its SHA-256 against the published checksum. Scoop
+  (`packaging/scoop/diffctx.json`) and AUR (`packaging/aur/`) manifests are
+  generated from the same checksums but are not published to any bucket or
+  to the AUR.
 
 ### Fixed
 
@@ -78,7 +231,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   log instead of re-running the builders, so generation cost stays 1x.
   Verified: gitpod 8000s-hang -> 35.7s,
   pytorch 1848s-SIGKILL -> 7.4s, mui/material-ui OOM class recovered.
-  Outputs are bit-identical (gated by `benchmarks/equivalence_gate.py`).
+  Outputs are bit-identical (gated by `eval/analysis/equivalence_gate.py`).
 
 ### Known limitations
 
@@ -117,7 +270,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **The default `--format` is now `md` (Markdown), previously `yaml`.** Markdown
   is ~7% more token-efficient than YAML on real diffs and is preferred by
   reviewers for code-fragment-heavy output (evidence:
-  `benchmarks/real_world_diff_bench/`). YAML remains available via `-f yaml`.
+  `datasets/real-world-diff/v1/`). YAML remains available via `-f yaml`.
   This also changes the default stdout format of `treemapper` (a passthrough
   wrapper over this engine); downstream scripts that parse the default output
   must now pass `-f yaml` explicitly. (#104)
@@ -252,7 +405,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   optional `diffctx-mcp` server confines its filesystem reach via
   `DIFFCTX_ALLOWED_PATHS`.
 - Footer of `README.md` links `CHANGELOG.md`, `SECURITY.md`, and
-  `docs/parameter-strategy.md` so they are no longer orphaned.
+  `docs/engineering/parameter-strategy.md` so they are no longer orphaned.
 
 ### Changed
 
@@ -311,5 +464,5 @@ Earlier releases shipped as `treemapper`; see
 <https://github.com/nikolay-e/diffctx/releases> for the corresponding GitHub
 release notes (`1.0.0` through `1.6.1`).
 
-[Unreleased]: https://github.com/nikolay-e/diffctx/compare/v1.7.0...HEAD
+[Unreleased]: https://github.com/nikolay-e/diffctx/compare/v1.12.1...HEAD
 [1.7.0]: https://github.com/nikolay-e/diffctx/compare/v1.6.1...v1.7.0

{diffctx-1.12.1/diffctx → diffctx-1.12.3}/Cargo.lock

@@ -123,9 +123,9 @@ checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
 
 [[package]]
 name = "clap"
-version = "4.6.2"
+version = "4.6.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dd059f9da4f5c36b3787f65d38ccaab1cc315f07b01f89abc8359ee6a8205011"
+checksum = "d91e0c145792ef73a6ad36d27c75ac09f1832222a3c209689d90f534685ee5b7"
 dependencies = [
  "clap_builder",
  "clap_derive",
@@ -145,14 +145,14 @@ dependencies = [
 
 [[package]]
 name = "clap_derive"
-version = "4.6.1"
+version = "4.6.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9"
+checksum = "d012d2b9d65aca7f18f4d9878a045bc17899bba951561ba5ec3c2ba1eed9a061"
 dependencies = [
  "heck",
  "proc-macro2",
  "quote",
- "syn",
+ "syn 3.0.3",
 ]
 
 [[package]]
@@ -179,9 +179,9 @@ dependencies = [
 
 [[package]]
 name = "crossbeam-epoch"
-version = "0.9.18"
+version = "0.9.20"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
+checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
 dependencies = [
  "crossbeam-utils",
 ]
@@ -194,7 +194,7 @@ checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
 
 [[package]]
 name = "diffctx"
-version = "1.12.1"
+version = "1.12.3"
 dependencies = [
  "anyhow",
  "clap",
@@ -330,9 +330,9 @@ dependencies = [
 
 [[package]]
 name = "glob"
-version = "0.3.3"
+version = "0.3.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
+checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
 
 [[package]]
 name = "hashbrown"
@@ -399,9 +399,9 @@ checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
 
 [[package]]
 name = "libc"
-version = "0.2.186"
+version = "0.2.189"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
 
 [[package]]
 name = "libmimalloc-sys"
@@ -500,7 +500,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
 dependencies = [
  "proc-macro2",
- "syn",
+ "syn 2.0.117",
 ]
 
 [[package]]
@@ -554,7 +554,7 @@ dependencies = [
  "proc-macro2",
  "pyo3-macros-backend",
  "quote",
- "syn",
+ "syn 2.0.117",
 ]
 
 [[package]]
@@ -566,7 +566,7 @@ dependencies = [
  "heck",
  "proc-macro2",
  "quote",
- "syn",
+ "syn 2.0.117",
 ]
 
 [[package]]
@@ -675,9 +675,9 @@ checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
 
 [[package]]
 name = "serde"
-version = "1.0.228"
+version = "1.0.229"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
+checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
 dependencies = [
  "serde_core",
  "serde_derive",
@@ -685,29 +685,29 @@ dependencies = [
 
 [[package]]
 name = "serde_core"
-version = "1.0.228"
+version = "1.0.229"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
+checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
 dependencies = [
  "serde_derive",
 ]
 
 [[package]]
 name = "serde_derive"
-version = "1.0.228"
+version = "1.0.229"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
+checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn",
+ "syn 3.0.3",
 ]
 
 [[package]]
 name = "serde_json"
-version = "1.0.150"
+version = "1.0.151"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"
+checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
 dependencies = [
  "indexmap",
  "itoa",
@@ -783,6 +783,17 @@ dependencies = [
  "unicode-ident",
 ]
 
+[[package]]
+name = "syn"
+version = "3.0.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "unicode-ident",
+]
+
 [[package]]
 name = "target-lexicon"
 version = "0.13.5"
@@ -819,7 +830,7 @@ checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn",
+ "syn 2.0.117",
 ]
 
 [[package]]
@@ -865,7 +876,7 @@ checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn",
+ "syn 2.0.117",
 ]
 
 [[package]]
@@ -1094,9 +1105,9 @@ dependencies = [
 
 [[package]]
 name = "tree-sitter-javascript"
-version = "0.23.1"
+version = "0.25.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bf40bf599e0416c16c125c3cec10ee5ddc7d1bb8b0c60fa5c4de249ad34dc1b1"
+checksum = "68204f2abc0627a90bdf06e605f5c470aa26fdcb2081ea553a04bdad756693f5"
 dependencies = [
  "cc",
  "tree-sitter-language",
@@ -1200,9 +1211,9 @@ dependencies = [
 
 [[package]]
 name = "tree-sitter-python"
-version = "0.23.6"
+version = "0.25.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3d065aaa27f3aaceaf60c1f0e0ac09e1cb9eb8ed28e7bcdaa52129cffc7f4b04"
+checksum = "6bf85fd39652e740bf60f46f4cda9492c3a9ad75880575bf14960f775cb74a1c"
 dependencies = [
  "cc",
  "tree-sitter-language",
@@ -1260,9 +1271,9 @@ dependencies = [
 
 [[package]]
 name = "tree-sitter-swift"
-version = "0.6.0"
+version = "0.7.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d65aeb41726119416567d0333ec17580ac4abfb96db1f67c4bd638c65f9992fe"
+checksum = "fe36052155b9dd69ca82b3b8f1b4ccfb2d867125ac1a4db1dd7331829242668c"
 dependencies = [
  "cc",
  "tree-sitter-language",
@@ -1459,7 +1470,7 @@ dependencies = [
  "heck",
  "indexmap",
  "prettyplease",
- "syn",
+ "syn 2.0.117",
  "wasm-metadata",
  "wit-bindgen-core",
  "wit-component",
@@ -1475,7 +1486,7 @@ dependencies = [
  "prettyplease",
  "proc-macro2",
  "quote",
- "syn",
+ "syn 2.0.117",
  "wit-bindgen-core",
  "wit-bindgen-rust",
 ]

diffctx-1.12.3/Cargo.toml

@@ -0,0 +1,20 @@
+[workspace]
+members = ["crates/diffctx-native"]
+resolver = "2"
+
+[profile.release]
+opt-level = 3
+lto = "fat"
+codegen-units = 1
+strip = true
+panic = "abort"
+debug = "line-tables-only"
+split-debuginfo = "packed"
+
+# `cargo test --release` with panic = "abort" compiles the whole dependency
+# graph twice (abort for the cdylib lib artifact, unwind for the test
+# universe); the hashless cdylib output names then collide (cargo#6313).
+# Tests run through this profile instead: same optimizations, one universe.
+[profile.release-unwind]
+inherits = "release"
+panic = "unwind"