deepdiff-rs 0.4.0__tar.gz → 0.5.0__tar.gz

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/Cargo.lock

@@ -127,7 +127,7 @@ checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
 
 [[package]]
 name = "onix-cli"
-version = "0.4.0"
+version = "0.5.0"
 dependencies = [
  "onix-core",
  "serde_json",
@@ -135,7 +135,7 @@ dependencies = [
 
 [[package]]
 name = "onix-core"
-version = "0.4.0"
+version = "0.5.0"
 dependencies = [
  "proptest",
  "serde",
@@ -145,7 +145,7 @@ dependencies = [
 
 [[package]]
 name = "onix-py"
-version = "0.4.0"
+version = "0.5.0"
 dependencies = [
  "onix-core",
  "pyo3",

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/Cargo.toml

@@ -3,7 +3,7 @@ resolver = "3"
 members = ["crates/onix-core", "crates/onix-py"]
 
 [workspace.package]
-version = "0.4.0"
+version = "0.5.0"
 edition = "2024"
 license = "MIT"
 

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: deepdiff-rs
-Version: 0.4.0
+Version: 0.5.0
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Rust
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
-Summary: onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4588x faster, with ignore_order support included.
+Summary: onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4245x faster, with ignore_order support included.
 Keywords: diff,deepdiff,json,rust,performance
 Author: Rodrigo Carvajal
 License: MIT
@@ -23,7 +23,7 @@ Project-URL: Repository, https://github.com/ksco92/onix
 [![license](https://img.shields.io/github/license/ksco92/onix.svg)](LICENSE)
 [![last commit](https://img.shields.io/github/last-commit/ksco92/onix.svg)](https://github.com/ksco92/onix/commits/main)
 
-**onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4588x faster, with `ignore_order` support included.** Install it as `deepdiff-rs`, a drop-in `DeepDiff` class for Python, or run the diff engine as the `onix` command-line tool.
+**onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4245x faster, with `ignore_order` support included.** Install it as `deepdiff-rs`, a drop-in `DeepDiff` class for Python, or run the diff engine as the `onix` command-line tool.
 
 `deepdiff-rs` reads live Python objects (or JSON) and produces the exact same report [DeepDiff](https://github.com/seperman/deepdiff) does at `verbose_level=2`, so it slots into code that already parses DeepDiff output while running dramatically faster on large or deeply nested inputs.
 
@@ -118,7 +118,7 @@ Pass `--ignore-order` to compare every list by value instead of by position, mir
 - A subclass of a supported type (a `tuple`, `set` or `frozenset` subclass including `namedtuple`, a `datetime`/`date` subclass such as pandas' `Timestamp`) raises `TypeError` rather than being diffed as its base type, because DeepDiff reports each value's own type name. A `type_changes` entry's `old_type`/`new_type` are type *names* in `to_dict()`, where DeepDiff returns the type objects. Both are described in [`tests/golden/README.md`](tests/golden/README.md).
 - **Datetimes** compare by instant, with a naive value read as UTC, matching DeepDiff. A changed pair is reported normalized to UTC (`to_json()` renders `...+00:00`, `to_dict()` returns UTC-aware `datetime`s); everywhere else a datetime keeps its raw value. Three deliberate departures: `to_json()` renders a `date` as `YYYY-MM-DD` where DeepDiff's own `to_json()` raises `TypeError` (a documented superset); a `zoneinfo`/`pytz` tzinfo comes back from `to_dict()` as a fixed-offset `datetime.timezone` carrying the offset it was in force at, not the original zone object; and a set holding both a naive and an aware value at one instant reports both as members, where DeepDiff's own digest cache can report only one (see [`crates/onix-py/src/convert.rs`](crates/onix-py/src/convert.rs)). Comparing two datetimes whose UTC form would leave year 1..=9999 raises `ValueError` naming the path, where DeepDiff raises `OverflowError`; under `ignore_order` DeepDiff's hasher normalizes every datetime and so raises for such a value even when it is only added, removed, or shuffled, where onix hashes by instant and reports it normally (see [`tests/golden/README.md`](tests/golden/README.md)). `truncate_datetime`, `time` and `timedelta` are not supported. The normalized-versus-raw split is documented in [`tests/golden/README.md`](tests/golden/README.md).
 - **Sets** are diffed deterministically, where DeepDiff's own answers depend on the order the running process happens to iterate a set in (hash order, and `PYTHONHASHSEED`-dependent for `str` members) or on how its digest cache/computation handles a tuple, frozenset, or calendar member independently of Python's own `==`. Each consequence — entry order, which member of an equality class is reported, set-versus-sequence coercion, and a tuple/frozenset member's own (positional, not order-/repetition-insensitive) matching rule — is shown with both tools' output in [`tests/golden/README.md`](tests/golden/README.md)'s "Set iteration order" section. A report holding a `frozenset` value also serializes to JSON here, where DeepDiff's own `to_json()` raises `TypeError` — a superset, not a difference in the findings.
-- Two distinct `str` values that differ only in lone surrogates (`'\ud800'` and `'\udc00'`) convert to the same string, so a set holding both loses a member and with it a finding. Tracked as [issue #27](https://github.com/ksco92/onix/issues/27).
+- A `str` containing a lone (unpaired) surrogate code point (e.g. `'\udc80'`, legal in Python but not encodable as UTF-8) raises `ValueError` naming the exact path on either side, before the two values are ever compared — including a pair DeepDiff would call equal and report as no change, since DeepDiff's scalar equality is plain Python `==` and never hits the encoding problem; DeepDiff does report a plain change for a *differing* pair, and crashes with an unhandled `UnicodeEncodeError` if such a string is ever hashed (a `set`/`frozenset` member). See [`tests/golden/README.md`](tests/golden/README.md)'s "Known DeepDiff quirks" section.
 - A `str` inside a `tuple` or `frozenset` set item is rendered with Python's `repr()`, which escapes every non-printable character; onix escapes those below `U+0100` (the complete set in that range) and passes higher non-printable code points through literally, since escaping them would mean carrying a Unicode category table. Exact for all of ASCII and all printable text. See [`crates/onix-core/src/path.rs`](crates/onix-core/src/path.rs).
 - Adversarially deep input raises `MaxDepthError` instead of crashing: the default `max_depth` is 512 and the hard ceiling is `MAX_DEPTH_CEILING` (20,000). See [`crates/onix-py/src/guard.rs`](crates/onix-py/src/guard.rs).
 - `ignore_order` pairing is `O(N^2)` in unpaired elements per side and carries a polynomial cost in both time and memory with input depth; it has no `max_passes`/`max_diffs` cutoff, so bound the size and depth of untrusted input yourself. See [`crates/onix-core/src/ignore_order/mod.rs`](crates/onix-core/src/ignore_order/mod.rs).
@@ -128,40 +128,46 @@ Pass `--ignore-order` to compare every list by value instead of by position, mir
 
 Two committed, regenerable reports back the numbers below; every figure here is copied verbatim from them.
 
-The Python bindings against real `deepdiff` on **live Python objects**, the number a real caller pays (source: [`crates/onix-py/benchmarks/bench_bindings.py`](crates/onix-py/benchmarks/bench_bindings.py), macOS 26.5.1, Apple M5 Max, median of 11 isolated subprocess runs per side):
+The Python bindings against real `deepdiff` on **live Python objects**, the number a real caller pays (source: [`crates/onix-py/benchmarks/bench_bindings.py`](crates/onix-py/benchmarks/bench_bindings.py), macOS 26.5.1, Apple M5 Max, median of 11 isolated subprocess runs per side, run on 2026-09-04):
 
 | Shape | deepdiff | deepdiff_rs | Speedup |
 | --- | --- | --- | --- |
-| `ignore_order`, 10k shuffled ints, ~5% mutated (live objects) | 3128.94ms | 82.57ms | **37.89x** |
-|   peak RSS | 228.0 MB | 141.2 MB | **1.61x** |
-|   CPU seconds | 3.128 s | 0.083 s | **37.88x** |
-| Heterogeneous API-payload records, n=20,000 (live objects) | 3466.52ms | 149.18ms | **23.24x** |
-|   peak RSS | 117.5 MB | 148.3 MB | **0.79x** |
-|   CPU seconds | 3.465 s | 0.149 s | **23.24x** |
-| Same `ignore_order` shape, via `diff_json` (JSON-string path) | 3133.45ms | 83.16ms | **37.68x** |
-|   peak RSS | 228.9 MB | 141.7 MB | **1.62x** |
-|   CPU seconds | 3.131 s | 0.083 s | **37.68x** |
-| Same API-payload shape, via `diff_json` (JSON-string path) | 4568.33ms | 86.33ms | **52.92x** |
-|   peak RSS | 139.0 MB | 141.7 MB | **0.98x** |
-|   CPU seconds | 4.566 s | 0.086 s | **52.90x** |
-| Same API-payload shape, both tools reading two JSON files from disk | 4571.28ms | 85.35ms | **53.56x** |
-|   peak RSS | 139.0 MB | 141.8 MB | **0.98x** |
-|   CPU seconds | 4.569 s | 0.085 s | **53.54x** |
+| `ignore_order`, 10k shuffled ints, ~5% mutated (live objects) | 3111.56ms | 71.68ms | **43.41x** |
+|   peak RSS | 228.5 MB | 93.2 MB | **2.45x** |
+|   CPU seconds | 3.110 s | 0.072 s | **43.42x** |
+| Heterogeneous API-payload records, n=20,000 (live objects) | 3439.96ms | 153.83ms | **22.36x** |
+|   peak RSS | 118.1 MB | 147.8 MB | **0.80x** |
+|   CPU seconds | 3.439 s | 0.154 s | **22.36x** |
+| Typed records (datetime/tuple/set fields), n=10,000 (live objects) | 795.17ms | 48.48ms | **16.40x** |
+|   peak RSS | 60.2 MB | 62.2 MB | **0.97x** |
+|   CPU seconds | 0.795 s | 0.048 s | **16.40x** |
+| Same typed-records shape, `ignore_order` (live objects) | 60506.65ms | 775.38ms | **78.03x** |
+|   peak RSS | 110.3 MB | 121.6 MB | **0.91x** |
+|   CPU seconds | 60.471 s | 0.774 s | **78.10x** |
+| Same `ignore_order` shape, via `diff_json` (JSON-string path) | 3116.22ms | 73.85ms | **42.20x** |
+|   peak RSS | 228.8 MB | 93.8 MB | **2.44x** |
+|   CPU seconds | 3.114 s | 0.074 s | **42.20x** |
+| Same API-payload shape, via `diff_json` (JSON-string path) | 4559.90ms | 87.02ms | **52.40x** |
+|   peak RSS | 139.5 MB | 140.9 MB | **0.99x** |
+|   CPU seconds | 4.558 s | 0.087 s | **52.58x** |
+| Same API-payload shape, both tools reading two JSON files from disk | 4555.37ms | 85.62ms | **53.20x** |
+|   peak RSS | 139.5 MB | 141.0 MB | **0.99x** |
+|   CPU seconds | 4.553 s | 0.086 s | **53.21x** |
 
 The engine's own diff-only time and peak resident memory against pinned `deepdiff` 9.1.0 (source: [`perf/RESULTS.md`](perf/RESULTS.md), same machine, median over tier-appropriate runs, diff time excluding process startup and JSON parsing on both sides):
 
 | Fixture | onix diff-only (median, min-max) | deepdiff diff-only (median, min-max) | Speedup | onix peak RSS | deepdiff peak RSS | Memory ratio | ≥5x threshold |
 |---|---|---|---|---|---|---|---|
-| `flat_dict_10k` | 3.093 ms (3.020 ms-3.153 ms) | 143.691 ms (140.322 ms-145.891 ms) | 46.46x | 5.72 MB | 39.46 MB | 6.90x | ✅ |
-| `flat_dict_100k` | 38.354 ms (37.963 ms-38.916 ms) | 1.596 s (1.584 s-1.686 s) | 41.60x | 40.53 MB | 110.95 MB | 2.74x | ✅ |
-| `flat_dict_1m` | 450.356 ms (449.689 ms-467.911 ms) | 16.936 s (16.889 s-17.154 s) | 37.60x | 478.10 MB | 753.75 MB | 1.58x | ✅ |
-| `flat_list_100k` | 81.787 ms (78.266 ms-89.593 ms) | 4.798 s (4.746 s-4.828 s) | 58.67x | 38.16 MB | 155.11 MB | 4.06x | ✅ |
-| `nested_uniform_d6_b10` | 208.770 ms (208.269 ms-209.842 ms) | 71.500 s (71.498 s-72.875 s) | 342.48x | 224.58 MB | 908.33 MB | 4.04x | ✅ |
-| `api_payloads` | 160.667 ms (153.309 ms-171.032 ms) | 94.138 s (93.625 s-94.669 s) | 585.92x | 269.39 MB | 544.98 MB | 2.02x | ✅ |
-| `deep_narrow_d120` | 0.027 ms (0.026 ms-0.028 ms) | 123.792 ms (123.234 ms-124.809 ms) | 4588.45x | 2.18 MB | 41.08 MB | 18.85x | ✅ |
-| `startup_trivial` | 0.001 ms (0.001 ms-0.001 ms) | 0.181 ms (0.174 ms-0.207 ms) | 177.24x | 2.18 MB | 32.69 MB | 15.00x | ✅ |
-| `ignore_order_10k` | 80.965 ms (80.117 ms-83.510 ms) | 13.010 s (12.900 s-13.023 s) | 160.68x | 108.82 MB | 345.41 MB | 3.17x | ✅ |
-| `identical_1m` | 6.996 ms (6.114 ms-7.731 ms) | 15.853 s (15.773 s-16.021 s) | 2266.18x | 315.39 MB | 503.28 MB | 1.60x | ✅ |
+| `flat_dict_10k` | 3.154 ms (3.058 ms-3.220 ms) | 141.155 ms (140.606 ms-142.781 ms) | 44.75x | 5.78 MB | 39.29 MB | 6.79x | ✅ |
+| `flat_dict_100k` | 38.440 ms (38.000 ms-38.806 ms) | 1.594 s (1.581 s-1.602 s) | 41.47x | 40.57 MB | 110.82 MB | 2.73x | ✅ |
+| `flat_dict_1m` | 460.082 ms (454.820 ms-465.133 ms) | 17.061 s (16.926 s-17.162 s) | 37.08x | 478.15 MB | 753.65 MB | 1.58x | ✅ |
+| `flat_list_100k` | 82.907 ms (81.131 ms-84.654 ms) | 4.751 s (4.715 s-4.820 s) | 57.31x | 38.17 MB | 154.95 MB | 4.06x | ✅ |
+| `nested_uniform_d6_b10` | 207.242 ms (205.249 ms-217.027 ms) | 71.458 s (70.959 s-71.599 s) | 344.80x | 227.41 MB | 868.32 MB | 3.82x | ✅ |
+| `api_payloads` | 162.489 ms (159.446 ms-178.736 ms) | 93.764 s (93.687 s-94.474 s) | 577.05x | 270.09 MB | 609.93 MB | 2.26x | ✅ |
+| `deep_narrow_d120` | 0.029 ms (0.028 ms-0.030 ms) | 123.650 ms (123.230 ms-125.018 ms) | 4245.55x | 2.15 MB | 41.27 MB | 19.23x | ✅ |
+| `startup_trivial` | 0.001 ms (0.001 ms-0.001 ms) | 0.175 ms (0.169 ms-0.183 ms) | 147.75x | 2.15 MB | 32.67 MB | 15.22x | ✅ |
+| `ignore_order_10k` | 73.475 ms (71.672 ms-73.940 ms) | 12.976 s (12.900 s-13.005 s) | 176.60x | 60.11 MB | 345.19 MB | 5.74x | ✅ |
+| `identical_1m` | 9.254 ms (6.726 ms-9.875 ms) | 15.790 s (15.660 s-15.989 s) | 1706.35x | 315.41 MB | 503.19 MB | 1.60x | ✅ |
 
 Both reports carry their full methodology, fairness rules, and the reproduce command. `perf/RESULTS.md` is an upper bound (JSON parsed straight into the engine, no Python-object conversion); the bindings table is the product-surface number. Regenerate them with `perf/run_bench.sh` and `crates/onix-py/benchmarks/bench_bindings.py` (see [CONTRIBUTING.md](CONTRIBUTING.md)).
 

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/README.md

@@ -7,7 +7,7 @@
 [![license](https://img.shields.io/github/license/ksco92/onix.svg)](LICENSE)
 [![last commit](https://img.shields.io/github/last-commit/ksco92/onix.svg)](https://github.com/ksco92/onix/commits/main)
 
-**onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4588x faster, with `ignore_order` support included.** Install it as `deepdiff-rs`, a drop-in `DeepDiff` class for Python, or run the diff engine as the `onix` command-line tool.
+**onix is a Rust rewrite of Python DeepDiff's core: byte-compatible output, 37-4245x faster, with `ignore_order` support included.** Install it as `deepdiff-rs`, a drop-in `DeepDiff` class for Python, or run the diff engine as the `onix` command-line tool.
 
 `deepdiff-rs` reads live Python objects (or JSON) and produces the exact same report [DeepDiff](https://github.com/seperman/deepdiff) does at `verbose_level=2`, so it slots into code that already parses DeepDiff output while running dramatically faster on large or deeply nested inputs.
 
@@ -102,7 +102,7 @@ Pass `--ignore-order` to compare every list by value instead of by position, mir
 - A subclass of a supported type (a `tuple`, `set` or `frozenset` subclass including `namedtuple`, a `datetime`/`date` subclass such as pandas' `Timestamp`) raises `TypeError` rather than being diffed as its base type, because DeepDiff reports each value's own type name. A `type_changes` entry's `old_type`/`new_type` are type *names* in `to_dict()`, where DeepDiff returns the type objects. Both are described in [`tests/golden/README.md`](tests/golden/README.md).
 - **Datetimes** compare by instant, with a naive value read as UTC, matching DeepDiff. A changed pair is reported normalized to UTC (`to_json()` renders `...+00:00`, `to_dict()` returns UTC-aware `datetime`s); everywhere else a datetime keeps its raw value. Three deliberate departures: `to_json()` renders a `date` as `YYYY-MM-DD` where DeepDiff's own `to_json()` raises `TypeError` (a documented superset); a `zoneinfo`/`pytz` tzinfo comes back from `to_dict()` as a fixed-offset `datetime.timezone` carrying the offset it was in force at, not the original zone object; and a set holding both a naive and an aware value at one instant reports both as members, where DeepDiff's own digest cache can report only one (see [`crates/onix-py/src/convert.rs`](crates/onix-py/src/convert.rs)). Comparing two datetimes whose UTC form would leave year 1..=9999 raises `ValueError` naming the path, where DeepDiff raises `OverflowError`; under `ignore_order` DeepDiff's hasher normalizes every datetime and so raises for such a value even when it is only added, removed, or shuffled, where onix hashes by instant and reports it normally (see [`tests/golden/README.md`](tests/golden/README.md)). `truncate_datetime`, `time` and `timedelta` are not supported. The normalized-versus-raw split is documented in [`tests/golden/README.md`](tests/golden/README.md).
 - **Sets** are diffed deterministically, where DeepDiff's own answers depend on the order the running process happens to iterate a set in (hash order, and `PYTHONHASHSEED`-dependent for `str` members) or on how its digest cache/computation handles a tuple, frozenset, or calendar member independently of Python's own `==`. Each consequence — entry order, which member of an equality class is reported, set-versus-sequence coercion, and a tuple/frozenset member's own (positional, not order-/repetition-insensitive) matching rule — is shown with both tools' output in [`tests/golden/README.md`](tests/golden/README.md)'s "Set iteration order" section. A report holding a `frozenset` value also serializes to JSON here, where DeepDiff's own `to_json()` raises `TypeError` — a superset, not a difference in the findings.
-- Two distinct `str` values that differ only in lone surrogates (`'\ud800'` and `'\udc00'`) convert to the same string, so a set holding both loses a member and with it a finding. Tracked as [issue #27](https://github.com/ksco92/onix/issues/27).
+- A `str` containing a lone (unpaired) surrogate code point (e.g. `'\udc80'`, legal in Python but not encodable as UTF-8) raises `ValueError` naming the exact path on either side, before the two values are ever compared — including a pair DeepDiff would call equal and report as no change, since DeepDiff's scalar equality is plain Python `==` and never hits the encoding problem; DeepDiff does report a plain change for a *differing* pair, and crashes with an unhandled `UnicodeEncodeError` if such a string is ever hashed (a `set`/`frozenset` member). See [`tests/golden/README.md`](tests/golden/README.md)'s "Known DeepDiff quirks" section.
 - A `str` inside a `tuple` or `frozenset` set item is rendered with Python's `repr()`, which escapes every non-printable character; onix escapes those below `U+0100` (the complete set in that range) and passes higher non-printable code points through literally, since escaping them would mean carrying a Unicode category table. Exact for all of ASCII and all printable text. See [`crates/onix-core/src/path.rs`](crates/onix-core/src/path.rs).
 - Adversarially deep input raises `MaxDepthError` instead of crashing: the default `max_depth` is 512 and the hard ceiling is `MAX_DEPTH_CEILING` (20,000). See [`crates/onix-py/src/guard.rs`](crates/onix-py/src/guard.rs).
 - `ignore_order` pairing is `O(N^2)` in unpaired elements per side and carries a polynomial cost in both time and memory with input depth; it has no `max_passes`/`max_diffs` cutoff, so bound the size and depth of untrusted input yourself. See [`crates/onix-core/src/ignore_order/mod.rs`](crates/onix-core/src/ignore_order/mod.rs).
@@ -112,40 +112,46 @@ Pass `--ignore-order` to compare every list by value instead of by position, mir
 
 Two committed, regenerable reports back the numbers below; every figure here is copied verbatim from them.
 
-The Python bindings against real `deepdiff` on **live Python objects**, the number a real caller pays (source: [`crates/onix-py/benchmarks/bench_bindings.py`](crates/onix-py/benchmarks/bench_bindings.py), macOS 26.5.1, Apple M5 Max, median of 11 isolated subprocess runs per side):
+The Python bindings against real `deepdiff` on **live Python objects**, the number a real caller pays (source: [`crates/onix-py/benchmarks/bench_bindings.py`](crates/onix-py/benchmarks/bench_bindings.py), macOS 26.5.1, Apple M5 Max, median of 11 isolated subprocess runs per side, run on 2026-09-04):
 
 | Shape | deepdiff | deepdiff_rs | Speedup |
 | --- | --- | --- | --- |
-| `ignore_order`, 10k shuffled ints, ~5% mutated (live objects) | 3128.94ms | 82.57ms | **37.89x** |
-|   peak RSS | 228.0 MB | 141.2 MB | **1.61x** |
-|   CPU seconds | 3.128 s | 0.083 s | **37.88x** |
-| Heterogeneous API-payload records, n=20,000 (live objects) | 3466.52ms | 149.18ms | **23.24x** |
-|   peak RSS | 117.5 MB | 148.3 MB | **0.79x** |
-|   CPU seconds | 3.465 s | 0.149 s | **23.24x** |
-| Same `ignore_order` shape, via `diff_json` (JSON-string path) | 3133.45ms | 83.16ms | **37.68x** |
-|   peak RSS | 228.9 MB | 141.7 MB | **1.62x** |
-|   CPU seconds | 3.131 s | 0.083 s | **37.68x** |
-| Same API-payload shape, via `diff_json` (JSON-string path) | 4568.33ms | 86.33ms | **52.92x** |
-|   peak RSS | 139.0 MB | 141.7 MB | **0.98x** |
-|   CPU seconds | 4.566 s | 0.086 s | **52.90x** |
-| Same API-payload shape, both tools reading two JSON files from disk | 4571.28ms | 85.35ms | **53.56x** |
-|   peak RSS | 139.0 MB | 141.8 MB | **0.98x** |
-|   CPU seconds | 4.569 s | 0.085 s | **53.54x** |
+| `ignore_order`, 10k shuffled ints, ~5% mutated (live objects) | 3111.56ms | 71.68ms | **43.41x** |
+|   peak RSS | 228.5 MB | 93.2 MB | **2.45x** |
+|   CPU seconds | 3.110 s | 0.072 s | **43.42x** |
+| Heterogeneous API-payload records, n=20,000 (live objects) | 3439.96ms | 153.83ms | **22.36x** |
+|   peak RSS | 118.1 MB | 147.8 MB | **0.80x** |
+|   CPU seconds | 3.439 s | 0.154 s | **22.36x** |
+| Typed records (datetime/tuple/set fields), n=10,000 (live objects) | 795.17ms | 48.48ms | **16.40x** |
+|   peak RSS | 60.2 MB | 62.2 MB | **0.97x** |
+|   CPU seconds | 0.795 s | 0.048 s | **16.40x** |
+| Same typed-records shape, `ignore_order` (live objects) | 60506.65ms | 775.38ms | **78.03x** |
+|   peak RSS | 110.3 MB | 121.6 MB | **0.91x** |
+|   CPU seconds | 60.471 s | 0.774 s | **78.10x** |
+| Same `ignore_order` shape, via `diff_json` (JSON-string path) | 3116.22ms | 73.85ms | **42.20x** |
+|   peak RSS | 228.8 MB | 93.8 MB | **2.44x** |
+|   CPU seconds | 3.114 s | 0.074 s | **42.20x** |
+| Same API-payload shape, via `diff_json` (JSON-string path) | 4559.90ms | 87.02ms | **52.40x** |
+|   peak RSS | 139.5 MB | 140.9 MB | **0.99x** |
+|   CPU seconds | 4.558 s | 0.087 s | **52.58x** |
+| Same API-payload shape, both tools reading two JSON files from disk | 4555.37ms | 85.62ms | **53.20x** |
+|   peak RSS | 139.5 MB | 141.0 MB | **0.99x** |
+|   CPU seconds | 4.553 s | 0.086 s | **53.21x** |
 
 The engine's own diff-only time and peak resident memory against pinned `deepdiff` 9.1.0 (source: [`perf/RESULTS.md`](perf/RESULTS.md), same machine, median over tier-appropriate runs, diff time excluding process startup and JSON parsing on both sides):
 
 | Fixture | onix diff-only (median, min-max) | deepdiff diff-only (median, min-max) | Speedup | onix peak RSS | deepdiff peak RSS | Memory ratio | ≥5x threshold |
 |---|---|---|---|---|---|---|---|
-| `flat_dict_10k` | 3.093 ms (3.020 ms-3.153 ms) | 143.691 ms (140.322 ms-145.891 ms) | 46.46x | 5.72 MB | 39.46 MB | 6.90x | ✅ |
-| `flat_dict_100k` | 38.354 ms (37.963 ms-38.916 ms) | 1.596 s (1.584 s-1.686 s) | 41.60x | 40.53 MB | 110.95 MB | 2.74x | ✅ |
-| `flat_dict_1m` | 450.356 ms (449.689 ms-467.911 ms) | 16.936 s (16.889 s-17.154 s) | 37.60x | 478.10 MB | 753.75 MB | 1.58x | ✅ |
-| `flat_list_100k` | 81.787 ms (78.266 ms-89.593 ms) | 4.798 s (4.746 s-4.828 s) | 58.67x | 38.16 MB | 155.11 MB | 4.06x | ✅ |
-| `nested_uniform_d6_b10` | 208.770 ms (208.269 ms-209.842 ms) | 71.500 s (71.498 s-72.875 s) | 342.48x | 224.58 MB | 908.33 MB | 4.04x | ✅ |
-| `api_payloads` | 160.667 ms (153.309 ms-171.032 ms) | 94.138 s (93.625 s-94.669 s) | 585.92x | 269.39 MB | 544.98 MB | 2.02x | ✅ |
-| `deep_narrow_d120` | 0.027 ms (0.026 ms-0.028 ms) | 123.792 ms (123.234 ms-124.809 ms) | 4588.45x | 2.18 MB | 41.08 MB | 18.85x | ✅ |
-| `startup_trivial` | 0.001 ms (0.001 ms-0.001 ms) | 0.181 ms (0.174 ms-0.207 ms) | 177.24x | 2.18 MB | 32.69 MB | 15.00x | ✅ |
-| `ignore_order_10k` | 80.965 ms (80.117 ms-83.510 ms) | 13.010 s (12.900 s-13.023 s) | 160.68x | 108.82 MB | 345.41 MB | 3.17x | ✅ |
-| `identical_1m` | 6.996 ms (6.114 ms-7.731 ms) | 15.853 s (15.773 s-16.021 s) | 2266.18x | 315.39 MB | 503.28 MB | 1.60x | ✅ |
+| `flat_dict_10k` | 3.154 ms (3.058 ms-3.220 ms) | 141.155 ms (140.606 ms-142.781 ms) | 44.75x | 5.78 MB | 39.29 MB | 6.79x | ✅ |
+| `flat_dict_100k` | 38.440 ms (38.000 ms-38.806 ms) | 1.594 s (1.581 s-1.602 s) | 41.47x | 40.57 MB | 110.82 MB | 2.73x | ✅ |
+| `flat_dict_1m` | 460.082 ms (454.820 ms-465.133 ms) | 17.061 s (16.926 s-17.162 s) | 37.08x | 478.15 MB | 753.65 MB | 1.58x | ✅ |
+| `flat_list_100k` | 82.907 ms (81.131 ms-84.654 ms) | 4.751 s (4.715 s-4.820 s) | 57.31x | 38.17 MB | 154.95 MB | 4.06x | ✅ |
+| `nested_uniform_d6_b10` | 207.242 ms (205.249 ms-217.027 ms) | 71.458 s (70.959 s-71.599 s) | 344.80x | 227.41 MB | 868.32 MB | 3.82x | ✅ |
+| `api_payloads` | 162.489 ms (159.446 ms-178.736 ms) | 93.764 s (93.687 s-94.474 s) | 577.05x | 270.09 MB | 609.93 MB | 2.26x | ✅ |
+| `deep_narrow_d120` | 0.029 ms (0.028 ms-0.030 ms) | 123.650 ms (123.230 ms-125.018 ms) | 4245.55x | 2.15 MB | 41.27 MB | 19.23x | ✅ |
+| `startup_trivial` | 0.001 ms (0.001 ms-0.001 ms) | 0.175 ms (0.169 ms-0.183 ms) | 147.75x | 2.15 MB | 32.67 MB | 15.22x | ✅ |
+| `ignore_order_10k` | 73.475 ms (71.672 ms-73.940 ms) | 12.976 s (12.900 s-13.005 s) | 176.60x | 60.11 MB | 345.19 MB | 5.74x | ✅ |
+| `identical_1m` | 9.254 ms (6.726 ms-9.875 ms) | 15.790 s (15.660 s-15.989 s) | 1706.35x | 315.41 MB | 503.19 MB | 1.60x | ✅ |
 
 Both reports carry their full methodology, fairness rules, and the reproduce command. `perf/RESULTS.md` is an upper bound (JSON parsed straight into the engine, no Python-object conversion); the bindings table is the product-surface number. Regenerate them with `perf/run_bench.sh` and `crates/onix-py/benchmarks/bench_bindings.py` (see [CONTRIBUTING.md](CONTRIBUTING.md)).
 

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/crates/onix-core/src/ignore_order/fxhash.rs

@@ -48,10 +48,13 @@ pub(crate) type HashSet<T> = std::collections::HashSet<T, BuildHasherDefault<FxH
 /// the pairing/`used` sets, and the distance memo — is reached **only** under
 /// `ignore_order=true`, the pairing hot path already bounded by the `O(N²)`
 /// caveat below; those keep `FxHash`, and their float-carrying keys
-/// ([`ItemKey`](super::hash::ItemKey)/[`ScalarKey`](super::hash::ScalarKey)) mix
-/// their bits first ([`mix_float_bits`](crate::lcs::mix_float_bits)) so a
-/// *non-adversarial* run of integral/half-integer floats — whose raw bit
-/// patterns share ~50 trailing zeros — does not accidentally collide.
+/// ([`ItemKey`](super::hash::ItemKey), [`ScalarKey`](super::hash::ScalarKey),
+/// and the distance memo's [`DistKey`](super::hash::DistKey)) mix their bits
+/// first ([`mix_float_bits`](crate::lcs::mix_float_bits)) — `DistKey`'s
+/// encoding routes every float through `number_key`, hence `ItemKey::Float`,
+/// hence `mix_float_bits`, so the hazard does not reach the new key either — a
+/// *non-adversarial* run of integral/half-integer floats, whose raw bit
+/// patterns share ~50 trailing zeros, does not accidentally collide.
 ///
 /// For those remaining `ignore_order`-only tables the trade is deliberate and
 /// measured. Re-keying them to `SipHash` (`RandomState`) added a material,

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/crates/onix-core/src/ignore_order/hash.rs

@@ -5,6 +5,7 @@
 //! how this fits into the algorithm end to end.
 
 use std::collections::{BTreeMap, BTreeSet};
+use std::hash::{Hash, Hasher};
 use std::rc::Rc;
 
 use crate::lcs::{ScalarKey, mix_float_bits, python_scalar_key};
@@ -13,6 +14,126 @@ use crate::value::Value;
 use super::IgnoreOrderMemo;
 use super::fxhash::HashMap;
 
+// ---------------------------------------------------------------------
+// Distance-memo cache key
+// ---------------------------------------------------------------------
+
+/// The distance memo's cache key for one side of a candidate pair: a value's
+/// **exact** structural identity.
+///
+/// [`ItemKey`] cannot serve here. It is deliberately order- and
+/// repetition-*insensitive* for a list/tuple (its `List`/`Tuple` payload is a
+/// [`BTreeSet`], matching `DeepHash`'s item-matching rules), but the distance a
+/// candidate pair is ranked by reads multiplicity — [`super::distance::rough_length`]
+/// counts every repeated element, and the trial diff's leaf count depends on
+/// each list's first-occurrence representative — so two values that share an
+/// `ItemKey` can have genuinely different distances. Keying the memo by
+/// `ItemKey` handed one such value's cached distance to the other (issue #31).
+///
+/// This keys by [`Value`]'s own `PartialEq` instead, which is exact:
+/// order- and repetition-preserving for lists and tuples, variant-sensitive
+/// for numbers, by-instant for datetimes. Two entries therefore share a cache
+/// slot only when their values are structurally identical and so their
+/// distance is provably equal — the memo is decision-neutral by construction.
+/// The [`Rc`] lets the shared value outlive the per-level [`HashedList`] that
+/// produced it and keeps a cache entry two pointers wide.
+#[derive(Clone)]
+pub(crate) struct DistKey(Rc<Value>);
+
+impl DistKey {
+    /// Interns a copy of `value` as a cache key: the value is cloned once per
+    /// distinct candidate (added/removed) entry, not once per pair, so the
+    /// `A * R` cache entries one pairing records cost only a refcount bump of
+    /// memory each. Per-*lookup* work is not constant, however — every probe
+    /// hashes the key ([`hash_value`] walks the whole value) and, on a bucket
+    /// match, compares two keys structurally — so a pairing's distance work is
+    /// proportional to record size, a constant-factor cost the `ignore_order`
+    /// input-size cap already covers (measured 2.3x-5.6x slower than a
+    /// deduplicating `ItemKey` key only on a crafted worst case: large values
+    /// whose `ItemKey` collapses to a tiny set; every realistic shape is
+    /// faster). The clone recurses natively like the report's own value clones
+    /// and is bounded by the same [`crate::diff::check_value_depth`] pre-pass;
+    /// the hashing the key is looked up by is iterative (see [`hash_value`]).
+    pub(crate) fn new(value: &Value) -> Self {
+        Self(Rc::new(value.clone()))
+    }
+
+    /// Wraps an already-owned value with no clone — the test hook that lets the
+    /// stack-safety probe hash a value deeper than a native clone could build.
+    #[cfg(test)]
+    pub(crate) fn from_rc(value: Rc<Value>) -> Self {
+        Self(value)
+    }
+}
+
+impl PartialEq for DistKey {
+    fn eq(&self, other: &Self) -> bool {
+        // `Value`'s own iterative structural equality — exact, and stack-safe
+        // on the deep values `Rc::ptr_eq` would miss across nesting levels.
+        self.0 == other.0
+    }
+}
+
+impl Eq for DistKey {}
+
+impl Hash for DistKey {
+    fn hash<H: Hasher>(&self, state: &mut H) {
+        hash_value(&self.0, state);
+    }
+}
+
+/// Hashes a value consistently with its structural `PartialEq` (equal values
+/// hash equal): a per-variant discriminant, then the fields that equality
+/// compares — numbers through [`number_key`] (so `-0.0`/`0.0` agree and an
+/// int and an equal-valued float stay distinct), a datetime by its instant, a
+/// date by its ordinal, a list/tuple by its length and elements (order and
+/// repetition preserving), a set/frozenset by its canonical members, a dict by
+/// its sorted keys and their values. The exact byte sequence is unspecified;
+/// only its determinism and its agreement with [`Value`]'s equality matter.
+///
+/// **Iterative** (an explicit work-stack, never native recursion), like the
+/// engine's own [`Value`] `Drop`/`PartialEq`: a value's nesting is
+/// user-controlled and can reach the caller-raised `max_depth`, so a recursive
+/// hasher would reopen the uncatchable native-stack-overflow class those
+/// iterative primitives exist to close — the default budget being safe is
+/// coincidence, not a property. The walk hashes each node before pushing its
+/// children, so two structurally equal values drive identical stack operations
+/// and hash identically; children are pushed so a container's length prefix and
+/// per-variant discriminant keep distinct shapes apart.
+fn hash_value<H: Hasher>(root: &Value, state: &mut H) {
+    let mut stack: Vec<&Value> = vec![root];
+    while let Some(value) = stack.pop() {
+        core::mem::discriminant(value).hash(state);
+        match value {
+            Value::Null => {}
+            Value::Bool(b) => b.hash(state),
+            Value::Number(n) => number_key(n).hash(state),
+            Value::Str(s) => s.hash(state),
+            Value::DateTime(dt) => dt.instant().hash(state),
+            Value::Date(date) => date.ordinal().hash(state),
+            Value::Array(items) | Value::Tuple(items) => {
+                items.len().hash(state);
+                stack.extend(items.iter());
+            }
+            Value::Set(items) | Value::FrozenSet(items) => {
+                items.len().hash(state);
+                stack.extend(items.iter());
+            }
+            Value::Object(map) => {
+                map.len().hash(state);
+                // Keys carry the association and are hashed here in sorted
+                // order; the values are pushed and hashed as they pop. The
+                // order values come back in is deterministic, which is all
+                // equality-consistency needs.
+                for (key, _) in map {
+                    key.hash(state);
+                }
+                stack.extend(map.values());
+            }
+        }
+    }
+}
+
 /// A canonical hash-equivalence key for one JSON value, matching
 /// `DeepHash`'s default semantics for **item matching** under
 /// `ignore_order=True` — deliberately **not** the same
@@ -667,9 +788,11 @@ fn tuple_keyed(items: &[Value], memo: &IgnoreOrderMemo) -> (ItemKey, Option<PyHa
 pub(crate) struct HashedList<'a> {
     /// Distinct keys, in first-occurrence (ascending original index) order
     /// — this is `SetOrdered(full_t{1,2}_hashtable.keys())`'s own iteration
-    /// order (a Python dict's insertion order).
-    pub(crate) distinct_order: Vec<ItemKey>,
-    info: HashMap<ItemKey, (usize, &'a Value)>,
+    /// order (a Python dict's insertion order). Each key sits behind an [`Rc`]
+    /// shared with [`Self::info`] and the distance-memo cache keys (see
+    /// [`super::memo::DistanceKey`]).
+    pub(crate) distinct_order: Vec<Rc<ItemKey>>,
+    info: HashMap<Rc<ItemKey>, (usize, &'a Value)>,
 }
 
 impl<'a> HashedList<'a> {
@@ -680,12 +803,12 @@ impl<'a> HashedList<'a> {
     /// builds its two hashtables against one shared `hashes` dict.
     pub(crate) fn build(items: &'a [Value], memo: &IgnoreOrderMemo) -> Self {
         let mut distinct_order = Vec::new();
-        let mut info: HashMap<ItemKey, (usize, &'a Value)> = HashMap::default();
+        let mut info: HashMap<Rc<ItemKey>, (usize, &'a Value)> = HashMap::default();
 
         for (idx, item) in items.iter().enumerate() {
-            let key = item_key(item, memo);
+            let key = Rc::new(item_key(item, memo));
 
-            if let std::collections::hash_map::Entry::Vacant(entry) = info.entry(key.clone()) {
+            if let std::collections::hash_map::Entry::Vacant(entry) = info.entry(Rc::clone(&key)) {
                 distinct_order.push(key);
                 entry.insert((idx, item));
             }

{deepdiff_rs-0.4.0 → deepdiff_rs-0.5.0}/crates/onix-core/src/ignore_order/memo.rs

@@ -40,12 +40,14 @@
 //!   `max_depth - depth - 1` — the trial always completes. So the structural
 //!   result depends only on content too.
 //!
-//! Since [`ItemKey`] is an *exact* structural identity (the full recursive
-//! key, not a lossy digest — and signed zeros are normalized the same way
-//! `numeric_distance` treats them), the `(removed, added)` `ItemKey` pair
-//! keys `rough_distance` losslessly. Caching by it is therefore
-//! observationally identical to recomputing — verified empirically by the
-//! with/without differential test in `super::tests`.
+//! Caching is therefore decision-neutral **as long as the key distinguishes
+//! every value pair whose distance differs** — which is exactly why the cache
+//! is keyed by [`super::hash::DistKey`] (a value's exact structural identity),
+//! not by the order- and repetition-insensitive `ItemKey` that conflated
+//! repetition-differing values into one slot (issue #31); see `DistKey`'s own
+//! doc for that rationale. Verified empirically by the with/without
+//! differential test in `super::tests` (including a repetition-only sibling
+//! divergence).
 //!
 //! # Tuple digests
 //!
@@ -129,11 +131,24 @@ use std::cell::RefCell;
 use std::collections::BTreeMap;
 
 use super::fxhash::HashMap;
-use super::hash::{ItemKey, MemberContent, MemberHashKey, NodeId, PyHashKey, RepId, TupleId};
+use super::hash::{
+    DistKey, ItemKey, MemberContent, MemberHashKey, NodeId, PyHashKey, RepId, TupleId,
+};
+
+/// A `(removed, added)` container-pair distance-cache key — see
+/// [`super::hash::DistKey`] for why each side is a value's exact structural
+/// identity (not its order/repetition-insensitive `ItemKey`). Memory per
+/// entry is a refcount bump: each side's value is interned once per candidate
+/// and the `A * R` entries one pairing records just share those [`Rc`]s. The
+/// *lookup* cost is not constant, though — every probe hashes both keys (a
+/// full walk of each value) and, on a bucket match, compares them structurally
+/// — so per-pair work is proportional to record size (see [`super::hash::DistKey`]).
+type DistanceKey = (DistKey, DistKey);
 
 /// The per-top-level-diff caches described in this module's doc: container-pair
-/// [`rough_distance`] results keyed by the `(removed, added)` [`ItemKey`]
-/// pair, the tuple-digest interning table for list-item matching, and the two
+/// [`rough_distance`] results keyed by the `(removed, added)` [`DistKey`]
+/// pair (each side a value's exact structural identity), the tuple-digest
+/// interning table for list-item matching, and the two
 /// set-member interning tables. Created in `crate::diff::diff_with_options`,
 /// threaded (by shared reference, interior mutability) through the whole
 /// recursive diff, and dropped when it returns. No eviction and no tuning
@@ -141,7 +156,7 @@ use super::hash::{ItemKey, MemberContent, MemberHashKey, NodeId, PyHashKey, RepI
 ///
 /// [`rough_distance`]: super::distance::rough_distance
 pub(crate) struct IgnoreOrderMemo {
-    cache: RefCell<HashMap<(ItemKey, ItemKey), f64>>,
+    cache: RefCell<HashMap<DistanceKey, f64>>,
     /// Interns each distinct hashable-tuple identity to its place in
     /// `tuple_digests`, so a nested tuple can be named by one [`TupleId`]
     /// inside its parent's identity instead of by a copy of its own.
@@ -194,22 +209,33 @@ impl IgnoreOrderMemo {
         }
     }
 
-    /// Whether a candidate pair should be routed through the cache: only when
-    /// enabled *and* both sides are containers. Scalar-involving pairs never
-    /// recurse (so never re-compute), so they skip the cache entirely — no
-    /// key clone, no map operation — which keeps flat `ignore_order` shapes
-    /// (a list of numbers, say) free of any memoization overhead.
-    pub(crate) fn should_cache(&self, removed: &ItemKey, added: &ItemKey) -> bool {
-        self.enabled && is_container(removed) && is_container(added)
+    /// Whether distance memoization is live for this run. A candidate pair is
+    /// additionally only cached when both sides are containers (see
+    /// [`is_container`]): scalar-involving pairs never recurse, so they never
+    /// re-compute and skip the cache entirely, keeping flat `ignore_order`
+    /// shapes (a list of numbers, say) free of any memoization overhead. The
+    /// `disabled()` cache reports `false` here so the with/without differential
+    /// test runs the identical code path with the cache inert.
+    pub(crate) fn caching_enabled(&self) -> bool {
+        self.enabled
+    }
+
+    /// The number of distinct container-pair distances currently memoized.
+    /// Test-only: lets the gate tests assert that scalar pairs are never
+    /// cached and that a `disabled()` memo caches nothing, pinning the two
+    /// conditions `caching_enabled`/`is_container` guard.
+    #[cfg(test)]
+    pub(crate) fn cache_len(&self) -> usize {
+        self.cache.borrow().len()
     }
 
     /// The cached distance for `key`, if present.
-    pub(crate) fn get(&self, key: &(ItemKey, ItemKey)) -> Option<f64> {
+    pub(crate) fn get(&self, key: &DistanceKey) -> Option<f64> {
         self.cache.borrow().get(key).copied()
     }
 
     /// Records `value` for `key` (moving the already-cloned key in).
-    pub(crate) fn put(&self, key: (ItemKey, ItemKey), value: f64) {
+    pub(crate) fn put(&self, key: DistanceKey, value: f64) {
         self.cache.borrow_mut().insert(key, value);
     }
 
@@ -286,7 +312,8 @@ impl IgnoreOrderMemo {
 }
 
 /// Whether `key` is a container (list/tuple/dict) rather than a scalar — the
-/// variants whose distance is computed by a recursive trial diff.
-fn is_container(key: &ItemKey) -> bool {
+/// variants whose distance is computed by a recursive trial diff, and so the
+/// only ones worth memoizing.
+pub(crate) fn is_container(key: &ItemKey) -> bool {
     matches!(key, ItemKey::List(_) | ItemKey::Tuple(_) | ItemKey::Dict(_))
 }