jsonl-diff 0.1.4__tar.gz → 0.1.5__tar.gz

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: jsonl-diff
-Version: 0.1.4
+Version: 0.1.5
 Summary: A lightweight, pure-Python tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory.
 Author-email: rmoralespp <rmoralespp@gmail.com>
 License: MIT License
@@ -140,12 +140,11 @@ Record order does not matter.
 * Exact **[RFC 6901](https://www.rfc-editor.org/info/rfc6901/)** JSON Pointer ignores.
 * Optional disk-backed observed-schema diff for fields, types, nullability, and requiredness.
 * Semantic number comparison using `Decimal`.
-* Optional `msgspec` accelerator for ~2x faster large diffs (`pip install "jsonl-diff[speedups]"`).
 * Deterministic summaries and change iteration.
 * Original OLD/NEW physical line numbers.
 * Machine-readable JSONL change log with `--details` ([format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md)).
 * Local, HTTP/HTTPS, file-like, and supported compressed sources.
-* Incremental top-level JSON array parsing with `--format json`.
+* Incremental parsing for JSONL/NDJSON by default, with support for top-level JSON arrays via `--format json`.
 * CLI and Python API using the same comparison engine.
 
 ## CLI
@@ -165,14 +164,14 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
 | `--key KEY`           | Required top-level identity field; repeat or comma-separate for composite keys |
 | `--ignore POINTER`    | RFC 6901 pointer to exclude from content comparison                            |
 | `--where EXPRESSION`  | JMESPath filter applied to each record                                         |
-| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`          |
-| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`        |
+| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`         |
+| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`       |
 | `--details FILE`      | Write deterministic machine-readable JSONL changes                             |
-| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                   |
-| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                      |
+| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                  |
+| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                     |
 | `--quiet`             | Suppress the normal summary                                                    |
 | `--max-temp BYTES`    | Best-effort budget for `jsonl-diff` workspace temporary storage                |
-| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`       |
+| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`          |
 
 Examples:
 
@@ -273,7 +272,9 @@ signature, result models, `Decimal` key semantics, and error hierarchy.
   elements are the records.
 * Blank lines, malformed JSON, `NaN`, and infinities are rejected.
 * Duplicate object property names within a record follow JSON's last-wins semantics: the last occurrence is kept.
-* Identity fields must be top-level scalar values (`null` allowed only as one component of a composite key).
+* Identity fields must be top-level JSON values. Object member order is ignored
+  and array order is significant (`null` allowed only as one component of a
+  composite key).
 * Identity types remain significant: `"1"` ≠ `1`, `true` ≠ `1`.
 * Duplicate identities fail by default; `--duplicates first`/`last` select one
   occurrence, report every discarded occurrence, and return exit code `1`.
@@ -293,9 +294,7 @@ order, and canonical number formatting.
 ## Sources & compression
 
 Supported sources include local paths and HTTP/HTTPS URLs. The Python API
-also accepts file-like objects. JSON arrays use `py-jsonl.open_stream()` for
-the source and `ijson` for incremental parsing, so compression and remote
-source handling are shared with JSONL input.
+also accepts file-like objects.
 
 Supported compression:
 
@@ -305,7 +304,7 @@ Supported compression:
 * Zstandard on Python 3.14
 
 See [architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) for the full source/compression
-support matrix and how sources are delegated to `py-jsonl`.
+support matrix and how sources are managed.
 
 ## Limitations
 

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/README.md

@@ -82,12 +82,11 @@ Record order does not matter.
 * Exact **[RFC 6901](https://www.rfc-editor.org/info/rfc6901/)** JSON Pointer ignores.
 * Optional disk-backed observed-schema diff for fields, types, nullability, and requiredness.
 * Semantic number comparison using `Decimal`.
-* Optional `msgspec` accelerator for ~2x faster large diffs (`pip install "jsonl-diff[speedups]"`).
 * Deterministic summaries and change iteration.
 * Original OLD/NEW physical line numbers.
 * Machine-readable JSONL change log with `--details` ([format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md)).
 * Local, HTTP/HTTPS, file-like, and supported compressed sources.
-* Incremental top-level JSON array parsing with `--format json`.
+* Incremental parsing for JSONL/NDJSON by default, with support for top-level JSON arrays via `--format json`.
 * CLI and Python API using the same comparison engine.
 
 ## CLI
@@ -107,14 +106,14 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
 | `--key KEY`           | Required top-level identity field; repeat or comma-separate for composite keys |
 | `--ignore POINTER`    | RFC 6901 pointer to exclude from content comparison                            |
 | `--where EXPRESSION`  | JMESPath filter applied to each record                                         |
-| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`          |
-| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`        |
+| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`         |
+| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`       |
 | `--details FILE`      | Write deterministic machine-readable JSONL changes                             |
-| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                   |
-| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                      |
+| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                  |
+| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                     |
 | `--quiet`             | Suppress the normal summary                                                    |
 | `--max-temp BYTES`    | Best-effort budget for `jsonl-diff` workspace temporary storage                |
-| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`       |
+| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`          |
 
 Examples:
 
@@ -215,7 +214,9 @@ signature, result models, `Decimal` key semantics, and error hierarchy.
   elements are the records.
 * Blank lines, malformed JSON, `NaN`, and infinities are rejected.
 * Duplicate object property names within a record follow JSON's last-wins semantics: the last occurrence is kept.
-* Identity fields must be top-level scalar values (`null` allowed only as one component of a composite key).
+* Identity fields must be top-level JSON values. Object member order is ignored
+  and array order is significant (`null` allowed only as one component of a
+  composite key).
 * Identity types remain significant: `"1"` ≠ `1`, `true` ≠ `1`.
 * Duplicate identities fail by default; `--duplicates first`/`last` select one
   occurrence, report every discarded occurrence, and return exit code `1`.
@@ -235,9 +236,7 @@ order, and canonical number formatting.
 ## Sources & compression
 
 Supported sources include local paths and HTTP/HTTPS URLs. The Python API
-also accepts file-like objects. JSON arrays use `py-jsonl.open_stream()` for
-the source and `ijson` for incremental parsing, so compression and remote
-source handling are shared with JSONL input.
+also accepts file-like objects.
 
 Supported compression:
 
@@ -247,7 +246,7 @@ Supported compression:
 * Zstandard on Python 3.14
 
 See [architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) for the full source/compression
-support matrix and how sources are delegated to `py-jsonl`.
+support matrix and how sources are managed.
 
 ## Limitations
 

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/jsonl_diff.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: jsonl-diff
-Version: 0.1.4
+Version: 0.1.5
 Summary: A lightweight, pure-Python tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory.
 Author-email: rmoralespp <rmoralespp@gmail.com>
 License: MIT License
@@ -140,12 +140,11 @@ Record order does not matter.
 * Exact **[RFC 6901](https://www.rfc-editor.org/info/rfc6901/)** JSON Pointer ignores.
 * Optional disk-backed observed-schema diff for fields, types, nullability, and requiredness.
 * Semantic number comparison using `Decimal`.
-* Optional `msgspec` accelerator for ~2x faster large diffs (`pip install "jsonl-diff[speedups]"`).
 * Deterministic summaries and change iteration.
 * Original OLD/NEW physical line numbers.
 * Machine-readable JSONL change log with `--details` ([format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md)).
 * Local, HTTP/HTTPS, file-like, and supported compressed sources.
-* Incremental top-level JSON array parsing with `--format json`.
+* Incremental parsing for JSONL/NDJSON by default, with support for top-level JSON arrays via `--format json`.
 * CLI and Python API using the same comparison engine.
 
 ## CLI
@@ -165,14 +164,14 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
 | `--key KEY`           | Required top-level identity field; repeat or comma-separate for composite keys |
 | `--ignore POINTER`    | RFC 6901 pointer to exclude from content comparison                            |
 | `--where EXPRESSION`  | JMESPath filter applied to each record                                         |
-| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`          |
-| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`        |
+| `--duplicates POLICY` | `error` (default), or select and report duplicates with `first`/`last`         |
+| `--missing-key POLICY`| `error` (default), or `null` to treat an absent identity field as `null`       |
 | `--details FILE`      | Write deterministic machine-readable JSONL changes                             |
-| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                   |
-| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                      |
+| `--schema-diff`       | Compare observed fields, types, nullability, and requiredness                  |
+| `--schema-ignore`     | RFC 6901 pointer to exclude from observed-schema profiling                     |
 | `--quiet`             | Suppress the normal summary                                                    |
 | `--max-temp BYTES`    | Best-effort budget for `jsonl-diff` workspace temporary storage                |
-| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`       |
+| `--format FORMAT`     | Input format: `jsonl` (default) or a top-level JSON array with `json`          |
 
 Examples:
 
@@ -273,7 +272,9 @@ signature, result models, `Decimal` key semantics, and error hierarchy.
   elements are the records.
 * Blank lines, malformed JSON, `NaN`, and infinities are rejected.
 * Duplicate object property names within a record follow JSON's last-wins semantics: the last occurrence is kept.
-* Identity fields must be top-level scalar values (`null` allowed only as one component of a composite key).
+* Identity fields must be top-level JSON values. Object member order is ignored
+  and array order is significant (`null` allowed only as one component of a
+  composite key).
 * Identity types remain significant: `"1"` ≠ `1`, `true` ≠ `1`.
 * Duplicate identities fail by default; `--duplicates first`/`last` select one
   occurrence, report every discarded occurrence, and return exit code `1`.
@@ -293,9 +294,7 @@ order, and canonical number formatting.
 ## Sources & compression
 
 Supported sources include local paths and HTTP/HTTPS URLs. The Python API
-also accepts file-like objects. JSON arrays use `py-jsonl.open_stream()` for
-the source and `ijson` for incremental parsing, so compression and remote
-source handling are shared with JSONL input.
+also accepts file-like objects.
 
 Supported compression:
 
@@ -305,7 +304,7 @@ Supported compression:
 * Zstandard on Python 3.14
 
 See [architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) for the full source/compression
-support matrix and how sources are delegated to `py-jsonl`.
+support matrix and how sources are managed.
 
 ## Limitations
 

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/jsonl_diff.py

@@ -26,9 +26,9 @@ try:  # Optional C-accelerated content canonicalization (Python 3.10+).
 except ImportError:
     _msgspec = None
 
-JsonScalar = Union[str, Decimal, bool, None]
+JsonIdentityValue = Union[str, Decimal, bool, int, None, list, dict]
 Number = Union[Decimal, int, float]
-IdentityKey = Tuple[JsonScalar, ...]
+IdentityKey = Tuple[JsonIdentityValue, ...]
 
 
 class _Num(Decimal):
@@ -605,8 +605,8 @@ def _identity(
             # record into one indistinguishable identity.
             if not composite:
                 raise ValueError("identity field {!r} must be a non-null scalar".format(name))
-        elif not isinstance(value, (str, Decimal, bool, int)):
-            raise ValueError("identity field {!r} must be a non-null scalar".format(name))
+        elif not isinstance(value, (str, Decimal, bool, int, list, dict)):
+            raise ValueError("identity field {!r} must be a JSON value".format(name))
         values.append(value)
     return tuple(values)
 

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "jsonl-diff"
-version = "0.1.4"
+version = "0.1.5"
 description = "A lightweight, pure-Python tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory."
 readme = "README.md"
 license = { file = "LICENSE" }

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/tests/test_cli.py

@@ -597,6 +597,26 @@ class TestDetailsOutput:
         # Assert
         assert records[1]["key"] == [12345678901234567890]
 
+    def test_structured_key_is_written_to_details_as_json(
+        self,
+        write_jsonl,
+        tmp_path,
+    ):
+        old = write_jsonl(
+            "old.jsonl",
+            [{"id": {"region": "ES", "customer": 7}, "value": "before"}],
+        )
+        new = write_jsonl(
+            "new.jsonl",
+            [{"id": {"customer": 7, "region": "ES"}, "value": "after"}],
+        )
+        details = tmp_path / "changes.jsonl"
+
+        main([str(old), str(new), "--key", "id", "--details", str(details), "--quiet"])
+        records = [json.loads(line) for line in details.read_text(encoding="utf-8").splitlines()]
+
+        assert records[1]["key"] == [{"customer": 7, "region": "ES"}]
+
     def test_duplicate_events_follow_meta_and_include_policy_outcome(
         self,
         write_jsonl,

{jsonl_diff-0.1.4 → jsonl_diff-0.1.5}/tests/test_diff.py

@@ -131,6 +131,50 @@ class TestDiffSummary:
 
 
 class TestIdentityKeys:
+    @pytest.mark.parametrize(
+        "identity",
+        [
+            {"region": "ES", "customer": 7},
+            ["ES", 7, {"active": True}],
+        ],
+    )
+    def test_structured_identity_values_are_supported(self, write_jsonl, identity):
+        old = write_jsonl("old.jsonl", [{"id": identity, "value": "before"}])
+        new = write_jsonl("new.jsonl", [{"id": identity, "value": "after"}])
+
+        with diff(old, new, key="id") as result:
+            change = next(result.changes())
+
+        assert change.key == (identity,)
+        assert change.operation == ChangeOperation.MODIFIED
+
+    def test_identity_object_member_order_is_ignored(self, write_jsonl):
+        old = write_jsonl("old.jsonl", ['{"id":{"a":1,"b":{"x":2,"y":3}}}'])
+        new = write_jsonl("new.jsonl", ['{"id":{"b":{"y":3,"x":2},"a":1}}'])
+
+        with diff(old, new, key="id") as result:
+            summary = result.summary
+
+        assert summary == Summary(equal=1, added=0, deleted=0, modified=0)
+
+    def test_identity_array_order_is_significant(self, write_jsonl):
+        old = write_jsonl("old.jsonl", ['{"id":[1,2]}'])
+        new = write_jsonl("new.jsonl", ['{"id":[2,1]}'])
+
+        with diff(old, new, key="id") as result:
+            summary = result.summary
+
+        assert summary == Summary(equal=0, added=1, deleted=1, modified=0)
+
+    def test_identity_nested_numbers_use_json_numeric_semantics(self, write_jsonl):
+        old = write_jsonl("old.jsonl", ['{"id":{"version":1,"values":[2.0]}}'])
+        new = write_jsonl("new.jsonl", ['{"id":{"version":1.0,"values":[2e0]}}'])
+
+        with diff(old, new, key="id") as result:
+            summary = result.summary
+
+        assert summary == Summary(equal=1, added=0, deleted=0, modified=0)
+
     def test_composite_key_is_returned_as_typed_tuple(self, write_jsonl):
         # Arrange
         old = write_jsonl("old.jsonl", [{"country": "ES", "customer": 7, "value": "before"}])
@@ -581,8 +625,6 @@ class TestInputValidation:
         [
             ({}, "missing identity field"),
             ({"id": None}, "must be a non-null scalar"),
-            ({"id": []}, "must be a non-null scalar"),
-            ({"id": {}}, "must be a non-null scalar"),
         ],
     )
     def test_invalid_identity_component_is_rejected(self, write_jsonl, record, message):