jsonl-diff 0.2.1__tar.gz → 0.2.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (17) hide show
  1. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/PKG-INFO +59 -43
  2. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/README.md +53 -38
  3. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/PKG-INFO +59 -43
  4. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/SOURCES.txt +1 -0
  5. jsonl_diff-0.2.2/jsonl_diff.egg-info/requires.txt +4 -0
  6. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.py +139 -73
  7. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/pyproject.toml +9 -5
  8. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/tests/test_cli.py +36 -0
  9. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/tests/test_diff.py +64 -0
  10. jsonl_diff-0.2.2/tests/test_result.py +413 -0
  11. jsonl_diff-0.2.1/jsonl_diff.egg-info/requires.txt +0 -4
  12. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/LICENSE +0 -0
  13. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/dependency_links.txt +0 -0
  14. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/entry_points.txt +0 -0
  15. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/top_level.txt +0 -0
  16. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/setup.cfg +0 -0
  17. {jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/tests/test_sources.py +0 -0
{jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/PKG-INFO RENAMED
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: jsonl-diff
3
- Version: 0.2.1
3
+ Version: 0.2.2
4
4
  Summary: A lightweight tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory.
5
5
  Author-email: rmoralespp <rmoralespp@gmail.com>
6
6
  License: MIT License
@@ -38,6 +38,7 @@ Classifier: Programming Language :: Python :: 3.11
38
38
  Classifier: Programming Language :: Python :: 3.12
39
39
  Classifier: Programming Language :: Python :: 3.13
40
40
  Classifier: Programming Language :: Python :: 3.14
41
+ Classifier: Programming Language :: Python :: 3.15
41
42
  Classifier: Intended Audience :: Developers
42
43
  Classifier: Intended Audience :: Information Technology
43
44
  Classifier: Topic :: Software Development :: Libraries :: Python Modules
@@ -47,18 +48,16 @@ Classifier: Topic :: File Formats :: JSON
47
48
  Requires-Python: >=3.10
48
49
  Description-Content-Type: text/markdown
49
50
  License-File: LICENSE
50
- Requires-Dist: ijson>=3.2
51
- Requires-Dist: py-jsonl==1.5.3
52
- Requires-Dist: jmespath>=1.0.0
53
- Requires-Dist: msgspec>=0.21.1
51
+ Requires-Dist: ijson>=3.6.0
52
+ Requires-Dist: py-jsonl==1.5.4
53
+ Requires-Dist: jmespath>=1.1.0
54
+ Requires-Dist: msgspec>=0.22.0
54
55
  Dynamic: license-file
55
56
 
56
57
  <div align="center">
57
58
 
58
59
  # jsonl-diff
59
60
 
60
- ### A lightweight Python CLI and library for identity-aware JSON diffs
61
-
62
61
  Compare large JSONL, NDJSON, and top-level JSON arrays by record identity—not
63
62
  by line position—without loading either complete dataset into memory. Strict,
64
63
  deterministic, and disk-backed.
@@ -67,7 +66,7 @@ deterministic, and disk-backed.
67
66
  [![Python](https://img.shields.io/pypi/pyversions/jsonl-diff?style=flat-square)](https://pypi.org/project/jsonl-diff/)
68
67
  [![CI](https://img.shields.io/github/actions/workflow/status/rmoralespp/jsonl-diff/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/rmoralespp/jsonl-diff/actions/workflows/ci.yml)
69
68
  [![Coverage](https://img.shields.io/codecov/c/github/rmoralespp/jsonl-diff?style=flat-square)](https://codecov.io/gh/rmoralespp/jsonl-diff)
70
- [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](LICENSE)
69
+ [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE)
71
70
 
72
71
  [Install](#install) · [Quick start](#quick-start) · [CLI](#cli) · [Python API](#python-api) · [Semantics](#comparison-semantics) · [Documentation](#documentation)
73
72
 
@@ -79,11 +78,6 @@ deterministic, and disk-backed.
79
78
  fields. Record order does not matter. It reports what is equal, added, deleted,
80
79
  modified, or duplicated while preserving the original OLD and NEW locations.
81
80
 
82
- ```text
83
- identity
84
- OLD ── parse ── index ───┬─── compare ── summary / changes
85
- NEW ── parse ── index ───┘
86
- ```
87
81
 
88
82
  ## Why jsonl-diff
89
83
 
@@ -171,7 +165,7 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
171
165
  | `--where EXPRESSION` | JMESPath filter evaluated against each record |
172
166
  | `--duplicates POLICY` | Reject duplicates or retain the `first`/`last` occurrence |
173
167
  | `--missing-key POLICY` | Reject absent identity fields or treat them as `null` |
174
- | `--details FILE` | Write deterministic machine-readable JSONL changes |
168
+ | `--details FILE` | Write the JSONL change report; must not be the OLD/NEW file |
175
169
  | `--schema-diff` | Compare observed fields, types, nullability, and requiredness |
176
170
  | `--schema-ignore POINTER` | Exclude a pointer from observed-schema profiling |
177
171
  | `--format FORMAT` | Read `jsonl` (default) or one top-level JSON array with `json` |
@@ -239,7 +233,27 @@ Using the files from the quick start, `changes.jsonl` contains:
239
233
 
240
234
  The stream starts with comparison metadata, emits only changed identities in
241
235
  deterministic order, and finishes with the complete summary. See the
242
- [details format](docs/details-format.md) for duplicate and schema events.
236
+ [details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) for duplicate and schema events.
237
+
238
+ **Get every report from a single indexing pass**
239
+
240
+ Indexing dominates the cost of a comparison, and each invocation indexes both
241
+ inputs again. Instead of running separate summary, schema, and details
242
+ commands, request them together and query the details file afterwards:
243
+
244
+ ```bash
245
+ jsonl-diff old.jsonl new.jsonl \
246
+ --key id \
247
+ --schema-diff \
248
+ --details report.jsonl
249
+
250
+ # Later queries read report.jsonl instead of re-indexing the inputs.
251
+ jq -c 'select(.type == "schema_change")' report.jsonl
252
+ jq -c 'select(.type == "change" and .op == "modified")' report.jsonl
253
+ jq -c 'select(.type == "summary")' report.jsonl
254
+ ```
255
+
256
+ See [Details format → Querying a report](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md#querying-a-report).
243
257
 
244
258
  **Read compressed inputs**
245
259
 
@@ -255,12 +269,12 @@ jsonl-diff old.json new.json --key id --format json
255
269
 
256
270
  ### Exit codes
257
271
 
258
- | Code | Meaning |
259
- |-----:|-------------------------------------------------------------|
260
- | `0` | No record, duplicate, or requested schema differences |
261
- | `1` | Record differences, tolerated duplicates, or schema changes |
262
- | `2` | Input, resource, output, or runtime failure |
263
- | `3` | Invalid configuration or CLI usage |
272
+ | Code | Meaning |
273
+ |-----:|------------------------------------------------------------------|
274
+ | `0` | No record, duplicate, or requested schema differences |
275
+ | `1` | Record differences, tolerated duplicates, or schema changes |
276
+ | `2` | Input, resource, output, or runtime failure |
277
+ | `3` | Invalid configuration or CLI usage (e.g. `--details` is OLD/NEW) |
264
278
 
265
279
  ## Python API
266
280
 
@@ -296,7 +310,7 @@ and schema changes are then iterated lazily instead of being materialized in
296
310
  memory.
297
311
 
298
312
  Applications must protect the entry point because indexing uses worker
299
- processes. See [Parallel indexing](docs/parallel-indexing.md#application-entry-points)
313
+ processes. See [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md#application-entry-points)
300
314
  for the multiprocessing requirements.
301
315
 
302
316
  The immutable result models include:
@@ -313,7 +327,7 @@ The public exception hierarchy is rooted at `JsonlDiffError`:
313
327
  - `DuplicateKeyError`
314
328
  - `ResourceError`
315
329
 
316
- See the [Python API reference](docs/python-api.md) for the complete signature,
330
+ See the [Python API reference](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) for the complete signature,
317
331
  models, iterator behavior, numeric key semantics, and error hierarchy.
318
332
 
319
333
  ## Comparison semantics
@@ -337,21 +351,21 @@ The comparison is intentionally strict:
337
351
  `--ignore` excludes fields from content comparison. `--schema-ignore`
338
352
  independently excludes fields from observed-schema profiling.
339
353
 
340
- Read [Comparison semantics](docs/comparison-semantics.md) for duplicate rules,
354
+ Read [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) for duplicate rules,
341
355
  ignore-pointer edge cases, filter order, schema inference, and numeric behavior.
342
356
 
343
357
  ## Sources and compression
344
358
 
345
- | Source | CLI | Python API |
346
- |--------------------|:-----------:|:-----------:|
347
- | Local path | Yes | Yes |
348
- | HTTP/HTTPS URL | Yes | Yes |
349
- | File-like object | No | Yes |
350
- | gzip (`.gz`) | Yes | Yes |
351
- | bzip2 (`.bz2`) | Yes | Yes |
352
- | xz (`.xz`) | Yes | Yes |
353
- | Zstandard (`.zst`) | Python 3.14 | Python 3.14 |
354
- | ZIP archive | No | No |
359
+ | Source | CLI | Python API |
360
+ |--------------------|:------------:|:------------:|
361
+ | Local path | Yes | Yes |
362
+ | HTTP/HTTPS URL | Yes | Yes |
363
+ | File-like object | No | Yes |
364
+ | gzip (`.gz`) | Yes | Yes |
365
+ | bzip2 (`.bz2`) | Yes | Yes |
366
+ | xz (`.xz`) | Yes | Yes |
367
+ | Zstandard (`.zst`) | Python 3.14+ | Python 3.14+ |
368
+ | ZIP archive | No | No |
355
369
 
356
370
  Source opening and decompression are delegated to `py-jsonl`. File-like inputs
357
371
  and stdin are staged in the private temporary workspace so both indexing
@@ -374,20 +388,22 @@ editor. It intentionally does not provide:
374
388
 
375
389
  ## Documentation
376
390
 
377
- | Guide | Covers |
378
- |------------------------------------------------------|--------------------------------------------------------------------|
379
- | [Comparison semantics](docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
380
- | [Disk-backed architecture](docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
381
- | [Parallel indexing](docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
382
- | [Details format](docs/details-format.md) | Machine-readable JSONL event contract |
383
- | [Python API](docs/python-api.md) | Complete signature, models, iterators, and errors |
391
+ | Guide | Covers |
392
+ |---------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
393
+ | [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
394
+ | [Disk-backed architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
395
+ | [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
396
+ | [Details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) | Machine-readable JSONL event contract |
397
+ | [Python API](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) | Complete signature, models, iterators, and errors |
384
398
 
385
399
  ## Development
386
400
 
387
401
  ```bash
388
402
  uv sync --group test --group lint
389
- uv run pytest
403
+ # Lint
390
404
  uv run ruff check --quiet --output-format=concise .
405
+ # Tests
406
+ uv run python -Wd -m pytest tests/ --cov
391
407
  ```
392
408
 
393
409
  The suite covers CLI behavior, identity and canonicalization, validation,
@@ -395,4 +411,4 @@ details output, source handling, compression, parallel indexing, and cleanup.
395
411
 
396
412
  ## License
397
413
 
398
- Licensed under the terms in [LICENSE](LICENSE).
414
+ Licensed under the terms in [LICENSE](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE).
{jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/README.md RENAMED
@@ -2,8 +2,6 @@
2
2
 
3
3
  # jsonl-diff
4
4
 
5
- ### A lightweight Python CLI and library for identity-aware JSON diffs
6
-
7
5
  Compare large JSONL, NDJSON, and top-level JSON arrays by record identity—not
8
6
  by line position—without loading either complete dataset into memory. Strict,
9
7
  deterministic, and disk-backed.
@@ -12,7 +10,7 @@ deterministic, and disk-backed.
12
10
  [![Python](https://img.shields.io/pypi/pyversions/jsonl-diff?style=flat-square)](https://pypi.org/project/jsonl-diff/)
13
11
  [![CI](https://img.shields.io/github/actions/workflow/status/rmoralespp/jsonl-diff/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/rmoralespp/jsonl-diff/actions/workflows/ci.yml)
14
12
  [![Coverage](https://img.shields.io/codecov/c/github/rmoralespp/jsonl-diff?style=flat-square)](https://codecov.io/gh/rmoralespp/jsonl-diff)
15
- [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](LICENSE)
13
+ [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE)
16
14
 
17
15
  [Install](#install) · [Quick start](#quick-start) · [CLI](#cli) · [Python API](#python-api) · [Semantics](#comparison-semantics) · [Documentation](#documentation)
18
16
 
@@ -24,11 +22,6 @@ deterministic, and disk-backed.
24
22
  fields. Record order does not matter. It reports what is equal, added, deleted,
25
23
  modified, or duplicated while preserving the original OLD and NEW locations.
26
24
 
27
- ```text
28
- identity
29
- OLD ── parse ── index ───┬─── compare ── summary / changes
30
- NEW ── parse ── index ───┘
31
- ```
32
25
 
33
26
  ## Why jsonl-diff
34
27
 
@@ -116,7 +109,7 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
116
109
  | `--where EXPRESSION` | JMESPath filter evaluated against each record |
117
110
  | `--duplicates POLICY` | Reject duplicates or retain the `first`/`last` occurrence |
118
111
  | `--missing-key POLICY` | Reject absent identity fields or treat them as `null` |
119
- | `--details FILE` | Write deterministic machine-readable JSONL changes |
112
+ | `--details FILE` | Write the JSONL change report; must not be the OLD/NEW file |
120
113
  | `--schema-diff` | Compare observed fields, types, nullability, and requiredness |
121
114
  | `--schema-ignore POINTER` | Exclude a pointer from observed-schema profiling |
122
115
  | `--format FORMAT` | Read `jsonl` (default) or one top-level JSON array with `json` |
@@ -184,7 +177,27 @@ Using the files from the quick start, `changes.jsonl` contains:
184
177
 
185
178
  The stream starts with comparison metadata, emits only changed identities in
186
179
  deterministic order, and finishes with the complete summary. See the
187
- [details format](docs/details-format.md) for duplicate and schema events.
180
+ [details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) for duplicate and schema events.
181
+
182
+ **Get every report from a single indexing pass**
183
+
184
+ Indexing dominates the cost of a comparison, and each invocation indexes both
185
+ inputs again. Instead of running separate summary, schema, and details
186
+ commands, request them together and query the details file afterwards:
187
+
188
+ ```bash
189
+ jsonl-diff old.jsonl new.jsonl \
190
+ --key id \
191
+ --schema-diff \
192
+ --details report.jsonl
193
+
194
+ # Later queries read report.jsonl instead of re-indexing the inputs.
195
+ jq -c 'select(.type == "schema_change")' report.jsonl
196
+ jq -c 'select(.type == "change" and .op == "modified")' report.jsonl
197
+ jq -c 'select(.type == "summary")' report.jsonl
198
+ ```
199
+
200
+ See [Details format → Querying a report](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md#querying-a-report).
188
201
 
189
202
  **Read compressed inputs**
190
203
 
@@ -200,12 +213,12 @@ jsonl-diff old.json new.json --key id --format json
200
213
 
201
214
  ### Exit codes
202
215
 
203
- | Code | Meaning |
204
- |-----:|-------------------------------------------------------------|
205
- | `0` | No record, duplicate, or requested schema differences |
206
- | `1` | Record differences, tolerated duplicates, or schema changes |
207
- | `2` | Input, resource, output, or runtime failure |
208
- | `3` | Invalid configuration or CLI usage |
216
+ | Code | Meaning |
217
+ |-----:|------------------------------------------------------------------|
218
+ | `0` | No record, duplicate, or requested schema differences |
219
+ | `1` | Record differences, tolerated duplicates, or schema changes |
220
+ | `2` | Input, resource, output, or runtime failure |
221
+ | `3` | Invalid configuration or CLI usage (e.g. `--details` is OLD/NEW) |
209
222
 
210
223
  ## Python API
211
224
 
@@ -241,7 +254,7 @@ and schema changes are then iterated lazily instead of being materialized in
241
254
  memory.
242
255
 
243
256
  Applications must protect the entry point because indexing uses worker
244
- processes. See [Parallel indexing](docs/parallel-indexing.md#application-entry-points)
257
+ processes. See [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md#application-entry-points)
245
258
  for the multiprocessing requirements.
246
259
 
247
260
  The immutable result models include:
@@ -258,7 +271,7 @@ The public exception hierarchy is rooted at `JsonlDiffError`:
258
271
  - `DuplicateKeyError`
259
272
  - `ResourceError`
260
273
 
261
- See the [Python API reference](docs/python-api.md) for the complete signature,
274
+ See the [Python API reference](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) for the complete signature,
262
275
  models, iterator behavior, numeric key semantics, and error hierarchy.
263
276
 
264
277
  ## Comparison semantics
@@ -282,21 +295,21 @@ The comparison is intentionally strict:
282
295
  `--ignore` excludes fields from content comparison. `--schema-ignore`
283
296
  independently excludes fields from observed-schema profiling.
284
297
 
285
- Read [Comparison semantics](docs/comparison-semantics.md) for duplicate rules,
298
+ Read [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) for duplicate rules,
286
299
  ignore-pointer edge cases, filter order, schema inference, and numeric behavior.
287
300
 
288
301
  ## Sources and compression
289
302
 
290
- | Source | CLI | Python API |
291
- |--------------------|:-----------:|:-----------:|
292
- | Local path | Yes | Yes |
293
- | HTTP/HTTPS URL | Yes | Yes |
294
- | File-like object | No | Yes |
295
- | gzip (`.gz`) | Yes | Yes |
296
- | bzip2 (`.bz2`) | Yes | Yes |
297
- | xz (`.xz`) | Yes | Yes |
298
- | Zstandard (`.zst`) | Python 3.14 | Python 3.14 |
299
- | ZIP archive | No | No |
303
+ | Source | CLI | Python API |
304
+ |--------------------|:------------:|:------------:|
305
+ | Local path | Yes | Yes |
306
+ | HTTP/HTTPS URL | Yes | Yes |
307
+ | File-like object | No | Yes |
308
+ | gzip (`.gz`) | Yes | Yes |
309
+ | bzip2 (`.bz2`) | Yes | Yes |
310
+ | xz (`.xz`) | Yes | Yes |
311
+ | Zstandard (`.zst`) | Python 3.14+ | Python 3.14+ |
312
+ | ZIP archive | No | No |
300
313
 
301
314
  Source opening and decompression are delegated to `py-jsonl`. File-like inputs
302
315
  and stdin are staged in the private temporary workspace so both indexing
@@ -319,20 +332,22 @@ editor. It intentionally does not provide:
319
332
 
320
333
  ## Documentation
321
334
 
322
- | Guide | Covers |
323
- |------------------------------------------------------|--------------------------------------------------------------------|
324
- | [Comparison semantics](docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
325
- | [Disk-backed architecture](docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
326
- | [Parallel indexing](docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
327
- | [Details format](docs/details-format.md) | Machine-readable JSONL event contract |
328
- | [Python API](docs/python-api.md) | Complete signature, models, iterators, and errors |
335
+ | Guide | Covers |
336
+ |---------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
337
+ | [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
338
+ | [Disk-backed architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
339
+ | [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
340
+ | [Details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) | Machine-readable JSONL event contract |
341
+ | [Python API](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) | Complete signature, models, iterators, and errors |
329
342
 
330
343
  ## Development
331
344
 
332
345
  ```bash
333
346
  uv sync --group test --group lint
334
- uv run pytest
347
+ # Lint
335
348
  uv run ruff check --quiet --output-format=concise .
349
+ # Tests
350
+ uv run python -Wd -m pytest tests/ --cov
336
351
  ```
337
352
 
338
353
  The suite covers CLI behavior, identity and canonicalization, validation,
@@ -340,4 +355,4 @@ details output, source handling, compression, parallel indexing, and cleanup.
340
355
 
341
356
  ## License
342
357
 
343
- Licensed under the terms in [LICENSE](LICENSE).
358
+ Licensed under the terms in [LICENSE](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE).
{jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/PKG-INFO RENAMED
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: jsonl-diff
3
- Version: 0.2.1
3
+ Version: 0.2.2
4
4
  Summary: A lightweight tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory.
5
5
  Author-email: rmoralespp <rmoralespp@gmail.com>
6
6
  License: MIT License
@@ -38,6 +38,7 @@ Classifier: Programming Language :: Python :: 3.11
38
38
  Classifier: Programming Language :: Python :: 3.12
39
39
  Classifier: Programming Language :: Python :: 3.13
40
40
  Classifier: Programming Language :: Python :: 3.14
41
+ Classifier: Programming Language :: Python :: 3.15
41
42
  Classifier: Intended Audience :: Developers
42
43
  Classifier: Intended Audience :: Information Technology
43
44
  Classifier: Topic :: Software Development :: Libraries :: Python Modules
@@ -47,18 +48,16 @@ Classifier: Topic :: File Formats :: JSON
47
48
  Requires-Python: >=3.10
48
49
  Description-Content-Type: text/markdown
49
50
  License-File: LICENSE
50
- Requires-Dist: ijson>=3.2
51
- Requires-Dist: py-jsonl==1.5.3
52
- Requires-Dist: jmespath>=1.0.0
53
- Requires-Dist: msgspec>=0.21.1
51
+ Requires-Dist: ijson>=3.6.0
52
+ Requires-Dist: py-jsonl==1.5.4
53
+ Requires-Dist: jmespath>=1.1.0
54
+ Requires-Dist: msgspec>=0.22.0
54
55
  Dynamic: license-file
55
56
 
56
57
  <div align="center">
57
58
 
58
59
  # jsonl-diff
59
60
 
60
- ### A lightweight Python CLI and library for identity-aware JSON diffs
61
-
62
61
  Compare large JSONL, NDJSON, and top-level JSON arrays by record identity—not
63
62
  by line position—without loading either complete dataset into memory. Strict,
64
63
  deterministic, and disk-backed.
@@ -67,7 +66,7 @@ deterministic, and disk-backed.
67
66
  [![Python](https://img.shields.io/pypi/pyversions/jsonl-diff?style=flat-square)](https://pypi.org/project/jsonl-diff/)
68
67
  [![CI](https://img.shields.io/github/actions/workflow/status/rmoralespp/jsonl-diff/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/rmoralespp/jsonl-diff/actions/workflows/ci.yml)
69
68
  [![Coverage](https://img.shields.io/codecov/c/github/rmoralespp/jsonl-diff?style=flat-square)](https://codecov.io/gh/rmoralespp/jsonl-diff)
70
- [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](LICENSE)
69
+ [![License](https://img.shields.io/github/license/rmoralespp/jsonl-diff?style=flat-square)](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE)
71
70
 
72
71
  [Install](#install) · [Quick start](#quick-start) · [CLI](#cli) · [Python API](#python-api) · [Semantics](#comparison-semantics) · [Documentation](#documentation)
73
72
 
@@ -79,11 +78,6 @@ deterministic, and disk-backed.
79
78
  fields. Record order does not matter. It reports what is equal, added, deleted,
80
79
  modified, or duplicated while preserving the original OLD and NEW locations.
81
80
 
82
- ```text
83
- identity
84
- OLD ── parse ── index ───┬─── compare ── summary / changes
85
- NEW ── parse ── index ───┘
86
- ```
87
81
 
88
82
  ## Why jsonl-diff
89
83
 
@@ -171,7 +165,7 @@ jsonl-diff [-h] --key KEY [--ignore IGNORE] [--where EXPRESSION]
171
165
  | `--where EXPRESSION` | JMESPath filter evaluated against each record |
172
166
  | `--duplicates POLICY` | Reject duplicates or retain the `first`/`last` occurrence |
173
167
  | `--missing-key POLICY` | Reject absent identity fields or treat them as `null` |
174
- | `--details FILE` | Write deterministic machine-readable JSONL changes |
168
+ | `--details FILE` | Write the JSONL change report; must not be the OLD/NEW file |
175
169
  | `--schema-diff` | Compare observed fields, types, nullability, and requiredness |
176
170
  | `--schema-ignore POINTER` | Exclude a pointer from observed-schema profiling |
177
171
  | `--format FORMAT` | Read `jsonl` (default) or one top-level JSON array with `json` |
@@ -239,7 +233,27 @@ Using the files from the quick start, `changes.jsonl` contains:
239
233
 
240
234
  The stream starts with comparison metadata, emits only changed identities in
241
235
  deterministic order, and finishes with the complete summary. See the
242
- [details format](docs/details-format.md) for duplicate and schema events.
236
+ [details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) for duplicate and schema events.
237
+
238
+ **Get every report from a single indexing pass**
239
+
240
+ Indexing dominates the cost of a comparison, and each invocation indexes both
241
+ inputs again. Instead of running separate summary, schema, and details
242
+ commands, request them together and query the details file afterwards:
243
+
244
+ ```bash
245
+ jsonl-diff old.jsonl new.jsonl \
246
+ --key id \
247
+ --schema-diff \
248
+ --details report.jsonl
249
+
250
+ # Later queries read report.jsonl instead of re-indexing the inputs.
251
+ jq -c 'select(.type == "schema_change")' report.jsonl
252
+ jq -c 'select(.type == "change" and .op == "modified")' report.jsonl
253
+ jq -c 'select(.type == "summary")' report.jsonl
254
+ ```
255
+
256
+ See [Details format → Querying a report](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md#querying-a-report).
243
257
 
244
258
  **Read compressed inputs**
245
259
 
@@ -255,12 +269,12 @@ jsonl-diff old.json new.json --key id --format json
255
269
 
256
270
  ### Exit codes
257
271
 
258
- | Code | Meaning |
259
- |-----:|-------------------------------------------------------------|
260
- | `0` | No record, duplicate, or requested schema differences |
261
- | `1` | Record differences, tolerated duplicates, or schema changes |
262
- | `2` | Input, resource, output, or runtime failure |
263
- | `3` | Invalid configuration or CLI usage |
272
+ | Code | Meaning |
273
+ |-----:|------------------------------------------------------------------|
274
+ | `0` | No record, duplicate, or requested schema differences |
275
+ | `1` | Record differences, tolerated duplicates, or schema changes |
276
+ | `2` | Input, resource, output, or runtime failure |
277
+ | `3` | Invalid configuration or CLI usage (e.g. `--details` is OLD/NEW) |
264
278
 
265
279
  ## Python API
266
280
 
@@ -296,7 +310,7 @@ and schema changes are then iterated lazily instead of being materialized in
296
310
  memory.
297
311
 
298
312
  Applications must protect the entry point because indexing uses worker
299
- processes. See [Parallel indexing](docs/parallel-indexing.md#application-entry-points)
313
+ processes. See [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md#application-entry-points)
300
314
  for the multiprocessing requirements.
301
315
 
302
316
  The immutable result models include:
@@ -313,7 +327,7 @@ The public exception hierarchy is rooted at `JsonlDiffError`:
313
327
  - `DuplicateKeyError`
314
328
  - `ResourceError`
315
329
 
316
- See the [Python API reference](docs/python-api.md) for the complete signature,
330
+ See the [Python API reference](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) for the complete signature,
317
331
  models, iterator behavior, numeric key semantics, and error hierarchy.
318
332
 
319
333
  ## Comparison semantics
@@ -337,21 +351,21 @@ The comparison is intentionally strict:
337
351
  `--ignore` excludes fields from content comparison. `--schema-ignore`
338
352
  independently excludes fields from observed-schema profiling.
339
353
 
340
- Read [Comparison semantics](docs/comparison-semantics.md) for duplicate rules,
354
+ Read [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) for duplicate rules,
341
355
  ignore-pointer edge cases, filter order, schema inference, and numeric behavior.
342
356
 
343
357
  ## Sources and compression
344
358
 
345
- | Source | CLI | Python API |
346
- |--------------------|:-----------:|:-----------:|
347
- | Local path | Yes | Yes |
348
- | HTTP/HTTPS URL | Yes | Yes |
349
- | File-like object | No | Yes |
350
- | gzip (`.gz`) | Yes | Yes |
351
- | bzip2 (`.bz2`) | Yes | Yes |
352
- | xz (`.xz`) | Yes | Yes |
353
- | Zstandard (`.zst`) | Python 3.14 | Python 3.14 |
354
- | ZIP archive | No | No |
359
+ | Source | CLI | Python API |
360
+ |--------------------|:------------:|:------------:|
361
+ | Local path | Yes | Yes |
362
+ | HTTP/HTTPS URL | Yes | Yes |
363
+ | File-like object | No | Yes |
364
+ | gzip (`.gz`) | Yes | Yes |
365
+ | bzip2 (`.bz2`) | Yes | Yes |
366
+ | xz (`.xz`) | Yes | Yes |
367
+ | Zstandard (`.zst`) | Python 3.14+ | Python 3.14+ |
368
+ | ZIP archive | No | No |
355
369
 
356
370
  Source opening and decompression are delegated to `py-jsonl`. File-like inputs
357
371
  and stdin are staged in the private temporary workspace so both indexing
@@ -374,20 +388,22 @@ editor. It intentionally does not provide:
374
388
 
375
389
  ## Documentation
376
390
 
377
- | Guide | Covers |
378
- |------------------------------------------------------|--------------------------------------------------------------------|
379
- | [Comparison semantics](docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
380
- | [Disk-backed architecture](docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
381
- | [Parallel indexing](docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
382
- | [Details format](docs/details-format.md) | Machine-readable JSONL event contract |
383
- | [Python API](docs/python-api.md) | Complete signature, models, iterators, and errors |
391
+ | Guide | Covers |
392
+ |---------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
393
+ | [Comparison semantics](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/comparison-semantics.md) | Identity, duplicates, ignores, filtering, numbers, and determinism |
394
+ | [Disk-backed architecture](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/architecture.md) | SQLite indexes, temporary storage, cleanup, and source handling |
395
+ | [Parallel indexing](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/parallel-indexing.md) | Worker lifecycle, database merge, recovery, and entry points |
396
+ | [Details format](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/details-format.md) | Machine-readable JSONL event contract |
397
+ | [Python API](https://github.com/rmoralespp/jsonl-diff/blob/main/docs/python-api.md) | Complete signature, models, iterators, and errors |
384
398
 
385
399
  ## Development
386
400
 
387
401
  ```bash
388
402
  uv sync --group test --group lint
389
- uv run pytest
403
+ # Lint
390
404
  uv run ruff check --quiet --output-format=concise .
405
+ # Tests
406
+ uv run python -Wd -m pytest tests/ --cov
391
407
  ```
392
408
 
393
409
  The suite covers CLI behavior, identity and canonicalization, validation,
@@ -395,4 +411,4 @@ details output, source handling, compression, parallel indexing, and cleanup.
395
411
 
396
412
  ## License
397
413
 
398
- Licensed under the terms in [LICENSE](LICENSE).
414
+ Licensed under the terms in [LICENSE](https://github.com/rmoralespp/jsonl-diff/blob/main/LICENSE).
{jsonl_diff-0.2.1 → jsonl_diff-0.2.2}/jsonl_diff.egg-info/SOURCES.txt RENAMED
@@ -10,4 +10,5 @@ jsonl_diff.egg-info/requires.txt
10
10
  jsonl_diff.egg-info/top_level.txt
11
11
  tests/test_cli.py
12
12
  tests/test_diff.py
13
+ tests/test_result.py
13
14
  tests/test_sources.py
jsonl_diff-0.2.2/jsonl_diff.egg-info/requires.txt ADDED
@@ -0,0 +1,4 @@
1
+ ijson>=3.6.0
2
+ py-jsonl==1.5.4
3
+ jmespath>=1.1.0
4
+ msgspec>=0.22.0