pagetrace 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,316 @@
1
+ Metadata-Version: 2.4
2
+ Name: pagetrace
3
+ Version: 0.1.0
4
+ Summary: Offline checker for API pagination traces. Never claims more than the trace proves.
5
+ Author: PageTrace maintainers
6
+ License-Expression: MIT
7
+ Classifier: Development Status :: 2 - Pre-Alpha
8
+ Classifier: Environment :: Console
9
+ Classifier: Intended Audience :: Developers
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Programming Language :: Python :: 3.11
12
+ Classifier: Programming Language :: Python :: 3.12
13
+ Classifier: Programming Language :: Python :: 3.13
14
+ Classifier: Programming Language :: Python :: 3.14
15
+ Classifier: Topic :: Software Development :: Testing
16
+ Classifier: Typing :: Typed
17
+ Requires-Dist: python-jsonpath>=2.2.1
18
+ Requires-Python: >=3.11
19
+ Project-URL: Homepage, https://github.com/OWNER/pagetrace
20
+ Project-URL: Changelog, https://github.com/OWNER/pagetrace/blob/main/CHANGELOG.md
21
+ Description-Content-Type: text/markdown
22
+
23
+ <div align="center">
24
+
25
+ <img src="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/header.svg" alt="PageTrace" width="720">
26
+
27
+ **English** · [Русский](https://github.com/Corner324/PageTrace/blob/main/README.ru.md) · [中文](https://github.com/Corner324/PageTrace/blob/main/README.zh.md)
28
+
29
+ **Prove what your API pagination actually did. Offline, from a trace you already have.**
30
+
31
+ [![CI](https://github.com/Corner324/PageTrace/actions/workflows/ci.yml/badge.svg)](https://github.com/Corner324/PageTrace/actions/workflows/ci.yml)
32
+ [![Coverage 98.97%](https://img.shields.io/badge/coverage-98.97%25%20branch-brightgreen.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md)
33
+ [![Tests 425](https://img.shields.io/badge/tests-425-brightgreen.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md)
34
+ [![Zero network](https://img.shields.io/badge/network-none-0aa344.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/security.md)
35
+ [![Python 3.11+](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md)
36
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/Corner324/PageTrace/blob/main/LICENSE)
37
+
38
+ [⚡ Install](#-install) · [🔍 How it works](#-how-it-works) · [🎯 Presets](#-presets) · [🧭 What it will not tell you](#-what-it-will-not-tell-you) · [🔒 Security](#-security) · [📐 Invariants](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md)
39
+
40
+ </div>
41
+
42
+ ---
43
+
44
+ ## 💡 Why this exists
45
+
46
+ Pagination fails quietly. Every page returns `200 OK`, the loop terminates, nothing is logged as
47
+ an error, and the export is short by four hundred rows. Nobody notices until someone reconciles
48
+ a total a month later, and by then the capture is gone.
49
+
50
+ The usual tools are the wrong shape for it. An integration test hits the live API and goes flaky.
51
+ Logs show you individual requests, never the relationship between them - and the bug always lives
52
+ in the relationship: page 3 asked for a token page 2 never handed out. Reading a HAR by hand works
53
+ right up to the twelfth page.
54
+
55
+ PageTrace takes the recording you already have and reports what it proves.
56
+
57
+ | You have heard | PageTrace answers, from the trace alone |
58
+ | -------------- | -------------------------------------- |
59
+ | "The export is missing rows" | whether a page was requested with a token the previous page never returned, whether the sequence revisited a state, whether the capture just stopped early |
60
+ | "The job runs forever" | whether the cursor stalled on one value, or the request sequence returned to a state it had already visited |
61
+ | "We get duplicates downstream" | which item identity appeared on which two pages |
62
+ | "Their API is broken" | which side the trace can actually attribute it to, which is often neither |
63
+
64
+ > [!IMPORTANT]
65
+ > **The one rule: never claim more than the trace proves.**
66
+ > Every finding carries a certainty and the exact observations behind it. There are no confidence
67
+ > scores anywhere in this tool. When the evidence is not there, it says
68
+ > `INSUFFICIENT_EVIDENCE` and names the flag you need to set, instead of guessing.
69
+
70
+ Three places it earns its keep: pinned to a regression test in CI, where a proven defect fails the
71
+ build; in your hand during an incident, where a colleague's HAR is the only evidence left; and
72
+ pointed at a vendor before you integrate, where you need the claim to survive being quoted back
73
+ at you.
74
+
75
+ ---
76
+
77
+ ## ⚡ See it in 30 seconds
78
+
79
+ ```bash
80
+ uv run pagetrace check examples/broken-cursor.jsonl
81
+ ```
82
+
83
+ ```
84
+ broken-cursor.jsonl: 3 pages, preset cursor
85
+
86
+ PROVEN ERROR CURSOR_CONTINUATION_MISMATCH [INV-CURSOR-001] pages 1, 2
87
+ Page 2 was requested with a continuation token that page 1 did not return.
88
+ page[1].response.next_cursor = "eyJvIjo0fQ"
89
+ page[2].request.cursor = "eyJvIjozfQ"
90
+
91
+ PROVEN WARNING DUPLICATE_ITEM_ACROSS_PAGES [INV-ITEM-001] pages 1, 2
92
+ An item observed on page 1 was observed again on page 2; its identity is in the evidence below. That the same item appears twice is a fact; whether it means overlapping pages depends on a uniqueness contract this trace does not contain.
93
+ page[1].items[0].id = 3
94
+ page[2].items[0].id = 3
95
+
96
+ 2 proven, 0 suspicious, 0 insufficient evidence
97
+ ```
98
+
99
+ Exit code 1. Note what the second finding refuses to do: it reports the duplicate as a fact and
100
+ declines to call it a pagination bug, because no trace can carry a uniqueness contract. That is
101
+ the whole design in one message.
102
+
103
+ `examples/clean-cursor.jsonl` is the same run without the defects, and exits 0.
104
+
105
+ ---
106
+
107
+ ## 🔍 How it works
108
+
109
+ <div align="center">
110
+ <picture>
111
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/flow.en.dark.svg">
112
+ <img alt="Recorded trace, parser, normalizer, invariant engine, report. No network call at any step." src="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/flow.en.light.svg" width="820">
113
+ </picture>
114
+ </div>
115
+
116
+ The important part is structural, not a promise. A detector receives normalized observations and
117
+ nothing else: `Page` has no field for a raw body or an `authorization` header, so the engine
118
+ physically cannot read one. See [docs/architecture.md](https://github.com/Corner324/PageTrace/blob/main/docs/architecture.md).
119
+
120
+ ---
121
+
122
+ ## ⚡ Install
123
+
124
+ > [!NOTE]
125
+ > Installing from PyPI is not available yet. The release workflow publishes through Trusted
126
+ > Publishing over OIDC with no token fallback, so nothing can be published until a trusted
127
+ > publisher is configured for the project and the `pypi` environment
128
+ > ([docs/compatibility.md](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md)).
129
+
130
+ From a clone, with [uv](https://docs.astral.sh/uv/):
131
+
132
+ ```bash
133
+ uv sync
134
+ uv run pagetrace check examples/broken-cursor.jsonl
135
+ ```
136
+
137
+ With pip into an environment of your own, or `pipx install .` for a standalone command:
138
+
139
+ ```bash
140
+ pip install .
141
+ ```
142
+
143
+ ---
144
+
145
+ ## 📼 Record a trace
146
+
147
+ A JSONL trace is one JSON object per line, one per request/response pair, in capture order:
148
+
149
+ ```json
150
+ {"v": 1, "seq": 0, "request": {"method": "GET", "url": "https://api.example.com/v1/items?limit=2"}, "response": {"status": 200, "headers": {"content-type": "application/json"}, "body": {"items": [{"id": 1, "name": "item-1"}, {"id": 2, "name": "item-2"}], "next_cursor": "eyJvIjoyfQ", "has_more": true, "total": 5}}}
151
+ ```
152
+
153
+ `v` and `response.status` are required; `seq` is optional and line order is authoritative. Full
154
+ rules: [specs/formats/jsonl-v1.md](https://github.com/Corner324/PageTrace/blob/main/specs/formats/jsonl-v1.md).
155
+
156
+ <details>
157
+ <summary><b>Or export a HAR from the browser</b> - no code to write, works against anyone's API</summary>
158
+
159
+ <br>
160
+
161
+ Open DevTools, go to the Network tab, reproduce the pagination, export the request list as HAR.
162
+ A browser capture holds every asset the page loaded, so filter it down to the API calls:
163
+
164
+ ```bash
165
+ uv run pagetrace check your-capture.har --url-filter /v1/items
166
+ ```
167
+
168
+ `OPTIONS` and `HEAD` requests are dropped before pages are numbered, so a CORS preflight never
169
+ looks like a client that failed to advance.
170
+
171
+ </details>
172
+
173
+ > [!WARNING]
174
+ > A HAR holds live credentials. PageTrace keeps exactly one response header and never renders a
175
+ > body, but the file on your disk still contains your `Authorization` header - treat it like a
176
+ > password, and do not attach it to a public issue.
177
+
178
+ ---
179
+
180
+ ## 🎯 Presets
181
+
182
+ Selectors are [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html) JSONPath. A preset default
183
+ that does not match resolves to "not found" and proves nothing, rather than becoming a guess.
184
+
185
+ | Preset | Default selectors | Fits |
186
+ | ------ | ----------------- | ---- |
187
+ | `cursor` <sub>default</sub> | `--cursor $.query.cursor` `--next $.next_cursor` `--items $.items` `--id $.id` `--total $.total` `--has-more $.has_more` | APIs that hand back an opaque continuation token |
188
+ | `page` | `--page $.query.page`, plus the same items / id / total / has-more defaults | page numbers, 0-based or 1-based |
189
+ | `offset` | `--offset $.query.offset`, plus the same items / id / total / has-more defaults | an offset and a limit |
190
+ | `link` | items / id / total / has-more only; continuation comes from the HTTP `Link` header | `rel="next"` in a Link header (RFC 8288) |
191
+
192
+ > [!TIP]
193
+ > Every default is overridable by flag, and `uv run pagetrace check --help` prints all of them
194
+ > with the preset each belongs to. If a finding says `INSUFFICIENT_EVIDENCE`, the flag to set is
195
+ > named in the message.
196
+
197
+ ---
198
+
199
+ ## 📊 Certainty, severity, exit codes
200
+
201
+ Two independent axes. Certainty is what the evidence supports; severity is how much it matters if
202
+ true. A finding is never a probability.
203
+
204
+ | Certainty | Meaning |
205
+ | --------- | ------- |
206
+ | `PROVEN` | the trace itself shows it, with the preconditions satisfied |
207
+ | `SUSPICIOUS` | consistent with a defect **and** with an innocent explanation |
208
+ | `INSUFFICIENT_EVIDENCE` | the selector or the trace did not supply what the check needs |
209
+
210
+ | Exit | Meaning |
211
+ | ---- | ------- |
212
+ | `0` | nothing both proven and severe |
213
+ | `1` | at least one `PROVEN` finding of severity `ERROR` |
214
+ | `2` | invalid input or configuration |
215
+
216
+ Exit 1 needs both axes. A duplicate item across pages is `PROVEN WARNING` and does not fail a
217
+ build: the finding declines to say whether overlapping windows are a bug, so failing on it would
218
+ be the tool claiming what its own message refuses to. A build that goes red on "this might be a
219
+ cache" trains people to ignore the tool.
220
+
221
+ For CI, take the machine formats rather than the terminal text, which is not a contract:
222
+
223
+ ```bash
224
+ uv run pagetrace check examples/broken-cursor.jsonl --format json --output report.json
225
+ ```
226
+
227
+ ---
228
+
229
+ ## 🧭 What it will not tell you
230
+
231
+ This section is the point of the project, not a disclaimer.
232
+
233
+ - **That pagination never terminates.** A finite trace can prove the sequence revisited a state,
234
+ and that is what `INV-CURSOR-003` says. A cycle back to the page-0 state after a `null` cursor
235
+ drops to `SUSPICIOUS`, because two concatenated runs look exactly the same.
236
+ - **That records are missing.** `INV-TOTAL-001` compares a declared total against observed
237
+ identities and is never worded as "N records are missing": PageTrace does not know that `total`
238
+ counts the same population, that the dataset held still, or that the same filters applied.
239
+ - **That the API is at fault.** `INV-CONT-001` and `INV-TRACE-001` report that the trace ended
240
+ while the server was still offering more, and name neither the API, the client, nor the recorder.
241
+ - **Anything about a duplicate beyond the duplicate itself**, without a uniqueness contract the
242
+ trace does not contain.
243
+ - **Anything from a page it could not read.** Non-2xx responses are excluded from every
244
+ body-derived check and reported once by `INV-TRACE-002`, so a trace nobody could read never
245
+ looks clean.
246
+ - **Anything a selector did not resolve.** Matching nothing, matching twice, or landing on an
247
+ object is `INSUFFICIENT_EVIDENCE` naming the flag, never a guess. JSON `null` is a different
248
+ state: an observation meaning "the server declared no continuation".
249
+ - **That `5` and `"5"` differ** - comparison is on the canonical text form, because a query
250
+ parameter can only carry text - or that a truthy `1` or `"true"` means `has_more`. Only a
251
+ boolean counts.
252
+
253
+ Every claim and every non-claim condition is in [specs/invariants.md](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md), the
254
+ normative contract for the tool. Eleven invariants, each with written preconditions.
255
+
256
+ ---
257
+
258
+ ## 🔒 Security
259
+
260
+ A HAR or JSONL trace is an untrusted, secret-bearing file. Parsers keep exactly one response
261
+ header, `link`, because exactly one is read; `authorization`, `cookie` and everything else is
262
+ dropped before the model is built. Raw bodies never reach a detector or a renderer, so only the
263
+ scalars you selected are displayed, with control characters escaped and lengths truncated. Input
264
+ is bounded per line and per decoded body, exceeding a limit is a usage error rather than a
265
+ traceback, and the package contains no networking dependency or code path at all. Threat table
266
+ and limits: [docs/security.md](https://github.com/Corner324/PageTrace/blob/main/docs/security.md).
267
+
268
+ ---
269
+
270
+ ## 📈 Performance
271
+
272
+ Measured 2026-09-11, one run per shape on a Windows 11 developer machine with CPython 3.12.6;
273
+ memory is peak Python allocation from `tracemalloc`. The shape of the numbers is the finding, not
274
+ the third digit.
275
+
276
+ | Shape | Pages | Items | Seconds | Peak MiB |
277
+ | ----- | ----: | ----: | ------: | -------: |
278
+ | wide | 1 000 | 100 000 | 2.6 | 16.1 |
279
+ | very wide | 1 000 | 1 000 000 | 20.9 | 156.8 |
280
+ | ten thousand pages | 10 000 | 1 000 000 | 28.3 | 142.8 |
281
+ | hundred thousand pages | 100 000 | 5 000 000 | 120.8 | 715.4 |
282
+
283
+ > [!NOTE]
284
+ > Three time targets are missed and stay written down as targets rather than quietly relaxed:
285
+ > 1k pages / 100k ids wants 2 s and takes 2.6 s; 10k / 1M wants 10 s and takes 28.3 s, about 3x
286
+ > over; the 100k / 5M stretch target wants 60 s and takes 120.8 s, about 2x over. Both memory
287
+ > targets are met with room to spare. Full table and reasoning:
288
+ > [docs/performance.md](https://github.com/Corner324/PageTrace/blob/main/docs/performance.md).
289
+
290
+ ---
291
+
292
+ ## 🛠 Contributing
293
+
294
+ ```bash
295
+ uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest
296
+ ```
297
+
298
+ 425 tests pass locally at 98.97% branch coverage, on 3.11, 3.12, 3.13 and 3.14 on Windows. The
299
+ property suite in `tests/property/` is the main correctness layer: a valid generated trace must
300
+ produce no `PROVEN` finding, and each mutation operator must wake exactly one detector. Mutation
301
+ score is 77.9% against the unit tests as of 2026-09-12, with the survivors sorted by hand in
302
+ [docs/testing.md](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md); mutmut refuses to run on Windows, so that run came from a
303
+ Linux container.
304
+
305
+ Before a new detector: an entry in [specs/invariants.md](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md) with its non-claim
306
+ conditions, then the test oracle, then the code. [CONTRIBUTING.md](https://github.com/Corner324/PageTrace/blob/main/CONTRIBUTING.md) has the
307
+ workflow and the definition of done, [AGENTS.md](https://github.com/Corner324/PageTrace/blob/main/AGENTS.md) the same for coding agents, and the
308
+ rest is in [docs/architecture.md](https://github.com/Corner324/PageTrace/blob/main/docs/architecture.md), [docs/testing.md](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md),
309
+ [docs/compatibility.md](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md), [docs/adr/](https://github.com/Corner324/PageTrace/tree/main/docs/adr/) and
310
+ [CHANGELOG.md](https://github.com/Corner324/PageTrace/blob/main/CHANGELOG.md).
311
+
312
+ <div align="center">
313
+
314
+ **MIT** · [LICENSE](https://github.com/Corner324/PageTrace/blob/main/LICENSE)
315
+
316
+ </div>
@@ -0,0 +1,294 @@
1
+ <div align="center">
2
+
3
+ <img src="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/header.svg" alt="PageTrace" width="720">
4
+
5
+ **English** · [Русский](https://github.com/Corner324/PageTrace/blob/main/README.ru.md) · [中文](https://github.com/Corner324/PageTrace/blob/main/README.zh.md)
6
+
7
+ **Prove what your API pagination actually did. Offline, from a trace you already have.**
8
+
9
+ [![CI](https://github.com/Corner324/PageTrace/actions/workflows/ci.yml/badge.svg)](https://github.com/Corner324/PageTrace/actions/workflows/ci.yml)
10
+ [![Coverage 98.97%](https://img.shields.io/badge/coverage-98.97%25%20branch-brightgreen.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md)
11
+ [![Tests 425](https://img.shields.io/badge/tests-425-brightgreen.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md)
12
+ [![Zero network](https://img.shields.io/badge/network-none-0aa344.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/security.md)
13
+ [![Python 3.11+](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md)
14
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/Corner324/PageTrace/blob/main/LICENSE)
15
+
16
+ [⚡ Install](#-install) · [🔍 How it works](#-how-it-works) · [🎯 Presets](#-presets) · [🧭 What it will not tell you](#-what-it-will-not-tell-you) · [🔒 Security](#-security) · [📐 Invariants](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md)
17
+
18
+ </div>
19
+
20
+ ---
21
+
22
+ ## 💡 Why this exists
23
+
24
+ Pagination fails quietly. Every page returns `200 OK`, the loop terminates, nothing is logged as
25
+ an error, and the export is short by four hundred rows. Nobody notices until someone reconciles
26
+ a total a month later, and by then the capture is gone.
27
+
28
+ The usual tools are the wrong shape for it. An integration test hits the live API and goes flaky.
29
+ Logs show you individual requests, never the relationship between them - and the bug always lives
30
+ in the relationship: page 3 asked for a token page 2 never handed out. Reading a HAR by hand works
31
+ right up to the twelfth page.
32
+
33
+ PageTrace takes the recording you already have and reports what it proves.
34
+
35
+ | You have heard | PageTrace answers, from the trace alone |
36
+ | -------------- | -------------------------------------- |
37
+ | "The export is missing rows" | whether a page was requested with a token the previous page never returned, whether the sequence revisited a state, whether the capture just stopped early |
38
+ | "The job runs forever" | whether the cursor stalled on one value, or the request sequence returned to a state it had already visited |
39
+ | "We get duplicates downstream" | which item identity appeared on which two pages |
40
+ | "Their API is broken" | which side the trace can actually attribute it to, which is often neither |
41
+
42
+ > [!IMPORTANT]
43
+ > **The one rule: never claim more than the trace proves.**
44
+ > Every finding carries a certainty and the exact observations behind it. There are no confidence
45
+ > scores anywhere in this tool. When the evidence is not there, it says
46
+ > `INSUFFICIENT_EVIDENCE` and names the flag you need to set, instead of guessing.
47
+
48
+ Three places it earns its keep: pinned to a regression test in CI, where a proven defect fails the
49
+ build; in your hand during an incident, where a colleague's HAR is the only evidence left; and
50
+ pointed at a vendor before you integrate, where you need the claim to survive being quoted back
51
+ at you.
52
+
53
+ ---
54
+
55
+ ## ⚡ See it in 30 seconds
56
+
57
+ ```bash
58
+ uv run pagetrace check examples/broken-cursor.jsonl
59
+ ```
60
+
61
+ ```
62
+ broken-cursor.jsonl: 3 pages, preset cursor
63
+
64
+ PROVEN ERROR CURSOR_CONTINUATION_MISMATCH [INV-CURSOR-001] pages 1, 2
65
+ Page 2 was requested with a continuation token that page 1 did not return.
66
+ page[1].response.next_cursor = "eyJvIjo0fQ"
67
+ page[2].request.cursor = "eyJvIjozfQ"
68
+
69
+ PROVEN WARNING DUPLICATE_ITEM_ACROSS_PAGES [INV-ITEM-001] pages 1, 2
70
+ An item observed on page 1 was observed again on page 2; its identity is in the evidence below. That the same item appears twice is a fact; whether it means overlapping pages depends on a uniqueness contract this trace does not contain.
71
+ page[1].items[0].id = 3
72
+ page[2].items[0].id = 3
73
+
74
+ 2 proven, 0 suspicious, 0 insufficient evidence
75
+ ```
76
+
77
+ Exit code 1. Note what the second finding refuses to do: it reports the duplicate as a fact and
78
+ declines to call it a pagination bug, because no trace can carry a uniqueness contract. That is
79
+ the whole design in one message.
80
+
81
+ `examples/clean-cursor.jsonl` is the same run without the defects, and exits 0.
82
+
83
+ ---
84
+
85
+ ## 🔍 How it works
86
+
87
+ <div align="center">
88
+ <picture>
89
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/flow.en.dark.svg">
90
+ <img alt="Recorded trace, parser, normalizer, invariant engine, report. No network call at any step." src="https://raw.githubusercontent.com/Corner324/PageTrace/main/docs/assets/flow.en.light.svg" width="820">
91
+ </picture>
92
+ </div>
93
+
94
+ The important part is structural, not a promise. A detector receives normalized observations and
95
+ nothing else: `Page` has no field for a raw body or an `authorization` header, so the engine
96
+ physically cannot read one. See [docs/architecture.md](https://github.com/Corner324/PageTrace/blob/main/docs/architecture.md).
97
+
98
+ ---
99
+
100
+ ## ⚡ Install
101
+
102
+ > [!NOTE]
103
+ > Installing from PyPI is not available yet. The release workflow publishes through Trusted
104
+ > Publishing over OIDC with no token fallback, so nothing can be published until a trusted
105
+ > publisher is configured for the project and the `pypi` environment
106
+ > ([docs/compatibility.md](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md)).
107
+
108
+ From a clone, with [uv](https://docs.astral.sh/uv/):
109
+
110
+ ```bash
111
+ uv sync
112
+ uv run pagetrace check examples/broken-cursor.jsonl
113
+ ```
114
+
115
+ With pip into an environment of your own, or `pipx install .` for a standalone command:
116
+
117
+ ```bash
118
+ pip install .
119
+ ```
120
+
121
+ ---
122
+
123
+ ## 📼 Record a trace
124
+
125
+ A JSONL trace is one JSON object per line, one per request/response pair, in capture order:
126
+
127
+ ```json
128
+ {"v": 1, "seq": 0, "request": {"method": "GET", "url": "https://api.example.com/v1/items?limit=2"}, "response": {"status": 200, "headers": {"content-type": "application/json"}, "body": {"items": [{"id": 1, "name": "item-1"}, {"id": 2, "name": "item-2"}], "next_cursor": "eyJvIjoyfQ", "has_more": true, "total": 5}}}
129
+ ```
130
+
131
+ `v` and `response.status` are required; `seq` is optional and line order is authoritative. Full
132
+ rules: [specs/formats/jsonl-v1.md](https://github.com/Corner324/PageTrace/blob/main/specs/formats/jsonl-v1.md).
133
+
134
+ <details>
135
+ <summary><b>Or export a HAR from the browser</b> - no code to write, works against anyone's API</summary>
136
+
137
+ <br>
138
+
139
+ Open DevTools, go to the Network tab, reproduce the pagination, export the request list as HAR.
140
+ A browser capture holds every asset the page loaded, so filter it down to the API calls:
141
+
142
+ ```bash
143
+ uv run pagetrace check your-capture.har --url-filter /v1/items
144
+ ```
145
+
146
+ `OPTIONS` and `HEAD` requests are dropped before pages are numbered, so a CORS preflight never
147
+ looks like a client that failed to advance.
148
+
149
+ </details>
150
+
151
+ > [!WARNING]
152
+ > A HAR holds live credentials. PageTrace keeps exactly one response header and never renders a
153
+ > body, but the file on your disk still contains your `Authorization` header - treat it like a
154
+ > password, and do not attach it to a public issue.
155
+
156
+ ---
157
+
158
+ ## 🎯 Presets
159
+
160
+ Selectors are [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html) JSONPath. A preset default
161
+ that does not match resolves to "not found" and proves nothing, rather than becoming a guess.
162
+
163
+ | Preset | Default selectors | Fits |
164
+ | ------ | ----------------- | ---- |
165
+ | `cursor` <sub>default</sub> | `--cursor $.query.cursor` `--next $.next_cursor` `--items $.items` `--id $.id` `--total $.total` `--has-more $.has_more` | APIs that hand back an opaque continuation token |
166
+ | `page` | `--page $.query.page`, plus the same items / id / total / has-more defaults | page numbers, 0-based or 1-based |
167
+ | `offset` | `--offset $.query.offset`, plus the same items / id / total / has-more defaults | an offset and a limit |
168
+ | `link` | items / id / total / has-more only; continuation comes from the HTTP `Link` header | `rel="next"` in a Link header (RFC 8288) |
169
+
170
+ > [!TIP]
171
+ > Every default is overridable by flag, and `uv run pagetrace check --help` prints all of them
172
+ > with the preset each belongs to. If a finding says `INSUFFICIENT_EVIDENCE`, the flag to set is
173
+ > named in the message.
174
+
175
+ ---
176
+
177
+ ## 📊 Certainty, severity, exit codes
178
+
179
+ Two independent axes. Certainty is what the evidence supports; severity is how much it matters if
180
+ true. A finding is never a probability.
181
+
182
+ | Certainty | Meaning |
183
+ | --------- | ------- |
184
+ | `PROVEN` | the trace itself shows it, with the preconditions satisfied |
185
+ | `SUSPICIOUS` | consistent with a defect **and** with an innocent explanation |
186
+ | `INSUFFICIENT_EVIDENCE` | the selector or the trace did not supply what the check needs |
187
+
188
+ | Exit | Meaning |
189
+ | ---- | ------- |
190
+ | `0` | nothing both proven and severe |
191
+ | `1` | at least one `PROVEN` finding of severity `ERROR` |
192
+ | `2` | invalid input or configuration |
193
+
194
+ Exit 1 needs both axes. A duplicate item across pages is `PROVEN WARNING` and does not fail a
195
+ build: the finding declines to say whether overlapping windows are a bug, so failing on it would
196
+ be the tool claiming what its own message refuses to. A build that goes red on "this might be a
197
+ cache" trains people to ignore the tool.
198
+
199
+ For CI, take the machine formats rather than the terminal text, which is not a contract:
200
+
201
+ ```bash
202
+ uv run pagetrace check examples/broken-cursor.jsonl --format json --output report.json
203
+ ```
204
+
205
+ ---
206
+
207
+ ## 🧭 What it will not tell you
208
+
209
+ This section is the point of the project, not a disclaimer.
210
+
211
+ - **That pagination never terminates.** A finite trace can prove the sequence revisited a state,
212
+ and that is what `INV-CURSOR-003` says. A cycle back to the page-0 state after a `null` cursor
213
+ drops to `SUSPICIOUS`, because two concatenated runs look exactly the same.
214
+ - **That records are missing.** `INV-TOTAL-001` compares a declared total against observed
215
+ identities and is never worded as "N records are missing": PageTrace does not know that `total`
216
+ counts the same population, that the dataset held still, or that the same filters applied.
217
+ - **That the API is at fault.** `INV-CONT-001` and `INV-TRACE-001` report that the trace ended
218
+ while the server was still offering more, and name neither the API, the client, nor the recorder.
219
+ - **Anything about a duplicate beyond the duplicate itself**, without a uniqueness contract the
220
+ trace does not contain.
221
+ - **Anything from a page it could not read.** Non-2xx responses are excluded from every
222
+ body-derived check and reported once by `INV-TRACE-002`, so a trace nobody could read never
223
+ looks clean.
224
+ - **Anything a selector did not resolve.** Matching nothing, matching twice, or landing on an
225
+ object is `INSUFFICIENT_EVIDENCE` naming the flag, never a guess. JSON `null` is a different
226
+ state: an observation meaning "the server declared no continuation".
227
+ - **That `5` and `"5"` differ** - comparison is on the canonical text form, because a query
228
+ parameter can only carry text - or that a truthy `1` or `"true"` means `has_more`. Only a
229
+ boolean counts.
230
+
231
+ Every claim and every non-claim condition is in [specs/invariants.md](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md), the
232
+ normative contract for the tool. Eleven invariants, each with written preconditions.
233
+
234
+ ---
235
+
236
+ ## 🔒 Security
237
+
238
+ A HAR or JSONL trace is an untrusted, secret-bearing file. Parsers keep exactly one response
239
+ header, `link`, because exactly one is read; `authorization`, `cookie` and everything else is
240
+ dropped before the model is built. Raw bodies never reach a detector or a renderer, so only the
241
+ scalars you selected are displayed, with control characters escaped and lengths truncated. Input
242
+ is bounded per line and per decoded body, exceeding a limit is a usage error rather than a
243
+ traceback, and the package contains no networking dependency or code path at all. Threat table
244
+ and limits: [docs/security.md](https://github.com/Corner324/PageTrace/blob/main/docs/security.md).
245
+
246
+ ---
247
+
248
+ ## 📈 Performance
249
+
250
+ Measured 2026-09-11, one run per shape on a Windows 11 developer machine with CPython 3.12.6;
251
+ memory is peak Python allocation from `tracemalloc`. The shape of the numbers is the finding, not
252
+ the third digit.
253
+
254
+ | Shape | Pages | Items | Seconds | Peak MiB |
255
+ | ----- | ----: | ----: | ------: | -------: |
256
+ | wide | 1 000 | 100 000 | 2.6 | 16.1 |
257
+ | very wide | 1 000 | 1 000 000 | 20.9 | 156.8 |
258
+ | ten thousand pages | 10 000 | 1 000 000 | 28.3 | 142.8 |
259
+ | hundred thousand pages | 100 000 | 5 000 000 | 120.8 | 715.4 |
260
+
261
+ > [!NOTE]
262
+ > Three time targets are missed and stay written down as targets rather than quietly relaxed:
263
+ > 1k pages / 100k ids wants 2 s and takes 2.6 s; 10k / 1M wants 10 s and takes 28.3 s, about 3x
264
+ > over; the 100k / 5M stretch target wants 60 s and takes 120.8 s, about 2x over. Both memory
265
+ > targets are met with room to spare. Full table and reasoning:
266
+ > [docs/performance.md](https://github.com/Corner324/PageTrace/blob/main/docs/performance.md).
267
+
268
+ ---
269
+
270
+ ## 🛠 Contributing
271
+
272
+ ```bash
273
+ uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest
274
+ ```
275
+
276
+ 425 tests pass locally at 98.97% branch coverage, on 3.11, 3.12, 3.13 and 3.14 on Windows. The
277
+ property suite in `tests/property/` is the main correctness layer: a valid generated trace must
278
+ produce no `PROVEN` finding, and each mutation operator must wake exactly one detector. Mutation
279
+ score is 77.9% against the unit tests as of 2026-09-12, with the survivors sorted by hand in
280
+ [docs/testing.md](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md); mutmut refuses to run on Windows, so that run came from a
281
+ Linux container.
282
+
283
+ Before a new detector: an entry in [specs/invariants.md](https://github.com/Corner324/PageTrace/blob/main/specs/invariants.md) with its non-claim
284
+ conditions, then the test oracle, then the code. [CONTRIBUTING.md](https://github.com/Corner324/PageTrace/blob/main/CONTRIBUTING.md) has the
285
+ workflow and the definition of done, [AGENTS.md](https://github.com/Corner324/PageTrace/blob/main/AGENTS.md) the same for coding agents, and the
286
+ rest is in [docs/architecture.md](https://github.com/Corner324/PageTrace/blob/main/docs/architecture.md), [docs/testing.md](https://github.com/Corner324/PageTrace/blob/main/docs/testing.md),
287
+ [docs/compatibility.md](https://github.com/Corner324/PageTrace/blob/main/docs/compatibility.md), [docs/adr/](https://github.com/Corner324/PageTrace/tree/main/docs/adr/) and
288
+ [CHANGELOG.md](https://github.com/Corner324/PageTrace/blob/main/CHANGELOG.md).
289
+
290
+ <div align="center">
291
+
292
+ **MIT** · [LICENSE](https://github.com/Corner324/PageTrace/blob/main/LICENSE)
293
+
294
+ </div>