logiparity 1.1.0 → 1.2.0

package/CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## [1.2.0] - 2026-11-08
+
+### Added
+
+- Calendar-day comparisons with `type: date`, per-source `dateFormat`/field `format`, and a fixed 1969–2068 two-digit-year window.
+- Opt-in REST `absentAsNull: true` and per-field `missing: null`, retaining strict extraction by default.
+- Exact decimal reconciliation for number sums in every report format; additive `exactValues` retains totals beyond combined JavaScript-number precision.
+- Separate formatting and repeated numeric-change patterns; opt-in `whitespaceFold` for string/enum fields.
+- A warning when page/offset pagination has no completeness metadata, including later short pages, and a Socrata recipe.
+
+### Fixed
+
+- Case/whitespace-only differences retain actual candidate spelling and no longer become stale or vocabulary diagnoses. Numeric fields never receive a vocabulary cause.
+- Consistent enum changes name the field and compare timing/definitions in both sources without assigning blame to the candidate.
+- Validation rejects datetime formats without a time and points to `type: date`.
+- Package verification pins optional peers to the recorded tested versions, matching the browser installed by CI and the offline kit.
+- Local checks permit only this version's exact development changelog heading; release preparation and publication still reject that heading.
+
 ## [1.1.0] - 2026-11-03
 
 ### Added

package/README.md

@@ -28,6 +28,14 @@ An endpoint test can pass while another surface shows different records. A contr
 
 One configuration compares records across browser tables, APIs, files and SQL. Scenario runs add invariant checks after disruptions, such as whether every order remains present and has one assignment.
 
+### When you may not need LogiParity
+
+- **A one-off check by someone fluent in SQL.** [DuckDB](https://duckdb.org/docs/) reads CSV, XLSX, JSON and PostgreSQL, and a `FULL OUTER JOIN` on the key finds missing rows and changed values. If you already know which defect you are looking for, a short query is often enough.
+- **Large tables between two databases.** [reladiff](https://github.com/erezsh/reladiff) compares tables by hashing segments, so it does not pull every row to your machine.
+- **Two DataFrames in Python.** [datacompy](https://github.com/capitalone/datacompy) reports row and column differences between pandas or Spark DataFrames.
+
+LogiParity is meant for the cases those tools leave to you: reading paginated REST APIs and browser tables, refusing money and time values it cannot parse unambiguously, grouping thousands of differences into a few likely causes with a next step, explaining a total's difference line by line, checking orders after disruptions, and returning CI-friendly exit codes and reports.
+
 ## Install
 
 Use Node.js 22 or later. CI tests Node 22, 24 and 26.
@@ -49,7 +57,7 @@ npm install --save-dev --save-exact pg@8.23.1 pg-query-stream@4.17.1 pgsql-ast-p
 
 These are optional peers loaded only for UI or SQL configurations. Missing peers fail with exit 2 and the exact install command. External runtime dependencies are exact-pinned ordinary dependencies, so consumer `overrides` can patch them. See [dependencies and sizes](docs/dependencies.md) and the [offline kit](docs/release.md#offline-installation).
 
-Version **1.1.0** adds the installed demo, optional browser parity and independent currency reconciliation.
+Version **1.2.0** adds calendar dates, opt-in absent REST properties, exact number-total breakdowns and more precise cause labels.
 
 ## Quickstart
 
@@ -81,7 +89,7 @@ cd logiparity-demo
 npx logiparity check --config parity-ui.yaml
 ```
 
-The login command records a private synthetic session in `logiparity-demo/auth/state.json`; restart invalidates it. The generated directory ignores auth and reports. Download CSV/XLSX as above before the UI check. With `--bugs`, the UI adds the stale-status cause to the API/export checks. `demo login` uses the generated service marker, including its actual port.
+The login command records a private synthetic session in `logiparity-demo/auth/state.json`; restart invalidates it. The generated directory ignores auth and reports. Download CSV/XLSX as above before the UI check. With `--bugs`, the UI adds the stale-status cause to the API/export checks. Its explanation asks to compare refresh timing in both sources and field definitions; the planted demo fixture establishes the UI lag, while this heuristic alone cannot assign blame in real datasets. `demo login` uses the generated service marker, including its actual port.
 
 For a source checkout:
 
@@ -178,7 +186,7 @@ npx logiparity validate --config parity.yaml
 npx logiparity check --config parity.yaml
 ```
 
-Read the export and API at the same cutoff time: freeze a snapshot or pass the same `as_of` value to both requests if the target supports it. Downloading them one after another does not freeze changing data. Otherwise compare a closed day after updates have stopped. `totalPath` detects capped pages and confirms completion. Money uses decimal strings by default; opt into numeric input only after reviewing precision. The [configuration reference](docs/configuration.md) also covers SQL, browser sources and `validate --allow-missing-env` for CI lint steps without secrets.
+Read the export and API at the same cutoff time: freeze a snapshot or pass the same `as_of` value to both requests if the target supports it. Downloading them one after another does not freeze changing data. Otherwise compare a closed day after updates have stopped. `totalPath` detects capped pages and confirms completion. Money uses decimal strings by default; opt into numeric input only after reviewing precision. Calendar dates use `type: date`; REST null omission can be accepted explicitly with `absentAsNull: true` or per-field `missing: null`. Reads without pagination metadata warn that completeness needs a separate count check. The [Socrata recipe](docs/recipes/socrata.md) covers these API shapes. The [configuration reference](docs/configuration.md) also covers SQL, browser sources and `validate --allow-missing-env` for CI lint steps without secrets.
 
 The [GitHub Actions template](templates/github-actions.yml) installs a pinned CLI, downloads a staging export and retains reports for seven days. Tokens are scoped to the download and comparison steps. Shared staging checks run manually or nightly; enable pull requests only with an isolated target. A [container job](templates/github-actions-container.yml) and a [GitLab job](templates/gitlab-ci.yml) are also available.
 
@@ -208,11 +216,11 @@ The [synthetic freight day](examples/freight/README.md) covers carrier settlemen
 
 ## How it is tested
 
-The automated suite passed 564 [unit](https://github.com/Nutts5796/logiparity/tree/main/tests/unit) and [adapter](https://github.com/Nutts5796/logiparity/tree/main/tests/adapters) tests. The 116 passing [end-to-end tests](https://github.com/Nutts5796/logiparity/tree/main/tests/e2e) exercise real Chromium, disposable PostgreSQL and synthetic HTTP services. Coverage includes core, adapters, scenarios and CLI; packages/ai is tested but excluded from this coverage figure. Unit/adapter line coverage is 91.84% (3,233/3,520), above the 80% gate: core 97.22%, adapters 90.93%, scenarios 82.29% and CLI 60.59%. Scenario runner.ts and CLI subprocess behavior are exercised by e2e and installed-package checks and do not increase unit-process coverage.
+The automated suite passed 593 [unit](https://github.com/Nutts5796/logiparity/tree/main/tests/unit) and [adapter](https://github.com/Nutts5796/logiparity/tree/main/tests/adapters) tests. The 117 passing [end-to-end tests](https://github.com/Nutts5796/logiparity/tree/main/tests/e2e) exercise real Chromium, disposable PostgreSQL and synthetic HTTP services. Coverage includes core, adapters, scenarios and CLI; packages/ai is tested but excluded from this coverage figure. Unit/adapter line coverage is 91.94% (3,321/3,612), above the 80% gate: core 97.14%, adapters 91.01%, scenarios 82.29% and CLI 60.59%. Scenario runner.ts and CLI subprocess behavior are exercised by e2e and installed-package checks and do not increase unit-process coverage.
 
-Installed-package checks cover both optional-peer profiles, public imports and declarations, installed demo/scenario commands and empty-cache offline installation. Six clean consumer profiles on npm 10.9.9 and 11.6.0 passed plain npm install, npm ci, npm audit and npm audit signatures, including coexistence with Playwright 1.63.0 and pg 8.16.0. The clean-source quickstart executed the README commands, including plain npm ci, and checked all report formats.
+Installed-package checks cover both optional-peer profiles, public imports and declarations, installed demo/scenario commands and empty-cache offline installation. The 1.2.0 package check uses npm 11.6.0 for plain npm install, npm ci, npm audit and npm audit signatures in default and optional-peer consumer profiles. The full profile uses the exact recorded tested peers (Playwright 1.64.1), matching the lockfile browser and offline kit; public peer ranges remain unchanged. Historical 1.1.0 verification also covered six npm 10/11 profiles and coexistence with older compatible peers. The clean-source quickstart executed the README commands, including plain npm ci, and checked all report formats.
 
-The seeded property-based test [filter-contributions.test.ts](https://github.com/Nutts5796/logiparity/blob/main/tests/unit/filter-contributions.test.ts) checks reconciliation while amounts and filter fields change independently. Regression cases include one-off differences that never become patterns, DST gaps and overlaps, capped pages, duplicate keys, source filters and currency partitions, XLSX amplification, temporary-file cleanup, path escape, redirect refusal and report symlinks. Tests and fixtures are synthetic; these results are not claims about production use. Live publication, Docker builds/image execution and real staging APIs are not tested by this review. Detailed commands and review limits are in the [milestone report](https://github.com/Nutts5796/logiparity/blob/main/docs/development/MILESTONE_REPORT.md).
+The seeded mixed-cause regression [m11-causes.test.ts](https://github.com/Nutts5796/logiparity/blob/main/tests/unit/m11-causes.test.ts) plants case, whitespace, numeric and consistent enum changes together. The seeded property-based test [filter-contributions.test.ts](https://github.com/Nutts5796/logiparity/blob/main/tests/unit/filter-contributions.test.ts) checks reconciliation while amounts and filter fields change independently. Regression cases include one-off differences that never become patterns, DST gaps and overlaps, capped pages, duplicate keys, source filters and currency partitions, XLSX amplification, temporary-file cleanup, path escape, redirect refusal and report symlinks. Tests and fixtures are synthetic; these results are not claims about production use. Live publication, Docker builds/image execution and real staging APIs are not tested by this review. Detailed commands and review limits are in the [milestone report](https://github.com/Nutts5796/logiparity/blob/main/docs/development/MILESTONE_REPORT.md).
 
 ## How it was built
 
@@ -243,7 +251,7 @@ Test and staging only. Read-only except scenario events you map yourself. No tel
 
 ## Status
 
-Version 1.1.0 — 2026-11-03. Maintained by Sergei Orekhovskii.
+Version 1.2.0 — 2026-11-08. Maintained by Sergei Orekhovskii.
 
 Pilots and feedback: share a sanitized use case or defect through [Issues](https://github.com/Nutts5796/logiparity/issues).
 
@@ -254,6 +262,7 @@ Sergei Orekhovskii, the maintainer, is a QA engineer who previously ran transpor
 ## Reference
 
 - [Configuration](docs/configuration.md), [scenarios](docs/scenarios.md) and [optional AI drafts](docs/ai.md)
+- [Socrata/FMCSA recipe](docs/recipes/socrata.md)
 - [Reports](docs/report-formats.md), [JSON schema](docs/report-schema.md) and [adapter development](docs/writing-adapters.md)
 - [CI](docs/ci.md), [Docker](docs/docker.md), [FAQ](docs/faq.md) and [release guide](docs/release.md)
 - [Dependencies](docs/dependencies.md), [audit notes](docs/dependency-audit.md) and [build history](https://github.com/Nutts5796/logiparity/blob/main/docs/development/milestones.md)

package/docs/configuration.md

@@ -45,7 +45,7 @@ Fields map logical names to paths, column headers or selectors. `{path: ..., lab
 
 ## Common source options and filters
 
-Every source accepts `label`, `fields`, `where`, `caseSensitive` (false), `unique` (empty), `rollup`, `maxRows` (100,000, maximum 10,000,000), `locale`, `numberFormat`, `datetimeFormat` and `currencyField`. Row limits count physical input records before filtering; exceeding a limit yields a source error. The engine indexes the reference and one candidate in memory. With Node's default heap (about 2 GB on an 8 GB Mac), two 1,000,000-row CSV sources with six columns ran out of memory; the same check passed with `NODE_OPTIONS=--max-old-space-size=6144`. Raise `maxRows` together with the heap size.
+Every source accepts `label`, `fields`, `where`, `caseSensitive` (false), `unique` (empty), `rollup`, `maxRows` (100,000, maximum 10,000,000), `locale`, `numberFormat`, `datetimeFormat`, `dateFormat` and `currencyField`. Row limits count physical input records before filtering; exceeding a limit yields a source error. The engine indexes the reference and one candidate in memory. With Node's default heap (about 2 GB on an 8 GB Mac), two 1,000,000-row CSV sources with six columns ran out of memory; the same check passed with `NODE_OPTIONS=--max-old-space-size=6144`. Raise `maxRows` together with the heap size.
 
 `where` combines fields with AND. Values can be scalar equality, a nonempty list, or `{not: value}` / `{not: [values]}`. The same predicate grammar applies to source, rollup and aggregate filters. Text equality is case-insensitive by default; set source `caseSensitive: true` for exact text matching. Typed filter values use their field normalization without comparison tolerances; untyped values use canonical strings. Quote money in major units: `{pay: "10.00"}`. Source filters run before unrelated cells are parsed; an all-filtered source warns with its entity/source and applied predicates.
 
@@ -57,7 +57,7 @@ datetimeFormat: 'dd.MM.yyyy HH:mm'
 maxRows: 100000
 ```
 
-A rule's explicit locale, numberFormat or format overrides the source default. Validation checks locales and formats. `unique: [tender_id]` finds a nonblank value reused across distinct keys in one source (`duplicate_value`); with rollups, keep that field in `by` or `first`. A repeated tender alone does not prove a duplicate payment.
+A rule's explicit locale, numberFormat or format overrides the source default. `dateFormat` is the default for calendar-date fields; `datetimeFormat` applies to datetime fields. Validation checks locales and formats. `unique: [tender_id]` finds a nonblank value reused across distinct keys in one source (`duplicate_value`); with rollups, keep that field in `by` or `first`. A repeated tender alone does not prove a duplicate payment.
 
 ## Source types
 
@@ -71,6 +71,18 @@ A rule's explicit locale, numberFormat or format overrides the source default. V
 
 REST mappings use supported JSONPath property/index/wildcard selections; unsupported syntax names its source and field. File/SQL mappings use headers/column names. UI mappings are selectors relative to each row; extraction reads rendered text. UI requires Playwright and Chromium; SQL requires the three optional PostgreSQL peers. Missing-peer errors give exact `--save-exact` install commands. Playwright supports `>=1.63.0 <2.0.0`; installed-package tests cover 1.63.0 and the pinned development version 1.64.1. Other peer ranges are in [dependencies](dependencies.md).
 
+### Absent REST properties
+
+REST mappings are strict by default. If an API omits null-valued JSON properties, opt in for the whole REST source with `absentAsNull: true`, or just one field with `missing: null`:
+
+```yaml
+fields:
+  id: $.id
+  name: {path: $.name, label: Name, missing: null}
+```
+
+An absent property, including an absent ancestor object, becomes null. Explicit null already maps to null. Wrong object/array shapes, multiple selected values and unreadable pages still fail; missing pagination metadata is never treated as a null field. The setting does not make a missing entity key valid. `missing: null` is a REST-only mapping option; `absentAsNull` defaults to false. The strict error names both opt-ins. Use a per-field opt-in when omitted properties might otherwise hide a misspelled mapping.
+
 ### REST completion and authentication
 
 ```yaml
@@ -84,6 +96,8 @@ pagination:
 
 Page pagination also supports `start: 1`; offset pagination uses `type: offset`, `offsetParam: offset`, `sizeParam`, `size`. Both support `totalPath`, `totalPagesPath` or `hasMorePath` completion signals. A short first page without a signal fails unless `allowShortFirstPage: true` explicitly opts into an unverified result (warning). Cursor pagination uses `type: cursor`, required `next`, `cursorParam: cursor`, `sizeParam: limit`, `size: 200`; link pagination uses `type: link`. Page sizes are 1–10,000. Exhausted page limits, loops and inconsistent completion fail instead of comparing partial data. Redirects are refused immediately; selected items are counted while decoding, before large arrays materialize.
 
+When page/offset pagination has none of those completion signals, a successful read emits one report warning: completeness is not verified. A short later page still ends the read; an empty probe page does too. A matching comparison is not proof of complete extraction. Run a separate count request with the same filter and a stable dataset version, then compare the count with the physical rows read. Cursor/Link endpoints retain their explicit next-pointer termination. The [Socrata recipe](recipes/socrata.md) shows count requests and stable ordering. There is no automatic `countUrl` option in this version.
+
 Use `auth: {type: bearer, tokenEnv: STAGING_TOKEN}` or `auth: {type: basic, usernameEnv: STAGING_USER, passwordEnv: STAGING_PASSWORD}`. `headersEnv: {Authorization: {env: STAGING_TOKEN, prefix: 'Bearer '}}` constructs a header; a string names an environment variable containing the complete value. Literal `headers` cannot contain sensitive auth/cookie/API-key headers. Credentials must not be embedded in URLs.
 
 ### Browser and SQL restrictions
@@ -96,11 +110,12 @@ SQL accepts the restricted read-only SELECT grammar and bound `params`; the adap
 
 | Type     | Behavior                                                                                                                                                                                                                                                                                                      |
 | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| string   | Trim text; optional `caseFold: true`. `emptyAsNull: true` by default.                                                                                                                                                                                                                                         |
-| enum     | Apply `map: {Delivered: delivered}` and optional case folding. Unknown values keep their trimmed text.                                                                                                                                                                                                        |
+| string   | Trim text; optional `caseFold: true` and `whitespaceFold: true` (collapse repeated whitespace to one space). `emptyAsNull: true` by default.                                                                                                                                                                  |
+| enum     | Apply `map: {Delivered: delivered}`, optional case folding and `whitespaceFold`. Unknown values keep their trimmed text.                                                                                                                                                                                      |
 | number   | Finite decimals with locale or explicit separators, Latin digits, valid grouping; `tolerance: 0`. Numeric JSON/exponent input is accepted; exponent text and currency symbols are rejected.                                                                                                                   |
 | money    | Required fallback `currency: USD`; decimal text becomes bigint minor units. `allowNumber: false`, `allowCurrencySymbol: false`, `negativeParentheses: false` by default. `tolerance: 0` in integer minor units. Excess nonzero decimal places fail unless `round: half-up` or `round: half-even` is selected. |
 | datetime | ISO 8601 becomes UTC epoch milliseconds; `toleranceSeconds: 0`. Offset-free values require `sourceTimezones`; missing zones, impossible dates, DST gaps and ambiguous wall clocks fail.                                                                                                                       |
+| date     | Calendar day normalized as `yyyy-MM-dd`, without a time zone, epoch or tolerance; see below.                                                                                                                                                                                                                  |
 
 Null compares equal only to null. Empty input becomes null by default; `emptyAsNull: false` preserves empty strings only for string/enum fields. Currency digits follow Intl, not a hard-coded two decimals. Numeric sums use exact canonical decimals before returning a finite number; overflow fails.
 
@@ -120,9 +135,17 @@ Map `currency` in every source using it. A source-level `currencyField` is also
 
 `allowNumber: true` converts finite numeric values through their canonical decimal text, including exponent notation. Decimal strings preserve original precision and remain preferable: conversion cannot recover precision already lost by JavaScript. For USD, 12.345 fails unless explicit rounding is configured; `0.1 + 0.2` can fail for the same reason. String inputs still use locale parsing. Rounding applies to input normalization, not to a combined total.
 
+### Calendar dates
+
+Use `type: date` for a calendar day, rather than an instant. The default input is strict ISO `yyyy-MM-dd`; month/year-only, timestamps and zone suffixes fail. `format: yyyyMMdd` or `format: {sms: dd-MMM-yy, census: yyyyMMdd}` selects supported calendar formats by source. The source-level `dateFormat` supplies a fallback; an explicit field format takes precedence. Formats must contain year, month and day and must not contain time or zone tokens. Impossible dates, including invalid leap days, become invalid-value findings. Null follows the normal null rules. Excel serial dates use their recorded calendar day; native JavaScript Date inputs use their UTC day.
+
+The `yy` rule is fixed: **00–68 means 2000–2068; 69–99 means 1969–1999**. It is independent of the current year, host time zone and Luxon global settings. Prefer four-digit years when century ambiguity matters. For example, `20250107` and `07-JAN-25` normalize to `2025-01-07`. A date rule rejects time zones, display-zone fields and nonzero time/numeric tolerances. JSON and every human format retain the canonical day. Calendar-day mismatches stay individual findings rather than timezone or vocabulary patterns.
+
+String/enum comparison still trims surrounding whitespace by default. Repeated internal whitespace remains significant unless `whitespaceFold: true`; letter case remains significant unless `caseFold: true`. Both fold settings default to false. Formatting patterns preserve the actual normalized spelling and suggest those opt-ins without automatically accepting differences.
+
 ### Datetimes and facility display
 
-`format: 'MM/dd/yyyy HH:mm'` applies a supported Luxon format to every source; `format: {export: 'MM/dd/yyyy HH:mm'}` applies only to that source. Other sources keep ISO parsing. Literal text must be quoted within a format. Explicit Z/±hh:mm offsets are authoritative, including XLSX `t="d"` cells. Naive cells and Excel serials use `sourceTimezones: {export: America/Chicago}`; ambiguous serials require a time zone column or unambiguous text, not a guessed offset.
+Validation rejects a datetime format without time tokens and recommends `type: date`, including a source's effective `datetimeFormat`. `format: 'MM/dd/yyyy HH:mm'` applies a supported Luxon format to every source; `format: {export: 'MM/dd/yyyy HH:mm'}` applies only to that source. Other sources keep ISO parsing. Literal text must be quoted within a format. Explicit Z/±hh:mm offsets are authoritative, including XLSX `t="d"` cells. Naive cells and Excel serials use `sourceTimezones: {export: America/Chicago}`; ambiguous serials require a time zone column or unambiguous text, not a guessed offset.
 
 `report.displayTimezone: America/New_York` changes human displays (default UTC), never parsing. A datetime rule's optional `displayTimezoneField: facility_zone` selects a mapped per-row IANA zone from the reference record for finding values and example charts; retain it in rollup `first`. Invalid/absent display metadata falls back to the report zone. This display-only option cannot repair offset-free input; configure source parsing separately. JSON retains normalized epoch milliseconds.
 

package/docs/recipes/socrata.md

@@ -0,0 +1,38 @@
+# Comparing Socrata data
+
+Socrata can omit null-valued JSON properties, return pages without completion metadata and publish calendar dates in different formats. Configure each behavior explicitly. The public FMCSA [Company Census File](https://data.transportation.gov/d/az4n-8mr2) and [SMS motor carrier census](https://data.transportation.gov/d/kjg3-diqy) illustrate these shapes.
+
+Use `$order` on the unique key when paging with `$offset`. Order does not freeze a changing dataset: compare the same filter and dataset version, and check update metadata before/after the run. A source fragment for Company Census is:
+
+```yaml
+type: rest
+baseUrl: https://data.transportation.gov
+path: "/resource/az4n-8mr2.json?$select=dot_number,power_units,mcs150_date&$where=phy_state='IA'&$order=dot_number"
+items: $[*]
+pagination:
+  type: offset
+  offsetParam: $offset
+  sizeParam: $limit
+  size: 5000
+  allowShortFirstPage: true
+fields:
+  id: $.dot_number
+  units: {path: $.power_units, missing: null}
+  filing_date: {path: $.mcs150_date, missing: null}
+```
+
+Map these fields in a second source and add entity rules such as `units: {type: number}` and `filing_date: {type: date, format: yyyyMMdd}`. For SMS versus Company Census, date formats can instead be `{sms: dd-MMM-yy, census: yyyyMMdd}`. Two-digit years use the fixed 1969–2068 window. Per-field `missing: null` tolerates absent properties selectively; source `absentAsNull: true` opts in for all mapped fields. Keys must still be present and valid.
+
+No total-count metadata is returned in the owner's Socrata observations. `allowShortFirstPage` accepts a small first page but does not verify completeness; a later short page also ends the read. LogiParity emits a warning in each report. Make a separate count request with the same filter and compare it with the physical source count:
+
+```sh
+curl --get 'https://data.transportation.gov/resource/az4n-8mr2.json' \
+  --data-urlencode '$select=count(*)' \
+  --data-urlencode "\$where=phy_state='IA'"
+```
+
+Use the matching `$where` for every source, including an active-record filter where applicable. A count from another dataset version cannot prove the earlier extraction complete. There is no automatic count request in LogiParity 1.2.0.
+
+The owner's archived 2026-11-08 investigation compared Iowa and Vermont. Company Census JSON and CSV matched on all 25 selected fields for 58,074 Iowa records and 8,378 Vermont records; independent server-side counts agreed. These are historical observations, not results of the smaller example above or claims about current counts. Both representations came from the same Socrata engine, so this checks extraction consistency rather than registration accuracy.
+
+SMS versus active Company Census produced 5,173 Iowa and 856 Vermont findings in that 1.1.0 investigation. Different populations, snapshot timing and field definitions matter: the hazmat definitions differ, and repeated power-unit changes do not imply a vocabulary mapping. Read the data dictionaries before treating one publication as authoritative. This recipe quotes only aggregates from the owner's `fmcsa_parity/findings.md`, with no carrier names or contact fields.

package/docs/release.md

@@ -1,6 +1,6 @@
 # Release procedure
 
-Release only reviewed changes. Version 1.1.0 is dated 2026-11-03; 1.0.1 was never published and its notes are folded into 1.1.0. Local preparation never pushes, tags or publishes. Use `npm run pack:artifact` for the sanitized CLI alone or `npm run release:prepare` for the complete release assets. Direct workspace `npm pack`/`npm publish` is rejected so a manifest with development fields cannot be distributed accidentally; release jobs publish the reviewed tarball.
+Release only reviewed changes. Development/package checks allow only the current version’s exact development heading in `CHANGELOG.md`; `check:release-text -- --publication`, release preparation, release builds and publishing reject it. Replace it with the actual date only after owner approval. Version 1.1.0 is dated 2026-11-03; 1.0.1 was never published and its notes are folded into 1.1.0. Local preparation never pushes, tags or publishes. Use `npm run pack:artifact` for the sanitized CLI alone or `npm run release:prepare` for the complete release assets. Direct workspace `npm pack`/`npm publish` is rejected so a manifest with development fields cannot be distributed accidentally; release jobs publish the reviewed tarball.
 
 ## One-time setup
 

package/docs/report-formats.md

@@ -6,11 +6,15 @@ HTML is a self-contained offline report with embedded licensed fonts, a strict C
 
 The hero leads with supported causes and individual review counts. A pattern needs at least `patterns.minCount` findings (default 3, minimum 3). A single overpayment or status mismatch stays under Needs review. Cause cards suggest a next step. Accounted for means each difference is traced to a cause above. Each cause still needs a fix. This status is neutral and does not change `failOn` or the exit code.
 
-Money and count totals show missing-row contributions and supported pattern deltas. Totals fully accounted for say Follows from; a nonzero remainder needs review. Blank or duplicate lines make only the affected keys uncertain. Totals reconcile certain keys and list Uncertain lines separately. Unmapped totals say Not mapped; connection failures say Not read. Zero contribution buckets are hidden.
+Money, count and number-sum totals show missing, extra and changed-value contributions, supported pattern deltas, uncertainty and an unexplained remainder in HTML, Markdown, console, JSON and JUnit. Number contributions use exact canonical decimal strings; numeric totals also expose `exactValues`, used for display and reconciliation. Totals fully accounted for say Follows from; a nonzero remainder needs review. Blank or duplicate lines make only the affected keys uncertain. Totals reconcile certain keys and list Uncertain lines separately. Unmapped totals say Not mapped; connection failures say Not read. HTML hides empty contribution buckets; text breakdowns retain explicit zero components.
+
+Case-only and whitespace-only patterns retain the actual candidate text and suggest explicit normalization. Numeric pairs are repeated numeric changes, never vocabulary mappings. Consistent enum changes suggest comparing refresh timing in both sources and field definitions; a pattern alone cannot identify which system is wrong.
+
+Page/offset reads without total-count or other completion metadata include a completeness warning even when every compared record matches. A separate count request can provide an independent check.
 
 Charts use each source's unique key count. The rounding waffle names its source and rounds down, with 1 square = 1%. Page-boundary charts show rows kept per page. Timezone fixes name the configuration property without guessing a zone. Source headers show row filters and rolled-up line-to-key counts.
 
-`report.displayTimezone` accepts an IANA zone and defaults to UTC. It changes human date display, including the run timestamp; it never changes normalization. Set `compare.<field>.sourceTimezones` to interpret a timezone-free source. JSON remains unchanged.
+`report.displayTimezone` accepts an IANA zone and defaults to UTC. It changes human date display, including the run timestamp; it never changes normalization. Set `compare.<field>.sourceTimezones` to interpret a timezone-free source. JSON retains canonical time units; calendar-date fields remain ISO days and have no display zone.
 
 Console and Markdown prioritize individual review, then provide examples of supported causes. Markdown uses Markdown escaping, hides zero-count finding types and limits individual rows to 50, with one example per cause. JUnit groups findings by entity and human-readable type; one finding uses singular grammar. Runtime/source errors are distinct from ordinary findings in every format. JUnit `time=0` is a schema placeholder, not measured execution time.
 

package/docs/report-schema.json

@@ -1,6 +1,6 @@
 {
   "$schema": "http://json-schema.org/draft-07/schema#",
-  "$id": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.1.0/docs/report-schema.json",
+  "$id": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.2.0/docs/report-schema.json",
   "title": "LogiParity parity report, schemaVersion 1",
   "type": "object",
   "required": ["schemaVersion", "entities", "findings"],
@@ -281,7 +281,15 @@
             "type": "string"
           },
           "kind": {
-            "enum": ["datetime_offset", "money_delta", "value_mapping", "missing_stride", "extra_stride"]
+            "enum": [
+              "datetime_offset",
+              "money_delta",
+              "value_mapping",
+              "missing_stride",
+              "extra_stride",
+              "formatting_difference",
+              "number_change"
+            ]
           },
           "count": {
             "type": "integer",
@@ -440,6 +448,13 @@
               "$ref": "#/definitions/reconciliation"
             }
           }
+        },
+        "exactValues": {
+          "type": "object",
+          "additionalProperties": {
+            "type": "string",
+            "pattern": "^-?[0-9]+(?:\\.[0-9]+)?$"
+          }
         }
       }
     },
@@ -472,7 +487,7 @@
           "minimum": 0
         },
         "amount": {
-          "$ref": "#/definitions/integerAmount"
+          "$ref": "#/definitions/decimalAmount"
         }
       }
     },
@@ -504,13 +519,13 @@
                 "minimum": 0
               },
               "amount": {
-                "$ref": "#/definitions/integerAmount"
+                "$ref": "#/definitions/decimalAmount"
               }
             }
           }
         },
         "remainder": {
-          "$ref": "#/definitions/integerAmount"
+          "$ref": "#/definitions/decimalAmount"
         },
         "uncertain": {
           "type": "object",
@@ -520,7 +535,7 @@
               "$ref": "#/definitions/keyParts"
             },
             "amount": {
-              "$ref": "#/definitions/integerAmount"
+              "$ref": "#/definitions/decimalAmount"
             }
           }
         },
@@ -531,6 +546,17 @@
           }
         }
       }
+    },
+    "decimalAmount": {
+      "anyOf": [
+        {
+          "type": "string",
+          "pattern": "^-?[0-9]+(?:\\.[0-9]+)?$"
+        },
+        {
+          "type": "integer"
+        }
+      ]
     }
   }
 }

package/docs/report-schema.md

@@ -1,14 +1,16 @@
 # JSON report schema, version 1
 
-The machine-readable [JSON Schema](report-schema.json) ships in the package. New optional properties may be added to schemaVersion 1; readers must tolerate unknown fields. Required meanings and money/time units remain stable. Removing fields or changing units requires a new schema version. Readers of older reports may encounter conflict `line` properties and display-string uncertain keys; new reports use physical `row` and key-part arrays. Version 1.1.0 uses this version-1 contract.
+The machine-readable [JSON Schema](report-schema.json) ships in the package. New optional properties may be added to schemaVersion 1; readers must tolerate unknown fields. Required meanings and money/time units remain stable. Removing fields or changing units requires a new schema version. Readers of older reports may encounter conflict `line` properties and display-string uncertain keys; new reports use physical `row` and key-part arrays. Versions 1.1.0 and 1.2.0 use this version-1 contract.
 
 The root requires schemaVersion, entities and findings. Optional patterns.minCount is at least 3 (default 3). groups preserve exact findingIndexes and never contain fewer members than minCount. coverage.explained counts the union of supported pattern members and accounted aggregate findings; needsReview counts the remaining findings. Accounting does not change failure policy.
 
 Each finding has type, entity, source, key, field and message. Types are missing_in_source, extra_in_source, field_mismatch, aggregate_mismatch, duplicate_key, duplicate_value, invalid_value and source_error. expected and actual are optional normalized scalars. invalid_value includes its physical row and, when available, the valid reference value. duplicate_value has relatedKeys and optional amounts; a repeated tender alone is not evidence of duplicate payment. Rollup conflicts use values containing row and value, rather than invented expected/actual values. Composite keys retain keyParts. Entity pluralLabel is an optional display noun.
 
-Money is a decimal integer minor-unit string. Datetimes are UTC epoch milliseconds. Optional currency identifies monetary findings. Aggregate valuesByCurrency and rawValuesByCurrency partition currencies; currencies are never converted or combined. rawValues, excludedRows, sourceStatus and uncertainKeys distinguish raw input, retained/certain inputs, unsupported mappings and failed reads. uncertainKeys contains arrays of key parts, including a one-element array for a simple key.
+Money is a decimal integer minor-unit string. Datetimes are UTC epoch milliseconds. Calendar dates are ISO `yyyy-MM-dd` strings without a zone. Number-sum `exactValues` contains canonical exact-decimal strings alongside compatibility `values` numbers; compare and display `exactValues` when present. Optional currency identifies monetary findings. Aggregate valuesByCurrency and rawValuesByCurrency partition currencies; currencies are never converted or combined. rawValues, excludedRows, sourceStatus and uncertainKeys distinguish raw input, retained/certain inputs, unsupported mappings and failed reads. uncertainKeys contains arrays of key parts, including a one-element array for a simple key.
 
-Reconciliation is keyed by source. Its identity is source total minus reference total equals missingRows.amount + extraRows.amount + valueDifferences.amount + sum(patterns.amount) + uncertain.amount + remainder. Each contribution is filtered by the aggregate's own where clause. Pattern amounts sum qualifying row contributions, rather than multiplying the whole pattern count. Tolerated or unsupported changes remain in the remainder. Zero remainder means the aggregate is accounted for; individual invalid values and one-off differences still require review. Mixed-currency totals remain partitioned and never receive a misleading combined reconciliation.
+Reconciliation is keyed by source. Its identity is source total minus reference total equals missingRows.amount + extraRows.amount + valueDifferences.amount + sum(patterns.amount) + uncertain.amount + remainder. Money/count contributions use integer strings; number-sum contributions and remainders use exact decimal strings. Each contribution is filtered by the aggregate's own where clause. Pattern amounts sum qualifying row contributions, rather than multiplying the whole pattern count. Tolerated or unsupported changes remain in the remainder. Zero remainder means the aggregate is accounted for; individual invalid values and one-off differences still require review. Mixed-currency totals remain partitioned and never receive a misleading combined reconciliation.
+
+New 1.2.0 group kinds are `formatting_difference` (cause `case_only`, `whitespace_only` or `case_whitespace`) and `number_change` (an exact repeated expected/actual pair). Both retain expected/actual and findingIndexes. `stale_value` remains a consistent-change heuristic that does not establish which side is stale or whether definitions differ.
 
 Patterns describe observations. Timezone hints require offset-free source values and a quarter-hour delta; page-boundary hints require pagination or matching configured page-size evidence. Blank values never form mapping or delta patterns. Groups retain every underlying finding and blocking status.
 
@@ -16,7 +18,7 @@ Object properties follow configuration order; findings sort naturally by entity,
 
 ## Package exports and scenario reports
 
-Import `logiparity/report-schema.json` and `logiparity/scenario-report-schema.json` (JSON import attributes depend on your runtime), or read the files under docs/ in the installed package. Their versioned raw `$id` URLs identify the 1.1.0 tag; they identify the corresponding versioned release. Validators can register both local schemas without network access.
+Import `logiparity/report-schema.json` and `logiparity/scenario-report-schema.json` (JSON import attributes depend on your runtime), or read the files under docs/ in the installed package. Their versioned raw `$id` URLs identify the 1.2.0 tag; they identify the corresponding versioned release. Validators can register both local schemas without network access.
 
 The [scenario schema](scenario-report-schema.json) describes `kind: scenario`, events, invariants, affectedOrderCount, runtime errors and optional nested Parity. Both wire schemas use version 1: consumers must tolerate additive optional fields; required meanings or money/time units cannot change without a schema version change. Runtime error reports may contain only the state collected before failure.
 

package/docs/scenario-report-schema.json

@@ -1,6 +1,6 @@
 {
   "$schema": "http://json-schema.org/draft-07/schema#",
-  "$id": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.1.0/docs/scenario-report-schema.json",
+  "$id": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.2.0/docs/scenario-report-schema.json",
   "title": "LogiParity scenario report, schemaVersion 1",
   "type": "object",
   "required": ["schemaVersion", "kind", "name", "target", "environment", "events", "invariants", "exitCode"],
@@ -46,7 +46,7 @@
       "type": "string"
     },
     "parity": {
-      "$ref": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.1.0/docs/report-schema.json"
+      "$ref": "https://raw.githubusercontent.com/Nutts5796/logiparity/v1.2.0/docs/report-schema.json"
     },
     "events": {
       "type": "array",

package/lib/adapters/dist/jsonpath.d.ts

@@ -1,3 +1,3 @@
 export declare function jsonPathTokens(path: string): string[];
-export declare function selectJson(value: unknown, path: string): unknown[];
-export declare function selectOne(value: unknown, path: string): unknown;
+export declare function selectJson(value: unknown, path: string, absentAsNull?: boolean): unknown[];
+export declare function selectOne(value: unknown, path: string, absentAsNull?: boolean): unknown;

package/lib/adapters/dist/jsonpath.js

@@ -16,25 +16,33 @@ export function jsonPathTokens(path) {
     }
     return tokens;
 }
-export function selectJson(value, path) {
+export function selectJson(value, path, absentAsNull = false) {
+    const absent = Symbol('absent');
     const tokens = jsonPathTokens(path);
     let nodes = [value];
     for (const token of tokens) {
         nodes = nodes.flatMap((node) => {
+            if (node === absent)
+                return [absent];
             if (token === '*') {
                 if (!Array.isArray(node))
                     throw new Error(`JSONPath ${path}: [*] requires an array; correct the extraction path.`);
                 return node;
             }
-            if (!node || typeof node !== 'object' || !Object.hasOwn(node, token))
-                throw new Error(`JSONPath ${path}: property ${token} is missing; correct the field mapping.`);
+            if (!node || typeof node !== 'object')
+                throw new Error(`JSONPath ${path}: property ${token} requires an object; correct the field mapping.`);
+            if (!Object.hasOwn(node, token)) {
+                if (absentAsNull)
+                    return [absent];
+                throw new Error(`JSONPath ${path}: property ${token} is missing; correct the field mapping or opt in with absentAsNull: true (source) or missing: null (field).`);
+            }
             return [Reflect.get(node, token)];
         });
     }
-    return nodes;
+    return nodes.map((node) => (node === absent ? null : node));
 }
-export function selectOne(value, path) {
-    const nodes = selectJson(value, path);
+export function selectOne(value, path, absentAsNull = false) {
+    const nodes = selectJson(value, path, absentAsNull);
     if (nodes.length !== 1)
         throw new Error(`JSONPath ${path} selected ${nodes.length} values; map exactly one scalar.`);
     return nodes[0];

package/lib/adapters/dist/rest.js

@@ -152,16 +152,22 @@ export async function* readRest(config, context) {
             if (signals.some((value) => value !== signals[0]))
                 throw new Error(`${location}; completion signals disagree. Correct the response metadata or pagination paths.`);
             more = signals[0];
+            const unverified = 'API returned no total count or completion metadata; completeness is not verified. Compare rows read with a separate count request using the same filter, or configure totalPath, totalPagesPath or hasMorePath.';
             if (more === undefined && page === 0 && items.length < pagination.size) {
                 const message = `${location}; no completion signal is configured. This is an unverified short first page. Set totalPath, totalPagesPath or hasMorePath to verify completeness.`;
                 if (!pagination.allowShortFirstPage)
                     throw new Error(`${message} Set size: ${items.length} to match the server cap, or allowShortFirstPage: true only to accept an unverified short first page.`);
-                (context.warn ?? console.warn)(safeConsole(message));
+                (context.warn ?? console.warn)(safeConsole(`${message} ${unverified}`));
             }
+            else if (more === undefined && page === 0)
+                (context.warn ?? console.warn)(safeConsole(unverified));
         }
         for (let index = 0; index < items.length; index++) {
             try {
-                yield Object.fromEntries(Object.entries(config.fields).map(([field, path]) => [field, selectOne(items[index], fieldPath(path))]));
+                yield Object.fromEntries(Object.entries(config.fields).map(([field, path]) => [
+                    field,
+                    selectOne(items[index], fieldPath(path), config.absentAsNull || (typeof path !== 'string' && path.missing === null)),
+                ]));
             }
             catch (error) {
                 throw new Error(`REST page ${page + 1}, record ${index + 1}: ${error instanceof Error ? error.message : String(error)}`);

package/lib/cli/dist/version.d.ts

@@ -1 +1 @@
-export declare const version = "1.1.0";
+export declare const version = "1.2.0";

package/lib/cli/dist/version.js

@@ -1,2 +1,2 @@
 // Generated by release:bump.
-export const version = '1.1.0';
+export const version = '1.2.0';

package/lib/core/dist/aggregates.d.ts

@@ -3,9 +3,14 @@ import type { NormalizedValue } from './types.js';
 export declare class DecimalSum {
     private units;
     private scale;
-    add(value: number): void;
+    add(value: number | bigint | string): void;
+    text(): string;
+    percentOf(other: DecimalSum): number;
     value(): number;
     within(tolerance: number): boolean;
 }
-export declare function measureAggregate(aggregate: AggregateConfig, entity: EntityConfig, source: string, config: SourceConfig, rows: Map<string, Record<string, NormalizedValue>>): number | bigint | undefined;
-export declare function aggregateEqual(expected: number | bigint, actual: number | bigint, config: AggregateConfig): boolean;
+export declare const negateDecimal: (value: bigint | string) => bigint | string;
+export declare function sumDecimal(...values: (bigint | string | number)[]): string;
+export declare function displayDecimal(value: bigint | string): string;
+export declare function measureAggregate(aggregate: AggregateConfig, entity: EntityConfig, source: string, config: SourceConfig, rows: Map<string, Record<string, NormalizedValue>>, exactSums?: Map<string, string>): number | bigint | undefined;
+export declare function aggregateEqual(expected: number | bigint | string, actual: number | bigint | string, config: AggregateConfig): boolean;

package/lib/core/dist/aggregates.js

@@ -19,6 +19,19 @@ export class DecimalSum {
         }
         this.units += units * 10n ** BigInt(this.scale - scale);
     }
+    text() {
+        const absolute = String(this.units < 0n ? -this.units : this.units).padStart(this.scale + 1, '0');
+        const decimal = this.scale
+            ? `${absolute.slice(0, -this.scale)}.${absolute.slice(-this.scale)}`.replace(/\.?0+$/, '')
+            : absolute;
+        return `${this.units < 0n ? '-' : ''}${decimal}`;
+    }
+    percentOf(other) {
+        if (other.units === 0n)
+            return 0;
+        const scale = Math.max(this.scale, other.scale);
+        return (Number((this.units * 10n ** BigInt(scale - this.scale) * 10000n) / (other.units * 10n ** BigInt(scale - other.scale))) / 100);
+    }
     value() {
         const value = Number(`${this.units}e-${this.scale}`);
         if (!Number.isFinite(value))
@@ -33,7 +46,19 @@ export class DecimalSum {
         return absolute * 10n ** BigInt(scale - this.scale) <= bound.units * 10n ** BigInt(scale - bound.scale);
     }
 }
-export function measureAggregate(aggregate, entity, source, config, rows) {
+export const negateDecimal = (value) => typeof value === 'bigint' ? -value : value.startsWith('-') ? value.slice(1) : '-' + value;
+export function sumDecimal(...values) {
+    const sum = new DecimalSum();
+    values.forEach((value) => sum.add(value));
+    return sum.text();
+}
+export function displayDecimal(value) {
+    const text = sumDecimal(value), [integer = '0', fraction] = text.split('.');
+    return ((text.startsWith('-') ? '−' : '') +
+        BigInt(integer.replace(/^-/, '')).toLocaleString('en-US') +
+        (fraction ? '.' + fraction : ''));
+}
+export function measureAggregate(aggregate, entity, source, config, rows, exactSums) {
     const required = [...Object.keys(aggregate.where), ...(aggregate.type === 'sum' ? [aggregate.field] : [])];
     if (required.some((field) => !Object.hasOwn(config.fields, field)))
         return undefined;
@@ -59,15 +84,17 @@ export function measureAggregate(aggregate, entity, source, config, rows) {
     }
     if (aggregate.type === 'count')
         return count;
+    if (entity.compare[aggregate.field]?.type === 'number')
+        exactSums?.set(aggregate.name, decimal.text());
     return entity.compare[aggregate.field]?.type === 'money' ? money : decimal.value();
 }
 export function aggregateEqual(expected, actual, config) {
     if (typeof expected === 'bigint' && typeof actual === 'bigint')
         return (expected > actual ? expected - actual : actual - expected) <= BigInt(config.tolerance);
-    if (typeof expected !== 'number' || typeof actual !== 'number')
+    if (typeof expected === 'bigint' || typeof actual === 'bigint')
         return false;
     const difference = new DecimalSum();
     difference.add(expected);
-    difference.add(-actual);
+    difference.add(negateDecimal(String(actual)));
     return difference.within(config.tolerance);
 }

package/lib/core/dist/calendar.d.ts

@@ -0,0 +1,3 @@
+export declare function formatHasTime(format: string): boolean;
+export declare function isCalendarFormat(format: string): boolean;
+export declare function normalizeDate(text: string, format: string | undefined, locale: string): string;