@cosyte/x12 0.0.5 → 0.0.7

package/CHANGELOG.md

@@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **`renderCallerJson`** (internal), the type-preserving half of the caller-value bound, held to the
+  same `BUILD_REFUSAL_VALUE_MAX_RENDERED` ceiling as `renderCallerValue`. It exists because
+  `defineProfile()` reports a bad `name` / `id` / `effect` / `fixture` with `JSON.stringify`, and that
+  distinction is diagnostically load-bearing: `null` and `"null"` are different mistakes, and a
+  coercing renderer would flatten them together. It bounds the JSON **text** rather than the argument,
+  never throws (a circular structure, a `BigInt`, a hostile `toJSON`), and fabricates no closing quote,
+  because JSON does not always open one.
+- **`requireCallerArray`** (internal), the single route a caller-supplied array takes into a builder
+  loop. Each builder passes its own `refuse` callback, so a forged list draws that module's existing
+  typed error and code rather than a new shared one.
 - **`renderCallerValue`**, plus the `BUILD_REFUSAL_VALUE_MAX_LENGTH` (63) and
   `BUILD_REFUSAL_VALUE_MAX_RENDERED` (90) bounds. This is the single sanctioned route a
   caller-supplied value takes into a `build*` refusal message, and both ceilings are exported so a
@@ -32,8 +42,144 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   a warning `message` is a frozen-registry lookup with positional metadata only, but an orphan is
   document content, verbatim, exactly like `tx.rawSegments`. Log `context` and `segmentIndex`.
 
+### Security
+
+- **The PHI scanner refuses an in-scope entry that is not a regular file, on both of its enumerating
+  routes.** A symbolic link under a scan root pointing at a PHI-bearing file used to scan CLEAN on
+  both, so the pre-commit gate and CI both reported "no hits" over a capture the scan never read.
+  Measured on `5779542`, against a throwaway repository laid out like this one, using a synthetic
+  `.edi` payload whose NM1 person name, DMG date of birth, PER phone and `REF*SY` SSN are all hits at
+  exit 1 when the same bytes sit at a regular file:
+  - the all-mode walk enumerates `Dirent.isFile()`, which is an lstat answer, so a link is neither a
+    file nor a directory and fell out of the loop silently. A link under `test/fixtures`, a link
+    under `src/`, and a linked DIRECTORY (which takes a whole subtree with it) each reported
+    `OK - no hits` at exit 0;
+  - `--staged` reads content with `git show :<path>`, and git stores a link as its TARGET PATH under
+    mode `120000`, so that route was handed the path text and never the target's bytes. A staged
+    link reported `OK - no hits` at exit 0.
+
+  Both routes now refuse the scan (exit 2, the existing "could not complete" code) and name every
+  offender, not just the first. Neither route is made to FOLLOW an ENTRY it enumerated: following
+  would read bytes the enumeration does not control, and git does not carry those bytes anyway, so a
+  hit on them would be a claim about something no commit contains. That is a statement about an
+  entry, not about a scan ROOT: a walk root that is itself a link is still followed, because
+  `existsSync` and `readdirSync` both follow. Measured identically before and after this change, that
+  direction produces a superset scan rather than a blind one (the target's files are enumerated under
+  their in-root names and hit, exit 1), so it is deliberately left alone.
+
+  **A refusal names the entry's own repo-relative path and a scanner-owned token for its kind. It
+  never reports the link target**, which is working-tree text that can itself carry PHI: a target
+  path of the shape `<surname>-<given>-<dob>.edi` is the whole reason. That shape is written out
+  rather than shown, because a diagnostic about a PHI leak is itself a PHI surface. The concern was
+  not hypothetical here: measured at base, a staged link whose target name was a dashed-SSN shape
+  exited 1 and printed that shape, because `git show` handed the path text straight to the
+  cross-cutting shape pass.
+
+  **The `--staged` filter is now `AMT`, and `T` is the one-letter difference that made the mode check
+  reachable.** Replacing a TRACKED regular file with a link is neither an add nor a modify: measured
+  on this tree, `git diff --cached --raw --diff-filter=AM` returned zero rows for that change while
+  the unfiltered `--raw` showed `:100644 120000 <sha> <sha> T`. Without `T` the record died before
+  any mode could be read and the hook passed a mode-`120000` blob green. Admitting `T` also covers
+  the reverse typechange, a link replaced by a real file bearing PHI. The route reads
+  `git diff --cached --raw -z` rather than `--name-only` because the destination mode is the only
+  thing that distinguishes a staged regular file from a staged link or gitlink, and
+  `git show :<path>` answers all three without complaint. A record that does not parse refuses rather
+  than being skipped.
+
+  **Each route keeps its own existing boundary**: the walk still excludes a gitignored entry (the same
+  rule that already excludes a gitignored file), and `--staged` still looks only at
+  `test/fixtures/**` and `src/**.ts`. This narrows what those scopes admit; it does not widen them.
+  A gitlink under a scanned prefix already exited 2 before this change, but by `git show` failing and
+  echoing git's own text; it is now refused at enumeration and named by kind.
+
+  **`paths` mode is deliberately unchanged, because it was never blind**: it reads with
+  `readFileSync`, which follows a link, so an explicitly named path that is a link to a PHI-bearing
+  file is scanned and hits (measured, exit 1).
+
+  **Not closed here, and stated rather than implied**: `R` (rename) and `C` (copy) are still not
+  enumerated by `--staged` at all, which is pre-existing and needs the two-path `--raw` record shape
+  handled. Measured, so the cost is not left to inference: renaming a fixture while substituting a
+  real name stages as `:100644 100644 <sha> <sha> R080 <old> <new>`, which both `AM` and `AMT` return
+  zero rows for, and `--staged` exits 0 over a payload that is a hit as an ordinary add; `git mv` of
+  an already-committed link into `test/fixtures/` is `R100` and is likewise not refused. The all-mode
+  sweep is the backstop for both (exit 1 and exit 2 respectively), so the gap is at pre-commit, not
+  in CI. Also unclosed: a scan that observed nothing is still reported clean rather than refused; and
+  the
+  enumerate-then-read window in all mode is untouched, because tolerating a failed read pulls the
+  opposite way from narrowing what the enumeration admits and belongs in its own change. No library
+  code changed and no published type changed.
+
 ### Fixed
 
+- **The `attw` publish gate no longer passes a tarball that carries no type declarations.** The
+  `attw` script was the bare CLI, and `@arethetypeswrong/cli` returns 0 whenever its analysis found
+  no types at all, before it reads the problem list. For a package that ships types, "This package
+  does not contain types." means the declarations were missing from the tarball, which is a broken
+  publish that the gate reported as a pass. Reproduced against this package with no concurrency
+  involved: with `dist/` removed, and with only `dist/index.d.ts` and `dist/index.d.cts` removed,
+  the CLI printed that sentence and exited 0 in both cases. The second is a state every build passes
+  through, because `tsup` writes the JS before the declarations (measured at 1.92 seconds apart on
+  one clean build here). `pnpm attw` now runs `scripts/attw.mjs`, which checks that every relative
+  path `package.json` promises exists and is non-empty before invoking the CLI, and fails afterwards
+  if the CLI still reports an untyped package. No change to the library or to any published type.
+
+- **A `defineProfile()` refusal message no longer grows with the value you passed in.** Twelve
+  refusal sites in `src/profiles/validate.ts` hold twenty-three caller-value holes between them, and
+  every one now routes through `renderCallerValue` or `renderCallerJson`. Re-derived on this tree
+  before the fix, driving the same thirteen cases the suite ships: the worst `X12ProfileError.message`
+  was **360,181 characters**, at the `fixture` refusal. Three of the thirteen exceed 360,000, and they
+  are the three that name **three** caller values (profile name + quirk id + a `JSON.stringify`d
+  value) rather than two; a 120,000-digit quirk id reaches them because `QUIRK_ID_RE` carries no
+  length bound. The same `fixture` refusal now measures **431 characters**.
+
+  **431 is a measurement, not a maximum.** The ` (N characters)` suffix widens with the decimal width
+  of the value's length, so the same refusal measures 434 at a 1,000,000-character value and 437 at
+  10,000,000. The site's ceiling, derived from its fixed text plus its three fragment ceilings, is
+  **443**; the suite asserts every one of the twelve under 500.
+
+  **The figure this was filed on, 120,093, did not reproduce**, the same way `X12-BUILDER-BOUNDS`'s
+  own filed figures did not: it depends on which site is hit and what the probe passes.
+
+  Scope it the way the builder half is scoped. This is **not** `PHI-WARNING-MESSAGE-LEAK`, where the
+  value was the document's: here you passed it in and still hold it, so bounding it **redacts
+  nothing**. What it buys is a fixed ceiling on anything reaching a log line, a crash report or a JSON
+  error envelope. The surviving characters are **not escaped**, and the bound is on UTF-16 **code
+  units, not bytes**. **`X12ProfileError.profileName` is deliberately left unbounded**, so it still
+  matches the name you passed.
+
+- **A builder handed a forged non-array now refuses instead of hanging.** Every domain builder took
+  its loop bound from a caller-supplied `.length`, so `{ length: "9".repeat(120000) }` coerced to
+  `Infinity`, every element read `undefined`, every guard `continue`d, and the builder **spun forever
+  rather than refusing**. Measured at base over the nineteen probes that drive a forged list at a
+  builder ENTRY point (17 `FORGED_ARRAY_CASES` + 2 `RESIDUAL_CASES`), each in a child process under a
+  20-second wall-clock timeout because a hang cannot be observed in-process: **16 of 19 hung** with no
+  refusal and the other **3 threw an untyped `TypeError`**. The suite ships three further forged
+  probes on optional LEAF arrays, which are `TypeError` at base and at head alike and move nothing:
+  counting all 22, base is 16 hung / 6 untyped and head is 17 typed / 5 untyped. All **32 indexed
+  loops across 7 builder modules** now take their bound from a `requireCallerArray` binding, and at
+  head the same nineteen give **17 typed, code-tagged refusals** (messages 169 to 194 characters) and
+  **2 untyped `TypeError`s**. `build835`'s `spec.traces` is the one that moved from the untyped group
+  to the typed one, because its guard reads the list; it never hung, and this changelog does not claim
+  it did.
+
+  A hang is a worse failure than a refusal: a refusal hands control back with something to branch on,
+  a hang takes the worker with it. But state the class correctly. This is a **forged non-array input,
+  not a mis-read clinical value** - nothing decodes a document differently because of it, and the
+  reachable harm is availability. It is unreachable from TypeScript and reachable from JavaScript,
+  JSON, and therefore `@cosyte/cli`.
+
+  **Disclosed rather than fixed, and identical at base and head:** where a builder reads a caller array
+  with `for...of` - `buildInterchange`'s `spec.groups`, `build999`'s
+  `functionalGroup.transactionResponses`, and every optional leaf array such as `claim.dates` - a
+  forged list throws `TypeError: ... is not iterable`. That terminates, so it is not the hang, but it
+  carries **no `code`**. Pinned by a test so it cannot quietly become a hang.
+
+- **The `QUIRK_ID_RE` comment claimed a length bound the pattern never had.** It said "2-64
+  lowercase-alphanumeric chars"; the regex accepts one character and it accepts 120,000, which was in
+  fact the path to the largest profile error message on the tree. The comment was corrected to the
+  code, not the grammar tightened: rejecting ids that define cleanly today is a separate decision from
+  bounding a message.
+
 - **A `build*` refusal message no longer grows with the value you passed in.** All twenty-three
   caller-value slots across the ten builder modules route through `renderCallerValue`, capping the
   rendered fragment at 90 characters. Nine are the `control number "…" exceeds the N-char spec limit`

package/README.md

@@ -45,7 +45,7 @@ That's the pitch: no schema upload, no spec knowledge. The parser accepts vendor
 - **Postel's Law**: a lenient parser (deviations → warnings with a stable code + positional context) and a conservative serializer. `serializeX12` is byte-faithful **for the segments on the model** by default: each comes back verbatim, including element padding, composites, and `?`-release escapes. **`serialize(parse(s)) === s` is not guaranteed in general.** Six constructs are known not to survive: line breaks between segments (any run of CR / LF between segments is absorbed at parse, so a pretty-printed or double-spaced file emits compact), a doubled terminator outside a transaction, a missing final terminator (the emit supplies one), post-IEA `trailingBytes` (re-joined, not verbatim), a TA1 that followed a functional group (emitted right after the ISA, so it is **reordered**, though nothing is lost), and a segment whose first element is empty outside a transaction (skipped entirely, with no warning at all). A **segment outside a transaction is not on that list**: it is reported as `X12_UNEXPECTED_SEGMENT`, kept on the model at `ix.orphanSegments` with a structural `anchor`, and re-emitted at that anchor, so the segment, its value and its warning all survive the round trip - placement is by the anchor and never by `segmentIndex`, which indexes the input stream the emit does not follow. The last five break the round trip on inputs with no line breaks at all, so do not treat "no line breaks" as sufficient; five of the six are silent, so a clean warnings list does not mean byte-exact either. Measured across this repo's 56 fixtures: every emit is a fixed point and re-parses to an identical model with an identical warning stream, the 14 with no line breaks return byte-identical, and the other 42 differ by line breaks and nothing else. See [Line endings between segments](./docs-content/spec-notes-envelope.md). `{ specClean: true }` reconciles the envelope, and `{ specClean: true, recomputeCounts: true }` also emits the corrected counts (`recomputeCounts` does nothing on its own). A mismatch is always warned, never silently corrected. Only **4 structural failures** are ever fatal.
 - **Money is exact**: every monetary/percent/quantity field decodes as `X12Decimal` (string-backed, BigInt arithmetic). The library **never `parseFloat`s** an EDI amount.
 - **Safety-critical fidelity**: TRN reassociation traces, 835 balance invariants, 837 HL hierarchy integrity, 834 maintenance types, and 278 certification actions are preserved **verbatim** and never inferred; ambiguity yields a warning or a typed refusal, never a confident wrong answer.
-- **PHI-disciplined**: synthetic-only fixtures, a PHI commit-gate, and a warning `message` that is a **lookup into a frozen registry**, not something built from your document. No warning factory in the library takes a value parameter, so no element can reach a diagnostic: the code and `position` say what and where, and the bytes stay on the model. `ALL_WARNING_MESSAGES` is exported so you can assert it. **Builder refusals are a different, and deliberately weaker, surface:** twenty-three refusal sites across ten builder modules name a value you passed in, so you can see which control number, count or code was refused. Each goes through `renderCallerValue`, capping the rendered **fragment** at `BUILD_REFUSAL_VALUE_MAX_RENDERED` (90 characters); all three names are exported so you can assert the ceiling. The whole `message` is that plus the site's own fixed text, so it is bounded by a constant but a larger one: a 120,000-character control number gave a 120,066-character `X12BuildError.message` before this and gives 150 now. That is robustness, not redaction: the value is one you supplied and bounding it hides nothing from you. It is also not escaped, and on the **ack** path it is not always strictly your own, since `build999`'s AK2-02 and `buildTA1` echo an inbound document's control numbers by design. Log `err.code`, not `err.message`, from a builder. Tracked in [KNOWN-LIMITATIONS.md](./KNOWN-LIMITATIONS.md). **The one deliberate exception:** `X12ParseError.snippet` on the four Tier-3 structural fatals is a bounded (≤ 64 character) copy of the start of the input, so on real traffic it can carry PHI. The library does not redact it. Redact at your call site, or log `err.code` and `err.position` instead. See [Keeping PHI out of logs](./docs-content/troubleshooting.md).
+- **PHI-disciplined**: synthetic-only fixtures, a PHI commit-gate, and a warning `message` that is a **lookup into a frozen registry**, not something built from your document. No warning factory in the library takes a value parameter, so no element can reach a diagnostic: the code and `position` say what and where, and the bytes stay on the model. `ALL_WARNING_MESSAGES` is exported so you can assert it. **Builder refusals are a different, and deliberately weaker, surface:** twenty-three refusal sites across ten builder modules name a value you passed in, so you can see which control number, count or code was refused. Each goes through `renderCallerValue`, capping the rendered **fragment** at `BUILD_REFUSAL_VALUE_MAX_RENDERED` (90 characters); all three names are exported so you can assert the ceiling. The whole `message` is that plus the site's own fixed text, so it is bounded by a constant but a larger one: a 120,000-character control number gave a 120,066-character `X12BuildError.message` before this and gives 150 now. That is robustness, not redaction: the value is one you supplied and bounding it hides nothing from you. It is also not escaped, and on the **ack** path it is not always strictly your own, since `build999`'s AK2-02 and `buildTA1` echo an inbound document's control numbers by design. Log `err.code`, not `err.message`, from a builder. **`defineProfile()` is bounded on the same terms since `0.0.6`** (twelve refusal sites, twenty-three caller values; the worst message measured **360,181 characters** before and **431** now, both at the `fixture` refusal, which names three caller values; the ceiling for that site is 443 and the suite asserts every site under 500; `X12ProfileError.profileName` is deliberately left unbounded so it still matches the name you passed). **And a forged non-array in a builder spec now refuses instead of hanging:** every indexed loop takes its bound from a checked array, so `{ length: "9".repeat(120000) }` draws a typed refusal rather than coercing to `Infinity` and looping forever. A few `for…of` reads still throw an untyped `TypeError` instead; both are in [KNOWN-LIMITATIONS.md](./KNOWN-LIMITATIONS.md). **The one deliberate exception:** `X12ParseError.snippet` on the four Tier-3 structural fatals is a bounded (≤ 64 character) copy of the start of the input, so on real traffic it can carry PHI. The library does not redact it. Redact at your call site, or log `err.code` and `err.position` instead. See [Keeping PHI out of logs](./docs-content/troubleshooting.md).
 
 See the [**Cookbook**](./docs-content/cookbook.md) for task-oriented recipes (post an 835, route 277CA rejections, round-trip a 271, walk an 837, read a 999) and [**KNOWN-LIMITATIONS.md**](./KNOWN-LIMITATIONS.md) for the honest do-not-over-trust list.