@cosyte/astm 0.0.11 → 0.0.14

package/CHANGELOG.md

@@ -11,6 +11,202 @@ this file is maintained by hand (Changesets handles the version bump and publish
 
 ### Fixed
 
+- **`composeAstmFrames` wrote an out-of-range `startFrameNumber` straight into the frame-number
+  position, so a value that was not a frame number produced a stream whose records the decoder
+  refused to emit** (`ASTM-FRAME-RESIDUALS`, defect 13). `PRE-EXISTING`, byte-identical on `64c2fd5`.
+  The option is now checked before `composeAstmFrames` reads a record, and refused with the new code
+  `ASTM_FRAME_INVALID_START_FRAME_NUMBER` (a fourth member of `AstmFrameEncodeErrorCode`) unless it is
+  a whole number from `0` to `7`.
+
+  **What went out instead, measured on this package's own round trip.** A frame's number is one ASCII
+  digit, written as `FN_ZERO + n`, and nothing checked `n`. `-1` put a `/` in the frame-number
+  position, and `NaN`, `Infinity` and `-Infinity` each put a `NUL` there in **every** frame of the
+  stream, after which `decodeAstmFrames` recognised no frame number at all, warned
+  `ASTM_FRAME_SEQUENCE_GAP` on every frame, and emitted **none** of the records: a whole four-record
+  message in, zero out, and `parseFramedAstm` throwing on empty input. **The recorded defect said an
+  out-of-range value writes a non-digit byte and named a space for `NaN`. It is a `NUL`**, which is
+  the difference between an ordinary inter-frame byte the decoder skips and a byte that costs the
+  stream every one of its records.
+
+  **The quieter half is why the check covers the whole domain rather than the ends.** Values that
+  truncated back onto a digit were accepted silently and behaved as some other start value: `1.5` and
+  `257` each produced the byte-for-byte stream a `startFrameNumber` of `1` produces. The option
+  documented `0`-`7` and enforced neither bound nor integrality in either direction.
+
+  **The domain is derived from what a frame can carry, and the refusal is not a clamp.** The writable
+  numbers are exactly the whole numbers `0`-`7`, the modulo-8 sequence `decodeAstmFrames` reads and
+  rolls over. Clamping or taking a modulo would pick a frame number the caller did not ask for, and
+  the frame number is the decoder's only evidence that no frame was dropped, so a stream numbered from
+  a value nobody chose is a stream whose sequence check certifies the wrong thing. The refusal is
+  raised before `composeAstmFrames` reads a record, so on that entry point it never depends on the
+  caller's data (`serializeFramedAstm` serializes every record first, so a record that cannot be
+  serialized at all is refused ahead of it on that route). Its message names the value received: that
+  value is the caller's own option, not stream content, and nothing from the records reaches it.
+
+  **The whole `0`-`7` range is still accepted, because a non-default start has a real use that was
+  measured rather than assumed.** It composes a **continuation** of a transfer already in progress:
+  `composeAstmFrames(head)` joined with `composeAstmFrames(tail, { startFrameNumber: n })`, where `n`
+  is the number after the last frame `head` used, is **byte-identical** to composing the whole list in
+  one call and decodes with an empty warnings array, across the `7 → 0` rollover included. Narrowing
+  the option to `1` would have removed that.
+
+  **What a continuation is not, now stated on the option rather than guarded against.** Read on its
+  own, a stream that starts anywhere but `1` opens on a sequence gap; the decoder never bridges a gap
+  silently, so it warns and does not emit that first record. **What `parseFramedAstm` does after that
+  varies with the message shape, and no rule is offered for it:** it may throw, under more than one
+  code (`ASTM_RECORD_NO_HEADER`, `EMPTY_INPUT` and `ASTM_RECORD_UNDECLARED_DELIMITERS` were each
+  measured), and it may return a message one record short. **The one statement that generalizes is
+  that the record layer never reports the loss**, because `parseFramedAstm` hands the record parser
+  only the frames the codec vouched for, so `message.warnings` carries what the surviving records
+  warrant and nothing about the record that did not survive. Read `frameWarnings`. Nothing returns the number to continue a sequence from either, and the frame count
+  is not it once a record splits, so the supported computation is named on the option. The
+  documented-valid `0` has always behaved that way. It is the cost of the option, not a defect in the
+  caller's records, and it is why `1` is the default.
+
+  **This is a narrowing on a published package, in a small population.** `1.5` and `257` used to
+  produce a working stream, the byte-for-byte start-at-`1` one, and now throw. Every value the check
+  turns away was already outside what the option documented: a frame number, a whole number `0`-`7`.
+
+  Both the old bytes and the new refusal are pinned in `test/frames/start-frame-number.test.ts`,
+  asserted on what the old encoder produced (rebuilt with the test-only `frame()` builder, so nothing
+  is transcribed twice), with a biconditional property: a value is refused **if and only if** it is
+  not a whole number in `0`-`7`. 14 of the 32 new tests are red against `64c2fd5`.
+
+- **Three `{@link}` targets in the published `.d.ts` did not name a symbol declared there**
+  (`ASTM-FRAME-RESIDUALS`, the sibling minors). `QuirkTolerance` is `AstmQuirkTolerance` (twice, in
+  `src/profiles/types.ts`); the `startFrameNumber` doc linked `FIRST_FRAME_NUMBER`, which this
+  package does not export, and that sentence was rewritten and no longer links it; and
+  `AstmMessage.profile` linked a bare `warnings`, meaning its sibling member, which resolves only for
+  a tool that resolves against the enclosing declaration. Every `{@link}` target in
+  `dist/index.d.ts` now names a symbol declared in that file.
+
+- **A delimiter set colliding with a record's type letter emitted a stream that read back as
+  DIFFERENT records, and in its silent branch it fabricated a final lab result out of patient
+  identifiers** (`ASTM-FRAME-RESIDUALS`, defect 5). `PRE-EXISTING`, byte-identical on `7253098`,
+  before any of `ASTM-EMIT-RESIDUALS`. `serializeAstmRecords` / `serializeAstmRecord` now throw
+  `AstmSerializeError` with the new code `ASTM_EMIT_TYPE_LETTER_COLLISION` (a third member of
+  `AstmSerializeErrorCode`), carrying the `recordIndex` and quoting nothing from the record: not the
+  value, and not the type letter either, which on an unsupported record is a byte off the wire.
+
+  **The recorded defect said the record "re-reads as an unsupported record, one result in, zero out".
+  That describes one branch of two, and the branch it does not describe is silent.** A record's type
+  letter is just another leaf to the escape encoder, so a set naming that letter escapes it away: an
+  `R` record emitted with `field` = `R` goes out as `&F&R1R…`, reads back as `unsupported`, and
+  `ASTM_RECORD_UNKNOWN_TYPE` does fire. But the encoder writes an escaped character as
+  escape + mnemonic + escape, so **when the escape character is itself a record type letter the
+  escaped type letter begins with a real letter** and no unknown-type warning fires at all. Measured:
+  a `P` record emitted with `field` = `P` and `escape` = `R` comes back as an `R` record whose
+  `value` is the patient's laboratory ID, whose `units` are the practice-assigned identifier, and
+  whose `resultStatus` is `F`, so `results()` returns a fabricated **final** result built out of
+  patient identifiers. The stream's only warning is `ASTM_NONSTANDARD_DELIMITERS`, which is what a
+  **clean** non-canonical stream carries too and which is on the profile safety gate's tolerable
+  allow-list, so `{ strict: true }` accepts it under a profile the gate permits. That allow-list
+  entry reasons that "a record's type letter is the first character of its line" is read before any
+  delimiter-driven tokenization: true of the parse, and no protection when the **emit** is what chose
+  that character.
+
+  **The rule is derived from the reader and checked on the bytes, not on the delimiter set.**
+  `parseAstmRecords` takes a record's type from `line.charAt(0)`, so the condition emit has to meet
+  is that the first character it writes is the letter the record models. Testing the output rather
+  than enumerating dangerous roles is what makes a type letter equal to the **escape** character fall
+  out as accepted without a special case: it is written starting with that letter, so the record
+  re-reads as its own type. The familiar `letter` + `E` + `letter` shape is not general and the guard
+  does not rely on it: the encoder protects the escape character it introduces but not the `E`/`F`/`S`
+  mnemonics, so under `{ field: "E", escape: "R" }` an `R` encodes to `RRFRR` and its type field
+  decodes back to `RER`, not `R`. What holds across every such set is the only thing checked, the
+  first character written. Measured over 137,632
+  delimiter sets (two roles at a time, every printable ASCII character each, against a stream
+  carrying one record of every modeled type plus an unsupported one), the refusal is
+  **biconditional** with the old serializer losing a type letter: zero over-refusals and zero
+  under-refusals. Over a second sweep of 3,690 emits across nine record-set shapes, 750 streams
+  previously read back as something other than the records that produced them, **303 of them accepted
+  by `{ strict: true }`**; all 750 are now refused and the remaining 2,940 are byte-unchanged.
+
+  This is a **narrowing on a published package**, in exactly the cases that were being corrupted.
+  **It is a transcoding condition, not a judgement on the set the caller passed, so it fires with no
+  delimiter argument at all** and "emit against the canonical delimiters" is not a remedy for it: a
+  stream whose header declares a vendor set and which carries one garbled line beginning `|` parses
+  to an unsupported record whose type letter is `|`, and the canonical set escapes that `|` away, so
+  `serializeAstmRecords(msg)`, `serializeAstmRecord(record)` and `serializeFramedAstm(msg)` all
+  refuse it where the base emitted a record whose `rawType` came back as `&`. Callers passing a set
+  explicitly (`serializeAstmRecords(msg, msg.delimiters)`) reach it too. The three
+  set-level conditions are unchanged and still raise `ASTM_EMIT_INVALID_DELIMITERS`; this is a
+  separate, per-record check because whether a set collides depends on which record is being written.
+  **What it does not promise, stated rather than left to be found:** it guarantees a record re-reads
+  as its own **type**, not that every field lands where it did (an escape sequence whose body is a
+  delimiter is still an opaque atom, reported on the parse side as `ASTM_UNKNOWN_ESCAPE_SEQUENCE`),
+  and `encodeComponent` / `serializeField` take no record, so a caller assembling a line out of them
+  is outside the check. Pinned in `test/records/type-letter-collision.test.ts`, asserted on what the
+  old serializer produced, rebuilt from the shipped `serializeField` so nothing is transcribed twice.
+  **No clause is claimed**: LIS02-A2 §5.4/§6.2 stay withheld from CLSI's free sample and the
+  paywalled editions were not read; the grounding is this package's own reader.
+
+- **A raw `STX`, `ETB` or `ETX` byte in a value was framed as given, and in its worst branch a whole
+  result record disappeared into the previous record's text with `warnings: []` at BOTH layers**
+  (`ASTM-RAW-ETX-SWALLOWS-A-RECORD`). `PRE-EXISTING`, shipped since the frame encoder did.
+  `composeAstmFrames` now throws `AstmFrameEncodeError` with the new code `ASTM_FRAME_RESERVED_BYTE`
+  (a third member of `AstmFrameEncodeErrorCode`), carrying `recordIndex` and `characterIndex` and
+  never the bytes.
+
+  **The recorded reason for deferring this said it "fails loudly". That was false, and the
+  correction is what re-ranked it.** A frame's text ends at the first `ETB`/`ETX` after its `STX`, so
+  an embedded one truncates the frame there, and the two bytes that follow are then read as that
+  short frame's checksum. When they match, the truncated frame **verifies**, `decode.ts` resumes at
+  `termIndex + 3` and skips the rest of the record as inter-frame bytes, and the next frame number is
+  still in sequence. Measured on this package's own round trip: a `C` comment ending `…HEMOLYZED` +
+  `ETX` + the two matching characters reassembled without its terminating `CR`, so the following `R`
+  merged into the comment's free text (`"SPECIMEN SLIGHTLY HEMOLYZEDR"`) and a `28.6 U/L` result
+  vanished, `results()` returning `[]` where the input carried one. **An embedded `ETB` reaches the
+  same silence by the other door**, which the defect record did not have: it leaves the record open,
+  so the _next_ record's text is appended to the truncated one and two records read back as one, with
+  every field of the result hanging off the comment. The precondition is a coincidence in the two
+  bytes that follow: they must both read as hex digits and must equal the truncated frame's checksum,
+  which is about 1 in 34,600 for uniform-random bytes (1 in 256 once they are hex digits, which is
+  where the earlier 1-in-256 figure came from), and trivially constructible on purpose. That bounds
+  the likelihood, not the harm.
+  Both silent branches, and the loud ones, are pinned in `test/frames/reserved-structure-byte.test.ts`
+  asserted on what the old encoder produced, so a weakened guard reds a test rather than passing
+  unnoticed.
+
+  **The refusal has no bytes-instead escape hatch, unlike the `U+00FF` one beside it.** A
+  `Uint8Array` is checked too, because the byte is unframable however it arrives: framing has no
+  escape sequence for it, and emit has no warning channel, so the only alternatives to refusing are
+  substituting a byte the sender did not send or dropping one they did. The three bytes are derived
+  from what `decodeAstmFrames` actually reads as structure, not from a control-character class:
+  `CR`/`LF` are deliberately absent (they are read only _after_ a frame's checksum, and a record's own
+  `CR` sits inside frame text on every stream this encoder writes), and `ENQ`/`ACK`/`NAK`/`EOT` are
+  absent too (structure only _between_ frames; measured to round-trip byte-exactly inside one).
+  **No clause is claimed:** LIS02-A2 §5.4/§6.2 are withheld from CLSI's free sample and the paywalled
+  editions were not read. The grounding is this library's own decoder, in this repo.
+
+  **The record layer is deliberately UNCHANGED, and that was the cost this slice turned on.**
+  Refusing the byte in `serializeAstmRecord` would refuse a byte the caller genuinely supplied, for
+  consumers who never frame anything. Measured, in every modeled value: all three
+  bytes round-trip through parse → serialize → parse byte for byte, value, units and status intact,
+  byte-stable, `warnings: []` on both generations. The byte becomes structure only when a frame is
+  built, and `composeAstmFrames` is the total gate on that route, including through
+  `serializeFramedAstm`. `CR`/`LF` remain the record layer's own refusal, because they end a _record_.
+
+  **That claim is about VALUES, and one position on a record line is not a value.** The surplus of a
+  header's delimiter declaration drops any control character silently on emit, so a header that
+  arrived as `H|\^&` + `ETX` goes back out without it, `warnings: []`, byte-stable, and the
+  round-trip is not byte-exact there. `PRE-EXISTING` and deliberately unchanged: that rule is argued
+  at `declarationResidual` and long predates the frame-layer refusal, and it is the better of the two
+  dispositions now, since carrying the byte through would turn a spec-clean header into a refused
+  stream. Pinned in `test/frames/reserved-structure-byte.test.ts` so the scoped sentence stays
+  measured. Found by the `conformance-refuter` grading this slice.
+
+  **What this does not close.** A delimiter colliding with a record's type letter still frames and
+  de-frames byte-exactly and re-reads as a different record; framing integrity is not record-layer
+  readback. And a caller reaching `composeAstmFrames` from JavaScript with some other typed array is
+  still outside the declared signature, and **this guard does not reach that route either**: the
+  check compares elements against the three byte values, so a `Uint16Array` element of `0x0103` is
+  not one of them and is not refused, while `Uint8Array.from` later takes its low byte and writes an
+  `ETX` after all. Measured on all three (`0x0102`/`0x0103`/`0x0117`): framed, then lost at decode
+  with a bad checksum or an unterminated frame. That is the same out-of-signature residue already
+  recorded for the low-byte truncation, unchanged here in either direction, and stated rather than
+  left to be found. Pass a `Uint8Array` or a string.
+
 - **A symbolic link under a scan root read CLEAN on BOTH of the PHI scanner's enumerating routes, so
   a link pointing at a file full of real identifiers passed the gate twice over**
   (`PHI-SCAN-SYMLINK-BLIND-ON-BOTH-ROUTES`). Development tooling only: `scripts/phi-scan.ts` ships in

package/README.md

@@ -39,10 +39,27 @@ reference parser, [`@cosyte/hl7`](https://github.com/cosyte/hl7).
   frame them with computed checksums, frame numbers, and the 240-byte split. Both layers round-trip by
   construction, and a delimiter set that fails any of the three conditions readback requires (one
   character per separator, no `CR`/`LF`, no two the same) is a typed error rather than bytes written
-  and lost. A frame carries **bytes**, so a record handed to `composeAstmFrames` as a `string` is one
+  and lost. Those three read the set alone, so each record is additionally checked against the set it
+  is written with: a separator equal to a record's own type letter escapes that letter away and the
+  record re-reads as a different record, which is `ASTM_EMIT_TYPE_LETTER_COLLISION` rather than
+  output. That second check is about **transcoding**, not about which set you asked for, so it can
+  fire with no delimiter argument at all: a stream read under a vendor's own set may carry a record
+  whose type letter the canonical set escapes away, and `serializeFramedAstm` refuses it for the same
+  reason. A frame carries **bytes**, so a record handed to `composeAstmFrames` as a `string` is one
   byte per character and a character above `U+00FF` is a typed error too: the encoder will not pick a
   character encoding for you, and it never quietly writes a different character than the one you gave
-  it. Encode such content yourself and pass the `Uint8Array`.
+  it. Encode such content yourself and pass the `Uint8Array`. A raw `STX`, `ETB` or `ETX` **byte** in a
+  record is a typed error as well, in either form: those three are what the decoder reads as the shape
+  of a frame, framing has no escape sequence for them, and writing one through truncated the frame at
+  that byte, silently losing a whole record whenever the two bytes after it happened to be the short
+  frame's checksum. The **record** layer still carries them, because a returned string is not yet on a
+  wire and a raw-transport consumer round-trips such a value exactly. `startFrameNumber` lets you
+  compose one transfer across several calls, and `composeAstmFrames` checks it before it reads a
+  record: a frame's number is a single ASCII digit, so a value that is not a whole number from `0` to
+  `7` is a typed error rather than whatever byte the arithmetic truncated to. The round trip above is
+  the **default** start; a non-default one writes a continuation of a sequence already in progress,
+  and a continuation read on its own opens on a frame-sequence gap, so its first record is warned
+  about and not emitted.
 - **Vendor profiles.** `defineAstmProfile()` builds a provenance-backed profile whose tolerances
   downgrade _expected_, non-safety-critical deviations to a `PROFILE_QUIRK_APPLIED` warning without
   ever altering a value, behind a safety gate that refuses to tolerate any result value, flag,

package/dist/index.cjs

@@ -1387,7 +1387,8 @@ var AstmSerializeError = class extends Error {
   /**
    * Stable discriminant. `ASTM_EMIT_UNENCODABLE_VALUE` for a `CR`/`LF` in a value;
    * `ASTM_EMIT_INVALID_DELIMITERS` for a delimiter set that cannot be emitted
-   * reversibly.
+   * reversibly; `ASTM_EMIT_TYPE_LETTER_COLLISION` for a record whose own type
+   * letter the set being emitted with would escape away.
    */
   code;
   /** 0-based ordinal of the record within the message, when known. */
@@ -1451,10 +1452,24 @@ function serializeAstmRecord(record, d = CANONICAL_DELIMITERS) {
   return serializeRecordChecked(record, d);
 }
 function serializeRecordChecked(record, d) {
+  return assertTypeLetterSurvives(serializeRecordText(record, d), record);
+}
+function serializeRecordText(record, d) {
   if (record.type === "M" || record.type === "S") return serializeVerbatimRecord(record, d);
   if (record.type === "H") return serializeHeader(record, d);
   return record.fields.map((f) => encodeField(f.repeats, d, record.recordIndex)).join(d.field);
 }
+function recordTypeLetter(record) {
+  return record.type === "unsupported" ? record.rawType : record.type;
+}
+function assertTypeLetterSurvives(text, record) {
+  if (text.charAt(0) === recordTypeLetter(record)) return text;
+  throw new AstmSerializeError(
+    "A record's type letter is escaped away by the delimiters being emitted with, so the record would read back as a different record. The record's index is on the error; the letter itself is not quoted, because on an unsupported record it is a byte off the wire.",
+    record.recordIndex,
+    "ASTM_EMIT_TYPE_LETTER_COLLISION"
+  );
+}
 function serializeVerbatimRecord(record, d) {
   if (recoversVerbatim(record, d)) return record.rawLine;
   return record.fields.map((f) => encodeField(f.repeats, d, record.recordIndex)).join(d.field);
@@ -2229,7 +2244,9 @@ var AstmFrameEncodeError = class extends Error {
   /**
    * Position of the offending character within that record, when applicable:
    * its index in the string, never the character itself and never its code
-   * point. Enough to find it in the caller's own data.
+   * point. Enough to find it in the caller's own data. For a record supplied as
+   * a `Uint8Array` this is the offending byte's index, which is the same
+   * position: a record string is a byte string, so the two indices coincide.
    */
   characterIndex;
   /** @internal */
@@ -2259,6 +2276,29 @@ function toBytes(input, recordIndex) {
   }
   return out;
 }
+var RESERVED_STRUCTURE_BYTES = /* @__PURE__ */ new Set([STX, ETB, ETX]);
+function assertFramable(bytes, recordIndex) {
+  for (let i = 0; i < bytes.length; i += 1) {
+    const byte = bytes[i];
+    if (byte !== void 0 && RESERVED_STRUCTURE_BYTES.has(byte)) {
+      throw new AstmFrameEncodeError(
+        "A record contains a byte the framing layer reads as structure (STX, ETB or ETX). A frame has no escape sequence for one, so it cannot be carried inside a frame: remove it from the record before framing.",
+        recordIndex,
+        "ASTM_FRAME_RESERVED_BYTE",
+        i
+      );
+    }
+  }
+}
+function assertStartFrameNumber(value) {
+  if (!Number.isInteger(value) || value < 0 || value > FRAME_NUMBER_MODULUS - 1) {
+    throw new AstmFrameEncodeError(
+      `startFrameNumber must be a whole number from 0 to 7: a frame's number is one ASCII digit and the sequence is modulo 8. Received: ${String(value)}.`,
+      void 0,
+      "ASTM_FRAME_INVALID_START_FRAME_NUMBER"
+    );
+  }
+}
 function encodeFrame(text, frameNumber, isFinal) {
   const term = isFinal ? ETX : ETB;
   const bytes = [STX, FN_ZERO + frameNumber, ...text, term];
@@ -2267,11 +2307,13 @@ function encodeFrame(text, frameNumber, isFinal) {
   return [...bytes, hex.charCodeAt(0), hex.charCodeAt(1), CR, LF];
 }
 function composeAstmFrames(records, options = {}) {
+  const start = options.startFrameNumber ?? FIRST_FRAME_NUMBER;
+  assertStartFrameNumber(start);
   if (records.length === 0) {
     throw new AstmFrameEncodeError("Cannot frame an empty record list: nothing to transmit.");
   }
   const out = [];
-  let frameNumber = options.startFrameNumber ?? FIRST_FRAME_NUMBER;
+  let frameNumber = start;
   records.forEach((record, recordIndex) => {
     const bytes = toBytes(record, recordIndex);
     if (bytes.length === 0) {
@@ -2280,6 +2322,7 @@ function composeAstmFrames(records, options = {}) {
         recordIndex
       );
     }
+    assertFramable(bytes, recordIndex);
     for (let offset = 0; offset < bytes.length; offset += MAX_FRAME_TEXT) {
       const chunk = bytes.subarray(offset, offset + MAX_FRAME_TEXT);
       const isFinal = offset + MAX_FRAME_TEXT >= bytes.length;
@@ -2315,7 +2358,7 @@ function concatBytes3(chunks) {
 }
 
 // src/index.ts
-var VERSION = "0.0.11";
+var VERSION = "0.0.14";
 
 exports.ALL_ASTM_WARNING_CODES = ALL_ASTM_WARNING_CODES;
 exports.AMBIGUOUS_CODES = AMBIGUOUS_CODES;