jspurefix 5.11.0 → 5.11.2

package/docs/instrumentation.md

@@ -0,0 +1,346 @@
+# Instrumentation — design note
+
+**Status: design, nothing implemented.** This is the shape being argued about before any
+code is written, because the contract has to be right in two engines at once. Comments
+and disagreement welcome on the tracking issue.
+
+The goal is that an operator running jspurefix in production can answer, without adding
+their own code: is the session up, how much is going through it, what is it costing me,
+and has anything started going wrong. Today they can answer none of those except by
+reading log prose.
+
+## What this is not
+
+[`docs/benchmarking.md`](benchmarking.md) already measures parser throughput and
+allocation. That is **offline**: `jsfix-bench` drives synthetic bytes under
+`--expose-gc` so a change to a hot path can be argued about with a number.
+
+This note is about the **live** engine — the same quantities, in a running process, at a
+cost low enough to leave switched on. The two should agree on names wherever they
+measure the same thing, so a bench figure and a production figure can be put side by
+side.
+
+## Principles
+
+1. **The snapshot is the contract.** Prometheus, OpenTelemetry, a JSON endpoint and a
+   CLI table are four *renderings* of one data structure. Design that structure; the
+   renderings are then cheap, independent, and none of them is in the way of the others.
+2. **Zero cost when off.** A disabled sink is a no-op method call, never a string format
+   or an object allocation. `EmptyLogger` is the existing precedent.
+3. **Nothing new in `dependencies`.** See [packaging](#packaging-boundary) — this is the
+   trap most likely to be walked into.
+4. **One spec, two engines.** Metric names and semantics are defined here and
+   implemented identically in cspurefix. A shared dashboard is most of the value.
+
+## Layer 0 — the snapshot
+
+A plain object, no dependencies, produced on demand. Everything else renders it.
+
+There is already a rough version of this: `TcpAcceptor` logs an *acceptor census*
+(`transports=1 [1@::1:57598 up 0s] sessions=0 []`). It is a stats snapshot that only
+knows how to be a log line.
+
+```ts
+interface ISessionStats {
+  sessionId: string          // BeginString-Sender-Target
+  peerCompId: string
+  role: 'initiator' | 'acceptor'
+  state: SessionState        // the enum, not a rendered string
+  upSince: Date | null
+  // sequence
+  lastSentSeqNum: number
+  lastPeerMsgSeqNum: number
+  // traffic, since process start
+  msgsSent: number
+  msgsReceived: number
+  bytesSent: number
+  bytesReceived: number
+  // health
+  resendRequestsSent: number
+  resendRequestsReceived: number
+  gapsDetected: number
+  reconnects: number
+  lastReceivedAt: Date | null
+  lastSentAt: Date | null
+}
+```
+
+Gauges are read from state that already exists — `FixSessionState` holds
+`lastPeerMsgSeqNum`, `LastSentAt`, `lastReceivedAt`, `peerCompId`; `SessionState` is the
+enum; `SessionRegistry` knows the live set. Counters are the new part.
+
+**Open question.** Whether the snapshot is per-session with an aggregate rolled up by the
+registry, or one flat structure. Per-session with rollup is more useful to a venue and
+more work; a single acceptor with one counterparty gets the same answer either way.
+
+## Layer 1 — chokepoints
+
+Counters are pushed through a narrow sink with a no-op default. The seams already exist;
+none of these needs a new abstraction to hang off.
+
+| what | where (jspurefix) | notes for the port |
+| --- | --- | --- |
+| bytes in, parse time | `AsciiParser.parseBuffer` / `parse` | the byte count is the write, not the message — a segment may hold several or half of one |
+| messages decoded, by msgType | `AsciiParser.msg` | bounded label set, ~30 values |
+| parse buffer high water | `ElasticBuffer.currentSize` vs `returnTo` | a buffer that keeps growing past `returnTo` is the early warning for a slow consumer |
+| messages encoded, bytes out | `MsgTransmitter.send` / `settle` | `settle` already carries the payload and any error |
+| session state transitions | `FixSession.setState` | transition counts, and time spent in each state |
+| application vs session traffic | `AsciiSession.dispatchMsg` | the split matters: heartbeats are not throughput |
+| gaps, resends, logon retries | `SessionSequenceCoordinator` | `pendingResendRequests`, `logonRetryCount` are already tracked |
+| store depth, put latency | `FixMsgMemoryStore` / `FileSessionStore` | file store latency is the one that will surprise people |
+| transports accepted, harvested | `TcpAcceptor` | already emits `transport` and `harvested` |
+| reconnects, recovery attempts | `RecoveringTcpInitiator` | see `recovery policy` logging added in 5.11.0 |
+
+**Bytes per second through the parser is the headline number** and deserves care: it is
+what an evaluator asks for, and it is the one most easily measured wrongly. Count bytes
+where they enter the parser, not where they leave the socket, and keep decode time
+separate from `toObject()` — the benchmarking doc already draws that line as
+`parse:view` against `parse:object`, and live instrumentation should draw it in the same
+place.
+
+## The metric set
+
+Names in Prometheus form; OpenTelemetry and the C# port use the same words. Prefix is
+`fix_` rather than `jspurefix_` so both engines populate one dashboard.
+
+| metric | type | labels |
+| --- | --- | --- |
+| `fix_session_up` | gauge (0/1) | `session`, `role` |
+| `fix_session_state` | gauge (enum value) | `session` |
+| `fix_messages_total` | counter | `session`, `direction`, `msg_type`, `class` (admin/app) |
+| `fix_bytes_total` | counter | `session`, `direction` |
+| `fix_parse_duration_seconds` | histogram | — |
+| `fix_encode_duration_seconds` | histogram | — |
+| `fix_seq_num` | gauge | `session`, `direction` |
+| `fix_resend_requests_total` | counter | `session`, `direction` |
+| `fix_gaps_detected_total` | counter | `session` |
+| `fix_reconnects_total` | counter | `session` |
+| `fix_store_depth` | gauge | `session` |
+| `fix_transports_open` | gauge | `listener` |
+
+Alerting mostly falls out of these: `fix_session_up == 0` during trading hours,
+`rate(fix_gaps_detected_total[5m]) > 0`, a seq num that stops advancing, a store depth
+that only grows.
+
+### Cardinality
+
+A wildcard acceptor (`TargetCompID: '*'`) can meet an unbounded set of counterparties, so
+**`session` is not automatically a safe label**. A venue with churny clients would grow
+the series set without limit. Options: cap the labelled set and bucket the rest as
+`other`, make per-session labelling opt-in, or expose per-session detail only through the
+diagnostics endpoint and keep the scrape aggregated. Decide before shipping, not after
+someone's Prometheus falls over.
+
+`msg_type` is safe — bounded by the dictionary.
+
+## Structured logging
+
+**This lands first, on its own.** Not because it is smaller — because it delivers the
+whole Elastic story by itself, and because the pass that adds fields is the pass that
+locates every future counter site.
+
+Worth separating two consumers that want opposite things:
+
+- **Filebeat and Elastic are push.** The engine writes JSON lines, Filebeat harvests,
+  Kibana does the drill-down. No server in the engine at all.
+- **A live diagnostics endpoint is pull.** For when there is no Elastic stack, or when
+  someone wants the state of a process now rather than after a harvest cycle.
+
+The click-through-and-drill-down picture is entirely the first of those. That makes the
+web server in [packaging](#packaging-boundary) an independent, later, optional thing
+rather than a prerequisite.
+
+### The leverage is bound context, not the call sites
+
+The engine has **307 logger call sites** (239 `info`, 39 `warning`, 15 `error`, 14 `debug`)
+but only **29 places a logger is constructed**. And the identity is already at those 29:
+
+```ts
+this.sessionLogger = config.logFactory.logger(`${this.me}:${peerCompId}:FixSession`)
+```
+
+That is the session identity, built once, and completely opaque downstream — the "regex
+over English" problem, except it is not even English, it is punctuation. So the first move
+is not widening 307 call sites, it is widening the 29. Bound context makes every one of the
+307 filterable without touching any of them.
+
+Per-call fields then matter only where a line carries a *value* worth aggregating —
+`msgType`, seq nums, byte counts, state transitions, gap ranges. Perhaps 25 lines, chosen
+deliberately. They are the same chokepoints [layer 1](#layer-1--chokepoints) lists, which
+is why doing this first leaves the metrics work half-located.
+
+### The interface
+
+```ts
+export type JsFixLogFields = Record<string, unknown>
+
+export interface IJsFixLogger {
+  info: (message: string, fields?: JsFixLogFields) => void
+  warning: (message: string, fields?: JsFixLogFields) => void
+  debug: (message: string, fields?: JsFixLogFields) => void
+  error: (e: Error, fields?: JsFixLogFields) => void
+}
+
+export abstract class JsFixLoggerFactory {
+  public abstract logger (type: string, context?: JsFixLogFields): IJsFixLogger
+}
+```
+
+Non-breaking, and worth being precise about why: TypeScript lets an implementation declare
+*fewer* parameters than the interface, so a client's existing `info(message: string)` still
+satisfies it, and their `JsFixLoggerFactory` subclass keeps working while ignoring the new
+argument. Callers are unaffected either way.
+
+`error` gains fields as well as the others. It currently takes only an `Error` — no
+message, no context — and those 15 sites are the ones most wanted at 3am.
+
+**Bound context, not a `child()` logger.** It fits the 29 construction sites, and it ports:
+a `child()` would put an object lifecycle in the contract. See
+[porting](#porting-to-cspurefix).
+
+### Not breaking the existing line
+
+The engine has been live for years; people have grep and scrapers built on the current
+output. That output must not move.
+
+```
+2026-08-16T14:14:56.102Z [skeleton_server:skeleton-client:FixSession] info: peer sent ...
+```
+
+The shipped format names exactly four fields:
+
+```ts
+printf(info => `${info.timestamp} [${info.type}] ${info.level}: ${info.message}`)
+```
+
+So as long as context and per-call fields go into the winston info object as **separate
+keys**, that printf never sees them. Verified: the same call with and without the new keys
+renders byte-identically apart from the millisecond.
+
+Three rules follow, and each has bitten already:
+
+1. **`type` stays verbatim at all 29 sites.** It cannot be composed from the fields — the
+   sites have no single rule between them (bare literals `'acceptor'`, one-part
+   `` `${this.me}` ``, two-part `` `${name}:TcpInitiator` ``, three-part
+   `` `${me}:${peer}:FixSession` ``, and one dotted `` `${type}.${t}` ``). Deriving it
+   means per-site rules, which is the same edit with more ways to get a string subtly
+   wrong. The resulting duplication between `type` and the fields is the price of not
+   breaking grep, and it is cheap. In a JSON rendering `type` is simply dropped, because
+   `component` and `app` carry it properly.
+2. **Fields are namespaced, never spread.** They travel under `context` and `fields` keys
+   and the *formatter* decides how to flatten. Spreading at the top level lets a caller's
+   `{ message: … }` clobber the real message. It also keeps the contract
+   transport-agnostic, which is what makes it port.
+3. **A winston format function must mutate `info`, not return a new object.** Returning a
+   fresh object drops `Symbol.for('level')` and winston discards the record silently — no
+   output, no error. Found the hard way while proving the above.
+
+### The default does not change
+
+`WinstonLogger.consoleOptions()` does **not** emit JSON today — it is a `printf` rendering
+prose, so "winston gives JSON for free" is not true as things stand. JSON arrives as a new
+opt-in `ecsOptions()`, and the console default stays exactly as it is. Not only for this
+release: changing a default log format is a major-version event at best and probably never
+worth it.
+
+`WinstonLogger.plain()` is message-only and used for FIX wire logs. That is a replay
+artifact, not structured-log material, and stays as it is.
+
+Two things make "same output after upgrade" a guarantee rather than a promise:
+
+- **A golden-output test** rendering lines through `consoleOptions()` with and without
+  context and fields, asserting the strings are identical. Anyone later improving
+  `appFormat` then fails a test instead of breaking a customer's scraper.
+- **The fields commit rewords no messages.** Mixing the two makes a before/after log diff
+  unreadable. Keep them separate so the diff at a given level is genuinely empty.
+
+### Field names
+
+ECS names where they exist — `service.name`, `log.level`, `error.message`,
+`error.stack_trace` — and everything engine-specific under `fix.*`. Context keys are plain
+words; the formatter applies the prefix.
+
+| context key | rendered | meaning |
+| --- | --- | --- |
+| `component` | `fix.component` | the class, e.g. `FixSession`, `TcpAcceptor` |
+| `app` | `fix.app` | application name from the session description |
+| `peer` | `fix.peer` | counterparty comp id, once known |
+| `role` | `fix.role` | `initiator` or `acceptor` |
+
+This is the metric prefix decision applied to logs: `fix.msg_type` in a log line,
+`fix_messages_total{msg_type}` in a metric — same words, each in its own convention's
+punctuation, so one dashboard serves both engines.
+
+`fix.session` in the canonical `BeginString-Sender-Target` form is deliberately **not** in
+the table above. A wildcard acceptor does not know its peer until bind time, so the field
+would be absent or wrong on exactly the sessions that need it most. It arrives with the
+metrics work, where `SessionId` is properly to hand — `AsciiSession` already rebuilds its
+logger at bind for this reason, so the seam is there.
+
+Rendered, one call site gives either:
+
+```
+2026-08-16T14:17:31.864Z [skeleton_server:skeleton-client:FixSession] info: peer sent ResetSeqNumFlag=Y with seqNum=1, weAlsoReset=true
+```
+
+```json
+{"@timestamp":"2026-08-16T14:17:59.278Z","log.level":"info","service.name":"jspurefix",
+ "message":"peer sent ResetSeqNumFlag=Y with seqNum=1, weAlsoReset=true",
+ "fix.component":"FixSession","fix.app":"skeleton_server","fix.peer":"skeleton-client",
+ "fix.reset_seq_num_flag":true,"fix.seq_num":1,"fix.we_also_reset":true}
+```
+
+## Packaging boundary
+
+`express` and `axios` are **already** in `dependencies`, which is the unfinished half of
+[#91](https://github.com/TimelordUK/jspurefix/issues/91): every consumer installs them
+whether or not they are used. A diagnostics web server is exactly how that happens a
+second time, and the fact that express is already present is what will make it tempting.
+
+Proposed:
+
+- **core** — snapshot, sink interface, counters. No new dependencies at all.
+- **exporters** — Prometheus text format is about fifty lines and needs nothing;
+  OpenTelemetry needs `@opentelemetry/api` as an **optional peer** dependency.
+- **diagnostics endpoint** — a separate package, or optional peer. It should also be off
+  by default and bind to loopback unless told otherwise: an unauthenticated page
+  describing your counterparties and sequence numbers is not something to open by
+  accident.
+
+## Porting to cspurefix
+
+The contract above is deliberately language-neutral. Points where the runtimes genuinely
+differ, and the spec should not pretend otherwise:
+
+- **GC and heap.** V8 exposes collections through `perf_hooks`; the CLR has its own
+  counters. Do not invent a common schema — report each natively and let the dashboard
+  hold two panels.
+- **Sink interface.** A plain interface with a no-op implementation in both; no
+  `EventEmitter` in the contract, since that would not port.
+- **Logging shape.** .NET expresses this differently: bound context is `BeginScope`, and
+  per-call fields come from message templates (`"session {SessionId} up"`) rather than a
+  separate object. Both can emit identical JSON, but they are not the same API. The
+  portable contract is *bound fields plus per-call fields*, and each engine renders it
+  idiomatically — forcing the TypeScript shape onto `Microsoft.Extensions.Logging` would
+  produce something unidiomatic that nobody on that side would want to use. What must
+  match is the emitted field names, not the call signature.
+- **Histograms.** Prometheus histograms need fixed buckets. Agree the bucket boundaries
+  here so latency panels are comparable between engines rather than merely similar.
+
+## Decisions still open
+
+1. Per-session snapshot with registry rollup, or one flat structure?
+2. Is `session` a metric label, and if so how is cardinality bounded?
+3. ~~Does structured logging land first, on its own?~~ **Yes** — see
+   [structured logging](#structured-logging). Bound context on the factory rather than a
+   `child()` logger, `type` kept verbatim, console format unchanged.
+4. Histogram buckets for parse and encode duration.
+5. Does the diagnostics endpoint live in this repository at all, or its own package? It
+   needs no framework — `node:http` and about a hundred lines — which is what stops
+   `express` being reached for a second time.
+6. Sampling: are parse timings measured on every message, or 1-in-N once throughput is
+   high enough for the measurement itself to show up in the measurement?
+7. Does the engine own the ECS field names, or hand people a formatter to remap? Someone
+   with an existing Splunk convention will want the latter.
+8. When per-call fields land, do the ~25 chosen lines also get their message text left
+   alone permanently, so the prose and the fields never drift apart?

package/docs/scattered-components.md

@@ -0,0 +1,280 @@
+# Scattered components — analysis and plan of attack
+
+**Status: analysis, with phase 0 landed.** Nothing in the parser has changed. This note
+sets out a generalisation of the fragmented component repair already shipped, why it is
+believed tractable, what it would cost, and the order the work would land in. It is
+written to port — the same shape applies to cspurefix.
+
+There is no urgency behind it. The library has been in use for eight years and exactly one
+counterparty has produced the shape that motivated the existing repair. This is the
+argument being written down while it is fresh, not a queue-jumping defect.
+
+## Where this starts
+
+A component is a fiction of the dictionary. Nothing goes on the wire to say one has begun
+or ended, so the segment parser infers a component's extent from *position*: a tag falling
+between the first and last tag attributed to a component belongs to it. Every well behaved
+counterparty emits a component's tags adjacently and the inference holds.
+
+One did not. It sent a message in which the `Instrument` tags arrived in two runs with
+unrelated fields between them, and the positional model could not represent that. The fix,
+ported from cspurefix, was to stop describing such a component by a range and describe it
+by the actual tags and their positions — `SegmentView` in
+[`src/buffer/segment/segment-view.ts`](../src/buffer/segment/segment-view.ts), built by
+`TagIndex` and attached in `AsciiSegmentParser.fragments()`. Applications see no
+difference; `MsgView` was already position-agnostic.
+
+That repair is deliberately narrow. It fires only for a component at depth 1 of the
+message, only when the same component is re-entered after being exited, and only once per
+component name.
+
+## The general shape
+
+The narrowness is the question. If `Instrument` can arrive scattered at the top level of a
+message, nothing stops `SecurityXLinkage` arriving scattered *within* a scattered
+`Instrument`, and nothing stops that recursing. Each level is a dictionary fiction with no
+wire representation, so each level has the same freedom. Russian dolls, laid out flat.
+
+The condition under which this is even possible is worth stating exactly, because it is
+also the condition that makes it solvable:
+
+> Contiguity is only forced where framing exists, and framing only exists where it is
+> needed.
+
+A repeating group is the sole self-describing construct in tag-value FIX — `NoXXX` plus a
+fixed delimiter tag. That is the one place the wire *requires* a bounded, ordered run.
+Everywhere else the layout is convention.
+
+The complementary half is what makes the general case tractable:
+
+> Within one level of a message, a component occurs at most once.
+
+If a sender needs it twice there is no mechanism but a group. So tag ambiguity — *which*
+`Instrument` does this tag belong to — arises only inside groups, and groups are exactly
+where scattering cannot happen. **Scattering and ambiguity are disjoint.** Where position
+stops being evidence, tag identity starts being sufficient.
+
+## The model
+
+`SegmentDescription` presently conflates two things: identity of a node in the dictionary
+tree, and a contiguous `[startPosition, endPosition]` interval. Containment is *inferred*
+from the interval — `contains()`, feeding `Structure.firstContainedWithin` and
+`MsgView.resolveChild`.
+
+The generalisation: a segment owns an **ordered set of positions**, and an interval is the
+compressed representation of the dense case. Containment becomes set inclusion. Stated in
+one line:
+
+> The moment position stops implying structure, the parent/child edges have to be
+> materialised instead of re-derived from offsets.
+
+`SegmentView` took the first step — positions instead of a range. It did not take the
+second, which is why it cannot nest.
+
+There is precedent in the codebase already. Group instances are not really intervals
+either: `SegmentDescription.getInstance` derives an instance's bounds by carving the parent
+with `delimiterPositions`. A segment whose extent is computed rather than assigned is a
+shape the engine already carries.
+
+### Own tags
+
+Define, for a dictionary set `S`, its **own tag set** `O(S)`: every simple tag reachable
+from `S` through components *without crossing a group boundary*, plus the `NoXXX` tag of
+each directly reachable group — but not the group's interior tags.
+
+This is the distinction `flattenedTag` does not draw, and it is the crux. A group's members
+are framed and are never evidence about the level above. `O(S)` is implemented as
+`FragmentSafety.ownTags`.
+
+It also subsumes a special case the current code carries. A component that wraps a single
+group owns exactly `{NoXXX}` — non-empty, and locatable — so `TagIndex.isComponentGroupWrapper`
+stops needing to exist.
+
+### The claim
+
+```
+claim(S, U):                       # U = ordered position set; root: the message body
+  for p in U with tag(p) ∈ O(S):
+    if tag(p) is NoXXX for group G:
+      run ← maximal delimiter-framed extent from p     # contiguous, guaranteed
+      assign run to G; for each instance i: claim(instance_i, run_i)
+    else:
+      assign p to the child of S owning tag(p)
+  for each component child C of S:
+    claim(C, positions assigned to C's subtree)        # ← universe restriction
+```
+
+Recursion is over the dictionary tree, with the universe narrowing at each step. The doll
+falls out for free: `SecurityXLinkage` is claimed from `Instrument`'s claimed positions,
+not from the message. Group interiors are consumed as whole runs before any component
+claim sees them, so framed tags cannot leak upward.
+
+When everything is contiguous this degenerates to what the linear parser already computes.
+
+## What the current code does not do
+
+Five things, roughly in order of how likely each is to be met:
+
+1. **`TagIndex.getInstance` claims by `flattenedTag` across the whole message.**
+   `flattenedTag` is transitive through groups, so a fragmented component containing a
+   group would claim every member tag in the message. `isComponentGroupWrapper` guards
+   this, but only for single-field wrappers. Universe restriction fixes it structurally.
+2. **Detection is depth-1 only.** `exitedDepth1Components` is keyed by bare name and only
+   populated at `structureStack.length === 1`. Keying by stack path registers re-entry at
+   any depth.
+3. **Shadow fragments survive.** Repair attaches a view to one description; the other
+   fragments keep their original ranges and still participate in `boundLayout`, so a
+   nested lookup can be handed the wrong one.
+4. **`contains` is interval arithmetic.** A scattered parent's hull can swallow unrelated
+   siblings, so `firstContainedWithin` can answer a lookup that should be null.
+5. **The one-instance-per-name guard is right for the wrong reason.** It reads as a
+   limitation; it is the at-most-once property above. Worth a comment before someone
+   "fixes" it.
+
+## Blast radius
+
+Measured, not estimated. Outside `src/buffer/segment/`, everything that *reads* an
+interval:
+
+| Site | What it does |
+| --- | --- |
+| `msg-view.ts` `allStrings` | already branches on `segmentView` |
+| `msg-view.ts` `binarySearch` | already branches on `segmentView` |
+| `msg-view.ts` `stringify` | display only |
+| `segment-description.ts` `contains` | feeds `Structure` and `resolveChild` |
+
+Everything else touching positions is a writer (`ascii-parser`, `fixml-parser`) or
+diagnostics (`SegmentSummary`). Two of the four read sites already carry the sparse branch,
+so this replaces an `if` rather than introducing a concept.
+
+Unchanged: the dictionary layer, the encoder, session and transport, every generated type,
+and the public `MsgView` API. The FIXML path builds structure from the XML tree and never
+has the problem.
+
+## Plan of attack
+
+Phasing works here because three of the four pieces have an existing oracle. Only one phase
+is irreducible.
+
+### Phase 0 — does the assumption hold? *(landed)*
+
+The claim attributes by tag identity, which is sound only while siblings own disjoint tags.
+FIX's numbering conventions — `Underlying*`, `Leg*`, `NestedParties2/3` — exist to
+guarantee that, but a house dictionary is under no obligation to honour them. So measure it
+before building on it.
+
+[`src/dictionary/fragment-safety.ts`](../src/dictionary/fragment-safety.ts) walks every
+message, component and group instance body in a dictionary and reports sibling tag
+collisions. It is exported, so anyone with a house dictionary can run it against their own.
+
+The QuickFIX rendering is the one that matters. A broker onboarding pack is now almost
+always a prepared QuickFIX XML rather than a reference to a spec version, so that is the
+flavour the engine actually meets. Every QuickFIX dictionary shipped here is covered, plus
+the repository renderings for comparison:
+
+| Dictionary | Form | Collisions |
+| --- | --- | --- |
+| `qf42`, `qf43`, `qf44` | QuickFIX | 0 |
+| `fix5-mod` (modified test dialect) | QuickFIX | 0 |
+| `repo44`, `repo50sp2` | repository | 0 |
+| `qf50sp2` | QuickFIX | **57** |
+
+(`data/FIX44-MD.xml` is byte-identical to `data/FIX44.xml`, so it is not a separate sample.)
+
+Every one of the 57 is the same root cause: the QuickFIX `FIX50SP2.xml` dictionary puts
+`Currency(15)` inside `Instrument` *and* declares it again beside the component, in
+`NewOrderSingle`, `ExecutionReport`, `Quote` and 54 other places. 14 of the 57 are inside
+group instance bodies rather than at message level — `NewOrderList.ListOrdGrp.NoOrders` and
+the like — which is exactly where a recursive repair would be working. The repository
+rendering of the same FIX version does not do it, so this is an authoring difference rather
+than a property of FIX 5.0.
+
+Two conclusions, and the second is the uncomfortable one.
+
+The positional parser is untroubled by any of this — whichever `15` falls within
+`Instrument`'s span is `Instrument`'s. A repair attributing by tag identity cannot tell. So
+`Instrument` has to be refused for repair in those sets specifically, which argues for the
+collision map being consulted per set at repair time rather than being a whole-dictionary
+yes/no.
+
+And the pattern that produced it — a field declared at message level that a component
+already carries — is ordinary dictionary authoring, not an exotic mistake. A broker adding
+fields to a message to suit their own flow is *more* likely to write it than a spec
+committee is, and the broker dialects are the ones most likely to also send the scattered
+messages this whole exercise is about. On the evidence here the collision map is not a
+defensive nicety at the edge of the design; it is load-bearing.
+
+### Phase 1 — the accessor seam
+
+Give `SegmentDescription` `owns(pos)` and `forEachPosition(cb)`; move the four `msg-view`
+sites onto them; dense implementation loops the range without allocating. Zero behaviour
+change, existing suite is a complete oracle. Ships alone, and afterwards every later change
+is local rather than cross-cutting.
+
+### Phase 2 — the index, behind the existing call site
+
+Rebuild `TagIndex` as the recursive, universe-restricted claim above, then keep
+`getInstance(name)` as a thin wrapper calling `claim(rootSet, wholeMessage)`. Same entry
+point, same callers, new engine.
+
+Not purely behaviour-preserving: universe restriction fixes the `flattenedTag` over-claim
+as a side effect, so it needs its own test rather than "existing fixtures still green".
+
+**The detail that makes this not-wasted work:** write `claim` with the `universe` parameter
+and a per-node result from day one, even though the phase 2 caller only ever passes the
+whole message and collapses the result. The signature carries the generality before a
+caller needs it. Built instead as a straight optimisation of today's global sweep, phase 3
+rewrites it.
+
+### Phase 3 — detection, repair at depth, containment edges
+
+Atomic. Path-keyed detection without repair-at-depth never fires; repair-at-depth without
+detection is dead code; and repair that leaves containment alone lets nested lookups pick
+up siblings that merely fall inside a scattered parent's hull.
+
+Since repair from the shallowest fragmented ancestor recursively fixes everything beneath,
+an over-eager detector is wasteful rather than wrong. Bias it toward false positives.
+
+## Why not replace the linear parser outright
+
+One algorithm would be cleaner than two. Three reasons against:
+
+1. **It is the hot path.** The linear stack pass is single-pass and allocation-light.
+   Claim-based parsing needs sorted indices and per-node universes.
+2. **The linear pass is what detects malformation.** A pure claim algorithm accepts almost
+   anything and produces a plausible object. `SegmentType.Gap` and unknown-tag handling
+   depend on stream order. Losing that turns protocol errors into silently wrong data,
+   which is strictly worse than the bug being fixed.
+3. **Repair is self-limiting.** Only flagged subtrees pay.
+
+## Honest limits
+
+**Repair is exact; detection is heuristic.** Given the at-most-once property, the claim is
+sound once pointed at a subtree. Deciding *which* subtrees to point it at relies on
+observing re-entry in the stack machine, and at depth the machine may unwind past the
+nested node before the stray tag arrives, so re-entry surfaces at whatever ancestor still
+claims the tag. That part wants a test matrix rather than an argument.
+
+**Round-tripping.** A repaired message re-encodes canonically contiguous. Byte-for-byte
+replay of the original is gone for those messages; retain the raw body where that matters.
+
+**Malformed groups are not scattering.** A broken delimiter run is corruption and must stay
+a hard parse error. The two look similar on the wire and conflating them would convert real
+protocol errors into wrong objects.
+
+## Open decisions
+
+1. Should a collision found by phase 0 refuse repair for the colliding pair only, or for
+   the whole set containing them? The `qf50sp2` result argues for the pair — refusing all
+   of `NewOrderSingle` over one contended tag would give up a great deal for very little.
+2. Should the collision map be computed once at dictionary load, or lazily on first repair?
+3. Should a dictionary with collisions say so at load time? An operator onboarding a broker
+   dialect would want to know before a live session does, and phase 0 is cheap enough to
+   run eagerly — but it is only meaningful to anyone who has read this note.
+4. Does a repaired segment report its hull or its true extent through `SegmentSummary`,
+   which is a diagnostic surface people read?
+5. Should scattering be *reported* at all — a counterparty emitting non-adjacent components
+   is worth knowing about even when the repair succeeds, and today it is silent.
+6. Is `stringify` on a scattered segment supposed to render wire order or claimed order?
+7. Does cspurefix take the same phase split, or land phases 1 and 2 together given its
+   different `MsgView`?