@tikoci/centrs 0.1.4 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/CHANGELOG.md +417 -1
  2. package/GLOSSARY.txt +40 -1
  3. package/README.md +16 -8
  4. package/commands/AGENTS.md +7 -4
  5. package/commands/api/AGENTS.md +13 -1
  6. package/commands/api/README.md +50 -10
  7. package/commands/api/examples.md +100 -2
  8. package/commands/devices/README.md +10 -1
  9. package/commands/devices/examples.md +33 -2
  10. package/commands/execute/README.md +68 -25
  11. package/commands/execute/examples.md +168 -7
  12. package/commands/explain/README.md +1206 -155
  13. package/commands/explain/examples.md +99 -3
  14. package/commands/mcp/README.md +9 -5
  15. package/commands/mcp/examples.md +2 -2
  16. package/commands/retrieve/examples.md +20 -0
  17. package/commands/terminal/examples.md +7 -1
  18. package/docs/CLI.md +3 -3
  19. package/docs/CONSTITUTION.md +83 -21
  20. package/docs/MATRIX.md +89 -52
  21. package/docs/errors/cdb/create-failed.md +20 -0
  22. package/docs/errors/cdb/file-exists.md +23 -0
  23. package/docs/errors/input/invalid-target.md +25 -3
  24. package/docs/errors/routeros/invalid-value.md +3 -2
  25. package/docs/errors/transport/cancel-unacknowledged.md +24 -0
  26. package/docs/errors/validation/syntax.md +29 -4
  27. package/package.json +13 -2
  28. package/src/api-fanout.ts +17 -4
  29. package/src/api.ts +346 -75
  30. package/src/cli/api.ts +1 -1
  31. package/src/cli/btest.ts +1 -1
  32. package/src/cli/devices.ts +16 -3
  33. package/src/cli/explain.ts +1 -1
  34. package/src/cli/terminal.ts +1 -1
  35. package/src/core/envelope.ts +53 -0
  36. package/src/core/error-catalog.ts +16 -1
  37. package/src/core/execute-command.ts +455 -0
  38. package/src/core/inspect.ts +58 -26
  39. package/src/core/routeros-errors.ts +66 -0
  40. package/src/core/version.ts +4 -0
  41. package/src/core/yaml.ts +86 -0
  42. package/src/data/winbox-cdb-write.ts +22 -8
  43. package/src/devices.ts +142 -28
  44. package/src/execute-fanout.ts +19 -5
  45. package/src/execute.ts +247 -295
  46. package/src/explain/arg-tokens.ts +62 -25
  47. package/src/explain/args.ts +398 -52
  48. package/src/explain/blocks.ts +54 -11
  49. package/src/explain/brace-tokens.ts +4 -3
  50. package/src/explain/catalog.ts +42 -44
  51. package/src/explain/coordinates.ts +159 -0
  52. package/src/explain/defects.ts +2 -1
  53. package/src/explain/escape-tokens.ts +77 -0
  54. package/src/explain/highlight-projection.ts +308 -0
  55. package/src/explain/menus.ts +9 -5
  56. package/src/explain/operator-tokens.ts +13 -7
  57. package/src/explain/operators.ts +27 -20
  58. package/src/explain/path-tokens.ts +1 -1
  59. package/src/explain/pathresolve.ts +358 -113
  60. package/src/explain/quoted-string.ts +110 -20
  61. package/src/explain/scope-brace.ts +175 -22
  62. package/src/explain/segment.ts +429 -21
  63. package/src/explain/separator.ts +187 -0
  64. package/src/explain/split-owner.ts +194 -0
  65. package/src/explain/string-tokens.ts +14 -4
  66. package/src/explain/symbol-values.ts +209 -173
  67. package/src/explain/symbols.ts +21 -1
  68. package/src/explain/transport.ts +17 -0
  69. package/src/explain/value-tokens.ts +15 -12
  70. package/src/explain/verbsplit.ts +165 -23
  71. package/src/explain/write.ts +47 -6
  72. package/src/explain.ts +522 -72
  73. package/src/index.ts +21 -2
  74. package/src/mcp/server.ts +2 -1
  75. package/src/mcp/tools.ts +1 -1
  76. package/src/offline-gate.ts +148 -0
  77. package/src/protocols/adapter.ts +12 -1
  78. package/src/protocols/mac-telnet-console.ts +6 -4
  79. package/src/protocols/native-api.ts +29 -3
  80. package/src/resolver/target.ts +32 -12
  81. package/src/retrieve-fanout.ts +1 -1
  82. package/src/retrieve.ts +18 -63
  83. package/src/settings.ts +2 -1
  84. package/src/transfer-fanout.ts +1 -1
  85. package/src/transfer.ts +1 -1
@@ -9,20 +9,13 @@ a path, with completion-style candidates for building the right command.
9
9
  validation): the cheap, safe knowledge tier in front of the runners
10
10
  (`execute` / `api` / `retrieve`).
11
11
 
12
- Status: `designed` over `rest-api` and `native-api` — the transports the live
13
- inspection probes ride; every other cell is `—`. The offline mode is
14
- protocol-connection-less, and **`centrs explain '<input>'` runs today** (issues #202b
15
- and #202c): the row stays `designed` because the grid tracks protocol cells and
16
- offline occupies none. See `docs/MATRIX.md` for the row. A first design round
17
- (2026-07-19, recorded in #90) settled the surface shape and the offline model;
18
- the **phase-0 canonicalization grounding lab (#185) is complete and this spec
19
- is now ratified** — every ratification-gating question was answered with cited
20
- evidence (see [Phase-0 ratification](#phase-0-ratification-185)). The findings
21
- folded in below; a few of them **amend** the surface (tristate `containsWrite`,
22
- an `ambiguous` canonicalizer verdict, a defect-region diagnostics contract, a
23
- byte-count-preserving coordinate rule, the native-api `:parse` readout). Flag
24
- names and sizing thresholds stay provisional — those are implementation opens,
25
- not ratification blockers.
12
+ Status: offline baseline `verified`; live probes over `rest-api` and `native-api`
13
+ `designed`. **`centrs explain '<input>'` runs today**, without a router or CDB.
14
+ [`docs/MATRIX.md`](../../docs/MATRIX.md#offline-analysis) tracks these separately;
15
+ [#90](https://github.com/tikoci/centrs/issues/90) indexes the remaining tasks.
16
+ The spec is ratified on the [phase-0 grounding lab](#phase-0-ratification-185).
17
+ The current finish line is [Offline baseline acceptance](#offline-baseline-acceptance),
18
+ not completion of the live protocol cells.
26
19
  Load-bearing rules — envelope, errors, settings precedence, identity,
27
20
  validation, protocol selection — live in
28
21
  [`docs/CONSTITUTION.md`](../../docs/CONSTITUTION.md).
@@ -62,6 +55,10 @@ validation, protocol selection — live in
62
55
 
63
56
  ## Where `explain` sits
64
57
 
58
+ - **Shared analysis across transports.** The offline result is independently
59
+ useful to libraries, agents, and editors. Its facts can later accompany
60
+ other commands' envelopes, irrespective of their transport; the rich
61
+ analysis is not currently applied as a runner policy gate.
65
62
  - **Never executes, never mutates.** All live probes (`/console/inspect`
66
63
  highlight/completion/syntax/child and `:put [:parse …]`) are read-only
67
64
  inspection. Execution-based probes (required-argument discovery via
@@ -84,6 +81,16 @@ validation, protocol selection — live in
84
81
  to **support one**: hover, diagnostics, completion, and semantic-token needs
85
82
  of `lsp-routeros-ts`/tikbook are first-class consumers of the export shape.
86
83
 
84
+ Normalized source structure is the foundation for possible configuration
85
+ checks. Preserve command order, selectors, source locations, literal/computed
86
+ value distinctions, and unknowns; an input fragment does not establish final
87
+ device state. Script-variable bindings are not yet an object-reference model
88
+ for interfaces, bridges, or other configuration entities. Security advice,
89
+ rule collections, and a browser/Monaco inspector are possible consumers, not
90
+ new requirements to implement those products during the offline baseline.
91
+ quickchr owns the CHR provisioning/test machinery; rosetta supplies docs and
92
+ reference knowledge. This direction adds no runtime dependency on either.
93
+
87
94
  ## Canonicalization is the core
88
95
 
89
96
  The engine is a **grown `canonicalizeExecuteCommand`** (decision: grow the
@@ -165,16 +172,16 @@ today's path/verb/args split and script-vs-structured gate:
165
172
  The *container* half of that pair is since decidable. `src/explain/menus.ts`
166
173
  (#207) bakes only the menu-vs-command node type — not a schema — and it is
167
174
  version-less because the pinned trees show no `dir`↔`cmd` flip across
168
- 7.10.2 → 7.24rc2 or across architectures. So `/ip/route` now resolves as
175
+ 7.10.2 → 7.25beta5 or across architectures. So `/ip/route` now resolves as
169
176
  `navigation` (#210). The floor is unmoved: the table is deliberately
170
177
  incomplete, absence from it decides nothing, and every rule above it still
171
178
  refuses rather than guesses.
172
179
 
173
180
  The *command* half is what #228 addressed, by adding a second first-order
174
181
  source rather than a schema. MikroTik's CLI Reference is generated from the
175
- definition structs, and measured against the same four pinned trees it has
176
- **zero navigation-vs-command contradictions** across 968 exactly-matching
177
- paths, three RouterOS versions and two architectures; all but one of the 102
182
+ definition structs, and measured against the same six pinned trees it has
183
+ **zero navigation-vs-command contradictions** across 959 exactly-matching
184
+ paths, four RouterOS versions and two architectures; all but one of the 94
178
185
  published paths absent from every tree carry a `package`/`conditions`/`syscap`
179
186
  gate — their own or an ancestor's — that predicts the absence.
180
187
  `src/explain/catalog.ts` unions the two with per-entry provenance, which is
@@ -198,6 +205,33 @@ today's path/verb/args split and script-vs-structured gate:
198
205
  inventory reshape fails loudly instead of shrinking the table (#285,
199
206
  tikoci/rosetta#137).
200
207
 
208
+ **Source refresh (#335, accepted 2026-09-13).** The inspect pins retain the
209
+ 7.10.2 and 7.16 history anchors, then pair x86 and arm64 for both current
210
+ stable 7.24.2 and prerelease 7.25beta3. Pairing architectures makes additions
211
+ attributable to version rather than to an architecture swap while preserving
212
+ the hardware-only menu floor. Reviewed independently, the inspect union moves
213
+ from 615 to 617 menus: 7.24.2 removes `/ip/ipsec/key/qkd`, and 7.25beta3 adds
214
+ `/interface/ethernet/pon`, `/interface/xfrm`, and `/ip/ssh/known-hosts`.
215
+ Separately, the current CLI Reference moves from 1,070 pages / 1,077 entries
216
+ to 1,053 / 1,060: it adds
217
+ `/interface/ethernet/switch/rule/reset-counters`, removes the five
218
+ `/interface/bridge/msrp` paths plus `/root`, moves 12 paths from `both` to
219
+ `inspect`, and changes 16 published gate strings. Those current-only
220
+ publication changes are accepted: none introduces a kind contradiction. The
221
+ coherent generated catalog has 1,120 paths: 959 `both`, 67 `inspect`, and 94
222
+ `published`; it still reports zero navigation-vs-command contradictions.
223
+
224
+ **Source refresh (accepted 2026-09-27).** The two current pins advance to the
225
+ channels then live — stable 7.24.2 → 7.24.4 and development 7.25beta3 →
226
+ 7.25beta5 — both still architecture-paired, both history anchors untouched.
227
+ The refresh is a structural no-op: the inspect union holds at 617 menus and
228
+ the catalog at 1,120 paths, with no path added or removed and no kind change.
229
+ What moved is current-only publication: the CLI Reference grows from 1,053
230
+ pages / 1,060 entries to 1,055 / 1,062, which documents
231
+ `/interface/ethernet/pon` and `/ip/ssh/known-hosts` for the first time and so
232
+ moves both from `inspect` to `both` — 961 `both`, 65 `inspect`, 94
233
+ `published`. Zero navigation-vs-command contradictions, unchanged.
234
+
201
235
  There are real unknowns in how far offline parsing can go (expression
202
236
  grammar, scope fidelity vs `:parse`, `[]`-nesting corner cases). Grounding
203
237
  those unknowns was **phase 0** (#185); its findings are folded in throughout
@@ -262,7 +296,10 @@ command accepts.
262
296
  Offline mode does ship two generated **structure** tables, baked at build time
263
297
  from pinned sources and read as ordinary closed lists. They may say whether a
264
298
  path is navigation or a command; they never describe what a command accepts.
265
- Absence from either abstains and never rejects.
299
+ Absence from either abstains and never rejects. Regenerate with
300
+ `bun run explain:menus` / `bun run explain:catalog`; `--check` drift-gates both
301
+ in QA. Do not hand-edit them — each file's header carries its source pin and
302
+ rationale.
266
303
 
267
304
  - `src/explain/menus.ts` (#207) — container paths from pinned restraml
268
305
  `/console/inspect` trees. **Read today** to confirm navigation, on its own by
@@ -499,7 +536,8 @@ Standard envelope (constitution: result envelope); `data` sketch:
499
536
  severity channels. **Malformed input carries a defect *region*** (Q14): each
500
537
  diagnostic points at the byte span of its defect, and the detectable classes
501
538
  are {unbalanced delimiter, unterminated string, invalid escape, invalid sigil,
502
- invalid unquoted hash, BOM, non-ASCII, over-depth nesting}. **Two classes named
539
+ invalid unquoted hash, invalid apostrophe (#355), BOM, non-ASCII,
540
+ over-depth nesting}. **Two classes named
503
541
  in the phase-0 draft
504
542
  of this list are deliberately deferred** (maintainer decision, #192), because
505
543
  each would require offline to assert something it cannot prove:
@@ -570,19 +608,24 @@ and its phase is named below.
570
608
  all: a `[…]`/`(…)`/`$x`/`{…}` value, an escape this phase does not decode, a
571
609
  left-hand side that is not a RouterOS name, a token split by a continuation.
572
610
  Never partially read, because a dropped argument silently changes what a
573
- rendered `curl` DOES. Two refusals are not lexical at all, but
574
- places where centrs's two readers disagree about an **unquoted** value: a
575
- `'`, which RouterOS treats as an ordinary character and the locked execute
576
- gate treats as a quote (`comment=it's` is `it's` here and `its` to the
577
- gate); and a `\f`/`\v`, which the gate splits tokens on (JavaScript `\s`)
578
- and every explain module does not (ASCII whitespace). The device-correct
579
- reading is the analysis's in both, but the gate cannot be corrected — so
580
- phase 1 publishes neither rather than putting two confident values in one
581
- result. Inside a `"…"` run the two agree and nothing is refused. The
582
- advisory value-anchor view is intentionally wider only for exact array
611
+ rendered `curl` DOES. One refusal is not lexical at all, but a place where
612
+ centrs's two readers disagree about an **unquoted** value: a `\f`/`\v`,
613
+ which the retired execute gate split tokens on (JavaScript `\s`) and every
614
+ explain module does not (ASCII whitespace). The device-correct reading is
615
+ the analysis's, but that gate could not be corrected — so phase 1 publishes
616
+ neither rather than putting two confident values in one result. Inside a
617
+ `"…"` run the two agree and nothing is refused.
618
+
619
+ The advisory value-anchor view is intentionally wider only for exact array
583
620
  literals: a non-empty `{…}` value or a parenthesized run with a depth-zero
584
621
  comma is locatable and hints `array`, while the strict REST view continues
585
622
  to refuse the whole list.
623
+
624
+ An unquoted `'` used to be a second such disagreement, described here as a
625
+ character RouterOS treats as ordinary and the gate treats as a quote. That
626
+ reading was wrong on both halves and #355 settled it: `'` is not a RouterOS
627
+ token at all, so `comment=it's` is REJECTED by the device rather than
628
+ carrying either value, and the quote-counting gate that disagreed is gone.
586
629
  - **A token is read but carries no `value`** — the list still reads. The token
587
630
  is delimited and classified; only its literal value is unknowable, as for
588
631
  the positional in `:log info "result: $[…]"`. **`value` absent means there
@@ -882,11 +925,11 @@ and its phase is named below.
882
925
  `bun run explain:value-census:readme` and gated against it by
883
926
  `bun run explain:value-census:readme:check`; the fixture itself is gated
884
927
  against a fresh corpus run by `bun run explain:value-census:check`. Of the
885
- 13,168 emitted values in a statement the strict argument lexer ALSO read, 0
928
+ 13,161 emitted values in a statement the strict argument lexer ALSO read, 0
886
929
  disagree with it on half-open byte span or decoded text, while the
887
- prefix-safe scan retains a further 6,540 values across 569 statements whose
888
- strict REST reading abstains. Of 19,708 emitted occurrences, 5,636 are array
889
- members (529 keyed, 1,147 nested inside another member) and 813 are arrays;
930
+ prefix-safe scan retains a further 6,538 values across 568 statements whose
931
+ strict REST reading abstains. Of 19,699 emitted occurrences, 5,636 are array
932
+ members (529 keyed, 1,147 nested inside another member) and 808 are arrays;
890
933
  the corpus holds no source-literal `id` example. The three structural
891
934
  counters — spans addressing bytes outside their own source, members naming a
892
935
  container that does not exist, members escaping the container they name —
@@ -1003,7 +1046,9 @@ navigated, so the context stays as unknown as it already was.
1003
1046
  (string-internal unknown/lowercase-hex/truncated — CHR `highlight` `error`
1004
1047
  `:parse` `expected message value`, grounded stable+testing #247),
1005
1048
  `bad-sigil`,
1006
- `invalid-hash`. Seven classes the device itself rejects.
1049
+ `invalid-hash`, `invalid-apostrophe` (#355, below),
1050
+ `missing-statement-separator` (#311, below). Nine classes
1051
+ the device itself rejects.
1007
1052
  - `warning` — `over-depth`, because it is centrs's own resource bound and says
1008
1053
  nothing about whether the input is legal; and an `ambiguous`/`unknown`
1009
1054
  resolution, never an error, so the default `--fail-on error` cannot fail a
@@ -1013,6 +1058,386 @@ navigated, so the context stays as unknown as it already was.
1013
1058
  fail), and `context-lost`, which reports a reading that is correct while the
1014
1059
  document's menu context was already gone.
1015
1060
 
1061
+ ### The missing statement separator (#311)
1062
+
1063
+ Two commands written with no `;` or newline between them used to read as one
1064
+ command with extra operands, and offline reported `pass`: `/ip/address add
1065
+ interface=ether1 /ip/route add gateway=192.168.88.1` published `gateway` as an
1066
+ attribute of `/ip/address add`. The rule that closes it is a known **menu path**
1067
+ in operand position followed by a console **verb** — never "a `/` after the
1068
+ verb", because a slash-shaped operand is legal and `/file remove
1069
+ /flash/skins/foo.html` lowers cleanly to `numbers=/flash/skins/foo.html`. The
1070
+ menu spelling may be slashed, spaced, or mixed (Q3 R4); the device rejects
1071
+ `/ip route add` exactly as it rejects `/ip/route add`.
1072
+
1073
+ **The device evidence is not what the report said, and the correction widened
1074
+ the rule.** #311 gave the error byte as the `=` of the trailing attribute
1075
+ (`expected end of command (line 1 column 55)`). That byte is an artefact of that
1076
+ head: `add` abbreviates `/ip/address/add`'s own `address=` argument, so the
1077
+ console reads `add gateway` as `address=gateway` and only complains at the `=`
1078
+ after it. Change the head so the second verb abbreviates nothing and the answer
1079
+ changes shape — `/system/note set note=hello /ip/route add gateway=1.2.3.4` is
1080
+ no hard reject at all; `:parse` returns IL carrying `bad parameter /ip/route
1081
+ (line 1 column 38)`, naming the second **path**. Both are refusals, and which
1082
+ one appears depends on per-menu argument names, which offline does not have. So
1083
+ the grounded claim is the negative one: across every probed spelling on CHR
1084
+ 7.24.2 and 7.23.5, RouterOS answers `bad parameter <path>` or `expected end of
1085
+ command`, and never accepts the shape. A trailing attribute is therefore **not**
1086
+ required, which #311's own example had made it look like.
1087
+
1088
+ The head-dependence goes one step further than "which wording", and it bounds
1089
+ what the diagnostic may say. On a head whose own value slot takes a bare
1090
+ positional, the second path is absorbed into that slot and the device refuses
1091
+ the **verb** instead: `:put /ip/route print` lowers to `/putmessage=/ip/route`
1092
+ and answers `bad parameter print`, and `:local a /ip/route add` lowers to
1093
+ `/localname=$a;value=/ip/route` and answers `bad parameter add`. Both are still
1094
+ refusals — `/system/script/run /ip/route`, the same shape with no verb after
1095
+ the path, lowers cleanly to `number=/ip/route` — so the offline `error` stands.
1096
+ But the message says the bytes **read as** a second command, which is centrs's
1097
+ heuristic reading, and never that the device starts one there or which token it
1098
+ will name. Every row above is pinned in
1099
+ `test/integration/explain-separator.test.ts`.
1100
+
1101
+ **The contradicted reading is withdrawn, not patched.** The statement stays
1102
+ `resolved` — `/ip/address add` really is the head, and `highlight` agrees. What
1103
+ is withdrawn is withdrawn by three separate gates, each with its own trigger:
1104
+
1105
+ 1. **The argument list.** `withoutSeparatedArguments` drops `command.args` and
1106
+ sets `arguments.read` to `false`. This is the same all-or-nothing move
1107
+ `enforceGateParity` makes for a gate/analysis disagreement, but it is a
1108
+ distinct gate on a distinct trigger — the two never consult each other. The
1109
+ device says the move is right rather than merely cautious:
1110
+ `… /ip/route add placeholder` lowers to
1111
+ `address=placeholder;;interface=ether1`, a phantom attribute the source never
1112
+ wrote, because the second run's words can abbreviate the HEAD's argument
1113
+ names.
1114
+ 2. **The invocation.** `rejectedBySeparator(runs)` sets the new
1115
+ `ExplainTransportInput.rejected`, which `classifyExplainTransport` honours
1116
+ ahead of every mapping rule and answers `unknown`. Without it the degraded
1117
+ `arguments` would fall through to `execute` and print a ready-to-paste
1118
+ `centrs execute '<router>' '<the malformed text>'` for bytes the same result
1119
+ calls a syntax error. **Unread is not rejected** — an unread argument list
1120
+ still legitimately rides the raw CLI surface, and that path is unchanged.
1121
+ 3. **The value facts from the run onward.** `values.occurrences` is an axis of
1122
+ its own, not a projection of `arguments` — the value census above pins how
1123
+ many values it retains in statements whose strict REST reading abstains — so
1124
+ gate 1 does not reach it and a cut is needed. The cut is the run's first byte, and
1125
+ the device draws the line in the same place: `:parse` reads `gateway` as the
1126
+ VALUE of `/ip/address/add`'s own `address=`, never as an attribute NAME,
1127
+ while the head's `interface=ether1` survives into the IL (as `note=hello`
1128
+ does in the `/system/note set` row). Values before the cut therefore stay;
1129
+ values reaching into or past it are dropped, and with them the `value` token
1130
+ class over those bytes.
1131
+
1132
+ **Abstentions, stated.** A menu path with no verb after it (`… /ip/route`) and a
1133
+ bare operand a menu does not accept (`… placeholder`, `/file remove a b`) also draw
1134
+ `bad parameter`, but that is the generic "no such positional" complaint;
1135
+ deciding it needs the per-menu positional list, which `catalog.ts` deliberately
1136
+ is not, and a separator is not necessarily the fix. Both abstain. The shape
1137
+ filter over a candidate path does admit a leading digit, because
1138
+ `/interface/6to4` is a real menu in both structure tables; presence in those
1139
+ tables, never the filter, is what decides. Over the 948-script pinned corpus the
1140
+ rule fires **0 times**, so no published census figure moves; the 38 corpus
1141
+ scripts the device answers `expected end of command` for are all other causes
1142
+ (line continuations, `<tab>`/`<VALUE>` placeholders, pasted shell and Python,
1143
+ stray parens), which is #203's genre bias showing.
1144
+
1145
+ **The reach limit is closed (#316).** This rule used to read
1146
+ `statement.arguments.tokens`, the all-or-nothing strict reading, so a statement
1147
+ holding a variable carried no tokens and the run in it was missed:
1148
+ `/ip/address add interface=ether1 /ip/route add gateway=$g` reported `pass`
1149
+ where its literal twin reported the separator. It now reads the skip-tolerant
1150
+ token stream described below, and both spellings report the separator — as does
1151
+ `/ip/address add interface=$x /ip/route add gateway=1.1.1.1`, where the
1152
+ undecodable token comes FIRST and a prefix-only reading would still have missed
1153
+ it. The corpus count is unchanged at **0** for the same reason it always was
1154
+ (#203's genre bias), so this buys reach rather than a number. What still stops
1155
+ the rule is an unterminated string or an unbalanced delimiter, because neither
1156
+ walk can find a boundary to resume from.
1157
+
1158
+ ### The token stream a rule reads (#316)
1159
+
1160
+ `lexArguments` is all-or-nothing by design: its consumer renders a runnable REST
1161
+ request, and a partially-read argument list changes what the rendered command
1162
+ DOES. That is the right reading for a renderer and the wrong one for a rule —
1163
+ the first token it declines discards every token already decoded, which put
1164
+ **48.2%** of the corpus's argument-bearing statements out of reach of every
1165
+ token-level rule and of the `arg` token fill.
1166
+
1167
+ `lexArgumentTokens` is a third reading over the same walk — alongside strict
1168
+ `lexArguments` and the prefix-safe `lexValueAnchors` values already use. Same
1169
+ text, same `from`, the same boundary rules and the same refusal strings,
1170
+ differing only in what a refusal does. Where the strict reading discards the list, this one publishes the token
1171
+ it could not decode with its bytes located, `undecided` set to that same reason
1172
+ and **no `value`**, then carries on to the next token. `statement.arguments` is
1173
+ unchanged and still strict; nothing that renders a command gained reach.
1174
+
1175
+ **Skip-and-continue, measured against stop-at-first-refusal.** #316 named two
1176
+ designs and said the fork was load-bearing rather than cosmetic. It is. Both are
1177
+ priced by `bun run explain:arg-reach` over the 948-script pinned corpus, from
1178
+ one implementation — design A is the tolerant token list truncated at its first
1179
+ `undecided` token, which is exactly the prefix the strict walk discards:
1180
+
1181
+ | | strict | A — prefix | B — skip |
1182
+ | --- | ---: | ---: | ---: |
1183
+ | statements reached, of 14,267 | 7,385 (51.8%) | 10,754 (75.4%) | 14,267 (100.0%) |
1184
+ | argument tokens | 16,652 | 20,481 | 31,310 |
1185
+ | missing-separator runs found | 0 | 0 | 0 |
1186
+
1187
+ A further 58 statements sit outside every column because their bytes are not
1188
+ addressable (normalized, or a span widened to the enclosing statement); the
1189
+ sweep counts them rather than dropping them silently, since `explain.ts` refuses
1190
+ those too and lexing them here would measure a reading the product does not
1191
+ offer.
1192
+
1193
+ A recovers 3,369 statements; the 3,513 statements whose FIRST interesting token
1194
+ is the undecodable one have an empty prefix and are reached only by B — so the
1195
+ cheaper design leaves slightly more behind than it recovers. The separator row
1196
+ is 0 across all three and is reported rather than buried: on this corpus neither
1197
+ design finds a run the strict reading missed, which is what #203's genre bias
1198
+ predicts for a typo class published scripts do not carry. The sweep also asserts
1199
+ what makes the A column trustworthy — that the strict tokens are the tolerant
1200
+ walk's prefix byte for byte, and that both walks name the same refusal reason —
1201
+ and fails if any statement disagrees (0 do).
1202
+
1203
+ **Where both walks still stop.** An unterminated string and an unbalanced
1204
+ `(`/`[`/`{` have no knowable end, so there is no boundary to resume from. A `;`
1205
+ is a third and deliberately so: it is a statement boundary the segmenter owns,
1206
+ and resuming after it would read the next statement's bytes as this one's
1207
+ arguments. No corpus statement reaches any of the three, and that is a property
1208
+ of the pipeline rather than luck: an unterminated string and an unbalanced
1209
+ delimiter are document defects the segmenter and resolver answer first, so no
1210
+ addressable statement with an argument list is produced at all, and a `;` is
1211
+ split into two statements before any argument is lexed. The unbalanced case is
1212
+ also the one place the two readings name a different reason, and it follows from
1213
+ what each needed: the strict walk refuses a `[…]` on sight and never asks where
1214
+ it ends, while the tolerant walk asks — because the answer is what it would
1215
+ resume from — and says `an unclosed structured argument value` when there is
1216
+ none.
1217
+
1218
+ **What it moved, including the cost.** The `arg` fill now paints the `name=` run
1219
+ of a token whose value only the device knows, so classified bytes go 69.77% →
1220
+ **71.73%** (`arg` tokens 11,438 → 16,526). On the device-agreement slice the
1221
+ bytes where the device decided and centrs abstained drop 5,540 → 4,158, and two
1222
+ whole cells (`unclassified` → `arg-dot`, `unclassified` → `arg-scope`) empty
1223
+ out. Against that, projection disagreements rise 143 → 360, and **every one of
1224
+ the 217 new bytes is the `=` of an attribute**: `arg` → `syntax-meta`, one cell,
1225
+ 358 runs. That is not a new wrong assertion. It is #293's provisional
1226
+ vocabulary — one `arg` class for both the name bytes and the `=` — meeting the
1227
+ device's context-dependent answer over 2.5x as many bytes as before, and it
1228
+ gave #264 B5 a concrete, counted reason to split the `=` into its own class.
1229
+ It did, and the relabel moved no byte coverage at all: disagreements fell
1230
+ 360 → 2 and the classified-byte census did not change by a digit.
1231
+
1232
+ **The one residue, sized so it is not mistaken for a defect.** A `\<newline>`
1233
+ continuation inside an argument leaves its token `undecided`, because the bytes
1234
+ of a continued value are not contiguous and a `valueSpan` over them would be a
1235
+ lie (`args.ts` → *What it refuses*). The conservative case is a token whose
1236
+ value is already complete BEFORE the continuation, which is then withheld
1237
+ needlessly. Measured on the 948-script corpus: 109 statements make the strict
1238
+ lexer refuse for this reason, the tolerant walk reaches 114 statements carrying
1239
+ 138 such tokens, and **all 138 put the continuation immediately after the `=`**
1240
+ — the value really does live on the next line. Exactly **one** token
1241
+ (`mode=none\<newline>`) has a complete value in front of its continuation. So
1242
+ narrowing the rule would recover one token, and deciding it correctly needs the
1243
+ device-grounded answer for what a continuation does to a value's INTERIOR, which
1244
+ is #225's probe-matrix territory rather than a lexer tweak. Recorded there; not
1245
+ carried as pending work here.
1246
+
1247
+ **What the public surface gained, and what it did not.** The package root
1248
+ exports the reading itself — `lexExplainArgumentTokens` and
1249
+ `ExplainArgumentTokenReading` — for the same reason `lexExplainValueAnchors` is
1250
+ exported: a consumer building its own token-level rule needs the reading a rule
1251
+ wants, not the one a renderer wants. What did NOT change is the *envelope*:
1252
+ `explainCommand`'s `structure.statements[].arguments` is still the strict
1253
+ reading, `ExplainArgumentToken` carries no `undecided` field, and `undecided`
1254
+ never appears on an `ExplainArguments` reading, because the strict lexer
1255
+ publishes no token it could not decide. Whether the tolerant stream should reach
1256
+ the envelope in its own right — so a browser consumer reading `explainCommand`'s
1257
+ result can see a located `gateway=` whose value is a runtime expression — was
1258
+ left to #264 B5, which decided it reaches the envelope **as the token
1259
+ partition** and nowhere else: naming the `=` `arg-sep` locates the argument
1260
+ without loosening `arguments`, and `undecided` stays off the envelope. See
1261
+ [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5).
1262
+
1263
+ ### The menu path with arguments (#324)
1264
+
1265
+ A run that is nothing but a **known menu**, with arguments after it and no
1266
+ verb, used to come back `resolved` with a verb the analyzer made up.
1267
+ `/ip/firewall/address-list name=ytkids timeout=1h` reported path
1268
+ `/ip/firewall`, verb `address-list`; the spaced spelling
1269
+ `/ip firewall address-list name=x` promoted `firewall`, one level higher
1270
+ still. The punctuation half of the ratified Q6 rule is what fires — "no space
1271
+ token, arguments follow, so the last run token is the verb", or "the first
1272
+ space token is the verb" — and neither premise it needs was ever checked:
1273
+
1274
+ ```text
1275
+ isKnownMenuPath(/ip/firewall/address-list) -> true (it is a menu)
1276
+ VERBS.has("address-list") -> false (it is not a verb)
1277
+ ```
1278
+
1279
+ This is the **symmetric half of R9**. R9 (#211 B1) made `pathresolve`'s
1280
+ `menuNavPath` consult `VERBS` so it could not claim navigation where
1281
+ `verbsplit` had decided a verb; this makes `resolveVerb` consult the menu
1282
+ tables so it cannot claim a verb where the whole run names a menu and the
1283
+ segment it would promote is not a verb. The reading degrades to **`ambiguous`**
1284
+ with the standard located `explain/canonicalizer/ambiguous-statement` warning,
1285
+ naming both premises. The adjacent readings are untouched —
1286
+ `/ip/firewall/address-list add name=x` resolves, the bare path is navigation,
1287
+ `/system/reboot` is a command — which is what makes this a seam rather than a
1288
+ parser limit.
1289
+
1290
+ **It abstains; it does not reject.** Offline knows it cannot name a verb here,
1291
+ and that needs no device evidence. Whether RouterOS refuses the shape outright
1292
+ is a *device* claim, so unlike #311's CHR-grounded `error` this stays a
1293
+ warning and the statement carries no verdict about acceptance.
1294
+
1295
+ **Four exclusions, each for its own reason.** A frozen-vocabulary hit is
1296
+ excluded by the `VERBS` premise itself (a vocabulary split promotes a `VERBS`
1297
+ member by construction). A **published command** is excluded by `catalogAt`:
1298
+ first-order evidence about that exact path outranks a container listing of its
1299
+ prefix, which is what keeps `/system/gps/monitor once` resolved. A scripting
1300
+ **directive** is excluded by R10, which is measured. And a `{` after the run is
1301
+ excluded by shape: that is a menu *container* body (`/system {identity print}`,
1302
+ H7) where the run legitimately names a menu and no verb is missing —
1303
+ abstaining there would drop the `invalid-command-brace` error the statement
1304
+ after it carries.
1305
+
1306
+ **Priced before landing, as #211 B2 asks.** Over the frozen dev/holdout
1307
+ partition (948-script corpus, 17,212 statements) **10 statements flip, all
1308
+ `resolved` → `ambiguous`**: 1 dev, 9 holdout. **None is a genuine no-verb
1309
+ command.** Three are real RouterOS missing a verb —
1310
+ `/ip/firewall/address-list name=ytkids …`, `/interface bridge [find]
1311
+ vlan-filtering=yes`, `/user-manager user [find …] attributes=…` — and seven
1312
+ are pasted non-RouterOS prose: five Python traceback `File "…"` lines,
1313
+ `User ID: 506`, and `port 80`. Those seven reach the rule at all because
1314
+ `/file`, `/user` and `/port` are real menus and the table lookup folds case,
1315
+ which is #203's genre bias arriving through a second door. Every one of the
1316
+ ten was a fabrication, so the abstention has no measured false-positive cost
1317
+ on this corpus.
1318
+
1319
+ **The execute gate is unmoved**, as the constitution requires: `canonical`
1320
+ still reports `mode: "structured"`, path `/ip/firewall`, verb `address-list`
1321
+ for this input, because that is what `canonicalizeExecuteCommand` decides and
1322
+ an explain-only abstention may not reinterpret it. `enforceGateParity` is
1323
+ about the argument *list* of a `kind: "command"` reading, so it does not fire
1324
+ on a statement that carries no reading at all — the gate and the analysis are
1325
+ two answers to two questions, and only one of them changed.
1326
+
1327
+ The invariant this closes — `pathresolve` and `verbsplit` never contradict
1328
+ each other on one statement — is asserted in
1329
+ `test/unit/explain-pathresolve.test.ts` beside R9's own cases. Presence in the
1330
+ shipped structure tables still narrows a reading and absence still decides
1331
+ nothing (#207/#228), so an unlisted menu spelled `…/address-list` is read
1332
+ exactly as before.
1333
+
1334
+ ### Two spellings the device accepts and offline used to reject (#347, #348)
1335
+
1336
+ Both were found the same way, and the method is worth keeping: run the analyzer
1337
+ over all 948 corpus scripts and compare its verdict to that build's own `:parse`
1338
+ verdict, which the corpus already stores. The number that matters is the **false
1339
+ reject** — a script the device ACCEPTS that offline analysis fails — because
1340
+ that is validation blocking input RouterOS is happy with. It was 4, identical on
1341
+ 7.23.5, 7.24.2 and 7.25beta3; it is now 1.
1342
+
1343
+ The converse count is not a parser score and is not quoted as one: the corpus is
1344
+ whole forum posts including prose output, so most of what the device rejects and
1345
+ offline passes is #203's genre bias rather than a gap.
1346
+
1347
+ **The `=` is optional on a scope slot (#347).** `:if (…) do={…} else={…}`,
1348
+ `:if (…) do={…} else {…}` and `:if (…) do {…} else {…}` are the same construct:
1349
+ `:parse` lowers all three to byte-identical IL (`…;do=;(evl …);else=;(evl …)`)
1350
+ on CHR 7.21.5, 7.23.5, 7.24.2 and 7.25beta3. The MikroTik manual documents only
1351
+ the `else={…}` spelling, so this is a real-but-undocumented form. The rule is
1352
+ deliberately narrower than "any word before a brace": only the four
1353
+ `SCOPE_ARG_NAMES` members may drop the `=`, and the word may not be the
1354
+ statement's own leading directive — otherwise `:local z {1;2}` would stop
1355
+ reading its brace as an array, and `:do {` would have its slot renamed away from
1356
+ the `command` the IL uses. A COLON also disqualifies the word: the device parses
1357
+ `:if (1=1) do {…}` but refuses `:if (1=1) :do {…}` with `expected end of command`
1358
+ on 7.21.5 and 7.24.2, because a colon makes it a directive and a directive
1359
+ cannot occupy an argument slot. The control is `elsy {`, rejected on all four
1360
+ builds, which is what makes the acceptance a device fact rather than a probe
1361
+ artefact.
1362
+
1363
+ **A relative menu-scope block composes the path (#348).** `/ip { address {
1364
+ print } }` is `(evl (evl (evl /ip/address/print)))` on device and
1365
+ `/interface { ethernet { print } }` the same way; the ABSOLUTE spelling
1366
+ `/ip/address { print }` already passed offline, and only the nested relative
1367
+ form failed. A menu block's body is a statement list, so it may open with a
1368
+ submenu, with one of the thirteen frozen CRUD verbs (`verbs.ts`), with a
1369
+ catalog-named command, with a `:` scripting directive, or with an absolute
1370
+ command — corpus 521 is the directive case, `ethernet { :put … }`.
1371
+
1372
+ That last rule is bounded by what a STATEMENT can start with, which is what
1373
+ keeps `{1;2}` an array literal: an array member opens with a digit or a quote
1374
+ and never reaches the identifier branch. It is also deliberately not widened
1375
+ into a claim of existence. Menu-scope resolution is schema-dependent —
1376
+ `/ip { zzznotamenu { print } }` is a `syntax error` on device — and offline has
1377
+ no schema, so a catalog MISS abstains rather than being decided either way.
1378
+ That is the catalog's own "a hit is decisive, a miss says nothing" contract, and
1379
+ turning such a miss into an acceptance would be the mirror-image defect of the
1380
+ one being fixed here.
1381
+
1382
+ ### `'` is not a RouterOS token (#355)
1383
+
1384
+ `'` (U+0027) is not a string delimiter and not a legal bare byte. Only `"` opens
1385
+ a string. `:parse` rejects every spelling of a bare apostrophe byte-identically
1386
+ on CHR 7.21.5, 7.23.5, 7.24.2 and 7.25beta3:
1387
+
1388
+ | input | device `:parse` | offline span |
1389
+ | --- | --- | --- |
1390
+ | `/system/identity/set name=don't` | `expected end of command (line 1 column 30)` | 29 |
1391
+ | `/system/identity/set name=don't'` | `expected end of command (line 1 column 30)` | 29 |
1392
+ | `/system/identity/set name='abc'` | `syntax error (line 1 column 27)` | 26 |
1393
+ | `:put 'abc'` | `syntax error (line 1 column 6)` | 5 |
1394
+
1395
+ The device's column is 1-based, so each offline span is the device's own byte.
1396
+ The accepting controls — `comment="it's here"` and a `# don't` comment line —
1397
+ parse cleanly on all four builds, which is what makes the rejection a device
1398
+ fact rather than a probe artefact. Rows 2–4 carry BALANCED apostrophes and are
1399
+ rejected anyway: this is a lexical rule, not a quote-balance one, and the remedy
1400
+ is always `"`, never a matching `'`.
1401
+
1402
+ Only the FIRST apostrophe is recorded, **document-wide**, the same way the hard
1403
+ `#` is: the device reports one column and it is the first one (row 2 reports
1404
+ column 30, not the trailing apostrophe), because classification stops there. A
1405
+ nested scope body is a separate scan (`segmentScopeBody`), so per-scan
1406
+ suppression is not enough on its own and the collapse to the earliest occurrence
1407
+ happens where the diagnostics are composed — which keeps each segmenter call
1408
+ pure. Apostrophes inside a `"`-string or a comment never reach the rule at all —
1409
+ both runs are consumed whole by the scan before it — so no special case is
1410
+ needed for either.
1411
+
1412
+ Symbol resolution stops at the apostrophe too, exactly as it stops at a
1413
+ malformed escape: the device classes nothing after a hard reject, so
1414
+ `:put 'abc'; :put $after` must not publish an occurrence for `$after`.
1415
+ Occurrences BEFORE the defect stand, which is the lab's X1 rule.
1416
+
1417
+ **This was a false ACCEPT, which the corpus join cannot see.** The false-reject
1418
+ measurement scores one direction only, so the analyzer passing all four forms
1419
+ with zero diagnostics never appeared in it. Over the pinned corpus the rule
1420
+ fires on 40 of the 7.24.2 scripts and **every one of them is a script the device
1421
+ rejects** — none of the 617 it accepts. It moves the false-reject count not at
1422
+ all (still 1, the extra `}` of corpus 228) while taking the caught count from 62
1423
+ to 93.
1424
+
1425
+ The token census moves, and only in the direction the fail-closed symbol stop
1426
+ predicts: 68 fewer `variable-*` tokens and their 306 bytes returning to
1427
+ `unclassified` (classified 1,023,413 → 1,023,107). Those are variable references
1428
+ sitting after a bare apostrophe, which the device never reaches and centrs
1429
+ therefore stops naming. The value, operator and bracket-equals censuses and
1430
+ `explain:arg-reach` are all unchanged.
1431
+
1432
+ **It also retires a wrong rule.** `execute` carried a private
1433
+ `hasUnbalancedQuotes` preflight that modelled `'` as a delimiter, moved into the
1434
+ offline gate by #354. It had no comment awareness, so `# don't` opened a quote
1435
+ it never closed: over the same corpus it false-rejected **51** of the 617
1436
+ device-accepted scripts. Deleting it costs 21 catches — all of them scripts the
1437
+ device rejects anyway, where the device stage is still the backstop — and takes
1438
+ the gate's false rejects from 52 to 1. A false reject blocks input RouterOS
1439
+ accepts, which is the figure that matters.
1440
+
1016
1441
  ### String escape validation (#247, #252)
1017
1442
 
1018
1443
  The accepted set is the **device's**, not the manual's. The
@@ -1061,8 +1486,9 @@ for the three issue rows `:put "\q"` / `:put "\x0a"` / `:put "\0a"` vs
1061
1486
  - Scoring a defect rule against the corpus needs the right oracle.
1062
1487
  `parseil_results.ok` is **not** one: `:parse` never throws, it returns a
1063
1488
  diagnostic *value*, so the capture records `ok=1 / status="ok"` for scripts
1064
- the device rejected (only 3 of 2739 rows are `ok=0`, and those are capture
1065
- failures). The verdict is in `il_text` — plain prose such as
1489
+ the device rejected (only 5 of 4635 rows are `ok=0`, and those are capture
1490
+ failures — one `edge-cases/oversize-32k.rsc` REST `413` per captured
1491
+ version). The verdict is in `il_text` — plain prose such as
1066
1492
  `expected input value (line 7 column 49)` means rejected, IL means accepted.
1067
1493
 
1068
1494
  ### The operator surface (#255)
@@ -1071,14 +1497,62 @@ Grounded, not transcribed, for the same reason the escape set above is: the
1071
1497
  [manual's operator list](https://manual.mikrotik.com/docs/developer-guides/scripting/index.md#operators)
1072
1498
  is a lower bound. `bun run explain:probe:operators` sweeps the manual's list
1073
1499
  **plus** plausible non-operators (`not`, `..`, `xor`, `mod`, `is`, `div`, …)
1074
- **plus** every IL head the corpus census saw, on CHR 7.23.3 stable, 7.24rc4
1075
- testing and 7.21.5 long-term — one build per release channel, so a claim about
1076
- any channel is evidence rather than an assumption. Re-cut the fixture with
1500
+ **plus** every IL head the corpus census saw, on CHR 7.24.2 stable, 7.23.5
1501
+ long-term and 7.25beta3 development — the same three builds the pinned corpus
1502
+ classes `complete`, so the operator axis and the token partition are finally
1503
+ scored on the same firmware (#342) — plus 7.21.5 as an **age anchor**, which is
1504
+ not a channel claim but the only live evidence that the surface behaves the same
1505
+ on a pre-7.23 branch. Re-cut the fixture with
1077
1506
  `bun run explain:operator-slice`; the table
1078
1507
  below is generated from `src/explain/operators.ts` by
1079
1508
  `bun run explain:operator-readme` and gated by
1080
1509
  `bun run explain:operator-readme:check`.
1081
1510
 
1511
+ **The census is a candidate generator, and boilerplate defeats its own
1512
+ tie-breaker.** `scripts/explain-operator-census.ts` reports every head-position
1513
+ token in the device IL, tagged by shape, because IL is a debug rendering that
1514
+ does not quote strings: a string literal is emitted bare, so string content is
1515
+ indistinguishable from a node head. Its stated honesty rule is that
1516
+ `distinctScripts` beats frequency — "a head seen in one script is one author's
1517
+ string, a head seen in three hundred is a language feature". Only the complete
1518
+ builds carry parseIL for the 35-script `tangentsoft` device-`/export` stratum,
1519
+ and that stratum showed the rule has a blind spot, because a stock string
1520
+ replicated across a whole stratum of device exports looks exactly like a
1521
+ language feature by script count:
1522
+
1523
+ | word-shaped head | scripts, partial builds → complete | what it actually is |
1524
+ | ---------------- | ---------------------------------: | ------------------- |
1525
+ | `for` | 2 → 22 | the defconf comment `accept to local loopback (for CAPsMAN)` — in 20 of the 35 device exports, plus the 2 forum scripts that already carried it |
1526
+ | `line` | 53 → 84 | device diagnostic prose, `bad parameter hw-offload (line 33 column 147)` |
1527
+
1528
+ Those arrows are a **coverage** step, not a version step: the left column is
1529
+ what a 913-script capture could see, the right what a 948-script one sees.
1530
+
1531
+ Neither is an operator, and both are `word`-shaped — the same bucket #255's
1532
+ `not` would have appeared in. So the sweep, not the census, still decides; a
1533
+ head that arrives with a whole stratum at once is boilerplate until the device
1534
+ says otherwise. The two heads that stratum added outright,
1535
+ `configuration.mode=` and `[system`, are `other`-shaped and are the documented
1536
+ string-noise class: a dotted wifi property name emitted bare, and text inside
1537
+ an embedded `/system script add source=` value.
1538
+
1539
+ **The census counts only the complete builds; a separate axis dates heads.**
1540
+ Occurrences are summed across versions, so admitting a partial capture would
1541
+ scale every figure by what happened to be captured — on this snapshot, letting
1542
+ all six builds in inflated the headline by ~20% while `distinctScripts`, the
1543
+ column that actually carries the argument, did not move at all. So
1544
+ `headOccurrences` / `headVersions` / `headArities` read
1545
+ `coverage_class = 'complete'` only, and the census states its version set in
1546
+ `versions` so a gate cannot silently compare two different universes.
1547
+
1548
+ Presence, unlike frequency, *is* monotone in coverage — a 913-of-948 capture
1549
+ that shows a head proves that build had it — so the partial builds keep one
1550
+ job: `headFirstSeen` dates each head against every captured build. That is what
1551
+ still grounds the `any` claim below at 7.20.8, a build the counting side no
1552
+ longer reads. Read a *late* first-seen as "not captured before here", never as
1553
+ "new in": the only two heads that do not reach 7.20.8 are the `tangentsoft`
1554
+ pair above, which no partial build could have seen.
1555
+
1082
1556
  **`:parse` IL is the oracle, and `highlight` cannot be.** IL is prefix form with
1083
1557
  the operator as its node's head, so it names the operator and shows its
1084
1558
  operands. `highlight`'s `syntax-meta` is the device's *residual structure*
@@ -1103,7 +1577,7 @@ Five things the device says that the manual does not:
1103
1577
  3. **`$`, `[`, `]` are syntax, not operator heads.** `$x` stays an atom in the
1104
1578
  IL and `[:tostr 1]` lowers to an `evl` node. Those bytes belong to the
1105
1579
  substitution axis.
1106
- 4. **`any` is an operator** (prefix, arity 1) and is not in the manual's list — a **nil-check**: `:typeof (any x)` is `bool`, `false` only for `nil`/`nothing` (the value of an undefined `:local` and of `[:nothing]`), `true` for everything else including `0`, `""` and `false`. It is the idiom `:if (any $x) ...` to test a variable that may be `nil`; `(true any false)` is not infix at all but juxtaposition `( true (any false))`, and `(1 . any [:nothing])` is concat `1`+`false`. Present since at least 7.20.8 (corpus `any|7.20.8:2`, and swept live on 7.21.5 long-term, 7.23.3 stable and 7.24rc4 testing with no difference between them).
1580
+ 4. **`any` is an operator** (prefix, arity 1) and is not in the manual's list — a **nil-check**: `:typeof (any x)` is `bool`, `false` only for `nil`/`nothing` (the value of an undefined `:local` and of `[:nothing]`), `true` for everything else including `0`, `""` and `false`. It is the idiom `:if (any $x) ...` to test a variable that may be `nil`; `(true any false)` is not infix at all but juxtaposition `( true (any false))`, and `(1 . any [:nothing])` is concat `1`+`false`. Present since at least 7.20.8 (corpus `headFirstSeen[any] = 7.20.8`, the oldest captured build, and swept live on 7.21.5 and 7.23.5 long-term, 7.24.2 stable and 7.25beta3 development with no difference between them).
1107
1581
  5. **`&&` and `||` are spellings**, lowering to the `and` and `or` nodes.
1108
1582
 
1109
1583
  The `(>…)` and `<%%` forms are in the table on the same footing as `+`.
@@ -1119,7 +1593,7 @@ their `precedence` is honestly `null`. That is "unmeasured", not "unknown": the
1119
1593
  sweep asks them separately, every `(U 1 B 2)` and `(1 B U 2)` for
1120
1594
  `U`∈`!`,`any`,`~`,`-`,`>` against all 24 binaries — 240 probes, recorded in the
1121
1595
  fixture's `unary` block. All 240 are accepted and the **binary is outer in every
1122
- one**, on all three versions, so each prefix operator binds tighter than every
1596
+ one**, on all four swept builds, so each prefix operator binds tighter than every
1123
1597
  binary including `->` (14) and `<%%` (13). The table stores one `precedence` per
1124
1598
  spelling, which is the binary level; `~`, `-` and `>` are here because their
1125
1599
  unary reading has no other record.
@@ -1131,8 +1605,14 @@ not concat at all** — `1.` lexes as a *variable name* and the row comes back a
1131
1605
  juxtaposition, `( $1. 2)`. A rule that claims every `.` byte for the operator
1132
1606
  emits a span the device does not have.
1133
1607
 
1134
- Everything in the sweep is identical across 7.21.5, 7.23.3 and 7.24rc4 except
1135
- one runtime row: `:put ({2;1} > {1;2;3})` evaluates to `true` only on 7.24rc4.
1608
+ Everything in the sweep is identical across all four builds except one runtime
1609
+ row: `:put ({2;1} > {1;2;3})` evaluates to `true` on 7.24.2 and 7.25beta3 and
1610
+ errors on both long-term builds. **Array comparison is not fixed on current
1611
+ long-term** — 7.23.5 is newer than the 7.23.3 this was first measured on and
1612
+ still errors — so the change rode the 7.24 line and was not backported. The two
1613
+ errors also differ: 7.23.5 says `cannot compare if array is more than array`,
1614
+ 7.21.5 says `cannot compare if nothing is more than nothing`, so on 7.21.5 the
1615
+ operands did not survive to the comparison at all.
1136
1616
  Identical is measured, not assumed — the slice diffs verdict, arity, precedence,
1137
1617
  associativity, `highlight` run, unary placement and op-axis per version. So the
1138
1618
  operator table needs no version gate; anything reporting what a comparison
@@ -1231,49 +1711,64 @@ And the grounded complement — asked, and refused:
1231
1711
  `bun run explain:token-census:readme` and gated against it by
1232
1712
  `bun run explain:token-census:readme:check`; the fixture itself is gated
1233
1713
  against a fresh corpus run by `bun run explain:token-census:check`. Of
1234
- 1,426,731 analyzed bytes, 990,840 are classified (69.45%), the remaining
1235
- 435,891 are `unclassified`. The census emits 205,072 tokens (avg 216.3 per
1714
+ 1,426,731 analyzed bytes, 1,023,107 are classified (71.71%), the remaining
1715
+ 403,624 are `unclassified`. The census emits 241,539 tokens (avg 254.8 per
1236
1716
  script). Every byte belongs to exactly one token — sorted by `start`, no
1237
- gaps, no overlaps, `join(slice) === input` — and the `class` field is
1238
- provisional until #264 B5. Each B2 fill should move the classified
1717
+ gaps, no overlaps, `join(slice) === input` — and the `class` field is the
1718
+ vocabulary #264 B5 settled. Each B2 fill should move the classified
1239
1719
  percentage.
1240
1720
  <!-- END GENERATED token-census -->
1241
1721
 
1242
1722
  B1's `data.tokens[]` is live behind `--tokens` — a total, gapless byte
1243
- partition whose `class` is provisional until #264 B5 (every unclaimed byte is
1244
- `unclassified`). Since B1 its only fill source was `data.spans[]` (comment runs
1723
+ partition whose `class` draws on the vocabulary #264 B5 settled (every unclaimed
1724
+ byte is `unclassified`). Since B1 its only fill source was `data.spans[]` (comment runs
1245
1725
  and resolved variable occurrences); **the path fill claims resolved menu and
1246
1726
  command-name bytes as provisional `dir` / `cmd` tokens**, including valid path
1247
1727
  slashes and nested command substitutions, while ambiguous, malformed, and
1248
- source-unmapped runs stay unclassified. **`#293`'s arg fill** in
1249
- `src/explain/arg-tokens.ts` then claims argument names and
1250
- their `=` (name run
1251
- `[span.start, valueSpan.start - 1)` plus the single `=` byte at
1252
- `valueSpan.start - 1`) on the residual left by spans, before operators see it
1728
+ source-unmapped runs stay unclassified. A substitution's bytes are located by
1729
+ the resolution's published **`innerSpan`**, never by arithmetic on its `span`:
1730
+ `span` covers `[…]` for a bare substitution but `$[…]` for one interpolated
1731
+ inside a string (the sigil is part of it), and `inner` is trimmed on top of
1732
+ that, so `span.start + 1` was one byte early in a string and two bytes wrong
1733
+ for `[ … ]`. Both refused the fill's slice check and fell through to the
1734
+ string fill, which is how a command `structure.subcommands` had already
1735
+ resolved came back as opaque `string` bytes (#325). The check itself stays:
1736
+ where `Loc` widens the resolution to its statement, the slice no longer equals
1737
+ `inner` and the bytes are left unclaimed rather than landed on the wrong
1738
+ place. **`#293`'s arg fill** in
1739
+ `src/explain/arg-tokens.ts` then claims argument names as `arg` (name run
1740
+ `[span.start, valueSpan.start - 1)`) and the single `=` byte at
1741
+ `valueSpan.start - 1` as `arg-sep` (#264 B5) on the residual left by spans,
1742
+ before operators see it
1253
1743
  — and **`#295`'s value fill follows it** — `src/explain/value-tokens.ts`
1254
1744
  claims argument value bytes and leaf array-literal members
1255
1745
  (`data.values.occurrences`, leaves only via `parent` containment,
1256
- quotes-included) on the residual left by `spans`+`path`+`arg`, before the
1746
+ quotes-included) on the residual left by `spans`+`path`+`arg`+`escaped`, before the
1257
1747
  **B3 string/brace fills** see it — `src/explain/string-tokens.ts` claims
1258
- quoted-string runs (delimiters included) and `src/explain/brace-tokens.ts`
1748
+ quoted-string runs (delimiters included, escapes already taken) and `src/explain/brace-tokens.ts`
1259
1749
  claims every residual brace delimiter (`{`/`}`), including delimiters from
1260
1750
  invalid braced references, on the residual left by
1261
- `spans`+`path`+`arg`+`value`, and finally `operator` sees only what none of
1751
+ `spans`+`path`+`arg`+`escaped`+`value`, and finally `operator` sees only what none of
1262
1752
  them wanted — which is why the operator fill can abstain on `, / = -` outside
1263
1753
  `( )`, on bytes glued after an argument `=`, and on bytes glued into an
1264
- argument name without a smarter operator scanner. Fill order is enforced by
1754
+ argument name without a smarter operator scanner. **`src/explain/escape-tokens.ts`
1755
+ claims valid string-internal escapes as `escaped`** between `arg` and `value`:
1756
+ an escape is interior to a quoted run, so it has to claim before the two fills
1757
+ that would otherwise swallow it whole, and it splits their runs into fragments
1758
+ around it the same way a `variable-*` span already did. Fill order is enforced by
1265
1759
  argument order to `buildTokens` ([#290 design decision 1](../../src/explain.ts)):
1266
- `spans` claims first, then `path`, `arg`, `value`, `string`, `brace`, and
1267
- finally `operator`.
1760
+ `spans` claims first, then `path`, `arg`, `escaped`, `value`, `string`, `brace`,
1761
+ and finally `operator`.
1268
1762
  `ExplainTokenClass` is
1269
- `ExplainSpanClass | "dir" | "cmd" | "operator" | "arg" | "value" | "string" |
1270
- "brace" | "unclassified"` — one
1271
- provisional `operator` class for all 26 spellings + 2 aliases, one provisional
1272
- `arg` class for both the name bytes and the `=` separator, one provisional
1273
- `value` class for every leaf value span, plus provisional `string` (quoted
1274
- runs) and `brace` (scope delimiters) — emit first, name later (whether the `=`
1275
- or per-shape `value` later deserve their own classes is #264 B5 and does not
1276
- move the byte coverage), not per-operator/per-category/shape (`#264` B5).
1763
+ `ExplainSpanClass | "dir" | "cmd" | "operator" | "arg" | "arg-sep" | "value" |
1764
+ "string" | "escaped" | "brace" | "unclassified"` — one `operator` class for all 26
1765
+ spellings + 2 aliases, `arg` for an argument's name bytes and `arg-sep` for the
1766
+ single `=` that binds it to its value, one `value` class for every leaf value
1767
+ span, plus `string` (quoted runs minus their escapes), `escaped` (one valid
1768
+ string-internal escape sequence) and `brace` (scope delimiters). #264
1769
+ settled which of those merges keep their name and which do not, on a stated
1770
+ rule rather than taste — see
1771
+ [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5).
1277
1772
  `ExplainSpanClass` and `data.spans[]` stay proof-only; `src/explain/operators.ts`
1278
1773
  remains data plus accessors, and the operator table above is its source. The
1279
1774
  `centrs → highlight` projection is B4 and reads both.
@@ -1289,11 +1784,413 @@ each grounded on the corpus device oracle (`parseil_results.il_text`):
1289
1784
  | A spelling glued immediately after an argument `=`. | `in-interface-list=!LAN`, `.id=*2`, `oid=.1.3.6.1.2.1` — the byte after an argument `=` starts the value. |
1290
1785
  | A spelling glued into an argument **name**. | `:foreach x in=$list` is `/foreach counter=$x` with no `(in …)` node anywhere; the IL keeps `security.authentication-types=wpa2-psk` as one name and renders `.id` as the single symbol `$.id`, never `(. …)`. |
1291
1786
 
1292
- The first abstention has a measured cost: `find where name="x"` inside `[ … ]`
1293
- *does* lower to a real `(= $name x)` node, so 224 of the corpus's 1,259
1294
- bracket-interior `=` bytes are genuine comparisons that now stay
1295
- `unclassified` — against 1,035 that were plain `arg=value`. A `where`-aware
1296
- fill can take them back later; claiming all 1,259 would be 82% wrong.
1787
+ The first abstention has a measured cost, and it is now generated rather than
1788
+ asserted. The figures below were prose with no script behind them and no gate
1789
+ on them for three releases, quoting a `224` / `1,259` / `1,035` split that no
1790
+ universe reproduces (#341). The abstention itself is as wide as it ever was —
1791
+ what was wrong was the accounting of it. They are re-derived by
1792
+ `bun run explain:bracket-equals` and gated by
1793
+ `bun run explain:bracket-equals:readme:check`.
1794
+
1795
+ <!-- BEGIN GENERATED bracket-equals — regenerate with `bun run explain:bracket-equals:readme` -->
1796
+ Measured by `bun run explain:bracket-equals` over the pinned corpus:
1797
+ **1,079** `=` bytes sit inside `[ … ]`, across
1798
+ 263 of 948 scripts, and **1,067** of them come back
1799
+ `unclassified` — the operator fill abstains on essentially all of it. No
1800
+ later fill rescues them: the `arg` fill claims **0**, because it offers only
1801
+ the `=` its own located argument token names and a `[ … ]` substitution is
1802
+ not where it locates arguments.
1803
+
1804
+ Of those 1,067, **448** are governed by a `find`/`where` query and are
1805
+ genuine comparisons — the device lowers `[find address=$IP]` to
1806
+ `(= $address $IP)` — against **619** plain `arg=value`. So abstaining is
1807
+ 42.0% wrong and claiming the lot would be 58.0% wrong; a `where`-aware
1808
+ fill can take the query half later. For contrast the fill does claim
1809
+ **1,402** `=` bytes whose innermost opener is `(`.
1810
+
1811
+ The query reading is the device's, not a guess: on 7.24.2, 93 of the
1812
+ 137 scripts carrying one have IL with at least as many `(= …)` nodes.
1813
+ Of the rest, 28 are scripts the console refused outright — their IL is prose,
1814
+ with no nodes of any kind — and 16 use a menu the capture build could not
1815
+ resolve (a forum snippet on `/interface wireless` answers `bad command name
1816
+ wireless`), so the device never reached the query to lower it. Neither is a
1817
+ counter-example to the reading. Whole-corpus `(= …)` head nodes, which also
1818
+ count expression comparisons the fill already claims:
1819
+ 7.23.5 1,233, 7.24.2 1,240, 7.25beta3 1,240.
1820
+ <!-- END GENERATED bracket-equals -->
1821
+
1822
+ ### Device agreement for the token partition (#264 B4, #263)
1823
+
1824
+ The census above counts what offline analysis *claimed*. This counts what the
1825
+ device *agreed with*. The oracle is the committed stratified slice of the Q13
1826
+ `/console/inspect request=highlight` captures
1827
+ ([`test/fixtures/explain/highlight-streams.slice.json`](../../test/fixtures/explain/highlight-streams.slice.json)),
1828
+ so the scorer needs no router, no corpus and no network, and gates in the unit
1829
+ suite rather than in the corpus job. Both oracles must see the same bytes, so
1830
+ the source text is reconstructed from the device's own run-length pairs — never
1831
+ from the corpus row, which still holds the non-ASCII the capture replaced
1832
+ (the #269 rule, one layer up).
1833
+
1834
+ **`highlight` is not one oracle; it is four kinds of answer.** Scoring a byte
1835
+ means first knowing what kind of answer came back, because three of the four
1836
+ are not judgments about syntax at all:
1837
+
1838
+ | Kind | Classes | Why it cannot be scored as agreement |
1839
+ | ---- | ------- | ------------------------------------ |
1840
+ | syntax | `cmd` `dir` `arg` `arg-dot` `arg-scope` `comment` `escaped` `syntax-meta` `variable-local` `variable-global` `variable-auto` `variable-parameter` | — this is the region that can be scored |
1841
+ | non-syntax | `obj-inactive` `obj-dynamic` `obj-disabled` `variable-undefined` `syntax-obsolete` | The source text alone does not decide these. centrs has no class for most of them at all — `ExplainSymbolClass` deliberately excludes `undefined` — and where it *does* claim the byte (an `arg` the device calls `obj-inactive`) it is the **scorer** that abstains, because the device was not making a syntax judgement there. *Why* each one is out of reach is a second question, answered per byte rather than per class; see applicability below. |
1842
+ | silence | `none` | The device's abstention, not a class named "none". Agreeing with silence is not agreement. |
1843
+ | parser-stop | `error` | The device gives up here — see below. |
1844
+
1845
+ **Applicability is a second axis, and it is not the class name.** #263's four
1846
+ categories — offline-decidable, schema-dependent, state-dependent,
1847
+ version-dependent — are assigned from the *pair* (what centrs read the byte as,
1848
+ what the device called it) plus whether the two captures even agreed, because
1849
+ one highlight class carries more than one category. Version disagreement is
1850
+ checked first, ahead even of the device's own silence: a byte one build
1851
+ classifies and the next leaves `none` is a version difference, not an absence.
1852
+ `obj-inactive` on an argument name means the menu's schema has no such field
1853
+ (**schema** — `default-vlan-id`, under an `/interface ethernet switch port` a
1854
+ CHR does not have); the same class on a menu or command name means the package
1855
+ is not installed on this box (**state**: `/zerotier`, `/user-manager`,
1856
+ `/container`). `obj-dynamic` on the routing table `main` is a live object
1857
+ (**state**).
1858
+ `syntax-obsolete` is this version's deprecation table talking, landing on the
1859
+ whitespace where modern syntax wants `=` (`} else {`, `:foreach conn in
1860
+ $conns`) — **version**. And `variable-undefined` is deliberately left
1861
+ **uncategorized**: an undefined reference may be an external global, a
1862
+ query/filter field name or a lexical spelling, nothing committed here separates
1863
+ the three, and an absent binding in *this* document is not evidence of a
1864
+ runtime undefined variable either. Leaving it explicit is the answer, not a gap
1865
+ to be filled by assumption.
1866
+
1867
+ **The device stops classifying at its first `error`, and mostly does not
1868
+ recover.** Measured over the slice, `error` appears **at most once per script
1869
+ and is always exactly one byte wide**, and on long-term 7.23.5 every byte after
1870
+ it comes back `none`. Most of the slice sits in that tail. A percentage computed
1871
+ over the whole stream therefore measures the device giving up, not the parser,
1872
+ which is why the tail is its own outcome and is never folded into agreement. The
1873
+ rule is not absolute across versions, so the report does not assert it: stable
1874
+ 7.24.2 and development 7.25beta3 classify again after the `error` in one run and
1875
+ stop on fewer scripts, and
1876
+ `parser-recovered` in the table below is the figure that says by how much —
1877
+ zero would have been the claim, and it is not zero. Both counts are generated,
1878
+ not quoted here, so neither can go stale against the fixture.
1879
+
1880
+ **The projection is declared, never fitted.** `src/explain/highlight-projection.ts`
1881
+ maps each centrs token class to the device classes it predicts, justified by
1882
+ what the class was *defined* to mean rather than by where it happened to land.
1883
+ A projection tuned to the measurement would make the measurement circular. So a
1884
+ class whose intent has no single device counterpart — `string`, which spans a
1885
+ delimiter the device calls `syntax-meta`, escapes it calls `escaped` and an
1886
+ interior it declines to classify, and `value`, for which the device has no class
1887
+ at all — projects nowhere, and its bytes are counted as `unprojected` with the
1888
+ device's own answers tabulated beside them. That table is the evidence #264 B5
1889
+ needs in order to decide which provisional merge deserves to be split; it is not
1890
+ a score.
1891
+
1892
+ <!-- cspell:disable — the example table quotes corpus script paths verbatim, forum-thread typos included -->
1893
+ <!-- BEGIN GENERATED highlight-agreement — regenerate with `bun run explain:highlight-agreement:readme` -->
1894
+ The agreement report is re-derivable with
1895
+ `bun run explain:highlight-agreement` and needs no router and no corpus: it
1896
+ scores the token partition against
1897
+ `test/fixtures/explain/highlight-streams.slice.json`, 70 stratified scripts of
1898
+ committed `/console/inspect request=highlight` output. The figures below are
1899
+ generated from `test/fixtures/explain/highlight-agreement.json` by
1900
+ `bun run explain:highlight-agreement:readme`, gated against it by
1901
+ `bun run explain:highlight-agreement:readme:check`, and the fixture itself is
1902
+ gated against a fresh measurement by
1903
+ `test/unit/explain-highlight-agreement.test.ts`. Of 85,529 bytes at 7.24.2,
1904
+ 27,974 are bytes **both** sides decided a syntax class for, and 99.99% of
1905
+ those agree (dev 99.99%, holdout 100.00%). Set `comment` aside — it is 65.63%
1906
+ of that decided region and mostly the corpus's harness-injected `# Source:`
1907
+ banner (#203) — and the remaining 9,615 bytes agree 99.98%. The device stops
1908
+ classifying at its one-byte `error`: 36 of 70 scripts carry one, and the
1909
+ 41,968 bytes from there on are not a judgment about anything. The oracle
1910
+ itself moves between captures: 36 stop at 7.24.2 and 43 stop at 7.23.5 and 36
1911
+ stop at 7.25beta3, and 27 of the 70 streams differ between them. **This is a
1912
+ trend line, not a pass gate** — the slice is a per-(split, class) quota over a
1913
+ corpus that is 96.8% two authors, so the percentage describes this slice.
1914
+
1915
+ | centrs class | projects to | agree | disagree | unprojected | of decided |
1916
+ | ------------ | ----------- | ----: | -------: | ----------: | ---------: |
1917
+ | `comment` | `comment` | 18,359 | 0 | 0 | 100.00% |
1918
+ | `variable-local` | `variable-local` | 1,123 | 0 | 0 | 100.00% |
1919
+ | `variable-global` | `variable-global` | 582 | 0 | 0 | 100.00% |
1920
+ | `variable-auto` | `variable-auto` | 92 | 0 | 0 | 100.00% |
1921
+ | `variable-parameter` | `variable-parameter` | 459 | 0 | 0 | 100.00% |
1922
+ | `dir` | `dir` | 1,677 | 0 | 0 | 100.00% |
1923
+ | `cmd` | `cmd` | 2,244 | 0 | 0 | 100.00% |
1924
+ | `arg` | `arg`, `arg-dot`, `arg-scope` | 2,188 | 0 | 0 | 100.00% |
1925
+ | `arg-sep` | `syntax-meta` | 382 | 0 | 0 | 100.00% |
1926
+ | `operator` | `syntax-meta` | 237 | 2 | 0 | 99.16% |
1927
+ | `brace` | `syntax-meta` | 211 | 0 | 0 | 100.00% |
1928
+ | `string` | *abstains* | 0 | 0 | 499 | — |
1929
+ | `escaped` | `escaped` | 418 | 0 | 0 | 100.00% |
1930
+ | `value` | *abstains* | 0 | 0 | 144 | — |
1931
+
1932
+ | outcome | bytes @7.24.2 | bytes @7.23.5 | bytes @7.25beta3 |
1933
+ | ------- | ----: | ----: | ----: |
1934
+ | agree — both decided, projection accepts | 27,972 | 27,218 | 27,972 |
1935
+ | disagree — both decided, projection rejects | 2 | 2 | 2 |
1936
+ | unprojected — no declared projection covers the pair | 643 | 633 | 643 |
1937
+ | offline-silent — device decided, centrs abstained | 4,300 | 4,150 | 4,300 |
1938
+ | non-syntax — the device answered something syntax cannot decide | 847 | 733 | 847 |
1939
+ | device-silent — centrs decided, device said `none` | 5,815 | 5,674 | 5,815 |
1940
+ | both-silent | 3,886 | 3,702 | 3,886 |
1941
+ | parser-stopped — at/after the device's `error` byte, and silent from there | 41,968 | 43,417 | 41,968 |
1942
+ | parser-recovered — past that `error`, and the device classified anyway | 96 | 0 | 96 |
1943
+
1944
+ Where the device decided and offline analysis did not (4,300 bytes at 7.24.2,
1945
+ the next fill's target list): `syntax-meta` 3,122, `comment` 355,
1946
+ `variable-local` 326, `arg` 197, `escaped` 145, `dir` 66, 89 across the rest.
1947
+ Most of the `syntax-meta` share is whitespace the device merged into an
1948
+ adjacent structure run rather than a token centrs missed.
1949
+
1950
+ **Applicability** is the second axis (#263): of the bytes the device did
1951
+ answer, what kind of fact was it answering? Assigned from the pair — what
1952
+ centrs read the byte as and what the device called it — plus whether the
1953
+ captures agreed, never by assigning a whole class to one category. It covers
1954
+ every measured byte, the post-`error` tail included: where one capture stops
1955
+ parsing and the next does not, that tail IS the version disagreement, which is
1956
+ why it dominates the category.
1957
+
1958
+ | applicability @7.24.2 | bytes | leading cells |
1959
+ | ---------------------------- | ----: | ------------- |
1960
+ | offline-decidable | 32,002 | `comment` → `comment` 18,006, `unclassified` → `syntax-meta` 2,999, `cmd` → `cmd` 2,132 |
1961
+ | schema-dependent | 326 | `arg` → `obj-inactive` 326 |
1962
+ | state-dependent | 168 | `dir` → `obj-inactive` 120, `cmd` → `obj-inactive` 21, `unclassified` → `obj-dynamic` 20 |
1963
+ | version-dependent | 1,158 | `comment` → `comment` 353, `unclassified` → `obj-inactive` 159, `unclassified` → `syntax-meta` 123 |
1964
+ | uncategorized | 213 | `unclassified` → `obj-inactive` 90, `unclassified` → `variable-undefined` 75, `value` → `variable-undefined` 21 |
1965
+ | no-device-answer | 51,662 | `none` where the captures agree, wherever it falls |
1966
+
1967
+ One representative run per cell, so B5 reads a fragment rather than a count.
1968
+ `runs` is how often that exact fragment occurs; the location is a slice script
1969
+ and a byte offset into the stream the device saw.
1970
+
1971
+ | outcome | cell | fragment | runs | first at |
1972
+ | ------- | ---- | -------- | ---: | -------- |
1973
+ | disagree | `operator` → `arg` | `"in"` | 1 | `forum/amm0/topic-169456-having-the-where-filter-in-scripting-signifantly-increases-the-execution-time-an/post-0006-snippet-01.rsc` @615 |
1974
+ | unprojected | `string` → `syntax-meta` | `"\""` | 311 | `eworm/ppp-on-up.rsc` @435 |
1975
+ | unprojected | `value` → `syntax-meta` | `"\""` | 88 | `forum/amm0/topic-153357-using-wifiwave2-to-bridge-two-audience-wirelessly-thoughts-4-address-mode/post-0001-snippet-01.rsc` @1436 |
1976
+ | unprojected | `string` → `arg` | `"as-value"` | 1 | `forum/amm0/topic-266499-how-to-condense-variables-passed-through-to-function/post-0020-snippet-01.rsc` @279 |
1977
+ | unprojected | `string` → `cmd` | `"len"` | 1 | `forum/amm0/topic-155978-zerotier-on-mikrotik-a-rosetta-stone-v7-1-1/post-0001-snippet-01.rsc` @1794 |
1978
+ | unprojected | `string` → `dir` | `":"` | 1 | `forum/amm0/topic-155978-zerotier-on-mikrotik-a-rosetta-stone-v7-1-1/post-0001-snippet-01.rsc` @1793 |
1979
+ <!-- END GENERATED highlight-agreement -->
1980
+ <!-- cspell:enable -->
1981
+
1982
+ **What the disagreements are.** Two cells carry all of them, and neither is a
1983
+ lexing accident:
1984
+
1985
+ - `arg` → `syntax-meta`: the `=` byte the `arg` class merges in. The device
1986
+ reads a separator as structure. This is the measured cost of the merge the
1987
+ token-census block already flags, and it is the single largest reason `arg`
1988
+ scores below the other name-coincident classes.
1989
+ - `operator` → `arg`: the space-separated `in` of `:foreach conn in $conns`,
1990
+ which the device reads as the argument **name** `in`. The two oracles
1991
+ genuinely differ here — `:parse` IL carries `in` in the operator set, and the
1992
+ operator-abstention table above already records that `:foreach x in=$list`
1993
+ lowers with no `(in …)` node.
1994
+
1995
+ The third cell is gone. `cmd` → `dir` was the whole of #324 — a known menu path
1996
+ carrying arguments and no verb, where `verbsplit` promoted the last segment;
1997
+ those 12 bytes now abstain instead of disagreeing, and the 14 `dir` bytes the
1998
+ same statements contributed to `agree` abstain with them. That is the price,
1999
+ and [the fix states it](#the-menu-path-with-arguments-324): withdrawing a
2000
+ fabricated verb also withdraws the menu reading it was built on.
2001
+
2002
+ The `unprojected` residue is what #264 B5 read to decide which merges to
2003
+ split — [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) records
2004
+ the verdict on each. It was also where #325 lived:
2005
+ a quoted string claimed any `$[…]` substitution inside it, so bytes the device
2006
+ resolves to `cmd` or `dir` came back as one opaque `string` run even though
2007
+ `structure.subcommands` had already resolved that same span. Those cells have
2008
+ largely emptied (`string` → `cmd` 42 → 3, `string` → `dir` 19 → 1, with the
2009
+ same fix recovering padded bare brackets `[ … ]` from `offline-silent`), and
2010
+ the two residues left are stated rather than fixed:
2011
+
2012
+ - `$(…)` — a **parenthesized** interpolation carrying a bracket, as in
2013
+ `apikey-len=$([:len $apikey])`. Only `$[` is scanned inside a string, so the
2014
+ inner command is not resolved there in the first place; this is a parser
2015
+ reach, not a fill-order question (4 bytes on the slice).
2016
+ - a bare-word **argument** inside a substitution — `as-value` in
2017
+ `:put "$[/ip address print as-value]"`. The path fill claims the path and
2018
+ verb; the argument fill reads `statements[].arguments`, and a substitution is
2019
+ not a statement, so it has no candidate to claim (8 bytes).
2020
+
2021
+ The `$` sigil and the `[` / `]` delimiters stay `string`, and `value` bytes
2022
+ are, to the device, simply unclassified — both settled by
2023
+ [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) below.
2024
+
2025
+ ### The vocabulary, and what earns a class (#264 B5)
2026
+
2027
+ `data.tokens[]` shipped with its `class` marked provisional from B1 onward —
2028
+ emit first, name later (#293 design decision 2). This is the "name later", and
2029
+ the vocabulary below is what `@tikoci/centrs/explain` offers a consumer. It is
2030
+ no longer provisional.
2031
+
2032
+ **The rule a class has to pass.** A merged class is split when both hold, and
2033
+ stays merged otherwise:
2034
+
2035
+ 1. **A consumer needs the distinction** — it changes what that consumer renders
2036
+ or decides, not just how a token dump reads.
2037
+ 2. **The envelope cannot already supply it** — the distinction is not
2038
+ recoverable by joining `tokens[]` to another published surface on byte
2039
+ offsets.
2040
+
2041
+ The second test is the one that does the work. A class that restates a fact
2042
+ `structure`, `symbols` or `values` already publishes is a second place for that
2043
+ fact to go stale, and the two copies can then disagree about the same bytes. So
2044
+ the question is never "would a name here be nice"; it is "what would a consumer
2045
+ have to do without it". Applied to the five merges the fills carried, exactly
2046
+ one passes:
2047
+
2048
+ | merge | what a consumer would do without it | verdict |
2049
+ | ----- | ----------------------------------- | ------- |
2050
+ | `arg` — the name bytes **and** the `=` | subtract one from `arguments[].valueSpan.start`, which exists only where the STRICT reading read. **5,088 of the 16,526 `=` runs the fill paints (30.8%, in 436 of 948 corpus scripts) sit in a statement whose `arguments` refused** — led by `do=` (2,393), `else=` (328) and `in=` (212), the block binder of every `:if`, `:foreach` and `:while`. For those there is nothing to subtract from. | **split** → `arg-sep` |
2051
+ | `operator` — 26 spellings + 2 aliases | slice the input at the token's span: the spelling *is* the bytes, and `src/explain/operators.ts` maps it to its IL node | keep |
2052
+ | `value` — every leaf, whatever its shape | join `data.values.occurrences[]` on the offset; #225's three axes (shape, observed, schema) live there, and a class per shape would be a fourth copy free to contradict them | keep |
2053
+ | `brace` — scope and array delimiters | `structure.blocks[]` locates every scope block; an array container is the `values.occurrences[]` entry that is some other occurrence's `parent` | keep |
2054
+ | `string` — delimiters, escapes, interior | the opening delimiter is the run's first byte and, where the run is closed, the last is its partner (an unterminated one is already a located defect). The **escapes were a different answer** — see below | **partly split** → `escaped`; delimiters and interior stay merged |
2055
+
2056
+ Re-derive the `arg-sep` figures with `bun run explain:arg-reach`, which prices
2057
+ the same gap it prices the reach fork with.
2058
+
2059
+ **What the split cost.** Nothing, which was the claim and is now the
2060
+ measurement. Classified bytes are identical — 1,023,405 of 1,426,731 (71.73%),
2061
+ equal to the last recorded decimal before and after — and `arg` gave up exactly
2062
+ the 16,526 bytes `arg-sep` claims, one per attribute. Token count rises 214,503
2063
+ → 231,026 because one run became two. **This is a retag.**
2064
+
2065
+ (Those totals are the corpus as it stood for THIS measurement. The classified
2066
+ total is 1,023,107 today: #347/#348 later classified 8 further bytes, and #355's
2067
+ fail-closed symbol stop then unclaimed 306 of them — variable bytes sitting
2068
+ after a bare apostrophe, which the device never reaches. The identity being
2069
+ claimed here is between the two sides of the split, not with any later total —
2070
+ restating it against today's figure would credit this retag with a movement a
2071
+ different change made. The generated block above carries the current numbers.)
2072
+
2073
+ One count is not a clean split: `arg`'s own run count falls 16,526 → 16,523.
2074
+ Three attributes lost their name run entirely, because an earlier `variable-*`
2075
+ span had already claimed the name bytes — `set $ifcId ssid=$ssid`, and two
2076
+ `on-error=` in one snippet — leaving only the `=` on the residual. Before the
2077
+ split those three emitted a token *called* `arg` that contained nothing but an
2078
+ `=`. That is the merge in miniature, and it is why the number goes down.
2079
+
2080
+ **The agreement number moved a long way, and it is still not a score.** Against
2081
+ the committed device slice, projection disagreements fall **360 → 2** and the
2082
+ decided region goes 98.66% → 99.99% (`arg` alone 85.50% → 100.00%). Read that
2083
+ as the withdrawal of a wrong assertion, not as a parser that got better: centrs
2084
+ stopped calling a separator a name over the 358 runs the device had been
2085
+ reading as `syntax-meta` all along, and not one byte changed which analyzer
2086
+ found it. The two survivors are the `operator` → `arg` `in` of
2087
+ `:foreach conn in $conns`, where the two oracles genuinely differ.
2088
+
2089
+ The projection entry was **declared before the measurement, never fitted to
2090
+ it**: `arg-sep` predicts `syntax-meta` because a separator is structure
2091
+ punctuation, the same reason `brace` and `operator` predict it. The slice then
2092
+ put 358 of 358 there.
2093
+
2094
+ **Where the tolerant reading reaches the envelope (#316's open decision).** It
2095
+ reaches it *as the token partition*, and that is the only place it should.
2096
+ `structure.statements[].arguments` stays the strict all-or-nothing reading,
2097
+ because its consumer renders a runnable request and a half-read argument list
2098
+ changes what the rendered command does. But with the `=` named, `tokens[]`
2099
+ locates the argument on its own:
2100
+
2101
+ ```text
2102
+ /ip route add dst-address=1.1.1.1 gateway=$gw
2103
+ arguments: { read: false, why: "a variable value" }
2104
+ tokens: … arg("gateway") arg-sep("=") unclassified("$") variable-parameter("gw")
2105
+
2106
+ :if ($x = 1) do={ :put "hi" }
2107
+ arguments: { read: false, why: "a substitution or expression value" }
2108
+ tokens: … operator("=") … arg("do") arg-sep("=") brace("{") …
2109
+ ```
2110
+
2111
+ Neither statement publishes a single argument, and in both a browser consumer
2112
+ can still see a located `gateway=` / `do=`. The second line is the
2113
+ discrimination working as well: the same `=` byte is `operator` inside
2114
+ `($x = 1)` and `arg-sep` in `do=`, decided by fill order and meaning rather
2115
+ than by a byte scan.
2116
+
2117
+ What `tokens[]` still does **not** carry is the *reason* a value was not
2118
+ decoded. `undecided` stays off `ExplainArgumentToken` and out of the
2119
+ envelope: the envelope's value-fact surface is `data.values.occurrences[]`,
2120
+ where absence at an offset already *is* the abstention, and a second spelling
2121
+ of the same abstention would be free to disagree with it. A consumer that needs
2122
+ the reason imports `lexExplainArgumentTokens` from the package root — which is
2123
+ why that export exists.
2124
+
2125
+ **The second split the rule admits, now made: `escaped`.** A *valid* string
2126
+ escape is recoverable from nothing. Only INVALID ones surface elsewhere, as
2127
+ `bad-string-escape` diagnostics, so a consumer holding `tokens[]` has no
2128
+ published surface to join against to find the valid ones — and the device does
2129
+ class them, as one of the 19 highlight classes. Both halves of the rule hold,
2130
+ so the class is earned.
2131
+
2132
+ Unlike `arg-sep` this is a **fill change** rather than a pure relabel of one
2133
+ analyzer's own output: the escape grammar (uppercase-hex `\XX`, the whitespace
2134
+ continuation, the twelve single-character forms) has to be walked, and the
2135
+ bytes come out of two fills at once rather than one.
2136
+ `src/explain/escape-tokens.ts` runs between `arg` and `value`.
2137
+
2138
+ **The walk is not new, and deliberately not a second one.**
2139
+ [String escape validation](#string-escape-validation-247-252) already owned this
2140
+ grammar for `bad-string-escape`. `walkStringEscapes` now returns *both* halves
2141
+ of that one reading — the valid escapes and the first invalid one — so an
2142
+ `escaped` token and a `bad-string-escape` diagnostic cannot disagree about which
2143
+ bytes are an escape. Deriving them from two walks would have made that
2144
+ disagreement possible, and the document is walked once rather than twice. The
2145
+ walk stops at the first invalid escape, so `escaped` is the prefix the validator
2146
+ vouched for: past a malformed escape the string boundaries are themselves in
2147
+ doubt, and painting there would be a claim the walk cannot support.
2148
+
2149
+ **What the split cost: nothing, and the census proves it.** On the corpus
2150
+ `escaped` claims **19,960 bytes over 7,408 tokens**, and every one of those
2151
+ bytes came from a fill that already held it — `string` gives up 17,672 and
2152
+ `value` 2,288, which is exactly 19,960. Classified bytes are **identical to the
2153
+ last decimal**, 1,023,405 of 1,426,731 (71.73076073906013%), so in byte terms
2154
+ this is a retag even though it took a new walk to make — again, the identity is
2155
+ between this change's own before and after, and the corpus total has since moved
2156
+ to 1,023,107 / 241,539 tokens under #347/#348 and #355. Only the token count
2157
+ moves, 231,026 → 241,663: the 7,408 new `escaped` tokens plus the 3,229
2158
+ fragments they leave behind when they split a run they sit inside (`string`
2159
+ +2,643, `value` +586). That second number is well under one per escape because
2160
+ escapes cluster — adjacent ones like `\C3\BD`, and ones flush against a
2161
+ delimiter, add no fragment at all.
2162
+
2163
+ **What it bought.** Against the committed device slice `escaped` claims **418
2164
+ bytes on 100.00% agreement, 0 disagreements** — the 53 `string` → `escaped`
2165
+ runs and the 1 `value` → `escaped` that were `unprojected` before, scored for
2166
+ the first time. Total `unprojected` falls **1,051 → 633 bytes** (−39.8%) and the
2167
+ both-decided region grows 26,794 → 27,212. No byte changed which analyzer found
2168
+ it and no disagreement was introduced.
2169
+
2170
+ The projection entry was again **declared before it was measured**: `escaped`
2171
+ predicts `escaped` because the class means one valid string-internal escape and
2172
+ that is the run the device gives the same name. `escaped` was already listed in
2173
+ `DEVICE_CLASS_KIND` as a syntax class; the agreement report's `unprojected`
2174
+ table showed the bytes were being merged away, it did not pick the target.
2175
+
2176
+ **The residue this does not claim, sized.** 145 bytes across 24 runs in 4 slice
2177
+ scripts stay `unclassified` where the device says `escaped`, and every one of
2178
+ them is the same shape: a `\`-newline **line continuation in code**, outside any
2179
+ string (`… != true) \<newline> do={`). The device's run there swallows the
2180
+ next line's indentation too, so the 145 bytes are 48 of `\`+terminator and 97 of
2181
+ trailing whitespace. That is a different claim from a string escape — centrs has
2182
+ no continuation token, and matching the device would mean deciding how much
2183
+ following whitespace a continuation owns — so it is **not** folded in here. It
2184
+ belongs with the continuation question #225 already owns (what a continuation
2185
+ does to a value's interior); this paragraph is its size, not a plan.
2186
+
2187
+ Delimiters are the near miss that shows the rule discriminates rather than
2188
+ rubber-stamps: `"` bytes are 311 `string` → `syntax-meta` runs and 88
2189
+ `value` → `syntax-meta`, the device classes them too, and they stay merged
2190
+ anyway, because a consumer can find them by looking at the run's own first and
2191
+ last byte. The `$` sigil and the `[` / `]` of a substitution stay `string` on
2192
+ the same ground — `structure.subcommands` already publishes the resolution, and
2193
+ its `innerSpan` locates the brackets around it.
1297
2194
 
1298
2195
  ### Designed, not implemented (the CLI surface, #202b)
1299
2196
 
@@ -1314,12 +2211,11 @@ That is the difference — an accepted flag must do something observable.
1314
2211
 
1315
2212
  ## MCP and library surfaces
1316
2213
 
1317
- - **The cross-project handoff is the structure layer, not the token layer.**
1318
- `explain`'s data is three layers: byte ranges and their classes
1319
- (`spans`, and `tokens` when #264 lands) — centrs-owned, and nothing outside
1320
- centrs consumes them; the resolved reading (`structure`, `symbols`,
1321
- `values`) — path, verb, argument names, value spans; and doc/version
1322
- enrichment, which is rosetta's (#175). A consumer that wants enrichment takes
2214
+ - **Consumers use the layer that answers their question.** Byte ranges and
2215
+ their classes (`spans` and the shipped `tokens` facet) serve editor/token
2216
+ consumers; the resolved reading (`structure`, `symbols`, `values`) carries
2217
+ path, verb, arguments, and value facts; doc/version enrichment belongs to
2218
+ rosetta (#175). A consumer that wants enrichment takes
1323
2219
  the **middle** layer: given `{path, verb, args}` plus a version it does its
1324
2220
  own lookups, whereas handing it tokens would make it re-derive from bytes a
1325
2221
  path centrs already resolved. So #223 is a decision about the structure
@@ -1350,8 +2246,102 @@ That is the difference — an accepted flag must do something observable.
1350
2246
  shapes. `lsp-routeros-ts` and
1351
2247
  tikbook are the intended external consumers (hover/diagnostics/completion/
1352
2248
  semantic tokens over these calls); an LSP *protocol* surface on centrs
1353
- stays out of scope (#90) — but the export shape is validated against a real
1354
- LSP consumption spike before it hardens (staging phase 4).
2249
+ stays out of scope (#90). A bounded offline browser/editor consumption check
2250
+ now precedes vocabulary stabilization (#264 B5); full live LSP integration
2251
+ remains later work. The entry is `@tikoci/centrs/explain` and it is browser-verified —
2252
+ see [The offline package entry](#the-offline-package-entry-312).
2253
+
2254
+ ### The offline package entry (#312)
2255
+
2256
+ **`@tikoci/centrs/explain` -> `src/explain.ts`** is the documented public entry for
2257
+ offline analysis. It is a real boundary, not a naming convention: the module
2258
+ graph reachable from it opens no connection, reads no CDB, and touches no
2259
+ filesystem, so it bundles and runs in a browser, a Worker, or an editor host.
2260
+
2261
+ ```sh
2262
+ bun run explain:browser-consumer # boundary + executed consumer report
2263
+ bun run explain:browser-consumer --json # machine-readable
2264
+ ```
2265
+
2266
+ Both halves are gated in the unit suite by
2267
+ `test/unit/explain-browser-entry.test.ts`, so neither needs a router or the
2268
+ corpus.
2269
+
2270
+ **Why a gate and not a note.** The entry reached the transport stack until this
2271
+ landed, for two *pure* helpers: `toYaml` (from `retrieve.ts`) and
2272
+ `canonicalizeExecuteCommand`/`isWriteShaped` (from `execute.ts`). Importing
2273
+ either dragged in `mac-telnet` (`node:dgram`, `node:os`), `native-api`
2274
+ (`node:crypto`) and `ssh`. Both now live in `src/core/` — `src/core/yaml.ts` and
2275
+ `src/core/execute-command.ts` — moved character for character, with `execute.ts`
2276
+ re-exporting the gate because it remains that gate's documented public home.
2277
+ **Nothing about the script-vs-structured gate moved with them**; the locked
2278
+ contract table in `test/unit/execute-canonicalize-contract.test.ts` still imports
2279
+ from `execute` and still passes.
2280
+
2281
+ That coupling was re-introduced once already by a one-line import, which is why
2282
+ the static half of the check names the offending **edge** (`explain.ts ->
2283
+ execute.ts -> mac-telnet-console.ts -> mac-telnet.ts`) rather than leaving a
2284
+ reader with the bundler's polyfill complaint four modules downstream.
2285
+
2286
+ **What "runs in a browser" is proven by.** Compilation is not the proof. A
2287
+ bundler silently substitutes browser polyfills, so the check bundles a worker
2288
+ entry around the module with `--target browser`, wraps the output in a function
2289
+ that shadows every host global (`Bun`, `process`, `require`, `module`,
2290
+ `exports`, `__dirname`, `__filename`), and **spawns that bundle as the Worker**.
2291
+ `Bun` is a non-configurable global — `delete globalThis.Bun` throws — so
2292
+ shadowing, not deletion, is what makes the absence real, and doing it in the
2293
+ bundler's banner keeps the harness free of `eval`/`new Function` while making
2294
+ the bundle text on disk the artifact that carries the claim.
2295
+
2296
+ It then compares the whole `ExplainData` against the Bun-native run, field for
2297
+ field: structure, diagnostics, tokens, spans, `canonical`, and
2298
+ `runtimeAcceptance`. Fourteen cases, covering normal and write-shaped commands,
2299
+ multi-statement and nested input, three malformed classes, Unicode, astral, a
2300
+ leading BOM, IPv6, empty input, and the facets switched off. All fourteen are
2301
+ byte-identical today.
2302
+
2303
+ Two things the bundle deliberately exercises rather than bypasses. It imports
2304
+ through the **published subpath** `@tikoci/centrs/explain`, not
2305
+ `../src/explain.ts`, so a typo in `package.json`'s `exports` map fails the gate
2306
+ instead of leaving it green while every real consumer breaks (the graph walk
2307
+ additionally asserts that the `exports` entry and the measured file are the same
2308
+ path). And it calls `resolveExplainFormat(undefined)`, whose `env` default is
2309
+ the `typeof`-guarded `hostEnv()` — nothing else reaches that parameter, so
2310
+ without this a regression to a bare `Bun.env` would pass every case here while a
2311
+ browser caller of that exported function got a `TypeError`.
2312
+
2313
+ The shadowing is **measured from inside the bundle**, not assumed: the worker
2314
+ reports which host globals it could still see, and an empty list is part of the
2315
+ gate. Without it, a wrapper that silently stopped being emitted would leave
2316
+ every "identical" above produced with host APIs in reach, proving nothing. Only
2317
+ `Bun` and `process` are probed, because they are the only two of the seven a
2318
+ bundle can honestly observe — Bun constant-folds `typeof require` to
2319
+ `"function"` at build time, mentioning `__dirname`/`__filename` makes it inject
2320
+ the build machine's absolute paths as local constants (the probe would cause
2321
+ what it reports), and mentioning `module`/`exports` reclassifies the entry as
2322
+ CommonJS, whose top-level `export default` is a syntax error inside the wrapper.
2323
+ All seven stay shadowed regardless.
2324
+
2325
+ The one tolerated host builtin is **`node:net`**, for `isIP` in
2326
+ `src/explain/values.ts` — a pure predicate with no socket in it. A bundler
2327
+ replaces it with a regex pair, so the browser build genuinely runs a *different*
2328
+ `isIP` than Bun does; the two IPv6 cases exist to price that substitution, which
2329
+ is the only place its verdict is observable. A builtin that polyfills to a stub
2330
+ (`node:fs`) or throws (`node:dgram`) could not be checked this way, so the
2331
+ allowlist is a boundary, not a list to extend.
2332
+
2333
+ **What it does not claim.** Not that the `centrs` package works in a browser —
2334
+ it does not, and is not meant to. Not DOM integration, no real browser engine.
2335
+ And these cases are a boundary corpus, not a parser-coverage corpus; measured
2336
+ parser evidence stays in the censuses and
2337
+ [Device agreement](#device-agreement-for-the-token-partition-264-b4-263).
2338
+
2339
+ **What this hands #264 B5.** The comparison serializes the entire public result,
2340
+ so it pins the vocabulary a consumer actually depends on: every `tokens[]` class
2341
+ name, every span pair, and the `ExplainData` key set are now load-bearing across
2342
+ a process boundary. A B5 rename is therefore a visible break here rather than a
2343
+ silent one — which is the point of settling the vocabulary before an external
2344
+ consumer commits to it.
1355
2345
 
1356
2346
  ## Non-goals
1357
2347
 
@@ -1418,7 +2408,8 @@ of 913). The ratified answers hold for what was measured; they are **not**
1418
2408
  evidence of coverage over device configuration, which is the genre most likely
1419
2409
  to be handed to `explain` and the one that concentrates Q4 path context and
1420
2410
  Q6's bare-path ambiguity. Quote phase-0 numbers per genre, never as one blended
1421
- figure. #203 tracks the capture and the re-score.
2411
+ figure. #203 records the capture decision and caveat; #336 owns the current
2412
+ corpus repin and remeasurement.
1422
2413
 
1423
2414
  Those figures are re-derivable, not eyeballed: `bun run explain:corpus-census`
1424
2415
  (`scripts/explain-corpus-census.ts`) reproduces the whole genre table from
@@ -1430,8 +2421,33 @@ literals, and DHCP client-ids as `:` directives, which under-counts exactly the
1430
2421
  config genre in question). Since phase 0 the corpus has grown to 948 with a
1431
2422
  `tangentsoft` stratum of 35 genuine `/export … terse` device captures, taking
1432
2423
  export-banner share to 4.6%; **that stratum is not part of any phase-0 figure
1433
- above**, and no promoted module has been scored against it yet (#203
1434
- deliverable 2).
2424
+ above**. Current whole-corpus offline censuses include its source text.
2425
+
2426
+ The pinned snapshot carries **three complete builds** — long-term `7.23.5`,
2427
+ stable `7.24.2` and development `7.25beta3` — each with *both* oracles over all
2428
+ 948 scripts, alongside six partial captures (parseIL at 7.20.8, 7.22.1,
2429
+ 7.23rc1; highlight at 7.9.2, 7.23.2, 7.24rc2) that cover only 913. Schema v4
2430
+ names that split rather than leaving it to be discovered by hand: the
2431
+ `v_version_coverage` view classifies each build, and
2432
+ `coverage_class = 'complete'` is the filter every version-comparable query uses.
2433
+ `source_scripts` is byte-identical across both repins that reached this
2434
+ snapshot, so every text-derived report — the token and value censuses,
2435
+ `explain:arg-reach`, the browser-consumer proof — is unchanged by construction,
2436
+ and only the IL-derived operator census moved.
2437
+
2438
+ **The three complete builds do not agree with each other, and that is the
2439
+ finding.** Scored over all 948 scripts:
2440
+
2441
+ | oracle | long-term 7.23.5 ↔ stable 7.24.2 | stable 7.24.2 ↔ development 7.25beta3 |
2442
+ | ------ | -------------------------------: | ------------------------------------: |
2443
+ | `:parse` IL (`il_sha256`) | 87 scripts differ | 11 |
2444
+ | `highlight` (count + class set) | 33 | 0 |
2445
+
2446
+ Stable and development are the same build for these purposes; long-term is the
2447
+ one that moves. So an operator-axis or token-partition figure is a claim *about
2448
+ a channel*, not about RouterOS — which is why the census reports its builds as
2449
+ separate columns and states its version set, rather than collapsing them to one
2450
+ number. Long-term is also the channel a real device most likely runs.
1435
2451
 
1436
2452
  The corpus itself is not in this repo and is not moving here — `lsp-routeros-ts`
1437
2453
  owns producing snapshots, centrs owns which snapshot it measures against.
@@ -1553,75 +2569,106 @@ validity check: `.scratch/q203-blocker-validity.ts`.
1553
2569
  **Non-blocking / deferred (not ratification-gating):** Q5 expression depth, Q7
1554
2570
  tokenizer corners, Q9 potential-command taxonomy, Q12 span-vocabulary draft —
1555
2571
  these refine the surface during implementation but did not gate ratification.
1556
- They keep their homes in staging: **Q7** is now concrete as #201 (lexical
1557
- boundaries — sigil spellings, escape validity, statement-start eligibility)
1558
- and #199 (the shared scanner's `$[…]`-in-string blind spot), both raised by the
1559
- Q13 promotion and both spanning already-promoted modules; **Q5** and **Q9** are
1560
- phase-1 measurements; **Q12** is spec open item 2 and hardens in phase 4. The
2572
+ Two of the four have since closed. **Q7** became #201 (lexical boundaries —
2573
+ sigil spellings, escape validity, statement-start eligibility) and #199 (the
2574
+ shared scanner's `$[…]`-in-string blind spot), both raised by the Q13 promotion
2575
+ and both now closed. **#199**'s fix is the one shared scanner,
2576
+ `src/explain/quoted-string.ts`, which is what keeps the segmenter, the brace
2577
+ classifier and the block scan from drifting on where a string ends; **#201** is
2578
+ the wider one, spanning statement context, sigil/escape reading and defect
2579
+ detection across several already-promoted modules.
2580
+ **Q12** closed with #264: the vocabulary is settled on a stated rule and is no
2581
+ longer provisional — see
2582
+ [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5). **Q5** and
2583
+ **Q9** remain phase-1 measurements. The
1561
2584
  probe framework's disposition (reuse of the mutation suite, coordinate
1562
2585
  fixtures, and adversarial generators as product test fixtures) is tracked in
1563
2586
  issue #186 rather than carried into mainline.
1564
2587
 
1565
2588
  ## Definition of done and staging
1566
2589
 
1567
- `designed` on the strength of this README. When implementation starts, offline
1568
- examples gate via unit/fixture tests and each live cell advances to
1569
- `CHR-passed` only when its `examples.md` entries run green via
1570
- `bun run test:integration` (constitution: done definition). Suggested staging
1571
- (sequence, not schedule):
1572
-
1573
- - **Phase 0 — canonicalization grounding lab — COMPLETE (#185).** Experiments (corpus +
1574
- CHR cross-checks against `:parse`/highlight) established what offline parsing
1575
- can actually achieve: statement segmentation, block/scope resolution,
1576
- sub-command path re-constitution, stateful-per-document path context, symbol
1577
- scope, transport classification, write-shape, malformed-input recovery,
1578
- coordinates, and stress invariants. Every ratification-gating question was
1579
- answered with cited evidence; the findings are folded into the surface above
1580
- and summarized in [Phase-0 ratification](#phase-0-ratification-185). This
1581
- spec is ratified on that basis.
1582
- - **Phase 0.5 — product contract fixtures — SUBSTANTIALLY COMPLETE (#186).**
1583
- Promote a minimal, reviewed subset of the lab into product-owned tests; do
1584
- not import or execute `.scratch/` code. One vertical-slice PR per lab suite,
1585
- each landing an exported module under `src/explain/` plus frozen fixtures
1586
- under `test/fixtures/explain/`. Shipped: Q15 coordinates (#188 →
1587
- `coordinates.ts`), Q1 segmentation (#189 → `segment.ts`), Q2–Q4 path
1588
- resolution (#191 → `pathresolve.ts`), Q17 single-pass container walker
1589
- (#194 → `blocks.ts`, closing #190), Q6 verb/menu boundary (#193 →
1590
- `verbsplit.ts`), Q16 write-shape tristate (#195 → `write.ts`), Q14 cascade /
1591
- context-certainty (#197), Q13 symbol scope (#200 → `symbols.ts`). These pin
1592
- the ratified thresholds: no confident command invented after a defect, zero
1593
- false negatives on statically obvious writes, 100% coordinate invariants,
1594
- deterministic/well-formed spans, bounded depth, no throw, and roughly linear
1595
- scaling. **Remaining:** #192 (Q14 defect-*region* spans and the
1596
- `ambiguous`/`unknown` verdict vocabulary — the verdict half wants the phase-1
1597
- envelope). The lab framework and its two ~7.5 MB per-character highlight
1598
- captures stay out of mainline (#186). What durable files CITE does not stay
1599
- out: the frozen dev/holdout split, the Q8 REST probe capture, a stratified
1600
- slice of the highlight streams, and the three device re-derivation probes
1601
- are committed under `test/fixtures/explain/` and `scripts/probes/`.
1602
- - **Phase 1 — offline core** — the grown canonicalizer: structure + gate
1603
- verdict + per-statement resolution and transport classification +
1604
- diagnostics (+ `--curl` rendering), and the CLI/MCP surface over it. Phase
1605
- 0.5 shipped the analysis modules as library exports only; phase 1 composes
1606
- them behind one entry point, adds the envelope and `src/cli/explain.ts`, and
1607
- turns `examples.md` green. **Landing in four PRs:** #192a defect regions
1608
- (#222), #202a composition + envelope (#224), **#202b the CLI surface + the
1609
- offline examples**, then #202c-1 argument reading and #202c-2 transport
1610
- classification + `--curl`, which close #202. Carried in with it: the lexical-boundary
1611
- hardening (#201) and the shared-scanner substitution blind spot (#199), both
1612
- found during Q13 and both spanning modules already promoted; plus the
1613
- deferred lab questions Q5 (expression depth) and Q9 (potential-command
1614
- taxonomy), which become focused tests as this surface hardens.
1615
- - **Phase 2 — live probes** — highlight + `:parse` (+
1616
- completion/child/syntax facets) over rest-api/native-api with the safety rules
1617
- above, including the
1618
- broad-query describe ladder, smart sizing, and CHR-backed anchors for Q6,
1619
- Q8, Q10, and Q11.
1620
- - **Phase 3 — facet polish** — `--complete`/`--schema` ergonomics, truncation
1621
- counts/hints, `--full`.
1622
- - **Phase 4 — library/LSP alignment** — export shape hardened against a real
1623
- lsp-routeros-ts consumption spike (semantic tokens via the centrs span
1624
- vocabulary + color map).
2590
+ Verification follows the [constitution](../../docs/CONSTITUTION.md#done-definition).
2591
+ The phase-0 grounding and promoted phase-1 CLI/library core (#185/#202) are
2592
+ already in the product. The remaining task graph is maintained in #90; issue
2593
+ closure for an earlier slice does not establish the baseline below.
2594
+
2595
+ ### Offline baseline acceptance
2596
+
2597
+ The offline capability can advance from `coded` to `verified` when:
2598
+
2599
+ - **Examples and diagnostics:** the offline examples and focused analysis/CLI
2600
+ contract tests pass. Known false confident readings in the baseline task
2601
+ list, starting with #311's missing separator, are fixed with source-located
2602
+ diagnostics and accepted/malformed controls. Unknown or ambiguous input
2603
+ remains explicit; no fabricated statements, paths, values, or runtime acceptance.
2604
+ - **Bounded execution:** the existing public `explainCommand` performance,
2605
+ coordinate, depth, no-throw, and determinism contracts pass. A reproduced
2606
+ performance failure is investigated before changing its threshold (#313). Cost
2607
+ must grow with input size and no faster: a wall-clock budget on a thermally
2608
+ throttled machine measures the machine, so the growth **shape** is pinned by a
2609
+ ratio between two sizes timed back to back, and an absolute threshold is never
2610
+ raised in place of finding the growing term. One input shape is not enough —
2611
+ #313's separator-free input carries no symbols, so it never reaches the
2612
+ symbol/value machinery where the next growth term was hiding (#317), and
2613
+ neither of those carries a `do={…}`, which is where the third one was (#320).
2614
+ A guarded shape is only guarded for the analyzers it actually runs, so a new
2615
+ growth guard states which analyzers its input reaches and is mutation-tested
2616
+ against the term it removed. **Met:** #322's fixed-size depth guard grows
2617
+ nesting 16× (depth 4 → 64) under the public token/curl path and stays below
2618
+ the 6× bound; deterministic companion guards cover colon-led, bare-directive,
2619
+ retry, array/substitution, bracket, and string-interpolation shapes and fail
2620
+ when child-scope traversal is restored.
2621
+ - **Measured token surface:** the total, gapless `--tokens` partition and
2622
+ generated censuses remain reproducible; #264 B4 / #263's projection and
2623
+ device-agreement report are in
2624
+ [Device agreement](#device-agreement-for-the-token-partition-264-b4-263),
2625
+ re-derivable without a router or the corpus. Report decided errors,
2626
+ abstentions, version/state differences, and corpus bias separately — the
2627
+ report's outcome buckets and its four applicability categories are that
2628
+ separation. Classified-byte percentage is not syntax coverage, and agreement
2629
+ is a trend, not a new numerical pass gate. A token-level rule and a token fill
2630
+ read the skip-tolerant argument stream, never the strict REST reading (#316);
2631
+ the two share one boundary walk, the strict reading's reach is unchanged, and
2632
+ the skip-vs-prefix fork is priced rather than assumed —
2633
+ [The token stream a rule reads](#the-token-stream-a-rule-reads-316). That
2634
+ stream reaches the envelope as the token partition and nowhere else, which is
2635
+ what makes the `=` boundary visible to a consumer whose statement published
2636
+ no arguments at all (#264 B5).
2637
+ - **Usable library contract:** one bounded browser/editor consumer imports a
2638
+ documented public offline entry point, runs analysis without transport/CDB
2639
+ dependencies, and verifies structure, diagnostics, tokens, and source-position
2640
+ mapping against the CLI/library result (#312). **Met** by `@tikoci/centrs/explain` and
2641
+ `bun run explain:browser-consumer` — see
2642
+ [The offline package entry](#the-offline-package-entry-312); the boundary is a
2643
+ module-graph gate and the consumer proof executes the browser bundle rather
2644
+ than only compiling it. The token vocabulary that consumer reads is settled by
2645
+ #264 — [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) —
2646
+ on a stated rule, with the two splits it makes (`arg-sep`, then `escaped`)
2647
+ and the merges it keeps each recorded there with the surface that already
2648
+ carries the distinction. A complete editor, LSP server, palette
2649
+ reproduction, or SCIP implementation is not required to prove this contract.
2650
+ - **Reachable evidence and explicit residue:** changed RouterOS semantic claims
2651
+ have committed CHR-grounded controls and a reproducible capture/replay path.
2652
+ Every remaining issue has a stated scope and dependency; #225 distinguishes
2653
+ offline value work from live types, #272 owns evidence reachability, and #211's
2654
+ unlisted-path decision retains the current abstention until separately decided.
2655
+
2656
+ Retain the existing checks: `bun run lint`, `bun run test`, `bun run build`, and
2657
+ `bun run lint:ci`, plus the affected CHR grounding/integration procedure from
2658
+ the constitution. Add the consumer and agreement checks with their implementation.
2659
+ No new rules engine, schema snapshot, final-state interpreter, or network-behavior
2660
+ test framework is part of this baseline.
2661
+
2662
+ ### Current and following capabilities
2663
+
2664
+ - **Current — live probes (#236, phase 2):** highlight + `:parse` and
2665
+ completion/child/syntax over rest-api/native-api. B4's projection/agreement
2666
+ informs this work; each live cell requires its own CHR-passing examples.
2667
+ - **MCP alignment (#223):** the deliberate flat-to-rich response migration,
2668
+ coordinated with rosetta. Independent of the offline token vocabulary.
2669
+ - **Facet polish (phase 3):** live sizing, `--full`, completion/schema ergonomics.
2670
+ - **Full LSP integration (phase 4):** consume live and offline results in
2671
+ `lsp-routeros-ts`, building on the earlier offline consumer check.
1625
2672
 
1626
2673
  ## Decisions (2026-07-19 round) and remaining opens
1627
2674
 
@@ -1689,13 +2736,17 @@ Still open (implementation opens — none block ratification):
1689
2736
 
1690
2737
  1. Final flag names (`--full`, `--curl`, facet names) and the smart-sizing
1691
2738
  thresholds (what counts as "cheap", what triggers truncation).
1692
- 2. The centrs span vocabulary itself: class list, mapping table from RouterOS
1693
- highlight classes (and their colors), and how unknown/new upstream classes
1694
- degrade. Phase 0 pinned the raw inputs (Q13's per-occurrence highlight-class
1695
- corpus; the observed 19 classes on 7.23.2 plus drift on 7.24rc2) but the
1696
- centrs-owned class list and color map are a **draft** (lab question Q12,
1697
- non-ratification-gating); they harden in staging phase 4 against the real
1698
- lsp-routeros-ts consumption spike.
2739
+ 2. The mapping table from RouterOS highlight classes to **colors**, and how
2740
+ unknown/new upstream classes degrade. Phase 0 pinned the raw inputs (Q13's
2741
+ per-occurrence highlight-class corpus; the observed 19 classes on 7.23.2 plus
2742
+ drift on 7.24rc2). The centrs-owned **class list is no longer open**: #264 B5
2743
+ settled it on a stated rule and `data.tokens[].class` is no longer provisional
2744
+ ([The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5)), and the
2745
+ projection onto the device's own classes is reported by
2746
+ `bun run explain:highlight-agreement`
2747
+ ([Device agreement](#device-agreement-for-the-token-partition-264-b4-263)).
2748
+ What stays open here is the color map and the drift policy. Live mappings
2749
+ and full LSP integration follow separately.
1699
2750
  3. Whether the live describe ladder needs result caching per target+version
1700
2751
  (probe cost vs freshness) — deferred to implementation evidence.
1701
2752