@tikoci/centrs 0.1.4 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +417 -1
- package/GLOSSARY.txt +40 -1
- package/README.md +16 -8
- package/commands/AGENTS.md +7 -4
- package/commands/api/AGENTS.md +13 -1
- package/commands/api/README.md +50 -10
- package/commands/api/examples.md +100 -2
- package/commands/devices/README.md +10 -1
- package/commands/devices/examples.md +33 -2
- package/commands/execute/README.md +68 -25
- package/commands/execute/examples.md +168 -7
- package/commands/explain/README.md +1206 -155
- package/commands/explain/examples.md +99 -3
- package/commands/mcp/README.md +9 -5
- package/commands/mcp/examples.md +2 -2
- package/commands/retrieve/examples.md +20 -0
- package/commands/terminal/examples.md +7 -1
- package/docs/CLI.md +3 -3
- package/docs/CONSTITUTION.md +83 -21
- package/docs/MATRIX.md +89 -52
- package/docs/errors/cdb/create-failed.md +20 -0
- package/docs/errors/cdb/file-exists.md +23 -0
- package/docs/errors/input/invalid-target.md +25 -3
- package/docs/errors/routeros/invalid-value.md +3 -2
- package/docs/errors/transport/cancel-unacknowledged.md +24 -0
- package/docs/errors/validation/syntax.md +29 -4
- package/package.json +13 -2
- package/src/api-fanout.ts +17 -4
- package/src/api.ts +346 -75
- package/src/cli/api.ts +1 -1
- package/src/cli/btest.ts +1 -1
- package/src/cli/devices.ts +16 -3
- package/src/cli/explain.ts +1 -1
- package/src/cli/terminal.ts +1 -1
- package/src/core/envelope.ts +53 -0
- package/src/core/error-catalog.ts +16 -1
- package/src/core/execute-command.ts +455 -0
- package/src/core/inspect.ts +58 -26
- package/src/core/routeros-errors.ts +66 -0
- package/src/core/version.ts +4 -0
- package/src/core/yaml.ts +86 -0
- package/src/data/winbox-cdb-write.ts +22 -8
- package/src/devices.ts +142 -28
- package/src/execute-fanout.ts +19 -5
- package/src/execute.ts +247 -295
- package/src/explain/arg-tokens.ts +62 -25
- package/src/explain/args.ts +398 -52
- package/src/explain/blocks.ts +54 -11
- package/src/explain/brace-tokens.ts +4 -3
- package/src/explain/catalog.ts +42 -44
- package/src/explain/coordinates.ts +159 -0
- package/src/explain/defects.ts +2 -1
- package/src/explain/escape-tokens.ts +77 -0
- package/src/explain/highlight-projection.ts +308 -0
- package/src/explain/menus.ts +9 -5
- package/src/explain/operator-tokens.ts +13 -7
- package/src/explain/operators.ts +27 -20
- package/src/explain/path-tokens.ts +1 -1
- package/src/explain/pathresolve.ts +358 -113
- package/src/explain/quoted-string.ts +110 -20
- package/src/explain/scope-brace.ts +175 -22
- package/src/explain/segment.ts +429 -21
- package/src/explain/separator.ts +187 -0
- package/src/explain/split-owner.ts +194 -0
- package/src/explain/string-tokens.ts +14 -4
- package/src/explain/symbol-values.ts +209 -173
- package/src/explain/symbols.ts +21 -1
- package/src/explain/transport.ts +17 -0
- package/src/explain/value-tokens.ts +15 -12
- package/src/explain/verbsplit.ts +165 -23
- package/src/explain/write.ts +47 -6
- package/src/explain.ts +522 -72
- package/src/index.ts +21 -2
- package/src/mcp/server.ts +2 -1
- package/src/mcp/tools.ts +1 -1
- package/src/offline-gate.ts +148 -0
- package/src/protocols/adapter.ts +12 -1
- package/src/protocols/mac-telnet-console.ts +6 -4
- package/src/protocols/native-api.ts +29 -3
- package/src/resolver/target.ts +32 -12
- package/src/retrieve-fanout.ts +1 -1
- package/src/retrieve.ts +18 -63
- package/src/settings.ts +2 -1
- package/src/transfer-fanout.ts +1 -1
- package/src/transfer.ts +1 -1
|
@@ -9,20 +9,13 @@ a path, with completion-style candidates for building the right command.
|
|
|
9
9
|
validation): the cheap, safe knowledge tier in front of the runners
|
|
10
10
|
(`execute` / `api` / `retrieve`).
|
|
11
11
|
|
|
12
|
-
Status: `
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
is now ratified** — every ratification-gating question was answered with cited
|
|
20
|
-
evidence (see [Phase-0 ratification](#phase-0-ratification-185)). The findings
|
|
21
|
-
folded in below; a few of them **amend** the surface (tristate `containsWrite`,
|
|
22
|
-
an `ambiguous` canonicalizer verdict, a defect-region diagnostics contract, a
|
|
23
|
-
byte-count-preserving coordinate rule, the native-api `:parse` readout). Flag
|
|
24
|
-
names and sizing thresholds stay provisional — those are implementation opens,
|
|
25
|
-
not ratification blockers.
|
|
12
|
+
Status: offline baseline `verified`; live probes over `rest-api` and `native-api`
|
|
13
|
+
`designed`. **`centrs explain '<input>'` runs today**, without a router or CDB.
|
|
14
|
+
[`docs/MATRIX.md`](../../docs/MATRIX.md#offline-analysis) tracks these separately;
|
|
15
|
+
[#90](https://github.com/tikoci/centrs/issues/90) indexes the remaining tasks.
|
|
16
|
+
The spec is ratified on the [phase-0 grounding lab](#phase-0-ratification-185).
|
|
17
|
+
The current finish line is [Offline baseline acceptance](#offline-baseline-acceptance),
|
|
18
|
+
not completion of the live protocol cells.
|
|
26
19
|
Load-bearing rules — envelope, errors, settings precedence, identity,
|
|
27
20
|
validation, protocol selection — live in
|
|
28
21
|
[`docs/CONSTITUTION.md`](../../docs/CONSTITUTION.md).
|
|
@@ -62,6 +55,10 @@ validation, protocol selection — live in
|
|
|
62
55
|
|
|
63
56
|
## Where `explain` sits
|
|
64
57
|
|
|
58
|
+
- **Shared analysis across transports.** The offline result is independently
|
|
59
|
+
useful to libraries, agents, and editors. Its facts can later accompany
|
|
60
|
+
other commands' envelopes, irrespective of their transport; the rich
|
|
61
|
+
analysis is not currently applied as a runner policy gate.
|
|
65
62
|
- **Never executes, never mutates.** All live probes (`/console/inspect`
|
|
66
63
|
highlight/completion/syntax/child and `:put [:parse …]`) are read-only
|
|
67
64
|
inspection. Execution-based probes (required-argument discovery via
|
|
@@ -84,6 +81,16 @@ validation, protocol selection — live in
|
|
|
84
81
|
to **support one**: hover, diagnostics, completion, and semantic-token needs
|
|
85
82
|
of `lsp-routeros-ts`/tikbook are first-class consumers of the export shape.
|
|
86
83
|
|
|
84
|
+
Normalized source structure is the foundation for possible configuration
|
|
85
|
+
checks. Preserve command order, selectors, source locations, literal/computed
|
|
86
|
+
value distinctions, and unknowns; an input fragment does not establish final
|
|
87
|
+
device state. Script-variable bindings are not yet an object-reference model
|
|
88
|
+
for interfaces, bridges, or other configuration entities. Security advice,
|
|
89
|
+
rule collections, and a browser/Monaco inspector are possible consumers, not
|
|
90
|
+
new requirements to implement those products during the offline baseline.
|
|
91
|
+
quickchr owns the CHR provisioning/test machinery; rosetta supplies docs and
|
|
92
|
+
reference knowledge. This direction adds no runtime dependency on either.
|
|
93
|
+
|
|
87
94
|
## Canonicalization is the core
|
|
88
95
|
|
|
89
96
|
The engine is a **grown `canonicalizeExecuteCommand`** (decision: grow the
|
|
@@ -165,16 +172,16 @@ today's path/verb/args split and script-vs-structured gate:
|
|
|
165
172
|
The *container* half of that pair is since decidable. `src/explain/menus.ts`
|
|
166
173
|
(#207) bakes only the menu-vs-command node type — not a schema — and it is
|
|
167
174
|
version-less because the pinned trees show no `dir`↔`cmd` flip across
|
|
168
|
-
7.10.2 → 7.
|
|
175
|
+
7.10.2 → 7.25beta5 or across architectures. So `/ip/route` now resolves as
|
|
169
176
|
`navigation` (#210). The floor is unmoved: the table is deliberately
|
|
170
177
|
incomplete, absence from it decides nothing, and every rule above it still
|
|
171
178
|
refuses rather than guesses.
|
|
172
179
|
|
|
173
180
|
The *command* half is what #228 addressed, by adding a second first-order
|
|
174
181
|
source rather than a schema. MikroTik's CLI Reference is generated from the
|
|
175
|
-
definition structs, and measured against the same
|
|
176
|
-
**zero navigation-vs-command contradictions** across
|
|
177
|
-
paths,
|
|
182
|
+
definition structs, and measured against the same six pinned trees it has
|
|
183
|
+
**zero navigation-vs-command contradictions** across 959 exactly-matching
|
|
184
|
+
paths, four RouterOS versions and two architectures; all but one of the 94
|
|
178
185
|
published paths absent from every tree carry a `package`/`conditions`/`syscap`
|
|
179
186
|
gate — their own or an ancestor's — that predicts the absence.
|
|
180
187
|
`src/explain/catalog.ts` unions the two with per-entry provenance, which is
|
|
@@ -198,6 +205,33 @@ today's path/verb/args split and script-vs-structured gate:
|
|
|
198
205
|
inventory reshape fails loudly instead of shrinking the table (#285,
|
|
199
206
|
tikoci/rosetta#137).
|
|
200
207
|
|
|
208
|
+
**Source refresh (#335, accepted 2026-09-13).** The inspect pins retain the
|
|
209
|
+
7.10.2 and 7.16 history anchors, then pair x86 and arm64 for both current
|
|
210
|
+
stable 7.24.2 and prerelease 7.25beta3. Pairing architectures makes additions
|
|
211
|
+
attributable to version rather than to an architecture swap while preserving
|
|
212
|
+
the hardware-only menu floor. Reviewed independently, the inspect union moves
|
|
213
|
+
from 615 to 617 menus: 7.24.2 removes `/ip/ipsec/key/qkd`, and 7.25beta3 adds
|
|
214
|
+
`/interface/ethernet/pon`, `/interface/xfrm`, and `/ip/ssh/known-hosts`.
|
|
215
|
+
Separately, the current CLI Reference moves from 1,070 pages / 1,077 entries
|
|
216
|
+
to 1,053 / 1,060: it adds
|
|
217
|
+
`/interface/ethernet/switch/rule/reset-counters`, removes the five
|
|
218
|
+
`/interface/bridge/msrp` paths plus `/root`, moves 12 paths from `both` to
|
|
219
|
+
`inspect`, and changes 16 published gate strings. Those current-only
|
|
220
|
+
publication changes are accepted: none introduces a kind contradiction. The
|
|
221
|
+
coherent generated catalog has 1,120 paths: 959 `both`, 67 `inspect`, and 94
|
|
222
|
+
`published`; it still reports zero navigation-vs-command contradictions.
|
|
223
|
+
|
|
224
|
+
**Source refresh (accepted 2026-09-27).** The two current pins advance to the
|
|
225
|
+
channels then live — stable 7.24.2 → 7.24.4 and development 7.25beta3 →
|
|
226
|
+
7.25beta5 — both still architecture-paired, both history anchors untouched.
|
|
227
|
+
The refresh is a structural no-op: the inspect union holds at 617 menus and
|
|
228
|
+
the catalog at 1,120 paths, with no path added or removed and no kind change.
|
|
229
|
+
What moved is current-only publication: the CLI Reference grows from 1,053
|
|
230
|
+
pages / 1,060 entries to 1,055 / 1,062, which documents
|
|
231
|
+
`/interface/ethernet/pon` and `/ip/ssh/known-hosts` for the first time and so
|
|
232
|
+
moves both from `inspect` to `both` — 961 `both`, 65 `inspect`, 94
|
|
233
|
+
`published`. Zero navigation-vs-command contradictions, unchanged.
|
|
234
|
+
|
|
201
235
|
There are real unknowns in how far offline parsing can go (expression
|
|
202
236
|
grammar, scope fidelity vs `:parse`, `[]`-nesting corner cases). Grounding
|
|
203
237
|
those unknowns was **phase 0** (#185); its findings are folded in throughout
|
|
@@ -262,7 +296,10 @@ command accepts.
|
|
|
262
296
|
Offline mode does ship two generated **structure** tables, baked at build time
|
|
263
297
|
from pinned sources and read as ordinary closed lists. They may say whether a
|
|
264
298
|
path is navigation or a command; they never describe what a command accepts.
|
|
265
|
-
Absence from either abstains and never rejects.
|
|
299
|
+
Absence from either abstains and never rejects. Regenerate with
|
|
300
|
+
`bun run explain:menus` / `bun run explain:catalog`; `--check` drift-gates both
|
|
301
|
+
in QA. Do not hand-edit them — each file's header carries its source pin and
|
|
302
|
+
rationale.
|
|
266
303
|
|
|
267
304
|
- `src/explain/menus.ts` (#207) — container paths from pinned restraml
|
|
268
305
|
`/console/inspect` trees. **Read today** to confirm navigation, on its own by
|
|
@@ -499,7 +536,8 @@ Standard envelope (constitution: result envelope); `data` sketch:
|
|
|
499
536
|
severity channels. **Malformed input carries a defect *region*** (Q14): each
|
|
500
537
|
diagnostic points at the byte span of its defect, and the detectable classes
|
|
501
538
|
are {unbalanced delimiter, unterminated string, invalid escape, invalid sigil,
|
|
502
|
-
invalid unquoted hash, BOM, non-ASCII,
|
|
539
|
+
invalid unquoted hash, invalid apostrophe (#355), BOM, non-ASCII,
|
|
540
|
+
over-depth nesting}. **Two classes named
|
|
503
541
|
in the phase-0 draft
|
|
504
542
|
of this list are deliberately deferred** (maintainer decision, #192), because
|
|
505
543
|
each would require offline to assert something it cannot prove:
|
|
@@ -570,19 +608,24 @@ and its phase is named below.
|
|
|
570
608
|
all: a `[…]`/`(…)`/`$x`/`{…}` value, an escape this phase does not decode, a
|
|
571
609
|
left-hand side that is not a RouterOS name, a token split by a continuation.
|
|
572
610
|
Never partially read, because a dropped argument silently changes what a
|
|
573
|
-
rendered `curl` DOES.
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
advisory value-anchor view is intentionally wider only for exact array
|
|
611
|
+
rendered `curl` DOES. One refusal is not lexical at all, but a place where
|
|
612
|
+
centrs's two readers disagree about an **unquoted** value: a `\f`/`\v`,
|
|
613
|
+
which the retired execute gate split tokens on (JavaScript `\s`) and every
|
|
614
|
+
explain module does not (ASCII whitespace). The device-correct reading is
|
|
615
|
+
the analysis's, but that gate could not be corrected — so phase 1 publishes
|
|
616
|
+
neither rather than putting two confident values in one result. Inside a
|
|
617
|
+
`"…"` run the two agree and nothing is refused.
|
|
618
|
+
|
|
619
|
+
The advisory value-anchor view is intentionally wider only for exact array
|
|
583
620
|
literals: a non-empty `{…}` value or a parenthesized run with a depth-zero
|
|
584
621
|
comma is locatable and hints `array`, while the strict REST view continues
|
|
585
622
|
to refuse the whole list.
|
|
623
|
+
|
|
624
|
+
An unquoted `'` used to be a second such disagreement, described here as a
|
|
625
|
+
character RouterOS treats as ordinary and the gate treats as a quote. That
|
|
626
|
+
reading was wrong on both halves and #355 settled it: `'` is not a RouterOS
|
|
627
|
+
token at all, so `comment=it's` is REJECTED by the device rather than
|
|
628
|
+
carrying either value, and the quote-counting gate that disagreed is gone.
|
|
586
629
|
- **A token is read but carries no `value`** — the list still reads. The token
|
|
587
630
|
is delimited and classified; only its literal value is unknowable, as for
|
|
588
631
|
the positional in `:log info "result: $[…]"`. **`value` absent means there
|
|
@@ -882,11 +925,11 @@ and its phase is named below.
|
|
|
882
925
|
`bun run explain:value-census:readme` and gated against it by
|
|
883
926
|
`bun run explain:value-census:readme:check`; the fixture itself is gated
|
|
884
927
|
against a fresh corpus run by `bun run explain:value-census:check`. Of the
|
|
885
|
-
13,
|
|
928
|
+
13,161 emitted values in a statement the strict argument lexer ALSO read, 0
|
|
886
929
|
disagree with it on half-open byte span or decoded text, while the
|
|
887
|
-
prefix-safe scan retains a further 6,
|
|
888
|
-
strict REST reading abstains. Of 19,
|
|
889
|
-
members (529 keyed, 1,147 nested inside another member) and
|
|
930
|
+
prefix-safe scan retains a further 6,538 values across 568 statements whose
|
|
931
|
+
strict REST reading abstains. Of 19,699 emitted occurrences, 5,636 are array
|
|
932
|
+
members (529 keyed, 1,147 nested inside another member) and 808 are arrays;
|
|
890
933
|
the corpus holds no source-literal `id` example. The three structural
|
|
891
934
|
counters — spans addressing bytes outside their own source, members naming a
|
|
892
935
|
container that does not exist, members escaping the container they name —
|
|
@@ -1003,7 +1046,9 @@ navigated, so the context stays as unknown as it already was.
|
|
|
1003
1046
|
(string-internal unknown/lowercase-hex/truncated — CHR `highlight` `error`
|
|
1004
1047
|
`:parse` `expected message value`, grounded stable+testing #247),
|
|
1005
1048
|
`bad-sigil`,
|
|
1006
|
-
`invalid-hash
|
|
1049
|
+
`invalid-hash`, `invalid-apostrophe` (#355, below),
|
|
1050
|
+
`missing-statement-separator` (#311, below). Nine classes
|
|
1051
|
+
the device itself rejects.
|
|
1007
1052
|
- `warning` — `over-depth`, because it is centrs's own resource bound and says
|
|
1008
1053
|
nothing about whether the input is legal; and an `ambiguous`/`unknown`
|
|
1009
1054
|
resolution, never an error, so the default `--fail-on error` cannot fail a
|
|
@@ -1013,6 +1058,386 @@ navigated, so the context stays as unknown as it already was.
|
|
|
1013
1058
|
fail), and `context-lost`, which reports a reading that is correct while the
|
|
1014
1059
|
document's menu context was already gone.
|
|
1015
1060
|
|
|
1061
|
+
### The missing statement separator (#311)
|
|
1062
|
+
|
|
1063
|
+
Two commands written with no `;` or newline between them used to read as one
|
|
1064
|
+
command with extra operands, and offline reported `pass`: `/ip/address add
|
|
1065
|
+
interface=ether1 /ip/route add gateway=192.168.88.1` published `gateway` as an
|
|
1066
|
+
attribute of `/ip/address add`. The rule that closes it is a known **menu path**
|
|
1067
|
+
in operand position followed by a console **verb** — never "a `/` after the
|
|
1068
|
+
verb", because a slash-shaped operand is legal and `/file remove
|
|
1069
|
+
/flash/skins/foo.html` lowers cleanly to `numbers=/flash/skins/foo.html`. The
|
|
1070
|
+
menu spelling may be slashed, spaced, or mixed (Q3 R4); the device rejects
|
|
1071
|
+
`/ip route add` exactly as it rejects `/ip/route add`.
|
|
1072
|
+
|
|
1073
|
+
**The device evidence is not what the report said, and the correction widened
|
|
1074
|
+
the rule.** #311 gave the error byte as the `=` of the trailing attribute
|
|
1075
|
+
(`expected end of command (line 1 column 55)`). That byte is an artefact of that
|
|
1076
|
+
head: `add` abbreviates `/ip/address/add`'s own `address=` argument, so the
|
|
1077
|
+
console reads `add gateway` as `address=gateway` and only complains at the `=`
|
|
1078
|
+
after it. Change the head so the second verb abbreviates nothing and the answer
|
|
1079
|
+
changes shape — `/system/note set note=hello /ip/route add gateway=1.2.3.4` is
|
|
1080
|
+
no hard reject at all; `:parse` returns IL carrying `bad parameter /ip/route
|
|
1081
|
+
(line 1 column 38)`, naming the second **path**. Both are refusals, and which
|
|
1082
|
+
one appears depends on per-menu argument names, which offline does not have. So
|
|
1083
|
+
the grounded claim is the negative one: across every probed spelling on CHR
|
|
1084
|
+
7.24.2 and 7.23.5, RouterOS answers `bad parameter <path>` or `expected end of
|
|
1085
|
+
command`, and never accepts the shape. A trailing attribute is therefore **not**
|
|
1086
|
+
required, which #311's own example had made it look like.
|
|
1087
|
+
|
|
1088
|
+
The head-dependence goes one step further than "which wording", and it bounds
|
|
1089
|
+
what the diagnostic may say. On a head whose own value slot takes a bare
|
|
1090
|
+
positional, the second path is absorbed into that slot and the device refuses
|
|
1091
|
+
the **verb** instead: `:put /ip/route print` lowers to `/putmessage=/ip/route`
|
|
1092
|
+
and answers `bad parameter print`, and `:local a /ip/route add` lowers to
|
|
1093
|
+
`/localname=$a;value=/ip/route` and answers `bad parameter add`. Both are still
|
|
1094
|
+
refusals — `/system/script/run /ip/route`, the same shape with no verb after
|
|
1095
|
+
the path, lowers cleanly to `number=/ip/route` — so the offline `error` stands.
|
|
1096
|
+
But the message says the bytes **read as** a second command, which is centrs's
|
|
1097
|
+
heuristic reading, and never that the device starts one there or which token it
|
|
1098
|
+
will name. Every row above is pinned in
|
|
1099
|
+
`test/integration/explain-separator.test.ts`.
|
|
1100
|
+
|
|
1101
|
+
**The contradicted reading is withdrawn, not patched.** The statement stays
|
|
1102
|
+
`resolved` — `/ip/address add` really is the head, and `highlight` agrees. What
|
|
1103
|
+
is withdrawn is withdrawn by three separate gates, each with its own trigger:
|
|
1104
|
+
|
|
1105
|
+
1. **The argument list.** `withoutSeparatedArguments` drops `command.args` and
|
|
1106
|
+
sets `arguments.read` to `false`. This is the same all-or-nothing move
|
|
1107
|
+
`enforceGateParity` makes for a gate/analysis disagreement, but it is a
|
|
1108
|
+
distinct gate on a distinct trigger — the two never consult each other. The
|
|
1109
|
+
device says the move is right rather than merely cautious:
|
|
1110
|
+
`… /ip/route add placeholder` lowers to
|
|
1111
|
+
`address=placeholder;;interface=ether1`, a phantom attribute the source never
|
|
1112
|
+
wrote, because the second run's words can abbreviate the HEAD's argument
|
|
1113
|
+
names.
|
|
1114
|
+
2. **The invocation.** `rejectedBySeparator(runs)` sets the new
|
|
1115
|
+
`ExplainTransportInput.rejected`, which `classifyExplainTransport` honours
|
|
1116
|
+
ahead of every mapping rule and answers `unknown`. Without it the degraded
|
|
1117
|
+
`arguments` would fall through to `execute` and print a ready-to-paste
|
|
1118
|
+
`centrs execute '<router>' '<the malformed text>'` for bytes the same result
|
|
1119
|
+
calls a syntax error. **Unread is not rejected** — an unread argument list
|
|
1120
|
+
still legitimately rides the raw CLI surface, and that path is unchanged.
|
|
1121
|
+
3. **The value facts from the run onward.** `values.occurrences` is an axis of
|
|
1122
|
+
its own, not a projection of `arguments` — the value census above pins how
|
|
1123
|
+
many values it retains in statements whose strict REST reading abstains — so
|
|
1124
|
+
gate 1 does not reach it and a cut is needed. The cut is the run's first byte, and
|
|
1125
|
+
the device draws the line in the same place: `:parse` reads `gateway` as the
|
|
1126
|
+
VALUE of `/ip/address/add`'s own `address=`, never as an attribute NAME,
|
|
1127
|
+
while the head's `interface=ether1` survives into the IL (as `note=hello`
|
|
1128
|
+
does in the `/system/note set` row). Values before the cut therefore stay;
|
|
1129
|
+
values reaching into or past it are dropped, and with them the `value` token
|
|
1130
|
+
class over those bytes.
|
|
1131
|
+
|
|
1132
|
+
**Abstentions, stated.** A menu path with no verb after it (`… /ip/route`) and a
|
|
1133
|
+
bare operand a menu does not accept (`… placeholder`, `/file remove a b`) also draw
|
|
1134
|
+
`bad parameter`, but that is the generic "no such positional" complaint;
|
|
1135
|
+
deciding it needs the per-menu positional list, which `catalog.ts` deliberately
|
|
1136
|
+
is not, and a separator is not necessarily the fix. Both abstain. The shape
|
|
1137
|
+
filter over a candidate path does admit a leading digit, because
|
|
1138
|
+
`/interface/6to4` is a real menu in both structure tables; presence in those
|
|
1139
|
+
tables, never the filter, is what decides. Over the 948-script pinned corpus the
|
|
1140
|
+
rule fires **0 times**, so no published census figure moves; the 38 corpus
|
|
1141
|
+
scripts the device answers `expected end of command` for are all other causes
|
|
1142
|
+
(line continuations, `<tab>`/`<VALUE>` placeholders, pasted shell and Python,
|
|
1143
|
+
stray parens), which is #203's genre bias showing.
|
|
1144
|
+
|
|
1145
|
+
**The reach limit is closed (#316).** This rule used to read
|
|
1146
|
+
`statement.arguments.tokens`, the all-or-nothing strict reading, so a statement
|
|
1147
|
+
holding a variable carried no tokens and the run in it was missed:
|
|
1148
|
+
`/ip/address add interface=ether1 /ip/route add gateway=$g` reported `pass`
|
|
1149
|
+
where its literal twin reported the separator. It now reads the skip-tolerant
|
|
1150
|
+
token stream described below, and both spellings report the separator — as does
|
|
1151
|
+
`/ip/address add interface=$x /ip/route add gateway=1.1.1.1`, where the
|
|
1152
|
+
undecodable token comes FIRST and a prefix-only reading would still have missed
|
|
1153
|
+
it. The corpus count is unchanged at **0** for the same reason it always was
|
|
1154
|
+
(#203's genre bias), so this buys reach rather than a number. What still stops
|
|
1155
|
+
the rule is an unterminated string or an unbalanced delimiter, because neither
|
|
1156
|
+
walk can find a boundary to resume from.
|
|
1157
|
+
|
|
1158
|
+
### The token stream a rule reads (#316)
|
|
1159
|
+
|
|
1160
|
+
`lexArguments` is all-or-nothing by design: its consumer renders a runnable REST
|
|
1161
|
+
request, and a partially-read argument list changes what the rendered command
|
|
1162
|
+
DOES. That is the right reading for a renderer and the wrong one for a rule —
|
|
1163
|
+
the first token it declines discards every token already decoded, which put
|
|
1164
|
+
**48.2%** of the corpus's argument-bearing statements out of reach of every
|
|
1165
|
+
token-level rule and of the `arg` token fill.
|
|
1166
|
+
|
|
1167
|
+
`lexArgumentTokens` is a third reading over the same walk — alongside strict
|
|
1168
|
+
`lexArguments` and the prefix-safe `lexValueAnchors` values already use. Same
|
|
1169
|
+
text, same `from`, the same boundary rules and the same refusal strings,
|
|
1170
|
+
differing only in what a refusal does. Where the strict reading discards the list, this one publishes the token
|
|
1171
|
+
it could not decode with its bytes located, `undecided` set to that same reason
|
|
1172
|
+
and **no `value`**, then carries on to the next token. `statement.arguments` is
|
|
1173
|
+
unchanged and still strict; nothing that renders a command gained reach.
|
|
1174
|
+
|
|
1175
|
+
**Skip-and-continue, measured against stop-at-first-refusal.** #316 named two
|
|
1176
|
+
designs and said the fork was load-bearing rather than cosmetic. It is. Both are
|
|
1177
|
+
priced by `bun run explain:arg-reach` over the 948-script pinned corpus, from
|
|
1178
|
+
one implementation — design A is the tolerant token list truncated at its first
|
|
1179
|
+
`undecided` token, which is exactly the prefix the strict walk discards:
|
|
1180
|
+
|
|
1181
|
+
| | strict | A — prefix | B — skip |
|
|
1182
|
+
| --- | ---: | ---: | ---: |
|
|
1183
|
+
| statements reached, of 14,267 | 7,385 (51.8%) | 10,754 (75.4%) | 14,267 (100.0%) |
|
|
1184
|
+
| argument tokens | 16,652 | 20,481 | 31,310 |
|
|
1185
|
+
| missing-separator runs found | 0 | 0 | 0 |
|
|
1186
|
+
|
|
1187
|
+
A further 58 statements sit outside every column because their bytes are not
|
|
1188
|
+
addressable (normalized, or a span widened to the enclosing statement); the
|
|
1189
|
+
sweep counts them rather than dropping them silently, since `explain.ts` refuses
|
|
1190
|
+
those too and lexing them here would measure a reading the product does not
|
|
1191
|
+
offer.
|
|
1192
|
+
|
|
1193
|
+
A recovers 3,369 statements; the 3,513 statements whose FIRST interesting token
|
|
1194
|
+
is the undecodable one have an empty prefix and are reached only by B — so the
|
|
1195
|
+
cheaper design leaves slightly more behind than it recovers. The separator row
|
|
1196
|
+
is 0 across all three and is reported rather than buried: on this corpus neither
|
|
1197
|
+
design finds a run the strict reading missed, which is what #203's genre bias
|
|
1198
|
+
predicts for a typo class published scripts do not carry. The sweep also asserts
|
|
1199
|
+
what makes the A column trustworthy — that the strict tokens are the tolerant
|
|
1200
|
+
walk's prefix byte for byte, and that both walks name the same refusal reason —
|
|
1201
|
+
and fails if any statement disagrees (0 do).
|
|
1202
|
+
|
|
1203
|
+
**Where both walks still stop.** An unterminated string and an unbalanced
|
|
1204
|
+
`(`/`[`/`{` have no knowable end, so there is no boundary to resume from. A `;`
|
|
1205
|
+
is a third and deliberately so: it is a statement boundary the segmenter owns,
|
|
1206
|
+
and resuming after it would read the next statement's bytes as this one's
|
|
1207
|
+
arguments. No corpus statement reaches any of the three, and that is a property
|
|
1208
|
+
of the pipeline rather than luck: an unterminated string and an unbalanced
|
|
1209
|
+
delimiter are document defects the segmenter and resolver answer first, so no
|
|
1210
|
+
addressable statement with an argument list is produced at all, and a `;` is
|
|
1211
|
+
split into two statements before any argument is lexed. The unbalanced case is
|
|
1212
|
+
also the one place the two readings name a different reason, and it follows from
|
|
1213
|
+
what each needed: the strict walk refuses a `[…]` on sight and never asks where
|
|
1214
|
+
it ends, while the tolerant walk asks — because the answer is what it would
|
|
1215
|
+
resume from — and says `an unclosed structured argument value` when there is
|
|
1216
|
+
none.
|
|
1217
|
+
|
|
1218
|
+
**What it moved, including the cost.** The `arg` fill now paints the `name=` run
|
|
1219
|
+
of a token whose value only the device knows, so classified bytes go 69.77% →
|
|
1220
|
+
**71.73%** (`arg` tokens 11,438 → 16,526). On the device-agreement slice the
|
|
1221
|
+
bytes where the device decided and centrs abstained drop 5,540 → 4,158, and two
|
|
1222
|
+
whole cells (`unclassified` → `arg-dot`, `unclassified` → `arg-scope`) empty
|
|
1223
|
+
out. Against that, projection disagreements rise 143 → 360, and **every one of
|
|
1224
|
+
the 217 new bytes is the `=` of an attribute**: `arg` → `syntax-meta`, one cell,
|
|
1225
|
+
358 runs. That is not a new wrong assertion. It is #293's provisional
|
|
1226
|
+
vocabulary — one `arg` class for both the name bytes and the `=` — meeting the
|
|
1227
|
+
device's context-dependent answer over 2.5x as many bytes as before, and it
|
|
1228
|
+
gave #264 B5 a concrete, counted reason to split the `=` into its own class.
|
|
1229
|
+
It did, and the relabel moved no byte coverage at all: disagreements fell
|
|
1230
|
+
360 → 2 and the classified-byte census did not change by a digit.
|
|
1231
|
+
|
|
1232
|
+
**The one residue, sized so it is not mistaken for a defect.** A `\<newline>`
|
|
1233
|
+
continuation inside an argument leaves its token `undecided`, because the bytes
|
|
1234
|
+
of a continued value are not contiguous and a `valueSpan` over them would be a
|
|
1235
|
+
lie (`args.ts` → *What it refuses*). The conservative case is a token whose
|
|
1236
|
+
value is already complete BEFORE the continuation, which is then withheld
|
|
1237
|
+
needlessly. Measured on the 948-script corpus: 109 statements make the strict
|
|
1238
|
+
lexer refuse for this reason, the tolerant walk reaches 114 statements carrying
|
|
1239
|
+
138 such tokens, and **all 138 put the continuation immediately after the `=`**
|
|
1240
|
+
— the value really does live on the next line. Exactly **one** token
|
|
1241
|
+
(`mode=none\<newline>`) has a complete value in front of its continuation. So
|
|
1242
|
+
narrowing the rule would recover one token, and deciding it correctly needs the
|
|
1243
|
+
device-grounded answer for what a continuation does to a value's INTERIOR, which
|
|
1244
|
+
is #225's probe-matrix territory rather than a lexer tweak. Recorded there; not
|
|
1245
|
+
carried as pending work here.
|
|
1246
|
+
|
|
1247
|
+
**What the public surface gained, and what it did not.** The package root
|
|
1248
|
+
exports the reading itself — `lexExplainArgumentTokens` and
|
|
1249
|
+
`ExplainArgumentTokenReading` — for the same reason `lexExplainValueAnchors` is
|
|
1250
|
+
exported: a consumer building its own token-level rule needs the reading a rule
|
|
1251
|
+
wants, not the one a renderer wants. What did NOT change is the *envelope*:
|
|
1252
|
+
`explainCommand`'s `structure.statements[].arguments` is still the strict
|
|
1253
|
+
reading, `ExplainArgumentToken` carries no `undecided` field, and `undecided`
|
|
1254
|
+
never appears on an `ExplainArguments` reading, because the strict lexer
|
|
1255
|
+
publishes no token it could not decide. Whether the tolerant stream should reach
|
|
1256
|
+
the envelope in its own right — so a browser consumer reading `explainCommand`'s
|
|
1257
|
+
result can see a located `gateway=` whose value is a runtime expression — was
|
|
1258
|
+
left to #264 B5, which decided it reaches the envelope **as the token
|
|
1259
|
+
partition** and nowhere else: naming the `=` `arg-sep` locates the argument
|
|
1260
|
+
without loosening `arguments`, and `undecided` stays off the envelope. See
|
|
1261
|
+
[The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5).
|
|
1262
|
+
|
|
1263
|
+
### The menu path with arguments (#324)
|
|
1264
|
+
|
|
1265
|
+
A run that is nothing but a **known menu**, with arguments after it and no
|
|
1266
|
+
verb, used to come back `resolved` with a verb the analyzer made up.
|
|
1267
|
+
`/ip/firewall/address-list name=ytkids timeout=1h` reported path
|
|
1268
|
+
`/ip/firewall`, verb `address-list`; the spaced spelling
|
|
1269
|
+
`/ip firewall address-list name=x` promoted `firewall`, one level higher
|
|
1270
|
+
still. The punctuation half of the ratified Q6 rule is what fires — "no space
|
|
1271
|
+
token, arguments follow, so the last run token is the verb", or "the first
|
|
1272
|
+
space token is the verb" — and neither premise it needs was ever checked:
|
|
1273
|
+
|
|
1274
|
+
```text
|
|
1275
|
+
isKnownMenuPath(/ip/firewall/address-list) -> true (it is a menu)
|
|
1276
|
+
VERBS.has("address-list") -> false (it is not a verb)
|
|
1277
|
+
```
|
|
1278
|
+
|
|
1279
|
+
This is the **symmetric half of R9**. R9 (#211 B1) made `pathresolve`'s
|
|
1280
|
+
`menuNavPath` consult `VERBS` so it could not claim navigation where
|
|
1281
|
+
`verbsplit` had decided a verb; this makes `resolveVerb` consult the menu
|
|
1282
|
+
tables so it cannot claim a verb where the whole run names a menu and the
|
|
1283
|
+
segment it would promote is not a verb. The reading degrades to **`ambiguous`**
|
|
1284
|
+
with the standard located `explain/canonicalizer/ambiguous-statement` warning,
|
|
1285
|
+
naming both premises. The adjacent readings are untouched —
|
|
1286
|
+
`/ip/firewall/address-list add name=x` resolves, the bare path is navigation,
|
|
1287
|
+
`/system/reboot` is a command — which is what makes this a seam rather than a
|
|
1288
|
+
parser limit.
|
|
1289
|
+
|
|
1290
|
+
**It abstains; it does not reject.** Offline knows it cannot name a verb here,
|
|
1291
|
+
and that needs no device evidence. Whether RouterOS refuses the shape outright
|
|
1292
|
+
is a *device* claim, so unlike #311's CHR-grounded `error` this stays a
|
|
1293
|
+
warning and the statement carries no verdict about acceptance.
|
|
1294
|
+
|
|
1295
|
+
**Four exclusions, each for its own reason.** A frozen-vocabulary hit is
|
|
1296
|
+
excluded by the `VERBS` premise itself (a vocabulary split promotes a `VERBS`
|
|
1297
|
+
member by construction). A **published command** is excluded by `catalogAt`:
|
|
1298
|
+
first-order evidence about that exact path outranks a container listing of its
|
|
1299
|
+
prefix, which is what keeps `/system/gps/monitor once` resolved. A scripting
|
|
1300
|
+
**directive** is excluded by R10, which is measured. And a `{` after the run is
|
|
1301
|
+
excluded by shape: that is a menu *container* body (`/system {identity print}`,
|
|
1302
|
+
H7) where the run legitimately names a menu and no verb is missing —
|
|
1303
|
+
abstaining there would drop the `invalid-command-brace` error the statement
|
|
1304
|
+
after it carries.
|
|
1305
|
+
|
|
1306
|
+
**Priced before landing, as #211 B2 asks.** Over the frozen dev/holdout
|
|
1307
|
+
partition (948-script corpus, 17,212 statements) **10 statements flip, all
|
|
1308
|
+
`resolved` → `ambiguous`**: 1 dev, 9 holdout. **None is a genuine no-verb
|
|
1309
|
+
command.** Three are real RouterOS missing a verb —
|
|
1310
|
+
`/ip/firewall/address-list name=ytkids …`, `/interface bridge [find]
|
|
1311
|
+
vlan-filtering=yes`, `/user-manager user [find …] attributes=…` — and seven
|
|
1312
|
+
are pasted non-RouterOS prose: five Python traceback `File "…"` lines,
|
|
1313
|
+
`User ID: 506`, and `port 80`. Those seven reach the rule at all because
|
|
1314
|
+
`/file`, `/user` and `/port` are real menus and the table lookup folds case,
|
|
1315
|
+
which is #203's genre bias arriving through a second door. Every one of the
|
|
1316
|
+
ten was a fabrication, so the abstention has no measured false-positive cost
|
|
1317
|
+
on this corpus.
|
|
1318
|
+
|
|
1319
|
+
**The execute gate is unmoved**, as the constitution requires: `canonical`
|
|
1320
|
+
still reports `mode: "structured"`, path `/ip/firewall`, verb `address-list`
|
|
1321
|
+
for this input, because that is what `canonicalizeExecuteCommand` decides and
|
|
1322
|
+
an explain-only abstention may not reinterpret it. `enforceGateParity` is
|
|
1323
|
+
about the argument *list* of a `kind: "command"` reading, so it does not fire
|
|
1324
|
+
on a statement that carries no reading at all — the gate and the analysis are
|
|
1325
|
+
two answers to two questions, and only one of them changed.
|
|
1326
|
+
|
|
1327
|
+
The invariant this closes — `pathresolve` and `verbsplit` never contradict
|
|
1328
|
+
each other on one statement — is asserted in
|
|
1329
|
+
`test/unit/explain-pathresolve.test.ts` beside R9's own cases. Presence in the
|
|
1330
|
+
shipped structure tables still narrows a reading and absence still decides
|
|
1331
|
+
nothing (#207/#228), so an unlisted menu spelled `…/address-list` is read
|
|
1332
|
+
exactly as before.
|
|
1333
|
+
|
|
1334
|
+
### Two spellings the device accepts and offline used to reject (#347, #348)
|
|
1335
|
+
|
|
1336
|
+
Both were found the same way, and the method is worth keeping: run the analyzer
|
|
1337
|
+
over all 948 corpus scripts and compare its verdict to that build's own `:parse`
|
|
1338
|
+
verdict, which the corpus already stores. The number that matters is the **false
|
|
1339
|
+
reject** — a script the device ACCEPTS that offline analysis fails — because
|
|
1340
|
+
that is validation blocking input RouterOS is happy with. It was 4, identical on
|
|
1341
|
+
7.23.5, 7.24.2 and 7.25beta3; it is now 1.
|
|
1342
|
+
|
|
1343
|
+
The converse count is not a parser score and is not quoted as one: the corpus is
|
|
1344
|
+
whole forum posts including prose output, so most of what the device rejects and
|
|
1345
|
+
offline passes is #203's genre bias rather than a gap.
|
|
1346
|
+
|
|
1347
|
+
**The `=` is optional on a scope slot (#347).** `:if (…) do={…} else={…}`,
|
|
1348
|
+
`:if (…) do={…} else {…}` and `:if (…) do {…} else {…}` are the same construct:
|
|
1349
|
+
`:parse` lowers all three to byte-identical IL (`…;do=;(evl …);else=;(evl …)`)
|
|
1350
|
+
on CHR 7.21.5, 7.23.5, 7.24.2 and 7.25beta3. The MikroTik manual documents only
|
|
1351
|
+
the `else={…}` spelling, so this is a real-but-undocumented form. The rule is
|
|
1352
|
+
deliberately narrower than "any word before a brace": only the four
|
|
1353
|
+
`SCOPE_ARG_NAMES` members may drop the `=`, and the word may not be the
|
|
1354
|
+
statement's own leading directive — otherwise `:local z {1;2}` would stop
|
|
1355
|
+
reading its brace as an array, and `:do {` would have its slot renamed away from
|
|
1356
|
+
the `command` the IL uses. A COLON also disqualifies the word: the device parses
|
|
1357
|
+
`:if (1=1) do {…}` but refuses `:if (1=1) :do {…}` with `expected end of command`
|
|
1358
|
+
on 7.21.5 and 7.24.2, because a colon makes it a directive and a directive
|
|
1359
|
+
cannot occupy an argument slot. The control is `elsy {`, rejected on all four
|
|
1360
|
+
builds, which is what makes the acceptance a device fact rather than a probe
|
|
1361
|
+
artefact.
|
|
1362
|
+
|
|
1363
|
+
**A relative menu-scope block composes the path (#348).** `/ip { address {
|
|
1364
|
+
print } }` is `(evl (evl (evl /ip/address/print)))` on device and
|
|
1365
|
+
`/interface { ethernet { print } }` the same way; the ABSOLUTE spelling
|
|
1366
|
+
`/ip/address { print }` already passed offline, and only the nested relative
|
|
1367
|
+
form failed. A menu block's body is a statement list, so it may open with a
|
|
1368
|
+
submenu, with one of the thirteen frozen CRUD verbs (`verbs.ts`), with a
|
|
1369
|
+
catalog-named command, with a `:` scripting directive, or with an absolute
|
|
1370
|
+
command — corpus 521 is the directive case, `ethernet { :put … }`.
|
|
1371
|
+
|
|
1372
|
+
That last rule is bounded by what a STATEMENT can start with, which is what
|
|
1373
|
+
keeps `{1;2}` an array literal: an array member opens with a digit or a quote
|
|
1374
|
+
and never reaches the identifier branch. It is also deliberately not widened
|
|
1375
|
+
into a claim of existence. Menu-scope resolution is schema-dependent —
|
|
1376
|
+
`/ip { zzznotamenu { print } }` is a `syntax error` on device — and offline has
|
|
1377
|
+
no schema, so a catalog MISS abstains rather than being decided either way.
|
|
1378
|
+
That is the catalog's own "a hit is decisive, a miss says nothing" contract, and
|
|
1379
|
+
turning such a miss into an acceptance would be the mirror-image defect of the
|
|
1380
|
+
one being fixed here.
|
|
1381
|
+
|
|
1382
|
+
### `'` is not a RouterOS token (#355)
|
|
1383
|
+
|
|
1384
|
+
`'` (U+0027) is not a string delimiter and not a legal bare byte. Only `"` opens
|
|
1385
|
+
a string. `:parse` rejects every spelling of a bare apostrophe byte-identically
|
|
1386
|
+
on CHR 7.21.5, 7.23.5, 7.24.2 and 7.25beta3:
|
|
1387
|
+
|
|
1388
|
+
| input | device `:parse` | offline span |
|
|
1389
|
+
| --- | --- | --- |
|
|
1390
|
+
| `/system/identity/set name=don't` | `expected end of command (line 1 column 30)` | 29 |
|
|
1391
|
+
| `/system/identity/set name=don't'` | `expected end of command (line 1 column 30)` | 29 |
|
|
1392
|
+
| `/system/identity/set name='abc'` | `syntax error (line 1 column 27)` | 26 |
|
|
1393
|
+
| `:put 'abc'` | `syntax error (line 1 column 6)` | 5 |
|
|
1394
|
+
|
|
1395
|
+
The device's column is 1-based, so each offline span is the device's own byte.
|
|
1396
|
+
The accepting controls — `comment="it's here"` and a `# don't` comment line —
|
|
1397
|
+
parse cleanly on all four builds, which is what makes the rejection a device
|
|
1398
|
+
fact rather than a probe artefact. Rows 2–4 carry BALANCED apostrophes and are
|
|
1399
|
+
rejected anyway: this is a lexical rule, not a quote-balance one, and the remedy
|
|
1400
|
+
is always `"`, never a matching `'`.
|
|
1401
|
+
|
|
1402
|
+
Only the FIRST apostrophe is recorded, **document-wide**, the same way the hard
|
|
1403
|
+
`#` is: the device reports one column and it is the first one (row 2 reports
|
|
1404
|
+
column 30, not the trailing apostrophe), because classification stops there. A
|
|
1405
|
+
nested scope body is a separate scan (`segmentScopeBody`), so per-scan
|
|
1406
|
+
suppression is not enough on its own and the collapse to the earliest occurrence
|
|
1407
|
+
happens where the diagnostics are composed — which keeps each segmenter call
|
|
1408
|
+
pure. Apostrophes inside a `"`-string or a comment never reach the rule at all —
|
|
1409
|
+
both runs are consumed whole by the scan before it — so no special case is
|
|
1410
|
+
needed for either.
|
|
1411
|
+
|
|
1412
|
+
Symbol resolution stops at the apostrophe too, exactly as it stops at a
|
|
1413
|
+
malformed escape: the device classes nothing after a hard reject, so
|
|
1414
|
+
`:put 'abc'; :put $after` must not publish an occurrence for `$after`.
|
|
1415
|
+
Occurrences BEFORE the defect stand, which is the lab's X1 rule.
|
|
1416
|
+
|
|
1417
|
+
**This was a false ACCEPT, which the corpus join cannot see.** The false-reject
|
|
1418
|
+
measurement scores one direction only, so the analyzer passing all four forms
|
|
1419
|
+
with zero diagnostics never appeared in it. Over the pinned corpus the rule
|
|
1420
|
+
fires on 40 of the 7.24.2 scripts and **every one of them is a script the device
|
|
1421
|
+
rejects** — none of the 617 it accepts. It moves the false-reject count not at
|
|
1422
|
+
all (still 1, the extra `}` of corpus 228) while taking the caught count from 62
|
|
1423
|
+
to 93.
|
|
1424
|
+
|
|
1425
|
+
The token census moves, and only in the direction the fail-closed symbol stop
|
|
1426
|
+
predicts: 68 fewer `variable-*` tokens and their 306 bytes returning to
|
|
1427
|
+
`unclassified` (classified 1,023,413 → 1,023,107). Those are variable references
|
|
1428
|
+
sitting after a bare apostrophe, which the device never reaches and centrs
|
|
1429
|
+
therefore stops naming. The value, operator and bracket-equals censuses and
|
|
1430
|
+
`explain:arg-reach` are all unchanged.
|
|
1431
|
+
|
|
1432
|
+
**It also retires a wrong rule.** `execute` carried a private
|
|
1433
|
+
`hasUnbalancedQuotes` preflight that modelled `'` as a delimiter, moved into the
|
|
1434
|
+
offline gate by #354. It had no comment awareness, so `# don't` opened a quote
|
|
1435
|
+
it never closed: over the same corpus it false-rejected **51** of the 617
|
|
1436
|
+
device-accepted scripts. Deleting it costs 21 catches — all of them scripts the
|
|
1437
|
+
device rejects anyway, where the device stage is still the backstop — and takes
|
|
1438
|
+
the gate's false rejects from 52 to 1. A false reject blocks input RouterOS
|
|
1439
|
+
accepts, which is the figure that matters.
|
|
1440
|
+
|
|
1016
1441
|
### String escape validation (#247, #252)
|
|
1017
1442
|
|
|
1018
1443
|
The accepted set is the **device's**, not the manual's. The
|
|
@@ -1061,8 +1486,9 @@ for the three issue rows `:put "\q"` / `:put "\x0a"` / `:put "\0a"` vs
|
|
|
1061
1486
|
- Scoring a defect rule against the corpus needs the right oracle.
|
|
1062
1487
|
`parseil_results.ok` is **not** one: `:parse` never throws, it returns a
|
|
1063
1488
|
diagnostic *value*, so the capture records `ok=1 / status="ok"` for scripts
|
|
1064
|
-
the device rejected (only
|
|
1065
|
-
failures
|
|
1489
|
+
the device rejected (only 5 of 4635 rows are `ok=0`, and those are capture
|
|
1490
|
+
failures — one `edge-cases/oversize-32k.rsc` REST `413` per captured
|
|
1491
|
+
version). The verdict is in `il_text` — plain prose such as
|
|
1066
1492
|
`expected input value (line 7 column 49)` means rejected, IL means accepted.
|
|
1067
1493
|
|
|
1068
1494
|
### The operator surface (#255)
|
|
@@ -1071,14 +1497,62 @@ Grounded, not transcribed, for the same reason the escape set above is: the
|
|
|
1071
1497
|
[manual's operator list](https://manual.mikrotik.com/docs/developer-guides/scripting/index.md#operators)
|
|
1072
1498
|
is a lower bound. `bun run explain:probe:operators` sweeps the manual's list
|
|
1073
1499
|
**plus** plausible non-operators (`not`, `..`, `xor`, `mod`, `is`, `div`, …)
|
|
1074
|
-
**plus** every IL head the corpus census saw, on CHR 7.
|
|
1075
|
-
|
|
1076
|
-
|
|
1500
|
+
**plus** every IL head the corpus census saw, on CHR 7.24.2 stable, 7.23.5
|
|
1501
|
+
long-term and 7.25beta3 development — the same three builds the pinned corpus
|
|
1502
|
+
classes `complete`, so the operator axis and the token partition are finally
|
|
1503
|
+
scored on the same firmware (#342) — plus 7.21.5 as an **age anchor**, which is
|
|
1504
|
+
not a channel claim but the only live evidence that the surface behaves the same
|
|
1505
|
+
on a pre-7.23 branch. Re-cut the fixture with
|
|
1077
1506
|
`bun run explain:operator-slice`; the table
|
|
1078
1507
|
below is generated from `src/explain/operators.ts` by
|
|
1079
1508
|
`bun run explain:operator-readme` and gated by
|
|
1080
1509
|
`bun run explain:operator-readme:check`.
|
|
1081
1510
|
|
|
1511
|
+
**The census is a candidate generator, and boilerplate defeats its own
|
|
1512
|
+
tie-breaker.** `scripts/explain-operator-census.ts` reports every head-position
|
|
1513
|
+
token in the device IL, tagged by shape, because IL is a debug rendering that
|
|
1514
|
+
does not quote strings: a string literal is emitted bare, so string content is
|
|
1515
|
+
indistinguishable from a node head. Its stated honesty rule is that
|
|
1516
|
+
`distinctScripts` beats frequency — "a head seen in one script is one author's
|
|
1517
|
+
string, a head seen in three hundred is a language feature". Only the complete
|
|
1518
|
+
builds carry parseIL for the 35-script `tangentsoft` device-`/export` stratum,
|
|
1519
|
+
and that stratum showed the rule has a blind spot, because a stock string
|
|
1520
|
+
replicated across a whole stratum of device exports looks exactly like a
|
|
1521
|
+
language feature by script count:
|
|
1522
|
+
|
|
1523
|
+
| word-shaped head | scripts, partial builds → complete | what it actually is |
|
|
1524
|
+
| ---------------- | ---------------------------------: | ------------------- |
|
|
1525
|
+
| `for` | 2 → 22 | the defconf comment `accept to local loopback (for CAPsMAN)` — in 20 of the 35 device exports, plus the 2 forum scripts that already carried it |
|
|
1526
|
+
| `line` | 53 → 84 | device diagnostic prose, `bad parameter hw-offload (line 33 column 147)` |
|
|
1527
|
+
|
|
1528
|
+
Those arrows are a **coverage** step, not a version step: the left column is
|
|
1529
|
+
what a 913-script capture could see, the right what a 948-script one sees.
|
|
1530
|
+
|
|
1531
|
+
Neither is an operator, and both are `word`-shaped — the same bucket #255's
|
|
1532
|
+
`not` would have appeared in. So the sweep, not the census, still decides; a
|
|
1533
|
+
head that arrives with a whole stratum at once is boilerplate until the device
|
|
1534
|
+
says otherwise. The two heads that stratum added outright,
|
|
1535
|
+
`configuration.mode=` and `[system`, are `other`-shaped and are the documented
|
|
1536
|
+
string-noise class: a dotted wifi property name emitted bare, and text inside
|
|
1537
|
+
an embedded `/system script add source=` value.
|
|
1538
|
+
|
|
1539
|
+
**The census counts only the complete builds; a separate axis dates heads.**
|
|
1540
|
+
Occurrences are summed across versions, so admitting a partial capture would
|
|
1541
|
+
scale every figure by what happened to be captured — on this snapshot, letting
|
|
1542
|
+
all six builds in inflated the headline by ~20% while `distinctScripts`, the
|
|
1543
|
+
column that actually carries the argument, did not move at all. So
|
|
1544
|
+
`headOccurrences` / `headVersions` / `headArities` read
|
|
1545
|
+
`coverage_class = 'complete'` only, and the census states its version set in
|
|
1546
|
+
`versions` so a gate cannot silently compare two different universes.
|
|
1547
|
+
|
|
1548
|
+
Presence, unlike frequency, *is* monotone in coverage — a 913-of-948 capture
|
|
1549
|
+
that shows a head proves that build had it — so the partial builds keep one
|
|
1550
|
+
job: `headFirstSeen` dates each head against every captured build. That is what
|
|
1551
|
+
still grounds the `any` claim below at 7.20.8, a build the counting side no
|
|
1552
|
+
longer reads. Read a *late* first-seen as "not captured before here", never as
|
|
1553
|
+
"new in": the only two heads that do not reach 7.20.8 are the `tangentsoft`
|
|
1554
|
+
pair above, which no partial build could have seen.
|
|
1555
|
+
|
|
1082
1556
|
**`:parse` IL is the oracle, and `highlight` cannot be.** IL is prefix form with
|
|
1083
1557
|
the operator as its node's head, so it names the operator and shows its
|
|
1084
1558
|
operands. `highlight`'s `syntax-meta` is the device's *residual structure*
|
|
@@ -1103,7 +1577,7 @@ Five things the device says that the manual does not:
|
|
|
1103
1577
|
3. **`$`, `[`, `]` are syntax, not operator heads.** `$x` stays an atom in the
|
|
1104
1578
|
IL and `[:tostr 1]` lowers to an `evl` node. Those bytes belong to the
|
|
1105
1579
|
substitution axis.
|
|
1106
|
-
4. **`any` is an operator** (prefix, arity 1) and is not in the manual's list — a **nil-check**: `:typeof (any x)` is `bool`, `false` only for `nil`/`nothing` (the value of an undefined `:local` and of `[:nothing]`), `true` for everything else including `0`, `""` and `false`. It is the idiom `:if (any $x) ...` to test a variable that may be `nil`; `(true any false)` is not infix at all but juxtaposition `( true (any false))`, and `(1 . any [:nothing])` is concat `1`+`false`. Present since at least 7.20.8 (corpus `any
|
|
1580
|
+
4. **`any` is an operator** (prefix, arity 1) and is not in the manual's list — a **nil-check**: `:typeof (any x)` is `bool`, `false` only for `nil`/`nothing` (the value of an undefined `:local` and of `[:nothing]`), `true` for everything else including `0`, `""` and `false`. It is the idiom `:if (any $x) ...` to test a variable that may be `nil`; `(true any false)` is not infix at all but juxtaposition `( true (any false))`, and `(1 . any [:nothing])` is concat `1`+`false`. Present since at least 7.20.8 (corpus `headFirstSeen[any] = 7.20.8`, the oldest captured build, and swept live on 7.21.5 and 7.23.5 long-term, 7.24.2 stable and 7.25beta3 development with no difference between them).
|
|
1107
1581
|
5. **`&&` and `||` are spellings**, lowering to the `and` and `or` nodes.
|
|
1108
1582
|
|
|
1109
1583
|
The `(>…)` and `<%%` forms are in the table on the same footing as `+`.
|
|
@@ -1119,7 +1593,7 @@ their `precedence` is honestly `null`. That is "unmeasured", not "unknown": the
|
|
|
1119
1593
|
sweep asks them separately, every `(U 1 B 2)` and `(1 B U 2)` for
|
|
1120
1594
|
`U`∈`!`,`any`,`~`,`-`,`>` against all 24 binaries — 240 probes, recorded in the
|
|
1121
1595
|
fixture's `unary` block. All 240 are accepted and the **binary is outer in every
|
|
1122
|
-
one**, on all
|
|
1596
|
+
one**, on all four swept builds, so each prefix operator binds tighter than every
|
|
1123
1597
|
binary including `->` (14) and `<%%` (13). The table stores one `precedence` per
|
|
1124
1598
|
spelling, which is the binary level; `~`, `-` and `>` are here because their
|
|
1125
1599
|
unary reading has no other record.
|
|
@@ -1131,8 +1605,14 @@ not concat at all** — `1.` lexes as a *variable name* and the row comes back a
|
|
|
1131
1605
|
juxtaposition, `( $1. 2)`. A rule that claims every `.` byte for the operator
|
|
1132
1606
|
emits a span the device does not have.
|
|
1133
1607
|
|
|
1134
|
-
Everything in the sweep is identical across
|
|
1135
|
-
|
|
1608
|
+
Everything in the sweep is identical across all four builds except one runtime
|
|
1609
|
+
row: `:put ({2;1} > {1;2;3})` evaluates to `true` on 7.24.2 and 7.25beta3 and
|
|
1610
|
+
errors on both long-term builds. **Array comparison is not fixed on current
|
|
1611
|
+
long-term** — 7.23.5 is newer than the 7.23.3 this was first measured on and
|
|
1612
|
+
still errors — so the change rode the 7.24 line and was not backported. The two
|
|
1613
|
+
errors also differ: 7.23.5 says `cannot compare if array is more than array`,
|
|
1614
|
+
7.21.5 says `cannot compare if nothing is more than nothing`, so on 7.21.5 the
|
|
1615
|
+
operands did not survive to the comparison at all.
|
|
1136
1616
|
Identical is measured, not assumed — the slice diffs verdict, arity, precedence,
|
|
1137
1617
|
associativity, `highlight` run, unary placement and op-axis per version. So the
|
|
1138
1618
|
operator table needs no version gate; anything reporting what a comparison
|
|
@@ -1231,49 +1711,64 @@ And the grounded complement — asked, and refused:
|
|
|
1231
1711
|
`bun run explain:token-census:readme` and gated against it by
|
|
1232
1712
|
`bun run explain:token-census:readme:check`; the fixture itself is gated
|
|
1233
1713
|
against a fresh corpus run by `bun run explain:token-census:check`. Of
|
|
1234
|
-
1,426,731 analyzed bytes,
|
|
1235
|
-
|
|
1714
|
+
1,426,731 analyzed bytes, 1,023,107 are classified (71.71%), the remaining
|
|
1715
|
+
403,624 are `unclassified`. The census emits 241,539 tokens (avg 254.8 per
|
|
1236
1716
|
script). Every byte belongs to exactly one token — sorted by `start`, no
|
|
1237
|
-
gaps, no overlaps, `join(slice) === input` — and the `class` field is
|
|
1238
|
-
|
|
1717
|
+
gaps, no overlaps, `join(slice) === input` — and the `class` field is the
|
|
1718
|
+
vocabulary #264 B5 settled. Each B2 fill should move the classified
|
|
1239
1719
|
percentage.
|
|
1240
1720
|
<!-- END GENERATED token-census -->
|
|
1241
1721
|
|
|
1242
1722
|
B1's `data.tokens[]` is live behind `--tokens` — a total, gapless byte
|
|
1243
|
-
partition whose `class`
|
|
1244
|
-
`unclassified`). Since B1 its only fill source was `data.spans[]` (comment runs
|
|
1723
|
+
partition whose `class` draws on the vocabulary #264 B5 settled (every unclaimed
|
|
1724
|
+
byte is `unclassified`). Since B1 its only fill source was `data.spans[]` (comment runs
|
|
1245
1725
|
and resolved variable occurrences); **the path fill claims resolved menu and
|
|
1246
1726
|
command-name bytes as provisional `dir` / `cmd` tokens**, including valid path
|
|
1247
1727
|
slashes and nested command substitutions, while ambiguous, malformed, and
|
|
1248
|
-
source-unmapped runs stay unclassified.
|
|
1249
|
-
|
|
1250
|
-
|
|
1251
|
-
|
|
1252
|
-
`
|
|
1728
|
+
source-unmapped runs stay unclassified. A substitution's bytes are located by
|
|
1729
|
+
the resolution's published **`innerSpan`**, never by arithmetic on its `span`:
|
|
1730
|
+
`span` covers `[…]` for a bare substitution but `$[…]` for one interpolated
|
|
1731
|
+
inside a string (the sigil is part of it), and `inner` is trimmed on top of
|
|
1732
|
+
that, so `span.start + 1` was one byte early in a string and two bytes wrong
|
|
1733
|
+
for `[ … ]`. Both refused the fill's slice check and fell through to the
|
|
1734
|
+
string fill, which is how a command `structure.subcommands` had already
|
|
1735
|
+
resolved came back as opaque `string` bytes (#325). The check itself stays:
|
|
1736
|
+
where `Loc` widens the resolution to its statement, the slice no longer equals
|
|
1737
|
+
`inner` and the bytes are left unclaimed rather than landed on the wrong
|
|
1738
|
+
place. **`#293`'s arg fill** in
|
|
1739
|
+
`src/explain/arg-tokens.ts` then claims argument names as `arg` (name run
|
|
1740
|
+
`[span.start, valueSpan.start - 1)`) and the single `=` byte at
|
|
1741
|
+
`valueSpan.start - 1` as `arg-sep` (#264 B5) on the residual left by spans,
|
|
1742
|
+
before operators see it
|
|
1253
1743
|
— and **`#295`'s value fill follows it** — `src/explain/value-tokens.ts`
|
|
1254
1744
|
claims argument value bytes and leaf array-literal members
|
|
1255
1745
|
(`data.values.occurrences`, leaves only via `parent` containment,
|
|
1256
|
-
quotes-included) on the residual left by `spans`+`path`+`arg`, before the
|
|
1746
|
+
quotes-included) on the residual left by `spans`+`path`+`arg`+`escaped`, before the
|
|
1257
1747
|
**B3 string/brace fills** see it — `src/explain/string-tokens.ts` claims
|
|
1258
|
-
quoted-string runs (delimiters included) and `src/explain/brace-tokens.ts`
|
|
1748
|
+
quoted-string runs (delimiters included, escapes already taken) and `src/explain/brace-tokens.ts`
|
|
1259
1749
|
claims every residual brace delimiter (`{`/`}`), including delimiters from
|
|
1260
1750
|
invalid braced references, on the residual left by
|
|
1261
|
-
`spans`+`path`+`arg`+`value`, and finally `operator` sees only what none of
|
|
1751
|
+
`spans`+`path`+`arg`+`escaped`+`value`, and finally `operator` sees only what none of
|
|
1262
1752
|
them wanted — which is why the operator fill can abstain on `, / = -` outside
|
|
1263
1753
|
`( )`, on bytes glued after an argument `=`, and on bytes glued into an
|
|
1264
|
-
argument name without a smarter operator scanner.
|
|
1754
|
+
argument name without a smarter operator scanner. **`src/explain/escape-tokens.ts`
|
|
1755
|
+
claims valid string-internal escapes as `escaped`** between `arg` and `value`:
|
|
1756
|
+
an escape is interior to a quoted run, so it has to claim before the two fills
|
|
1757
|
+
that would otherwise swallow it whole, and it splits their runs into fragments
|
|
1758
|
+
around it the same way a `variable-*` span already did. Fill order is enforced by
|
|
1265
1759
|
argument order to `buildTokens` ([#290 design decision 1](../../src/explain.ts)):
|
|
1266
|
-
`spans` claims first, then `path`, `arg`, `value`, `string`, `brace`,
|
|
1267
|
-
finally `operator`.
|
|
1760
|
+
`spans` claims first, then `path`, `arg`, `escaped`, `value`, `string`, `brace`,
|
|
1761
|
+
and finally `operator`.
|
|
1268
1762
|
`ExplainTokenClass` is
|
|
1269
|
-
`ExplainSpanClass | "dir" | "cmd" | "operator" | "arg" | "
|
|
1270
|
-
"brace" | "unclassified"` — one
|
|
1271
|
-
|
|
1272
|
-
|
|
1273
|
-
`
|
|
1274
|
-
|
|
1275
|
-
|
|
1276
|
-
|
|
1763
|
+
`ExplainSpanClass | "dir" | "cmd" | "operator" | "arg" | "arg-sep" | "value" |
|
|
1764
|
+
"string" | "escaped" | "brace" | "unclassified"` — one `operator` class for all 26
|
|
1765
|
+
spellings + 2 aliases, `arg` for an argument's name bytes and `arg-sep` for the
|
|
1766
|
+
single `=` that binds it to its value, one `value` class for every leaf value
|
|
1767
|
+
span, plus `string` (quoted runs minus their escapes), `escaped` (one valid
|
|
1768
|
+
string-internal escape sequence) and `brace` (scope delimiters). #264
|
|
1769
|
+
settled which of those merges keep their name and which do not, on a stated
|
|
1770
|
+
rule rather than taste — see
|
|
1771
|
+
[The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5).
|
|
1277
1772
|
`ExplainSpanClass` and `data.spans[]` stay proof-only; `src/explain/operators.ts`
|
|
1278
1773
|
remains data plus accessors, and the operator table above is its source. The
|
|
1279
1774
|
`centrs → highlight` projection is B4 and reads both.
|
|
@@ -1289,11 +1784,413 @@ each grounded on the corpus device oracle (`parseil_results.il_text`):
|
|
|
1289
1784
|
| A spelling glued immediately after an argument `=`. | `in-interface-list=!LAN`, `.id=*2`, `oid=.1.3.6.1.2.1` — the byte after an argument `=` starts the value. |
|
|
1290
1785
|
| A spelling glued into an argument **name**. | `:foreach x in=$list` is `/foreach counter=$x` with no `(in …)` node anywhere; the IL keeps `security.authentication-types=wpa2-psk` as one name and renders `.id` as the single symbol `$.id`, never `(. …)`. |
|
|
1291
1786
|
|
|
1292
|
-
The first abstention has a measured cost
|
|
1293
|
-
|
|
1294
|
-
|
|
1295
|
-
|
|
1296
|
-
|
|
1787
|
+
The first abstention has a measured cost, and it is now generated rather than
|
|
1788
|
+
asserted. The figures below were prose with no script behind them and no gate
|
|
1789
|
+
on them for three releases, quoting a `224` / `1,259` / `1,035` split that no
|
|
1790
|
+
universe reproduces (#341). The abstention itself is as wide as it ever was —
|
|
1791
|
+
what was wrong was the accounting of it. They are re-derived by
|
|
1792
|
+
`bun run explain:bracket-equals` and gated by
|
|
1793
|
+
`bun run explain:bracket-equals:readme:check`.
|
|
1794
|
+
|
|
1795
|
+
<!-- BEGIN GENERATED bracket-equals — regenerate with `bun run explain:bracket-equals:readme` -->
|
|
1796
|
+
Measured by `bun run explain:bracket-equals` over the pinned corpus:
|
|
1797
|
+
**1,079** `=` bytes sit inside `[ … ]`, across
|
|
1798
|
+
263 of 948 scripts, and **1,067** of them come back
|
|
1799
|
+
`unclassified` — the operator fill abstains on essentially all of it. No
|
|
1800
|
+
later fill rescues them: the `arg` fill claims **0**, because it offers only
|
|
1801
|
+
the `=` its own located argument token names and a `[ … ]` substitution is
|
|
1802
|
+
not where it locates arguments.
|
|
1803
|
+
|
|
1804
|
+
Of those 1,067, **448** are governed by a `find`/`where` query and are
|
|
1805
|
+
genuine comparisons — the device lowers `[find address=$IP]` to
|
|
1806
|
+
`(= $address $IP)` — against **619** plain `arg=value`. So abstaining is
|
|
1807
|
+
42.0% wrong and claiming the lot would be 58.0% wrong; a `where`-aware
|
|
1808
|
+
fill can take the query half later. For contrast the fill does claim
|
|
1809
|
+
**1,402** `=` bytes whose innermost opener is `(`.
|
|
1810
|
+
|
|
1811
|
+
The query reading is the device's, not a guess: on 7.24.2, 93 of the
|
|
1812
|
+
137 scripts carrying one have IL with at least as many `(= …)` nodes.
|
|
1813
|
+
Of the rest, 28 are scripts the console refused outright — their IL is prose,
|
|
1814
|
+
with no nodes of any kind — and 16 use a menu the capture build could not
|
|
1815
|
+
resolve (a forum snippet on `/interface wireless` answers `bad command name
|
|
1816
|
+
wireless`), so the device never reached the query to lower it. Neither is a
|
|
1817
|
+
counter-example to the reading. Whole-corpus `(= …)` head nodes, which also
|
|
1818
|
+
count expression comparisons the fill already claims:
|
|
1819
|
+
7.23.5 1,233, 7.24.2 1,240, 7.25beta3 1,240.
|
|
1820
|
+
<!-- END GENERATED bracket-equals -->
|
|
1821
|
+
|
|
1822
|
+
### Device agreement for the token partition (#264 B4, #263)
|
|
1823
|
+
|
|
1824
|
+
The census above counts what offline analysis *claimed*. This counts what the
|
|
1825
|
+
device *agreed with*. The oracle is the committed stratified slice of the Q13
|
|
1826
|
+
`/console/inspect request=highlight` captures
|
|
1827
|
+
([`test/fixtures/explain/highlight-streams.slice.json`](../../test/fixtures/explain/highlight-streams.slice.json)),
|
|
1828
|
+
so the scorer needs no router, no corpus and no network, and gates in the unit
|
|
1829
|
+
suite rather than in the corpus job. Both oracles must see the same bytes, so
|
|
1830
|
+
the source text is reconstructed from the device's own run-length pairs — never
|
|
1831
|
+
from the corpus row, which still holds the non-ASCII the capture replaced
|
|
1832
|
+
(the #269 rule, one layer up).
|
|
1833
|
+
|
|
1834
|
+
**`highlight` is not one oracle; it is four kinds of answer.** Scoring a byte
|
|
1835
|
+
means first knowing what kind of answer came back, because three of the four
|
|
1836
|
+
are not judgments about syntax at all:
|
|
1837
|
+
|
|
1838
|
+
| Kind | Classes | Why it cannot be scored as agreement |
|
|
1839
|
+
| ---- | ------- | ------------------------------------ |
|
|
1840
|
+
| syntax | `cmd` `dir` `arg` `arg-dot` `arg-scope` `comment` `escaped` `syntax-meta` `variable-local` `variable-global` `variable-auto` `variable-parameter` | — this is the region that can be scored |
|
|
1841
|
+
| non-syntax | `obj-inactive` `obj-dynamic` `obj-disabled` `variable-undefined` `syntax-obsolete` | The source text alone does not decide these. centrs has no class for most of them at all — `ExplainSymbolClass` deliberately excludes `undefined` — and where it *does* claim the byte (an `arg` the device calls `obj-inactive`) it is the **scorer** that abstains, because the device was not making a syntax judgement there. *Why* each one is out of reach is a second question, answered per byte rather than per class; see applicability below. |
|
|
1842
|
+
| silence | `none` | The device's abstention, not a class named "none". Agreeing with silence is not agreement. |
|
|
1843
|
+
| parser-stop | `error` | The device gives up here — see below. |
|
|
1844
|
+
|
|
1845
|
+
**Applicability is a second axis, and it is not the class name.** #263's four
|
|
1846
|
+
categories — offline-decidable, schema-dependent, state-dependent,
|
|
1847
|
+
version-dependent — are assigned from the *pair* (what centrs read the byte as,
|
|
1848
|
+
what the device called it) plus whether the two captures even agreed, because
|
|
1849
|
+
one highlight class carries more than one category. Version disagreement is
|
|
1850
|
+
checked first, ahead even of the device's own silence: a byte one build
|
|
1851
|
+
classifies and the next leaves `none` is a version difference, not an absence.
|
|
1852
|
+
`obj-inactive` on an argument name means the menu's schema has no such field
|
|
1853
|
+
(**schema** — `default-vlan-id`, under an `/interface ethernet switch port` a
|
|
1854
|
+
CHR does not have); the same class on a menu or command name means the package
|
|
1855
|
+
is not installed on this box (**state**: `/zerotier`, `/user-manager`,
|
|
1856
|
+
`/container`). `obj-dynamic` on the routing table `main` is a live object
|
|
1857
|
+
(**state**).
|
|
1858
|
+
`syntax-obsolete` is this version's deprecation table talking, landing on the
|
|
1859
|
+
whitespace where modern syntax wants `=` (`} else {`, `:foreach conn in
|
|
1860
|
+
$conns`) — **version**. And `variable-undefined` is deliberately left
|
|
1861
|
+
**uncategorized**: an undefined reference may be an external global, a
|
|
1862
|
+
query/filter field name or a lexical spelling, nothing committed here separates
|
|
1863
|
+
the three, and an absent binding in *this* document is not evidence of a
|
|
1864
|
+
runtime undefined variable either. Leaving it explicit is the answer, not a gap
|
|
1865
|
+
to be filled by assumption.
|
|
1866
|
+
|
|
1867
|
+
**The device stops classifying at its first `error`, and mostly does not
|
|
1868
|
+
recover.** Measured over the slice, `error` appears **at most once per script
|
|
1869
|
+
and is always exactly one byte wide**, and on long-term 7.23.5 every byte after
|
|
1870
|
+
it comes back `none`. Most of the slice sits in that tail. A percentage computed
|
|
1871
|
+
over the whole stream therefore measures the device giving up, not the parser,
|
|
1872
|
+
which is why the tail is its own outcome and is never folded into agreement. The
|
|
1873
|
+
rule is not absolute across versions, so the report does not assert it: stable
|
|
1874
|
+
7.24.2 and development 7.25beta3 classify again after the `error` in one run and
|
|
1875
|
+
stop on fewer scripts, and
|
|
1876
|
+
`parser-recovered` in the table below is the figure that says by how much —
|
|
1877
|
+
zero would have been the claim, and it is not zero. Both counts are generated,
|
|
1878
|
+
not quoted here, so neither can go stale against the fixture.
|
|
1879
|
+
|
|
1880
|
+
**The projection is declared, never fitted.** `src/explain/highlight-projection.ts`
|
|
1881
|
+
maps each centrs token class to the device classes it predicts, justified by
|
|
1882
|
+
what the class was *defined* to mean rather than by where it happened to land.
|
|
1883
|
+
A projection tuned to the measurement would make the measurement circular. So a
|
|
1884
|
+
class whose intent has no single device counterpart — `string`, which spans a
|
|
1885
|
+
delimiter the device calls `syntax-meta`, escapes it calls `escaped` and an
|
|
1886
|
+
interior it declines to classify, and `value`, for which the device has no class
|
|
1887
|
+
at all — projects nowhere, and its bytes are counted as `unprojected` with the
|
|
1888
|
+
device's own answers tabulated beside them. That table is the evidence #264 B5
|
|
1889
|
+
needs in order to decide which provisional merge deserves to be split; it is not
|
|
1890
|
+
a score.
|
|
1891
|
+
|
|
1892
|
+
<!-- cspell:disable — the example table quotes corpus script paths verbatim, forum-thread typos included -->
|
|
1893
|
+
<!-- BEGIN GENERATED highlight-agreement — regenerate with `bun run explain:highlight-agreement:readme` -->
|
|
1894
|
+
The agreement report is re-derivable with
|
|
1895
|
+
`bun run explain:highlight-agreement` and needs no router and no corpus: it
|
|
1896
|
+
scores the token partition against
|
|
1897
|
+
`test/fixtures/explain/highlight-streams.slice.json`, 70 stratified scripts of
|
|
1898
|
+
committed `/console/inspect request=highlight` output. The figures below are
|
|
1899
|
+
generated from `test/fixtures/explain/highlight-agreement.json` by
|
|
1900
|
+
`bun run explain:highlight-agreement:readme`, gated against it by
|
|
1901
|
+
`bun run explain:highlight-agreement:readme:check`, and the fixture itself is
|
|
1902
|
+
gated against a fresh measurement by
|
|
1903
|
+
`test/unit/explain-highlight-agreement.test.ts`. Of 85,529 bytes at 7.24.2,
|
|
1904
|
+
27,974 are bytes **both** sides decided a syntax class for, and 99.99% of
|
|
1905
|
+
those agree (dev 99.99%, holdout 100.00%). Set `comment` aside — it is 65.63%
|
|
1906
|
+
of that decided region and mostly the corpus's harness-injected `# Source:`
|
|
1907
|
+
banner (#203) — and the remaining 9,615 bytes agree 99.98%. The device stops
|
|
1908
|
+
classifying at its one-byte `error`: 36 of 70 scripts carry one, and the
|
|
1909
|
+
41,968 bytes from there on are not a judgment about anything. The oracle
|
|
1910
|
+
itself moves between captures: 36 stop at 7.24.2 and 43 stop at 7.23.5 and 36
|
|
1911
|
+
stop at 7.25beta3, and 27 of the 70 streams differ between them. **This is a
|
|
1912
|
+
trend line, not a pass gate** — the slice is a per-(split, class) quota over a
|
|
1913
|
+
corpus that is 96.8% two authors, so the percentage describes this slice.
|
|
1914
|
+
|
|
1915
|
+
| centrs class | projects to | agree | disagree | unprojected | of decided |
|
|
1916
|
+
| ------------ | ----------- | ----: | -------: | ----------: | ---------: |
|
|
1917
|
+
| `comment` | `comment` | 18,359 | 0 | 0 | 100.00% |
|
|
1918
|
+
| `variable-local` | `variable-local` | 1,123 | 0 | 0 | 100.00% |
|
|
1919
|
+
| `variable-global` | `variable-global` | 582 | 0 | 0 | 100.00% |
|
|
1920
|
+
| `variable-auto` | `variable-auto` | 92 | 0 | 0 | 100.00% |
|
|
1921
|
+
| `variable-parameter` | `variable-parameter` | 459 | 0 | 0 | 100.00% |
|
|
1922
|
+
| `dir` | `dir` | 1,677 | 0 | 0 | 100.00% |
|
|
1923
|
+
| `cmd` | `cmd` | 2,244 | 0 | 0 | 100.00% |
|
|
1924
|
+
| `arg` | `arg`, `arg-dot`, `arg-scope` | 2,188 | 0 | 0 | 100.00% |
|
|
1925
|
+
| `arg-sep` | `syntax-meta` | 382 | 0 | 0 | 100.00% |
|
|
1926
|
+
| `operator` | `syntax-meta` | 237 | 2 | 0 | 99.16% |
|
|
1927
|
+
| `brace` | `syntax-meta` | 211 | 0 | 0 | 100.00% |
|
|
1928
|
+
| `string` | *abstains* | 0 | 0 | 499 | — |
|
|
1929
|
+
| `escaped` | `escaped` | 418 | 0 | 0 | 100.00% |
|
|
1930
|
+
| `value` | *abstains* | 0 | 0 | 144 | — |
|
|
1931
|
+
|
|
1932
|
+
| outcome | bytes @7.24.2 | bytes @7.23.5 | bytes @7.25beta3 |
|
|
1933
|
+
| ------- | ----: | ----: | ----: |
|
|
1934
|
+
| agree — both decided, projection accepts | 27,972 | 27,218 | 27,972 |
|
|
1935
|
+
| disagree — both decided, projection rejects | 2 | 2 | 2 |
|
|
1936
|
+
| unprojected — no declared projection covers the pair | 643 | 633 | 643 |
|
|
1937
|
+
| offline-silent — device decided, centrs abstained | 4,300 | 4,150 | 4,300 |
|
|
1938
|
+
| non-syntax — the device answered something syntax cannot decide | 847 | 733 | 847 |
|
|
1939
|
+
| device-silent — centrs decided, device said `none` | 5,815 | 5,674 | 5,815 |
|
|
1940
|
+
| both-silent | 3,886 | 3,702 | 3,886 |
|
|
1941
|
+
| parser-stopped — at/after the device's `error` byte, and silent from there | 41,968 | 43,417 | 41,968 |
|
|
1942
|
+
| parser-recovered — past that `error`, and the device classified anyway | 96 | 0 | 96 |
|
|
1943
|
+
|
|
1944
|
+
Where the device decided and offline analysis did not (4,300 bytes at 7.24.2,
|
|
1945
|
+
the next fill's target list): `syntax-meta` 3,122, `comment` 355,
|
|
1946
|
+
`variable-local` 326, `arg` 197, `escaped` 145, `dir` 66, 89 across the rest.
|
|
1947
|
+
Most of the `syntax-meta` share is whitespace the device merged into an
|
|
1948
|
+
adjacent structure run rather than a token centrs missed.
|
|
1949
|
+
|
|
1950
|
+
**Applicability** is the second axis (#263): of the bytes the device did
|
|
1951
|
+
answer, what kind of fact was it answering? Assigned from the pair — what
|
|
1952
|
+
centrs read the byte as and what the device called it — plus whether the
|
|
1953
|
+
captures agreed, never by assigning a whole class to one category. It covers
|
|
1954
|
+
every measured byte, the post-`error` tail included: where one capture stops
|
|
1955
|
+
parsing and the next does not, that tail IS the version disagreement, which is
|
|
1956
|
+
why it dominates the category.
|
|
1957
|
+
|
|
1958
|
+
| applicability @7.24.2 | bytes | leading cells |
|
|
1959
|
+
| ---------------------------- | ----: | ------------- |
|
|
1960
|
+
| offline-decidable | 32,002 | `comment` → `comment` 18,006, `unclassified` → `syntax-meta` 2,999, `cmd` → `cmd` 2,132 |
|
|
1961
|
+
| schema-dependent | 326 | `arg` → `obj-inactive` 326 |
|
|
1962
|
+
| state-dependent | 168 | `dir` → `obj-inactive` 120, `cmd` → `obj-inactive` 21, `unclassified` → `obj-dynamic` 20 |
|
|
1963
|
+
| version-dependent | 1,158 | `comment` → `comment` 353, `unclassified` → `obj-inactive` 159, `unclassified` → `syntax-meta` 123 |
|
|
1964
|
+
| uncategorized | 213 | `unclassified` → `obj-inactive` 90, `unclassified` → `variable-undefined` 75, `value` → `variable-undefined` 21 |
|
|
1965
|
+
| no-device-answer | 51,662 | `none` where the captures agree, wherever it falls |
|
|
1966
|
+
|
|
1967
|
+
One representative run per cell, so B5 reads a fragment rather than a count.
|
|
1968
|
+
`runs` is how often that exact fragment occurs; the location is a slice script
|
|
1969
|
+
and a byte offset into the stream the device saw.
|
|
1970
|
+
|
|
1971
|
+
| outcome | cell | fragment | runs | first at |
|
|
1972
|
+
| ------- | ---- | -------- | ---: | -------- |
|
|
1973
|
+
| disagree | `operator` → `arg` | `"in"` | 1 | `forum/amm0/topic-169456-having-the-where-filter-in-scripting-signifantly-increases-the-execution-time-an/post-0006-snippet-01.rsc` @615 |
|
|
1974
|
+
| unprojected | `string` → `syntax-meta` | `"\""` | 311 | `eworm/ppp-on-up.rsc` @435 |
|
|
1975
|
+
| unprojected | `value` → `syntax-meta` | `"\""` | 88 | `forum/amm0/topic-153357-using-wifiwave2-to-bridge-two-audience-wirelessly-thoughts-4-address-mode/post-0001-snippet-01.rsc` @1436 |
|
|
1976
|
+
| unprojected | `string` → `arg` | `"as-value"` | 1 | `forum/amm0/topic-266499-how-to-condense-variables-passed-through-to-function/post-0020-snippet-01.rsc` @279 |
|
|
1977
|
+
| unprojected | `string` → `cmd` | `"len"` | 1 | `forum/amm0/topic-155978-zerotier-on-mikrotik-a-rosetta-stone-v7-1-1/post-0001-snippet-01.rsc` @1794 |
|
|
1978
|
+
| unprojected | `string` → `dir` | `":"` | 1 | `forum/amm0/topic-155978-zerotier-on-mikrotik-a-rosetta-stone-v7-1-1/post-0001-snippet-01.rsc` @1793 |
|
|
1979
|
+
<!-- END GENERATED highlight-agreement -->
|
|
1980
|
+
<!-- cspell:enable -->
|
|
1981
|
+
|
|
1982
|
+
**What the disagreements are.** Two cells carry all of them, and neither is a
|
|
1983
|
+
lexing accident:
|
|
1984
|
+
|
|
1985
|
+
- `arg` → `syntax-meta`: the `=` byte the `arg` class merges in. The device
|
|
1986
|
+
reads a separator as structure. This is the measured cost of the merge the
|
|
1987
|
+
token-census block already flags, and it is the single largest reason `arg`
|
|
1988
|
+
scores below the other name-coincident classes.
|
|
1989
|
+
- `operator` → `arg`: the space-separated `in` of `:foreach conn in $conns`,
|
|
1990
|
+
which the device reads as the argument **name** `in`. The two oracles
|
|
1991
|
+
genuinely differ here — `:parse` IL carries `in` in the operator set, and the
|
|
1992
|
+
operator-abstention table above already records that `:foreach x in=$list`
|
|
1993
|
+
lowers with no `(in …)` node.
|
|
1994
|
+
|
|
1995
|
+
The third cell is gone. `cmd` → `dir` was the whole of #324 — a known menu path
|
|
1996
|
+
carrying arguments and no verb, where `verbsplit` promoted the last segment;
|
|
1997
|
+
those 12 bytes now abstain instead of disagreeing, and the 14 `dir` bytes the
|
|
1998
|
+
same statements contributed to `agree` abstain with them. That is the price,
|
|
1999
|
+
and [the fix states it](#the-menu-path-with-arguments-324): withdrawing a
|
|
2000
|
+
fabricated verb also withdraws the menu reading it was built on.
|
|
2001
|
+
|
|
2002
|
+
The `unprojected` residue is what #264 B5 read to decide which merges to
|
|
2003
|
+
split — [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) records
|
|
2004
|
+
the verdict on each. It was also where #325 lived:
|
|
2005
|
+
a quoted string claimed any `$[…]` substitution inside it, so bytes the device
|
|
2006
|
+
resolves to `cmd` or `dir` came back as one opaque `string` run even though
|
|
2007
|
+
`structure.subcommands` had already resolved that same span. Those cells have
|
|
2008
|
+
largely emptied (`string` → `cmd` 42 → 3, `string` → `dir` 19 → 1, with the
|
|
2009
|
+
same fix recovering padded bare brackets `[ … ]` from `offline-silent`), and
|
|
2010
|
+
the two residues left are stated rather than fixed:
|
|
2011
|
+
|
|
2012
|
+
- `$(…)` — a **parenthesized** interpolation carrying a bracket, as in
|
|
2013
|
+
`apikey-len=$([:len $apikey])`. Only `$[` is scanned inside a string, so the
|
|
2014
|
+
inner command is not resolved there in the first place; this is a parser
|
|
2015
|
+
reach, not a fill-order question (4 bytes on the slice).
|
|
2016
|
+
- a bare-word **argument** inside a substitution — `as-value` in
|
|
2017
|
+
`:put "$[/ip address print as-value]"`. The path fill claims the path and
|
|
2018
|
+
verb; the argument fill reads `statements[].arguments`, and a substitution is
|
|
2019
|
+
not a statement, so it has no candidate to claim (8 bytes).
|
|
2020
|
+
|
|
2021
|
+
The `$` sigil and the `[` / `]` delimiters stay `string`, and `value` bytes
|
|
2022
|
+
are, to the device, simply unclassified — both settled by
|
|
2023
|
+
[The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) below.
|
|
2024
|
+
|
|
2025
|
+
### The vocabulary, and what earns a class (#264 B5)
|
|
2026
|
+
|
|
2027
|
+
`data.tokens[]` shipped with its `class` marked provisional from B1 onward —
|
|
2028
|
+
emit first, name later (#293 design decision 2). This is the "name later", and
|
|
2029
|
+
the vocabulary below is what `@tikoci/centrs/explain` offers a consumer. It is
|
|
2030
|
+
no longer provisional.
|
|
2031
|
+
|
|
2032
|
+
**The rule a class has to pass.** A merged class is split when both hold, and
|
|
2033
|
+
stays merged otherwise:
|
|
2034
|
+
|
|
2035
|
+
1. **A consumer needs the distinction** — it changes what that consumer renders
|
|
2036
|
+
or decides, not just how a token dump reads.
|
|
2037
|
+
2. **The envelope cannot already supply it** — the distinction is not
|
|
2038
|
+
recoverable by joining `tokens[]` to another published surface on byte
|
|
2039
|
+
offsets.
|
|
2040
|
+
|
|
2041
|
+
The second test is the one that does the work. A class that restates a fact
|
|
2042
|
+
`structure`, `symbols` or `values` already publishes is a second place for that
|
|
2043
|
+
fact to go stale, and the two copies can then disagree about the same bytes. So
|
|
2044
|
+
the question is never "would a name here be nice"; it is "what would a consumer
|
|
2045
|
+
have to do without it". Applied to the five merges the fills carried, exactly
|
|
2046
|
+
one passes:
|
|
2047
|
+
|
|
2048
|
+
| merge | what a consumer would do without it | verdict |
|
|
2049
|
+
| ----- | ----------------------------------- | ------- |
|
|
2050
|
+
| `arg` — the name bytes **and** the `=` | subtract one from `arguments[].valueSpan.start`, which exists only where the STRICT reading read. **5,088 of the 16,526 `=` runs the fill paints (30.8%, in 436 of 948 corpus scripts) sit in a statement whose `arguments` refused** — led by `do=` (2,393), `else=` (328) and `in=` (212), the block binder of every `:if`, `:foreach` and `:while`. For those there is nothing to subtract from. | **split** → `arg-sep` |
|
|
2051
|
+
| `operator` — 26 spellings + 2 aliases | slice the input at the token's span: the spelling *is* the bytes, and `src/explain/operators.ts` maps it to its IL node | keep |
|
|
2052
|
+
| `value` — every leaf, whatever its shape | join `data.values.occurrences[]` on the offset; #225's three axes (shape, observed, schema) live there, and a class per shape would be a fourth copy free to contradict them | keep |
|
|
2053
|
+
| `brace` — scope and array delimiters | `structure.blocks[]` locates every scope block; an array container is the `values.occurrences[]` entry that is some other occurrence's `parent` | keep |
|
|
2054
|
+
| `string` — delimiters, escapes, interior | the opening delimiter is the run's first byte and, where the run is closed, the last is its partner (an unterminated one is already a located defect). The **escapes were a different answer** — see below | **partly split** → `escaped`; delimiters and interior stay merged |
|
|
2055
|
+
|
|
2056
|
+
Re-derive the `arg-sep` figures with `bun run explain:arg-reach`, which prices
|
|
2057
|
+
the same gap it prices the reach fork with.
|
|
2058
|
+
|
|
2059
|
+
**What the split cost.** Nothing, which was the claim and is now the
|
|
2060
|
+
measurement. Classified bytes are identical — 1,023,405 of 1,426,731 (71.73%),
|
|
2061
|
+
equal to the last recorded decimal before and after — and `arg` gave up exactly
|
|
2062
|
+
the 16,526 bytes `arg-sep` claims, one per attribute. Token count rises 214,503
|
|
2063
|
+
→ 231,026 because one run became two. **This is a retag.**
|
|
2064
|
+
|
|
2065
|
+
(Those totals are the corpus as it stood for THIS measurement. The classified
|
|
2066
|
+
total is 1,023,107 today: #347/#348 later classified 8 further bytes, and #355's
|
|
2067
|
+
fail-closed symbol stop then unclaimed 306 of them — variable bytes sitting
|
|
2068
|
+
after a bare apostrophe, which the device never reaches. The identity being
|
|
2069
|
+
claimed here is between the two sides of the split, not with any later total —
|
|
2070
|
+
restating it against today's figure would credit this retag with a movement a
|
|
2071
|
+
different change made. The generated block above carries the current numbers.)
|
|
2072
|
+
|
|
2073
|
+
One count is not a clean split: `arg`'s own run count falls 16,526 → 16,523.
|
|
2074
|
+
Three attributes lost their name run entirely, because an earlier `variable-*`
|
|
2075
|
+
span had already claimed the name bytes — `set $ifcId ssid=$ssid`, and two
|
|
2076
|
+
`on-error=` in one snippet — leaving only the `=` on the residual. Before the
|
|
2077
|
+
split those three emitted a token *called* `arg` that contained nothing but an
|
|
2078
|
+
`=`. That is the merge in miniature, and it is why the number goes down.
|
|
2079
|
+
|
|
2080
|
+
**The agreement number moved a long way, and it is still not a score.** Against
|
|
2081
|
+
the committed device slice, projection disagreements fall **360 → 2** and the
|
|
2082
|
+
decided region goes 98.66% → 99.99% (`arg` alone 85.50% → 100.00%). Read that
|
|
2083
|
+
as the withdrawal of a wrong assertion, not as a parser that got better: centrs
|
|
2084
|
+
stopped calling a separator a name over the 358 runs the device had been
|
|
2085
|
+
reading as `syntax-meta` all along, and not one byte changed which analyzer
|
|
2086
|
+
found it. The two survivors are the `operator` → `arg` `in` of
|
|
2087
|
+
`:foreach conn in $conns`, where the two oracles genuinely differ.
|
|
2088
|
+
|
|
2089
|
+
The projection entry was **declared before the measurement, never fitted to
|
|
2090
|
+
it**: `arg-sep` predicts `syntax-meta` because a separator is structure
|
|
2091
|
+
punctuation, the same reason `brace` and `operator` predict it. The slice then
|
|
2092
|
+
put 358 of 358 there.
|
|
2093
|
+
|
|
2094
|
+
**Where the tolerant reading reaches the envelope (#316's open decision).** It
|
|
2095
|
+
reaches it *as the token partition*, and that is the only place it should.
|
|
2096
|
+
`structure.statements[].arguments` stays the strict all-or-nothing reading,
|
|
2097
|
+
because its consumer renders a runnable request and a half-read argument list
|
|
2098
|
+
changes what the rendered command does. But with the `=` named, `tokens[]`
|
|
2099
|
+
locates the argument on its own:
|
|
2100
|
+
|
|
2101
|
+
```text
|
|
2102
|
+
/ip route add dst-address=1.1.1.1 gateway=$gw
|
|
2103
|
+
arguments: { read: false, why: "a variable value" }
|
|
2104
|
+
tokens: … arg("gateway") arg-sep("=") unclassified("$") variable-parameter("gw")
|
|
2105
|
+
|
|
2106
|
+
:if ($x = 1) do={ :put "hi" }
|
|
2107
|
+
arguments: { read: false, why: "a substitution or expression value" }
|
|
2108
|
+
tokens: … operator("=") … arg("do") arg-sep("=") brace("{") …
|
|
2109
|
+
```
|
|
2110
|
+
|
|
2111
|
+
Neither statement publishes a single argument, and in both a browser consumer
|
|
2112
|
+
can still see a located `gateway=` / `do=`. The second line is the
|
|
2113
|
+
discrimination working as well: the same `=` byte is `operator` inside
|
|
2114
|
+
`($x = 1)` and `arg-sep` in `do=`, decided by fill order and meaning rather
|
|
2115
|
+
than by a byte scan.
|
|
2116
|
+
|
|
2117
|
+
What `tokens[]` still does **not** carry is the *reason* a value was not
|
|
2118
|
+
decoded. `undecided` stays off `ExplainArgumentToken` and out of the
|
|
2119
|
+
envelope: the envelope's value-fact surface is `data.values.occurrences[]`,
|
|
2120
|
+
where absence at an offset already *is* the abstention, and a second spelling
|
|
2121
|
+
of the same abstention would be free to disagree with it. A consumer that needs
|
|
2122
|
+
the reason imports `lexExplainArgumentTokens` from the package root — which is
|
|
2123
|
+
why that export exists.
|
|
2124
|
+
|
|
2125
|
+
**The second split the rule admits, now made: `escaped`.** A *valid* string
|
|
2126
|
+
escape is recoverable from nothing. Only INVALID ones surface elsewhere, as
|
|
2127
|
+
`bad-string-escape` diagnostics, so a consumer holding `tokens[]` has no
|
|
2128
|
+
published surface to join against to find the valid ones — and the device does
|
|
2129
|
+
class them, as one of the 19 highlight classes. Both halves of the rule hold,
|
|
2130
|
+
so the class is earned.
|
|
2131
|
+
|
|
2132
|
+
Unlike `arg-sep` this is a **fill change** rather than a pure relabel of one
|
|
2133
|
+
analyzer's own output: the escape grammar (uppercase-hex `\XX`, the whitespace
|
|
2134
|
+
continuation, the twelve single-character forms) has to be walked, and the
|
|
2135
|
+
bytes come out of two fills at once rather than one.
|
|
2136
|
+
`src/explain/escape-tokens.ts` runs between `arg` and `value`.
|
|
2137
|
+
|
|
2138
|
+
**The walk is not new, and deliberately not a second one.**
|
|
2139
|
+
[String escape validation](#string-escape-validation-247-252) already owned this
|
|
2140
|
+
grammar for `bad-string-escape`. `walkStringEscapes` now returns *both* halves
|
|
2141
|
+
of that one reading — the valid escapes and the first invalid one — so an
|
|
2142
|
+
`escaped` token and a `bad-string-escape` diagnostic cannot disagree about which
|
|
2143
|
+
bytes are an escape. Deriving them from two walks would have made that
|
|
2144
|
+
disagreement possible, and the document is walked once rather than twice. The
|
|
2145
|
+
walk stops at the first invalid escape, so `escaped` is the prefix the validator
|
|
2146
|
+
vouched for: past a malformed escape the string boundaries are themselves in
|
|
2147
|
+
doubt, and painting there would be a claim the walk cannot support.
|
|
2148
|
+
|
|
2149
|
+
**What the split cost: nothing, and the census proves it.** On the corpus
|
|
2150
|
+
`escaped` claims **19,960 bytes over 7,408 tokens**, and every one of those
|
|
2151
|
+
bytes came from a fill that already held it — `string` gives up 17,672 and
|
|
2152
|
+
`value` 2,288, which is exactly 19,960. Classified bytes are **identical to the
|
|
2153
|
+
last decimal**, 1,023,405 of 1,426,731 (71.73076073906013%), so in byte terms
|
|
2154
|
+
this is a retag even though it took a new walk to make — again, the identity is
|
|
2155
|
+
between this change's own before and after, and the corpus total has since moved
|
|
2156
|
+
to 1,023,107 / 241,539 tokens under #347/#348 and #355. Only the token count
|
|
2157
|
+
moves, 231,026 → 241,663: the 7,408 new `escaped` tokens plus the 3,229
|
|
2158
|
+
fragments they leave behind when they split a run they sit inside (`string`
|
|
2159
|
+
+2,643, `value` +586). That second number is well under one per escape because
|
|
2160
|
+
escapes cluster — adjacent ones like `\C3\BD`, and ones flush against a
|
|
2161
|
+
delimiter, add no fragment at all.
|
|
2162
|
+
|
|
2163
|
+
**What it bought.** Against the committed device slice `escaped` claims **418
|
|
2164
|
+
bytes on 100.00% agreement, 0 disagreements** — the 53 `string` → `escaped`
|
|
2165
|
+
runs and the 1 `value` → `escaped` that were `unprojected` before, scored for
|
|
2166
|
+
the first time. Total `unprojected` falls **1,051 → 633 bytes** (−39.8%) and the
|
|
2167
|
+
both-decided region grows 26,794 → 27,212. No byte changed which analyzer found
|
|
2168
|
+
it and no disagreement was introduced.
|
|
2169
|
+
|
|
2170
|
+
The projection entry was again **declared before it was measured**: `escaped`
|
|
2171
|
+
predicts `escaped` because the class means one valid string-internal escape and
|
|
2172
|
+
that is the run the device gives the same name. `escaped` was already listed in
|
|
2173
|
+
`DEVICE_CLASS_KIND` as a syntax class; the agreement report's `unprojected`
|
|
2174
|
+
table showed the bytes were being merged away, it did not pick the target.
|
|
2175
|
+
|
|
2176
|
+
**The residue this does not claim, sized.** 145 bytes across 24 runs in 4 slice
|
|
2177
|
+
scripts stay `unclassified` where the device says `escaped`, and every one of
|
|
2178
|
+
them is the same shape: a `\`-newline **line continuation in code**, outside any
|
|
2179
|
+
string (`… != true) \<newline> do={`). The device's run there swallows the
|
|
2180
|
+
next line's indentation too, so the 145 bytes are 48 of `\`+terminator and 97 of
|
|
2181
|
+
trailing whitespace. That is a different claim from a string escape — centrs has
|
|
2182
|
+
no continuation token, and matching the device would mean deciding how much
|
|
2183
|
+
following whitespace a continuation owns — so it is **not** folded in here. It
|
|
2184
|
+
belongs with the continuation question #225 already owns (what a continuation
|
|
2185
|
+
does to a value's interior); this paragraph is its size, not a plan.
|
|
2186
|
+
|
|
2187
|
+
Delimiters are the near miss that shows the rule discriminates rather than
|
|
2188
|
+
rubber-stamps: `"` bytes are 311 `string` → `syntax-meta` runs and 88
|
|
2189
|
+
`value` → `syntax-meta`, the device classes them too, and they stay merged
|
|
2190
|
+
anyway, because a consumer can find them by looking at the run's own first and
|
|
2191
|
+
last byte. The `$` sigil and the `[` / `]` of a substitution stay `string` on
|
|
2192
|
+
the same ground — `structure.subcommands` already publishes the resolution, and
|
|
2193
|
+
its `innerSpan` locates the brackets around it.
|
|
1297
2194
|
|
|
1298
2195
|
### Designed, not implemented (the CLI surface, #202b)
|
|
1299
2196
|
|
|
@@ -1314,12 +2211,11 @@ That is the difference — an accepted flag must do something observable.
|
|
|
1314
2211
|
|
|
1315
2212
|
## MCP and library surfaces
|
|
1316
2213
|
|
|
1317
|
-
- **
|
|
1318
|
-
`
|
|
1319
|
-
(`
|
|
1320
|
-
|
|
1321
|
-
|
|
1322
|
-
enrichment, which is rosetta's (#175). A consumer that wants enrichment takes
|
|
2214
|
+
- **Consumers use the layer that answers their question.** Byte ranges and
|
|
2215
|
+
their classes (`spans` and the shipped `tokens` facet) serve editor/token
|
|
2216
|
+
consumers; the resolved reading (`structure`, `symbols`, `values`) carries
|
|
2217
|
+
path, verb, arguments, and value facts; doc/version enrichment belongs to
|
|
2218
|
+
rosetta (#175). A consumer that wants enrichment takes
|
|
1323
2219
|
the **middle** layer: given `{path, verb, args}` plus a version it does its
|
|
1324
2220
|
own lookups, whereas handing it tokens would make it re-derive from bytes a
|
|
1325
2221
|
path centrs already resolved. So #223 is a decision about the structure
|
|
@@ -1350,8 +2246,102 @@ That is the difference — an accepted flag must do something observable.
|
|
|
1350
2246
|
shapes. `lsp-routeros-ts` and
|
|
1351
2247
|
tikbook are the intended external consumers (hover/diagnostics/completion/
|
|
1352
2248
|
semantic tokens over these calls); an LSP *protocol* surface on centrs
|
|
1353
|
-
stays out of scope (#90)
|
|
1354
|
-
|
|
2249
|
+
stays out of scope (#90). A bounded offline browser/editor consumption check
|
|
2250
|
+
now precedes vocabulary stabilization (#264 B5); full live LSP integration
|
|
2251
|
+
remains later work. The entry is `@tikoci/centrs/explain` and it is browser-verified —
|
|
2252
|
+
see [The offline package entry](#the-offline-package-entry-312).
|
|
2253
|
+
|
|
2254
|
+
### The offline package entry (#312)
|
|
2255
|
+
|
|
2256
|
+
**`@tikoci/centrs/explain` -> `src/explain.ts`** is the documented public entry for
|
|
2257
|
+
offline analysis. It is a real boundary, not a naming convention: the module
|
|
2258
|
+
graph reachable from it opens no connection, reads no CDB, and touches no
|
|
2259
|
+
filesystem, so it bundles and runs in a browser, a Worker, or an editor host.
|
|
2260
|
+
|
|
2261
|
+
```sh
|
|
2262
|
+
bun run explain:browser-consumer # boundary + executed consumer report
|
|
2263
|
+
bun run explain:browser-consumer --json # machine-readable
|
|
2264
|
+
```
|
|
2265
|
+
|
|
2266
|
+
Both halves are gated in the unit suite by
|
|
2267
|
+
`test/unit/explain-browser-entry.test.ts`, so neither needs a router or the
|
|
2268
|
+
corpus.
|
|
2269
|
+
|
|
2270
|
+
**Why a gate and not a note.** The entry reached the transport stack until this
|
|
2271
|
+
landed, for two *pure* helpers: `toYaml` (from `retrieve.ts`) and
|
|
2272
|
+
`canonicalizeExecuteCommand`/`isWriteShaped` (from `execute.ts`). Importing
|
|
2273
|
+
either dragged in `mac-telnet` (`node:dgram`, `node:os`), `native-api`
|
|
2274
|
+
(`node:crypto`) and `ssh`. Both now live in `src/core/` — `src/core/yaml.ts` and
|
|
2275
|
+
`src/core/execute-command.ts` — moved character for character, with `execute.ts`
|
|
2276
|
+
re-exporting the gate because it remains that gate's documented public home.
|
|
2277
|
+
**Nothing about the script-vs-structured gate moved with them**; the locked
|
|
2278
|
+
contract table in `test/unit/execute-canonicalize-contract.test.ts` still imports
|
|
2279
|
+
from `execute` and still passes.
|
|
2280
|
+
|
|
2281
|
+
That coupling was re-introduced once already by a one-line import, which is why
|
|
2282
|
+
the static half of the check names the offending **edge** (`explain.ts ->
|
|
2283
|
+
execute.ts -> mac-telnet-console.ts -> mac-telnet.ts`) rather than leaving a
|
|
2284
|
+
reader with the bundler's polyfill complaint four modules downstream.
|
|
2285
|
+
|
|
2286
|
+
**What "runs in a browser" is proven by.** Compilation is not the proof. A
|
|
2287
|
+
bundler silently substitutes browser polyfills, so the check bundles a worker
|
|
2288
|
+
entry around the module with `--target browser`, wraps the output in a function
|
|
2289
|
+
that shadows every host global (`Bun`, `process`, `require`, `module`,
|
|
2290
|
+
`exports`, `__dirname`, `__filename`), and **spawns that bundle as the Worker**.
|
|
2291
|
+
`Bun` is a non-configurable global — `delete globalThis.Bun` throws — so
|
|
2292
|
+
shadowing, not deletion, is what makes the absence real, and doing it in the
|
|
2293
|
+
bundler's banner keeps the harness free of `eval`/`new Function` while making
|
|
2294
|
+
the bundle text on disk the artifact that carries the claim.
|
|
2295
|
+
|
|
2296
|
+
It then compares the whole `ExplainData` against the Bun-native run, field for
|
|
2297
|
+
field: structure, diagnostics, tokens, spans, `canonical`, and
|
|
2298
|
+
`runtimeAcceptance`. Fourteen cases, covering normal and write-shaped commands,
|
|
2299
|
+
multi-statement and nested input, three malformed classes, Unicode, astral, a
|
|
2300
|
+
leading BOM, IPv6, empty input, and the facets switched off. All fourteen are
|
|
2301
|
+
byte-identical today.
|
|
2302
|
+
|
|
2303
|
+
Two things the bundle deliberately exercises rather than bypasses. It imports
|
|
2304
|
+
through the **published subpath** `@tikoci/centrs/explain`, not
|
|
2305
|
+
`../src/explain.ts`, so a typo in `package.json`'s `exports` map fails the gate
|
|
2306
|
+
instead of leaving it green while every real consumer breaks (the graph walk
|
|
2307
|
+
additionally asserts that the `exports` entry and the measured file are the same
|
|
2308
|
+
path). And it calls `resolveExplainFormat(undefined)`, whose `env` default is
|
|
2309
|
+
the `typeof`-guarded `hostEnv()` — nothing else reaches that parameter, so
|
|
2310
|
+
without this a regression to a bare `Bun.env` would pass every case here while a
|
|
2311
|
+
browser caller of that exported function got a `TypeError`.
|
|
2312
|
+
|
|
2313
|
+
The shadowing is **measured from inside the bundle**, not assumed: the worker
|
|
2314
|
+
reports which host globals it could still see, and an empty list is part of the
|
|
2315
|
+
gate. Without it, a wrapper that silently stopped being emitted would leave
|
|
2316
|
+
every "identical" above produced with host APIs in reach, proving nothing. Only
|
|
2317
|
+
`Bun` and `process` are probed, because they are the only two of the seven a
|
|
2318
|
+
bundle can honestly observe — Bun constant-folds `typeof require` to
|
|
2319
|
+
`"function"` at build time, mentioning `__dirname`/`__filename` makes it inject
|
|
2320
|
+
the build machine's absolute paths as local constants (the probe would cause
|
|
2321
|
+
what it reports), and mentioning `module`/`exports` reclassifies the entry as
|
|
2322
|
+
CommonJS, whose top-level `export default` is a syntax error inside the wrapper.
|
|
2323
|
+
All seven stay shadowed regardless.
|
|
2324
|
+
|
|
2325
|
+
The one tolerated host builtin is **`node:net`**, for `isIP` in
|
|
2326
|
+
`src/explain/values.ts` — a pure predicate with no socket in it. A bundler
|
|
2327
|
+
replaces it with a regex pair, so the browser build genuinely runs a *different*
|
|
2328
|
+
`isIP` than Bun does; the two IPv6 cases exist to price that substitution, which
|
|
2329
|
+
is the only place its verdict is observable. A builtin that polyfills to a stub
|
|
2330
|
+
(`node:fs`) or throws (`node:dgram`) could not be checked this way, so the
|
|
2331
|
+
allowlist is a boundary, not a list to extend.
|
|
2332
|
+
|
|
2333
|
+
**What it does not claim.** Not that the `centrs` package works in a browser —
|
|
2334
|
+
it does not, and is not meant to. Not DOM integration, no real browser engine.
|
|
2335
|
+
And these cases are a boundary corpus, not a parser-coverage corpus; measured
|
|
2336
|
+
parser evidence stays in the censuses and
|
|
2337
|
+
[Device agreement](#device-agreement-for-the-token-partition-264-b4-263).
|
|
2338
|
+
|
|
2339
|
+
**What this hands #264 B5.** The comparison serializes the entire public result,
|
|
2340
|
+
so it pins the vocabulary a consumer actually depends on: every `tokens[]` class
|
|
2341
|
+
name, every span pair, and the `ExplainData` key set are now load-bearing across
|
|
2342
|
+
a process boundary. A B5 rename is therefore a visible break here rather than a
|
|
2343
|
+
silent one — which is the point of settling the vocabulary before an external
|
|
2344
|
+
consumer commits to it.
|
|
1355
2345
|
|
|
1356
2346
|
## Non-goals
|
|
1357
2347
|
|
|
@@ -1418,7 +2408,8 @@ of 913). The ratified answers hold for what was measured; they are **not**
|
|
|
1418
2408
|
evidence of coverage over device configuration, which is the genre most likely
|
|
1419
2409
|
to be handed to `explain` and the one that concentrates Q4 path context and
|
|
1420
2410
|
Q6's bare-path ambiguity. Quote phase-0 numbers per genre, never as one blended
|
|
1421
|
-
figure. #203
|
|
2411
|
+
figure. #203 records the capture decision and caveat; #336 owns the current
|
|
2412
|
+
corpus repin and remeasurement.
|
|
1422
2413
|
|
|
1423
2414
|
Those figures are re-derivable, not eyeballed: `bun run explain:corpus-census`
|
|
1424
2415
|
(`scripts/explain-corpus-census.ts`) reproduces the whole genre table from
|
|
@@ -1430,8 +2421,33 @@ literals, and DHCP client-ids as `:` directives, which under-counts exactly the
|
|
|
1430
2421
|
config genre in question). Since phase 0 the corpus has grown to 948 with a
|
|
1431
2422
|
`tangentsoft` stratum of 35 genuine `/export … terse` device captures, taking
|
|
1432
2423
|
export-banner share to 4.6%; **that stratum is not part of any phase-0 figure
|
|
1433
|
-
above
|
|
1434
|
-
|
|
2424
|
+
above**. Current whole-corpus offline censuses include its source text.
|
|
2425
|
+
|
|
2426
|
+
The pinned snapshot carries **three complete builds** — long-term `7.23.5`,
|
|
2427
|
+
stable `7.24.2` and development `7.25beta3` — each with *both* oracles over all
|
|
2428
|
+
948 scripts, alongside six partial captures (parseIL at 7.20.8, 7.22.1,
|
|
2429
|
+
7.23rc1; highlight at 7.9.2, 7.23.2, 7.24rc2) that cover only 913. Schema v4
|
|
2430
|
+
names that split rather than leaving it to be discovered by hand: the
|
|
2431
|
+
`v_version_coverage` view classifies each build, and
|
|
2432
|
+
`coverage_class = 'complete'` is the filter every version-comparable query uses.
|
|
2433
|
+
`source_scripts` is byte-identical across both repins that reached this
|
|
2434
|
+
snapshot, so every text-derived report — the token and value censuses,
|
|
2435
|
+
`explain:arg-reach`, the browser-consumer proof — is unchanged by construction,
|
|
2436
|
+
and only the IL-derived operator census moved.
|
|
2437
|
+
|
|
2438
|
+
**The three complete builds do not agree with each other, and that is the
|
|
2439
|
+
finding.** Scored over all 948 scripts:
|
|
2440
|
+
|
|
2441
|
+
| oracle | long-term 7.23.5 ↔ stable 7.24.2 | stable 7.24.2 ↔ development 7.25beta3 |
|
|
2442
|
+
| ------ | -------------------------------: | ------------------------------------: |
|
|
2443
|
+
| `:parse` IL (`il_sha256`) | 87 scripts differ | 11 |
|
|
2444
|
+
| `highlight` (count + class set) | 33 | 0 |
|
|
2445
|
+
|
|
2446
|
+
Stable and development are the same build for these purposes; long-term is the
|
|
2447
|
+
one that moves. So an operator-axis or token-partition figure is a claim *about
|
|
2448
|
+
a channel*, not about RouterOS — which is why the census reports its builds as
|
|
2449
|
+
separate columns and states its version set, rather than collapsing them to one
|
|
2450
|
+
number. Long-term is also the channel a real device most likely runs.
|
|
1435
2451
|
|
|
1436
2452
|
The corpus itself is not in this repo and is not moving here — `lsp-routeros-ts`
|
|
1437
2453
|
owns producing snapshots, centrs owns which snapshot it measures against.
|
|
@@ -1553,75 +2569,106 @@ validity check: `.scratch/q203-blocker-validity.ts`.
|
|
|
1553
2569
|
**Non-blocking / deferred (not ratification-gating):** Q5 expression depth, Q7
|
|
1554
2570
|
tokenizer corners, Q9 potential-command taxonomy, Q12 span-vocabulary draft —
|
|
1555
2571
|
these refine the surface during implementation but did not gate ratification.
|
|
1556
|
-
|
|
1557
|
-
|
|
1558
|
-
|
|
1559
|
-
|
|
1560
|
-
|
|
2572
|
+
Two of the four have since closed. **Q7** became #201 (lexical boundaries —
|
|
2573
|
+
sigil spellings, escape validity, statement-start eligibility) and #199 (the
|
|
2574
|
+
shared scanner's `$[…]`-in-string blind spot), both raised by the Q13 promotion
|
|
2575
|
+
and both now closed. **#199**'s fix is the one shared scanner,
|
|
2576
|
+
`src/explain/quoted-string.ts`, which is what keeps the segmenter, the brace
|
|
2577
|
+
classifier and the block scan from drifting on where a string ends; **#201** is
|
|
2578
|
+
the wider one, spanning statement context, sigil/escape reading and defect
|
|
2579
|
+
detection across several already-promoted modules.
|
|
2580
|
+
**Q12** closed with #264: the vocabulary is settled on a stated rule and is no
|
|
2581
|
+
longer provisional — see
|
|
2582
|
+
[The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5). **Q5** and
|
|
2583
|
+
**Q9** remain phase-1 measurements. The
|
|
1561
2584
|
probe framework's disposition (reuse of the mutation suite, coordinate
|
|
1562
2585
|
fixtures, and adversarial generators as product test fixtures) is tracked in
|
|
1563
2586
|
issue #186 rather than carried into mainline.
|
|
1564
2587
|
|
|
1565
2588
|
## Definition of done and staging
|
|
1566
2589
|
|
|
1567
|
-
|
|
1568
|
-
|
|
1569
|
-
|
|
1570
|
-
|
|
1571
|
-
|
|
1572
|
-
|
|
1573
|
-
|
|
1574
|
-
|
|
1575
|
-
|
|
1576
|
-
|
|
1577
|
-
|
|
1578
|
-
|
|
1579
|
-
|
|
1580
|
-
|
|
1581
|
-
|
|
1582
|
-
|
|
1583
|
-
|
|
1584
|
-
|
|
1585
|
-
|
|
1586
|
-
|
|
1587
|
-
|
|
1588
|
-
|
|
1589
|
-
|
|
1590
|
-
|
|
1591
|
-
|
|
1592
|
-
|
|
1593
|
-
|
|
1594
|
-
|
|
1595
|
-
|
|
1596
|
-
|
|
1597
|
-
|
|
1598
|
-
|
|
1599
|
-
|
|
1600
|
-
|
|
1601
|
-
|
|
1602
|
-
-
|
|
1603
|
-
|
|
1604
|
-
|
|
1605
|
-
|
|
1606
|
-
|
|
1607
|
-
|
|
1608
|
-
|
|
1609
|
-
|
|
1610
|
-
|
|
1611
|
-
|
|
1612
|
-
|
|
1613
|
-
|
|
1614
|
-
|
|
1615
|
-
|
|
1616
|
-
|
|
1617
|
-
|
|
1618
|
-
|
|
1619
|
-
|
|
1620
|
-
-
|
|
1621
|
-
|
|
1622
|
-
|
|
1623
|
-
|
|
1624
|
-
|
|
2590
|
+
Verification follows the [constitution](../../docs/CONSTITUTION.md#done-definition).
|
|
2591
|
+
The phase-0 grounding and promoted phase-1 CLI/library core (#185/#202) are
|
|
2592
|
+
already in the product. The remaining task graph is maintained in #90; issue
|
|
2593
|
+
closure for an earlier slice does not establish the baseline below.
|
|
2594
|
+
|
|
2595
|
+
### Offline baseline acceptance
|
|
2596
|
+
|
|
2597
|
+
The offline capability can advance from `coded` to `verified` when:
|
|
2598
|
+
|
|
2599
|
+
- **Examples and diagnostics:** the offline examples and focused analysis/CLI
|
|
2600
|
+
contract tests pass. Known false confident readings in the baseline task
|
|
2601
|
+
list, starting with #311's missing separator, are fixed with source-located
|
|
2602
|
+
diagnostics and accepted/malformed controls. Unknown or ambiguous input
|
|
2603
|
+
remains explicit; no fabricated statements, paths, values, or runtime acceptance.
|
|
2604
|
+
- **Bounded execution:** the existing public `explainCommand` performance,
|
|
2605
|
+
coordinate, depth, no-throw, and determinism contracts pass. A reproduced
|
|
2606
|
+
performance failure is investigated before changing its threshold (#313). Cost
|
|
2607
|
+
must grow with input size and no faster: a wall-clock budget on a thermally
|
|
2608
|
+
throttled machine measures the machine, so the growth **shape** is pinned by a
|
|
2609
|
+
ratio between two sizes timed back to back, and an absolute threshold is never
|
|
2610
|
+
raised in place of finding the growing term. One input shape is not enough —
|
|
2611
|
+
#313's separator-free input carries no symbols, so it never reaches the
|
|
2612
|
+
symbol/value machinery where the next growth term was hiding (#317), and
|
|
2613
|
+
neither of those carries a `do={…}`, which is where the third one was (#320).
|
|
2614
|
+
A guarded shape is only guarded for the analyzers it actually runs, so a new
|
|
2615
|
+
growth guard states which analyzers its input reaches and is mutation-tested
|
|
2616
|
+
against the term it removed. **Met:** #322's fixed-size depth guard grows
|
|
2617
|
+
nesting 16× (depth 4 → 64) under the public token/curl path and stays below
|
|
2618
|
+
the 6× bound; deterministic companion guards cover colon-led, bare-directive,
|
|
2619
|
+
retry, array/substitution, bracket, and string-interpolation shapes and fail
|
|
2620
|
+
when child-scope traversal is restored.
|
|
2621
|
+
- **Measured token surface:** the total, gapless `--tokens` partition and
|
|
2622
|
+
generated censuses remain reproducible; #264 B4 / #263's projection and
|
|
2623
|
+
device-agreement report are in
|
|
2624
|
+
[Device agreement](#device-agreement-for-the-token-partition-264-b4-263),
|
|
2625
|
+
re-derivable without a router or the corpus. Report decided errors,
|
|
2626
|
+
abstentions, version/state differences, and corpus bias separately — the
|
|
2627
|
+
report's outcome buckets and its four applicability categories are that
|
|
2628
|
+
separation. Classified-byte percentage is not syntax coverage, and agreement
|
|
2629
|
+
is a trend, not a new numerical pass gate. A token-level rule and a token fill
|
|
2630
|
+
read the skip-tolerant argument stream, never the strict REST reading (#316);
|
|
2631
|
+
the two share one boundary walk, the strict reading's reach is unchanged, and
|
|
2632
|
+
the skip-vs-prefix fork is priced rather than assumed —
|
|
2633
|
+
[The token stream a rule reads](#the-token-stream-a-rule-reads-316). That
|
|
2634
|
+
stream reaches the envelope as the token partition and nowhere else, which is
|
|
2635
|
+
what makes the `=` boundary visible to a consumer whose statement published
|
|
2636
|
+
no arguments at all (#264 B5).
|
|
2637
|
+
- **Usable library contract:** one bounded browser/editor consumer imports a
|
|
2638
|
+
documented public offline entry point, runs analysis without transport/CDB
|
|
2639
|
+
dependencies, and verifies structure, diagnostics, tokens, and source-position
|
|
2640
|
+
mapping against the CLI/library result (#312). **Met** by `@tikoci/centrs/explain` and
|
|
2641
|
+
`bun run explain:browser-consumer` — see
|
|
2642
|
+
[The offline package entry](#the-offline-package-entry-312); the boundary is a
|
|
2643
|
+
module-graph gate and the consumer proof executes the browser bundle rather
|
|
2644
|
+
than only compiling it. The token vocabulary that consumer reads is settled by
|
|
2645
|
+
#264 — [The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5) —
|
|
2646
|
+
on a stated rule, with the two splits it makes (`arg-sep`, then `escaped`)
|
|
2647
|
+
and the merges it keeps each recorded there with the surface that already
|
|
2648
|
+
carries the distinction. A complete editor, LSP server, palette
|
|
2649
|
+
reproduction, or SCIP implementation is not required to prove this contract.
|
|
2650
|
+
- **Reachable evidence and explicit residue:** changed RouterOS semantic claims
|
|
2651
|
+
have committed CHR-grounded controls and a reproducible capture/replay path.
|
|
2652
|
+
Every remaining issue has a stated scope and dependency; #225 distinguishes
|
|
2653
|
+
offline value work from live types, #272 owns evidence reachability, and #211's
|
|
2654
|
+
unlisted-path decision retains the current abstention until separately decided.
|
|
2655
|
+
|
|
2656
|
+
Retain the existing checks: `bun run lint`, `bun run test`, `bun run build`, and
|
|
2657
|
+
`bun run lint:ci`, plus the affected CHR grounding/integration procedure from
|
|
2658
|
+
the constitution. Add the consumer and agreement checks with their implementation.
|
|
2659
|
+
No new rules engine, schema snapshot, final-state interpreter, or network-behavior
|
|
2660
|
+
test framework is part of this baseline.
|
|
2661
|
+
|
|
2662
|
+
### Current and following capabilities
|
|
2663
|
+
|
|
2664
|
+
- **Current — live probes (#236, phase 2):** highlight + `:parse` and
|
|
2665
|
+
completion/child/syntax over rest-api/native-api. B4's projection/agreement
|
|
2666
|
+
informs this work; each live cell requires its own CHR-passing examples.
|
|
2667
|
+
- **MCP alignment (#223):** the deliberate flat-to-rich response migration,
|
|
2668
|
+
coordinated with rosetta. Independent of the offline token vocabulary.
|
|
2669
|
+
- **Facet polish (phase 3):** live sizing, `--full`, completion/schema ergonomics.
|
|
2670
|
+
- **Full LSP integration (phase 4):** consume live and offline results in
|
|
2671
|
+
`lsp-routeros-ts`, building on the earlier offline consumer check.
|
|
1625
2672
|
|
|
1626
2673
|
## Decisions (2026-07-19 round) and remaining opens
|
|
1627
2674
|
|
|
@@ -1689,13 +2736,17 @@ Still open (implementation opens — none block ratification):
|
|
|
1689
2736
|
|
|
1690
2737
|
1. Final flag names (`--full`, `--curl`, facet names) and the smart-sizing
|
|
1691
2738
|
thresholds (what counts as "cheap", what triggers truncation).
|
|
1692
|
-
2. The
|
|
1693
|
-
|
|
1694
|
-
|
|
1695
|
-
|
|
1696
|
-
|
|
1697
|
-
|
|
1698
|
-
|
|
2739
|
+
2. The mapping table from RouterOS highlight classes to **colors**, and how
|
|
2740
|
+
unknown/new upstream classes degrade. Phase 0 pinned the raw inputs (Q13's
|
|
2741
|
+
per-occurrence highlight-class corpus; the observed 19 classes on 7.23.2 plus
|
|
2742
|
+
drift on 7.24rc2). The centrs-owned **class list is no longer open**: #264 B5
|
|
2743
|
+
settled it on a stated rule and `data.tokens[].class` is no longer provisional
|
|
2744
|
+
([The vocabulary](#the-vocabulary-and-what-earns-a-class-264-b5)), and the
|
|
2745
|
+
projection onto the device's own classes is reported by
|
|
2746
|
+
`bun run explain:highlight-agreement`
|
|
2747
|
+
([Device agreement](#device-agreement-for-the-token-partition-264-b4-263)).
|
|
2748
|
+
What stays open here is the color map and the drift policy. Live mappings
|
|
2749
|
+
and full LSP integration follow separately.
|
|
1699
2750
|
3. Whether the live describe ladder needs result caching per target+version
|
|
1700
2751
|
(probe cost vs freshness) — deferred to implementation evidence.
|
|
1701
2752
|
|