ibex 0.2.0 → 0.3.0

data/docs/error-ux-round2.md

@@ -0,0 +1,111 @@
+# Error UX evidence round 2
+
+H003 expands repository-owned diagnostic and repair evidence beyond R001's ten
+JSON cases. The machine-readable result is
+[`error-ux-round2-v1.json`](error-ux-round2-v1.json), validated by
+[`error-ux-round2-v1.schema.json`](../schema/error-ux-round2-v1.schema.json).
+It does not replace or revise the R001 snapshot, rubric, review records, or
+release gate.
+
+## Status and authority
+
+The deterministic repository capture is **complete**. The external subjective
+gate is separately **HOLD**: there are no external review records, and every
+case has `status: pending` with an empty label list. Repository observations
+cannot be promoted into external `useful`, `misleading`, `unsafe`, or `unclear`
+labels.
+
+The immutable machine capture intentionally has no slot for later subjective
+labels. H003's separate
+[`review status registry`](error-ux-round2-review-status-v1.json) and
+[`review schema`](../schema/error-ux-round2-review-v1.schema.json) provide that
+pathway without rewriting the capture. The registry binds the exact capture
+SHA-256 and required case inventory. It remains `HOLD` with `records: []`.
+
+The fixed corpus covers these distinct dimensions:
+
+| Case | Shape | Observation | Fresh reparse |
+| --- | --- | --- | --- |
+| `H003-DELIMITER-01` | nested delimiter-heavy calls | mismatched closer | accepted |
+| `H003-STATEMENT-01` | assignment statement | missing semantic value | accepted |
+| `H003-STATEFUL-01` | stateful string lexer | EOF in the string state | accepted |
+| `H003-EOF-01` | delimiter EOF | missing closer | progress; the proposed edit over-closes |
+| `H003-MULTI-01` | synchronized statements | two errors at distinct offsets, then continuation | accepted |
+| `H003-UNKNOWN-01` | external token adapter | unknown token after a complete word | accepted |
+| `H003-LEXER-01` | generated lexer | no matching lexer rule | accepted |
+
+Indentation-sensitive input is explicitly `excluded-unimplemented`. Ibex has
+no indentation-tokenization contract, so the corpus does not simulate one.
+
+## Capture semantics
+
+Each case binds the grammar and input digests, exact runtime expected tokens,
+full diagnostic message, source line and caret, bounded repair result, a
+repository-authored byte-range edit, the outcome of applying that edit and
+parsing with a fresh parser, semantic-value risk, and external-review state.
+The capture also binds the exact frontend, normalization, code generator,
+runtime parser, generated lexer, repair implementation, and H003 generator
+source bytes used to produce those observations.
+
+Parser expected-token arrays come directly from the existing runtime API. A
+generated-lexer failure occurs before a parser state exists, so its exact array
+is empty with the closed reason
+`lexer-failure-precedes-parser-state`; it is not presented as a parser
+prediction. Runtime repairs remain distinct from proposed source edits.
+Insertion can supply nil, replacement retains the original runtime value, and
+deletion can discard intent. A successful fresh parse establishes syntax only,
+not that an invented literal, deleted token, or replacement matches the user's
+intended semantics.
+
+The multi-error case overrides the normal error hook only to record the same
+runtime token, expected-token, state, and location fields while allowing the
+grammar's explicit `error ';'` synchronization path to continue. The committed
+capture requires at least two diagnostics at strictly increasing byte offsets
+and a non-nil result after recovery; it cannot be satisfied by duplicating one
+error.
+
+## Trust boundary
+
+The capture compiles and executes generated lexer and parser action code.
+Execution is allowlisted to the four repository-owned fixtures under
+`test/fixtures/error_ux_round2/`. It does not download or execute external
+grammars, reviewer payloads, or user-supplied code. These fixtures are trusted
+test programs, not a sandbox claim.
+
+The existing R001 normative snapshot is bound to its unchanged SHA-256. H003
+generation fails if those bytes change. H003 uses its own corpus, schema,
+evidence ID, generator, and quality gate so comparative R001 evidence and this
+repository study cannot silently become duplicate authorities.
+
+## Limits and kill conditions
+
+This evidence has no external workloads or reviewers, observes one bounded
+repair plan rather than every equal-cost plan, and does not infer intent from
+acceptance. It must fail closed when fixture/corpus bytes drift, a required
+dimension disappears, R001 changes, review state is overstated, synchronized
+diagnostics collapse, or the committed fresh-reparse outcome changes.
+
+The same quality gate validates the external registry fail-closed. `PASS`
+requires at least two complete, independent, external records from
+reviewers whose names remain distinct after Unicode NFKC normalization, full
+case folding, trimming, and whitespace collapse. Every record must assess all seven cases with a
+label, rationale, and semantic-value risk assessment. If reviewer labels
+differ, every reviewer-pair disagreement must remain explicit rather than
+being reduced to a majority conclusion. See the
+[`records workflow`](error-ux-round2-reviews/v1/records/README.md).
+
+Independence and identity remain human attestations: schema validation can
+require the declarations and reject duplicate normalized names, but cannot
+prove who authored a review. A record must never be added merely to clear the
+gate. Evidence drift, incomplete case coverage, duplicate reviewer identity,
+or missing disagreement entries kills `PASS`.
+
+Regenerate only after reviewing intentional parser, lexer, diagnostic, or
+repair changes:
+
+```sh
+bundle exec ruby tool/error_ux_round2.rb --write
+bundle exec rake quality:error_ux_round2
+```
+
+The ordinary quality command never writes evidence.

data/docs/error-ux.md

@@ -1,3 +1,8 @@
+---
+title: Error experience evidence
+description: Bounded diagnostics, repair behavior, and the evidence boundary for parser errors.
+---
+
 # JSON error UX evidence
 
 This report fixes the Phase 14 error-experience baseline against ten malformed
@@ -12,8 +17,56 @@ Use `--write` only after reviewing a deliberate diagnostic, parser-table, racc,
 or repair-policy change. CI regenerates the evidence and requires byte-for-byte
 equality.
 
+## Independent review gate
+
+The fixed observations include a maintainer repair assessment. They do not
+include an independent diagnostic or repair assessment.
+
+<!-- r001-review-status:start -->
+R001: `HOLD` — [`awaiting_independent_review`](error-ux-review-status-v1.json).
+<!-- r001-review-status:end -->
+
+Third-party reviewers use the versioned
+[`rubric`](error-ux-review-rubric-v1.md) and closed
+[`review-record schema`](../schema/error-ux-review-v1.schema.json). Run:
+
+```sh
+bundle exec rake quality:error_ux_review_kit
+bundle exec rake quality:error_ux_review_status
+bundle exec ruby tool/error_ux_review.rb template review.json
+```
+
+The generated payload contains the reviewer identity, reviewed maintainer
+roster, structured consent, labels, rationales, and disagreements, but no
+permalink. Publication metadata is kept separately in the status registry so a
+reviewer can commit the final payload without a self-referential SHA. The first
+command verifies the immutable snapshot, corpus identities, schema, local
+byte/digest bindings, reports, claims, and truthful status entirely offline
+while allowing the expected HOLD in ordinary CI.
+
+`bundle exec rake release:error_ux_review` is the separate promotion gate and
+returns nonzero until a valid published external payload exists. When PASS is
+claimed, it additionally fetches the full-SHA blob without redirects, compares
+the exact imported bytes, and requires the source repository owner, GitHub
+commit API author, status publisher login, and reviewer login to agree
+case-insensitively. This verifies control of the named GitHub namespace and
+GitHub account metadata, not a cryptographic identity or signature. Network or
+account-metadata uncertainty fails closed. Neither path changes the normative
+observation fixture.
+
 ## Comparison method
 
+<!-- comparative-evidence:racc-error-ux-json-v1:start -->
+At Ibex revision cc20c5eb799cc218ebea665df64261f10d030f75, on ten fixed malformed
+JSON inputs with the Ruby, OS, CPU, kernel, processor-count, and YJIT environment unrecorded,
+the committed result artifact records Ibex diagnostics,
+Racc 1.8.1's public on_error callback values, and a maintainer assessment of ten bounded repairs;
+independent subjective review is pending, so this is not a completed comparative UX claim.
+
+Evidence record ID: `racc-error-ux-json-v1`. The observations and assessment
+are registered in [`claims.yml`](claims.yml), including every unrecorded
+environment field and the still-pending independent review.
+
 Ibex uses [`examples/json.y`](../examples/json.y), including its generated
 lexer locations. The migration from its previous handwritten lexer is guarded
 by exact location and error-snapshot tests. The comparison uses the self-authored compatible grammar
@@ -58,3 +111,4 @@ SP-4 is **go** for the existing opt-in bounded single-plan repair described by
 experimental: a selected edit can require a nil semantic value, and the search
 does not enumerate every equal-cost CPCT+ repair. The stable default answer
 remains exact expected tokens plus explicit yacc/synchronization recovery.
+<!-- comparative-evidence:racc-error-ux-json-v1:end -->

data/docs/getting-started.md

@@ -0,0 +1,80 @@
+---
+title: Getting started
+description: Install Ibex, generate a first parser, and choose a grammar mode.
+---
+
+# Getting started
+
+Ibex turns a racc-compatible grammar into a Pure Ruby LR parser. This guide
+gets from an installed gem to a working parser before explaining the optional
+extended surface.
+
+## Install the generator
+
+```sh
+gem install ibex
+```
+
+For an application that only runs generated parsers, install `ibex-runtime`
+instead. A project can add either dependency with `bundle add ibex` or
+`bundle add ibex-runtime`.
+
+## Generate a calculator
+
+Create `calculator.y` with a class, token declaration, rules, and a small
+handwritten pull lexer:
+
+```text
+class Calculator
+token NUM
+preclow
+  left '+'
+  left '*'
+prechigh
+rule
+  expr : expr '+' expr { result = val[0] + val[2] }
+       | expr '*' expr { result = val[0] * val[2] }
+       | NUM { result = val[0] }
+end
+---- inner
+def parse_tokens(tokens)
+  @tokens = tokens
+  do_parse
+end
+
+def next_token
+  @tokens.shift
+end
+---- footer
+if $PROGRAM_NAME == __FILE__
+  tokens = [[:NUM, 2], ['+', nil], [:NUM, 3], ['*', nil], [:NUM, 4]]
+  puts Calculator.new.parse_tokens(tokens)
+end
+```
+
+Generate and run it:
+
+```sh
+ibex -o calculator.rb calculator.y
+ruby calculator.rb
+# 14
+```
+
+Run `ibex --check -o calculator.rb calculator.y` in CI. It compares the
+reproducible output without rewriting the committed file.
+
+## Choose a mode
+
+The default mode is the conservative racc migration surface. Add
+`pragma extended` immediately after the class header, or pass
+`--mode=extended`, when the grammar needs EBNF groups, imports, generated
+lexers, typed trees, or other opt-in features. The [grammar reference](grammar-reference.md)
+lists the exact syntax and the [maturity audit](maturity.md) records support
+level per API.
+
+## Next steps
+
+- [Migrate an existing racc grammar](racc-migration.md).
+- [Configure grammar-owned and invocation-owned settings](configuration-model.md).
+- [Inspect a grammar locally in the Playground](https://ydah.github.io/ibex/playground/).
+- [Read the runtime and grammar reference](grammar-reference.md).

data/docs/grammar-reference.md

@@ -1,5 +1,12 @@
+---
+title: Grammar reference
+description: Write compatible and extended Ibex grammars and understand their generated contracts.
+---
+
 # Grammar reference
 
+<!-- stable:compatible-parser:v2 -->
+
 Ibex's `default` mode accepts the compatible grammar described here. `--mode=extended` or an explicit grammar-file
 `pragma extended` adds the marked syntax; extensions are never inferred from a production.
 
@@ -64,8 +71,8 @@ lossless segment spans rather than rescanning raw text. Documentation works in r
 `AST::Rule`.
 
 For repeated definitions, every normalized user production keeps its definition's documentation. The nonterminal symbol uses
-the first nonnil text; a different later nonnil text is a positioned error, while the same text is accepted. Grammar IR v2
-serializes `doc` on symbols and productions. Version 1 omits those fields.
+the first nonnil text; a different later nonnil text is a positioned error, while the same text is accepted. The current
+Grammar IR serializes `doc` on symbols and productions.
 
 `ibex doc [--format=markdown|html|railroad] [-o FILE] [--mode=MODE] grammar.y` renders the canonical resolved grammar. Output
 defaults to Markdown on stdout. HTML is accessible and self-contained, railroad output includes visible wrapped descriptions,
@@ -106,16 +113,37 @@ includes and emits the first cross-file security, missing-target, cycle, or frag
 `frontend.resolution_error`; cross-file recovery is intentionally bounded to that one record. Permission and other actual
 filesystem read failures remain CLI invocation errors on stderr and do not produce a JSON envelope.
 
+## Generated parser execution boundary
+
+Static frontend, formatting, LSP, documentation, IR, and analysis operations
+treat parser actions, generated lexer actions, and user-code sections as opaque
+data and do not execute them. Every semantic parse executes parser production
+actions. Generated lexer actions additionally execute for `parse(source)`,
+generated-lexer `lex(source).do_parse`, and generated-lexer
+`parse_with_syntax(source)`. A handwritten `next_token` used by `do_parse` or
+no-argument `parse_with_syntax` does not invoke generated lexer actions, nor do
+the caller-fed `yyparse` and `push` / `finish` APIs.
+
+Loading the generated file may execute `header`, `inner`, and `footer` code on
+all of these paths. Generated runtime execution is trusted application code,
+not a sandbox.
+
+Generated `parse_syntax` and `incremental_session` are syntax-only: they
+suppress parser production actions but still execute generated lexer actions.
+They may also inherit side effects from loading user sections. A future
+nonexecuting syntax profile would require a declarative built-in-only lexer and
+no user-code sections; the current generated parser API is not that profile.
+
 ## Declarations
 
 - `pragma extended` enables extended syntax for this grammar even when the CLI uses its default or explicit `--mode=default`.
   It must immediately follow the class header, before every ordinary declaration. Unknown, duplicate, and misplaced pragmas
-  are positioned errors. The frontend records the effective mode on the root AST, and Grammar IR v2 records extended mode
+  are positioned errors. The frontend records the effective mode on the root AST, and the current Grammar IR records extended mode
   additively so downstream generators preserve its runtime behavior.
 - `pragma cst` enables extended syntax and builds a pure-syntax Red/Green tree
   in parallel with the ordinary semantic value stack. Distinct pragmas may be
   combined in the class header; repeating either one is an error. Grammar IR
-  v2 stores the optional `cst: true` setting.
+  The current Grammar IR stores the optional `cst: true` setting.
 - `import "relative/path.y"` inserts one explicit fragment through the canonical resolver. `include` is an accepted
   compatibility spelling. Imports are available only in extended mode. Parsing source text alone performs no filesystem
   access; path-based callers use `Frontend::Resolver` to resolve the import graph.
@@ -133,6 +161,15 @@ filesystem read failures remain CLI invocation errors on stderr and do not produ
   retained in Automaton IR but are not counted.
 - Extended `%expect-rr N` records the expected reduce/reduce count. Under `--warnings=error`, generation succeeds only when
   both declared counts match.
+- Extended roots accept one `parser ... end` block before `rule`. `algorithm`
+  accepts exactly `slr`, `lalr`, `ielr`, or `lr1`; `entries` accepts exactly
+  `shared` or `isolated`. Keys may appear in either order, but duplicate blocks,
+  duplicate keys, unknown keys or values, and `algorithm auto` are positioned
+  errors. `entries isolated` requires at least two declared start symbols.
+  Fragments cannot own parser-wide construction. During canonical generation a
+  matching `--algorithm` or `--entry-isolation` is accepted and a conflicting
+  request is rejected. Analysis and `ibex test` may explicitly choose another
+  algorithm and report the declared and selected values as noncanonical.
 - `start name` overrides the first rule as the start symbol. Extended mode accepts an ordered list such as
   `start program expression`. The first name remains the primary entry for `do_parse`; generated parsers also expose
   `parse_program` and `parse_expression`. Shared construction attributes each conflict to its reachable entries and marks a
@@ -155,8 +192,8 @@ filesystem read failures remain CLI invocation errors on stderr and do not produ
   each later declaration has higher priority. A uniquely highest-priority completed production fills only table cells that
   would otherwise be errors, so explicit shifts, reductions, accepts, and conflict decisions remain authoritative.
 - Extended roots accept ordered `%test accept "source"` and `%test reject "source"` declarations. Sources must use
-  double-quoted Ruby literals and exact duplicate expectation/source pairs are rejected. Grammar IR v2 retains their decoded
-  source and location; ordinary generated parser tables do not.
+  double-quoted Ruby literals and exact duplicate expectation/source pairs are rejected. The current Grammar IR retains their
+  decoded source and location; ordinary generated parser tables do not.
 
 ## Generated lexer (extended mode)
 
@@ -333,8 +370,8 @@ helper and production order while avoiding dependence on the Ruby stack. The def
 1,000 distinct specializations; programmatic callers can configure the positive-Integer
 `max_parameter_specializations:`. Argument-changing recursive instantiation is rejected by structural cycle detection rather
 than an arbitrary depth boundary. Specialized productions retain template actions,
-precedence, types, documentation, locations, and include chains. Grammar IR v2 records
-`expansion.parameter {rule, arguments}`; v1 output omits the expansion record.
+precedence, types, documentation, locations, and include chains. The current Grammar IR records
+`expansion.parameter {rule, arguments}`.
 
 ### Inline rules (extended mode)
 
@@ -362,10 +399,9 @@ grammar nesting is independent of the Ruby call stack.
 
 Eliminated reductions still run their explicit or implicit actions in logical post-order. Named references, `val`, `_values`,
 `@N`, `@$`, empty spans, middle actions, `result`/`no_result_var`, and parser instance methods retain their logical rule view.
-Grammar IR v2 serializes the executable sequence in `action.composition.plan` and records
+The current Grammar IR serializes the executable sequence in `action.composition.plan` and records
 `expansion.inline {rule}`; dump/load followed by code generation preserves it. `yyaccept` and `yyerror` stop the remaining
-logical fragments and caller after the current fragment completes, and `yyerrok` does not erase that `yyerror`. Version 1 omits
-both metadata families.
+logical fragments and caller after the current fragment completes, and `yyerrok` does not erase that `yyerror`.
 
 The action scanner handles nested braces, quoted/backtick strings and interpolation, `%q/%Q/%w/%W/%i/%I/%x/%r/%s`, regular
 expressions, comments, character literals, and unquoted, single-quoted, double-quoted, or backtick heredocs. Indented, squiggly,
@@ -399,8 +435,8 @@ For external tooling, `observe { |event| ... }` registers an ordered observer an
 `unobserve`. Events are immutable, sequence-numbered per parse session, and cover `start`, `shift`, `reduce`, `error`, `recover`,
 `discard`, `accept`, and `reject`. Semantic values and locations are bounded JSON summaries rather than live objects.
 `Ibex::Runtime::EventJSONLTracer.attach(parser, io:)` writes the versioned schema at
-`schema/runtime-event-v1.schema.json`; write and serialization failures propagate. This API is separate from the legacy
-hook-shaped `Runtime::JSONLTracer`. See [ADR 0010](decisions/0010-committed-runtime-observation.md).
+`schema/runtime-event-v1.schema.json`; write and serialization failures propagate. See
+[ADR 0010](decisions/0010-committed-runtime-observation.md).
 
 Every parser instance owns immutable `Runtime::ResourceLimits`. The defaults allow a 10,000-entry LR state stack and 100
 recovery entries per parse. Pass `resource_limits:` to the generated parser constructor or replace it while the instance is
@@ -494,6 +530,10 @@ the manifest without rewriting them. `Ibex::GenerationManifest.validate_file(pat
 artifact bytes. For coherent concurrent reads, read the manifest, verify every entry, and retry from a newly read manifest if
 anything is missing or mismatched.
 
+The version-1 `options` object is the existing open map for additive generation evidence. It now always records the effective
+`cst_trivia` value: an omitted option uses the built-in `leading` default, and the compatibility spelling `attach` is recorded
+canonically as `leading`. Adding this entry changes neither the manifest's required root shape nor its schema version.
+
 `--watch` repeatedly applies the same transaction to Ruby file generation. It observes the root, the latest successful include
 closure, unresolved include attempts, an optional messages file, and repairable output paths. Failed candidates leave the last
 successful generation intact; an unchanged failure is reported once. Source changes during render or publication retry after
@@ -533,6 +573,137 @@ error is rejected with an instruction to run the updater.
 
 ## Analysis and visualizations
 
+`ibex samples grammar.y` emits JSON token arrays derived from Grammar IR.
+`--strategy=random|coverage` selects random alternatives or least-covered
+production paths; `--path-length=1|2`, token, depth, expansion, count, and seed
+options make the search finite and reproducible.
+
+`ibex fuzz grammar.y` without `--against` feeds bounded generated sentences and
+single-token insert/delete/replace mutations to SLR, LALR, IELR, and LR(1)
+table simulators. It does not execute production actions. `--coverage-guided`
+selects uncovered production paths. The JSON report records every effective
+bound and returns 0 when no difference is found within those bounds, 1 with a
+concrete differential witness, and 2 when a budget prevents completion.
+`--format=text` renders the same result and bounds for a terminal; JSON remains
+the default versioned contract.
+`--against=COMMAND` is an explicit unsafe opt-in that sends each token array as
+JSON to the supplied subprocess. It executes arbitrary application code with
+the invoking user's host permissions and is not a sandbox. The command is
+split into an executable and arguments without an implicit shell, but the
+selected executable can itself access the filesystem, network, environment,
+and other processes. Exit 0 means accepted and exit 1 means rejected. It
+requires `--against-runtime=DESCRIPTION`, and the report records that target
+runtime, the exact command, and the Ibex host runtime. Each invocation is
+bounded by `--against-timeout=SECONDS` (default 10) and
+`--against-max-output=N` (default 1,048,576 bytes); either limit exits 2
+instead of treating a stuck or noisy target as a language result. These limits
+and process-group cleanup bound resources but do not confine side effects. On
+process-group platforms, descendants are terminated with the target. A
+difference is automatically delta-minimized while preserving its mismatch kind
+and outcomes, then written atomically to `test/fuzz/regressions` with its seed
+and effective bounds.
+`--max-reduction-trials=N` bounds that work, `--regression-dir=DIR` selects
+another destination, and `--no-save-regression` disables persistence. An
+incomplete minimization remains a concrete difference and records
+`complete: false`; it is never described as minimal.
+
+`ibex reduce --command=COMMAND input` is an explicit unsafe opt-in that
+repeatedly executes the supplied checker to perform trial-bounded delta
+debugging. The checker may run arbitrary application code with the invoking
+user's host permissions; it is not sandboxed. As with external fuzzing,
+resource limits and process cleanup do not confine filesystem, network,
+process, or other side effects.
+Input mode is `tokens` (a JSON string array), `lines`, or `bytes`. A nonzero
+normal subprocess exit means the failure persists; a signal is an invocation
+error rather than evidence. The version-2 report contains the reduced
+sequence, trial count, original/final sizes, every effective limit, and whether
+the configured trial budget allowed completion. A trial-limited result is
+reported as `incomplete`, never as `minimized`. The checker timeout
+(default 10 seconds), checker output (default 1,048,576 bytes), and input
+(default 10 MiB) are bounded by `--timeout`, `--max-output-bytes`, and
+`--max-input-bytes`. Exceeding any configured budget exits 2 with a distinct
+reason. Report v1 remains the prior read-only contract.
+`--format=text` selects a human-readable report; JSON remains the default.
+
+`ibex verify automaton.json` validates table semantics by independently
+deriving the relevant LR item collection from the embedded Grammar IR. Default
+checks cover soundness, lookaheads, default reductions and explicit error
+masks, reachability/productivity, epsilon termination, and conflict
+resolution consistency. Both modes rebuild and compare plain/compact rows;
+`--strict` adds complete collection checks and reports row mismatches under
+the strict V5 identifier. `--max-states` and `--max-items` bound only the
+reference derivation. JSON is the default report format; exit 0 means valid, 1
+means a concrete violation, and 2 means the reference budget was exhausted.
+Opaque semantic actions are never executed. See the
+[verifier trust boundary](verifier-trust-boundary.md) for the exact TCB,
+algorithm-specific strength, resource non-goals, and generated-artifact
+boundary.
+
+`ibex equiv LEFT RIGHT` accepts grammar source, Grammar IR, or Automaton IR.
+It combines normalized structural comparison, deterministic samples generated
+in both directions, and a breadth-first product search over the two LR state
+stacks. A difference returns 1 with a shortest token witness found within
+`--max-tokens` and `--max-configurations`; an incomplete search returns 2.
+`--samples`, `--seed`, `--max-actions`, and `--max-stack` expose the remaining
+bounds. `--map=old_rule=new_rule` requests reduction-tree comparison for a
+declared one-to-one rule correspondence. Without a map, only language
+acceptance is compared. A successful bounded search is explicitly not a proof
+of equivalence.
+
+`ibex diff OLD NEW` accepts the same three input forms and classifies symbols,
+rules, conflicts, and normalization warnings into `added`, `removed`, and
+`changed`. It also reports before/after/delta counts for states, productions,
+warnings, and unresolved conflicts. Conflict identities use token and
+production shapes rather than unstable state numbers.
+
+`ibex metrics GRAMMAR` reports normalized rule and alternative counts, average
+and maximum branching, epsilon productions, recursive nonterminals, the
+longest dependency path after collapsing mutually recursive components, and
+deterministic Automaton IR cell/conflict counts. The metric is deliberately
+structural; it contains no timing or memory threshold. Both commands default
+to their closed version-1 JSON schemas and support `--format=text`.
+
+`ibex fix GRAMMAR` selects an unresolved conflict and searches a finite set of
+precedence declarations and overrides, algorithm changes, and `%inline`
+rewrites. Rewrites that would move opaque semantic actions, including
+recursion reversal and source factoring, are deliberately excluded. A repair
+proposal is emitted only if the target disappears, no other conflict fingerprint
+increases, the resulting table passes the independent verifier, and bounded
+language plus mapped reduction-tree comparison finds no difference. `%expect`
+and recovery-quality suggestions are reported separately as non-repair advice;
+they never enter the verified proposal or `--apply` path because they do not
+eliminate the selected conflict. The JSON
+report includes the exact bounds, unified diff, eliminated fingerprint, state
+delta, and rejection reasons. `--messages=FILE` measures newly moved,
+uncovered, and unreachable message-catalog entries against the original table.
+`--apply[=FXNNN]` transactionally applies a source proposal and refuses
+symlinks or files with multiple hard links. Candidate, independent-verifier,
+or equivalence budget exhaustion exits 2. `--verify-max-states` and
+`--verify-max-items` bound the independent collection used for every
+candidate. The advice/repair separation and independent-verifier bounds are
+reported by the closed `schema/fix-v3.schema.json`; versions 1 and 2 remain
+available as prior report contracts. Every successful report states that
+bounded search is not a proof of equivalence.
+
+`ibex import bison [--format=source|json] [-o FILE] grammar.y` performs a
+one-way, analysis-only conversion. C actions are mechanically reference-mapped
+but remain opaque, and Ruby generation is refused. Unsupported directives are
+reported at every source position; structural gaps are distinguished from
+generator-only controls, and `fix` refuses structurally incomplete imports.
+Input bytes, structural tokens, rule groups, and actions have independent
+positive budgets. Read-only grammar analysis commands auto-detect two Bison
+`%%` section markers. The complete directive, naming, external-corpus, and
+CRuby `parse.y` contracts are in the
+[Bison import guide](bison-import.md).
+
+All CLI forms accept `--lang=LANG`; `IBEX_LANG` supplies the default. Built-in
+diagnostic catalogs ship for `en` and `ja`. Locale suffixes such as
+`ja_JP.UTF-8` select their base language, while unavailable translations fall
+back to English silently. Stable machine-readable diagnostic codes and report
+schemas do not change with the display language. The built-in
+`diagnostic.*`, `warning.*`, `conflict.*`, and `note.*` IDs are an internal
+translation namespace and never overlap the user-owned `E00xx` message IDs.
+
 `--emit=sets` writes deterministic JSON containing nullable nonterminals and their FIRST and FOLLOW sets. `--dot=FILE` and
 `--mermaid=FILE` write automaton graphs. `--html=FILE` writes a self-contained report with state search, conflict highlighting,
 and a filter that keeps a selected conflict state and its one-hop neighbors. All three visualizations can be produced while

data/docs/ielr-design.md

@@ -0,0 +1,43 @@
+# Direct IELR(1) implementation design
+
+This document is the maintained implementation summary for the paper-derived
+IELR(1) pipeline. The Japanese work order in `.idea/ielr1-implementation-design-ja.md`
+contains the full definition and fixture tables; this public copy records the
+decisions that are part of the shipped implementation.
+
+## Contract
+
+Direct IELR starts from an LR(0) collection, computes DeRemer–Pennello
+successor/internal/includes and goto-follow closures, annotates grammar-relative
+inadequacies, splits incompatible isocores, and reruns item lookahead
+propagation. Conflict resolution, error reductions, default reductions, and
+Automaton IR serialization are shared with the existing builder. Canonical LR(1)
+is not constructed on the normal direct path.
+
+The implementation is divided into `LR0Collection`, `GotoFollows`,
+`IELR::Annotator`, `IELR::StateSplitter`, and `LookaheadPropagation`.
+Integer bitsets are used for terminal and kernel-item sets. `--remove-unreachable`
+is an explicit, default-off post-resolution compaction because state numbers are
+part of diagnostics and generated artifacts.
+
+## Verification and limits
+
+`GotoFollows#reduction_lookaheads` is cross-checked against the existing direct
+item propagator. Paper fixtures cover the split and no-split cases, and the
+acceptance matrix checks direct IELR against LALR for the four conflict examples.
+The independent verifier still uses its canonical reference collection for V1/V2;
+strict IELR verification additionally runs the bounded V9 acceptance witness.
+`Verify::ActionCorrespondence` and `LALR::InadequacyReport` remain bounded
+diagnostic tools, not a scale-independent proof. Global minimum state merging and
+remergeable state optimization are intentionally out of scope.
+
+## Operational decision
+
+`ielr_strategy: :partition` remains the default. `:direct` is an experimental,
+explicit strategy exposed by `--ielr-strategy=direct`; no state-count or release
+readiness claim is made. The decision and provenance boundary are recorded in
+[ADR 0024](decisions/0024-direct-ielr-construction.md) and the direct-IELR dossier.
+
+The implementation is independently derived from the published IELR and
+DeRemer–Pennello papers and repository parser specifications. GPL implementation
+source is not a design input.