ibex 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 46d7505f93788dea378cffff477d2d881b786445c3fd1fc805de78267784d11a
-  data.tar.gz: 853d1ad44d6224c7feba09fb00e012aa883ad3a86bbcbbaedc878b3f151f7bd6
+  metadata.gz: 305b0c04e2cc0e6340b5584b9c2f159c48c7525d34d5212eb7969393e1205244
+  data.tar.gz: 2a2993d999dedb2decf5bcda1356a4b36aba03edc30225c9d7548ace0587c925
 SHA512:
-  metadata.gz: 6b4f1e69a42798bc4b8b81be22ce21ce18a3a63e4fab0684e22e9d092055eba37d7a3d7341c3aec54bd04177553e8d8570419564130e1c8369e27ef742dffee9
-  data.tar.gz: 26d34199203cea82c17da593b06c067e24c661496ea13a1f0ec9bb2a4884730fb439ce5427c15c4bc193400ab365e237677462e540b539c1a31e74fa02b6e9c0
+  metadata.gz: 7ed7961859d052b5b205d0a3114c22e527fa627d2ec5c0dcd936e3480f0ba3dc453fccb5306b91bea0caece79f72f71e2670258a481a5a31dd4bdb9c981613fa
+  data.tar.gz: 420b15173d5da4ea674df973b48c7aaadb0d3f6f9c010654556e6ca4138aed08efd54e55198009e86413a76639d59401b7252725c371881db5289c11d1025137

data/README.md

@@ -1,71 +1,55 @@
 # Ibex
 
-[![Gem Version](https://badge.fury.io/rb/ibex.svg)](https://badge.fury.io/rb/ibex)
+[![Gem](https://badge.fury.io/rb/ibex.svg)](https://rubygems.org/gems/ibex)
 [![CI](https://github.com/ydah/ibex/actions/workflows/main.yml/badge.svg)](https://github.com/ydah/ibex/actions/workflows/main.yml)
+[![Docs](https://img.shields.io/badge/docs-GitHub%20Pages-4ade80)](https://ydah.github.io/ibex/)
+[![License](https://img.shields.io/badge/license-MIT-4ade80.svg)](LICENSE.txt)
 
 Ibex is a Pure Ruby LR parser generator. It reads racc-compatible grammar
 files, generates parsers with the familiar `do_parse` / `yyparse` API, and
-requires no C or Java extension.
-
-> [!IMPORTANT]
-> Ibex is pre-1.0. The current v1.0 decision is
-> [**HOLD**](docs/release-readiness.md): compatibility and scale evidence pass,
-> but cold generation and new-instance runtime still trail Racc's Ruby backend,
-> and the error UX evidence still needs independent review. Compatible mode is
-> the stable baseline; opt-in features have the maturity levels documented in
-> the [stability policy](docs/stability.md).
-
-- Try grammar analysis in the
-  [browser playground](https://ydah.github.io/ibex/playground/).
-- Start with the [grammar reference](docs/grammar-reference.md).
-- Build lossless and incremental tooling with the
-  [Red/Green CST guide](docs/cst.md).
-- Migrate an existing parser with the
-  [racc migration guide](docs/racc-migration.md).
-- Browse the [API reference](https://ydah.github.io/ibex/api/).
-
-The playground runs in a local Web Worker. It does not upload grammar source
-or execute semantic action bodies.
-
-## At a glance
-
-| Concern | Ibex default |
-| --- | --- |
-| Grammar input | racc-compatible `.y` source |
-| Construction | Direct LALR(1) |
-| Generated tables | Compact, immutable Ruby data |
-| Parser API | Pull parsing with `do_parse` / `yyparse` |
-| Generated dependency | The separately versioned `ibex-runtime` gem |
-| Ruby support | Ruby 3.0 or later |
-| Extensions | Disabled unless `pragma extended` or `--mode=extended` is used |
-| Concrete syntax | Stable format-v6 Red/Green CST with `pragma cst` |
-| Semantic actions | Opaque during generation and analysis; executed only by a running parser |
-
-The generator and runtime are deliberately separate:
+requires no native extension.
 
-```text
-grammar.y + lexer contract
-          |
-          v
-        ibex  -----> parser.rb --------> ibex-runtime
-          |
-          +--------> optional RBS, IR, reports, diagrams, and docs
-```
+The shortest path is the [Getting Started guide](https://ydah.github.io/ibex/getting-started/):
+install the gem, try the calculator grammar, and then choose the compatible or
+extended contract you need. You can also [inspect a grammar in the browser](https://ydah.github.io/ibex/playground/)
+without uploading its source.
+
+## Release status
 
-Use `-E` when a generated parser must embed the runtime instead of depending
-on `ibex-runtime`.
+Ibex is pre-1.0. The v1.0 publication decision is on hold pending an
+independent review of the published error-experience evidence; feature
+development continues. The default compatible mode and its current IR
+contracts are the conservative adoption baseline. See the [release status](https://ydah.github.io/ibex/project/status/)
+and [stability policy](docs/stability.md) for the human-readable boundaries.
 
-## Quick start from a checkout
+<!-- r001-review-status:start -->
+R001: `HOLD` — [`awaiting_independent_review`](docs/error-ux-review-status-v1.json).
+<!-- r001-review-status:end -->
 
-Clone the repository and install its locked development dependencies:
+<!-- type-stats:start -->
+The current whole-library `steep stats` result is 28,836 typed calls and 2,734 untyped calls out of 31,610 (91.2% typed).
+The generated signature tree contains 492 explicit `untyped` occurrences across 39 files.
+<!-- type-stats:end -->
+
+## Install
+
+Add the generator to an application or tool project:
 
 ```sh
-git clone https://github.com/ydah/ibex.git
-cd ibex
-bundle install
+gem install ibex
+# or, in a Gemfile:
+bundle add ibex
+```
+
+Applications that only run generated parsers can depend on the smaller runtime:
+
+```sh
+bundle add ibex-runtime
 ```
 
-Save this grammar as `calculator.y`:
+## First parser
+
+Save this as `calculator.y`:
 
 <!-- calculator-grammar:start -->
 ```text
@@ -97,241 +81,80 @@ end
 ```
 <!-- calculator-grammar:end -->
 
-Generate and run the parser without installing local gems:
-
-```sh
-bundle exec ruby -Ilib exe/ibex calculator.y
-bundle exec ruby -Ilib calculator.rb
-# 14
-```
-
-With the local gems installed, the equivalent commands are:
+Generate and run it:
 
 ```sh
-ibex calculator.y
+ibex -o calculator.rb calculator.y
 ruby calculator.rb
+# 14
 ```
 
-Ibex writes `calculator.rb`, uses compact tables, and generates a parser that
-requires `ibex-runtime`. Use `-o PATH` to choose the output, `--table=plain`
-for inspectable Hash rows, `--check` to compare without rewriting, or `-E` to
-embed the runtime.
-
-## Generated parser contract
-
-### Tokens and lexers
-
-A handwritten pull lexer implements `next_token` and returns:
-
-```ruby
-[token, value]
-[token, value, location]
-```
-
-`false` or `nil` marks EOF. Bare grammar tokens use Ruby symbols such as
-`:NUM`; quoted terminals use strings such as `'+'`. Locations are
-application-owned objects and remain optional.
-
-Existing lexers can use the [lexer migration guide](docs/lexer-migration.md).
-Extended grammars may instead generate a streaming lexer from declarative
-regular-expression rules.
-
-### Semantic actions and locations
-
-Actions read RHS values through `val`. `@1`, `@2`, and so on address parallel
-semantic locations, while `@$` is the span of the current reduction. Action
-source remains opaque Ruby: Ibex preserves its source mapping but does not
-parse, type-check, or sandbox it.
-
-### Parse lifecycles and errors
-
-Generated parsers support:
-
-- `do_parse` and yielding `yyparse` for racc-compatible pull parsing;
-- `push(token, value, location)` and `finish(location:)` for caller-driven
-  streaming;
-- structured `Ibex::ParseError` values containing the token, expected tokens,
-  state, location, and spelling suggestions;
-- yacc-style recovery through `on_error`, `yyerror`, `yyerrok`, and
-  `yyaccept`; and
-- immutable, shareable generated tables.
-
-Use one parser instance per concurrent parse. Isolated runtime sessions and
-resource budgets are opt-in when application code needs explicit execution
-boundaries.
+The generated parser uses compact immutable tables and depends on
+`ibex-runtime`. Use `-E` when a single self-contained generated file is more
+convenient. Use `--check` in CI to detect stale generated output.
 
-## Choose a grammar mode
+## Choose a contract
 
-| Mode | Use it for | Contract |
+| Surface | Use it for | Maturity |
 | --- | --- | --- |
-| `default` | Existing racc grammars and conservative migrations | Compatible syntax and runtime behavior |
-| `extended` | New grammars that need composition or generated tooling | Preview, explicitly enabled |
-
-Enable extended syntax with `pragma extended` in the grammar or
-`--mode=extended` on the command line:
-
-```text
-class ExtendedParser
-pragma extended
-token NUM KEY VALUE
-rule
-  arguments : separated_list(NUM, ',')
-  sum       : NUM:left '+' NUM:right { result = left + right }
-  maybe     : NUM?
-  many      : NUM*
-  groups    : (KEY VALUE)+
-  list(X)   : X | list(X) ',' X
-  %inline signed(X): '-' X { result = -val[1] }
-  negative  : signed(NUM)
-end
-```
-
-Extended mode includes nested EBNF groups, named references, parameterized and
-inline rules, multiple entry points, canonical fragment imports, generated
-lexers, CST or typed `Data` AST generation, declarative recovery, semantic
-types, and grammar-declared tests. These features do not silently change the
-default compatible frontend. Maturity is assigned per API: format-v6 batch
-Red/Green CST is Stable, while the wider extended grammar surface remains
-Preview and incremental CST sessions remain Experimental.
-
-## Choose a construction algorithm
-
-| Option | When to use it |
-| --- | --- |
-| `--algorithm=lalr` | Default direct LALR(1) construction |
-| `--algorithm=slr` | Grammars whose FOLLOW-set reductions are sufficient |
-| `--algorithm=ielr` | Preview backend for conflicts introduced by LALR state merging |
-| `--algorithm=lr1` | Canonical LR(1) analysis when the larger automaton is acceptable |
-
-`--suggest-ielr` checks whether IELR removes an unexpected LALR conflict and
-prints an advisory note when it does.
-That note does not change the selected algorithm or exit status.
-
-Conflict freedom is not the same as a proof of unambiguity. `ibex check
---ambiguity` searches for a concrete sentence with two interpretations, but
-the token and configuration budgets make every result explicitly bounded.
-
-## Integrate generation into a build
-
-Treat grammar source as the input and generated Ruby as a reproducible build
-artifact:
-
-```sh
-ibex --warnings=error -o lib/calculator.rb grammar/calculator.y
-ibex --warnings=error --check -o lib/calculator.rb grammar/calculator.y
-```
-
-The first command writes the parser. The second is suitable for CI and fails
-when the committed output is stale without modifying it.
-
-Rake projects can declare the dependency directly:
-
-```ruby
-require "ibex/rake_task"
-
-Ibex::RakeTask.new("lib/calculator.rb") do |task|
-  task.grammar = "grammar/calculator.y"
-  task.options = ["--warnings=error"]
-end
-```
-
-The task follows the grammar's fragment dependency closure. Multi-output
-generation can additionally publish a manifest containing input identities,
-output paths, sizes, and SHA-256 digests. Generation renders all requested
-outputs before publishing them; `--watch` retains the last successful output
-when a later edit is invalid.
-
-Applications that only execute generated parsers need `ibex-runtime`, not the
-generator. To build and install both current local gems:
-
-```sh
-gem build ibex-runtime.gemspec
-gem build ibex.gemspec
-gem install ./ibex-runtime-0.1.0.gem
-gem install ./ibex-0.1.0.gem
-```
-
-## Diagnose, verify, and inspect
-
-| Goal | Command |
-| --- | --- |
-| Promote grammar warnings to errors | `ibex --warnings=error -C grammar.y` |
-| Collect structured frontend diagnostics | `ibex diagnose --format=json grammar.y` |
-| Explain one conflict | `ibex explain --state=7 --token=ELSE grammar.y` |
-| Search for a concrete ambiguity | `ibex check --ambiguity --algorithm=lr1 grammar.y` |
-| Run grammar-declared tests | `ibex test --coverage=100 grammar.y` |
-| Check canonical formatting | `ibex fmt --check grammar.y` |
-| Render grammar documentation | `ibex doc --format=html -o grammar.html grammar.y` |
-| Emit Automaton IR | `ibex --emit=automaton-ir grammar.y > automaton.json` |
-| Simulate table behavior without actions | `ibex debug automaton.json NUMBER PLUS NUMBER` |
-| Check a racc migration | `ibex migrate-check grammar.y` |
-| Start the language server | `ibex lsp --stdio` |
-
-Grammar IR, Automaton IR, generated table formats, diagnostic JSON, runtime
-events, coverage, and manifests are versioned contracts. Reports can also be
-rendered as text, DOT, Mermaid, HTML, railroad SVG, samples, or conflict
-counterexamples.
-
-## Safety boundaries and non-goals
-
-- Frontend parsing, diagnosis, formatting, documentation, LSP, static
-  migration checks, IR validation, and table simulation do not execute
-  semantic actions or user-code sections.
-- Generated parsers, grammar-declared tests, and explicit migration harnesses
-  do execute application Ruby. They are not sandboxes.
-- Fragment imports are confined to the declared source root and reject
-  traversal, cycles, and symlink escapes.
-- Counterexample, ambiguity, sample, repair, and runtime-budget searches are
-  bounded. Exhausting a budget is distinct from finding no witness within it.
-- Ibex generates deterministic LR parsers. It does not provide GLR or
-  generalized ambiguity handling. Incremental CST sessions are syntax-only
-  and experimental.
-- racc compatibility is a migration surface, not a claim that every generated
-  parser is a byte-for-byte or adapter-free replacement.
-- Preview and experimental features may have weaker compatibility guarantees
-  than the default mode.
-
-## Documentation by task
-
-| When you need to... | Start here |
-| --- | --- |
-| Write a grammar or use the runtime | [Grammar reference](docs/grammar-reference.md) |
-| Build, edit, serialize, or incrementally update syntax trees | [Red/Green CST guide](docs/cst.md) and [migration guide](docs/cst-migration.md) |
-| Learn from executable grammars | [Examples](examples/README.md) |
-| Migrate from racc | [racc migration guide](docs/racc-migration.md) |
-| Adapt a handwritten lexer | [Lexer migration guide](docs/lexer-migration.md) |
-| Configure an editor or LSP client | [Editor setup](docs/editor-setup.md) |
-| Understand the pipeline, IR, algorithms, or tables | [Architecture](docs/architecture.md) |
-| Check API maturity and deprecation rules | [Stability policy](docs/stability.md) |
-| Evaluate readiness, performance, or error UX | [Release readiness](docs/release-readiness.md), [benchmarks](benchmark/README.md), and [error UX](docs/error-ux.md) |
-| Contribute or run every quality gate | [Development guide](docs/development.md) |
-| Review implementation decisions | [Architecture decision records](docs/decisions/README.md) |
-
-The published documentation separates the
-[compatibility contract](https://ydah.github.io/ibex/compatibility/),
-[opt-in extensions](https://ydah.github.io/ibex/extensions/), and
-[experimental surfaces](https://ydah.github.io/ibex/experimental/). The
-[grammar gallery](https://ydah.github.io/ibex/gallery/) contains executable
-examples whose declared tests reach complete production coverage.
+| Compatible mode | Existing racc grammars and conservative migrations | Stable baseline |
+| Extended mode | EBNF, imports, generated lexers, trees, types, and tooling | Preview; explicit opt-in |
+| Browser playground and incremental CST sessions | Local analysis and evaluation | Preview/Experimental; bounded |
+
+The default frontend remains unchanged unless a grammar opts into
+`pragma extended` or the command line uses `--mode=extended`. Read the
+[grammar reference](https://ydah.github.io/ibex/docs/grammar-reference/),
+[racc migration guide](https://ydah.github.io/ibex/docs/racc-migration/), and
+[maturity audit](https://ydah.github.io/ibex/docs/maturity/) before adopting a
+Preview or Experimental surface.
+
+IELR is also explicit: `--algorithm=ielr` uses the conservative partition
+strategy by default, while `--ielr-strategy=direct` enables the experimental
+LR(0)-based construction. Direct IELR is profiled and reviewable, but it is not
+the default release strategy; see the [IELR guide](docs/ielr.md).
+
+## Trust boundary
+
+Static frontend, formatting, documentation, LSP, IR, verification, and
+browser analysis treat grammar actions and user sections as opaque source.
+Generated parsers and semantic parses execute application Ruby and are not
+sandboxes. The playground runs in a local Web Worker, does not upload grammar
+source, and never executes parser actions or user-code sections. Its limits and
+non-goals are documented in the [playground guide](https://ydah.github.io/ibex/playground/).
+
+## Where to go next
+
+- [Documentation hub](https://ydah.github.io/ibex/docs/) — task-oriented entry points.
+- [Grammar gallery](https://ydah.github.io/ibex/gallery/) — executable examples with declared tests.
+- [API reference](https://ydah.github.io/ibex/api/) — generated Ruby API documentation.
+- [Configuration model](https://ydah.github.io/ibex/docs/configuration-model/) — grammar-owned and invocation-owned settings.
+- [Project status and evidence](https://ydah.github.io/ibex/project/) — release, roadmap, and bounded claims.
+- [Contributing](CONTRIBUTING.md) and [security reporting](SECURITY.md) — community workflow.
 
 ## Development
 
-The [development guide](docs/development.md) contains frontend regeneration,
-RBS/Steep validation, grammar coverage, browser checks, mutation analysis, and
-workflow lint commands. The default local gate is:
+Clone the repository only when you need to contribute or run the complete
+quality suite:
 
 ```sh
+git clone https://github.com/ydah/ibex.git
+cd ibex
 bundle install
 bundle exec rake
+npm ci
+npm run test:site
 ```
 
-<!-- type-stats:start -->
-The current whole-library `steep stats` result is 20,629 typed calls and 2,275 untyped calls out of 22,904 (90.1% typed).
-The generated signature tree contains 2,084 explicit `untyped` occurrences across 103 files.
-<!-- type-stats:end -->
+The [development guide](https://ydah.github.io/ibex/docs/development/) lists
+the focused frontend, type, evidence, browser, and workflow checks.
+
+## Decisions and evidence
 
-Performance measurements are evidence, not portable scores or CI timing
-thresholds. Follow the [benchmark guide](benchmark/README.md) to reproduce and
-compare results under the same environment and configuration.
+The public reference also indexes the [direct IELR decision](docs/direct-ielr-decision.md),
+[direct multi-entry decision](docs/direct-multi-entry-decision.md),
+[verifier trust boundary](docs/verifier-trust-boundary.md), and the
+[error-experience review status](docs/error-ux-review-status-v1.json). These
+records scope claims to their evidence and revision; they do not change the
+compatible installation path above.
 
 Ibex is available under the [MIT License](LICENSE.txt).