ibex 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +95 -272
- data/docs/architecture.md +187 -35
- data/docs/bison-import.md +103 -0
- data/docs/comparison-policy.md +127 -0
- data/docs/configuration-model.md +59 -0
- data/docs/configuration-report.md +28 -0
- data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
- data/docs/conflict-explanation-study.md +62 -0
- data/docs/construction-profiling.md +157 -0
- data/docs/cst-migration.md +3 -4
- data/docs/cst.md +24 -0
- data/docs/decisions/0000-template.md +24 -0
- data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
- data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
- data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
- data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
- data/docs/decisions/0005-contained-grammar-composition.md +34 -0
- data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
- data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
- data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
- data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
- data/docs/decisions/0010-committed-runtime-observation.md +34 -0
- data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
- data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
- data/docs/decisions/0013-transactional-generation-publication.md +37 -0
- data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
- data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
- data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
- data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
- data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
- data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
- data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
- data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
- data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
- data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
- data/docs/decisions/0024-direct-ielr-construction.md +42 -0
- data/docs/decisions/README.md +74 -0
- data/docs/declarative-configuration.md +262 -0
- data/docs/development.md +179 -2
- data/docs/direct-ielr-decision.md +120 -0
- data/docs/direct-multi-entry-decision.md +82 -0
- data/docs/editor-setup.md +2 -1
- data/docs/error-ux-review-rubric-v1.md +117 -0
- data/docs/error-ux-reviews/v1/records/README.md +22 -0
- data/docs/error-ux-round2-review-status-v1.json +27 -0
- data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
- data/docs/error-ux-round2-v1.json +1581 -0
- data/docs/error-ux-round2.md +111 -0
- data/docs/error-ux.md +54 -0
- data/docs/getting-started.md +80 -0
- data/docs/grammar-reference.md +184 -13
- data/docs/ielr-design.md +43 -0
- data/docs/ielr.md +92 -0
- data/docs/ja/bison-import.md +51 -0
- data/docs/ja/stable-api.md +53 -0
- data/docs/lexer-construction-profile.md +109 -0
- data/docs/lexer-migration.md +5 -0
- data/docs/maturity.md +128 -0
- data/docs/project-site-strategy.md +54 -0
- data/docs/racc-migration-evidence.md +45 -0
- data/docs/racc-migration.md +11 -0
- data/docs/release-readiness.md +126 -68
- data/docs/repair-semantics.md +123 -0
- data/docs/runtime-abi-evolution.md +307 -0
- data/docs/stability.md +144 -34
- data/docs/status.md +43 -0
- data/docs/syntax-sessions.md +183 -0
- data/docs/table-artifact.md +125 -0
- data/docs/test-interactions.md +219 -0
- data/docs/verification-report.md +137 -0
- data/docs/verifier-trust-boundary.md +199 -0
- data/docs/workloads.md +132 -0
- data/lib/ibex/analysis/digraph.rb +128 -0
- data/lib/ibex/analysis.rb +1 -0
- data/lib/ibex/bison_import/importer.rb +516 -0
- data/lib/ibex/bison_import/tokenizer.rb +260 -0
- data/lib/ibex/bison_import.rb +200 -0
- data/lib/ibex/bounded_subprocess.rb +171 -0
- data/lib/ibex/cli/ambiguity.rb +14 -5
- data/lib/ibex/cli/analysis.rb +134 -0
- data/lib/ibex/cli/bison_import.rb +121 -0
- data/lib/ibex/cli/config.rb +128 -0
- data/lib/ibex/cli/diagnostics.rb +31 -11
- data/lib/ibex/cli/documentation.rb +11 -8
- data/lib/ibex/cli/equiv.rb +169 -0
- data/lib/ibex/cli/error_messages.rb +11 -4
- data/lib/ibex/cli/explain.rb +14 -5
- data/lib/ibex/cli/fix.rb +203 -0
- data/lib/ibex/cli/formatting.rb +11 -10
- data/lib/ibex/cli/fuzz.rb +200 -0
- data/lib/ibex/cli/fuzz_regressions.rb +145 -0
- data/lib/ibex/cli/generation_artifacts.rb +53 -2
- data/lib/ibex/cli/generation_error_messages.rb +1 -1
- data/lib/ibex/cli/grammar_tests.rb +32 -14
- data/lib/ibex/cli/ir_tools.rb +5 -61
- data/lib/ibex/cli/outputs.rb +60 -33
- data/lib/ibex/cli/reduce.rb +202 -0
- data/lib/ibex/cli/reduce_reporting.rb +71 -0
- data/lib/ibex/cli/samples.rb +49 -23
- data/lib/ibex/cli/verify.rb +116 -0
- data/lib/ibex/cli/watch.rb +6 -5
- data/lib/ibex/cli.rb +357 -56
- data/lib/ibex/codegen/action_locations.rb +1 -1
- data/lib/ibex/codegen/action_method_source.rb +5 -6
- data/lib/ibex/codegen/action_source.rb +3 -2
- data/lib/ibex/codegen/ambiguity.rb +5 -4
- data/lib/ibex/codegen/explain.rb +123 -55
- data/lib/ibex/codegen/generated_action_abi.rb +10 -5
- data/lib/ibex/codegen/rbs.rb +15 -7
- data/lib/ibex/codegen/report.rb +25 -13
- data/lib/ibex/codegen/ruby.rb +28 -64
- data/lib/ibex/codegen/ruby_actions.rb +9 -6
- data/lib/ibex/codegen/ruby_syntax.rb +15 -9
- data/lib/ibex/configuration/analysis_grammar.rb +35 -0
- data/lib/ibex/configuration/explanation.rb +421 -0
- data/lib/ibex/configuration/inspector.rb +232 -0
- data/lib/ibex/configuration.rb +580 -0
- data/lib/ibex/coverage/collector.rb +21 -11
- data/lib/ibex/coverage/event_stream.rb +13 -7
- data/lib/ibex/coverage/report.rb +26 -18
- data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
- data/lib/ibex/delta_reducer.rb +99 -0
- data/lib/ibex/diff.rb +140 -0
- data/lib/ibex/equiv/machine.rb +135 -0
- data/lib/ibex/equiv.rb +373 -0
- data/lib/ibex/fix.rb +557 -0
- data/lib/ibex/frontend/ast.rb +22 -2
- data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
- data/lib/ibex/frontend/diagnostic.rb +1 -1
- data/lib/ibex/frontend/formatter.rb +72 -26
- data/lib/ibex/frontend/generated_parser.rb +147 -117
- data/lib/ibex/frontend/generated_parser_base.rb +7 -3
- data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
- data/lib/ibex/frontend/generation.rb +1 -1
- data/lib/ibex/frontend/parser/declarations.rb +17 -2
- data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
- data/lib/ibex/frontend/regenerator.rb +1 -0
- data/lib/ibex/frontend/resolution.rb +2 -1
- data/lib/ibex/frontend/resolver.rb +1 -1
- data/lib/ibex/frontend/rule_documentation.rb +2 -1
- data/lib/ibex/frontend/source_cursor.rb +2 -2
- data/lib/ibex/frontend/source_span.rb +9 -1
- data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
- data/lib/ibex/frontend/token_adapter.rb +10 -0
- data/lib/ibex/frontend.rb +1 -0
- data/lib/ibex/fuzz.rb +219 -0
- data/lib/ibex/generation_input.rb +1 -1
- data/lib/ibex/generation_manifest.rb +36 -25
- data/lib/ibex/generation_transaction.rb +1 -1
- data/lib/ibex/generation_transaction_recovery.rb +4 -4
- data/lib/ibex/generation_transaction_validation.rb +2 -2
- data/lib/ibex/grammar_tests.rb +1 -1
- data/lib/ibex/ir/automaton_ir.rb +61 -19
- data/lib/ibex/ir/grammar_ir.rb +76 -45
- data/lib/ibex/ir/lexer_ir.rb +5 -5
- data/lib/ibex/ir/parser_contract.rb +135 -0
- data/lib/ibex/ir/serialize.rb +203 -76
- data/lib/ibex/ir/validator/automaton.rb +64 -34
- data/lib/ibex/ir/validator/base.rb +51 -33
- data/lib/ibex/ir/validator/grammar.rb +113 -83
- data/lib/ibex/ir/validator/lexer.rb +2 -2
- data/lib/ibex/ir/validator.rb +2 -1
- data/lib/ibex/ir.rb +16 -3
- data/lib/ibex/lalr/build_metrics.rb +50 -2
- data/lib/ibex/lalr/builder.rb +191 -56
- data/lib/ibex/lalr/conflict_search.rb +15 -4
- data/lib/ibex/lalr/counterexample.rb +19 -14
- data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
- data/lib/ibex/lalr/goto_follows.rb +229 -0
- data/lib/ibex/lalr/ielr/annotator.rb +214 -0
- data/lib/ibex/lalr/ielr/bits.rb +28 -0
- data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
- data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
- data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
- data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
- data/lib/ibex/lalr/ielr/split_state.rb +20 -0
- data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
- data/lib/ibex/lalr/ielr_partition.rb +22 -8
- data/lib/ibex/lalr/inadequacy_report.rb +50 -0
- data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
- data/lib/ibex/lalr/lr0_collection.rb +121 -0
- data/lib/ibex/lalr/unreachable_states.rb +80 -0
- data/lib/ibex/lalr.rb +52 -1
- data/lib/ibex/location.rb +3 -3
- data/lib/ibex/lsp/document_handlers.rb +9 -7
- data/lib/ibex/lsp/initialization_handlers.rb +9 -7
- data/lib/ibex/lsp/navigation_handlers.rb +25 -11
- data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
- data/lib/ibex/lsp/position_codec.rb +6 -2
- data/lib/ibex/lsp/request_handlers.rb +3 -2
- data/lib/ibex/lsp/request_support.rb +14 -7
- data/lib/ibex/lsp/server.rb +13 -11
- data/lib/ibex/lsp/symbol_index.rb +13 -13
- data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
- data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
- data/lib/ibex/lsp/transport.rb +13 -3
- data/lib/ibex/lsp/workspace.rb +5 -4
- data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
- data/lib/ibex/lsp.rb +1 -0
- data/lib/ibex/messages/en.yml +28 -0
- data/lib/ibex/messages/ja.yml +28 -0
- data/lib/ibex/messages.rb +68 -0
- data/lib/ibex/metrics.rb +175 -0
- data/lib/ibex/normalize/declarations.rb +2 -0
- data/lib/ibex/normalize/diagnostics.rb +63 -38
- data/lib/ibex/normalize/expander.rb +5 -4
- data/lib/ibex/normalize/expression.rb +8 -6
- data/lib/ibex/normalize/grammar_builder.rb +26 -0
- data/lib/ibex/normalize/inline_expansion.rb +100 -49
- data/lib/ibex/normalize/lexer.rb +3 -3
- data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
- data/lib/ibex/normalize/parameter_lowering.rb +40 -23
- data/lib/ibex/normalize/parameters.rb +31 -4
- data/lib/ibex/normalize/parser_configuration.rb +70 -0
- data/lib/ibex/normalize.rb +10 -20
- data/lib/ibex/racc_migration/report.rb +6 -3
- data/lib/ibex/rake_task.rb +1 -1
- data/lib/ibex/samples.rb +43 -13
- data/lib/ibex/table_artifact/builder.rb +266 -0
- data/lib/ibex/table_artifact/cst_projection.rb +70 -0
- data/lib/ibex/table_artifact/document.rb +39 -0
- data/lib/ibex/table_artifact/executor.rb +187 -0
- data/lib/ibex/table_artifact/serializer.rb +57 -0
- data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
- data/lib/ibex/table_artifact/validator/support.rb +86 -0
- data/lib/ibex/table_artifact/validator/tables.rb +238 -0
- data/lib/ibex/table_artifact/validator.rb +253 -0
- data/lib/ibex/table_artifact.rb +88 -0
- data/lib/ibex/table_simulation/result.rb +6 -2
- data/lib/ibex/table_simulation/step.rb +3 -1
- data/lib/ibex/tables.rb +14 -0
- data/lib/ibex/verifiable_generation_bundle.rb +86 -0
- data/lib/ibex/verification_report/builder.rb +159 -0
- data/lib/ibex/verification_report/canonical_ir.rb +123 -0
- data/lib/ibex/verification_report/logical_path.rb +68 -0
- data/lib/ibex/verification_report/validator.rb +329 -0
- data/lib/ibex/verification_report.rb +69 -0
- data/lib/ibex/verify/action_correspondence.rb +138 -0
- data/lib/ibex/verify/language_witness.rb +295 -0
- data/lib/ibex/verify/reference_collection.rb +190 -0
- data/lib/ibex/verify/result.rb +73 -0
- data/lib/ibex/verify/verifier.rb +598 -0
- data/lib/ibex/verify.rb +15 -0
- data/lib/ibex/version.rb +1 -1
- data/lib/ibex/watch/runner.rb +10 -1
- data/lib/ibex/watch/source_snapshot.rb +13 -8
- data/lib/ibex.rb +10 -0
- data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
- data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
- data/schema/bison-import-v1.schema.json +106 -0
- data/schema/conflict-explanation-review-v1.schema.json +196 -0
- data/schema/conflict-explanation-study-v1.schema.json +528 -0
- data/schema/construction-profile-v1.schema.json +779 -0
- data/schema/diff-v1.schema.json +41 -0
- data/schema/direct-ielr-decision-v1.schema.json +200 -0
- data/schema/equiv-v1.schema.json +64 -0
- data/schema/error-ux-review-v1.schema.json +609 -0
- data/schema/error-ux-round2-review-v1.schema.json +191 -0
- data/schema/error-ux-round2-v1.schema.json +519 -0
- data/schema/explain-v1.schema.json +63 -1
- data/schema/fix-v1.schema.json +64 -0
- data/schema/fix-v2.schema.json +66 -0
- data/schema/fix-v3.schema.json +101 -0
- data/schema/fuzz-regression-v1.schema.json +79 -0
- data/schema/fuzz-v1.schema.json +114 -0
- data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
- data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
- data/schema/grammar-ir.schema.json +271 -0
- data/schema/ielr-benchmark-v1.schema.json +85 -0
- data/schema/lexer-profile-v1.schema.json +491 -0
- data/schema/metrics-v1.schema.json +60 -0
- data/schema/reduce-v1.schema.json +22 -0
- data/schema/reduce-v2.schema.json +63 -0
- data/schema/table-artifact-v1.schema.json +377 -0
- data/schema/verification-report-v1.schema.json +222 -0
- data/schema/verify-v1.schema.json +42 -0
- data/sig/ibex/analysis/digraph.rbs +19 -0
- data/sig/ibex/bison_import/importer.rbs +140 -0
- data/sig/ibex/bison_import/tokenizer.rbs +91 -0
- data/sig/ibex/bison_import.rbs +95 -0
- data/sig/ibex/bounded_subprocess.rbs +76 -0
- data/sig/ibex/cli/ambiguity.rbs +6 -0
- data/sig/ibex/cli/analysis.rbs +43 -0
- data/sig/ibex/cli/bison_import.rbs +32 -0
- data/sig/ibex/cli/config.rbs +35 -0
- data/sig/ibex/cli/diagnostics.rbs +16 -3
- data/sig/ibex/cli/documentation.rbs +5 -1
- data/sig/ibex/cli/equiv.rbs +49 -0
- data/sig/ibex/cli/error_messages.rbs +6 -0
- data/sig/ibex/cli/explain.rbs +6 -0
- data/sig/ibex/cli/fix.rbs +53 -0
- data/sig/ibex/cli/formatting.rbs +5 -1
- data/sig/ibex/cli/fuzz.rbs +47 -0
- data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
- data/sig/ibex/cli/generation_artifacts.rbs +21 -2
- data/sig/ibex/cli/generation_error_messages.rbs +2 -2
- data/sig/ibex/cli/grammar_tests.rbs +15 -3
- data/sig/ibex/cli/ir_tools.rbs +7 -19
- data/sig/ibex/cli/outputs.rbs +9 -3
- data/sig/ibex/cli/reduce.rbs +60 -0
- data/sig/ibex/cli/reduce_reporting.rbs +27 -0
- data/sig/ibex/cli/samples.rbs +8 -0
- data/sig/ibex/cli/verify.rbs +28 -0
- data/sig/ibex/cli/watch.rbs +2 -2
- data/sig/ibex/cli.rbs +79 -5
- data/sig/ibex/codegen/action_method_source.rbs +4 -4
- data/sig/ibex/codegen/explain.rbs +45 -25
- data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
- data/sig/ibex/codegen/rbs.rbs +12 -10
- data/sig/ibex/codegen/report.rbs +12 -12
- data/sig/ibex/codegen/ruby.rbs +18 -15
- data/sig/ibex/codegen/ruby_actions.rbs +6 -6
- data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
- data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
- data/sig/ibex/configuration/explanation.rbs +195 -0
- data/sig/ibex/configuration/inspector.rbs +53 -0
- data/sig/ibex/configuration.rbs +242 -0
- data/sig/ibex/coverage/collector.rbs +16 -14
- data/sig/ibex/coverage/event_stream.rbs +12 -10
- data/sig/ibex/coverage/report.rbs +20 -18
- data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
- data/sig/ibex/delta_reducer.rbs +38 -0
- data/sig/ibex/diff.rbs +47 -0
- data/sig/ibex/equiv/machine.rbs +54 -0
- data/sig/ibex/equiv.rbs +129 -0
- data/sig/ibex/fix.rbs +138 -0
- data/sig/ibex/frontend/ast.rbs +33 -7
- data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
- data/sig/ibex/frontend/diagnostic.rbs +2 -2
- data/sig/ibex/frontend/formatter.rbs +38 -30
- data/sig/ibex/frontend/generated_parser.rbs +87 -77
- data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
- data/sig/ibex/frontend/parser/declarations.rbs +3 -0
- data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
- data/sig/ibex/frontend/resolution.rbs +3 -2
- data/sig/ibex/frontend/source_cursor.rbs +2 -2
- data/sig/ibex/frontend/source_span.rbs +5 -2
- data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
- data/sig/ibex/frontend/token_adapter.rbs +6 -0
- data/sig/ibex/fuzz.rbs +71 -0
- data/sig/ibex/generation_input.rbs +2 -2
- data/sig/ibex/generation_manifest.rbs +35 -32
- data/sig/ibex/generation_transaction_recovery.rbs +2 -2
- data/sig/ibex/generation_transaction_validation.rbs +2 -2
- data/sig/ibex/grammar_tests.rbs +2 -2
- data/sig/ibex/ir/automaton_ir.rbs +30 -11
- data/sig/ibex/ir/grammar_ir.rbs +41 -27
- data/sig/ibex/ir/lexer_ir.rbs +8 -8
- data/sig/ibex/ir/parser_contract.rbs +69 -0
- data/sig/ibex/ir/serialize.rbs +63 -28
- data/sig/ibex/ir/validator/automaton.rbs +47 -44
- data/sig/ibex/ir/validator/base.rbs +35 -31
- data/sig/ibex/ir/validator/grammar.rbs +78 -67
- data/sig/ibex/ir/validator/lexer.rbs +4 -4
- data/sig/ibex/ir.rbs +9 -3
- data/sig/ibex/lalr/build_metrics.rbs +42 -2
- data/sig/ibex/lalr/builder.rbs +75 -24
- data/sig/ibex/lalr/conflict_search.rbs +5 -2
- data/sig/ibex/lalr/counterexample.rbs +17 -14
- data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
- data/sig/ibex/lalr/goto_follows.rbs +89 -0
- data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
- data/sig/ibex/lalr/ielr/bits.rbs +14 -0
- data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
- data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
- data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
- data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
- data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
- data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
- data/sig/ibex/lalr/ielr_partition.rbs +17 -5
- data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
- data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
- data/sig/ibex/lalr/lr0_collection.rbs +50 -0
- data/sig/ibex/lalr/unreachable_states.rbs +25 -0
- data/sig/ibex/lalr.rbs +11 -1
- data/sig/ibex/location.rbs +6 -6
- data/sig/ibex/lsp/document_handlers.rbs +8 -8
- data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
- data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
- data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
- data/sig/ibex/lsp/position_codec.rbs +8 -4
- data/sig/ibex/lsp/request_handlers.rbs +4 -4
- data/sig/ibex/lsp/request_support.rbs +8 -8
- data/sig/ibex/lsp/server.rbs +16 -16
- data/sig/ibex/lsp/symbol_index.rbs +20 -21
- data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
- data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
- data/sig/ibex/lsp/transport.rbs +18 -6
- data/sig/ibex/lsp/workspace.rbs +3 -2
- data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
- data/sig/ibex/messages.rbs +26 -0
- data/sig/ibex/metrics.rbs +50 -0
- data/sig/ibex/normalize/diagnostics.rbs +4 -4
- data/sig/ibex/normalize/expander.rbs +2 -2
- data/sig/ibex/normalize/expression.rbs +10 -8
- data/sig/ibex/normalize/grammar_builder.rbs +11 -0
- data/sig/ibex/normalize/inline_expansion.rbs +59 -50
- data/sig/ibex/normalize/lexer.rbs +4 -4
- data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
- data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
- data/sig/ibex/normalize/parameters.rbs +13 -8
- data/sig/ibex/normalize/parser_configuration.rbs +23 -0
- data/sig/ibex/normalize.rbs +11 -5
- data/sig/ibex/racc_migration/report.rbs +8 -6
- data/sig/ibex/rake_task.rbs +1 -1
- data/sig/ibex/samples.rbs +13 -4
- data/sig/ibex/table_artifact/builder.rbs +93 -0
- data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
- data/sig/ibex/table_artifact/document.rbs +27 -0
- data/sig/ibex/table_artifact/executor.rbs +66 -0
- data/sig/ibex/table_artifact/serializer.rbs +25 -0
- data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
- data/sig/ibex/table_artifact/validator/support.rbs +46 -0
- data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
- data/sig/ibex/table_artifact/validator.rbs +66 -0
- data/sig/ibex/table_artifact.rbs +32 -0
- data/sig/ibex/table_simulation/result.rbs +8 -4
- data/sig/ibex/table_simulation/step.rbs +4 -2
- data/sig/ibex/tables.rbs +10 -9
- data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
- data/sig/ibex/verification_report/builder.rbs +53 -0
- data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
- data/sig/ibex/verification_report/logical_path.rbs +34 -0
- data/sig/ibex/verification_report/validator.rbs +114 -0
- data/sig/ibex/verification_report.rbs +44 -0
- data/sig/ibex/verify/action_correspondence.rbs +50 -0
- data/sig/ibex/verify/language_witness.rbs +112 -0
- data/sig/ibex/verify/reference_collection.rbs +63 -0
- data/sig/ibex/verify/result.rbs +52 -0
- data/sig/ibex/verify/verifier.rbs +170 -0
- data/sig/ibex/verify.rbs +6 -0
- data/sig/ibex/watch/runner.rbs +12 -2
- data/sig/ibex/watch/source_snapshot.rbs +13 -9
- metadata +232 -12
- data/lib/ibex/ir/migration.rb +0 -120
- data/sig/ibex/ir/migration.rbs +0 -34
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# ADR 0008: Separate the runtime behind a versioned parser-table ABI
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Applications executing a generated parser do not need the grammar frontend or
|
|
9
|
+
generator. Independently installed generated code and runtime code still need a
|
|
10
|
+
precise compatibility boundary; otherwise a table-shape change can fail or be
|
|
11
|
+
misinterpreted during a parse.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
The dependency-free parser runtime ships as `ibex-runtime`; the generator
|
|
16
|
+
depends on it but runtime deployments do not depend on generator code. Normal
|
|
17
|
+
generated parsers require the runtime package, while embedded output carries
|
|
18
|
+
the same runtime implementation.
|
|
19
|
+
|
|
20
|
+
Every generated table declares a parser-table format version. The runtime
|
|
21
|
+
validates the version and marker combinations before consuming input. Changes
|
|
22
|
+
to required shape, action calling convention, or table meaning increment that
|
|
23
|
+
version.
|
|
24
|
+
|
|
25
|
+
Plain and compact tables are two encodings of the same lookup contract.
|
|
26
|
+
Compression may change representation only when every declared and unknown
|
|
27
|
+
token retains the same shift, reduce, accept, or error result. In particular,
|
|
28
|
+
explicit error cells must mask default reductions.
|
|
29
|
+
|
|
30
|
+
## Consequences
|
|
31
|
+
|
|
32
|
+
- Generator and runtime packaging can evolve independently within the declared
|
|
33
|
+
dependency and table-format contracts.
|
|
34
|
+
- Unsupported generated parsers fail before lexer or semantic side effects.
|
|
35
|
+
- Compact representation remains replaceable and optimizable without becoming
|
|
36
|
+
a second parser semantics.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# ADR 0009: Isolate mutable parser sessions from immutable parser data
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Generated tables are immutable program data, while LR stacks, lexer state,
|
|
9
|
+
lookahead, recovery state, observers, and incremental caches are per parse.
|
|
10
|
+
Supporting pull, yielding, and push input through separate engines would make
|
|
11
|
+
their recovery and callback behavior drift.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
Generated parser classes own recursively frozen tables. Each parser instance
|
|
16
|
+
owns one mutable session and permits only one active driver. Pull, yielding, and
|
|
17
|
+
push APIs feed the same LR transition and recovery machinery; nested or mixed
|
|
18
|
+
drivers fail before mutating state.
|
|
19
|
+
|
|
20
|
+
State, semantic value, optional location, and optional Green stacks remain
|
|
21
|
+
index-aligned. Capabilities allocate their parallel state only when the
|
|
22
|
+
generated grammar or runtime tooling requires it.
|
|
23
|
+
|
|
24
|
+
Per-instance resource limits bound stack growth, recovery attempts, and other
|
|
25
|
+
session-owned search or memo structures. Exceeding a bound produces a
|
|
26
|
+
structured failure rather than unbounded work. Sharing a parser instance or
|
|
27
|
+
its application callbacks across concurrent parses is outside the contract.
|
|
28
|
+
|
|
29
|
+
## Consequences
|
|
30
|
+
|
|
31
|
+
- Parser classes and eligible table graphs can be shared while sessions remain
|
|
32
|
+
independently mutable.
|
|
33
|
+
- All input lifecycles preserve one action, recovery, and callback ordering.
|
|
34
|
+
- Applications choose stricter or larger limits without regenerating tables.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# ADR 0010: Observe only committed runtime transitions
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Hooks, tracing, coverage, and debugging need parser activity without depending
|
|
9
|
+
on private mutable stacks. Emitting speculative events makes failed semantic
|
|
10
|
+
actions and recovery attempts indistinguishable from committed parser state.
|
|
11
|
+
Optional observation must also remain removable from the normal hot path.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
Runtime hooks and versioned events are emitted at documented committed
|
|
16
|
+
boundaries for shifts, reductions, errors, recovery, discard, acceptance, and
|
|
17
|
+
rejection. Payloads are immutable snapshots or bounded JSON-safe summaries;
|
|
18
|
+
they never expose live parser stacks or retain arbitrary application objects.
|
|
19
|
+
Observer failures are visible to the parser caller.
|
|
20
|
+
|
|
21
|
+
Tracing and coverage consume this public boundary rather than adding parser
|
|
22
|
+
drivers. When no observer or hook is active, the runtime does not construct
|
|
23
|
+
event payloads.
|
|
24
|
+
|
|
25
|
+
The generic runtime is the semantic reference. A fast path may run only after
|
|
26
|
+
proving that every omitted observable capability is inactive, and must fall
|
|
27
|
+
back before a newly enabled capability could miss its documented next event.
|
|
28
|
+
|
|
29
|
+
## Consequences
|
|
30
|
+
|
|
31
|
+
- Runtime tools compose without reading implementation state.
|
|
32
|
+
- Coverage and traces describe completed transitions rather than intentions.
|
|
33
|
+
- Performance work must preserve invalidation rules whenever a new observation
|
|
34
|
+
capability is added.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# ADR 0011: Compile opaque actions once behind a versioned calling boundary
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Generated semantic actions need grammar-source backtraces, access to reduction
|
|
9
|
+
values and locations, and efficient calls from the runtime. Re-evaluating
|
|
10
|
+
source on every reduction is expensive, while changing generated method
|
|
11
|
+
arguments can silently break older tables or application callables.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
The generator builds each opaque action method once and compiles it when the
|
|
16
|
+
generated class loads, preserving the grammar filename and line. Direct
|
|
17
|
+
generated-file mode emits the equivalent method body for inspection. Static
|
|
18
|
+
action-shadow output uses the same method-source builder and is never executed
|
|
19
|
+
by Ibex.
|
|
20
|
+
|
|
21
|
+
Generated action call shapes are explicit production metadata covered by the
|
|
22
|
+
parser-table format version. The runtime validates markers before input and
|
|
23
|
+
grants generated-only ABIs only to generated action methods. Handwritten
|
|
24
|
+
methods and callables keep their documented application ABI.
|
|
25
|
+
|
|
26
|
+
Action dependency analysis is conservative. It may select a narrower values or
|
|
27
|
+
positional ABI only when lexical evidence proves omitted stacks, locations, or
|
|
28
|
+
containers are unused; unsupported or unparsable source uses the general ABI.
|
|
29
|
+
Location references are rewritten only at Ruby code tokens, not inside literal
|
|
30
|
+
or comment content.
|
|
31
|
+
|
|
32
|
+
## Consequences
|
|
33
|
+
|
|
34
|
+
- Actions compile once while retaining useful source-mapped failures.
|
|
35
|
+
- Generated ABI optimization cannot silently change application callables.
|
|
36
|
+
- Conservative fallback can miss an optimization but cannot omit an input the
|
|
37
|
+
action may observe.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# ADR 0012: Keep parser analysis bounded and free of semantic execution
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Conflict counterexamples, sample generation, table debugging, ambiguity
|
|
9
|
+
checks, migration inspection, and automatic repair explore potentially
|
|
10
|
+
unbounded parser or grammar configurations. Executing semantic actions on
|
|
11
|
+
speculative paths would duplicate application side effects and make read-only
|
|
12
|
+
tools unsafe.
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
Static tools operate on validated Grammar IR, Automaton IR, or parser-table
|
|
17
|
+
state only. They never load generated application code or invoke semantic
|
|
18
|
+
actions. Every search has explicit token, configuration, stack, step, or
|
|
19
|
+
expansion limits and terminates when a limit is reached.
|
|
20
|
+
|
|
21
|
+
Results distinguish a proof from bounded evidence and from exhaustion. Failure
|
|
22
|
+
to find an ambiguity is never reported as proof of unambiguity. Table
|
|
23
|
+
simulation applies the same explicit-cell-before-default lookup rule as the
|
|
24
|
+
runtime.
|
|
25
|
+
|
|
26
|
+
Optional runtime repair also searches table configurations without actions.
|
|
27
|
+
Only the selected repair is replayed through the ordinary runtime, where
|
|
28
|
+
semantic actions and observers execute once on the committed path.
|
|
29
|
+
|
|
30
|
+
## Consequences
|
|
31
|
+
|
|
32
|
+
- Analysis is suitable for untrusted validated artifacts within caller-supplied
|
|
33
|
+
input-size limits.
|
|
34
|
+
- Deep grammars may require explicitly larger budgets or produce an
|
|
35
|
+
inconclusive result.
|
|
36
|
+
- Runtime and tooling share table semantics without sharing mutable parser
|
|
37
|
+
internals.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# ADR 0013: Publish generated artifacts as a verified transaction
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
One generation can produce Ruby, RBS, static action source, reports, and
|
|
9
|
+
metadata. Writing outputs as renderers finish can expose a mixed generation,
|
|
10
|
+
and filesystem aliases or a source change during generation can invalidate the
|
|
11
|
+
candidate being published.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
Generation renders every requested artifact in memory before changing output
|
|
16
|
+
paths. It records the exact input closure and revalidates it before
|
|
17
|
+
publication. Targets are resolved and checked for input aliases and
|
|
18
|
+
cross-output collisions.
|
|
19
|
+
|
|
20
|
+
Stages and recovery backups are created exclusively beside their targets.
|
|
21
|
+
Publication uses ordered locks, flushed and synchronized stages, atomic
|
|
22
|
+
renames, directory synchronization, and reverse-order restoration on failure.
|
|
23
|
+
Because a portable filesystem cannot atomically replace several paths, an
|
|
24
|
+
optional final manifest is the coherence marker: readers that need one
|
|
25
|
+
generation verify artifact digests against a stable manifest read.
|
|
26
|
+
|
|
27
|
+
Watch mode runs this same transaction only after a stable source closure is
|
|
28
|
+
observed; it never publishes a candidate built from changing inputs.
|
|
29
|
+
|
|
30
|
+
## Consequences
|
|
31
|
+
|
|
32
|
+
- Rendering, source races, and recoverable write failures do not intentionally
|
|
33
|
+
publish a partial candidate as complete.
|
|
34
|
+
- Multi-file coherence requires the manifest protocol rather than an
|
|
35
|
+
unsupported cross-file atomicity claim.
|
|
36
|
+
- Transaction and watch code pay filesystem synchronization and hashing costs
|
|
37
|
+
in exchange for a recoverable publication boundary.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# ADR 0014: Model generated lexers as an independently versioned stage
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Handwritten token sources keep the parser runtime general, but a generated
|
|
9
|
+
lexer needs durable matching, state, streaming, and location semantics.
|
|
10
|
+
Embedding those details directly in Grammar or Automaton IR would tie lexer
|
|
11
|
+
evolution to parser construction.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
Generated lexer declarations normalize into immutable Lexer IR with its own
|
|
16
|
+
schema version. Grammar IR may embed that document, while Lexer IR can also be
|
|
17
|
+
validated and consumed independently.
|
|
18
|
+
|
|
19
|
+
Rules are ordered and state-scoped. At one input position the lexer chooses the
|
|
20
|
+
longest match, then the lowest rule id; empty matches are invalid. Lexical state
|
|
21
|
+
is explicit per parser instance. Streaming input retries a match when the
|
|
22
|
+
current chunk may be a strict prefix, so tokenization does not depend on chunk
|
|
23
|
+
boundaries.
|
|
24
|
+
|
|
25
|
+
Lexer actions execute only in the generated runtime boundary. Static tools may
|
|
26
|
+
inspect their opaque source but do not run it.
|
|
27
|
+
|
|
28
|
+
## Consequences
|
|
29
|
+
|
|
30
|
+
- Lexer format can evolve without changing Automaton IR or handwritten token
|
|
31
|
+
sources.
|
|
32
|
+
- Matching is deterministic but intentionally uses a simple ordered rule model
|
|
33
|
+
rather than a hidden engine-specific DFA contract.
|
|
34
|
+
- Regular-expression safety remains an application responsibility; static
|
|
35
|
+
warnings are not a proof against pathological expressions.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# ADR 0015: Isolate browser analysis in a replaceable worker
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Ruby running through WebAssembly can consume substantial CPU and memory.
|
|
9
|
+
Running grammar analysis on the main browser thread would let a pathological
|
|
10
|
+
input freeze the interface, and interpolating grammar text into Ruby source
|
|
11
|
+
would create an unnecessary execution boundary.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
The Ruby VM and grammar-analysis pipeline run in a dedicated module worker.
|
|
16
|
+
The main thread owns the timeout and replaces the worker on expiry. Input,
|
|
17
|
+
diagnostic, conflict, and search sizes are bounded. Grammar bytes cross the
|
|
18
|
+
JavaScript/Ruby boundary as data and semantic action bodies are never
|
|
19
|
+
evaluated.
|
|
20
|
+
|
|
21
|
+
The worker returns serialized analysis results only. Browser assets are
|
|
22
|
+
self-hosted under a restrictive Content Security Policy, and no submitted
|
|
23
|
+
grammar is sent to or stored by an application server.
|
|
24
|
+
|
|
25
|
+
## Consequences
|
|
26
|
+
|
|
27
|
+
- Pathological analysis cannot retain Ruby state after worker replacement and
|
|
28
|
+
cannot block the main thread indefinitely.
|
|
29
|
+
- Worker termination is the isolation boundary, not a claim that WebAssembly
|
|
30
|
+
is a security sandbox.
|
|
31
|
+
- Browser support depends on Web Workers, WebAssembly, module scripts, and the
|
|
32
|
+
shipped Ruby VM asset.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# ADR 0016: Represent concrete syntax with immutable Green data and lazy Red views
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
A lossless syntax tree must support sharing, navigation, recovery, and editing.
|
|
9
|
+
Nodes containing absolute offsets, parents, or semantic action results cannot
|
|
10
|
+
be safely interned or reused at multiple source occurrences.
|
|
11
|
+
|
|
12
|
+
## Decision
|
|
13
|
+
|
|
14
|
+
Green nodes, tokens, and trivia are immutable, position-independent values.
|
|
15
|
+
They contain integer kinds, source bytes, child structure, flags, and derived
|
|
16
|
+
widths, but no parent, absolute location, parser instance, or semantic value.
|
|
17
|
+
Tokens own trivia deterministically, including final trivia on EOF.
|
|
18
|
+
|
|
19
|
+
Red values are lazy occurrence views over Green values. They provide parents,
|
|
20
|
+
child indexes, and absolute offsets without changing Green identity.
|
|
21
|
+
|
|
22
|
+
CST-aware parsers maintain a Green stack parallel to the semantic stack.
|
|
23
|
+
Every physical reduction builds syntax independently of whether a semantic
|
|
24
|
+
action exists. Recovery and repair produce flagged syntax, and the public
|
|
25
|
+
syntax root is a synthetic `source_file` containing the selected start node
|
|
26
|
+
and EOF. Structured CST metadata in the parser-table ABI selects this path.
|
|
27
|
+
|
|
28
|
+
## Consequences
|
|
29
|
+
|
|
30
|
+
- Equal Green subtrees can be interned, shared across Ractors, and reused at
|
|
31
|
+
multiple Red occurrences.
|
|
32
|
+
- Syntax shape does not depend on semantic action return values.
|
|
33
|
+
- Absolute navigation allocates Red wrappers lazily, while Green construction
|
|
34
|
+
must preserve exact source width and trivia ownership.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# ADR 0017: Persist and edit syntax through Green structure
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Mutable syntax nodes make snapshots and sharing difficult. Serializing Red
|
|
9
|
+
parents and offsets would duplicate derived occurrence state and tie the file
|
|
10
|
+
format to one navigation implementation.
|
|
11
|
+
|
|
12
|
+
## Decision
|
|
13
|
+
|
|
14
|
+
Syntax edits replace one Green occurrence and path-copy only its ancestor
|
|
15
|
+
chain. Batch rewriting resolves replacements by Red occurrence; unchanged
|
|
16
|
+
subtrees retain Green identity. Annotations are occurrence-scoped identities,
|
|
17
|
+
exclude annotated Green values from interning, and are not serialized.
|
|
18
|
+
|
|
19
|
+
Concrete syntax uses an independently versioned closed schema. It records
|
|
20
|
+
grammar and parser compatibility metadata, kind metadata, trivia policy, and
|
|
21
|
+
one Green root. Source bytes use a canonical UTF-8-or-Base64 representation.
|
|
22
|
+
Loading validates the closed shape and compatibility, then recomputes widths,
|
|
23
|
+
descendant counts, and aggregate flags through Green constructors. Red parents,
|
|
24
|
+
offsets, cached wrappers, and annotations are deliberately absent.
|
|
25
|
+
|
|
26
|
+
Text diffs descend past identical Green values and emit byte-oriented edits
|
|
27
|
+
whose application must reconstruct the new source exactly.
|
|
28
|
+
|
|
29
|
+
## Consequences
|
|
30
|
+
|
|
31
|
+
- Small edits allocate in proportion to the changed occurrence path.
|
|
32
|
+
- Serialized trees are independent of Red navigation caches and preserve
|
|
33
|
+
arbitrary source bytes.
|
|
34
|
+
- Structural editing may produce grammar-invalid trees; parsing validation is
|
|
35
|
+
a separate operation.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# ADR 0018: Reuse incremental syntax only with conservative proofs
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-07-28
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Parser production actions may depend on arbitrary mutable state. Replaying or
|
|
9
|
+
skipping them during incremental parsing can duplicate side effects or return a
|
|
10
|
+
stale semantic value. Reusing a syntax subtree is also unsafe unless lexer and
|
|
11
|
+
LR state match at its boundaries.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
Incremental sessions are syntax-only: parser production actions are suppressed
|
|
16
|
+
and no semantic value is returned. Generated lexer actions still run because
|
|
17
|
+
they define token emission and lexical state. A fresh syntax parse remains the
|
|
18
|
+
reference result.
|
|
19
|
+
|
|
20
|
+
Relexing may reuse an old token suffix only after exact source-boundary, lexer
|
|
21
|
+
state, kind, text, flags, and trivia agreement. Green interning may preserve
|
|
22
|
+
equal token and node identity independently of parser reuse.
|
|
23
|
+
|
|
24
|
+
Subtree blending additionally requires the recorded left LR state to match the
|
|
25
|
+
live state, an unchanged following lookahead, a positive-width complete
|
|
26
|
+
nonterminal, and no error, missing, or skipped flags. Parse memo identity is by
|
|
27
|
+
occurrence position, not Green object identity. Every cache, decomposition, and
|
|
28
|
+
memo has a resource bound; uncertainty or exhaustion falls back to the already
|
|
29
|
+
available fresh path.
|
|
30
|
+
|
|
31
|
+
Incremental sessions and their caches are single-owner mutable objects.
|
|
32
|
+
|
|
33
|
+
## Consequences
|
|
34
|
+
|
|
35
|
+
- Incremental parsing cannot silently reuse semantic side effects or values.
|
|
36
|
+
- Valid reuse is intentionally missed whenever proof metadata is insufficient.
|
|
37
|
+
- Correctness is defined by equality with a fresh syntax parse; performance is
|
|
38
|
+
evidence for optimization, not part of this decision.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# ADR 0019: Keep generated-language syntax sessions in the runtime
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-08-05
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
The Red/Green CST and incremental parser already provide syntax-only parsing,
|
|
9
|
+
but application integrations otherwise have to assemble trust checks, edits,
|
|
10
|
+
failure expectations, fallback evidence, cancellation, and resource bounds.
|
|
11
|
+
This boundary is for one loaded generated language. It is not the existing
|
|
12
|
+
grammar-authoring LSP and has no workspace or protocol semantics.
|
|
13
|
+
|
|
14
|
+
Generated lexer actions and generated-file user sections are arbitrary Ruby.
|
|
15
|
+
A session-time flag cannot make an already loaded parser class safe.
|
|
16
|
+
|
|
17
|
+
## Decision
|
|
18
|
+
|
|
19
|
+
Add an Experimental `SyntaxSession` façade to `ibex-runtime`, backed solely by
|
|
20
|
+
the existing `IncrementalParseSession`. Current parser classes expose
|
|
21
|
+
`:trusted_application_code` and require that exact profile as an explicit
|
|
22
|
+
caller acknowledgement. No `:declarative` mode is exposed until the generator
|
|
23
|
+
can produce a separate action-free syntax artifact with a declarative lexer and
|
|
24
|
+
without user-code sections.
|
|
25
|
+
|
|
26
|
+
Return immutable result and metric snapshots. Treat syntax errors as results,
|
|
27
|
+
but represent cancellation and hard resource exhaustion with distinct
|
|
28
|
+
exceptions. Preserve the previous completed session state when an edit aborts.
|
|
29
|
+
|
|
30
|
+
Keep the boundary in `ibex-runtime`: deployed generated languages need the
|
|
31
|
+
runtime but not grammar construction. Do not create `ibex-workbench` until a
|
|
32
|
+
larger editor-service boundary has independent demand. Do not add LSP,
|
|
33
|
+
workspace, indexing, query, or formatting behavior here.
|
|
34
|
+
|
|
35
|
+
## Consequences
|
|
36
|
+
|
|
37
|
+
- Parser production actions remain suppressed; lexer actions and runtime hooks
|
|
38
|
+
remain explicit trusted application execution.
|
|
39
|
+
- Fresh syntax parsing remains the correctness oracle and the existing CST
|
|
40
|
+
implementation remains the only parser/tree engine.
|
|
41
|
+
- Parser-table format v6 and normal semantic parse behavior do not change. The
|
|
42
|
+
installed-runtime API is additive, and feature-off non-embedded generated
|
|
43
|
+
bytes remain unchanged.
|
|
44
|
+
- Embedded generated bytes do change because they copy the runtime sources.
|
|
45
|
+
Existing embedded artifacts continue to parse but must be regenerated and
|
|
46
|
+
redeployed to acquire `SyntaxSession` and its fixes. A non-embedded generated
|
|
47
|
+
parser acquires the additive API by upgrading its compatible runtime package.
|
|
48
|
+
- Cancellation is cooperative and cannot preempt a nonreturning Ruby lexer
|
|
49
|
+
action.
|
|
50
|
+
- The façade remains part of the existing Experimental incremental-CST product
|
|
51
|
+
surface rather than claiming a new Stable editor platform.
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# ADR 0020: Persist parser contracts in Grammar IR
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-08-05
|
|
5
|
+
|
|
6
|
+
> The pre-release compatibility readers and in-process IR migration path were
|
|
7
|
+
> retired before v1.0. Racc-compatible grammar input remains supported.
|
|
8
|
+
|
|
9
|
+
## Context
|
|
10
|
+
|
|
11
|
+
Parser construction algorithm, multiple-entry construction, and CST trivia
|
|
12
|
+
ownership are durable grammar-owned contracts. They must survive normalization,
|
|
13
|
+
IR-only generation, Automaton IR embedding, and static inspection before a
|
|
14
|
+
source declaration can be released. The current Grammar IR stores these
|
|
15
|
+
contracts at the root so a resumed pipeline sees the same explicit and
|
|
16
|
+
unspecified states as the source pipeline.
|
|
17
|
+
|
|
18
|
+
Three persistence boundaries were credible:
|
|
19
|
+
|
|
20
|
+
1. add one root `parser_contract` to Grammar IR;
|
|
21
|
+
2. publish a separately versioned Parser Contract IR; or
|
|
22
|
+
3. wrap grammar and contract documents in a grammar-bundle envelope.
|
|
23
|
+
|
|
24
|
+
The choice crosses persisted schemas, grammar digests, resumed generation, and
|
|
25
|
+
the Automaton IR boundary. Reversing it requires coordinated changes to the
|
|
26
|
+
writer, reader, schemas, fixtures, and downstream consumers.
|
|
27
|
+
|
|
28
|
+
## Decision
|
|
29
|
+
|
|
30
|
+
The current Grammar IR owns one closed, root-level `parser_contract`. Its admitted
|
|
31
|
+
fields are parser algorithm, entry construction, and CST trivia ownership.
|
|
32
|
+
Each field records an explicit value and source location, or an explicit
|
|
33
|
+
unspecified state. Unspecified contract state contains no built-in default;
|
|
34
|
+
defaults and invocation requests remain inputs to the typed configuration
|
|
35
|
+
resolver.
|
|
36
|
+
|
|
37
|
+
Automaton IR embeds the exact current Grammar IR document. Its grammar digest
|
|
38
|
+
therefore binds the parser contract. It also records whether entry construction
|
|
39
|
+
was shared or isolated; the existing algorithm field remains the selected
|
|
40
|
+
automaton fact.
|
|
41
|
+
|
|
42
|
+
There is one reader and one writer for the current format. `Grammar.new` and
|
|
43
|
+
`Automaton.new` both construct the current format directly.
|
|
44
|
+
|
|
45
|
+
A separate Parser Contract IR is rejected because it would create two
|
|
46
|
+
authorities that must be joined atomically and included independently in every
|
|
47
|
+
digest. A bundle envelope is rejected because Grammar IR already contains the
|
|
48
|
+
normalized root and resolved fragment result; an additional container would
|
|
49
|
+
add identity rules without an independently lifecycled artifact.
|
|
50
|
+
|
|
51
|
+
The contract remains generator-side data. It does not change parser-table
|
|
52
|
+
format, generated runtime constants, or the `ibex-runtime` ABI. Static readers
|
|
53
|
+
validate it without evaluating grammar actions or application code.
|
|
54
|
+
|
|
55
|
+
## Consequences
|
|
56
|
+
|
|
57
|
+
- Source and IR-only pipelines share one normalized contract and one grammar
|
|
58
|
+
digest.
|
|
59
|
+
- Unspecified source contract remains distinct from a newly selected built-in
|
|
60
|
+
default.
|
|
61
|
+
- Fragments cannot acquire parser-wide authority; only the normalized root has
|
|
62
|
+
a parser contract.
|
|
63
|
+
- Grammar and Automaton IR each have one closed current schema.
|
|
64
|
+
- Revisit the rejected sidecar only if parser contracts gain an independent
|
|
65
|
+
lifecycle or a non-grammar authority with multiple real consumers.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# ADR 0021: Persist executable parser tables as a data-only sidecar
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-08-05
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Automaton IR is suitable for construction analysis, but a consumer must still
|
|
9
|
+
build runtime tables from it. Generated Ruby already contains executable
|
|
10
|
+
tables, but inspecting that file safely would require parsing Ruby and loading
|
|
11
|
+
it may execute application code. Keeping only Automaton IR provides no
|
|
12
|
+
separate object that a publication report can identify as the executed table.
|
|
13
|
+
|
|
14
|
+
The alternatives were to keep rebuilding tables from Automaton IR, treat
|
|
15
|
+
generated Ruby as the table container, or persist the table projection as a
|
|
16
|
+
closed data format. The last option crosses persistence, execution, packaging,
|
|
17
|
+
and trust boundaries and therefore needs a durable decision.
|
|
18
|
+
|
|
19
|
+
## Decision
|
|
20
|
+
|
|
21
|
+
Adopt an Experimental, separately versioned JSON sidecar as the data-only
|
|
22
|
+
authority for parser recognition tables. It records canonical internal token
|
|
23
|
+
IDs, actions, GOTOs, defaults, production stack effects, entries, static CST
|
|
24
|
+
and recovery metadata, and explicit source/table/metadata identities.
|
|
25
|
+
|
|
26
|
+
Semantic action bodies, generated wrappers, lexers, external token conversion,
|
|
27
|
+
and application hooks remain outside the artifact. Productions identify only
|
|
28
|
+
opaque wrapper action slots. The prototype executor recognizes internal token
|
|
29
|
+
ID sequences and never reconstructs Automaton IR or evaluates an action.
|
|
30
|
+
|
|
31
|
+
Artifact schema versioning is independent of the generated parser Stable ABI.
|
|
32
|
+
Version 1 names parser-table format v6 only to pin action and production
|
|
33
|
+
semantics; the current runtime does not accept the JSON representation. The
|
|
34
|
+
sidecar remains outside default generation and runtime packaging until another
|
|
35
|
+
decision defines compatibility and publication binding.
|
|
36
|
+
|
|
37
|
+
## Consequences
|
|
38
|
+
|
|
39
|
+
- Consumers can validate and exercise table data without loading generated
|
|
40
|
+
application code, and later reports have a non-Ruby table identity to bind.
|
|
41
|
+
- The sidecar has real execution value beyond Automaton IR, while retaining the
|
|
42
|
+
wrapper as authority for semantic actions and tokenization.
|
|
43
|
+
- Source digests in a standalone artifact are linkage claims, not proofs or
|
|
44
|
+
signatures; verification against source IR requires that IR as another
|
|
45
|
+
input.
|
|
46
|
+
- The prototype duplicates some Automaton IR projection data and introduces a
|
|
47
|
+
schema/loader compatibility obligation.
|
|
48
|
+
- Runtime consumption, semantic action ABI binding, default emission, or a
|
|
49
|
+
stability promotion must revisit this decision and define migration without
|
|
50
|
+
silently changing existing generated bytes.
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# ADR 0022: Bind scoped verification evidence through the generation manifest
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-08-05
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
The independent Automaton IR verifier and the data-only parser table sidecar
|
|
9
|
+
have useful but separate identities. A published parser also includes a Ruby
|
|
10
|
+
wrapper containing opaque actions. Consumers need to detect a table, report,
|
|
11
|
+
wrapper, or input taken from another generation without loading the wrapper or
|
|
12
|
+
overstating what the verifier checks.
|
|
13
|
+
|
|
14
|
+
A report could contain every artifact digest including its manifest, be
|
|
15
|
+
embedded in generated Ruby, use physical paths as artifact identities, or be
|
|
16
|
+
published as a separate manifest-bound artifact. Including the manifest
|
|
17
|
+
digest creates a hash cycle because the manifest must hash the report.
|
|
18
|
+
Embedding evidence in Ruby makes nonexecuting inspection depend on a source
|
|
19
|
+
container that also carries opaque code. Physical paths make equivalent
|
|
20
|
+
evidence differ between checkouts.
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
Adopt an Experimental, separately versioned scoped verification report and an
|
|
25
|
+
explicit verifiable-generation-bundle API. The report records logical input
|
|
26
|
+
identities, Grammar and Automaton IR digests, the data-only table artifact and
|
|
27
|
+
payload digests, checker version, verification profile, completed checks,
|
|
28
|
+
bounds, outcome, and a closed excluded-trust list. That machine-readable
|
|
29
|
+
exclusion includes semantic derivation or language equivalence from Automaton
|
|
30
|
+
IR to persisted table data; version 1 validates table structure and digest
|
|
31
|
+
linkage but does not independently prove that projection.
|
|
32
|
+
|
|
33
|
+
The report never contains the manifest digest or path. Generation renders the
|
|
34
|
+
table, wrapper, and report in memory, then renders a generation manifest that
|
|
35
|
+
lists and hashes all three. The existing transaction publishes non-marker
|
|
36
|
+
artifacts and the wrapper before installing that manifest as the final
|
|
37
|
+
coherence marker.
|
|
38
|
+
|
|
39
|
+
Canonical report evidence uses ordered logical identities rather than
|
|
40
|
+
absolute paths: indexed input basenames and a role-qualified table basename.
|
|
41
|
+
Version 1 closes both forms to one basename, exactly four input-index digits,
|
|
42
|
+
and at most 10,000 inputs.
|
|
43
|
+
|
|
44
|
+
IR claims use the explicitly named `source-logical-v1` identity scope. Bundle
|
|
45
|
+
construction rebuilds an immutable Automaton IR copy from the supplied IR and
|
|
46
|
+
`source_records`: every location file becomes its ordered logical input
|
|
47
|
+
identity and each non-null source root becomes `input`. The table and verifier
|
|
48
|
+
consume that same copy, so its Grammar IR digest, embedded Automaton IR
|
|
49
|
+
grammar digest, Automaton IR digest, table identity, report claims, and report
|
|
50
|
+
evidence digest cannot disagree. An unmapped or ambiguous source location is
|
|
51
|
+
an error. The original Automaton, its diagnostic provenance, and ordinary
|
|
52
|
+
V002 table construction remain unchanged.
|
|
53
|
+
|
|
54
|
+
The bundle validator checks report self-identity, manifest artifact bytes,
|
|
55
|
+
input identity, table identity and payload, and IR/table linkage without
|
|
56
|
+
parsing or loading generated Ruby.
|
|
57
|
+
|
|
58
|
+
The API stays outside the default `ibex` require path and generator CLI. An
|
|
59
|
+
application must explicitly require and invoke it. This preserves existing
|
|
60
|
+
generation bytes and leaves CLI/stability promotion to a later compatibility
|
|
61
|
+
decision.
|
|
62
|
+
|
|
63
|
+
## Consequences
|
|
64
|
+
|
|
65
|
+
- A manifest is the single non-cyclic publication marker for table, wrapper,
|
|
66
|
+
and report coherence.
|
|
67
|
+
- Equivalent source records produce identical table and report evidence
|
|
68
|
+
across checkout roots when their ordered logical basenames and bytes match.
|
|
69
|
+
- Missing, stale, duplicated, or cross-generation table/report/manifest
|
|
70
|
+
combinations are rejected before generated Ruby is loaded.
|
|
71
|
+
- A passing report remains bounded evidence for named V1-V8 checks, not proof
|
|
72
|
+
of table semantic derivation, wrapper, action, lexer, runtime, application,
|
|
73
|
+
source-to-IR, authenticity, or grammar-unambiguity correctness.
|
|
74
|
+
- The new schema and validator are persistence and trusted-computing-base
|
|
75
|
+
obligations. Future schema versions may extend evidence only without
|
|
76
|
+
silently widening version 1 claims.
|
|
77
|
+
- Default CLI generation and the Stable generated-parser ABI do not change.
|