ibex 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +95 -272
- data/docs/architecture.md +187 -35
- data/docs/bison-import.md +103 -0
- data/docs/comparison-policy.md +127 -0
- data/docs/configuration-model.md +59 -0
- data/docs/configuration-report.md +28 -0
- data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
- data/docs/conflict-explanation-study.md +62 -0
- data/docs/construction-profiling.md +157 -0
- data/docs/cst-migration.md +3 -4
- data/docs/cst.md +24 -0
- data/docs/decisions/0000-template.md +24 -0
- data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
- data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
- data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
- data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
- data/docs/decisions/0005-contained-grammar-composition.md +34 -0
- data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
- data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
- data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
- data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
- data/docs/decisions/0010-committed-runtime-observation.md +34 -0
- data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
- data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
- data/docs/decisions/0013-transactional-generation-publication.md +37 -0
- data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
- data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
- data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
- data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
- data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
- data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
- data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
- data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
- data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
- data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
- data/docs/decisions/0024-direct-ielr-construction.md +42 -0
- data/docs/decisions/README.md +74 -0
- data/docs/declarative-configuration.md +262 -0
- data/docs/development.md +179 -2
- data/docs/direct-ielr-decision.md +120 -0
- data/docs/direct-multi-entry-decision.md +82 -0
- data/docs/editor-setup.md +2 -1
- data/docs/error-ux-review-rubric-v1.md +117 -0
- data/docs/error-ux-reviews/v1/records/README.md +22 -0
- data/docs/error-ux-round2-review-status-v1.json +27 -0
- data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
- data/docs/error-ux-round2-v1.json +1581 -0
- data/docs/error-ux-round2.md +111 -0
- data/docs/error-ux.md +54 -0
- data/docs/getting-started.md +80 -0
- data/docs/grammar-reference.md +184 -13
- data/docs/ielr-design.md +43 -0
- data/docs/ielr.md +92 -0
- data/docs/ja/bison-import.md +51 -0
- data/docs/ja/stable-api.md +53 -0
- data/docs/lexer-construction-profile.md +109 -0
- data/docs/lexer-migration.md +5 -0
- data/docs/maturity.md +128 -0
- data/docs/project-site-strategy.md +54 -0
- data/docs/racc-migration-evidence.md +45 -0
- data/docs/racc-migration.md +11 -0
- data/docs/release-readiness.md +126 -68
- data/docs/repair-semantics.md +123 -0
- data/docs/runtime-abi-evolution.md +307 -0
- data/docs/stability.md +144 -34
- data/docs/status.md +43 -0
- data/docs/syntax-sessions.md +183 -0
- data/docs/table-artifact.md +125 -0
- data/docs/test-interactions.md +219 -0
- data/docs/verification-report.md +137 -0
- data/docs/verifier-trust-boundary.md +199 -0
- data/docs/workloads.md +132 -0
- data/lib/ibex/analysis/digraph.rb +128 -0
- data/lib/ibex/analysis.rb +1 -0
- data/lib/ibex/bison_import/importer.rb +516 -0
- data/lib/ibex/bison_import/tokenizer.rb +260 -0
- data/lib/ibex/bison_import.rb +200 -0
- data/lib/ibex/bounded_subprocess.rb +171 -0
- data/lib/ibex/cli/ambiguity.rb +14 -5
- data/lib/ibex/cli/analysis.rb +134 -0
- data/lib/ibex/cli/bison_import.rb +121 -0
- data/lib/ibex/cli/config.rb +128 -0
- data/lib/ibex/cli/diagnostics.rb +31 -11
- data/lib/ibex/cli/documentation.rb +11 -8
- data/lib/ibex/cli/equiv.rb +169 -0
- data/lib/ibex/cli/error_messages.rb +11 -4
- data/lib/ibex/cli/explain.rb +14 -5
- data/lib/ibex/cli/fix.rb +203 -0
- data/lib/ibex/cli/formatting.rb +11 -10
- data/lib/ibex/cli/fuzz.rb +200 -0
- data/lib/ibex/cli/fuzz_regressions.rb +145 -0
- data/lib/ibex/cli/generation_artifacts.rb +53 -2
- data/lib/ibex/cli/generation_error_messages.rb +1 -1
- data/lib/ibex/cli/grammar_tests.rb +32 -14
- data/lib/ibex/cli/ir_tools.rb +5 -61
- data/lib/ibex/cli/outputs.rb +60 -33
- data/lib/ibex/cli/reduce.rb +202 -0
- data/lib/ibex/cli/reduce_reporting.rb +71 -0
- data/lib/ibex/cli/samples.rb +49 -23
- data/lib/ibex/cli/verify.rb +116 -0
- data/lib/ibex/cli/watch.rb +6 -5
- data/lib/ibex/cli.rb +357 -56
- data/lib/ibex/codegen/action_locations.rb +1 -1
- data/lib/ibex/codegen/action_method_source.rb +5 -6
- data/lib/ibex/codegen/action_source.rb +3 -2
- data/lib/ibex/codegen/ambiguity.rb +5 -4
- data/lib/ibex/codegen/explain.rb +123 -55
- data/lib/ibex/codegen/generated_action_abi.rb +10 -5
- data/lib/ibex/codegen/rbs.rb +15 -7
- data/lib/ibex/codegen/report.rb +25 -13
- data/lib/ibex/codegen/ruby.rb +28 -64
- data/lib/ibex/codegen/ruby_actions.rb +9 -6
- data/lib/ibex/codegen/ruby_syntax.rb +15 -9
- data/lib/ibex/configuration/analysis_grammar.rb +35 -0
- data/lib/ibex/configuration/explanation.rb +421 -0
- data/lib/ibex/configuration/inspector.rb +232 -0
- data/lib/ibex/configuration.rb +580 -0
- data/lib/ibex/coverage/collector.rb +21 -11
- data/lib/ibex/coverage/event_stream.rb +13 -7
- data/lib/ibex/coverage/report.rb +26 -18
- data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
- data/lib/ibex/delta_reducer.rb +99 -0
- data/lib/ibex/diff.rb +140 -0
- data/lib/ibex/equiv/machine.rb +135 -0
- data/lib/ibex/equiv.rb +373 -0
- data/lib/ibex/fix.rb +557 -0
- data/lib/ibex/frontend/ast.rb +22 -2
- data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
- data/lib/ibex/frontend/diagnostic.rb +1 -1
- data/lib/ibex/frontend/formatter.rb +72 -26
- data/lib/ibex/frontend/generated_parser.rb +147 -117
- data/lib/ibex/frontend/generated_parser_base.rb +7 -3
- data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
- data/lib/ibex/frontend/generation.rb +1 -1
- data/lib/ibex/frontend/parser/declarations.rb +17 -2
- data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
- data/lib/ibex/frontend/regenerator.rb +1 -0
- data/lib/ibex/frontend/resolution.rb +2 -1
- data/lib/ibex/frontend/resolver.rb +1 -1
- data/lib/ibex/frontend/rule_documentation.rb +2 -1
- data/lib/ibex/frontend/source_cursor.rb +2 -2
- data/lib/ibex/frontend/source_span.rb +9 -1
- data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
- data/lib/ibex/frontend/token_adapter.rb +10 -0
- data/lib/ibex/frontend.rb +1 -0
- data/lib/ibex/fuzz.rb +219 -0
- data/lib/ibex/generation_input.rb +1 -1
- data/lib/ibex/generation_manifest.rb +36 -25
- data/lib/ibex/generation_transaction.rb +1 -1
- data/lib/ibex/generation_transaction_recovery.rb +4 -4
- data/lib/ibex/generation_transaction_validation.rb +2 -2
- data/lib/ibex/grammar_tests.rb +1 -1
- data/lib/ibex/ir/automaton_ir.rb +61 -19
- data/lib/ibex/ir/grammar_ir.rb +76 -45
- data/lib/ibex/ir/lexer_ir.rb +5 -5
- data/lib/ibex/ir/parser_contract.rb +135 -0
- data/lib/ibex/ir/serialize.rb +203 -76
- data/lib/ibex/ir/validator/automaton.rb +64 -34
- data/lib/ibex/ir/validator/base.rb +51 -33
- data/lib/ibex/ir/validator/grammar.rb +113 -83
- data/lib/ibex/ir/validator/lexer.rb +2 -2
- data/lib/ibex/ir/validator.rb +2 -1
- data/lib/ibex/ir.rb +16 -3
- data/lib/ibex/lalr/build_metrics.rb +50 -2
- data/lib/ibex/lalr/builder.rb +191 -56
- data/lib/ibex/lalr/conflict_search.rb +15 -4
- data/lib/ibex/lalr/counterexample.rb +19 -14
- data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
- data/lib/ibex/lalr/goto_follows.rb +229 -0
- data/lib/ibex/lalr/ielr/annotator.rb +214 -0
- data/lib/ibex/lalr/ielr/bits.rb +28 -0
- data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
- data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
- data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
- data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
- data/lib/ibex/lalr/ielr/split_state.rb +20 -0
- data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
- data/lib/ibex/lalr/ielr_partition.rb +22 -8
- data/lib/ibex/lalr/inadequacy_report.rb +50 -0
- data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
- data/lib/ibex/lalr/lr0_collection.rb +121 -0
- data/lib/ibex/lalr/unreachable_states.rb +80 -0
- data/lib/ibex/lalr.rb +52 -1
- data/lib/ibex/location.rb +3 -3
- data/lib/ibex/lsp/document_handlers.rb +9 -7
- data/lib/ibex/lsp/initialization_handlers.rb +9 -7
- data/lib/ibex/lsp/navigation_handlers.rb +25 -11
- data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
- data/lib/ibex/lsp/position_codec.rb +6 -2
- data/lib/ibex/lsp/request_handlers.rb +3 -2
- data/lib/ibex/lsp/request_support.rb +14 -7
- data/lib/ibex/lsp/server.rb +13 -11
- data/lib/ibex/lsp/symbol_index.rb +13 -13
- data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
- data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
- data/lib/ibex/lsp/transport.rb +13 -3
- data/lib/ibex/lsp/workspace.rb +5 -4
- data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
- data/lib/ibex/lsp.rb +1 -0
- data/lib/ibex/messages/en.yml +28 -0
- data/lib/ibex/messages/ja.yml +28 -0
- data/lib/ibex/messages.rb +68 -0
- data/lib/ibex/metrics.rb +175 -0
- data/lib/ibex/normalize/declarations.rb +2 -0
- data/lib/ibex/normalize/diagnostics.rb +63 -38
- data/lib/ibex/normalize/expander.rb +5 -4
- data/lib/ibex/normalize/expression.rb +8 -6
- data/lib/ibex/normalize/grammar_builder.rb +26 -0
- data/lib/ibex/normalize/inline_expansion.rb +100 -49
- data/lib/ibex/normalize/lexer.rb +3 -3
- data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
- data/lib/ibex/normalize/parameter_lowering.rb +40 -23
- data/lib/ibex/normalize/parameters.rb +31 -4
- data/lib/ibex/normalize/parser_configuration.rb +70 -0
- data/lib/ibex/normalize.rb +10 -20
- data/lib/ibex/racc_migration/report.rb +6 -3
- data/lib/ibex/rake_task.rb +1 -1
- data/lib/ibex/samples.rb +43 -13
- data/lib/ibex/table_artifact/builder.rb +266 -0
- data/lib/ibex/table_artifact/cst_projection.rb +70 -0
- data/lib/ibex/table_artifact/document.rb +39 -0
- data/lib/ibex/table_artifact/executor.rb +187 -0
- data/lib/ibex/table_artifact/serializer.rb +57 -0
- data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
- data/lib/ibex/table_artifact/validator/support.rb +86 -0
- data/lib/ibex/table_artifact/validator/tables.rb +238 -0
- data/lib/ibex/table_artifact/validator.rb +253 -0
- data/lib/ibex/table_artifact.rb +88 -0
- data/lib/ibex/table_simulation/result.rb +6 -2
- data/lib/ibex/table_simulation/step.rb +3 -1
- data/lib/ibex/tables.rb +14 -0
- data/lib/ibex/verifiable_generation_bundle.rb +86 -0
- data/lib/ibex/verification_report/builder.rb +159 -0
- data/lib/ibex/verification_report/canonical_ir.rb +123 -0
- data/lib/ibex/verification_report/logical_path.rb +68 -0
- data/lib/ibex/verification_report/validator.rb +329 -0
- data/lib/ibex/verification_report.rb +69 -0
- data/lib/ibex/verify/action_correspondence.rb +138 -0
- data/lib/ibex/verify/language_witness.rb +295 -0
- data/lib/ibex/verify/reference_collection.rb +190 -0
- data/lib/ibex/verify/result.rb +73 -0
- data/lib/ibex/verify/verifier.rb +598 -0
- data/lib/ibex/verify.rb +15 -0
- data/lib/ibex/version.rb +1 -1
- data/lib/ibex/watch/runner.rb +10 -1
- data/lib/ibex/watch/source_snapshot.rb +13 -8
- data/lib/ibex.rb +10 -0
- data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
- data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
- data/schema/bison-import-v1.schema.json +106 -0
- data/schema/conflict-explanation-review-v1.schema.json +196 -0
- data/schema/conflict-explanation-study-v1.schema.json +528 -0
- data/schema/construction-profile-v1.schema.json +779 -0
- data/schema/diff-v1.schema.json +41 -0
- data/schema/direct-ielr-decision-v1.schema.json +200 -0
- data/schema/equiv-v1.schema.json +64 -0
- data/schema/error-ux-review-v1.schema.json +609 -0
- data/schema/error-ux-round2-review-v1.schema.json +191 -0
- data/schema/error-ux-round2-v1.schema.json +519 -0
- data/schema/explain-v1.schema.json +63 -1
- data/schema/fix-v1.schema.json +64 -0
- data/schema/fix-v2.schema.json +66 -0
- data/schema/fix-v3.schema.json +101 -0
- data/schema/fuzz-regression-v1.schema.json +79 -0
- data/schema/fuzz-v1.schema.json +114 -0
- data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
- data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
- data/schema/grammar-ir.schema.json +271 -0
- data/schema/ielr-benchmark-v1.schema.json +85 -0
- data/schema/lexer-profile-v1.schema.json +491 -0
- data/schema/metrics-v1.schema.json +60 -0
- data/schema/reduce-v1.schema.json +22 -0
- data/schema/reduce-v2.schema.json +63 -0
- data/schema/table-artifact-v1.schema.json +377 -0
- data/schema/verification-report-v1.schema.json +222 -0
- data/schema/verify-v1.schema.json +42 -0
- data/sig/ibex/analysis/digraph.rbs +19 -0
- data/sig/ibex/bison_import/importer.rbs +140 -0
- data/sig/ibex/bison_import/tokenizer.rbs +91 -0
- data/sig/ibex/bison_import.rbs +95 -0
- data/sig/ibex/bounded_subprocess.rbs +76 -0
- data/sig/ibex/cli/ambiguity.rbs +6 -0
- data/sig/ibex/cli/analysis.rbs +43 -0
- data/sig/ibex/cli/bison_import.rbs +32 -0
- data/sig/ibex/cli/config.rbs +35 -0
- data/sig/ibex/cli/diagnostics.rbs +16 -3
- data/sig/ibex/cli/documentation.rbs +5 -1
- data/sig/ibex/cli/equiv.rbs +49 -0
- data/sig/ibex/cli/error_messages.rbs +6 -0
- data/sig/ibex/cli/explain.rbs +6 -0
- data/sig/ibex/cli/fix.rbs +53 -0
- data/sig/ibex/cli/formatting.rbs +5 -1
- data/sig/ibex/cli/fuzz.rbs +47 -0
- data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
- data/sig/ibex/cli/generation_artifacts.rbs +21 -2
- data/sig/ibex/cli/generation_error_messages.rbs +2 -2
- data/sig/ibex/cli/grammar_tests.rbs +15 -3
- data/sig/ibex/cli/ir_tools.rbs +7 -19
- data/sig/ibex/cli/outputs.rbs +9 -3
- data/sig/ibex/cli/reduce.rbs +60 -0
- data/sig/ibex/cli/reduce_reporting.rbs +27 -0
- data/sig/ibex/cli/samples.rbs +8 -0
- data/sig/ibex/cli/verify.rbs +28 -0
- data/sig/ibex/cli/watch.rbs +2 -2
- data/sig/ibex/cli.rbs +79 -5
- data/sig/ibex/codegen/action_method_source.rbs +4 -4
- data/sig/ibex/codegen/explain.rbs +45 -25
- data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
- data/sig/ibex/codegen/rbs.rbs +12 -10
- data/sig/ibex/codegen/report.rbs +12 -12
- data/sig/ibex/codegen/ruby.rbs +18 -15
- data/sig/ibex/codegen/ruby_actions.rbs +6 -6
- data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
- data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
- data/sig/ibex/configuration/explanation.rbs +195 -0
- data/sig/ibex/configuration/inspector.rbs +53 -0
- data/sig/ibex/configuration.rbs +242 -0
- data/sig/ibex/coverage/collector.rbs +16 -14
- data/sig/ibex/coverage/event_stream.rbs +12 -10
- data/sig/ibex/coverage/report.rbs +20 -18
- data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
- data/sig/ibex/delta_reducer.rbs +38 -0
- data/sig/ibex/diff.rbs +47 -0
- data/sig/ibex/equiv/machine.rbs +54 -0
- data/sig/ibex/equiv.rbs +129 -0
- data/sig/ibex/fix.rbs +138 -0
- data/sig/ibex/frontend/ast.rbs +33 -7
- data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
- data/sig/ibex/frontend/diagnostic.rbs +2 -2
- data/sig/ibex/frontend/formatter.rbs +38 -30
- data/sig/ibex/frontend/generated_parser.rbs +87 -77
- data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
- data/sig/ibex/frontend/parser/declarations.rbs +3 -0
- data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
- data/sig/ibex/frontend/resolution.rbs +3 -2
- data/sig/ibex/frontend/source_cursor.rbs +2 -2
- data/sig/ibex/frontend/source_span.rbs +5 -2
- data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
- data/sig/ibex/frontend/token_adapter.rbs +6 -0
- data/sig/ibex/fuzz.rbs +71 -0
- data/sig/ibex/generation_input.rbs +2 -2
- data/sig/ibex/generation_manifest.rbs +35 -32
- data/sig/ibex/generation_transaction_recovery.rbs +2 -2
- data/sig/ibex/generation_transaction_validation.rbs +2 -2
- data/sig/ibex/grammar_tests.rbs +2 -2
- data/sig/ibex/ir/automaton_ir.rbs +30 -11
- data/sig/ibex/ir/grammar_ir.rbs +41 -27
- data/sig/ibex/ir/lexer_ir.rbs +8 -8
- data/sig/ibex/ir/parser_contract.rbs +69 -0
- data/sig/ibex/ir/serialize.rbs +63 -28
- data/sig/ibex/ir/validator/automaton.rbs +47 -44
- data/sig/ibex/ir/validator/base.rbs +35 -31
- data/sig/ibex/ir/validator/grammar.rbs +78 -67
- data/sig/ibex/ir/validator/lexer.rbs +4 -4
- data/sig/ibex/ir.rbs +9 -3
- data/sig/ibex/lalr/build_metrics.rbs +42 -2
- data/sig/ibex/lalr/builder.rbs +75 -24
- data/sig/ibex/lalr/conflict_search.rbs +5 -2
- data/sig/ibex/lalr/counterexample.rbs +17 -14
- data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
- data/sig/ibex/lalr/goto_follows.rbs +89 -0
- data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
- data/sig/ibex/lalr/ielr/bits.rbs +14 -0
- data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
- data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
- data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
- data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
- data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
- data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
- data/sig/ibex/lalr/ielr_partition.rbs +17 -5
- data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
- data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
- data/sig/ibex/lalr/lr0_collection.rbs +50 -0
- data/sig/ibex/lalr/unreachable_states.rbs +25 -0
- data/sig/ibex/lalr.rbs +11 -1
- data/sig/ibex/location.rbs +6 -6
- data/sig/ibex/lsp/document_handlers.rbs +8 -8
- data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
- data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
- data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
- data/sig/ibex/lsp/position_codec.rbs +8 -4
- data/sig/ibex/lsp/request_handlers.rbs +4 -4
- data/sig/ibex/lsp/request_support.rbs +8 -8
- data/sig/ibex/lsp/server.rbs +16 -16
- data/sig/ibex/lsp/symbol_index.rbs +20 -21
- data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
- data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
- data/sig/ibex/lsp/transport.rbs +18 -6
- data/sig/ibex/lsp/workspace.rbs +3 -2
- data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
- data/sig/ibex/messages.rbs +26 -0
- data/sig/ibex/metrics.rbs +50 -0
- data/sig/ibex/normalize/diagnostics.rbs +4 -4
- data/sig/ibex/normalize/expander.rbs +2 -2
- data/sig/ibex/normalize/expression.rbs +10 -8
- data/sig/ibex/normalize/grammar_builder.rbs +11 -0
- data/sig/ibex/normalize/inline_expansion.rbs +59 -50
- data/sig/ibex/normalize/lexer.rbs +4 -4
- data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
- data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
- data/sig/ibex/normalize/parameters.rbs +13 -8
- data/sig/ibex/normalize/parser_configuration.rbs +23 -0
- data/sig/ibex/normalize.rbs +11 -5
- data/sig/ibex/racc_migration/report.rbs +8 -6
- data/sig/ibex/rake_task.rbs +1 -1
- data/sig/ibex/samples.rbs +13 -4
- data/sig/ibex/table_artifact/builder.rbs +93 -0
- data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
- data/sig/ibex/table_artifact/document.rbs +27 -0
- data/sig/ibex/table_artifact/executor.rbs +66 -0
- data/sig/ibex/table_artifact/serializer.rbs +25 -0
- data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
- data/sig/ibex/table_artifact/validator/support.rbs +46 -0
- data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
- data/sig/ibex/table_artifact/validator.rbs +66 -0
- data/sig/ibex/table_artifact.rbs +32 -0
- data/sig/ibex/table_simulation/result.rbs +8 -4
- data/sig/ibex/table_simulation/step.rbs +4 -2
- data/sig/ibex/tables.rbs +10 -9
- data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
- data/sig/ibex/verification_report/builder.rbs +53 -0
- data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
- data/sig/ibex/verification_report/logical_path.rbs +34 -0
- data/sig/ibex/verification_report/validator.rbs +114 -0
- data/sig/ibex/verification_report.rbs +44 -0
- data/sig/ibex/verify/action_correspondence.rbs +50 -0
- data/sig/ibex/verify/language_witness.rbs +112 -0
- data/sig/ibex/verify/reference_collection.rbs +63 -0
- data/sig/ibex/verify/result.rbs +52 -0
- data/sig/ibex/verify/verifier.rbs +170 -0
- data/sig/ibex/verify.rbs +6 -0
- data/sig/ibex/watch/runner.rbs +12 -2
- data/sig/ibex/watch/source_snapshot.rbs +13 -9
- metadata +232 -12
- data/lib/ibex/ir/migration.rb +0 -120
- data/sig/ibex/ir/migration.rbs +0 -34
|
@@ -0,0 +1,307 @@
|
|
|
1
|
+
# Runtime ABI evolution
|
|
2
|
+
|
|
3
|
+
This policy separates durable serialized input, generated parser tables, and
|
|
4
|
+
the Ruby runtime API. The current format is the only supported Grammar and
|
|
5
|
+
Automaton IR boundary; the numeric field is a wire discriminator, not a promise
|
|
6
|
+
to read old files.
|
|
7
|
+
|
|
8
|
+
<!-- ibex-runtime-abi-contract:start -->
|
|
9
|
+
```yaml
|
|
10
|
+
contract_version: 1
|
|
11
|
+
ir:
|
|
12
|
+
grammar:
|
|
13
|
+
current_writer: 1
|
|
14
|
+
readable: [1]
|
|
15
|
+
migrations: []
|
|
16
|
+
preserve_loaded_version: false
|
|
17
|
+
automaton:
|
|
18
|
+
current_writer: 1
|
|
19
|
+
readable: [1]
|
|
20
|
+
migrations: []
|
|
21
|
+
preserve_loaded_version: false
|
|
22
|
+
lexer:
|
|
23
|
+
current_writer: 1
|
|
24
|
+
readable: [1]
|
|
25
|
+
migrations: []
|
|
26
|
+
standalone: true
|
|
27
|
+
embedded_in_grammar: true
|
|
28
|
+
parser_tables:
|
|
29
|
+
current_writer: 6
|
|
30
|
+
readable: [6]
|
|
31
|
+
cst_readable: [6]
|
|
32
|
+
fail_before_input: true
|
|
33
|
+
versions:
|
|
34
|
+
generator: "0.3.0"
|
|
35
|
+
runtime: "0.3.0"
|
|
36
|
+
runtime_dependency: "~> 0.3.0"
|
|
37
|
+
runtime_paths:
|
|
38
|
+
- .github/pull_request_template.md
|
|
39
|
+
- .github/workflows/main.yml
|
|
40
|
+
- Rakefile
|
|
41
|
+
- docs/runtime-abi-evolution.md
|
|
42
|
+
- docs/test-interactions.md
|
|
43
|
+
- tool/quality/runtime_abi.rb
|
|
44
|
+
- tool/quality/runtime_abi/**/*
|
|
45
|
+
- test/quality/runtime_abi*_test.rb
|
|
46
|
+
- test/support/runtime_abi_test_project.rb
|
|
47
|
+
- test/fixtures/runtime_abi/**/*
|
|
48
|
+
- ibex.gemspec
|
|
49
|
+
- ibex-runtime.gemspec
|
|
50
|
+
- lib/ibex/version.rb
|
|
51
|
+
- lib/ibex/runtime.rb
|
|
52
|
+
- lib/ibex/runtime/**/*
|
|
53
|
+
- lib/ibex/runtime/version.rb
|
|
54
|
+
- lib/ibex/tables.rb
|
|
55
|
+
- lib/ibex/tables/**/*
|
|
56
|
+
- sig/ibex/tables.rbs
|
|
57
|
+
- sig/ibex/runtime.rbs
|
|
58
|
+
- sig/ibex/runtime/**/*
|
|
59
|
+
- sig/ibex/tables/**/*
|
|
60
|
+
- lib/ibex/codegen.rb
|
|
61
|
+
- lib/ibex/codegen/**/*
|
|
62
|
+
- sig/ibex/codegen.rbs
|
|
63
|
+
- sig/ibex/codegen/**/*
|
|
64
|
+
- lib/ibex/frontend/generated_parser.rb
|
|
65
|
+
- lib/ibex/ir.rb
|
|
66
|
+
- lib/ibex/ir/**/*
|
|
67
|
+
- sig/ibex/ir.rbs
|
|
68
|
+
- sig/ibex/ir/**/*
|
|
69
|
+
- schema/grammar-ir.schema.json
|
|
70
|
+
- schema/grammar-ir-foundation.schema.json
|
|
71
|
+
- schema/grammar-ir-extensions.schema.json
|
|
72
|
+
- schema/automaton-ir.schema.json
|
|
73
|
+
- schema/automaton-ir-definitions.schema.json
|
|
74
|
+
- schema/lexer-ir-v*.schema.json
|
|
75
|
+
- schema/cst-v*.json
|
|
76
|
+
- test/matrix.yml
|
|
77
|
+
- test/support/matrix_contract.rb
|
|
78
|
+
- test/support/matrix_runner.rb
|
|
79
|
+
- test/tooling/matrix_runner_test.rb
|
|
80
|
+
- tool/quality/golden.rb
|
|
81
|
+
- test/golden/**/*
|
|
82
|
+
assessment:
|
|
83
|
+
states: [compatible, breaking, not_applicable]
|
|
84
|
+
surfaces: [parser_table, grammar_ir, automaton_ir, lexer_ir, runtime_api, embedded_runtime, generation_metadata, cst, test_matrix, policy, none]
|
|
85
|
+
abi_choices: [current_contract, new_table_format, new_ir_version, new_runtime_major, sidecar, none]
|
|
86
|
+
regeneration: [required, not_required, not_applicable]
|
|
87
|
+
required_fields: [state, surfaces, abi_choice, regeneration, rationale, affected_interactions, evidence, tests, verification]
|
|
88
|
+
```
|
|
89
|
+
<!-- ibex-runtime-abi-contract:end -->
|
|
90
|
+
|
|
91
|
+
The fenced record above is validated against the constants, schemas, gemspecs,
|
|
92
|
+
test declaration, and pull-request policy in this repository. Edit it only as
|
|
93
|
+
part of the implementation change that moves the corresponding boundary.
|
|
94
|
+
|
|
95
|
+
## IR read, write, migration, and freeze policy
|
|
96
|
+
|
|
97
|
+
| Contract | Current writer | Current reader | Migration | Freeze |
|
|
98
|
+
| --- | ---: | --- | --- | --- |
|
|
99
|
+
| Grammar IR | current format | current format only | none | the current schema is closed; older documents are rejected |
|
|
100
|
+
| Automaton IR | current format | current format only | none | the current schema is closed; older documents are rejected |
|
|
101
|
+
| Lexer IR | schema 1 | schema 1 | none | the closed schema changes only through a new schema |
|
|
102
|
+
|
|
103
|
+
The generator and runtime intentionally share one Grammar and Automaton IR
|
|
104
|
+
format. Older documents are rejected before construction; there is no in-process
|
|
105
|
+
IR migration command or compatibility reader. The current Grammar IR carries a
|
|
106
|
+
required root `parser_contract` whose individual fields
|
|
107
|
+
are either explicit with source locations or explicitly unspecified. Automaton
|
|
108
|
+
IR embeds the current Grammar IR, includes that contract in `grammar_digest`,
|
|
109
|
+
and records `entry_construction` as `shared` or `isolated`. Unknown versions and
|
|
110
|
+
construction values are rejected.
|
|
111
|
+
|
|
112
|
+
An extended root `parser ... end` declaration now writes explicit
|
|
113
|
+
`parser.algorithm` and `parser.entries` members into the current contract. The
|
|
114
|
+
declaration is generator input only: it selects existing construction paths and
|
|
115
|
+
does not add a runtime lookup, generated parser method, action ABI field, or
|
|
116
|
+
parser-table datum. Parser-table format v6 therefore remains the sole writer,
|
|
117
|
+
and declaration-free grammar, generated Ruby, and table golden bytes remain
|
|
118
|
+
unchanged.
|
|
119
|
+
|
|
120
|
+
Resuming from the current Grammar IR resolves explicit contract values through the same
|
|
121
|
+
typed configuration algebra as CLI values; a matching CLI value is accepted and
|
|
122
|
+
a conflicting canonical-generation value fails at the recorded declaration
|
|
123
|
+
location. Explicit analysis or grammar-test algorithm selection is reported as
|
|
124
|
+
noncanonical and never reclassified as canonical generation. Resuming from a constructed Automaton IR rejects
|
|
125
|
+
construction flags because they cannot rebuild the embedded tables. Its manifest
|
|
126
|
+
keeps the grammar contract, embedded construction facts, and effective codegen
|
|
127
|
+
configuration separate. These schema additions do not change parser-table
|
|
128
|
+
format v6 or the runtime action ABI.
|
|
129
|
+
|
|
130
|
+
Lexer IR schema 1 is independently versioned. It can be validated and serialized as
|
|
131
|
+
a standalone `ibex_ir: lexer` document, and Grammar IR can carry the same
|
|
132
|
+
versioned document in its optional `lexer` field. There is no Lexer IR
|
|
133
|
+
migration today. Changing lexer rule meaning or its required shape therefore
|
|
134
|
+
requires Lexer IR v2; changing only parser construction does not.
|
|
135
|
+
|
|
136
|
+
The JSON schemas under `schema/` are the published closed shapes. An optional
|
|
137
|
+
field may be added to a future version, but an existing closed schema is not
|
|
138
|
+
made open to avoid assigning new meaning to an old version.
|
|
139
|
+
|
|
140
|
+
## Parser-table formats
|
|
141
|
+
|
|
142
|
+
The generator emits format v6, and the runtime recognizes only that current
|
|
143
|
+
format. Older generated tables fail before input so the runtime does not carry
|
|
144
|
+
historical action-ABI branches indefinitely:
|
|
145
|
+
|
|
146
|
+
| Format | Non-CST runtime contract | CST contract | Current writer |
|
|
147
|
+
| ---: | --- | --- | --- |
|
|
148
|
+
| v6 | retains the preceding action contracts | structured CST metadata is accepted | yes |
|
|
149
|
+
|
|
150
|
+
Plain and compact tables are encodings of the same lookup contract, not
|
|
151
|
+
separate ABI versions. A change to compression is format-preserving only when
|
|
152
|
+
every known and unknown token retains the same shift, reduce, accept, or error
|
|
153
|
+
result. Required table shape, action calling convention, marker meaning, or
|
|
154
|
+
runtime lookup meaning requires a new parser-table format.
|
|
155
|
+
|
|
156
|
+
Every pull, `yyparse`, push, finish, and syntax entry starts a parser session by
|
|
157
|
+
validating its table object. A missing or unsupported `format_version`, an
|
|
158
|
+
invalid generated action marker combination, or a legacy/boolean CST shape is
|
|
159
|
+
rejected before the runtime asks a pull lexer for a token or consumes a
|
|
160
|
+
caller-supplied push token. Current generated action marker combinations are
|
|
161
|
+
checked at that boundary. The error directs the application to regenerate.
|
|
162
|
+
Validation cannot undo side effects the caller performed while constructing an
|
|
163
|
+
argument before calling the runtime.
|
|
164
|
+
|
|
165
|
+
Regeneration is required when:
|
|
166
|
+
|
|
167
|
+
- a parser has no table-format version or uses a version the installed runtime
|
|
168
|
+
does not list;
|
|
169
|
+
- any CST-aware parser predates structured format-v6 CST metadata;
|
|
170
|
+
- generated action markers do not satisfy their current calling contract;
|
|
171
|
+
- an embedded parser must receive runtime fixes or a newer runtime ABI; or
|
|
172
|
+
- a release note for a new format explicitly retires an older reader.
|
|
173
|
+
|
|
174
|
+
Non-CST v1-v5 tables are outside the pre-v1 runtime compatibility obligation
|
|
175
|
+
and require regeneration. Applications that need to keep an old generated
|
|
176
|
+
artifact can pin the matching pre-v1 runtime package instead of making the
|
|
177
|
+
current reader retain every historical table branch.
|
|
178
|
+
|
|
179
|
+
## Generator and runtime versions
|
|
180
|
+
|
|
181
|
+
| Generated artifact | Runtime used | Status |
|
|
182
|
+
| --- | --- | --- |
|
|
183
|
+
| Ibex 0.3.0 non-embedded output (table v6) | `ibex-runtime` 0.3.0 | supported and covered by packaging/runtime tests |
|
|
184
|
+
| Ibex 0.3.0 non-embedded output | a future version admitted by `~> 0.3.0` | dependency resolution permits it; compatibility is an obligation of that future release, not current execution evidence |
|
|
185
|
+
| Ibex 0.3.0 embedded output | runtime sources copied by Ibex 0.3.0 | self-contained and covered by packaging/runtime tests |
|
|
186
|
+
| Ibex 0.2.0 non-embedded output (table v6) | `ibex-runtime` 0.2.0 | supported and covered by packaging/runtime tests |
|
|
187
|
+
| Ibex 0.2.0 non-embedded output | a future version admitted by `~> 0.2.0` | dependency resolution permits it; compatibility is an obligation of that future release, not current execution evidence |
|
|
188
|
+
| Ibex 0.2.0 embedded output | runtime sources copied by Ibex 0.2.0 | self-contained and covered by packaging/runtime tests |
|
|
189
|
+
| old non-CST or CST table v1-v5 | `ibex-runtime` 0.3.0 | rejected before input; regenerate or pin the matching pre-v1 runtime |
|
|
190
|
+
| table v7 or later | `ibex-runtime` 0.3.0 | rejected before input; upgrade the runtime or regenerate to a supported format |
|
|
191
|
+
|
|
192
|
+
`ibex` 0.3.0 declares `ibex-runtime ~> 0.3.0`; in RubyGems terms that admits
|
|
193
|
+
runtime releases from 0.3.0 up to, but not including, 0.4.0. The previous
|
|
194
|
+
0.2.0 line remains a historical compatibility boundary. The table format is
|
|
195
|
+
still the executable compatibility check. A package requirement alone does not
|
|
196
|
+
prove that an unpublished runtime can execute a generated parser.
|
|
197
|
+
|
|
198
|
+
Normal generated output requires `ibex/runtime`. Embedded output concatenates
|
|
199
|
+
the runtime files named by `Runtime::EmbeddedSource` in dependency order. Thus
|
|
200
|
+
both modes use the same repository sources at generation time. Embedded output
|
|
201
|
+
does not consult an installed runtime and does not acquire later security,
|
|
202
|
+
correctness, or performance fixes: the application that chooses embedding owns
|
|
203
|
+
regeneration and redeployment. Loading embedded and installed copies into one
|
|
204
|
+
process is not a supported upgrade mechanism.
|
|
205
|
+
|
|
206
|
+
### Syntax-session ABI classification
|
|
207
|
+
|
|
208
|
+
Adding `SyntaxSession` and its syntax-only repair proposal keeps parser-table
|
|
209
|
+
format v6 unchanged and adds methods and immutable result types to the current Ruby runtime API. Existing
|
|
210
|
+
non-embedded generated source remains byte-identical and acquires the API from
|
|
211
|
+
a compatible runtime-package upgrade. Embedded generated source copies runtime
|
|
212
|
+
implementation bytes, so its bytes change and regeneration plus redeployment
|
|
213
|
+
is required to acquire the API or later session correctness fixes. Existing
|
|
214
|
+
embedded parsers that are not regenerated retain their previous parsing
|
|
215
|
+
behavior but do not gain the new façade.
|
|
216
|
+
|
|
217
|
+
The repository contract can represent that combined assessment without
|
|
218
|
+
claiming a table-format bump. The conservative `regeneration: required` value
|
|
219
|
+
applies to the affected embedded artifacts; the rationale records why the
|
|
220
|
+
non-embedded path only needs a runtime upgrade:
|
|
221
|
+
|
|
222
|
+
```yaml
|
|
223
|
+
state: compatible
|
|
224
|
+
surfaces: [runtime_api, embedded_runtime, cst]
|
|
225
|
+
abi_choice: current_contract
|
|
226
|
+
regeneration: required
|
|
227
|
+
rationale: Table v6 is unchanged and the runtime API is additive; embedded output bytes change and must be regenerated to acquire SyntaxSession.
|
|
228
|
+
affected_interactions: [incremental_cst, syntax_session, embedded_runtime]
|
|
229
|
+
evidence: [lib/ibex/runtime/cst/incremental/relexer.rb, lib/ibex/runtime/cst/incremental/session.rb, lib/ibex/runtime/syntax_session.rb, lib/ibex/runtime/parser.rb, lib/ibex/runtime/embedded_source.rb, test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
|
|
230
|
+
tests: [test/runtime/cst_incremental_test.rb, test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
|
|
231
|
+
verification: [bundle exec rake quality:runtime_abi, bundle exec ruby -Itest test/runtime/cst_incremental_test.rb, bundle exec ruby -Itest test/runtime/syntax_session_test.rb, bundle exec ruby -Itest test/packaging/runtime_gem_test.rb]
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
## Sidecar, IR version, or table-format version
|
|
235
|
+
|
|
236
|
+
Choose the boundary by its consumer:
|
|
237
|
+
|
|
238
|
+
- Use a sidecar for optional generation, review, or editor metadata that the
|
|
239
|
+
parser does not need to accept input and whose absence preserves behavior.
|
|
240
|
+
- Use a new Grammar, Automaton, or Lexer IR version when persisted generator
|
|
241
|
+
input changes shape or meaning, even if generated runtime tables do not.
|
|
242
|
+
- Use a new parser-table format when runtime execution needs a required field,
|
|
243
|
+
new action convention, or changed table meaning before or during parsing.
|
|
244
|
+
- Use both IR and table versions when the persisted construction contract and
|
|
245
|
+
the runtime execution contract both change.
|
|
246
|
+
|
|
247
|
+
An optional table field is not automatically safe: if its absence selects new
|
|
248
|
+
behavior or changes execution, it belongs behind a new format. Conversely,
|
|
249
|
+
large diagnostics that never affect execution should not inflate every parser
|
|
250
|
+
table merely to avoid a sidecar.
|
|
251
|
+
|
|
252
|
+
## Pull-request ABI assessment
|
|
253
|
+
|
|
254
|
+
The pull-request template contains a delimited YAML assessment. CI parses only
|
|
255
|
+
that block and requires the exact declared fields when a changed path matches
|
|
256
|
+
`runtime_paths` above. Free-form prose outside the block is not evidence.
|
|
257
|
+
|
|
258
|
+
`compatible` means the change keeps a current contract or adds a sidecar.
|
|
259
|
+
`breaking` selects a new table format, IR version, or runtime major version. A
|
|
260
|
+
new table format requires regeneration. An IR-only or Ruby-runtime-API break
|
|
261
|
+
must still make an explicit `required`/`not_required` regeneration decision;
|
|
262
|
+
application migration can be required even when parser regeneration is not.
|
|
263
|
+
`not_applicable` is accepted only when no declared runtime-facing path changed.
|
|
264
|
+
Every changed runtime-facing path must appear in `evidence`; additional evidence
|
|
265
|
+
must be a changed path or an existing regression test. `affected_interactions`
|
|
266
|
+
uses ids from the test-interaction contract, and every listed interaction owns
|
|
267
|
+
at least one path in `tests`. `verification` accepts only reviewed repository
|
|
268
|
+
commands and must run the ABI gate plus the owned tests (or the full suite).
|
|
269
|
+
The validator checks these relationships, while reviewers remain responsible
|
|
270
|
+
for judging the rationale and whether all affected surfaces were identified.
|
|
271
|
+
The deterministic rationale screen checks only structure: the value must be a
|
|
272
|
+
non-empty string containing a Unicode letter, and it must not equal the exact
|
|
273
|
+
repository-owned template sentinel after whitespace normalization. It does not
|
|
274
|
+
classify general placeholders, identifiers, repetition, entropy, or semantic
|
|
275
|
+
substance. Japanese, Arabic, and other writing systems are accepted. Reviewers
|
|
276
|
+
must decide whether the rationale actually explains compatibility and whether
|
|
277
|
+
TODO-like text is acceptable in context.
|
|
278
|
+
|
|
279
|
+
The structured choice table is closed:
|
|
280
|
+
|
|
281
|
+
| `abi_choice` | Required state | Required surface | Regeneration |
|
|
282
|
+
| --- | --- | --- | --- |
|
|
283
|
+
| `current_contract` | `compatible` | one or more concrete surfaces | `required` or `not_required` must be decided |
|
|
284
|
+
| `sidecar` | `compatible` | exactly `generation_metadata` | `not_required` |
|
|
285
|
+
| `new_table_format` | `breaking` | includes `parser_table` | `required` |
|
|
286
|
+
| `new_ir_version` | `breaking` | includes `grammar_ir`, `automaton_ir`, or `lexer_ir` | `required` or `not_required` must be decided |
|
|
287
|
+
| `new_runtime_major` | `breaking` | includes `runtime_api` or `embedded_runtime` | `required` or `not_required` must be decided |
|
|
288
|
+
| `none` | only a non-runtime change | `none` | `not_applicable` |
|
|
289
|
+
|
|
290
|
+
Runtime-facing changes cannot use `none` or `not_applicable`. A rationale must
|
|
291
|
+
replace the exact template sentinel and contain a Unicode letter; required human
|
|
292
|
+
review decides whether it is substantive. Verification commands are parsed as
|
|
293
|
+
arguments, not executed by the validator; shell composition, arbitrary
|
|
294
|
+
commands, unowned test files, and evidence-only README links are rejected.
|
|
295
|
+
|
|
296
|
+
Run the contract-only `bundle exec rake quality:runtime_abi` locally.
|
|
297
|
+
Pull-request CI uses a minimal dedicated job to invoke the separate
|
|
298
|
+
`quality:runtime_abi_pr` entry with its event path and name explicitly. That PR
|
|
299
|
+
gate derives the changed path list from
|
|
300
|
+
the event's exact base and head SHAs without GitHub API writes or elevated
|
|
301
|
+
permissions. It parses `runtime_paths` from the exact
|
|
302
|
+
trusted base revision and unions those paths with the head policy, so a pull
|
|
303
|
+
request cannot exempt its own changes merely by shrinking the head contract.
|
|
304
|
+
Only the first explicit addition of this contract can use the fixed bootstrap
|
|
305
|
+
path set. Repository rules must still require this check and review changes to
|
|
306
|
+
the workflow and validator: code running solely from an untrusted head cannot
|
|
307
|
+
defend against deletion or malicious replacement of the gate that launches it.
|
data/docs/stability.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Stability policy
|
|
3
|
+
description: Compatibility, promotion, deprecation, and feature-development rules for Ibex.
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# Stability, compatibility, and deprecation
|
|
2
7
|
|
|
3
8
|
Ibex separates support level from activation. Opt-in controls do not determine
|
|
@@ -8,7 +13,7 @@ maturity: a documented opt-in API can be Stable, Preview, or Experimental.
|
|
|
8
13
|
| Level | Activation | Guarantee |
|
|
9
14
|
|---|---|---|
|
|
10
15
|
| Stable | Default compatible mode or a documented stable API | Semantic versioning; compatible-mode behavior remains unchanged |
|
|
11
|
-
| Preview |
|
|
16
|
+
| Preview | Feature-specific; compatible/default or explicit as recorded in the maturity audit | Before v1.0: changeable with a migration note; after v1.0: one-minor notice |
|
|
12
17
|
| Experimental | Explicit policy/object or research entry point | May change without notice; budgets and failure modes are part of the experiment |
|
|
13
18
|
|
|
14
19
|
After v1.0, promotion requires representative use in a shadow or gallery
|
|
@@ -20,6 +25,61 @@ support where applicable, and complete public documentation replace the
|
|
|
20
25
|
impossible two-prior-release requirement. Only Stable features may be adopted
|
|
21
26
|
by the production self-hosted grammar.
|
|
22
27
|
|
|
28
|
+
Before v1.0, Preview is an evolving product boundary rather than a general
|
|
29
|
+
backward-compatibility promise. A pre-v1 Preview change is allowed when the
|
|
30
|
+
same change includes a release-note or migration note, updated black-box
|
|
31
|
+
regressions, and a refreshed maturity record. Consumers of a Preview feature
|
|
32
|
+
must therefore pin a pre-release or be prepared to migrate. After v1.0,
|
|
33
|
+
Preview receives the one-minor notice described in the deprecation policy.
|
|
34
|
+
|
|
35
|
+
Middle actions are accepted in compatible/default grammar productions and
|
|
36
|
+
therefore remain under the Stable compatibility lock; their Preview row is
|
|
37
|
+
retained only for traceability while its redundant classification is
|
|
38
|
+
redesigned. Most other Preview surfaces require an extended declaration,
|
|
39
|
+
command, or option. The [maturity audit](maturity.md) records activation and
|
|
40
|
+
Stable overlap separately.
|
|
41
|
+
|
|
42
|
+
## Execution trust is independent of maturity
|
|
43
|
+
|
|
44
|
+
Stable, Preview, and Experimental describe compatibility and promotion, not
|
|
45
|
+
sandbox strength. Static grammar and IR tools do not execute parser actions,
|
|
46
|
+
generated lexer actions, or `header` / `inner` / `footer` sections. Generated
|
|
47
|
+
lexer semantic parses execute parser and lexer actions. Handwritten pull and
|
|
48
|
+
caller-fed semantic parses execute parser actions but do not invoke the
|
|
49
|
+
generated lexer. Generated syntax-only parses suppress parser production
|
|
50
|
+
actions but still execute generated lexer actions. All generated runtime paths
|
|
51
|
+
may load user sections and are trusted application code, not sandboxes.
|
|
52
|
+
|
|
53
|
+
A future nonexecuting syntax profile is a separate product contract. It must
|
|
54
|
+
require a declarative built-in-only lexer and reject all user-code sections;
|
|
55
|
+
neither the Stable batch CST API nor the Experimental incremental API currently
|
|
56
|
+
makes that guarantee. See the [execution trust matrix](../README.md#execution-trust-matrix).
|
|
57
|
+
|
|
58
|
+
## Feature development budget
|
|
59
|
+
|
|
60
|
+
Feature development is not frozen. A `HOLD` release decision blocks publication
|
|
61
|
+
of that release candidate, not investigation or feature development. The
|
|
62
|
+
following limits keep new work reviewable:
|
|
63
|
+
|
|
64
|
+
- at most three active new Preview development tracks;
|
|
65
|
+
- at most one of those tracks may introduce grammar syntax; and
|
|
66
|
+
- at most five Experimental product features.
|
|
67
|
+
|
|
68
|
+
A development track starts when its first user-visible change is merged and
|
|
69
|
+
ends when the feature becomes Stable or Experimental, or is removed. An
|
|
70
|
+
investigation, an ADR, and an unmerged spike do not consume the budget. An
|
|
71
|
+
existing Preview feature consumes a track only while its specification is being
|
|
72
|
+
changed; promotion work that preserves its specification does not.
|
|
73
|
+
|
|
74
|
+
There is currently one active Preview development track: root declarative
|
|
75
|
+
parser construction. It consumes the one grammar-syntax slot while extending
|
|
76
|
+
the existing Preview IELR and multiple-entry surfaces; it is not a Stable
|
|
77
|
+
promotion. A pull request that starts or ends a track must update this
|
|
78
|
+
statement and the inventory below.
|
|
79
|
+
These limits do not relax the core IR contracts described under
|
|
80
|
+
[Core IR freeze](#core-ir-freeze), but they do permit retiring an unpromoted
|
|
81
|
+
Preview implementation when its maintenance cost exceeds its evidence.
|
|
82
|
+
|
|
23
83
|
## v1 inventory
|
|
24
84
|
|
|
25
85
|
Stable:
|
|
@@ -28,26 +88,45 @@ Stable:
|
|
|
28
88
|
- default direct LALR construction, parser tables, recovery callbacks,
|
|
29
89
|
observation events, resource limits, migration checks, and bounded
|
|
30
90
|
counterexample/ambiguity analysis;
|
|
31
|
-
-
|
|
91
|
+
- current core Grammar IR and Automaton IR, independently versioned Lexer IR, table formats, report
|
|
32
92
|
schemas, and their validators;
|
|
33
93
|
- format-v6 Red/Green batch CST parsing, typed syntax views, persistent editing
|
|
34
94
|
and diffing, and the closed `ibex_cst` schema v1 serialization contract.
|
|
35
95
|
|
|
36
|
-
Preview:
|
|
37
|
-
|
|
38
|
-
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
96
|
+
Preview and Experimental:
|
|
97
|
+
|
|
98
|
+
The machine-readable [maturity audit](maturity.md) re-evaluates every current
|
|
99
|
+
feature. All eighteen Preview and both Experimental features remain at their
|
|
100
|
+
current maturity. The canonical summary below is generated from that registry
|
|
101
|
+
and checked against both public documents.
|
|
102
|
+
|
|
103
|
+
<!-- maturity-summary:start -->
|
|
104
|
+
Inventory: **18 Preview, 2 Experimental**. Active new Preview tracks: **1/3** (grammar syntax: **1/1**). Experimental product features: **2/5**.
|
|
105
|
+
Release dependency state: R001 **hold_external**; R002 **pending_exact_revision**; no feature is promoted by this audit.
|
|
106
|
+
|
|
107
|
+
| Stable ID | Feature | Current maturity | Decision | External use | Release gate |
|
|
108
|
+
| --- | --- | --- | --- | --- | --- |
|
|
109
|
+
| `ebnf-groups` | EBNF groups | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
110
|
+
| `parameterized-rules` | parameterized rules | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
111
|
+
| `inline-rules` | inline rules | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
112
|
+
| `middle-actions` | middle actions | Preview | Redesign Preview | not demonstrated | Blocked: R001, R002 |
|
|
113
|
+
| `multiple-entries` | multiple entries | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
114
|
+
| `canonical-imports` | canonical imports | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
115
|
+
| `generated-lexers` | generated lexers | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
116
|
+
| `semantic-locations-types` | semantic locations/types | Preview | Redesign Preview | not demonstrated | Blocked: R001, R002 |
|
|
117
|
+
| `ast-generation` | AST generation | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
118
|
+
| `grammar-tests` | grammar tests | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
119
|
+
| `documentation-tooling` | documentation tooling | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
120
|
+
| `ielr` | IELR | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
121
|
+
| `lsp` | LSP | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
122
|
+
| `watch` | watch | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
123
|
+
| `debug` | debug | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
124
|
+
| `coverage` | coverage | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
125
|
+
| `browser-playground` | browser playground | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
126
|
+
| `action-shadow` | action-shadow | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
|
|
127
|
+
| `bounded-repair` | bounded repair | Experimental | Keep Experimental | not demonstrated | Blocked: R001, R002 |
|
|
128
|
+
| `incremental-cst` | incremental CST | Experimental | Keep Experimental | not demonstrated | Blocked: R001, R002 |
|
|
129
|
+
<!-- maturity-summary:end -->
|
|
51
130
|
|
|
52
131
|
The batch CST contract is selected for the initial v1 API under the
|
|
53
132
|
initial-major evidence rule above. The remaining Preview features have not
|
|
@@ -57,6 +136,20 @@ useful and inserted semantic values may be nil.
|
|
|
57
136
|
|
|
58
137
|
## Research decisions
|
|
59
138
|
|
|
139
|
+
- The Stage A matrix, golden, reproducibility, schema, adversarial, gallery,
|
|
140
|
+
fuzzing, dependency, and network gates now make feature-off and
|
|
141
|
+
algorithm-crossing regressions reviewable before promotion.
|
|
142
|
+
- Independent verification detects all twenty committed structural table
|
|
143
|
+
faults across every gallery/algorithm/table combination.
|
|
144
|
+
- Bounded grammar comparison distinguishes concrete shortest differences from
|
|
145
|
+
completed searches with no difference and never labels the latter a proof.
|
|
146
|
+
- Conflict repair passed its first fixed twenty-case capability measurement at
|
|
147
|
+
20/20. The corpus is intentionally described as a regression baseline, not
|
|
148
|
+
a general repair-rate claim.
|
|
149
|
+
- The analysis-only Bison adapter imports and builds five checksum-pinned
|
|
150
|
+
external grammars. Bison-era CRuby production counts match, its one-state
|
|
151
|
+
acceptance-convention delta is explained, and current Lrama-only `%rule`
|
|
152
|
+
structure is reported as incomplete instead of receiving unsafe repairs.
|
|
60
153
|
- IELR passed its correctness/state-bound spike and remains preview while it
|
|
61
154
|
gains field experience. Direct LALR remains the default.
|
|
62
155
|
- Bounded repair passed SP-4 with 8/10 useful plans and remains experimental.
|
|
@@ -71,16 +164,20 @@ The supported alternatives are canonical LR(1) or IELR for LALR inadequacy,
|
|
|
71
164
|
bounded ambiguity/counterexample analysis for ambiguous grammars, and
|
|
72
165
|
`--watch` with full deterministic reparsing for editing workflows.
|
|
73
166
|
|
|
74
|
-
## Core IR
|
|
167
|
+
## Core IR contract
|
|
75
168
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
169
|
+
Grammar IR and Automaton IR each have one current closed format. Their required
|
|
170
|
+
fields, meanings, identity rules, ordering, and validation semantics are
|
|
171
|
+
reviewed as one contract. Older documents are rejected at the load boundary;
|
|
172
|
+
there is no compatibility reader or migration command for either document.
|
|
80
173
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
174
|
+
Before v1.0, an incompatible change to either current IR is a coordinated
|
|
175
|
+
contract change: update the writer, reader, schemas, fixtures, and downstream
|
|
176
|
+
consumers together. No legacy format is retained only to avoid a pre-release
|
|
177
|
+
breaking change.
|
|
178
|
+
|
|
179
|
+
Additive fields and meaning changes are deliberate schema changes with updated
|
|
180
|
+
evidence. Current schemas remain closed and reject unknown fields.
|
|
84
181
|
|
|
85
182
|
The `x-` namespace is reserved for future experimental data and is not frozen.
|
|
86
183
|
Current closed schemas intentionally reject unknown fields and do not emit
|
|
@@ -91,6 +188,8 @@ a later schema version.
|
|
|
91
188
|
|
|
92
189
|
## Compatibility policy
|
|
93
190
|
|
|
191
|
+
<!-- stable:compatibility-policy:v1 -->
|
|
192
|
+
|
|
94
193
|
Compatible mode is the permanent default. Opt-in extensions, exact lookahead
|
|
95
194
|
defaults, repair, and research algorithms do not silently replace compatible
|
|
96
195
|
behavior. Public migration evidence uses public commands and black-box
|
|
@@ -101,13 +200,23 @@ An undeclared invalid token intentionally calls `on_error` before ordinary yacc
|
|
|
101
200
|
recovery. This is the documented recommended behavior and is not changed by
|
|
102
201
|
the freeze.
|
|
103
202
|
|
|
104
|
-
Parser-table
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
contract. Additive meaning requires a new schema version; readers
|
|
110
|
-
accept unknown fields.
|
|
203
|
+
Parser-table format v6 is the only runtime table contract and the only table
|
|
204
|
+
writer. Older generated tables fail before token consumption with a
|
|
205
|
+
regeneration instruction; they are not a pre-v1 compatibility obligation.
|
|
206
|
+
The current Grammar IR adds generator-owned parser configuration without changing that
|
|
207
|
+
runtime table contract. The closed `ibex_cst` schema v1 is a versioned
|
|
208
|
+
interchange contract. Additive meaning requires a new schema version; readers
|
|
209
|
+
do not accept unknown fields.
|
|
210
|
+
|
|
211
|
+
## Evidence is not compatibility
|
|
212
|
+
|
|
213
|
+
Exact source digests and clean-revision bindings establish what an evidence
|
|
214
|
+
record measured. They are audit-integrity requirements, not promises that a
|
|
215
|
+
consumer can execute an old generated artifact or reuse an old working-tree
|
|
216
|
+
capture. Historical evidence remains immutable for auditability; current
|
|
217
|
+
evidence must be regenerated and rebound after a relevant source or policy
|
|
218
|
+
change. A stale-evidence failure therefore means “refresh the measurement,”
|
|
219
|
+
not “preserve the old implementation forever.”
|
|
111
220
|
|
|
112
221
|
## Deprecation policy
|
|
113
222
|
|
|
@@ -116,8 +225,9 @@ two minor releases. The release notes and documentation must name the first
|
|
|
116
225
|
warning release, replacement, migration command or procedure, and earliest
|
|
117
226
|
removal release. Automated migration is supplied when practical; policy does
|
|
118
227
|
not promise a command that the product does not provide. Preview features
|
|
119
|
-
receive at least one minor release of notice
|
|
120
|
-
|
|
228
|
+
receive at least one minor release of notice only after v1.0; before v1.0 the
|
|
229
|
+
release-note and maturity-evidence rule above applies. Experimental features
|
|
230
|
+
may change without notice.
|
|
121
231
|
|
|
122
232
|
The pre-v1 mixed semantic/syntax CST was a Preview contract and is removed
|
|
123
233
|
while selecting the initial stable API. Its parser tables are rejected with a
|
data/docs/status.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Project status
|
|
3
|
+
description: Human-readable release status, maturity boundaries, and evidence links for Ibex.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Project status
|
|
7
|
+
|
|
8
|
+
Ibex is pre-1.0. The default compatible mode is the conservative adoption
|
|
9
|
+
baseline, while Preview and Experimental features require explicit activation
|
|
10
|
+
and have feature-specific boundaries.
|
|
11
|
+
|
|
12
|
+
## Release decision
|
|
13
|
+
|
|
14
|
+
The v1.0 publication decision is currently on hold because the published error
|
|
15
|
+
experience evidence still needs an independent review. This blocks publication
|
|
16
|
+
of that release candidate; it does not freeze feature development. The current
|
|
17
|
+
Grammar IR, Automaton IR, table, report, and runtime contracts remain closed
|
|
18
|
+
and validated while Preview work continues under the stability policy.
|
|
19
|
+
|
|
20
|
+
The complete rationale and evidence limitations are in the [release readiness
|
|
21
|
+
report](release-readiness.md). The [error UX review status](error-ux-review-status-v1.json)
|
|
22
|
+
is the machine-readable record; this page is the human entry point.
|
|
23
|
+
|
|
24
|
+
## Maturity at a glance
|
|
25
|
+
|
|
26
|
+
- **Stable baseline:** racc-compatible grammar input, default LALR construction,
|
|
27
|
+
current parser/runtime contracts, and format-v6 batch CST.
|
|
28
|
+
- **Preview:** extended grammar syntax, generated lexers, IELR, LSP, watch,
|
|
29
|
+
diagnostics, and the browser playground. These require explicit activation
|
|
30
|
+
or are bounded tools.
|
|
31
|
+
- **Experimental:** syntax-only incremental CST sessions and selected research
|
|
32
|
+
surfaces. They may change without the Stable compatibility promise.
|
|
33
|
+
|
|
34
|
+
Consult [maturity.md](maturity.md) and [stability.md](stability.md) for the
|
|
35
|
+
canonical feature registry, activation mechanism, and promotion requirements.
|
|
36
|
+
|
|
37
|
+
## Evidence boundaries
|
|
38
|
+
|
|
39
|
+
Repository tests and bounded benchmarks are evidence for the recorded revision,
|
|
40
|
+
not universal performance claims. Comparative statements link to the relevant
|
|
41
|
+
[comparison policy](comparison-policy.md), [workload registry](workloads.md),
|
|
42
|
+
and release artifact. Ibex does not claim to be a sandbox: generated semantic
|
|
43
|
+
parsers execute application Ruby.
|