ibex 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +95 -272
- data/docs/architecture.md +187 -35
- data/docs/bison-import.md +103 -0
- data/docs/comparison-policy.md +127 -0
- data/docs/configuration-model.md +59 -0
- data/docs/configuration-report.md +28 -0
- data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
- data/docs/conflict-explanation-study.md +62 -0
- data/docs/construction-profiling.md +157 -0
- data/docs/cst-migration.md +3 -4
- data/docs/cst.md +24 -0
- data/docs/decisions/0000-template.md +24 -0
- data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
- data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
- data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
- data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
- data/docs/decisions/0005-contained-grammar-composition.md +34 -0
- data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
- data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
- data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
- data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
- data/docs/decisions/0010-committed-runtime-observation.md +34 -0
- data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
- data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
- data/docs/decisions/0013-transactional-generation-publication.md +37 -0
- data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
- data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
- data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
- data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
- data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
- data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
- data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
- data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
- data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
- data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
- data/docs/decisions/0024-direct-ielr-construction.md +42 -0
- data/docs/decisions/README.md +74 -0
- data/docs/declarative-configuration.md +262 -0
- data/docs/development.md +179 -2
- data/docs/direct-ielr-decision.md +120 -0
- data/docs/direct-multi-entry-decision.md +82 -0
- data/docs/editor-setup.md +2 -1
- data/docs/error-ux-review-rubric-v1.md +117 -0
- data/docs/error-ux-reviews/v1/records/README.md +22 -0
- data/docs/error-ux-round2-review-status-v1.json +27 -0
- data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
- data/docs/error-ux-round2-v1.json +1581 -0
- data/docs/error-ux-round2.md +111 -0
- data/docs/error-ux.md +54 -0
- data/docs/getting-started.md +80 -0
- data/docs/grammar-reference.md +184 -13
- data/docs/ielr-design.md +43 -0
- data/docs/ielr.md +92 -0
- data/docs/ja/bison-import.md +51 -0
- data/docs/ja/stable-api.md +53 -0
- data/docs/lexer-construction-profile.md +109 -0
- data/docs/lexer-migration.md +5 -0
- data/docs/maturity.md +128 -0
- data/docs/project-site-strategy.md +54 -0
- data/docs/racc-migration-evidence.md +45 -0
- data/docs/racc-migration.md +11 -0
- data/docs/release-readiness.md +126 -68
- data/docs/repair-semantics.md +123 -0
- data/docs/runtime-abi-evolution.md +307 -0
- data/docs/stability.md +144 -34
- data/docs/status.md +43 -0
- data/docs/syntax-sessions.md +183 -0
- data/docs/table-artifact.md +125 -0
- data/docs/test-interactions.md +219 -0
- data/docs/verification-report.md +137 -0
- data/docs/verifier-trust-boundary.md +199 -0
- data/docs/workloads.md +132 -0
- data/lib/ibex/analysis/digraph.rb +128 -0
- data/lib/ibex/analysis.rb +1 -0
- data/lib/ibex/bison_import/importer.rb +516 -0
- data/lib/ibex/bison_import/tokenizer.rb +260 -0
- data/lib/ibex/bison_import.rb +200 -0
- data/lib/ibex/bounded_subprocess.rb +171 -0
- data/lib/ibex/cli/ambiguity.rb +14 -5
- data/lib/ibex/cli/analysis.rb +134 -0
- data/lib/ibex/cli/bison_import.rb +121 -0
- data/lib/ibex/cli/config.rb +128 -0
- data/lib/ibex/cli/diagnostics.rb +31 -11
- data/lib/ibex/cli/documentation.rb +11 -8
- data/lib/ibex/cli/equiv.rb +169 -0
- data/lib/ibex/cli/error_messages.rb +11 -4
- data/lib/ibex/cli/explain.rb +14 -5
- data/lib/ibex/cli/fix.rb +203 -0
- data/lib/ibex/cli/formatting.rb +11 -10
- data/lib/ibex/cli/fuzz.rb +200 -0
- data/lib/ibex/cli/fuzz_regressions.rb +145 -0
- data/lib/ibex/cli/generation_artifacts.rb +53 -2
- data/lib/ibex/cli/generation_error_messages.rb +1 -1
- data/lib/ibex/cli/grammar_tests.rb +32 -14
- data/lib/ibex/cli/ir_tools.rb +5 -61
- data/lib/ibex/cli/outputs.rb +60 -33
- data/lib/ibex/cli/reduce.rb +202 -0
- data/lib/ibex/cli/reduce_reporting.rb +71 -0
- data/lib/ibex/cli/samples.rb +49 -23
- data/lib/ibex/cli/verify.rb +116 -0
- data/lib/ibex/cli/watch.rb +6 -5
- data/lib/ibex/cli.rb +357 -56
- data/lib/ibex/codegen/action_locations.rb +1 -1
- data/lib/ibex/codegen/action_method_source.rb +5 -6
- data/lib/ibex/codegen/action_source.rb +3 -2
- data/lib/ibex/codegen/ambiguity.rb +5 -4
- data/lib/ibex/codegen/explain.rb +123 -55
- data/lib/ibex/codegen/generated_action_abi.rb +10 -5
- data/lib/ibex/codegen/rbs.rb +15 -7
- data/lib/ibex/codegen/report.rb +25 -13
- data/lib/ibex/codegen/ruby.rb +28 -64
- data/lib/ibex/codegen/ruby_actions.rb +9 -6
- data/lib/ibex/codegen/ruby_syntax.rb +15 -9
- data/lib/ibex/configuration/analysis_grammar.rb +35 -0
- data/lib/ibex/configuration/explanation.rb +421 -0
- data/lib/ibex/configuration/inspector.rb +232 -0
- data/lib/ibex/configuration.rb +580 -0
- data/lib/ibex/coverage/collector.rb +21 -11
- data/lib/ibex/coverage/event_stream.rb +13 -7
- data/lib/ibex/coverage/report.rb +26 -18
- data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
- data/lib/ibex/delta_reducer.rb +99 -0
- data/lib/ibex/diff.rb +140 -0
- data/lib/ibex/equiv/machine.rb +135 -0
- data/lib/ibex/equiv.rb +373 -0
- data/lib/ibex/fix.rb +557 -0
- data/lib/ibex/frontend/ast.rb +22 -2
- data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
- data/lib/ibex/frontend/diagnostic.rb +1 -1
- data/lib/ibex/frontend/formatter.rb +72 -26
- data/lib/ibex/frontend/generated_parser.rb +147 -117
- data/lib/ibex/frontend/generated_parser_base.rb +7 -3
- data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
- data/lib/ibex/frontend/generation.rb +1 -1
- data/lib/ibex/frontend/parser/declarations.rb +17 -2
- data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
- data/lib/ibex/frontend/regenerator.rb +1 -0
- data/lib/ibex/frontend/resolution.rb +2 -1
- data/lib/ibex/frontend/resolver.rb +1 -1
- data/lib/ibex/frontend/rule_documentation.rb +2 -1
- data/lib/ibex/frontend/source_cursor.rb +2 -2
- data/lib/ibex/frontend/source_span.rb +9 -1
- data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
- data/lib/ibex/frontend/token_adapter.rb +10 -0
- data/lib/ibex/frontend.rb +1 -0
- data/lib/ibex/fuzz.rb +219 -0
- data/lib/ibex/generation_input.rb +1 -1
- data/lib/ibex/generation_manifest.rb +36 -25
- data/lib/ibex/generation_transaction.rb +1 -1
- data/lib/ibex/generation_transaction_recovery.rb +4 -4
- data/lib/ibex/generation_transaction_validation.rb +2 -2
- data/lib/ibex/grammar_tests.rb +1 -1
- data/lib/ibex/ir/automaton_ir.rb +61 -19
- data/lib/ibex/ir/grammar_ir.rb +76 -45
- data/lib/ibex/ir/lexer_ir.rb +5 -5
- data/lib/ibex/ir/parser_contract.rb +135 -0
- data/lib/ibex/ir/serialize.rb +203 -76
- data/lib/ibex/ir/validator/automaton.rb +64 -34
- data/lib/ibex/ir/validator/base.rb +51 -33
- data/lib/ibex/ir/validator/grammar.rb +113 -83
- data/lib/ibex/ir/validator/lexer.rb +2 -2
- data/lib/ibex/ir/validator.rb +2 -1
- data/lib/ibex/ir.rb +16 -3
- data/lib/ibex/lalr/build_metrics.rb +50 -2
- data/lib/ibex/lalr/builder.rb +191 -56
- data/lib/ibex/lalr/conflict_search.rb +15 -4
- data/lib/ibex/lalr/counterexample.rb +19 -14
- data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
- data/lib/ibex/lalr/goto_follows.rb +229 -0
- data/lib/ibex/lalr/ielr/annotator.rb +214 -0
- data/lib/ibex/lalr/ielr/bits.rb +28 -0
- data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
- data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
- data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
- data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
- data/lib/ibex/lalr/ielr/split_state.rb +20 -0
- data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
- data/lib/ibex/lalr/ielr_partition.rb +22 -8
- data/lib/ibex/lalr/inadequacy_report.rb +50 -0
- data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
- data/lib/ibex/lalr/lr0_collection.rb +121 -0
- data/lib/ibex/lalr/unreachable_states.rb +80 -0
- data/lib/ibex/lalr.rb +52 -1
- data/lib/ibex/location.rb +3 -3
- data/lib/ibex/lsp/document_handlers.rb +9 -7
- data/lib/ibex/lsp/initialization_handlers.rb +9 -7
- data/lib/ibex/lsp/navigation_handlers.rb +25 -11
- data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
- data/lib/ibex/lsp/position_codec.rb +6 -2
- data/lib/ibex/lsp/request_handlers.rb +3 -2
- data/lib/ibex/lsp/request_support.rb +14 -7
- data/lib/ibex/lsp/server.rb +13 -11
- data/lib/ibex/lsp/symbol_index.rb +13 -13
- data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
- data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
- data/lib/ibex/lsp/transport.rb +13 -3
- data/lib/ibex/lsp/workspace.rb +5 -4
- data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
- data/lib/ibex/lsp.rb +1 -0
- data/lib/ibex/messages/en.yml +28 -0
- data/lib/ibex/messages/ja.yml +28 -0
- data/lib/ibex/messages.rb +68 -0
- data/lib/ibex/metrics.rb +175 -0
- data/lib/ibex/normalize/declarations.rb +2 -0
- data/lib/ibex/normalize/diagnostics.rb +63 -38
- data/lib/ibex/normalize/expander.rb +5 -4
- data/lib/ibex/normalize/expression.rb +8 -6
- data/lib/ibex/normalize/grammar_builder.rb +26 -0
- data/lib/ibex/normalize/inline_expansion.rb +100 -49
- data/lib/ibex/normalize/lexer.rb +3 -3
- data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
- data/lib/ibex/normalize/parameter_lowering.rb +40 -23
- data/lib/ibex/normalize/parameters.rb +31 -4
- data/lib/ibex/normalize/parser_configuration.rb +70 -0
- data/lib/ibex/normalize.rb +10 -20
- data/lib/ibex/racc_migration/report.rb +6 -3
- data/lib/ibex/rake_task.rb +1 -1
- data/lib/ibex/samples.rb +43 -13
- data/lib/ibex/table_artifact/builder.rb +266 -0
- data/lib/ibex/table_artifact/cst_projection.rb +70 -0
- data/lib/ibex/table_artifact/document.rb +39 -0
- data/lib/ibex/table_artifact/executor.rb +187 -0
- data/lib/ibex/table_artifact/serializer.rb +57 -0
- data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
- data/lib/ibex/table_artifact/validator/support.rb +86 -0
- data/lib/ibex/table_artifact/validator/tables.rb +238 -0
- data/lib/ibex/table_artifact/validator.rb +253 -0
- data/lib/ibex/table_artifact.rb +88 -0
- data/lib/ibex/table_simulation/result.rb +6 -2
- data/lib/ibex/table_simulation/step.rb +3 -1
- data/lib/ibex/tables.rb +14 -0
- data/lib/ibex/verifiable_generation_bundle.rb +86 -0
- data/lib/ibex/verification_report/builder.rb +159 -0
- data/lib/ibex/verification_report/canonical_ir.rb +123 -0
- data/lib/ibex/verification_report/logical_path.rb +68 -0
- data/lib/ibex/verification_report/validator.rb +329 -0
- data/lib/ibex/verification_report.rb +69 -0
- data/lib/ibex/verify/action_correspondence.rb +138 -0
- data/lib/ibex/verify/language_witness.rb +295 -0
- data/lib/ibex/verify/reference_collection.rb +190 -0
- data/lib/ibex/verify/result.rb +73 -0
- data/lib/ibex/verify/verifier.rb +598 -0
- data/lib/ibex/verify.rb +15 -0
- data/lib/ibex/version.rb +1 -1
- data/lib/ibex/watch/runner.rb +10 -1
- data/lib/ibex/watch/source_snapshot.rb +13 -8
- data/lib/ibex.rb +10 -0
- data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
- data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
- data/schema/bison-import-v1.schema.json +106 -0
- data/schema/conflict-explanation-review-v1.schema.json +196 -0
- data/schema/conflict-explanation-study-v1.schema.json +528 -0
- data/schema/construction-profile-v1.schema.json +779 -0
- data/schema/diff-v1.schema.json +41 -0
- data/schema/direct-ielr-decision-v1.schema.json +200 -0
- data/schema/equiv-v1.schema.json +64 -0
- data/schema/error-ux-review-v1.schema.json +609 -0
- data/schema/error-ux-round2-review-v1.schema.json +191 -0
- data/schema/error-ux-round2-v1.schema.json +519 -0
- data/schema/explain-v1.schema.json +63 -1
- data/schema/fix-v1.schema.json +64 -0
- data/schema/fix-v2.schema.json +66 -0
- data/schema/fix-v3.schema.json +101 -0
- data/schema/fuzz-regression-v1.schema.json +79 -0
- data/schema/fuzz-v1.schema.json +114 -0
- data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
- data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
- data/schema/grammar-ir.schema.json +271 -0
- data/schema/ielr-benchmark-v1.schema.json +85 -0
- data/schema/lexer-profile-v1.schema.json +491 -0
- data/schema/metrics-v1.schema.json +60 -0
- data/schema/reduce-v1.schema.json +22 -0
- data/schema/reduce-v2.schema.json +63 -0
- data/schema/table-artifact-v1.schema.json +377 -0
- data/schema/verification-report-v1.schema.json +222 -0
- data/schema/verify-v1.schema.json +42 -0
- data/sig/ibex/analysis/digraph.rbs +19 -0
- data/sig/ibex/bison_import/importer.rbs +140 -0
- data/sig/ibex/bison_import/tokenizer.rbs +91 -0
- data/sig/ibex/bison_import.rbs +95 -0
- data/sig/ibex/bounded_subprocess.rbs +76 -0
- data/sig/ibex/cli/ambiguity.rbs +6 -0
- data/sig/ibex/cli/analysis.rbs +43 -0
- data/sig/ibex/cli/bison_import.rbs +32 -0
- data/sig/ibex/cli/config.rbs +35 -0
- data/sig/ibex/cli/diagnostics.rbs +16 -3
- data/sig/ibex/cli/documentation.rbs +5 -1
- data/sig/ibex/cli/equiv.rbs +49 -0
- data/sig/ibex/cli/error_messages.rbs +6 -0
- data/sig/ibex/cli/explain.rbs +6 -0
- data/sig/ibex/cli/fix.rbs +53 -0
- data/sig/ibex/cli/formatting.rbs +5 -1
- data/sig/ibex/cli/fuzz.rbs +47 -0
- data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
- data/sig/ibex/cli/generation_artifacts.rbs +21 -2
- data/sig/ibex/cli/generation_error_messages.rbs +2 -2
- data/sig/ibex/cli/grammar_tests.rbs +15 -3
- data/sig/ibex/cli/ir_tools.rbs +7 -19
- data/sig/ibex/cli/outputs.rbs +9 -3
- data/sig/ibex/cli/reduce.rbs +60 -0
- data/sig/ibex/cli/reduce_reporting.rbs +27 -0
- data/sig/ibex/cli/samples.rbs +8 -0
- data/sig/ibex/cli/verify.rbs +28 -0
- data/sig/ibex/cli/watch.rbs +2 -2
- data/sig/ibex/cli.rbs +79 -5
- data/sig/ibex/codegen/action_method_source.rbs +4 -4
- data/sig/ibex/codegen/explain.rbs +45 -25
- data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
- data/sig/ibex/codegen/rbs.rbs +12 -10
- data/sig/ibex/codegen/report.rbs +12 -12
- data/sig/ibex/codegen/ruby.rbs +18 -15
- data/sig/ibex/codegen/ruby_actions.rbs +6 -6
- data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
- data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
- data/sig/ibex/configuration/explanation.rbs +195 -0
- data/sig/ibex/configuration/inspector.rbs +53 -0
- data/sig/ibex/configuration.rbs +242 -0
- data/sig/ibex/coverage/collector.rbs +16 -14
- data/sig/ibex/coverage/event_stream.rbs +12 -10
- data/sig/ibex/coverage/report.rbs +20 -18
- data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
- data/sig/ibex/delta_reducer.rbs +38 -0
- data/sig/ibex/diff.rbs +47 -0
- data/sig/ibex/equiv/machine.rbs +54 -0
- data/sig/ibex/equiv.rbs +129 -0
- data/sig/ibex/fix.rbs +138 -0
- data/sig/ibex/frontend/ast.rbs +33 -7
- data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
- data/sig/ibex/frontend/diagnostic.rbs +2 -2
- data/sig/ibex/frontend/formatter.rbs +38 -30
- data/sig/ibex/frontend/generated_parser.rbs +87 -77
- data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
- data/sig/ibex/frontend/parser/declarations.rbs +3 -0
- data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
- data/sig/ibex/frontend/resolution.rbs +3 -2
- data/sig/ibex/frontend/source_cursor.rbs +2 -2
- data/sig/ibex/frontend/source_span.rbs +5 -2
- data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
- data/sig/ibex/frontend/token_adapter.rbs +6 -0
- data/sig/ibex/fuzz.rbs +71 -0
- data/sig/ibex/generation_input.rbs +2 -2
- data/sig/ibex/generation_manifest.rbs +35 -32
- data/sig/ibex/generation_transaction_recovery.rbs +2 -2
- data/sig/ibex/generation_transaction_validation.rbs +2 -2
- data/sig/ibex/grammar_tests.rbs +2 -2
- data/sig/ibex/ir/automaton_ir.rbs +30 -11
- data/sig/ibex/ir/grammar_ir.rbs +41 -27
- data/sig/ibex/ir/lexer_ir.rbs +8 -8
- data/sig/ibex/ir/parser_contract.rbs +69 -0
- data/sig/ibex/ir/serialize.rbs +63 -28
- data/sig/ibex/ir/validator/automaton.rbs +47 -44
- data/sig/ibex/ir/validator/base.rbs +35 -31
- data/sig/ibex/ir/validator/grammar.rbs +78 -67
- data/sig/ibex/ir/validator/lexer.rbs +4 -4
- data/sig/ibex/ir.rbs +9 -3
- data/sig/ibex/lalr/build_metrics.rbs +42 -2
- data/sig/ibex/lalr/builder.rbs +75 -24
- data/sig/ibex/lalr/conflict_search.rbs +5 -2
- data/sig/ibex/lalr/counterexample.rbs +17 -14
- data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
- data/sig/ibex/lalr/goto_follows.rbs +89 -0
- data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
- data/sig/ibex/lalr/ielr/bits.rbs +14 -0
- data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
- data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
- data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
- data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
- data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
- data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
- data/sig/ibex/lalr/ielr_partition.rbs +17 -5
- data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
- data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
- data/sig/ibex/lalr/lr0_collection.rbs +50 -0
- data/sig/ibex/lalr/unreachable_states.rbs +25 -0
- data/sig/ibex/lalr.rbs +11 -1
- data/sig/ibex/location.rbs +6 -6
- data/sig/ibex/lsp/document_handlers.rbs +8 -8
- data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
- data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
- data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
- data/sig/ibex/lsp/position_codec.rbs +8 -4
- data/sig/ibex/lsp/request_handlers.rbs +4 -4
- data/sig/ibex/lsp/request_support.rbs +8 -8
- data/sig/ibex/lsp/server.rbs +16 -16
- data/sig/ibex/lsp/symbol_index.rbs +20 -21
- data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
- data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
- data/sig/ibex/lsp/transport.rbs +18 -6
- data/sig/ibex/lsp/workspace.rbs +3 -2
- data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
- data/sig/ibex/messages.rbs +26 -0
- data/sig/ibex/metrics.rbs +50 -0
- data/sig/ibex/normalize/diagnostics.rbs +4 -4
- data/sig/ibex/normalize/expander.rbs +2 -2
- data/sig/ibex/normalize/expression.rbs +10 -8
- data/sig/ibex/normalize/grammar_builder.rbs +11 -0
- data/sig/ibex/normalize/inline_expansion.rbs +59 -50
- data/sig/ibex/normalize/lexer.rbs +4 -4
- data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
- data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
- data/sig/ibex/normalize/parameters.rbs +13 -8
- data/sig/ibex/normalize/parser_configuration.rbs +23 -0
- data/sig/ibex/normalize.rbs +11 -5
- data/sig/ibex/racc_migration/report.rbs +8 -6
- data/sig/ibex/rake_task.rbs +1 -1
- data/sig/ibex/samples.rbs +13 -4
- data/sig/ibex/table_artifact/builder.rbs +93 -0
- data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
- data/sig/ibex/table_artifact/document.rbs +27 -0
- data/sig/ibex/table_artifact/executor.rbs +66 -0
- data/sig/ibex/table_artifact/serializer.rbs +25 -0
- data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
- data/sig/ibex/table_artifact/validator/support.rbs +46 -0
- data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
- data/sig/ibex/table_artifact/validator.rbs +66 -0
- data/sig/ibex/table_artifact.rbs +32 -0
- data/sig/ibex/table_simulation/result.rbs +8 -4
- data/sig/ibex/table_simulation/step.rbs +4 -2
- data/sig/ibex/tables.rbs +10 -9
- data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
- data/sig/ibex/verification_report/builder.rbs +53 -0
- data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
- data/sig/ibex/verification_report/logical_path.rbs +34 -0
- data/sig/ibex/verification_report/validator.rbs +114 -0
- data/sig/ibex/verification_report.rbs +44 -0
- data/sig/ibex/verify/action_correspondence.rbs +50 -0
- data/sig/ibex/verify/language_witness.rbs +112 -0
- data/sig/ibex/verify/reference_collection.rbs +63 -0
- data/sig/ibex/verify/result.rbs +52 -0
- data/sig/ibex/verify/verifier.rbs +170 -0
- data/sig/ibex/verify.rbs +6 -0
- data/sig/ibex/watch/runner.rbs +12 -2
- data/sig/ibex/watch/source_snapshot.rbs +13 -9
- metadata +232 -12
- data/lib/ibex/ir/migration.rb +0 -120
- data/sig/ibex/ir/migration.rbs +0 -34
|
@@ -0,0 +1,183 @@
|
|
|
1
|
+
# Generated-language syntax sessions
|
|
2
|
+
|
|
3
|
+
The Experimental syntax-session API is a generated-language service boundary,
|
|
4
|
+
not an LSP implementation. It reuses the existing incremental Red/Green CST
|
|
5
|
+
engine and returns syntax only: parser production actions never run and no
|
|
6
|
+
semantic value is available.
|
|
7
|
+
|
|
8
|
+
## Trust must be acknowledged
|
|
9
|
+
|
|
10
|
+
Current generated parser classes report exactly one execution profile:
|
|
11
|
+
|
|
12
|
+
```ruby
|
|
13
|
+
Calculator.syntax_execution_profile
|
|
14
|
+
# => :trusted_application_code
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Opening a session requires the caller to acknowledge that profile:
|
|
18
|
+
|
|
19
|
+
```ruby
|
|
20
|
+
session = Calculator.syntax_session(
|
|
21
|
+
"1 + 2",
|
|
22
|
+
execution_profile: :trusted_application_code
|
|
23
|
+
)
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Generated lexer actions execute while opening and editing a session because
|
|
27
|
+
they define token emission, conversion, and lexer-state changes. Loading the
|
|
28
|
+
generated Ruby class may already have executed `header`, `inner`, or `footer`
|
|
29
|
+
sections. The explicit keyword records caller intent; it is not a sandbox flag.
|
|
30
|
+
Omitting it raises `Ibex::Runtime::SyntaxSessionTrustError` before lexing.
|
|
31
|
+
|
|
32
|
+
There is no `:declarative` switch. That profile will require a separately
|
|
33
|
+
generated action-free parser artifact, a declarative built-in-only lexer, and
|
|
34
|
+
no user-code sections. The current generator does not produce such an artifact,
|
|
35
|
+
and overriding `syntax_execution_profile` cannot enable it.
|
|
36
|
+
|
|
37
|
+
Parser callbacks and generated lexer hooks are application code too. The
|
|
38
|
+
syntax-only guarantee is specifically that grammar production actions are
|
|
39
|
+
suppressed. Use this API only with generated artifacts already trusted by the
|
|
40
|
+
host process.
|
|
41
|
+
|
|
42
|
+
## Opening, editing, and results
|
|
43
|
+
|
|
44
|
+
Input is a `String` or `Ibex::Runtime::CST::SourceText`. Edits are
|
|
45
|
+
`Ibex::Runtime::CST::TextEdit` values expressed as zero-based, half-open byte
|
|
46
|
+
ranges against the current source:
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
edit = Ibex::Runtime::CST::TextEdit.new(
|
|
50
|
+
start: 4,
|
|
51
|
+
delete_length: 1,
|
|
52
|
+
insert_text: "3"
|
|
53
|
+
)
|
|
54
|
+
result = session.apply_edits([edit])
|
|
55
|
+
|
|
56
|
+
result.revision # 1
|
|
57
|
+
result.syntax_root # immutable Red root
|
|
58
|
+
result.diagnostics # immutable diagnostics
|
|
59
|
+
result.expected_tokens # expected terminals observed at parser failures
|
|
60
|
+
result.status # :ok or :syntax_error
|
|
61
|
+
result.metrics # immutable reuse/fallback evidence
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`expected_tokens` reuses the generated parser's current ordinary or exact
|
|
65
|
+
lookahead-correction policy. With several recoverable failures it is the
|
|
66
|
+
first-seen union for that operation; it is not a completion ranking.
|
|
67
|
+
|
|
68
|
+
The result and metrics snapshots are frozen. Metrics always describe the
|
|
69
|
+
installed result of the completed operation. `token_count` is the number of
|
|
70
|
+
entries in that result's final token memo, including parser-supporting tokens
|
|
71
|
+
represented there. `reused_tokens` is the number of those entries whose Green
|
|
72
|
+
token identity was reused from the preceding memo. It is zero for an initial
|
|
73
|
+
open and for a full fresh-parse fallback. `reused_ratio` is the parser's
|
|
74
|
+
completed structural-reuse ratio (or lexical ratio when the Blender is
|
|
75
|
+
disabled), so it is not generally `reused_tokens / token_count`.
|
|
76
|
+
|
|
77
|
+
`fallback?` and `fallback_reasons` distinguish a clean incremental operation
|
|
78
|
+
from a successful fallback. A full lexical-error fallback reports no reuse.
|
|
79
|
+
A decomposition or memo-budget fallback may still report independently
|
|
80
|
+
validated lexical token reuse even though subtree reuse or memo retention was
|
|
81
|
+
declined. Every fallback remains a valid syntax result.
|
|
82
|
+
|
|
83
|
+
Fresh syntax sessions are the reference semantics. Incremental results must
|
|
84
|
+
match fresh results in source bytes, Green structure and flags, and diagnostic
|
|
85
|
+
content.
|
|
86
|
+
|
|
87
|
+
## Syntax-only repair proposals
|
|
88
|
+
|
|
89
|
+
`SyntaxSession#repair` is an additive Experimental operation for editor-style
|
|
90
|
+
syntax proposals. It runs the bounded repair search on a fresh private parser,
|
|
91
|
+
captures token identity and byte ranges, applies the resulting
|
|
92
|
+
`CST::TextEdit` values to a new `CST::SourceText`, and validates the edited
|
|
93
|
+
source with another fresh syntax session. The originating session and source
|
|
94
|
+
are unchanged.
|
|
95
|
+
|
|
96
|
+
```ruby
|
|
97
|
+
proposal = session.repair(token_text: { "PLUS" => "+" })
|
|
98
|
+
proposal.status # :accepted, :progress, :rejected, or a bounded failure
|
|
99
|
+
proposal.text_edits # immutable byte edits
|
|
100
|
+
proposal.updated_source # exact edited bytes, when validation ran
|
|
101
|
+
proposal.validation # fresh syntax-only result, when validation ran
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
The result is deliberately syntax-only: it has no semantic `value`, never
|
|
105
|
+
replays production actions, and never invokes the application's `on_repair`
|
|
106
|
+
callback. `token_text` supplies source bytes only. Named inserted or replaced
|
|
107
|
+
tokens fail closed when no spelling is supplied; punctuation literals may be
|
|
108
|
+
derived conservatively. `:exhausted` and `:not_found` remain distinct bounded
|
|
109
|
+
outcomes, while overlapping edits and multiple selected repair segments are
|
|
110
|
+
reported as unavailable.
|
|
111
|
+
|
|
112
|
+
An accepted result requires zero diagnostics and complete source consumption.
|
|
113
|
+
For progress or rejection, `updated_source` retains the exact edited bytes and
|
|
114
|
+
the validation CST may expose only its consumed prefix, as permitted by the
|
|
115
|
+
existing error/early-accept CST contract. Cancellation and service limits raise
|
|
116
|
+
their existing exceptions and are never returned as successful proposals.
|
|
117
|
+
|
|
118
|
+
## Validation, cancellation, and limits
|
|
119
|
+
|
|
120
|
+
A source passed directly as a String must have an ASCII-compatible Ruby
|
|
121
|
+
encoding. `SourceText` and `TextEdit` are explicit byte containers and have
|
|
122
|
+
already normalized their text to binary, so binary strings and invalid UTF-8
|
|
123
|
+
bytes remain supported. A direct non-ASCII-compatible source such as UTF-16 is
|
|
124
|
+
rejected instead of being silently reinterpreted. Edit shape, nonnegative
|
|
125
|
+
offsets and lengths, overlap, and source bounds are validated by the existing
|
|
126
|
+
`TextEdit` and `SourceText` contracts.
|
|
127
|
+
|
|
128
|
+
`SyntaxSessionLimits` bounds source size, edit count, and inserted bytes. The
|
|
129
|
+
existing `ResourceLimits` independently bounds parser stack/recovery work,
|
|
130
|
+
incremental decomposition, and memo storage:
|
|
131
|
+
|
|
132
|
+
```ruby
|
|
133
|
+
limits = Ibex::Runtime::SyntaxSessionLimits.new(
|
|
134
|
+
max_source_bytes: 2 * 1024 * 1024,
|
|
135
|
+
max_edits_per_operation: 128,
|
|
136
|
+
max_inserted_bytes: 256 * 1024
|
|
137
|
+
)
|
|
138
|
+
parser_limits = Ibex::Runtime::ResourceLimits.new(
|
|
139
|
+
max_incremental_decomposed_nodes: 20_000,
|
|
140
|
+
max_session_memo_bytes: 8 * 1024 * 1024
|
|
141
|
+
)
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Exceeding a service or parser bound raises
|
|
145
|
+
`SyntaxSessionResourceLimitError`. It is never returned as an `:ok` result.
|
|
146
|
+
Incremental decomposition or memo exhaustion may instead use the existing
|
|
147
|
+
sound fresh-path fallback and records that decision in metrics.
|
|
148
|
+
|
|
149
|
+
Cancellation is cooperative:
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
token = Ibex::Runtime::CancellationToken.new
|
|
153
|
+
session = Calculator.syntax_session(
|
|
154
|
+
source,
|
|
155
|
+
execution_profile: :trusted_application_code,
|
|
156
|
+
cancellation: token
|
|
157
|
+
)
|
|
158
|
+
|
|
159
|
+
token.cancel!
|
|
160
|
+
session.apply_edits(edits) # raises SyntaxSessionCancelled
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Checks run before an operation and at synchronous parser runtime events. A
|
|
164
|
+
cancellation requested after the final event belongs to the next operation;
|
|
165
|
+
this keeps completion and publication one atomic session transition. A Ruby
|
|
166
|
+
lexer action that does not return cannot be preempted; resource bounds and
|
|
167
|
+
cancellation do not isolate its side effects. An edit cancelled at a checkpoint
|
|
168
|
+
or stopped by a hard resource error leaves the last completed source and result
|
|
169
|
+
installed, and is never confused with a successful parse. Calls to
|
|
170
|
+
`apply_edits`, `source_text`, and `result` are serialized by the session.
|
|
171
|
+
|
|
172
|
+
## Scope and packaging
|
|
173
|
+
|
|
174
|
+
The prototype lives in `ibex-runtime`. A deployed generated parser needs only
|
|
175
|
+
the runtime's parser tables, lexer, CST engine, and session façade; requiring
|
|
176
|
+
the generator gem would unnecessarily add grammar construction concerns.
|
|
177
|
+
A separate `ibex-workbench` package is deferred until there is evidence for a
|
|
178
|
+
larger editor-service product boundary.
|
|
179
|
+
|
|
180
|
+
This API does not define LSP transport, workspace or file-system semantics,
|
|
181
|
+
cross-file indexing, query languages, formatting, or a declarative lexer. The
|
|
182
|
+
existing grammar-authoring LSP remains a separate static tool and does not load
|
|
183
|
+
generated application parsers.
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# Data-only parser table artifact
|
|
2
|
+
|
|
3
|
+
The table artifact is an Experimental, separately versioned JSON sidecar for
|
|
4
|
+
inspecting and recognizing parser token streams without parsing or loading
|
|
5
|
+
generated Ruby. Version 1 is a prototype persistence and execution boundary;
|
|
6
|
+
it is not accepted directly by the current parser runtime and is not part of
|
|
7
|
+
the Stable generated-parser ABI.
|
|
8
|
+
|
|
9
|
+
Load the prototype explicitly:
|
|
10
|
+
|
|
11
|
+
```ruby
|
|
12
|
+
require "ibex/table_artifact"
|
|
13
|
+
|
|
14
|
+
document = Ibex::TableArtifact.build(automaton, representation: :compact)
|
|
15
|
+
loaded = Ibex::TableArtifact.load(document.dump)
|
|
16
|
+
result = Ibex::TableArtifact::Executor.new(loaded).recognize([number_token_id])
|
|
17
|
+
result.accepted?
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
`recognize` consumes internal terminal IDs. It does not lex source text,
|
|
21
|
+
convert external token values, run semantic actions, build semantic values, or
|
|
22
|
+
perform recovery search. Callers may select a named entry and must treat
|
|
23
|
+
`max_steps` as the execution bound for hostile or cyclic input.
|
|
24
|
+
|
|
25
|
+
## Authority and representation
|
|
26
|
+
|
|
27
|
+
The artifact is executable table authority for its limited recognition
|
|
28
|
+
profile. The executor reads its actions, defaults, GOTOs, productions, and
|
|
29
|
+
entry states directly; it does not reconstruct an automaton or call the parser
|
|
30
|
+
builder. This is the execution benefit over retaining only Automaton IR.
|
|
31
|
+
|
|
32
|
+
The canonical payload contains:
|
|
33
|
+
|
|
34
|
+
- all symbol and terminal names, display names, and internal IDs;
|
|
35
|
+
- entry names and initial states;
|
|
36
|
+
- signed action codes, GOTOs, and default actions in either closed sparse rows
|
|
37
|
+
or row-displacement arrays;
|
|
38
|
+
- production left sides, right sides, stack lengths, and opaque action slots;
|
|
39
|
+
- static CST kind/field metadata and recovery declaration IDs;
|
|
40
|
+
- Grammar IR, Automaton IR, whole-payload, CST-metadata, and recovery-metadata
|
|
41
|
+
SHA-256 identities; and
|
|
42
|
+
- static byte, occupied-cell, lookup, and recognition-cost descriptions.
|
|
43
|
+
|
|
44
|
+
Signed action codes use `0` for accept, `-1` for error, `1 + state` for shift,
|
|
45
|
+
and `-2 - production` for reduce. Artifact schema version 1 pins those meanings
|
|
46
|
+
to parser-table format version 6. This is a semantic identity, not a claim that
|
|
47
|
+
the JSON object has the Ruby hash/object shape accepted by the v6 runtime.
|
|
48
|
+
|
|
49
|
+
Objects are serialized with recursively sorted string keys. Array order is
|
|
50
|
+
part of the format and is either semantic or validated as canonical. The
|
|
51
|
+
authoritative closed contract is
|
|
52
|
+
[`schema/table-artifact-v1.schema.json`](../schema/table-artifact-v1.schema.json),
|
|
53
|
+
and the Ruby loader additionally checks referential and canonical invariants
|
|
54
|
+
that JSON Schema cannot express.
|
|
55
|
+
|
|
56
|
+
`canonical_payload_bytes` is the exact byte count of compact canonical JSON for
|
|
57
|
+
the payload only. Compact lookup is an O(1) row-displacement probe; plain sparse
|
|
58
|
+
rows use O(log row width) binary search. These complexity strings and the
|
|
59
|
+
occupied-cell counts are static descriptions. `measurement: not-measured`
|
|
60
|
+
means none of them is a
|
|
61
|
+
wall-clock, allocation, RSS, or end-to-end parser benchmark.
|
|
62
|
+
|
|
63
|
+
## Validation and trust boundary
|
|
64
|
+
|
|
65
|
+
The loader applies a 16 MiB default byte bound, parses JSON, rejects unknown
|
|
66
|
+
fields, checks table encodings and references, recomputes payload/CST/recovery
|
|
67
|
+
digests and static cost fields, then deeply freezes the document. It never
|
|
68
|
+
loads generated classes or evaluates action strings. Semantic action slots are
|
|
69
|
+
production-ID bindings with `verified: false`; action bodies are deliberately
|
|
70
|
+
absent.
|
|
71
|
+
|
|
72
|
+
Compact tables must use the unique minimal row-displacement layout produced by
|
|
73
|
+
the v1 packer; unused padding is not canonical. Accept is valid only on `$eof`,
|
|
74
|
+
`$eof` cannot be shifted, and default actions are limited to errors and
|
|
75
|
+
reductions. The executor also converts any attempt to run beyond the bounded
|
|
76
|
+
input into a rejected result rather than leaking an indexing exception.
|
|
77
|
+
|
|
78
|
+
The grammar and automaton digests are provenance linkage claims copied from
|
|
79
|
+
the builder input. An artifact alone cannot recompute either source object or
|
|
80
|
+
establish its authenticity. A caller that has the source IR must recompute and
|
|
81
|
+
compare those digests, and a publication system must separately authenticate
|
|
82
|
+
the artifact. Digests detect inconsistency; they are not signatures.
|
|
83
|
+
|
|
84
|
+
Ordinary V002 construction hashes the supplied IR, including its diagnostic
|
|
85
|
+
source locations. The explicit verifiable-generation-bundle API instead
|
|
86
|
+
passes a `source-logical-v1` Automaton copy to this unchanged builder. Its
|
|
87
|
+
table identities are therefore path-neutral and exactly match the companion
|
|
88
|
+
report without changing the meaning of standalone `TableArtifact.build`.
|
|
89
|
+
|
|
90
|
+
The validator checks structural and internal table consistency. It does not
|
|
91
|
+
prove that the table implements a claimed grammar, that CST or recovery
|
|
92
|
+
metadata was derived correctly, or that the parser runtime and semantic
|
|
93
|
+
actions are correct. The separately recomputed CST and recovery digests make
|
|
94
|
+
those metadata regions explicit identities for later bundle binding, but do
|
|
95
|
+
not make them independently true.
|
|
96
|
+
|
|
97
|
+
This does not extend the claims of [`ibex verify`](verifier-trust-boundary.md).
|
|
98
|
+
Default and strict verification still rebuild tables from Automaton IR; strict
|
|
99
|
+
still adds collection-completeness checks only. Neither mode consumes this
|
|
100
|
+
sidecar. The separate [scoped generation verification
|
|
101
|
+
report](verification-report.md) binds a verifier outcome to this table's
|
|
102
|
+
identity and lets the generation manifest bind both to a wrapper without
|
|
103
|
+
changing that distinction.
|
|
104
|
+
|
|
105
|
+
## Compatibility and stop-condition result
|
|
106
|
+
|
|
107
|
+
The prototype is intentionally absent from the default `ibex` require path,
|
|
108
|
+
the generator CLI, `Codegen::Ruby`, generated constants, and `ibex-runtime`.
|
|
109
|
+
The explicitly required bundle API can publish it as a companion artifact.
|
|
110
|
+
Existing generation therefore emits byte-for-byte the same output when the
|
|
111
|
+
feature is not explicitly used, and runtime packaging has no new dependency.
|
|
112
|
+
|
|
113
|
+
The V002 stop conditions do not apply to this scoped adoption:
|
|
114
|
+
|
|
115
|
+
- wrapper/table separation does not change the Stable ABI because the sidecar
|
|
116
|
+
is separate Experimental data;
|
|
117
|
+
- direct bounded recognition supplies execution value beyond duplicating
|
|
118
|
+
Automaton IR; and
|
|
119
|
+
- action bodies remain opaque wrapper authority, so loading or parsing them is
|
|
120
|
+
not required.
|
|
121
|
+
|
|
122
|
+
Promoting the sidecar into default generated output, accepting it in
|
|
123
|
+
`ibex-runtime`, binding action calling conventions, or promising compatibility
|
|
124
|
+
beyond schema version 1 requires a separate compatibility decision and
|
|
125
|
+
migration plan.
|
|
@@ -0,0 +1,219 @@
|
|
|
1
|
+
# Runtime test interactions
|
|
2
|
+
|
|
3
|
+
The suite uses a bounded product for the common generated-parser pipeline and
|
|
4
|
+
focused tests for runtime features that do not fit that product. Matrix size is
|
|
5
|
+
not a proxy for coverage: every runtime-facing feature still needs an explicit
|
|
6
|
+
interaction assessment.
|
|
7
|
+
|
|
8
|
+
<!-- ibex-test-interaction-contract:start -->
|
|
9
|
+
```yaml
|
|
10
|
+
contract_version: 1
|
|
11
|
+
matrix:
|
|
12
|
+
axes:
|
|
13
|
+
algorithm: [slr, lalr, ielr, lr1]
|
|
14
|
+
table: [plain, compact]
|
|
15
|
+
cst: ["off", "on"]
|
|
16
|
+
locations: ["off", "on"]
|
|
17
|
+
entries: [single, multi, isolated]
|
|
18
|
+
expected_cases: 96
|
|
19
|
+
representative_cases: 12
|
|
20
|
+
roles:
|
|
21
|
+
normal: representative
|
|
22
|
+
scheduled: full
|
|
23
|
+
promotion: full_review
|
|
24
|
+
scheduled_long_gates: ["test:matrix:full", "test:adversarial", "fuzz:long"]
|
|
25
|
+
golden:
|
|
26
|
+
task: "test:zero_cost"
|
|
27
|
+
policy: feature_off_byte_identity
|
|
28
|
+
digest_path: test/golden/digests.yml
|
|
29
|
+
sources:
|
|
30
|
+
- test/fixtures/compat/calculator.y
|
|
31
|
+
- benchmark/grammars/representative.y
|
|
32
|
+
- gallery/json/grammar.y
|
|
33
|
+
update_command: "bundle exec rake golden:update"
|
|
34
|
+
interactions:
|
|
35
|
+
- id: parser_semantics
|
|
36
|
+
axes: [algorithm, table, cst, locations, entries]
|
|
37
|
+
coverage: matrix
|
|
38
|
+
tests: [test/tooling/matrix_runner_test.rb]
|
|
39
|
+
- id: generated_lexer
|
|
40
|
+
axes: [algorithm, table, cst, locations, entries]
|
|
41
|
+
coverage: matrix_and_focused
|
|
42
|
+
tests: [test/codegen/lexer_test.rb, test/frontend/lexer_test.rb]
|
|
43
|
+
- id: semantic_actions
|
|
44
|
+
axes: [table, locations, cst]
|
|
45
|
+
coverage: matrix_and_focused
|
|
46
|
+
tests: [test/runtime/action_contract_test.rb, test/analysis_no_exec_test.rb]
|
|
47
|
+
- id: table_encoding
|
|
48
|
+
axes: [algorithm, table, cst, entries]
|
|
49
|
+
coverage: matrix_and_focused
|
|
50
|
+
tests: [test/tables_test.rb, test/runtime/table_format_test.rb]
|
|
51
|
+
- id: locations
|
|
52
|
+
axes: [locations, table, cst, entries]
|
|
53
|
+
coverage: matrix_and_focused
|
|
54
|
+
tests: [test/location_test.rb, test/runtime/action_contract_test.rb]
|
|
55
|
+
- id: cst
|
|
56
|
+
axes: [cst, algorithm, table, locations, entries]
|
|
57
|
+
coverage: matrix_and_focused
|
|
58
|
+
tests: [test/codegen/cst_runtime_integration_test.rb, test/runtime/cst_serialize_test.rb]
|
|
59
|
+
- id: entry_modes
|
|
60
|
+
axes: [entries, algorithm, table, cst]
|
|
61
|
+
coverage: matrix_and_focused
|
|
62
|
+
tests: [test/codegen/multiple_start_test.rb]
|
|
63
|
+
- id: parser_drivers
|
|
64
|
+
axes: [table, cst, locations]
|
|
65
|
+
coverage: focused_regression
|
|
66
|
+
tests: [test/runtime/parser_test.rb, test/runtime/push_parser_test.rb, test/runtime/parser_table_session_test.rb]
|
|
67
|
+
- id: generated_ast
|
|
68
|
+
axes: [algorithm, table, locations]
|
|
69
|
+
coverage: focused_regression
|
|
70
|
+
tests: [test/codegen/ast_test.rb, test/tooling/matrix_runner_test.rb]
|
|
71
|
+
- id: recovery
|
|
72
|
+
axes: [algorithm, table, cst, entries]
|
|
73
|
+
coverage: focused_regression
|
|
74
|
+
tests: [test/runtime/repair_test.rb, test/runtime/sync_recovery_test.rb]
|
|
75
|
+
- id: resource_limits
|
|
76
|
+
axes: [algorithm, table, cst, entries]
|
|
77
|
+
coverage: matrix_and_focused
|
|
78
|
+
tests: [test/runtime/resource_limits_test.rb]
|
|
79
|
+
- id: observation
|
|
80
|
+
axes: [table, cst, locations]
|
|
81
|
+
coverage: focused_regression
|
|
82
|
+
tests: [test/runtime/observation_test.rb, test/codegen/embedded_tracer_test.rb]
|
|
83
|
+
- id: incremental_cst
|
|
84
|
+
axes: [cst, table, locations]
|
|
85
|
+
coverage: focused_regression
|
|
86
|
+
tests: [test/runtime/cst_incremental_test.rb]
|
|
87
|
+
- id: syntax_session
|
|
88
|
+
axes: [algorithm, table, cst, locations, entries]
|
|
89
|
+
coverage: focused_regression
|
|
90
|
+
tests: [test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
|
|
91
|
+
- id: embedded_runtime
|
|
92
|
+
axes: [table, cst]
|
|
93
|
+
coverage: focused_regression
|
|
94
|
+
tests: [test/packaging/runtime_gem_test.rb, test/codegen/ractor_shareability_test.rb]
|
|
95
|
+
```
|
|
96
|
+
<!-- ibex-test-interaction-contract:end -->
|
|
97
|
+
|
|
98
|
+
The quality validator compares this record with `test/matrix.yml`, the CI
|
|
99
|
+
workflow, the golden generator, and the files named by every interaction.
|
|
100
|
+
|
|
101
|
+
## The 96-case product
|
|
102
|
+
|
|
103
|
+
The full matrix is exactly:
|
|
104
|
+
|
|
105
|
+
```text
|
|
106
|
+
4 algorithms × 2 table encodings × 2 CST states × 2 location states × 3 entry modes = 96
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
The algorithm axis is SLR, LALR, IELR, and canonical LR(1). The table axis is
|
|
110
|
+
plain and compact. CST and semantic locations are independently off or on. The
|
|
111
|
+
entry axis is a single start, shared multi-start automaton, or isolated
|
|
112
|
+
multi-start construction.
|
|
113
|
+
|
|
114
|
+
Each case normalizes and round-trips Grammar IR, builds and round-trips
|
|
115
|
+
Automaton IR, proves deterministic generated bytes, executes valid and invalid
|
|
116
|
+
input, compares algorithm/table behavior within the same CST/location/entry
|
|
117
|
+
shape, and enforces a stack resource limit. The runner separately checks
|
|
118
|
+
generated AST node equivalence for every algorithm/table pair.
|
|
119
|
+
|
|
120
|
+
The normal pull-request gate chooses twelve deterministic representatives from
|
|
121
|
+
the ordered product. Twelve is a fast regression net, not a claim of pairwise
|
|
122
|
+
coverage and not a substitute for feature-focused tests. Scheduled CI runs all
|
|
123
|
+
96 cases plus bounded hostile-input tests and the 100,000-sentence fuzz run.
|
|
124
|
+
Promotion of a Preview or Experimental runtime feature requires a reviewed
|
|
125
|
+
full-96 result on the candidate revision even if scheduled CI was recently
|
|
126
|
+
green.
|
|
127
|
+
|
|
128
|
+
## Interaction map and covering rules
|
|
129
|
+
|
|
130
|
+
The machine-readable map names the current runtime-facing families and their
|
|
131
|
+
owned tests. `matrix` means the 96-product is the primary coverage.
|
|
132
|
+
`matrix_and_focused` combines product coverage with semantic edge cases.
|
|
133
|
+
`focused_regression` means the feature is not an independent matrix axis and
|
|
134
|
+
the named suite owns its current protocol and edge cases. It does not claim
|
|
135
|
+
that every listed axis pair is already covered; a change to that row must apply
|
|
136
|
+
the covering rules below and record the chosen combinations in its tests.
|
|
137
|
+
|
|
138
|
+
For a new runtime-facing feature:
|
|
139
|
+
|
|
140
|
+
1. List every existing axis and feature family whose state or observable result
|
|
141
|
+
it can affect.
|
|
142
|
+
2. Add it as a matrix axis only when all values are cheap, orthogonal, and
|
|
143
|
+
meaningful for the common pipeline. Update the declared product count in
|
|
144
|
+
the same change.
|
|
145
|
+
3. Otherwise select a deterministic covering set containing every new value,
|
|
146
|
+
every affected existing value, and each pair the implementation says can
|
|
147
|
+
interact. Record the selection rule and own it in a focused test.
|
|
148
|
+
4. Use feature-specific exhaustive tests for small finite state spaces,
|
|
149
|
+
protocol transitions, version readers, and failure-before-input behavior.
|
|
150
|
+
5. Add scheduled long fuzz/property cases only when shorter deterministic tests
|
|
151
|
+
cannot provide the same signal. Seeds, counts, and budgets stay fixed.
|
|
152
|
+
|
|
153
|
+
Pairwise coverage is the minimum for declared interactions, not permission to
|
|
154
|
+
ignore a known three-way dependency. A bug involving three or more settings is
|
|
155
|
+
promoted to a permanent focused regression and its interaction is added to the
|
|
156
|
+
map. Unsupported combinations must be rejected explicitly and tested; omitting
|
|
157
|
+
them from a generator is not coverage.
|
|
158
|
+
|
|
159
|
+
Current non-product boundaries include embedded versus installed runtime,
|
|
160
|
+
pull/`yyparse`/push driver transitions, action marker versions, recovery and
|
|
161
|
+
repair state, observation hooks, incremental reuse, serialization, Ractor
|
|
162
|
+
shareability, and packaging. Their focused suites remain mandatory even though
|
|
163
|
+
they do not multiply the 96 common cases.
|
|
164
|
+
|
|
165
|
+
Syntax sessions require CST and a generated lexer, so a disabled-CST value is
|
|
166
|
+
rejected by the service boundary rather than multiplied into the common
|
|
167
|
+
product. `test/runtime/syntax_session_test.rb` exhaustively crosses all four
|
|
168
|
+
algorithms, both table encodings, and shared versus isolated multi-entry
|
|
169
|
+
construction (16 cases). A separate four-case cross covers both table encodings
|
|
170
|
+
with grammar actions that omit or reference semantic locations. Production
|
|
171
|
+
actions, including their semantic-location references, never execute in a
|
|
172
|
+
syntax session, so that location-action choice is independent of algorithm and
|
|
173
|
+
entry construction; CST source locations remain mandatory internally. The
|
|
174
|
+
remaining tests own trust acknowledgement, fresh-result equivalence, fallback
|
|
175
|
+
metrics, cancellation, resource bounds, and malformed edit sequences.
|
|
176
|
+
`test/packaging/runtime_gem_test.rb` owns the embedded-runtime acquisition path.
|
|
177
|
+
|
|
178
|
+
## Normal, scheduled, and promotion gates
|
|
179
|
+
|
|
180
|
+
Normal pushes and pull requests run the twelve representative cases and the
|
|
181
|
+
feature-off golden check as part of the bounded safety net. The weekly schedule
|
|
182
|
+
runs `test:matrix:full`, long adversarial settings, and `fuzz:long`. Manual
|
|
183
|
+
workflow dispatch does not currently run the full matrix or long fuzz merely by
|
|
184
|
+
being manual; use the explicit local tasks when preparing a promotion.
|
|
185
|
+
|
|
186
|
+
A promotion review records the exact candidate revision and includes:
|
|
187
|
+
|
|
188
|
+
- all 96 matrix cases;
|
|
189
|
+
- every focused suite named by the promoted feature's interaction row;
|
|
190
|
+
- version-reader, invalid-version, and regeneration diagnostics when an ABI is
|
|
191
|
+
involved;
|
|
192
|
+
- the feature-off golden diff; and
|
|
193
|
+
- any scheduled long property or fuzz evidence assigned to the feature.
|
|
194
|
+
|
|
195
|
+
Scheduled success is supporting evidence only. It is not a silent maturity
|
|
196
|
+
promotion and does not replace review of the exact candidate change.
|
|
197
|
+
|
|
198
|
+
## Golden zero-cost policy
|
|
199
|
+
|
|
200
|
+
`test:zero_cost` regenerates three committed parsers and requires both exact
|
|
201
|
+
bytes and their SHA-256 index to match. When a new feature is disabled, its
|
|
202
|
+
implementation must not change these reviewed generated bytes. This is a
|
|
203
|
+
source-size and emitted-code zero-cost contract for the fixed corpus, not a
|
|
204
|
+
runtime performance or allocation claim.
|
|
205
|
+
|
|
206
|
+
If a cross-cutting correction intentionally changes feature-off output, run
|
|
207
|
+
`bundle exec rake golden:update` and review the generated source diff and
|
|
208
|
+
digest index in the same pull request. Never update the digest merely to make a
|
|
209
|
+
failure green. Add a new representative source when the existing three cannot
|
|
210
|
+
exercise a stable feature-off boundary.
|
|
211
|
+
|
|
212
|
+
## Merge rule
|
|
213
|
+
|
|
214
|
+
A change under the runtime-facing paths in
|
|
215
|
+
[the ABI policy](runtime-abi-evolution.md#pull-request-abi-assessment) cannot
|
|
216
|
+
merge without a valid structured ABI assessment. The automated check verifies
|
|
217
|
+
the declared versions, matrix arithmetic, scheduled commands, golden inputs,
|
|
218
|
+
and assessment structure. Human review still decides whether the chosen ABI
|
|
219
|
+
boundary and interaction set match the behavior of the change.
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Verification report
|
|
3
|
+
description: Validate parser-table artifacts and understand the independent verifier trust boundary.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Scoped generation verification report
|
|
7
|
+
|
|
8
|
+
Ibex can explicitly render a verifiable generation bundle containing a
|
|
9
|
+
data-only parser table, generated Ruby wrapper, scoped verification report,
|
|
10
|
+
and generation manifest. This Experimental API is opt-in and is not part of
|
|
11
|
+
default CLI generation:
|
|
12
|
+
|
|
13
|
+
```ruby
|
|
14
|
+
require "ibex/verifiable_generation_bundle"
|
|
15
|
+
|
|
16
|
+
bundle = Ibex::VerifiableGenerationBundle.new(
|
|
17
|
+
automaton,
|
|
18
|
+
wrapper_path: "build/parser.rb",
|
|
19
|
+
wrapper_source: generated_ruby,
|
|
20
|
+
table_path: "build/parser.tables.ibex.json",
|
|
21
|
+
report_path: "build/parser.verification.ibex.json",
|
|
22
|
+
manifest_path: "build/parser.ibex.json",
|
|
23
|
+
source_records: generation_inputs,
|
|
24
|
+
manifest_options: { "table" => "compact" },
|
|
25
|
+
strict: true
|
|
26
|
+
)
|
|
27
|
+
bundle.publish
|
|
28
|
+
|
|
29
|
+
Ibex::VerifiableGenerationBundle.validate_file("build/parser.ibex.json")
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
`render` returns the four in-memory artifacts without publishing them.
|
|
33
|
+
`publish` passes the complete artifact set and exact input records to the
|
|
34
|
+
existing generation transaction. The transaction installs the table and
|
|
35
|
+
report, then the parser wrapper, and installs the manifest last as the
|
|
36
|
+
coherence marker. Rendering or verification failure changes no target.
|
|
37
|
+
|
|
38
|
+
## Claims and outcomes
|
|
39
|
+
|
|
40
|
+
The version 1 report is a closed JSON document described by
|
|
41
|
+
[`schema/verification-report-v1.schema.json`](../schema/verification-report-v1.schema.json).
|
|
42
|
+
It records:
|
|
43
|
+
|
|
44
|
+
- logical input identities, content digests, and byte sizes;
|
|
45
|
+
- canonical Grammar IR and Automaton IR digests and the construction
|
|
46
|
+
algorithm;
|
|
47
|
+
- the data-only table artifact and payload digests, schema version, and
|
|
48
|
+
representation;
|
|
49
|
+
- the Ibex checker version, default or strict profile, requested checks,
|
|
50
|
+
completed checks, and state/item bounds; and
|
|
51
|
+
- a distinct `pass`, `violations`, or `exhausted` outcome.
|
|
52
|
+
|
|
53
|
+
A completed default report names V1, V3, V4, V6, V7, and V8. A completed
|
|
54
|
+
strict report additionally names V2, V5, and V9. V9 executes only for IELR
|
|
55
|
+
and records a bounded canonical acceptance witness. An exhausted report conservatively
|
|
56
|
+
records no completed checks, includes the requested profile and bounds, and
|
|
57
|
+
is not a pass. The meanings and limitations of V1-V9 remain those in
|
|
58
|
+
[`verifier-trust-boundary.md`](verifier-trust-boundary.md).
|
|
59
|
+
|
|
60
|
+
The report's `evidence_digest` covers the complete report except that digest
|
|
61
|
+
field itself. Input identities use `input/NNNN/BASENAME`, and the table uses
|
|
62
|
+
`table/BASENAME`. These logical identities preserve ordering and artifact
|
|
63
|
+
roles without embedding an absolute checkout path. Both forms are closed:
|
|
64
|
+
`NNNN` is exactly four decimal digits, a report contains at most 10,000 input
|
|
65
|
+
records, and `BASENAME` is one non-empty component other than `.` or `..`
|
|
66
|
+
without slash, backslash, or control characters.
|
|
67
|
+
|
|
68
|
+
The `ir.identity_scope` value `source-logical-v1` defines the versioned digest
|
|
69
|
+
scope. Before building the bundle table, Ibex rebuilds an immutable Automaton
|
|
70
|
+
IR copy in which every source-location `file` is the corresponding indexed
|
|
71
|
+
input identity and every non-null source-provenance `root` is `input`. The
|
|
72
|
+
Grammar IR digest, embedded Automaton IR grammar digest, Automaton IR digest,
|
|
73
|
+
table source identities, report IR claims, and evidence digest all derive
|
|
74
|
+
from that same copy. Every IR source file must map uniquely to
|
|
75
|
+
`source_records`; an absent or ambiguous mapping is rejected.
|
|
76
|
+
|
|
77
|
+
This normalization is limited to the explicit bundle/report boundary. It does
|
|
78
|
+
not mutate the supplied Automaton, change diagnostic locations, or alter the
|
|
79
|
+
ordinary V002 `TableArtifact.build(automaton)` identity. Lower-level callers
|
|
80
|
+
that render a report directly can obtain the required copy with
|
|
81
|
+
`VerificationReport.canonical_automaton`; pairing a raw-path table with the
|
|
82
|
+
canonical report is rejected as an IR/table mismatch.
|
|
83
|
+
|
|
84
|
+
## Non-cyclic manifest binding
|
|
85
|
+
|
|
86
|
+
The report binds input, Grammar IR, Automaton IR, table artifact, and table
|
|
87
|
+
payload identities. It deliberately contains neither a manifest path nor a
|
|
88
|
+
manifest digest. After the report is rendered, the generation manifest lists
|
|
89
|
+
and hashes the table, wrapper, and report. This gives one direction of
|
|
90
|
+
dependency:
|
|
91
|
+
|
|
92
|
+
```text
|
|
93
|
+
input + IR -> table -> verification report -> generation manifest
|
|
94
|
+
wrapper ---------------> generation manifest
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Adding the manifest digest to the report would require the report hash before
|
|
98
|
+
the manifest can be completed and the manifest hash before the report can be
|
|
99
|
+
completed. Version 1 rejects that circular format.
|
|
100
|
+
|
|
101
|
+
`validate_bundle` checks supplied manifest, report, and table bytes without
|
|
102
|
+
loading generated Ruby. `validate_bundle_file` additionally resolves all
|
|
103
|
+
artifacts from the manifest and verifies every manifest byte count and digest.
|
|
104
|
+
Both reject a missing or duplicate report/table/wrapper entry, stale report or
|
|
105
|
+
table bytes, mismatched logical table identity, input mismatch, IR/table
|
|
106
|
+
identity mismatch, payload mismatch, and an invalid report evidence digest.
|
|
107
|
+
Digests detect inconsistency; they are not signatures or authenticity claims.
|
|
108
|
+
|
|
109
|
+
## Excluded trust surface
|
|
110
|
+
|
|
111
|
+
This artifact is a **verification report**, not a proof-carrying parser. The
|
|
112
|
+
closed `excluded_trust` field always identifies the following boundaries:
|
|
113
|
+
|
|
114
|
+
- source parsing and normalization into IR;
|
|
115
|
+
- semantic derivation or language equivalence from Automaton IR to the
|
|
116
|
+
persisted table (the sidecar receives closed structural validation and
|
|
117
|
+
digest binding only);
|
|
118
|
+
- generated wrapper structure and behavior;
|
|
119
|
+
- semantic action meaning, safety, termination, and side effects;
|
|
120
|
+
- generated or handwritten lexer actions and tokenization;
|
|
121
|
+
- the Ruby and Ibex runtime implementations;
|
|
122
|
+
- application hooks; and
|
|
123
|
+
- grammar unambiguity.
|
|
124
|
+
|
|
125
|
+
The manifest binds the wrapper's bytes to the same publication, but neither
|
|
126
|
+
the report builder nor validator parses, loads, or executes those bytes.
|
|
127
|
+
Likewise, the table sidecar exposes opaque production action slots with
|
|
128
|
+
`verified: false`. V1-V9 rebuild internal table views but do not consume the
|
|
129
|
+
persisted sidecar or prove the builder's Automaton-IR-to-sidecar projection.
|
|
130
|
+
A `pass` applies only to the named V1-V9 checks over the supplied Automaton IR
|
|
131
|
+
plus the reported structural validation and digest relationships.
|
|
132
|
+
|
|
133
|
+
The checker, IR validator and serializer, reference collection, set analysis,
|
|
134
|
+
table construction/validation, JSON and digest implementations, host runtime,
|
|
135
|
+
and publication filesystem remain trusted as documented. Hostile input also
|
|
136
|
+
requires outer byte, time, memory, and process limits beyond the two reference
|
|
137
|
+
collection bounds.
|