ibex 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (461) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +100 -272
  3. data/docs/architecture.md +187 -35
  4. data/docs/bison-import.md +103 -0
  5. data/docs/comparison-policy.md +127 -0
  6. data/docs/configuration-model.md +59 -0
  7. data/docs/configuration-report.md +28 -0
  8. data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
  9. data/docs/conflict-explanation-study.md +62 -0
  10. data/docs/construction-profiling.md +157 -0
  11. data/docs/cst-migration.md +3 -4
  12. data/docs/cst.md +24 -0
  13. data/docs/decisions/0000-template.md +24 -0
  14. data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
  15. data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
  16. data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
  17. data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
  18. data/docs/decisions/0005-contained-grammar-composition.md +34 -0
  19. data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
  20. data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
  21. data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
  22. data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
  23. data/docs/decisions/0010-committed-runtime-observation.md +34 -0
  24. data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
  25. data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
  26. data/docs/decisions/0013-transactional-generation-publication.md +37 -0
  27. data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
  28. data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
  29. data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
  30. data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
  31. data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
  32. data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
  33. data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
  34. data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
  35. data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
  36. data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
  37. data/docs/decisions/0024-direct-ielr-construction.md +42 -0
  38. data/docs/decisions/README.md +74 -0
  39. data/docs/declarative-configuration.md +274 -0
  40. data/docs/development.md +196 -2
  41. data/docs/direct-ielr-decision.md +120 -0
  42. data/docs/direct-multi-entry-decision.md +82 -0
  43. data/docs/editor-setup.md +2 -1
  44. data/docs/error-ux-review-rubric-v1.md +117 -0
  45. data/docs/error-ux-reviews/v1/records/README.md +22 -0
  46. data/docs/error-ux-round2-review-status-v1.json +27 -0
  47. data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
  48. data/docs/error-ux-round2-v1.json +1617 -0
  49. data/docs/error-ux-round2.md +111 -0
  50. data/docs/error-ux.md +54 -0
  51. data/docs/getting-started.md +80 -0
  52. data/docs/grammar-reference.md +184 -13
  53. data/docs/ielr-design.md +43 -0
  54. data/docs/ielr.md +92 -0
  55. data/docs/impact.md +101 -0
  56. data/docs/ja/bison-import.md +51 -0
  57. data/docs/ja/stable-api.md +53 -0
  58. data/docs/lexer-construction-profile.md +109 -0
  59. data/docs/lexer-migration.md +5 -0
  60. data/docs/maturity.md +129 -0
  61. data/docs/project-site-strategy.md +54 -0
  62. data/docs/racc-migration-evidence.md +45 -0
  63. data/docs/racc-migration.md +11 -0
  64. data/docs/release-readiness.md +126 -68
  65. data/docs/repair-semantics.md +123 -0
  66. data/docs/runtime-abi-evolution.md +310 -0
  67. data/docs/stability.md +145 -34
  68. data/docs/status.md +43 -0
  69. data/docs/syntax-sessions.md +183 -0
  70. data/docs/table-artifact.md +125 -0
  71. data/docs/test-interactions.md +219 -0
  72. data/docs/verification-report.md +137 -0
  73. data/docs/verifier-trust-boundary.md +199 -0
  74. data/docs/workloads.md +132 -0
  75. data/lib/ibex/analysis/digraph.rb +128 -0
  76. data/lib/ibex/analysis/sets.rb +14 -0
  77. data/lib/ibex/analysis.rb +1 -0
  78. data/lib/ibex/bison_import/importer.rb +516 -0
  79. data/lib/ibex/bison_import/tokenizer.rb +260 -0
  80. data/lib/ibex/bison_import.rb +200 -0
  81. data/lib/ibex/bounded_subprocess.rb +171 -0
  82. data/lib/ibex/cli/ambiguity.rb +14 -5
  83. data/lib/ibex/cli/analysis.rb +134 -0
  84. data/lib/ibex/cli/bison_import.rb +121 -0
  85. data/lib/ibex/cli/config.rb +128 -0
  86. data/lib/ibex/cli/diagnostics.rb +31 -11
  87. data/lib/ibex/cli/documentation.rb +11 -8
  88. data/lib/ibex/cli/equiv.rb +169 -0
  89. data/lib/ibex/cli/error_messages.rb +11 -4
  90. data/lib/ibex/cli/explain.rb +14 -5
  91. data/lib/ibex/cli/fix.rb +203 -0
  92. data/lib/ibex/cli/formatting.rb +11 -10
  93. data/lib/ibex/cli/fuzz.rb +200 -0
  94. data/lib/ibex/cli/fuzz_regressions.rb +145 -0
  95. data/lib/ibex/cli/generation_artifacts.rb +53 -2
  96. data/lib/ibex/cli/generation_error_messages.rb +1 -1
  97. data/lib/ibex/cli/grammar_tests.rb +32 -14
  98. data/lib/ibex/cli/impact.rb +393 -0
  99. data/lib/ibex/cli/ir_tools.rb +5 -61
  100. data/lib/ibex/cli/outputs.rb +60 -33
  101. data/lib/ibex/cli/reduce.rb +202 -0
  102. data/lib/ibex/cli/reduce_reporting.rb +71 -0
  103. data/lib/ibex/cli/samples.rb +49 -23
  104. data/lib/ibex/cli/verify.rb +116 -0
  105. data/lib/ibex/cli/watch.rb +6 -5
  106. data/lib/ibex/cli.rb +362 -56
  107. data/lib/ibex/codegen/action_locations.rb +1 -1
  108. data/lib/ibex/codegen/action_method_source.rb +5 -6
  109. data/lib/ibex/codegen/action_source.rb +3 -2
  110. data/lib/ibex/codegen/ambiguity.rb +5 -4
  111. data/lib/ibex/codegen/explain.rb +123 -55
  112. data/lib/ibex/codegen/generated_action_abi.rb +10 -5
  113. data/lib/ibex/codegen/rbs.rb +15 -7
  114. data/lib/ibex/codegen/report.rb +25 -13
  115. data/lib/ibex/codegen/ruby.rb +28 -64
  116. data/lib/ibex/codegen/ruby_actions.rb +9 -6
  117. data/lib/ibex/codegen/ruby_syntax.rb +15 -9
  118. data/lib/ibex/configuration/analysis_grammar.rb +35 -0
  119. data/lib/ibex/configuration/explanation.rb +421 -0
  120. data/lib/ibex/configuration/inspector.rb +232 -0
  121. data/lib/ibex/configuration.rb +580 -0
  122. data/lib/ibex/coverage/collector.rb +21 -11
  123. data/lib/ibex/coverage/event_stream.rb +13 -7
  124. data/lib/ibex/coverage/report.rb +26 -18
  125. data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
  126. data/lib/ibex/delta_reducer.rb +99 -0
  127. data/lib/ibex/diff.rb +141 -0
  128. data/lib/ibex/equiv/machine.rb +135 -0
  129. data/lib/ibex/equiv.rb +373 -0
  130. data/lib/ibex/fix.rb +557 -0
  131. data/lib/ibex/frontend/ast.rb +22 -2
  132. data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
  133. data/lib/ibex/frontend/diagnostic.rb +1 -1
  134. data/lib/ibex/frontend/formatter.rb +72 -26
  135. data/lib/ibex/frontend/generated_parser.rb +147 -117
  136. data/lib/ibex/frontend/generated_parser_base.rb +7 -3
  137. data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
  138. data/lib/ibex/frontend/generation.rb +1 -1
  139. data/lib/ibex/frontend/parser/declarations.rb +17 -2
  140. data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
  141. data/lib/ibex/frontend/regenerator.rb +1 -0
  142. data/lib/ibex/frontend/resolution.rb +2 -1
  143. data/lib/ibex/frontend/resolver.rb +1 -1
  144. data/lib/ibex/frontend/rule_documentation.rb +2 -1
  145. data/lib/ibex/frontend/source_cursor.rb +2 -2
  146. data/lib/ibex/frontend/source_span.rb +9 -1
  147. data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
  148. data/lib/ibex/frontend/token_adapter.rb +10 -0
  149. data/lib/ibex/frontend.rb +1 -0
  150. data/lib/ibex/fuzz.rb +219 -0
  151. data/lib/ibex/generation_input.rb +1 -1
  152. data/lib/ibex/generation_manifest.rb +36 -25
  153. data/lib/ibex/generation_transaction.rb +1 -1
  154. data/lib/ibex/generation_transaction_recovery.rb +4 -4
  155. data/lib/ibex/generation_transaction_validation.rb +2 -2
  156. data/lib/ibex/grammar_tests.rb +1 -1
  157. data/lib/ibex/impact/action_impact.rb +148 -0
  158. data/lib/ibex/impact/automaton_impact.rb +57 -0
  159. data/lib/ibex/impact/coverage_impact.rb +53 -0
  160. data/lib/ibex/impact/graph.rb +132 -0
  161. data/lib/ibex/impact/propagation.rb +147 -0
  162. data/lib/ibex/impact/report.rb +273 -0
  163. data/lib/ibex/impact/seeds.rb +69 -0
  164. data/lib/ibex/impact/severity.rb +139 -0
  165. data/lib/ibex/impact.rb +16 -0
  166. data/lib/ibex/ir/automaton_ir.rb +61 -19
  167. data/lib/ibex/ir/grammar_ir.rb +76 -45
  168. data/lib/ibex/ir/lexer_ir.rb +5 -5
  169. data/lib/ibex/ir/parser_contract.rb +135 -0
  170. data/lib/ibex/ir/serialize.rb +203 -76
  171. data/lib/ibex/ir/validator/automaton.rb +64 -34
  172. data/lib/ibex/ir/validator/base.rb +51 -33
  173. data/lib/ibex/ir/validator/grammar.rb +113 -83
  174. data/lib/ibex/ir/validator/lexer.rb +2 -2
  175. data/lib/ibex/ir/validator.rb +2 -1
  176. data/lib/ibex/ir.rb +16 -3
  177. data/lib/ibex/lalr/build_metrics.rb +50 -2
  178. data/lib/ibex/lalr/builder.rb +191 -56
  179. data/lib/ibex/lalr/conflict_search.rb +15 -4
  180. data/lib/ibex/lalr/counterexample.rb +19 -14
  181. data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
  182. data/lib/ibex/lalr/goto_follows.rb +229 -0
  183. data/lib/ibex/lalr/ielr/annotator.rb +214 -0
  184. data/lib/ibex/lalr/ielr/bits.rb +28 -0
  185. data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
  186. data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
  187. data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
  188. data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
  189. data/lib/ibex/lalr/ielr/split_state.rb +20 -0
  190. data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
  191. data/lib/ibex/lalr/ielr_partition.rb +22 -8
  192. data/lib/ibex/lalr/inadequacy_report.rb +50 -0
  193. data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
  194. data/lib/ibex/lalr/lr0_collection.rb +121 -0
  195. data/lib/ibex/lalr/unreachable_states.rb +80 -0
  196. data/lib/ibex/lalr.rb +52 -1
  197. data/lib/ibex/location.rb +3 -3
  198. data/lib/ibex/lsp/document_handlers.rb +9 -7
  199. data/lib/ibex/lsp/initialization_handlers.rb +9 -7
  200. data/lib/ibex/lsp/navigation_handlers.rb +25 -11
  201. data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
  202. data/lib/ibex/lsp/position_codec.rb +6 -2
  203. data/lib/ibex/lsp/request_handlers.rb +3 -2
  204. data/lib/ibex/lsp/request_support.rb +14 -7
  205. data/lib/ibex/lsp/server.rb +13 -11
  206. data/lib/ibex/lsp/symbol_index.rb +13 -13
  207. data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
  208. data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
  209. data/lib/ibex/lsp/transport.rb +13 -3
  210. data/lib/ibex/lsp/workspace.rb +5 -4
  211. data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
  212. data/lib/ibex/lsp.rb +1 -0
  213. data/lib/ibex/messages/en.yml +28 -0
  214. data/lib/ibex/messages/ja.yml +28 -0
  215. data/lib/ibex/messages.rb +68 -0
  216. data/lib/ibex/metrics.rb +175 -0
  217. data/lib/ibex/normalize/declarations.rb +2 -0
  218. data/lib/ibex/normalize/diagnostics.rb +63 -38
  219. data/lib/ibex/normalize/expander.rb +5 -4
  220. data/lib/ibex/normalize/expression.rb +8 -6
  221. data/lib/ibex/normalize/grammar_builder.rb +26 -0
  222. data/lib/ibex/normalize/inline_expansion.rb +100 -49
  223. data/lib/ibex/normalize/lexer.rb +3 -3
  224. data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
  225. data/lib/ibex/normalize/parameter_lowering.rb +40 -23
  226. data/lib/ibex/normalize/parameters.rb +31 -4
  227. data/lib/ibex/normalize/parser_configuration.rb +70 -0
  228. data/lib/ibex/normalize.rb +10 -20
  229. data/lib/ibex/racc_migration/report.rb +6 -3
  230. data/lib/ibex/rake_task.rb +1 -1
  231. data/lib/ibex/samples.rb +43 -13
  232. data/lib/ibex/table_artifact/builder.rb +266 -0
  233. data/lib/ibex/table_artifact/cst_projection.rb +70 -0
  234. data/lib/ibex/table_artifact/document.rb +39 -0
  235. data/lib/ibex/table_artifact/executor.rb +187 -0
  236. data/lib/ibex/table_artifact/serializer.rb +57 -0
  237. data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
  238. data/lib/ibex/table_artifact/validator/support.rb +86 -0
  239. data/lib/ibex/table_artifact/validator/tables.rb +238 -0
  240. data/lib/ibex/table_artifact/validator.rb +253 -0
  241. data/lib/ibex/table_artifact.rb +88 -0
  242. data/lib/ibex/table_simulation/result.rb +6 -2
  243. data/lib/ibex/table_simulation/step.rb +3 -1
  244. data/lib/ibex/tables.rb +14 -0
  245. data/lib/ibex/verifiable_generation_bundle.rb +86 -0
  246. data/lib/ibex/verification_report/builder.rb +159 -0
  247. data/lib/ibex/verification_report/canonical_ir.rb +123 -0
  248. data/lib/ibex/verification_report/logical_path.rb +68 -0
  249. data/lib/ibex/verification_report/validator.rb +329 -0
  250. data/lib/ibex/verification_report.rb +69 -0
  251. data/lib/ibex/verify/action_correspondence.rb +138 -0
  252. data/lib/ibex/verify/language_witness.rb +295 -0
  253. data/lib/ibex/verify/reference_collection.rb +190 -0
  254. data/lib/ibex/verify/result.rb +73 -0
  255. data/lib/ibex/verify/verifier.rb +598 -0
  256. data/lib/ibex/verify.rb +15 -0
  257. data/lib/ibex/version.rb +1 -1
  258. data/lib/ibex/watch/runner.rb +10 -1
  259. data/lib/ibex/watch/source_snapshot.rb +13 -8
  260. data/lib/ibex.rb +11 -0
  261. data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
  262. data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
  263. data/schema/bison-import-v1.schema.json +106 -0
  264. data/schema/conflict-explanation-review-v1.schema.json +196 -0
  265. data/schema/conflict-explanation-study-v1.schema.json +528 -0
  266. data/schema/construction-profile-v1.schema.json +779 -0
  267. data/schema/diff-v1.schema.json +41 -0
  268. data/schema/direct-ielr-decision-v1.schema.json +200 -0
  269. data/schema/equiv-v1.schema.json +64 -0
  270. data/schema/error-ux-review-v1.schema.json +609 -0
  271. data/schema/error-ux-round2-review-v1.schema.json +191 -0
  272. data/schema/error-ux-round2-v1.schema.json +519 -0
  273. data/schema/explain-v1.schema.json +63 -1
  274. data/schema/fix-v1.schema.json +64 -0
  275. data/schema/fix-v2.schema.json +66 -0
  276. data/schema/fix-v3.schema.json +101 -0
  277. data/schema/fuzz-regression-v1.schema.json +79 -0
  278. data/schema/fuzz-v1.schema.json +114 -0
  279. data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
  280. data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
  281. data/schema/grammar-ir.schema.json +271 -0
  282. data/schema/ielr-benchmark-v1.schema.json +85 -0
  283. data/schema/impact-v1.schema.json +208 -0
  284. data/schema/lexer-profile-v1.schema.json +491 -0
  285. data/schema/metrics-v1.schema.json +60 -0
  286. data/schema/reduce-v1.schema.json +22 -0
  287. data/schema/reduce-v2.schema.json +63 -0
  288. data/schema/table-artifact-v1.schema.json +377 -0
  289. data/schema/verification-report-v1.schema.json +222 -0
  290. data/schema/verify-v1.schema.json +42 -0
  291. data/sig/ibex/analysis/digraph.rbs +19 -0
  292. data/sig/ibex/analysis/sets.rbs +10 -0
  293. data/sig/ibex/bison_import/importer.rbs +140 -0
  294. data/sig/ibex/bison_import/tokenizer.rbs +91 -0
  295. data/sig/ibex/bison_import.rbs +95 -0
  296. data/sig/ibex/bounded_subprocess.rbs +76 -0
  297. data/sig/ibex/cli/ambiguity.rbs +6 -0
  298. data/sig/ibex/cli/analysis.rbs +43 -0
  299. data/sig/ibex/cli/bison_import.rbs +32 -0
  300. data/sig/ibex/cli/config.rbs +35 -0
  301. data/sig/ibex/cli/diagnostics.rbs +16 -3
  302. data/sig/ibex/cli/documentation.rbs +5 -1
  303. data/sig/ibex/cli/equiv.rbs +49 -0
  304. data/sig/ibex/cli/error_messages.rbs +6 -0
  305. data/sig/ibex/cli/explain.rbs +6 -0
  306. data/sig/ibex/cli/fix.rbs +53 -0
  307. data/sig/ibex/cli/formatting.rbs +5 -1
  308. data/sig/ibex/cli/fuzz.rbs +47 -0
  309. data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
  310. data/sig/ibex/cli/generation_artifacts.rbs +21 -2
  311. data/sig/ibex/cli/generation_error_messages.rbs +2 -2
  312. data/sig/ibex/cli/grammar_tests.rbs +15 -3
  313. data/sig/ibex/cli/impact.rbs +95 -0
  314. data/sig/ibex/cli/ir_tools.rbs +7 -19
  315. data/sig/ibex/cli/outputs.rbs +9 -3
  316. data/sig/ibex/cli/reduce.rbs +60 -0
  317. data/sig/ibex/cli/reduce_reporting.rbs +27 -0
  318. data/sig/ibex/cli/samples.rbs +8 -0
  319. data/sig/ibex/cli/verify.rbs +28 -0
  320. data/sig/ibex/cli/watch.rbs +2 -2
  321. data/sig/ibex/cli.rbs +79 -5
  322. data/sig/ibex/codegen/action_method_source.rbs +4 -4
  323. data/sig/ibex/codegen/explain.rbs +45 -25
  324. data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
  325. data/sig/ibex/codegen/rbs.rbs +12 -10
  326. data/sig/ibex/codegen/report.rbs +12 -12
  327. data/sig/ibex/codegen/ruby.rbs +18 -15
  328. data/sig/ibex/codegen/ruby_actions.rbs +6 -6
  329. data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
  330. data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
  331. data/sig/ibex/configuration/explanation.rbs +195 -0
  332. data/sig/ibex/configuration/inspector.rbs +53 -0
  333. data/sig/ibex/configuration.rbs +242 -0
  334. data/sig/ibex/coverage/collector.rbs +16 -14
  335. data/sig/ibex/coverage/event_stream.rbs +12 -10
  336. data/sig/ibex/coverage/report.rbs +20 -18
  337. data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
  338. data/sig/ibex/delta_reducer.rbs +38 -0
  339. data/sig/ibex/diff.rbs +47 -0
  340. data/sig/ibex/equiv/machine.rbs +54 -0
  341. data/sig/ibex/equiv.rbs +129 -0
  342. data/sig/ibex/fix.rbs +138 -0
  343. data/sig/ibex/frontend/ast.rbs +33 -7
  344. data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
  345. data/sig/ibex/frontend/diagnostic.rbs +2 -2
  346. data/sig/ibex/frontend/formatter.rbs +38 -30
  347. data/sig/ibex/frontend/generated_parser.rbs +87 -77
  348. data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
  349. data/sig/ibex/frontend/parser/declarations.rbs +3 -0
  350. data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
  351. data/sig/ibex/frontend/resolution.rbs +3 -2
  352. data/sig/ibex/frontend/source_cursor.rbs +2 -2
  353. data/sig/ibex/frontend/source_span.rbs +5 -2
  354. data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
  355. data/sig/ibex/frontend/token_adapter.rbs +6 -0
  356. data/sig/ibex/fuzz.rbs +71 -0
  357. data/sig/ibex/generation_input.rbs +2 -2
  358. data/sig/ibex/generation_manifest.rbs +35 -32
  359. data/sig/ibex/generation_transaction_recovery.rbs +2 -2
  360. data/sig/ibex/generation_transaction_validation.rbs +2 -2
  361. data/sig/ibex/grammar_tests.rbs +2 -2
  362. data/sig/ibex/impact/action_impact.rbs +58 -0
  363. data/sig/ibex/impact/automaton_impact.rbs +37 -0
  364. data/sig/ibex/impact/coverage_impact.rbs +28 -0
  365. data/sig/ibex/impact/graph.rbs +67 -0
  366. data/sig/ibex/impact/propagation.rbs +56 -0
  367. data/sig/ibex/impact/report.rbs +107 -0
  368. data/sig/ibex/impact/seeds.rbs +36 -0
  369. data/sig/ibex/impact/severity.rbs +55 -0
  370. data/sig/ibex/impact.rbs +6 -0
  371. data/sig/ibex/ir/automaton_ir.rbs +30 -11
  372. data/sig/ibex/ir/grammar_ir.rbs +41 -27
  373. data/sig/ibex/ir/lexer_ir.rbs +8 -8
  374. data/sig/ibex/ir/parser_contract.rbs +69 -0
  375. data/sig/ibex/ir/serialize.rbs +63 -28
  376. data/sig/ibex/ir/validator/automaton.rbs +47 -44
  377. data/sig/ibex/ir/validator/base.rbs +35 -31
  378. data/sig/ibex/ir/validator/grammar.rbs +78 -67
  379. data/sig/ibex/ir/validator/lexer.rbs +4 -4
  380. data/sig/ibex/ir.rbs +9 -3
  381. data/sig/ibex/lalr/build_metrics.rbs +42 -2
  382. data/sig/ibex/lalr/builder.rbs +75 -24
  383. data/sig/ibex/lalr/conflict_search.rbs +5 -2
  384. data/sig/ibex/lalr/counterexample.rbs +17 -14
  385. data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
  386. data/sig/ibex/lalr/goto_follows.rbs +89 -0
  387. data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
  388. data/sig/ibex/lalr/ielr/bits.rbs +14 -0
  389. data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
  390. data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
  391. data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
  392. data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
  393. data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
  394. data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
  395. data/sig/ibex/lalr/ielr_partition.rbs +17 -5
  396. data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
  397. data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
  398. data/sig/ibex/lalr/lr0_collection.rbs +50 -0
  399. data/sig/ibex/lalr/unreachable_states.rbs +25 -0
  400. data/sig/ibex/lalr.rbs +11 -1
  401. data/sig/ibex/location.rbs +6 -6
  402. data/sig/ibex/lsp/document_handlers.rbs +8 -8
  403. data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
  404. data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
  405. data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
  406. data/sig/ibex/lsp/position_codec.rbs +8 -4
  407. data/sig/ibex/lsp/request_handlers.rbs +4 -4
  408. data/sig/ibex/lsp/request_support.rbs +8 -8
  409. data/sig/ibex/lsp/server.rbs +16 -16
  410. data/sig/ibex/lsp/symbol_index.rbs +20 -21
  411. data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
  412. data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
  413. data/sig/ibex/lsp/transport.rbs +18 -6
  414. data/sig/ibex/lsp/workspace.rbs +3 -2
  415. data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
  416. data/sig/ibex/messages.rbs +26 -0
  417. data/sig/ibex/metrics.rbs +50 -0
  418. data/sig/ibex/normalize/diagnostics.rbs +4 -4
  419. data/sig/ibex/normalize/expander.rbs +2 -2
  420. data/sig/ibex/normalize/expression.rbs +10 -8
  421. data/sig/ibex/normalize/grammar_builder.rbs +11 -0
  422. data/sig/ibex/normalize/inline_expansion.rbs +59 -50
  423. data/sig/ibex/normalize/lexer.rbs +4 -4
  424. data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
  425. data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
  426. data/sig/ibex/normalize/parameters.rbs +13 -8
  427. data/sig/ibex/normalize/parser_configuration.rbs +23 -0
  428. data/sig/ibex/normalize.rbs +11 -5
  429. data/sig/ibex/racc_migration/report.rbs +8 -6
  430. data/sig/ibex/rake_task.rbs +1 -1
  431. data/sig/ibex/samples.rbs +13 -4
  432. data/sig/ibex/table_artifact/builder.rbs +93 -0
  433. data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
  434. data/sig/ibex/table_artifact/document.rbs +27 -0
  435. data/sig/ibex/table_artifact/executor.rbs +66 -0
  436. data/sig/ibex/table_artifact/serializer.rbs +25 -0
  437. data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
  438. data/sig/ibex/table_artifact/validator/support.rbs +46 -0
  439. data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
  440. data/sig/ibex/table_artifact/validator.rbs +66 -0
  441. data/sig/ibex/table_artifact.rbs +32 -0
  442. data/sig/ibex/table_simulation/result.rbs +8 -4
  443. data/sig/ibex/table_simulation/step.rbs +4 -2
  444. data/sig/ibex/tables.rbs +10 -9
  445. data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
  446. data/sig/ibex/verification_report/builder.rbs +53 -0
  447. data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
  448. data/sig/ibex/verification_report/logical_path.rbs +34 -0
  449. data/sig/ibex/verification_report/validator.rbs +114 -0
  450. data/sig/ibex/verification_report.rbs +44 -0
  451. data/sig/ibex/verify/action_correspondence.rbs +50 -0
  452. data/sig/ibex/verify/language_witness.rbs +112 -0
  453. data/sig/ibex/verify/reference_collection.rbs +63 -0
  454. data/sig/ibex/verify/result.rbs +52 -0
  455. data/sig/ibex/verify/verifier.rbs +170 -0
  456. data/sig/ibex/verify.rbs +6 -0
  457. data/sig/ibex/watch/runner.rbs +12 -2
  458. data/sig/ibex/watch/source_snapshot.rbs +13 -9
  459. metadata +254 -12
  460. data/lib/ibex/ir/migration.rb +0 -120
  461. data/sig/ibex/ir/migration.rbs +0 -34
@@ -0,0 +1,183 @@
1
+ # Generated-language syntax sessions
2
+
3
+ The Experimental syntax-session API is a generated-language service boundary,
4
+ not an LSP implementation. It reuses the existing incremental Red/Green CST
5
+ engine and returns syntax only: parser production actions never run and no
6
+ semantic value is available.
7
+
8
+ ## Trust must be acknowledged
9
+
10
+ Current generated parser classes report exactly one execution profile:
11
+
12
+ ```ruby
13
+ Calculator.syntax_execution_profile
14
+ # => :trusted_application_code
15
+ ```
16
+
17
+ Opening a session requires the caller to acknowledge that profile:
18
+
19
+ ```ruby
20
+ session = Calculator.syntax_session(
21
+ "1 + 2",
22
+ execution_profile: :trusted_application_code
23
+ )
24
+ ```
25
+
26
+ Generated lexer actions execute while opening and editing a session because
27
+ they define token emission, conversion, and lexer-state changes. Loading the
28
+ generated Ruby class may already have executed `header`, `inner`, or `footer`
29
+ sections. The explicit keyword records caller intent; it is not a sandbox flag.
30
+ Omitting it raises `Ibex::Runtime::SyntaxSessionTrustError` before lexing.
31
+
32
+ There is no `:declarative` switch. That profile will require a separately
33
+ generated action-free parser artifact, a declarative built-in-only lexer, and
34
+ no user-code sections. The current generator does not produce such an artifact,
35
+ and overriding `syntax_execution_profile` cannot enable it.
36
+
37
+ Parser callbacks and generated lexer hooks are application code too. The
38
+ syntax-only guarantee is specifically that grammar production actions are
39
+ suppressed. Use this API only with generated artifacts already trusted by the
40
+ host process.
41
+
42
+ ## Opening, editing, and results
43
+
44
+ Input is a `String` or `Ibex::Runtime::CST::SourceText`. Edits are
45
+ `Ibex::Runtime::CST::TextEdit` values expressed as zero-based, half-open byte
46
+ ranges against the current source:
47
+
48
+ ```ruby
49
+ edit = Ibex::Runtime::CST::TextEdit.new(
50
+ start: 4,
51
+ delete_length: 1,
52
+ insert_text: "3"
53
+ )
54
+ result = session.apply_edits([edit])
55
+
56
+ result.revision # 1
57
+ result.syntax_root # immutable Red root
58
+ result.diagnostics # immutable diagnostics
59
+ result.expected_tokens # expected terminals observed at parser failures
60
+ result.status # :ok or :syntax_error
61
+ result.metrics # immutable reuse/fallback evidence
62
+ ```
63
+
64
+ `expected_tokens` reuses the generated parser's current ordinary or exact
65
+ lookahead-correction policy. With several recoverable failures it is the
66
+ first-seen union for that operation; it is not a completion ranking.
67
+
68
+ The result and metrics snapshots are frozen. Metrics always describe the
69
+ installed result of the completed operation. `token_count` is the number of
70
+ entries in that result's final token memo, including parser-supporting tokens
71
+ represented there. `reused_tokens` is the number of those entries whose Green
72
+ token identity was reused from the preceding memo. It is zero for an initial
73
+ open and for a full fresh-parse fallback. `reused_ratio` is the parser's
74
+ completed structural-reuse ratio (or lexical ratio when the Blender is
75
+ disabled), so it is not generally `reused_tokens / token_count`.
76
+
77
+ `fallback?` and `fallback_reasons` distinguish a clean incremental operation
78
+ from a successful fallback. A full lexical-error fallback reports no reuse.
79
+ A decomposition or memo-budget fallback may still report independently
80
+ validated lexical token reuse even though subtree reuse or memo retention was
81
+ declined. Every fallback remains a valid syntax result.
82
+
83
+ Fresh syntax sessions are the reference semantics. Incremental results must
84
+ match fresh results in source bytes, Green structure and flags, and diagnostic
85
+ content.
86
+
87
+ ## Syntax-only repair proposals
88
+
89
+ `SyntaxSession#repair` is an additive Experimental operation for editor-style
90
+ syntax proposals. It runs the bounded repair search on a fresh private parser,
91
+ captures token identity and byte ranges, applies the resulting
92
+ `CST::TextEdit` values to a new `CST::SourceText`, and validates the edited
93
+ source with another fresh syntax session. The originating session and source
94
+ are unchanged.
95
+
96
+ ```ruby
97
+ proposal = session.repair(token_text: { "PLUS" => "+" })
98
+ proposal.status # :accepted, :progress, :rejected, or a bounded failure
99
+ proposal.text_edits # immutable byte edits
100
+ proposal.updated_source # exact edited bytes, when validation ran
101
+ proposal.validation # fresh syntax-only result, when validation ran
102
+ ```
103
+
104
+ The result is deliberately syntax-only: it has no semantic `value`, never
105
+ replays production actions, and never invokes the application's `on_repair`
106
+ callback. `token_text` supplies source bytes only. Named inserted or replaced
107
+ tokens fail closed when no spelling is supplied; punctuation literals may be
108
+ derived conservatively. `:exhausted` and `:not_found` remain distinct bounded
109
+ outcomes, while overlapping edits and multiple selected repair segments are
110
+ reported as unavailable.
111
+
112
+ An accepted result requires zero diagnostics and complete source consumption.
113
+ For progress or rejection, `updated_source` retains the exact edited bytes and
114
+ the validation CST may expose only its consumed prefix, as permitted by the
115
+ existing error/early-accept CST contract. Cancellation and service limits raise
116
+ their existing exceptions and are never returned as successful proposals.
117
+
118
+ ## Validation, cancellation, and limits
119
+
120
+ A source passed directly as a String must have an ASCII-compatible Ruby
121
+ encoding. `SourceText` and `TextEdit` are explicit byte containers and have
122
+ already normalized their text to binary, so binary strings and invalid UTF-8
123
+ bytes remain supported. A direct non-ASCII-compatible source such as UTF-16 is
124
+ rejected instead of being silently reinterpreted. Edit shape, nonnegative
125
+ offsets and lengths, overlap, and source bounds are validated by the existing
126
+ `TextEdit` and `SourceText` contracts.
127
+
128
+ `SyntaxSessionLimits` bounds source size, edit count, and inserted bytes. The
129
+ existing `ResourceLimits` independently bounds parser stack/recovery work,
130
+ incremental decomposition, and memo storage:
131
+
132
+ ```ruby
133
+ limits = Ibex::Runtime::SyntaxSessionLimits.new(
134
+ max_source_bytes: 2 * 1024 * 1024,
135
+ max_edits_per_operation: 128,
136
+ max_inserted_bytes: 256 * 1024
137
+ )
138
+ parser_limits = Ibex::Runtime::ResourceLimits.new(
139
+ max_incremental_decomposed_nodes: 20_000,
140
+ max_session_memo_bytes: 8 * 1024 * 1024
141
+ )
142
+ ```
143
+
144
+ Exceeding a service or parser bound raises
145
+ `SyntaxSessionResourceLimitError`. It is never returned as an `:ok` result.
146
+ Incremental decomposition or memo exhaustion may instead use the existing
147
+ sound fresh-path fallback and records that decision in metrics.
148
+
149
+ Cancellation is cooperative:
150
+
151
+ ```ruby
152
+ token = Ibex::Runtime::CancellationToken.new
153
+ session = Calculator.syntax_session(
154
+ source,
155
+ execution_profile: :trusted_application_code,
156
+ cancellation: token
157
+ )
158
+
159
+ token.cancel!
160
+ session.apply_edits(edits) # raises SyntaxSessionCancelled
161
+ ```
162
+
163
+ Checks run before an operation and at synchronous parser runtime events. A
164
+ cancellation requested after the final event belongs to the next operation;
165
+ this keeps completion and publication one atomic session transition. A Ruby
166
+ lexer action that does not return cannot be preempted; resource bounds and
167
+ cancellation do not isolate its side effects. An edit cancelled at a checkpoint
168
+ or stopped by a hard resource error leaves the last completed source and result
169
+ installed, and is never confused with a successful parse. Calls to
170
+ `apply_edits`, `source_text`, and `result` are serialized by the session.
171
+
172
+ ## Scope and packaging
173
+
174
+ The prototype lives in `ibex-runtime`. A deployed generated parser needs only
175
+ the runtime's parser tables, lexer, CST engine, and session façade; requiring
176
+ the generator gem would unnecessarily add grammar construction concerns.
177
+ A separate `ibex-workbench` package is deferred until there is evidence for a
178
+ larger editor-service product boundary.
179
+
180
+ This API does not define LSP transport, workspace or file-system semantics,
181
+ cross-file indexing, query languages, formatting, or a declarative lexer. The
182
+ existing grammar-authoring LSP remains a separate static tool and does not load
183
+ generated application parsers.
@@ -0,0 +1,125 @@
1
+ # Data-only parser table artifact
2
+
3
+ The table artifact is an Experimental, separately versioned JSON sidecar for
4
+ inspecting and recognizing parser token streams without parsing or loading
5
+ generated Ruby. Version 1 is a prototype persistence and execution boundary;
6
+ it is not accepted directly by the current parser runtime and is not part of
7
+ the Stable generated-parser ABI.
8
+
9
+ Load the prototype explicitly:
10
+
11
+ ```ruby
12
+ require "ibex/table_artifact"
13
+
14
+ document = Ibex::TableArtifact.build(automaton, representation: :compact)
15
+ loaded = Ibex::TableArtifact.load(document.dump)
16
+ result = Ibex::TableArtifact::Executor.new(loaded).recognize([number_token_id])
17
+ result.accepted?
18
+ ```
19
+
20
+ `recognize` consumes internal terminal IDs. It does not lex source text,
21
+ convert external token values, run semantic actions, build semantic values, or
22
+ perform recovery search. Callers may select a named entry and must treat
23
+ `max_steps` as the execution bound for hostile or cyclic input.
24
+
25
+ ## Authority and representation
26
+
27
+ The artifact is executable table authority for its limited recognition
28
+ profile. The executor reads its actions, defaults, GOTOs, productions, and
29
+ entry states directly; it does not reconstruct an automaton or call the parser
30
+ builder. This is the execution benefit over retaining only Automaton IR.
31
+
32
+ The canonical payload contains:
33
+
34
+ - all symbol and terminal names, display names, and internal IDs;
35
+ - entry names and initial states;
36
+ - signed action codes, GOTOs, and default actions in either closed sparse rows
37
+ or row-displacement arrays;
38
+ - production left sides, right sides, stack lengths, and opaque action slots;
39
+ - static CST kind/field metadata and recovery declaration IDs;
40
+ - Grammar IR, Automaton IR, whole-payload, CST-metadata, and recovery-metadata
41
+ SHA-256 identities; and
42
+ - static byte, occupied-cell, lookup, and recognition-cost descriptions.
43
+
44
+ Signed action codes use `0` for accept, `-1` for error, `1 + state` for shift,
45
+ and `-2 - production` for reduce. Artifact schema version 1 pins those meanings
46
+ to parser-table format version 6. This is a semantic identity, not a claim that
47
+ the JSON object has the Ruby hash/object shape accepted by the v6 runtime.
48
+
49
+ Objects are serialized with recursively sorted string keys. Array order is
50
+ part of the format and is either semantic or validated as canonical. The
51
+ authoritative closed contract is
52
+ [`schema/table-artifact-v1.schema.json`](../schema/table-artifact-v1.schema.json),
53
+ and the Ruby loader additionally checks referential and canonical invariants
54
+ that JSON Schema cannot express.
55
+
56
+ `canonical_payload_bytes` is the exact byte count of compact canonical JSON for
57
+ the payload only. Compact lookup is an O(1) row-displacement probe; plain sparse
58
+ rows use O(log row width) binary search. These complexity strings and the
59
+ occupied-cell counts are static descriptions. `measurement: not-measured`
60
+ means none of them is a
61
+ wall-clock, allocation, RSS, or end-to-end parser benchmark.
62
+
63
+ ## Validation and trust boundary
64
+
65
+ The loader applies a 16 MiB default byte bound, parses JSON, rejects unknown
66
+ fields, checks table encodings and references, recomputes payload/CST/recovery
67
+ digests and static cost fields, then deeply freezes the document. It never
68
+ loads generated classes or evaluates action strings. Semantic action slots are
69
+ production-ID bindings with `verified: false`; action bodies are deliberately
70
+ absent.
71
+
72
+ Compact tables must use the unique minimal row-displacement layout produced by
73
+ the v1 packer; unused padding is not canonical. Accept is valid only on `$eof`,
74
+ `$eof` cannot be shifted, and default actions are limited to errors and
75
+ reductions. The executor also converts any attempt to run beyond the bounded
76
+ input into a rejected result rather than leaking an indexing exception.
77
+
78
+ The grammar and automaton digests are provenance linkage claims copied from
79
+ the builder input. An artifact alone cannot recompute either source object or
80
+ establish its authenticity. A caller that has the source IR must recompute and
81
+ compare those digests, and a publication system must separately authenticate
82
+ the artifact. Digests detect inconsistency; they are not signatures.
83
+
84
+ Ordinary V002 construction hashes the supplied IR, including its diagnostic
85
+ source locations. The explicit verifiable-generation-bundle API instead
86
+ passes a `source-logical-v1` Automaton copy to this unchanged builder. Its
87
+ table identities are therefore path-neutral and exactly match the companion
88
+ report without changing the meaning of standalone `TableArtifact.build`.
89
+
90
+ The validator checks structural and internal table consistency. It does not
91
+ prove that the table implements a claimed grammar, that CST or recovery
92
+ metadata was derived correctly, or that the parser runtime and semantic
93
+ actions are correct. The separately recomputed CST and recovery digests make
94
+ those metadata regions explicit identities for later bundle binding, but do
95
+ not make them independently true.
96
+
97
+ This does not extend the claims of [`ibex verify`](verifier-trust-boundary.md).
98
+ Default and strict verification still rebuild tables from Automaton IR; strict
99
+ still adds collection-completeness checks only. Neither mode consumes this
100
+ sidecar. The separate [scoped generation verification
101
+ report](verification-report.md) binds a verifier outcome to this table's
102
+ identity and lets the generation manifest bind both to a wrapper without
103
+ changing that distinction.
104
+
105
+ ## Compatibility and stop-condition result
106
+
107
+ The prototype is intentionally absent from the default `ibex` require path,
108
+ the generator CLI, `Codegen::Ruby`, generated constants, and `ibex-runtime`.
109
+ The explicitly required bundle API can publish it as a companion artifact.
110
+ Existing generation therefore emits byte-for-byte the same output when the
111
+ feature is not explicitly used, and runtime packaging has no new dependency.
112
+
113
+ The V002 stop conditions do not apply to this scoped adoption:
114
+
115
+ - wrapper/table separation does not change the Stable ABI because the sidecar
116
+ is separate Experimental data;
117
+ - direct bounded recognition supplies execution value beyond duplicating
118
+ Automaton IR; and
119
+ - action bodies remain opaque wrapper authority, so loading or parsing them is
120
+ not required.
121
+
122
+ Promoting the sidecar into default generated output, accepting it in
123
+ `ibex-runtime`, binding action calling conventions, or promising compatibility
124
+ beyond schema version 1 requires a separate compatibility decision and
125
+ migration plan.
@@ -0,0 +1,219 @@
1
+ # Runtime test interactions
2
+
3
+ The suite uses a bounded product for the common generated-parser pipeline and
4
+ focused tests for runtime features that do not fit that product. Matrix size is
5
+ not a proxy for coverage: every runtime-facing feature still needs an explicit
6
+ interaction assessment.
7
+
8
+ <!-- ibex-test-interaction-contract:start -->
9
+ ```yaml
10
+ contract_version: 1
11
+ matrix:
12
+ axes:
13
+ algorithm: [slr, lalr, ielr, lr1]
14
+ table: [plain, compact]
15
+ cst: ["off", "on"]
16
+ locations: ["off", "on"]
17
+ entries: [single, multi, isolated]
18
+ expected_cases: 96
19
+ representative_cases: 12
20
+ roles:
21
+ normal: representative
22
+ scheduled: full
23
+ promotion: full_review
24
+ scheduled_long_gates: ["test:matrix:full", "test:adversarial", "fuzz:long"]
25
+ golden:
26
+ task: "test:zero_cost"
27
+ policy: feature_off_byte_identity
28
+ digest_path: test/golden/digests.yml
29
+ sources:
30
+ - test/fixtures/compat/calculator.y
31
+ - benchmark/grammars/representative.y
32
+ - gallery/json/grammar.y
33
+ update_command: "bundle exec rake golden:update"
34
+ interactions:
35
+ - id: parser_semantics
36
+ axes: [algorithm, table, cst, locations, entries]
37
+ coverage: matrix
38
+ tests: [test/tooling/matrix_runner_test.rb]
39
+ - id: generated_lexer
40
+ axes: [algorithm, table, cst, locations, entries]
41
+ coverage: matrix_and_focused
42
+ tests: [test/codegen/lexer_test.rb, test/frontend/lexer_test.rb]
43
+ - id: semantic_actions
44
+ axes: [table, locations, cst]
45
+ coverage: matrix_and_focused
46
+ tests: [test/runtime/action_contract_test.rb, test/analysis_no_exec_test.rb]
47
+ - id: table_encoding
48
+ axes: [algorithm, table, cst, entries]
49
+ coverage: matrix_and_focused
50
+ tests: [test/tables_test.rb, test/runtime/table_format_test.rb]
51
+ - id: locations
52
+ axes: [locations, table, cst, entries]
53
+ coverage: matrix_and_focused
54
+ tests: [test/location_test.rb, test/runtime/action_contract_test.rb]
55
+ - id: cst
56
+ axes: [cst, algorithm, table, locations, entries]
57
+ coverage: matrix_and_focused
58
+ tests: [test/codegen/cst_runtime_integration_test.rb, test/runtime/cst_serialize_test.rb]
59
+ - id: entry_modes
60
+ axes: [entries, algorithm, table, cst]
61
+ coverage: matrix_and_focused
62
+ tests: [test/codegen/multiple_start_test.rb]
63
+ - id: parser_drivers
64
+ axes: [table, cst, locations]
65
+ coverage: focused_regression
66
+ tests: [test/runtime/parser_test.rb, test/runtime/push_parser_test.rb, test/runtime/parser_table_session_test.rb]
67
+ - id: generated_ast
68
+ axes: [algorithm, table, locations]
69
+ coverage: focused_regression
70
+ tests: [test/codegen/ast_test.rb, test/tooling/matrix_runner_test.rb]
71
+ - id: recovery
72
+ axes: [algorithm, table, cst, entries]
73
+ coverage: focused_regression
74
+ tests: [test/runtime/repair_test.rb, test/runtime/sync_recovery_test.rb]
75
+ - id: resource_limits
76
+ axes: [algorithm, table, cst, entries]
77
+ coverage: matrix_and_focused
78
+ tests: [test/runtime/resource_limits_test.rb]
79
+ - id: observation
80
+ axes: [table, cst, locations]
81
+ coverage: focused_regression
82
+ tests: [test/runtime/observation_test.rb, test/codegen/embedded_tracer_test.rb]
83
+ - id: incremental_cst
84
+ axes: [cst, table, locations]
85
+ coverage: focused_regression
86
+ tests: [test/runtime/cst_incremental_test.rb]
87
+ - id: syntax_session
88
+ axes: [algorithm, table, cst, locations, entries]
89
+ coverage: focused_regression
90
+ tests: [test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
91
+ - id: embedded_runtime
92
+ axes: [table, cst]
93
+ coverage: focused_regression
94
+ tests: [test/packaging/runtime_gem_test.rb, test/codegen/ractor_shareability_test.rb]
95
+ ```
96
+ <!-- ibex-test-interaction-contract:end -->
97
+
98
+ The quality validator compares this record with `test/matrix.yml`, the CI
99
+ workflow, the golden generator, and the files named by every interaction.
100
+
101
+ ## The 96-case product
102
+
103
+ The full matrix is exactly:
104
+
105
+ ```text
106
+ 4 algorithms × 2 table encodings × 2 CST states × 2 location states × 3 entry modes = 96
107
+ ```
108
+
109
+ The algorithm axis is SLR, LALR, IELR, and canonical LR(1). The table axis is
110
+ plain and compact. CST and semantic locations are independently off or on. The
111
+ entry axis is a single start, shared multi-start automaton, or isolated
112
+ multi-start construction.
113
+
114
+ Each case normalizes and round-trips Grammar IR, builds and round-trips
115
+ Automaton IR, proves deterministic generated bytes, executes valid and invalid
116
+ input, compares algorithm/table behavior within the same CST/location/entry
117
+ shape, and enforces a stack resource limit. The runner separately checks
118
+ generated AST node equivalence for every algorithm/table pair.
119
+
120
+ The normal pull-request gate chooses twelve deterministic representatives from
121
+ the ordered product. Twelve is a fast regression net, not a claim of pairwise
122
+ coverage and not a substitute for feature-focused tests. Scheduled CI runs all
123
+ 96 cases plus bounded hostile-input tests and the 100,000-sentence fuzz run.
124
+ Promotion of a Preview or Experimental runtime feature requires a reviewed
125
+ full-96 result on the candidate revision even if scheduled CI was recently
126
+ green.
127
+
128
+ ## Interaction map and covering rules
129
+
130
+ The machine-readable map names the current runtime-facing families and their
131
+ owned tests. `matrix` means the 96-product is the primary coverage.
132
+ `matrix_and_focused` combines product coverage with semantic edge cases.
133
+ `focused_regression` means the feature is not an independent matrix axis and
134
+ the named suite owns its current protocol and edge cases. It does not claim
135
+ that every listed axis pair is already covered; a change to that row must apply
136
+ the covering rules below and record the chosen combinations in its tests.
137
+
138
+ For a new runtime-facing feature:
139
+
140
+ 1. List every existing axis and feature family whose state or observable result
141
+ it can affect.
142
+ 2. Add it as a matrix axis only when all values are cheap, orthogonal, and
143
+ meaningful for the common pipeline. Update the declared product count in
144
+ the same change.
145
+ 3. Otherwise select a deterministic covering set containing every new value,
146
+ every affected existing value, and each pair the implementation says can
147
+ interact. Record the selection rule and own it in a focused test.
148
+ 4. Use feature-specific exhaustive tests for small finite state spaces,
149
+ protocol transitions, version readers, and failure-before-input behavior.
150
+ 5. Add scheduled long fuzz/property cases only when shorter deterministic tests
151
+ cannot provide the same signal. Seeds, counts, and budgets stay fixed.
152
+
153
+ Pairwise coverage is the minimum for declared interactions, not permission to
154
+ ignore a known three-way dependency. A bug involving three or more settings is
155
+ promoted to a permanent focused regression and its interaction is added to the
156
+ map. Unsupported combinations must be rejected explicitly and tested; omitting
157
+ them from a generator is not coverage.
158
+
159
+ Current non-product boundaries include embedded versus installed runtime,
160
+ pull/`yyparse`/push driver transitions, action marker versions, recovery and
161
+ repair state, observation hooks, incremental reuse, serialization, Ractor
162
+ shareability, and packaging. Their focused suites remain mandatory even though
163
+ they do not multiply the 96 common cases.
164
+
165
+ Syntax sessions require CST and a generated lexer, so a disabled-CST value is
166
+ rejected by the service boundary rather than multiplied into the common
167
+ product. `test/runtime/syntax_session_test.rb` exhaustively crosses all four
168
+ algorithms, both table encodings, and shared versus isolated multi-entry
169
+ construction (16 cases). A separate four-case cross covers both table encodings
170
+ with grammar actions that omit or reference semantic locations. Production
171
+ actions, including their semantic-location references, never execute in a
172
+ syntax session, so that location-action choice is independent of algorithm and
173
+ entry construction; CST source locations remain mandatory internally. The
174
+ remaining tests own trust acknowledgement, fresh-result equivalence, fallback
175
+ metrics, cancellation, resource bounds, and malformed edit sequences.
176
+ `test/packaging/runtime_gem_test.rb` owns the embedded-runtime acquisition path.
177
+
178
+ ## Normal, scheduled, and promotion gates
179
+
180
+ Normal pushes and pull requests run the twelve representative cases and the
181
+ feature-off golden check as part of the bounded safety net. The weekly schedule
182
+ runs `test:matrix:full`, long adversarial settings, and `fuzz:long`. Manual
183
+ workflow dispatch does not currently run the full matrix or long fuzz merely by
184
+ being manual; use the explicit local tasks when preparing a promotion.
185
+
186
+ A promotion review records the exact candidate revision and includes:
187
+
188
+ - all 96 matrix cases;
189
+ - every focused suite named by the promoted feature's interaction row;
190
+ - version-reader, invalid-version, and regeneration diagnostics when an ABI is
191
+ involved;
192
+ - the feature-off golden diff; and
193
+ - any scheduled long property or fuzz evidence assigned to the feature.
194
+
195
+ Scheduled success is supporting evidence only. It is not a silent maturity
196
+ promotion and does not replace review of the exact candidate change.
197
+
198
+ ## Golden zero-cost policy
199
+
200
+ `test:zero_cost` regenerates three committed parsers and requires both exact
201
+ bytes and their SHA-256 index to match. When a new feature is disabled, its
202
+ implementation must not change these reviewed generated bytes. This is a
203
+ source-size and emitted-code zero-cost contract for the fixed corpus, not a
204
+ runtime performance or allocation claim.
205
+
206
+ If a cross-cutting correction intentionally changes feature-off output, run
207
+ `bundle exec rake golden:update` and review the generated source diff and
208
+ digest index in the same pull request. Never update the digest merely to make a
209
+ failure green. Add a new representative source when the existing three cannot
210
+ exercise a stable feature-off boundary.
211
+
212
+ ## Merge rule
213
+
214
+ A change under the runtime-facing paths in
215
+ [the ABI policy](runtime-abi-evolution.md#pull-request-abi-assessment) cannot
216
+ merge without a valid structured ABI assessment. The automated check verifies
217
+ the declared versions, matrix arithmetic, scheduled commands, golden inputs,
218
+ and assessment structure. Human review still decides whether the chosen ABI
219
+ boundary and interaction set match the behavior of the change.
@@ -0,0 +1,137 @@
1
+ ---
2
+ title: Verification report
3
+ description: Validate parser-table artifacts and understand the independent verifier trust boundary.
4
+ ---
5
+
6
+ # Scoped generation verification report
7
+
8
+ Ibex can explicitly render a verifiable generation bundle containing a
9
+ data-only parser table, generated Ruby wrapper, scoped verification report,
10
+ and generation manifest. This Experimental API is opt-in and is not part of
11
+ default CLI generation:
12
+
13
+ ```ruby
14
+ require "ibex/verifiable_generation_bundle"
15
+
16
+ bundle = Ibex::VerifiableGenerationBundle.new(
17
+ automaton,
18
+ wrapper_path: "build/parser.rb",
19
+ wrapper_source: generated_ruby,
20
+ table_path: "build/parser.tables.ibex.json",
21
+ report_path: "build/parser.verification.ibex.json",
22
+ manifest_path: "build/parser.ibex.json",
23
+ source_records: generation_inputs,
24
+ manifest_options: { "table" => "compact" },
25
+ strict: true
26
+ )
27
+ bundle.publish
28
+
29
+ Ibex::VerifiableGenerationBundle.validate_file("build/parser.ibex.json")
30
+ ```
31
+
32
+ `render` returns the four in-memory artifacts without publishing them.
33
+ `publish` passes the complete artifact set and exact input records to the
34
+ existing generation transaction. The transaction installs the table and
35
+ report, then the parser wrapper, and installs the manifest last as the
36
+ coherence marker. Rendering or verification failure changes no target.
37
+
38
+ ## Claims and outcomes
39
+
40
+ The version 1 report is a closed JSON document described by
41
+ [`schema/verification-report-v1.schema.json`](../schema/verification-report-v1.schema.json).
42
+ It records:
43
+
44
+ - logical input identities, content digests, and byte sizes;
45
+ - canonical Grammar IR and Automaton IR digests and the construction
46
+ algorithm;
47
+ - the data-only table artifact and payload digests, schema version, and
48
+ representation;
49
+ - the Ibex checker version, default or strict profile, requested checks,
50
+ completed checks, and state/item bounds; and
51
+ - a distinct `pass`, `violations`, or `exhausted` outcome.
52
+
53
+ A completed default report names V1, V3, V4, V6, V7, and V8. A completed
54
+ strict report additionally names V2, V5, and V9. V9 executes only for IELR
55
+ and records a bounded canonical acceptance witness. An exhausted report conservatively
56
+ records no completed checks, includes the requested profile and bounds, and
57
+ is not a pass. The meanings and limitations of V1-V9 remain those in
58
+ [`verifier-trust-boundary.md`](verifier-trust-boundary.md).
59
+
60
+ The report's `evidence_digest` covers the complete report except that digest
61
+ field itself. Input identities use `input/NNNN/BASENAME`, and the table uses
62
+ `table/BASENAME`. These logical identities preserve ordering and artifact
63
+ roles without embedding an absolute checkout path. Both forms are closed:
64
+ `NNNN` is exactly four decimal digits, a report contains at most 10,000 input
65
+ records, and `BASENAME` is one non-empty component other than `.` or `..`
66
+ without slash, backslash, or control characters.
67
+
68
+ The `ir.identity_scope` value `source-logical-v1` defines the versioned digest
69
+ scope. Before building the bundle table, Ibex rebuilds an immutable Automaton
70
+ IR copy in which every source-location `file` is the corresponding indexed
71
+ input identity and every non-null source-provenance `root` is `input`. The
72
+ Grammar IR digest, embedded Automaton IR grammar digest, Automaton IR digest,
73
+ table source identities, report IR claims, and evidence digest all derive
74
+ from that same copy. Every IR source file must map uniquely to
75
+ `source_records`; an absent or ambiguous mapping is rejected.
76
+
77
+ This normalization is limited to the explicit bundle/report boundary. It does
78
+ not mutate the supplied Automaton, change diagnostic locations, or alter the
79
+ ordinary V002 `TableArtifact.build(automaton)` identity. Lower-level callers
80
+ that render a report directly can obtain the required copy with
81
+ `VerificationReport.canonical_automaton`; pairing a raw-path table with the
82
+ canonical report is rejected as an IR/table mismatch.
83
+
84
+ ## Non-cyclic manifest binding
85
+
86
+ The report binds input, Grammar IR, Automaton IR, table artifact, and table
87
+ payload identities. It deliberately contains neither a manifest path nor a
88
+ manifest digest. After the report is rendered, the generation manifest lists
89
+ and hashes the table, wrapper, and report. This gives one direction of
90
+ dependency:
91
+
92
+ ```text
93
+ input + IR -> table -> verification report -> generation manifest
94
+ wrapper ---------------> generation manifest
95
+ ```
96
+
97
+ Adding the manifest digest to the report would require the report hash before
98
+ the manifest can be completed and the manifest hash before the report can be
99
+ completed. Version 1 rejects that circular format.
100
+
101
+ `validate_bundle` checks supplied manifest, report, and table bytes without
102
+ loading generated Ruby. `validate_bundle_file` additionally resolves all
103
+ artifacts from the manifest and verifies every manifest byte count and digest.
104
+ Both reject a missing or duplicate report/table/wrapper entry, stale report or
105
+ table bytes, mismatched logical table identity, input mismatch, IR/table
106
+ identity mismatch, payload mismatch, and an invalid report evidence digest.
107
+ Digests detect inconsistency; they are not signatures or authenticity claims.
108
+
109
+ ## Excluded trust surface
110
+
111
+ This artifact is a **verification report**, not a proof-carrying parser. The
112
+ closed `excluded_trust` field always identifies the following boundaries:
113
+
114
+ - source parsing and normalization into IR;
115
+ - semantic derivation or language equivalence from Automaton IR to the
116
+ persisted table (the sidecar receives closed structural validation and
117
+ digest binding only);
118
+ - generated wrapper structure and behavior;
119
+ - semantic action meaning, safety, termination, and side effects;
120
+ - generated or handwritten lexer actions and tokenization;
121
+ - the Ruby and Ibex runtime implementations;
122
+ - application hooks; and
123
+ - grammar unambiguity.
124
+
125
+ The manifest binds the wrapper's bytes to the same publication, but neither
126
+ the report builder nor validator parses, loads, or executes those bytes.
127
+ Likewise, the table sidecar exposes opaque production action slots with
128
+ `verified: false`. V1-V9 rebuild internal table views but do not consume the
129
+ persisted sidecar or prove the builder's Automaton-IR-to-sidecar projection.
130
+ A `pass` applies only to the named V1-V9 checks over the supplied Automaton IR
131
+ plus the reported structural validation and digest relationships.
132
+
133
+ The checker, IR validator and serializer, reference collection, set analysis,
134
+ table construction/validation, JSON and digest implementations, host runtime,
135
+ and publication filesystem remain trusted as documented. Hostile input also
136
+ requires outer byte, time, memory, and process limits beyond the two reference
137
+ collection bounds.