ibex 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (461) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +100 -272
  3. data/docs/architecture.md +187 -35
  4. data/docs/bison-import.md +103 -0
  5. data/docs/comparison-policy.md +127 -0
  6. data/docs/configuration-model.md +59 -0
  7. data/docs/configuration-report.md +28 -0
  8. data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
  9. data/docs/conflict-explanation-study.md +62 -0
  10. data/docs/construction-profiling.md +157 -0
  11. data/docs/cst-migration.md +3 -4
  12. data/docs/cst.md +24 -0
  13. data/docs/decisions/0000-template.md +24 -0
  14. data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
  15. data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
  16. data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
  17. data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
  18. data/docs/decisions/0005-contained-grammar-composition.md +34 -0
  19. data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
  20. data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
  21. data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
  22. data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
  23. data/docs/decisions/0010-committed-runtime-observation.md +34 -0
  24. data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
  25. data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
  26. data/docs/decisions/0013-transactional-generation-publication.md +37 -0
  27. data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
  28. data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
  29. data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
  30. data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
  31. data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
  32. data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
  33. data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
  34. data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
  35. data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
  36. data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
  37. data/docs/decisions/0024-direct-ielr-construction.md +42 -0
  38. data/docs/decisions/README.md +74 -0
  39. data/docs/declarative-configuration.md +274 -0
  40. data/docs/development.md +196 -2
  41. data/docs/direct-ielr-decision.md +120 -0
  42. data/docs/direct-multi-entry-decision.md +82 -0
  43. data/docs/editor-setup.md +2 -1
  44. data/docs/error-ux-review-rubric-v1.md +117 -0
  45. data/docs/error-ux-reviews/v1/records/README.md +22 -0
  46. data/docs/error-ux-round2-review-status-v1.json +27 -0
  47. data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
  48. data/docs/error-ux-round2-v1.json +1617 -0
  49. data/docs/error-ux-round2.md +111 -0
  50. data/docs/error-ux.md +54 -0
  51. data/docs/getting-started.md +80 -0
  52. data/docs/grammar-reference.md +184 -13
  53. data/docs/ielr-design.md +43 -0
  54. data/docs/ielr.md +92 -0
  55. data/docs/impact.md +101 -0
  56. data/docs/ja/bison-import.md +51 -0
  57. data/docs/ja/stable-api.md +53 -0
  58. data/docs/lexer-construction-profile.md +109 -0
  59. data/docs/lexer-migration.md +5 -0
  60. data/docs/maturity.md +129 -0
  61. data/docs/project-site-strategy.md +54 -0
  62. data/docs/racc-migration-evidence.md +45 -0
  63. data/docs/racc-migration.md +11 -0
  64. data/docs/release-readiness.md +126 -68
  65. data/docs/repair-semantics.md +123 -0
  66. data/docs/runtime-abi-evolution.md +310 -0
  67. data/docs/stability.md +145 -34
  68. data/docs/status.md +43 -0
  69. data/docs/syntax-sessions.md +183 -0
  70. data/docs/table-artifact.md +125 -0
  71. data/docs/test-interactions.md +219 -0
  72. data/docs/verification-report.md +137 -0
  73. data/docs/verifier-trust-boundary.md +199 -0
  74. data/docs/workloads.md +132 -0
  75. data/lib/ibex/analysis/digraph.rb +128 -0
  76. data/lib/ibex/analysis/sets.rb +14 -0
  77. data/lib/ibex/analysis.rb +1 -0
  78. data/lib/ibex/bison_import/importer.rb +516 -0
  79. data/lib/ibex/bison_import/tokenizer.rb +260 -0
  80. data/lib/ibex/bison_import.rb +200 -0
  81. data/lib/ibex/bounded_subprocess.rb +171 -0
  82. data/lib/ibex/cli/ambiguity.rb +14 -5
  83. data/lib/ibex/cli/analysis.rb +134 -0
  84. data/lib/ibex/cli/bison_import.rb +121 -0
  85. data/lib/ibex/cli/config.rb +128 -0
  86. data/lib/ibex/cli/diagnostics.rb +31 -11
  87. data/lib/ibex/cli/documentation.rb +11 -8
  88. data/lib/ibex/cli/equiv.rb +169 -0
  89. data/lib/ibex/cli/error_messages.rb +11 -4
  90. data/lib/ibex/cli/explain.rb +14 -5
  91. data/lib/ibex/cli/fix.rb +203 -0
  92. data/lib/ibex/cli/formatting.rb +11 -10
  93. data/lib/ibex/cli/fuzz.rb +200 -0
  94. data/lib/ibex/cli/fuzz_regressions.rb +145 -0
  95. data/lib/ibex/cli/generation_artifacts.rb +53 -2
  96. data/lib/ibex/cli/generation_error_messages.rb +1 -1
  97. data/lib/ibex/cli/grammar_tests.rb +32 -14
  98. data/lib/ibex/cli/impact.rb +393 -0
  99. data/lib/ibex/cli/ir_tools.rb +5 -61
  100. data/lib/ibex/cli/outputs.rb +60 -33
  101. data/lib/ibex/cli/reduce.rb +202 -0
  102. data/lib/ibex/cli/reduce_reporting.rb +71 -0
  103. data/lib/ibex/cli/samples.rb +49 -23
  104. data/lib/ibex/cli/verify.rb +116 -0
  105. data/lib/ibex/cli/watch.rb +6 -5
  106. data/lib/ibex/cli.rb +362 -56
  107. data/lib/ibex/codegen/action_locations.rb +1 -1
  108. data/lib/ibex/codegen/action_method_source.rb +5 -6
  109. data/lib/ibex/codegen/action_source.rb +3 -2
  110. data/lib/ibex/codegen/ambiguity.rb +5 -4
  111. data/lib/ibex/codegen/explain.rb +123 -55
  112. data/lib/ibex/codegen/generated_action_abi.rb +10 -5
  113. data/lib/ibex/codegen/rbs.rb +15 -7
  114. data/lib/ibex/codegen/report.rb +25 -13
  115. data/lib/ibex/codegen/ruby.rb +28 -64
  116. data/lib/ibex/codegen/ruby_actions.rb +9 -6
  117. data/lib/ibex/codegen/ruby_syntax.rb +15 -9
  118. data/lib/ibex/configuration/analysis_grammar.rb +35 -0
  119. data/lib/ibex/configuration/explanation.rb +421 -0
  120. data/lib/ibex/configuration/inspector.rb +232 -0
  121. data/lib/ibex/configuration.rb +580 -0
  122. data/lib/ibex/coverage/collector.rb +21 -11
  123. data/lib/ibex/coverage/event_stream.rb +13 -7
  124. data/lib/ibex/coverage/report.rb +26 -18
  125. data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
  126. data/lib/ibex/delta_reducer.rb +99 -0
  127. data/lib/ibex/diff.rb +141 -0
  128. data/lib/ibex/equiv/machine.rb +135 -0
  129. data/lib/ibex/equiv.rb +373 -0
  130. data/lib/ibex/fix.rb +557 -0
  131. data/lib/ibex/frontend/ast.rb +22 -2
  132. data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
  133. data/lib/ibex/frontend/diagnostic.rb +1 -1
  134. data/lib/ibex/frontend/formatter.rb +72 -26
  135. data/lib/ibex/frontend/generated_parser.rb +147 -117
  136. data/lib/ibex/frontend/generated_parser_base.rb +7 -3
  137. data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
  138. data/lib/ibex/frontend/generation.rb +1 -1
  139. data/lib/ibex/frontend/parser/declarations.rb +17 -2
  140. data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
  141. data/lib/ibex/frontend/regenerator.rb +1 -0
  142. data/lib/ibex/frontend/resolution.rb +2 -1
  143. data/lib/ibex/frontend/resolver.rb +1 -1
  144. data/lib/ibex/frontend/rule_documentation.rb +2 -1
  145. data/lib/ibex/frontend/source_cursor.rb +2 -2
  146. data/lib/ibex/frontend/source_span.rb +9 -1
  147. data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
  148. data/lib/ibex/frontend/token_adapter.rb +10 -0
  149. data/lib/ibex/frontend.rb +1 -0
  150. data/lib/ibex/fuzz.rb +219 -0
  151. data/lib/ibex/generation_input.rb +1 -1
  152. data/lib/ibex/generation_manifest.rb +36 -25
  153. data/lib/ibex/generation_transaction.rb +1 -1
  154. data/lib/ibex/generation_transaction_recovery.rb +4 -4
  155. data/lib/ibex/generation_transaction_validation.rb +2 -2
  156. data/lib/ibex/grammar_tests.rb +1 -1
  157. data/lib/ibex/impact/action_impact.rb +148 -0
  158. data/lib/ibex/impact/automaton_impact.rb +57 -0
  159. data/lib/ibex/impact/coverage_impact.rb +53 -0
  160. data/lib/ibex/impact/graph.rb +132 -0
  161. data/lib/ibex/impact/propagation.rb +147 -0
  162. data/lib/ibex/impact/report.rb +273 -0
  163. data/lib/ibex/impact/seeds.rb +69 -0
  164. data/lib/ibex/impact/severity.rb +139 -0
  165. data/lib/ibex/impact.rb +16 -0
  166. data/lib/ibex/ir/automaton_ir.rb +61 -19
  167. data/lib/ibex/ir/grammar_ir.rb +76 -45
  168. data/lib/ibex/ir/lexer_ir.rb +5 -5
  169. data/lib/ibex/ir/parser_contract.rb +135 -0
  170. data/lib/ibex/ir/serialize.rb +203 -76
  171. data/lib/ibex/ir/validator/automaton.rb +64 -34
  172. data/lib/ibex/ir/validator/base.rb +51 -33
  173. data/lib/ibex/ir/validator/grammar.rb +113 -83
  174. data/lib/ibex/ir/validator/lexer.rb +2 -2
  175. data/lib/ibex/ir/validator.rb +2 -1
  176. data/lib/ibex/ir.rb +16 -3
  177. data/lib/ibex/lalr/build_metrics.rb +50 -2
  178. data/lib/ibex/lalr/builder.rb +191 -56
  179. data/lib/ibex/lalr/conflict_search.rb +15 -4
  180. data/lib/ibex/lalr/counterexample.rb +19 -14
  181. data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
  182. data/lib/ibex/lalr/goto_follows.rb +229 -0
  183. data/lib/ibex/lalr/ielr/annotator.rb +214 -0
  184. data/lib/ibex/lalr/ielr/bits.rb +28 -0
  185. data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
  186. data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
  187. data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
  188. data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
  189. data/lib/ibex/lalr/ielr/split_state.rb +20 -0
  190. data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
  191. data/lib/ibex/lalr/ielr_partition.rb +22 -8
  192. data/lib/ibex/lalr/inadequacy_report.rb +50 -0
  193. data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
  194. data/lib/ibex/lalr/lr0_collection.rb +121 -0
  195. data/lib/ibex/lalr/unreachable_states.rb +80 -0
  196. data/lib/ibex/lalr.rb +52 -1
  197. data/lib/ibex/location.rb +3 -3
  198. data/lib/ibex/lsp/document_handlers.rb +9 -7
  199. data/lib/ibex/lsp/initialization_handlers.rb +9 -7
  200. data/lib/ibex/lsp/navigation_handlers.rb +25 -11
  201. data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
  202. data/lib/ibex/lsp/position_codec.rb +6 -2
  203. data/lib/ibex/lsp/request_handlers.rb +3 -2
  204. data/lib/ibex/lsp/request_support.rb +14 -7
  205. data/lib/ibex/lsp/server.rb +13 -11
  206. data/lib/ibex/lsp/symbol_index.rb +13 -13
  207. data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
  208. data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
  209. data/lib/ibex/lsp/transport.rb +13 -3
  210. data/lib/ibex/lsp/workspace.rb +5 -4
  211. data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
  212. data/lib/ibex/lsp.rb +1 -0
  213. data/lib/ibex/messages/en.yml +28 -0
  214. data/lib/ibex/messages/ja.yml +28 -0
  215. data/lib/ibex/messages.rb +68 -0
  216. data/lib/ibex/metrics.rb +175 -0
  217. data/lib/ibex/normalize/declarations.rb +2 -0
  218. data/lib/ibex/normalize/diagnostics.rb +63 -38
  219. data/lib/ibex/normalize/expander.rb +5 -4
  220. data/lib/ibex/normalize/expression.rb +8 -6
  221. data/lib/ibex/normalize/grammar_builder.rb +26 -0
  222. data/lib/ibex/normalize/inline_expansion.rb +100 -49
  223. data/lib/ibex/normalize/lexer.rb +3 -3
  224. data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
  225. data/lib/ibex/normalize/parameter_lowering.rb +40 -23
  226. data/lib/ibex/normalize/parameters.rb +31 -4
  227. data/lib/ibex/normalize/parser_configuration.rb +70 -0
  228. data/lib/ibex/normalize.rb +10 -20
  229. data/lib/ibex/racc_migration/report.rb +6 -3
  230. data/lib/ibex/rake_task.rb +1 -1
  231. data/lib/ibex/samples.rb +43 -13
  232. data/lib/ibex/table_artifact/builder.rb +266 -0
  233. data/lib/ibex/table_artifact/cst_projection.rb +70 -0
  234. data/lib/ibex/table_artifact/document.rb +39 -0
  235. data/lib/ibex/table_artifact/executor.rb +187 -0
  236. data/lib/ibex/table_artifact/serializer.rb +57 -0
  237. data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
  238. data/lib/ibex/table_artifact/validator/support.rb +86 -0
  239. data/lib/ibex/table_artifact/validator/tables.rb +238 -0
  240. data/lib/ibex/table_artifact/validator.rb +253 -0
  241. data/lib/ibex/table_artifact.rb +88 -0
  242. data/lib/ibex/table_simulation/result.rb +6 -2
  243. data/lib/ibex/table_simulation/step.rb +3 -1
  244. data/lib/ibex/tables.rb +14 -0
  245. data/lib/ibex/verifiable_generation_bundle.rb +86 -0
  246. data/lib/ibex/verification_report/builder.rb +159 -0
  247. data/lib/ibex/verification_report/canonical_ir.rb +123 -0
  248. data/lib/ibex/verification_report/logical_path.rb +68 -0
  249. data/lib/ibex/verification_report/validator.rb +329 -0
  250. data/lib/ibex/verification_report.rb +69 -0
  251. data/lib/ibex/verify/action_correspondence.rb +138 -0
  252. data/lib/ibex/verify/language_witness.rb +295 -0
  253. data/lib/ibex/verify/reference_collection.rb +190 -0
  254. data/lib/ibex/verify/result.rb +73 -0
  255. data/lib/ibex/verify/verifier.rb +598 -0
  256. data/lib/ibex/verify.rb +15 -0
  257. data/lib/ibex/version.rb +1 -1
  258. data/lib/ibex/watch/runner.rb +10 -1
  259. data/lib/ibex/watch/source_snapshot.rb +13 -8
  260. data/lib/ibex.rb +11 -0
  261. data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
  262. data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
  263. data/schema/bison-import-v1.schema.json +106 -0
  264. data/schema/conflict-explanation-review-v1.schema.json +196 -0
  265. data/schema/conflict-explanation-study-v1.schema.json +528 -0
  266. data/schema/construction-profile-v1.schema.json +779 -0
  267. data/schema/diff-v1.schema.json +41 -0
  268. data/schema/direct-ielr-decision-v1.schema.json +200 -0
  269. data/schema/equiv-v1.schema.json +64 -0
  270. data/schema/error-ux-review-v1.schema.json +609 -0
  271. data/schema/error-ux-round2-review-v1.schema.json +191 -0
  272. data/schema/error-ux-round2-v1.schema.json +519 -0
  273. data/schema/explain-v1.schema.json +63 -1
  274. data/schema/fix-v1.schema.json +64 -0
  275. data/schema/fix-v2.schema.json +66 -0
  276. data/schema/fix-v3.schema.json +101 -0
  277. data/schema/fuzz-regression-v1.schema.json +79 -0
  278. data/schema/fuzz-v1.schema.json +114 -0
  279. data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
  280. data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
  281. data/schema/grammar-ir.schema.json +271 -0
  282. data/schema/ielr-benchmark-v1.schema.json +85 -0
  283. data/schema/impact-v1.schema.json +208 -0
  284. data/schema/lexer-profile-v1.schema.json +491 -0
  285. data/schema/metrics-v1.schema.json +60 -0
  286. data/schema/reduce-v1.schema.json +22 -0
  287. data/schema/reduce-v2.schema.json +63 -0
  288. data/schema/table-artifact-v1.schema.json +377 -0
  289. data/schema/verification-report-v1.schema.json +222 -0
  290. data/schema/verify-v1.schema.json +42 -0
  291. data/sig/ibex/analysis/digraph.rbs +19 -0
  292. data/sig/ibex/analysis/sets.rbs +10 -0
  293. data/sig/ibex/bison_import/importer.rbs +140 -0
  294. data/sig/ibex/bison_import/tokenizer.rbs +91 -0
  295. data/sig/ibex/bison_import.rbs +95 -0
  296. data/sig/ibex/bounded_subprocess.rbs +76 -0
  297. data/sig/ibex/cli/ambiguity.rbs +6 -0
  298. data/sig/ibex/cli/analysis.rbs +43 -0
  299. data/sig/ibex/cli/bison_import.rbs +32 -0
  300. data/sig/ibex/cli/config.rbs +35 -0
  301. data/sig/ibex/cli/diagnostics.rbs +16 -3
  302. data/sig/ibex/cli/documentation.rbs +5 -1
  303. data/sig/ibex/cli/equiv.rbs +49 -0
  304. data/sig/ibex/cli/error_messages.rbs +6 -0
  305. data/sig/ibex/cli/explain.rbs +6 -0
  306. data/sig/ibex/cli/fix.rbs +53 -0
  307. data/sig/ibex/cli/formatting.rbs +5 -1
  308. data/sig/ibex/cli/fuzz.rbs +47 -0
  309. data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
  310. data/sig/ibex/cli/generation_artifacts.rbs +21 -2
  311. data/sig/ibex/cli/generation_error_messages.rbs +2 -2
  312. data/sig/ibex/cli/grammar_tests.rbs +15 -3
  313. data/sig/ibex/cli/impact.rbs +95 -0
  314. data/sig/ibex/cli/ir_tools.rbs +7 -19
  315. data/sig/ibex/cli/outputs.rbs +9 -3
  316. data/sig/ibex/cli/reduce.rbs +60 -0
  317. data/sig/ibex/cli/reduce_reporting.rbs +27 -0
  318. data/sig/ibex/cli/samples.rbs +8 -0
  319. data/sig/ibex/cli/verify.rbs +28 -0
  320. data/sig/ibex/cli/watch.rbs +2 -2
  321. data/sig/ibex/cli.rbs +79 -5
  322. data/sig/ibex/codegen/action_method_source.rbs +4 -4
  323. data/sig/ibex/codegen/explain.rbs +45 -25
  324. data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
  325. data/sig/ibex/codegen/rbs.rbs +12 -10
  326. data/sig/ibex/codegen/report.rbs +12 -12
  327. data/sig/ibex/codegen/ruby.rbs +18 -15
  328. data/sig/ibex/codegen/ruby_actions.rbs +6 -6
  329. data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
  330. data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
  331. data/sig/ibex/configuration/explanation.rbs +195 -0
  332. data/sig/ibex/configuration/inspector.rbs +53 -0
  333. data/sig/ibex/configuration.rbs +242 -0
  334. data/sig/ibex/coverage/collector.rbs +16 -14
  335. data/sig/ibex/coverage/event_stream.rbs +12 -10
  336. data/sig/ibex/coverage/report.rbs +20 -18
  337. data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
  338. data/sig/ibex/delta_reducer.rbs +38 -0
  339. data/sig/ibex/diff.rbs +47 -0
  340. data/sig/ibex/equiv/machine.rbs +54 -0
  341. data/sig/ibex/equiv.rbs +129 -0
  342. data/sig/ibex/fix.rbs +138 -0
  343. data/sig/ibex/frontend/ast.rbs +33 -7
  344. data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
  345. data/sig/ibex/frontend/diagnostic.rbs +2 -2
  346. data/sig/ibex/frontend/formatter.rbs +38 -30
  347. data/sig/ibex/frontend/generated_parser.rbs +87 -77
  348. data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
  349. data/sig/ibex/frontend/parser/declarations.rbs +3 -0
  350. data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
  351. data/sig/ibex/frontend/resolution.rbs +3 -2
  352. data/sig/ibex/frontend/source_cursor.rbs +2 -2
  353. data/sig/ibex/frontend/source_span.rbs +5 -2
  354. data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
  355. data/sig/ibex/frontend/token_adapter.rbs +6 -0
  356. data/sig/ibex/fuzz.rbs +71 -0
  357. data/sig/ibex/generation_input.rbs +2 -2
  358. data/sig/ibex/generation_manifest.rbs +35 -32
  359. data/sig/ibex/generation_transaction_recovery.rbs +2 -2
  360. data/sig/ibex/generation_transaction_validation.rbs +2 -2
  361. data/sig/ibex/grammar_tests.rbs +2 -2
  362. data/sig/ibex/impact/action_impact.rbs +58 -0
  363. data/sig/ibex/impact/automaton_impact.rbs +37 -0
  364. data/sig/ibex/impact/coverage_impact.rbs +28 -0
  365. data/sig/ibex/impact/graph.rbs +67 -0
  366. data/sig/ibex/impact/propagation.rbs +56 -0
  367. data/sig/ibex/impact/report.rbs +107 -0
  368. data/sig/ibex/impact/seeds.rbs +36 -0
  369. data/sig/ibex/impact/severity.rbs +55 -0
  370. data/sig/ibex/impact.rbs +6 -0
  371. data/sig/ibex/ir/automaton_ir.rbs +30 -11
  372. data/sig/ibex/ir/grammar_ir.rbs +41 -27
  373. data/sig/ibex/ir/lexer_ir.rbs +8 -8
  374. data/sig/ibex/ir/parser_contract.rbs +69 -0
  375. data/sig/ibex/ir/serialize.rbs +63 -28
  376. data/sig/ibex/ir/validator/automaton.rbs +47 -44
  377. data/sig/ibex/ir/validator/base.rbs +35 -31
  378. data/sig/ibex/ir/validator/grammar.rbs +78 -67
  379. data/sig/ibex/ir/validator/lexer.rbs +4 -4
  380. data/sig/ibex/ir.rbs +9 -3
  381. data/sig/ibex/lalr/build_metrics.rbs +42 -2
  382. data/sig/ibex/lalr/builder.rbs +75 -24
  383. data/sig/ibex/lalr/conflict_search.rbs +5 -2
  384. data/sig/ibex/lalr/counterexample.rbs +17 -14
  385. data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
  386. data/sig/ibex/lalr/goto_follows.rbs +89 -0
  387. data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
  388. data/sig/ibex/lalr/ielr/bits.rbs +14 -0
  389. data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
  390. data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
  391. data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
  392. data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
  393. data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
  394. data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
  395. data/sig/ibex/lalr/ielr_partition.rbs +17 -5
  396. data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
  397. data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
  398. data/sig/ibex/lalr/lr0_collection.rbs +50 -0
  399. data/sig/ibex/lalr/unreachable_states.rbs +25 -0
  400. data/sig/ibex/lalr.rbs +11 -1
  401. data/sig/ibex/location.rbs +6 -6
  402. data/sig/ibex/lsp/document_handlers.rbs +8 -8
  403. data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
  404. data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
  405. data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
  406. data/sig/ibex/lsp/position_codec.rbs +8 -4
  407. data/sig/ibex/lsp/request_handlers.rbs +4 -4
  408. data/sig/ibex/lsp/request_support.rbs +8 -8
  409. data/sig/ibex/lsp/server.rbs +16 -16
  410. data/sig/ibex/lsp/symbol_index.rbs +20 -21
  411. data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
  412. data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
  413. data/sig/ibex/lsp/transport.rbs +18 -6
  414. data/sig/ibex/lsp/workspace.rbs +3 -2
  415. data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
  416. data/sig/ibex/messages.rbs +26 -0
  417. data/sig/ibex/metrics.rbs +50 -0
  418. data/sig/ibex/normalize/diagnostics.rbs +4 -4
  419. data/sig/ibex/normalize/expander.rbs +2 -2
  420. data/sig/ibex/normalize/expression.rbs +10 -8
  421. data/sig/ibex/normalize/grammar_builder.rbs +11 -0
  422. data/sig/ibex/normalize/inline_expansion.rbs +59 -50
  423. data/sig/ibex/normalize/lexer.rbs +4 -4
  424. data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
  425. data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
  426. data/sig/ibex/normalize/parameters.rbs +13 -8
  427. data/sig/ibex/normalize/parser_configuration.rbs +23 -0
  428. data/sig/ibex/normalize.rbs +11 -5
  429. data/sig/ibex/racc_migration/report.rbs +8 -6
  430. data/sig/ibex/rake_task.rbs +1 -1
  431. data/sig/ibex/samples.rbs +13 -4
  432. data/sig/ibex/table_artifact/builder.rbs +93 -0
  433. data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
  434. data/sig/ibex/table_artifact/document.rbs +27 -0
  435. data/sig/ibex/table_artifact/executor.rbs +66 -0
  436. data/sig/ibex/table_artifact/serializer.rbs +25 -0
  437. data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
  438. data/sig/ibex/table_artifact/validator/support.rbs +46 -0
  439. data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
  440. data/sig/ibex/table_artifact/validator.rbs +66 -0
  441. data/sig/ibex/table_artifact.rbs +32 -0
  442. data/sig/ibex/table_simulation/result.rbs +8 -4
  443. data/sig/ibex/table_simulation/step.rbs +4 -2
  444. data/sig/ibex/tables.rbs +10 -9
  445. data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
  446. data/sig/ibex/verification_report/builder.rbs +53 -0
  447. data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
  448. data/sig/ibex/verification_report/logical_path.rbs +34 -0
  449. data/sig/ibex/verification_report/validator.rbs +114 -0
  450. data/sig/ibex/verification_report.rbs +44 -0
  451. data/sig/ibex/verify/action_correspondence.rbs +50 -0
  452. data/sig/ibex/verify/language_witness.rbs +112 -0
  453. data/sig/ibex/verify/reference_collection.rbs +63 -0
  454. data/sig/ibex/verify/result.rbs +52 -0
  455. data/sig/ibex/verify/verifier.rbs +170 -0
  456. data/sig/ibex/verify.rbs +6 -0
  457. data/sig/ibex/watch/runner.rbs +12 -2
  458. data/sig/ibex/watch/source_snapshot.rbs +13 -9
  459. metadata +254 -12
  460. data/lib/ibex/ir/migration.rb +0 -120
  461. data/sig/ibex/ir/migration.rbs +0 -34
@@ -0,0 +1,310 @@
1
+ # Runtime ABI evolution
2
+
3
+ This policy separates durable serialized input, generated parser tables, and
4
+ the Ruby runtime API. The current format is the only supported Grammar and
5
+ Automaton IR boundary; the numeric field is a wire discriminator, not a promise
6
+ to read old files.
7
+
8
+ <!-- ibex-runtime-abi-contract:start -->
9
+ ```yaml
10
+ contract_version: 1
11
+ ir:
12
+ grammar:
13
+ current_writer: 1
14
+ readable: [1]
15
+ migrations: []
16
+ preserve_loaded_version: false
17
+ automaton:
18
+ current_writer: 1
19
+ readable: [1]
20
+ migrations: []
21
+ preserve_loaded_version: false
22
+ lexer:
23
+ current_writer: 1
24
+ readable: [1]
25
+ migrations: []
26
+ standalone: true
27
+ embedded_in_grammar: true
28
+ parser_tables:
29
+ current_writer: 6
30
+ readable: [6]
31
+ cst_readable: [6]
32
+ fail_before_input: true
33
+ versions:
34
+ generator: "0.4.0"
35
+ runtime: "0.4.0"
36
+ runtime_dependency: "~> 0.4.0"
37
+ runtime_paths:
38
+ - .github/pull_request_template.md
39
+ - .github/workflows/main.yml
40
+ - Rakefile
41
+ - docs/runtime-abi-evolution.md
42
+ - docs/test-interactions.md
43
+ - tool/quality/runtime_abi.rb
44
+ - tool/quality/runtime_abi/**/*
45
+ - test/quality/runtime_abi*_test.rb
46
+ - test/support/runtime_abi_test_project.rb
47
+ - test/fixtures/runtime_abi/**/*
48
+ - ibex.gemspec
49
+ - ibex-runtime.gemspec
50
+ - lib/ibex/version.rb
51
+ - lib/ibex/runtime.rb
52
+ - lib/ibex/runtime/**/*
53
+ - lib/ibex/runtime/version.rb
54
+ - lib/ibex/tables.rb
55
+ - lib/ibex/tables/**/*
56
+ - sig/ibex/tables.rbs
57
+ - sig/ibex/runtime.rbs
58
+ - sig/ibex/runtime/**/*
59
+ - sig/ibex/tables/**/*
60
+ - lib/ibex/codegen.rb
61
+ - lib/ibex/codegen/**/*
62
+ - sig/ibex/codegen.rbs
63
+ - sig/ibex/codegen/**/*
64
+ - lib/ibex/frontend/generated_parser.rb
65
+ - lib/ibex/ir.rb
66
+ - lib/ibex/ir/**/*
67
+ - sig/ibex/ir.rbs
68
+ - sig/ibex/ir/**/*
69
+ - schema/grammar-ir.schema.json
70
+ - schema/grammar-ir-foundation.schema.json
71
+ - schema/grammar-ir-extensions.schema.json
72
+ - schema/automaton-ir.schema.json
73
+ - schema/automaton-ir-definitions.schema.json
74
+ - schema/lexer-ir-v*.schema.json
75
+ - schema/cst-v*.json
76
+ - test/matrix.yml
77
+ - test/support/matrix_contract.rb
78
+ - test/support/matrix_runner.rb
79
+ - test/tooling/matrix_runner_test.rb
80
+ - tool/quality/golden.rb
81
+ - test/golden/**/*
82
+ assessment:
83
+ states: [compatible, breaking, not_applicable]
84
+ surfaces: [parser_table, grammar_ir, automaton_ir, lexer_ir, runtime_api, embedded_runtime, generation_metadata, cst, test_matrix, policy, none]
85
+ abi_choices: [current_contract, new_table_format, new_ir_version, new_runtime_major, sidecar, none]
86
+ regeneration: [required, not_required, not_applicable]
87
+ required_fields: [state, surfaces, abi_choice, regeneration, rationale, affected_interactions, evidence, tests, verification]
88
+ ```
89
+ <!-- ibex-runtime-abi-contract:end -->
90
+
91
+ The fenced record above is validated against the constants, schemas, gemspecs,
92
+ test declaration, and pull-request policy in this repository. Edit it only as
93
+ part of the implementation change that moves the corresponding boundary.
94
+
95
+ ## IR read, write, migration, and freeze policy
96
+
97
+ | Contract | Current writer | Current reader | Migration | Freeze |
98
+ | --- | ---: | --- | --- | --- |
99
+ | Grammar IR | current format | current format only | none | the current schema is closed; older documents are rejected |
100
+ | Automaton IR | current format | current format only | none | the current schema is closed; older documents are rejected |
101
+ | Lexer IR | schema 1 | schema 1 | none | the closed schema changes only through a new schema |
102
+
103
+ The generator and runtime intentionally share one Grammar and Automaton IR
104
+ format. Older documents are rejected before construction; there is no in-process
105
+ IR migration command or compatibility reader. The current Grammar IR carries a
106
+ required root `parser_contract` whose individual fields
107
+ are either explicit with source locations or explicitly unspecified. Automaton
108
+ IR embeds the current Grammar IR, includes that contract in `grammar_digest`,
109
+ and records `entry_construction` as `shared` or `isolated`. Unknown versions and
110
+ construction values are rejected.
111
+
112
+ An extended root `parser ... end` declaration now writes explicit
113
+ `parser.algorithm` and `parser.entries` members into the current contract. The
114
+ declaration is generator input only: it selects existing construction paths and
115
+ does not add a runtime lookup, generated parser method, action ABI field, or
116
+ parser-table datum. Parser-table format v6 therefore remains the sole writer,
117
+ and declaration-free grammar, generated Ruby, and table golden bytes remain
118
+ unchanged.
119
+
120
+ Resuming from the current Grammar IR resolves explicit contract values through the same
121
+ typed configuration algebra as CLI values; a matching CLI value is accepted and
122
+ a conflicting canonical-generation value fails at the recorded declaration
123
+ location. Explicit analysis or grammar-test algorithm selection is reported as
124
+ noncanonical and never reclassified as canonical generation. Resuming from a constructed Automaton IR rejects
125
+ construction flags because they cannot rebuild the embedded tables. Its manifest
126
+ keeps the grammar contract, embedded construction facts, and effective codegen
127
+ configuration separate. These schema additions do not change parser-table
128
+ format v6 or the runtime action ABI.
129
+
130
+ Lexer IR schema 1 is independently versioned. It can be validated and serialized as
131
+ a standalone `ibex_ir: lexer` document, and Grammar IR can carry the same
132
+ versioned document in its optional `lexer` field. There is no Lexer IR
133
+ migration today. Changing lexer rule meaning or its required shape therefore
134
+ requires Lexer IR v2; changing only parser construction does not.
135
+
136
+ The JSON schemas under `schema/` are the published closed shapes. An optional
137
+ field may be added to a future version, but an existing closed schema is not
138
+ made open to avoid assigning new meaning to an old version.
139
+
140
+ ## Parser-table formats
141
+
142
+ The generator emits format v6, and the runtime recognizes only that current
143
+ format. Older generated tables fail before input so the runtime does not carry
144
+ historical action-ABI branches indefinitely:
145
+
146
+ | Format | Non-CST runtime contract | CST contract | Current writer |
147
+ | ---: | --- | --- | --- |
148
+ | v6 | retains the preceding action contracts | structured CST metadata is accepted | yes |
149
+
150
+ Plain and compact tables are encodings of the same lookup contract, not
151
+ separate ABI versions. A change to compression is format-preserving only when
152
+ every known and unknown token retains the same shift, reduce, accept, or error
153
+ result. Required table shape, action calling convention, marker meaning, or
154
+ runtime lookup meaning requires a new parser-table format.
155
+
156
+ Every pull, `yyparse`, push, finish, and syntax entry starts a parser session by
157
+ validating its table object. A missing or unsupported `format_version`, an
158
+ invalid generated action marker combination, or a legacy/boolean CST shape is
159
+ rejected before the runtime asks a pull lexer for a token or consumes a
160
+ caller-supplied push token. Current generated action marker combinations are
161
+ checked at that boundary. The error directs the application to regenerate.
162
+ Validation cannot undo side effects the caller performed while constructing an
163
+ argument before calling the runtime.
164
+
165
+ Regeneration is required when:
166
+
167
+ - a parser has no table-format version or uses a version the installed runtime
168
+ does not list;
169
+ - any CST-aware parser predates structured format-v6 CST metadata;
170
+ - generated action markers do not satisfy their current calling contract;
171
+ - an embedded parser must receive runtime fixes or a newer runtime ABI; or
172
+ - a release note for a new format explicitly retires an older reader.
173
+
174
+ Non-CST v1-v5 tables are outside the pre-v1 runtime compatibility obligation
175
+ and require regeneration. Applications that need to keep an old generated
176
+ artifact can pin the matching pre-v1 runtime package instead of making the
177
+ current reader retain every historical table branch.
178
+
179
+ ## Generator and runtime versions
180
+
181
+ | Generated artifact | Runtime used | Status |
182
+ | --- | --- | --- |
183
+ | Ibex 0.4.0 non-embedded output (table v6) | `ibex-runtime` 0.4.0 | supported and covered by packaging/runtime tests |
184
+ | Ibex 0.4.0 non-embedded output | a future version admitted by `~> 0.4.0` | dependency resolution permits it; compatibility is an obligation of that future release, not current execution evidence |
185
+ | Ibex 0.4.0 embedded output | runtime sources copied by Ibex 0.4.0 | self-contained and covered by packaging/runtime tests |
186
+ | Ibex 0.3.0 non-embedded output (table v6) | `ibex-runtime` 0.3.0 | supported and covered by packaging/runtime tests |
187
+ | Ibex 0.3.0 non-embedded output | a future version admitted by `~> 0.3.0` | dependency resolution permits it; compatibility is an obligation of that future release, not current execution evidence |
188
+ | Ibex 0.3.0 embedded output | runtime sources copied by Ibex 0.3.0 | self-contained and covered by packaging/runtime tests |
189
+ | Ibex 0.2.0 non-embedded output (table v6) | `ibex-runtime` 0.2.0 | supported and covered by packaging/runtime tests |
190
+ | Ibex 0.2.0 non-embedded output | a future version admitted by `~> 0.2.0` | dependency resolution permits it; compatibility is an obligation of that future release, not current execution evidence |
191
+ | Ibex 0.2.0 embedded output | runtime sources copied by Ibex 0.2.0 | self-contained and covered by packaging/runtime tests |
192
+ | old non-CST or CST table v1-v5 | `ibex-runtime` 0.4.0 | rejected before input; regenerate or pin the matching pre-v1 runtime |
193
+ | table v7 or later | `ibex-runtime` 0.4.0 | rejected before input; upgrade the runtime or regenerate to a supported format |
194
+
195
+ `ibex` 0.4.0 declares `ibex-runtime ~> 0.4.0`; in RubyGems terms that admits
196
+ runtime releases from 0.4.0 up to, but not including, 0.5.0. The previous
197
+ 0.3.0 and 0.2.0 lines remain historical compatibility boundaries. The table format is
198
+ still the executable compatibility check. A package requirement alone does not
199
+ prove that an unpublished runtime can execute a generated parser.
200
+
201
+ Normal generated output requires `ibex/runtime`. Embedded output concatenates
202
+ the runtime files named by `Runtime::EmbeddedSource` in dependency order. Thus
203
+ both modes use the same repository sources at generation time. Embedded output
204
+ does not consult an installed runtime and does not acquire later security,
205
+ correctness, or performance fixes: the application that chooses embedding owns
206
+ regeneration and redeployment. Loading embedded and installed copies into one
207
+ process is not a supported upgrade mechanism.
208
+
209
+ ### Syntax-session ABI classification
210
+
211
+ Adding `SyntaxSession` and its syntax-only repair proposal keeps parser-table
212
+ format v6 unchanged and adds methods and immutable result types to the current Ruby runtime API. Existing
213
+ non-embedded generated source remains byte-identical and acquires the API from
214
+ a compatible runtime-package upgrade. Embedded generated source copies runtime
215
+ implementation bytes, so its bytes change and regeneration plus redeployment
216
+ is required to acquire the API or later session correctness fixes. Existing
217
+ embedded parsers that are not regenerated retain their previous parsing
218
+ behavior but do not gain the new façade.
219
+
220
+ The repository contract can represent that combined assessment without
221
+ claiming a table-format bump. The conservative `regeneration: required` value
222
+ applies to the affected embedded artifacts; the rationale records why the
223
+ non-embedded path only needs a runtime upgrade:
224
+
225
+ ```yaml
226
+ state: compatible
227
+ surfaces: [runtime_api, embedded_runtime, cst]
228
+ abi_choice: current_contract
229
+ regeneration: required
230
+ rationale: Table v6 is unchanged and the runtime API is additive; embedded output bytes change and must be regenerated to acquire SyntaxSession.
231
+ affected_interactions: [incremental_cst, syntax_session, embedded_runtime]
232
+ evidence: [lib/ibex/runtime/cst/incremental/relexer.rb, lib/ibex/runtime/cst/incremental/session.rb, lib/ibex/runtime/syntax_session.rb, lib/ibex/runtime/parser.rb, lib/ibex/runtime/embedded_source.rb, test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
233
+ tests: [test/runtime/cst_incremental_test.rb, test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
234
+ verification: [bundle exec rake quality:runtime_abi, bundle exec ruby -Itest test/runtime/cst_incremental_test.rb, bundle exec ruby -Itest test/runtime/syntax_session_test.rb, bundle exec ruby -Itest test/packaging/runtime_gem_test.rb]
235
+ ```
236
+
237
+ ## Sidecar, IR version, or table-format version
238
+
239
+ Choose the boundary by its consumer:
240
+
241
+ - Use a sidecar for optional generation, review, or editor metadata that the
242
+ parser does not need to accept input and whose absence preserves behavior.
243
+ - Use a new Grammar, Automaton, or Lexer IR version when persisted generator
244
+ input changes shape or meaning, even if generated runtime tables do not.
245
+ - Use a new parser-table format when runtime execution needs a required field,
246
+ new action convention, or changed table meaning before or during parsing.
247
+ - Use both IR and table versions when the persisted construction contract and
248
+ the runtime execution contract both change.
249
+
250
+ An optional table field is not automatically safe: if its absence selects new
251
+ behavior or changes execution, it belongs behind a new format. Conversely,
252
+ large diagnostics that never affect execution should not inflate every parser
253
+ table merely to avoid a sidecar.
254
+
255
+ ## Pull-request ABI assessment
256
+
257
+ The pull-request template contains a delimited YAML assessment. CI parses only
258
+ that block and requires the exact declared fields when a changed path matches
259
+ `runtime_paths` above. Free-form prose outside the block is not evidence.
260
+
261
+ `compatible` means the change keeps a current contract or adds a sidecar.
262
+ `breaking` selects a new table format, IR version, or runtime major version. A
263
+ new table format requires regeneration. An IR-only or Ruby-runtime-API break
264
+ must still make an explicit `required`/`not_required` regeneration decision;
265
+ application migration can be required even when parser regeneration is not.
266
+ `not_applicable` is accepted only when no declared runtime-facing path changed.
267
+ Every changed runtime-facing path must appear in `evidence`; additional evidence
268
+ must be a changed path or an existing regression test. `affected_interactions`
269
+ uses ids from the test-interaction contract, and every listed interaction owns
270
+ at least one path in `tests`. `verification` accepts only reviewed repository
271
+ commands and must run the ABI gate plus the owned tests (or the full suite).
272
+ The validator checks these relationships, while reviewers remain responsible
273
+ for judging the rationale and whether all affected surfaces were identified.
274
+ The deterministic rationale screen checks only structure: the value must be a
275
+ non-empty string containing a Unicode letter, and it must not equal the exact
276
+ repository-owned template sentinel after whitespace normalization. It does not
277
+ classify general placeholders, identifiers, repetition, entropy, or semantic
278
+ substance. Japanese, Arabic, and other writing systems are accepted. Reviewers
279
+ must decide whether the rationale actually explains compatibility and whether
280
+ TODO-like text is acceptable in context.
281
+
282
+ The structured choice table is closed:
283
+
284
+ | `abi_choice` | Required state | Required surface | Regeneration |
285
+ | --- | --- | --- | --- |
286
+ | `current_contract` | `compatible` | one or more concrete surfaces | `required` or `not_required` must be decided |
287
+ | `sidecar` | `compatible` | exactly `generation_metadata` | `not_required` |
288
+ | `new_table_format` | `breaking` | includes `parser_table` | `required` |
289
+ | `new_ir_version` | `breaking` | includes `grammar_ir`, `automaton_ir`, or `lexer_ir` | `required` or `not_required` must be decided |
290
+ | `new_runtime_major` | `breaking` | includes `runtime_api` or `embedded_runtime` | `required` or `not_required` must be decided |
291
+ | `none` | only a non-runtime change | `none` | `not_applicable` |
292
+
293
+ Runtime-facing changes cannot use `none` or `not_applicable`. A rationale must
294
+ replace the exact template sentinel and contain a Unicode letter; required human
295
+ review decides whether it is substantive. Verification commands are parsed as
296
+ arguments, not executed by the validator; shell composition, arbitrary
297
+ commands, unowned test files, and evidence-only README links are rejected.
298
+
299
+ Run the contract-only `bundle exec rake quality:runtime_abi` locally.
300
+ Pull-request CI uses a minimal dedicated job to invoke the separate
301
+ `quality:runtime_abi_pr` entry with its event path and name explicitly. That PR
302
+ gate derives the changed path list from
303
+ the event's exact base and head SHAs without GitHub API writes or elevated
304
+ permissions. It parses `runtime_paths` from the exact
305
+ trusted base revision and unions those paths with the head policy, so a pull
306
+ request cannot exempt its own changes merely by shrinking the head contract.
307
+ Only the first explicit addition of this contract can use the fixed bootstrap
308
+ path set. Repository rules must still require this check and review changes to
309
+ the workflow and validator: code running solely from an untrusted head cannot
310
+ defend against deletion or malicious replacement of the gate that launches it.
data/docs/stability.md CHANGED
@@ -1,3 +1,8 @@
1
+ ---
2
+ title: Stability policy
3
+ description: Compatibility, promotion, deprecation, and feature-development rules for Ibex.
4
+ ---
5
+
1
6
  # Stability, compatibility, and deprecation
2
7
 
3
8
  Ibex separates support level from activation. Opt-in controls do not determine
@@ -8,7 +13,7 @@ maturity: a documented opt-in API can be Stable, Preview, or Experimental.
8
13
  | Level | Activation | Guarantee |
9
14
  |---|---|---|
10
15
  | Stable | Default compatible mode or a documented stable API | Semantic versioning; compatible-mode behavior remains unchanged |
11
- | Preview | Explicit pragma, mode, command, or algorithm | Breaking changes require notice one minor release in advance |
16
+ | Preview | Feature-specific; compatible/default or explicit as recorded in the maturity audit | Before v1.0: changeable with a migration note; after v1.0: one-minor notice |
12
17
  | Experimental | Explicit policy/object or research entry point | May change without notice; budgets and failure modes are part of the experiment |
13
18
 
14
19
  After v1.0, promotion requires representative use in a shadow or gallery
@@ -20,6 +25,61 @@ support where applicable, and complete public documentation replace the
20
25
  impossible two-prior-release requirement. Only Stable features may be adopted
21
26
  by the production self-hosted grammar.
22
27
 
28
+ Before v1.0, Preview is an evolving product boundary rather than a general
29
+ backward-compatibility promise. A pre-v1 Preview change is allowed when the
30
+ same change includes a release-note or migration note, updated black-box
31
+ regressions, and a refreshed maturity record. Consumers of a Preview feature
32
+ must therefore pin a pre-release or be prepared to migrate. After v1.0,
33
+ Preview receives the one-minor notice described in the deprecation policy.
34
+
35
+ Middle actions are accepted in compatible/default grammar productions and
36
+ therefore remain under the Stable compatibility lock; their Preview row is
37
+ retained only for traceability while its redundant classification is
38
+ redesigned. Most other Preview surfaces require an extended declaration,
39
+ command, or option. The [maturity audit](maturity.md) records activation and
40
+ Stable overlap separately.
41
+
42
+ ## Execution trust is independent of maturity
43
+
44
+ Stable, Preview, and Experimental describe compatibility and promotion, not
45
+ sandbox strength. Static grammar and IR tools do not execute parser actions,
46
+ generated lexer actions, or `header` / `inner` / `footer` sections. Generated
47
+ lexer semantic parses execute parser and lexer actions. Handwritten pull and
48
+ caller-fed semantic parses execute parser actions but do not invoke the
49
+ generated lexer. Generated syntax-only parses suppress parser production
50
+ actions but still execute generated lexer actions. All generated runtime paths
51
+ may load user sections and are trusted application code, not sandboxes.
52
+
53
+ A future nonexecuting syntax profile is a separate product contract. It must
54
+ require a declarative built-in-only lexer and reject all user-code sections;
55
+ neither the Stable batch CST API nor the Experimental incremental API currently
56
+ makes that guarantee. See the [execution trust matrix](../README.md#execution-trust-matrix).
57
+
58
+ ## Feature development budget
59
+
60
+ Feature development is not frozen. A `HOLD` release decision blocks publication
61
+ of that release candidate, not investigation or feature development. The
62
+ following limits keep new work reviewable:
63
+
64
+ - at most three active new Preview development tracks;
65
+ - at most one of those tracks may introduce grammar syntax; and
66
+ - at most five Experimental product features.
67
+
68
+ A development track starts when its first user-visible change is merged and
69
+ ends when the feature becomes Stable or Experimental, or is removed. An
70
+ investigation, an ADR, and an unmerged spike do not consume the budget. An
71
+ existing Preview feature consumes a track only while its specification is being
72
+ changed; promotion work that preserves its specification does not.
73
+
74
+ There is currently one active Preview development track: root declarative
75
+ parser construction. It consumes the one grammar-syntax slot while extending
76
+ the existing Preview IELR and multiple-entry surfaces; it is not a Stable
77
+ promotion. A pull request that starts or ends a track must update this
78
+ statement and the inventory below.
79
+ These limits do not relax the core IR contracts described under
80
+ [Core IR freeze](#core-ir-freeze), but they do permit retiring an unpromoted
81
+ Preview implementation when its maintenance cost exceeds its evidence.
82
+
23
83
  ## v1 inventory
24
84
 
25
85
  Stable:
@@ -28,26 +88,46 @@ Stable:
28
88
  - default direct LALR construction, parser tables, recovery callbacks,
29
89
  observation events, resource limits, migration checks, and bounded
30
90
  counterexample/ambiguity analysis;
31
- - versioned core Grammar IR, Automaton IR, Lexer IR, table formats, report
91
+ - current core Grammar IR and Automaton IR, independently versioned Lexer IR, table formats, report
32
92
  schemas, and their validators;
33
93
  - format-v6 Red/Green batch CST parsing, typed syntax views, persistent editing
34
94
  and diffing, and the closed `ibex_cst` schema v1 serialization contract.
35
95
 
36
- Preview:
37
-
38
- - `pragma extended` / `--mode=extended`, including EBNF groups, parameterized
39
- and inline rules, middle actions, multiple entries, canonical imports,
40
- generated lexers, semantic locations/types, AST generation, grammar tests,
41
- and documentation tooling;
42
- - the conservative `--algorithm=ielr` backend;
43
- - LSP, watch, debug, coverage, browser playground, and static action-shadow
44
- integration.
45
-
46
- Experimental:
47
-
48
- - opt-in bounded insertion/deletion/replacement repair through
49
- `Runtime::RepairPolicy`;
50
- - syntax-only incremental CST sessions and conservative Blender subtree reuse.
96
+ Preview and Experimental:
97
+
98
+ The machine-readable [maturity audit](maturity.md) re-evaluates every current
99
+ feature. All eighteen Preview and both Experimental features remain at their
100
+ current maturity. The canonical summary below is generated from that registry
101
+ and checked against both public documents.
102
+
103
+ <!-- maturity-summary:start -->
104
+ Inventory: **19 Preview, 2 Experimental**. Active new Preview tracks: **2/3** (grammar syntax: **1/1**). Experimental product features: **2/5**.
105
+ Release dependency state: R001 **hold_external**; R002 **pending_exact_revision**; no feature is promoted by this audit.
106
+
107
+ | Stable ID | Feature | Current maturity | Decision | External use | Release gate |
108
+ | --- | --- | --- | --- | --- | --- |
109
+ | `ebnf-groups` | EBNF groups | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
110
+ | `parameterized-rules` | parameterized rules | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
111
+ | `inline-rules` | inline rules | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
112
+ | `middle-actions` | middle actions | Preview | Redesign Preview | not demonstrated | Blocked: R001, R002 |
113
+ | `multiple-entries` | multiple entries | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
114
+ | `canonical-imports` | canonical imports | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
115
+ | `generated-lexers` | generated lexers | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
116
+ | `semantic-locations-types` | semantic locations/types | Preview | Redesign Preview | not demonstrated | Blocked: R001, R002 |
117
+ | `ast-generation` | AST generation | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
118
+ | `grammar-tests` | grammar tests | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
119
+ | `documentation-tooling` | documentation tooling | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
120
+ | `impact` | Grammar impact analysis | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
121
+ | `ielr` | IELR | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
122
+ | `lsp` | LSP | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
123
+ | `watch` | watch | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
124
+ | `debug` | debug | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
125
+ | `coverage` | coverage | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
126
+ | `browser-playground` | browser playground | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
127
+ | `action-shadow` | action-shadow | Preview | Keep Preview | not demonstrated | Blocked: R001, R002 |
128
+ | `bounded-repair` | bounded repair | Experimental | Keep Experimental | not demonstrated | Blocked: R001, R002 |
129
+ | `incremental-cst` | incremental CST | Experimental | Keep Experimental | not demonstrated | Blocked: R001, R002 |
130
+ <!-- maturity-summary:end -->
51
131
 
52
132
  The batch CST contract is selected for the initial v1 API under the
53
133
  initial-major evidence rule above. The remaining Preview features have not
@@ -57,6 +137,20 @@ useful and inserted semantic values may be nil.
57
137
 
58
138
  ## Research decisions
59
139
 
140
+ - The Stage A matrix, golden, reproducibility, schema, adversarial, gallery,
141
+ fuzzing, dependency, and network gates now make feature-off and
142
+ algorithm-crossing regressions reviewable before promotion.
143
+ - Independent verification detects all twenty committed structural table
144
+ faults across every gallery/algorithm/table combination.
145
+ - Bounded grammar comparison distinguishes concrete shortest differences from
146
+ completed searches with no difference and never labels the latter a proof.
147
+ - Conflict repair passed its first fixed twenty-case capability measurement at
148
+ 20/20. The corpus is intentionally described as a regression baseline, not
149
+ a general repair-rate claim.
150
+ - The analysis-only Bison adapter imports and builds five checksum-pinned
151
+ external grammars. Bison-era CRuby production counts match, its one-state
152
+ acceptance-convention delta is explained, and current Lrama-only `%rule`
153
+ structure is reported as incomplete instead of receiving unsafe repairs.
60
154
  - IELR passed its correctness/state-bound spike and remains preview while it
61
155
  gains field experience. Direct LALR remains the default.
62
156
  - Bounded repair passed SP-4 with 8/10 useful plans and remains experimental.
@@ -71,16 +165,20 @@ The supported alternatives are canonical LR(1) or IELR for LALR inadequacy,
71
165
  bounded ambiguity/counterexample analysis for ambiguous grammars, and
72
166
  `--watch` with full deterministic reparsing for editing workflows.
73
167
 
74
- ## Core IR freeze
168
+ ## Core IR contract
75
169
 
76
- The required fields, meanings, identity rules, ordering, and validation
77
- semantics of the published core Grammar IR and Automaton IR versions are
78
- frozen. Existing version-1 and version-2 documents retain byte-stable
79
- round-trips and one-generation read compatibility.
170
+ Grammar IR and Automaton IR each have one current closed format. Their required
171
+ fields, meanings, identity rules, ordering, and validation semantics are
172
+ reviewed as one contract. Older documents are rejected at the load boundary;
173
+ there is no compatibility reader or migration command for either document.
80
174
 
81
- Additive optional core fields require a minor release. Meaning changes,
82
- required-field changes, or removals require a new major schema version and
83
- continued reading of the immediately preceding major version.
175
+ Before v1.0, an incompatible change to either current IR is a coordinated
176
+ contract change: update the writer, reader, schemas, fixtures, and downstream
177
+ consumers together. No legacy format is retained only to avoid a pre-release
178
+ breaking change.
179
+
180
+ Additive fields and meaning changes are deliberate schema changes with updated
181
+ evidence. Current schemas remain closed and reject unknown fields.
84
182
 
85
183
  The `x-` namespace is reserved for future experimental data and is not frozen.
86
184
  Current closed schemas intentionally reject unknown fields and do not emit
@@ -91,6 +189,8 @@ a later schema version.
91
189
 
92
190
  ## Compatibility policy
93
191
 
192
+ <!-- stable:compatibility-policy:v1 -->
193
+
94
194
  Compatible mode is the permanent default. Opt-in extensions, exact lookahead
95
195
  defaults, repair, and research algorithms do not silently replace compatible
96
196
  behavior. Public migration evidence uses public commands and black-box
@@ -101,13 +201,23 @@ An undeclared invalid token intentionally calls `on_error` before ordinary yacc
101
201
  recovery. This is the documented recommended behavior and is not changed by
102
202
  the freeze.
103
203
 
104
- Parser-table formats v1 through v6 remain readable for non-CST parsers. CST
105
- tables must use the current format v6 structured metadata; older CST tables
106
- and the boolean `cst: true` shape fail before token consumption with a
107
- regeneration instruction. Format v6 is the only writer and does not change
108
- Grammar IR v2. The closed `ibex_cst` schema v1 is a versioned interchange
109
- contract. Additive meaning requires a new schema version; readers do not
110
- accept unknown fields.
204
+ Parser-table format v6 is the only runtime table contract and the only table
205
+ writer. Older generated tables fail before token consumption with a
206
+ regeneration instruction; they are not a pre-v1 compatibility obligation.
207
+ The current Grammar IR adds generator-owned parser configuration without changing that
208
+ runtime table contract. The closed `ibex_cst` schema v1 is a versioned
209
+ interchange contract. Additive meaning requires a new schema version; readers
210
+ do not accept unknown fields.
211
+
212
+ ## Evidence is not compatibility
213
+
214
+ Exact source digests and clean-revision bindings establish what an evidence
215
+ record measured. They are audit-integrity requirements, not promises that a
216
+ consumer can execute an old generated artifact or reuse an old working-tree
217
+ capture. Historical evidence remains immutable for auditability; current
218
+ evidence must be regenerated and rebound after a relevant source or policy
219
+ change. A stale-evidence failure therefore means “refresh the measurement,”
220
+ not “preserve the old implementation forever.”
111
221
 
112
222
  ## Deprecation policy
113
223
 
@@ -116,8 +226,9 @@ two minor releases. The release notes and documentation must name the first
116
226
  warning release, replacement, migration command or procedure, and earliest
117
227
  removal release. Automated migration is supplied when practical; policy does
118
228
  not promise a command that the product does not provide. Preview features
119
- receive at least one minor release of notice, and Experimental features may
120
- change without notice.
229
+ receive at least one minor release of notice only after v1.0; before v1.0 the
230
+ release-note and maturity-evidence rule above applies. Experimental features
231
+ may change without notice.
121
232
 
122
233
  The pre-v1 mixed semantic/syntax CST was a Preview contract and is removed
123
234
  while selecting the initial stable API. Its parser tables are rejected with a
data/docs/status.md ADDED
@@ -0,0 +1,43 @@
1
+ ---
2
+ title: Project status
3
+ description: Human-readable release status, maturity boundaries, and evidence links for Ibex.
4
+ ---
5
+
6
+ # Project status
7
+
8
+ Ibex is pre-1.0. The default compatible mode is the conservative adoption
9
+ baseline, while Preview and Experimental features require explicit activation
10
+ and have feature-specific boundaries.
11
+
12
+ ## Release decision
13
+
14
+ The v1.0 publication decision is currently on hold because the published error
15
+ experience evidence still needs an independent review. This blocks publication
16
+ of that release candidate; it does not freeze feature development. The current
17
+ Grammar IR, Automaton IR, table, report, and runtime contracts remain closed
18
+ and validated while Preview work continues under the stability policy.
19
+
20
+ The complete rationale and evidence limitations are in the [release readiness
21
+ report](release-readiness.md). The [error UX review status](error-ux-review-status-v1.json)
22
+ is the machine-readable record; this page is the human entry point.
23
+
24
+ ## Maturity at a glance
25
+
26
+ - **Stable baseline:** racc-compatible grammar input, default LALR construction,
27
+ current parser/runtime contracts, and format-v6 batch CST.
28
+ - **Preview:** extended grammar syntax, generated lexers, IELR, LSP, watch,
29
+ diagnostics, and the browser playground. These require explicit activation
30
+ or are bounded tools.
31
+ - **Experimental:** syntax-only incremental CST sessions and selected research
32
+ surfaces. They may change without the Stable compatibility promise.
33
+
34
+ Consult [maturity.md](maturity.md) and [stability.md](stability.md) for the
35
+ canonical feature registry, activation mechanism, and promotion requirements.
36
+
37
+ ## Evidence boundaries
38
+
39
+ Repository tests and bounded benchmarks are evidence for the recorded revision,
40
+ not universal performance claims. Comparative statements link to the relevant
41
+ [comparison policy](comparison-policy.md), [workload registry](workloads.md),
42
+ and release artifact. Ibex does not claim to be a sandbox: generated semantic
43
+ parsers execute application Ruby.