ibex 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (437) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +95 -272
  3. data/docs/architecture.md +187 -35
  4. data/docs/bison-import.md +103 -0
  5. data/docs/comparison-policy.md +127 -0
  6. data/docs/configuration-model.md +59 -0
  7. data/docs/configuration-report.md +28 -0
  8. data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
  9. data/docs/conflict-explanation-study.md +62 -0
  10. data/docs/construction-profiling.md +157 -0
  11. data/docs/cst-migration.md +3 -4
  12. data/docs/cst.md +24 -0
  13. data/docs/decisions/0000-template.md +24 -0
  14. data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
  15. data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
  16. data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
  17. data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
  18. data/docs/decisions/0005-contained-grammar-composition.md +34 -0
  19. data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
  20. data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
  21. data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
  22. data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
  23. data/docs/decisions/0010-committed-runtime-observation.md +34 -0
  24. data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
  25. data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
  26. data/docs/decisions/0013-transactional-generation-publication.md +37 -0
  27. data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
  28. data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
  29. data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
  30. data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
  31. data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
  32. data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
  33. data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
  34. data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
  35. data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
  36. data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
  37. data/docs/decisions/0024-direct-ielr-construction.md +42 -0
  38. data/docs/decisions/README.md +74 -0
  39. data/docs/declarative-configuration.md +262 -0
  40. data/docs/development.md +179 -2
  41. data/docs/direct-ielr-decision.md +120 -0
  42. data/docs/direct-multi-entry-decision.md +82 -0
  43. data/docs/editor-setup.md +2 -1
  44. data/docs/error-ux-review-rubric-v1.md +117 -0
  45. data/docs/error-ux-reviews/v1/records/README.md +22 -0
  46. data/docs/error-ux-round2-review-status-v1.json +27 -0
  47. data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
  48. data/docs/error-ux-round2-v1.json +1581 -0
  49. data/docs/error-ux-round2.md +111 -0
  50. data/docs/error-ux.md +54 -0
  51. data/docs/getting-started.md +80 -0
  52. data/docs/grammar-reference.md +184 -13
  53. data/docs/ielr-design.md +43 -0
  54. data/docs/ielr.md +92 -0
  55. data/docs/ja/bison-import.md +51 -0
  56. data/docs/ja/stable-api.md +53 -0
  57. data/docs/lexer-construction-profile.md +109 -0
  58. data/docs/lexer-migration.md +5 -0
  59. data/docs/maturity.md +128 -0
  60. data/docs/project-site-strategy.md +54 -0
  61. data/docs/racc-migration-evidence.md +45 -0
  62. data/docs/racc-migration.md +11 -0
  63. data/docs/release-readiness.md +126 -68
  64. data/docs/repair-semantics.md +123 -0
  65. data/docs/runtime-abi-evolution.md +307 -0
  66. data/docs/stability.md +144 -34
  67. data/docs/status.md +43 -0
  68. data/docs/syntax-sessions.md +183 -0
  69. data/docs/table-artifact.md +125 -0
  70. data/docs/test-interactions.md +219 -0
  71. data/docs/verification-report.md +137 -0
  72. data/docs/verifier-trust-boundary.md +199 -0
  73. data/docs/workloads.md +132 -0
  74. data/lib/ibex/analysis/digraph.rb +128 -0
  75. data/lib/ibex/analysis.rb +1 -0
  76. data/lib/ibex/bison_import/importer.rb +516 -0
  77. data/lib/ibex/bison_import/tokenizer.rb +260 -0
  78. data/lib/ibex/bison_import.rb +200 -0
  79. data/lib/ibex/bounded_subprocess.rb +171 -0
  80. data/lib/ibex/cli/ambiguity.rb +14 -5
  81. data/lib/ibex/cli/analysis.rb +134 -0
  82. data/lib/ibex/cli/bison_import.rb +121 -0
  83. data/lib/ibex/cli/config.rb +128 -0
  84. data/lib/ibex/cli/diagnostics.rb +31 -11
  85. data/lib/ibex/cli/documentation.rb +11 -8
  86. data/lib/ibex/cli/equiv.rb +169 -0
  87. data/lib/ibex/cli/error_messages.rb +11 -4
  88. data/lib/ibex/cli/explain.rb +14 -5
  89. data/lib/ibex/cli/fix.rb +203 -0
  90. data/lib/ibex/cli/formatting.rb +11 -10
  91. data/lib/ibex/cli/fuzz.rb +200 -0
  92. data/lib/ibex/cli/fuzz_regressions.rb +145 -0
  93. data/lib/ibex/cli/generation_artifacts.rb +53 -2
  94. data/lib/ibex/cli/generation_error_messages.rb +1 -1
  95. data/lib/ibex/cli/grammar_tests.rb +32 -14
  96. data/lib/ibex/cli/ir_tools.rb +5 -61
  97. data/lib/ibex/cli/outputs.rb +60 -33
  98. data/lib/ibex/cli/reduce.rb +202 -0
  99. data/lib/ibex/cli/reduce_reporting.rb +71 -0
  100. data/lib/ibex/cli/samples.rb +49 -23
  101. data/lib/ibex/cli/verify.rb +116 -0
  102. data/lib/ibex/cli/watch.rb +6 -5
  103. data/lib/ibex/cli.rb +357 -56
  104. data/lib/ibex/codegen/action_locations.rb +1 -1
  105. data/lib/ibex/codegen/action_method_source.rb +5 -6
  106. data/lib/ibex/codegen/action_source.rb +3 -2
  107. data/lib/ibex/codegen/ambiguity.rb +5 -4
  108. data/lib/ibex/codegen/explain.rb +123 -55
  109. data/lib/ibex/codegen/generated_action_abi.rb +10 -5
  110. data/lib/ibex/codegen/rbs.rb +15 -7
  111. data/lib/ibex/codegen/report.rb +25 -13
  112. data/lib/ibex/codegen/ruby.rb +28 -64
  113. data/lib/ibex/codegen/ruby_actions.rb +9 -6
  114. data/lib/ibex/codegen/ruby_syntax.rb +15 -9
  115. data/lib/ibex/configuration/analysis_grammar.rb +35 -0
  116. data/lib/ibex/configuration/explanation.rb +421 -0
  117. data/lib/ibex/configuration/inspector.rb +232 -0
  118. data/lib/ibex/configuration.rb +580 -0
  119. data/lib/ibex/coverage/collector.rb +21 -11
  120. data/lib/ibex/coverage/event_stream.rb +13 -7
  121. data/lib/ibex/coverage/report.rb +26 -18
  122. data/lib/ibex/coverage/runtime_event_validator.rb +30 -21
  123. data/lib/ibex/delta_reducer.rb +99 -0
  124. data/lib/ibex/diff.rb +140 -0
  125. data/lib/ibex/equiv/machine.rb +135 -0
  126. data/lib/ibex/equiv.rb +373 -0
  127. data/lib/ibex/fix.rb +557 -0
  128. data/lib/ibex/frontend/ast.rb +22 -2
  129. data/lib/ibex/frontend/bootstrap_parser.rb +6 -13
  130. data/lib/ibex/frontend/diagnostic.rb +1 -1
  131. data/lib/ibex/frontend/formatter.rb +72 -26
  132. data/lib/ibex/frontend/generated_parser.rb +147 -117
  133. data/lib/ibex/frontend/generated_parser_base.rb +7 -3
  134. data/lib/ibex/frontend/generated_parser_includes.rb +1 -11
  135. data/lib/ibex/frontend/generation.rb +1 -1
  136. data/lib/ibex/frontend/parser/declarations.rb +17 -2
  137. data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
  138. data/lib/ibex/frontend/regenerator.rb +1 -0
  139. data/lib/ibex/frontend/resolution.rb +2 -1
  140. data/lib/ibex/frontend/resolver.rb +1 -1
  141. data/lib/ibex/frontend/rule_documentation.rb +2 -1
  142. data/lib/ibex/frontend/source_cursor.rb +2 -2
  143. data/lib/ibex/frontend/source_span.rb +9 -1
  144. data/lib/ibex/frontend/token_adapter/declaration_state.rb +38 -6
  145. data/lib/ibex/frontend/token_adapter.rb +10 -0
  146. data/lib/ibex/frontend.rb +1 -0
  147. data/lib/ibex/fuzz.rb +219 -0
  148. data/lib/ibex/generation_input.rb +1 -1
  149. data/lib/ibex/generation_manifest.rb +36 -25
  150. data/lib/ibex/generation_transaction.rb +1 -1
  151. data/lib/ibex/generation_transaction_recovery.rb +4 -4
  152. data/lib/ibex/generation_transaction_validation.rb +2 -2
  153. data/lib/ibex/grammar_tests.rb +1 -1
  154. data/lib/ibex/ir/automaton_ir.rb +61 -19
  155. data/lib/ibex/ir/grammar_ir.rb +76 -45
  156. data/lib/ibex/ir/lexer_ir.rb +5 -5
  157. data/lib/ibex/ir/parser_contract.rb +135 -0
  158. data/lib/ibex/ir/serialize.rb +203 -76
  159. data/lib/ibex/ir/validator/automaton.rb +64 -34
  160. data/lib/ibex/ir/validator/base.rb +51 -33
  161. data/lib/ibex/ir/validator/grammar.rb +113 -83
  162. data/lib/ibex/ir/validator/lexer.rb +2 -2
  163. data/lib/ibex/ir/validator.rb +2 -1
  164. data/lib/ibex/ir.rb +16 -3
  165. data/lib/ibex/lalr/build_metrics.rb +50 -2
  166. data/lib/ibex/lalr/builder.rb +191 -56
  167. data/lib/ibex/lalr/conflict_search.rb +15 -4
  168. data/lib/ibex/lalr/counterexample.rb +19 -14
  169. data/lib/ibex/lalr/direct_lookaheads.rb +110 -57
  170. data/lib/ibex/lalr/goto_follows.rb +229 -0
  171. data/lib/ibex/lalr/ielr/annotator.rb +214 -0
  172. data/lib/ibex/lalr/ielr/bits.rb +28 -0
  173. data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
  174. data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
  175. data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
  176. data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
  177. data/lib/ibex/lalr/ielr/split_state.rb +20 -0
  178. data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
  179. data/lib/ibex/lalr/ielr_partition.rb +22 -8
  180. data/lib/ibex/lalr/inadequacy_report.rb +50 -0
  181. data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
  182. data/lib/ibex/lalr/lr0_collection.rb +121 -0
  183. data/lib/ibex/lalr/unreachable_states.rb +80 -0
  184. data/lib/ibex/lalr.rb +52 -1
  185. data/lib/ibex/location.rb +3 -3
  186. data/lib/ibex/lsp/document_handlers.rb +9 -7
  187. data/lib/ibex/lsp/initialization_handlers.rb +9 -7
  188. data/lib/ibex/lsp/navigation_handlers.rb +25 -11
  189. data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
  190. data/lib/ibex/lsp/position_codec.rb +6 -2
  191. data/lib/ibex/lsp/request_handlers.rb +3 -2
  192. data/lib/ibex/lsp/request_support.rb +14 -7
  193. data/lib/ibex/lsp/server.rb +13 -11
  194. data/lib/ibex/lsp/symbol_index.rb +13 -13
  195. data/lib/ibex/lsp/symbol_index_builder.rb +30 -20
  196. data/lib/ibex/lsp/symbol_occurrence.rb +8 -2
  197. data/lib/ibex/lsp/transport.rb +13 -3
  198. data/lib/ibex/lsp/workspace.rb +5 -4
  199. data/lib/ibex/lsp/workspace_analyzer.rb +1 -1
  200. data/lib/ibex/lsp.rb +1 -0
  201. data/lib/ibex/messages/en.yml +28 -0
  202. data/lib/ibex/messages/ja.yml +28 -0
  203. data/lib/ibex/messages.rb +68 -0
  204. data/lib/ibex/metrics.rb +175 -0
  205. data/lib/ibex/normalize/declarations.rb +2 -0
  206. data/lib/ibex/normalize/diagnostics.rb +63 -38
  207. data/lib/ibex/normalize/expander.rb +5 -4
  208. data/lib/ibex/normalize/expression.rb +8 -6
  209. data/lib/ibex/normalize/grammar_builder.rb +26 -0
  210. data/lib/ibex/normalize/inline_expansion.rb +100 -49
  211. data/lib/ibex/normalize/lexer.rb +3 -3
  212. data/lib/ibex/normalize/parameter_ebnf_lowering.rb +11 -10
  213. data/lib/ibex/normalize/parameter_lowering.rb +40 -23
  214. data/lib/ibex/normalize/parameters.rb +31 -4
  215. data/lib/ibex/normalize/parser_configuration.rb +70 -0
  216. data/lib/ibex/normalize.rb +10 -20
  217. data/lib/ibex/racc_migration/report.rb +6 -3
  218. data/lib/ibex/rake_task.rb +1 -1
  219. data/lib/ibex/samples.rb +43 -13
  220. data/lib/ibex/table_artifact/builder.rb +266 -0
  221. data/lib/ibex/table_artifact/cst_projection.rb +70 -0
  222. data/lib/ibex/table_artifact/document.rb +39 -0
  223. data/lib/ibex/table_artifact/executor.rb +187 -0
  224. data/lib/ibex/table_artifact/serializer.rb +57 -0
  225. data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
  226. data/lib/ibex/table_artifact/validator/support.rb +86 -0
  227. data/lib/ibex/table_artifact/validator/tables.rb +238 -0
  228. data/lib/ibex/table_artifact/validator.rb +253 -0
  229. data/lib/ibex/table_artifact.rb +88 -0
  230. data/lib/ibex/table_simulation/result.rb +6 -2
  231. data/lib/ibex/table_simulation/step.rb +3 -1
  232. data/lib/ibex/tables.rb +14 -0
  233. data/lib/ibex/verifiable_generation_bundle.rb +86 -0
  234. data/lib/ibex/verification_report/builder.rb +159 -0
  235. data/lib/ibex/verification_report/canonical_ir.rb +123 -0
  236. data/lib/ibex/verification_report/logical_path.rb +68 -0
  237. data/lib/ibex/verification_report/validator.rb +329 -0
  238. data/lib/ibex/verification_report.rb +69 -0
  239. data/lib/ibex/verify/action_correspondence.rb +138 -0
  240. data/lib/ibex/verify/language_witness.rb +295 -0
  241. data/lib/ibex/verify/reference_collection.rb +190 -0
  242. data/lib/ibex/verify/result.rb +73 -0
  243. data/lib/ibex/verify/verifier.rb +598 -0
  244. data/lib/ibex/verify.rb +15 -0
  245. data/lib/ibex/version.rb +1 -1
  246. data/lib/ibex/watch/runner.rb +10 -1
  247. data/lib/ibex/watch/source_snapshot.rb +13 -8
  248. data/lib/ibex.rb +10 -0
  249. data/schema/{automaton-ir-v1.schema.json → automaton-ir-definitions.schema.json} +6 -6
  250. data/schema/{automaton-ir-v2.schema.json → automaton-ir.schema.json} +14 -7
  251. data/schema/bison-import-v1.schema.json +106 -0
  252. data/schema/conflict-explanation-review-v1.schema.json +196 -0
  253. data/schema/conflict-explanation-study-v1.schema.json +528 -0
  254. data/schema/construction-profile-v1.schema.json +779 -0
  255. data/schema/diff-v1.schema.json +41 -0
  256. data/schema/direct-ielr-decision-v1.schema.json +200 -0
  257. data/schema/equiv-v1.schema.json +64 -0
  258. data/schema/error-ux-review-v1.schema.json +609 -0
  259. data/schema/error-ux-round2-review-v1.schema.json +191 -0
  260. data/schema/error-ux-round2-v1.schema.json +519 -0
  261. data/schema/explain-v1.schema.json +63 -1
  262. data/schema/fix-v1.schema.json +64 -0
  263. data/schema/fix-v2.schema.json +66 -0
  264. data/schema/fix-v3.schema.json +101 -0
  265. data/schema/fuzz-regression-v1.schema.json +79 -0
  266. data/schema/fuzz-v1.schema.json +114 -0
  267. data/schema/{grammar-ir-v2.schema.json → grammar-ir-extensions.schema.json} +20 -65
  268. data/schema/{grammar-ir-v1.schema.json → grammar-ir-foundation.schema.json} +3 -3
  269. data/schema/grammar-ir.schema.json +271 -0
  270. data/schema/ielr-benchmark-v1.schema.json +85 -0
  271. data/schema/lexer-profile-v1.schema.json +491 -0
  272. data/schema/metrics-v1.schema.json +60 -0
  273. data/schema/reduce-v1.schema.json +22 -0
  274. data/schema/reduce-v2.schema.json +63 -0
  275. data/schema/table-artifact-v1.schema.json +377 -0
  276. data/schema/verification-report-v1.schema.json +222 -0
  277. data/schema/verify-v1.schema.json +42 -0
  278. data/sig/ibex/analysis/digraph.rbs +19 -0
  279. data/sig/ibex/bison_import/importer.rbs +140 -0
  280. data/sig/ibex/bison_import/tokenizer.rbs +91 -0
  281. data/sig/ibex/bison_import.rbs +95 -0
  282. data/sig/ibex/bounded_subprocess.rbs +76 -0
  283. data/sig/ibex/cli/ambiguity.rbs +6 -0
  284. data/sig/ibex/cli/analysis.rbs +43 -0
  285. data/sig/ibex/cli/bison_import.rbs +32 -0
  286. data/sig/ibex/cli/config.rbs +35 -0
  287. data/sig/ibex/cli/diagnostics.rbs +16 -3
  288. data/sig/ibex/cli/documentation.rbs +5 -1
  289. data/sig/ibex/cli/equiv.rbs +49 -0
  290. data/sig/ibex/cli/error_messages.rbs +6 -0
  291. data/sig/ibex/cli/explain.rbs +6 -0
  292. data/sig/ibex/cli/fix.rbs +53 -0
  293. data/sig/ibex/cli/formatting.rbs +5 -1
  294. data/sig/ibex/cli/fuzz.rbs +47 -0
  295. data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
  296. data/sig/ibex/cli/generation_artifacts.rbs +21 -2
  297. data/sig/ibex/cli/generation_error_messages.rbs +2 -2
  298. data/sig/ibex/cli/grammar_tests.rbs +15 -3
  299. data/sig/ibex/cli/ir_tools.rbs +7 -19
  300. data/sig/ibex/cli/outputs.rbs +9 -3
  301. data/sig/ibex/cli/reduce.rbs +60 -0
  302. data/sig/ibex/cli/reduce_reporting.rbs +27 -0
  303. data/sig/ibex/cli/samples.rbs +8 -0
  304. data/sig/ibex/cli/verify.rbs +28 -0
  305. data/sig/ibex/cli/watch.rbs +2 -2
  306. data/sig/ibex/cli.rbs +79 -5
  307. data/sig/ibex/codegen/action_method_source.rbs +4 -4
  308. data/sig/ibex/codegen/explain.rbs +45 -25
  309. data/sig/ibex/codegen/generated_action_abi.rbs +14 -10
  310. data/sig/ibex/codegen/rbs.rbs +12 -10
  311. data/sig/ibex/codegen/report.rbs +12 -12
  312. data/sig/ibex/codegen/ruby.rbs +18 -15
  313. data/sig/ibex/codegen/ruby_actions.rbs +6 -6
  314. data/sig/ibex/codegen/ruby_syntax.rbs +10 -8
  315. data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
  316. data/sig/ibex/configuration/explanation.rbs +195 -0
  317. data/sig/ibex/configuration/inspector.rbs +53 -0
  318. data/sig/ibex/configuration.rbs +242 -0
  319. data/sig/ibex/coverage/collector.rbs +16 -14
  320. data/sig/ibex/coverage/event_stream.rbs +12 -10
  321. data/sig/ibex/coverage/report.rbs +20 -18
  322. data/sig/ibex/coverage/runtime_event_validator.rbs +38 -36
  323. data/sig/ibex/delta_reducer.rbs +38 -0
  324. data/sig/ibex/diff.rbs +47 -0
  325. data/sig/ibex/equiv/machine.rbs +54 -0
  326. data/sig/ibex/equiv.rbs +129 -0
  327. data/sig/ibex/fix.rbs +138 -0
  328. data/sig/ibex/frontend/ast.rbs +33 -7
  329. data/sig/ibex/frontend/bootstrap_parser.rbs +2 -0
  330. data/sig/ibex/frontend/diagnostic.rbs +2 -2
  331. data/sig/ibex/frontend/formatter.rbs +38 -30
  332. data/sig/ibex/frontend/generated_parser.rbs +87 -77
  333. data/sig/ibex/frontend/generated_parser_base.rbs +4 -2
  334. data/sig/ibex/frontend/parser/declarations.rbs +3 -0
  335. data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
  336. data/sig/ibex/frontend/resolution.rbs +3 -2
  337. data/sig/ibex/frontend/source_cursor.rbs +2 -2
  338. data/sig/ibex/frontend/source_span.rbs +5 -2
  339. data/sig/ibex/frontend/token_adapter/declaration_state.rbs +13 -1
  340. data/sig/ibex/frontend/token_adapter.rbs +6 -0
  341. data/sig/ibex/fuzz.rbs +71 -0
  342. data/sig/ibex/generation_input.rbs +2 -2
  343. data/sig/ibex/generation_manifest.rbs +35 -32
  344. data/sig/ibex/generation_transaction_recovery.rbs +2 -2
  345. data/sig/ibex/generation_transaction_validation.rbs +2 -2
  346. data/sig/ibex/grammar_tests.rbs +2 -2
  347. data/sig/ibex/ir/automaton_ir.rbs +30 -11
  348. data/sig/ibex/ir/grammar_ir.rbs +41 -27
  349. data/sig/ibex/ir/lexer_ir.rbs +8 -8
  350. data/sig/ibex/ir/parser_contract.rbs +69 -0
  351. data/sig/ibex/ir/serialize.rbs +63 -28
  352. data/sig/ibex/ir/validator/automaton.rbs +47 -44
  353. data/sig/ibex/ir/validator/base.rbs +35 -31
  354. data/sig/ibex/ir/validator/grammar.rbs +78 -67
  355. data/sig/ibex/ir/validator/lexer.rbs +4 -4
  356. data/sig/ibex/ir.rbs +9 -3
  357. data/sig/ibex/lalr/build_metrics.rbs +42 -2
  358. data/sig/ibex/lalr/builder.rbs +75 -24
  359. data/sig/ibex/lalr/conflict_search.rbs +5 -2
  360. data/sig/ibex/lalr/counterexample.rbs +17 -14
  361. data/sig/ibex/lalr/direct_lookaheads.rbs +35 -28
  362. data/sig/ibex/lalr/goto_follows.rbs +89 -0
  363. data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
  364. data/sig/ibex/lalr/ielr/bits.rbs +14 -0
  365. data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
  366. data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
  367. data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
  368. data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
  369. data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
  370. data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
  371. data/sig/ibex/lalr/ielr_partition.rbs +17 -5
  372. data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
  373. data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
  374. data/sig/ibex/lalr/lr0_collection.rbs +50 -0
  375. data/sig/ibex/lalr/unreachable_states.rbs +25 -0
  376. data/sig/ibex/lalr.rbs +11 -1
  377. data/sig/ibex/location.rbs +6 -6
  378. data/sig/ibex/lsp/document_handlers.rbs +8 -8
  379. data/sig/ibex/lsp/initialization_handlers.rbs +12 -12
  380. data/sig/ibex/lsp/navigation_handlers.rbs +15 -12
  381. data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
  382. data/sig/ibex/lsp/position_codec.rbs +8 -4
  383. data/sig/ibex/lsp/request_handlers.rbs +4 -4
  384. data/sig/ibex/lsp/request_support.rbs +8 -8
  385. data/sig/ibex/lsp/server.rbs +16 -16
  386. data/sig/ibex/lsp/symbol_index.rbs +20 -21
  387. data/sig/ibex/lsp/symbol_index_builder.rbs +9 -8
  388. data/sig/ibex/lsp/symbol_occurrence.rbs +11 -5
  389. data/sig/ibex/lsp/transport.rbs +18 -6
  390. data/sig/ibex/lsp/workspace.rbs +3 -2
  391. data/sig/ibex/lsp/workspace_analyzer.rbs +1 -1
  392. data/sig/ibex/messages.rbs +26 -0
  393. data/sig/ibex/metrics.rbs +50 -0
  394. data/sig/ibex/normalize/diagnostics.rbs +4 -4
  395. data/sig/ibex/normalize/expander.rbs +2 -2
  396. data/sig/ibex/normalize/expression.rbs +10 -8
  397. data/sig/ibex/normalize/grammar_builder.rbs +11 -0
  398. data/sig/ibex/normalize/inline_expansion.rbs +59 -50
  399. data/sig/ibex/normalize/lexer.rbs +4 -4
  400. data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +15 -14
  401. data/sig/ibex/normalize/parameter_lowering.rbs +18 -18
  402. data/sig/ibex/normalize/parameters.rbs +13 -8
  403. data/sig/ibex/normalize/parser_configuration.rbs +23 -0
  404. data/sig/ibex/normalize.rbs +11 -5
  405. data/sig/ibex/racc_migration/report.rbs +8 -6
  406. data/sig/ibex/rake_task.rbs +1 -1
  407. data/sig/ibex/samples.rbs +13 -4
  408. data/sig/ibex/table_artifact/builder.rbs +93 -0
  409. data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
  410. data/sig/ibex/table_artifact/document.rbs +27 -0
  411. data/sig/ibex/table_artifact/executor.rbs +66 -0
  412. data/sig/ibex/table_artifact/serializer.rbs +25 -0
  413. data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
  414. data/sig/ibex/table_artifact/validator/support.rbs +46 -0
  415. data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
  416. data/sig/ibex/table_artifact/validator.rbs +66 -0
  417. data/sig/ibex/table_artifact.rbs +32 -0
  418. data/sig/ibex/table_simulation/result.rbs +8 -4
  419. data/sig/ibex/table_simulation/step.rbs +4 -2
  420. data/sig/ibex/tables.rbs +10 -9
  421. data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
  422. data/sig/ibex/verification_report/builder.rbs +53 -0
  423. data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
  424. data/sig/ibex/verification_report/logical_path.rbs +34 -0
  425. data/sig/ibex/verification_report/validator.rbs +114 -0
  426. data/sig/ibex/verification_report.rbs +44 -0
  427. data/sig/ibex/verify/action_correspondence.rbs +50 -0
  428. data/sig/ibex/verify/language_witness.rbs +112 -0
  429. data/sig/ibex/verify/reference_collection.rbs +63 -0
  430. data/sig/ibex/verify/result.rbs +52 -0
  431. data/sig/ibex/verify/verifier.rbs +170 -0
  432. data/sig/ibex/verify.rbs +6 -0
  433. data/sig/ibex/watch/runner.rbs +12 -2
  434. data/sig/ibex/watch/source_snapshot.rbs +13 -9
  435. metadata +232 -12
  436. data/lib/ibex/ir/migration.rb +0 -120
  437. data/sig/ibex/ir/migration.rbs +0 -34
@@ -0,0 +1,36 @@
1
+ # ADR 0008: Separate the runtime behind a versioned parser-table ABI
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Applications executing a generated parser do not need the grammar frontend or
9
+ generator. Independently installed generated code and runtime code still need a
10
+ precise compatibility boundary; otherwise a table-shape change can fail or be
11
+ misinterpreted during a parse.
12
+
13
+ ## Decision
14
+
15
+ The dependency-free parser runtime ships as `ibex-runtime`; the generator
16
+ depends on it but runtime deployments do not depend on generator code. Normal
17
+ generated parsers require the runtime package, while embedded output carries
18
+ the same runtime implementation.
19
+
20
+ Every generated table declares a parser-table format version. The runtime
21
+ validates the version and marker combinations before consuming input. Changes
22
+ to required shape, action calling convention, or table meaning increment that
23
+ version.
24
+
25
+ Plain and compact tables are two encodings of the same lookup contract.
26
+ Compression may change representation only when every declared and unknown
27
+ token retains the same shift, reduce, accept, or error result. In particular,
28
+ explicit error cells must mask default reductions.
29
+
30
+ ## Consequences
31
+
32
+ - Generator and runtime packaging can evolve independently within the declared
33
+ dependency and table-format contracts.
34
+ - Unsupported generated parsers fail before lexer or semantic side effects.
35
+ - Compact representation remains replaceable and optimizable without becoming
36
+ a second parser semantics.
@@ -0,0 +1,34 @@
1
+ # ADR 0009: Isolate mutable parser sessions from immutable parser data
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Generated tables are immutable program data, while LR stacks, lexer state,
9
+ lookahead, recovery state, observers, and incremental caches are per parse.
10
+ Supporting pull, yielding, and push input through separate engines would make
11
+ their recovery and callback behavior drift.
12
+
13
+ ## Decision
14
+
15
+ Generated parser classes own recursively frozen tables. Each parser instance
16
+ owns one mutable session and permits only one active driver. Pull, yielding, and
17
+ push APIs feed the same LR transition and recovery machinery; nested or mixed
18
+ drivers fail before mutating state.
19
+
20
+ State, semantic value, optional location, and optional Green stacks remain
21
+ index-aligned. Capabilities allocate their parallel state only when the
22
+ generated grammar or runtime tooling requires it.
23
+
24
+ Per-instance resource limits bound stack growth, recovery attempts, and other
25
+ session-owned search or memo structures. Exceeding a bound produces a
26
+ structured failure rather than unbounded work. Sharing a parser instance or
27
+ its application callbacks across concurrent parses is outside the contract.
28
+
29
+ ## Consequences
30
+
31
+ - Parser classes and eligible table graphs can be shared while sessions remain
32
+ independently mutable.
33
+ - All input lifecycles preserve one action, recovery, and callback ordering.
34
+ - Applications choose stricter or larger limits without regenerating tables.
@@ -0,0 +1,34 @@
1
+ # ADR 0010: Observe only committed runtime transitions
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Hooks, tracing, coverage, and debugging need parser activity without depending
9
+ on private mutable stacks. Emitting speculative events makes failed semantic
10
+ actions and recovery attempts indistinguishable from committed parser state.
11
+ Optional observation must also remain removable from the normal hot path.
12
+
13
+ ## Decision
14
+
15
+ Runtime hooks and versioned events are emitted at documented committed
16
+ boundaries for shifts, reductions, errors, recovery, discard, acceptance, and
17
+ rejection. Payloads are immutable snapshots or bounded JSON-safe summaries;
18
+ they never expose live parser stacks or retain arbitrary application objects.
19
+ Observer failures are visible to the parser caller.
20
+
21
+ Tracing and coverage consume this public boundary rather than adding parser
22
+ drivers. When no observer or hook is active, the runtime does not construct
23
+ event payloads.
24
+
25
+ The generic runtime is the semantic reference. A fast path may run only after
26
+ proving that every omitted observable capability is inactive, and must fall
27
+ back before a newly enabled capability could miss its documented next event.
28
+
29
+ ## Consequences
30
+
31
+ - Runtime tools compose without reading implementation state.
32
+ - Coverage and traces describe completed transitions rather than intentions.
33
+ - Performance work must preserve invalidation rules whenever a new observation
34
+ capability is added.
@@ -0,0 +1,37 @@
1
+ # ADR 0011: Compile opaque actions once behind a versioned calling boundary
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Generated semantic actions need grammar-source backtraces, access to reduction
9
+ values and locations, and efficient calls from the runtime. Re-evaluating
10
+ source on every reduction is expensive, while changing generated method
11
+ arguments can silently break older tables or application callables.
12
+
13
+ ## Decision
14
+
15
+ The generator builds each opaque action method once and compiles it when the
16
+ generated class loads, preserving the grammar filename and line. Direct
17
+ generated-file mode emits the equivalent method body for inspection. Static
18
+ action-shadow output uses the same method-source builder and is never executed
19
+ by Ibex.
20
+
21
+ Generated action call shapes are explicit production metadata covered by the
22
+ parser-table format version. The runtime validates markers before input and
23
+ grants generated-only ABIs only to generated action methods. Handwritten
24
+ methods and callables keep their documented application ABI.
25
+
26
+ Action dependency analysis is conservative. It may select a narrower values or
27
+ positional ABI only when lexical evidence proves omitted stacks, locations, or
28
+ containers are unused; unsupported or unparsable source uses the general ABI.
29
+ Location references are rewritten only at Ruby code tokens, not inside literal
30
+ or comment content.
31
+
32
+ ## Consequences
33
+
34
+ - Actions compile once while retaining useful source-mapped failures.
35
+ - Generated ABI optimization cannot silently change application callables.
36
+ - Conservative fallback can miss an optimization but cannot omit an input the
37
+ action may observe.
@@ -0,0 +1,37 @@
1
+ # ADR 0012: Keep parser analysis bounded and free of semantic execution
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Conflict counterexamples, sample generation, table debugging, ambiguity
9
+ checks, migration inspection, and automatic repair explore potentially
10
+ unbounded parser or grammar configurations. Executing semantic actions on
11
+ speculative paths would duplicate application side effects and make read-only
12
+ tools unsafe.
13
+
14
+ ## Decision
15
+
16
+ Static tools operate on validated Grammar IR, Automaton IR, or parser-table
17
+ state only. They never load generated application code or invoke semantic
18
+ actions. Every search has explicit token, configuration, stack, step, or
19
+ expansion limits and terminates when a limit is reached.
20
+
21
+ Results distinguish a proof from bounded evidence and from exhaustion. Failure
22
+ to find an ambiguity is never reported as proof of unambiguity. Table
23
+ simulation applies the same explicit-cell-before-default lookup rule as the
24
+ runtime.
25
+
26
+ Optional runtime repair also searches table configurations without actions.
27
+ Only the selected repair is replayed through the ordinary runtime, where
28
+ semantic actions and observers execute once on the committed path.
29
+
30
+ ## Consequences
31
+
32
+ - Analysis is suitable for untrusted validated artifacts within caller-supplied
33
+ input-size limits.
34
+ - Deep grammars may require explicitly larger budgets or produce an
35
+ inconclusive result.
36
+ - Runtime and tooling share table semantics without sharing mutable parser
37
+ internals.
@@ -0,0 +1,37 @@
1
+ # ADR 0013: Publish generated artifacts as a verified transaction
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ One generation can produce Ruby, RBS, static action source, reports, and
9
+ metadata. Writing outputs as renderers finish can expose a mixed generation,
10
+ and filesystem aliases or a source change during generation can invalidate the
11
+ candidate being published.
12
+
13
+ ## Decision
14
+
15
+ Generation renders every requested artifact in memory before changing output
16
+ paths. It records the exact input closure and revalidates it before
17
+ publication. Targets are resolved and checked for input aliases and
18
+ cross-output collisions.
19
+
20
+ Stages and recovery backups are created exclusively beside their targets.
21
+ Publication uses ordered locks, flushed and synchronized stages, atomic
22
+ renames, directory synchronization, and reverse-order restoration on failure.
23
+ Because a portable filesystem cannot atomically replace several paths, an
24
+ optional final manifest is the coherence marker: readers that need one
25
+ generation verify artifact digests against a stable manifest read.
26
+
27
+ Watch mode runs this same transaction only after a stable source closure is
28
+ observed; it never publishes a candidate built from changing inputs.
29
+
30
+ ## Consequences
31
+
32
+ - Rendering, source races, and recoverable write failures do not intentionally
33
+ publish a partial candidate as complete.
34
+ - Multi-file coherence requires the manifest protocol rather than an
35
+ unsupported cross-file atomicity claim.
36
+ - Transaction and watch code pay filesystem synchronization and hashing costs
37
+ in exchange for a recoverable publication boundary.
@@ -0,0 +1,35 @@
1
+ # ADR 0014: Model generated lexers as an independently versioned stage
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Handwritten token sources keep the parser runtime general, but a generated
9
+ lexer needs durable matching, state, streaming, and location semantics.
10
+ Embedding those details directly in Grammar or Automaton IR would tie lexer
11
+ evolution to parser construction.
12
+
13
+ ## Decision
14
+
15
+ Generated lexer declarations normalize into immutable Lexer IR with its own
16
+ schema version. Grammar IR may embed that document, while Lexer IR can also be
17
+ validated and consumed independently.
18
+
19
+ Rules are ordered and state-scoped. At one input position the lexer chooses the
20
+ longest match, then the lowest rule id; empty matches are invalid. Lexical state
21
+ is explicit per parser instance. Streaming input retries a match when the
22
+ current chunk may be a strict prefix, so tokenization does not depend on chunk
23
+ boundaries.
24
+
25
+ Lexer actions execute only in the generated runtime boundary. Static tools may
26
+ inspect their opaque source but do not run it.
27
+
28
+ ## Consequences
29
+
30
+ - Lexer format can evolve without changing Automaton IR or handwritten token
31
+ sources.
32
+ - Matching is deterministic but intentionally uses a simple ordered rule model
33
+ rather than a hidden engine-specific DFA contract.
34
+ - Regular-expression safety remains an application responsibility; static
35
+ warnings are not a proof against pathological expressions.
@@ -0,0 +1,32 @@
1
+ # ADR 0015: Isolate browser analysis in a replaceable worker
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Ruby running through WebAssembly can consume substantial CPU and memory.
9
+ Running grammar analysis on the main browser thread would let a pathological
10
+ input freeze the interface, and interpolating grammar text into Ruby source
11
+ would create an unnecessary execution boundary.
12
+
13
+ ## Decision
14
+
15
+ The Ruby VM and grammar-analysis pipeline run in a dedicated module worker.
16
+ The main thread owns the timeout and replaces the worker on expiry. Input,
17
+ diagnostic, conflict, and search sizes are bounded. Grammar bytes cross the
18
+ JavaScript/Ruby boundary as data and semantic action bodies are never
19
+ evaluated.
20
+
21
+ The worker returns serialized analysis results only. Browser assets are
22
+ self-hosted under a restrictive Content Security Policy, and no submitted
23
+ grammar is sent to or stored by an application server.
24
+
25
+ ## Consequences
26
+
27
+ - Pathological analysis cannot retain Ruby state after worker replacement and
28
+ cannot block the main thread indefinitely.
29
+ - Worker termination is the isolation boundary, not a claim that WebAssembly
30
+ is a security sandbox.
31
+ - Browser support depends on Web Workers, WebAssembly, module scripts, and the
32
+ shipped Ruby VM asset.
@@ -0,0 +1,34 @@
1
+ # ADR 0016: Represent concrete syntax with immutable Green data and lazy Red views
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ A lossless syntax tree must support sharing, navigation, recovery, and editing.
9
+ Nodes containing absolute offsets, parents, or semantic action results cannot
10
+ be safely interned or reused at multiple source occurrences.
11
+
12
+ ## Decision
13
+
14
+ Green nodes, tokens, and trivia are immutable, position-independent values.
15
+ They contain integer kinds, source bytes, child structure, flags, and derived
16
+ widths, but no parent, absolute location, parser instance, or semantic value.
17
+ Tokens own trivia deterministically, including final trivia on EOF.
18
+
19
+ Red values are lazy occurrence views over Green values. They provide parents,
20
+ child indexes, and absolute offsets without changing Green identity.
21
+
22
+ CST-aware parsers maintain a Green stack parallel to the semantic stack.
23
+ Every physical reduction builds syntax independently of whether a semantic
24
+ action exists. Recovery and repair produce flagged syntax, and the public
25
+ syntax root is a synthetic `source_file` containing the selected start node
26
+ and EOF. Structured CST metadata in the parser-table ABI selects this path.
27
+
28
+ ## Consequences
29
+
30
+ - Equal Green subtrees can be interned, shared across Ractors, and reused at
31
+ multiple Red occurrences.
32
+ - Syntax shape does not depend on semantic action return values.
33
+ - Absolute navigation allocates Red wrappers lazily, while Green construction
34
+ must preserve exact source width and trivia ownership.
@@ -0,0 +1,35 @@
1
+ # ADR 0017: Persist and edit syntax through Green structure
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Mutable syntax nodes make snapshots and sharing difficult. Serializing Red
9
+ parents and offsets would duplicate derived occurrence state and tie the file
10
+ format to one navigation implementation.
11
+
12
+ ## Decision
13
+
14
+ Syntax edits replace one Green occurrence and path-copy only its ancestor
15
+ chain. Batch rewriting resolves replacements by Red occurrence; unchanged
16
+ subtrees retain Green identity. Annotations are occurrence-scoped identities,
17
+ exclude annotated Green values from interning, and are not serialized.
18
+
19
+ Concrete syntax uses an independently versioned closed schema. It records
20
+ grammar and parser compatibility metadata, kind metadata, trivia policy, and
21
+ one Green root. Source bytes use a canonical UTF-8-or-Base64 representation.
22
+ Loading validates the closed shape and compatibility, then recomputes widths,
23
+ descendant counts, and aggregate flags through Green constructors. Red parents,
24
+ offsets, cached wrappers, and annotations are deliberately absent.
25
+
26
+ Text diffs descend past identical Green values and emit byte-oriented edits
27
+ whose application must reconstruct the new source exactly.
28
+
29
+ ## Consequences
30
+
31
+ - Small edits allocate in proportion to the changed occurrence path.
32
+ - Serialized trees are independent of Red navigation caches and preserve
33
+ arbitrary source bytes.
34
+ - Structural editing may produce grammar-invalid trees; parsing validation is
35
+ a separate operation.
@@ -0,0 +1,38 @@
1
+ # ADR 0018: Reuse incremental syntax only with conservative proofs
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-07-28
5
+
6
+ ## Context
7
+
8
+ Parser production actions may depend on arbitrary mutable state. Replaying or
9
+ skipping them during incremental parsing can duplicate side effects or return a
10
+ stale semantic value. Reusing a syntax subtree is also unsafe unless lexer and
11
+ LR state match at its boundaries.
12
+
13
+ ## Decision
14
+
15
+ Incremental sessions are syntax-only: parser production actions are suppressed
16
+ and no semantic value is returned. Generated lexer actions still run because
17
+ they define token emission and lexical state. A fresh syntax parse remains the
18
+ reference result.
19
+
20
+ Relexing may reuse an old token suffix only after exact source-boundary, lexer
21
+ state, kind, text, flags, and trivia agreement. Green interning may preserve
22
+ equal token and node identity independently of parser reuse.
23
+
24
+ Subtree blending additionally requires the recorded left LR state to match the
25
+ live state, an unchanged following lookahead, a positive-width complete
26
+ nonterminal, and no error, missing, or skipped flags. Parse memo identity is by
27
+ occurrence position, not Green object identity. Every cache, decomposition, and
28
+ memo has a resource bound; uncertainty or exhaustion falls back to the already
29
+ available fresh path.
30
+
31
+ Incremental sessions and their caches are single-owner mutable objects.
32
+
33
+ ## Consequences
34
+
35
+ - Incremental parsing cannot silently reuse semantic side effects or values.
36
+ - Valid reuse is intentionally missed whenever proof metadata is insufficient.
37
+ - Correctness is defined by equality with a fresh syntax parse; performance is
38
+ evidence for optimization, not part of this decision.
@@ -0,0 +1,51 @@
1
+ # ADR 0019: Keep generated-language syntax sessions in the runtime
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-08-05
5
+
6
+ ## Context
7
+
8
+ The Red/Green CST and incremental parser already provide syntax-only parsing,
9
+ but application integrations otherwise have to assemble trust checks, edits,
10
+ failure expectations, fallback evidence, cancellation, and resource bounds.
11
+ This boundary is for one loaded generated language. It is not the existing
12
+ grammar-authoring LSP and has no workspace or protocol semantics.
13
+
14
+ Generated lexer actions and generated-file user sections are arbitrary Ruby.
15
+ A session-time flag cannot make an already loaded parser class safe.
16
+
17
+ ## Decision
18
+
19
+ Add an Experimental `SyntaxSession` façade to `ibex-runtime`, backed solely by
20
+ the existing `IncrementalParseSession`. Current parser classes expose
21
+ `:trusted_application_code` and require that exact profile as an explicit
22
+ caller acknowledgement. No `:declarative` mode is exposed until the generator
23
+ can produce a separate action-free syntax artifact with a declarative lexer and
24
+ without user-code sections.
25
+
26
+ Return immutable result and metric snapshots. Treat syntax errors as results,
27
+ but represent cancellation and hard resource exhaustion with distinct
28
+ exceptions. Preserve the previous completed session state when an edit aborts.
29
+
30
+ Keep the boundary in `ibex-runtime`: deployed generated languages need the
31
+ runtime but not grammar construction. Do not create `ibex-workbench` until a
32
+ larger editor-service boundary has independent demand. Do not add LSP,
33
+ workspace, indexing, query, or formatting behavior here.
34
+
35
+ ## Consequences
36
+
37
+ - Parser production actions remain suppressed; lexer actions and runtime hooks
38
+ remain explicit trusted application execution.
39
+ - Fresh syntax parsing remains the correctness oracle and the existing CST
40
+ implementation remains the only parser/tree engine.
41
+ - Parser-table format v6 and normal semantic parse behavior do not change. The
42
+ installed-runtime API is additive, and feature-off non-embedded generated
43
+ bytes remain unchanged.
44
+ - Embedded generated bytes do change because they copy the runtime sources.
45
+ Existing embedded artifacts continue to parse but must be regenerated and
46
+ redeployed to acquire `SyntaxSession` and its fixes. A non-embedded generated
47
+ parser acquires the additive API by upgrading its compatible runtime package.
48
+ - Cancellation is cooperative and cannot preempt a nonreturning Ruby lexer
49
+ action.
50
+ - The façade remains part of the existing Experimental incremental-CST product
51
+ surface rather than claiming a new Stable editor platform.
@@ -0,0 +1,65 @@
1
+ # ADR 0020: Persist parser contracts in Grammar IR
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-08-05
5
+
6
+ > The pre-release compatibility readers and in-process IR migration path were
7
+ > retired before v1.0. Racc-compatible grammar input remains supported.
8
+
9
+ ## Context
10
+
11
+ Parser construction algorithm, multiple-entry construction, and CST trivia
12
+ ownership are durable grammar-owned contracts. They must survive normalization,
13
+ IR-only generation, Automaton IR embedding, and static inspection before a
14
+ source declaration can be released. The current Grammar IR stores these
15
+ contracts at the root so a resumed pipeline sees the same explicit and
16
+ unspecified states as the source pipeline.
17
+
18
+ Three persistence boundaries were credible:
19
+
20
+ 1. add one root `parser_contract` to Grammar IR;
21
+ 2. publish a separately versioned Parser Contract IR; or
22
+ 3. wrap grammar and contract documents in a grammar-bundle envelope.
23
+
24
+ The choice crosses persisted schemas, grammar digests, resumed generation, and
25
+ the Automaton IR boundary. Reversing it requires coordinated changes to the
26
+ writer, reader, schemas, fixtures, and downstream consumers.
27
+
28
+ ## Decision
29
+
30
+ The current Grammar IR owns one closed, root-level `parser_contract`. Its admitted
31
+ fields are parser algorithm, entry construction, and CST trivia ownership.
32
+ Each field records an explicit value and source location, or an explicit
33
+ unspecified state. Unspecified contract state contains no built-in default;
34
+ defaults and invocation requests remain inputs to the typed configuration
35
+ resolver.
36
+
37
+ Automaton IR embeds the exact current Grammar IR document. Its grammar digest
38
+ therefore binds the parser contract. It also records whether entry construction
39
+ was shared or isolated; the existing algorithm field remains the selected
40
+ automaton fact.
41
+
42
+ There is one reader and one writer for the current format. `Grammar.new` and
43
+ `Automaton.new` both construct the current format directly.
44
+
45
+ A separate Parser Contract IR is rejected because it would create two
46
+ authorities that must be joined atomically and included independently in every
47
+ digest. A bundle envelope is rejected because Grammar IR already contains the
48
+ normalized root and resolved fragment result; an additional container would
49
+ add identity rules without an independently lifecycled artifact.
50
+
51
+ The contract remains generator-side data. It does not change parser-table
52
+ format, generated runtime constants, or the `ibex-runtime` ABI. Static readers
53
+ validate it without evaluating grammar actions or application code.
54
+
55
+ ## Consequences
56
+
57
+ - Source and IR-only pipelines share one normalized contract and one grammar
58
+ digest.
59
+ - Unspecified source contract remains distinct from a newly selected built-in
60
+ default.
61
+ - Fragments cannot acquire parser-wide authority; only the normalized root has
62
+ a parser contract.
63
+ - Grammar and Automaton IR each have one closed current schema.
64
+ - Revisit the rejected sidecar only if parser contracts gain an independent
65
+ lifecycle or a non-grammar authority with multiple real consumers.
@@ -0,0 +1,50 @@
1
+ # ADR 0021: Persist executable parser tables as a data-only sidecar
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-08-05
5
+
6
+ ## Context
7
+
8
+ Automaton IR is suitable for construction analysis, but a consumer must still
9
+ build runtime tables from it. Generated Ruby already contains executable
10
+ tables, but inspecting that file safely would require parsing Ruby and loading
11
+ it may execute application code. Keeping only Automaton IR provides no
12
+ separate object that a publication report can identify as the executed table.
13
+
14
+ The alternatives were to keep rebuilding tables from Automaton IR, treat
15
+ generated Ruby as the table container, or persist the table projection as a
16
+ closed data format. The last option crosses persistence, execution, packaging,
17
+ and trust boundaries and therefore needs a durable decision.
18
+
19
+ ## Decision
20
+
21
+ Adopt an Experimental, separately versioned JSON sidecar as the data-only
22
+ authority for parser recognition tables. It records canonical internal token
23
+ IDs, actions, GOTOs, defaults, production stack effects, entries, static CST
24
+ and recovery metadata, and explicit source/table/metadata identities.
25
+
26
+ Semantic action bodies, generated wrappers, lexers, external token conversion,
27
+ and application hooks remain outside the artifact. Productions identify only
28
+ opaque wrapper action slots. The prototype executor recognizes internal token
29
+ ID sequences and never reconstructs Automaton IR or evaluates an action.
30
+
31
+ Artifact schema versioning is independent of the generated parser Stable ABI.
32
+ Version 1 names parser-table format v6 only to pin action and production
33
+ semantics; the current runtime does not accept the JSON representation. The
34
+ sidecar remains outside default generation and runtime packaging until another
35
+ decision defines compatibility and publication binding.
36
+
37
+ ## Consequences
38
+
39
+ - Consumers can validate and exercise table data without loading generated
40
+ application code, and later reports have a non-Ruby table identity to bind.
41
+ - The sidecar has real execution value beyond Automaton IR, while retaining the
42
+ wrapper as authority for semantic actions and tokenization.
43
+ - Source digests in a standalone artifact are linkage claims, not proofs or
44
+ signatures; verification against source IR requires that IR as another
45
+ input.
46
+ - The prototype duplicates some Automaton IR projection data and introduces a
47
+ schema/loader compatibility obligation.
48
+ - Runtime consumption, semantic action ABI binding, default emission, or a
49
+ stability promotion must revisit this decision and define migration without
50
+ silently changing existing generated bytes.
@@ -0,0 +1,77 @@
1
+ # ADR 0022: Bind scoped verification evidence through the generation manifest
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-08-05
5
+
6
+ ## Context
7
+
8
+ The independent Automaton IR verifier and the data-only parser table sidecar
9
+ have useful but separate identities. A published parser also includes a Ruby
10
+ wrapper containing opaque actions. Consumers need to detect a table, report,
11
+ wrapper, or input taken from another generation without loading the wrapper or
12
+ overstating what the verifier checks.
13
+
14
+ A report could contain every artifact digest including its manifest, be
15
+ embedded in generated Ruby, use physical paths as artifact identities, or be
16
+ published as a separate manifest-bound artifact. Including the manifest
17
+ digest creates a hash cycle because the manifest must hash the report.
18
+ Embedding evidence in Ruby makes nonexecuting inspection depend on a source
19
+ container that also carries opaque code. Physical paths make equivalent
20
+ evidence differ between checkouts.
21
+
22
+ ## Decision
23
+
24
+ Adopt an Experimental, separately versioned scoped verification report and an
25
+ explicit verifiable-generation-bundle API. The report records logical input
26
+ identities, Grammar and Automaton IR digests, the data-only table artifact and
27
+ payload digests, checker version, verification profile, completed checks,
28
+ bounds, outcome, and a closed excluded-trust list. That machine-readable
29
+ exclusion includes semantic derivation or language equivalence from Automaton
30
+ IR to persisted table data; version 1 validates table structure and digest
31
+ linkage but does not independently prove that projection.
32
+
33
+ The report never contains the manifest digest or path. Generation renders the
34
+ table, wrapper, and report in memory, then renders a generation manifest that
35
+ lists and hashes all three. The existing transaction publishes non-marker
36
+ artifacts and the wrapper before installing that manifest as the final
37
+ coherence marker.
38
+
39
+ Canonical report evidence uses ordered logical identities rather than
40
+ absolute paths: indexed input basenames and a role-qualified table basename.
41
+ Version 1 closes both forms to one basename, exactly four input-index digits,
42
+ and at most 10,000 inputs.
43
+
44
+ IR claims use the explicitly named `source-logical-v1` identity scope. Bundle
45
+ construction rebuilds an immutable Automaton IR copy from the supplied IR and
46
+ `source_records`: every location file becomes its ordered logical input
47
+ identity and each non-null source root becomes `input`. The table and verifier
48
+ consume that same copy, so its Grammar IR digest, embedded Automaton IR
49
+ grammar digest, Automaton IR digest, table identity, report claims, and report
50
+ evidence digest cannot disagree. An unmapped or ambiguous source location is
51
+ an error. The original Automaton, its diagnostic provenance, and ordinary
52
+ V002 table construction remain unchanged.
53
+
54
+ The bundle validator checks report self-identity, manifest artifact bytes,
55
+ input identity, table identity and payload, and IR/table linkage without
56
+ parsing or loading generated Ruby.
57
+
58
+ The API stays outside the default `ibex` require path and generator CLI. An
59
+ application must explicitly require and invoke it. This preserves existing
60
+ generation bytes and leaves CLI/stability promotion to a later compatibility
61
+ decision.
62
+
63
+ ## Consequences
64
+
65
+ - A manifest is the single non-cyclic publication marker for table, wrapper,
66
+ and report coherence.
67
+ - Equivalent source records produce identical table and report evidence
68
+ across checkout roots when their ordered logical basenames and bytes match.
69
+ - Missing, stale, duplicated, or cross-generation table/report/manifest
70
+ combinations are rejected before generated Ruby is loaded.
71
+ - A passing report remains bounded evidence for named V1-V8 checks, not proof
72
+ of table semantic derivation, wrapper, action, lexer, runtime, application,
73
+ source-to-IR, authenticity, or grammar-unambiguity correctness.
74
+ - The new schema and validator are persistence and trusted-computing-base
75
+ obligations. Future schema versions may extend evidence only without
76
+ silently widening version 1 claims.
77
+ - Default CLI generation and the Stable generated-parser ABI do not change.