ibex 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (603) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +89 -123
  3. data/docs/architecture.md +477 -31
  4. data/docs/bison-import.md +103 -0
  5. data/docs/comparison-policy.md +127 -0
  6. data/docs/configuration-model.md +59 -0
  7. data/docs/configuration-report.md +28 -0
  8. data/docs/conflict-explanation-reviews/v1/records/README.md +28 -0
  9. data/docs/conflict-explanation-study.md +62 -0
  10. data/docs/construction-profiling.md +157 -0
  11. data/docs/cst-migration.md +106 -0
  12. data/docs/cst.md +237 -0
  13. data/docs/decisions/0000-template.md +24 -0
  14. data/docs/decisions/0001-separate-ir-pipeline.md +34 -0
  15. data/docs/decisions/0002-opaque-user-code-boundary.md +32 -0
  16. data/docs/decisions/0003-self-hosted-grammar-frontend.md +32 -0
  17. data/docs/decisions/0004-shared-semantic-and-lossless-source-model.md +33 -0
  18. data/docs/decisions/0005-contained-grammar-composition.md +34 -0
  19. data/docs/decisions/0006-bounded-structural-grammar-lowering.md +32 -0
  20. data/docs/decisions/0007-shared-parser-construction-pipeline.md +33 -0
  21. data/docs/decisions/0008-versioned-runtime-package-boundary.md +36 -0
  22. data/docs/decisions/0009-isolated-parser-sessions.md +34 -0
  23. data/docs/decisions/0010-committed-runtime-observation.md +34 -0
  24. data/docs/decisions/0011-versioned-semantic-action-boundary.md +37 -0
  25. data/docs/decisions/0012-bounded-nonexecuting-analysis.md +37 -0
  26. data/docs/decisions/0013-transactional-generation-publication.md +37 -0
  27. data/docs/decisions/0014-versioned-generated-lexer.md +35 -0
  28. data/docs/decisions/0015-worker-isolated-browser-analysis.md +32 -0
  29. data/docs/decisions/0016-red-green-concrete-syntax.md +34 -0
  30. data/docs/decisions/0017-persistent-syntax-artifacts.md +35 -0
  31. data/docs/decisions/0018-conservative-incremental-syntax-reuse.md +38 -0
  32. data/docs/decisions/0019-runtime-syntax-session-boundary.md +51 -0
  33. data/docs/decisions/0020-grammar-ir-parser-contract.md +65 -0
  34. data/docs/decisions/0021-data-only-parser-table-sidecar.md +50 -0
  35. data/docs/decisions/0022-manifest-bound-verification-report.md +77 -0
  36. data/docs/decisions/0023-syntax-only-repair-results.md +48 -0
  37. data/docs/decisions/0024-direct-ielr-construction.md +42 -0
  38. data/docs/decisions/README.md +74 -0
  39. data/docs/declarative-configuration.md +262 -0
  40. data/docs/development.md +306 -0
  41. data/docs/direct-ielr-decision.md +120 -0
  42. data/docs/direct-multi-entry-decision.md +82 -0
  43. data/docs/editor-setup.md +26 -0
  44. data/docs/error-ux-review-rubric-v1.md +117 -0
  45. data/docs/error-ux-reviews/v1/records/README.md +22 -0
  46. data/docs/error-ux-round2-review-status-v1.json +27 -0
  47. data/docs/error-ux-round2-reviews/v1/records/README.md +30 -0
  48. data/docs/error-ux-round2-v1.json +1581 -0
  49. data/docs/error-ux-round2.md +111 -0
  50. data/docs/error-ux.md +114 -0
  51. data/docs/getting-started.md +80 -0
  52. data/docs/grammar-reference.md +629 -18
  53. data/docs/ielr-design.md +43 -0
  54. data/docs/ielr.md +92 -0
  55. data/docs/ja/bison-import.md +51 -0
  56. data/docs/ja/stable-api.md +53 -0
  57. data/docs/lexer-construction-profile.md +109 -0
  58. data/docs/lexer-migration.md +56 -0
  59. data/docs/maturity.md +128 -0
  60. data/docs/project-site-strategy.md +54 -0
  61. data/docs/racc-migration-evidence.md +45 -0
  62. data/docs/racc-migration.md +45 -6
  63. data/docs/release-readiness.md +221 -0
  64. data/docs/repair-semantics.md +123 -0
  65. data/docs/runtime-abi-evolution.md +307 -0
  66. data/docs/stability.md +234 -0
  67. data/docs/status.md +43 -0
  68. data/docs/syntax-sessions.md +183 -0
  69. data/docs/table-artifact.md +125 -0
  70. data/docs/test-interactions.md +219 -0
  71. data/docs/verification-report.md +137 -0
  72. data/docs/verifier-trust-boundary.md +199 -0
  73. data/docs/workloads.md +132 -0
  74. data/examples/README.md +60 -0
  75. data/examples/calculator.y +44 -0
  76. data/examples/csv.y +32 -0
  77. data/examples/ini.y +70 -0
  78. data/examples/json.y +48 -0
  79. data/examples/tiny_language.y +75 -0
  80. data/lib/ibex/analysis/digraph.rb +128 -0
  81. data/lib/ibex/analysis/sets.rb +5 -4
  82. data/lib/ibex/analysis.rb +1 -0
  83. data/lib/ibex/artifact_set.rb +63 -0
  84. data/lib/ibex/bison_import/importer.rb +516 -0
  85. data/lib/ibex/bison_import/tokenizer.rb +260 -0
  86. data/lib/ibex/bison_import.rb +200 -0
  87. data/lib/ibex/bounded_subprocess.rb +171 -0
  88. data/lib/ibex/cli/ambiguity.rb +75 -0
  89. data/lib/ibex/cli/analysis.rb +134 -0
  90. data/lib/ibex/cli/bison_import.rb +121 -0
  91. data/lib/ibex/cli/config.rb +128 -0
  92. data/lib/ibex/cli/counterexample_options.rb +6 -2
  93. data/lib/ibex/cli/coverage.rb +173 -0
  94. data/lib/ibex/cli/debug.rb +106 -0
  95. data/lib/ibex/cli/diagnostics.rb +136 -0
  96. data/lib/ibex/cli/documentation.rb +70 -0
  97. data/lib/ibex/cli/equiv.rb +169 -0
  98. data/lib/ibex/cli/error_messages.rb +184 -0
  99. data/lib/ibex/cli/explain.rb +85 -0
  100. data/lib/ibex/cli/fix.rb +203 -0
  101. data/lib/ibex/cli/formatting.rb +387 -0
  102. data/lib/ibex/cli/fuzz.rb +200 -0
  103. data/lib/ibex/cli/fuzz_regressions.rb +145 -0
  104. data/lib/ibex/cli/generation_artifacts.rb +189 -0
  105. data/lib/ibex/cli/generation_error_messages.rb +39 -0
  106. data/lib/ibex/cli/grammar_tests.rb +140 -0
  107. data/lib/ibex/cli/ir_tools.rb +128 -0
  108. data/lib/ibex/cli/lsp.rb +33 -0
  109. data/lib/ibex/cli/outputs.rb +133 -23
  110. data/lib/ibex/cli/racc_migration.rb +87 -0
  111. data/lib/ibex/cli/reduce.rb +202 -0
  112. data/lib/ibex/cli/reduce_reporting.rb +71 -0
  113. data/lib/ibex/cli/samples.rb +118 -0
  114. data/lib/ibex/cli/verify.rb +116 -0
  115. data/lib/ibex/cli/watch.rb +104 -0
  116. data/lib/ibex/cli.rb +825 -55
  117. data/lib/ibex/codegen/action_locations.rb +103 -0
  118. data/lib/ibex/codegen/action_method_source.rb +209 -0
  119. data/lib/ibex/codegen/action_source.rb +119 -0
  120. data/lib/ibex/codegen/ambiguity.rb +174 -0
  121. data/lib/ibex/codegen/cst_metadata.rb +171 -0
  122. data/lib/ibex/codegen/documentation.rb +170 -0
  123. data/lib/ibex/codegen/explain.rb +380 -0
  124. data/lib/ibex/codegen/generated_action_abi.rb +293 -0
  125. data/lib/ibex/codegen/html.rb +94 -10
  126. data/lib/ibex/codegen/mermaid.rb +43 -0
  127. data/lib/ibex/codegen/railroad.rb +197 -0
  128. data/lib/ibex/codegen/railroad_documentation.rb +59 -0
  129. data/lib/ibex/codegen/rbs.rb +331 -2
  130. data/lib/ibex/codegen/report.rb +67 -15
  131. data/lib/ibex/codegen/ruby.rb +305 -66
  132. data/lib/ibex/codegen/ruby_actions.rb +146 -0
  133. data/lib/ibex/codegen/ruby_ast.rb +121 -0
  134. data/lib/ibex/codegen/ruby_error_messages.rb +28 -0
  135. data/lib/ibex/codegen/ruby_lexer.rb +83 -0
  136. data/lib/ibex/codegen/ruby_syntax.rb +96 -0
  137. data/lib/ibex/codegen/ruby_table_metadata.rb +57 -0
  138. data/lib/ibex/codegen/ruby_value_printers.rb +56 -0
  139. data/lib/ibex/codegen/symbol_labels.rb +1 -1
  140. data/lib/ibex/configuration/analysis_grammar.rb +35 -0
  141. data/lib/ibex/configuration/explanation.rb +421 -0
  142. data/lib/ibex/configuration/inspector.rb +232 -0
  143. data/lib/ibex/configuration.rb +580 -0
  144. data/lib/ibex/coverage/collector.rb +198 -0
  145. data/lib/ibex/coverage/event_stream.rb +103 -0
  146. data/lib/ibex/coverage/report.rb +267 -0
  147. data/lib/ibex/coverage/runtime_event_validator.rb +169 -0
  148. data/lib/ibex/coverage.rb +13 -0
  149. data/lib/ibex/delta_reducer.rb +99 -0
  150. data/lib/ibex/diff.rb +140 -0
  151. data/lib/ibex/equiv/machine.rb +135 -0
  152. data/lib/ibex/equiv.rb +373 -0
  153. data/lib/ibex/error_messages/parser.rb +159 -0
  154. data/lib/ibex/error_messages/parser_v2.rb +198 -0
  155. data/lib/ibex/error_messages/renderer.rb +65 -0
  156. data/lib/ibex/error_messages/sentence_search.rb +196 -0
  157. data/lib/ibex/error_messages/update.rb +169 -0
  158. data/lib/ibex/error_messages.rb +165 -0
  159. data/lib/ibex/fix.rb +557 -0
  160. data/lib/ibex/frontend/ast.rb +166 -7
  161. data/lib/ibex/frontend/bootstrap_parser.rb +40 -4
  162. data/lib/ibex/frontend/diagnostic.rb +81 -0
  163. data/lib/ibex/frontend/diagnostic_recovery.rb +268 -0
  164. data/lib/ibex/frontend/dsl.rb +66 -7
  165. data/lib/ibex/frontend/formatter.rb +453 -0
  166. data/lib/ibex/frontend/generated_parser.rb +644 -190
  167. data/lib/ibex/frontend/generated_parser_base.rb +170 -32
  168. data/lib/ibex/frontend/generated_parser_includes.rb +51 -0
  169. data/lib/ibex/frontend/generated_parser_metadata.rb +61 -0
  170. data/lib/ibex/frontend/generated_parser_parameters.rb +60 -0
  171. data/lib/ibex/frontend/generation.rb +33 -0
  172. data/lib/ibex/frontend/lexer.rb +162 -10
  173. data/lib/ibex/frontend/lexer_recovery.rb +84 -0
  174. data/lib/ibex/frontend/parser/declarations.rb +262 -11
  175. data/lib/ibex/frontend/parser/parameters.rb +82 -0
  176. data/lib/ibex/frontend/parser/rules.rb +43 -6
  177. data/lib/ibex/frontend/parser.rb +182 -4
  178. data/lib/ibex/frontend/parser_configuration_support.rb +56 -0
  179. data/lib/ibex/frontend/regenerator.rb +27 -1
  180. data/lib/ibex/frontend/resolution.rb +70 -0
  181. data/lib/ibex/frontend/resolver.rb +217 -0
  182. data/lib/ibex/frontend/rule_documentation.rb +104 -0
  183. data/lib/ibex/frontend/source_cursor.rb +133 -9
  184. data/lib/ibex/frontend/source_document.rb +229 -0
  185. data/lib/ibex/frontend/source_loader.rb +150 -0
  186. data/lib/ibex/frontend/source_span.rb +89 -0
  187. data/lib/ibex/frontend/token_adapter/declaration_document_state.rb +47 -0
  188. data/lib/ibex/frontend/token_adapter/declaration_lexer_state.rb +83 -0
  189. data/lib/ibex/frontend/token_adapter/declaration_state.rb +244 -22
  190. data/lib/ibex/frontend/token_adapter/delimiter_tracker.rb +8 -2
  191. data/lib/ibex/frontend/token_adapter/rule_state.rb +60 -2
  192. data/lib/ibex/frontend/token_adapter.rb +18 -3
  193. data/lib/ibex/frontend.rb +14 -2
  194. data/lib/ibex/fuzz.rb +219 -0
  195. data/lib/ibex/generation_input.rb +57 -0
  196. data/lib/ibex/generation_manifest.rb +211 -0
  197. data/lib/ibex/generation_transaction.rb +261 -0
  198. data/lib/ibex/generation_transaction_recovery.rb +109 -0
  199. data/lib/ibex/generation_transaction_validation.rb +196 -0
  200. data/lib/ibex/grammar_tests.rb +206 -0
  201. data/lib/ibex/ir/automaton_ir.rb +86 -11
  202. data/lib/ibex/ir/grammar_ir.rb +170 -26
  203. data/lib/ibex/ir/lexer_ir.rb +76 -0
  204. data/lib/ibex/ir/parser_contract.rb +135 -0
  205. data/lib/ibex/ir/serialize.rb +268 -50
  206. data/lib/ibex/ir/validator/automaton.rb +375 -0
  207. data/lib/ibex/ir/validator/base.rb +147 -0
  208. data/lib/ibex/ir/validator/grammar.rb +634 -0
  209. data/lib/ibex/ir/validator/lexer.rb +113 -0
  210. data/lib/ibex/ir/validator.rb +63 -0
  211. data/lib/ibex/ir.rb +71 -5
  212. data/lib/ibex/lalr/build_metrics.rb +71 -0
  213. data/lib/ibex/lalr/builder.rb +514 -52
  214. data/lib/ibex/lalr/conflict.rb +1 -0
  215. data/lib/ibex/lalr/conflict_search.rb +26 -9
  216. data/lib/ibex/lalr/counterexample.rb +38 -18
  217. data/lib/ibex/lalr/direct_lookaheads.rb +289 -0
  218. data/lib/ibex/lalr/goto_follows.rb +229 -0
  219. data/lib/ibex/lalr/ielr/annotator.rb +214 -0
  220. data/lib/ibex/lalr/ielr/bits.rb +28 -0
  221. data/lib/ibex/lalr/ielr/inadequacy.rb +47 -0
  222. data/lib/ibex/lalr/ielr/item_lookaheads.rb +78 -0
  223. data/lib/ibex/lalr/ielr/pipeline.rb +75 -0
  224. data/lib/ibex/lalr/ielr/split_stability.rb +78 -0
  225. data/lib/ibex/lalr/ielr/split_state.rb +20 -0
  226. data/lib/ibex/lalr/ielr/state_splitter.rb +258 -0
  227. data/lib/ibex/lalr/ielr_partition.rb +166 -0
  228. data/lib/ibex/lalr/inadequacy_report.rb +50 -0
  229. data/lib/ibex/lalr/lookahead_propagation.rb +111 -0
  230. data/lib/ibex/lalr/lr0_collection.rb +121 -0
  231. data/lib/ibex/lalr/on_error_reductions.rb +74 -0
  232. data/lib/ibex/lalr/unreachable_states.rb +80 -0
  233. data/lib/ibex/lalr.rb +59 -1
  234. data/lib/ibex/location.rb +129 -0
  235. data/lib/ibex/lsp/document_handlers.rb +68 -0
  236. data/lib/ibex/lsp/document_store.rb +264 -0
  237. data/lib/ibex/lsp/document_store_diagnostics.rb +44 -0
  238. data/lib/ibex/lsp/document_store_validation.rb +61 -0
  239. data/lib/ibex/lsp/initialization_handlers.rb +80 -0
  240. data/lib/ibex/lsp/navigation_handlers.rb +70 -0
  241. data/lib/ibex/lsp/parser_configuration_assistance.rb +149 -0
  242. data/lib/ibex/lsp/position_codec.rb +113 -0
  243. data/lib/ibex/lsp/protocol_error.rb +33 -0
  244. data/lib/ibex/lsp/request_handlers.rb +46 -0
  245. data/lib/ibex/lsp/request_support.rb +94 -0
  246. data/lib/ibex/lsp/server.rb +148 -0
  247. data/lib/ibex/lsp/symbol_index.rb +241 -0
  248. data/lib/ibex/lsp/symbol_index_builder.rb +277 -0
  249. data/lib/ibex/lsp/symbol_index_precedence_references.rb +44 -0
  250. data/lib/ibex/lsp/symbol_index_source_queries.rb +61 -0
  251. data/lib/ibex/lsp/symbol_occurrence.rb +23 -0
  252. data/lib/ibex/lsp/transport.rb +128 -0
  253. data/lib/ibex/lsp/workspace.rb +128 -0
  254. data/lib/ibex/lsp/workspace_analyzer.rb +199 -0
  255. data/lib/ibex/lsp.rb +33 -0
  256. data/lib/ibex/messages/en.yml +28 -0
  257. data/lib/ibex/messages/ja.yml +28 -0
  258. data/lib/ibex/messages.rb +68 -0
  259. data/lib/ibex/metrics.rb +175 -0
  260. data/lib/ibex/normalize/declarations.rb +142 -7
  261. data/lib/ibex/normalize/diagnostics.rb +94 -24
  262. data/lib/ibex/normalize/expander.rb +64 -59
  263. data/lib/ibex/normalize/expression.rb +63 -40
  264. data/lib/ibex/normalize/grammar_builder.rb +26 -0
  265. data/lib/ibex/normalize/inline_expansion.rb +465 -0
  266. data/lib/ibex/normalize/inline_validation.rb +174 -0
  267. data/lib/ibex/normalize/lexer.rb +131 -0
  268. data/lib/ibex/normalize/named_references.rb +60 -0
  269. data/lib/ibex/normalize/nodes.rb +46 -0
  270. data/lib/ibex/normalize/parameter_ebnf_lowering.rb +70 -0
  271. data/lib/ibex/normalize/parameter_lowering.rb +143 -0
  272. data/lib/ibex/normalize/parameter_substitution.rb +125 -0
  273. data/lib/ibex/normalize/parameter_validation.rb +140 -0
  274. data/lib/ibex/normalize/parameters.rb +226 -0
  275. data/lib/ibex/normalize/parser_configuration.rb +70 -0
  276. data/lib/ibex/normalize/recovery_declarations.rb +84 -0
  277. data/lib/ibex/normalize.rb +172 -17
  278. data/lib/ibex/racc_migration/checker.rb +122 -0
  279. data/lib/ibex/racc_migration/harness.rb +177 -0
  280. data/lib/ibex/racc_migration/report.rb +97 -0
  281. data/lib/ibex/racc_migration.rb +12 -0
  282. data/lib/ibex/rake_task.rb +116 -0
  283. data/lib/ibex/samples.rb +216 -0
  284. data/lib/ibex/table_artifact/builder.rb +266 -0
  285. data/lib/ibex/table_artifact/cst_projection.rb +70 -0
  286. data/lib/ibex/table_artifact/document.rb +39 -0
  287. data/lib/ibex/table_artifact/executor.rb +187 -0
  288. data/lib/ibex/table_artifact/serializer.rb +57 -0
  289. data/lib/ibex/table_artifact/validator/metadata.rb +179 -0
  290. data/lib/ibex/table_artifact/validator/support.rb +86 -0
  291. data/lib/ibex/table_artifact/validator/tables.rb +238 -0
  292. data/lib/ibex/table_artifact/validator.rb +253 -0
  293. data/lib/ibex/table_artifact.rb +88 -0
  294. data/lib/ibex/table_simulation/result.rb +55 -0
  295. data/lib/ibex/table_simulation/simulator.rb +253 -0
  296. data/lib/ibex/table_simulation/step.rb +62 -0
  297. data/lib/ibex/table_simulation/text.rb +31 -0
  298. data/lib/ibex/table_simulation.rb +13 -0
  299. data/lib/ibex/tables.rb +22 -69
  300. data/lib/ibex/verifiable_generation_bundle.rb +86 -0
  301. data/lib/ibex/verification_report/builder.rb +159 -0
  302. data/lib/ibex/verification_report/canonical_ir.rb +123 -0
  303. data/lib/ibex/verification_report/logical_path.rb +68 -0
  304. data/lib/ibex/verification_report/validator.rb +329 -0
  305. data/lib/ibex/verification_report.rb +69 -0
  306. data/lib/ibex/verify/action_correspondence.rb +138 -0
  307. data/lib/ibex/verify/language_witness.rb +295 -0
  308. data/lib/ibex/verify/reference_collection.rb +190 -0
  309. data/lib/ibex/verify/result.rb +73 -0
  310. data/lib/ibex/verify/verifier.rb +598 -0
  311. data/lib/ibex/verify.rb +15 -0
  312. data/lib/ibex/version.rb +1 -1
  313. data/lib/ibex/watch/runner.rb +181 -0
  314. data/lib/ibex/watch/source_snapshot.rb +98 -0
  315. data/lib/ibex/watch.rb +11 -0
  316. data/lib/ibex.rb +35 -1
  317. data/schema/automaton-ir-definitions.schema.json +401 -0
  318. data/schema/automaton-ir.schema.json +65 -0
  319. data/schema/benchmark-v1.schema.json +212 -0
  320. data/schema/benchmark-v2.schema.json +61 -0
  321. data/schema/bison-import-v1.schema.json +106 -0
  322. data/schema/conflict-explanation-review-v1.schema.json +196 -0
  323. data/schema/conflict-explanation-study-v1.schema.json +528 -0
  324. data/schema/construction-profile-v1.schema.json +779 -0
  325. data/schema/cst-v1.json +170 -0
  326. data/schema/diff-v1.schema.json +41 -0
  327. data/schema/direct-ielr-decision-v1.schema.json +200 -0
  328. data/schema/equiv-v1.schema.json +64 -0
  329. data/schema/error-ux-review-v1.schema.json +609 -0
  330. data/schema/error-ux-round2-review-v1.schema.json +191 -0
  331. data/schema/error-ux-round2-v1.schema.json +519 -0
  332. data/schema/error-ux-v1.schema.json +258 -0
  333. data/schema/explain-v1.schema.json +495 -0
  334. data/schema/fix-v1.schema.json +64 -0
  335. data/schema/fix-v2.schema.json +66 -0
  336. data/schema/fix-v3.schema.json +101 -0
  337. data/schema/frontend-diagnostics-v1.schema.json +154 -0
  338. data/schema/fuzz-regression-v1.schema.json +79 -0
  339. data/schema/fuzz-v1.schema.json +114 -0
  340. data/schema/generation-manifest-v1.schema.json +115 -0
  341. data/schema/grammar-ir-extensions.schema.json +734 -0
  342. data/schema/grammar-ir-foundation.schema.json +426 -0
  343. data/schema/grammar-ir.schema.json +271 -0
  344. data/schema/ielr-benchmark-v1.schema.json +85 -0
  345. data/schema/lexer-ir-v1.schema.json +215 -0
  346. data/schema/lexer-profile-v1.schema.json +491 -0
  347. data/schema/metrics-v1.schema.json +60 -0
  348. data/schema/migration-check-v1.schema.json +60 -0
  349. data/schema/performance-comparison-v1.schema.json +395 -0
  350. data/schema/public-performance-comparison-v1.schema.json +506 -0
  351. data/schema/public-performance-profile-v1.schema.json +360 -0
  352. data/schema/reduce-v1.schema.json +22 -0
  353. data/schema/reduce-v2.schema.json +63 -0
  354. data/schema/runtime-coverage-v1.schema.json +86 -0
  355. data/schema/runtime-event-v1.schema.json +308 -0
  356. data/schema/table-artifact-v1.schema.json +377 -0
  357. data/schema/table-simulation-v1.schema.json +70 -0
  358. data/schema/verification-report-v1.schema.json +222 -0
  359. data/schema/verify-v1.schema.json +42 -0
  360. data/sig/ibex/analysis/digraph.rbs +19 -0
  361. data/sig/ibex/artifact_set.rbs +37 -0
  362. data/sig/ibex/bison_import/importer.rbs +140 -0
  363. data/sig/ibex/bison_import/tokenizer.rbs +91 -0
  364. data/sig/ibex/bison_import.rbs +95 -0
  365. data/sig/ibex/bounded_subprocess.rbs +76 -0
  366. data/sig/ibex/cli/ambiguity.rbs +28 -0
  367. data/sig/ibex/cli/analysis.rbs +43 -0
  368. data/sig/ibex/cli/bison_import.rbs +32 -0
  369. data/sig/ibex/cli/config.rbs +35 -0
  370. data/sig/ibex/cli/counterexample_options.rbs +2 -0
  371. data/sig/ibex/cli/coverage.rbs +53 -0
  372. data/sig/ibex/cli/debug.rbs +28 -0
  373. data/sig/ibex/cli/diagnostics.rbs +51 -0
  374. data/sig/ibex/cli/documentation.rbs +29 -0
  375. data/sig/ibex/cli/equiv.rbs +49 -0
  376. data/sig/ibex/cli/error_messages.rbs +63 -0
  377. data/sig/ibex/cli/explain.rbs +31 -0
  378. data/sig/ibex/cli/fix.rbs +53 -0
  379. data/sig/ibex/cli/formatting.rbs +107 -0
  380. data/sig/ibex/cli/fuzz.rbs +47 -0
  381. data/sig/ibex/cli/fuzz_regressions.rbs +40 -0
  382. data/sig/ibex/cli/generation_artifacts.rbs +69 -0
  383. data/sig/ibex/cli/generation_error_messages.rbs +19 -0
  384. data/sig/ibex/cli/grammar_tests.rbs +48 -0
  385. data/sig/ibex/cli/ir_tools.rbs +39 -0
  386. data/sig/ibex/cli/lsp.rbs +14 -0
  387. data/sig/ibex/cli/outputs.rbs +26 -3
  388. data/sig/ibex/cli/racc_migration.rbs +30 -0
  389. data/sig/ibex/cli/reduce.rbs +60 -0
  390. data/sig/ibex/cli/reduce_reporting.rbs +27 -0
  391. data/sig/ibex/cli/samples.rbs +38 -0
  392. data/sig/ibex/cli/verify.rbs +28 -0
  393. data/sig/ibex/cli/watch.rbs +43 -0
  394. data/sig/ibex/cli.rbs +182 -9
  395. data/sig/ibex/codegen/action_locations.rbs +45 -0
  396. data/sig/ibex/codegen/action_method_source.rbs +65 -0
  397. data/sig/ibex/codegen/action_source.rbs +50 -0
  398. data/sig/ibex/codegen/ambiguity.rbs +60 -0
  399. data/sig/ibex/codegen/cst_metadata.rbs +59 -0
  400. data/sig/ibex/codegen/documentation.rbs +50 -0
  401. data/sig/ibex/codegen/explain.rbs +105 -0
  402. data/sig/ibex/codegen/generated_action_abi.rbs +105 -0
  403. data/sig/ibex/codegen/html.rbs +18 -2
  404. data/sig/ibex/codegen/mermaid.rbs +16 -0
  405. data/sig/ibex/codegen/railroad.rbs +82 -0
  406. data/sig/ibex/codegen/railroad_documentation.rbs +31 -0
  407. data/sig/ibex/codegen/rbs.rbs +86 -4
  408. data/sig/ibex/codegen/report.rbs +16 -8
  409. data/sig/ibex/codegen/ruby.rbs +105 -28
  410. data/sig/ibex/codegen/ruby_actions.rbs +54 -0
  411. data/sig/ibex/codegen/ruby_ast.rbs +34 -0
  412. data/sig/ibex/codegen/ruby_error_messages.rbs +16 -0
  413. data/sig/ibex/codegen/ruby_lexer.rbs +25 -0
  414. data/sig/ibex/codegen/ruby_syntax.rbs +27 -0
  415. data/sig/ibex/codegen/ruby_table_metadata.rbs +26 -0
  416. data/sig/ibex/codegen/ruby_value_printers.rbs +28 -0
  417. data/sig/ibex/configuration/analysis_grammar.rbs +13 -0
  418. data/sig/ibex/configuration/explanation.rbs +195 -0
  419. data/sig/ibex/configuration/inspector.rbs +53 -0
  420. data/sig/ibex/configuration.rbs +242 -0
  421. data/sig/ibex/coverage/collector.rbs +78 -0
  422. data/sig/ibex/coverage/event_stream.rbs +44 -0
  423. data/sig/ibex/coverage/report.rbs +102 -0
  424. data/sig/ibex/coverage/runtime_event_validator.rbs +70 -0
  425. data/sig/ibex/coverage.rbs +7 -0
  426. data/sig/ibex/delta_reducer.rbs +38 -0
  427. data/sig/ibex/diff.rbs +47 -0
  428. data/sig/ibex/equiv/machine.rbs +54 -0
  429. data/sig/ibex/equiv.rbs +129 -0
  430. data/sig/ibex/error_messages/parser.rbs +58 -0
  431. data/sig/ibex/error_messages/parser_v2.rbs +67 -0
  432. data/sig/ibex/error_messages/renderer.rbs +23 -0
  433. data/sig/ibex/error_messages/sentence_search.rbs +80 -0
  434. data/sig/ibex/error_messages/update.rbs +43 -0
  435. data/sig/ibex/error_messages.rbs +85 -0
  436. data/sig/ibex/fix.rbs +138 -0
  437. data/sig/ibex/frontend/ast.rbs +237 -22
  438. data/sig/ibex/frontend/bootstrap_parser.rbs +13 -0
  439. data/sig/ibex/frontend/diagnostic.rbs +53 -0
  440. data/sig/ibex/frontend/diagnostic_recovery.rbs +98 -0
  441. data/sig/ibex/frontend/dsl.rbs +33 -4
  442. data/sig/ibex/frontend/formatter.rbs +143 -0
  443. data/sig/ibex/frontend/generated_parser.rbs +228 -68
  444. data/sig/ibex/frontend/generated_parser_base.rbs +65 -12
  445. data/sig/ibex/frontend/generated_parser_includes.rbs +23 -0
  446. data/sig/ibex/frontend/generated_parser_metadata.rbs +23 -0
  447. data/sig/ibex/frontend/generated_parser_parameters.rbs +24 -0
  448. data/sig/ibex/frontend/generation.rbs +6 -0
  449. data/sig/ibex/frontend/lexer.rbs +49 -2
  450. data/sig/ibex/frontend/lexer_recovery.rbs +25 -0
  451. data/sig/ibex/frontend/parser/declarations.rbs +57 -0
  452. data/sig/ibex/frontend/parser/parameters.rbs +28 -0
  453. data/sig/ibex/frontend/parser/rules.rbs +3 -0
  454. data/sig/ibex/frontend/parser.rbs +66 -0
  455. data/sig/ibex/frontend/parser_configuration_support.rbs +27 -0
  456. data/sig/ibex/frontend/regenerator.rbs +11 -0
  457. data/sig/ibex/frontend/resolution.rbs +34 -0
  458. data/sig/ibex/frontend/resolver.rbs +91 -0
  459. data/sig/ibex/frontend/rule_documentation.rbs +42 -0
  460. data/sig/ibex/frontend/source_cursor.rbs +38 -5
  461. data/sig/ibex/frontend/source_document.rbs +119 -0
  462. data/sig/ibex/frontend/source_loader.rbs +66 -0
  463. data/sig/ibex/frontend/source_span.rbs +56 -0
  464. data/sig/ibex/frontend/token_adapter/declaration_document_state.rbs +21 -0
  465. data/sig/ibex/frontend/token_adapter/declaration_lexer_state.rbs +27 -0
  466. data/sig/ibex/frontend/token_adapter/declaration_state.rbs +83 -3
  467. data/sig/ibex/frontend/token_adapter/rule_state.rbs +20 -0
  468. data/sig/ibex/frontend/token_adapter.rbs +11 -2
  469. data/sig/ibex/frontend.rbs +1 -1
  470. data/sig/ibex/fuzz.rbs +71 -0
  471. data/sig/ibex/generation_input.rbs +37 -0
  472. data/sig/ibex/generation_manifest.rbs +70 -0
  473. data/sig/ibex/generation_transaction.rbs +82 -0
  474. data/sig/ibex/generation_transaction_recovery.rbs +36 -0
  475. data/sig/ibex/generation_transaction_validation.rbs +65 -0
  476. data/sig/ibex/grammar_tests.rbs +93 -0
  477. data/sig/ibex/ir/automaton_ir.rbs +35 -10
  478. data/sig/ibex/ir/grammar_ir.rbs +96 -22
  479. data/sig/ibex/ir/lexer_ir.rbs +57 -0
  480. data/sig/ibex/ir/parser_contract.rbs +69 -0
  481. data/sig/ibex/ir/serialize.rbs +75 -23
  482. data/sig/ibex/ir/validator/automaton.rbs +112 -0
  483. data/sig/ibex/ir/validator/base.rbs +69 -0
  484. data/sig/ibex/ir/validator/grammar.rbs +195 -0
  485. data/sig/ibex/ir/validator/lexer.rbs +37 -0
  486. data/sig/ibex/ir/validator.rbs +16 -0
  487. data/sig/ibex/ir.rbs +45 -5
  488. data/sig/ibex/lalr/build_metrics.rbs +60 -0
  489. data/sig/ibex/lalr/builder.rbs +150 -19
  490. data/sig/ibex/lalr/conflict_search.rbs +12 -5
  491. data/sig/ibex/lalr/counterexample.rbs +20 -14
  492. data/sig/ibex/lalr/direct_lookaheads.rbs +93 -0
  493. data/sig/ibex/lalr/goto_follows.rbs +89 -0
  494. data/sig/ibex/lalr/ielr/annotator.rbs +67 -0
  495. data/sig/ibex/lalr/ielr/bits.rbs +14 -0
  496. data/sig/ibex/lalr/ielr/inadequacy.rbs +24 -0
  497. data/sig/ibex/lalr/ielr/item_lookaheads.rbs +35 -0
  498. data/sig/ibex/lalr/ielr/pipeline.rbs +17 -0
  499. data/sig/ibex/lalr/ielr/split_stability.rbs +27 -0
  500. data/sig/ibex/lalr/ielr/split_state.rbs +14 -0
  501. data/sig/ibex/lalr/ielr/state_splitter.rbs +76 -0
  502. data/sig/ibex/lalr/ielr_partition.rbs +71 -0
  503. data/sig/ibex/lalr/inadequacy_report.rbs +21 -0
  504. data/sig/ibex/lalr/lookahead_propagation.rbs +38 -0
  505. data/sig/ibex/lalr/lr0_collection.rbs +50 -0
  506. data/sig/ibex/lalr/on_error_reductions.rbs +22 -0
  507. data/sig/ibex/lalr/unreachable_states.rbs +25 -0
  508. data/sig/ibex/lalr.rbs +17 -1
  509. data/sig/ibex/location.rbs +67 -0
  510. data/sig/ibex/lsp/document_handlers.rbs +24 -0
  511. data/sig/ibex/lsp/document_store.rbs +94 -0
  512. data/sig/ibex/lsp/document_store_diagnostics.rbs +20 -0
  513. data/sig/ibex/lsp/document_store_validation.rbs +26 -0
  514. data/sig/ibex/lsp/initialization_handlers.rbs +30 -0
  515. data/sig/ibex/lsp/navigation_handlers.rbs +33 -0
  516. data/sig/ibex/lsp/parser_configuration_assistance.rbs +49 -0
  517. data/sig/ibex/lsp/position_codec.rbs +44 -0
  518. data/sig/ibex/lsp/protocol_error.rbs +32 -0
  519. data/sig/ibex/lsp/request_handlers.rbs +26 -0
  520. data/sig/ibex/lsp/request_support.rbs +40 -0
  521. data/sig/ibex/lsp/server.rbs +55 -0
  522. data/sig/ibex/lsp/symbol_index.rbs +77 -0
  523. data/sig/ibex/lsp/symbol_index_builder.rbs +88 -0
  524. data/sig/ibex/lsp/symbol_index_precedence_references.rbs +18 -0
  525. data/sig/ibex/lsp/symbol_index_source_queries.rbs +26 -0
  526. data/sig/ibex/lsp/symbol_occurrence.rbs +31 -0
  527. data/sig/ibex/lsp/transport.rbs +47 -0
  528. data/sig/ibex/lsp/workspace.rbs +42 -0
  529. data/sig/ibex/lsp/workspace_analyzer.rbs +69 -0
  530. data/sig/ibex/lsp.rbs +7 -0
  531. data/sig/ibex/messages.rbs +26 -0
  532. data/sig/ibex/metrics.rbs +50 -0
  533. data/sig/ibex/normalize/declarations.rbs +31 -0
  534. data/sig/ibex/normalize/diagnostics.rbs +13 -4
  535. data/sig/ibex/normalize/expander.rbs +22 -16
  536. data/sig/ibex/normalize/expression.rbs +11 -9
  537. data/sig/ibex/normalize/grammar_builder.rbs +11 -0
  538. data/sig/ibex/normalize/inline_expansion.rbs +129 -0
  539. data/sig/ibex/normalize/inline_validation.rbs +40 -0
  540. data/sig/ibex/normalize/lexer.rbs +37 -0
  541. data/sig/ibex/normalize/named_references.rbs +20 -0
  542. data/sig/ibex/normalize/nodes.rbs +14 -0
  543. data/sig/ibex/normalize/parameter_ebnf_lowering.rbs +33 -0
  544. data/sig/ibex/normalize/parameter_lowering.rbs +35 -0
  545. data/sig/ibex/normalize/parameter_substitution.rbs +42 -0
  546. data/sig/ibex/normalize/parameter_validation.rbs +44 -0
  547. data/sig/ibex/normalize/parameters.rbs +51 -0
  548. data/sig/ibex/normalize/parser_configuration.rbs +23 -0
  549. data/sig/ibex/normalize/recovery_declarations.rbs +24 -0
  550. data/sig/ibex/normalize.rbs +129 -18
  551. data/sig/ibex/racc_migration/checker.rbs +36 -0
  552. data/sig/ibex/racc_migration/harness.rbs +16 -0
  553. data/sig/ibex/racc_migration/report.rbs +55 -0
  554. data/sig/ibex/racc_migration.rbs +8 -0
  555. data/sig/ibex/rake_task.rbs +51 -0
  556. data/sig/ibex/samples.rbs +60 -0
  557. data/sig/ibex/table_artifact/builder.rbs +93 -0
  558. data/sig/ibex/table_artifact/cst_projection.rbs +20 -0
  559. data/sig/ibex/table_artifact/document.rbs +27 -0
  560. data/sig/ibex/table_artifact/executor.rbs +66 -0
  561. data/sig/ibex/table_artifact/serializer.rbs +25 -0
  562. data/sig/ibex/table_artifact/validator/metadata.rbs +52 -0
  563. data/sig/ibex/table_artifact/validator/support.rbs +46 -0
  564. data/sig/ibex/table_artifact/validator/tables.rbs +63 -0
  565. data/sig/ibex/table_artifact/validator.rbs +66 -0
  566. data/sig/ibex/table_artifact.rbs +32 -0
  567. data/sig/ibex/table_simulation/result.rbs +36 -0
  568. data/sig/ibex/table_simulation/simulator.rbs +98 -0
  569. data/sig/ibex/table_simulation/step.rbs +42 -0
  570. data/sig/ibex/table_simulation/text.rbs +14 -0
  571. data/sig/ibex/table_simulation.rbs +7 -0
  572. data/sig/ibex/tables.rbs +10 -35
  573. data/sig/ibex/verifiable_generation_bundle.rbs +24 -0
  574. data/sig/ibex/verification_report/builder.rbs +53 -0
  575. data/sig/ibex/verification_report/canonical_ir.rbs +41 -0
  576. data/sig/ibex/verification_report/logical_path.rbs +34 -0
  577. data/sig/ibex/verification_report/validator.rbs +114 -0
  578. data/sig/ibex/verification_report.rbs +44 -0
  579. data/sig/ibex/verify/action_correspondence.rbs +50 -0
  580. data/sig/ibex/verify/language_witness.rbs +112 -0
  581. data/sig/ibex/verify/reference_collection.rbs +63 -0
  582. data/sig/ibex/verify/result.rbs +52 -0
  583. data/sig/ibex/verify/verifier.rbs +170 -0
  584. data/sig/ibex/verify.rbs +6 -0
  585. data/sig/ibex/watch/runner.rbs +58 -0
  586. data/sig/ibex/watch/source_snapshot.rbs +46 -0
  587. data/sig/ibex/watch.rbs +7 -0
  588. data/sig/ibex.rbs +2 -0
  589. metadata +524 -19
  590. data/.rubocop.yml +0 -43
  591. data/CHANGELOG.md +0 -30
  592. data/Rakefile +0 -25
  593. data/Steepfile +0 -10
  594. data/docs/compat-notes.md +0 -37
  595. data/docs/lexer-coverage.md +0 -14
  596. data/docs/phase10-extensions.md +0 -27
  597. data/gemfiles/Gemfile +0 -7
  598. data/gemfiles/Gemfile.lock +0 -98
  599. data/lib/ibex/frontend/grammar.y +0 -156
  600. data/lib/ibex/runtime/parser.rb +0 -360
  601. data/lib/ibex/runtime.rb +0 -8
  602. data/sig/ibex/runtime/parser.rbs +0 -167
  603. data/sig/ibex/runtime.rbs +0 -6
@@ -0,0 +1,103 @@
1
+ # Bison grammar import for analysis
2
+
3
+ `ibex import bison` converts Bison-shaped grammar structure into ordinary
4
+ extended Ibex source for analysis. It is a one-way adapter, not a parser
5
+ generator compatibility mode. Imported C actions remain opaque, are never
6
+ executed by analysis, and make Ruby generation fail before code is emitted.
7
+
8
+ ```sh
9
+ ibex import bison parser.y -o parser.analysis.y
10
+ ibex import bison --format=json parser.y > import-report.json
11
+ ibex explain --state=STATE parser.y
12
+ ibex metrics parser.y
13
+ ```
14
+
15
+ `explain`, `metrics`, `diff`, and the other read-only grammar analysis paths
16
+ recognize a file with two Bison `%%` section markers and import it in memory.
17
+ Source-repair commands need a canonical imported file because edits against
18
+ the generated analysis source do not map byte-for-byte onto the Bison input.
19
+
20
+ ## Conversion contract
21
+
22
+ | Classification | Directives | Treatment |
23
+ |---|---|---|
24
+ | converted | `%token`, `%left`, `%right`, `%nonassoc`, `%precedence`, `%start`, `%expect`, `%expect-rr`, `%empty`, `%prec` | existing extended declarations or production syntax |
25
+ | recognized metadata | `%nterm`, `%type`, `%union`, `%destructor`, `%printer`, `%param`, `%parse-param`, `%lex-param`, `%define`, `%code` | positioned report entries; no new Ibex syntax is invented |
26
+ | unsupported | `%dprec`, `%merge`, GLR controls, generator/output controls, and every unknown directive | every occurrence is reported with its original line and column |
27
+
28
+ `$$`, `$1`, `$<type>1`, and `@$` become `result`, `val[0]`, `val[0]`, and
29
+ `result_loc` inside an opaque hexadecimal action sentinel. Ordinary `@1`
30
+ locations remain `@1`. Balanced action scanning handles nested braces,
31
+ strings, character literals, and C comments iteratively.
32
+
33
+ Bison permits uppercase nonterminals, lowercase terminals, names that collide
34
+ after punctuation removal, and names that are Ibex declaration words. The
35
+ adapter deterministically namespaces nonterminals as `bison_nt_*`, maps
36
+ lowercase terminals into `BISON_T_*`, and adds stable numeric suffixes to
37
+ sanitization collisions. Named LHS forms such as `expression[result]:` are
38
+ recognized.
39
+
40
+ The source and JSON report distinguish general unsupported directives from
41
+ structural gaps. Output/language controls and opaque initialization hooks do
42
+ not make the recovered production graph incomplete. Unknown directives,
43
+ `%dprec`, `%merge`, GLR semantics, and grammar-macro directives do. A generated
44
+ header therefore contains exactly one of:
45
+
46
+ ```text
47
+ # ibex-bison-structural-status: complete
48
+ # ibex-bison-structural-status: incomplete
49
+ ```
50
+
51
+ `ibex fix` refuses a structurally incomplete import. This prevents a repair
52
+ from being called safe when the bounded comparison only covered an
53
+ approximation of the source grammar.
54
+
55
+ ## Limits and failure behavior
56
+
57
+ The defaults are 20 MiB of input, 1,000,000 structural tokens, 50,000 rule
58
+ groups, and 100,000 actions. Override them with `--max-bytes`,
59
+ `--max-tokens`, `--max-rules`, and `--max-actions`; every value must be
60
+ positive. Exhaustion produces a version-1 JSON envelope and exits 2.
61
+ Successful import exits 0 even when unsupported directives are present,
62
+ because the report—not silence—is the result. Invocation and malformed-input
63
+ errors go to stderr and exit 1.
64
+
65
+ `-o` writes atomically, rejects an input/output alias, and refuses symlink
66
+ targets or files with multiple hard links. Third-party grammar files are not
67
+ packaged or committed.
68
+
69
+ ## External evidence and the `parse.y` boundary
70
+
71
+ The scheduled external gate downloads five checksum-pinned GNU Bison grammars
72
+ listed in [`gallery/EXTERNAL.md`](../gallery/EXTERNAL.md). It validates the
73
+ import report, parses the generated source, normalizes it, and builds an Ibex
74
+ LALR automaton. Current aggregate evidence is:
75
+
76
+ | Grammar | Rule groups | Actions | Productions | Ibex states | Unresolved S/R |
77
+ |---|---:|---:|---:|---:|---:|
78
+ | GNU Bison calc | 5 | 7 | 13 | 22 | 0 |
79
+ | jq | 29 | 167 | 167 | 311 | 408 |
80
+ | PHP | 177 | 552 | 635 | 1,203 | 0 |
81
+ | PostgreSQL | 795 | 2,436 | 3,640 | 6,942 | 0 |
82
+ | CRuby Bison-era `parse.y` | 223 | 594 | 781 | 1,303 | 0 |
83
+
84
+ For the pinned Bison-era CRuby grammar, GNU Bison 3.8.2 and Ibex both recover
85
+ 781 non-augmented productions and zero unresolved conflicts. Bison reports
86
+ 1,304 states while Ibex reports 1,303. The one-state difference is explained
87
+ by the acceptance convention: Bison shifts end-of-input into a separate
88
+ accept state, while Ibex accepts from the completed augmented start item.
89
+
90
+ Current CRuby `parse.y` is an Lrama input, not raw GNU Bison input. The pinned
91
+ current file imports 231 rule groups and 486 actions, but reports 22
92
+ structurally unsupported occurrences, including `%rule` grammar macros. The
93
+ approximate automaton has 1,152 states and 27 unresolved S/R conflicts.
94
+ Selecting state 413 keeps `explain` bounded and yields one nonunifying
95
+ reachability witness for the `'rescue' modifier` shift/reduce choice within 8
96
+ tokens and 10,000 configurations. That is useful diagnostic evidence about
97
+ the recovered approximation, not a claim about CRuby's actual Lrama table.
98
+ `fix` refuses this input because proposing a language-preserving repair across
99
+ unexpanded `%rule` definitions would be unsound.
100
+
101
+ The external gate reads public grammar text and GNU Bison's black-box report
102
+ only. It does not inspect Bison implementation source or generated C, and
103
+ publishes only aggregate counts.
@@ -0,0 +1,127 @@
1
+ # Comparative claim policy
2
+
3
+ This policy defines when Ibex documentation may compare Ibex with another
4
+ parser tool. The machine-readable registry is [`claims.yml`](claims.yml), and
5
+ `bundle exec rake quality:comparative_claims` validates the registry and its
6
+ public claim markers.
7
+
8
+ The initial comparison set is Racc, Lrama, GNU Bison, Menhir, Tree-sitter, and
9
+ ANTLR. A tool being in the set does not imply that a comparison has been made.
10
+ An unmeasured tool remains `not_compared`; unknown facts remain `unknown`.
11
+ Each entry also owns its canonical public aliases. README scanning recognizes
12
+ only this declared set and aliases; adding a tool or spelling requires a
13
+ reviewed registry change.
14
+
15
+ Tool-level state is derived from every registered claim that names that tool.
16
+ `pending_claims` is the canonical ordered list of claim IDs that are not yet
17
+ `measured`, and the reason is generated deterministically from those IDs,
18
+ their pending state, and their declared missing evidence. A tool with no claims
19
+ is `not_compared` with an empty pending list and unknown identity. A tool with
20
+ any pending claim is `evidence_pending`, even if another claim for that tool is
21
+ measured. `compared` is valid only when the tool has at least one claim, every
22
+ one is measured, and `pending_claims` is empty.
23
+
24
+ ## Required record
25
+
26
+ Every measured, evidence-pending, or review-pending record contains:
27
+
28
+ - an immutable claim ID and narrowly scoped wording;
29
+ - each subject's exact released version and, when applicable, repository
30
+ revision;
31
+ - one public command represented as an executable and an ordered argv list;
32
+ - fixed corpus paths and external revisions;
33
+ - known environment values and an explicit list of unrecorded values;
34
+ - excluded or unsupported semantics;
35
+ - the subjective review method and its completion state;
36
+ - repository-relative evidence paths and explicit limitations; and
37
+ - the SHA-256 of the canonical marker body, including every table and note; and
38
+ - an exact-revision validity scope, expiry statement, and conditions that
39
+ require review.
40
+
41
+ Repository subjects and corpora use immutable 40- or 64-hex revisions. Released
42
+ tools use an exact release version and may use `not_applicable` only for the
43
+ repository revision. It must not conceal missing information. Every required
44
+ environment identity is either recorded as a non-placeholder value or listed
45
+ as `unknown`, and each unknown field is repeated in the public limitations.
46
+
47
+ No extra JSON Schema is used for this YAML registry. Its cross-file rules—file
48
+ existence, canonical ordering, public markers, and wording restrictions—need an
49
+ explicit repository validator and would not be made clearer by duplicating
50
+ them in a data-only schema.
51
+
52
+ ## Public interface and corpus rules
53
+
54
+ Comparisons use public commands, public callbacks, or documented artifact
55
+ formats. Do not inspect another tool's implementation or generated source to
56
+ manufacture an advantage. If a public interface cannot expose a value, record
57
+ that limitation instead of inferring it.
58
+
59
+ Inputs, grammar revisions, lifecycle choices, lexer inclusion, runtime backend,
60
+ warm-up, repetitions, random seeds, and instrumentation must be fixed before a
61
+ measurement. Failure rows stay in the evidence. Results from different
62
+ environments or configurations form separate observations and must not be
63
+ silently merged.
64
+
65
+ ## Subjective review
66
+
67
+ Diagnostic and repair usefulness are human judgments. The review method must
68
+ fix case IDs and the allowed labels, retain rationale and disagreements, and
69
+ separate maintainer assessment from independent assessment. A record whose
70
+ required independent review is missing stays `review_pending`; a record missing
71
+ its direct result artifact stays `evidence_pending`. Neither state can use a
72
+ public strong-claim marker or be worded as a completed comparative conclusion.
73
+
74
+ Performance statistics and deterministic behavior digests do not require a
75
+ subjective reviewer, but their scope and limitations still require ordinary
76
+ code review.
77
+
78
+ ## Scoped wording
79
+
80
+ Comparative wording names the claim ID and states the relevant revision or
81
+ release, corpus, environment, measurement category, and important limitation.
82
+ For example:
83
+
84
+ > At revision X, on corpus Y and environment Z, metric M had the recorded
85
+ > relationship to tool version V. This does not describe other revisions,
86
+ > workloads, backends, or semantic capabilities.
87
+
88
+ Do not publish wording such as “faster than Racc” or “better diagnostics”
89
+ without those boundaries. README comparative strength wording is enclosed by
90
+ `comparative-claim` markers. Only a `measured` record with a direct result
91
+ artifact may use that marker. Pending material instead uses a
92
+ `comparative-evidence` marker around the full table or conclusion it records.
93
+ The validator requires each marker body to contain the registry wording exactly
94
+ and every registered table anchor. It also canonicalizes line endings and
95
+ trailing whitespace and requires the complete body SHA-256 to match, so any
96
+ other prose, table-cell, row, or blank-line change fails independently of its
97
+ vocabulary. It always scans README, even when no claim targets README, and
98
+ rejects unmarked paragraphs that combine a declared tool alias with English or
99
+ Japanese strength wording.
100
+
101
+ ## No cross-category ordering
102
+
103
+ <!-- comparison-policy:forbidden-terms:start -->
104
+ Do not collapse performance, diagnostics, recovery, migration, ecosystem, and
105
+ verification into an aggregate score or tool ranking. The categories have
106
+ different semantics and trust boundaries. Publish each observation and its
107
+ limitations separately. Numeric totals never imply semantic equivalence.
108
+ <!-- comparison-policy:forbidden-terms:end -->
109
+
110
+ ## Review and update workflow
111
+
112
+ 1. Add evidence without deleting unfavorable or failed rows.
113
+ 2. Add or update the claim in canonical claim-ID order. Keep corpus and evidence
114
+ entries ordered by ID and path.
115
+ 3. Mark unavailable tools or facts `not_compared` or `unknown`; do not estimate
116
+ values.
117
+ 4. Bind measured wording with claim markers or pending tables and conclusions
118
+ with evidence markers. The body must retain the exact registered wording.
119
+ 5. After reviewing an intentional marker-body diff, run the quality task. Its
120
+ mismatch reports both the registered and actual SHA-256; copy the actual
121
+ digest into `body_sha256` only in the same reviewed change. Never refresh a
122
+ digest merely to make the task pass.
123
+ 6. Run `bundle exec rake quality:comparative_claims`, the focused tests, and
124
+ documentation coverage.
125
+ 7. When a review condition fires, either remeasure under a new claim ID or
126
+ narrow the old wording to its historical scope. Historical observations are
127
+ retained rather than rewritten as current results.
@@ -0,0 +1,59 @@
1
+ ---
2
+ title: Configuration model
3
+ description: Decide which parser settings belong to grammar source and which belong to an invocation.
4
+ ---
5
+
6
+ # Configuration model
7
+
8
+ Ibex keeps reproducible grammar intent in the grammar and keeps environment or
9
+ review choices at the invocation boundary. This separation prevents a build
10
+ flag from silently changing a source-owned parser contract.
11
+
12
+ ## Grammar-owned settings
13
+
14
+ Declare settings in an extended grammar when every consumer should observe the
15
+ same choice:
16
+
17
+ ```text
18
+ class ExampleParser
19
+ pragma extended
20
+ parser
21
+ algorithm ielr
22
+ entries isolated
23
+ end
24
+ ```
25
+
26
+ The current grammar-owned parser block supports `slr`, `lalr`, `ielr`, or
27
+ `lr1` construction and `shared` or `isolated` entry handling. The declaration
28
+ is part of normalized Grammar IR and is visible in generated reports.
29
+
30
+ ## Invocation-owned settings
31
+
32
+ Keep these choices at the CLI or API call site because they describe a local
33
+ operation rather than the grammar's meaning:
34
+
35
+ - output path and embedded-runtime selection (`-o`, `-E`);
36
+ - table representation and diagnostic format;
37
+ - warnings, verification strictness, resource budgets, and report output;
38
+ - a temporary analysis algorithm used for comparison or investigation; and
39
+ - external command execution for explicitly unsafe fuzz or reduction workflows.
40
+
41
+ An invocation may select an analysis algorithm, but it must report that the
42
+ selection is noncanonical when it differs from the grammar declaration. A
43
+ conflicting canonical selection is rejected at the declaration location.
44
+
45
+ ## Review checklist
46
+
47
+ When adding a setting, ask:
48
+
49
+ 1. Would two independent users of the same grammar need the same value? If so,
50
+ it is grammar-owned.
51
+ 2. Is the value about this run, host, output, or evidence capture? If so, it is
52
+ invocation-owned.
53
+ 3. Does changing it alter generated parser behavior, persisted IR, or runtime
54
+ ABI? Update the relevant schema, fixture, maturity entry, and migration note.
55
+ 4. Is the setting opt-in? Keep the compatible default unchanged and document
56
+ the activation boundary next to the option.
57
+
58
+ See the [stability policy](stability.md), [grammar reference](grammar-reference.md),
59
+ and [development guide](development.md) for contract and verification rules.
@@ -0,0 +1,28 @@
1
+ # Effective configuration reports
2
+
3
+ `ibex config` explains every key in `Ibex::Configuration::Registry` without generating or loading a parser:
4
+
5
+ ```console
6
+ ibex config grammar.y
7
+ ibex config --format=json grammar.y
8
+ ibex config --from=grammar-ir --format=json grammar.json
9
+ ```
10
+
11
+ The command has a `static-no-user-code` trust boundary. For grammar source it uses the contained source resolver, so
12
+ the root grammar and its import closure are parsed as data while parent traversal, absolute imports, glob imports,
13
+ and symlink escapes are rejected. Parser actions, lexer actions, user-code sections, and generated parser classes are
14
+ never executed or required. Grammar IR input is accepted only through the normal current-format validator.
15
+
16
+ Each setting reports its effective value, owner, override policy, origin and source location when recorded,
17
+ explicit or implicit selection, canonical or noncanonical conformance, recording status, and source evidence.
18
+ Evidence is classified as `accepted`, `ignored`, `duplicate`, or `conflicting`. The JSON projection is deterministic,
19
+ uses schema version 1, and always lists canonical keys in lexical order.
20
+
21
+ The current Grammar IR `parser_contract` entries are authoritative for `parser.algorithm`, `parser.entries`, and `cst.trivia`.
22
+ A matching command-line request is retained as accepted evidence. A contradictory request produces a positioned,
23
+ structured conflict report and exits nonzero. An unspecified entry remains explicitly `unspecified`; a document without
24
+ the current contract is rejected. A current builtin or CLI value is never presented as a historical fact.
25
+
26
+ The source grammar syntax supports the root-only declarative parser block in extended mode. CST ownership is
27
+ declared as `cst_trivia leading|balanced|drop` and requires `pragma cst`; `ibex config` reports the resulting
28
+ value and source location without executing user code. It does not recognize generic or unknown settings.
@@ -0,0 +1,28 @@
1
+ # H004 independent review records
2
+
3
+ During blind collection, review only
4
+ `docs/conflict-explanation-reviews/v1/blind-study-v1.json`. Do not inspect the
5
+ source study, review registry, implementation, or repository history until the
6
+ submission has been sealed. The blind artifact uses neutral IDs and excludes
7
+ the source case ID, shape, grammar path, and maintainer hypothesis. For every
8
+ case, use its state items, competing actions, bounded witness, and repair
9
+ section to identify the cause and choose an edit without executing grammar
10
+ actions.
11
+
12
+ A record must match `schema/conflict-explanation-review-v1.schema.json`'s
13
+ `review_record` definition and contain exactly one review for each of
14
+ `H004-BLIND-01` through `H004-BLIND-04`. Copy the blind artifact SHA-256 into
15
+ the record, set `submitted_before_reveal` to true only when accurate, record
16
+ your own cause and edit text, and leave `reveal_comparison.status` as `pending`.
17
+ If the machine artifact offers no verified repair, `chosen_edit` may describe a
18
+ manual edit and `repair_usefulness` should remain `no_verified_repair` unless
19
+ there is an actual proposal to assess.
20
+
21
+ The registry remains in `blind_collection` and `HOLD` until it has at least two
22
+ records from distinct reviewers. Only then may the source study be revealed.
23
+ During `reveal_comparison`, map every neutral ID to its source case, bind the
24
+ source study digest, and record cause/edit alignment plus notes without
25
+ rewriting the original blind answers. The registry reaches `review_complete`
26
+ and `PASS` only when every record has a completed reveal comparison. Do not
27
+ collapse disagreements into a majority label; preserve them in each record and
28
+ in the registry summary.
@@ -0,0 +1,62 @@
1
+ # Conflict-explanation usefulness study
2
+
3
+ H004 separates reproducible machine evidence from subjective usefulness
4
+ review. The committed version-1 corpus contains four grammar shapes and five
5
+ conflicts:
6
+
7
+ - an ambiguous expression and dangling `else` shift/reduce conflict;
8
+ - an ambiguous reduce/reduce choice;
9
+ - two nonunifying reduce/reduce conflicts introduced by LALR context merging.
10
+
11
+ Each case binds its grammar bytes, Automaton IR digest, conflicting state items,
12
+ competing actions, resolution, and bounded witness. It also runs the existing
13
+ `fix` candidate search with fixed limits. Only the expression case currently
14
+ has a machine-verified proposal: declaring right associativity for `PLUS`
15
+ removes the conflict and finds no language or mapped-tree difference within the
16
+ recorded bounds. The other cases truthfully record `no_safe_proposal`; advice
17
+ that merely acknowledges a conflict or changes recovery is labeled as not a
18
+ verified repair.
19
+
20
+ Every witness records its typed search outcome, explored configuration count,
21
+ exhaustion flag, and token/configuration bounds. A configuration-budget
22
+ exhaustion is `inconclusive`; it is never relabeled as a nonunifying witness or
23
+ accepted into the fixed study capture.
24
+
25
+ Regenerate or verify the machine capture with:
26
+
27
+ ```sh
28
+ bundle exec ruby tool/conflict_explanation_study.rb --write
29
+ bundle exec rake quality:conflict_explanations
30
+ ```
31
+
32
+ The normative machine artifact is
33
+ [`study-v1.json`](../test/fixtures/conflict_explanations/study-v1.json),
34
+ validated by the closed
35
+ [`schema`](../schema/conflict-explanation-study-v1.schema.json). The capture
36
+ uses repository-owned grammars and never executes grammar actions. Search,
37
+ candidate, build, equivalence, and verifier limits are explicit; exhaustion is
38
+ not success. The detailed pre-implementation contract and kill conditions are
39
+ recorded in the [investigation](investigations/H004-conflict-explanation-study.md).
40
+
41
+ ## Independent human gate
42
+
43
+ The machine capture proves reproducibility, not usefulness. Independent
44
+ reviewers receive only the deterministic
45
+ [`blind-study-v1.json`](conflict-explanation-reviews/v1/blind-study-v1.json)
46
+ view. It uses neutral case IDs and omits the repository case ID, shape, grammar
47
+ path, and `maintainer_hypothesis`. Reviewers use its state/items and witness to
48
+ identify the cause, then choose an edit and explain whether the explanation and
49
+ proposed repair helped. Each submission binds the blind artifact digest and is
50
+ sealed before any expected answer is revealed.
51
+ The [record instructions](conflict-explanation-reviews/v1/records/README.md)
52
+ and [review schema](../schema/conflict-explanation-review-v1.schema.json)
53
+ preserve per-case rationales and reviewer disagreement.
54
+
55
+ The current subjective gate is **HOLD** with zero external records, as recorded
56
+ in the [status registry](conflict-explanation-review-status-v1.json). Once at
57
+ least two distinct blind submissions cover every neutral case, the repository
58
+ case mapping and maintainer hypotheses may be revealed. Each reviewer record
59
+ must then retain a per-case comparison against that revealed material. PASS is
60
+ possible only after every comparison is complete; disagreement cannot be
61
+ replaced with a majority percentage. Until then no usefulness rate,
62
+ comparative superiority, or general conflict-repair claim is publishable.
@@ -0,0 +1,157 @@
1
+ # IELR and multi-entry construction profiling
2
+
3
+ This document closes H005's profiling work without selecting or implementing a
4
+ new construction algorithm. The machine-readable authority is
5
+ [`construction-profile-v1.json`](../tool/profile/evidence/construction-profile-v1.json),
6
+ validated by
7
+ [`construction-profile-v1.schema.json`](../schema/construction-profile-v1.schema.json).
8
+ It is an `internal_local_observation`, not release or adoption evidence.
9
+
10
+ ## Public profiler
11
+
12
+ Run the repository-owned and synthetic profiles with:
13
+
14
+ ```sh
15
+ bundle exec ruby tool/construction_profile.rb \
16
+ --wall-seconds=60 \
17
+ --output=tool/profile/evidence/construction-profile-v1.json
18
+ bundle exec rake quality:construction_profile
19
+ ```
20
+
21
+ Third-party observations require a full checkout that the existing public
22
+ workload manifest can verify. Each checkout must have the exact revision,
23
+ origin, clean status, grammar digest, and structural baseline:
24
+
25
+ ```sh
26
+ bundle exec ruby tool/construction_profile.rb \
27
+ --checkout=namae=/path/to/namae \
28
+ --checkout=bcdice_command=/path/to/BCDice \
29
+ --checkout=nokogiri_css=/path/to/nokogiri
30
+ ```
31
+
32
+ Omitting a checkout records its exact registry metadata as `not_run`; it does
33
+ not silently profile a cached file or substitute a synthetic grammar.
34
+
35
+ The artifact separates reconstructible source provenance from host-bound
36
+ observations. `provenance.base_revision` is only the Git base: when capture is
37
+ dirty, every H005 contract path records its observed digest, base-object
38
+ digest, and `base` / `modified` / `untracked` state. The clean flag is derived
39
+ from the exact porcelain status; the status, bound-path set, implementation
40
+ subset, host observation, and measurement policy each have an integrity
41
+ digest. The evidence file is excluded from the source and implementation
42
+ digests, so no digest refers to the bytes that contain itself.
43
+
44
+ The quality gate reconstructs the base objects, requires the base to be an
45
+ ancestor of the checked revision, verifies current contract bytes against the
46
+ bound paths, and checks all derived digests and workload/run entry identities.
47
+ Ruby version and options, kernel, CPU, and other host fields remain explicitly
48
+ host-bound rather than cross-platform goldens, but their recorded values are
49
+ validated against `environment_observation_sha256` before that observation is
50
+ excluded from the deterministic structural comparison.
51
+
52
+ Every construction has an explicit wall-time limit. Exceeding it records the
53
+ resource and limit. `NoMemoryError` and `SystemStackError` are recorded as
54
+ resource exhaustion. Those results establish only that a bound was observed;
55
+ they are not negative proofs about the language or algorithm.
56
+
57
+ ## Counting contract
58
+
59
+ The profiler records these structural quantities from the construction that
60
+ already exists:
61
+
62
+ | Field | Meaning |
63
+ | --- | --- |
64
+ | `lr0_states` | Distinct LR(0) cores materialized directly or represented by the canonical collection. |
65
+ | `lr0_items` | Core-item occurrences across those distinct cores. |
66
+ | `canonical_states` / `canonical_items` | Canonical LR(1) states and item triples, only when that collection completes. |
67
+ | `final_states` / `final_items` | States and core-item occurrences after merge or IELR partitioning. |
68
+ | `final_lookahead_items` | Lookahead memberships across final core items. |
69
+ | `propagation_edges` | Deduplicated direct-LALR lookahead propagation edges; canonical paths report `not_applicable`. |
70
+ | `ielr_initial_partitions` / `ielr_final_partitions` | Compatible partitions before and after transition refinement. |
71
+
72
+ `not_applicable` is reserved for metrics that the selected construction path
73
+ does not perform, such as canonical-state counts in direct LALR. A metric that
74
+ would apply but could not be collected because construction timed out or
75
+ failed is `not_measured`; failure never masquerades as algorithmic absence.
76
+
77
+ Elapsed seconds are retained only as host-bound observations. The schema fixes
78
+ `release_gate` to `false`, and the quality gate deliberately removes elapsed
79
+ values before comparing evidence. Ruby's live-slot counters are neither a
80
+ peak measurement nor reproducible retained-object accounting, so peak retained
81
+ objects are truthfully `not_measured`. The current builders also expose no
82
+ state budget or portable per-build heap limit; both omissions are explicit in
83
+ the evidence.
84
+
85
+ Profiling is opt-in through `Builder(profile: true)`. Ordinary generation does
86
+ not calculate the additional item/core summaries. Parser tables, conflicts,
87
+ generated source, runtime ABI, Grammar IR, and Automaton IR are unchanged.
88
+
89
+ ## Current observations
90
+
91
+ Synthetic and real workloads are separate cohorts. All repository grammar
92
+ revisions and SHA-256 digests come from the public workload registry. The
93
+ multi-entry diagnostic is the existing representative matrix grammar at the
94
+ recorded exact revision, not a new language fixture.
95
+
96
+ | Workload | Class | LR(0) states/items | Direct propagation edges | Canonical LR(1) states/items | IELR partitions initial/final | Final states |
97
+ | --- | --- | ---: | ---: | ---: | ---: | ---: |
98
+ | `gallery-calc` | repository synthetic | 15 / 76 | 99 | 27 / 674 | 15 / 15 | 15 |
99
+ | `gallery-json` | repository synthetic | 27 / 82 | 111 | 57 / 272 | 27 / 27 | 27 |
100
+ | `gallery-sql-lite` | repository synthetic | 26 / 53 | 55 | 26 / 115 | 26 / 26 | 26 |
101
+ | `matrix-multi-entry` | repository synthetic | 9 / 14 | n/a | 9 / 20 | 9 / 9 | 9 |
102
+ | `ibex-frontend` | repository real | 227 / 633 | 749 | 456 / 10,482 | 227 / 227 | 227 |
103
+
104
+ The table pairs direct-LALR edge counts with the canonical collection used by
105
+ current IELR. The multi-entry row reports the shared result; its committed run
106
+ records also contain isolated LALR and IELR. On this diagnostic all four
107
+ shared/isolated combinations retain 9 states, 14 final core items, and 20 final
108
+ lookahead memberships. That bounded synthetic equality is not evidence that a
109
+ direct multi-entry implementation lacks value on real grammars.
110
+
111
+ The three `public_real` identities—Namae, BCDice command, and Nokogiri CSS—are
112
+ recorded as `not_run` because no checkout satisfying the public manifest was
113
+ supplied for this evidence capture. They contribute no state, item, edge,
114
+ partition, failure, or timing observation. The only measured real workload is
115
+ the repository's production frontend; it has no unresolved conflict requiring
116
+ IELR, and current IELR partitions its 456 canonical states back to the same 227
117
+ final states as direct LALR.
118
+
119
+ ## Decisions
120
+
121
+ Direct IELR is **NO-GO** under the current verifier and workload evidence.
122
+ Direct multi-entry is **MORE DATA**.
123
+
124
+ ### Direct IELR — NO-GO
125
+
126
+ The threshold is at least two verified representative real grammars that both
127
+ show a practical canonical-collection cost and require IELR for semantically
128
+ meaningful conflicts. The current artifact has one measured real grammar, zero
129
+ real IELR-required conflict cases, and no verified public checkout.
130
+
131
+ [V001](verifier-trust-boundary.md) is complete, so the trust-boundary threshold
132
+ is satisfied. It also establishes a current NO-GO condition: default and
133
+ strict `ielr1` verification both enumerate the full canonical LR(1)
134
+ collection. A direct builder would therefore not remove canonical scale cost
135
+ from the current verified workflow, and the verifier does not establish IELR
136
+ adequacy or split witnesses. Together with no real IELR-required workload,
137
+ that makes direct IELR unjustified now rather than merely unmeasured.
138
+
139
+ Evidence that can change the decision must include verified real grammars,
140
+ material canonical state/item cost or observed exhaustion, meaningful
141
+ conflicts removed by IELR, a bounded verification plan that does not make full
142
+ canonical enumeration mandatory, and an owner accepting the specification and
143
+ maintenance budget. State-count vanity alone is insufficient.
144
+
145
+ ### Direct multi-entry — MORE DATA
146
+
147
+ The threshold is at least two verified real multi-entry grammars plus either an
148
+ observed construction bound or at least twofold structural overhead in the
149
+ current shared path relative to an admitted direct plan. The registry contains
150
+ zero real multi-entry workloads. The existing synthetic matrix exercises both
151
+ shared and isolated modes, but neither its elapsed time nor its equal state
152
+ counts justify a production algorithm.
153
+
154
+ Evidence that can change the decision must add verified real multi-entry
155
+ grammars, preserve entry dispatch and shared conflict attribution, demonstrate
156
+ material structural cost, and identify an owner for the semantic and
157
+ maintenance plan.