kumi 0.0.31 → 0.0.35

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4b228964d7834bd37eb2b5f969fb288edbb4dab6e27ef72d45a04fa9fb87a2d0
-  data.tar.gz: 79e0b9db5acedb69e61eb6799844fa6a849fca4ed604e79522af03147e3e0436
+  metadata.gz: 74f061f34a9936380a89aff28563594be891ac849f233613edf5ad9b52acce64
+  data.tar.gz: 8c7ee4b31d80293c18d8ea5914ae51ec80a489f6950352dd47044aa1b1aecba1
 SHA512:
-  metadata.gz: b28eb238da8e21b0ec51dd1f5c3e2ac769fd07bef7461787a456fded7dba4a5d7603e2d3eff110e23bbe69af8d28e303834d0aa12b2a06fe6ee7d8c441eae289
-  data.tar.gz: 98fa5a9f34f3ab1c8d9aea17ce888025713d73665de5a85608a4108fdcd697d587986ba24d404f505b35cabffadb8880765ec7f9cd803a232d2295bd4b6b36fc
+  metadata.gz: 70164a71693f388d25f2b2c8fc3c7d938860bb6e0f06344f06f7be96d1edb25a1e88686339fb81e919f6fde6d11510d14174cea22c3acc9f200bcc5c6fbf8724
+  data.tar.gz: 02df5faaef97f3d951a5fb022d7621daf1cec37ee482d0aa8db366c0ff7e0f44f6dc301302745249de1901c63c6e40dc52cff3e8d0019712ddae364196a4f642

data/CHANGELOG.md

@@ -1,4 +1,42 @@
 ## [Unreleased]
+
+## [0.0.35] – 2026-06-11
+### Added
+- **Dataflow IR Pipeline:** New DF IR graph, typed operations, validation, import inlining, broadcast simplification, CSE, load deduplication, tuple/object canonicalization, and golden artifact coverage.
+- **Vector IR Pipeline:** New VecIR builder, lowerer, validator, axis canonicalization, constant propagation, GVN, DCE, peephole simplification, stencil detection, and focused specs.
+- **Loop IR Pipeline:** New LoopIR builder/lowerer/validator and Ruby/JavaScript loop codegen path replacing the legacy LIR codegen pipeline.
+- **IR Tooling:** Shared IR base types, pass pipeline infrastructure, pretty printers, SNAST factory helpers, and expanded golden v2 output artifacts for DF/Vec/Loop stages.
+- **Regression Coverage:** External installed-gem loading spec that builds and installs the gem in an isolated `GEM_HOME` so runtime packaging issues are caught.
+
+### Changed
+- Removed the legacy LIR pipeline and refreshed golden outputs to the DF/Vec/Loop representation stack.
+- Updated generated Ruby and JavaScript output to use the LoopIR-backed codegen path.
+- Hardened kernel definitions and docs around numeric reductions, string aggregations, arithmetic helpers, and min/max empty-array behavior.
+- Revised architecture, schema import, golden test, and development documentation around the new IR pipeline.
+
+### Fixed
+- Runtime `require "kumi"` no longer loads golden-test support files or leaks the Zeitwerk loader as a top-level `AUTOLOADER` constant.
+- Golden shared schemas are loaded only through `Kumi.load_shared_schemas!`, preserving test support behavior without coupling it to normal gem loading.
+
+## [0.0.33] – 2025-11-05
+### Added
+- **Peephole Optimization Helper:** `Kumi::Core::LIR::Peephole` plus specs for safer peephole transforms, inline Ruby clamp kernel, richer LIR dumps, and refreshed golden files
+- **Import Diagnostics:** Clearer errors with available declaration hints when imports fail
+
+### Changed
+- Default `Kumi.configure.compilation_mode` now `:jit`; override via config or `KUMI_COMPILATION_MODE`
+- Improved constant propagation output formatting for Ruby/JS codegen
+
+## [0.0.32] – 2025-10-23
+### Added
+- **Schema Imports:** Reuse declarations from other schemas with `import :name, from: Module` syntax
+- **Improved Import Errors:** Clearer messages with available declaration hints when imports fail
+- Documentation: `docs/COMPOSED_SCHEMAS.md` and `docs/SCHEMA_IMPORTS.md`
+
+## [0.0.31] – 2025-10-22
+### Changed
+- Documentation for schema imports in README and SYNTAX
+
 ## [0.0.30] – 2025-10-21
 ### Changed
 - **Analyzer Refactoring:** PassManager now centralizes pass orchestration, replacing manual orchestration in Analyzer.run_analysis_passes
@@ -174,4 +212,4 @@ Fix - Remove require of pry gem on runtime.
 - Ruby >= 3.1 (Was >= 3.0)
 
 ### Notes
-- No expected DSL changes for typical schemas; report regressions.
+- No expected DSL changes for typical schemas; report regressions.

data/README.md

@@ -10,116 +10,44 @@
 
 ## What is Kumi?
 
-Kumi is a **declarative DSL for building calculation systems** that are:
-- **Typed & verifiable at compile time** (catch errors before they hit production)
-- **Vectorized** (arrays and nested data structures work naturally)
-- **Transparent** (inspect generated code and execution order)
-- **Portable** (compile the same schema to Ruby or JavaScript)
+Kumi is a **declarative DSL for calculation logic** — tax rules, pricing, scoring, financial projections — that compiles to plain Ruby and JavaScript.
 
-Instead of writing procedural formulas, you declare *what* values depend on *what*, and Kumi figures out the computation order, validates types, detects impossible constraints, and generates efficient code.
-
-## Why Kumi Exists
-
-Calculation systems are everywhere: tax engines, pricing models, financial projections, compliance checks. They're usually:
-- Hard to verify (logic spread across multiple files)
-- Fragile (changing one formula breaks hidden dependencies)
-- Duplicated (same logic needed in backend and frontend)
-- Opaque (hard to audit which rules applied to which data)
-
-Kumi makes them explicit, testable, and portable.
-
----
-
-**Status**: experimental. Public API may change. Typing and some static checks are still evolving.
-
-**Feedback**: have a use case or hit a rough edge? Open an issue or reach out.
-
----
-
-## Example: US Tax Calculator (2024)
-
-A single schema computes federal, state, FICA taxes across multiple filing statuses—all types verified at compile time. Try it in the [interactive demo](https://kumi-play-web.fly.dev/) or inspect the [full schema, input, output, and generated code](golden/us_tax_2024/).
-
-<details>
-<summary><strong>Schema</strong></summary>
+You declare the shape of your input data and the values you want computed. The compiler determines evaluation order, checks types, detects impossible conditions, and emits dependency-free code for each target.
 
 ```ruby
 schema do
   input do
-    float  :income
-    float  :state_rate
-    float  :local_rate
-    float  :retirement_contrib
-    string :filing_status
-
-    array :statuses do
-      hash :status do
-        string :name
-        float  :std
-        float  :addl_threshold
-        array  :rates do
-          hash :bracket do
-            float :lo
-            float :hi # -1 = open-ended
-            float :rate
-          end
-        end
+    array :items do
+      hash :item do
+        integer :quantity
+        decimal :unit_price
       end
     end
   end
 
-  # shared
-  let :big_hi, 100_000_000_000.0
-  let :state_tax, input.income * input.state_rate
-  let :local_tax, input.income * input.local_rate
-
-  # FICA constants
-  let :ss_wage_base, 168_600.0
-  let :ss_rate, 0.062
-  let :med_base_rate, 0.0145
-  let :addl_med_rate, 0.009
-
-  # per-status federal
-  let :taxable,   fn(:max, [input.income - input.statuses.status.std, 0])
-  let :lo,        input.statuses.status.rates.bracket.lo
-  let :hi,        input.statuses.status.rates.bracket.hi
-  let :rate,      input.statuses.status.rates.bracket.rate
-  let :hi_eff,    select(hi == -1, big_hi, hi)
-  let :amt,       fn(:clamp, taxable - lo, 0, hi_eff - lo)
-  let :fed_tax,   fn(:sum, amt * rate)
-  let :in_br,     (taxable >= lo) & (taxable < hi_eff)
-  let :fed_marg,  fn(:sum_if, rate, in_br)
-  let :fed_eff,   fed_tax / fn(:max, [input.income, 1.0])
-
-  # per-status FICA
-  let :ss_tax,         fn(:min, [input.income, ss_wage_base]) * ss_rate
-  let :med_tax,        input.income * med_base_rate
-  let :addl_med_tax,   fn(:max, [input.income - input.statuses.status.addl_threshold, 0]) * addl_med_rate
-  let :fica_tax,       ss_tax + med_tax + addl_med_tax
-  let :fica_eff,       fica_tax / fn(:max, [input.income, 1.0])
-
-  # totals per status
-  let :total_tax,  fed_tax + fica_tax + state_tax + local_tax
-  let :total_eff,  total_tax / fn(:max, [input.income, 1.0])
-  let :after_tax,  input.income - total_tax
-  let :take_home,  after_tax - input.retirement_contrib
-
-  # array of result objects, one per status
-  value :summary, {
-    filing_status: input.statuses.status.name,
-    federal: { marginal: fed_marg, effective: fed_eff, tax: fed_tax },
-    fica: { effective: fica_eff, tax: fica_tax },
-    state: { marginal: input.state_rate, effective: input.state_rate, tax: state_tax },
-    local: { marginal: input.local_rate, effective: input.local_rate, tax: local_tax },
-    total: { effective: total_eff, tax: total_tax },
-    after_tax: after_tax,
-    retirement_contrib: input.retirement_contrib,
-    take_home: take_home
-  }
+  value :line_totals, input.items.item.quantity * input.items.item.unit_price
+  value :subtotal, fn(:sum, line_totals)
 end
 ```
 
-</details>
+No loops, no iteration plumbing: `line_totals` is computed per item because that's where the data lives, and `fn(:sum, ...)` collapses it back to a scalar. This works through arbitrarily nested arrays.
+
+## Why
+
+- **One source of truth, two targets.** The same schema compiles to Ruby and JavaScript with identical semantics — write pricing logic once, run it in your backend and in the browser preview.
+- **Broadcasting from data shape.** Operations align over arrays automatically based on the declared input structure, including nested and ragged data.
+- **Static checks at compile time.** Type checking, dependency cycle detection, and unsatisfiable-constraint detection happen when the schema is defined, not in production.
+- **Boring generated code.** Output is deterministic, dependency-free, straight-line code with explicit loops. What you read is what runs.
+
+## Use Cases
+
+Tax engines, pricing models, financial projections, compliance rules, insurance underwriting, shipping rate calculators — anywhere calculation logic must be correct, auditable, and consistent across platforms.
+
+---
+
+**Status**: experimental. Public API may change. Typing and some static checks are still evolving.
+
+**Feedback**: have a use case or hit a rough edge? Open an issue or reach out (andremuta+kumi@gmail.com).
 
 ---
 
@@ -129,7 +57,7 @@ end
 gem install kumi
 ```
 
-Requires Ruby 3.1+. No external dependencies.
+Requires Ruby 3.1+. Runtime dependencies: `mutex_m` and `zeitwerk` (bundled via Rubygems).
 
 ## Quick Start
 
@@ -149,22 +77,38 @@ end
 result = Double.from(x: 5)
 result[:doubled]  # => 10
 
+# or just call the method directly
+Double._doubled(x: 5) # => 10
+
 # Export to JavaScript (same logic)
 Double.write_source("output.mjs", platform: :javascript)
+# ./output.mjs
+# export function _doubled(input) {
+#   let t1 = input["x"];
+#   let t3 = t1 * 2;
+#   return t3;
+# }
 ```
 
-Try the [interactive demo](https://kumi-play-web.fly.dev/) (no setup required).
+You can also override the compilation strategy without touching code by setting
+`KUMI_COMPILATION_MODE` to `jit` or `aot` (e.g. `export KUMI_COMPILATION_MODE=aot`).
+
+## Examples
+
+- **US Tax Calculator (2024)** — a single schema computes federal, state, and FICA taxes across multiple filing statuses. [Open in the demo](https://kumi-play-web.fly.dev/?example=us-federal-tax-2024).
+- **Monte Carlo Portfolio** — probabilistic simulations and table visualizations. [Open in the demo](https://kumi-play-web.fly.dev/?example=monte-carlo-simulation).
+- **Conway's Game of Life** — array operations powering a grid-based simulation. [Open in the demo](https://kumi-play-web.fly.dev/?example=game-of-life).
 
 ---
 
 ## Documentation
 
-- **[Syntax Reference](docs/SYNTAX.md)** - DSL syntax, types, operators, and functions
-- **[Functions Reference](docs/FUNCTIONS.md)** - Auto-generated docs for all functions and kernels
-- **[functions-reference.json](docs/functions-reference.json)** - Machine-readable format for IDEs (VSCode, Monaco, etc.)
-- **[Development Guide](docs/DEVELOPMENT.md)** - Testing, debugging, and contributing
-
-To regenerate function docs: `bin/kumi-doc-gen`
+- **[Syntax Reference](docs/SYNTAX.md)** — DSL syntax, types, operators, functions
+- **[Functions Reference](docs/FUNCTIONS.md)** — auto-generated docs for all functions and kernels ([machine-readable JSON](docs/functions-reference.json))
+- **[Schema Imports](docs/SCHEMA_IMPORTS.md)** — composing and reusing schemas
+- **[Architecture](docs/ARCHITECTURE.md)** — the compiler pipeline and IR stack
+- **[Golden Tests](docs/GOLDEN_TESTS.md)** — the end-to-end test harness
+- **[Development Guide](docs/DEVELOPMENT.md)** — tooling, docs generation, IDE integration
 
 ---
 

data/Rakefile

@@ -9,4 +9,6 @@ require "rubocop/rake_task"
 
 RuboCop::RakeTask.new
 
+Dir[File.join(__dir__, "tasks/**/*.rake")].each { |task_file| import task_file }
+
 task default: %i[spec rubocop]

data/data/functions/agg/numeric.yaml

@@ -7,6 +7,8 @@ functions:
       param: source_value
     reduction_strategy: identity # This function has a true identity value.
     aliases: ["sum"]
+    options:
+      tuple_fold_combiner: core.add
 
   - id: agg.count
     kind: reduce
@@ -25,6 +27,8 @@ functions:
       param: source_value
     reduction_strategy: first_element
     aliases: ["min"]
+    options:
+      tuple_fold_combiner: core.min
 
   - id: agg.max
     kind: reduce
@@ -34,6 +38,8 @@ functions:
       param: source_value
     reduction_strategy: first_element
     aliases: ["max"]
+    options:
+      tuple_fold_combiner: core.max
 
   - id: agg.mean
     kind: reduce

data/data/functions/core/arithmetic.yaml

@@ -7,6 +7,32 @@ functions:
       param: number
     aliases: ["abs"]
 
+  - id: core.max
+    kind: elementwise
+    params:
+      - { name: left_operand, dtype: numeric }
+      - { name: right_operand, dtype: numeric }
+    dtype:
+      rule: promote
+      params: [left_operand, right_operand]
+    constraint_semantics:
+      pure_combiner: true
+      commutativity: true
+      associativity: true
+
+  - id: core.min
+    kind: elementwise
+    params:
+      - { name: left_operand, dtype: numeric }
+      - { name: right_operand, dtype: numeric }
+    dtype:
+      rule: promote
+      params: [left_operand, right_operand]
+    constraint_semantics:
+      pure_combiner: true
+      commutativity: true
+      associativity: true
+
   - id: core.add
     kind: elementwise
     params:

data/data/kernels/javascript/agg/numeric.yaml

@@ -16,12 +16,16 @@ kernels:
 
   - id: agg.min:javascript:v1
     fn: agg.min
-    # Handles first element being null/undefined
-    inline: "= ($0 === null || $1 < $0) ? $1 : $0"
+    # Skips null values (an inner reduction over an empty array yields null);
+    # the $1 !== null guard is required because null coerces to 0 in
+    # relational comparisons (null < 5 is true).
+    inline: "= ($1 !== null && ($0 === null || $1 < $0)) ? $1 : $0"
     fold_inline: "= Math.min(...$0)"
 
   - id: agg.max:javascript:v1
     fn: agg.max
-    # Handles first element being null/undefined
-    inline: "= ($0 === null || $1 > $0) ? $1 : $0"
+    # Skips null values (an inner reduction over an empty array yields null);
+    # the $1 !== null guard is required because null coerces to 0 in
+    # relational comparisons (null > -5 is true).
+    inline: "= ($1 !== null && ($0 === null || $1 > $0)) ? $1 : $0"
     fold_inline: "= Math.max(...$0)"

data/data/kernels/javascript/agg/string.yaml

@@ -2,4 +2,7 @@ kernels:
   - id: agg.join:javascript:v1
     fn: agg.join
     inline: "+= $1"
-    fold_inline: "= $0.join('')"
+    fold_inline: "= $0.join('')"
+    # String concatenation identity: join over an empty array is "".
+    identity:
+      string: ""

data/data/kernels/javascript/core/arithmetic.yaml

@@ -7,6 +7,14 @@ kernels:
     fn: core.mod
     inline: "= (($0 % $1) + $1) % $1"
 
+  - id: max:javascript:v1
+    fn: core.max
+    inline: "= Math.max($0, $1)"
+
+  - id: min:javascript:v1
+    fn: core.min
+    inline: "= Math.min($0, $1)"
+
   - id: add:javascript:v1
     fn: core.add
     inline: "= $0 + $1"

data/data/kernels/ruby/agg/numeric.yaml

@@ -18,12 +18,16 @@ kernels:
 
   - id: agg.min:ruby:v1
     fn: agg.min
-    inline: "= $1 if $1 < $0" 
+    # Skips nil values (an inner reduction over an empty array yields nil),
+    # matching the javascript kernel's null handling.
+    inline: "= $1 if !$1.nil? && $1 < $0"
     impl: "(a, b)\n  a < b ? a : b"
     fold_inline: "= $0.min"
 
   - id: agg.max:ruby:v1
     fn: agg.max
-    inline: "= $1 if $1 > $0" 
+    # Skips nil values (an inner reduction over an empty array yields nil),
+    # matching the javascript kernel's null handling.
+    inline: "= $1 if !$1.nil? && $1 > $0"
     impl: "(a, b)\n  a > b ? a : b"
     fold_inline: "= $0.max"

data/data/kernels/ruby/agg/string.yaml

@@ -2,4 +2,7 @@ kernels:
   - id: agg.join:ruby:v1
     fn: agg.join
     inline: "+= $1"
-    fold_inline: "= $0.join"
+    fold_inline: "= $0.join"
+    # String concatenation identity: join over an empty array is "".
+    identity:
+      string: ""

data/data/kernels/ruby/core/arithmetic.yaml

@@ -7,6 +7,16 @@ kernels:
     fn: core.mod
     inline: "= $0 % $1"
   
+  - id: max:ruby:v1
+    fn: core.max
+    inline: "= $0 > $1 ? $0 : $1"
+    impl: "(a, b)\n  a > b ? a : b"
+
+  - id: min:ruby:v1
+    fn: core.min
+    inline: "= $0 < $1 ? $0 : $1"
+    impl: "(a, b)\n  a < b ? a : b"
+
   - id: add:ruby:v1
     fn: core.add
     inline: "= $0 + $1"
@@ -39,5 +49,5 @@ kernels:
   
   - id: clamp:ruby:v1
     fn: core.clamp
-    inline: "= [[ $0, $1 ].max, $2 ].min"
-    impl: "(x, lo, hi)\n  [[x, lo].max, hi].min"
+    inline: "= $0.clamp($1, $2)"
+    impl: "(x, lo, hi)\n  x.clamp(lo, hi)"

data/docs/AGENTS.md

@@ -0,0 +1,60 @@
+# Agent Reference
+
+A toolbox for inspecting and debugging the compiler pipeline. The pipeline is
+SNAST → DFIR → VecIR → LoopIR → Ruby/JS emitters; see
+[ARCHITECTURE.md](ARCHITECTURE.md).
+
+## Inspect One Layer for One Schema
+
+```bash
+bundle exec bin/kumi pp <repr> <schema.kumi>
+# reprs: ast, input_plan, nast, snast, dfir, dfir_optimized, vecir, loopir,
+#        schema_ruby, schema_javascript
+bundle exec bin/kumi pp loopir golden/multi_loop_reduction/schema.kumi
+```
+
+This is the fastest way to surface a lowering crash in a specific layer.
+
+## Sweep for Crashes
+
+```bash
+for d in golden/*/; do
+  bundle exec bin/kumi pp loopir "$d/schema.kumi" >/dev/null || echo "$d"
+done
+```
+
+## Phase-Scoped Golden Checks
+
+```bash
+bundle exec bin/kumi golden_v2 verify --repr loop            # one layer, all schemas
+bundle exec bin/kumi golden_v2 diff --repr df <schema>       # unified diff
+bundle exec bin/kumi golden_v2 update --repr vec,loop        # regenerate layers
+```
+
+Groups: `frontend`, `df`, `vec`, `loop`, `codegen`, `all`.
+
+## Runtime Ground Truth
+
+```bash
+bundle exec bin/kumi golden test            # regenerate + execute Ruby and JS
+bundle exec bin/kumi golden test <schema>
+```
+
+## Search Goldens Instead of Reading Them
+
+Some golden artifacts are large (`game_of_life`, `us_tax_2024`) — grep them:
+
+```bash
+rg -n "reduce" golden -g"dfir*.txt"
+rg -n "axis_shift" golden -g"vecir.txt"
+rg -n "function" golden/<schema>/expected/loopir.txt
+```
+
+## Other Useful Commands
+
+```bash
+bundle exec bin/kumi analyze <schema> --dump <state_key>   # inspect analyzer state
+bundle exec rspec                                          # full test suite
+```
+
+`tmp/` is gitignored and a good place for debug scripts.