kumi 0.0.37 → 0.1.0

data/CHANGELOG.md

@@ -1,5 +1,42 @@
 ## [Unreleased]
 
+## [0.1.0] – 2026-06-20
+### Added
+- **Parser rewrite integration** (kumi-parser ≥ 0.1.0): new lexer/parser/grammar with located, framed parse errors. Semantic and parse errors now render on one `file:line:col` format; no more `?:?`.
+- **Cross-target semantics doc + parity tests** (`docs/CROSS_TARGET_SEMANTICS.md`): pinned Ruby/JS contracts for `to_string(float)`, `to_integer`/`to_float` of strings, and `pow` with a negative base and fractional exponent, each backed by a spec running Ruby against real node JS.
+- **Cross-references** beyond plain input fields: cross-axis (`A × A'`) iteration in Loop lowering, usable through `let`.
+- **Optimization passes**: algebraic identity folding (`x*1`, `x/1`, `x-0`, integer `x+0`/`x*0`) in Vec constant propagation and a Loop-layer copy-cleanup pass.
+- `SchemaMetadata`: the schema's algebra exposed as read-only data.
+
+### Changed
+- **Type system consolidated** under one `Core::Types` namespace (Registry/Profile/System/DtypeRule) with a per-target `Profile` seam; decimal > float > integer promotion lattice.
+- Compiler invariants, backend capability limits, and registry/schema/IR faults are now typed errors that break loudly instead of silently degrading.
+- `__select__` condition is typed `boolean`; the JS Loop emitter supports `impl`-only kernels (emitted once as a module-level helper), mirroring Ruby.
+
+### Fixed
+- `to_string(float)`, string `to_integer`/`to_float`, and `pow` of a negative base produced divergent results between Ruby and JS; kernels rewritten so both targets agree.
+- Typo'd input paths (`input.nope`) report a clean located user error instead of a compiler bug deep in dimensional analysis.
+- `PeepholeSimplify` could drop a def while keeping a stale use.
+
+### Removed
+- Vestigial `return_stamp` IR field, dead IRv2/pack tooling, and duplicated `RegisterGenerator`/terminal-instruction lookups across IR passes.
+
+## [0.0.40] – 2026-06-15
+### Added
+- LoopIR optimization pipeline (`Kumi::IR::Loop::Pipeline`), previously empty, now runs two passes on every compile:
+  - **LoopFusion** — merges sibling loops over the same axis and source into one traversal, hoisting independent scalar barriers above the loop. Stencil consumers (shifts, lengths, re-iteration of a partially built array) and accumulator reads correctly block fusion.
+  - **ArrayContraction** — replaces intermediate arrays that are filled and read back at the same index within one loop with the scalar that fills them, eliminating the materialization entirely.
+- `outer(...)` usable from text schemas (requires kumi-parser ≥ 0.0.33, now the pinned version). Golden `outer_let` covers the text path with Ruby/JS parity.
+
+### Changed
+- Streaming exports throw a `TypeError` when the target aliases an input array (silent zero-length corruption before; feedback loops must double-buffer) and when a typed-array target is passed for record outputs (silently produced an empty buffer before).
+- Array input arity errors now explain the element rule (Kumi maps over arrays by default, so the element must be named) and show the correct single-child form, instead of a cryptic type error.
+- The schema digest (compiled module name + compile-cache key) no longer folds in `RUBY_VERSION`. Generated code is plain Ruby with identical semantics across supported Rubies, so the digest is now Ruby-version-independent; this also lets the codegen goldens verify across the full CI Ruby matrix.
+
+### Fixed
+- Ruby DSL `let` raised `ArgumentError: struct size differs` since 0.0.37 (`inline: true` leaked into the `ValueDeclaration` struct instead of the `hints:` channel; the text frontend was unaffected).
+- `outer`/`cross` value used through a `let` that lives purely on the inner pairing axis (no dependence on the outer-consuming axis) read `nil` past the inner array's length (`nil can't be coerced into Float`). The materialized read indexed by positional loop depth instead of by the value's own axis; it now matches by axis. Affected both Ruby and JS codegen.
+
 ## [0.0.37] – 2026-06-11
 ### Added
 - Streaming hints for JS codegen

data/README.md

@@ -175,7 +175,10 @@ The playground's example picker groups these by theme (language tour, business l
 ## Documentation
 
 - **[Syntax Reference](docs/SYNTAX.md)** — DSL syntax, types, operators, functions
+- **[Syntax Notes](docs/SYNTAX_NOTES.md)** — parser differences, nested input recipes, expression literals, and post-parse errors
+- **[Input Shapes](docs/INPUTS.md)** — declaring scalars, arrays, hashes, and the element rule
 - **[Functions Reference](docs/FUNCTIONS.md)** — auto-generated docs for all functions and kernels ([machine-readable JSON](docs/functions-reference.json))
+- **[Cross-Target Semantics](docs/CROSS_TARGET_SEMANTICS.md)** — where Ruby and JS would diverge (float `to_string`, string conversions, `pow`) and how the kernels keep them identical
 - **[Schema Imports](docs/SCHEMA_IMPORTS.md)** — composing and reusing schemas
 - **[Architecture](docs/ARCHITECTURE.md)** — the compiler pipeline and IR stack
 - **[Golden Tests](docs/GOLDEN_TESTS.md)** — the end-to-end test harness

data/data/functions/core/arithmetic.yaml

@@ -7,6 +7,54 @@ functions:
       param: number
     aliases: ["abs"]
 
+  - id: core.sqrt
+    kind: elementwise
+    params: [{ name: number, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["sqrt"]
+
+  - id: core.sin
+    kind: elementwise
+    params: [{ name: angle, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["sin"]
+
+  - id: core.cos
+    kind: elementwise
+    params: [{ name: angle, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["cos"]
+
+  - id: core.exp
+    kind: elementwise
+    params: [{ name: number, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["exp"]
+
+  - id: core.log
+    kind: elementwise
+    params: [{ name: number, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["log"]
+
+  - id: core.tanh
+    kind: elementwise
+    params: [{ name: number, dtype: numeric }]
+    dtype:
+      rule: scalar
+      kind: float
+    aliases: ["tanh"]
+
   - id: core.max
     kind: elementwise
     params:

data/data/functions/core/conversion.yaml

@@ -30,3 +30,16 @@ functions:
       rule: scalar
       kind: string
     aliases: ["to_string", "to_str"]
+
+  # Float overload: a whole-valued float must stringify as "3.0", not "3", and
+  # both targets must agree. Ruby's Float#to_s already does this; the JS kernel
+  # reproduces Ruby's exact float formatting (see data/kernels/.../coercion.yaml
+  # and docs/CROSS_TARGET_SEMANTICS.md). Picked over the generic overload for
+  # float arguments by dtype-based overload resolution.
+  - id: core.to_string:float
+    kind: elementwise
+    params: [{ name: value, dtype: float }]
+    dtype:
+      rule: scalar
+      kind: string
+    aliases: ["to_string", "to_str"]

data/data/functions/core/select.yaml

@@ -1,7 +1,7 @@
 functions:
   - id: __select__
     kind: elementwise
-    params: [{ name: condition_mask }, { name: value_when_true }, { name: value_when_false }]
+    params: [{ name: condition_mask, dtype: boolean }, { name: value_when_true }, { name: value_when_false }]
     dtype:
       rule: same_as
       param: value_when_true

data/data/functions/core/stencil.yaml

@@ -9,6 +9,38 @@ functions:
       policy: wrap          # wrap|clamp
       axis_offset: 0
 
+  # Re-exposes the source array under a fresh, independent axis (the
+  # broadcast-dual of a reduction). Lowered specially: it introduces a new
+  # innermost loop over the same carrier so an array can be combined with
+  # itself (e.g. all-pairs / N-body). Axis stamping lives in the dimensional
+  # analyzer; this entry exists so resolution/validation treat cross as a
+  # known unary call.
+  - id: cross
+    kind: elementwise
+    params: [{ name: source }]
+    dtype:
+      rule: same_as
+      param: source
+    constraint_semantics:
+      domain_effect: NONE
+      pure_combiner: false
+
+  # Re-exposes a value from a DIFFERENT array as a fresh inner axis, so two
+  # distinct arrays can be paired all-pairs (A x B). Where `cross` self-joins one
+  # array, `outer` joins two. Lowered specially: it opens a new innermost loop
+  # over the *other* array's carrier. Axis stamping lives in the dimensional
+  # analyzer; this entry exists so resolution/validation treat outer as a known
+  # unary call.
+  - id: outer
+    kind: elementwise
+    params: [{ name: source }]
+    dtype:
+      rule: same_as
+      param: source
+    constraint_semantics:
+      domain_effect: NONE
+      pure_combiner: false
+
   - id: shift
     kind: elementwise
     params: [{ name: source }, { name: offset }]

data/data/kernels/javascript/agg/numeric.yaml

@@ -6,6 +6,7 @@ kernels:
     identity:
       float: 0.0
       integer: 0
+      decimal: 0.0
 
   - id: agg.count:javascript:v1
     fn: agg.count

data/data/kernels/javascript/core/arithmetic.yaml

@@ -3,6 +3,30 @@ kernels:
     fn: core.abs
     inline: "= Math.abs($0)"
 
+  - id: sqrt:javascript:v1
+    fn: core.sqrt
+    inline: "= Math.sqrt($0)"
+
+  - id: sin:javascript:v1
+    fn: core.sin
+    inline: "= Math.sin($0)"
+
+  - id: cos:javascript:v1
+    fn: core.cos
+    inline: "= Math.cos($0)"
+
+  - id: exp:javascript:v1
+    fn: core.exp
+    inline: "= Math.exp($0)"
+
+  - id: log:javascript:v1
+    fn: core.log
+    inline: "= Math.log($0)"
+
+  - id: tanh:javascript:v1
+    fn: core.tanh
+    inline: "= Math.tanh($0)"
+
   - id: mod:javascript:v1
     fn: core.mod
     inline: "= (($0 % $1) + $1) % $1"

data/data/kernels/javascript/core/coercion.yaml

@@ -4,17 +4,31 @@ kernels:
     inline: "= typeof $0 === 'string' ? parseFloat($0) : Number($0)"
     impl: "(value) {\n  if (typeof value === 'string') return parseFloat(value);\n  return Number(value);\n}"
 
+  # Match Ruby String#to_i / Numeric#to_i (and to_f): a non-numeric string is 0
+  # (not NaN), parsing is base-10 (so "0x1f" is 0, not 31), and a numeric value
+  # truncates toward zero without scientific-notation surprises (Math.trunc, not
+  # parseInt — parseInt(1e-7) is 1). Emitted as module-level helpers (no inline).
+  # Contract documented in docs/CROSS_TARGET_SEMANTICS.md.
   - id: to_integer:javascript:v1
     fn: core.to_integer
-    inline: "= parseInt($0)"
-    impl: "(value) { return parseInt(value); }"
+    impl: "(value) {\n  if (typeof value === 'number') return Math.trunc(value) || 0;\n  const m = String(value).trim().match(/^[+-]?\\d+/);\n  return m ? parseInt(m[0], 10) : 0;\n}"
 
   - id: to_float:javascript:v1
     fn: core.to_float
-    inline: "= parseFloat($0)"
-    impl: "(value) { return parseFloat(value); }"
+    impl: "(value) {\n  if (typeof value === 'number') return value;\n  const f = parseFloat(value);\n  return Number.isNaN(f) ? 0.0 : f;\n}"
 
   - id: to_string:javascript:v1
     fn: core.to_string
     inline: "= String($0)"
     impl: "(value) { return String(value); }"
+
+  # Reproduce Ruby's Float#to_s so to_string(float) matches across targets:
+  #   - whole floats keep ".0"  (3.0 -> "3.0", not JS's "3")
+  #   - negative zero keeps its sign  (-0.0 -> "-0.0")
+  #   - exponent form is "1.0e+21" / "1.0e-07"  (mantissa gets ".0", exponent
+  #     is signed and zero-padded to >= 2 digits)
+  # No `inline`: this is emitted once as a module-level helper function. The
+  # contract is documented in docs/CROSS_TARGET_SEMANTICS.md.
+  - id: to_string_float:javascript:v1
+    fn: core.to_string:float
+    impl: "(value) {\n  if (Number.isNaN(value)) return 'NaN';\n  if (value === Infinity) return 'Infinity';\n  if (value === -Infinity) return '-Infinity';\n  if (Object.is(value, -0)) return '-0.0';\n  let s = String(value);\n  const e = s.indexOf('e');\n  if (e !== -1) {\n    let mant = s.slice(0, e);\n    let exp = s.slice(e + 1);\n    if (!mant.includes('.')) mant += '.0';\n    const sign = exp[0] === '-' ? '-' : '+';\n    let digits = exp.replace(/^[+-]/, '');\n    if (digits.length < 2) digits = '0' + digits;\n    return mant + 'e' + sign + digits;\n  }\n  return s.includes('.') ? s : s + '.0';\n}"

data/data/kernels/ruby/agg/numeric.yaml

@@ -7,7 +7,8 @@ kernels:
     identity:
       float: 0.0
       integer: 0
-      
+      decimal: 0
+
   - id: agg.count:ruby:v1
     fn: agg.count
     inline: "+= 1"

data/data/kernels/ruby/core/arithmetic.yaml

@@ -2,7 +2,31 @@ kernels:
   - id: abs:ruby:v1
     fn: core.abs
     inline: "= $0.abs"
-  
+
+  - id: sqrt:ruby:v1
+    fn: core.sqrt
+    inline: "= Math.sqrt($0)"
+
+  - id: sin:ruby:v1
+    fn: core.sin
+    inline: "= Math.sin($0)"
+
+  - id: cos:ruby:v1
+    fn: core.cos
+    inline: "= Math.cos($0)"
+
+  - id: exp:ruby:v1
+    fn: core.exp
+    inline: "= Math.exp($0)"
+
+  - id: log:ruby:v1
+    fn: core.log
+    inline: "= Math.log($0)"
+
+  - id: tanh:ruby:v1
+    fn: core.tanh
+    inline: "= Math.tanh($0)"
+
   - id: mod.ruby:v1
     fn: core.mod
     inline: "= $0 % $1"
@@ -37,10 +61,14 @@ kernels:
     inline: "= $0 * $1"
     impl: "(str, count)\n  str * count"
 
+  # A negative base with a fractional exponent makes Ruby return a Complex,
+  # whereas JS Math.pow returns NaN. A numeric DSL should not silently produce
+  # complex numbers, so collapse the Complex case to NaN to match JS.
+  # Contract documented in docs/CROSS_TARGET_SEMANTICS.md.
   - id: pow:ruby:v1
     fn: core.pow
-    inline: "= $0 ** $1"
-    impl: "(a, b)\n  a ** b"
+    inline: "= (($0) ** ($1)).then { |r| r.is_a?(Complex) ? Float::NAN : r }"
+    impl: "(a, b)\n  r = a ** b\n  r.is_a?(Complex) ? Float::NAN : r"
 
   - id: div:ruby:v1
     fn: core.div

data/data/kernels/ruby/core/coercion.yaml

@@ -18,3 +18,11 @@ kernels:
     fn: core.to_string
     inline: "= $0.to_s"
     impl: "(value)\n  value.to_s"
+
+  # Float#to_s already renders Ruby's canonical float form ("3.0", "1.0e+21"),
+  # which IS the cross-target contract; the JS kernel reproduces it exactly.
+  # See docs/CROSS_TARGET_SEMANTICS.md.
+  - id: to_string_float:ruby:v1
+    fn: core.to_string:float
+    inline: "= $0.to_s"
+    impl: "(value)\n  value.to_s"

data/docs/CROSS_TARGET_SEMANTICS.md

@@ -0,0 +1,99 @@
+# Cross-Target Semantics
+
+Kumi compiles one schema to **both Ruby and JavaScript**, and the central promise
+is that the two targets compute **identical results**. Most operations are
+identical for free (arithmetic, comparisons, array iteration). A handful are not,
+because the two host languages disagree on edge-case behavior. This document is
+the single place that records those cases and where each is pinned.
+
+## Where the contract lives
+
+Per-operation semantics live in **kernels**, one YAML entry per `(function,
+target)`:
+
+```
+data/kernels/ruby/**/*.yaml         # Ruby implementation of each function
+data/kernels/javascript/**/*.yaml   # JavaScript implementation of each function
+```
+
+A kernel has an `inline` form (expanded at the call site) and/or an `impl` form
+(a named helper function emitted once per module). When Ruby and JS would
+otherwise diverge, the kernels are written so **both produce the agreed result**,
+and the YAML carries a comment pointing back here.
+
+This is the configurable seam: changing a cross-target rule means editing the two
+kernels (and the golden snapshots), not the compiler. Nothing in the analyzer or
+emitters hard-codes these semantics.
+
+> The Ruby and JS Loop emitters both support `impl`-only kernels (a kernel with
+> no `inline` is emitted once as a module-level helper, `__<fn_id>(...)`), so a
+> non-trivial rule can be a clean named function on both targets instead of an
+> inline blob. See `codegen/loop/{ruby,js}/emitter.rb`.
+
+## The pinned cases
+
+### `to_string` of a float — keep the `.0`
+
+A whole-valued float must stringify the same way on both targets.
+
+| value     | Ruby `to_s` | JS `String()` | Kumi (both) |
+| --------- | ----------- | ------------- | ----------- |
+| `3.0`     | `"3.0"`     | `"3"`         | `"3.0"`     |
+| `100.0`   | `"100.0"`   | `"100"`       | `"100.0"`   |
+| `-0.0`    | `"-0.0"`    | `"0"`         | `"-0.0"`    |
+| `1e21`    | `"1.0e+21"` | `"1e+21"`     | `"1.0e+21"` |
+| `1e-7`    | `"1.0e-07"` | `"1e-7"`      | `"1.0e-07"` |
+
+The contract is **Ruby's `Float#to_s`** (the float dtype stays visible in the
+string). JS cannot distinguish `3` (integer) from `3.0` (float) at runtime, so
+this is a **dtype-dispatched overload**: `core.to_string:float` (param
+`dtype: float`) is selected for float arguments by overload resolution, and its
+JS kernel reproduces Ruby's formatting (mantissa `.0`, signed zero-padded
+exponent, `-0.0` sign). Non-float `to_string` keeps the plain `String($0)`.
+
+Kernels: `to_string_float:ruby:v1`, `to_string_float:javascript:v1`.
+
+### `to_integer` / `to_float` of a string — Ruby parsing, not JS parsing
+
+`String#to_i` / `String#to_f` and JS `parseInt` / `parseFloat` disagree:
+
+| input      | Ruby `to_i` | JS `parseInt` | Kumi (both) |
+| ---------- | ----------- | ------------- | ----------- |
+| `"abc"`    | `0`         | `NaN`         | `0`         |
+| `""`       | `0`         | `NaN`         | `0`         |
+| `"0x1f"`   | `0`         | `31` (hex)    | `0`         |
+| `"12abc"`  | `12`        | `12`          | `12`        |
+
+The contract is **Ruby's** behavior: a non-numeric string is the type's zero
+(`0` / `0.0`), parsing is base-10 (no `0x` hex), and a numeric value truncates
+toward zero with `Math.trunc` (JS `parseInt(1e-7)` is the wrong `1`). The JS
+kernels are written to match.
+
+Kernels: `to_integer:javascript:v1`, `to_float:javascript:v1`.
+
+### `pow` of a negative base with a fractional exponent — NaN, not Complex
+
+`(-8.0) ** (1.0/3)` is mathematically complex. Ruby returns a `Complex`; JS
+`Math.pow` returns `NaN`. A numeric DSL should not silently produce complex
+numbers, so the **contract is `NaN`** (matching JS). The Ruby kernel collapses a
+`Complex` result to `Float::NAN`.
+
+Kernel: `pow:ruby:v1`.
+
+## Not divergences (documented to prevent false alarms)
+
+- **`mean` / `sum` / `min` / `max` of an empty array.** `mean([])` is `NaN` on
+  *both* targets (`0.0 / 0`). When probing via JSON, note `JSON.stringify(NaN)`
+  is `"null"`, so a Ruby `NaN` looks like a JS `null` through a JSON round-trip —
+  that is a serialization artifact, not a semantic difference.
+
+## Adding or changing a rule
+
+1. Decide the agreed result for both targets.
+2. Edit the Ruby kernel and the JS kernel so each produces it (add a `# see
+   docs/CROSS_TARGET_SEMANTICS.md` comment and a row in this file).
+3. Regenerate the golden snapshots: `bin/kumi golden_v2 update`. The
+   `runtime.json` of any affected golden should change only where the old
+   behavior was actually wrong.
+4. Add a parity case (Ruby vs generated-JS over edge inputs) so the contract is
+   tested, not just documented.

data/docs/FORM_SCHEMA.md

@@ -52,13 +52,18 @@ bin/kumi analyze schema.kumi --dump input_form_schema
 
 **Schema:**
 ```kumi
-input items: array {
-  item: hash {
-    name: string
-    price: float
-  }
-}
-input discount: float
+schema do
+  input do
+    array :items do
+      hash :item do
+        string :name
+        float :price
+      end
+    end
+
+    float :discount
+  end
+end
 ```
 
 **Generated Form Schema:**
@@ -82,4 +87,4 @@ input discount: float
 
 ## Implementation
 
-The `InputFormSchemaPass` runs after `InputCollector` in the analyzer pipeline and produces a clean schema containing only type information needed for form generation, excluding internal metadata like access modes and navigation steps.
+The `InputFormSchemaPass` runs after `InputCollectorPass` in the analyzer pipeline and produces a clean schema containing only type information needed for form generation, excluding internal metadata like access modes and navigation steps.