kumi 0.0.13 → 0.0.14

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7af4bdb11ef5da5a25b799cef8387752636a8224634c70656efc203b9d923185
-  data.tar.gz: 4e604513e42dcb8754fab54ead4140d5ca2d8443ae4d0461e62dc3b48bb925d4
+  metadata.gz: ec64db5a7b14df1a8656e2cedc569153a0f092d69a99544c5aa103241f61d931
+  data.tar.gz: 84bd4c7360eb74364e47103901fe8ccbcf6282a9cb40111d42b05357a2172c9e
 SHA512:
-  metadata.gz: dcbd93268081ff8f0bcf112101603b180932196b1d6fcb59aae2f363d6bbfe4e9ea07cf3cbe811290222dcd15cdf132ea04f409bea34f81599412f729211a5fa
-  data.tar.gz: 3e0c2e91609e742c2852457b2dd61a114a03b988ae983e0cfb23c9bbf46bc5c6c1714636961ed46b4b9ae924c6a77a79dc0aab9013cadbb73e49d5374f8b8dbc
+  metadata.gz: da7a9f3135e63779676ea5a802f48288ff2675935ab8dd8dcc751aaf309392275589f08fb92fede4dce351707369b5113e28540e5f4853629a02f4a1945ca75a
+  data.tar.gz: 5a45952056f81d9b4551bc7b48d3fd35c25065eb058e3cd6737879377a5c6e5f61d1cbba5b51955f54ad6ca14a797254e3381c69cd6b9eb145f3929e6e962dae

data/.rspec

@@ -1,3 +1,2 @@
---format documentation
 --color
 --require spec_helper

data/BACKLOG.md

@@ -0,0 +1,34 @@
+# Kumi Development Backlog
+
+## High Priority
+
+### RegistryV2 Migration Cleanup
+- [ ] Remove legacy `lib/kumi/core/function_registry.rb` and related modules after RegistryV2 is fully integrated
+- [ ] Clean up bridge code in `FunctionSignaturePass` once all functions are migrated
+- [ ] Remove `create_basic_signature` fallback logic
+- [ ] Update all tests to use RegistryV2 functions
+
+## Medium Priority
+
+### Function Registry Enhancements
+- [ ] Add YAML linting for function definitions
+- [ ] Implement golden-IR test fixtures for signature resolution
+- [ ] Add Arrow/JS kernel mapping support
+- [ ] Implement type promotion rules from YAML dtypes
+
+### Performance Optimizations
+- [ ] Cache RegistryV2 loading for repeated schema compilations
+- [ ] Optimize signature resolution for large function sets
+- [ ] Implement lazy loading of function kernels
+
+## Low Priority
+
+### Documentation
+- [ ] Auto-generate function reference docs from YAML
+- [ ] Add examples for each function in registry
+- [ ] Document kernel implementation guidelines
+
+### Testing
+- [ ] Add comprehensive NEP-20 compliance test suite
+- [ ] Stress test RegistryV2 with large function sets
+- [ ] Add performance benchmarks for signature resolution

data/CLAUDE.md

@@ -1,11 +1,9 @@
 # CLAUDE.md
 
-!! Important:
-!! Remember, this gem is not on production yet, so no backward compatilibity is necessary. But do not change the public interfaces (e.g. DSL, Schema) without explicitly requested or demanded. 
-!! We are using zeitwerk, i.e.: no requires
-!! Disregard linting or coverage issues unless asked to do so.
-!! Communication style - Write direct, factual statements. Avoid promotional language, unnecessary claims, or marketing speak. State what the system does, not what benefits it provides. Use TODOs for missing information rather than placeholder claims.
-!! See all Available Functions in docs/FUNCTIONS.md
+IMPORTANT!: Use `kumi parse` to see IR or Analysis States Diff - see docs/dev/parse-command.md
+Remember, this gem is not on production yet, so no backward compatilibity is necessary. But do not change the public interfaces (e.g. DSL, Schema) without explicitly requested or demanded. 
+We are using zeitwerk, i.e.: no requires
+See all Available Functions in docs/FUNCTIONS.md
 
 ## Project Overview
 

data/README.md

@@ -436,24 +436,6 @@ runner[:after_tax]     # => 196,844.80 (cached)
 - Sequential procedural workflows  
 - High-frequency processing
 
-## JavaScript Transpiler (V 0.0.10)
-Note: On the current 0.0.11 this is disabled but will be back in later versions. reason: IR/Interpreter update.
-Transpiles compiled schemas to standalone JavaScript code. See [docs/features/javascript-transpiler.md](docs/features/javascript-transpiler.md) for details.
-
-```ruby
-Kumi::Js.export_to_file(FederalTax2024, "federal-tax-2024.js")
-```
-
-```javascript
-const { schema } = require('./federal-tax-2024.js');
-const calculator = schema.from({ income: 100_000, filing_status: "single" });
-console.log(calculator.fetch('total_tax'));   // 21491
-```
-
-Generated JavaScript includes only functions used by the schema.
-
-All tests run in dual mode to verify compiled schemas produce identical results in both Ruby and JavaScript.
-
 ## Performance
 
 Benchmarks on Linux with Ruby 3.3.8 on a Dell Latitude 7450:

data/config/functions.yaml

@@ -0,0 +1,352 @@
+# ==================== CORE - SCALAR (elementwise) ====================
+
+- name: core.add
+  domain: core
+  opset: 1
+  class: scalar
+  summary: Numeric addition with broadcasting
+  signature: ["(),()->()", "(i),(i)->(i)", "(i,j),(i,j)->(i,j)"]
+  type_vars: {T: Numeric, U: Numeric}
+  dtypes: {result: "promote(T,U)", casting: "same_kind"}
+  null_policy: propagate
+  algebra: {associative: true, commutative: true, identity: 0}
+  vectorization: {mask_aware: true}
+  kernels: 
+    - backend: ruby
+      impl: kumi_add
+      priority: 10
+
+- name: core.sub
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)", "(i,j),(i,j)->(i,j)"]
+  type_vars: {T: Numeric, U: Numeric}
+  dtypes: {result: "promote(T,U)", casting: "same_kind"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: kumi_sub
+      priority: 10
+
+- name: core.mul
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)", "(i,j),(i,j)->(i,j)"]
+  type_vars: {T: Numeric, U: Numeric}
+  dtypes: {result: "promote(T,U)", casting: "same_kind"}
+  null_policy: propagate
+  algebra: {associative: true, commutative: true, identity: 1, annihilator: 0}
+  kernels:
+    - backend: ruby
+      impl: kumi_mul
+      priority: 10
+
+- name: core.div
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)", "(i,j),(i,j)->(i,j)"]
+  type_vars: {T: Numeric, U: Numeric}
+  dtypes: {result: "promote_float(T,U)", casting: "same_kind"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: kumi_div
+      priority: 10
+
+- name: core.eq
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)", "(i,j),(i,j)->(i,j)"]
+  type_vars: {T: Any, U: Any}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  algebra: {idempotent: true}
+  kernels:
+    - backend: ruby
+      impl: kumi_eq
+      priority: 10
+
+- name: core.gt
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {T: Orderable, U: Orderable}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: kumi_gt
+      priority: 10
+
+- name: core.gte
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {T: Orderable, U: Orderable}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: kumi_gte
+      priority: 10
+
+- name: core.and
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {B1: Bool, B2: Bool}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  algebra: {associative: true, commutative: true, identity: true, annihilator: false}
+  vectorization: {short_circuit: left_to_right, mask_aware: true}
+  kernels:
+    - backend: ruby
+      impl: kumi_and
+      priority: 10
+
+- name: core.or
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {B1: Bool, B2: Bool}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  algebra: {associative: true, commutative: true, identity: false, annihilator: true}
+  vectorization: {short_circuit: left_to_right, mask_aware: true}
+  kernels:
+    - backend: ruby
+      impl: kumi_or
+      priority: 10
+
+- name: core.not
+  domain: core
+  opset: 1
+  class: scalar
+  signature: ["()->()", "(i)->(i)"]
+  type_vars: {B: Bool}
+  dtypes: {result: "bool"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: kumi_not
+      priority: 10
+
+# ==================== AGGREGATES (reduce last axis in signature) ==============
+
+- name: agg.sum
+  domain: core
+  opset: 1
+  class: aggregate
+  signature: ["(i)->()", "(i,j)->(i)"]
+  type_vars: {T: Numeric}
+  dtypes: {result: "promote(T)"}
+  null_policy: skip
+  options: {min_count: 0}
+  algebra: {associative: true, commutative: true, identity: 0}
+  kernels:
+    - backend: ruby
+      impl: kumi_sum
+      priority: 10
+
+- name: agg.min
+  domain: core
+  opset: 1
+  class: aggregate
+  signature: ["(i)->()", "(i,j)->(i)"]
+  type_vars: {T: Orderable}
+  dtypes: {result: "T"}
+  null_policy: skip
+  kernels:
+    - backend: ruby
+      impl: kumi_min
+      priority: 10
+
+- name: agg.max
+  domain: core
+  opset: 1
+  class: aggregate
+  signature: ["(i)->()", "(i,j)->(i)"]
+  type_vars: {T: Orderable}
+  dtypes: {result: "T"}
+  null_policy: skip
+  kernels:
+    - backend: ruby
+      impl: kumi_max
+      priority: 10
+
+- name: agg.mean
+  domain: core
+  opset: 1
+  class: aggregate
+  signature: ["(i)->()", "(i,j)->(i)"]
+  type_vars: {T: Numeric}
+  dtypes: {result: "float"}
+  null_policy: skip
+  options: {min_count: 0}
+  kernels:
+    - backend: ruby
+      impl: kumi_mean
+      priority: 10
+
+# ==================== STRUCTURE / SHAPE / JOIN ================================
+
+- name: struct.join
+  domain: struct
+  opset: 1
+  class: structure
+  summary: Align inputs by axes using zip or product
+  signature: ["(i),(i)->(i)@zip", "(i),(j)->(i,j)@product"]
+  null_policy: error_on_missing
+  kernels:
+    - backend: ruby
+      impl: join_zip
+      when: {policy: zip}
+      priority: 10
+    - backend: ruby
+      impl: join_product
+      when: {policy: product}
+      priority: 10
+
+- name: struct.align_to
+  domain: struct
+  opset: 1
+  class: structure
+  signature: ["(i)->(i)", "(i,j)->(i,j)"]
+  traits: {reorders: false}
+  null_policy: error_on_missing
+  kernels:
+    - backend: ruby
+      impl: align_to
+      priority: 10
+
+- name: struct.lift
+  domain: struct
+  opset: 1
+  class: structure
+  signature: ["(i)->(i)", "(i,j)->(i,j)"]
+  summary: Group flat rows back to nested collections based on indices
+  kernels:
+    - backend: ruby
+      impl: lift
+      priority: 10
+
+- name: struct.flatten
+  domain: struct
+  opset: 1
+  class: structure
+  signature: ["(i,j)->(k)"]
+  shape_fn: "flatten_axes(i,j)->k"
+  kernels:
+    - backend: ruby
+      impl: flatten
+      priority: 10
+
+- name: struct.take
+  domain: struct
+  opset: 1
+  class: structure
+  signature: ["(i),(i)->(i)"]  # take(values, indices) elemwise
+  type_vars: {T: Any}
+  dtypes: {result: "T"}
+  null_policy: error
+  kernels:
+    - backend: ruby
+      impl: take
+      priority: 10
+
+- name: struct.size
+  domain: struct
+  opset: 1
+  class: vector
+  signature: ["(i)->()", "(i,j)->()"]
+  dtypes: {result: "int"}
+  null_policy: error
+  kernels:
+    - backend: ruby
+      impl: size
+      priority: 10
+
+# ==================== MASK / CONDITIONAL ======================================
+
+- name: mask.where
+  domain: mask
+  opset: 1
+  class: scalar
+  summary: Ternary select with broadcasting of then/else
+  signature:
+    - "(i),(i),(i)->(i)"
+    - "(i),(),()->(i)"
+    - "(),(i),()->(i)"
+    - "(),(),()->()"
+  type_vars: {B: Bool, T: Any, U: Any}
+  dtypes: {result: "unify(T,U)"}
+  null_policy: propagate
+  vectorization: {mask_aware: true, short_circuit: left_to_right}
+  kernels:
+    - backend: ruby
+      impl: where
+      priority: 10
+
+# ==================== STRING (minimal) ========================================
+
+- name: string.concat
+  domain: string
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {S1: String, S2: String}
+  dtypes: {result: "string"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: str_concat
+      priority: 10
+
+- name: string.length
+  domain: string
+  opset: 1
+  class: vector
+  signature: ["(i)->(i)", "()->()"]
+  type_vars: {S: String}
+  dtypes: {result: "int"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: str_length
+      priority: 10
+
+# ==================== DATETIME (minimal) ======================================
+
+- name: datetime.add_days
+  domain: datetime
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {D: DateLike, N: IntLike}
+  dtypes: {result: "datetime"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: dt_add_days
+      priority: 10
+
+- name: datetime.diff_days
+  domain: datetime
+  opset: 1
+  class: scalar
+  signature: ["(),()->()", "(i),(i)->(i)"]
+  type_vars: {D1: DateLike, D2: DateLike}
+  dtypes: {result: "int"}
+  null_policy: propagate
+  kernels:
+    - backend: ruby
+      impl: dt_diff_days
+      priority: 10

data/docs/dev/analyzer-debug.md

@@ -0,0 +1,52 @@
+# Analyzer Debug System
+
+## Debug Module
+
+**File**: `lib/kumi/core/analyzer/debug.rb`
+
+**Enable**: `KUMI_DEBUG_STATE=1`
+
+**Output**: JSONL to stdout with state diffs and timing per pass
+
+**Configuration**:
+- `KUMI_DEBUG_REQUIRE_FROZEN=1` - Enforce state immutability checks
+
+## Checkpoint System
+
+**Files**: 
+- `lib/kumi/core/analyzer/checkpoint.rb`
+- `lib/kumi/core/analyzer/state_serde.rb`
+
+**Enable**: `KUMI_CHECKPOINT=1`
+
+**Configuration**:
+- `KUMI_CHECKPOINT_DIR=path` - Output directory (default: `/tmp/kumi_checkpoints`)
+- `KUMI_CHECKPOINT_FORMAT=marshal|json|both` - File format (default: `marshal`)
+- `KUMI_CHECKPOINT_PHASE=before|after|both` - When to save (default: `both`)
+
+**Resume/Stop**:
+- `KUMI_RESUME_FROM=file.msh` - Resume from checkpoint file
+- `KUMI_RESUME_AT=PassName` - Skip to specific pass
+- `KUMI_STOP_AFTER=PassName` - Stop after specific pass
+
+## Object Printers
+
+**File**: `spec/support/debug_printers.rb`
+
+Handles clean output for debug logs. Add new object types here when they appear as `#<Object:0x...>` in debug output.
+
+## Usage
+
+```bash
+# Basic debug
+KUMI_DEBUG_STATE=1 bundle exec ruby script.rb
+
+# Debug with checkpoints
+KUMI_DEBUG_STATE=1 KUMI_CHECKPOINT=1 bundle exec ruby script.rb
+
+# Resume from checkpoint
+KUMI_RESUME_FROM=/tmp/kumi_checkpoints/005_TypeChecker_after.msh bundle exec ruby script.rb
+
+# Stop at specific pass for debugging
+KUMI_STOP_AFTER=BroadcastDetector bundle exec ruby script.rb
+```

data/docs/dev/parse-command.md

@@ -0,0 +1,64 @@
+# Parse Command
+
+One-command developer loop for text schemas: parse → analyze → render IR → diff.
+
+## Usage
+
+```bash
+# Default: show diff vs golden file (or print if no golden)
+kumi parse golden/simple_math/schema.kumi
+
+# Write/overwrite golden file  
+kumi parse golden/simple_math/schema.kumi --write
+
+# Update golden only if different
+kumi parse golden/simple_math/schema.kumi --update
+
+# JSON format instead of text
+kumi parse golden/simple_math/schema.kumi --json --write
+
+# Enable state tracing
+kumi parse golden/simple_math/schema.kumi --trace
+
+# Snapshot analysis passes
+kumi parse golden/simple_math/schema.kumi --snap after --snap-dir debug/
+```
+
+## Exit Codes
+
+- `0` - No changes or successful write
+- `1` - Diff mismatch or analysis error
+
+## Output
+
+**No changes:**
+```
+No changes (golden/simple_math/expected/ir.txt)
+```
+
+**Diff mismatch:**
+```
+--- expected
++++ actual
+@@ -6,7 +6,7 @@
+   2: map argc=2 fn=:add [0, 1]
++  2: map argc=2 fn=:power [0, 1]
+```
+
+**Write mode:**
+```
+Wrote: golden/simple_math/expected/ir.txt
+```
+
+## IR Format
+
+Deterministic text representation:
+```
+IR Module
+decls: 3
+decl[0] value:sum shape=scalar ops=4
+  0: load_input has_idx=false is_scalar=true plan_id=x:read scope=[] []
+  1: load_input has_idx=false is_scalar=true plan_id=y:read scope=[] []
+  2: map argc=2 fn=:add [0, 1]
+  3: store name=:sum [2]
+```