kumi 0.0.18 → 0.0.19

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f48b3c1d3dc46f7725b1f138fd4eafd2f1c197ed9bdcc0465a26206ed0b3ec56
-  data.tar.gz: 758a5ed295dda7ab9ca97f8caa888b502c0485abfb80ad13ccf6e0a0c03a716e
+  metadata.gz: fdbfa79ff2c48eb20c3d54fbae9ff658ea1227331444aa6b6079e71a0b24b9ce
+  data.tar.gz: d3e732f5d9a334fd376f351dc4dcf4a3dde7ad0a24694d9c845826d4602baa8d
 SHA512:
-  metadata.gz: ebdd2e94fe3e3146018fbb4487601b8cbb5c10b1574c3fe7858cc230d8b69f1884d4bab7e8f9f352523754e4a6f1ed39d512659812569248f41584a4c692b341
-  data.tar.gz: 3d588dac5c01a4c6773d43383b3f0ed54ea21c43bf6d8aeefbc933518b74d00cd2b5b3b02cac5ce86cf633d930c4733c8a55d851e770063325b4a94d56f64d15
+  metadata.gz: 53c9e49c36dace2082d2abfbe4b8754ad8b0fb347bd2f2b3aa38eb29e097fb9669b3499846ebd4ceb81867e2ff679d2e1b2181ce716be5f7fdf793c60ae39095
+  data.tar.gz: 9addd404751ca0c864a047224cc602060ff4083ff2750609f7d01d0731f0c976d35a9acafa15f3c41fae91805304e79c971886b5a165cf7b016cc96017d463e9

data/.rubocop.yml

@@ -115,4 +115,7 @@ RSpec/ExpectActual:
   Enabled: true   # Keep this - it's a good practice
 
 RSpec/IdenticalEqualityAssertion:
-  Enabled: false  # Sometimes needed for testing type systems
+  Enabled: false  # Sometimes needed for testing type systems
+
+RSpec/PendingWithoutReason:
+  Enabled: false

data/.ruby-version

@@ -0,0 +1 @@
+3.4.5

data/CHANGELOG.md

@@ -1,5 +1,30 @@
 ## [Unreleased]
 
+## [0.0.19] – 2025-10-13
+### Added
+- **Ruby & JavaScript Code Generation:** The compiler now directly generates clean, idiomatic Ruby and JavaScript (MJS) code from Kumi schemas, removing the need for a complex runtime interpreter.
+- **IRv2:** A new, backend-agnostic intermediate representation (IR) that simplifies the compilation process and improves performance.
+- **New Language Features:**
+    - First-class `select` expressions for conditional logic.
+    - `fold` expressions for reductions over tuples.
+    - Hash expressions for creating and manipulating hash maps.
+    - Tuple-based syntax for array literals.
+- **Tooling:**
+    - `InputFormSchemaPass` for generating minimal form schemas from Kumi files.
+    - `OutputSchemaPass` for extracting output metadata.
+
+### Changed
+- **Compiler Architecture:** The compiler has been redesigned around the new IRv2 and code generation backend.
+- **Syntax:** Array literals now use a tuple-based syntax `(1, 2, 3)` instead of `[1, 2, 3]`.
+
+### Removed
+- **Legacy Runtime:** The old, interpreter-based runtime and legacy FunctionRegistry have been removed in favor of direct code generation.
+- **`align_to`:** The `align_to` operator has been removed from the language.
+
+### Performance
+- **Reduced Overhead:** By generating direct Ruby/JavaScript code, the runtime overhead of the interpreter is eliminated, resulting in significant performance improvements.
+- **Optimizations:** New optimization passes, including loop fusion and inlining, have been added to the compiler.
+
 ## [0.0.18] – 2025-09-03
 - Fixed bug missing updated Gemfile.lock
 

data/CLAUDE.md

@@ -1,154 +1,41 @@
-# CLAUDE.md
+# Important
+Ignore Linting issues for now.
+Dont invent backward compatibility unless that is the plan.
 
-IMPORTANT!: Use `kumi parse` to see IR or Analysis States Diff - see docs/dev/parse-command.md
-Remember, this gem is not on production yet, so no backward compatilibity is necessary. But do not change the public interfaces (e.g. DSL, Schema) without explicitly requested or demanded. 
-We are using zeitwerk, i.e.: no requires
-See all Available Functions in docs/FUNCTIONS.md
+# Vector Semantics
+Axes align by identity (lineage), not by name
 
-## Project Overview
+# Analyzer
+> lib/kumi/analyzer.rb
+> lib/kumi/core/analyzer/passes/pass_base.rb
 
-Kumi is a Declarative logic and rules engine framework with static analysis for Ruby.
+# For a schema.kumi
+- To view a IRV2 `bin/kumi pp irv2 <schema.kumi>`
 
-## Development Commands
+# Golden tests
+- First udpate `bin/kumi golden update <empty for all | golden_name>` then `bin/kumi golden test <same>`
 
-### Testing
-- `bundle exec rspec` - Run all tests
-- `bundle exec rspec spec/path/to/specific_spec.rb` - Run specific test file
-- `bundle exec rspec spec/path/to/specific_spec.rb:123` - Run specific test at line
 
-## Architecture Overview
 
-### Core Components
+# Debug Tools:
+`bin/kumi pp <ast|nast|snast|ir> <schema>` - Pretty print representations
+`bin/kumi analyze <schema> --dump <state_key>` - Dump analyzer state (call_table, declaration_table, snast_module, etc.)
+`bin/kumi golden list` - List all golden test schemas
+`bin/kumi golden record [name]` - Record expected representations
+`bin/kumi golden verify [name]` - Verify current vs expected
+`bin/kumi golden diff <name>` - Show diffs when verification fails
 
-**Schema System** (`lib/kumi/schema.rb`):
-- Entry point that ties together parsing, analysis, and compilation
-- DSL method `schema(&block)` builds the syntax tree, runs analysis, and compiles to executable form
-- Generates a `Runner` instance for executing queries against input data
+# Kernels Invariants
+All reducers are pure binary combiner f : T × T → T applied over the last axis of a value. Example: agg.sum(a,b) = a+b.
 
-**Parser** (`lib/kumi/ruby_parser{/*,.rb}`):
 
-**Syntax Tree** (`lib/kumi/syntax/`):
-- `node.rb` - Base node class with location tracking
-- `root.rb` - Root schema node containing inputs, attributes, and traits
-- `value_declaration.rb` - Value declaration nodes (formerly Attribute)
-- `trait_declaration.rb` - Trait declaration nodes (formerly Trait)
-- `input_declaration.rb` - Input field declaration nodes (formerly FieldDecl)
-- `call_expression.rb` - Function call expression nodes
-- `array_expression.rb` - Array expression nodes (formerly ListExpression)
-- `hash_expression.rb` - Hash expression nodes (for future hash literals) (currently not used)
-- `cascade_expression.rb` - Cascade expression nodes (conditional values)
-- `case_expression.rb` - Case expression nodes (formerly WhenCaseExpression)
-- `literal.rb` - Literal value nodes
-- `input_reference.rb` - Input field reference nodes (formerly FieldRef) 
-- `input_element_reference.rb` - Reference to nested input field (array -> obj.field) 
-- `declaration_reference.rb` - Declaration reference value or trait nodes
+# KernelRegistry:
+You can use the KernelRegistry like this:
+> registry = Kumi::KernelRegistry.load_ruby
+> registry.impl_for("agg.sum:ruby:v1")
+=> "->(a,b) {a + b}"
 
-**Analyzer** (`lib/kumi/analyzer.rb`):
-- Multi-pass analysis system that validates schemas and builds dependency graphs
-- **Pass 1**: `name_indexer.rb` - Find all names, check for duplicates
-- **Pass 2**: `input_collector.rb` - Collect field metadata, validate conflicts
-- **Pass 3**: `definition_validator.rb` - Validate basic structure
-- **Pass 4**: `dependency_resolver.rb` - Build dependency graph
-- **Pass 5**: `cycle_detector.rb` - Find circular dependencies
-- **Pass 6**: `toposorter.rb` - Create evaluation order
-- **Pass 7**: `type_inferencer.rb` - Infer types for all declarations
-- **Pass 8**: `type_checker.rb` - Validate function types and compatibility using inferred types
 
-**Compiler** (`lib/kumi/compiler.rb`):
-- Compiles analyzed syntax tree into executable lambda functions
-- Maps each expression type to a compilation method
-- Handles function calls via `Kumi::Registry`
-- Produces `CompiledSchema` with executable bindings
 
-**Function Registry** (`lib/kumi/function_registry.rb`):
-- Registry of available functions (operators, math, string, logical, collection operations)
-- Supports custom function registration with type metadata
-- Each function includes param_types, return_type, arity, and description
-- Core functions include: `==`, `>`, `<`, `add`, `multiply`, `and`, `or`, `clamp`, etc.
-- Function documents are generated by the script ./scripts/generate_function_docs.rb
 
-### Key Patterns
-**DSL Structure**:
-```ruby
-schema do
-  input do
-    # Recommended type-specific DSL methods
-    string  :field_name
-    integer :number_field, domain: 0..100
-    array   :scores, elem: { type: :float }
-    hash    :metadata, key: { type: :string }, val: { type: :any }
 
-    array :line_items do
-      float   :price
-      integer :quantity
-      string  :category
-    end
-
-    # Fields with no declared type
-    any     :misc_field
-  end
-
-  trait :name, (expression)  # Boolean conditions
-  value :name, expression    # Computed values
-  value :name do             # Conditional logic
-    on condition, result     # on <trait> ?,<trait> , <expr>
-    base default_result      # base <expr>
-  end
-end
-```
-
-**IMPORTANT CASCADE CONDITION SYNTAX:**
-In cascade expressions (`value :name do ... end`), trait references use bare identifiers:
-
-
-**Expression Types**:
-- `input.field_name` - Access input data with operator methods (>=, <=, >, <, ==, !=)
-- `ref(:name)` - Reference other declarations
-- `fn(:name, args...)` - Function calls
-- `(expr1) & (expr2)` - Logical AND chaining
-- `[element1, element2]` - Lists
-- Literals (numbers, strings, booleans)
-
-**Analysis Flow**:
-1. Parse DSL → Syntax Tree
-2. Analyze Syntax Tree → Analysis Result (dependency graph, type information, topo order)
-3. Compile → Executable Schema  
-4. Execute with Runner
-
-**Type System** (`lib/kumi/types.rb`):
-- Symbol-based type system
-- **Dual Type System**: Declared types (from input blocks) and inferred types (from expressions)
-- Type inference for all declarations based on expression analysis
-- Type primitives: `:string`, `:integer`, `:float`, `:boolean`, `:any`, `:symbol`, `:regexp`, `:time`, `:date`, `:datetime`
-- Collection types: `array(:element_type)` and `hash(:key_type, :value_type)` helper functions
-
-## Files for Understanding
-
-. `docs/*` - Documents about Kumi, its features, DSL syntax, ... 
-- `examples/*` Random examples of diverse contexts.
-
-### Troubleshooting Schema Issues
-DEBUG, DEBUG. DEBUG LOGS!
-
-
-### Array Broadcasting System
-
-**Vectorization**: Field access on array inputs (`input.items.price`) applies operations element-wise with map/reduce detection.
-
-**Basic Broadcasting**:
-```ruby
-input do
-  array :line_items do
-    float   :price
-    integer :quantity  
-    string  :category
-    array   :prices do
-      element :integer, :val
-    end
-  end
-end
-
-# Element-wise computation - broadcasts over each item
-value :subtotals, input.line_items.price * input.line_items.quantity
-trait :is_taxable, (input.line_items.category != "digital")
-```