kumi 0.0.9 → 0.0.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c5f9f087001a482c906f041da8cb229511966a11c56e3ced99930b59d4141543
-  data.tar.gz: fe5fafe03a5ca1e414b5bc3af9eccd830553aa8f44762e9d9b38e589fa342fd1
+  metadata.gz: 3eb46e14716bf14c3d9165ffac957211f41ad5a21a74ad37c47b37c37e01b312
+  data.tar.gz: fd0e36d65ac41079c27cf9699a3e092ff762bc76b3af8fc90df2ec763fed3806
 SHA512:
-  metadata.gz: b26279c08a9387616104252d24147c920f27b0f66574cde1987024325c5fd1ba87aff876b39fdc3d4f472d9aba2c713b19a5105890b66ac3ef10a3018a0b1bca
-  data.tar.gz: d4c8db755880cd7088fd4b6087d2f38e86a4fdee11f867d58a480e143494ca2b85440abd9249b8c02a469a9ad21642c287d5ef8f30948bdc6d67695a895c1c13
+  metadata.gz: 54ed5e72d6acf863e0f7ed0986c8db83fab22526bcffb97b4c1b96343e724b0ae505804baed5554933c452478ced8c46ec24a6568426813e69c4007994588de6
+  data.tar.gz: '09aabb772643aab71d957060c934f05be5838a1056b7c635822b75759106a3119844cc7e20f96b8990ec658d969f2d2e0143ceb43d36f8d48da061c2bb80cb6a'

data/CHANGELOG.md

@@ -0,0 +1,18 @@
+## [0.0.11] – 2025-08-13
+### Added
+- Intermediate Representation (IR) and slot-based VM interpreter.
+- Scope-aware vector semantics (alignment, lift, hierarchical indices).
+- Debug tooling: IR dump, VM/lowering traces via DEBUG_* flags.
+
+### Changed
+- Analyzer now lowers to IR via `LowerToIRPass`.
+- Access modes: `:read`, `:ravel`, `:each_indexed`, `:materialize`.
+
+### Removed (BREAKING)
+- JavaScript transpiler (legacy compiler).
+
+### Requirements
+- Ruby >= 3.1 (Was >= 3.0)
+
+### Notes
+- No expected DSL changes for typical schemas; report regressions.

data/CLAUDE.md

@@ -1,9 +1,11 @@
 # CLAUDE.md
 
+!! Important:
 !! Remember, this gem is not on production yet, so no backward compatilibity is necessary. But do not change the public interfaces (e.g. DSL, Schema) without explicitly requested or demanded. 
 !! We are using zeitwerk, i.e.: no requires
-!! Do not care about linter or coverage unless asked to do so.
-!! IMPORTANT: Communication style - Write direct, factual statements. Avoid promotional language, unnecessary claims, or marketing speak. State what the system does, not what benefits it provides. Use TODOs for missing information rather than placeholder claims.
+!! Disregard linting or coverage issues unless asked to do so.
+!! Communication style - Write direct, factual statements. Avoid promotional language, unnecessary claims, or marketing speak. State what the system does, not what benefits it provides. Use TODOs for missing information rather than placeholder claims.
+!! See all Available Functions in docs/FUNCTIONS.md
 
 ## Project Overview
 
@@ -16,18 +18,13 @@ Kumi is a Declarative logic and rules engine framework with static analysis for
 - `bundle exec rspec spec/path/to/specific_spec.rb` - Run specific test file
 - `bundle exec rspec spec/path/to/specific_spec.rb:123` - Run specific test at line
 
-### Gem Management
-- `bundle install` - Install dependencies
-- `gem build kumi.gemspec` - Build the gem
-- `gem install ./kumi-*.gem` - Install locally built gem
-
 ## Architecture Overview
 
 ### Core Components
 
 **Schema System** (`lib/kumi/schema.rb`):
 - Entry point that ties together parsing, analysis, and compilation
-- Provides the `schema(&block)` DSL method that builds the syntax tree, runs analysis, and compiles to executable form
+- DSL method `schema(&block)` builds the syntax tree, runs analysis, and compiles to executable form
 - Generates a `Runner` instance for executing queries against input data
 
 **Parser** (`lib/kumi/ruby_parser{/*,.rb}`):
@@ -60,71 +57,18 @@ Kumi is a Declarative logic and rules engine framework with static analysis for
 - **Pass 8**: `type_checker.rb` - Validate function types and compatibility using inferred types
 
 **Compiler** (`lib/kumi/compiler.rb`):
-- Transforms analyzed syntax tree into executable lambda functions
+- Compiles analyzed syntax tree into executable lambda functions
 - Maps each expression type to a compilation method
 - Handles function calls via `Kumi::Registry`
 - Produces `CompiledSchema` with executable bindings
 
 **Function Registry** (`lib/kumi/function_registry.rb`):
 - Registry of available functions (operators, math, string, logical, collection operations)
-- Supports custom function registration with comprehensive type metadata
+- Supports custom function registration with type metadata
 - Each function includes param_types, return_type, arity, and description
 - Core functions include: `==`, `>`, `<`, `add`, `multiply`, `and`, `or`, `clamp`, etc.
-- Maintains backward compatibility with legacy type checking system
 - Function documents are generated by the script ./scripts/generate_function_docs.rb
 
-**Runner** (`lib/kumi/runner.rb`):
-- Executes compiled schemas against input data
-- Provides `fetch(key)` for individual value retrieval with caching
-- Provides `slice(*keys)` for batch evaluation
-- Provides `explain(key)` for detailed execution tracing
-
-**Input Validation System** (`lib/kumi/input/` and `lib/kumi/domain/`):
-- `input/validator.rb` - Main validation coordinator for type and domain checking
-- `input/type_matcher.rb` - Type validation logic for primitive and complex types
-- `input/violation_creator.rb` - Creates standardized violation objects with detailed messages
-- `domain/validator.rb` - Domain constraint validation (ranges, arrays, procs)
-- `domain/range_analyzer.rb` - Range domain analysis and validation
-- `domain/enum_analyzer.rb` - Enumeration domain analysis and validation
-- `domain/violation_formatter.rb` - Formats domain violation error messages
-
-## DSL Syntax Requirements
-
-### Critical Syntax Rules
-
-**Module Definition Structure** (REQUIRED for CLI):
-```ruby
-# CORRECT - CLI can find and load this
-module SchemaName
-  extend Kumi::Schema
-  
-  schema do
-    # schema definition here
-  end
-end
-
-# INCORRECT - CLI cannot load standalone schemas
-schema do  # This won't work with CLI
-  # schema definition
-end
-```
-
-**Function Call Syntax**:
-- **Symbol style**: `fn(:function_name, arg1, arg2, ...)` - The only supported function call syntax 
-
-**Arithmetic Operations**:
-- **Sugar Syntax**: `input.field1 + input.field2` - Works for input fields and value references
-- **Function Syntax**: `fn(:add, input.field1, input.field2)` - Always works, more explicit
-- **Mixed**: Use sugar for simple operations, functions for complex ones
-
-**Cascade Condition Syntax**:
-```ruby
-value :status do
-  on trait_name, "Result"
-  base "Default"
-end
-```
-
 ### Key Patterns
 **DSL Structure**:
 ```ruby
@@ -142,38 +86,22 @@ schema do
       string  :category
     end
 
-
-
     # Fields with no declared type
     any     :misc_field
   end
 
-  trait :name, (expression)  # Boolean conditions with new syntax
+  trait :name, (expression)  # Boolean conditions
   value :name, expression    # Computed values
   value :name do             # Conditional logic
-    on condition, result
-    base default_result
+    on condition, result     # on <trait> ?,<trait> , <expr>
+    base default_result      # base <expr>
   end
 end
 ```
 
 **IMPORTANT CASCADE CONDITION SYNTAX:**
 In cascade expressions (`value :name do ... end`), trait references use bare identifiers:
-```ruby
-value :status do
-  on adult, "Adult Status"      
-  on verified, "Verified User"
-  base "Unverified"
-end
-```
 
-**Input Block System**:
-- **Required**: All schemas must have an `input` block declaring expected fields
-- **Type Declarations**: Preferred via type-specific methods (e.g. `integer :field`, `string :name`, `any :field` for untyped fields)
-- **Complex Types**: Use helper functions: `array(:element_type)` and `hash(:key_type, :value_type)`
-- **Domain Constraints**: Fields can have domains: `integer :age, domain: 18..65` (validated at runtime)
-- **Field Access**: Use `input.field_name` to reference input fields in expressions
-- **Separation**: Input metadata (types, domains) is separate from business logic
 
 **Expression Types**:
 - `input.field_name` - Access input data with operator methods (>=, <=, >, <, ==, !=)
@@ -190,86 +118,24 @@ end
 4. Execute with Runner
 
 **Type System** (`lib/kumi/types.rb`):
-- Simple symbol-based type system for clean and intuitive declaration
+- Symbol-based type system
 - **Dual Type System**: Declared types (from input blocks) and inferred types (from expressions)
-- Automatic type inference for all declarations based on expression analysis
+- Type inference for all declarations based on expression analysis
 - Type primitives: `:string`, `:integer`, `:float`, `:boolean`, `:any`, `:symbol`, `:regexp`, `:time`, `:date`, `:datetime`
 - Collection types: `array(:element_type)` and `hash(:key_type, :value_type)` helper functions
-- Type compatibility checking and unification algorithms for numeric types
-- Enhanced error messages showing type provenance (declared vs inferred)
-- Legacy compatibility constants maintained for backward compatibility
-
-### Examples Directory
-
-The `examples/` directory contains comprehensive examples showing Kumi usage patterns:
-- `cascade_demonstration.rb` - Demonstrates cascade logic with UnsatDetector fixes (working)
-- `working_comprehensive_schema.rb` - Complete feature showcase (current best practices, working)
-- Mathematical predicate examples - Safe mutual recursion patterns using cascade mutual exclusion
-- `federal_tax_calculator_2024.rb` - Real-world tax calculation example (working)
-- `tax_2024.rb` - Simple tax example with explain functionality (working)
-- `wide_schema_compilation_and_evaluation_benchmark.rb` - Performance benchmark for wide schemas (compilation and evaluation scaling)
-- `deep_schema_compilation_and_evaluation_benchmark.rb` - Performance benchmark for deep dependency chains (stack-safe evaluation)
-- `comprehensive_god_schema.rb` - Complex example (currently has UnsatDetector semantic errors)
-
-*Note: Some examples may use deprecated syntax and should be updated to use the new input block system.*
-
-## Test Structure
-
-- `spec/kumi/` - Unit tests for core components
-- `spec/integration/` - Integration tests for full workflows
-- `spec/fixtures/` - Test fixtures and sample schemas
-- `spec/support/` - Test helpers (`ast_factory.rb`, `schema_generator.rb`)
 
 ## Files for Understanding
 
 . `docs/*` - Documents about Kumi, its features, DSL syntax, ... 
-- `examples/*` Random examples of very diverse contexts.
+- `examples/*` Random examples of diverse contexts.
 
 ### Troubleshooting Schema Issues
-- **Parse Errors**: Check function syntax (avoid empty `fn()` calls)
-- **Module Not Found**: Ensure proper module structure and naming, see examples
-- **UnsatDetector Errors**: Review trait logic for contradictions, add debugs!
-- **Type Errors**: Check input block type declarations match usage, add debugs!
-- **Runtime Errors**: Use explain to trace computation dependencies, add debugs!
+DEBUG, DEBUG. DEBUG LOGS!
 
-## Input Block System Details
-
-### Required Input Blocks
-- **All schemas must have an input block** -
-- Input blocks declare expected fields with optional type and domain constraints
-- **Empty input blocks are allowed** -`input {}` Even if its not very useful.
-- Fields are accessed via `input.field_name` or `input.field.nested_field.nested_nested_field` which
-works for referencing nested array input declarations.
-
-### Type System Integration
-- **Declared Types**: Explicit type declarations in input blocks (e.g. `integer :field`, `string :name`, `any :field`)
-- **Inferred Types**: Types automatically inferred from expression analysis
-- **Type Checking**: Validates compatibility between declared and inferred types
-- **Enhanced Errors**: Error messages show type provenance (declared vs inferred)
-- **Helper Functions**: Use `array(:type)` and `hash(:key_type, :value_type)` for complex types
-
-### Parser Components
-See `lib/kumi/ruby_parser/parser.rb`
-
-### Domain Constraints
-- Can be declared: `integer :age, domain: 18..65`
-- Supports Range domains (`18..65`), Array domains (`%w[active inactive]`), and Proc domains for custom validation
-- Analyzer do some limited domain UNSAT detection, and its used to validated against input at Runtime
-### Type Examples
-```ruby
-input do
-  string       :name
-  integer      :age, domain: 18..65
-  hash         :metadata, key: { type: :string }, val: { type: :any }
-
-  #generic type
-  any          :misc # this will make Kumi lose most of its analyze/inference power
-end
-```
 
 ### Array Broadcasting System
 
-**Automatic Vectorization**: Field access on array inputs (`input.items.price`) applies operations element-wise with intelligent map/reduce detection.
+**Vectorization**: Field access on array inputs (`input.items.price`) applies operations element-wise with map/reduce detection.
 
 **Basic Broadcasting**:
 ```ruby
@@ -278,6 +144,9 @@ input do
     float   :price
     integer :quantity  
     string  :category
+    array   :prices do
+      element :integer, :val
+    end
   end
 end
 
@@ -285,112 +154,3 @@ end
 value :subtotals, input.line_items.price * input.line_items.quantity
 trait :is_taxable, (input.line_items.category != "digital")
 ```
-
-**Aggregation Operations**: Functions consuming arrays automatically detected:
-```ruby
-value :total_subtotal, fn(:sum, subtotals)
-value :avg_price, fn(:avg, input.line_items.price)
-value :max_quantity, fn(:max, input.line_items.quantity)
-```
-
-**Implementation Components**:
-- **InputElementReference** AST nodes for nested field access paths
-- **BroadcastDetector** analyzer pass identifies vectorized vs scalar operations  
-- **Compiler** generates appropriate map/reduce functions based on usage context
-- **Type Inference** automatically infers types for array element operations
-- Supports arbitrary depth field access with nested arrays and hashes
-
-### Trait Syntax Evolution
-
-**Current Syntax** (recommended):
-```ruby
-trait :adult, (input.age >= 18)
-trait :qualified, (input.age >= 21) & (input.score > 80) & (input.verified == true)
-```
-
-**Composite Trait Syntax** (NEW - bare identifier references):
-```ruby
-# Base traits
-trait :adult, (input.age >= 18)
-trait :verified, (input.verified == true)
-trait :high_score, (input.score > 80)
-
-# Composite traits using bare identifier syntax
-trait :eligible, adult & verified & high_score
-trait :mixed, adult & (input.income > 50_000) & verified
-
-# Backward compatibility - both syntaxes work together
-trait :legacy_mix, adult & ref(:verified) & (input.score > 90)
-```
-
-**Deprecated Syntax** (with warnings):
-```ruby
-trait :adult, input.age, :>=, 18                    # OLD - shows deprecation warning
-trait :qualified, input.age, :>=, 21, input.score  # OLD - shows deprecation warning
-```
-
-**Key Changes**:
-- **NEW**: Bare identifier syntax allows direct trait reference: `adult` instead of `ref(:adult)`
-- New syntax uses parenthesized expressions: `trait :name, (expression)`  
-- FieldRef nodes have operator methods that create CallExpression nodes
-- Logical AND chaining via `&` operator (Ruby limitation prevents `&&`)
-- Only AND operations supported to maintain constraint satisfaction system
-- **Backward Compatible**: Both `trait_name` and `ref(:trait_name)` work together
-- Old syntax maintained with deprecation warnings for backward compatibility
-
-## Common Development Tasks
-
-### Adding New Analyzer Passes
-1. Create pass class inheriting from `PassBase` in `lib/kumi/analyzer/passes/`
-2. Implement `run(errors)` method that calls `set_state(key, value)` to store results
-3. Add pass to `PASSES` array in `lib/kumi/analyzer.rb` in correct order
-4. Consider dependencies on other passes (e.g., TypeChecker needs TypeInferencer)
-
-## Architecture Design Principles
-
-- **Multi-pass Analysis**: Each analysis pass has a single responsibility and builds on previous passes
-- **Immutable Syntax Tree**: AST nodes are immutable; analysis results stored separately in analyzer state
-- **Dependency-driven Evaluation**: All computation follows dependency graph to ensure correct order
-- **Type Safety**: Optional but comprehensive type checking without breaking existing schemas
-- **Ruby Integration**: Leverages Ruby's metaprogramming while providing structured analysis
-- **Unified Error Reporting**: Consistent, localized error messages throughout the system with clear interface patterns
-
-## Code Organization Patterns
-
-### Testing Best Practices
-- **Spec Organization**: Tests organized by component with clear separation between unit and integration tests
-- **Error Variable Extraction**: RSpec patterns avoid multiline block chains by extracting error variables for assertion
-
-## Development Guides and Standards
-
-### Error Reporting Standards
-**For Parser Classes**:
-```ruby
-class MyParser
-  include ErrorReporting
-  
-  def parse_something
-    # Immediate error raising
-    raise_syntax_error("Invalid syntax", location: current_location)
-  end
-end
-```
-
-**For Analyzer Passes**:
-```ruby
-class MyAnalyzerPass < PassBase
-  def run(errors)
-    # Error accumulation with enhanced location
-    report_error(errors, "semantic error", location: node.loc, type: :semantic)
-    
-    # Backward compatible method
-    add_error(errors, node.loc, "legacy format error")
-  end
-end
-```
-### Testing Error Scenarios
-- Use `spec/integration/dsl_breakage_spec.rb` patterns for comprehensive error testing
-- Use `spec/integration/potential_breakage_spec.rb` for edge cases break
-- Use `spec/fixtures/location_tracking_test_schema.rb` fixture for testing different syntax error types  
-
-#