dentaku 4.0.0 → 4.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a911d2d58c12144c0692b27a9abe5ad08716e1800deb9e9bda08bee04735b9ac
-  data.tar.gz: 8e55f39625ebb6a6a5f4d38b820137566c9bd2d529d411cb4c40c89770dbf048
+  metadata.gz: 8c52e5ed43e8cc6d9ac6f3cf81ae522e408860ab24bf59109bdd97301e0d6e12
+  data.tar.gz: fa376e35d073d5c0ebce858f96c4c1322ed1bba2a202dc6e41d265c1e5fa24f3
 SHA512:
-  metadata.gz: ac372c840c68d33bb1a6577640ea17812e0aab7b1f82c1979d1caac5ef670148743ecc77f87c8cf32a33e5e90d6f9f35f5d6da85da3e345beb64cf0a0d7d2b78
-  data.tar.gz: 4d9d05f5b7a79751cfc4f82576b6bddd5d621e02df4ee98282052d55cbe87c250d5f0906960726278ee078d9162f369a9efd42f166d7446d189aeb80de7c166d
+  metadata.gz: 040e836d96fad9833095285a2d3a9b6b7e7a3c4bffcd84cac9603bc16128308acb719103eb96693cc04910c028265231f72f982658dd34523d3685ac4dea8e7d
+  data.tar.gz: 66aa9ee21ce77b2d9dd25d44b3a7f1db7212e7776614839d4c92e403ed0d4e5561fee07e28a50eb81c1a886bf889c1ae285c4a89aa78a59910f5636de66159d4

data/CHANGELOG.md

@@ -1,5 +1,39 @@
 # Change Log
 
+## [v4.0.1] 2026-08-02
+- parsing no longer executes user functions. The three parse-time operand
+  validators (`Arithmetic`, `Negation`, and the logical combinators) asked
+  each operand for its dependencies without static mode, which evaluates
+  short-circuit guards -- so an `IF` or `CASE` used as an operand ran its
+  predicate while the parser was still building the node. `Calculator#ast`
+  and `Calculator#identifiers` could therefore raise from inside a
+  user-registered function, contrary to the documented contract that
+  `identifiers` never evaluates guards or functions (#197). Regression
+  introduced in 3.5.6 (ecd46bf); evaluation and short-circuiting during
+  `evaluate`/`dependencies` are unchanged.
+- functions registered under a name that another calculator already registered
+  no longer render as `#<Class:0x...>` in error messages; every generated
+  function class now has a stable `to_s` of
+  `Dentaku::AST::Function::<Name>` (#264)
+- arithmetic operations that reject an otherwise well-typed operand (notably
+  `^` with an oversized exponent) now raise `Dentaku::ArgumentError` instead of
+  leaking a raw Ruby `ArgumentError`, so the non-bang `Calculator#evaluate`
+  returns `nil` as documented rather than raising (#332)
+- `Calculator#store` restores only the keys it set instead of snapshotting the
+  whole memory hash, so evaluation cost no longer scales with the number of
+  stored variables (#336). Measured on 20k evaluations of a two-variable
+  formula: 34.5 -> 18.6 us/eval with 20,000 variables in memory, and flat
+  across memory sizes. Block-scoping semantics are unchanged; when the
+  identifier cache is enabled the previous whole-hash snapshot is still used,
+  because evaluation writes into memory in that mode.
+- document what the `add_function` return type argument actually does: it is
+  consulted at parse time to decide whether a call is a valid arithmetic
+  operand, is never coerced or checked at runtime, and has no observable
+  effect outside arithmetic (#303)
+- the test suite no longer depends on example ordering: the module-level
+  caching opt-ins are reset before each example, since they have no public
+  disable and leaked between examples
+
 ## [v4.0.0] 2026-08-01
 BREAKING CHANGES
 - require Ruby 3.2 or newer

data/README.md

@@ -347,6 +347,42 @@ function)
 Functions can be added individually using Calculator#add_function, or en masse
 using Calculator#add_functions.
 
+### The return type argument
+
+The second argument is the function's *declared* return type.  It is used at
+parse time to decide whether a call to the function is a valid operand of an
+arithmetic operator -- only `:numeric`, `:integer`, `:array`, and `nil` are
+accepted there.  Declaring anything else means the function cannot be used in
+arithmetic:
+
+```ruby
+> c = Dentaku::Calculator.new
+> c.add_function(:label, :string, ->(x) { "item#{x}" })
+> c.evaluate!('label(1)')
+#=> "item1"
+> c.evaluate!("label(1) = 'item1'")     # comparison: type is not consulted
+#=> true
+> c.evaluate!('label(1) + 1')           # arithmetic: rejected at parse time
+#=> Dentaku::ParseError: Dentaku::AST::Addition requires operands that are
+#   numeric or compatible types, not string
+```
+
+The type is a declaration, not a guarantee: Dentaku does not coerce the return
+value and does not check it at runtime.  Declaring `:numeric` for a function
+that actually returns a String defers the failure to whatever consumes it:
+
+```ruby
+> c.add_function(:mislabeled, :numeric, ->(x) { "item#{x}" })
+> c.evaluate!('mislabeled(1) + 1')
+#=> Dentaku::ArgumentError: String input 'item1' is not coercible to numeric
+```
+
+So the type only has an observable effect when the result feeds an arithmetic
+operator.  A function used standalone, in comparisons, or as a function
+argument behaves the same whatever type is declared -- which is why an
+incorrect type can go unnoticed for a long time.  Declare the type your lambda
+actually returns.
+
 Dentaku assumes registered functions are pure (see DEPENDENCY ANALYSIS AND
 SHORT-CIRCUITING above).  A function that reads external state, performs
 I/O, or returns different values across calls must be declared volatile so

data/lib/dentaku/ast/arithmetic.rb

@@ -38,10 +38,21 @@ module Dentaku
         l = cast(left_value)
         r = cast(right_value)
 
-        l.public_send(operator, r)
-      rescue ::TypeError => e
-        # Right cannot be converted to a suitable type for left. e.g. [] + 1
-        raise Dentaku::ArgumentError.for(:incompatible_type, actual: r, expected: l.class), e.message
+        begin
+          l.public_send(operator, r)
+        rescue ::TypeError => e
+          # Right cannot be converted to a suitable type for left. e.g. [] + 1
+          raise Dentaku::ArgumentError.for(:incompatible_type, actual: r, expected: l.class), e.message
+        rescue Dentaku::Error
+          # already one of ours (e.g. from arithmetic on a custom class)
+          raise
+        rescue ::ArgumentError, ::RangeError, ::FloatDomainError => e
+          # the operation itself rejected an otherwise well-typed operand,
+          # e.g. BigDecimal#** with an oversized exponent. Without this, a raw
+          # Ruby exception escapes the non-bang Calculator#evaluate, which
+          # promises to return nil rather than raise.
+          raise Dentaku::ArgumentError.for(:invalid_value, actual: r), e.message
+        end
       end
 
       def cast(val)
@@ -66,8 +77,11 @@ module Dentaku
       def valid_node?(node)
         return false unless node
 
-        # Allow nodes with dependencies (identifiers that will be resolved later)
-        return true if node.dependencies.any?
+        # Allow nodes with dependencies (identifiers that will be resolved later).
+        # Ask statically: this runs while the parser is building the node, and
+        # a non-static dependency check evaluates short-circuit guards, which
+        # would execute user functions at parse time (#197).
+        return true if node.dependencies(Node::STATIC_CONTEXT).any?
 
         # Allow compatible types
         return true if [:numeric, :integer, :array].include?(node.type)

data/lib/dentaku/ast/combinators.rb

@@ -57,8 +57,10 @@ module Dentaku
 
       private
 
+      # static dependency check: this runs at parse time, and evaluating
+      # short-circuit guards here would execute user functions (#197)
       def valid_node?(node)
-        node && (node.dependencies.any? || node.type == :logical)
+        node && (node.dependencies(Node::STATIC_CONTEXT).any? || node.type == :logical)
       end
 
       # whether a single operand with this value already determines the

data/lib/dentaku/ast/function_registry.rb

@@ -18,6 +18,21 @@ module Dentaku
             @name
           end
 
+          # only the first registration of a given name claims the constant
+          # under Dentaku::AST::Function, so later ones stay anonymous and
+          # would otherwise render as "#<Class:0x...>" in error messages
+          def self.display_name=(display_name)
+            @display_name = display_name
+          end
+
+          def self.to_s
+            @display_name || super
+          end
+
+          def self.inspect
+            to_s
+          end
+
           def self.implementation=(impl)
             @implementation = impl
           end
@@ -72,9 +87,11 @@ module Dentaku
           end
         end
 
+        function.name = name
+        function.display_name = "#{Function}::#{normalize_name(name)}"
+
         define_class(name, function)
 
-        function.name = name
         function.type = type
         function.implementation = implementation
         function.callback = callback

data/lib/dentaku/ast/negation.rb

@@ -51,8 +51,10 @@ module Dentaku
         @node.pure?
       end
 
+      # static dependency check: this runs at parse time, and evaluating
+      # short-circuit guards here would execute user functions (#197)
       def valid_node?(node)
-        node && (node.dependencies.any? || node.type == :numeric)
+        node && (node.dependencies(Node::STATIC_CONTEXT).any? || node.type == :numeric)
       end
     end
   end

data/lib/dentaku/calculator.rb

@@ -102,7 +102,9 @@ module Dentaku
     end
 
     def dependencies(expression, context = {})
-      test_context = context.nil? ? {} : store(context) { memory }
+      # dup inside the block: `store` now restores in place, so the caller
+      # needs its own copy of the merged context rather than the live memory
+      test_context = context.nil? ? {} : store(context) { memory.dup }
 
       case expression
       when Dentaku::AST::Node
@@ -169,32 +171,51 @@ module Dentaku
     end
 
     def store(key_or_hash, value = nil)
-      restore = Hash[memory]
+      pairs = pairs_to_store(key_or_hash, value)
 
-      if value.nil?
-        key_or_hash = FlatHash.from_hash_with_intermediates(key_or_hash) if nested_data_support
-        key_or_hash.each do |key, val|
-          memory[standardize_case(key.to_s)] = val
-        end
-      else
-        memory[standardize_case(key_or_hash.to_s)] = value
+      unless block_given?
+        pairs.each { |key, val| memory[key] = val }
+        return self
       end
 
-      if block_given?
+      # `evaluate!` routes every call through here, so snapshotting the whole
+      # memory hash makes each evaluation scale with the number of stored
+      # variables (#336). Undoing just the keys this call touched is O(pairs)
+      # instead of O(memory) -- but it is only equivalent while nothing else
+      # writes into memory during the block. The identifier cache does exactly
+      # that, so when it is enabled we still need the wholesale snapshot to
+      # keep cached values scoped to a single evaluation.
+      if Dentaku.cache_identifier?
+        restore = Hash[memory]
+        pairs.each { |key, val| memory[key] = val }
+
         begin
-          result = yield
+          yield
+        ensure
           @memory = restore
-          return result
-        rescue => e
-          @memory = restore
-          raise e
         end
-      end
+      else
+        undo = pairs.map { |key, _| [key, memory.key?(key), memory[key]] }
+        pairs.each { |key, val| memory[key] = val }
 
-      self
+        begin
+          yield
+        ensure
+          undo.each { |key, present, val| present ? memory[key] = val : memory.delete(key) }
+        end
+      end
     end
     alias_method :bind, :store
 
+    private def pairs_to_store(key_or_hash, value)
+      if value.nil?
+        key_or_hash = FlatHash.from_hash_with_intermediates(key_or_hash) if nested_data_support
+        key_or_hash.map { |key, val| [standardize_case(key.to_s), val] }
+      else
+        [[standardize_case(key_or_hash.to_s), value]]
+      end
+    end
+
     def store_formula(key, formula)
       store(key, ast(formula))
     end

data/lib/dentaku/version.rb

@@ -1,3 +1,3 @@
 module Dentaku
-  VERSION = "4.0.0"
+  VERSION = "4.0.1"
 end

data/spec/ast/arithmetic_spec.rb

@@ -123,6 +123,99 @@ describe Dentaku::AST::Arithmetic do
     expect { add(x, one, 'x' => [1]) }.to raise_error(Dentaku::ArgumentError)
   end
 
+  describe 'parse-time operand validation (#197)' do
+    let(:calculator) do
+      Dentaku::Calculator.new.tap do |c|
+        c.add_function(:explode, :numeric, ->(*) { raise "function executed at parse time" })
+        c.add_function(:explode_flag, :logical, ->(*) { raise "function executed at parse time" })
+      end
+    end
+
+    it 'does not execute functions when an IF is an arithmetic operand' do
+      expect {
+        calculator.ast("IF(explode(1) = 1, 1, 2) + category")
+      }.not_to raise_error
+    end
+
+    it 'does not execute functions when an IF is negated' do
+      expect {
+        calculator.ast("-IF(explode(1) = 1, 1, 2)")
+      }.not_to raise_error
+    end
+
+    it 'does not execute functions when an IF is a combinator operand' do
+      expect {
+        calculator.ast("IF(explode_flag(), true, false) AND x")
+      }.not_to raise_error
+    end
+
+    it 'does not execute functions when a CASE is an arithmetic operand' do
+      expect {
+        calculator.ast("(CASE explode(1) WHEN 1 THEN 1 ELSE 2 END) + category")
+      }.not_to raise_error
+    end
+
+    it 'reports identifiers across both branches without evaluating' do
+      expect(calculator.identifiers("IF(explode(1) = 1, 1, 2) + category")).to eq(["category"])
+    end
+
+    it 'still validates operand types' do
+      expect { Dentaku::Calculator.new.ast("'abc' + 1") }.to raise_error(Dentaku::ParseError)
+    end
+
+    it 'leaves evaluation and short-circuiting unchanged' do
+      plain = Dentaku::Calculator.new
+
+      expect(plain.evaluate!("IF(a = 1, 10, 20) + 5", a: 1)).to eq(15)
+      expect(plain.evaluate!("IF(a = 1, 10, 20) + 5", a: 2)).to eq(25)
+      expect(plain.evaluate!("-IF(a = 1, 10, 20)", a: 1)).to eq(-10)
+      expect(plain.dependencies("IF(a, b, c)", a: true)).to eq(["b"])
+    end
+  end
+
+  describe 'operations that reject well-typed operands (#332)' do
+    let(:oversized) { "999999999999999 ^ 99999999999999" }
+
+    it 'wraps the raw Ruby error so it is a Dentaku::Error' do
+      expect {
+        Dentaku::Calculator.new.evaluate!(oversized)
+      }.to raise_error(Dentaku::ArgumentError, /exponent is too large/)
+    end
+
+    it 'is rescuable as Dentaku::Error' do
+      error = begin
+        Dentaku::Calculator.new.evaluate!(oversized)
+        nil
+      rescue Dentaku::Error => e
+        e
+      end
+
+      expect(error).to be_a(Dentaku::ArgumentError)
+    end
+
+    it 'returns nil from the non-bang evaluate rather than raising' do
+      expect(Dentaku::Calculator.new.evaluate(oversized)).to be_nil
+      expect(Dentaku::Calculator.new.evaluate("2 ^ 99999999999")).to be_nil
+    end
+
+    it 'yields to the block form' do
+      handled = nil
+      Dentaku::Calculator.new.evaluate(oversized) { |_expr, ex| handled = ex }
+
+      expect(handled).to be_a(Dentaku::ArgumentError)
+    end
+
+    it 'still evaluates exponentiation that fits' do
+      expect(Dentaku::Calculator.new.evaluate!("2 ^ 10")).to eq(1024)
+    end
+
+    it 'does not swallow Dentaku errors raised by the operation itself' do
+      expect {
+        Dentaku::Calculator.new.evaluate!("1 / 0")
+      }.to raise_error(Dentaku::ZeroDivisionError)
+    end
+  end
+
   private
 
   def add(left, right, context = ctx)

data/spec/calculator_spec.rb

@@ -1076,5 +1076,84 @@ describe Dentaku::Calculator do
         called
       }.from(12).to(1)
     end
+
+    it 'does not carry cached identifier values across evaluations' do
+      # there is no public disable, so save/restore around the example
+      was_enabled = Dentaku.cache_identifier?
+      Dentaku.enable_identifier_cache!
+      called = 0
+      calculator.store_formula("A1", "B1+B1")
+      calculator.store_formula("B1", "C1")
+      calculator.store("C1", proc { called += 1; 1 })
+
+      calculator.evaluate("A1")
+      after_first = called
+      calculator.evaluate("A1")
+
+      expect(after_first).to be > 0
+      expect(called).to eq(after_first * 2)
+    ensure
+      Dentaku.instance_variable_set(:@enable_identifier_caching, was_enabled)
+    end
+  end
+
+  describe 'block-scoped store (#336)' do
+    it 'removes keys the block introduced' do
+      calculator.store(a: 1)
+      result = calculator.store(b: 2) { calculator.evaluate!("a + b") }
+
+      expect(result).to eq(3)
+      expect(calculator.memory.keys).to eq(["a"])
+    end
+
+    it 'restores keys the block shadowed' do
+      calculator.store(a: 1)
+      inner = calculator.store(a: 99) { calculator.evaluate!("a") }
+
+      expect(inner).to eq(99)
+      expect(calculator.evaluate!("a")).to eq(1)
+    end
+
+    it 'restores when the block raises' do
+      calculator.store(a: 1)
+
+      expect {
+        calculator.store(a: 99) { raise "boom" }
+      }.to raise_error("boom")
+
+      expect(calculator.evaluate!("a")).to eq(1)
+      expect(calculator.memory.keys).to eq(["a"])
+    end
+
+    it 'restores correctly when nested' do
+      calculator.store(a: 1)
+      innermost = calculator.store(a: 2) { calculator.store(a: 3) { calculator.evaluate!("a") } }
+
+      expect(innermost).to eq(3)
+      expect(calculator.evaluate!("a")).to eq(1)
+    end
+
+    it 'does not leak the internal evaluation mode key into memory' do
+      calculator.store(a: 1)
+      calculator.evaluate("a + 1")
+
+      expect(calculator.memory.keys).to eq(["a"])
+    end
+
+    it 'restores in place rather than copying the whole memory hash' do
+      # the old implementation snapshotted memory up front and reassigned
+      # @memory to the snapshot, so evaluation cost scaled with memory size.
+      # Restoring in place keeps the same hash object throughout.
+      was_enabled = Dentaku.cache_identifier?
+      Dentaku.instance_variable_set(:@enable_identifier_caching, false)
+
+      calculator.store(a: 1, b: 2)
+      before = calculator.memory.object_id
+
+      expect(calculator.evaluate!("a + b")).to eq(3)
+      expect(calculator.memory.object_id).to eq(before)
+    ensure
+      Dentaku.instance_variable_set(:@enable_identifier_caching, was_enabled)
+    end
   end
 end

data/spec/external_function_spec.rb

@@ -109,6 +109,7 @@ describe Dentaku::Calculator do
       end
 
       it 'exposes the `callback` method of a function' do
+        custom_calculator # the `let` is lazy, and it is what registers the function
         expect(Dentaku::AST::Function::Callback_lambda.callback.call()).to eq("lambda executed")
       end
 
@@ -155,6 +156,38 @@ describe Dentaku::Calculator do
       }.to raise_error(Dentaku::ParseError)
     end
 
+    describe 'error messages (#264)' do
+      it 'names the function even when another calculator claimed the constant' do
+        messages = 2.times.map do
+          calculator = described_class.new
+          calculator.add_function(:custom, :integer, -> { 1 })
+
+          begin
+            calculator.ast("CUSTOM(1,2)")
+            nil
+          rescue Dentaku::ParseError => e
+            e.message
+          end
+        end
+
+        expect(messages).to all(include("Dentaku::AST::Function::Custom"))
+        messages.each { |message| expect(message).not_to include("#<Class:") }
+      end
+
+      it 'gives re-registered functions a stable string representation' do
+        first = described_class.new
+        first.add_function(:repeated, :numeric, ->(x) { x })
+        second = described_class.new
+        second.add_function(:repeated, :numeric, ->(x) { x * 2 })
+
+        registry = ->(c) { c.instance_variable_get(:@function_registry).get("repeated") }
+
+        expect(registry.call(second).to_s).to eq("Dentaku::AST::Function::Repeated")
+        expect(registry.call(second).to_s).to eq(registry.call(first).to_s)
+        expect(registry.call(second).name).to eq("repeated")
+      end
+    end
+
     describe 'Dentaku::Calculator.add_function' do
       it 'adds a function to default/global function registry' do
         described_class.add_function(:global_function, :numeric, ->(x) { 10 + x**2 })

data/spec/spec_helper.rb

@@ -20,6 +20,15 @@ RSpec.configure do |c|
       Dentaku.aliases = { roundup: ['roundupup'] }
     end
   }
+
+  # the caching opt-ins are module-level and have no public "disable", so an
+  # example that turns one on leaks into every example that runs after it.
+  # Reset to the documented defaults so the suite does not depend on order.
+  c.before(:each) {
+    Dentaku.instance_variable_set(:@enable_ast_caching, false)
+    Dentaku.instance_variable_set(:@enable_dependency_order_caching, false)
+    Dentaku.instance_variable_set(:@enable_identifier_caching, false)
+  }
 end
 
 # automatically create a token stream from bare values

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dentaku
 version: !ruby/object:Gem::Version
-  version: 4.0.0
+  version: 4.0.1
 platform: ruby
 authors:
 - Solomon White