namo 0.28.0 → 0.30.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cf138a52a691a4d50f093e506e5018d652a66ab23846593e2622cbe23710c8ef
-  data.tar.gz: a8bbdc8cdf8af2b8c3df9881d468cc0445ffca7dd1f4523c614a2e71a8acc4c5
+  metadata.gz: e6f9ea9ed29bd86c309865e0c49d03122b7fa50460e34fd7a42a9a273bb220c0
+  data.tar.gz: 99d5e8191c3458b498c6fe297c2af8b94d8bb73aced942d9f3da67af94ace454
 SHA512:
-  metadata.gz: 58c3cb00aee14efb44155a9a4a8e5eaa748aa51389f0ac19c27f4d8a077970cac3b605e7d1ffad2c95329782fdf04755ee1e8975b9e2d34209705a1e8f780014
-  data.tar.gz: 7492096f83494f4c5a976b602796e3611c4eef1e920cc6b6179ff1b63e7f7fcd7a7f1e12ab2d637d3a1ef8fb4a0eeeb0364a0068ff3cbe12332b8e4b0e0d20c2
+  metadata.gz: 9e4aca2b13be76dc10a2fcf3eb7122045e600b1ed2410c2f6e82554c7dfc679249bf714fc52939fd4ee058fa6925b1958935bd91281dbf67f48634aec4cd4ea9
+  data.tar.gz: d3432cb0f092521adb1e2f60dccbc5b37aff2e4f91511189300b0693d374124e76cf1fde57d64709606ea504aae5db85d9c47e7aba99efcfdfa68b6fcbc38eef

data/CHANGELOG

@@ -1,6 +1,33 @@
 # CHANGELOG
 
-## 20260711
+## 20260715
+
+0.30.0: Assemble Collection objects in a single pass when doing Namo::Enumerable#group_by.
+
+Appending one member per group re-materialised Collection#<<'s @data = detail.data over all members-so-far g times; group_by now appends all the members in one << operation and materialises once, causing it be performed in O(rows) instead of O(rows·groups).
+
+1. ~ Namo::Enumerable#group_by: map the groups to members, then append them all in one collection << members, rather than << per group — each of which rebuilt @data over every member-so-far.
+2. ~ test/namo_test.rb: + a single-pass assembly pin under #group_by — the assembly materialises the detail once whatever the number of groups (not once per group).
+3. ~ ROADMAP.md: + the 0.30.0 entry; current-state bumped to 0.30.0.
+4. ~ docs/: regenerate the print-ready PDFs.
+5. ~ Namo::VERSION: /0.29.0/0.30.0/
+
+## 20260715
+
+0.29.0: Per access indexed materialisation.
+
+Materialise each column once per top-level access, collapsing an O(n²) parent-scanning formula to O(n), with no value retained across accesses.
+
+1. ~ Namo#values: wrap the body in a private materialising scope.
+2. ~ Namo#values_for: memoise its column build into @materialised[dim] ||= (the data/derived branch that was its body).
+3. + Namo#materialising (a per-access scope that sets @materialised, yields, and clears it in an ensure; a nested values call reuses the open scope rather than starting another). A formula that scans its parent asks for the same column once per row it visits; the scope materialises that column once and reuses it for the rest of the access, then discards it — so every access still recomputes from current state and nothing can go stale. Enumerable is untouched: iteration paths (each/select/map/group_by) do not yet open the scope.
+4. ~ test/namo_test.rb: + an "access-scoped materialisation" describe — a referenced column materialises once per access, not once per referencing row; nothing is retained across accesses (a later access recomputes from current state); the scope clears on a raising materialisation, leaving later accesses working.
+5. ~ README.md: ~ the cross-row liveness note — "with no caching" becomes "with no caching across accesses", the within-access materialisation being a loop-invariant hoist, not a cache.
+6. ~ ROADMAP.md: + the 0.29.0 entry; current-state bumped to 0.29.0; the 1.x pure-live passages refined from "no memoisation anywhere" to the invariant "no derived value outlives the access that produced it"; the Live-data-and-cache-invalidation open question gains the access-scoped resolution; + forward notes on streaming performance vs incremental maintenance (a commercial-tier candidate) and on formula opacity as the shared wall behind ===, dependency-aware projection, serialisation, and incremental streaming.
+7. ~ docs/: regenerate the print-ready PDFs.
+8. ~ Namo::VERSION: /0.28.0/0.29.0/
+
+## 20260713
 
 0.28.0: ~ Namo::Collection#detail/#as_detail: Accept the by label positionally or by keyword.
 

data/README.md

@@ -688,7 +688,7 @@ prices.values(:sma)
 # => [10.0, 15.0, 20.0]
 ```
 
-`namo` is the Namo that yielded the row, live — so the window always reflects the current state of the object you ask through. A filtered Namo's rows window over the filtered rows; an operator result's rows window over the result. Appending a row changes every cross-row value on the next access, with no caching.
+`namo` is the Namo that yielded the row, live — so the window always reflects the current state of the object you ask through. A filtered Namo's rows window over the filtered rows; an operator result's rows window over the result. Appending a row changes every cross-row value on the next access, with no caching across accesses.
 
 One-arity formulae are unchanged, and the two forms mix freely — a one-arity formula can reference a two-arity one, and a two-arity formula can reference a one-arity one, by name.
 

data/lib/Namo/Enumerable.rb

@@ -72,10 +72,12 @@ class Namo
     def group_by(dimension)
       collection = Collection.new
       source = derived_dimensions.include?(dimension) ? self[*data_dimensions, dimension] : self
-      source.data.group_by{|row_data| row_data[dimension]}.each do |value, rows|
-        collection << self.class.new(rows, formulae: source.formulae.dup, name: value)
-      end
-      collection
+      members = (
+        source.data.group_by{|row_data| row_data[dimension]}.map do |value, rows|
+          self.class.new(rows, formulae: source.formulae.dup, name: value)
+        end
+      )
+      collection << members
     end
   end
 end

data/lib/Namo/VERSION.rb

@@ -2,5 +2,5 @@
 # Namo::VERSION
 
 class Namo
-  VERSION = '0.28.0'
+  VERSION = '0.30.0'
 end

data/lib/namo.rb

@@ -30,12 +30,14 @@ class Namo
   end
 
   def values(*dims)
-    if dims.empty?
-      materialisable_dimensions.each_with_object({}){|dim, hash| hash[dim] = values_for(dim)}
-    elsif dims.length == 1
-      values_for(dims.first)
-    else
-      dims.each_with_object({}){|dim, hash| hash[dim] = values_for(dim)}
+    materialising do
+      if dims.empty?
+        materialisable_dimensions.each_with_object({}){|dim, hash| hash[dim] = values_for(dim)}
+      elsif dims.length == 1
+        values_for(dims.first)
+      else
+        dims.each_with_object({}){|dim, hash| hash[dim] = values_for(dim)}
+      end
     end
   end
 
@@ -290,14 +292,26 @@ class Namo
     end
   end
 
-  def values_for(dim)
-    if data_dimensions.include?(dim)
-      @data.map{|row_data| row_data[dim]}
-    else
-      @data.map{|row_data| Row.new(row_data, @formulae, self)[dim]}
+  def materialising
+    return yield if @materialised
+    @materialised = {}
+    begin
+      yield
+    ensure
+      @materialised = nil
     end
   end
 
+  def values_for(dim)
+    @materialised[dim] ||= (
+      if data_dimensions.include?(dim)
+        @data.map{|row_data| row_data[dim]}
+      else
+        @data.map{|row_data| Row.new(row_data, @formulae, self)[dim]}
+      end
+    )
+  end
+
   def materialisable_dimensions
     dimensions.reject{|dim| requires_arguments?(dim)}
   end

data/test/namo_test.rb

@@ -287,6 +287,44 @@ describe Namo do
     end
   end
 
+  describe "access-scoped materialisation" do
+    it "materialises a referenced column once per access, not once per referencing row" do
+      calls = Hash.new(0)
+      namo = Namo.new([{n: 1}, {n: 2}, {n: 3}])
+      namo[:base] = proc{|row| calls[:base] += 1; row[:n] * 10}
+      namo[:sum_base] = proc{|row, namo| namo.values(:base).sum}
+      _(namo.values(:sum_base)).must_equal [60, 60, 60]
+      _(calls[:base]).must_equal 3
+    end
+
+    it "retains nothing across accesses — a later access recomputes from current state" do
+      namo = Namo.new([{n: 1}, {n: 2}])
+      namo[:total] = proc{|row, namo| namo.values(:n).sum}
+      _(namo.values(:total)).must_equal [3, 3]
+      namo.data << {n: 4}
+      _(namo.values(:total)).must_equal [7, 7, 7]
+    end
+
+    it "clears the access scope when materialisation raises, leaving later accesses working" do
+      namo = Namo.new([{n: 1}])
+      namo[:boom] = proc{|row| raise 'materialisation blew up'}
+      _(->{namo.values(:boom)}).must_raise RuntimeError
+      namo[:ok] = proc{|row| row[:n]}
+      _(namo.values(:ok)).must_equal [1]
+    end
+
+    it "keeps one scope when a formula's nested values call runs during a multi-dimension access" do
+      calls = 0
+      namo = Namo.new([{x: 1}, {x: 2}, {x: 3}])
+      namo[:base] = proc{|row| calls += 1; row[:x] * 2}
+      namo[:agg] = proc{|row, namo| namo.values(:base).first}
+      # :agg's formula calls values(:base) mid-materialisation; that nested call must
+      # reuse the open scope, not reset it, so the :x column that follows still builds.
+      _(namo.values(:agg, :x)).must_equal(agg: [2, 2, 2], x: [1, 2, 3])
+      _(calls).must_equal 3
+    end
+  end
+
   describe "#to_h" do
     it "returns the full values Hash" do
       _(sales.to_h).must_equal sales.values
@@ -3160,5 +3198,27 @@ describe Namo do
         _(collection.detail.values(:close)).must_equal [42.5, 43.0, 118.3, 100.0]
       end
     end
+
+    context "single-pass assembly" do
+      # Collection#<< rebuilds @data = detail.data over all members-so-far on every
+      # append, so appending one member per group re-materialised the data g times
+      # (O(rows·groups)). group_by now hands all members to a single <<: one
+      # materialisation whatever the number of groups.
+      def detail_calls_assembling(namo, dimension)
+        calls = 0
+        tracer = TracePoint.new(:call) do |tp|
+          calls += 1 if tp.method_id == :detail && tp.defined_class == Namo::Collection
+        end
+        tracer.enable{namo.group_by(dimension)}
+        calls
+      end
+
+      it "materialises the assembled data once, whatever the number of groups" do
+        few = Namo.new((1..3).map{|i| {symbol: "S#{i}", close: i.to_f}})
+        many = Namo.new((1..30).map{|i| {symbol: "S#{i}", close: i.to_f}})
+        _(detail_calls_assembling(few, :symbol)).must_equal 1
+        _(detail_calls_assembling(many, :symbol)).must_equal 1
+      end
+    end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: namo
 version: !ruby/object:Gem::Version
-  version: 0.28.0
+  version: 0.30.0
 platform: ruby
 authors:
 - thoran