carray-jit 0.1.4 → 0.1.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 85e10fcc54b5ddc4ef74d6a2ddf169b56891eef480471ac91b8b834055677da1
-  data.tar.gz: 2c6a3c4a390c4c3cffd3b1621656e06ea627a44789db2450721ec6c97bfbfc3f
+  metadata.gz: e052fd85b01164a6147a30ca655b8997ba7c87ac9c7ec8354d73447d1208b5ca
+  data.tar.gz: 2a6ebb524fb6afee5c871c0a45ed13ecf2f120f9ef0194e36f486ac3b015b378
 SHA512:
-  metadata.gz: 9d20ccad82918edc5df3038d35204a419fa9e4d92645c5d3a2691c97d251d31624acee7b9bf28b1bef35958b255ef62ed10006ac15983786ffaf3bfcbd3df2b3
-  data.tar.gz: d67ec3082b6e76bd10175c593cbfabfe13c1eec9efdd9c91edb344cad5b3ea6e1b7fecbd4b1f1b7a16b8413e4c180f8449dcc8e03f386ca798a2c2149476db79
+  metadata.gz: bd5017f5251ad7b9f4263bfc25091a938d3bccd7f18ef5d83a5a68b63f57e0df6c1ee4278ccd1b903a68cb77a5a156a47c98cbc4832edabbf144593778fed1b5
+  data.tar.gz: d8d2733f8a98376561ccb183895e1e963eff8a1caea636d066e88ce4852f167fb8755c6f1225beaf0c89aea51bf18dab4482bde0a079983ff507e9cd66769b2f

data/CHANGELOG.md

@@ -37,6 +37,54 @@ version you have and a newer one.
      a release needs is said in the gemspec, and an entry says so only
      when the answer changes. -->
 
+## 0.1.6
+
+- Change: needs CArray 3.0.5 or later in the 3.0 series; install it
+  alongside this release. A Float stored into an integer array in a block
+  is refused as CArray 3.0.5's own store refuses it, and under 3.0.4 the
+  block would raise where CArray's store does not.
+
+- Change: a Float stored into an integer array in a block, captured or
+  local, that the type has no integer for raises `RangeError`, or
+  `FloatDomainError` for a NaN or an infinity, as CArray 3.0.5's own
+  store does. Before, it stored whatever the machine's C conversion gave,
+  which differed between x86 and ARM. Round and clamp first where the
+  value may leave the range. A masked cell, and a `jit_function`'s return
+  value or pointer parameter, are not checked.
+
+- Fix: an int64 array handed to a C function declared to take
+  `long long *` compiles with gcc, as it did with clang; gcc 14 raised
+  CompilationError. An array of the wrong element type raises Unsupported,
+  naming the type the function takes, under gcc as under clang.
+
+- Fix: `finite?` on an integer cell in a block compiles with gcc; it
+  raised CompilationError there, while clang accepted it. It is true for
+  every integer, as in Ruby.
+
+- Fix: a lazy expression or `CArray.fuse` computed here now gives the
+  same bits as CArray's own walk where gcc is the compiler. On x86 with
+  `-march=native`, a multiply followed by an add could come out one
+  rounding off in the last bit. Builds with clang were not affected.
+
+## 0.1.5
+
+- Change: needs CArray 3.0.4 or later in the 3.0 series; install it
+  alongside this release. 0.1.4 accepts CArray 3.0.4 but does not run
+  under it, so the two are upgraded together. A store in a block into a
+  cell that cannot hold the value still wraps as C does, where CArray
+  3.0.4's own `int8[0] = 300` raises RangeError.
+
+- Change: `jit_each` and `jit_map` called again with operands of the same
+  shapes, types and masking cost less per call -- about 9 us rather than 11
+  for a small two-array block -- which a time-stepping loop calling one
+  block thousands of times will notice. Nothing changes in what they
+  compute; a scalar's value may differ from call to call as before.
+
+- Fix: an element-wise `jit_each` or `jit_map` run through CArray's sweep
+  could read its scalars and C functions from where they had been before
+  GC compaction moved them, and answer wrongly or crash. Only a process
+  that compacts (`GC.compact`, `GC.auto_compact = true`) was exposed.
+
 ## 0.1.4
 
 - New: a lazy `then_else` (a lazy condition or branch) is compiled with

data/README.md

@@ -8,7 +8,7 @@ The block is read with Prism, translated to C if it falls inside that subset, co
 
 ## Status
 
-0.1.4 is the current release, and it still moves: behaviour can change between releases.
+0.1.6 is the current release, and it still moves: behaviour can change between releases.
 
 ## Features
 
@@ -35,7 +35,7 @@ gem "carray-jit"
 Requires:
 
 - Ruby >= 3.2
-- CArray >= 3.0.3, < 3.1
+- CArray >= 3.0.5, < 3.1
 - A C compiler
 - Prism and Fiddle (both ship with Ruby; Fiddle is a bundled gem)
 

data/carray-jit.gemspec

@@ -33,7 +33,7 @@ Gem::Specification.new do |spec|
   spec.executables   = ["carray-jit"]
 
   # A kernel reaches CArray's C by address, so the ceiling is the next minor.
-  spec.add_dependency "carray", ">= 3.0.3", "< 3.1"
+  spec.add_dependency "carray", ">= 3.0.5", "< 3.1"
   # fiddle ships as a bundled gem; depend on it explicitly so Ruby 3.5+ resolves it.
   spec.add_dependency "fiddle"
 end

data/docs/03_SupportedFeatures.md

@@ -1223,6 +1223,6 @@ Everything else: `for` and `until`, `begin ... end while`, strings, hashes, symb
 - **A local array's subscript counts from the start.** CArray's `w[-1]` is the last cell; a kernel has no such reading, so a literal negative subscript is refused by name and one the kernel works out raises `IndexError`. This is the difference the captured arrays already have. Write `w[n - 1]`.
 - **`CArray.empty` leaves its cells as the stack left them.** Reading one before writing it is out of contract, the way reading under a mask is: the value is whatever was there. What it saves is the clearing, which for a 256-cell array inside a per-cell loop is 2 KiB a cell.
 - **The stack limits are per function, and a chain of them is not counted.** One local array up to 4 KiB stands in a frame, and the arrays of one kernel body or one function body up to 16 KiB together; past either, a kernel allocates the array at its entry, while a function body -- which is called once per cell -- refuses it and asks the calling kernel for a pointer instead. A pasted function called from a kernel is a second frame, and a chain of three such functions can therefore stand 48 KiB deep while each of them is inside its own limit -- counting along the chain was considered and dropped, since the depth is not knowable where recursion is in the subset at all. What that costs is the bargain a deep recursion already takes: too deep is a SIGSEGV rather than a `SystemStackError`, and a body carrying arrays reaches it sooner. Both numbers are provisional, and neither counts a future thread pool's stacks.
-- **Storing a real number into an integer array is C's conversion.** A value past the type's range, or a NaN, is undefined behaviour rather than a number -- and the same is true of a captured integer array, so a kernel brings no new hazard here. Round and clamp before storing where the value may leave the range. The rounding methods are not this case: `floor`, `ceil`, `round`, `truncate` and `to_i` on a Float raise `FloatDomainError` for a NaN or an infinity, as Ruby does, and `RangeError` where the Integer is past int64 -- except written straight into a float cell, which holds it whole (`1e20.floor` is `1e20`). The check costs a comparison per call, and in a loop that does little else it shows.
+- **Storing a real number into an integer array is checked.** It keeps the integer part, as CArray's own store does, and a value past the type's range raises `RangeError` -- `FloatDomainError` for a NaN or an infinity -- where CArray's store raises the same; C leaves that conversion undefined, and x86 and ARM answer it differently. Like any error at a cell it is raised when the pass ends, and the cell holds 0 in the meantime. A masked cell is not checked. The check is a comparison per store; round and clamp before storing where the value may leave the range and the cost shows. A compiled function's return value and a write through its pointer parameter are not this case: they are C's conversion, as the declaration says. Nor are the rounding methods: `floor`, `ceil`, `round`, `truncate` and `to_i` on a Float raise `FloatDomainError` for a NaN or an infinity, as Ruby does, and `RangeError` where the Integer is past int64 -- except written straight into a float cell, which holds it whole (`1e20.floor` is `1e20`). The check costs a comparison per call, and in a loop that does little else it shows.
 - **A block's source must be recoverable.** Blocks defined in `eval` or in a console have no file to read back; set `RubyVM.keep_script_lines = true` before defining them, or hand the text to `CArray::JIT.compile`, which takes a kernel as a string rather than as a block. This also ties the gem to CRuby, which CArray requires anyway.
 - **Nothing existing is replaced.** `jit_for` is a new method, not a faster `each_index`: the two differ in what they reject, and a caller should be able to choose.

data/docs/04_Compiling.md

@@ -67,7 +67,7 @@ In `~/.cache/carray-jit` (honouring `XDG_CACHE_HOME`), one directory per version
 
 A kernel is only good for the version that generated it and the architecture it was built for, so those get their own directories. CArray's version is there for the same reason and one of its own: a kernel is handed CArray's memory, on layouts CArray decides, so one compiled against one version and run against the next would answer wrongly rather than fail to load.
 
-The two numbers say different things and move on their own clocks: the first is the version that generated the kernel, the second the version it was generated against. This gem is not versioned with CArray, and one release of it meets more than one -- the dependency is `>= 3.0.3, < 3.1`, so 3.0.3 and 3.0.4 both satisfy it -- and a layout the kernel reaches into can move between them. A new release does not spend its cache budget on entries nothing can reach any more, and a home directory shared between machines -- over NFS, or between Rosetta and native -- does not have one architecture evicting the other's kernels. A directory whose newest entry has not been touched in 30 days (`CARRAY_JIT_CACHE_MAX_AGE_DAYS`) is removed.
+The two numbers say different things and move on their own clocks: the first is the version that generated the kernel, the second the version it was generated against. This gem is not versioned with CArray, and one release of it meets more than one -- the dependency is `>= 3.0.5, < 3.1`, so 3.0.5 and 3.0.6 both satisfy it -- and a layout the kernel reaches into can move between them. A new release does not spend its cache budget on entries nothing can reach any more, and a home directory shared between machines -- over NFS, or between Rosetta and native -- does not have one architecture evicting the other's kernels. A directory whose newest entry has not been touched in 30 days (`CARRAY_JIT_CACHE_MAX_AGE_DAYS`) is removed.
 
 ```ruby
 CArray::JIT.cache_root                 #=> "/home/you/.cache/carray-jit"

data/docs/05_DesignNotes.md

@@ -4,7 +4,7 @@ Decisions that were not obvious, and why.
 
 ## The type is C's, and so is the arithmetic where the width is real
 
-The storage type is CArray's, mapped to C exactly -- `int64_t`, `uint8_t`, `float` -- and everything the type itself decides follows from that: the width, the wrap on store, the bit patterns. There a kernel agrees with CArray, because both are the same C. A shift agrees too, though the two get there differently: a kernel shifts in the width it computes in, `int64_t`, and narrows on store, where CArray shifts in the cell's own width -- and since both take a shift to mean what Ruby's Integer means by it, the low bits that survive the narrowing are the same.
+The storage type is CArray's, mapped to C exactly -- `int64_t`, `uint8_t`, `float` -- and everything the type itself decides follows from that: the width, the wrap on store, the bit patterns. There a kernel agrees with CArray's conversion between arrays (`to_type`), because both are the same C; CArray's store of a single Ruby Integer refuses one the cell cannot hold, where a kernel narrows it. A shift agrees too, though the two get there differently: a kernel shifts in the width it computes in, `int64_t`, and narrows on store, where CArray shifts in the cell's own width -- and since both take a shift to mean what Ruby's Integer means by it, the low bits that survive the narrowing are the same.
 
 The arithmetic follows the same rule where the width makes a difference to the answer, and Ruby's where it does not. That splits the types in two.
 

data/docs/08_LanguageReference.md

@@ -395,7 +395,7 @@ These are deliberate. Everything not listed here gives Ruby's answer, to the las
 - **A complex raised to a power** uses C's `cpow`, which can differ from Ruby in the last bit.
 - **A sine beside a cosine of the same argument** may be computed together by the C compiler, which can change the last bit.
 - **A reduction is reassociated** into partial sums unless `reassociate: false` is given, so its last bits can differ from adding in order.
-- **A real stored into an integer array** is C's conversion: past the type's range, or for a NaN, the result depends on the machine. Ruby would raise or grow.
+- **A real stored into an integer array** keeps its integer part, as CArray's store does, and one past the type's range raises `RangeError` where Ruby's `to_i` would grow. A NaN or an infinity raises `FloatDomainError`, as `to_i` does.
 
 **The language**
 
@@ -417,6 +417,7 @@ A block reads like Ruby run over one cell at a time, and it answers as that Ruby
 | `int8` cells: `a * 2 / 4` with `a = 100` | `-14`: each operation wraps at 8 bits | `50`: computed in `int64`, wrapped only when stored |
 | `int8` + `uint8` | computed and returned in `uint8` | computed in `int64` |
 | a uint8 cell `+ 300` | `RangeError`: 300 does not fit uint8 | `301`, computed in `int64` |
+| `int8` cell `= 300` | `RangeError`: 300 does not fit int8 (`to_type` gives `44`) | `44`: narrowed as C converts |
 | `x.round`, `floor`, `ceil`, `truncate` | a float array | an Integer, as Ruby |
 | boolean arithmetic, `b + b` | `2`: a boolean is 0/1 | refused, as Ruby refuses `true + true` |
 | a boolean compared with a number, `b == 1` | compares as 0/1 | refused: in Ruby `true == 1` is false |
@@ -424,4 +425,4 @@ A block reads like Ruby run over one cell at a time, and it answers as that Ruby
 | `\|` and `&` of a masked boolean (`UNDEF \| true`) | Kleene: `true` | `\|\|` and `&&`: masked whenever an operand is |
 | a `uint64` compared with a negative `int64` | compares by value: `7 < -7` is false | compared as two `uint64`: `7 < -7` is true |
 
-Where they agree, the block keeps CArray's answer: integer `/` and `%` floor, a shift means what it means to `Integer`, `INT64_MIN / -1` is `INT64_MIN`, overflow wraps, an Integer meets a Float as two doubles, narrow types compute narrow, `Math` outside its domain is NaN, a real out of range stored into an integer cell is C's conversion, and a masked operand gives a masked result.
+Where they agree, the block keeps CArray's answer: integer `/` and `%` floor, a shift means what it means to `Integer`, `INT64_MIN / -1` is `INT64_MIN`, overflow wraps, an Integer meets a Float as two doubles, narrow types compute narrow, `Math` outside its domain is NaN, a real stored into an integer cell keeps its integer part and raises `RangeError` past the type's range (`FloatDomainError` for a NaN or an infinity), and a masked operand gives a masked result.

data/lib/carray/jit/c_function.rb

@@ -55,6 +55,13 @@ class CArray
         "#{text.sub(/\s*\[.*\]\z/, "")} #{name}[#{array if sized?}]"
       end
 
+      # The pointer an array argument decays to, as C spells it:
+      # `const double coef[3]` takes a `const double *`.
+      def decayed
+        return text unless array
+        "#{text.sub(/\s*\[.*\]\z/, "")} *"
+      end
+
       # @return [String] the type, as C spells it.
       def to_s
         text

data/lib/carray/jit/c_generator.rb

@@ -421,6 +421,7 @@ class CArray
         @uses_real_arg = false
         @uses_real_power_complex = false
         @clamp_types = []
+        @store_types = []
         @gamma_types = []
         @complex_helpers = []
         @uses_floor_divide = false
@@ -727,6 +728,7 @@ class CArray
           :real_arg => @uses_real_arg,
           :real_power_complex => @uses_real_power_complex,
           :clamp => @clamp_types.dup,
+          :store => @store_types.dup,
           :gamma => @gamma_types.dup,
           :complex => @complex_helpers.dup,
           :index_check => @uses_index_check,
@@ -839,6 +841,7 @@ class CArray
           @uses_real_arg ||= needs[:real_arg]
           @uses_real_power_complex ||= needs[:real_power_complex]
           @clamp_types |= needs[:clamp] || []
+          @store_types |= needs[:store] || []
           @gamma_types |= needs[:gamma] || []
           @complex_helpers |= needs[:complex] || []
           @uses_index_check ||= needs[:index_check]
@@ -1095,6 +1098,31 @@ class CArray
           C
         end
         text << intrinsic_helpers
+        @store_types.each do |storage|
+          low, high = STORE_BOUNDS.fetch(storage)
+          c_type = storage_c_type(storage)
+          text << <<~C
+            /* A real number stored into an #{storage} cell keeps its integer
+               part, as CArray's store does.  One with no #{storage} to keep --
+               a NaN, an infinity, a number past the range -- is refused as
+               CArray refuses it: C leaves that conversion undefined, and the
+               machines answer it differently. */
+            static inline #{c_type}
+            carray_jit_store_#{storage} (double value, int32_t *error)
+            {
+              if ( value > #{low} && value < #{high} ) {
+                return (#{c_type}) value;
+              }
+              if ( error ) {
+                *error = isnan(value) ? #{FLOAT_NAN_CODE} :
+                         isinf(value) ? ( value > 0 ? #{FLOAT_INFINITY_CODE} : #{FLOAT_NEGATIVE_INFINITY_CODE} ) :
+                         #{STORE_RANGE_CODES.fetch(storage)};
+              }
+              return 0;
+            }
+
+          C
+        end
         @clamp_types.each do |type|
           suffix, c_type, has_nan = CLAMP_C_TYPES.fetch(type)
           refusal = lambda { |test, code|
@@ -1834,6 +1862,14 @@ class CArray
       FLOAT_NEGATIVE_INFINITY_CODE = -5
       INTEGER_RANGE_CODE = -6
 
+      # And what a real number stored into an integer cell reports where the
+      # cell's type has no integer for it, one code for each type, so the
+      # message can name the type as CArray's own store does.
+      STORE_RANGE_CODES = {
+        "int8"  => -7,  "int16"  => -8,  "int32"  => -9,  "int64"  => -10,
+        "uint8" => -11, "uint16" => -12, "uint32" => -13, "uint64" => -14,
+      }.freeze
+
       # The fixed failures a body reports, as the exception each one raises.
       # A kernel and a compiled function both look here, so the two say the
       # same thing for the same cell.
@@ -1842,6 +1878,24 @@ class CArray
         FLOAT_INFINITY_CODE => [FloatDomainError, "Infinity"],
         FLOAT_NEGATIVE_INFINITY_CODE => [FloatDomainError, "-Infinity"],
         INTEGER_RANGE_CODE => [RangeError, "a rounded Float is past the range of int64"],
+        **STORE_RANGE_CODES.to_h { |storage, code|
+          [code, [RangeError, "a Float is out of range for #{storage}"]]
+        },
+      }.freeze
+
+      # The bounds a real number has to fall strictly between for its integer
+      # part to be a value of the type, written as doubles, which hold every
+      # one of them exactly.  CArray's store keeps the integer part, so -0.5
+      # goes into a uint8 cell as 0, and 255.9 as 255.
+      STORE_BOUNDS = {
+        "int8"   => ["-129.0", "128.0"],
+        "int16"  => ["-32769.0", "32768.0"],
+        "int32"  => ["-2147483649.0", "2147483648.0"],
+        "int64"  => ["-9223372036854777856.0", "9223372036854775808.0"],
+        "uint8"  => ["-1.0", "256.0"],
+        "uint16" => ["-1.0", "65536.0"],
+        "uint32" => ["-1.0", "4294967296.0"],
+        "uint64" => ["-1.0", "18446744073709551616.0"],
       }.freeze
 
       RAISE_CODE_FLOOR = 3
@@ -2588,9 +2642,9 @@ class CArray
         # expression written into it carried, and the branches the write
         # stands in.  A write reached because a missing cell said so was
         # reached for no reason, whichever of the three it writes.
+        masked = @masked ? combine_masks(@carried_masks + [emit_mask(write.expression)]) : nil
         mask = if @masked
-                 "#{indent}#{local_array_reference(write, :mask)} = " \
-                 "#{combine_masks(@carried_masks + [emit_mask(write.expression)])};\n"
+                 "#{indent}#{local_array_reference(write, :mask)} = #{masked};\n"
                else
                  ""
                end
@@ -2600,10 +2654,10 @@ class CArray
           "#{indent}#{COMPUTATION_C_TYPES.fetch(type)} #{temporary};\n" +
             emit_conditional_statement(expression, temporary, type, indent) +
             "#{indent}#{target} = " \
-            "#{cast_to_storage(temporary, type, write.storage)};\n" + mask
+            "#{cast_to_storage(temporary, type, write.storage, masked: masked)};\n" + mask
         else
           type = complex_storage_of_power(expression, write.storage) || expression.type
-          value = cast_to_storage(emit(expression, type), type, write.storage)
+          value = cast_to_storage(emit(expression, type), type, write.storage, masked: masked)
           "#{indent}#{target} = #{value};\n" + mask
         end
       end
@@ -2999,13 +3053,41 @@ class CArray
         @storage_types.fetch(array)
       end
 
-      def cast_to_storage (text, type, storage)
+      # A captured array's element type is matched at the call
+      # (`address_buffer`), after the build, and raises before the kernel
+      # runs.  Where the declaration spells the pointer differently from the
+      # array's own, the address is cast to it, so that check is the one that
+      # answers: gcc refuses an incompatible pointer at compile time where
+      # clang only warns.  `long long *` is such a spelling for an int64
+      # array, which the check accepts and gcc does not.  A body's own
+      # pointer parameter is no captured array and is handed on as it is:
+      # its type was declared in C.
+      def emit_array_argument (argument, parameter)
+        address = emit(argument, :address)
+        return address unless @storage_types.key?(argument.array)
+        own = "#{storage_c_type(array_storage(argument.array))} *"
+        declared = parameter.decayed.sub(/\Aconst\s+/, "")
+        declared == own ? address : "(#{parameter.decayed})#{address}"
+      end
+
+      # `masked` is the C of the written cell's mask, for a write whose mask
+      # the error slot does not already know about: a missing cell holds
+      # whatever was under its mask, which is no value to refuse.
+      def cast_to_storage (text, type, storage, masked: nil)
         # Storing into a boolean array normalises: CArray's boolean is a byte
         # holding 0 or 1, and a kernel is not the place that invariant stops
         # being true.
         return "(uint8_t)((#{text}) ? 1 : 0)" if boolean_storage?(storage)
         target = storage_c_type(storage)
         return text if COMPUTATION_C_TYPES.fetch(type) == target
+        if [:double, :float].include?(type) and STORE_BOUNDS.key?(storage)
+          @store_types |= [storage]
+          error = error_argument
+          if masked and masked != "0"
+            error = "(#{masked} ? (int32_t *) 0 : #{error})"
+          end
+          return "carray_jit_store_#{storage}(#{text}, #{error})"
+        end
         "(#{target})(#{text})"
       end
 
@@ -3563,8 +3645,9 @@ class CArray
           c_function = @c_functions.fetch(node.name)
           arguments = node.arguments.zip(c_function.parameters)
                           .map { |argument, parameter|
-                            if argument.is_a?(ArrayAddress) ||
-                               argument.is_a?(LocalArrayAddress)
+                            if argument.is_a?(ArrayAddress)
+                              emit_array_argument(argument, parameter)
+                            elsif argument.is_a?(LocalArrayAddress)
                               emit(argument, :address)
                             else
                               emit(argument, parameter.computation)
@@ -3623,12 +3706,15 @@ class CArray
          "#{emit(node.high, type)}, #{error_argument})", LEAF_PRECEDENCE]
       end
 
-      # `isnan` and `isfinite` are C's own, and take a number of any width.
-      # A Complex answers `finite?` the way Ruby answers it -- both parts
-      # finite -- through a helper, so that the number is computed once.
-      # An Integer is finite whatever it holds, and goes through `isfinite`
-      # all the same rather than being answered here: the read it stands for
-      # is a read, and dropping it would drop what the read reports.
+      # `isnan` and `isfinite` are C's own, and take a real floating number
+      # of either width.  A Complex answers `finite?` the way Ruby answers
+      # it -- both parts finite -- through a helper, so that the number is
+      # computed once.  An Integer is finite whatever it holds, and goes
+      # through `isfinite` all the same rather than being answered here: the
+      # read it stands for is a read, and dropping it would drop what the
+      # read reports.  It goes in as a double, because C asks for a floating
+      # argument and gcc refuses an integer one where clang lets it pass;
+      # every integer converts to a finite double.
       def emit_numeric_predicate (node)
         type = node.operand.type
         function = node.name == :nan? ? "isnan" : "isfinite"
@@ -3636,7 +3722,11 @@ class CArray
           return emit_helper_call("complex_finite", [emit(node.operand, type)],
                                   type == :float_complex)
         end
-        ["#{function}(#{emit(node.operand, type)})", LEAF_PRECEDENCE]
+        operand = emit(node.operand, type)
+        if TypeAssignment::INTEGER_TYPES.include?(type)
+          operand = "(double)(#{operand})"
+        end
+        ["#{function}(#{operand})", LEAF_PRECEDENCE]
       end
 
       def emit_mask_test (node)

data/lib/carray/jit/expression.rb

@@ -40,7 +40,17 @@ class CArray
       # multiply and an add into one rounding, so it compiles with
       # -ffp-contract=off.  Here the reference is the eager kernel, which was
       # built with whatever CArray settled on.
-      FLAGS = ["-fPIC", "-shared", *CArray::BUILD_FLAGS.split, *DYNAMIC_LOOKUP].freeze
+      #
+      # Except between nodes.  CArray computes each node in a kernel of its
+      # own, so no node's multiply meets the next node's add; this object
+      # writes the expression as one statement per node in a single loop.
+      # clang contracts within a statement by default, which leaves that
+      # alone, but gcc defaults to -ffp-contract=fast and contracts across
+      # statements: on x86 with -march=native, `a * b + c` lost a rounding in
+      # 3296 of 30000 cells.  -ffp-contract=on keeps a contraction inside a
+      # node's own body, as CArray's kernel has it, and none between nodes.
+      FLAGS = ["-fPIC", "-shared", *CArray::BUILD_FLAGS.split,
+               "-ffp-contract=on", *DYNAMIC_LOOKUP].freeze
 
       C_TYPES = {
         float64: "double",  float32: "float",

data/lib/carray/jit/kernel.rb

@@ -235,9 +235,10 @@ class CArray
 
         held = [buffer(reals), buffer(integers), buffer(functions),
                 buffer(data), masks, masks, error]
-        context = held.map { |item| Fiddle::Pointer[item].to_i }.pack("Q*")
-
-        Sweep.call(slab_pointer, fsync, arrays, Fiddle::Pointer[context])
+        error = in_one_block(held) { |context, slots|
+          Sweep.call(slab_pointer, fsync, arrays, context)
+          slots.last[0, error.bytesize]
+        }
 
         packed.each_with_index do |(array, item), index|
           next if array.equal?(item)
@@ -558,6 +559,31 @@ class CArray
         packed.empty? ? "\0" * 8 : packed
       end
 
+      # Copies `parts` into one block of C memory, each part 8-byte aligned,
+      # after a table of their addresses, and yields a pointer to the table
+      # and a pointer to each part; the block is freed when this returns.
+      #
+      # For a call that is handed addresses inside other memory rather than
+      # arguments: a Ruby String's bytes may be moved by GC compaction, and
+      # an address taken from one and packed elsewhere is not one Ruby knows
+      # to keep in place.  Memory from malloc is never moved.
+      def in_one_block (parts)
+        table = parts.size * Fiddle::SIZEOF_VOIDP
+        offsets = parts.each_with_object([]) { |part, list|
+          last = list.empty? ? table : list.last + parts[list.size - 1].bytesize
+          list << ((last + 7) & ~7)
+        }
+        size = offsets.last + parts.last.bytesize
+        Fiddle::Pointer.malloc(size, Fiddle::RUBY_FREE) do |block|
+          slots = parts.zip(offsets).map { |part, offset|
+            block[offset, part.bytesize] = part
+            block + offset
+          }
+          block[0, table] = slots.map(&:to_i).pack("Q*")
+          yield block, slots
+        end
+      end
+
       # The xfer tier transfers a box rather than the whole view.  The box
       # is the loop range grown by how far the kernel reaches from the cell it
       # is on -- per array, because two arrays in one kernel need not be read
@@ -778,9 +804,9 @@ class CArray
           unless array.is_a?(CArray)
             raise Unsupported, "`#{name}` is not a CArray"
           end
-          unless array.rank == array_rank(name)
+          unless array.ndim == array_rank(name)
             raise Unsupported,
-                  "`#{name}` has rank #{array.rank}, but is indexed with " \
+                  "`#{name}` has rank #{array.ndim}, but is indexed with " \
                   "#{array_rank(name)} #{array_rank(name) == 1 ? 'index' : 'indices'}"
           end
           unless array.data_type_name == @storage_types.fetch(name)

data/lib/carray/jit/version.rb

@@ -1,5 +1,5 @@
 class CArray
   module JIT
-    VERSION = "0.1.4"
+    VERSION = "0.1.6"
   end
 end

data/lib/carray/jit.rb

@@ -638,10 +638,10 @@ class CArray
                   "the indices of a term are symbols naming its axes, as in " \
                   "`[a, [:i, :k]]`; got #{subscripts.inspect}"
           end
-          unless subscripts.size == array.rank
+          unless subscripts.size == array.ndim
             raise Unsupported,
                   "a term names one index per axis: this array has rank " \
-                  "#{array.rank} and #{subscripts.size} " \
+                  "#{array.ndim} and #{subscripts.size} " \
                   "#{subscripts.size == 1 ? 'index' : 'indices'} were given"
           end
         end
@@ -680,9 +680,9 @@ class CArray
           raise Unsupported, "`into:` is an array to write into"
         end
         expected = free.empty? ? 1 : free.size
-        unless into.rank == expected
+        unless into.ndim == expected
           raise Unsupported,
-                "`into:` has rank #{into.rank}, and the result's axes are " \
+                "`into:` has rank #{into.ndim}, and the result's axes are " \
                 "#{free.empty? ? 'none, which is one cell' : free.map { |name| "`#{name}`" }.join(', ')}"
         end
       end
@@ -909,7 +909,7 @@ class CArray
         return false if masked
         return false if arrays.empty? || arrays.size > Sweep::MAX_ARITY
         return false unless arrays.each_value.all? { |array|
-          array.rank.zero? || array.dim == shape
+          array.ndim.zero? || array.dim == shape
         }
         walkable_in_place?(arrays.each_value)
       end
@@ -933,7 +933,7 @@ class CArray
       # 0.9 ns/cell with the driver here against 6.1 ns/cell swept, and no
       # scratch either way -- there was nothing the re-gather was buying.
       def walkable_in_place? (arrays)
-        tiers = arrays.map { |array| Access.classify(array)[:tier] }
+        tiers = arrays.map { |array| Access.tier(array) }
         return true if tiers.include?(Access::TIER_XFER)
         tiers.none? { |tier| tier == Access::TIER_STRIDE }
       end
@@ -1214,6 +1214,23 @@ class CArray
       # common case allocates nothing.
       EMPTY_ARRAYS = {}.freeze
 
+      # What an element-wise pass works out from its operands' shapes and
+      # types, and not from their data: which of them are walked and which
+      # are handed to a C function whole, the shape the pass covers, whether
+      # lining the walked ones up to it changes any of them, which loop runs
+      # it, the kernel, and what type a map's result is made as.
+      #
+      # @private
+      RunPlan = Data.define(:walked, :addressed, :stretched, :shape,
+                            :sweeping, :kernel, :result_type)
+
+      # How many plans one block keeps.  Not one: a caller alternating
+      # between two model states would miss on every call, and pay for the
+      # asking on top of what every call paid before.  Eight, as a kernel
+      # keeps of its own calls (CompiledKernel::PLAN_LIMIT), and for no
+      # better-measured reason.
+      RUN_PLAN_LIMIT = 8
+
       # @private
       def run_over_whole_arrays (block, map: false)
         node, source, origin = read_block(block)
@@ -1238,11 +1255,185 @@ class CArray
         # is before the broadcast.
         made = local_array_names(source, node)
         refuse_a_shadowed_array(made, outside)
-        arrays = arrays.merge(assigned_arrays(assigned - made, outside))
+        written = assigned_arrays(made.empty? ? assigned : assigned - made,
+                                  outside)
+        arrays = arrays.merge(written) unless written.empty?
         if arrays.empty?
           raise Unsupported, "the block reaches no array"
         end
 
+        # Everything from here to the kernel answers the same for two calls
+        # whose operands agree on shape and type, so it is worked out once
+        # per such agreement rather than once per call (#plan_run).  What the
+        # plan leaves to the call is what it has to: the operands themselves,
+        # the views the broadcast makes of them, a generator's state, the
+        # map's result, and the masks.
+        key = run_key(arrays, scalars, c_functions, randoms, map)
+        plan = key && found_run_plan(source, key)
+        unless plan
+          plan = plan_run(source, node, origin, arrays, scalars, c_functions,
+                          randoms, map)
+          keep_run_plan(source, key, plan) if key
+        end
+
+        walked = plan.addressed.empty? ? arrays : arrays.slice(*plan.walked)
+        # Where no operand is stretched the broadcast hands back the operands
+        # it was given, so it is not asked.  Where one is, the views it makes
+        # are views of these arrays and are made again: a view is exactly
+        # what a plan must not keep.
+        #
+        # Unstretched, `aligned` can be `arrays` itself.  That is sound only
+        # because every write into it below puts back the array already
+        # there.
+        aligned = plan.stretched ? broadcast(walked).first : walked
+        plan.addressed.each { |name| aligned[name] = arrays.fetch(name) }
+        # A generator's state joins after the broadcast, never before it.
+        # It is passed whole rather than walked, so the expression's shape
+        # has nothing to say about it -- and it has a shape of its own that
+        # would not line up anyway: four cells, against however many the
+        # expression covers.  Lining it up first is what `jit_each` did to a
+        # one-cell state before 314d6cb, and a four-cell one does not even
+        # stretch.
+        unless randoms.empty?
+          states = random_states(randoms)
+          arrays = arrays.merge(states)
+          aligned = aligned.merge(states)
+        end
+        shape = plan.shape
+        if map
+          result = CArray.send(plan.result_type, *(shape.empty? ? [1] : shape))
+          arrays = arrays.merge(MAP_RESULT => result)
+          aligned = aligned.merge(MAP_RESULT => result)
+        end
+
+        kernel = plan.kernel
+        # The shape of each of these was checked against the pass's when the
+        # plan was made, and the key holds the shapes.
+        kernel.written_arrays.each do |name|
+          next if name == MAP_RESULT
+          aligned[name] = arrays.fetch(name)
+        end
+
+        aligned = read_once_where_the_pass_writes(aligned, kernel.written_arrays)
+
+        # An array handed to a C function by address is passed whole rather
+        # than walked, so the expression's shape has nothing to say about it.
+        # Broadcasting it would stretch a one-cell state array into a
+        # read-only `CARepeat`, which the copy-back after the call cannot
+        # write through -- and a state array is exactly the shape a caller
+        # reaches for.
+        kernel.address_arrays.each do |name|
+          aligned[name] = arrays.fetch(name)
+        end
+
+        if kernel.masked
+          kernel.written_arrays.each do |name|
+            array = arrays.fetch(name)
+            array.mask = 0 unless array.has_mask?
+          end
+        end
+
+        result.mask = 0 if map && kernel.masked && !result.has_mask?
+        drive(kernel, aligned, scalars, c_functions, shape, plan.sweeping)
+        map ? result : kernel
+      end
+
+      # What two calls of one block have to agree on for one to be run as
+      # the other was.
+      #
+      # Each operand's name, type and shape, because that is what the kernel
+      # is compiled for and the broadcast lines up; whether it is a CScalar,
+      # because the broadcast stretches one where it leaves a one-cell array
+      # standing.  Whether it has a mask, because a mask is runtime state that
+      # leaves the shape and the type alone -- one appearing makes a masked
+      # kernel, and nothing else here would notice it.  And the tier it would
+      # be opened at, because that decides whether CArray's sweep or the
+      # tiers here run the pass (#walkable_in_place?).  All of it read off
+      # the operand on this call; none of it kept from the last.
+      #
+      # Then the type of each scalar, each C function's signature and each
+      # generator's kind, which is what the kernel is compiled for of those.
+      # Not a scalar's value: a clock or a coefficient changes every step,
+      # and keying on it would miss on every call of the loop this is for.
+      #
+      # Values only, never an operand: a block's plans live as long as the
+      # process, and holding an array would hold its memory as long.  Shapes
+      # from `ndim`, `elements` and `dim0`.. rather than `dim`, which builds
+      # an Array on every call.
+      #
+      # nil where a scalar is not one a kernel takes, so that the call is
+      # refused where and as it always was rather than here.
+      def run_key (arrays, scalars, c_functions, randoms, map)
+        key = [map, arrays.size]
+        arrays.each do |name, array|
+          rank = array.ndim
+          key << name << array.data_type << rank << array.is_a?(CScalar) <<
+                 array.has_mask? << Access.tier(array)
+          case rank
+          when 1 then key << array.elements
+          when 2 then key << array.dim0 << array.dim1
+          when 3 then key << array.dim0 << array.dim1 << array.dim2
+          else        key.concat(array.dim)
+          end
+        end
+        key << scalars.size
+        scalars.each do |name, value|
+          key << name << TypeAssignment.scalar_type(value)
+        end
+        key << c_functions.size
+        c_functions.each { |name, function| key << name << function.kernel_key }
+        randoms.each { |name, generator| key << name << generator.generator }
+        key
+      rescue Unsupported
+        nil
+      end
+
+      # @private
+      def run_plans
+        @run_plans ||= {}
+      end
+
+      # The block's plan for this key, if it has one.  The list a plan sits
+      # in is never changed once it is published -- a kernel's call lets go
+      # of the GVL, so another thread may be reading it -- and bringing a
+      # plan to the front replaces the list rather than editing it.
+      def found_run_plan (source, key)
+        plans = run_plans[source]
+        return nil unless plans
+        found = plans.assoc(key)
+        return nil unless found
+        unless plans.first.equal?(found)
+          run_plans[source] =
+            [found, *plans.reject { |entry| entry.equal?(found) }].freeze
+        end
+        found[1]
+      end
+
+      # @private
+      def keep_run_plan (source, key, plan)
+        entry = [frozen_throughout(key), plan].freeze
+        run_plans[source] =
+          [entry, *run_plans[source]].first(RUN_PLAN_LIMIT).freeze
+      end
+
+      # A copy that nothing outside can change under it, and that changes
+      # nothing outside by being frozen: a C function's signature is its own
+      # strings, and freezing those in place would freeze them in the
+      # function.
+      def frozen_throughout (value)
+        case value
+        when Array  then value.map { |item| frozen_throughout(item) }.freeze
+        when String then value.frozen? ? value : value.dup.freeze
+        else value
+        end
+      end
+
+      # Works out what #run_over_whole_arrays keeps between calls, the way it
+      # always worked it out on every call.  Every refusal a pass can meet is
+      # met here, before a plan exists, so a plan is only ever of a call that
+      # ran.
+      def plan_run (source, node, origin, arrays, scalars, c_functions,
+                    randoms, map)
         # An array the block only hands to a C function whole is not walked,
         # so the expression's shape has nothing to say about it -- and it has
         # a shape of its own that need not line up with anything.  Left in,
@@ -1250,15 +1441,10 @@ class CArray
         # axis rather than the call it was written for.
         #
         # So it joins after the broadcast, which is what a generator's state
-        # already does and for the same reason.  The re-substitution further
-        # down stays: this is a static reading, and the analyzer's answer is
-        # the final one.
+        # already does and for the same reason.  The re-substitution in the
+        # caller stays: this is a static reading, and the analyzer's answer
+        # is the final one.
         handed_over = address_array_names(source, node, c_functions)
-        # The common case is that there are none, and it pays nothing: no
-        # hash is built and the broadcast is handed the operands it always
-        # was.  Every call comes through here -- `jit_each` is re-entered per
-        # call -- so the work that is only needed sometimes is done only
-        # then.
         addressed = handed_over.empty? ? EMPTY_ARRAYS
                                        : arrays.slice(*handed_over)
         walked = addressed.empty? ? arrays
@@ -1274,22 +1460,13 @@ class CArray
         end
 
         aligned, shape = broadcast(walked)
-        addressed.each { |name, array| aligned[name] = array }
-        # A generator's state joins after the broadcast, never before it.
-        # It is passed whole rather than walked, so the expression's shape
-        # has nothing to say about it -- and it has a shape of its own that
-        # would not line up anyway: four cells, against however many the
-        # expression covers.  Lining it up first is what `jit_each` did to a
-        # one-cell state before 314d6cb, and a four-cell one does not even
-        # stretch.
-        states = random_states(randoms)
-        arrays = arrays.merge(states)
-        aligned = aligned.merge(states)
+        stretched = aligned.any? { |name, array| !array.equal?(walked[name]) }
+        arrays = arrays.merge(random_states(randoms))
         if map
-          result = allocate_map_result(source, node, arrays, scalars, shape,
-                                       c_functions, randoms)
+          result_type = probe_map(source, node, arrays, scalars, c_functions,
+                                  randoms)
+          result = CArray.send(result_type, *(shape.empty? ? [1] : shape))
           arrays = arrays.merge(MAP_RESULT => result)
-          aligned = aligned.merge(MAP_RESULT => result)
         end
         # Both questions are about the arrays the loop walks.  An array
         # handed over whole is not one of them: its mask is not something the
@@ -1321,31 +1498,13 @@ class CArray
                   "expression covers #{shape.inspect}; a stretched array " \
                   "cannot be written to"
           end
-          aligned[name] = written
-        end
-
-        aligned = read_once_where_the_pass_writes(aligned, kernel.written_arrays)
-
-        # An array handed to a C function by address is passed whole rather
-        # than walked, so the expression's shape has nothing to say about it.
-        # Broadcasting it would stretch a one-cell state array into a
-        # read-only `CARepeat`, which the copy-back after the call cannot
-        # write through -- and a state array is exactly the shape a caller
-        # reaches for.
-        kernel.address_arrays.each do |name|
-          aligned[name] = arrays.fetch(name)
-        end
-
-        if kernel.masked
-          kernel.written_arrays.each do |name|
-            array = arrays.fetch(name)
-            array.mask = 0 unless array.has_mask?
-          end
         end
 
-        result.mask = 0 if map && kernel.masked && !result.has_mask?
-        drive(kernel, aligned, scalars, c_functions, shape, sweeping)
-        map ? result : kernel
+        RunPlan.new(walked: walked.keys.freeze,
+                    addressed: addressed.keys.freeze,
+                    stretched: stretched, shape: shape.freeze,
+                    sweeping: sweeping, kernel: kernel,
+                    result_type: result_type)
       end
 
       # `a = b * 2.0` reads the whole of the right-hand side and then assigns,
@@ -1394,7 +1553,7 @@ class CArray
         return if written.empty?
         roots = written.map { |array| root_of(array) }
         arrays.each do |name, array|
-          next unless Access.classify(array)[:tier] == Access::TIER_XFER
+          next unless Access.tier(array) == Access::TIER_XFER
           root = root_of(array)
           # Another operand on the same storage, and one of the two written:
           # a copy taken before the loop cannot answer for either of them.
@@ -1434,7 +1593,7 @@ class CArray
       def broadcast (arrays)
         values = arrays.values
         rank = values.reject { |value| value.is_a?(CScalar) }
-                     .map(&:rank).max || 1
+                     .map(&:ndim).max || 1
         values = values.map { |value|
           value.is_a?(CScalar) ? value.refer(value.data_type, [1] * rank) : value
         }
@@ -1695,6 +1854,9 @@ class CArray
         @undef_cache = {}
         @probe_cache = {}
         @local_array_name_cache = {}
+        # A plan holds the kernel it runs, so one left behind would hand the
+        # next call a kernel the registry no longer has.
+        @run_plans = {}
       end
 
       # @return [String] the directory this environment's kernels are kept in,

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: carray-jit
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.1.6
 platform: ruby
 authors:
 - himotoyoshi
@@ -15,7 +15,7 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 3.0.3
+        version: 3.0.5
     - - "<"
       - !ruby/object:Gem::Version
         version: '3.1'
@@ -25,7 +25,7 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 3.0.3
+        version: 3.0.5
     - - "<"
       - !ruby/object:Gem::Version
         version: '3.1'