carray 3.0.3 → 3.0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76ca6c2983cbd1e07acc64daeedc7328bee9239369f56f3af661ed1cc41e0bfb
-  data.tar.gz: 1506fa95c68148469451631fb375826e2599a979b4593db8c7508cf16d3c7545
+  metadata.gz: ccd30e8ea5b53c3a2654a86a0ab7473b3073f7feffa7db7be0e5c9aef5bc17c8
+  data.tar.gz: ffcd91fe93ca10b4b12dbff035ffc25533699472b364482a59bf2aa635296032
 SHA512:
-  metadata.gz: 66a3078a1bdd6459ab0c7526a588cc81191a4e043a17028b769b87b5d32874a9fd20c2b4828217dfa573457871f476155d005db518da600bbef38baca5551eb8
-  data.tar.gz: 23479b0a9b229489ff5ed1afa7fdc0a30150b9d3ba32803b089f1f9e5d7a955cd556c478c7cb5346e516830632a4ba35cb54d186f54a783b589d7cabec9498d5
+  metadata.gz: 53c08f4ccf6a4c25219cef4e8f27e7de9740dc8c69ee493ff8e3631c87499cd15ed121d5f7707bd1017f77e0309dc1be2628fa3611b9b3c979e0e075008842d6
+  data.tar.gz: 2200b047a456603d6274d9dfaf8f93c011ce1cc1eb457c454fa7f9cdc9c75ea4b96c9bbe1dab0b37710f8b872ac5e8b630db5c0dacae12a9012b1b34d4c7f1ec

data/CHANGELOG.md

@@ -34,6 +34,475 @@ and a newer one. The 1.x history, up to the 2.0.0 release, is in
      neighbour ("as well", "the producer above") or leave unnamed the
      method, class or keyword it is about. -->
 
+## 3.0.5
+
+- New: `GroupedFrame#aggregate` takes a block instead of a spec. The block
+  is given the grouping once and returns a Hash of output name => a
+  CArray with one value per group, built from `g["col"]` iterators and
+  CArray arithmetic (`g["temp"].max - g["temp"].min`). `grp[array]`
+  groups any CArray with one entry per row, such as a value computed
+  from columns or a column with cells masked.
+
+- New: a `CAFrame` grouped reduction takes a Hash of column name => value
+  as `fill_value:`, filling only the named columns. A single
+  `fill_value:` still goes to every column and has to suit each.
+
+- New: `CArray.meld` brings pieces of different data types to a common
+  one, as `CArray.stack` does, and takes `data_type:`. `CAFrame.meld` and
+  a frame splice still refuse a column whose pieces differ in data type;
+  cast the column first, or use `concatenate`.
+
+- New: `scatter_replace!` takes an object or fixed-length array, as
+  `put_nd` and `put_along_axis` do. The accumulating members
+  (`scatter_add!` and the others) still take numeric arrays only.
+
+- New: `CAConstString#count` and `CACategorical#count` with no argument
+  count the cells that are not masked, as `CArray#count` does.
+
+- New: `CAFixlenString` answers `minmax`, `cummax` and `cummin`, as
+  `CAString` does.
+
+- New (C extensions): `carray.h` defines `ca_u_<type>` cell types and
+  `CA_U(type)`, of the same size and value as the plain types but with no
+  alignment requirement, for reading cells that need not be aligned (a
+  refer with a byte offset, a struct member, a wrapped buffer). The slab
+  macros in `ca_kernel_iterator.h` use them for their pointers.
+
+- Change: `CACategorical#sort_addr` is renamed `group_addr`, the order of
+  the cells by category. `sort_addr` on a categorical now raises, as
+  `sort_index` does, since a categorical has no order of values.
+
+- Change: a Face's `parent` (of a `CATime`, `CARecord`, ...) is a
+  read-only view of its storage. Write the storage of a time through
+  `CATime#ticks`.
+
+- Change: an operator that cannot bring two data types together raises
+  `CArray::DataTypeError` instead of `TypeError`. `DataTypeError` is a
+  `TypeError`, so `rescue TypeError` still catches it.
+
+- Change: a float cell with no integer of the target type -- an infinity,
+  or a value out of the type's range -- is now treated the way NaN
+  already was. `to_type` (`int32`, `as_int64`, ...) masks it to `UNDEF`;
+  storing it into an integer array raises `RangeError`
+  (`FloatDomainError` for an infinity). Before, it became whatever the
+  machine's C conversion gave, which differed between x86 and ARM.
+  `clip_int64` and the other `clip_<type>` methods saturate a float
+  source to the type's limits on every machine; integer `linspace` raises
+  `RangeError` for a span the type cannot hold.
+
+- Change: a Ruby Array operand of a numeric or boolean array is read as
+  its cells: `a.eq([1, 5, 3])` compares cell by cell (it compared every
+  cell with the whole Array), and `a + [1, 2, 3]` adds (it raised).
+  Nested Arrays give the axes. An object array still takes an Array as
+  one value.
+
+- Change: `window`'s `bounds:` is a policy named by a Symbol or String,
+  matched whole; an Integer and an abbreviation raise. A `fill_value:`
+  given with a policy that does not use it (`:nearest`, `:strict`, ...)
+  raises instead of being dropped.
+
+- Change: a length, count, offset or shift given to a view
+  (`sliding_windows`, `block_view`, `tile`, `roll`, `shift`, `field`,
+  `diagonal`) must be an Integer; a Float raises `TypeError` instead of
+  being truncated. `transpose` reads its axes as the other axis arguments
+  are read: negative from the end, an Integer only.
+
+- Change: `drop_axis` on an array whose every axis has size 1 keeps one
+  axis of size 1, and `reshape` with no axes raises.
+
+- Change: in an index of one key per axis, an index array of more than
+  one axis on one axis raises `ArgumentError`; it was read flat. Pass
+  `idx.flatten` for the old reading.
+
+- Change: `elem_fetch`, `elem_store` and the other `elem_*` methods read
+  an index as `[]` reads a key: a Float with no fraction is an index, a
+  Float with a fraction raises `IndexError`.
+
+- Change: two string arrays of different kinds (`CAString`,
+  `CAFixlenString`, `CAConstString`) or of different widths compare by
+  their strings. A String query longer than a fixed-width cell matches
+  no cell in `eq`, `count`, `search`, `bsearch` and `is_in`; it was cut
+  to the cell's width.
+
+- Change: `str_index` and `rindex` give `UNDEF` for a cell that does not
+  hold the substring; they raised for the whole array.
+
+- Change: `bsearch` and `bsearch_addr` answer the first of several equal
+  cells.
+
+- Change: `search` and `search_addr` raise `ArgumentError` for a negative
+  or NaN `eps`, which acted as an exact match.
+
+- Change: `inherit_mask` and `inherit_mask_replace` raise `ArgumentError`
+  for a source whose number of cells differs from the receiver's (a
+  scalar source still applies to every cell).
+
+- Change: a reduction's `fill_value:` that its result cannot hold raises
+  even when no cell needs it, and `fill` on an empty array checks its
+  value as an assignment does.
+
+- Change: rounding a `:M` or `:Y` `CATime` to a calendar bucket
+  (`round(unit: :Y)`, `"3 months"`) measures the distance by the instant,
+  so it agrees with the same dates on `:D`.
+
+- Change: a `CATime` reduction (`mean`, `median`, ...) sends a tie to the
+  later tick on both sides of 1970; before 1970 it went to the earlier
+  one.
+
+- Change: a duration String added to or subtracted from a `CATime` is
+  one step of its own resolution, so `"15 minutes"` on a `"10 minutes"`
+  grid raises, as the same duration as a `CATimedelta` does. It was read
+  as fifteen one-minute steps and cut to the grid.
+
+- Change: of `-0.0` and `0.0`, `pmax` gives `0.0` and `pmin` gives
+  `-0.0`, in either order and on every platform, as `maximum` and
+  `minimum` do. Which zero `min` and `max` answer is not specified.
+
+- Change: a one-sided `clip` passes a NaN cell through, as a two-sided
+  `clip` does.
+
+- Change: a categorical's `fill_value:` (of `shift`, `window`, `project`)
+  is a label; a `CAConstString` takes only `UNDEF` as a fill value.
+
+- Change: a lazy operator with a Face operand (a categorical, a time, a
+  string array) answers as the eager operator does; it read the Face's
+  storage. `CAConstString#eq` with another array compares cell by cell.
+
+- Change: a boolean member of a record (or `field(offset, :boolean)` of a
+  fixed-length array) raises `RangeError` when read as a byte that is not
+  0 or 1, instead of answering differently in each method.
+
+- Change: unmasking a cell of a `CAString` that holds no String gives an
+  empty String.
+
+- Change: `swap_bytes` keeps the attributes, as `refer` and `bitfield`
+  do. An array with no cells has no say in the attributes a `meld`,
+  `stack` or `concatenate` keeps.
+
+- Change: `dim_view` keeps a Face, as `transpose` does.
+
+- Change: `CArray.string(n)` without a block, and `CArray.string("ab")`,
+  raise; `fixlen_string` and `to_fixlen_string` raise for `bytes:` below 1.
+  `project` raises for a keyword argument, `paste` into a frozen array
+  raises `FrozenError`, and `CArray.result_type` refuses a MemoryView
+  operand whose format carray cannot read, as the operators do.
+
+- Change: `CArray.sort_addr` with numeric keys is two to three times as fast, and
+  `nlargest` / `nsmallest` read each fiber once with a work area of a
+  few times n.
+
+- Fix: x86-64 Linux. Inputs that reached an integer division by zero
+  inside the library stopped the process with SIGFPE there (on ARM they
+  were refused later or gave a wrong answer): a sum over a view with a
+  zero-length axis, `wsum` / `wmean` with 1-D weights over a zero-length
+  axis, a view of a stack of empty arrays, `window` with a step of 0, and
+  `wrap_readonly(String, CA_FIXLEN)`. Each now answers or raises
+  `ArgumentError`. The extension builds with GCC 14 without warnings.
+
+- Fix: big-endian hosts. `bitfield` read 0 and lost writes, a struct's
+  bit members overwrote each other, and an element-wise operation applied
+  in place could give a wrong cell.
+
+- Fix: `save` and `dump` with `endian: :little` or `:big` wrote the other
+  byte order. `load` reads the marker, so files written so far still load
+  correctly.
+
+- Fix: running out of memory inside a kernel left a view attached, lost
+  the keywords of the call (`a.sum(axis: 1)` returned the total), or
+  leaked the array being built.
+
+- Fix: an array with 16 axes (the maximum) given to `bitarray`, `field`,
+  a narrowing `refer` or a `:_` index wrote past an internal array.
+
+- Fix: a lazy chain whose step narrows the cells (`a.lazy.abs` of a
+  complex array) overran the heap; `seq` on an empty object array wrote
+  past its end.
+
+- Fix: a CAObject's copy and sync hooks: a view the hook kept of the
+  array it was lent, or the lent array's mask, pointed at freed memory
+  after the hook returned.
+
+- Fix: `CArray.fuse` crashed when several Ractors ran a fuse block for the
+  first time at once.
+
+- Fix: a grouped `variancep` / `stddevp` matches `CArray#variancep` to the
+  last bit.
+
+- Fix: `count` with an array of values honours `keep_axis:`.
+
+- Fix: `CARecord.wrap` of a record array wraps its storage, not the
+  record array.
+
+- Fix: an unnamed struct's `order_by` survives `save` / `load` and
+  Marshal.
+
+- Fix: `CAFrame.from_csv` and `cast(name => :time)` read a leap second
+  (`23:59:60`) as `CArray.time` does, into the next minute.
+
+## 3.0.4
+
+- New: `CAFrame#verify(rules)` judges every row against named rules (each
+  a callable that returns a boolean CArray) and returns the verdicts as a
+  frame, one column per rule; a row whose rule read a masked value is
+  `UNDEF`, so `count(false)` counts the rows that break it.
+
+- New: `CAFrame#join` takes `how: :semi` and `:anti` (the left rows that
+  have, or have no, match), several key columns (`on: ["station",
+  "date"]`), and the index as a key by the row axis name.
+  `CAFrame#join_asof` takes the index too, and `by:` columns that must
+  match exactly before the nearest key is taken.
+
+- New: a `CAFrame` grouping has more verbs. The short form of `prod`,
+  `accumulate`, `median`, `variance`, `stddev`, `variancep`, `stddevp`,
+  `first`, `last`, `nunique`, `count`, `all` and `any`, beside `sum` /
+  `mean` / `min` / `max`; `filter` (keep the groups a block chooses);
+  `lag`, `order`, `cumsum` and the other running values, and
+  `fill(name, :ffill)` / `:bfill`, all kept on the frame's rows; and
+  `to_frame`. The reductions take column names (`grp.mean("temp")` is a
+  CArray) and `keep_rows: true`, which gives each row its own group's
+  result.
+
+- New: `CAFrame#resample` takes `by:`, columns whose groups are binned
+  apart: `df.resample("time", "1 day", by: "station").max("temp")`.
+
+- New: `CArray#windows(on:, before:, after:)` bounds each window by a
+  width of a key (a time, a duration or a number) instead of a count of
+  cells, with `by:`, `exclude:`, `interval:` and `at:` (windows placed at
+  given points, which may overlap). The key need not be sorted, and the
+  value may have several columns (shape `(N, ...)`). `CAFrame#windows(on:
+  "time", ...)` does the same for a frame's columns.
+
+- New: `CArray#lag(n = 1, axis:)` is the value `n` steps earlier along an
+  axis, `UNDEF` where there is none, so `x - x.lag` starts with `UNDEF`.
+
+- New: the slab, block, segment, categorical and axis-group iterators
+  have `lag`, `order` (each cell's rank within its piece) and
+  `strip_mask(method: :ffill / :bfill)`; each of them works within one
+  piece and never takes a value from another. Every iterator has `first`,
+  `last` and `nunique`.
+
+- New: `CArray#is_between(lo, hi, exclude:)` (or a Range) is true where a
+  value lies between two bounds; `exclude: :lower`, `:upper` or `:both`
+  leaves bounds out, and `nil` or an open end leaves a side unbounded.
+
+- New: `CArray#round(ndigits)` rounds each element to `ndigits` decimal
+  places (negative for tens, hundreds, ...), as `Float#round` and
+  `Integer#round` do.
+
+- New: `CArray#to_frame(*names, axis: -1)` shows an array as a CAFrame
+  whose columns are views of it, one per position along `axis`.
+
+- New: `CACategorical#contrast(base:, full:)` gives one int8 column per
+  category (1 in the cell's category, the base left out unless `full:
+  true`), and `CAFrame#contrast_columns(name, ...)` puts those columns in
+  the frame in place of the categorical.
+
+- New: `CAFrame#is_duplicate(*names, except:)` is true for the rows that
+  repeat an earlier row in the named columns; `except: :last` or `:none`
+  chooses which occurrence stays unmarked.
+
+- New: `CAFrame#append(name, col, before:, after:)` puts the column next
+  to another one.
+
+- New: a `CAFrame` carries attributes of the frame as a whole (`attr`,
+  `set_attr`, `attrs`, ..., and `CAFrame.new(..., attrs:)`), separate from
+  each column's. A row selection or a change of columns keeps them; a
+  summary, a join or a reshape does not.
+
+- New: `CAFrame#to_csv(attrs: [:units, ...])` writes one line per
+  attribute key below the names, and `CAFrame.from_csv(attrs: [...])`
+  reads them back as column attributes; in a reading block,
+  `r.header(:units)` does the same for a line anywhere in the header.
+
+- New: `CAFrame.from_csv` `columns:` takes a Hash that selects and names
+  columns, and the names `c0`, `c1`, ... of a file without a header;
+  `cast` and `from_csv` `types:` take `:categorical`. `CAFrame.new` takes
+  `types:` and `on_error:`, as `from_records` does.
+
+- New: `CArray#delete_attr(key)` removes an attribute and returns its
+  value.
+
+- New: `CATime#end_of(unit:, origin:)` (the last tick of each time's
+  bucket), `days_in_month`, `quarter`, and the ISO 8601 week `iso_week`
+  with its year `iso_year`.
+
+- New: a time or duration array takes a single value in `+` and `-`
+  (`t + "3 days"`, `t - t.min`, `t - Time.utc(2024, 1, 1)`), and a month
+  or year duration moves a time on a day or finer grid as `Date#>>` does
+  (`2024-01-31 + 1 month` is `2024-02-29`). A bare number is still
+  refused.
+
+- New: a time array compares with a value off its grid (`days.eq(
+  "2024-01-03T05:00")`, `<`, `search_nearest`, `is_in`, ...) in the finer
+  unit the two share, where it raised `ArgumentError`. An assignment
+  still raises.
+
+- New: `CATime#pmax` / `pmin` (and `maximum` / `minimum` / `fmax` /
+  `fmin`) give the later / earlier time cell by cell.
+
+- New: CArray can be used in a Ractor other than the main one, `fuse`
+  included, and a frozen array is shared between Ractors without
+  copying. See `docs/topics/Ractor.md`.
+
+- New: C extensions: `CA_FOR_EACH_WHOLE` and `CA_FOR_EACH_WHOLE_MASKED`
+  hand a kernel the whole array as one contiguous run.
+
+- Change: `CArray#rank` is removed; use `#ndim`.
+
+- Change: `CArray::TableMethods` (`column_names=`, `column`, `row`, `rows`
+  and `select` on a 2-D array) is removed; use `a.to_frame("x", "y")`,
+  then `df["x"]` for a column or `df.to_records[i]` for a row.
+
+- Change: `CArray.load_arrow_tensor`, `CArray#save_arrow_tensor` and
+  `CArray::ArrowTensor`, the experimental Arrow tensor IPC reader and
+  writer, are removed.
+
+- Change: methods whose names start and end with `__` that were public
+  internals are now private, renamed or removed. Call the public method
+  they served instead.
+
+- Change: CAFrame takes column names as Strings only. A Symbol name
+  (`df.drop(:a)`, `join(on: :a)`, `CAFrame.new(axis_name: :row)`) raises
+  `TypeError`; write `"a"`. Record keys given to `from_records` may still
+  be Symbols, and `df[:x]` stays the predicate syntax.
+
+- Change: a masked cell of an integer index array keeps its place:
+  `x[i]`, the grid forms and `CAFrame` row selection `df[i]` give a
+  result shaped like `i`, `UNDEF` where `i` is masked, where they dropped
+  those positions. To drop them as before, index by
+  `i[i.is_not_masked]`.
+
+- Change: an index beside a CArray key is read as it is without one: a
+  range ending before its start reads backwards (`a[i, -1..1]`), and an
+  Integer key drops its axis (`a[i, 1]` has the shape of `i`; add `:_`
+  or use `1..1` to keep it).
+
+- Change: an index with a Float that has a fraction (`a[1.5..3]`, a step
+  of `0.5`) raises `IndexError`; it was cut to an integer. Convert with
+  `to_i` or `round` first. A Float that holds an integer, such as a count
+  from `sum`, is still that position.
+
+- Change: an index step says the direction, as in Ruby: a step pointing
+  away from the end reads nothing (`a[(0..4).step(-1)]` read `a[0]`).
+
+- Change: a boolean selector of more than one axis must have the array's
+  shape; it picked cells by flat position. Flatten both sides to keep the
+  old reading.
+
+- Change: storing an integer a cell cannot hold raises `RangeError`
+  (`CA_INT8([300])`, `a[0] = 300`, `fill`, `scatter_add!`, a record
+  member); it wrapped (`300` into int8 was `44`). Storing or reading a
+  NaN into an integer cell (`ints[] = floats`, `as_type`, `fake`) raises
+  `FloatDomainError`. To keep C's conversion, use `to_type`, which is
+  unchanged.
+
+- Change: `CArray.time` reads an Integer as a count of ticks of `unit:`
+  (`CArray.time(CA_INT64([0, 10]), unit: :m)` is 00:00 and 00:10); it
+  read Unix seconds. To read Unix seconds onto another unit, write
+  `CArray.time(secs).to_unit(:m)`.
+
+- Change: `CArray.time` without `unit:`, and `CAFrame#parse_to_time` with
+  a format or `:mixed`, read the values in the finest unit they show
+  (dates alone `:D`, fractions of a second `:ms` / `:us` / `:ns`); it was
+  always `:s`. Pass `unit: :s` to keep the old unit.
+
+- Change: adding a duration that is coarser than a time's unit and not a
+  whole number of its ticks (`"1 day"` on a `"7 hours"` time) raises, and
+  so does `CArray#time(origin:)` between two ticks of `unit:`; both were
+  cut to the grid. Convert the time to a unit both share first.
+
+- Change: `order(descending: true)` puts NaN last and keeps tied cells in
+  their original order (`[5, 3, 5, 1, 5]` gives `[0, 3, 1, 4, 2]`, was
+  `[2, 3, 1, 4, 0]`); with `method: :dense` the largest value is 0 and
+  the ranks have no gaps. Code that used `(n - 1) - order` for the
+  descending rank needs that formula written out. `nlargest` no longer
+  returns NaN or masked cells first, and `nsmallest` / `nlargest` and
+  their `_index` forms break ties by position.
+
+- Change: a CAFrame column built from Ruby values is typed by them:
+  Integers `int64`, numbers `float64`, Strings a text column (`CAString`),
+  arrays of one shape an N-D column, anything else `:object`; it was
+  `:object`. This holds for `CAFrame.new("x" => [...])`, `df["x"] = [...]`,
+  a group's `table` block and a Proc in `aggregate`. Wrap the values in
+  `CA_OBJECT(...)` to keep an object column. A returned array is read
+  from the N-D column as `col[i, nil]`.
+
+- Change: `CAFrame#melt` names each row's source in a text column
+  (`CAString`); it was an object column.
+
+- Change: `CAFrame#group_by` on several keys gives a result with no
+  index: each key is a column, before the reduced columns. Read the keys
+  with `result["station"]` and so on, where the index held tuples.
+
+- Change: `CAFrame#data_types`, `describe` and `p df` name a Face column
+  by its class (`CATime[D]`, `CAString`) where they gave its storage
+  `data_type` (`:fixlen` for most). Read the storage through the Face's
+  accessor (`ticks`, `codes`, ...).
+
+- Change: a CAFrame verb that takes a list of columns refuses a name
+  given twice (`select("a", "a")`, `group_by("a", "a")`) with
+  `ArgumentError`. `df["a", "a"]` still gives the column twice.
+
+- Change: a CAFrame group or window reduction over a column whose type
+  lacks it raises `CArray::DataTypeError`; it raised `ArgumentError`.
+
+- Change: a `CAString` holds only Strings: storing `nil`, a number or an
+  Array into it raises (use `UNDEF` to mask a cell). A selection among
+  Strings (`then_else`, `choose`, `select`, `conditional`) and a
+  `CAString`'s String operators give a `CAString`, where they gave an
+  object array.
+
+- Change: attributes follow conversions, views and copies but not
+  reductions: `categorize`, `cast`, `parse_to_time`, `dup`, Marshal and a
+  Face's views keep them, and `first` / `last` of a grouping no longer
+  carry them. A joined or concatenated array shows the keys every piece
+  shows with the same value. An attribute value is stored frozen; change
+  it by setting it again.
+
+- Change: an error names the method you called, and the common refusals
+  open with it ("+: shape mismatch between operands ...", "fill: can not
+  modify read-only array"). The exception classes are unchanged; code
+  that matches a message needs the new text.
+
+- Change: `clip` refuses `UNDEF` or a masked cell as a bound with
+  `ArgumentError`. To mask the cells out of range, pass `UNDEF` as the
+  fill (`clip(0, 10, UNDEF)`).
+
+- Change: `is_in`, `intersection`, `difference` and `union` refuse a
+  Range; pass an Array (`(1..3).to_a`), or use `is_between` for an
+  interval.
+
+- Change: `median`, `percentile` and `quantile` along the only axis of a
+  1-D array give a bare value, as `sum` does. `nunique(axis:)` naming
+  every axis gives an Integer; pass `keep_axis: true` to keep an array.
+
+- Change: `valid_index?` and `valid_addr?` count a negative index from
+  the end, as `[]` does. `elem_fetch` on an empty array raises
+  `IndexError`; it returned `nil`.
+
+- Change: `seq` / `seq!` on an integer array take a Float or Rational
+  start or step as `init_val + step*i` cut toward zero
+  (`int32.seq(0.5, 0.5)` is `[0, 1, 1, 2]`, was `[0, 0, 1, 1]`), and
+  raise `RangeError` for a start or Integer step the type cannot hold.
+
+- Change: `random!` on an integer array raises for bounds the type
+  cannot hold or a Float bound, and clears the mask. `bin`, `bincount`,
+  `bincount_nd` and the histograms raise for a non-Integer count and for
+  edges that decrease.
+
+- Change: MemoryView: `wrap_memory_view` refuses a boolean buffer (any
+  byte other than 0 or 1 cannot be told apart; use `from_memory_view`,
+  which reads it as true), and a wrap with a read-only `mask:` is
+  read-only. `CArray.memory_view_reject_reason` returns `nil` for an
+  object that is not a CArray.
+
+- Change: C extensions: fifteen kernel iterator shorthand macros
+  (`CA_SLAB_REDUCE_F64`, `CA_SLAB_REDUCE_T_PLUS`,
+  `CA_L2_FOR_EACH_UNMASKED` and others) are removed; write the macro they
+  stood for (`CA_SLAB_REDUCE_T` with `double`, ...).
+  `ca_iter_state_next_slab_strided` is removed, and
+  `ca_iter_state_init_l2` takes `CA_SLAB_AXES` only.
+
 ## 3.0.3
 
 - New: `CAFrame.from_csv` reads messy tables: `types: :infer` (numbers,

data/README.md

@@ -10,11 +10,11 @@ Ruby/CArray is an extension library for the multi-dimensional array class. It pr
 
 * Every array carries a per-element mask for missing values, respected by reductions and statistics
 * Views compose without copying — a write through the outermost view reaches the source data
-* MemoryView protocol on both sides: share buffers with other numerical libraries without copying
+* [MemoryView protocol](guides/users/15_memory_view.md) on both sides: share buffers with other numerical libraries without copying
 * Define your own array class in pure Ruby while keeping the full CArray interface
 * Kernel-style iteration: run a Ruby block over each sub-array spanning the axes you choose
 * Faces: array types for time, strings and categories, with methods of their own on top of everything CArray does
-* Comes with a DataFrame (`CAFrame`) whose columns are plain CArrays, so masks and views keep working on them
+* Comes with a DataFrame ([`CAFrame`](guides/dataframe/README.md)) whose columns are plain CArrays, so masks and views keep working on them
 
 Each of these is shown, briefly and with a runnable example, in [A tour of the features](docs/FeatureTour.md).
 
@@ -119,6 +119,7 @@ b.strip_mask(Float::NAN).sum(axis: 0)
 * [Views](guides/users/06_views.md) — referring to data without copying
 * [Reduction and statistics](guides/users/04_reduction_and_statistics.md) — summaries, whole or per axis
 * [Masks and missing values](guides/users/05_masks.md) — undefined elements, and how calculations treat them
+* [Data frames](guides/dataframe/README.md) — tables of named columns, each column a CArray
 
 ## Contributing
 

data/carray.gemspec

@@ -1,6 +1,6 @@
 Gem::Specification.new do |s|
   s.name        = "carray"
-  s.version     = "3.0.3"
+  s.version     = "3.0.5"
   s.authors      = ["himotoyoshi"]
   s.email       = ["himotoyoshi@users.noreply.github.com"]
   s.summary     = "Multi-dimensional numerical array class for Ruby"

data/ext/ca_array_pool.c

@@ -43,18 +43,42 @@
 ---------------------------------------------------------------------------- */
 
 #include "carray.h"
+#include "carray_internal.h"
 #include <string.h>
 
 /* Allocate struct + pool in one shot, then run pool_init.
  *
  * Called by per-class C `*_new` constructors. */
+typedef struct {
+  CArray *ca;
+  int8_t  obj_type;
+  int8_t  ndim;
+} ca_array_alloc_arg;
+
+static VALUE
+ca_array_alloc_pool (VALUE varg)
+{
+  ca_array_alloc_arg *a = (ca_array_alloc_arg *) varg;
+  ca_array_pool_alloc(a->ca, a->obj_type, a->ndim);
+  return Qnil;
+}
+
 void *
 ca_array_alloc (int8_t obj_type, int8_t ndim)
 {
   ca_operation_function_t *func = &ca_func[obj_type];
   CArray *ca = xmalloc(func->struct_size);
+  ca_array_alloc_arg a;
   memset(ca, 0, func->struct_size);
-  ca_array_pool_alloc(ca, obj_type, ndim);
+  /* The type is set now so that a constructor whose setup raises can hand
+     the struct to ca_free, which dispatches on it.  The setup sets it
+     again. */
+  ca->obj_type = obj_type;
+  /* The pool is a second allocation; when it fails, the struct goes too. */
+  a.ca = ca;
+  a.obj_type = obj_type;
+  a.ndim = ndim;
+  ca_fill_or_xfree(ca, ca_array_alloc_pool, (VALUE) &a);
   return ca;
 }
 

data/ext/ca_axis_descriptor.h

@@ -2,13 +2,10 @@
 
   ca_axis_descriptor.h
 
-  Per-axis descriptor framework (D1 + D2 + D3) — shared C-level vocabulary
-  that lets CSA, CAGrid, CASelectAxis, CAWindow, CAShift, and future
+  Per-axis descriptor framework — shared C-level vocabulary that lets
+  CASelectAxis, CAGrid, CASelect, CAWindow, CAShift, and future
   per-axis-mixed views describe each axis the same way so a common
-  attach dispatch engine (D3, ca_axis_dispatch.c) can drive them
-  uniformly.
-
-  See devel/PROPOSAL_AXIS_DESCRIPTOR_FRAMEWORK.md for the design.
+  attach dispatch engine (ca_axis_dispatch.c) can drive them uniformly.
 
   Status: TWO-TIER FREEZE CONTRACT (3.0 onward), the same split
   ca_kernel_iterator.h declares, for the same reason: this header is
@@ -50,14 +47,14 @@
    When this header is included from carray.h mid-file, those typedefs
    are already in scope by the time we reach the include site. */
 
-/* ---- D1: per-axis descriptor types ----------------------------------------
+/* ---- per-axis descriptor types --------------------------------------------
 
    STRIDE = linear stride axis: parent index = start + i*step,
             i in [0, count).  Subsumes nil / Integer / Range index args.
    INDEX  = integer index axis:  parent index = indices[i],
             i in [0, count).  Subsumes integer-array (CAGrid)
             and boolean-mask-after-snapshot (CASelectAxis) axes.
-   SHIFT  = bound-aware stride axis (CAWindow/CAShift, Tier 2.B):
+   SHIFT  = bound-aware stride axis (CAWindow/CAShift):
             parent index = start + i*step normalised by `policy`
             (CA_BOUNDS_FILL / PERIODIC / REFLECT / NEAREST / ...)
             against `size0` (= parent dim along this axis).  When
@@ -86,16 +83,17 @@ typedef struct {
   uint8_t         policy;    /* CA_BOUNDS_FILL / PERIODIC / REFLECT / ... */
 } ca_axis_desc_t;
 
-/* ---- D2: producer interface ----------------------------------------------
+/* ---- producer interface --------------------------------------------------
 
-   Both implementations write ndim entries into `out` (caller-allocated)
-   and `out_parent_dims` (caller-allocated, sized CA_RANK_MAX).  See
-   devel/PROPOSAL_AXIS_DESCRIPTOR_FRAMEWORK.md §5 Framework Phase 1 and
-   devel/PROPOSAL_FLAT_INDEX_VIEWS.md §2.3 for rationale.
+   A producer writes one entry per descriptor axis into `out`
+   (caller-allocated) and `out_parent_dims` (caller-allocated, sized
+   CA_RANK_MAX).  The producers declared here are CASelectAxis and CAGrid;
+   CASelect and CAWindow have their own (ca_select_describe_axes /
+   ca_window_describe_axes).
 
    `out_parent_dims[k]` is the **effective** parent dim along descriptor
-   axis k — for CSA / CAGrid this is just `parent->dim[k]`, but for
-   flat-index views (CASelect / CAMapping, Tier 2.A.3+) it is
+   axis k — for CASelectAxis / CAGrid this is just `parent->dim[k]`, but
+   for a flat-index view (CASelect) it is
    `{parent->elements}` (a single-axis flat view of parent).  This
    decouples descriptor.ndim from parent.ndim so the engine can drive
    views whose iteration axes don't match the parent's physical shape.
@@ -103,13 +101,17 @@ typedef struct {
    Pointer arguments are typed `void *` here to avoid pulling the
    private CASelectAxis / CAGrid struct definitions (which live in
    their respective .c files) into this header.  Each implementation
-   casts to its concrete type inside the .c file. */
+   casts to its concrete type inside the .c file.
+
+   An INDEX axis has no way to say "no cell": a CAGrid index that is
+   masked (a hole) is written as index 0, so a caller driving a holed grid
+   through the engine reads and writes parent cell 0 there. */
 void ca_select_axis_describe_axes (void *ca, ca_axis_desc_t *out,
                                    ca_size_t *out_parent_dims);
 void ca_grid_describe_axes        (void *ca, ca_axis_desc_t *out,
                                    ca_size_t *out_parent_dims);
 
-/* ---- D3: common attach engine --------------------------------------------
+/* ---- common attach engine -------------------------------------------------
 
    Drives gather from `parent` into an output buffer driven by the
    per-axis descriptor array.  Caller must have parent attached before
@@ -119,14 +121,15 @@ void ca_grid_describe_axes        (void *ca, ca_axis_desc_t *out,
    descriptor axis (see producer interface above).  The engine never
    reads `parent->dim` directly — it only reads `parent->ptr` for
    data access — which is what allows flat-index views to lie about
-   parent's apparent shape (Tier 2.A, PROPOSAL_FLAT_INDEX_VIEWS.md).
+   parent's apparent shape.
 
    `bound_fill` is the bytes-wide value written into gather output
-   cells whose SHIFT axes resolve to an out-of-bounds parent index
-   (Tier 2.B, PROPOSAL_BOUND_AWARE_VIEWS.md).  Pass NULL when no
-   SHIFT axis is present (CSA / CAGrid / CASelect / CAMapping); the
-   engine ignores it.  On scatter, the engine simply skips parent
-   writes for OOB cells regardless of bound_fill.
+   cells whose SHIFT axes resolve to an out-of-bounds parent index.
+   Pass NULL when no SHIFT axis is present (CASelectAxis / CAGrid /
+   CASelect); the engine ignores it.  On scatter, the engine simply skips
+   parent writes for OOB cells regardless of bound_fill.  A SHIFT axis
+   under the RUBY or STRICT policy raises IndexError part way through the
+   walk for an index outside the parent.
 
    _gather — caller provides out_buf (must be sized total_elements*bytes)
    _attach — wrapper that mallocs the buffer; caller xfrees
@@ -160,7 +163,7 @@ void  ca_axis_dispatch_scatter (struct _CArray  *parent,
                                 ca_size_t        total_elements,
                                 const char      *in_buf);
 /* Broadcast scatter: write the single value `val` (bytes wide) into
-   every selected parent cell.  See D6 of the framework piece.
+   every selected parent cell.
    (SHIFT axis OOB cells are skipped — fill semantics aren't
    meaningful for broadcast-write since the destination value already
    IS the broadcast value.) */