carray 3.0.2 → 3.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 728236f069adb2a801da715596ac5ab87d8cf7c460e70de8667d3afa23e9b19f
-  data.tar.gz: fdf25919377fe2d90919712ecf8c50a0ed183c15a57ec925cefedc4a7066fb90
+  metadata.gz: d2edeb545754cf31d330c667913471c1a0035c97de72dabad7e7b2afb4fe272b
+  data.tar.gz: 8b34390fcabbb04c5e1f0926ae103052345196112ec3a6a16f244fb310df5583
 SHA512:
-  metadata.gz: 26f8ae296774c82b5bba4f9521cf2b53ec0b1804115c0a4a2f39e29d67419932487a4ff892cd115ab3eb34242c942711504c05945b5e95d44653e546e7c70d66
-  data.tar.gz: e8109043b73d2f71d395ee0cc5711e395f6964ba0cef7c2aea2d35759dadedca5aa649ec3ebe14de66b6034a788213fb0c0fa1a1ef2499eef03d8b0b021d4bc4
+  metadata.gz: 0f7430ff0556faf7b609673fb8cf9ef08279e226192995ea58a5dd203e5285733582c5cff0405fc753c39cad7056acd9194f55ba26ce9eb1d1a27cf1de8e37f0
+  data.tar.gz: 48d35829402ba2949e865712d8aa89dd9d4228f0ca4abf62719d3b2d32280541d1c6b7b01dba3e2a68927a89e258e0d6b474a20fc6177415154e1ab57f2b380d

data/CHANGELOG.md

@@ -34,6 +34,574 @@ and a newer one. The 1.x history, up to the 2.0.0 release, is in
      neighbour ("as well", "the producer above") or leave unnamed the
      method, class or keyword it is about. -->
 
+## 3.0.4
+
+- New: `CAFrame#verify(rules)` judges every row against named rules (each
+  a callable that returns a boolean CArray) and returns the verdicts as a
+  frame, one column per rule; a row whose rule read a masked value is
+  `UNDEF`, so `count(false)` counts the rows that break it.
+
+- New: `CAFrame#join` takes `how: :semi` and `:anti` (the left rows that
+  have, or have no, match), several key columns (`on: ["station",
+  "date"]`), and the index as a key by the row axis name.
+  `CAFrame#join_asof` takes the index too, and `by:` columns that must
+  match exactly before the nearest key is taken.
+
+- New: a `CAFrame` grouping has more verbs. The short form of `prod`,
+  `accumulate`, `median`, `variance`, `stddev`, `variancep`, `stddevp`,
+  `first`, `last`, `nunique`, `count`, `all` and `any`, beside `sum` /
+  `mean` / `min` / `max`; `filter` (keep the groups a block chooses);
+  `lag`, `order`, `cumsum` and the other running values, and
+  `fill(name, :ffill)` / `:bfill`, all kept on the frame's rows; and
+  `to_frame`. The reductions take column names (`grp.mean("temp")` is a
+  CArray) and `keep_rows: true`, which gives each row its own group's
+  result.
+
+- New: `CAFrame#resample` takes `by:`, columns whose groups are binned
+  apart: `df.resample("time", "1 day", by: "station").max("temp")`.
+
+- New: `CArray#windows(on:, before:, after:)` bounds each window by a
+  width of a key (a time, a duration or a number) instead of a count of
+  cells, with `by:`, `exclude:`, `interval:` and `at:` (windows placed at
+  given points, which may overlap). The key need not be sorted, and the
+  value may have several columns (shape `(N, ...)`). `CAFrame#windows(on:
+  "time", ...)` does the same for a frame's columns.
+
+- New: `CArray#lag(n = 1, axis:)` is the value `n` steps earlier along an
+  axis, `UNDEF` where there is none, so `x - x.lag` starts with `UNDEF`.
+
+- New: the slab, block, segment, categorical and axis-group iterators
+  have `lag`, `order` (each cell's rank within its piece) and
+  `strip_mask(method: :ffill / :bfill)`; each of them works within one
+  piece and never takes a value from another. Every iterator has `first`,
+  `last` and `nunique`.
+
+- New: `CArray#is_between(lo, hi, exclude:)` (or a Range) is true where a
+  value lies between two bounds; `exclude: :lower`, `:upper` or `:both`
+  leaves bounds out, and `nil` or an open end leaves a side unbounded.
+
+- New: `CArray#round(ndigits)` rounds each element to `ndigits` decimal
+  places (negative for tens, hundreds, ...), as `Float#round` and
+  `Integer#round` do.
+
+- New: `CArray#to_frame(*names, axis: -1)` shows an array as a CAFrame
+  whose columns are views of it, one per position along `axis`.
+
+- New: `CACategorical#contrast(base:, full:)` gives one int8 column per
+  category (1 in the cell's category, the base left out unless `full:
+  true`), and `CAFrame#contrast_columns(name, ...)` puts those columns in
+  the frame in place of the categorical.
+
+- New: `CAFrame#is_duplicate(*names, except:)` is true for the rows that
+  repeat an earlier row in the named columns; `except: :last` or `:none`
+  chooses which occurrence stays unmarked.
+
+- New: `CAFrame#append(name, col, before:, after:)` puts the column next
+  to another one.
+
+- New: a `CAFrame` carries attributes of the frame as a whole (`attr`,
+  `set_attr`, `attrs`, ..., and `CAFrame.new(..., attrs:)`), separate from
+  each column's. A row selection or a change of columns keeps them; a
+  summary, a join or a reshape does not.
+
+- New: `CAFrame#to_csv(attrs: [:units, ...])` writes one line per
+  attribute key below the names, and `CAFrame.from_csv(attrs: [...])`
+  reads them back as column attributes; in a reading block,
+  `r.header(:units)` does the same for a line anywhere in the header.
+
+- New: `CAFrame.from_csv` `columns:` takes a Hash that selects and names
+  columns, and the names `c0`, `c1`, ... of a file without a header;
+  `cast` and `from_csv` `types:` take `:categorical`. `CAFrame.new` takes
+  `types:` and `on_error:`, as `from_records` does.
+
+- New: `CArray#delete_attr(key)` removes an attribute and returns its
+  value.
+
+- New: `CATime#end_of(unit:, origin:)` (the last tick of each time's
+  bucket), `days_in_month`, `quarter`, and the ISO 8601 week `iso_week`
+  with its year `iso_year`.
+
+- New: a time or duration array takes a single value in `+` and `-`
+  (`t + "3 days"`, `t - t.min`, `t - Time.utc(2024, 1, 1)`), and a month
+  or year duration moves a time on a day or finer grid as `Date#>>` does
+  (`2024-01-31 + 1 month` is `2024-02-29`). A bare number is still
+  refused.
+
+- New: a time array compares with a value off its grid (`days.eq(
+  "2024-01-03T05:00")`, `<`, `search_nearest`, `is_in`, ...) in the finer
+  unit the two share, where it raised `ArgumentError`. An assignment
+  still raises.
+
+- New: `CATime#pmax` / `pmin` (and `maximum` / `minimum` / `fmax` /
+  `fmin`) give the later / earlier time cell by cell.
+
+- New: CArray can be used in a Ractor other than the main one, `fuse`
+  included, and a frozen array is shared between Ractors without
+  copying. See `docs/topics/Ractor.md`.
+
+- New: C extensions: `CA_FOR_EACH_WHOLE` and `CA_FOR_EACH_WHOLE_MASKED`
+  hand a kernel the whole array as one contiguous run.
+
+- Change: `CArray#rank` is removed; use `#ndim`.
+
+- Change: `CArray::TableMethods` (`column_names=`, `column`, `row`, `rows`
+  and `select` on a 2-D array) is removed; use `a.to_frame("x", "y")`,
+  then `df["x"]` for a column or `df.to_records[i]` for a row.
+
+- Change: `CArray.load_arrow_tensor`, `CArray#save_arrow_tensor` and
+  `CArray::ArrowTensor`, the experimental Arrow tensor IPC reader and
+  writer, are removed.
+
+- Change: methods whose names start and end with `__` that were public
+  internals are now private, renamed or removed. Call the public method
+  they served instead.
+
+- Change: CAFrame takes column names as Strings only. A Symbol name
+  (`df.drop(:a)`, `join(on: :a)`, `CAFrame.new(axis_name: :row)`) raises
+  `TypeError`; write `"a"`. Record keys given to `from_records` may still
+  be Symbols, and `df[:x]` stays the predicate syntax.
+
+- Change: a masked cell of an integer index array keeps its place:
+  `x[i]`, the grid forms and `CAFrame` row selection `df[i]` give a
+  result shaped like `i`, `UNDEF` where `i` is masked, where they dropped
+  those positions. To drop them as before, index by
+  `i[i.is_not_masked]`.
+
+- Change: an index beside a CArray key is read as it is without one: a
+  range ending before its start reads backwards (`a[i, -1..1]`), and an
+  Integer key drops its axis (`a[i, 1]` has the shape of `i`; add `:_`
+  or use `1..1` to keep it).
+
+- Change: an index with a Float that has a fraction (`a[1.5..3]`, a step
+  of `0.5`) raises `IndexError`; it was cut to an integer. Convert with
+  `to_i` or `round` first. A Float that holds an integer, such as a count
+  from `sum`, is still that position.
+
+- Change: an index step says the direction, as in Ruby: a step pointing
+  away from the end reads nothing (`a[(0..4).step(-1)]` read `a[0]`).
+
+- Change: a boolean selector of more than one axis must have the array's
+  shape; it picked cells by flat position. Flatten both sides to keep the
+  old reading.
+
+- Change: storing an integer a cell cannot hold raises `RangeError`
+  (`CA_INT8([300])`, `a[0] = 300`, `fill`, `scatter_add!`, a record
+  member); it wrapped (`300` into int8 was `44`). Storing or reading a
+  NaN into an integer cell (`ints[] = floats`, `as_type`, `fake`) raises
+  `FloatDomainError`. To keep C's conversion, use `to_type`, which is
+  unchanged.
+
+- Change: `CArray.time` reads an Integer as a count of ticks of `unit:`
+  (`CArray.time(CA_INT64([0, 10]), unit: :m)` is 00:00 and 00:10); it
+  read Unix seconds. To read Unix seconds onto another unit, write
+  `CArray.time(secs).to_unit(:m)`.
+
+- Change: `CArray.time` without `unit:`, and `CAFrame#parse_to_time` with
+  a format or `:mixed`, read the values in the finest unit they show
+  (dates alone `:D`, fractions of a second `:ms` / `:us` / `:ns`); it was
+  always `:s`. Pass `unit: :s` to keep the old unit.
+
+- Change: adding a duration that is coarser than a time's unit and not a
+  whole number of its ticks (`"1 day"` on a `"7 hours"` time) raises, and
+  so does `CArray#time(origin:)` between two ticks of `unit:`; both were
+  cut to the grid. Convert the time to a unit both share first.
+
+- Change: `order(descending: true)` puts NaN last and keeps tied cells in
+  their original order (`[5, 3, 5, 1, 5]` gives `[0, 3, 1, 4, 2]`, was
+  `[2, 3, 1, 4, 0]`); with `method: :dense` the largest value is 0 and
+  the ranks have no gaps. Code that used `(n - 1) - order` for the
+  descending rank needs that formula written out. `nlargest` no longer
+  returns NaN or masked cells first, and `nsmallest` / `nlargest` and
+  their `_index` forms break ties by position.
+
+- Change: a CAFrame column built from Ruby values is typed by them:
+  Integers `int64`, numbers `float64`, Strings a text column (`CAString`),
+  arrays of one shape an N-D column, anything else `:object`; it was
+  `:object`. This holds for `CAFrame.new("x" => [...])`, `df["x"] = [...]`,
+  a group's `table` block and a Proc in `aggregate`. Wrap the values in
+  `CA_OBJECT(...)` to keep an object column. A returned array is read
+  from the N-D column as `col[i, nil]`.
+
+- Change: `CAFrame#melt` names each row's source in a text column
+  (`CAString`); it was an object column.
+
+- Change: `CAFrame#group_by` on several keys gives a result with no
+  index: each key is a column, before the reduced columns. Read the keys
+  with `result["station"]` and so on, where the index held tuples.
+
+- Change: `CAFrame#data_types`, `describe` and `p df` name a Face column
+  by its class (`CATime[D]`, `CAString`) where they gave its storage
+  `data_type` (`:fixlen` for most). Read the storage through the Face's
+  accessor (`ticks`, `codes`, ...).
+
+- Change: a CAFrame verb that takes a list of columns refuses a name
+  given twice (`select("a", "a")`, `group_by("a", "a")`) with
+  `ArgumentError`. `df["a", "a"]` still gives the column twice.
+
+- Change: a CAFrame group or window reduction over a column whose type
+  lacks it raises `CArray::DataTypeError`; it raised `ArgumentError`.
+
+- Change: a `CAString` holds only Strings: storing `nil`, a number or an
+  Array into it raises (use `UNDEF` to mask a cell). A selection among
+  Strings (`then_else`, `choose`, `select`, `conditional`) and a
+  `CAString`'s String operators give a `CAString`, where they gave an
+  object array.
+
+- Change: attributes follow conversions, views and copies but not
+  reductions: `categorize`, `cast`, `parse_to_time`, `dup`, Marshal and a
+  Face's views keep them, and `first` / `last` of a grouping no longer
+  carry them. A joined or concatenated array shows the keys every piece
+  shows with the same value. An attribute value is stored frozen; change
+  it by setting it again.
+
+- Change: an error names the method you called, and the common refusals
+  open with it ("+: shape mismatch between operands ...", "fill: can not
+  modify read-only array"). The exception classes are unchanged; code
+  that matches a message needs the new text.
+
+- Change: `clip` refuses `UNDEF` or a masked cell as a bound with
+  `ArgumentError`. To mask the cells out of range, pass `UNDEF` as the
+  fill (`clip(0, 10, UNDEF)`).
+
+- Change: `is_in`, `intersection`, `difference` and `union` refuse a
+  Range; pass an Array (`(1..3).to_a`), or use `is_between` for an
+  interval.
+
+- Change: `median`, `percentile` and `quantile` along the only axis of a
+  1-D array give a bare value, as `sum` does. `nunique(axis:)` naming
+  every axis gives an Integer; pass `keep_axis: true` to keep an array.
+
+- Change: `valid_index?` and `valid_addr?` count a negative index from
+  the end, as `[]` does. `elem_fetch` on an empty array raises
+  `IndexError`; it returned `nil`.
+
+- Change: `seq` / `seq!` on an integer array take a Float or Rational
+  start or step as `init_val + step*i` cut toward zero
+  (`int32.seq(0.5, 0.5)` is `[0, 1, 1, 2]`, was `[0, 0, 1, 1]`), and
+  raise `RangeError` for a start or Integer step the type cannot hold.
+
+- Change: `random!` on an integer array raises for bounds the type
+  cannot hold or a Float bound, and clears the mask. `bin`, `bincount`,
+  `bincount_nd` and the histograms raise for a non-Integer count and for
+  edges that decrease.
+
+- Change: MemoryView: `wrap_memory_view` refuses a boolean buffer (any
+  byte other than 0 or 1 cannot be told apart; use `from_memory_view`,
+  which reads it as true), and a wrap with a read-only `mask:` is
+  read-only. `CArray.memory_view_reject_reason` returns `nil` for an
+  object that is not a CArray.
+
+- Change: C extensions: fifteen kernel iterator shorthand macros
+  (`CA_SLAB_REDUCE_F64`, `CA_SLAB_REDUCE_T_PLUS`,
+  `CA_L2_FOR_EACH_UNMASKED` and others) are removed; write the macro they
+  stood for (`CA_SLAB_REDUCE_T` with `double`, ...).
+  `ca_iter_state_next_slab_strided` is removed, and
+  `ca_iter_state_init_l2` takes `CA_SLAB_AXES` only.
+
+## 3.0.3
+
+- New: `CAFrame.from_csv` reads messy tables: `types: :infer` (numbers,
+  booleans, year-first dates and times), `types: { default: ... }`,
+  `missing:` tokens, `on_error:` (`:mask`, `:warn`, `:raise`), `columns:` to
+  read only some columns, and `header:` / `data:` / `column_names:` by line.
+  `CAFrame#cast` takes the same map; `parse_to_time(name, :infer)` finds a
+  column's date format; `describe` and `split_column` are new; `to_csv`
+  takes `encoding:` and `missing:`.
+
+- New: `CAFrame` reshapes and resamples: `pivot` / `pivot_grid` / `melt`,
+  `stack_rows(by:)` / `unstack_rows`, `stack_columns` / `unstack_column`,
+  `CAFrame.stack` / `unstack` (frames as layers), and `resample(name, unit)`
+  for time bins. The stacked forms are views. `protect` makes a frame
+  read-only.
+
+- New: `CAFrame` group reductions and `aggregate` reduce N-D columns along
+  the rows (a column `(N, L)` gives `(groups, L)`), take `min_count:` and
+  `fill_value:`, and pass a reduction its arguments
+  (`"p90" => ["temp", :percentile, 90.0]`).
+
+- New: `value.segments(offsets:)` / `segments(lengths:)` iterate over
+  consecutive runs of cells, with `CArray.segment_offsets` /
+  `segment_index` to convert between lengths, boundaries and segment
+  numbers. The scans, `map`, `median` and `percentile` of
+  `group_by_category` take `axis:`, and a window's `median`, `percentile`
+  and `quantile` answer under the default `bounds: :skip`.
+
+- New: `bitfield(range, type)` reads a field as a signed or wider integer;
+  `set_attrs(hash)`; `order(kind:)`; `tolerance: "10 minutes"` for time keys
+  in `locate_nearest_addr` and `CAFrame#join_asof`. A `CARecord` answers
+  `unique`, `value_counts`, `mode`, `is_in`, the set operations,
+  `categorize`, `count(v)` and `locate_addr`.
+
+- New: for whoever registers a `CArray.expression_evaluator` (carray-jit):
+  mixed data types, integer powers, comparisons, `then_else`, `shift`, and
+  reductions, scans and sorts of an expression now reach the evaluator. An
+  evaluator that does not know a node declines, and CArray computes the
+  expression as before.
+
+- New: for C extensions: `ca_iter_ensure`, `ca_sync_detach`,
+  `ca_attach_n` / `ca_allocate_n`, `ca_iter_state_init_l2_paired`,
+  `ca_stride_setup(NULL, ...)` and `ca_check_uninitialized` let an
+  extension clean up when its code raises.
+
+- Change: `CAFrame` names its columns "columns": `variable_names` is now
+  `column_names`, `variables` is `columns` and `nvar` is `ncol`. The old
+  names are gone.
+
+- Change: `sort_index`, `rank_index`, `partition_index`, `partition` and
+  `partition_copy` without `axis:` work on the whole array, as `sort` does;
+  they worked along axis 0. On an N-D array pass `axis: 0` to keep the old
+  result.
+
+- Change: errors a caller can cause no longer raise `RuntimeError`: a bad
+  shape, step, axis, option or data type raises `ArgumentError`,
+  `TypeError`, `RangeError` or `CArray::DataTypeError`, and every index out
+  of range raises `IndexError` (`take_along_axis`, `axis2addr` and
+  `addr2index` raised others). `CArray::DataTypeError` is now a
+  `TypeError`. A write to a read-only array still raises `RuntimeError`.
+  Messages name the method you called and the value you gave. Update a
+  `rescue` that matched the old class or wording.
+
+- Change: arguments that were truncated or ignored now raise: an axis,
+  `min_count:`, `kth` or `n` that is not an Integer; a misspelt keyword; a
+  keyword a method does not take (`min_count:` to `all` / `any` of an
+  iterator, as to the core); a Symbol column name in `cast` and `types:`;
+  `windows` given anything but Ranges (an exclusive range now leaves out
+  its end); `CA_INT32(0, 2)`; a fixlen array without `bytes:`. A negative
+  axis counts from the end everywhere. The indexer's method form
+  (`a[:eq, 0]`) calls public methods only.
+
+- Change: a Hash passed as a positional argument is an argument, as in
+  Ruby 3, not keywords: `a.sum({axis: 0})` raises (write `a.sum(axis: 0)`
+  or `a.sum(**opts)`). An object array of Hashes can now be searched and
+  counted.
+
+- Change: values that cannot be held raise instead of being bent: storing
+  `nil` into a numeric or boolean array, an Integer operand the array's type
+  cannot hold (`RangeError`; the comparisons answer by value instead), and a
+  String that is not a decimal number (`"010"` is now 10, and `to_type` gives
+  `UNDEF`). `CAFrame#cast` to an integer makes `2.5` `UNDEF` instead of 2.
+  Store `Float::NAN` or `UNDEF` for a missing value.
+
+- Change: reductions with too few values: `variance` / `stddev` of fewer
+  than two values is `UNDEF` (pass `fill_value: 0.0` for the old answer),
+  and `minmax` of nothing is `[UNDEF, UNDEF]`. `fill_value:` is stored into
+  the result's type with or without `axis:`; `fill_value: UNDEF`,
+  `strip_mask(UNDEF)` and `unmask(UNDEF)` leave the cells masked.
+
+- Change: arrays of mixed types meet in a type that holds both: signed with
+  unsigned integers (`uint8` with `int8` is `int16`, `uint64` with `int64` is
+  `:object`) and 32-bit or wider integers with `float32` (in `float64`) when
+  joined or matched. `CArray.result_type` lets a scalar take the arrays'
+  type, as an operator does (`result_type(float32_array, 0.1)` is
+  `:float32`). Arithmetic is unchanged.
+
+- Change: integer and float edge cases no longer depend on the machine or
+  the argument order: a shift count means what it does to `Integer#<<`,
+  signed `rcp` / `rcp_mul` round down as `/` does, `to_type` to an integer
+  makes NaN `UNDEF`, `maximum` / `minimum` of two zeros are `0.0` / `-0.0`,
+  and `clip` raises when `min` exceeds `max` (a one-sided fill clamps the
+  other side).
+
+- Change: `CAFrame#join` matches as SQL joins do: a key on 2 left and 3
+  right rows gives 6 rows, and a masked key matches nothing and stays as an
+  unmatched row. When the other frame's key is unique, `:left` gives the
+  same rows as before.
+
+- Change: `CAFrame.from_csv`'s reading block takes the reader as its
+  parameter and its `body` verb is `data`
+  (`from_csv(path) { |r| r.skip 2; r.header; r.data }`). Text columns come
+  back as `CAString`. Contents that cannot be read raise
+  `CAFrame::UnreadableColumn` or `CAFrame::AmbiguousTimeFormat`, which are
+  not `ArgumentError`.
+
+- Change: reading time from text is stricter: `parse_to_time` without a
+  format reads year-first text only (pass a format, or `:mixed` for the old
+  guess) in the finest unit the text shows; `CArray.time` and strptime
+  formats refuse trailing text and a time of day they cannot read; a date
+  the column's unit cannot hold raises `RangeError`. Before, these cells
+  were guessed, floored or masked.
+
+- Change: `CAFrame#to_csv` writes a boolean as `1` / `0` and a float32 as
+  its shortest decimal, refuses a record column, and raises naming the row
+  and column of a cell it cannot encode. `extract` masks a cell its pattern
+  does not match, where it gave `""`.
+
+- Change: `CAFrame`: a frozen frame refuses `[]=`, `set_index`, `cast` and
+  the other in-place verbs; `at` on a numeric index takes numbers only; the
+  index of a `group_by` result keeps the key's type (a `CATime` key gives a
+  `CATime` index); group `sum` / `mean` / `min` / `max` include every column
+  whose type has the reduction (times and strings among them).
+
+- Change: `CAStruct` / `CARecord`: records are ordered only by the members
+  named in `CArray.struct(order_by:)`; without it sorting, `min` / `max` and
+  comparison raise. An element's `[]` takes a String or a position (a
+  Symbol raises), a member named like a method (`hash`, `size`) gets no
+  accessor, a nested member reads as a view (`dup` it for a copy), boolean
+  and 1-bit members read as `true` / `false`, and `UNDEF` cannot be stored
+  in a member. Records of another struct are refused.
+
+- Change: a `CARecord` exported through MemoryView uses Ruby's own format
+  (`ruby/memory_view.h`) instead of PEP 3118 `T{...}`; the member names are
+  `CARecord#memory_view_field_names`. A union is no longer exported.
+
+- Change: a time array compares with, searches for and stores a `String` or
+  `Date` as well as a `Time`, at the precision the value carries: a value
+  off the array's grid (09:00 against a day array) raises, where a `Time`
+  was floored. `union` of two units gives the finer unit. `inspect` shows
+  the tick and each cell as `to_s` does.
+
+- Change: `==` on a Face (a time, string or categorical array) compares the
+  values it holds, and only with a Face of its own class: time arrays of
+  different units holding the same instants are `==`, and a `CAString` is
+  no longer `==` to an object array of the same strings (compare
+  `to_type(:object)` for that).
+
+- Change: Faces (`CATime`, `CATimedelta`, `CACategorical`, string arrays):
+  `.lazy` returns the Face and computes eagerly; storing one into a
+  non-object plain array raises (convert with `to_type(:object)` or
+  `ticks`); `CArray.save`, `dump` and `Marshal.dump` refuse one, and a
+  `CAWrap` (save `.parent` or `.copy`); `CArray.stack([face])` returns the
+  Face, and mixing Faces raises.
+
+- Change: for whoever writes a Face: the hooks are renamed
+  `storage_to_element` / `element_to_storage`, and a stored value goes
+  through `element_to_storage` once (it went through twice). A Face written
+  in Ruby on `CAObject` over a numeric storage refuses arithmetic, math and
+  reductions unless it passes `numeric: true`.
+
+- Change: views: a view of a frozen array is read-only rather than frozen
+  (`RuntimeError`, `frozen?` false), so `sort`, selections and lazy
+  expressions work on a frozen array. `then_else` and `sort` on a lazy
+  expression return a read-only lazy view (call `copy` for a snapshot), and
+  `then_else` branches broadcast as operands do. `dup` of a `CAWrap` is a
+  plain `CArray`. Attributes survive `copy` and `to_type`.
+
+- Change: a `bitarray` view has no mask and refuses a masked array (use
+  `value.bitarray`); `pack_bits` raises on masked cells (pack the mask with
+  `validity_bits`). `bitfield` refuses a complex array.
+
+- Change: on an object array, `min`, `max`, `cummin`, `pmin` and the rest
+  compare with `<=>`, as `sort` does; a pair that cannot be ordered raises
+  `ArgumentError`. `median`, `percentile` and `quantile` answer in the
+  elements' own arithmetic (`CA_OBJECT([1, 2]).median` is `3/2`; convert
+  with `to_type(:float64)` for a Float). `&`, `|` and `^` with a boolean
+  array read its cells as `true` / `false`, eager and lazy.
+
+- Change: smaller behaviour changes: `locate_nearest_addr` (and `join_asof`,
+  `snap_to`) with `direction:` matches beyond the reference's ends (limit it
+  with `tolerance:`); `map_slab` carries the mask of the block's result; a
+  masked cell of a one-axis boolean selector selects nothing; `shift`,
+  `roll`, `tile` and `window` on a zero-length axis return an empty view;
+  boolean sums in `axis_group` and categorical scans are uint64, as in the
+  core; `group_by_category` raises `TypeError` for a non-categorical.
+
+- Change: an operand that exports MemoryView and defines `to_ca` (an Arrow
+  array, with a bridge) is taken through `to_ca`, so Arrow nulls arrive as
+  masked cells and time columns as `CATime`.
+
+- Change: for C extensions: `CArray::VERSION_CODE` is
+  `major*10000 + minor*100 + teeny` (`30003`); the `CA_FOR_EACH_ELEMENT`
+  macros, `CA_WITH_BUFFER` and `CA_CHECK_BOUND` are removed (use
+  `ca_call_cslab_N_r`, `ca_call_cfunc_N_r` or `rb_ca_call_with_buffer`);
+  the `_INOUT` and `_PAIR` fiber macros raise on a shape mismatch; kernels
+  are built with `-fwrapv`. Rebuild the extension.
+
+- Change: faster, with the same results: `from_csv` with `types:` (two to
+  five times), text to float (eight times), `CAConstString#to_string`,
+  reductions over gathered, `shift` and `window` views, a repeated
+  `CArray.fuse` block, and selecting from a lazy expression.
+
+- Fix: wrong answers that came without an error. An in-place operation
+  reading an argument that shares its memory read cells it had already
+  written (`a[1..5].add!(a[0..4])` gave a running sum; `scatter_*!`,
+  `inherit_mask` and `mask=` too); `refer` to a wider type on a transposed
+  or stepped parent read neighbouring values; views of a stack or meld,
+  and a slice of a lazy expression, returned other cells; a Float operand
+  of an integer array was truncated (`count(1.5)` found `1`); `mask=` with
+  another shape copied cells by count; a masked index read cell 0 in
+  `gather_nd`; `stddev` of NaN data was 0.0.
+
+- Fix: wrong answers that came without an error, time and strings: time
+  keys of different units matched or not depending on the side in `is_in`,
+  `join` and `align`; `locate_nearest_addr` picked a wrong row for a NaN,
+  an infinite or a nanosecond key; `unmask(time)`, `resize(fill_value:)`
+  and `insert_block` read a `Time` as a count; `CAConstString` comparisons
+  compared buffer positions; a `CAString` compared against a time,
+  categorical or const string compared their encodings.
+
+- Fix: writes that were lost now reach the array: through a grid, selection
+  or `select_axis` over a reshaped transpose, through a stack or meld of
+  views, into a nested struct member, into the mask copy of a stack or meld,
+  and the masked cells of `c[] = c.flip(0)`.
+
+- Fix: crashes and reads outside an array: a garbage collection during an
+  operation on a converted or computed array (`CAObject`, lazy, object
+  data); a slab kept after `each_slab`; a `CARecord` after `Marshal.load`;
+  shapes and indices whose arithmetic overflows; `refer(:object)` and
+  `bitfield` on object data; `scatter_*!` on complex; `MIN / -1` on Linux
+  x86; struct members outside the record; fixlen stores of another width;
+  reductions in several Ractors.
+
+- Fix: an operation that raises part way (a block, a conversion, a
+  division by zero) leaves its arrays usable and frees its memory. Before,
+  an array could stay attached, and after 32 such errors every lazy
+  expression failed with "all 32 slots in use".
+
+- Fix: iterator reductions answer as the same reduction on an array: every
+  one takes `min_count:` and `fill_value:` (`blocks(n).percentile(p,
+  min_count: k)` read them as a second percentile), a group's counts apply
+  `min_count:` to the right cells, a group's `wmean` no longer takes
+  `fill_value:` as a weight, a window of only NaN gives NaN, and object
+  weights in `segments` stay exact.
+
+- Fix: `CAFrame` round trips through CSV and records: quotes, separators of
+  several characters or spaces, lines of spaces, a duplicated header name,
+  complex `Infinity` / `NaN`, months and years, hour 24, a one-column file
+  with a blank name; nested arrays, `Complex` and `Rational` in
+  `from_records`. `fill`, `to_table` and `filter(keep_masked:)` work with
+  N-D and categorical columns; `meld` and `concatenate` take empty frames;
+  `group_by` takes a categorical key and treats NaN in a composite key as
+  one value.
+
+- Fix: Faces stay Faces: the reductions of `windows` and `blocks`,
+  `partition_copy`, `then_else`, `clip`, `mask_eq` and `meld` return the
+  Face, and a Face's own state survives `min(axis:)`, `cummax` and
+  `partition_copy`. `axis_group` and categorical reductions take boolean,
+  complex, object and Face values; a time array's `fill_value:` is a time
+  on every path. `CAFrame#fill(:linear)` interpolates any Face that defines
+  `linear_fetch`. A categorical refuses `clip` and `minmax` as it refuses
+  `min`.
+
+- Fix: types kept through operations: complex arrays work with `unique`,
+  `is_in` and the rest of that family; `meld` reductions answer as on a
+  copy; fixlen strings of different widths match; `atan2`, `hypot` and
+  others keep float32 and integer types; a record member declared with
+  `endian:` reads, sorts, saves and loads correctly.
+
+- Fix: NaN and masked cells are treated the same everywhere: NaN is last in
+  object sorts and `categorize`, loses `min` / `max`, and is found by
+  `CAFrame#at(Float::NAN)`; `min_index` never points at a masked cell or a
+  NaN; `min_count:` and `fill_value:` are honoured where they were ignored;
+  a lazy expression's mask follows its operands; a selection or `shift`
+  follows later mask changes; `invert_mask` works on a selection.
+
+- Fix: MemoryView: a `CAFixlenString` exports as bytes and a frozen fixlen
+  array can be exported; `CATime`, `CACategorical` and `CAConstString` are
+  refused by `memory_view_available?` with the reason, instead of answering
+  `true` and failing.
+
+- Fix: with carray-jit: two expressions that differ only in which operand an
+  operation reads no longer share a kernel, and a store whose destination
+  overlaps an operand is computed by CArray.
+
+- Fix: for C extensions: a `CA_KERNEL_WRITE` walk over a view that holds a
+  cell more than once keeps the cells the kernel wrote; a walk left with
+  `break` writes back its slab; writes through a `CAGrid`, `CAWindow` or
+  `CAShift` over a view reach the array; the sweep engine and paired walks
+  read an argument sharing the output's memory as it was; a view class
+  that is not registered with `ca_iter_register_source_kind` is still
+  accepted. Rebuild the extension.
+
 ## 3.0.2
 
 - New: `CArray::AddressBasis`, for a C extension whose code addresses cells

data/README.md

@@ -4,18 +4,17 @@ Ruby/CArray is an extension library for the multi-dimensional array class. It pr
 
 ## Status
 
-3.0.x still moves: behavior can change between releases — see [CHANGELOG.md](CHANGELOG.md). 3.1 is the first release meant to be depended on. Until then, treat it as a place to try things out.
+3.0.x still moves: behavior can change between releases. 3.1 is the first release meant to be depended on. Until then, treat it as a place to try things out.
 
 ## Features
 
 * Every array carries a per-element mask for missing values, respected by reductions and statistics
 * Views compose without copying — a write through the outermost view reaches the source data
-* MemoryView protocol on both sides: share buffers with other numerical libraries without copying
-* Kernel-style iteration: run a Ruby block over each sub-array spanning the axes you choose
-* Attach domain meaning (time, angle, quantity with units…) without changing storage — a Face
+* [MemoryView protocol](guides/users/15_memory_view.md) on both sides: share buffers with other numerical libraries without copying
 * Define your own array class in pure Ruby while keeping the full CArray interface
-* Pack multiple values into one element as a record type
-* Comes with a DataFrame (`CAFrame`) whose columns are plain CArrays, so masks and views keep working on them
+* Kernel-style iteration: run a Ruby block over each sub-array spanning the axes you choose
+* Faces: array types for time, strings and categories, with methods of their own on top of everything CArray does
+* Comes with a DataFrame ([`CAFrame`](guides/dataframe/README.md)) whose columns are plain CArrays, so masks and views keep working on them
 
 Each of these is shown, briefly and with a runnable example, in [A tour of the features](docs/FeatureTour.md).
 
@@ -120,6 +119,7 @@ b.strip_mask(Float::NAN).sum(axis: 0)
 * [Views](guides/users/06_views.md) — referring to data without copying
 * [Reduction and statistics](guides/users/04_reduction_and_statistics.md) — summaries, whole or per axis
 * [Masks and missing values](guides/users/05_masks.md) — undefined elements, and how calculations treat them
+* [Data frames](guides/dataframe/README.md) — tables of named columns, each column a CArray
 
 ## Contributing
 

data/carray.gemspec

@@ -1,6 +1,6 @@
 Gem::Specification.new do |s|
   s.name        = "carray"
-  s.version     = "3.0.2"
+  s.version     = "3.0.4"
   s.authors      = ["himotoyoshi"]
   s.email       = ["himotoyoshi@users.noreply.github.com"]
   s.summary     = "Multi-dimensional numerical array class for Ruby"