netcdf 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 81d6a201281ded6b844b92d6d66413b81aa9c89c3b2faff4c76d783fdefea7d3
4
- data.tar.gz: bbe34f91b073bfb82daa5825dd1826c19b6d02cfce585e8df919170207749ea6
3
+ metadata.gz: ede50e0a5d1f1fca85a4210bc6012277979828fbe02a3a47803d3189568bc7c5
4
+ data.tar.gz: d6041661c7e95d520d816ebe2f4384095502d96600b174a35ddef06e20fee968
5
5
  SHA512:
6
- metadata.gz: 3fdb1a510287e5f1b886ae66cadf6864a5efaa896452f4f9aa158bb88ccef40b6fef6a0c8dff536cef464b6db7360bc800b64a7656d6c53f8d47223c0ffccee1
7
- data.tar.gz: 3dd9e164eefbec8f3868deac541ace703b831fa67bd390319be2073af12bf31937622ef24342fc5344a302e2df7607f5978597e393363a9d45347eb500f51c85
6
+ metadata.gz: d0819e52979e171112c2eaddb0cee8354155c14dd6131355f2a1ab1202918e3d056e2aab38ef24e8f9013046c5fe9ea387e80e0558f5c3e9b312652f4da87fbb
7
+ data.tar.gz: 9d2d0520db997082134630cbe1a94b8e7b09e026dad206668f2455d77735b14abd486d6c9113abbd591d7b61784c521a99518c8f3e4bc7588d963095f25f84a7
data/CHANGELOG.md CHANGED
@@ -29,6 +29,45 @@ version you have and a newer one.
29
29
  libnetcdf a release needs is said in extconf.rb, and an entry says
30
30
  so only when the answer changes. -->
31
31
 
32
+ ## 0.2.0
33
+
34
+ - New: the `NC_64BIT_DATA` creation flag, and its alias `NC_CDF5`, for
35
+ `nc_create` to write a CDF-5 file. Defined only when libnetcdf has
36
+ them (4.4 or later); `NC_FORMAT_64BIT_DATA` was already there.
37
+
38
+ - New: calls for the compression filters szip, zstandard, bzip2 and
39
+ blosc: `nc_def_var_szip` / `nc_inq_var_szip`, and the same pair for
40
+ `zstandard`, `bzip2` and `blosc` (libnetcdf 4.9 or later). New constants
41
+ give the filter ids (`H5Z_FILTER_*`) and the szip and blosc options.
42
+ When libnetcdf has no such filter, the def calls raise
43
+ `NetCDF::Error` (`NC_ENOFILTER`) instead of writing uncompressed data.
44
+ For szip this check needs libnetcdf 4.7 or later.
45
+
46
+ - New: `NetCDF.nc_copy_var(src_ncid, varid, dst_ncid)` copies one variable
47
+ -- its definition, attributes and data -- to another file. The
48
+ destination needs dimensions with the same names.
49
+
50
+ - New: `NetCDF.nc_set_default_format(format)` sets the format that
51
+ `nc_create` uses when the mode names none, and returns the old one.
52
+ The setting is for the whole process, not for one file.
53
+
54
+ - New: `NetCDF.nc_inq_type_equal(ncid1, typeid1, ncid2, typeid2)` returns
55
+ `true` if the two types are the same, also when they are in different
56
+ files or groups. A user-defined type can be compared only after
57
+ `nc_enddef`; in define mode the call raises `NetCDF::Error`.
58
+
59
+ - New: `NetCDF.nc_inq_unlimdims(ncid)` returns the ids of all unlimited
60
+ dimensions as an Array, or `[]` if there are none. A NetCDF-4 file can
61
+ have more than one unlimited dimension; `nc_inq_unlimdim` still returns
62
+ only one of them.
63
+
64
+ - Fix: an attribute of enum type, such as the `_FillValue` netCDF4-python
65
+ writes for an enum variable, is read as Integers of the enum's base
66
+ type by `nc_get_att` and by the base type's typed getter, instead of
67
+ raising `NotImplementedError`. `nc_def_var_fill` and `nc_inq_var_fill`
68
+ accept an enum variable in the same way. Writing an enum attribute
69
+ other than `_FillValue` is still not supported.
70
+
32
71
  ## 0.1.0
33
72
 
34
73
  First release.
data/README.md CHANGED
@@ -1,20 +1,25 @@
1
1
  # netcdf
2
2
 
3
- A Ruby binding to the [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) C library that passes data through Ruby's MemoryView protocol. An array library's own memory is handed to libnetcdf, which reads or writes it where it lies: nothing is copied, and no Ruby Array stands in between.
3
+ A Ruby binding to the [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) C library that passes data through Ruby's MemoryView protocol.
4
4
 
5
- The binding is nearly one to one, for code that wants to be written the way NetCDF itself is written; [Unidata's C documentation](https://docs.unidata.ucar.edu/netcdf-c/current/) reads as documentation for it.
5
+ The binding is nearly one to one, each C function corresponding to a Ruby method of the same name, so [Unidata's C documentation](https://docs.unidata.ucar.edu/netcdf-c/current/) reads as documentation for it.
6
6
 
7
7
  ## Features
8
8
 
9
- * Programs are written the way NetCDF's own are: every `nc_*` function under its own name, taking its arguments in its own order, so a call in a C example is written here as it stands
10
- * Any MemoryView producer is a buffer, with no adapter written for it, and libnetcdf reads or writes its memory where it lies
11
- * Array I/O releases the GVL, so other threads keep running through a long read or write; netCDF calls themselves are serialized, since libnetcdf is not thread-safe
12
-
13
- What libnetcdf offers is bound and covered by the test suite, apart from a handful of features listed at the end of the [reference](docs/03_reference.md#not-yet-implemented).
9
+ * Provides the same `nc_*` functions as the NetCDF C library, with their arguments in the same order as the originals
10
+ * Variable data is read and written with any buffer object that supports MemoryView
11
+ * Ruby's GVL (Global VM Lock) is released while variable data is read or written
12
+ * NetCDF calls from different threads run one at a time (libnetcdf is not thread-safe)
14
13
 
15
14
  ## Install
16
15
 
17
- Install libnetcdf and its headers first, then the gem:
16
+ Requires:
17
+
18
+ - Ruby >= 3.1
19
+ - libnetcdf >= 4.1, with its headers (`nc-config` or `pkg-config` on PATH)
20
+ - A C compiler
21
+
22
+ With those in place, install the gem:
18
23
 
19
24
  ```
20
25
  gem install netcdf
@@ -26,19 +31,13 @@ Or add it to your `Gemfile`:
26
31
  gem "netcdf"
27
32
  ```
28
33
 
29
- Requires:
30
-
31
- - Ruby >= 3.1
32
- - libnetcdf >= 4.1, with its headers (`nc-config` or `pkg-config` on PATH)
33
- - A C compiler
34
-
35
34
  ## Example
36
35
 
37
- Arrays are passed to and from this gem through [MemoryView](https://github.com/ruby/ruby/blob/master/include/ruby/memory_view.h), the protocol Ruby 3.0 added for sharing an array's memory between libraries. An array library that implements it works here as it is, with no adapter written on either side; the gem itself depends on no array library. The example below takes [CArray](https://github.com/himotoyoshi/carray) as one such library: it writes a 3 × 4 variable with two attributes, and reads it back.
36
+ Variable data is read and written through the [MemoryView](https://github.com/ruby/ruby/blob/master/include/ruby/memory_view.h) protocol, so any buffer object that supports MemoryView can be the other end of the binary I/O. The example below uses [CArray](https://github.com/himotoyoshi/carray) to show a NetCDF variable being written and read back.
38
37
 
39
38
  ```ruby
40
39
  require "netcdf"
41
- require "carray" # a MemoryView producer/consumer
40
+ require "carray" # an array library that supports MemoryView
42
41
 
43
42
  ncid = NetCDF.nc_create("out.nc", NetCDF::NC_CLOBBER | NetCDF::NC_NETCDF4)
44
43
 
@@ -60,18 +59,16 @@ NetCDF.nc_get_vara(ncid, v, [0, 0], [3, 4], dst)
60
59
  NetCDF.nc_close(ncid)
61
60
  ```
62
61
 
63
- libnetcdf reads and writes the array's own memory, with nothing copied in between.
64
-
65
62
  ## Documentation
66
63
 
67
- * [Introduction](docs/00_introduction.md) — what this gem is, what it leaves to a layer above, and what is not implemented
64
+ * [Introduction](docs/00_introduction.md) — what the netcdf gem is, what it leaves to libraries built on it, and what is not implemented
68
65
  * [Getting started](docs/01_getting_started.md) — install, the shape of the API, writing and reading your first file
69
66
  * [Buffers and MemoryView](docs/02_buffers.md) — how data actually moves: what a buffer must satisfy, format strings, and handing an array library's memory straight over
70
- * [Reference](docs/03_reference.md) — every function, grouped by subsystem, plus every constant
67
+ * [Reference](docs/03_reference.md) — every function, grouped by subsystem, and every constant
71
68
 
72
69
  ## Contributing
73
70
 
74
- Bug reports, feature requests and pull requests are welcome.
71
+ Bug reports, feature requests and pull requests are welcome on [GitHub](https://github.com/himotoyoshi/netcdf/issues).
75
72
 
76
73
  ## Credits
77
74
 
@@ -79,4 +76,4 @@ netcdf was designed and reviewed by a human developer; the implementation was pr
79
76
 
80
77
  ## License
81
78
 
82
- MIT
79
+ [MIT](LICENSE)
@@ -1,28 +1,28 @@
1
1
  # Introduction
2
2
 
3
- netcdf is a Ruby binding to the [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) C library. It takes NetCDF itself as known: what this guide has to explain is the Ruby side of the boundary — how a `nc_*` function reads as a Ruby call, what it returns, what it raises, and how bulk data crosses without being copied.
3
+ The netcdf gem is a Ruby binding to the [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) C library. This guide covers the Ruby side of the boundary: how a `nc_*` function reads as a Ruby call, what it returns, what it raises, and how variable data crosses without being copied.
4
4
 
5
5
  ## A literal translation of libnetcdf
6
6
 
7
- The gem translates libnetcdf into Ruby word for word. Each `nc_*` function is a `NetCDF.nc_*` method with the same name, the same arguments in the same order, and the same meaning. Only the grammar changes: what C returns through a pointer argument comes back as a Ruby value, and what C reports as a non-zero status is raised. The vocabulary does not — nothing is grouped, renamed, or given a friendlier spelling — so a call you read in C documentation, in an example, or in someone's Fortran-shaped code is written here as it stands. The few places where a Ruby call does more than its C counterpart — `nc_put_att` taking the type from the value, an `NC_STRING` read returning Ruby Strings — are marked in the [reference](03_reference.md) where they occur.
7
+ The gem translates libnetcdf into Ruby word for word. Each `nc_*` function in the NetCDF C library is a `NetCDF.nc_*` method with the same name and the same arguments in the same order. Only the grammar changes: what C returns through a pointer argument comes back as a Ruby value, and what C reports as a non-zero status is raised as a `NetCDF::Error`. The vocabulary does not, so a call you read in C documentation or in an example is written here as it stands. The few places where a Ruby call differs from its C counterpart — `nc_put_att` taking the type from the value, an `NC_STRING` read returning Ruby Strings, a lookup that finds nothing returning `nil` — are marked in the [reference](03_reference.md) where they occur.
8
8
 
9
9
  Translating rather than redesigning settles three things:
10
10
 
11
- - **Ids stay the integers libnetcdf uses.** `ncid`, `dimid`, `varid` and `grpid` are passed to you and back unchanged, so a file's identity is the same value on both sides of the boundary and yours to keep wherever you like.
12
- - **Data stays where your array library holds it.** A buffer here is a block of memory the library reads or writes in place; element access, arithmetic, slicing and type coercion stay with the array class that owns the data.
13
- - **Attributes are stored and returned verbatim.** `_FillValue`, `scale_factor`, `add_offset` and CF time come back exactly as they were written — a convention is yours to apply, at the point where you know which one the file follows.
11
+ - **Ids stay the integers libnetcdf uses.** `ncid`, `dimid`, `varid` and `grpid` are plain Integers, passed to you and back unchanged, with no Ruby object wrapped around them.
12
+ - **Data stays where your array library holds it.** libnetcdf reads or writes its memory in place; element access, arithmetic, slicing and type coercion stay with the array class that owns the data.
13
+ - **Attributes are stored and returned verbatim.** `_FillValue`, `scale_factor`, `add_offset` and CF time come back exactly as they were written — a convention is yours to apply.
14
14
 
15
- A translation this close can be checked against the C API function by function, and it never stands between you and a feature it was not designed for. The conveniences — an object-oriented surface, masks, CF/COARDS decoding — belong to a layer written above this one.
15
+ A translation this close can be checked against the C API function by function. The conveniences — an object-oriented API, masks, CF/COARDS decoding — belong in a separate library built on this gem.
16
16
 
17
- The practical consequence is that [Unidata's C documentation](https://docs.unidata.ucar.edu/netcdf-c/current/) is documentation for this gem. When you want to know what `nc_def_var_chunking` does to a file, that is where the answer is; what this guide covers is what the Ruby call returns, what it raises, and how the buffer works.
17
+ The practical consequence is that [Unidata's C documentation](https://docs.unidata.ucar.edu/netcdf-c/current/) is documentation for this gem. When you want to know what `nc_def_var_chunking` does to a file, that is where the answer is.
18
18
 
19
19
  ## Passing data through the MemoryView protocol
20
20
 
21
- Metadata crosses as ordinary Ruby values: names are Strings, ids and lengths are Integers, an attribute goes in as a Ruby value and comes back as one. Bulk numeric data does not — it is handed over through Ruby's MemoryView protocol, which is what lets a whole array cross the boundary as memory rather than as a converted Ruby object.
21
+ Metadata crosses as ordinary Ruby values: names are Strings, ids and lengths are Integers, an attribute goes in as a Ruby value and comes back as one. Variable data does not — it is handed over through the MemoryView protocol, which is what lets a whole array cross the boundary as memory rather than as a converted Ruby object.
22
22
 
23
- A buffer, in this gem, is any object that exposes MemoryView: the gem asks it for the pointer, checks that the memory is row-major contiguous and that its format agrees with the variable, and lets libnetcdf read or write it where it lies. The gem copies nothing in either direction — what libnetcdf does with the bytes after that, type conversion included, is libnetcdf's business — and the call runs with the GVL released. An array class that exposes the protocol is therefore usable as a buffer with no adapter written for it.
23
+ A buffer, in this gem, is any object that supports MemoryView: the gem asks it for the pointer, checks that the memory is row-major contiguous and that its format is one the gem can read, and lets libnetcdf read or write it where it lies. The gem copies nothing in either direction — what libnetcdf does with the bytes after that, type conversion included, is libnetcdf's business.
24
24
 
25
- The gem allocates no buffers of its own: a `get` reads into memory you hand it, as a `put` writes from it. The gem depends on no array library; this guide's examples take CArray as one that implements MemoryView.
25
+ The gem allocates no buffers of its own: a `get` reads into memory you hand it, as a `put` writes from it. The gem depends on no array library; this guide's examples take [CArray](https://github.com/himotoyoshi/carray) as one that supports MemoryView. What a buffer must satisfy is in [Buffers and MemoryView](02_buffers.md).
26
26
 
27
27
  ## What is not implemented
28
28
 
@@ -1,6 +1,6 @@
1
1
  # Getting started
2
2
 
3
- This page goes from an empty directory to a NetCDF file written and read back. NetCDF itself is taken as known; what is shown is how a `nc_*` call is written in Ruby, and which of its edges are Ruby's rather than C's.
3
+ This page goes from an empty directory to a NetCDF file written and read back, showing how a `nc_*` call is written in Ruby and where it differs from C.
4
4
 
5
5
  ## Install
6
6
 
@@ -17,7 +17,7 @@ require "netcdf"
17
17
  NetCDF.nc_inq_libvers # => "4.9.2 of Mar 15 2024 ..."
18
18
  ```
19
19
 
20
- The examples below use [CArray](https://github.com/himotoyoshi/carray), a multi-dimensional array library that can be both a producer and a consumer of MemoryView:
20
+ The examples below use [CArray](https://github.com/himotoyoshi/carray), a multi-dimensional array library that supports MemoryView, so its arrays can be both written from and read into:
21
21
 
22
22
  ```
23
23
  gem install carray
@@ -25,30 +25,27 @@ gem install carray
25
25
 
26
26
  ## What is Ruby rather than C
27
27
 
28
- `NetCDF.nc_open`, `NetCDF.nc_def_var`, `NetCDF.nc_put_vara`: the names and the argument order are the ones you already know. Four things are spelled Ruby's way instead of C's.
28
+ `NetCDF.nc_open`, `NetCDF.nc_def_var`, `NetCDF.nc_put_vara`: the names and the argument order are those of the C library. Three things differ from C:
29
29
 
30
30
  - **Status codes are exceptions.** A call that fails raises `NetCDF::Error`; there is no return value to check.
31
31
  - **Output parameters are return values.** `nc_inq_var` hands back its five values as an Array instead of filling five pointers.
32
- - **A lookup that finds nothing returns `nil`.** `nc_inq_varid` on a name that is not in the file, for one — absence is not an error.
33
- - **Bulk data moves through MemoryView.** A buffer is memory libnetcdf reads or writes where it lies, so an array library's own storage is handed over as it stands.
32
+ - **A lookup that finds nothing returns `nil`.** `nc_inq_varid` on a name that is not in the file returns `nil` rather than raising.
34
33
 
35
34
  ## Define mode and data mode
36
35
 
37
- `nc_enddef` and `nc_redef` are here as they are in C, and the gem does nothing to hide the two phases — but how strictly they are enforced depends on the format, which is worth seeing before the first example. A classic (NetCDF-3) file refuses a write before `nc_enddef`:
36
+ `nc_enddef` and `nc_redef` are here as they are in C, and the gem does nothing to hide the two phases — but how strictly they are enforced depends on the file format. A classic (NetCDF-3) file refuses a write before `nc_enddef`:
38
37
 
39
38
  ```
40
39
  NetCDF::Error: NetCDF: Operation not allowed in define mode # status -39
41
40
  ```
42
41
 
43
- A NetCDF-4 file does not — libnetcdf switches modes for you there, and the write goes through. Call `nc_enddef` anyway, at the point where the schema is complete: it is what makes the same code work against both formats, and it marks the boundary for a reader.
44
-
45
- Opening an existing file skips all of this — the file is already in data mode.
42
+ A NetCDF-4 file does not — libnetcdf switches modes for you there, and the write goes through. Call `nc_enddef` anyway, at the point where the schema is complete: it is what makes the same code work against both file formats.
46
43
 
47
44
  ## Writing a file
48
45
 
49
46
  ```ruby
50
47
  require "netcdf"
51
- require "carray" # a MemoryView producer/consumer
48
+ require "carray" # an array library that supports MemoryView
52
49
 
53
50
  ncid = NetCDF.nc_create("temp.nc", NetCDF::NC_CLOBBER | NetCDF::NC_NETCDF4)
54
51
 
@@ -65,7 +62,7 @@ NetCDF.nc_put_att(ncid, NetCDF::NC_GLOBAL, "title", "example field")
65
62
  NetCDF.nc_enddef(ncid)
66
63
 
67
64
  # --- data mode ---
68
- values = CArray.double(3, 4).seq(271.0)
65
+ values = CArray.double(3, 4).seq(271.0) # 271.0, 272.0, ..., 282.0
69
66
  NetCDF.nc_put_var(ncid, v, values)
70
67
 
71
68
  NetCDF.nc_close(ncid)
@@ -75,14 +72,14 @@ Four things in there are worth naming:
75
72
 
76
73
  - Creation flags are OR-ed together as in C: `NC_CLOBBER | NC_NETCDF4` here. A constant is `NetCDF::NC_*`, and every one libnetcdf defines is in the [reference](03_reference.md).
77
74
  - Dimension ids come back from `nc_def_dim` and go into `nc_def_var` as an Array, in the order the variable is indexed.
78
- - `nc_put_att` picks the on-disk type from the Ruby value — `String` becomes `NC_CHAR`, `Float` becomes `NC_DOUBLE`, an Array of Floats becomes an `NC_DOUBLE` vector. When you need to pin the type down, the typed calls (`nc_put_att_float` and friends) are in the [reference](03_reference.md#typed-putget).
79
- - The data goes in as a `CArray`, not a Ruby Array: libnetcdf reads the array's own memory. Any other MemoryView producer would do; that is the subject of the [next page](02_buffers.md).
75
+ - `nc_put_att` picks the on-disk type from the Ruby value — `String` becomes `NC_CHAR`, `Integer` becomes `NC_INT`, `Float` becomes `NC_DOUBLE`, an Array of Floats becomes an `NC_DOUBLE` vector. When you need to pin the type down, the typed calls (`nc_put_att_float` and friends) are in the [reference](03_reference.md#typed-putget).
76
+ - The data goes in as a `CArray`; a Ruby Array is not a buffer and raises `TypeError`. Any other object that supports MemoryView would do; that is the subject of the [next page](02_buffers.md).
80
77
 
81
78
  ## Reading it back
82
79
 
83
80
  ```ruby
84
81
  require "netcdf"
85
- require "carray" # a MemoryView producer/consumer
82
+ require "carray" # an array library that supports MemoryView
86
83
 
87
84
  ncid = NetCDF.nc_open("temp.nc", NetCDF::NC_NOWRITE)
88
85
 
@@ -95,29 +92,34 @@ NetCDF.nc_get_att(ncid, varid, "units") # => "K"
95
92
 
96
93
  buf = CArray.double(*shape)
97
94
  NetCDF.nc_get_var(ncid, varid, buf) # => buf, filled
98
- buf[0, nil].to_a # => [271.0, 272.0, 273.0, 274.0]
95
+ buf[0, nil].to_a # => [271.0, 272.0, 273.0, 274.0] (row 0, every x)
99
96
 
100
97
  NetCDF.nc_close(ncid)
101
98
  ```
102
99
 
103
- A `get` reads into the buffer you pass, as a `put` writes from it, and returns that buffer. The gem allocates none of its own: the shape comes from the dimensions, and the element type is yours to choose — libnetcdf converts from the variable's type to it. The one exception is an `NC_STRING` variable, which comes back as Ruby Strings and takes no buffer.
100
+ `nc_get_var` returns the buffer it filled. The shape comes from the dimensions, and the element type is yours to choose — libnetcdf converts from the variable's type to it. The one case that differs is an `NC_STRING` variable, which comes back as Ruby Strings and takes no buffer.
104
101
 
105
102
  ## Reading part of a variable
106
103
 
107
- `nc_get_vara` takes a corner and a shape as Ruby Arrays, zero-based as in C:
104
+ `nc_get_vara` takes `start` and `count` as Ruby Arrays, zero-based as in C:
108
105
 
109
106
  ```ruby
107
+ ncid = NetCDF.nc_open("temp.nc", NetCDF::NC_NOWRITE)
108
+ varid = NetCDF.nc_inq_varid(ncid, "temperature")
109
+
110
110
  rows = CArray.double(2, 4)
111
111
  NetCDF.nc_get_vara(ncid, varid, [1, 0], [2, 4], rows) # last two rows
112
+
113
+ NetCDF.nc_close(ncid)
112
114
  ```
113
115
 
114
- `nc_get_vars` adds a stride, and `nc_get_varm` a mapping vector for transposing on the way out. Each has a `put` twin taking the same arguments plus the buffer.
116
+ `nc_get_vars` adds a stride, and `nc_get_varm` a mapping vector for transposing on the way out. Each has a `put` twin taking the same arguments.
115
117
 
116
118
  ## When something fails
117
119
 
118
120
  ```ruby
119
121
  begin
120
- NetCDF.nc_open("missing.nc")
122
+ NetCDF.nc_open("missing.nc", NetCDF::NC_NOWRITE)
121
123
  rescue NetCDF::Error => e
122
124
  e.message # => "No such file or directory" — libnetcdf's own text
123
125
  e.status # => 2 the raw libnetcdf status
@@ -125,3 +127,5 @@ end
125
127
  ```
126
128
 
127
129
  `#status` is what to match on when you care about a specific failure rather than any failure. NetCDF's own conditions are negative (`-39` is "operation not allowed in define mode"); a positive status is an errno the C library passed through, as above.
130
+
131
+ Next, [Buffers and MemoryView](02_buffers.md) covers what a buffer must satisfy, and the [reference](03_reference.md) lists every function and constant.
data/docs/02_buffers.md CHANGED
@@ -1,27 +1,27 @@
1
1
  # Buffers and MemoryView
2
2
 
3
- Every numeric value this gem reads or writes passes through a buffer, and a buffer is any object implementing Ruby's MemoryView protocol. That is the whole interop story: an array library that exposes MemoryView hands libnetcdf its own memory, and the gem copies nothing on the way in or out.
3
+ Variable data passes through a buffer, and a buffer is any object that supports the MemoryView protocol. The gem passes the buffer's memory straight to libnetcdf, which reads or writes it in place; nothing is copied on the way in or out.
4
4
 
5
- This page is that surface: what a buffer has to satisfy, what to pass when you have no array library, how a format string is read, and where an array library fits.
5
+ This page covers what a buffer has to satisfy, what to pass when you have no array library, how a format string is read, and where an array library fits.
6
6
 
7
- The gem allocates no buffers itself. A `get` reads into the buffer you pass and returns it, just as a `put` writes from one; leaving the buffer off raises `ArgumentError: buffer is required`. `NC_STRING` variables are the exception (see [the end of this page](#types-that-are-not-buffers)).
7
+ Leaving the buffer off raises `ArgumentError: buffer is required`, except for `NC_STRING` variables, which take none (see [below](#types-that-are-not-buffers)).
8
8
 
9
9
  ## A buffer is a block of memory
10
10
 
11
- Not a collection. A Ruby `Array` of Floats is a list of boxed objects; there is nothing in it to hand to a C function that wants twelve contiguous 8-byte values, and packing one is a copy plus a per-element conversion — work that belongs to whatever owns the data, not to this boundary.
11
+ A Ruby `Array` of Floats is a list of Ruby objects, not a block of memory, so it is not a buffer. Put the values in an array library's array, or pack them into a `Fiddle::Pointer` ([below](#without-an-array-library)).
12
12
 
13
- A `String` *is* a block of bytes, but it carries no shape, no element type, and no writability guarantee, so the array I/O surface does not take one. Hand the bytes over through a producer instead — an array library's array, or a `Fiddle::Pointer` ([below](#without-an-array-library)).
13
+ A `String` is a block of bytes, but it does not support MemoryView, so it is not a buffer either; copy the bytes into a `Fiddle::Pointer` instead.
14
14
 
15
- Strings do appear elsewhere — `NC_CHAR` and `NC_STRING` values, opaque scalars, attribute text — because there the data really is text or an opaque blob. It is only the numeric array surface that refuses them.
15
+ Strings are taken elsewhere — attribute text, `NC_STRING` values, and a single `NC_CHAR` or opaque element through `nc_put_var1` — because there the data really is text or an opaque blob. Only array I/O refuses them.
16
16
 
17
17
  ## What a buffer has to satisfy
18
18
 
19
- - **A MemoryView producer.** Anything else raises `TypeError: buffer must provide a MemoryView`. Note that a library's view objects are often not producers even when its arrays are — a `CArray` is one, `CArray#transpose` is not.
20
- - **Row-major (C-contiguous).** libnetcdf reads and writes a flat run of elements, so a strided or column-major view has no meaning at this boundary. A producer that hands over a strided view — including a 1-D one that gives strides without a shape — is refused with `ArgumentError: buffer must be row-major (C-contiguous)` rather than read wrongly. Make a contiguous copy first — `CArray#copy`, for a CArray view.
21
- - **Large enough.** The element count implied by `count` (or by the whole variable) must fit, or `ArgumentError: buffer too small` says how many bytes were expected.
22
- - **Writable, on a `get`.** A producer that refuses a writable view raises `TypeError: buffer must provide a writable MemoryView`. One that hands back a view marked read-only anyway — `rb_memory_view_get` does not hold a producer to the request — raises `TypeError: buffer is read-only`, rather than being written into. `Fiddle::Pointer` is one up to fiddle 1.1.8: it marks every view read-only, so it serves as the source of a `put` but not as the destination of a `get`. fiddle 1.1.9 hands out a writable view when asked.
19
+ - **Supports MemoryView.** Anything else raises `TypeError: buffer must provide a MemoryView`. A library's derived arrays may not support it even when its plain arrays do: a `CArray` does, but the array `CArray#transpose` returns does not — pass its `#copy`.
20
+ - **Row-major (C-contiguous).** libnetcdf reads and writes a flat run of elements, so strided or column-major memory is refused with `ArgumentError: buffer must be row-major (C-contiguous)` rather than read wrongly. Make a contiguous copy first (`CArray#copy`, for a CArray).
21
+ - **Large enough.** The element count implied by `count` (or by the whole variable) must fit, or `ArgumentError: buffer too small` says how many bytes were expected. Only the size is checked, not the shape: a 4 × 3 buffer serves a 3 × 4 read.
22
+ - **Writable, on a `get`.** A buffer that cannot be written raises `TypeError` (`buffer must provide a writable MemoryView`, or `buffer is read-only`). `Fiddle::Pointer` is read-only, so it serves as the source of a `put` but not as the destination of a `get`.
23
23
 
24
- The element *type* need not match the variable's: libnetcdf converts. Writing a `"f"` buffer into an `NC_DOUBLE` variable widens on the way in, and reading that variable into a `"f"` buffer narrows on the way out. What the buffer's format decides is how libnetcdf interprets your memory, not what ends up on disk.
24
+ The element *type* need not match the variable's: libnetcdf converts. Writing a float (`"f"`) buffer into an `NC_DOUBLE` variable widens on the way in, and reading that variable into a `"f"` buffer narrows on the way out. What the buffer's format decides is how libnetcdf interprets your memory, not what ends up on disk. A value that does not fit the type it is converted to raises `NetCDF::Error: NetCDF: Numeric conversion not representable`.
25
25
 
26
26
  ## Without an array library
27
27
 
@@ -35,11 +35,11 @@ src[0, 12 * 8] = (1..12).map(&:to_f).pack("d*")
35
35
  NetCDF.nc_put_vara(ncid, varid, [0, 0], [3, 4], src)
36
36
  ```
37
37
 
38
- It serves as the source of a `put`. It can be read into only from fiddle 1.1.9 on (see [what a buffer has to satisfy](#what-a-buffer-has-to-satisfy)); `#to_str(nbytes)` then copies the bytes out.
38
+ Nothing checks that the bytes are of the variable's type, so pack them as that type: doubles written into an `NC_FLOAT` variable are stored wrong, with no error. It serves as the source of a `put`; it cannot be read into (see [what a buffer has to satisfy](#what-a-buffer-has-to-satisfy)).
39
39
 
40
40
  ## Format strings
41
41
 
42
- A buffer describes its element type with a MemoryView `format` string. The vocabulary is the one `ruby/memory_view.h` specifies — pack-template specifiers — and this gem expects it:
42
+ A buffer describes its element type with a MemoryView `format` string. The vocabulary is the one `ruby/memory_view.h` specifies — pack-template specifiers:
43
43
 
44
44
  | format | NetCDF type |
45
45
  |--------|-------------|
@@ -50,23 +50,23 @@ A buffer describes its element type with a MemoryView `format` string. The voca
50
50
  | `f` | `NC_FLOAT` |
51
51
  | `d` | `NC_DOUBLE` |
52
52
 
53
- `l` / `L` are read as the 32-bit integers they are in that vocabulary, and `e` / `E` / `g` / `G` as the floats that carry their own byte order (below). PEP 3118's `b` / `B` and `h` / `H` are accepted too, for producers that spell the 1- and 2-byte integers PEP's way.
53
+ `l` / `L` are read as the 32-bit integers they are in that vocabulary, and `e` / `E` / `g` / `G` as the floats that carry their own byte order (below). The `b` / `B` and `h` / `H` of PEP 3118 (Python's buffer format) are accepted too, for buffers that spell the 1- and 2-byte integers that way.
54
54
 
55
- A format that disagrees with the buffer's own `item_size` — numpy's 64-bit `l` arriving untranslated, say — raises `ArgumentError` rather than being misread.
55
+ A format that disagrees with the buffer's element size — Python's 64-bit `l`, say — raises `ArgumentError` rather than being misread.
56
56
 
57
- A format may name a byte order. Ruby spells it as a suffix on an integer (`q<`, `s>`) and as a letter of its own on a float (`e` / `E` little-endian single / double, `g` / `G` big-endian); a suffix on a float (`d<`) is not Ruby's and is refused as unsupported. PEP 3118 spells it as a prefix on the whole string (`<d`, plus the order-neutral `=` `@` `|` `^`). All of these are read. One naming the *foreign* order raises `ArgumentError: buffer format "..." is not native-endian` — libnetcdf transfers the buffer as native memory, so byte-swapping is the producer's job.
57
+ A format may name a byte order, in Ruby's spelling (`q<`, `s>`, and `e` / `E` / `g` / `G` for floats) or PEP 3118's (`<d`). Native order is accepted; the foreign order raises `ArgumentError: buffer format "..." is not native-endian`, so swap the bytes before passing them.
58
58
 
59
- Only a single numeric element is accepted. Repeat counts, records (`T{...}`), and mixed formats (`dd`) have no netCDF counterpart and are rejected.
59
+ Only a single numeric element is accepted. Repeat counts, records (`T{...}`), and mixed formats (`dd`) have no NetCDF counterpart and raise `ArgumentError: unsupported buffer format`.
60
60
 
61
- A buffer with no format at all is passed through as raw bytes of the variable's own type. So is any buffer for an opaque variable, whose element size netCDF already knows and whose data has no element type for a format to name — see [opaque I/O](03_reference.md#opaque-fixed-size-raw-bytes). An enum variable moves its base type whatever the buffer holds, so a formatted buffer for one must name that base type; another raises `ArgumentError` rather than being reinterpreted. An `NC_CHAR` buffer moves bytes, so its format, when it has one, must name a 1-byte element.
61
+ A buffer with no format at all is passed through as raw bytes of the variable's own type. So is any buffer for an opaque variable, whose element size NetCDF already knows and whose data has no element type for a format to name — see [opaque I/O](03_reference.md#opaque-fixed-size-raw-bytes). An enum variable is read and written as its base type whatever the buffer holds, so a formatted buffer for one must name that base type; another raises `ArgumentError` rather than being reinterpreted. An `NC_CHAR` buffer holds bytes, so its format, when it has one, must name a 1-byte element.
62
62
 
63
63
  ## Array libraries
64
64
 
65
- An array class that exposes MemoryView natively is a buffer here with no adapter written for it and nothing copied on the way through. Pass the array where a buffer goes; the example takes CArray as one such class:
65
+ An array class that supports MemoryView is a buffer here. Pass the array where a buffer goes; the example takes CArray as one such class:
66
66
 
67
67
  ```ruby
68
68
  require "netcdf"
69
- require "carray" # a MemoryView producer/consumer
69
+ require "carray" # an array library that supports MemoryView
70
70
 
71
71
  src = CArray.double(3, 4).seq(1.0)
72
72
  NetCDF.nc_put_vara(ncid, varid, [0, 0], [3, 4], src)
@@ -75,9 +75,7 @@ dst = CArray.double(3, 4)
75
75
  NetCDF.nc_get_vara(ncid, varid, [0, 0], [3, 4], dst) # filled in place
76
76
  ```
77
77
 
78
- An array class that does not expose it has its data copied into one that does, once in each direction.
79
-
80
- The gem's own code names no library: which of the two paths yours takes is decided by the protocol alone, and it moves to the first the moment it grows a MemoryView entry.
78
+ An array class that does not support it cannot be passed: copy its data into one that does before a `put`, and out of it after a `get`.
81
79
 
82
80
  ## Types that are not buffers
83
81
 
@@ -86,10 +84,10 @@ Two of NetCDF's types do not fit into a single block of memory, and the gem hand
86
84
  - **`NC_STRING`** — each element is a separately allocated C string, so a `get` takes no buffer and returns a `String` or a nested `Array<String>`, and a `put` accepts the same. See [NC_STRING variables](03_reference.md#nc_string-variables).
87
85
  - **Vlen** — a variable-length sequence per element. Type definition works; data I/O is not wired through yet.
88
86
 
89
- ## The GVL
87
+ ## Threads and the GVL
90
88
 
91
- Array I/O releases the GVL around the libnetcdf call, so a large read or write does not block other Ruby threads. The buffer must not be touched from another thread while the call is in flight — the gem holds a raw pointer into it for the duration.
89
+ Ruby's GVL (Global VM Lock) lets only one thread run Ruby code at a time, and a C call that holds it on the way through stops every other thread until it returns. Array I/O releases the GVL around the libnetcdf call, so while a large read or write waits on the disk or decompresses chunks, other Ruby threads keep running — one serving requests, one reporting progress, one working on data already read. The buffer must not be touched from another thread while the call is in flight — the gem holds a raw pointer into it for the duration.
92
90
 
93
- libnetcdf, and the HDF5 under it, are not thread-safe: two threads inside them at once corrupt process-wide state, even when they work on different files. So every `NetCDF` call — array I/O, and inquiry, definition, open and close alike — takes one process-wide lock. A thread that calls into NetCDF while another is in the middle of a read waits for it, with the GVL released; threads doing anything else keep running. Threads therefore do not speed up netCDF I/O itself, only the Ruby work around it. The lock covers one call at a time: a sequence of calls that must not interleave with another thread's — define, then write, say — still needs a lock of your own.
91
+ libnetcdf, and the HDF5 under it, are not thread-safe: two threads inside them at once corrupt process-wide state, even when they work on different files. So every `NetCDF` call — array I/O, and inquiry, definition, open and close alike — takes one process-wide lock. A thread that calls into NetCDF while another is in the middle of a read waits for it, with the GVL released; threads doing anything else keep running. Threads therefore do not speed up NetCDF I/O itself, only the Ruby work around it. The lock covers one call at a time: a sequence of calls that must not interleave with another thread's — define, then write, say — still needs a lock of your own.
94
92
 
95
- `NetCDF` can be called only from the main Ractor. A call from any other Ractor raises an error. For netCDF I/O that runs in parallel, use separate processes.
93
+ `NetCDF` can be called only from the main Ractor. A call from any other Ractor raises an error. For NetCDF I/O that runs in parallel, use separate processes.
data/docs/03_reference.md CHANGED
@@ -24,9 +24,9 @@ Every function this gem defines, grouped by subsystem. [Getting started](01_get
24
24
  - `ncid`, `dimid`, `varid`, `attnum`, `grpid` are the integers libnetcdf uses, passed to and from these calls unchanged.
25
25
  - Functions that take an `Array` of indices (`start`, `count`, `stride`, etc.) expect a Ruby `Array` of `Integer` whose length equals the variable's rank.
26
26
  - Smart-typing generics (`nc_put_att`, `nc_get_att`, `nc_put_var1`, `nc_get_var1`, `nc_put_var{,a,s,m}`, `nc_get_var{,a,s,m}`) dispatch on the Ruby value type or the variable's `nc_type`. There are no typed Ruby variants for variables — Ruby's type system already covers the relevant cases. Attributes do expose typed variants because the put-call itself determines the on-disk attribute type.
27
- - Buffer arguments for array I/O are MemoryView producers. A raw Ruby `String` is *not* accepted there; see [Buffers and MemoryView](02_buffers.md).
27
+ - Buffer arguments for array I/O are objects that support MemoryView. A raw Ruby `String` is *not* accepted there; see [Buffers and MemoryView](02_buffers.md).
28
28
  - `get` calls read into the buffer passed and return it; the gem allocates none of its own, and omitting the buffer raises `ArgumentError`. `NC_STRING` variables are the exception: they take no buffer and return a nested Ruby `Array<String>` (`char**` does not map to MemoryView).
29
- - Array I/O releases the GVL around the underlying libnetcdf call. Every call is serialized on one process-wide lock, because libnetcdf is not thread-safe; see [the GVL](02_buffers.md#the-gvl).
29
+ - Array I/O releases the GVL around the underlying libnetcdf call. Every call is serialized on one process-wide lock, because libnetcdf is not thread-safe; see [Threads and the GVL](02_buffers.md#threads-and-the-gvl).
30
30
 
31
31
  ## File
32
32
 
@@ -44,8 +44,10 @@ NetCDF.nc_inq_ndims(ncid) # => Integer
44
44
  NetCDF.nc_inq_nvars(ncid) # => Integer
45
45
  NetCDF.nc_inq_natts(ncid) # => Integer
46
46
  NetCDF.nc_inq_unlimdim(ncid) # => Integer | nil
47
+ NetCDF.nc_inq_unlimdims(ncid) # => [Integer, ...] (every unlimited dim; [] if none)
47
48
 
48
49
  oldmode = NetCDF.nc_set_fill(ncid, mode)
50
+ oldfmt = NetCDF.nc_set_default_format(fmt) # fmt: NC_FORMAT_*; for every later nc_create in the process
49
51
 
50
52
  path = NetCDF.nc_inq_path(ncid) # libnetcdf >= 4.1.2, => String
51
53
  fmt = NetCDF.nc_inq_format(ncid) # => NC_FORMAT_*
@@ -128,8 +130,11 @@ NetCDF.nc_inq_varndims(ncid, varid)
128
130
  NetCDF.nc_inq_vardimid(ncid, varid)
129
131
  NetCDF.nc_inq_varnatts(ncid, varid)
130
132
  NetCDF.nc_rename_var(ncid, varid, newname)
133
+ NetCDF.nc_copy_var(src_ncid, varid, dst_ncid) # => nil
131
134
  ```
132
135
 
136
+ `nc_copy_var` copies the definition, the attributes and the data of one variable to another file. The destination must already have dimensions with the same names. It may be in define mode or in data mode. A variable of the same name already there raises `NetCDF::Error`. To find the new variable, use `nc_inq_varid` with its name.
137
+
133
138
  ### Variable scalar I/O
134
139
 
135
140
  ```
@@ -153,7 +158,7 @@ NetCDF.nc_get_vars(ncid, varid, start, count, stride[, buffer])
153
158
  NetCDF.nc_get_varm(ncid, varid, start, count, stride, imap[, buffer])
154
159
  ```
155
160
 
156
- `buffer` is any object that implements the Ruby MemoryView protocol. It must be sized to hold the requested elements, must be writable on `get` calls, and must be row-major C-contiguous; [Buffers and MemoryView](02_buffers.md) has the rules and the errors they raise.
161
+ `buffer` is any object that supports MemoryView. It must be sized to hold the requested elements, must be writable on `get` calls, and must be row-major C-contiguous; [Buffers and MemoryView](02_buffers.md) has the rules and the errors they raise.
157
162
 
158
163
  On `get`, the read fills `buffer` and returns it. The argument is written as optional only because `NC_STRING` variables take none and return a nested `Array<String>` instead (see below); for any other type, omitting it raises `ArgumentError: buffer is required`.
159
164
 
@@ -192,6 +197,16 @@ NetCDF.nc_inq_var_filter_info(ncid, varid, filter_id) # libnetcdf >=
192
197
  NetCDF.nc_inq_var_filter_ids(ncid, varid) # libnetcdf >= 4.7, => Array<Integer>
193
198
  NetCDF.nc_inq_filter_avail(ncid, filter_id) # libnetcdf >= 4.7, => true | false
194
199
 
200
+ NetCDF.nc_def_var_szip(ncid, varid, options_mask, pixels_per_block) # options_mask: NC_SZIP_NN or NC_SZIP_EC
201
+ NetCDF.nc_inq_var_szip(ncid, varid) # => [options_mask, pixels_per_block]
202
+ NetCDF.nc_def_var_zstandard(ncid, varid, level) # libnetcdf >= 4.9
203
+ NetCDF.nc_inq_var_zstandard(ncid, varid) # libnetcdf >= 4.9, => [has_filter, level]
204
+ NetCDF.nc_def_var_bzip2(ncid, varid, level) # libnetcdf >= 4.9
205
+ NetCDF.nc_inq_var_bzip2(ncid, varid) # libnetcdf >= 4.9, => [has_filter, level]
206
+ NetCDF.nc_def_var_blosc(ncid, varid, subcompressor, level, blocksize, shuffle) # libnetcdf >= 4.9
207
+ NetCDF.nc_inq_var_blosc(ncid, varid) # libnetcdf >= 4.9,
208
+ # => [has_filter, subcompressor, level, blocksize, shuffle]
209
+
195
210
  NetCDF.nc_def_var_quantize(ncid, varid, mode, nsd) # libnetcdf >= 4.9
196
211
  NetCDF.nc_inq_var_quantize(ncid, varid) # libnetcdf >= 4.9, => [mode, nsd]
197
212
 
@@ -199,6 +214,13 @@ NetCDF.nc_set_var_chunk_cache(ncid, varid, size, nelems, preemption)
199
214
  NetCDF.nc_get_var_chunk_cache(ncid, varid) # => [size, nelems, preemption]
200
215
  ```
201
216
 
217
+ The calls for szip, zstandard, bzip2 and blosc are there even when libnetcdf was built without the filter. To check whether a filter can be used, call `nc_inq_filter_avail` with its `H5Z_FILTER_*` id. When the filter is missing:
218
+
219
+ - The four def calls raise `NetCDF::Error` with status `NC_ENOFILTER` (-136). The gem checks this itself, because a libnetcdf without blosc returns no error from `nc_def_var_blosc` and writes the data uncompressed. For szip the check needs libnetcdf 4.7 or later.
220
+ - The `nc_inq_var_*` calls report that the variable does not have the filter.
221
+
222
+ A filter needs chunked storage (`nc_def_var_chunking` with `NC_CHUNKED`). A classic-format file has no filters.
223
+
202
224
  Global chunk cache (process-wide HDF5 cache parameters):
203
225
 
204
226
  ```
@@ -286,10 +308,13 @@ Strings read back — `NC_CHAR` and `NC_STRING` attributes, and `NC_CHAR` and `N
286
308
  name, size, base_type, nfields, klass = NetCDF.nc_inq_user_type(ncid, typeid)
287
309
  typeid = NetCDF.nc_inq_typeid(ncid, name) # => Integer | nil
288
310
  NetCDF.nc_inq_typeids(ncid) # => Array<Integer>
311
+ NetCDF.nc_inq_type_equal(ncid1, typeid1, ncid2, typeid2) # => true | false
289
312
  ```
290
313
 
291
314
  `klass` is one of `NC_VLEN`, `NC_OPAQUE`, `NC_ENUM`, `NC_COMPOUND`.
292
315
 
316
+ `nc_inq_type_equal` compares a user-defined type only after `nc_enddef` on its file. In define mode it raises `NetCDF::Error` ("HDF error"), even when both arguments name the same type.
317
+
293
318
  ### Opaque (fixed-size raw bytes)
294
319
 
295
320
  ```
@@ -314,7 +339,7 @@ NetCDF.nc_get_vara(ncid, v, [0], [3], out)
314
339
  out.to_s # => "ABCDEFGHIJKL"
315
340
  ```
316
341
 
317
- Raw Ruby Strings are not accepted on the array I/O surface, the same as for any other type.
342
+ Raw Ruby Strings are not accepted by array I/O, the same as for any other type.
318
343
 
319
344
  ### Vlen (variable-length sequence)
320
345
 
@@ -339,6 +364,8 @@ NetCDF.nc_inq_enum_ident(ncid, typeid, value) # => String | nil
339
364
 
340
365
  Variables of enum type are read and written as plain Integers — internally the gem uses libnetcdf's untyped scalar/array I/O to satisfy the enum type check. That I/O moves elements of the base type with no conversion, so an array buffer for an enum variable holds the base type: a formatted buffer must name it (`"i"` for an `NC_INT` base), and one naming another type raises `ArgumentError`. A formatless buffer is read as the base type.
341
366
 
367
+ Attributes of enum type are read the same way. libnetcdf's typed getters refuse one, but here `nc_get_att` returns its values as Integers of the base type, and the typed getter for the base type (`nc_get_att_uchar` for an `NC_UBYTE` base) reads it too; any other typed getter raises `NetCDF::Error`. The usual case is a `_FillValue` on an enum variable, which netCDF4-python writes with the enum's type. `nc_def_var_fill` and `nc_inq_var_fill` on an enum variable take and return the fill value as an Integer of the base type, and the default fill is the base type's. There is no call yet to write an enum attribute other than through `nc_def_var_fill`.
368
+
342
369
  ## Group (NetCDF-4 only)
343
370
 
344
371
  ```
@@ -359,7 +386,7 @@ A group's `ncid` is interchangeable with the file's `ncid` for almost all calls
359
386
 
360
387
  ## Constants
361
388
 
362
- Open / create modes: `NC_NOWRITE`, `NC_WRITE`, `NC_CLOBBER`, `NC_NOCLOBBER`, `NC_SHARE`, `NC_NETCDF4`, `NC_CLASSIC_MODEL`, `NC_64BIT_OFFSET`.
389
+ Open / create modes: `NC_NOWRITE`, `NC_WRITE`, `NC_CLOBBER`, `NC_NOCLOBBER`, `NC_SHARE`, `NC_NETCDF4`, `NC_CLASSIC_MODEL`, `NC_64BIT_OFFSET`, `NC_64BIT_DATA` (alias `NC_CDF5`; CDF-5, defined only when the linked libnetcdf provides it).
363
390
 
364
391
  Diskless / in-memory modes (defined only when the linked libnetcdf provides them): `NC_DISKLESS`, `NC_PERSIST`, `NC_MMAP`, `NC_INMEMORY`.
365
392
 
@@ -381,6 +408,12 @@ Endian: `NC_ENDIAN_NATIVE`, `NC_ENDIAN_LITTLE`, `NC_ENDIAN_BIG`.
381
408
 
382
409
  Quantize modes: `NC_NOQUANTIZE`, `NC_QUANTIZE_BITGROOM`, `NC_QUANTIZE_GRANULARBR`, `NC_QUANTIZE_BITROUND`.
383
410
 
411
+ Filter ids (for `nc_inq_filter_avail` and `nc_inq_var_filter_ids`): `H5Z_FILTER_DEFLATE`, `H5Z_FILTER_SHUFFLE`, `H5Z_FILTER_FLETCHER32`, `H5Z_FILTER_SZIP`, `H5Z_FILTER_ZSTD`, `H5Z_FILTER_BZIP2`, `H5Z_FILTER_BLOSC`.
412
+
413
+ Szip options: `NC_SZIP_NN`, `NC_SZIP_EC`.
414
+
415
+ Blosc subcompressors: `BLOSC_LZ`, `BLOSC_LZ4`, `BLOSC_LZ4HC`, `BLOSC_SNAPPY`, `BLOSC_ZLIB`, `BLOSC_ZSTD`. Blosc shuffle: `BLOSC_NOSHUFFLE`, `BLOSC_SHUFFLE`, `BLOSC_BITSHUFFLE`. (libnetcdf >= 4.9)
416
+
384
417
  Limits: `NC_MAX_NAME`, `NC_MAX_VAR_DIMS`.
385
418
 
386
419
  ## Buffers
@@ -45,6 +45,14 @@ have_func("nc_def_var_filter", filter_hdrs) # 4.6+
45
45
  have_func("nc_inq_var_filter_info", filter_hdrs) # 4.6+
46
46
  have_func("nc_inq_var_filter_ids", filter_hdrs) # 4.6+
47
47
  have_func("nc_inq_filter_avail", filter_hdrs) # 4.7+
48
+ have_func("nc_def_var_szip", "netcdf.h")
49
+ have_func("nc_inq_var_szip", "netcdf.h")
50
+ have_func("nc_def_var_zstandard", filter_hdrs) # 4.9+
51
+ have_func("nc_inq_var_zstandard", filter_hdrs) # 4.9+
52
+ have_func("nc_def_var_bzip2", filter_hdrs) # 4.9+
53
+ have_func("nc_inq_var_bzip2", filter_hdrs) # 4.9+
54
+ have_func("nc_def_var_blosc", filter_hdrs) # 4.9+
55
+ have_func("nc_inq_var_blosc", filter_hdrs) # 4.9+
48
56
 
49
57
  # In-memory file APIs. nc_open_mem arrived in 4.4; nc_create_mem and
50
58
  # nc_close_memio (with NC_memio) arrived in 4.6.2. netcdf_mem.h needs
@@ -15,7 +15,10 @@
15
15
  #if defined(HAVE_NC_DEF_VAR_FILTER) || \
16
16
  defined(HAVE_NC_INQ_VAR_FILTER_INFO) || \
17
17
  defined(HAVE_NC_INQ_VAR_FILTER_IDS) || \
18
- defined(HAVE_NC_INQ_FILTER_AVAIL)
18
+ defined(HAVE_NC_INQ_FILTER_AVAIL) || \
19
+ defined(HAVE_NC_DEF_VAR_ZSTANDARD) || \
20
+ defined(HAVE_NC_DEF_VAR_BZIP2) || \
21
+ defined(HAVE_NC_DEF_VAR_BLOSC)
19
22
  # include <netcdf_filter.h>
20
23
  #endif
21
24
  #ifdef HAVE_NETCDF_MEM_H
@@ -41,6 +44,7 @@ void nc_raise(int status) __attribute__((noreturn));
41
44
  void ary_to_sizet(VALUE ary, int ndims, const char *role, size_t *out);
42
45
  bool ary_to_ptrdiff(VALUE ary, int ndims, const char *role, ptrdiff_t *out);
43
46
  ssize_t nc_xtype_size(nc_type xt);
47
+ nc_type resolve_enum_base(int ncid, nc_type xt);
44
48
  void indices_to_c(int ncid, int varid, VALUE index_ary, size_t *out);
45
49
 
46
50
  /* Range-checked integer conversions (defined in netcdf.c); each raises
data/ext/netcdf/nc_att.c CHANGED
@@ -119,9 +119,39 @@ rb_nc_get_att_text (VALUE self, VALUE rncid, VALUE rvarid, VALUE rname)
119
119
  return out;
120
120
  }
121
121
 
122
+ /* libnetcdf refuses to read an enum attribute through a typed getter
123
+ (NC_EBADTYPE). Its values are stored as the enum's base type, so when
124
+ that base type is `memtype` read the raw elements with nc_get_att;
125
+ otherwise keep NC_EBADTYPE. Returns a status rather than raising so
126
+ that the caller can free its buffer first. */
127
+ static int
128
+ get_att_enum_as (int ncid, int varid, const char *name, nc_type memtype, void *buf)
129
+ {
130
+ nc_type xt, base;
131
+ int klass, st;
132
+
133
+ st = nc_inq_atttype(ncid, varid, name, &xt);
134
+ if ( st != NC_NOERR ) {
135
+ return st;
136
+ }
137
+ if ( xt < NC_FIRSTUSERTYPEID ) {
138
+ return NC_EBADTYPE;
139
+ }
140
+ st = nc_inq_user_type(ncid, xt, NULL, NULL, &base, NULL, &klass);
141
+ if ( st != NC_NOERR ) {
142
+ return st;
143
+ }
144
+ if ( klass != NC_ENUM || base != memtype ) {
145
+ return NC_EBADTYPE;
146
+ }
147
+ return nc_get_att(ncid, varid, name, buf);
148
+ }
149
+
122
150
  /* As in C, `xtype` is the attribute's type on disk and the suffix names
123
- the type of the values in memory; libnetcdf converts between them. */
124
- #define DEFINE_ATT_NUM(suffix, ctype, num2c, c2num) \
151
+ the type of the values in memory; libnetcdf converts between them.
152
+ The getter also reads an enum attribute whose base type is the
153
+ suffix's type (`memtype`). */
154
+ #define DEFINE_ATT_NUM(suffix, ctype, memtype, num2c, c2num) \
125
155
  static VALUE \
126
156
  rb_nc_put_att_##suffix (VALUE self, VALUE rncid, VALUE rvarid, \
127
157
  VALUE rname, VALUE rxtype, VALUE rvalue) \
@@ -161,6 +191,10 @@ rb_nc_get_att_text (VALUE self, VALUE rncid, VALUE rvarid, VALUE rname)
161
191
  buf = ALLOC_N(ctype, (len > 0 ? len : 1)); \
162
192
  st = nc_get_att_##suffix(NUM2INT(rncid), NUM2INT(rvarid), \
163
193
  StringValueCStr(rname), buf); \
194
+ if ( st == NC_EBADTYPE ) { \
195
+ st = get_att_enum_as(NUM2INT(rncid), NUM2INT(rvarid), \
196
+ StringValueCStr(rname), memtype, buf); \
197
+ } \
164
198
  if ( st != NC_NOERR ) { \
165
199
  xfree(buf); \
166
200
  nc_raise(st); \
@@ -173,16 +207,16 @@ rb_nc_get_att_text (VALUE self, VALUE rncid, VALUE rvarid, VALUE rname)
173
207
  return out; \
174
208
  }
175
209
 
176
- DEFINE_ATT_NUM(schar, signed char, num_to_schar, INT2NUM)
177
- DEFINE_ATT_NUM(uchar, unsigned char, num_to_uchar, UINT2NUM)
178
- DEFINE_ATT_NUM(short, short, num_to_short, INT2NUM)
179
- DEFINE_ATT_NUM(ushort, unsigned short, num_to_ushort, UINT2NUM)
180
- DEFINE_ATT_NUM(int, int, NUM2INT, INT2NUM)
181
- DEFINE_ATT_NUM(uint, unsigned int, num_to_uint, UINT2NUM)
182
- DEFINE_ATT_NUM(longlong, long long, NUM2LL, LL2NUM)
183
- DEFINE_ATT_NUM(ulonglong, unsigned long long, num_to_ulonglong, ULL2NUM)
184
- DEFINE_ATT_NUM(float, float, NUM2DBL, DBL2NUM)
185
- DEFINE_ATT_NUM(double, double, NUM2DBL, DBL2NUM)
210
+ DEFINE_ATT_NUM(schar, signed char, NC_BYTE, num_to_schar, INT2NUM)
211
+ DEFINE_ATT_NUM(uchar, unsigned char, NC_UBYTE, num_to_uchar, UINT2NUM)
212
+ DEFINE_ATT_NUM(short, short, NC_SHORT, num_to_short, INT2NUM)
213
+ DEFINE_ATT_NUM(ushort, unsigned short, NC_USHORT, num_to_ushort, UINT2NUM)
214
+ DEFINE_ATT_NUM(int, int, NC_INT, NUM2INT, INT2NUM)
215
+ DEFINE_ATT_NUM(uint, unsigned int, NC_UINT, num_to_uint, UINT2NUM)
216
+ DEFINE_ATT_NUM(longlong, long long, NC_INT64, NUM2LL, LL2NUM)
217
+ DEFINE_ATT_NUM(ulonglong, unsigned long long, NC_UINT64, num_to_ulonglong, ULL2NUM)
218
+ DEFINE_ATT_NUM(float, float, NC_FLOAT, NUM2DBL, DBL2NUM)
219
+ DEFINE_ATT_NUM(double, double, NC_DOUBLE, NUM2DBL, DBL2NUM)
186
220
 
187
221
  static VALUE
188
222
  rb_nc_put_att_string (VALUE self, VALUE rncid, VALUE rvarid, VALUE rname, VALUE rvalue)
@@ -369,6 +403,9 @@ rb_nc_get_att (VALUE self, VALUE rncid, VALUE rvarid, VALUE rname)
369
403
  return out;
370
404
  }
371
405
 
406
+ /* an enum attribute is read as its base type, which the typed getters
407
+ below accept */
408
+ xt = resolve_enum_base(NUM2INT(rncid), xt);
372
409
  switch ( xt ) {
373
410
  case NC_BYTE: out = rb_nc_get_att_schar(self, rncid, rvarid, rname); break;
374
411
  case NC_UBYTE: out = rb_nc_get_att_uchar(self, rncid, rvarid, rname); break;
data/ext/netcdf/nc_file.c CHANGED
@@ -230,6 +230,28 @@ rb_nc_inq_unlimdim (VALUE self, VALUE rncid)
230
230
  return ( d < 0 ) ? Qnil : INT2NUM(d);
231
231
  }
232
232
 
233
+ static VALUE
234
+ rb_nc_inq_unlimdims (VALUE self, VALUE rncid)
235
+ {
236
+ VALUE out;
237
+ int n, st, i;
238
+ int *ids;
239
+
240
+ NC_CHECK(nc_inq_unlimdims(NUM2INT(rncid), &n, NULL));
241
+ ids = ALLOC_N(int, n > 0 ? n : 1);
242
+ st = nc_inq_unlimdims(NUM2INT(rncid), NULL, ids);
243
+ if ( st != NC_NOERR ) {
244
+ xfree(ids);
245
+ nc_raise(st);
246
+ }
247
+ out = rb_ary_new_capa(n);
248
+ for (i=0; i<n; i++) {
249
+ rb_ary_push(out, INT2NUM(ids[i]));
250
+ }
251
+ xfree(ids);
252
+ return out;
253
+ }
254
+
233
255
  #ifdef HAVE_NC_INQ_PATH
234
256
  static VALUE
235
257
  rb_nc_inq_path (VALUE self, VALUE rncid)
@@ -290,6 +312,16 @@ rb_nc_set_fill (VALUE self, VALUE rncid, VALUE rmode)
290
312
  return INT2NUM(old);
291
313
  }
292
314
 
315
+ /* Process-global format used by nc_create when the mode names none. */
316
+ static VALUE
317
+ rb_nc_set_default_format (VALUE self, VALUE rformat)
318
+ {
319
+ int old;
320
+
321
+ NC_CHECK(nc_set_default_format(NUM2INT(rformat), &old));
322
+ return INT2NUM(old);
323
+ }
324
+
293
325
  /* Process-global HDF5 chunk cache parameters. */
294
326
  static VALUE
295
327
  rb_nc_set_chunk_cache (VALUE self, VALUE rsize, VALUE rnelems, VALUE rpreemption)
@@ -334,6 +366,7 @@ init_nc_file (void)
334
366
  rb_define_module_function(mNetCDF, "nc_inq_nvars", rb_nc_inq_nvars, 1);
335
367
  rb_define_module_function(mNetCDF, "nc_inq_natts", rb_nc_inq_natts, 1);
336
368
  rb_define_module_function(mNetCDF, "nc_inq_unlimdim", rb_nc_inq_unlimdim, 1);
369
+ rb_define_module_function(mNetCDF, "nc_inq_unlimdims", rb_nc_inq_unlimdims, 1);
337
370
  #ifdef HAVE_NC_INQ_PATH
338
371
  rb_define_module_function(mNetCDF, "nc_inq_path", rb_nc_inq_path, 1);
339
372
  #endif
@@ -344,6 +377,7 @@ init_nc_file (void)
344
377
  rb_define_module_function(mNetCDF, "nc_inq_libvers", rb_nc_inq_libvers, 0);
345
378
  rb_define_module_function(mNetCDF, "nc_inq_type", rb_nc_inq_type, 2);
346
379
  rb_define_module_function(mNetCDF, "nc_set_fill", rb_nc_set_fill, 2);
380
+ rb_define_module_function(mNetCDF, "nc_set_default_format", rb_nc_set_default_format, 1);
347
381
  rb_define_module_function(mNetCDF, "nc_set_chunk_cache", rb_nc_set_chunk_cache, 3);
348
382
  rb_define_module_function(mNetCDF, "nc_get_chunk_cache", rb_nc_get_chunk_cache, 0);
349
383
  }
data/ext/netcdf/nc_io.c CHANGED
@@ -1267,6 +1267,45 @@ rb_nc_get_var (int argc, VALUE *argv, VALUE self)
1267
1267
  return rb_nc_var_io(rncid, rvarid, rbuf, false);
1268
1268
  }
1269
1269
 
1270
+ /* ---------------------------------------------------------- */
1271
+ /* Whole-variable copy between files */
1272
+ /* ---------------------------------------------------------- */
1273
+
1274
+ typedef struct {
1275
+ int ncid_in, varid, ncid_out;
1276
+ int status;
1277
+ } copy_var_args_t;
1278
+
1279
+ static void *
1280
+ nogvl_copy_var (void *p)
1281
+ {
1282
+ copy_var_args_t *a;
1283
+
1284
+ a = (copy_var_args_t *) p;
1285
+ a->status = nc_copy_var(a->ncid_in, a->varid, a->ncid_out);
1286
+ return NULL;
1287
+ }
1288
+
1289
+ /* Copies the definition, the attributes and the data. Nothing is held
1290
+ across the call, so an interrupt that kept it from running is raised,
1291
+ or the copy tried again, with nothing to clean up first. */
1292
+ static VALUE
1293
+ rb_nc_copy_var (VALUE self, VALUE rncid_in, VALUE rvarid, VALUE rncid_out)
1294
+ {
1295
+ copy_var_args_t a;
1296
+
1297
+ a.ncid_in = NUM2INT(rncid_in);
1298
+ a.varid = NUM2INT(rvarid);
1299
+ a.ncid_out = NUM2INT(rncid_out);
1300
+ a.status = NC_NOERR;
1301
+ while ( ! call_without_gvl(nogvl_copy_var, &a) ) {
1302
+ rb_thread_check_ints();
1303
+ }
1304
+ rb_thread_check_ints();
1305
+ NC_CHECK(a.status);
1306
+ return Qnil;
1307
+ }
1308
+
1270
1309
  void
1271
1310
  init_nc_io (void)
1272
1311
  {
@@ -1281,4 +1320,6 @@ init_nc_io (void)
1281
1320
  rb_define_module_function(mNetCDF, "nc_get_vars", rb_nc_get_vars, -1);
1282
1321
  rb_define_module_function(mNetCDF, "nc_put_varm", rb_nc_put_varm, 7);
1283
1322
  rb_define_module_function(mNetCDF, "nc_get_varm", rb_nc_get_varm, -1);
1323
+
1324
+ rb_define_module_function(mNetCDF, "nc_copy_var", rb_nc_copy_var, 3);
1284
1325
  }
@@ -67,6 +67,17 @@ rb_nc_inq_typeids (VALUE self, VALUE rncid)
67
67
  return out;
68
68
  }
69
69
 
70
+ /* The two types may live in different files or groups. */
71
+ static VALUE
72
+ rb_nc_inq_type_equal (VALUE self, VALUE rncid1, VALUE rtypeid1, VALUE rncid2, VALUE rtypeid2)
73
+ {
74
+ int equal;
75
+
76
+ NC_CHECK(nc_inq_type_equal(NUM2INT(rncid1), NUM2INT(rtypeid1),
77
+ NUM2INT(rncid2), NUM2INT(rtypeid2), &equal));
78
+ return equal ? Qtrue : Qfalse;
79
+ }
80
+
70
81
  /* ---------------------------------------------------------- */
71
82
  /* Opaque (fixed-size byte type) */
72
83
  /* ---------------------------------------------------------- */
@@ -263,6 +274,7 @@ init_nc_types (void)
263
274
  rb_define_module_function(mNetCDF, "nc_inq_user_type", rb_nc_inq_user_type, 2);
264
275
  rb_define_module_function(mNetCDF, "nc_inq_typeid", rb_nc_inq_typeid, 2);
265
276
  rb_define_module_function(mNetCDF, "nc_inq_typeids", rb_nc_inq_typeids, 1);
277
+ rb_define_module_function(mNetCDF, "nc_inq_type_equal", rb_nc_inq_type_equal, 4);
266
278
 
267
279
  rb_define_module_function(mNetCDF, "nc_def_opaque", rb_nc_def_opaque, 3);
268
280
  rb_define_module_function(mNetCDF, "nc_inq_opaque", rb_nc_inq_opaque, 2);
data/ext/netcdf/nc_var.c CHANGED
@@ -150,6 +150,8 @@ rb_nc_def_var_fill (VALUE self, VALUE rncid, VALUE rvarid, VALUE rno_fill, VALUE
150
150
  no_fill = RTEST(rno_fill) ? 1 : 0;
151
151
 
152
152
  NC_CHECK(nc_inq_vartype(ncid, varid, &xt));
153
+ /* an enum variable's fill value is an element of its base type */
154
+ xt = resolve_enum_base(ncid, xt);
153
155
 
154
156
  if ( NIL_P(rfill) ) {
155
157
  NC_CHECK(nc_def_var_fill(ncid, varid, no_fill, NULL));
@@ -201,6 +203,7 @@ rb_nc_inq_var_fill (VALUE self, VALUE rncid, VALUE rvarid)
201
203
  varid = NUM2INT(rvarid);
202
204
 
203
205
  NC_CHECK(nc_inq_vartype(ncid, varid, &xt));
206
+ xt = resolve_enum_base(ncid, xt);
204
207
  /* libnetcdf writes a whole element of the variable's type into `v`,
205
208
  so refuse any type the union below cannot hold before asking --
206
209
  an opaque's element can be any size. */
@@ -451,6 +454,137 @@ rb_nc_inq_filter_avail (VALUE self, VALUE rncid, VALUE rid)
451
454
  }
452
455
  #endif
453
456
 
457
+ /* ---------------------------------------------------------- */
458
+ /* Compression filters with their own calls */
459
+ /* ---------------------------------------------------------- */
460
+
461
+ /* Whether the filter can be used depends on how libnetcdf was built,
462
+ but the calls are there either way. A libnetcdf built without blosc
463
+ returns NC_NOERR from both blosc calls and sets nothing, so every inq
464
+ here starts its outputs at 0: a variable without the filter then
465
+ reads back as not having it.
466
+
467
+ Each def call here first asks whether the filter is there, and
468
+ raises NC_ENOFILTER when it is not: the blosc def would otherwise
469
+ return without adding the filter, and the data would be written
470
+ uncompressed. The check needs nc_inq_filter_avail (4.7+); the szip
471
+ def in an older libnetcdf goes without it. */
472
+
473
+ #ifdef HAVE_NC_INQ_FILTER_AVAIL
474
+ static void
475
+ check_filter_available (int ncid, unsigned id)
476
+ {
477
+ NC_CHECK(nc_inq_filter_avail(ncid, id));
478
+ }
479
+ #endif
480
+
481
+ #ifdef HAVE_NC_DEF_VAR_SZIP
482
+ static VALUE
483
+ rb_nc_def_var_szip (VALUE self, VALUE rncid, VALUE rvarid,
484
+ VALUE roptions_mask, VALUE rpixels_per_block)
485
+ {
486
+ #ifdef HAVE_NC_INQ_FILTER_AVAIL
487
+ check_filter_available(NUM2INT(rncid), H5Z_FILTER_SZIP);
488
+ #endif
489
+ NC_CHECK(nc_def_var_szip(NUM2INT(rncid), NUM2INT(rvarid),
490
+ NUM2INT(roptions_mask), NUM2INT(rpixels_per_block)));
491
+ return Qnil;
492
+ }
493
+ #endif
494
+
495
+ #ifdef HAVE_NC_INQ_VAR_SZIP
496
+ static VALUE
497
+ rb_nc_inq_var_szip (VALUE self, VALUE rncid, VALUE rvarid)
498
+ {
499
+ int options_mask, pixels_per_block;
500
+
501
+ options_mask = 0;
502
+ pixels_per_block = 0;
503
+ NC_CHECK(nc_inq_var_szip(NUM2INT(rncid), NUM2INT(rvarid),
504
+ &options_mask, &pixels_per_block));
505
+ return rb_ary_new_from_args(2, INT2NUM(options_mask), INT2NUM(pixels_per_block));
506
+ }
507
+ #endif
508
+
509
+ #ifdef HAVE_NC_DEF_VAR_ZSTANDARD
510
+ static VALUE
511
+ rb_nc_def_var_zstandard (VALUE self, VALUE rncid, VALUE rvarid, VALUE rlevel)
512
+ {
513
+ check_filter_available(NUM2INT(rncid), H5Z_FILTER_ZSTD);
514
+ NC_CHECK(nc_def_var_zstandard(NUM2INT(rncid), NUM2INT(rvarid), NUM2INT(rlevel)));
515
+ return Qnil;
516
+ }
517
+ #endif
518
+
519
+ #ifdef HAVE_NC_INQ_VAR_ZSTANDARD
520
+ static VALUE
521
+ rb_nc_inq_var_zstandard (VALUE self, VALUE rncid, VALUE rvarid)
522
+ {
523
+ int has, level;
524
+
525
+ has = 0;
526
+ level = 0;
527
+ NC_CHECK(nc_inq_var_zstandard(NUM2INT(rncid), NUM2INT(rvarid), &has, &level));
528
+ return rb_ary_new_from_args(2, has ? Qtrue : Qfalse, INT2NUM(level));
529
+ }
530
+ #endif
531
+
532
+ #ifdef HAVE_NC_DEF_VAR_BZIP2
533
+ static VALUE
534
+ rb_nc_def_var_bzip2 (VALUE self, VALUE rncid, VALUE rvarid, VALUE rlevel)
535
+ {
536
+ check_filter_available(NUM2INT(rncid), H5Z_FILTER_BZIP2);
537
+ NC_CHECK(nc_def_var_bzip2(NUM2INT(rncid), NUM2INT(rvarid), NUM2INT(rlevel)));
538
+ return Qnil;
539
+ }
540
+ #endif
541
+
542
+ #ifdef HAVE_NC_INQ_VAR_BZIP2
543
+ static VALUE
544
+ rb_nc_inq_var_bzip2 (VALUE self, VALUE rncid, VALUE rvarid)
545
+ {
546
+ int has, level;
547
+
548
+ has = 0;
549
+ level = 0;
550
+ NC_CHECK(nc_inq_var_bzip2(NUM2INT(rncid), NUM2INT(rvarid), &has, &level));
551
+ return rb_ary_new_from_args(2, has ? Qtrue : Qfalse, INT2NUM(level));
552
+ }
553
+ #endif
554
+
555
+ #ifdef HAVE_NC_DEF_VAR_BLOSC
556
+ static VALUE
557
+ rb_nc_def_var_blosc (VALUE self, VALUE rncid, VALUE rvarid, VALUE rsubcompressor,
558
+ VALUE rlevel, VALUE rblocksize, VALUE raddshuffle)
559
+ {
560
+ check_filter_available(NUM2INT(rncid), H5Z_FILTER_BLOSC);
561
+ NC_CHECK(nc_def_var_blosc(NUM2INT(rncid), NUM2INT(rvarid),
562
+ NUM2UINT(rsubcompressor), NUM2UINT(rlevel),
563
+ NUM2UINT(rblocksize), NUM2UINT(raddshuffle)));
564
+ return Qnil;
565
+ }
566
+ #endif
567
+
568
+ #ifdef HAVE_NC_INQ_VAR_BLOSC
569
+ static VALUE
570
+ rb_nc_inq_var_blosc (VALUE self, VALUE rncid, VALUE rvarid)
571
+ {
572
+ int has;
573
+ unsigned subcompressor, level, blocksize, addshuffle;
574
+
575
+ has = 0;
576
+ subcompressor = 0;
577
+ level = 0;
578
+ blocksize = 0;
579
+ addshuffle = 0;
580
+ NC_CHECK(nc_inq_var_blosc(NUM2INT(rncid), NUM2INT(rvarid), &has,
581
+ &subcompressor, &level, &blocksize, &addshuffle));
582
+ return rb_ary_new_from_args(5, has ? Qtrue : Qfalse,
583
+ UINT2NUM(subcompressor), UINT2NUM(level),
584
+ UINT2NUM(blocksize), UINT2NUM(addshuffle));
585
+ }
586
+ #endif
587
+
454
588
  /* ---------------------------------------------------------- */
455
589
  /* Quantize (lossy bit-grooming for floating-point data) */
456
590
  /* ---------------------------------------------------------- */
@@ -539,6 +673,31 @@ init_nc_var (void)
539
673
  rb_define_module_function(mNetCDF, "nc_inq_filter_avail", rb_nc_inq_filter_avail, 2);
540
674
  #endif
541
675
 
676
+ #ifdef HAVE_NC_DEF_VAR_SZIP
677
+ rb_define_module_function(mNetCDF, "nc_def_var_szip", rb_nc_def_var_szip, 4);
678
+ #endif
679
+ #ifdef HAVE_NC_INQ_VAR_SZIP
680
+ rb_define_module_function(mNetCDF, "nc_inq_var_szip", rb_nc_inq_var_szip, 2);
681
+ #endif
682
+ #ifdef HAVE_NC_DEF_VAR_ZSTANDARD
683
+ rb_define_module_function(mNetCDF, "nc_def_var_zstandard", rb_nc_def_var_zstandard, 3);
684
+ #endif
685
+ #ifdef HAVE_NC_INQ_VAR_ZSTANDARD
686
+ rb_define_module_function(mNetCDF, "nc_inq_var_zstandard", rb_nc_inq_var_zstandard, 2);
687
+ #endif
688
+ #ifdef HAVE_NC_DEF_VAR_BZIP2
689
+ rb_define_module_function(mNetCDF, "nc_def_var_bzip2", rb_nc_def_var_bzip2, 3);
690
+ #endif
691
+ #ifdef HAVE_NC_INQ_VAR_BZIP2
692
+ rb_define_module_function(mNetCDF, "nc_inq_var_bzip2", rb_nc_inq_var_bzip2, 2);
693
+ #endif
694
+ #ifdef HAVE_NC_DEF_VAR_BLOSC
695
+ rb_define_module_function(mNetCDF, "nc_def_var_blosc", rb_nc_def_var_blosc, 6);
696
+ #endif
697
+ #ifdef HAVE_NC_INQ_VAR_BLOSC
698
+ rb_define_module_function(mNetCDF, "nc_inq_var_blosc", rb_nc_inq_var_blosc, 2);
699
+ #endif
700
+
542
701
  #ifdef HAVE_NC_DEF_VAR_QUANTIZE
543
702
  rb_define_module_function(mNetCDF, "nc_def_var_quantize", rb_nc_def_var_quantize, 4);
544
703
  #endif
data/ext/netcdf/netcdf.c CHANGED
@@ -73,6 +73,25 @@ nc_xtype_size (nc_type xt)
73
73
  }
74
74
  }
75
75
 
76
+ /* The base type of the enum type `xt`, or `xt` itself for any other type.
77
+ An enum value is stored as its base type, so code that reads or writes
78
+ one element can treat it as that integer type. */
79
+ nc_type
80
+ resolve_enum_base (int ncid, nc_type xt)
81
+ {
82
+ nc_type base;
83
+ int klass;
84
+
85
+ if ( xt < NC_FIRSTUSERTYPEID ) {
86
+ return xt;
87
+ }
88
+ NC_CHECK(nc_inq_user_type(ncid, xt, NULL, NULL, &base, NULL, &klass));
89
+ if ( klass != NC_ENUM ) {
90
+ return xt;
91
+ }
92
+ return base;
93
+ }
94
+
76
95
  /* Integer conversions that refuse a value the C type cannot hold. A cast
77
96
  would store 300 as NC_BYTE 44, and Ruby's own NUM2UINT / NUM2ULL wrap a
78
97
  negative value around rather than raise, so -1 would be stored as the
@@ -173,6 +192,12 @@ Init_netcdf (void)
173
192
  DEF_CONST(NC_NETCDF4);
174
193
  DEF_CONST(NC_CLASSIC_MODEL);
175
194
  DEF_CONST(NC_64BIT_OFFSET);
195
+ #ifdef NC_64BIT_DATA
196
+ DEF_CONST(NC_64BIT_DATA); /* CDF-5; libnetcdf 4.4+ */
197
+ #endif
198
+ #ifdef NC_CDF5
199
+ DEF_CONST(NC_CDF5); /* alias of NC_64BIT_DATA */
200
+ #endif
176
201
  #ifdef NC_DISKLESS
177
202
  DEF_CONST(NC_DISKLESS);
178
203
  #endif
@@ -279,6 +304,51 @@ Init_netcdf (void)
279
304
  DEF_CONST(NC_QUANTIZE_BITROUND);
280
305
  #endif
281
306
 
307
+ /* Filter ids, for nc_inq_filter_avail and nc_inq_var_filter_ids */
308
+ #ifdef H5Z_FILTER_DEFLATE
309
+ DEF_CONST(H5Z_FILTER_DEFLATE);
310
+ #endif
311
+ #ifdef H5Z_FILTER_SHUFFLE
312
+ DEF_CONST(H5Z_FILTER_SHUFFLE);
313
+ #endif
314
+ #ifdef H5Z_FILTER_FLETCHER32
315
+ DEF_CONST(H5Z_FILTER_FLETCHER32);
316
+ #endif
317
+ #ifdef H5Z_FILTER_SZIP
318
+ DEF_CONST(H5Z_FILTER_SZIP);
319
+ #endif
320
+ #ifdef H5Z_FILTER_ZSTD
321
+ DEF_CONST(H5Z_FILTER_ZSTD);
322
+ #endif
323
+ #ifdef H5Z_FILTER_BZIP2
324
+ DEF_CONST(H5Z_FILTER_BZIP2);
325
+ #endif
326
+ #ifdef H5Z_FILTER_BLOSC
327
+ DEF_CONST(H5Z_FILTER_BLOSC);
328
+ #endif
329
+
330
+ /* szip options_mask */
331
+ #ifdef NC_SZIP_EC
332
+ DEF_CONST(NC_SZIP_EC);
333
+ #endif
334
+ #ifdef NC_SZIP_NN
335
+ DEF_CONST(NC_SZIP_NN);
336
+ #endif
337
+
338
+ /* Blosc subcompressors and shuffle modes. These are enum members,
339
+ not macros, so they are gated on the call that takes them. */
340
+ #ifdef HAVE_NC_DEF_VAR_BLOSC
341
+ DEF_CONST(BLOSC_LZ);
342
+ DEF_CONST(BLOSC_LZ4);
343
+ DEF_CONST(BLOSC_LZ4HC);
344
+ DEF_CONST(BLOSC_SNAPPY);
345
+ DEF_CONST(BLOSC_ZLIB);
346
+ DEF_CONST(BLOSC_ZSTD);
347
+ DEF_CONST(BLOSC_NOSHUFFLE);
348
+ DEF_CONST(BLOSC_SHUFFLE);
349
+ DEF_CONST(BLOSC_BITSHUFFLE);
350
+ #endif
351
+
282
352
  /* User-defined type classes (returned by nc_inq_user_type) */
283
353
  DEF_CONST(NC_VLEN);
284
354
  DEF_CONST(NC_OPAQUE);
@@ -1,3 +1,3 @@
1
1
  module NetCDF
2
- VERSION = "0.1.0"
2
+ VERSION = "0.2.0"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: netcdf
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - himotoyoshi