carray-netcdf 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +73 -2
- data/CONTRIBUTING.md +3 -3
- data/README.md +53 -21
- data/bin/ncheader +43 -8
- data/bin/ncvar +48 -12
- data/carray-netcdf.gemspec +4 -4
- data/docs/00_introduction.md +19 -30
- data/docs/01_getting_started.md +16 -25
- data/docs/02_reading.md +26 -26
- data/docs/03_writing.md +53 -44
- data/docs/04_codec.md +74 -82
- data/docs/05_groups.md +5 -5
- data/docs/06_low_level_api.md +161 -0
- data/docs/07_cookbook.md +167 -126
- data/docs/08_cheatsheet.md +77 -52
- data/docs/09_cli.md +4 -4
- data/lib/carray/netcdf/codec.rb +223 -29
- data/lib/carray/netcdf/dim.rb +8 -5
- data/lib/carray/netcdf/file.rb +184 -57
- data/lib/carray/netcdf/group.rb +301 -43
- data/lib/carray/netcdf/var.rb +311 -31
- data/lib/carray/netcdf/version.rb +1 -1
- data/lib/carray/netcdf.rb +99 -17
- metadata +7 -7
- data/docs/06_layer1.md +0 -160
data/docs/08_cheatsheet.md
CHANGED
|
@@ -4,38 +4,36 @@ Tables of everything this gem gives you to call. Read [getting started](01_getti
|
|
|
4
4
|
|
|
5
5
|
Conventions:
|
|
6
6
|
|
|
7
|
-
- **
|
|
8
|
-
- **
|
|
9
|
-
- **
|
|
7
|
+
- **The netcdf gem** = [netcdf](https://github.com/himotoyoshi/netcdf) (`NetCDF`), a near one-to-one binding to libnetcdf.
|
|
8
|
+
- **Low-level API** = `CA::NetCDF`; the netcdf gem's API with the variable-data get / put calls overridden to speak CArray. Everything else passes through because `CA::NetCDF` `include`s the netcdf gem.
|
|
9
|
+
- **High-level API** = the `CA::NetCDF::File`, `Group`, `Var` and `Dim` classes, and the codecs in `CA::NetCDF::Codec`.
|
|
10
10
|
|
|
11
11
|
In every table, `→` is the return value.
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
-
##
|
|
15
|
+
## The netcdf gem
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
This is the [netcdf](https://github.com/himotoyoshi/netcdf) gem itself: `NetCDF`, one module function per libnetcdf function, plus the `NC_*` constants. Its own reference lists all of them, and is the place to look them up.
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
## Low-level API — `CA::NetCDF`
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
The whole `::NetCDF` module is `include`d into `CA::NetCDF`, so everything from L0 above is callable — as `CA::NetCDF.nc_open(…)` or via mixin (`include CA::NetCDF` and call `nc_open(…)` bare). Only the array I/O calls are overridden to speak CArray.
|
|
21
|
+
The whole `NetCDF` module is `include`d into `CA::NetCDF`, so everything from the netcdf gem above is callable via mixin (`include CA::NetCDF` and call `nc_open(…)` bare). Called on the module, only the overridden functions below are public (`CA::NetCDF.nc_get_vara(…)`); call the rest on `NetCDF` (`NetCDF.nc_open(…)`).
|
|
24
22
|
|
|
25
23
|
### Overridden functions
|
|
26
24
|
|
|
27
25
|
| Call | Behavior |
|
|
28
26
|
|---|---|
|
|
29
|
-
| `nc_get_var(ncid, varid, buffer=nil)` | Without buffer:
|
|
27
|
+
| `nc_get_var(ncid, varid, buffer=nil)` | Without buffer: a new CArray of the variable's type (`NC_CHAR` → `:uint8`, an enum → its base type). With a buffer (a CArray, or any MemoryView provider): reads into it and returns it. `NC_STRING` raises `NotImplementedError`; read it with `NetCDF.nc_get_var`. |
|
|
30
28
|
| `nc_get_vara(ncid, varid, start, count, buffer=nil)` | Same, hyperslab. |
|
|
31
29
|
| `nc_get_vars(ncid, varid, start, count, stride, buffer=nil)` | Same, strided. |
|
|
32
30
|
| `nc_get_varm(ncid, varid, start, count, stride, imap, buffer=nil)` | Same, mapped. |
|
|
33
|
-
| `nc_put_var(ncid, varid, ca)` |
|
|
31
|
+
| `nc_put_var(ncid, varid, ca)` | A contiguous CArray is written as it is; a view is copied into one block first. Masked → `ArgumentError` (mask/fill belongs to the high-level API). |
|
|
34
32
|
| `nc_put_vara(ncid, varid, start, count, ca)` | Same. |
|
|
35
33
|
| `nc_put_vars(ncid, varid, start, count, stride, ca)` | Same. |
|
|
36
34
|
| `nc_put_varm(ncid, varid, start, count, stride, imap, ca)` | Same. |
|
|
37
35
|
|
|
38
|
-
Everything else on
|
|
36
|
+
Everything else on `NetCDF` (see [the netcdf gem](#the-netcdf-gem) above) is a passthrough. All the `NC_*` constants are visible via the include:
|
|
39
37
|
|
|
40
38
|
```ruby
|
|
41
39
|
CA::NetCDF::NC_DOUBLE # => 6
|
|
@@ -52,13 +50,13 @@ CA::NetCDF.nc_get_vara(ncid, varid, [0, 0], [3, 4]) # => CArray
|
|
|
52
50
|
class Reader
|
|
53
51
|
include CA::NetCDF
|
|
54
52
|
def read(ncid, varid) = nc_get_var(ncid, varid) # bare -> CArray
|
|
55
|
-
def define(ncid,
|
|
53
|
+
def define(ncid, name) = nc_def_dim(ncid, name, 100) # passthrough
|
|
56
54
|
end
|
|
57
55
|
```
|
|
58
56
|
|
|
59
57
|
---
|
|
60
58
|
|
|
61
|
-
##
|
|
59
|
+
## High-level API — `File`, `Group`, `Var`, `Dim` and `Codec`
|
|
62
60
|
|
|
63
61
|
### `CA::NetCDF::File` (class methods)
|
|
64
62
|
|
|
@@ -66,51 +64,53 @@ end
|
|
|
66
64
|
|---|---|
|
|
67
65
|
| `File.open(path, mode = :read, codec: :auto)` | File; `mode` is `:read` / `:write` / `:append` |
|
|
68
66
|
| `File.open(path, ...) { \|nc\| ... }` | value of block; auto-closes on exit / exception |
|
|
69
|
-
| `File.create(path, codec: :auto, clobber: true, netcdf4:
|
|
70
|
-
| `File.create(path, ...) { \|nc\| ... }` | value of block; auto-closes.
|
|
67
|
+
| `File.create(path, codec: :auto, clobber: true, netcdf4: nil, format: nil, atomic: false, template: nil, dims: true, atts: true, vars: [], **var_defaults)` | File in define mode. See details below. |
|
|
68
|
+
| `File.create(path, ...) { \|nc\| ... }` | value of block; auto-closes. Under `atomic: true`: commits on successful block exit, aborts (unlinks tmp) on exception. Under `clobber: :skip` with existing target: block is not called; returns `nil`. |
|
|
71
69
|
|
|
72
70
|
Kwargs on `File.create`:
|
|
73
71
|
|
|
74
72
|
| Kwarg | Meaning |
|
|
75
73
|
|---|---|
|
|
76
|
-
| `codec:` | codec dispatch (`:auto` / `:identity` / `:cf` / `[:cf, opts]` / ...) — see [
|
|
74
|
+
| `codec:` | codec dispatch (`:auto` / `:identity` / `:cf` / `[:cf, opts]` / ...) — see the [Codecs chapter](04_codec.md) |
|
|
77
75
|
| `clobber: true` (default) | overwrite an existing target file |
|
|
78
76
|
| `clobber: false` | raise `Errno::EEXIST` if target exists |
|
|
79
77
|
| `clobber: :skip` | if target exists, don't call the block, return `nil` (restartable loops) |
|
|
80
|
-
| `netcdf4: true` | create as NetCDF-4 (
|
|
78
|
+
| `netcdf4: true / false` | create as NetCDF-4 / classic; left out (`nil`), it follows `template:`'s format |
|
|
79
|
+
| `format: :classic / :offset64 / :cdf5 / :netcdf4 / :netcdf4_classic` | create in that format (instead of `netcdf4:`); a template's format is kept the same way |
|
|
81
80
|
| `atomic: true` | write to same-dir tmp; rename on successful close; unlink tmp on exception |
|
|
82
81
|
| `template: file_or_group` | inherit skeleton (dims + atts + format) from another open File/Group |
|
|
83
82
|
| `dims: true / false / [names]` | which template dims to copy (default all) |
|
|
84
83
|
| `atts: true / false / {overrides}` | which template atts to copy; Hash merges overrides |
|
|
85
|
-
| `vars: [names]` | template vars to
|
|
86
|
-
| `chunks: / deflate: / shuffle: / fletcher32: / endian: / no_fill:` | file-wide var defaults, applied to every `def_var` (per-var kwarg wins) |
|
|
84
|
+
| `vars: [names]` | template vars to copy in full (definition, storage settings and stored values) |
|
|
85
|
+
| `chunks: / deflate: / shuffle: / zstandard: / bzip2: / szip: / blosc: / fletcher32: / endian: / no_fill:` | file-wide var defaults, applied to every `def_var` (per-var kwarg wins); values as in [Storage options](#storage-options) below |
|
|
87
86
|
|
|
88
|
-
`path` may be a Zarr URL (`"file:///data/t.zarr#mode=zarr,file"`) for `open` and `create` alike; `atomic:` and `clobber: false / :skip` then raise `ArgumentError`.
|
|
87
|
+
`path` may be a Zarr URL (`"file:///data/t.zarr#mode=zarr,file"`) for `open` and `create` alike; `atomic:` and `clobber: false / :skip` then raise `ArgumentError`. See [Zarr stores](03_writing.md#zarr-stores).
|
|
89
88
|
|
|
90
89
|
### `File` instance (adds these to what `Group` provides)
|
|
91
90
|
|
|
92
91
|
| Call | → |
|
|
93
92
|
|---|---|
|
|
94
93
|
| `#filename` | on-disk path |
|
|
95
|
-
| `#format` | `
|
|
94
|
+
| `#format` | the `NC_FORMAT_*` Integer |
|
|
96
95
|
| `#conventions` | tokenized `Conventions` attribute (`["CF-1.7", "ACDD-1.3"]`); `[]` if absent |
|
|
97
|
-
| `#codec` |
|
|
98
|
-
| `#
|
|
99
|
-
| `#
|
|
96
|
+
| `#codec` | the codec object (`Codec::Identity`, or a `Codec::CF` / `Codec::COARDS` instance) |
|
|
97
|
+
| `#netcdf4?` | true for a NetCDF-4 or NetCDF-4 classic model file |
|
|
98
|
+
| `#close` | close the file (idempotent). For an atomic-write File, renames tmp → filename here. |
|
|
99
|
+
| `#abort` | roll back definitions not yet committed and close (`nc_abort`); a file created here still in define mode is removed, and an atomic write's tmp unlinked. Idempotent. |
|
|
100
100
|
| `#closed?` | Boolean |
|
|
101
101
|
| `#atomic?` | Boolean — is this an atomic-write File that has not yet committed? |
|
|
102
102
|
| `#tmp_path` | the tmp path an atomic write is going to (nil for regular / after commit) |
|
|
103
103
|
| `#var_defaults` | file-wide var option defaults (frozen Hash) |
|
|
104
|
-
| `#
|
|
105
|
-
| `#
|
|
104
|
+
| `#reload` | read dims / vars / groups / attributes again after low-level changes, and choose the codec again; returns `self` (also on `Group`) |
|
|
105
|
+
| `#ensure_define_mode` / `#ensure_data_mode` | switch modes, when you mix low-level calls into a `File` (also on `Group`) |
|
|
106
106
|
|
|
107
|
-
### `Group` (
|
|
107
|
+
### `Group` (a `File` is a `Group`, the root one)
|
|
108
108
|
|
|
109
109
|
**Identity**
|
|
110
110
|
|
|
111
111
|
| Call | → |
|
|
112
112
|
|---|---|
|
|
113
|
-
| `#ncid` |
|
|
113
|
+
| `#ncid` | the group's id, for the low-level API |
|
|
114
114
|
| `#file` | root `File` |
|
|
115
115
|
| `#parent` | parent `Group`, `nil` for root |
|
|
116
116
|
| `#name` | `"foo"` (or `"/"` for root) |
|
|
@@ -121,7 +121,7 @@ Kwargs on `File.create`:
|
|
|
121
121
|
|
|
122
122
|
| Call | → |
|
|
123
123
|
|---|---|
|
|
124
|
-
| `#dims` | `{String => Dim}` |
|
|
124
|
+
| `#dims` | `{String => Dim}` (a frozen copy, as are `vars` and `groups`) |
|
|
125
125
|
| `#vars` | `{String => Var}` |
|
|
126
126
|
| `#groups` | `{String => Group}` |
|
|
127
127
|
| `#atts` | `{String => value}` (frozen), this group's globals |
|
|
@@ -134,7 +134,7 @@ Kwargs on `File.create`:
|
|
|
134
134
|
|
|
135
135
|
| Call | → |
|
|
136
136
|
|---|---|
|
|
137
|
-
| `#[](name)` | Var — accepts `"temp"`, `"foo/temp"`, `"/foo/temp"` |
|
|
137
|
+
| `#[](name)` | Var — accepts `"temp"`, `"foo/temp"`, `"/foo/temp"`; raises `KeyError` if there is none |
|
|
138
138
|
| `#[]=(name, ca)` | whole-variable write |
|
|
139
139
|
| `#[]=(name, *index, ca)` | hyperslab write (delegates to `Var#[]=`) |
|
|
140
140
|
|
|
@@ -142,13 +142,14 @@ Kwargs on `File.create`:
|
|
|
142
142
|
|
|
143
143
|
| Call | → |
|
|
144
144
|
|---|---|
|
|
145
|
-
| `#def_dim(name, length)` | Dim; `length
|
|
146
|
-
| `#def_var(name, type, dims=
|
|
147
|
-
| `#def_var(name, template: src_var, **overrides)` | copy a definition (type, dims, attributes) from another Var;
|
|
148
|
-
| `#
|
|
145
|
+
| `#def_dim(name, length)` | Dim; `length` is `:unlimited` or an Integer |
|
|
146
|
+
| `#def_var(name, type = nil, dims = nil, template: nil, **atts)` | Var; `type` is a Symbol or String (`:float`, `"short"`, …) or an `NC_*` constant; `type:` and `dims:` are accepted as keywords too. Storage options ([below](#storage-options)) are reserved kwargs; every other kwarg is a variable attribute. |
|
|
147
|
+
| `#def_var(name, template: src_var, **overrides)` | copy a definition (type, dims, attributes) from another Var; overrides merge into / replace its attributes, and `type` / `dims` can be given too. Between two NetCDF-4 files the storage settings come along; the chunk sizes only when the new dimensions have the template's lengths. |
|
|
148
|
+
| `#def_var(name, template: group)` | the same, with the variable of the same name in that File / Group |
|
|
149
|
+
| `#def_att(name, value)` | value; writes to this group's attributes (a CArray value keeps its own type). On a File, redefining `Conventions` chooses the codec again. |
|
|
149
150
|
| `#def_group(name)` | Group; creates a subgroup |
|
|
150
151
|
| `#copy_dim(dim_or_name, from: nil, name: nil)` | copy a dimension (unlimited status preserved); optionally rename |
|
|
151
|
-
| `#copy_var(var_or_name, from: nil, name: nil, **overrides)` | copy a variable's definition
|
|
152
|
+
| `#copy_var(var_or_name, from: nil, name: nil, **overrides)` | copy a variable's definition and its stored values unchanged; the dimensions must already exist. Optionally rename / override attributes and storage options. |
|
|
152
153
|
|
|
153
154
|
### `Var`
|
|
154
155
|
|
|
@@ -157,15 +158,19 @@ Kwargs on `File.create`:
|
|
|
157
158
|
| Call | → |
|
|
158
159
|
|---|---|
|
|
159
160
|
| `#name` | variable name |
|
|
160
|
-
| `#shape` |
|
|
161
|
+
| `#shape` | current shape, asked of the file each time, so the growth of an unlimited dimension shows |
|
|
161
162
|
| `#ndim` | rank |
|
|
162
|
-
| `#nc_type` |
|
|
163
|
+
| `#nc_type` | the `NC_*` type constant |
|
|
164
|
+
| `#varid` | the variable's id, for the low-level API |
|
|
165
|
+
| `#scalar?` | true for a variable with no dimensions |
|
|
163
166
|
| `#data_type` | CArray Symbol (`:float32`, `:int16`, …) |
|
|
164
167
|
| `#dims` | Array of Dim (with dim inheritance resolved) |
|
|
165
168
|
| `#atts` | `{String => value}` (frozen) |
|
|
166
|
-
| `#att(name)` | one attribute or `nil` |
|
|
169
|
+
| `#att(name)` | one attribute (frozen) or `nil` |
|
|
170
|
+
| `#att_type(name)` | the `NC_*` type the attribute is stored in, or `nil` |
|
|
167
171
|
| `#file` / `#group` | enclosing File / Group |
|
|
168
172
|
| `#codec` | delegates to `file.codec` |
|
|
173
|
+
| `#storage_options` | the variable's chunking / compression / fletcher32 / endian / no_fill settings, in `def_var`'s keyword form (`{}` in a classic file) |
|
|
169
174
|
|
|
170
175
|
**Read (`[]` — lazy hyperslab + decode)**
|
|
171
176
|
|
|
@@ -179,7 +184,7 @@ Kwargs on `File.create`:
|
|
|
179
184
|
| `#[(a..b).step(s), ...]` | strided hyperslab |
|
|
180
185
|
| `#[bool_or_fancy]` | reads whole var + does CArray indexing on the result |
|
|
181
186
|
| `#raw` / `#raw(*idx)` | same indexing as `#[]`, but **bypasses the codec**; `v[...] == v.codec.decode(v, v.raw(...))` |
|
|
182
|
-
|
|
|
187
|
+
| any CArray method (`var.mean`, `var.sum(0)`, …) | called on `to_ca`, reading the whole variable each time; in-place methods (`fill`, `*!`, `*=`) raise `NoMethodError` |
|
|
183
188
|
|
|
184
189
|
**Write (`[]=`)**
|
|
185
190
|
|
|
@@ -215,6 +220,7 @@ Kwargs on `File.create`:
|
|
|
215
220
|
| `:coards` | `Codec::COARDS.new` |
|
|
216
221
|
| `[:cf, opts]` | `Codec::CF.new(**opts)` |
|
|
217
222
|
| `[:coards, opts]` | `Codec::COARDS.new(**opts)` |
|
|
223
|
+
| a codec Class (e.g. `Codec::CF`) | an instance, made with no arguments |
|
|
218
224
|
| a stateless codec `Module` (e.g. `Codec::Identity`) | itself |
|
|
219
225
|
| any object with `#decode(var, ca)` and `#encode(var, ca)` | itself |
|
|
220
226
|
|
|
@@ -222,15 +228,15 @@ Kwargs on `File.create`:
|
|
|
222
228
|
|
|
223
229
|
| Class method | Purpose |
|
|
224
230
|
|---|---|
|
|
225
|
-
| `self.claims?(conventions_string)` |
|
|
231
|
+
| `self.claims?(conventions_string)` | true if this codec should be chosen for this `Conventions` string (called with the whole string, then with each word). Needed only for a registered codec. |
|
|
226
232
|
| Instance methods | |
|
|
227
|
-
| `#decode(var, ca)` |
|
|
228
|
-
| `#encode(var, ca)` |
|
|
233
|
+
| `#decode(var, ca)` | stored values → what a read returns (use `var`'s own attributes only) |
|
|
234
|
+
| `#encode(var, ca)` | what is written → the values to store (the same) |
|
|
229
235
|
|
|
230
236
|
**Registration**
|
|
231
237
|
|
|
232
238
|
```ruby
|
|
233
|
-
CA::NetCDF::Codec.register(MyCodec) # later
|
|
239
|
+
CA::NetCDF::Codec.register(MyCodec) # registered later = tried first, by `claims?`
|
|
234
240
|
```
|
|
235
241
|
|
|
236
242
|
**Built-in: `Codec::Identity`** — module, stateless; `decode`/`encode` pass values through unchanged.
|
|
@@ -240,7 +246,7 @@ CA::NetCDF::Codec.register(MyCodec) # later registrations win on `applies?`
|
|
|
240
246
|
| Call | Purpose |
|
|
241
247
|
|---|---|
|
|
242
248
|
| `CF.new(mask_valid_range: false)` | Options |
|
|
243
|
-
| `CF.claims?(s)` | `"CF
|
|
249
|
+
| `CF.claims?(s)` | a string starting `"CF-"` or `"CF "`, or `"CF"` (case-sensitive) |
|
|
244
250
|
| `#mask_valid_range` | Boolean reader |
|
|
245
251
|
| `#decode(var, ca)` | fill → mask, (opt-in) valid_range → mask, unpack scale/offset, CF time → CATime |
|
|
246
252
|
| `#encode(var, ca)` | reverse (CATime → count, pack, mask → fill) |
|
|
@@ -249,14 +255,14 @@ CF attributes recognized (decode order):
|
|
|
249
255
|
|
|
250
256
|
| Attribute | Effect (decode → encode) |
|
|
251
257
|
|---|---|
|
|
252
|
-
| `_Unsigned` | signed on-disk → unsigned user (bit-reinterpret); reverse on write.
|
|
258
|
+
| `_Unsigned` | signed on-disk → unsigned user (bit-reinterpret); reverse on write. Applied before the others, so they see unsigned values. |
|
|
253
259
|
| `_FillValue`, `missing_value` | mask ↔ sentinel (in unsigned space if `_Unsigned` was set) |
|
|
254
260
|
| `valid_range` / `valid_min` / `valid_max` | (opt-in) out-of-range → UNDEF, same space as fill; encode not defined by CF |
|
|
255
261
|
| `scale_factor`, `add_offset` | `v = packed*sf + ao` ↔ `packed = round((v-ao)/sf)` (round only when var is integer) |
|
|
256
262
|
| `units: "<u> since <ref>"` + `calendar` | CATime ↔ count-since-ref int |
|
|
257
263
|
| `units` = anything else | raw (no UDUNITS conversion) |
|
|
258
264
|
|
|
259
|
-
Supported calendars: `standard`, `gregorian`, `proleptic_gregorian
|
|
265
|
+
Supported calendars: `standard`, `gregorian`, `proleptic_gregorian` (any case), missing → CATime (proleptic Gregorian, UTC). Everything else (`noleap`, `360_day`, `julian`, …) warns once per file and returns raw values on read; write raises.
|
|
260
266
|
|
|
261
267
|
CF time unit words → CATime base:
|
|
262
268
|
|
|
@@ -272,9 +278,28 @@ CF time unit words → CATime base:
|
|
|
272
278
|
| `microsecond` … `usec` … `us` | `:us` |
|
|
273
279
|
| `nanosecond` … `nsec` … `ns` | `:ns` |
|
|
274
280
|
|
|
275
|
-
An `<count> <base>` form like `"5 days since 2024-06-15"` is stored as CATime at the base (`:D`) with the count (`5`) folded into the numeric values — see [
|
|
281
|
+
An `<count> <base>` form like `"5 days since 2024-06-15"` is stored as CATime at the base (`:D`) with the count (`5`) folded into the numeric values — see the [Codecs chapter](04_codec.md#n-of-a-unit-intervals-5-days-since-).
|
|
282
|
+
|
|
283
|
+
**Built-in: `Codec::COARDS < CF`** — same transforms; chosen for a string starting with `COARDS`, or for one starting with neither `COARDS` nor `CF` that has a word starting with `COARDS` (tried before CF).
|
|
284
|
+
|
|
285
|
+
---
|
|
276
286
|
|
|
277
|
-
|
|
287
|
+
## Storage options
|
|
288
|
+
|
|
289
|
+
Reserved kwargs of `def_var`, and file-wide defaults on `File.create`. All but `no_fill:` and `deflate: 0` need a NetCDF-4 file; zstandard, bzip2 and blosc need libnetcdf 4.9, and a compressor libnetcdf lacks raises `NetCDF::Error`. `Var#storage_options` gives them back in this form.
|
|
290
|
+
|
|
291
|
+
| Kwarg | Values |
|
|
292
|
+
|---|---|
|
|
293
|
+
| `chunks:` | `[sizes]`, `:contiguous` or `:compact` |
|
|
294
|
+
| `deflate:` | level 1..9 (`0` / `nil` = off) |
|
|
295
|
+
| `shuffle:` | `true` / `false` |
|
|
296
|
+
| `zstandard:` | level (`0` / `nil` = off) |
|
|
297
|
+
| `bzip2:` | level 1..9 (`0` / `nil` = off) |
|
|
298
|
+
| `szip:` | `{ coding: :nn \| :ec, pixels_per_block: 32 }` |
|
|
299
|
+
| `blosc:` | `{ compressor: :lz \| :lz4 \| :lz4hc \| :snappy \| :zlib \| :zstd, level: 5, blocksize: 0, shuffle: :none \| :byte \| :bit }` |
|
|
300
|
+
| `fletcher32:` | `true` / `false` |
|
|
301
|
+
| `endian:` | `:native`, `:little` or `:big` |
|
|
302
|
+
| `no_fill:` | `true` / `false` |
|
|
278
303
|
|
|
279
304
|
---
|
|
280
305
|
|
|
@@ -311,9 +336,9 @@ An `<count> <base>` form like `"5 days since 2024-06-15"` is stored as CATime at
|
|
|
311
336
|
| `NC_FLOAT` | `:float32` |
|
|
312
337
|
| `NC_DOUBLE` | `:float64` |
|
|
313
338
|
|
|
314
|
-
##
|
|
339
|
+
## Attributes written in the variable's type
|
|
315
340
|
|
|
316
|
-
|
|
341
|
+
`def_var(..., **atts)` and `Var#def_att` write these attributes in the variable's on-disk type, as CF §2.5.1 asks (libnetcdf requires it of `_FillValue`):
|
|
317
342
|
|
|
318
343
|
| Attribute name | Written as |
|
|
319
344
|
|---|---|
|
|
@@ -323,4 +348,4 @@ L2's `def_var(..., **atts)` writes these attribute names via the typed `nc_put_a
|
|
|
323
348
|
| `valid_min` | " |
|
|
324
349
|
| `valid_max` | " |
|
|
325
350
|
|
|
326
|
-
|
|
351
|
+
This holds for a CArray value too. Other attributes take a NetCDF type inferred from the Ruby value; a CArray value keeps its own type, and must be unmasked (a multi-dimensional one is flattened).
|
data/docs/09_cli.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Command-line tools
|
|
2
2
|
|
|
3
|
-
The gem installs two executables. Both work on any NetCDF file, whoever wrote it, and both are thin scripts — `ncheader` over the netcdf gem's `nc_inq_*` calls, `ncvar` over
|
|
3
|
+
The gem installs two executables. Both work on any NetCDF file, whoever wrote it, and both are thin scripts — `ncheader` over the netcdf gem's `nc_inq_*` calls, `ncvar` over the low-level API, which reads into a CArray — so read them (`gem contents carray-netcdf`) if you want a worked example of walking a file.
|
|
4
4
|
|
|
5
5
|
## ncheader — what is in this file
|
|
6
6
|
|
|
@@ -30,7 +30,7 @@ variables:
|
|
|
30
30
|
}
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
The JSON and YAML forms are there for scripting — a schema check in CI, a diff between two files' structure, a generator that reads the header and writes something else.
|
|
33
|
+
The JSON and YAML forms are there for scripting — a schema check in CI, a diff between two files' structure, a generator that reads the header and writes something else. A NaN or infinite attribute (a common `_FillValue`) is written as a bare `NaN` / `Infinity`, as Python's `json` does; parse it with `allow_nan` (Ruby) or Python's default. A byte in a text attribute that is not part of a UTF-8 character, as in an old Latin-1 file, is written as `\xNN`, in every format.
|
|
34
34
|
|
|
35
35
|
## ncvar — get the values out
|
|
36
36
|
|
|
@@ -51,7 +51,7 @@ ncvar -s 0,0 -c 10,20 FILE.nc temperature # a 10x20 corner
|
|
|
51
51
|
ncvar -s 0,0 -c 10,10 -S 1,2 FILE.nc temperature # every second column of it
|
|
52
52
|
```
|
|
53
53
|
|
|
54
|
-
Omitted, `-s` is all zeros and `-c` is the rest of each dimension.
|
|
54
|
+
Omitted, `-s` is all zeros and `-c` is the rest of each dimension, taking every stride-th element, so `-S 2` alone reads every second element to the end. A given `-c` counts the elements actually read, not the span they cover, so a stride of 2 goes with half the count — asking for 20 with `-S 1,2` reaches past the end and raises `NetCDF: Start+count exceeds dimension bound`. A `-s` past the end of a dimension is refused with a message.
|
|
55
55
|
|
|
56
56
|
A variable inside a group is named by its path:
|
|
57
57
|
|
|
@@ -76,4 +76,4 @@ Since the stream carries no shape or type, take those from `ncheader` (or `-f js
|
|
|
76
76
|
|
|
77
77
|
### Character and string variables
|
|
78
78
|
|
|
79
|
-
`NC_CHAR` is bytes, and comes out as bytes: raw mode writes them unchanged, `-f text` writes them as one string, `-f json` puts that string in `data`. `NC_STRING` is a sequence of separate strings, so `-f text` prints one per line, `-f json` keeps the nesting of the variable's shape, and raw mode joins them with a NUL — the one case where the raw stream needs a separator to be readable at all.
|
|
79
|
+
`NC_CHAR` is bytes, and comes out as bytes: raw mode writes them unchanged, `-f text` writes them as one string, `-f json` puts that string in `data`, with any byte that is not UTF-8 written as `\xNN`. `NC_STRING` is a sequence of separate strings, so `-f text` prints one per line, `-f json` keeps the nesting of the variable's shape, and raw mode joins them with a NUL — the one case where the raw stream needs a separator to be readable at all.
|