hypertabular-wasm 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE-UNICODE +39 -0
- data/README.md +278 -34
- data/ext/hypertabular_native/3.4/hypertabular_native.a.gz +0 -0
- data/ext/hypertabular_native/4.0/hypertabular_native.a.gz +0 -0
- data/lib/hypertabular/batch.rb +219 -24
- data/lib/hypertabular/column.rb +166 -21
- data/lib/hypertabular/delimited_reader.rb +319 -94
- data/lib/hypertabular/dialect.rb +92 -10
- data/lib/hypertabular/guarded.rb +116 -0
- data/lib/hypertabular/header.rb +59 -22
- data/lib/hypertabular/row.rb +17 -4
- data/lib/hypertabular/runtime/columns.rb +78 -12
- data/lib/hypertabular/runtime/delimited.rb +63 -37
- data/lib/hypertabular/runtime/workbook.rb +39 -15
- data/lib/hypertabular/runtime.rb +255 -17
- data/lib/hypertabular/suggest.rb +283 -0
- data/lib/hypertabular/tabular_error.rb +9 -3
- data/lib/hypertabular/workbook.rb +185 -45
- data/lib/hypertabular.rb +14 -8
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 323ab4a467db67bc9a44a14beb074f70c384fa0d0db88768a5421e6e5c7898c6
|
|
4
|
+
data.tar.gz: b040fc258c4c0712b5eb6edd0f14bfb869024d8080c001a9cb18fe49bb27257a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 901604507e37a99aa8d3df6799bca5f0882c932f80be3c4c2c8179e4ecc3d662d665125eb097f81fc6c0ed6ed6ff475df0e377eb3d6b866baba6fcbbdea48d41
|
|
7
|
+
data.tar.gz: 2ad307124bac01323ce160996c239186e9e2e1e15752040209d3f59650f7fd54053d5765f8605f51bb2f0a90d5b479cd7d58475930e9241646a0025bebbd5a5c
|
data/LICENSE-UNICODE
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
UNICODE LICENSE V3
|
|
2
|
+
|
|
3
|
+
COPYRIGHT AND PERMISSION NOTICE
|
|
4
|
+
|
|
5
|
+
Copyright © 1991-2026 Unicode, Inc.
|
|
6
|
+
|
|
7
|
+
NOTICE TO USER: Carefully read the following legal agreement. BY
|
|
8
|
+
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
|
|
9
|
+
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
|
|
10
|
+
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
|
|
11
|
+
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
|
|
12
|
+
|
|
13
|
+
Permission is hereby granted, free of charge, to any person obtaining a
|
|
14
|
+
copy of data files and any associated documentation (the "Data Files") or
|
|
15
|
+
software and any associated documentation (the "Software") to deal in the
|
|
16
|
+
Data Files or Software without restriction, including without limitation
|
|
17
|
+
the rights to use, copy, modify, merge, publish, distribute, and/or sell
|
|
18
|
+
copies of the Data Files or Software, and to permit persons to whom the
|
|
19
|
+
Data Files or Software are furnished to do so, provided that either (a)
|
|
20
|
+
this copyright and permission notice appear with all copies of the Data
|
|
21
|
+
Files or Software, or (b) this copyright and permission notice appear in
|
|
22
|
+
associated Documentation.
|
|
23
|
+
|
|
24
|
+
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
|
|
25
|
+
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
26
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
|
|
27
|
+
THIRD PARTY RIGHTS.
|
|
28
|
+
|
|
29
|
+
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
|
|
30
|
+
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
|
|
31
|
+
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
|
|
32
|
+
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
|
|
33
|
+
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
|
|
34
|
+
FILES OR SOFTWARE.
|
|
35
|
+
|
|
36
|
+
Except as contained in this notice, the name of a copyright holder shall
|
|
37
|
+
not be used in advertising or otherwise to promote the sale, use or other
|
|
38
|
+
dealings in these Data Files or Software without prior written
|
|
39
|
+
authorization of the copyright holder.
|
data/README.md
CHANGED
|
@@ -4,12 +4,30 @@ Delimited text — CSV, TSV, any single-byte ASCII separator — and workbooks
|
|
|
4
4
|
read a batch at a time into typed columns, with a
|
|
5
5
|
[HyperCast](https://github.com/SkunkWerkx/HyperCast) verdict for every cell.
|
|
6
6
|
|
|
7
|
+
```sh
|
|
8
|
+
gem install hypertabular
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
The examples read this file, saved as `people.csv`; the second row's score does not cast,
|
|
12
|
+
so the fault path has something to print:
|
|
13
|
+
|
|
14
|
+
```csv
|
|
15
|
+
id,name,score
|
|
16
|
+
1,alice,2.5
|
|
17
|
+
2,bob,oops
|
|
18
|
+
3,carol,4.75
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
and `people.xlsx`, a workbook with one sheet named `People` holding the same table, header
|
|
22
|
+
row first (open `people.csv` in a spreadsheet app, rename the sheet and save it as `.xlsx`).
|
|
23
|
+
|
|
7
24
|
```ruby
|
|
8
25
|
require "hypertabular"
|
|
9
26
|
|
|
10
|
-
HyperTabular::DelimitedReader.open("
|
|
27
|
+
HyperTabular::DelimitedReader.open("people.csv", HyperTabular::Dialect::CSV) do |reader|
|
|
11
28
|
# The header is read first; the plan is built from where its names are.
|
|
12
|
-
header = reader.header
|
|
29
|
+
header = reader.header # => ["id", "name", "score"]
|
|
30
|
+
puts "header: #{header.inspect}"
|
|
13
31
|
reader.bind([
|
|
14
32
|
HyperTabular::Column.i32(header.ordinal("id")),
|
|
15
33
|
HyperTabular::Column.text(header.ordinal("name")),
|
|
@@ -18,32 +36,44 @@ HyperTabular::DelimitedReader.open("orders.csv", HyperTabular::Dialect::CSV) do
|
|
|
18
36
|
|
|
19
37
|
while (batch = reader.read)
|
|
20
38
|
# A column at a time, decoded in one pass — nil where a cell did not cast…
|
|
21
|
-
|
|
39
|
+
names = batch.values(1)
|
|
40
|
+
puts "ids: #{batch.values(0).inspect}"
|
|
41
|
+
puts "faults: #{batch.verdicts(2).count { |verdict| verdict.is_a?(HyperCast::Fault) }}"
|
|
22
42
|
|
|
23
43
|
# …or a cell at a time, as HyperCast's union.
|
|
24
44
|
batch.rows.times do |row|
|
|
25
45
|
case batch.get(2, row)
|
|
26
46
|
in HyperCast::Success(value:)
|
|
27
|
-
puts "#{
|
|
28
|
-
in HyperCast::Fault(reason
|
|
29
|
-
|
|
47
|
+
puts "#{names[row]}: #{value}"
|
|
48
|
+
in HyperCast::Fault(reason:)
|
|
49
|
+
puts "line #{batch.line(row)}: #{reason} in #{batch.raw(2, row).inspect}"
|
|
30
50
|
end
|
|
31
51
|
end
|
|
32
52
|
end
|
|
33
53
|
end
|
|
34
54
|
|
|
35
|
-
# Or a row at a time: a reader, a sheet and a batch are Enumerable over Rows.
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
55
|
+
# Or a row at a time: a reader, a sheet and a batch are Enumerable over Rows. A plan known
|
|
56
|
+
# up front goes straight to the factory, bound before the header is read.
|
|
57
|
+
plan = [HyperTabular::Column.i32(0), HyperTabular::Column.text(1), HyperTabular::Column.f64(2)]
|
|
58
|
+
HyperTabular::DelimitedReader.open("people.csv", HyperTabular::Dialect::CSV, plan) do |reader|
|
|
59
|
+
reader.each { |row| puts "line #{row.line}: #{row.value(1)}" }
|
|
60
|
+
end
|
|
39
61
|
|
|
40
62
|
# A workbook reads into the same batch — from a path, a String or an IO.
|
|
41
|
-
book = HyperTabular::Workbook.open("
|
|
42
|
-
book.sheets
|
|
43
|
-
sheet = book.sheet("
|
|
44
|
-
sheet.bind([
|
|
45
|
-
|
|
46
|
-
|
|
63
|
+
book = HyperTabular::Workbook.open("people.xlsx")
|
|
64
|
+
puts "sheets: #{book.sheets.map(&:name).inspect}" # SheetInfo name="People", hidden=false
|
|
65
|
+
sheet = book.sheet("People", HyperTabular::SheetOptions::DEFAULT)
|
|
66
|
+
sheet.bind([
|
|
67
|
+
HyperTabular::Column.text(sheet.header.ordinal("name")),
|
|
68
|
+
HyperTabular::Column.decimal(sheet.header.ordinal("score"))
|
|
69
|
+
])
|
|
70
|
+
sheet.each do |row|
|
|
71
|
+
case row.get(1)
|
|
72
|
+
in HyperCast::Success(value:)
|
|
73
|
+
puts "#{row.value(0)}: #{value}"
|
|
74
|
+
in HyperCast::Fault(reason:)
|
|
75
|
+
puts "line #{row.line}: #{reason}"
|
|
76
|
+
end
|
|
47
77
|
end
|
|
48
78
|
```
|
|
49
79
|
|
|
@@ -52,10 +82,12 @@ A plan known up front can be handed over at once —
|
|
|
52
82
|
before the header is read.
|
|
53
83
|
|
|
54
84
|
- **The header** (`reader.header`, `sheet.header`) is a `HyperTabular::Header`: a frozen
|
|
55
|
-
Array of frozen names, which also says where a name is. `header.ordinal(name)`
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
`
|
|
85
|
+
Array of frozen names, which also says where a name is. `header.ordinal(name)` is the column
|
|
86
|
+
with exactly that name — case and spaces included — and a `KeyError` naming it when there is
|
|
87
|
+
none (or, with a block, the block's value, as `Hash#fetch` does). A name two columns share is
|
|
88
|
+
a `HyperTabular::AmbiguousColumn`, an `ArgumentError` with `name`, `first` and `second`, whose
|
|
89
|
+
message names both positions, because a plan built from names mustn't read the wrong column:
|
|
90
|
+
say which by position. `header.find_ordinal(name)` answers the first match, or `nil`.
|
|
59
91
|
- **Binding** happens once, before the first read: `bind` again raises `RuntimeError`, and so
|
|
60
92
|
does a `read` before `bind` — which `bind` then cures. A plan that is not Columns is an
|
|
61
93
|
`ArgumentError` and leaves the reader unbound; so is, once a delimited header has fixed
|
|
@@ -64,9 +96,18 @@ before the header is read.
|
|
|
64
96
|
column reads as empty, and a plan reaching past 256 source columns reads shorter batches,
|
|
65
97
|
as many rows as about 8 MiB of cell table holds. `bound?` says which it is. A `nil` plan
|
|
66
98
|
means "bind later"; `[]` is an empty plan, bound at once.
|
|
99
|
+
`batch_rows` sizes the arrays a bound plan reads into. The core judges them before anything is
|
|
100
|
+
allocated: a batch asking for more than 4 GiB, and a `batch_rows` or `buffer_bytes` of 0, are an
|
|
101
|
+
`ArgumentError` from the core, at open or at `bind`. Below that they're asked for zeroed and aren't
|
|
102
|
+
written to until a row reaches them, so a large batch costs what the source holds, and one no memory
|
|
103
|
+
holds is a `NoMemoryError`. Through Fiddle on 64-bit Windows, one array stops at 2 GiB. A delimited
|
|
104
|
+
plan of no columns has no arrays, and reads at any `batch_rows` up to 2^64. `suggest` is never refused
|
|
105
|
+
for the size of its own batches: the core says what fits and the sampler reads as many as it takes. A
|
|
106
|
+
row that alone is past the limit is an `ArgumentError` naming its width.
|
|
67
107
|
`reader.column_count` is the header's count, or the first record's once it has been read.
|
|
68
108
|
- **Rows.** `batch.each`, `reader.each` and `sheet.each` yield a `HyperTabular::Row` —
|
|
69
|
-
`index`, `line`, `get(column)` (also `[]` and `verdict`), `
|
|
109
|
+
`index`, `line`, `get(column)` (also `[]` and `verdict`), `get_checked(column)` (a
|
|
110
|
+
[constrained](#constraints) column's cell), `value(column)`, `raw(column)`, `origin(column)`,
|
|
70
111
|
`to_a` — each answering what the batch's own accessor does for that row. A batch is a copy,
|
|
71
112
|
so a row stays good after the read has moved on. A Row deconstructs as its verdicts, for
|
|
72
113
|
`case row in [HyperCast::Success(value: id), *]`. `reader.each_row` yields each row as
|
|
@@ -75,6 +116,205 @@ before the header is read.
|
|
|
75
116
|
String the workbook owns, as soon as it is opened; `close_source: true` closes the IO once it
|
|
76
117
|
has been read. The kind of workbook is read from its bytes, never from a name.
|
|
77
118
|
|
|
119
|
+
## A messier file
|
|
120
|
+
|
|
121
|
+
Everything about a file is still declared, on the `Dialect` and on the `SheetOptions`, whose defaults
|
|
122
|
+
read as before: lines to pass over, records to discard, what a record of the wrong width is, the
|
|
123
|
+
encoding, and what stands for no value.
|
|
124
|
+
|
|
125
|
+
```ruby
|
|
126
|
+
require "hypertabular"
|
|
127
|
+
|
|
128
|
+
# Windows-1252, with a title line above the table, NA for nothing and rows that stop short.
|
|
129
|
+
dialect = HyperTabular::Dialect.new(separator: ",", skip: 1, short_rows: :pad, encoding: :windows_1252,
|
|
130
|
+
tokens: ["NA"])
|
|
131
|
+
text = "Exported by hand\nname,amount\ncaf\xE9,NA\ntea\nmilk,12x4\n".b
|
|
132
|
+
plan = [
|
|
133
|
+
HyperTabular::Column.text(0),
|
|
134
|
+
HyperTabular::Column.f64(1).with_tokens(%w[NA -]) # in place of the dialect's
|
|
135
|
+
]
|
|
136
|
+
reader = HyperTabular::DelimitedReader.new(text, dialect, plan)
|
|
137
|
+
while (batch = reader.read)
|
|
138
|
+
batch.each do |row|
|
|
139
|
+
cell = row.get(1)
|
|
140
|
+
puts "#{row.value(0)} #{cell.reason} #{row.origin(1)} #{cell.detail}"
|
|
141
|
+
end
|
|
142
|
+
# What went wrong, and only that: a blank, a token and a padded cell aren't problems.
|
|
143
|
+
batch.faults.each do |problem|
|
|
144
|
+
puts "line #{problem.line} column #{problem.source_column} #{problem.reason} #{problem.detail}"
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
# A plan to look at, from a sample of the text. It reads nothing until it's handed to a read.
|
|
149
|
+
found = HyperTabular::DelimitedReader.suggest_string(%(id,total,day\n1,"1,234.50",2026-01-07\n2,12.00,2026-01-08\n),
|
|
150
|
+
HyperTabular::Dialect::CSV)
|
|
151
|
+
found.report.each do |column|
|
|
152
|
+
puts "#{column.name} #{column.status} #{column.door} #{column.matched} of #{column.sampled}"
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
# A sheet takes the part of it that's the table; its plan's ordinals count from B here.
|
|
156
|
+
cut = HyperTabular::Range.parse("B2:D40")
|
|
157
|
+
options = HyperTabular::SheetOptions.new(range: cut)
|
|
158
|
+
puts "#{options.range.first_column} #{options.range.last_row}"
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
A `TabularError` of kind `:encoding` comes after every intact row before it, and its `byte` alone
|
|
162
|
+
counts the original bytes: a read's spans are all the decoded text's. A declared encoding decodes a
|
|
163
|
+
String too, so it isn't read in place. A sheet's `Range` is parsed by the core, so
|
|
164
|
+
`Range.parse("A5:F200")` means what it means in every binding, with `Range.new(first_row:,
|
|
165
|
+
first_column:, last_row:, last_column:)`, 1-based and `0` for no limit, for what that can't say, such
|
|
166
|
+
as every row from the fifth. A plan's ordinals count from the range's first column, while
|
|
167
|
+
`Batch#line` and a problem's `source_column` stay the sheet's own.
|
|
168
|
+
|
|
169
|
+
A `Fault` carries HyperCast's `detail`, and `batch.origin(column, row)` and `row.origin(column)` say
|
|
170
|
+
where a cell's verdict came from: `:cell`, `:token`, `:padded` or `:uncached`. `batch.faults` is a
|
|
171
|
+
frozen Array of `HyperTabular::Problem`s, the row, the line, the plan and source columns, the reason,
|
|
172
|
+
detail, origin and span, a long row reported with its `expected` and `found`, and a broken constraint
|
|
173
|
+
with its `keyword`. It's worked out when
|
|
174
|
+
it's first asked for and kept. A batch is a copy, so the list and the batch both stay right after the
|
|
175
|
+
next read and after the reader has gone; nothing here goes stale. A span counts characters, as a
|
|
176
|
+
`Fault`'s does, on the verdict and the list alike, so `raw[problem.offset, problem.length]` slices it.
|
|
177
|
+
|
|
178
|
+
`notes: false`, on a reader or in `SheetOptions`, drops `detail` and `origin`, which then say `nil` and
|
|
179
|
+
`:cell`: the read costs a little less, and `faults` can't list a formula with no cached value.
|
|
180
|
+
`Column#with_date_options` hands a date door HyperCast's `DateOptions` and `MonthNames`, and on a sheet
|
|
181
|
+
it and `with_tokens` reach only a text cell; `with_tokens([])` says a column has none, the dialect's
|
|
182
|
+
included. Tokens and month names last as long as the read that holds them: this binding keeps nothing
|
|
183
|
+
for the life of the process, on either backend.
|
|
184
|
+
|
|
185
|
+
A stream that stalls still hands over every row it has finished: a read asks the IO for more only when
|
|
186
|
+
it has no finished row in hand, and each ask is one `#readpartial` or `#read`, declared encodings
|
|
187
|
+
included. Whatever the IO's read raises goes to the caller and is final, as in Rust and Go, a
|
|
188
|
+
transient `IOError` or `SystemCallError` (`Errno::EAGAIN`, `Errno::ETIMEDOUT`) included: Ruby can't
|
|
189
|
+
tell one from an exception that landed in the read (below), which may have taken bytes with it, so
|
|
190
|
+
the reader isn't resumable after it. Every later read raises `RuntimeError` naming it; open the
|
|
191
|
+
source again to go on. A broken record is final too, the same failure on every later read.
|
|
192
|
+
|
|
193
|
+
A reader or a sheet is called into once at a time. A call back into it from the IO it reads or from
|
|
194
|
+
a plan's `each` while it reads or binds (`read`, `each`, `bind`, `close`, a sheet's `suggest`)
|
|
195
|
+
raises `RuntimeError`, "the reader is in the middle of a call; nothing that call runs can call back
|
|
196
|
+
into it" ("the sheet …" for a sheet), and the call it came from goes on as it was. A call from your
|
|
197
|
+
own block, between batches, is yours to make. An exception that lands mid-read, such as
|
|
198
|
+
`Timeout.timeout`, `Thread#raise` or Ctrl-C's `Interrupt`, waits until the core and the reader agree
|
|
199
|
+
again, so no row is lost or numbered wrong inside the read: a batch it lands on is handed over by
|
|
200
|
+
the next `read`, with its own lines. The one place it doesn't wait is the IO's own read while that
|
|
201
|
+
blocks, so a timeout still breaks a read waiting on a pipe. Ruby may drop bytes it had already read
|
|
202
|
+
from the IO when an exception lands there, so that read raises the exception and every later read
|
|
203
|
+
(`each` and the rest included) raises `RuntimeError` naming it: "the reader can't read on: Interrupt
|
|
204
|
+
(…) was raised while it was reading its IO, and what that read had taken from the IO may be lost". Once
|
|
205
|
+
the IO has answered, the exception waits until its answer is held. Anything raised inside the reader
|
|
206
|
+
once the core has read past a batch, such as a `NoMemoryError` while the batch is copied, makes
|
|
207
|
+
every later read raise `RuntimeError` naming it. One gap no Ruby method can close is the caller's
|
|
208
|
+
own: an exception that lands after `read` has let go of its batch and before the caller holds it
|
|
209
|
+
loses that batch. Its rows are missing, never wrong, and the lines `Batch#line` gives the next batch
|
|
210
|
+
skip them. A caller that can't lose one holds such exceptions back itself around the read and
|
|
211
|
+
whatever keeps its batch, as in
|
|
212
|
+
`Thread.handle_interrupt(Object => :never) { batches << reader.read }`; the read hands the batch
|
|
213
|
+
over whole, and the exception lands after.
|
|
214
|
+
|
|
215
|
+
A walk through the rows (`each`, `each_row`, a sheet's `each` and `each_batch`) left part-way
|
|
216
|
+
through a batch, by `break`, `first`, `take` or an exception from the block, leaves the rest of that
|
|
217
|
+
batch behind, as the Rust core and Python's `reader.rows()` do: the next walk or `read` starts at
|
|
218
|
+
the next batch, and the lines `Batch#line` gives show the skip. An Enumerator from `each` without a
|
|
219
|
+
block, kept and advanced with `next`, is one walk and goes on where it was.
|
|
220
|
+
|
|
221
|
+
Declaring the locale, a number format and a date order, is the recommended route, and then nothing
|
|
222
|
+
needs inferring. A suggestion is a starting point for a plan to inspect, and it's never applied. Read
|
|
223
|
+
the report first: only a resolved column reads through its door in the plan, the rest as text. With no
|
|
224
|
+
format declared, each of two notations has to read the whole column: `1,234,567` and `1,234.5` read
|
|
225
|
+
only with `,` grouping, `1.234.567`, `1.234,5`, `2,5` and `0,125` only with `,` as the decimal, and the
|
|
226
|
+
plan is in whichever is left. `1,234` and `1.234` read both ways, so they come back `:ambiguous` and
|
|
227
|
+
read as text until a format is declared or another cell settles them, which takes in a column of prices
|
|
228
|
+
at three decimals; `SuggestOptions.new(date_order: ...)` does the same for `01/02/2026`. `2,5` beside
|
|
229
|
+
`2.5` is `:unresolved`, and `1,23,456`, Indian grouping, is text until the column declares Indian
|
|
230
|
+
grouping, because nothing says its culture.
|
|
231
|
+
|
|
232
|
+
`DelimitedReader.suggest(path, dialect, options)` and `suggest_string(text, dialect, options)` sample a
|
|
233
|
+
file or a String, and `sheet.suggest(options)` a sheet, whatever it has been up to, leaving it where it
|
|
234
|
+
was. `SuggestOptions.new(sample_rows: 0)` means 1 and a negative count is refused; options that declare
|
|
235
|
+
only a format still sample a thousand rows, at most a batch. A headerless file is opened twice, so no
|
|
236
|
+
suggestion is offered of a stream. A failure among the sampled rows is raised, and one past them only
|
|
237
|
+
says `truncated`. `SuggestOptions`' `column_formats` and `column_date_orders` take a Hash or an Array
|
|
238
|
+
of pairs, the last for a column winning, and a format anywhere in them that the core refuses is an
|
|
239
|
+
`ArgumentError` before anything is opened, whether or not the sample would reach its column. Compare
|
|
240
|
+
two formats with `==`: the policy counts.
|
|
241
|
+
|
|
242
|
+
A sheet's number cell has no notation, so a column of them is a number under any format, where text
|
|
243
|
+
holding the same digits asks. A column of `0` and `1` is bool, `007` and an integer past int64 are text,
|
|
244
|
+
and a typed date in a sheet is text, for a plan to change by hand. With nothing to choose between, a
|
|
245
|
+
resolved number is planned in `,` groups of three, and `SuggestedColumn#format` names what the plan
|
|
246
|
+
uses: a later `1,23,456` is refused.
|
|
247
|
+
|
|
248
|
+
**To keep every row of a sheet, set `skip_empty_rows: false`.** By default a row whose only cells are
|
|
249
|
+
formulas with no cached value is filtered out as empty, with nothing to say so; off, those rows are
|
|
250
|
+
delivered, their cells at origin `:uncached`.
|
|
251
|
+
|
|
252
|
+
A column given no format is Western, the same as `HyperCast::NumFormat::INVARIANT`: `1,234,567` is
|
|
253
|
+
1234567 and `1,23,456` is `:malformed`. If a column of Indian numbers comes back refused, declare it,
|
|
254
|
+
one column at a time, so an Indian column reads beside a Western one in the same read:
|
|
255
|
+
|
|
256
|
+
```ruby
|
|
257
|
+
indian = HyperCast::NumFormat::INVARIANT.with(grouping: :indian)
|
|
258
|
+
either = [
|
|
259
|
+
HyperTabular::Column.i32(0, indian), # 1,23,456 is 123456, and 1,234,567 is :malformed
|
|
260
|
+
HyperTabular::Column.i32(1) # 1,234,567 is 1234567, and 1,23,456 is :malformed
|
|
261
|
+
]
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
Neither policy reads `0,125`, `12,34`, `1,2,345` or `1234,567` as a number. `HyperCast::NumFormat::DETECT` isn't a
|
|
265
|
+
policy: a column that declares it infers `,` and `.` from each cell, so `Column.f64` reads `1,2345` as
|
|
266
|
+
1.2345, `12,34` as 12.34 and `0,125` as 0.125. A format holding the retired grouping bit `256` (the
|
|
267
|
+
words `351` and `863`), which only a 1.0 Marshal payload or a hand-packed format can hold, raises
|
|
268
|
+
`ArgumentError` when the plan is bound, before any cell, and in a suggest's options before it opens
|
|
269
|
+
anything. HyperCast's
|
|
270
|
+
[grouping notes](https://github.com/SkunkWerkx/HyperCast/blob/master/docs/grouping.md) have every shape
|
|
271
|
+
and the locales neither policy matches.
|
|
272
|
+
|
|
273
|
+
## Constraints
|
|
274
|
+
|
|
275
|
+
A numeric or text column can declare HyperCast's `Constraints`, the OpenAPI 3.2 keywords
|
|
276
|
+
(`minimum`, `exclusive_minimum`, `maximum`, `exclusive_maximum`, `multiple_of` with `rounding`,
|
|
277
|
+
`min_length`, `max_length`, `enum`, with `trim` and `case_sensitive` for text), through the column
|
|
278
|
+
factory's `constraints:`. The declaration is for the column's door: `Constraints` for `:i32` on a
|
|
279
|
+
`Column.u8` is an `ArgumentError`, when the column is made and again when its plan is bound, and a
|
|
280
|
+
door that takes no constraints has no factory keyword for them. Each cell is cast first, so text that
|
|
281
|
+
isn't the type is a `Fault` as ever, and then judged:
|
|
282
|
+
|
|
283
|
+
```ruby
|
|
284
|
+
require "hypertabular"
|
|
285
|
+
|
|
286
|
+
plan = [
|
|
287
|
+
HyperTabular::Column.i32(0, constraints: HyperCast::Constraints.new(door: :i32, minimum: 1, maximum: 10)),
|
|
288
|
+
HyperTabular::Column.text(1, constraints: HyperCast::Constraints.new(door: :text, enum: %w[Female Male]))
|
|
289
|
+
]
|
|
290
|
+
batch = HyperTabular::DelimitedReader.new("qty,sex\n12,MALE\n7,x\n", HyperTabular::Dialect::CSV, plan).read
|
|
291
|
+
batch.each do |row|
|
|
292
|
+
case row.get_checked(0)
|
|
293
|
+
in HyperCast::Checked(value:) then print value
|
|
294
|
+
in HyperCast::Violation(keyword:, value:) then print "#{value} breaks #{keyword}"
|
|
295
|
+
in HyperCast::Fault(reason:) then print reason
|
|
296
|
+
end
|
|
297
|
+
# "MALE" matches the entry, and reads as "Male".
|
|
298
|
+
case row.get_checked(1)
|
|
299
|
+
in HyperCast::Checked(value:) then puts " #{value}"
|
|
300
|
+
in rejection then puts " #{rejection.inspect}"
|
|
301
|
+
end
|
|
302
|
+
end
|
|
303
|
+
batch.faults.each { |problem| puts "line #{problem.line} column #{problem.source_column} #{problem.keyword}" }
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
It prints `12 breaks maximum Male`,
|
|
307
|
+
`7 #<data HyperCast::Violation keyword=HyperCast::Keyword::ENUM, value="x", offset=0, length=1>`,
|
|
308
|
+
`line 2 column 0 maximum` and `line 3 column 1 enum`. A violation isn't a fault: the cell cast, so
|
|
309
|
+
`values(column)` holds its value exactly as cast (never rounded), and the span is the judged text's, in
|
|
310
|
+
characters as a `Fault`'s is. A cell `multiple_of` rounded passes as `Checked` with `adjusted` set, and
|
|
311
|
+
a text `enum` match reads as the entry as declared, the very String the `Constraints` holds, with
|
|
312
|
+
`enum_index` naming it. `faults` lists a violation with its `keyword` and a `reason` of `nil`.
|
|
313
|
+
`batch.get`, `batch.verdicts` and a row's `get` raise `TypeError` on a constrained column, since they
|
|
314
|
+
would read a violation as passed; `get_checked`, and `checked_verdicts(column)` for the whole column
|
|
315
|
+
at once, read an unconstrained column too, where every cell that cast passes, unadjusted. A row's `to_a`, and so `reader.each_row` and a Row's pattern match, give
|
|
316
|
+
each column as its own getter reads it.
|
|
317
|
+
|
|
78
318
|
## The shape
|
|
79
319
|
|
|
80
320
|
The native core (`libhypertabular`) owns no memory and reads no files. `DelimitedReader`
|
|
@@ -84,14 +324,15 @@ native call per batch, and a column comes out of its buffer in one `String#unpac
|
|
|
84
324
|
boundary is crossed once per few thousand rows, not once per cell.
|
|
85
325
|
|
|
86
326
|
- **One batch class.** `read` returns a `HyperTabular::Batch` — `rows`, `columns`,
|
|
87
|
-
`line(row)`, `values(column)`, `verdicts(column)`, `
|
|
88
|
-
row)`, and a `Row` for each row through `each`
|
|
89
|
-
for delimited text and a sheet alike. A batch owns what
|
|
90
|
-
reader has moved on.
|
|
91
|
-
- **Nothing is sniffed.** The `Dialect` states the separator, the quoting
|
|
92
|
-
`SheetOptions` states a sheet's header, whether empty rows
|
|
93
|
-
the plan
|
|
94
|
-
|
|
327
|
+
`line(row)`, `values(column)`, `verdicts(column)`, `checked_verdicts(column)`, `get(column, row)`,
|
|
328
|
+
`get_checked(column, row)`, `raw(column, row)`, `origin(column, row)`, `faults`, and a `Row` for each row through `each`
|
|
329
|
+
— or `nil` once there are no more rows, for delimited text and a sheet alike. A batch owns what
|
|
330
|
+
it shows: it stays good after the reader has moved on.
|
|
331
|
+
- **Nothing is sniffed.** The `Dialect` states the separator, the quoting, the header and what
|
|
332
|
+
[a messier file](#a-messier-file) needs; `SheetOptions` states a sheet's header, whether empty rows
|
|
333
|
+
are skipped, the batch size and the rest of that; the plan states each column's door and, for
|
|
334
|
+
numbers, its `HyperCast::NumFormat`. A suggestion is a plan to inspect, and nothing reads through
|
|
335
|
+
one until it's handed over.
|
|
95
336
|
- **HyperCast is the judge.** `HyperCast::Success`, `HyperCast::Fault`,
|
|
96
337
|
`HyperCast::NumFormat`, `HyperCast::Decimal` and the declared options (`:milliseconds`,
|
|
97
338
|
`:day_month_year`, `:y1904`) are the `hypercast` gem's own. A text cell means exactly
|
|
@@ -106,7 +347,7 @@ boundary is crossed once per few thousand rows, not once per cell.
|
|
|
106
347
|
from, whatever its door and verdict, and a `Fault`'s span indexes it in the units
|
|
107
348
|
`String#[]` slices by, as HyperCast's gem does: `raw[fault.offset, fault.length]` is the
|
|
108
349
|
offending text.
|
|
109
|
-
- **A String is read in place
|
|
350
|
+
- **A String is read in place**, unless its dialect declares an encoding. An IO is read forward only, through a buffer that grows
|
|
110
351
|
when a record does not fit it.
|
|
111
352
|
- **No async API, and none needed.** An IO is read with `#readpartial` or `#read`, which yield
|
|
112
353
|
to the Fiber scheduler (the `async` gem's, or any other) instead of blocking, and a task
|
|
@@ -144,9 +385,12 @@ Selection happens once, at `require`, in that order. Both backends read a batch
|
|
|
144
385
|
call and hand back the same bytes; everything above that crossing — the readers, the
|
|
145
386
|
workbook, `Batch` and every value and verdict it builds — is the same Ruby on either, and
|
|
146
387
|
`spec/native_backend_spec.rb` holds the two to the same answer over every cell of both corpora.
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
388
|
+
Either backend crosses once per batch, yet they don't read at the same speed, and the cause
|
|
389
|
+
hasn't been looked into. Through `benchmark/workbook_benchmark.rb`'s eight-column plan on an Apple
|
|
390
|
+
M6 on 2026-11-08, making every value, Fiddle is the slower: an xlsx sheet of 300k rows reads in
|
|
391
|
+
704 ms against the extension's 458 ms (54% slower), an ods one in 1.02 s against 787 ms (30%), and a
|
|
392
|
+
Numbers one in 887 ms against 875 ms. The extension also buys a platform gem with no shared library
|
|
393
|
+
for Fiddle to find, and Ruby in the browser, where there is no Fiddle at all.
|
|
150
394
|
|
|
151
395
|
`HYPERTABULAR_PURE` forces `:fiddle`, as `HYPERCAST_PURE` does for hypercast. It is a testing
|
|
152
396
|
and diagnostic switch — CI runs the whole suite through it — read for presence, not value. A
|
|
@@ -193,7 +437,7 @@ delimited text from a String or any IO.
|
|
|
193
437
|
- **No Rust toolchain.** The gem's `extconf.rb` only hands rbwasm the prebuilt archive. The
|
|
194
438
|
first `rbwasm build` compiles Ruby itself and takes 15–20 minutes; later builds reuse it.
|
|
195
439
|
- **macOS with Homebrew: limit pkg-config to `openssl@3`.** Run the build as
|
|
196
|
-
`PKG_CONFIG_LIBDIR="$(brew --prefix openssl@3)/lib/pkgconfig" bundle exec rbwasm build
|
|
440
|
+
`PKG_CONFIG_LIBDIR="$(brew --prefix openssl@3)/lib/pkgconfig" bundle exec rbwasm build --ruby-version 4.0 -o ruby.wasm`.
|
|
197
441
|
Without it Homebrew's pkg-config files reach into the build twice: its `openssl` is OpenSSL
|
|
198
442
|
4, which the native Ruby rbwasm builds first does not compile against, and its `libyaml`
|
|
199
443
|
is a macOS library that psych then links into the wasm interpreter, which fails for Ruby
|
|
Binary file
|
|
Binary file
|