edf2csv 0.4.64 → 0.4.67
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1282 -0
- package/dist/cli/report.js.map +1 -1
- package/dist/cli.js.map +1 -1
- package/dist/convert/channels.js.map +1 -1
- package/dist/convert/options.js.map +1 -1
- package/dist/convert/plan.js.map +1 -1
- package/dist/convert/run.js.map +1 -1
- package/dist/convert/time-range.js.map +1 -1
- package/dist/convert/timing.js.map +1 -1
- package/dist/edf/annotations.js.map +1 -1
- package/dist/edf/bytes.js.map +1 -1
- package/dist/edf/errors.js.map +1 -1
- package/dist/edf/header.js.map +1 -1
- package/dist/edf/reader.js.map +1 -1
- package/dist/edf/scale.js.map +1 -1
- package/dist/format/csv.js.map +1 -1
- package/dist/format/list.js.map +1 -1
- package/dist/format/number.js.map +1 -1
- package/dist/index.js.map +1 -1
- package/dist/version.js.map +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,1288 @@
|
|
|
3
3
|
Notable changes to edf2csv. Versions follow [semantic versioning](https://semver.org); while the
|
|
4
4
|
major version is 0, a minor bump may contain breaking changes.
|
|
5
5
|
|
|
6
|
+
## 0.4.67
|
|
7
|
+
|
|
8
|
+
### Fixed: a documentation link pointed at a page the site does not serve
|
|
9
|
+
|
|
10
|
+
The site serves its pages under `/docs/`, and one link in warnings-and-errors.md — added in
|
|
11
|
+
0.4.42 — pointed at `/cli-reference#synopsis`. Every other internal link on the site uses the
|
|
12
|
+
prefix. The markdown rendered, the sentence read sensibly, and only a click found out.
|
|
13
|
+
|
|
14
|
+
The landing page also still showed 129,536 as the number of sample values checked against
|
|
15
|
+
pyEDFlib. 0.4.44 corrected that figure to 16,943 on the correctness page and nowhere else, so
|
|
16
|
+
the two pages disagreed by 7.6x about the tool's headline claim.
|
|
17
|
+
|
|
18
|
+
A test now walks every internal link in the content, checking the prefix, the page and the
|
|
19
|
+
heading it names, since a heading can be renamed without the links to it moving.
|
|
20
|
+
|
|
21
|
+
## 0.4.66
|
|
22
|
+
|
|
23
|
+
### Fixed: the source maps in the published package pointed at nothing
|
|
24
|
+
|
|
25
|
+
Every `.js.map` named `../src/cli.ts` and its siblings as their sources, and `src` is not in
|
|
26
|
+
package.json's `files`. So the maps shipped — 38 of them, 137 kB — and resolved to nothing
|
|
27
|
+
once installed: a stack frame inside edf2csv followed a map to a file that is not there and
|
|
28
|
+
fell back to the compiled output.
|
|
29
|
+
|
|
30
|
+
`inlineSources` puts the TypeScript into the map itself, which a debugger prefers over
|
|
31
|
+
fetching the path, so a frame now lands on the line that produced it. It costs 92 kB packed,
|
|
32
|
+
186.5 kB to 278.3 kB. A test walks every emitted map and fails if one names sources the
|
|
33
|
+
package neither ships nor carries.
|
|
34
|
+
|
|
35
|
+
## 0.4.65
|
|
36
|
+
|
|
37
|
+
### Fixed: the changelog stopped forty-five versions ago
|
|
38
|
+
|
|
39
|
+
This file records what each version changed, and its newest entry was 0.4.19 while the
|
|
40
|
+
package was at 0.4.64. Every release since had notes on GitHub and nothing here — in the one
|
|
41
|
+
place the repository presents as the record, and the one a reader without a browser has.
|
|
42
|
+
|
|
43
|
+
The forty-five entries are written from the commit messages, which were composed in this
|
|
44
|
+
file's voice and carry the same reproductions. Nothing is reconstructed from memory: each
|
|
45
|
+
entry says what its commit said.
|
|
46
|
+
|
|
47
|
+
A test now compares the newest heading here against the version in `package.json`, so the
|
|
48
|
+
file can fall behind by the release being prepared and no further. Two of the numbers on the
|
|
49
|
+
correctness page had drifted the same way and were caught the same way, twice; the difference
|
|
50
|
+
between a number that stays true and one that rots is whether something reads it.
|
|
51
|
+
|
|
52
|
+
## 0.4.64
|
|
53
|
+
|
|
54
|
+
### Fixed: the correctness page says how long its own suite takes
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
`npm test` ... runs the three test files ... It finishes in about a second on a laptop
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
There are six files and it takes about twenty seconds. Both halves drifted while the page's
|
|
61
|
+
test counts were being corrected twice, because those were the numbers a test was watching
|
|
62
|
+
and these were not.
|
|
63
|
+
|
|
64
|
+
Twenty seconds is not a regression, and saying so is the point: almost all of it sits in
|
|
65
|
+
three files that do expensive things on purpose. cli.test.js spawns the built binary as a
|
|
66
|
+
subprocess for every case and interrupts a thirty-file batch to watch it stop. large.test.js
|
|
67
|
+
builds and reads multi-gigabyte recordings. stdout-audit.test.js creates and mounts a disk
|
|
68
|
+
image so it can fill it up. The parser, the planning, the CSV contents and the documentation
|
|
69
|
+
checks together still run in about a second, which is the number the old sentence was
|
|
70
|
+
describing before those three arrived.
|
|
71
|
+
|
|
72
|
+
The file count is now checked, spelled out in words the way the sentence reads. The duration
|
|
73
|
+
is not: it is a property of the machine, and a test asserting a wall-clock figure fails on a
|
|
74
|
+
slow CI box for no reason anyone can act on.
|
|
75
|
+
|
|
76
|
+
## 0.4.63
|
|
77
|
+
|
|
78
|
+
### Added: the promise that the digital codes are recoverable is now checked
|
|
79
|
+
|
|
80
|
+
The FAQ says "the rounding recovers the original integer exactly, because the written
|
|
81
|
+
decimals are always fine enough to keep adjacent digital codes distinct", and prints the
|
|
82
|
+
arithmetic. It is the reason the tool offers no raw-digital output mode: the claim is that
|
|
83
|
+
you do not need one. Nothing checked it.
|
|
84
|
+
|
|
85
|
+
`npm run roundtrip` does — 12,096 cells over 756 combinations of digital and physical bounds,
|
|
86
|
+
EDF and BDF, each converted and then recovered with exactly the arithmetic the page prints.
|
|
87
|
+
Every one comes back as the code the file holds. Narrowing the derived precision by two places
|
|
88
|
+
makes it fail, so it is a check rather than a demonstration.
|
|
89
|
+
|
|
90
|
+
It also found two conditions the page did not state, both of which will bite someone.
|
|
91
|
+
|
|
92
|
+
Take the gain from channels.csv, not from what you believe the range to be. EDF's physical
|
|
93
|
+
bound fields are 8 characters, so a header asked for -0.000001 stores -0. The first version of
|
|
94
|
+
this harness computed the gain from the values it had passed to the writer and reported 620
|
|
95
|
+
failures, every one of them its own — the tool was right and the test was measuring its own
|
|
96
|
+
intent.
|
|
97
|
+
|
|
98
|
+
And leave --decimals alone. The promise is about the precision derived per channel; force a
|
|
99
|
+
coarser one and the codes stop being recoverable with nothing to indicate it. --decimals 0 on
|
|
100
|
+
a 256 Hz EEG channel gets 645 of 768 samples wrong. The page warned that flag would not give
|
|
101
|
+
you integers; it did not say it silently breaks the recipe printed below it.
|
|
102
|
+
|
|
103
|
+
## 0.4.62
|
|
104
|
+
|
|
105
|
+
### Fixed: the fallback for an inexact rate reaches far enough to do its job
|
|
106
|
+
|
|
107
|
+
A rate whose sample interval has no terminating decimal expansion falls back to, in the
|
|
108
|
+
comment's own words, "enough places to keep consecutive samples distinct". It stopped at
|
|
109
|
+
nine. At 3e10 Hz the interval is 3.3e-11, so nine places rounded every sample in a record to
|
|
110
|
+
the same timestamp — a column that cannot tell two samples apart is not keeping them
|
|
111
|
+
distinct, which is the one thing that branch exists to do.
|
|
112
|
+
|
|
113
|
+
Both halves of the function now reach fifteen: the search for an exact expansion, since
|
|
114
|
+
0.4.59, and the fallback. 3e10 Hz gets fourteen places and six distinct timestamps where it
|
|
115
|
+
had three.
|
|
116
|
+
|
|
117
|
+
TIME_RESOLUTION keeps a trigger, because a bound that nothing reaches is a warning nobody
|
|
118
|
+
maintains. It now takes a rate whose interval is finer than fifteen places can express —
|
|
119
|
+
3e15 Hz — which is nine orders of magnitude past anything that records biosignals, and the
|
|
120
|
+
fixture moved there so the warning still has something that raises it.
|
|
121
|
+
|
|
122
|
+
## 0.4.61
|
|
123
|
+
|
|
124
|
+
### Fixed: three things the FAQ told people that its own examples contradict
|
|
125
|
+
|
|
126
|
+
The digital-recovery snippet reads sleep-study_csv/signals.csv, from the recording the same
|
|
127
|
+
page opens by showing as a mixed-rate file whose 256 Hz channels land in signals_256hz.csv.
|
|
128
|
+
Two earlier snippets on the page read that name correctly. This one, the only snippet doing
|
|
129
|
+
arithmetic worth checking, pointed at a file that recording does not produce, so pasting it
|
|
130
|
+
gets a FileNotFoundError. It now reads the right table and says why, and running it against
|
|
131
|
+
mixed-rates.edf recovers the digital codes exactly — [0, 74, 147, 219, 290], which is what
|
|
132
|
+
the file holds.
|
|
133
|
+
|
|
134
|
+
The leftover-files section is headed "Why is there a leftover signals_256hz.csv next to my
|
|
135
|
+
new signals.csv?" and its example warning lists signals_128hz.csv and signals_1hz.csv. The
|
|
136
|
+
one file the reader came to that section about was missing from the message. The tool names
|
|
137
|
+
all three; so does the example now.
|
|
138
|
+
|
|
139
|
+
And the CSV size factor was given as four here and seven in recipes.md. Neither is a
|
|
140
|
+
constant: every row carries one time_s cell however many channels share it, so a 23-channel
|
|
141
|
+
256 Hz montage measures 4.3x while a single-channel recording of the same length is 10x.
|
|
142
|
+
Both pages now say that, and say what it turns on, rather than picking a number.
|
|
143
|
+
|
|
144
|
+
## 0.4.60
|
|
145
|
+
|
|
146
|
+
### Fixed: one recording's length stops being rendered two ways in one session
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
$ edf2csv rec.edf --info
|
|
150
|
+
Duration 6m 40s (400 records of 1s)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
$ edf2csv rec.edf --start 10m --out csv
|
|
155
|
+
error: --start "10m" is at or past the end of this 400s recording.
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
The message whose job is to say how long the recording actually is gave a bare number of
|
|
159
|
+
seconds, while --info gave the humanised form for the same file. On an overnight recording it
|
|
160
|
+
read "7950s recording", leaving the reader to divide by 3600 to judge whether their --start
|
|
161
|
+
was reasonable — which is the one question the message exists to answer. cli-reference.md has
|
|
162
|
+
documented it humanised since it was written ("2h 12m 30s"), a form no input could produce.
|
|
163
|
+
|
|
164
|
+
It now uses the same formatter --info does. The typed value keeps its quotation marks, since
|
|
165
|
+
that is the user's own text and should come back exactly as written, and a recording short
|
|
166
|
+
enough that the two renderings coincide is unchanged — which is the case the reference's
|
|
167
|
+
other example uses.
|
|
168
|
+
|
|
169
|
+
The neighbouring row in the usage-error table was stale in the other direction. Quoting the
|
|
170
|
+
window values was deliberate, added along with the quoting for --start, and only one of the
|
|
171
|
+
two adjacent rows was updated at the time. The table is maintained as exact strings, so the
|
|
172
|
+
row now carries the quotation marks the tool emits.
|
|
173
|
+
|
|
174
|
+
## 0.4.59
|
|
175
|
+
|
|
176
|
+
### Fixed: the time column is exact at 1024 Hz and 2048 Hz, which is what a BioSemi records
|
|
177
|
+
|
|
178
|
+
timeDecimals looks for the number of decimal places in which 1/rate terminates, so sample
|
|
179
|
+
times are written exactly and `time_s * rate` comes back a whole number. The search stopped
|
|
180
|
+
at nine places, and the comment beside it said "every rate in common use clears this — 256 Hz
|
|
181
|
+
needs 8 places, 512 Hz needs 9".
|
|
182
|
+
|
|
183
|
+
The next two powers of two do not. 1/1024 is 0.0009765625, which needs ten; 1/2048 needs
|
|
184
|
+
eleven. Both fell through to the rounding fallback and were written as 0.0009766 and
|
|
185
|
+
0.0004883 — precisely the behaviour the function exists to avoid, on the rates an ActiveTwo
|
|
186
|
+
records at by default.
|
|
187
|
+
|
|
188
|
+
The search now goes to fifteen places, which covers every power of two through 32768 Hz.
|
|
189
|
+
Fifteen and not more because the test has to stay exact: 10^16 is past 2^53, and
|
|
190
|
+
`Number.isInteger(10 ** 17 / 3)` is true, so a larger bound would claim a terminating
|
|
191
|
+
expansion for 3 Hz and ask for seventeen decimals of a number that repeats forever.
|
|
192
|
+
|
|
193
|
+
It also subsumes most of what 0.4.55 warned about. A 1e10 Hz recording terminates at ten
|
|
194
|
+
places, so it now gets a column that separates every sample rather than twenty rows sharing
|
|
195
|
+
three timestamps. TIME_RESOLUTION still has a real trigger — a rate above a gigahertz whose
|
|
196
|
+
expansion repeats, 3e10 Hz say — and a fixture now holds one, since a warning nothing can
|
|
197
|
+
raise is a warning nobody will maintain.
|
|
198
|
+
|
|
199
|
+
## 0.4.58
|
|
200
|
+
|
|
201
|
+
### Added: the header length the file claims is available beside the one that is used
|
|
202
|
+
|
|
203
|
+
api.md says "Every field is read straight from the 256-byte fixed header plus the per-signal
|
|
204
|
+
block. Nothing is normalised except where noted", and lists headerBytes with no note. It is
|
|
205
|
+
the one field that is computed rather than read: 256 for the fixed header plus 256 per
|
|
206
|
+
signal.
|
|
207
|
+
|
|
208
|
+
The computation is right, and has to be. Every data record offset is derived from it, a
|
|
209
|
+
writer that fills the length field in carelessly is common enough to have its own warning,
|
|
210
|
+
and believing the field over the arithmetic would put every sample at the wrong offset. What
|
|
211
|
+
was missing is the field's own value — which is a fact about the file, and is the thing
|
|
212
|
+
HEADER_BYTES_MISMATCH is comparing against.
|
|
213
|
+
|
|
214
|
+
`declaredHeaderBytes` now carries it, exactly as `declaredRecordCount` already carries what
|
|
215
|
+
the record-count field said. A caller auditing how a recording was written can see both; a
|
|
216
|
+
well-formed file has them equal, which is what makes the pair worth having.
|
|
217
|
+
|
|
218
|
+
## 0.4.57
|
|
219
|
+
|
|
220
|
+
### Fixed: the reference describes the inversion the code actually looks for
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
INVERTED_PHYSICAL_RANGE — A channel's physical minimum sits above its physical maximum
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
That is not the condition. The gain is (physMax - physMin) / (digMax - digMin), so what
|
|
227
|
+
inverts a channel is the sign of that fraction. Reversing exactly one of the two pairs makes
|
|
228
|
+
it negative; reversing both leaves it positive, and such a channel is not inverted at all.
|
|
229
|
+
The code has known this for some versions — its comment says warning on the physical pair
|
|
230
|
+
alone "was wrong in both directions" — and the page went on stating the version that was
|
|
231
|
+
wrong, in two places.
|
|
232
|
+
|
|
233
|
+
Someone matching on the described condition would raise a warning on a file with both pairs
|
|
234
|
+
reversed, which is correct data, and miss one with only its digital bounds reversed, which is
|
|
235
|
+
genuinely sign-flipped EEG. The page now gives the gain, a table of the three cases, and a
|
|
236
|
+
note that the message names whichever pair is actually reversed.
|
|
237
|
+
|
|
238
|
+
A fixture holds all three so the distinction is checked rather than described. Two of its
|
|
239
|
+
columns are constant, because the fixture writer clamps samples to the declared digital
|
|
240
|
+
range and a reversed range clamps every one of them to the same value — said so in the
|
|
241
|
+
fixture and in the test, since a constant column that looks like a converter bug and is not
|
|
242
|
+
one is worth labelling.
|
|
243
|
+
|
|
244
|
+
## 0.4.56
|
|
245
|
+
|
|
246
|
+
### Fixed: a channel with no samples stops being described as one nobody asked for
|
|
247
|
+
|
|
248
|
+
```
|
|
249
|
+
$ edf2csv rec.edf --info --channels unused
|
|
250
|
+
1 unused unused uV 0 Hz -100 to 100 (not selected)
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
warning: Signal 1 ("unused") carries no samples at all (0 per data record).
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
The channel named on --channels was reported as not selected, in a table printed directly
|
|
258
|
+
above a warning explaining why it has nothing to contribute. It was selected; the file gives
|
|
259
|
+
it nothing. The column now says "(no samples)", which is true whether or not it was asked
|
|
260
|
+
for, and a channel that does carry samples and was not chosen still reads "(not selected)".
|
|
261
|
+
|
|
262
|
+
The conversion had the same gap from the other side. Selecting only such a channel leaves no
|
|
263
|
+
table to write, so the run produced channels.csv and metadata.json and no signals.csv —
|
|
264
|
+
while output-files.md says signals.csv is written unless --annotations-only was passed.
|
|
265
|
+
Nothing accounted for the missing file. It does now, and the page carries the exception.
|
|
266
|
+
|
|
267
|
+
Same shape as 0.4.51: a report describing what it measures rather than what the run did.
|
|
268
|
+
|
|
269
|
+
## 0.4.55
|
|
270
|
+
|
|
271
|
+
### Fixed: the slack at a window boundary stops being wider than the samples
|
|
272
|
+
|
|
273
|
+
Deciding which samples fall inside a requested window used a flat nanosecond of slack,
|
|
274
|
+
applied whatever the sampling rate. A nanosecond is far below any real interval — 20 kHz is
|
|
275
|
+
50 microseconds — but the format does not oblige it to be: EDF's record duration is an
|
|
276
|
+
8-character field that accepts 1e-9.
|
|
277
|
+
|
|
278
|
+
A recording of two 1 ns records holding ten samples each wrote ten of its twenty rows. The
|
|
279
|
+
window ends at 2e-9, the comparison asked for `time < 2e-9 - 1e-9`, and the whole second
|
|
280
|
+
record failed it. Exit 0, no warning, half the samples gone.
|
|
281
|
+
|
|
282
|
+
Slack that reaches the next sample is not slack, so it is now capped at half a sample
|
|
283
|
+
interval. Same lesson as 0.4.45: a constant chosen for one scale is a bug at another, and the
|
|
284
|
+
scale that matters is what the recording can express.
|
|
285
|
+
|
|
286
|
+
Fixing that exposes the other half. Sample times are written to at most nine decimal places,
|
|
287
|
+
so above a gigahertz the column repeats — those twenty rows carry three distinct times.
|
|
288
|
+
Nothing is lost, and every sample is written in order, but `time_s` stops identifying a row,
|
|
289
|
+
so joining or plotting on it collapses samples that are genuinely distinct. TIME_RESOLUTION
|
|
290
|
+
now says so and points at the row number instead. No shipped fixture raises it, and the
|
|
291
|
+
whole fixture set was checked to be sure.
|
|
292
|
+
|
|
293
|
+
## 0.4.54
|
|
294
|
+
|
|
295
|
+
### Fixed: the row buffer is emptied when it fills, not when the record ends
|
|
296
|
+
|
|
297
|
+
BufferedLineWriter exists to hold a bounded amount of formatted text, and the loop drained it
|
|
298
|
+
once per data record. Within a record nothing emptied it, so memory followed
|
|
299
|
+
samples-per-record rather than the buffer's own threshold.
|
|
300
|
+
|
|
301
|
+
```
|
|
302
|
+
16,000 records of 1,000 samples -> 283 MB CSV, converts under a 256 MB heap
|
|
303
|
+
1 record of 16,000,000 samples -> JavaScript heap out of memory
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
The same 32 MB of samples either way. The format allows both layouts and says nothing about
|
|
307
|
+
which to expect: samples-per-record is an 8-character field, and a writer that puts an entire
|
|
308
|
+
recording in one record is producing a legal file.
|
|
309
|
+
|
|
310
|
+
The buffer is now emptied wherever it fills. `full` is a synchronous read of the pending size,
|
|
311
|
+
so the twenty million rows that are not at a boundary cost a comparison rather than a
|
|
312
|
+
microtask each — a 92 MB conversion times the same as before.
|
|
313
|
+
|
|
314
|
+
The test caps the heap at 256 MB, because without a limit the machine's own memory lets the
|
|
315
|
+
old code through and the test becomes a demonstration of nothing.
|
|
316
|
+
|
|
317
|
+
The FAQ said one thing scales with the recording. Two do, and neither is its length: the
|
|
318
|
+
annotation list, and one record held whole while it is read. Both are now named there.
|
|
319
|
+
|
|
320
|
+
## 0.4.53
|
|
321
|
+
|
|
322
|
+
### Fixed: a data record larger than one read no longer takes the process down
|
|
323
|
+
|
|
324
|
+
```
|
|
325
|
+
$ edf2csv rec.edf --out csv
|
|
326
|
+
node[87678]: void node::fs::Read(...) at ../src/node_file.cc:2632
|
|
327
|
+
Assertion failed: args[3]->IsInt32()
|
|
328
|
+
----- Native stack trace -----
|
|
329
|
+
$ echo $?
|
|
330
|
+
134
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
Nothing written, and exit 134 — which is none of the three codes this tool documents. Worse
|
|
334
|
+
through the library: the assertion is raised in C++, not thrown, so neither a try/catch
|
|
335
|
+
around readRecords nor an uncaughtException handler ever runs. A consumer's whole process
|
|
336
|
+
goes down with it.
|
|
337
|
+
|
|
338
|
+
A record is read in one call when it exceeds the chunk budget, there being nothing smaller to
|
|
339
|
+
divide it by: a record is the unit the format is addressed in. EDF's samples-per-record field
|
|
340
|
+
is 8 characters, so eleven channels at 99,999,999 samples make a record of 2,199,999,978
|
|
341
|
+
bytes — and a long record duration at ordinary rates reaches the same place. `fs.read` takes
|
|
342
|
+
a length that must fit in a signed 32-bit integer, and asserts rather than throwing when it
|
|
343
|
+
does not.
|
|
344
|
+
|
|
345
|
+
readFully already looped to handle a short read, so the fix is to cap what it asks for at a
|
|
346
|
+
gigabyte and go round again. The 2.2 GB record now converts in 1.2s. The boundary is exact,
|
|
347
|
+
so the test checks both sides of it: 2,199,999,978 bytes must convert, and 2,134,000,000 —
|
|
348
|
+
which always worked — must keep working, since a fix that refused large records rather than
|
|
349
|
+
reading them would pass the first and fail the second.
|
|
350
|
+
|
|
351
|
+
The FAQ said memory does not scale with the recording, with the annotation list as the one
|
|
352
|
+
exception. There is a second: one record is held whole. It is now named there.
|
|
353
|
+
|
|
354
|
+
## 0.4.52
|
|
355
|
+
|
|
356
|
+
### Added: the metadata.json the documentation prints is checked against one a conversion writes
|
|
357
|
+
|
|
358
|
+
output-files.md prints a whole metadata.json as its explanation of the format, and that
|
|
359
|
+
transcript is what someone reads before writing code against it. A key added to the record
|
|
360
|
+
and not to the page reads as a key that does not exist; one removed reads as a key they can
|
|
361
|
+
rely on. api.md had exactly that happen to its `window` object, which lost two fields for
|
|
362
|
+
several versions before an audit noticed — and the only reason this record has not is that
|
|
363
|
+
nobody has changed it lately.
|
|
364
|
+
|
|
365
|
+
The docs test now converts a fixture and compares the key structure of the result against
|
|
366
|
+
the sample on the page. The shape, not the values: the sample describes an eight-hour sleep
|
|
367
|
+
study that is not in this repository, and rewriting it to match a two-record fixture would
|
|
368
|
+
make it a worse explanation of the format, which is what it is there for.
|
|
369
|
+
|
|
370
|
+
The fixture is the discontinuous one, chosen so every part of the record the page shows is
|
|
371
|
+
populated — it raises a diagnostic, so `notes` is not empty, and carries annotations, so the
|
|
372
|
+
annotation fields are real. An empty array cannot say what its entries look like, and that is
|
|
373
|
+
the half of the shape worth checking.
|
|
374
|
+
|
|
375
|
+
Confirmed capable of failing: deleting one key from the sample fails the test.
|
|
376
|
+
|
|
377
|
+
## 0.4.51
|
|
378
|
+
|
|
379
|
+
### Fixed: --info stops promising nothing for a run that writes a file
|
|
380
|
+
|
|
381
|
+
```
|
|
382
|
+
$ edf2csv annotations.edf --info --annotations-only
|
|
383
|
+
Would write 0 rows, roughly 0 B.
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
```
|
|
387
|
+
$ edf2csv annotations.edf --out csv --annotations-only
|
|
388
|
+
Wrote csv
|
|
389
|
+
annotations.csv 3 rows
|
|
390
|
+
channels.csv 1 rows
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
The estimate describes the signal tables, and under --annotations-only there are none, so
|
|
394
|
+
zero was true of what it measures and false of the run. --info exists to say what a
|
|
395
|
+
conversion will do before you commit to it; asserting it will write nothing, when it will
|
|
396
|
+
write a file, is the one thing it must not do.
|
|
397
|
+
|
|
398
|
+
How many events there are cannot be answered from the header — the annotation channel has to
|
|
399
|
+
be read record by record, which is the scan --info exists to avoid. So it names the files and
|
|
400
|
+
says the count is not knowable this cheaply, rather than inventing a zero:
|
|
401
|
+
|
|
402
|
+
```
|
|
403
|
+
Would write annotations.csv and channels.csv, and no signal data. How many events there
|
|
404
|
+
are cannot be told from the header.
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
A recording with no annotation channel is told that instead, since it gets no annotations.csv
|
|
408
|
+
at all. An ordinary --info is untouched, --gzip keeps its "before compression" note, and a
|
|
409
|
+
test that had encoded "Would write 0 rows" as correct is replaced by one that checks the run
|
|
410
|
+
it describes really does write three events.
|
|
411
|
+
|
|
412
|
+
## 0.4.50
|
|
413
|
+
|
|
414
|
+
### Added: what --info predicts is checked against what a conversion writes
|
|
415
|
+
|
|
416
|
+
Two promises live in that one line, and they are not the same promise. The row count is
|
|
417
|
+
arithmetic on the header, so a conversion doing the same arithmetic has to land on the same
|
|
418
|
+
number — close is wrong. The byte count is documented as an approximation, and the direction
|
|
419
|
+
it errs in is the whole point: people read it to decide whether they have room, so reading
|
|
420
|
+
low is a defect even though "approximate" would excuse it.
|
|
421
|
+
|
|
422
|
+
Neither had anything checking it. `npm run estimate` crosses every fixture with every option
|
|
423
|
+
combination — 192 predictions over 34 recordings — and asserts the row count exactly and the
|
|
424
|
+
byte count as a bound. Sizes read 20% high on average, which is the side to be on.
|
|
425
|
+
|
|
426
|
+
It found the header row being measured as the raw labels rather than as the line that gets
|
|
427
|
+
written. A column name is quoted when it contains a comma, a quote, a newline or an edge
|
|
428
|
+
space, and every quote inside it is doubled; three channels labelled `a,b,c,d,e`, `x"y` and
|
|
429
|
+
`plain` write a 32-byte header and were budgeted 27. EDF labels are free text, so commas in
|
|
430
|
+
them are ordinary. csvRow writes that line, so csvRow now measures it — nothing else is in a
|
|
431
|
+
position to stay right when the quoting rules change.
|
|
432
|
+
|
|
433
|
+
The sweep carries no allowlist. There is a case the estimate cannot bound — a recording whose
|
|
434
|
+
samples fall outside the digital range its own header declares — but no fixture does that, and
|
|
435
|
+
an exemption nobody has to earn is how a regression gets in wearing the name of a known case.
|
|
436
|
+
|
|
437
|
+
## 0.4.49
|
|
438
|
+
|
|
439
|
+
### Fixed: two names a macOS filesystem makes one are refused, not merged
|
|
440
|
+
|
|
441
|
+
The guard that refuses two recordings landing in the same directory case-folded and stopped
|
|
442
|
+
there. HFS+ and APFS fold Unicode normalisation as well, so `café` written as e + U+0301 and
|
|
443
|
+
`café` written as U+00E9 are one directory — while remaining two different JavaScript
|
|
444
|
+
strings, which is all the guard compared.
|
|
445
|
+
|
|
446
|
+
```
|
|
447
|
+
study/café.edf (NFC) -> csv/café/signals.csv
|
|
448
|
+
study/café.bdf (NFD) -> csv/café/signals_256hz.csv, _128hz, _1hz
|
|
449
|
+
```
|
|
450
|
+
|
|
451
|
+
One directory holding two recordings, under a single metadata.json naming one of them,
|
|
452
|
+
reported as "Converted 2 of 2 recordings" and exit 0 under --force. The extensions differ,
|
|
453
|
+
which is what lets both files exist while their stems collide.
|
|
454
|
+
|
|
455
|
+
Without --force the second conversion happened to hit "already exists" — the accidental save
|
|
456
|
+
rather than the check doing its job, and it named the wrong problem: a directory in the way,
|
|
457
|
+
rather than two recordings claiming one name. It now says what is actually wrong, before
|
|
458
|
+
anything is written, with or without --force.
|
|
459
|
+
|
|
460
|
+
Folded only on darwin. On Linux those are genuinely two directories and refusing them would
|
|
461
|
+
be inventing a collision that is not there; Windows preserves normalisation too. That is the
|
|
462
|
+
same platform-shaped assumption the case fold already made, and it keeps the same limit — a
|
|
463
|
+
volume that normalises while the running platform does not is covered by neither.
|
|
464
|
+
|
|
465
|
+
## 0.4.48
|
|
466
|
+
|
|
467
|
+
### Fixed: the read buffer is the size of the data, not the size of the budget
|
|
468
|
+
|
|
469
|
+
`chunkBytes` is a ceiling on how much to read at once. The buffer was however many records
|
|
470
|
+
would fit in it — whether or not the file had that many.
|
|
471
|
+
|
|
472
|
+
```
|
|
473
|
+
a 848-byte fixture, 2 records, read with a 512 MB budget
|
|
474
|
+
allocated 536,870,880 bytes
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
and every ordinary read of a small recording reserved the full 8 MB default for a file that
|
|
478
|
+
might be a tenth of a kilobyte. Nothing was wrong with the data; the memory simply had
|
|
479
|
+
nothing to do with it. A batch of five hundred short recordings paid it five hundred times.
|
|
480
|
+
|
|
481
|
+
Bounded now by what is actually going to be read, so the fixture above allocates 80 bytes and
|
|
482
|
+
a windowed read of two records allocates two records. Large files are untouched: their record
|
|
483
|
+
count exceeds what the budget allows either way, so the budget is still what decides.
|
|
484
|
+
|
|
485
|
+
The samples cannot move, and are checked not to: the same five recordings read with a
|
|
486
|
+
one-byte budget, a one-megabyte budget and a 512 MB budget produce identical sample
|
|
487
|
+
sequences. The suite already held that check for two of those; this adds the third.
|
|
488
|
+
|
|
489
|
+
## 0.4.47
|
|
490
|
+
|
|
491
|
+
### Fixed: two ways a batch still depended on things the command did not say
|
|
492
|
+
|
|
493
|
+
--out went back to meaning the output directory itself when a named folder turned out to
|
|
494
|
+
hold nothing.
|
|
495
|
+
|
|
496
|
+
```
|
|
497
|
+
edf2csv study named.edf --out csv study holds recordings -> csv/named/signals.csv
|
|
498
|
+
edf2csv blank named.edf --out csv blank holds none -> csv/signals.csv
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
0.4.20 took this decision off the recording count, and the flag it left behind answered "did
|
|
502
|
+
any input come from a directory" rather than "was a directory named" — the same question
|
|
503
|
+
only when the directory yielded something. So whether an unrelated folder happened to
|
|
504
|
+
contain anything decided where a different recording's output went, silently, exit 0. It is
|
|
505
|
+
now decided by what was named, which is what 0.4.20 said it was.
|
|
506
|
+
|
|
507
|
+
And a child was handed its options as two arguments each, so a value beginning with a dash
|
|
508
|
+
became another option in the child's parser:
|
|
509
|
+
|
|
510
|
+
```
|
|
511
|
+
edf2csv study --out ./-nightly converts
|
|
512
|
+
edf2csv study --out ./-nightly --jobs 2 Option '--out' argument is ambiguous.
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
The same command, converting everything one way and nothing the other. A leading dash is not
|
|
516
|
+
exotic — path.join produces one from a folder given as `.`, and directories get named after
|
|
517
|
+
dates and flags. 0.4.19 fixed this for the recording's own path with a `--` separator; every
|
|
518
|
+
option that carries a value had it too, and they now go over as `--flag=value`, which cannot
|
|
519
|
+
be misread. Serial and parallel produce identical bytes with such a destination.
|
|
520
|
+
|
|
521
|
+
## 0.4.46
|
|
522
|
+
|
|
523
|
+
### Fixed: --info and the conversion agree again about where a recording starts
|
|
524
|
+
|
|
525
|
+
0.4.41 taught a conversion to take a continuous recording's origin from whichever record
|
|
526
|
+
first states one, so a single unreadable timekeeping entry no longer costs the file its
|
|
527
|
+
position. --info was left reading record 0 and stopping there, so from that version the two
|
|
528
|
+
halves of the tool described the same file differently.
|
|
529
|
+
|
|
530
|
+
```
|
|
531
|
+
$ edf2csv rec.edf --info --start 3
|
|
532
|
+
error: --start "3" is at or past the end of this 3s recording.
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
```
|
|
536
|
+
$ edf2csv rec.edf --out csv --start 3
|
|
537
|
+
Wrote csv
|
|
538
|
+
signals.csv 2 rows
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
--info found nothing at record 0 and reported a recording starting at zero; the conversion
|
|
542
|
+
took the origin from record 1 and timed it from 0.5s. The flag exists to tell you what a
|
|
543
|
+
conversion will do before you commit to it, so disagreeing with the conversion is the one
|
|
544
|
+
thing it must not do.
|
|
545
|
+
|
|
546
|
+
It now reads on past record 0 until a record states its own time, up to sixteen of them.
|
|
547
|
+
Continuity is what makes that cheap and correct: record i beginning at t puts the origin at
|
|
548
|
+
t - i * duration, so any one of them settles it, and sixteen small reads keep --info a header
|
|
549
|
+
read rather than the per-record scan it was deliberately spared. On the 205 KB fixture it
|
|
550
|
+
still returns in 0.05s.
|
|
551
|
+
|
|
552
|
+
readFirstRecordStart is now readOrigin, since it returns the recording's origin rather than
|
|
553
|
+
one record's start. It was never documented.
|
|
554
|
+
|
|
555
|
+
## 0.4.45
|
|
556
|
+
|
|
557
|
+
### Fixed: an ordinary recording stops being accused of contradicting its own continuity
|
|
558
|
+
|
|
559
|
+
0.4.41 added a check that an EDF+C file's records sit where continuity puts them, and wrote
|
|
560
|
+
it as an equality between two doubles. They are not equal. A recording of 0.1s records
|
|
561
|
+
sitting at 0.1, 0.2, 0.3 ... is contiguous by construction, and 0.1 + 2 * 0.1 is
|
|
562
|
+
0.30000000000000004.
|
|
563
|
+
|
|
564
|
+
```
|
|
565
|
+
warning: This file is marked continuous (EDF+C), but 2 of its 8 data records say they
|
|
566
|
+
start somewhere other than where continuity puts them.
|
|
567
|
+
```
|
|
568
|
+
|
|
569
|
+
On a file with nothing whatever wrong with it — and under --strict, a failed run. Fractional
|
|
570
|
+
record durations are ordinary; the format has an 8-character field for exactly that.
|
|
571
|
+
|
|
572
|
+
The comparison is now against what the recording can express: one sample of its fastest
|
|
573
|
+
channel is the shortest span it distinguishes, and anything below half of that is arithmetic
|
|
574
|
+
rather than a gap. canCarry already refuses origins where the double spacing swamps that
|
|
575
|
+
interval, so the representation error is under the tolerance by construction rather than by
|
|
576
|
+
hope — the two checks now share the same notion of "the finest thing this file can say".
|
|
577
|
+
|
|
578
|
+
A record that really is somewhere else is still caught: a file marked EDF+C whose records
|
|
579
|
+
jump from 1.5s to 10.5s reports one of three.
|
|
580
|
+
|
|
581
|
+
Shipped three versions ago and mine. The lesson is the ordinary one about comparing computed
|
|
582
|
+
floating point for equality, which is why the fixture that catches it is now in the suite in
|
|
583
|
+
both directions.
|
|
584
|
+
|
|
585
|
+
## 0.4.44
|
|
586
|
+
|
|
587
|
+
### Fixed: the numbers the documentation prints are numbers this repository produces
|
|
588
|
+
|
|
589
|
+
Five figures, each verified by running the thing that produces it.
|
|
590
|
+
|
|
591
|
+
The --info footer said "roughly 27.4 KB" on two pages where the tool says 22.2 KB, and
|
|
592
|
+
api.md printed estimate.bytes of 28095 against an actual 22749. The estimate changed and the
|
|
593
|
+
transcripts did not.
|
|
594
|
+
|
|
595
|
+
The convert() example in api.md was wrong three ways in one block. Its `window` object had
|
|
596
|
+
lost `recordingStartSeconds` and `recordingEndSeconds`, added to ResolvedRange since. Its
|
|
597
|
+
estimate said 6165 against an actual 5172. And its warning read "3 different sampling rates
|
|
598
|
+
(256 Hz, 128 Hz, 1 Hz)" for a call that selects two channels at two rates — where the prose
|
|
599
|
+
directly below it says "Two channels were requested at two different rates". That is the
|
|
600
|
+
0.4.x change making the mixed-rate warning describe the conversion rather than the file, and
|
|
601
|
+
the sample output was never updated, so the page contradicted itself in adjacent lines. The
|
|
602
|
+
note explaining why it is two rather than three is now there.
|
|
603
|
+
|
|
604
|
+
And the correctness page's test counts, wrong again: 179 against 197, having been 148 against
|
|
605
|
+
179 before 0.4.34 corrected them. So this time the docs test checks them — per file, against
|
|
606
|
+
the summary, and against the prose that repeats the total. It caught its own new test on the
|
|
607
|
+
first run, which is what it is for.
|
|
608
|
+
|
|
609
|
+
## 0.4.43
|
|
610
|
+
|
|
611
|
+
### Added: the documentation's lists are checked against the source's
|
|
612
|
+
|
|
613
|
+
Four times in the 0.4 line a new diagnostic shipped and one of the three places that
|
|
614
|
+
enumerate them was not updated — the table in warnings-and-errors.md, the code list in
|
|
615
|
+
cli-reference.md, the block in api.md. Nothing failed, because nothing checked. The lists
|
|
616
|
+
are prose, and prose does not compile.
|
|
617
|
+
|
|
618
|
+
Now a test reads both directions: every code in the source is named on all three pages, and
|
|
619
|
+
every code the pages list exists. Same for the conversion error codes against api.md, every
|
|
620
|
+
flag in --help against the README and the reference, and the set of exit codes against the
|
|
621
|
+
table that explains them.
|
|
622
|
+
|
|
623
|
+
It reads the three enumerating constructs rather than sweeping the pages for upper-case
|
|
624
|
+
words, because those are also --info column headings, errno names and the format's own
|
|
625
|
+
vocabulary — a check that caught those would need an allowlist that grows with the prose,
|
|
626
|
+
which is the kind of test people delete.
|
|
627
|
+
|
|
628
|
+
On its first run it found NONSTANDARD_UNIT declared in DiagnosticCode and named on two pages
|
|
629
|
+
out of three. It has been dead since 0.1: reserved for a physical dimension outside the set
|
|
630
|
+
the spec recommends, never raised, and a warning on every unusual unit would fire on most
|
|
631
|
+
real recordings, which is presumably why. 0.4.37 took the other dead code, NONPRINTABLE_LABEL,
|
|
632
|
+
and implemented it. This one is removed instead, so the type stops offering a value nothing
|
|
633
|
+
can produce and the reference loses its "codes that exist but are never raised" section
|
|
634
|
+
along with the last thing in it.
|
|
635
|
+
|
|
636
|
+
Nothing that ever occurred at runtime changes: no conversion could produce that value.
|
|
637
|
+
|
|
638
|
+
## 0.4.42
|
|
639
|
+
|
|
640
|
+
### Fixed: the reference stops describing a version of the tool that no longer exists
|
|
641
|
+
|
|
642
|
+
```
|
|
643
|
+
The input path must be a regular file that can be read. A directory, a missing path or
|
|
644
|
+
a special file is a file error (exit 1), not a usage error.
|
|
645
|
+
```
|
|
646
|
+
|
|
647
|
+
Every clause of that is wrong for a directory, and has been since folders became inputs. A
|
|
648
|
+
folder of recordings converts and exits 0. A folder holding none exits 2, not 1. The exit-1
|
|
649
|
+
list said the same thing a second time — "the input can't be read: ... it's a directory" —
|
|
650
|
+
and the UNREADABLE section printed an example the command line cannot produce.
|
|
651
|
+
|
|
652
|
+
That last one is worth keeping rather than deleting: `EdfFile.open` does still refuse a
|
|
653
|
+
directory, because the library takes one recording and a directory is not one. What changed
|
|
654
|
+
is that the CLI expands a directory before it gets there. The section now says which of the
|
|
655
|
+
two you are looking at.
|
|
656
|
+
|
|
657
|
+
A page that describes the wrong tool is worse than a page with a gap in it — the gap sends
|
|
658
|
+
you to try it, and this sent you to conclude the feature was not there. A test pins the
|
|
659
|
+
contract in all four of its parts: a folder converts, an empty folder is 2, a missing file
|
|
660
|
+
is 1, and the library still refuses a directory.
|
|
661
|
+
|
|
662
|
+
## 0.4.41
|
|
663
|
+
|
|
664
|
+
### Fixed: one unreadable timekeeping entry stops costing a recording its origin
|
|
665
|
+
|
|
666
|
+
The EDF+C origin came from recordStarts[0] and nowhere else. When that one entry could not
|
|
667
|
+
be decoded, the whole file was timed from zero — while records 1 and 2, saying plainly that
|
|
668
|
+
they begin at 1.5s and 2.5s, went unread.
|
|
669
|
+
|
|
670
|
+
A recording whose records sit at 0.5s, 1.5s and 2.5s came out with every sample half a
|
|
671
|
+
second earlier than the file states, against annotation onsets that kept their true values,
|
|
672
|
+
so an event at +0.75 fell between rows. That is exactly the mismatch 0.4.9 fixed, arriving
|
|
673
|
+
through the one hole left in it — and the byte-identical EDF+D twin timed it correctly,
|
|
674
|
+
which is what gives it away.
|
|
675
|
+
|
|
676
|
+
Continuity is what makes it recoverable: record i sits at origin + i * duration, so any
|
|
677
|
+
readable record fixes the origin for all of them. The EDF+D fallback for a record with no
|
|
678
|
+
readable time now starts from that origin too, rather than assuming zero, so the twins agree
|
|
679
|
+
about record 0 as well.
|
|
680
|
+
|
|
681
|
+
Two things fall out of reading the other records. A file marked EDF+C whose records say they
|
|
682
|
+
sit somewhere other than where continuity puts them is now reported rather than quietly
|
|
683
|
+
timed as contiguous. And a timekeeping entry is counted apart from the events, because
|
|
684
|
+
calling it an annotation that "could not be exported" was wrong twice: a file with one bad
|
|
685
|
+
timekeeping entry and three good events reported one entry lost while exporting all three,
|
|
686
|
+
and never mentioned the timing that had actually gone. The EDF+D path keeps its own
|
|
687
|
+
per-record message, which is more specific; saying both would report one problem twice.
|
|
688
|
+
|
|
689
|
+
## 0.4.40
|
|
690
|
+
|
|
691
|
+
### Fixed: a --stdout failure stops describing files it never wrote
|
|
692
|
+
|
|
693
|
+
```
|
|
694
|
+
error: Writing to stdout failed: ENOSPC: no space left on device, write
|
|
695
|
+
The files written so far are incomplete and should not be used. The destination
|
|
696
|
+
is out of space; free some up or choose another with --out.
|
|
697
|
+
```
|
|
698
|
+
|
|
699
|
+
Both halves of that hint were written for --out. There are no "files written so far" on this
|
|
700
|
+
path — the conversion writes to a stream the shell redirected — and --out is the flag whose
|
|
701
|
+
absence is the reason the message exists at all. Someone following it goes looking for a
|
|
702
|
+
partial directory that was never created, and is pointed at a mode they deliberately did not
|
|
703
|
+
use.
|
|
704
|
+
|
|
705
|
+
It now names what there actually is:
|
|
706
|
+
|
|
707
|
+
```
|
|
708
|
+
What reached stdout before it failed is incomplete and should not be used. The
|
|
709
|
+
destination is out of space; free some up or redirect it somewhere else.
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
Same class as the disk-space hint 0.4.36 replaced, one flag over: advice that fits one path
|
|
713
|
+
being given on another. The --out wording is untouched, and every errno keeps the sentence
|
|
714
|
+
0.4.36 gave it.
|
|
715
|
+
|
|
716
|
+
## 0.4.39
|
|
717
|
+
|
|
718
|
+
### Fixed: --stdout stops reporting success for bytes that never arrived
|
|
719
|
+
|
|
720
|
+
```
|
|
721
|
+
$ edf2csv long-stream.edf --stdout > /Volumes/small/sig.csv
|
|
722
|
+
Wrote 102,400 rows to stdout.
|
|
723
|
+
$ echo $?
|
|
724
|
+
0
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
94,977 rows on disk. The file ends mid-row at "371.00390625," with no trailing newline,
|
|
728
|
+
150,904 bytes short, and the line on stderr overstates the count by 7,423 rows. The same
|
|
729
|
+
recording onto the same volume through --out fails correctly — exit 1, ENOSPC named — which
|
|
730
|
+
is what gives it away.
|
|
731
|
+
|
|
732
|
+
POSIX write returns a short count rather than an error when the filesystem fills partway
|
|
733
|
+
through a single call; only the NEXT write raises ENOSPC. --out always has a next write,
|
|
734
|
+
since channels.csv and metadata.json come after the samples, so it always finds out.
|
|
735
|
+
--stdout has nothing after it. And when fd 1 is a regular file, Node's stdout is a
|
|
736
|
+
SyncWriteStream whose _write discards the count writeSync returns, so no error is raised at
|
|
737
|
+
all — which means checking the stream's recorded failure would not have caught it either.
|
|
738
|
+
|
|
739
|
+
What can be checked is the descriptor: how much it grew against how much it was handed. Only
|
|
740
|
+
for a regular file, since a pipe or a terminal has no size to compare and cannot lose a
|
|
741
|
+
write this way without saying so; appending with >> is fine, because the starting size is
|
|
742
|
+
taken first. Under --gzip the bytes counted are the compressor's output rather than the
|
|
743
|
+
CSV's, since those are what stdout is given.
|
|
744
|
+
|
|
745
|
+
Two guards keep it off the healthy paths: the audit declines anything that is not a regular
|
|
746
|
+
file, and it is skipped entirely when the reader hung up, because `--stdout | head -1` is a
|
|
747
|
+
shell idiom and not a failure. Reporting a failure for a command that worked would be worse
|
|
748
|
+
than the bug.
|
|
749
|
+
|
|
750
|
+
BufferedLineWriter.end() also returned early for stdout without consulting the failure the
|
|
751
|
+
stream's error listener had recorded, so an error not yet surfaced by a later flush was
|
|
752
|
+
dropped. It looks now.
|
|
753
|
+
|
|
754
|
+
This is a narrower window than "any full disk": when an earlier flush crosses the boundary a
|
|
755
|
+
later write does raise ENOSPC and that was always handled. It is the case where the output
|
|
756
|
+
very nearly fits.
|
|
757
|
+
|
|
758
|
+
## 0.4.38
|
|
759
|
+
|
|
760
|
+
### Fixed: the library stops blaming the destination for the caller's own failures
|
|
761
|
+
|
|
762
|
+
Two places where an answer came from the wrong layer.
|
|
763
|
+
|
|
764
|
+
`changedSinceOpen` returned false on a closed file. `convert` closes the file before it
|
|
765
|
+
returns, so `result.file.changedSinceOpen()` denied the very change the INPUT_CHANGED
|
|
766
|
+
diagnostic in the same result object had just reported — one object, two answers. False is
|
|
767
|
+
not something a closed descriptor can know. It now remembers the last answer it computed,
|
|
768
|
+
and `convert` always asks on the way out, so the result agrees with itself. A file closed
|
|
769
|
+
without ever being asked throws instead of guessing.
|
|
770
|
+
|
|
771
|
+
A progress callback that threw came back as
|
|
772
|
+
|
|
773
|
+
```
|
|
774
|
+
ConversionError WRITE_FAILED: Writing to "out" failed: caller bug
|
|
775
|
+
The files written so far are incomplete and should not be used. Check the destination
|
|
776
|
+
and run the conversion again.
|
|
777
|
+
```
|
|
778
|
+
|
|
779
|
+
The destination was working perfectly; the caller's own callback threw. onProgress ran
|
|
780
|
+
inside the same try that turns a stream failure into WRITE_FAILED, which is the same
|
|
781
|
+
misattribution the write hints carried until 0.4.36, one layer up. It now raises
|
|
782
|
+
CALLBACK_FAILED naming the callback, keeps the original as `cause` so the stack that matters
|
|
783
|
+
survives, and still stops the conversion — writing on into a directory whose owner has just
|
|
784
|
+
failed is not an improvement. A ConversionError that arrives at the outer catch already
|
|
785
|
+
saying what went wrong is passed through rather than wrapped a second time.
|
|
786
|
+
|
|
787
|
+
Found by an auditor looking at the library surface; a third report from the same pass — a
|
|
788
|
+
raw ENOENT escaping when the input is deleted mid-run — did not reproduce, since an open
|
|
789
|
+
descriptor outlives both unlink and rename.
|
|
790
|
+
|
|
791
|
+
## 0.4.37
|
|
792
|
+
|
|
793
|
+
### Fixed: a label made of control characters is reported instead of passed on in silence
|
|
794
|
+
|
|
795
|
+
NONPRINTABLE_LABEL has been declared in the source and documented as reserved since 0.1.
|
|
796
|
+
Nothing raised it.
|
|
797
|
+
|
|
798
|
+
`--info` has escaped control bytes since it was written, on the reasoning that an ANSI
|
|
799
|
+
escape in a header can drive the reader's terminal — \x1b[2J clears the screen, which is
|
|
800
|
+
enough to hide the rest of the output. The CSV had no such protection and needs none for
|
|
801
|
+
correctness: quoting makes any byte safe for a parser, and passing the label through exactly
|
|
802
|
+
as the file has it is the right call, since losing what the header says is not an
|
|
803
|
+
improvement.
|
|
804
|
+
|
|
805
|
+
What was missing is the sentence saying so. A recording whose channel is labelled
|
|
806
|
+
\x1b[2Jgone converted with no warning at all, and `cat signals.csv` then cleared the
|
|
807
|
+
terminal — while a script referencing that column by name carried an invisible control
|
|
808
|
+
character in it. The two halves of the tool disagreed about whether this was worth
|
|
809
|
+
mentioning.
|
|
810
|
+
|
|
811
|
+
Now every affected channel gets a warning naming the bytes, escaped, and saying to address
|
|
812
|
+
it by position with --channels "#N" since the name cannot be typed. Tab is included: it is
|
|
813
|
+
harmless to a terminal but it makes a column name nothing can match reliably. Nothing is
|
|
814
|
+
rewritten — the label still reaches the CSV as the file has it.
|
|
815
|
+
|
|
816
|
+
That leaves NONSTANDARD_UNIT as the only code still declared and never raised, and the
|
|
817
|
+
documentation now says so rather than saying two.
|
|
818
|
+
|
|
819
|
+
## 0.4.36
|
|
820
|
+
|
|
821
|
+
### Fixed: a failed write says what went wrong, not "free up space" every time
|
|
822
|
+
|
|
823
|
+
Every write failure carried one hint: "Free up space or choose another destination with
|
|
824
|
+
--out". That fits exactly one errno. A directory sitting where signals.csv belongs produced
|
|
825
|
+
|
|
826
|
+
```
|
|
827
|
+
error: Writing to "csv" failed: EISDIR: illegal operation on a directory, open "csv/signals.csv"
|
|
828
|
+
The files written so far are incomplete and should not be used. Free up space or
|
|
829
|
+
choose another destination with --out, then run the conversion again.
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
and so did a read-only volume, a permission denial, a path too long for the filesystem, and
|
|
833
|
+
running out of file descriptors. Wrong advice is worse than none: it sends someone to check
|
|
834
|
+
`df` on a disk that is fine, while the thing that actually failed stays unexamined.
|
|
835
|
+
|
|
836
|
+
The errno is the one part of the failure that names the cause, so it is what picks the
|
|
837
|
+
sentence now — out of space, over quota, no permission, read-only, a directory in the way,
|
|
838
|
+
a path that vanished, a name too long, too many open files (which a recording with many
|
|
839
|
+
sampling rates can reach, since it opens one output file per rate, so --channels narrows
|
|
840
|
+
it). Anything unrecognised keeps the general form rather than guessing.
|
|
841
|
+
|
|
842
|
+
What does not change is the part that matters: whatever the cause, the files written so far
|
|
843
|
+
are incomplete and must not be used.
|
|
844
|
+
|
|
845
|
+
## 0.4.35
|
|
846
|
+
|
|
847
|
+
### Fixed: a window that lands where there is no data says so
|
|
848
|
+
|
|
849
|
+
`--start` at or past the end of the recording is a usage error and stops the run. This is
|
|
850
|
+
the narrower case it leaves behind: a window that lies inside the recording but selects
|
|
851
|
+
nothing. Between the last sample and the nominal end of the last record —
|
|
852
|
+
|
|
853
|
+
```
|
|
854
|
+
edf2csv tiny.edf --start 1.95 --out csv
|
|
855
|
+
```
|
|
856
|
+
|
|
857
|
+
— or, on a discontinuous file whose records sit at 0s, 1s and 10s, anywhere in the eight
|
|
858
|
+
second gap:
|
|
859
|
+
|
|
860
|
+
```
|
|
861
|
+
edf2csv study.edf --start 2 --end 10 --out csv
|
|
862
|
+
```
|
|
863
|
+
|
|
864
|
+
Both produced a signals.csv holding its header and nothing else, exit 0, no warning, and
|
|
865
|
+
--strict passing. That is exactly what a successful extraction of an empty range looks like.
|
|
866
|
+
The summary does print "signals.csv 0 rows" and --json carries rows: 0, so it was not
|
|
867
|
+
invisible — but everywhere else that a request produces nothing, this tool says so: a
|
|
868
|
+
--channels term matching nothing is an error, and --annotations-only on a file with no
|
|
869
|
+
events raises NO_ANNOTATIONS.
|
|
870
|
+
|
|
871
|
+
EMPTY_WINDOW now covers it, quoting the window back and pointing at --info to see where the
|
|
872
|
+
records really sit. A warning rather than an error, because a batch of five hundred
|
|
873
|
+
recordings should not stop for the one whose gap lines up with the window; --strict makes it
|
|
874
|
+
a failure for those who want that.
|
|
875
|
+
|
|
876
|
+
The paths that write no signal table by design are left alone: --annotations-only, and a
|
|
877
|
+
file whose only channel carries annotations.
|
|
878
|
+
|
|
879
|
+
## 0.4.34
|
|
880
|
+
|
|
881
|
+
### Fixed: the correctness page states numbers this repository can produce
|
|
882
|
+
|
|
883
|
+
Five things had drifted apart from what the commands actually print.
|
|
884
|
+
|
|
885
|
+
```
|
|
886
|
+
"Three separate claims", followed by five numbered ones. The list grew as the batch and
|
|
887
|
+
fuzz harnesses were added and the heading did not.
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
```
|
|
891
|
+
"4,000 runs over 1,000 corrupted recordings" for `npm run fuzz`, which does 1,200 over 300
|
|
892
|
+
at its default seed. The larger figure is reachable — `npm run fuzz -- 42 2000` — so the
|
|
893
|
+
page now says which is the default and how to ask for more.
|
|
894
|
+
```
|
|
895
|
+
|
|
896
|
+
```
|
|
897
|
+
"ℹ tests 148" and a per-file table adding to 148, against a suite of 179. The table is
|
|
898
|
+
regenerated from what the files hold: 35, 59, 85.
|
|
899
|
+
```
|
|
900
|
+
|
|
901
|
+
```
|
|
902
|
+
129,536 sample values in the README's accuracy section, a number from a recording set that
|
|
903
|
+
is not in this repository. Both places now say 16,943 across the 75 recordings that are.
|
|
904
|
+
```
|
|
905
|
+
|
|
906
|
+
```
|
|
907
|
+
The README put `npm test` under the pyEDFlib claim, and `npm test` has never run that
|
|
908
|
+
check — it is deliberately kept out so the package can stay dependency-free. It now shows
|
|
909
|
+
`pip install pyedflib && npm run crossvalidate`, which is the command that does it.
|
|
910
|
+
```
|
|
911
|
+
|
|
912
|
+
A number nobody can reproduce is worse than no number, on a page whose subject is what has
|
|
913
|
+
actually been verified.
|
|
914
|
+
|
|
915
|
+
## 0.4.33
|
|
916
|
+
|
|
917
|
+
### Fixed: the library checks its options too, before anything is written
|
|
918
|
+
|
|
919
|
+
The command line has always rejected these values — `--decimals 1.5` is a usage error and
|
|
920
|
+
always has been — and the library did not, so the same value behaved differently depending
|
|
921
|
+
on how it arrived:
|
|
922
|
+
|
|
923
|
+
```
|
|
924
|
+
decimals: NaN resolved successfully, having written whole numbers into a column the
|
|
925
|
+
caller had asked for decimals in. No error, no warning, and output that
|
|
926
|
+
looks like a deliberate choice.
|
|
927
|
+
decimals: -1 came back as a bare RangeError from inside toFixed, naming nothing the
|
|
928
|
+
caller had written.
|
|
929
|
+
start: NaN created the output directory, wrote signals.csv, and then failed saying
|
|
930
|
+
the input was unreadable — a partial conversion, blamed on the file.
|
|
931
|
+
```
|
|
932
|
+
|
|
933
|
+
The first is the one that matters: a conversion that succeeds and is wrong.
|
|
934
|
+
|
|
935
|
+
`assertOptions` now runs at the top of buildPlan, which every path goes through before a
|
|
936
|
+
directory is created or a stream is opened, so a rejected option leaves nothing behind. It
|
|
937
|
+
throws `OptionError`, which the CLI already declared privately and now shares with the
|
|
938
|
+
library and exports, so a bad option is one error type whichever way it arrived — and the
|
|
939
|
+
CLI keeps exit 2 for it.
|
|
940
|
+
|
|
941
|
+
## 0.4.32
|
|
942
|
+
|
|
943
|
+
### Changed: the bit-for-bit cross-check actually compares bits
|
|
944
|
+
|
|
945
|
+
The correctness page has said, since it was written, that edf2csv's physical values match
|
|
946
|
+
pyEDFlib's "to the last bit ... not equal to within a tolerance, not numpy.allclose". It even
|
|
947
|
+
printed the method: dump the doubles through the API, compare the 64 bits.
|
|
948
|
+
|
|
949
|
+
`npm run crossvalidate` did something else. It converted with --decimals 20, parsed the cells
|
|
950
|
+
back into floats, and accepted anything within abs(reference) * 1e-9 — and skipped empty
|
|
951
|
+
cells without counting them as anything. That cannot be exact whatever the tolerance: a cell
|
|
952
|
+
is a rounded decimal rendering, so reading it back gives the nearest double to the printed
|
|
953
|
+
digits rather than the double that was computed. The page described a check that did not
|
|
954
|
+
exist.
|
|
955
|
+
|
|
956
|
+
The recipe the page prints is now checked in as test/crossvalidate/dump-doubles.mjs and is
|
|
957
|
+
what the checker runs, so the documented method and the executed one are the same code. Every
|
|
958
|
+
value is compared as its 64 bits, addressed by the signal's position rather than its label —
|
|
959
|
+
labels need not be unique, and matching on them could compare one channel against another's
|
|
960
|
+
samples.
|
|
961
|
+
|
|
962
|
+
Confirmed capable of failing before being trusted: flipping the lowest mantissa bit of every
|
|
963
|
+
scaled value is caught on the first sample of every recording, including differences no
|
|
964
|
+
decimal rendering shows — pyEDFlib -1.0 against edf2csv -1.0000000000000002, which the old
|
|
965
|
+
tolerance passed in silence.
|
|
966
|
+
|
|
967
|
+
```
|
|
968
|
+
Compared 16,943 sample values bit for bit, and 120 annotations, across 75 recordings.
|
|
969
|
+
Every value agreed.
|
|
970
|
+
```
|
|
971
|
+
|
|
972
|
+
The README and the correctness page said 129,536 values, a number from a recording set that
|
|
973
|
+
is not in the repository. Both now state what the shipped command actually compares, and the
|
|
974
|
+
README no longer files the pyEDFlib claim under `npm test`, which never ran it.
|
|
975
|
+
|
|
976
|
+
## 0.4.31
|
|
977
|
+
|
|
978
|
+
### Fixed: metadata describes the file that was converted, or admits it cannot
|
|
979
|
+
|
|
980
|
+
The size, timestamp and SHA-256 in metadata.json came from re-opening the input path once
|
|
981
|
+
the CSVs were written. That describes whatever answers to that name by then, which need not
|
|
982
|
+
be what was converted. A recording still being written grew from 2,000 records to 3,000
|
|
983
|
+
mid-conversion, and metadata.json recorded:
|
|
984
|
+
|
|
985
|
+
```
|
|
986
|
+
"bytes": 1536512, "sha256": "28535bb4...", <- the 3,000-record file
|
|
987
|
+
"data_records": 2000 <- what the CSV holds
|
|
988
|
+
```
|
|
989
|
+
|
|
990
|
+
Two halves of one provenance record describing two different files, with nothing to say so.
|
|
991
|
+
Replacing the file at that path did the same thing more thoroughly. That is the opposite of
|
|
992
|
+
what --checksum is for.
|
|
993
|
+
|
|
994
|
+
`bytes` and `modified` now come from the descriptor state at open — the same size every
|
|
995
|
+
record count and window in the output was derived from — so the record is internally
|
|
996
|
+
consistent whatever happens to the path.
|
|
997
|
+
|
|
998
|
+
The hash needed more than that. A file overwritten in place keeps its inode, so the open
|
|
999
|
+
descriptor sees the new bytes too and the converted ones are gone; no post-hoc hash can
|
|
1000
|
+
recover them. So it is taken before the first record is read, and published only if the file
|
|
1001
|
+
held still: if size or modification time moved at any point, `sha256` is null and the run
|
|
1002
|
+
raises INPUT_CHANGED saying why. A plausible hash of the wrong bytes is worse than none.
|
|
1003
|
+
`sha256` present now means the file demonstrably did not change while it was read.
|
|
1004
|
+
|
|
1005
|
+
The change is reported even without --checksum, since a conversion of a file that moved
|
|
1006
|
+
under it is worth knowing about on its own. The CSVs stay correct for the records that were
|
|
1007
|
+
read either way.
|
|
1008
|
+
|
|
1009
|
+
## 0.4.30
|
|
1010
|
+
|
|
1011
|
+
### Fixed: the folder above the recording stops choosing its name by accident too
|
|
1012
|
+
|
|
1013
|
+
0.4.29 settled which of two names for one recording the output is called after, and left the
|
|
1014
|
+
directory above it deciding the same question the old way. The walk was a stack popped from
|
|
1015
|
+
the back, so which of two names for one folder was visited first came down to the order
|
|
1016
|
+
readdir returned them — and the loser was skipped as already seen, taking its name out of
|
|
1017
|
+
the run.
|
|
1018
|
+
|
|
1019
|
+
```
|
|
1020
|
+
study/aaa-real/rec.edf
|
|
1021
|
+
study/zzz-alias -> study/aaa-real
|
|
1022
|
+
```
|
|
1023
|
+
|
|
1024
|
+
```
|
|
1025
|
+
edf2csv study --out ./out -> out/zzz-alias/rec/signals.csv
|
|
1026
|
+
```
|
|
1027
|
+
|
|
1028
|
+
The link's name, chosen by a hash order that differs between filesystems. So the release
|
|
1029
|
+
note for 0.4.29 was true of files and not of the folders holding them, which is the same
|
|
1030
|
+
defect one level up.
|
|
1031
|
+
|
|
1032
|
+
Breadth first now, with each directory's children entered real names before links and
|
|
1033
|
+
alphabetically within each, so the surviving name is a property of the tree: the shallowest,
|
|
1034
|
+
then the one that is not a link, then the first in sort order. Both orderings of the names
|
|
1035
|
+
are tested — aaa-real beside zzz-alias, and zzz-real beside aaa-alias — so passing cannot be
|
|
1036
|
+
the sort order agreeing with the answer by luck.
|
|
1037
|
+
|
|
1038
|
+
## 0.4.29
|
|
1039
|
+
|
|
1040
|
+
### Fixed: one recording, two names — the output directory stops depending on the order
|
|
1041
|
+
|
|
1042
|
+
A recording reachable more than one way is converted once, and the surviving name was
|
|
1043
|
+
whichever arrived first. That made the output directory a function of enumeration order:
|
|
1044
|
+
|
|
1045
|
+
```
|
|
1046
|
+
edf2csv data/one.edf data/alias.edf --out ./out -> out/one
|
|
1047
|
+
edf2csv data/alias.edf data/one.edf --out ./out -> out/alias
|
|
1048
|
+
```
|
|
1049
|
+
|
|
1050
|
+
A shell orders a glob however it likes. Inside a folder it was whatever readdir returned,
|
|
1051
|
+
which is hash order on APFS, a different hash on ext4, and creation order elsewhere — so
|
|
1052
|
+
copying a study to another machine could rename its output, and a script that read
|
|
1053
|
+
out/one/signals.csv found nothing there.
|
|
1054
|
+
|
|
1055
|
+
The winner is now decided by the names themselves. A name the recording actually has beats
|
|
1056
|
+
a link pointing at it, since that is the name the file has; two links are settled by the
|
|
1057
|
+
path that sorts first. Both are properties of the file set, so the answer does not move.
|
|
1058
|
+
|
|
1059
|
+
One behaviour change falls out of it: a recording reachable as `link.edf` and as
|
|
1060
|
+
`linkdir/actual.edf` now converts into `<out>/linkdir/actual` rather than `<out>/link`,
|
|
1061
|
+
which is also the layout the folder has.
|
|
1062
|
+
|
|
1063
|
+
## 0.4.28
|
|
1064
|
+
|
|
1065
|
+
### Changed: a request that cannot be carried out exits 2, like every other one
|
|
1066
|
+
|
|
1067
|
+
Exit 1 means "the file or the destination is the problem"; exit 2 means "the command line
|
|
1068
|
+
is the problem". Both --stdout refusals were filed under 1:
|
|
1069
|
+
|
|
1070
|
+
```
|
|
1071
|
+
error: --stdout has no signal data to write because --annotations-only was given.
|
|
1072
|
+
Drop one of the two flags.
|
|
1073
|
+
```
|
|
1074
|
+
|
|
1075
|
+
```
|
|
1076
|
+
error: --stdout needs exactly one table, but this recording produces 3, one for each
|
|
1077
|
+
sampling rate its channels use (256 Hz, 128 Hz, 1 Hz).
|
|
1078
|
+
Narrow it to one rate with --channels, or convert to a directory instead.
|
|
1079
|
+
```
|
|
1080
|
+
|
|
1081
|
+
Both hints say, in as many words, to change the flags — and a script reading the exit code
|
|
1082
|
+
went looking at the disk instead. `--stdout --json` was already 2 for exactly this reason,
|
|
1083
|
+
so the three conflicts disagreed with each other.
|
|
1084
|
+
|
|
1085
|
+
Exit 2 has always covered checks that need the header first: a --channels term matching
|
|
1086
|
+
nothing, a --start past the end of the recording. These belong with them. The library keeps
|
|
1087
|
+
throwing a ConversionError so nothing about its API changes, under a new code
|
|
1088
|
+
UNSUPPORTED_REQUEST that says which kind of problem it is; a destination that genuinely
|
|
1089
|
+
cannot be written is still exit 1.
|
|
1090
|
+
|
|
1091
|
+
## 0.4.27
|
|
1092
|
+
|
|
1093
|
+
### Changed: --json is shaped by what you named, and the JSON Lines batch is documented
|
|
1094
|
+
|
|
1095
|
+
Two things, one cause.
|
|
1096
|
+
|
|
1097
|
+
The batch flag was `inputs.length > 1`, so the shape of --json depended on the contents of a
|
|
1098
|
+
folder rather than on the command. A study holding one night printed an indented document;
|
|
1099
|
+
the same study holding two printed JSON Lines. A script written against one broke on the
|
|
1100
|
+
other — on the day a recording was added, not the day the script changed — and an input
|
|
1101
|
+
going missing did it in reverse. That is the same count 0.4.20 took out of --out, for the
|
|
1102
|
+
same reason, and it is now decided the same way: naming a folder means a batch whether it
|
|
1103
|
+
holds one recording or fifty.
|
|
1104
|
+
|
|
1105
|
+
And the documentation never mentioned JSON Lines. It said "the whole result of a successful
|
|
1106
|
+
run is one parseable document on stdout", which is true of one recording and false of a
|
|
1107
|
+
folder: `json.load` fails on the second line. One object per line as each recording finishes
|
|
1108
|
+
is the right shape for a batch — five hundred recordings can be consumed while the run is
|
|
1109
|
+
still going, and `jq` reads it a record at a time — so the docs now say so, and say what to
|
|
1110
|
+
use in Python.
|
|
1111
|
+
|
|
1112
|
+
## 0.4.26
|
|
1113
|
+
|
|
1114
|
+
### Fixed: the signal count is read the way every other header number is read
|
|
1115
|
+
|
|
1116
|
+
EdfFile.open has to know how many signals there are before it can know how much header to
|
|
1117
|
+
read, and it worked that out with its own Number(). That parse tolerated the NUL padding
|
|
1118
|
+
sloppy writers emit — the reason it existed — but not the comma decimal separator, which
|
|
1119
|
+
every other numeric field here accepts and which the documentation lists this field among.
|
|
1120
|
+
|
|
1121
|
+
So a header written with a comma never got its signal headers read at all, and the file
|
|
1122
|
+
died on a message whose arithmetic refuted itself:
|
|
1123
|
+
|
|
1124
|
+
```
|
|
1125
|
+
error: File declares 2 signals, which needs a 768-byte header, but the file is
|
|
1126
|
+
only 848 bytes.
|
|
1127
|
+
```
|
|
1128
|
+
|
|
1129
|
+
848 is larger than 768. Following that message leads to a truncation that is not there,
|
|
1130
|
+
while parseHeader, handed the same bytes directly, read the file correctly and raised
|
|
1131
|
+
COMMA_DECIMAL. The reader and the parser disagreed about which files were readable.
|
|
1132
|
+
|
|
1133
|
+
Both now share one parse, so they cannot drift apart again. The size error also names
|
|
1134
|
+
whichever of the two is actually short — the file, or the bytes the parser was handed.
|
|
1135
|
+
|
|
1136
|
+
## 0.4.25
|
|
1137
|
+
|
|
1138
|
+
### Fixed: the _ch suffix is checked against the file, not only against the label it fixes
|
|
1139
|
+
|
|
1140
|
+
Duplicated labels get a `_ch<index>` suffix, which is unique among the channels sharing that
|
|
1141
|
+
label — and nothing stopped it from landing on a label some other channel already had. EDF
|
|
1142
|
+
labels are free text and nothing enforces uniqueness, so a file carrying T8, T8 and a third
|
|
1143
|
+
channel genuinely labelled T8_ch0 is well-formed. It produced:
|
|
1144
|
+
|
|
1145
|
+
```
|
|
1146
|
+
time_s,T8_ch0,T8_ch1,T8_ch0
|
|
1147
|
+
```
|
|
1148
|
+
|
|
1149
|
+
Two columns, one name, and the warning beside it said the suffix kept them
|
|
1150
|
+
"distinguishable". channels.csv listed T8_ch0 against two signal indices, so the join it
|
|
1151
|
+
exists for could not resolve it either; metadata.json recorded the name twice; and a
|
|
1152
|
+
name-based lookup returns one of the two with nothing to say which. Exit 0 throughout.
|
|
1153
|
+
|
|
1154
|
+
Names are now made unique across the whole file: anything still shared after the first pass
|
|
1155
|
+
takes its own position too, which is unique by construction. And the channel that lost its
|
|
1156
|
+
own label to another's suffix is named, because its column is the one thing in the output
|
|
1157
|
+
that no longer matches the file:
|
|
1158
|
+
|
|
1159
|
+
```
|
|
1160
|
+
warning: Signal 2 is labelled "T8_ch0", which is also the column name another channel's
|
|
1161
|
+
"_ch" suffix produces, so its column is "T8_ch0_ch2".
|
|
1162
|
+
```
|
|
1163
|
+
|
|
1164
|
+
## 0.4.24
|
|
1165
|
+
|
|
1166
|
+
### Fixed: an origin the file's own arithmetic cannot hold is reported, not acted on
|
|
1167
|
+
|
|
1168
|
+
0.4.9 taught continuous recordings to honour the first timekeeping TAL, which is right, but
|
|
1169
|
+
it took the number at face value. A double spaces its values further apart the larger they
|
|
1170
|
+
get: at 1e16 the gap is two seconds, so `t + 1` is `t`. Past that point an origin stops
|
|
1171
|
+
being a position and becomes a wall.
|
|
1172
|
+
|
|
1173
|
+
Two silent failures came out of that. At 1e16 the collapse is partial, and the test for
|
|
1174
|
+
"does this record overlap the window" — `start + recordDuration > windowStart` — is false
|
|
1175
|
+
for every record that rounded onto its neighbour. A twelve-row recording wrote four rows,
|
|
1176
|
+
exit 0, no warning; the eight that vanished looked exactly like a file that never had them.
|
|
1177
|
+
|
|
1178
|
+
At 1e17 every record lands on one instant, so the recording measures zero seconds long, and
|
|
1179
|
+
the window resolver had no reason to suspect the recording rather than the request:
|
|
1180
|
+
|
|
1181
|
+
```
|
|
1182
|
+
error: --start 100000000000000000s is at or past the end of this 100000000000000000s
|
|
1183
|
+
recording.
|
|
1184
|
+
```
|
|
1185
|
+
|
|
1186
|
+
--start was never passed.
|
|
1187
|
+
|
|
1188
|
+
Both paths now check that the origin can still separate two consecutive samples of the
|
|
1189
|
+
fastest channel. When it cannot, the recording is timed from zero — what it did before
|
|
1190
|
+
0.4.9, and the only column that can hold distinct values at that magnitude — and says so,
|
|
1191
|
+
pointing at annotations.csv for the absolute onsets. An origin merely large is kept: at 1e15
|
|
1192
|
+
the gap is an eighth of a second, so a 4 Hz recording's quarter-second steps survive.
|
|
1193
|
+
|
|
1194
|
+
## 0.4.23
|
|
1195
|
+
|
|
1196
|
+
### Fixed: one offset cache for the whole conversion, not one per sampling rate
|
|
1197
|
+
|
|
1198
|
+
The time-column cache was capped at 2^20 offsets per rate group. A file may hold as many
|
|
1199
|
+
rate groups as it has channels, so nothing bounded the total.
|
|
1200
|
+
|
|
1201
|
+
Twelve channels at twelve rates just under the cap — a 25 MB file — peaked at 1.66 GB and
|
|
1202
|
+
took 36 seconds. A 92 MB file at a single rate, four times the data, peaks at 283 MB and
|
|
1203
|
+
finishes in a fraction of that. Twenty-four rates never finished at all: it spent two
|
|
1204
|
+
minutes swapping and was killed. A per-group limit is not a limit.
|
|
1205
|
+
|
|
1206
|
+
One budget for the conversion, spent in the order the groups ask. They are already sorted
|
|
1207
|
+
fastest-rate first, so the cache goes to the tables with the most rows to write and the
|
|
1208
|
+
ones that miss out are the ones that would have gained least from it — they format the
|
|
1209
|
+
same text the slow way.
|
|
1210
|
+
|
|
1211
|
+
```
|
|
1212
|
+
12 rates, 25 MB 1.66 GB / 36.0s -> 1.02 GB / 5.5s
|
|
1213
|
+
24 rates, 50 MB did not finish -> 1.20 GB / 8.5s
|
|
1214
|
+
```
|
|
1215
|
+
|
|
1216
|
+
A file with one rate group behaves exactly as it did: same budget, same cache, and 75
|
|
1217
|
+
fixture and flag combinations byte-identical to 0.4.22.
|
|
1218
|
+
|
|
1219
|
+
## 0.4.22
|
|
1220
|
+
|
|
1221
|
+
### Fixed: the time column stops writing "1e+21.000"
|
|
1222
|
+
|
|
1223
|
+
`fixed` has guarded value cells against the 1e21 cliff since 0.3.x, where toFixed switches
|
|
1224
|
+
to exponent notation. The time column lost that guard in 0.4.1, when the per-row toFixed
|
|
1225
|
+
was replaced by a cached decomposition: whole seconds plus printed fraction, concatenated.
|
|
1226
|
+
The concatenation is an implicit Number-to-String, which switches to exponent form at the
|
|
1227
|
+
same 1e21 — and the cached fraction is then glued onto the end of it:
|
|
1228
|
+
|
|
1229
|
+
```
|
|
1230
|
+
time_s,ch1
|
|
1231
|
+
750000000000000100000.000,0.300
|
|
1232
|
+
1e+21.000,0.400
|
|
1233
|
+
1.25e+21.000,0.500
|
|
1234
|
+
```
|
|
1235
|
+
|
|
1236
|
+
"1e+21.000" is not a number in any notation. pandas and R both read it as NaN, in a column
|
|
1237
|
+
whose every other cell is plain fixed-decimal, so a reader has no reason to look for it.
|
|
1238
|
+
|
|
1239
|
+
Reachable because EDF's record-duration field is 8 characters and exponent form fits, so a
|
|
1240
|
+
header may legitimately say 1e21; three records get there. One comparison per row hands
|
|
1241
|
+
those to the slow path, which already expands them with BigInt, and leaves the cache doing
|
|
1242
|
+
its job for the other twenty million.
|
|
1243
|
+
|
|
1244
|
+
## 0.4.21
|
|
1245
|
+
|
|
1246
|
+
### Fixed: a conversion killed mid-flight names itself, and the directory it left
|
|
1247
|
+
|
|
1248
|
+
A process that dies by signal exits with a null code and prints nothing on its way out.
|
|
1249
|
+
The parent read that as an ordinary failure with empty output, so a batch whose child was
|
|
1250
|
+
killed printed "Converted 1 of 2 recordings; 1 failed." and nothing else — not which
|
|
1251
|
+
recording, not why, and not that out/b held a 194 MB signals.csv cut off mid-row with no
|
|
1252
|
+
channels.csv beside it. Half a CSV opens in pandas exactly like a whole one.
|
|
1253
|
+
|
|
1254
|
+
The out-of-memory killer, a job scheduler's time limit and `kill` all arrive this way, and
|
|
1255
|
+
they arrive on the machines where batches are largest. The close handler now reads the
|
|
1256
|
+
signal it was given and says so, in the same words the interrupt handler uses for a run
|
|
1257
|
+
stopped from the keyboard:
|
|
1258
|
+
|
|
1259
|
+
```
|
|
1260
|
+
error: in/b.edf: stopped by SIGKILL before it finished.
|
|
1261
|
+
Incomplete, and should not be used: out/b
|
|
1262
|
+
```
|
|
1263
|
+
|
|
1264
|
+
Ctrl-C keeps its single message: the interrupt handler names every abandoned directory at
|
|
1265
|
+
once, and a per-child line under it would repeat that once per job.
|
|
1266
|
+
|
|
1267
|
+
## 0.4.20
|
|
1268
|
+
|
|
1269
|
+
### Changed: a link that leads nowhere is reported, and --out stops depending on what was found
|
|
1270
|
+
|
|
1271
|
+
A study kept as one folder per night, with one night linked to an external drive that
|
|
1272
|
+
happened not to be mounted, converted the nights that were there and said nothing about
|
|
1273
|
+
the one that was not. The walk only reported entries whose names ended in .edf or .bdf,
|
|
1274
|
+
and a directory carries no such name.
|
|
1275
|
+
|
|
1276
|
+
Losing that input made it worse than a silent omission. --out decided between "the output
|
|
1277
|
+
directory" and "a parent to fill" by counting the recordings, so dropping one left a single
|
|
1278
|
+
recording and moved the survivor as well: csv/signals.csv instead of csv/night-01/rec/.
|
|
1279
|
+
Whether a drive was mounted changed both what was converted and where it went, exit 0.
|
|
1280
|
+
|
|
1281
|
+
Two changes. The walk reports anything it cannot inspect, whatever it is called, and that
|
|
1282
|
+
counts against the run. And what --out means is now decided by what was named rather than
|
|
1283
|
+
by what was found: one recording names the output directory itself, a folder or several
|
|
1284
|
+
recordings name a parent. `edf2csv study --out csv` writes csv/night-01/rec/ whether the
|
|
1285
|
+
study holds one night or fifty — adding a second night no longer moves the first one's
|
|
1286
|
+
output, and neither does an input going missing.
|
|
1287
|
+
|
|
6
1288
|
## 0.4.19
|
|
7
1289
|
|
|
8
1290
|
### Fixed: the nesting guard could be stepped past by a sibling
|