@forzalabs/remora 2.0.0 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/index.js +455 -300
- package/package.json +1 -1
- package/workers/ExecutorWorker.js +193 -34
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,18 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
|
|
|
6
6
|
|
|
7
7
|
## Unreleased
|
|
8
8
|
|
|
9
|
+
## V 2.0.1 - 2026-09-01
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
|
|
13
|
+
- A configuration that fails its JSON schema now reports **the mistake** rather than every reason it wasn't something else. A `oneOf` — a consumer `transform`, one alternative per transformation — makes Ajv explain why all 27 alternatives were rejected, so a `"cast": "date"` typo came out as 82 complaints whose first three were `must be equal to one of the allowed values; must have required property 'multiply'; must NOT have additional properties`, naming a transformation the field never mentioned and burying the one line that mattered. The alternatives' rejections are now collapsed: what is reported is whatever an alternative said about a value *inside* the branch — the one that got far enough in to find the real mistake — and when none did, the branch itself, listing what was actually allowed there and what the field has instead. Messages also carry the offending **value** (`must be one of: string, number, datetime, boolean (got "date")`), the **file** the resource was read from, since two resources can carry the same `name` and the one to go and edit is a path, and the **name** of the array item the pointer lands in, so `/fields/2` reads as `/fields/2 (field "created_at")` instead of sending you to count array elements
|
|
14
|
+
- `remora compile` no longer prints a configuration error twice, once under `❌ Unexpected error during validation`. A resource that fails its schema is an expected outcome of `compile`, and is now reported in the same list as every other validation error
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- A debug-mode run (`REMORA_DEBUG_MODE=true`) could fail every chunk with `write after end`, reported as `Affirm failed: The worker pool ran 7 thread(s) at once but was sized 2`. The executor worker **closed** the file log sink at the end of every task, but a thread outlives its task under the run-long pool, and a closed winston logger still accepts lines and writes them into an ended stream — `ERR_STREAM_WRITE_AFTER_END`, thrown from a stream nothing listens to, which inside a worker thread kills the thread and fails the chunk it was running. The sink is now flushed rather than closed; closing it resets it, so a line logged after a close re-opens a fresh sink instead of ending the process; and pointing the sink at a file it already writes no longer builds a second transport for it, which a reused thread was doing once per task. A failing sink can no longer take the process with it either — winston re-emits transport failures, and an unhandled `error` on a stream is an uncaughtException
|
|
19
|
+
- A run that lost a worker thread now reports the **chunk error that killed it** instead of `The worker pool ran N thread(s) at once but was sized M`. The pool never oversubscribed: the thread tally counted spawns as concurrency, and workerpool drops a dead worker from its list synchronously while only firing its termination callback from a handler an already-exited thread never triggers — so every crashed-and-replaced thread inflated the count by one and the assertion fired ahead of the real error. Concurrency is now measured on the pool itself, and a run with dead threads says how many chunks failed and how many threads were spawned to serve them
|
|
20
|
+
|
|
9
21
|
## V 2.0.0 - 2026-09-01
|
|
10
22
|
|
|
11
23
|
**Upgrading from 1.x.** Four changes can alter what an existing project does. JSON schema validation now actually runs at load, so a configuration that was silently schema-invalid fails at `compile` — run `remora compile` (or `remora graph`, which lists every problem at once without failing) before upgrading. `filters[].sql` and the `"sql"` filter operator are gone; rewrite them as `rule` filters. Filters on `number`, `datetime` and `boolean` fields now match, so a consumer whose filters were silently matching nothing will start returning rows. And `remora run -l` is now the whole run's limit rather than each worker's, so a limited run returns the number of rows asked for instead of that number multiplied by the chunk count. The `mock` command is removed — use `synth`.
|