@t4h.framework/fs 0.9.5 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. package/.ai/skills/framework-fs-split/SKILL.md +2 -1
  2. package/CHANGELOG.md +31 -0
  3. package/README.md +11 -1
  4. package/dist/activities/FileSplitActivity.d.ts +23 -1
  5. package/dist/activities/FileSplitActivity.d.ts.map +1 -1
  6. package/dist/activities/FileSplitActivity.js +36 -4
  7. package/dist/activities/FileSplitActivity.js.map +1 -1
  8. package/dist/constants/operations.d.ts +1 -0
  9. package/dist/constants/operations.d.ts.map +1 -1
  10. package/dist/constants/operations.js +6 -1
  11. package/dist/constants/operations.js.map +1 -1
  12. package/dist/fs.d.ts +1 -1
  13. package/dist/fs.d.ts.map +1 -1
  14. package/dist/fs.js +1 -1
  15. package/dist/fs.js.map +1 -1
  16. package/dist/helpers/{byte-delimiter-scanner.d.ts → ByteDelimiterScanner.d.ts} +10 -3
  17. package/dist/helpers/ByteDelimiterScanner.d.ts.map +1 -0
  18. package/dist/helpers/{byte-delimiter-scanner.js → ByteDelimiterScanner.js} +30 -12
  19. package/dist/helpers/ByteDelimiterScanner.js.map +1 -0
  20. package/dist/helpers/ByteRecordFilter.d.ts +32 -0
  21. package/dist/helpers/ByteRecordFilter.d.ts.map +1 -0
  22. package/dist/helpers/ByteRecordFilter.js +89 -0
  23. package/dist/helpers/ByteRecordFilter.js.map +1 -0
  24. package/dist/tools/assert-where-serializable.d.ts +1 -1
  25. package/dist/tools/assert-where-serializable.d.ts.map +1 -1
  26. package/dist/tools/assert-where-serializable.js +7 -7
  27. package/dist/tools/assert-where-serializable.js.map +1 -1
  28. package/package.json +3 -3
  29. package/dist/helpers/byte-delimiter-scanner.d.ts.map +0 -1
  30. package/dist/helpers/byte-delimiter-scanner.js.map +0 -1
@@ -137,7 +137,8 @@ multi-byte encodings cannot skew the ranges a ranged download will use.
137
137
  | `every` | `1` | Cut only every N delimiter occurrences — groups N records per child. One million lines with `every: 10_000` start 100 children, each receiving a raw block (inner delimiters untouched) to iterate locally. `keep` arithmetic applies only at actual cut points. |
138
138
  | `trim` | none | `{ start?, end? }` **bytes** cut off each end of the file before scanning — a positional header/trailer. Byte-based on purpose: a record-start delimiter (`'\n10'`) fuses the trailer into the last record's slice, so a record-count skip would drop a valid record along with it. Trimmed content never becomes a slice nor counts toward `every`; fetch it with `read`. Positional layouts make the byte counts constants — getting them right is the caller's business. |
139
139
  | `skipEmpty` | `true` | Drops zero-length slices (consecutive delimiters, file starting/ending with one) so they never become child workflows. Turn off when slice index must align 1:1 with record position. "Empty" means zero bytes — a whitespace-only line is a regular slice. |
140
- | `encoding` | `'utf8'` | Converts the delimiter to bytes and tells the runtime how to hydrate each slice for the child (string when set on the slice). |
140
+ | `encoding` | `'utf8'` | Converts the delimiter to bytes and decodes each slice into the string the child receives. |
141
+ | `inlineMaxBytes` | `16_384` | A slice up to this many bytes is handed to the child as a string straight from the scan; a larger one as a reference to its byte range in the durable copy, read back where the child runs. Reading a few hundred bytes back costs a row lookup and a ranged object read per child, which is what a million-line split spends its CPU on. `0` hands every slice by range. Keep it well under the runtime's fan-out message budget (1 MiB by default). |
141
142
  | `timeoutInMs` | none | Max milliseconds to await the batch; on expiry the activity **rejects** (reason `timeout`) instead of suspending forever — the safety net for a child that never reaches a terminal state. |
142
143
 
143
144
  ---
package/CHANGELOG.md CHANGED
@@ -1,5 +1,36 @@
1
1
  # @t4h.framework/fs
2
2
 
3
+ ## 0.11.0
4
+
5
+ ### Minor Changes
6
+
7
+ - [#188](https://github.com/tech4humans-brasil/framework/pull/188) [`814b5e5`](https://github.com/tech4humans-brasil/framework/commit/814b5e5fd46eccb925041b695e03306e44b0432f) Thanks [@gusteycamargo](https://github.com/gusteycamargo)! - `split` accepts a `where` that drops junk records before any child workflow is
8
+ started.
9
+
10
+ The query is the same declarative MongoDB-style subset as `join`'s, plus
11
+ `$regex`/`$options`, tested against each record as a string after `trim`.
12
+ `every` groups only the kept records, and the durable copy is the filtered
13
+ file. Like `join`, a function or `RegExp` is rejected in `toInput`, since the
14
+ options are stored in the history.
15
+
16
+ ## 0.10.0
17
+
18
+ ### Minor Changes
19
+
20
+ - [#181](https://github.com/tech4humans-brasil/framework/pull/181) [`9449e4e`](https://github.com/tech4humans-brasil/framework/commit/9449e4e9889f42dea1d52418140f47d25dff0535) Thanks [@gusteycamargo](https://github.com/gusteycamargo)! - `split` hands a slice to its child as a string when it fits `inlineMaxBytes`
21
+ (default 16 KiB) instead of a range read back from the durable copy.
22
+
23
+ Every child paid a lookup and a ranged read to fetch a few hundred bytes, so a
24
+ large file cost as many object reads as it has lines. The scanner keeps the
25
+ bytes of the open slice while it is under the limit, so memory stays bounded
26
+ by `inlineMaxBytes` whatever the file size. A larger slice still travels by
27
+ range; `inlineMaxBytes: 0` restores the previous behavior.
28
+
29
+ ### Patch Changes
30
+
31
+ - Updated dependencies [[`9449e4e`](https://github.com/tech4humans-brasil/framework/commit/9449e4e9889f42dea1d52418140f47d25dff0535), [`9449e4e`](https://github.com/tech4humans-brasil/framework/commit/9449e4e9889f42dea1d52418140f47d25dff0535)]:
32
+ - @t4h.framework/core@0.15.0
33
+
3
34
  ## 0.9.5
4
35
 
5
36
  ### Patch Changes
package/README.md CHANGED
@@ -223,6 +223,7 @@ await split(file, workflow, {
223
223
  trim: { start: 241, end: 241 }, // bytes cut off each end before scanning
224
224
  skipEmpty: false, // keep zero-length slices (default: skipped)
225
225
  encoding: 'latin1', // delimiter bytes + slice hydration (default: utf8)
226
+ inlineMaxBytes: 16_384, // slices up to this size travel as strings (default: 16 KiB)
226
227
  timeoutInMs: 3_600_000, // max ms to await the batch; rejects on expiry
227
228
  })
228
229
  ```
@@ -256,6 +257,15 @@ await split(file, workflow, {
256
257
  - **`encoding`** converts the delimiter to bytes and tells the runtime how to
257
258
  hydrate each slice for the child. Offsets are always computed over raw
258
259
  bytes, so multi-byte encodings never skew the ranges.
260
+ - **`inlineMaxBytes`** (default 16 KiB) is the size up to which a slice is
261
+ handed to the child as a string, straight from the scan. A slice above it is
262
+ handed as a reference to its byte range in the durable copy, read back
263
+ where the child runs. A line of a CSV is a few hundred bytes: reading it
264
+ back would cost a row lookup and a ranged object read per child, and that,
265
+ not the scan, is what a million-line split spends its CPU on. Set it to `0`
266
+ to hand every slice by range. Keep it well under the fan-out message budget
267
+ of the runtime (`WORKFLOW_BATCH_FANOUT_CHUNK_MAX_BYTES`, 1 MiB by default):
268
+ an inline slice that does not fit a message is refused.
259
269
  - **`timeoutInMs`** is the maximum time in milliseconds to await the batch. On
260
270
  expiry the activity rejects (reason `timeout`) instead of suspending
261
271
  forever — the safety net for a child that never reaches a terminal state.
@@ -380,5 +390,5 @@ const scanner = new ByteDelimiterScanner({ delimiter: { value: '\n' } })
380
390
 
381
391
  await pipeline(source, scanner, destination)
382
392
 
383
- scanner.ranges // [{ start: 0, end: 3 }, ...]
393
+ scanner.slices // [{ start: 0, end: 3, content: <Buffer> }, ...]
384
394
  ```
@@ -1,11 +1,31 @@
1
1
  import { AsyncActivity, type Binary, type Workflow, WorkflowBatchRef, workflowBatchSchema } from '@t4h.framework/core';
2
- import { type ByteTrim, type FileSplitDelimiter } from '../helpers/byte-delimiter-scanner.js';
2
+ import { type ByteTrim, type FileSplitDelimiter } from '../helpers/ByteDelimiterScanner.js';
3
+ type FileSplitWhereOperators = {
4
+ $eq?: string;
5
+ $ne?: string;
6
+ $lt?: string;
7
+ $lte?: string;
8
+ $gt?: string;
9
+ $gte?: string;
10
+ $in?: string[];
11
+ $nin?: string[];
12
+ $regex?: string;
13
+ $options?: string;
14
+ $not?: FileSplitWhere;
15
+ };
16
+ export type FileSplitWhere = string | (FileSplitWhereOperators & {
17
+ $and?: FileSplitWhere[];
18
+ $or?: FileSplitWhere[];
19
+ $nor?: FileSplitWhere[];
20
+ });
3
21
  export type FileSplitOptions = {
4
22
  delimiter: FileSplitDelimiter;
5
23
  every?: number;
24
+ where?: FileSplitWhere;
6
25
  trim?: ByteTrim;
7
26
  skipEmpty?: boolean;
8
27
  encoding?: BufferEncoding;
28
+ inlineMaxBytes?: number;
9
29
  timeoutInMs?: number;
10
30
  };
11
31
  export type FileSplitActivityInput = {
@@ -28,6 +48,7 @@ export declare class FileSplitActivity extends AsyncActivity<readonly [
28
48
  batch: import("typebox").TString;
29
49
  count: import("typebox").TNumber;
30
50
  }>> | import("@t4h.framework/core").AsyncActivityResultFulfilled<FileSplitBatch | undefined>>;
51
+ private toFilter;
31
52
  advance(payload: FileSplitBatch): import("@t4h.framework/core").AsyncActivityResultFulfilled<FileSplitBatch | undefined>;
32
53
  toOutput(output: FileSplitBatch): WorkflowBatchRef;
33
54
  }
@@ -36,4 +57,5 @@ export declare namespace Split {
36
57
  const line: (value?: string) => FileSplitDelimiter;
37
58
  const char: (value: string, keep?: FileSplitDelimiter["keep"]) => FileSplitDelimiter;
38
59
  }
60
+ export {};
39
61
  //# sourceMappingURL=FileSplitActivity.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"FileSplitActivity.d.ts","sourceRoot":"","sources":["../../src/activities/FileSplitActivity.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,aAAa,EACb,KAAK,MAAM,EAGX,KAAK,QAAQ,EACb,gBAAgB,EAEhB,mBAAmB,EACpB,MAAM,qBAAqB,CAAA;AAG5B,OAAO,EAEL,KAAK,QAAQ,EACb,KAAK,kBAAkB,EACxB,MAAM,sCAAsC,CAAA;AAE7C,MAAM,MAAM,gBAAgB,GAAG;IAC7B,SAAS,EAAE,kBAAkB,CAAA;IAC7B,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,IAAI,CAAC,EAAE,QAAQ,CAAA;IACf,SAAS,CAAC,EAAE,OAAO,CAAA;IACnB,QAAQ,CAAC,EAAE,cAAc,CAAA;IACzB,WAAW,CAAC,EAAE,MAAM,CAAA;CACrB,CAAA;AAED,MAAM,MAAM,sBAAsB,GAAG;IACnC,MAAM,EAAE,MAAM,CAAA;IACd,QAAQ,EAAE,MAAM,CAAA;IAChB,OAAO,EAAE,gBAAgB,CAAA;CAC1B,CAAA;AAED,MAAM,MAAM,cAAc,GAAG;IAC3B,KAAK,EAAE,MAAM,CAAA;IACb,KAAK,EAAE,MAAM,CAAA;CACd,CAAA;AAED,qBAAa,iBAAkB,SAAQ,aAAa,CAClD,SAAS;IACP,MAAM,EAAE,MAAM;IACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC;IAC5B,OAAO,EAAE,gBAAgB;CAC1B,EACD,sBAAsB,EACtB,cAAc,EACd,gBAAgB,EAChB,sBAAsB,EACtB,OAAO,mBAAmB,CAC3B;IACC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAGrB;IAEK,OAAO,CACZ,MAAM,EAAE,MAAM,EACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC,EAC5B,OAAO,EAAE,gBAAgB,GACxB,sBAAsB;IASZ,SAAS,CAAC,EACrB,MAAM,EACN,QAAQ,EACR,OAAO,GACR,EAAE,sBAAsB;;;;IAoClB,OAAO,CAAC,OAAO,EAAE,cAAc;IAI/B,QAAQ,CAAC,MAAM,EAAE,cAAc,GAAG,gBAAgB;CAG1D;AAED,wBAAsB,KAAK,CACzB,MAAM,EAAE,MAAM,EACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC,EAC5B,OAAO,EAAE,gBAAgB,GACxB,OAAO,CAAC,gBAAgB,CAAC,CAS3B;AAED,yBAAiB,KAAK,CAAC;IACd,MAAM,IAAI,GAAI,cAAY,KAAG,kBAAiC,CAAA;IAE9D,MAAM,IAAI,GACf,OAAO,MAAM,EACb,OAAO,kBAAkB,CAAC,MAAM,CAAC,KAChC,kBAAuC,CAAA;CAC3C"}
1
+ {"version":3,"file":"FileSplitActivity.d.ts","sourceRoot":"","sources":["../../src/activities/FileSplitActivity.ts"],"names":[],"mappings":"AAEA,OAAO,EACL,aAAa,EACb,KAAK,MAAM,EAGX,KAAK,QAAQ,EACb,gBAAgB,EAEhB,mBAAmB,EACpB,MAAM,qBAAqB,CAAA;AAM5B,OAAO,EAEL,KAAK,QAAQ,EACb,KAAK,kBAAkB,EACxB,MAAM,oCAAoC,CAAA;AAM3C,KAAK,uBAAuB,GAAG;IAC7B,GAAG,CAAC,EAAE,MAAM,CAAA;IACZ,GAAG,CAAC,EAAE,MAAM,CAAA;IACZ,GAAG,CAAC,EAAE,MAAM,CAAA;IACZ,IAAI,CAAC,EAAE,MAAM,CAAA;IACb,GAAG,CAAC,EAAE,MAAM,CAAA;IACZ,IAAI,CAAC,EAAE,MAAM,CAAA;IACb,GAAG,CAAC,EAAE,MAAM,EAAE,CAAA;IACd,IAAI,CAAC,EAAE,MAAM,EAAE,CAAA;IACf,MAAM,CAAC,EAAE,MAAM,CAAA;IACf,QAAQ,CAAC,EAAE,MAAM,CAAA;IACjB,IAAI,CAAC,EAAE,cAAc,CAAA;CACtB,CAAA;AAED,MAAM,MAAM,cAAc,GACtB,MAAM,GACN,CAAC,uBAAuB,GAAG;IACzB,IAAI,CAAC,EAAE,cAAc,EAAE,CAAA;IACvB,GAAG,CAAC,EAAE,cAAc,EAAE,CAAA;IACtB,IAAI,CAAC,EAAE,cAAc,EAAE,CAAA;CACxB,CAAC,CAAA;AAEN,MAAM,MAAM,gBAAgB,GAAG;IAC7B,SAAS,EAAE,kBAAkB,CAAA;IAC7B,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,KAAK,CAAC,EAAE,cAAc,CAAA;IACtB,IAAI,CAAC,EAAE,QAAQ,CAAA;IACf,SAAS,CAAC,EAAE,OAAO,CAAA;IACnB,QAAQ,CAAC,EAAE,cAAc,CAAA;IACzB,cAAc,CAAC,EAAE,MAAM,CAAA;IACvB,WAAW,CAAC,EAAE,MAAM,CAAA;CACrB,CAAA;AAED,MAAM,MAAM,sBAAsB,GAAG;IACnC,MAAM,EAAE,MAAM,CAAA;IACd,QAAQ,EAAE,MAAM,CAAA;IAChB,OAAO,EAAE,gBAAgB,CAAA;CAC1B,CAAA;AAED,MAAM,MAAM,cAAc,GAAG;IAC3B,KAAK,EAAE,MAAM,CAAA;IACb,KAAK,EAAE,MAAM,CAAA;CACd,CAAA;AAED,qBAAa,iBAAkB,SAAQ,aAAa,CAClD,SAAS;IACP,MAAM,EAAE,MAAM;IACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC;IAC5B,OAAO,EAAE,gBAAgB;CAC1B,EACD,sBAAsB,EACtB,cAAc,EACd,gBAAgB,EAChB,sBAAsB,EACtB,OAAO,mBAAmB,CAC3B;IACC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAGrB;IAEK,OAAO,CACZ,MAAM,EAAE,MAAM,EACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC,EAC5B,OAAO,EAAE,gBAAgB,GACxB,sBAAsB;IAsBZ,SAAS,CAAC,EACrB,MAAM,EACN,QAAQ,EACR,OAAO,GACR,EAAE,sBAAsB;;;;IA6CzB,OAAO,CAAC,QAAQ;IAkBT,OAAO,CAAC,OAAO,EAAE,cAAc;IAI/B,QAAQ,CAAC,MAAM,EAAE,cAAc,GAAG,gBAAgB;CAG1D;AAED,wBAAsB,KAAK,CACzB,MAAM,EAAE,MAAM,EACd,QAAQ,EAAE,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC,EAC5B,OAAO,EAAE,gBAAgB,GACxB,OAAO,CAAC,gBAAgB,CAAC,CAS3B;AAED,yBAAiB,KAAK,CAAC;IACd,MAAM,IAAI,GAAI,cAAY,KAAG,kBAAiC,CAAA;IAE9D,MAAM,IAAI,GACf,OAAO,MAAM,EACb,OAAO,kBAAkB,CAAC,MAAM,CAAC,KAChC,kBAAuC,CAAA;CAC3C"}
@@ -1,6 +1,12 @@
1
+ import { pipeline } from 'node:stream';
1
2
  import { AsyncActivity, Claim, History, WorkflowBatchRef, WorkflowClaim, workflowBatchSchema, } from '@t4h.framework/core';
3
+ import sift from 'sift';
4
+ import { STRING_OPERATIONS } from '../constants/operations.js';
2
5
  import { FileSystemClaim } from '../FileSystemClaim.js';
3
- import { ByteDelimiterScanner, } from '../helpers/byte-delimiter-scanner.js';
6
+ import { ByteDelimiterScanner, } from '../helpers/ByteDelimiterScanner.js';
7
+ import { ByteRecordFilter } from '../helpers/ByteRecordFilter.js';
8
+ import { assertWhereSerializable } from '../tools/assert-where-serializable.js';
9
+ const { createQueryTester } = sift;
4
10
  export class FileSplitActivity extends AsyncActivity {
5
11
  claims = new Claim({
6
12
  fs: FileSystemClaim,
@@ -9,20 +15,33 @@ export class FileSplitActivity extends AsyncActivity {
9
15
  toInput(binary, workflow, options) {
10
16
  if (!workflow.id)
11
17
  throw new TypeError('Could not split a file into a workflow without an id');
18
+ if (options.where !== undefined) {
19
+ assertWhereSerializable(options.where, 'where', 'Split where', STRING_OPERATIONS);
20
+ createQueryTester(options.where, {
21
+ operations: STRING_OPERATIONS,
22
+ });
23
+ }
12
24
  return { binary, workflow: workflow.id, options };
13
25
  }
14
26
  async bootstrap({ binary, workflow, options, }) {
15
27
  const encoding = options.encoding ?? 'utf8';
28
+ const filter = this.toFilter(options.where, options);
16
29
  const scanner = new ByteDelimiterScanner({
17
30
  delimiter: options.delimiter,
18
31
  every: options.every,
19
- trim: options.trim,
32
+ trim: filter ? undefined : options.trim,
20
33
  skipEmpty: options.skipEmpty,
21
34
  encoding,
35
+ inlineMaxBytes: options.inlineMaxBytes,
22
36
  });
23
37
  const source = await binary.stream();
24
- const durable = await this.claims.fs.write(`split/${this.id}`, source.pipe(scanner));
25
- const slices = await Promise.all(scanner.ranges.map(range => this.claims.fs.slice(durable, range, encoding)));
38
+ const durable = await this.claims.fs.write(`split/${this.id}`, filter
39
+ ? pipeline(source, filter, scanner, () => { })
40
+ : source.pipe(scanner));
41
+ const kept = filter && filter.accepted === 0 ? [] : scanner.slices;
42
+ const slices = await Promise.all(kept.map(({ start, end, content }) => content
43
+ ? content.toString(encoding)
44
+ : this.claims.fs.slice(durable, { start, end }, encoding)));
26
45
  const batch = await this.claims.workflow.startMany(workflow, slices);
27
46
  if (batch.count === 0)
28
47
  return this.done({ batch: batch.id, count: 0 });
@@ -33,6 +52,19 @@ export class FileSplitActivity extends AsyncActivity {
33
52
  }),
34
53
  });
35
54
  }
55
+ toFilter(where, { delimiter, trim, encoding = 'utf8' }) {
56
+ if (!where)
57
+ return;
58
+ const matches = createQueryTester(where, {
59
+ operations: STRING_OPERATIONS,
60
+ });
61
+ return new ByteRecordFilter({
62
+ delimiter,
63
+ trim,
64
+ encoding,
65
+ accept: record => matches(record.toString(encoding)),
66
+ });
67
+ }
36
68
  advance(payload) {
37
69
  return this.done(payload);
38
70
  }
@@ -1 +1 @@
1
- {"version":3,"file":"FileSplitActivity.js","sourceRoot":"","sources":["../../src/activities/FileSplitActivity.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,aAAa,EAEb,KAAK,EACL,OAAO,EAEP,gBAAgB,EAChB,aAAa,EACb,mBAAmB,GACpB,MAAM,qBAAqB,CAAA;AAE5B,OAAO,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAA;AACvD,OAAO,EACL,oBAAoB,GAGrB,MAAM,sCAAsC,CAAA;AAsB7C,MAAM,OAAO,iBAAkB,SAAQ,aAWtC;IACkB,MAAM,GAAG,IAAI,KAAK,CAAC;QAClC,EAAE,EAAE,eAAe;QACnB,QAAQ,EAAE,aAAa;KACxB,CAAC,CAAA;IAEK,OAAO,CACZ,MAAc,EACd,QAA4B,EAC5B,OAAyB;QAEzB,IAAI,CAAC,QAAQ,CAAC,EAAE;YACd,MAAM,IAAI,SAAS,CACjB,sDAAsD,CACvD,CAAA;QAEH,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,EAAE,OAAO,EAAE,CAAA;IACnD,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,EACrB,MAAM,EACN,QAAQ,EACR,OAAO,GACgB;QACvB,MAAM,QAAQ,GAAG,OAAO,CAAC,QAAQ,IAAI,MAAM,CAAA;QAE3C,MAAM,OAAO,GAAG,IAAI,oBAAoB,CAAC;YACvC,SAAS,EAAE,OAAO,CAAC,SAAS;YAC5B,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,IAAI,EAAE,OAAO,CAAC,IAAI;YAClB,SAAS,EAAE,OAAO,CAAC,SAAS;YAC5B,QAAQ;SACT,CAAC,CAAA;QAEF,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,MAAM,EAAE,CAAA;QAEpC,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,CACxC,SAAS,IAAI,CAAC,EAAE,EAAE,EAClB,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CACrB,CAAA;QAED,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,GAAG,CAC9B,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CACzB,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,CAAC,OAAO,EAAE,KAAK,EAAE,QAAQ,CAAC,CAC/C,CACF,CAAA;QAED,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAA;QAEpE,IAAI,KAAK,CAAC,KAAK,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,CAAC,CAAA;QAEtE,OAAO,IAAI,CAAC,OAAO,CAAC;YAClB,MAAM,EAAE,mBAAmB;YAC3B,GAAG,CAAC,OAAO,CAAC,WAAW,IAAI;gBACzB,SAAS,EAAE,IAAI,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC,WAAW,CAAC;aACtD,CAAC;SACH,CAAC,CAAA;IACJ,CAAC;IAEM,OAAO,CAAC,OAAuB;QACpC,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;IAEM,QAAQ,CAAC,MAAsB;QACpC,OAAO,IAAI,gBAAgB,CAAC,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,KAAK,CAAC,CAAA;IACzD,CAAC;CACF;AAED,MAAM,CAAC,KAAK,UAAU,KAAK,CACzB,MAAc,EACd,QAA4B,EAC5B,OAAyB;IAEzB,MAAM,SAAS,GAAG,MAAM,OAAO,CAAC,UAAU,CACxC,iBAAiB,EACjB,MAAM,EACN,QAAQ,EACR,OAAO,CACR,CAAA;IAED,OAAO,MAAM,SAAS,CAAC,IAAI,EAAE,CAAA;AAC/B,CAAC;AAED,MAAM,KAAW,KAAK,CAOrB;AAPD,WAAiB,KAAK;IACP,UAAI,GAAG,CAAC,KAAK,GAAG,IAAI,EAAsB,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAA;IAExD,UAAI,GAAG,CAClB,KAAa,EACb,IAAiC,EACb,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAA;AAC5C,CAAC,EAPgB,KAAK,KAAL,KAAK,QAOrB"}
1
+ {"version":3,"file":"FileSplitActivity.js","sourceRoot":"","sources":["../../src/activities/FileSplitActivity.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAA;AAEtC,OAAO,EACL,aAAa,EAEb,KAAK,EACL,OAAO,EAEP,gBAAgB,EAChB,aAAa,EACb,mBAAmB,GACpB,MAAM,qBAAqB,CAAA;AAC5B,OAAO,IAAI,MAAM,MAAM,CAAA;AAGvB,OAAO,EAAE,iBAAiB,EAAE,MAAM,4BAA4B,CAAA;AAC9D,OAAO,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAA;AACvD,OAAO,EACL,oBAAoB,GAGrB,MAAM,oCAAoC,CAAA;AAC3C,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAA;AACjE,OAAO,EAAE,uBAAuB,EAAE,MAAM,uCAAuC,CAAA;AAE/E,MAAM,EAAE,iBAAiB,EAAE,GAAG,IAAI,CAAA;AA8ClC,MAAM,OAAO,iBAAkB,SAAQ,aAWtC;IACkB,MAAM,GAAG,IAAI,KAAK,CAAC;QAClC,EAAE,EAAE,eAAe;QACnB,QAAQ,EAAE,aAAa;KACxB,CAAC,CAAA;IAEK,OAAO,CACZ,MAAc,EACd,QAA4B,EAC5B,OAAyB;QAEzB,IAAI,CAAC,QAAQ,CAAC,EAAE;YACd,MAAM,IAAI,SAAS,CACjB,sDAAsD,CACvD,CAAA;QAEH,IAAI,OAAO,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;YAChC,uBAAuB,CACrB,OAAO,CAAC,KAAK,EACb,OAAO,EACP,aAAa,EACb,iBAAiB,CAClB,CAAA;YAED,iBAAiB,CAAC,OAAO,CAAC,KAAsB,EAAE;gBAChD,UAAU,EAAE,iBAAiB;aAC9B,CAAC,CAAA;QACJ,CAAC;QAED,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,EAAE,OAAO,EAAE,CAAA;IACnD,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,EACrB,MAAM,EACN,QAAQ,EACR,OAAO,GACgB;QACvB,MAAM,QAAQ,GAAG,OAAO,CAAC,QAAQ,IAAI,MAAM,CAAA;QAE3C,MAAM,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,CAAA;QAEpD,MAAM,OAAO,GAAG,IAAI,oBAAoB,CAAC;YACvC,SAAS,EAAE,OAAO,CAAC,SAAS;YAC5B,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI;YACvC,SAAS,EAAE,OAAO,CAAC,SAAS;YAC5B,QAAQ;YACR,cAAc,EAAE,OAAO,CAAC,cAAc;SACvC,CAAC,CAAA;QAEF,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,MAAM,EAAE,CAAA;QAEpC,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,CACxC,SAAS,IAAI,CAAC,EAAE,EAAE,EAClB,MAAM;YACJ,CAAC,CAAC,QAAQ,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,GAAG,EAAE,GAAE,CAAC,CAAC;YAC7C,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CACzB,CAAA;QAED,MAAM,IAAI,GAAG,MAAM,IAAI,MAAM,CAAC,QAAQ,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,CAAA;QAElE,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,GAAG,CAC9B,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,EAAE,EAAE,CACnC,OAAO;YACL,CAAC,CAAC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC;YAC5B,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,CAAC,OAAO,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,EAAE,QAAQ,CAAC,CAC5D,CACF,CAAA;QAED,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAA;QAEpE,IAAI,KAAK,CAAC,KAAK,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,CAAC,CAAA;QAEtE,OAAO,IAAI,CAAC,OAAO,CAAC;YAClB,MAAM,EAAE,mBAAmB;YAC3B,GAAG,CAAC,OAAO,CAAC,WAAW,IAAI;gBACzB,SAAS,EAAE,IAAI,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC,WAAW,CAAC;aACtD,CAAC;SACH,CAAC,CAAA;IACJ,CAAC;IAEO,QAAQ,CACd,KAAiC,EACjC,EAAE,SAAS,EAAE,IAAI,EAAE,QAAQ,GAAG,MAAM,EAAoB;QAExD,IAAI,CAAC,KAAK;YAAE,OAAM;QAElB,MAAM,OAAO,GAAG,iBAAiB,CAAC,KAAsB,EAAE;YACxD,UAAU,EAAE,iBAAiB;SAC9B,CAAC,CAAA;QAEF,OAAO,IAAI,gBAAgB,CAAC;YAC1B,SAAS;YACT,IAAI;YACJ,QAAQ;YACR,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;SACrD,CAAC,CAAA;IACJ,CAAC;IAEM,OAAO,CAAC,OAAuB;QACpC,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;IAEM,QAAQ,CAAC,MAAsB;QACpC,OAAO,IAAI,gBAAgB,CAAC,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,KAAK,CAAC,CAAA;IACzD,CAAC;CACF;AAED,MAAM,CAAC,KAAK,UAAU,KAAK,CACzB,MAAc,EACd,QAA4B,EAC5B,OAAyB;IAEzB,MAAM,SAAS,GAAG,MAAM,OAAO,CAAC,UAAU,CACxC,iBAAiB,EACjB,MAAM,EACN,QAAQ,EACR,OAAO,CACR,CAAA;IAED,OAAO,MAAM,SAAS,CAAC,IAAI,EAAE,CAAA;AAC/B,CAAC;AAED,MAAM,KAAW,KAAK,CAOrB;AAPD,WAAiB,KAAK;IACP,UAAI,GAAG,CAAC,KAAK,GAAG,IAAI,EAAsB,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAA;IAExD,UAAI,GAAG,CAClB,KAAa,EACb,IAAiC,EACb,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAA;AAC5C,CAAC,EAPgB,KAAK,KAAL,KAAK,QAOrB"}
@@ -1,3 +1,4 @@
1
1
  import type { OperationCreator } from 'sift/lib/core.js';
2
2
  export declare const OPERATIONS: Record<string, OperationCreator<any>>;
3
+ export declare const STRING_OPERATIONS: Record<string, OperationCreator<any>>;
3
4
  //# sourceMappingURL=operations.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"operations.d.ts","sourceRoot":"","sources":["../../src/constants/operations.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAA;AAkBxD,eAAO,MAAM,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,gBAAgB,CAAC,GAAG,CAAC,CAc5D,CAAA"}
1
+ {"version":3,"file":"operations.d.ts","sourceRoot":"","sources":["../../src/constants/operations.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAA;AAoBxD,eAAO,MAAM,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,gBAAgB,CAAC,GAAG,CAAC,CAc5D,CAAA;AAED,eAAO,MAAM,iBAAiB,EAAE,MAAM,CAAC,MAAM,EAAE,gBAAgB,CAAC,GAAG,CAAC,CAInE,CAAA"}
@@ -1,5 +1,5 @@
1
1
  import sift from 'sift';
2
- const { $and, $eq, $exists, $gt, $gte, $in, $lt, $lte, $ne, $nin, $nor, $not, $or, } = sift;
2
+ const { $and, $eq, $exists, $gt, $gte, $in, $lt, $lte, $ne, $nin, $nor, $not, $options, $or, $regex, } = sift;
3
3
  export const OPERATIONS = {
4
4
  $and,
5
5
  $eq,
@@ -15,4 +15,9 @@ export const OPERATIONS = {
15
15
  $not,
16
16
  $or,
17
17
  };
18
+ export const STRING_OPERATIONS = {
19
+ ...OPERATIONS,
20
+ $options,
21
+ $regex,
22
+ };
18
23
  //# sourceMappingURL=operations.js.map
@@ -1 +1 @@
1
- {"version":3,"file":"operations.js","sourceRoot":"","sources":["../../src/constants/operations.ts"],"names":[],"mappings":"AAAA,OAAO,IAAI,MAAM,MAAM,CAAA;AAGvB,MAAM,EACJ,IAAI,EACJ,GAAG,EACH,OAAO,EACP,GAAG,EACH,IAAI,EACJ,GAAG,EACH,GAAG,EACH,IAAI,EACJ,GAAG,EACH,IAAI,EACJ,IAAI,EACJ,IAAI,EACJ,GAAG,GACJ,GAAG,IAAI,CAAA;AAER,MAAM,CAAC,MAAM,UAAU,GAA0C;IAC/D,IAAI;IACJ,GAAG;IACH,OAAO;IACP,GAAG;IACH,IAAI;IACJ,GAAG;IACH,GAAG;IACH,IAAI;IACJ,GAAG;IACH,IAAI;IACJ,IAAI;IACJ,IAAI;IACJ,GAAG;CACJ,CAAA"}
1
+ {"version":3,"file":"operations.js","sourceRoot":"","sources":["../../src/constants/operations.ts"],"names":[],"mappings":"AAAA,OAAO,IAAI,MAAM,MAAM,CAAA;AAGvB,MAAM,EACJ,IAAI,EACJ,GAAG,EACH,OAAO,EACP,GAAG,EACH,IAAI,EACJ,GAAG,EACH,GAAG,EACH,IAAI,EACJ,GAAG,EACH,IAAI,EACJ,IAAI,EACJ,IAAI,EACJ,QAAQ,EACR,GAAG,EACH,MAAM,GACP,GAAG,IAAI,CAAA;AAER,MAAM,CAAC,MAAM,UAAU,GAA0C;IAC/D,IAAI;IACJ,GAAG;IACH,OAAO;IACP,GAAG;IACH,IAAI;IACJ,GAAG;IACH,GAAG;IACH,IAAI;IACJ,GAAG;IACH,IAAI;IACJ,IAAI;IACJ,IAAI;IACJ,GAAG;CACJ,CAAA;AAED,MAAM,CAAC,MAAM,iBAAiB,GAA0C;IACtE,GAAG,UAAU;IACb,QAAQ;IACR,MAAM;CACP,CAAA"}
package/dist/fs.d.ts CHANGED
@@ -4,5 +4,5 @@ export * from './activities/FileJoinActivity.js';
4
4
  export * from './activities/FileReadActivity.js';
5
5
  export * from './activities/FileSplitActivity.js';
6
6
  export * from './activities/FileWriteActivity.js';
7
- export * from './helpers/byte-delimiter-scanner.js';
7
+ export * from './helpers/ByteDelimiterScanner.js';
8
8
  //# sourceMappingURL=fs.d.ts.map
package/dist/fs.d.ts.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"fs.d.ts","sourceRoot":"","sources":["../src/fs.ts"],"names":[],"mappings":"AAAA,cAAc,iBAAiB,CAAA;AAC/B,cAAc,sBAAsB,CAAA;AACpC,cAAc,kCAAkC,CAAA;AAChD,cAAc,kCAAkC,CAAA;AAChD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA;AACjD,cAAc,qCAAqC,CAAA"}
1
+ {"version":3,"file":"fs.d.ts","sourceRoot":"","sources":["../src/fs.ts"],"names":[],"mappings":"AAAA,cAAc,iBAAiB,CAAA;AAC/B,cAAc,sBAAsB,CAAA;AACpC,cAAc,kCAAkC,CAAA;AAChD,cAAc,kCAAkC,CAAA;AAChD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA"}
package/dist/fs.js CHANGED
@@ -4,5 +4,5 @@ export * from './activities/FileJoinActivity.js';
4
4
  export * from './activities/FileReadActivity.js';
5
5
  export * from './activities/FileSplitActivity.js';
6
6
  export * from './activities/FileWriteActivity.js';
7
- export * from './helpers/byte-delimiter-scanner.js';
7
+ export * from './helpers/ByteDelimiterScanner.js';
8
8
  //# sourceMappingURL=fs.js.map
package/dist/fs.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"fs.js","sourceRoot":"","sources":["../src/fs.ts"],"names":[],"mappings":"AAAA,cAAc,iBAAiB,CAAA;AAC/B,cAAc,sBAAsB,CAAA;AACpC,cAAc,kCAAkC,CAAA;AAChD,cAAc,kCAAkC,CAAA;AAChD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA;AACjD,cAAc,qCAAqC,CAAA"}
1
+ {"version":3,"file":"fs.js","sourceRoot":"","sources":["../src/fs.ts"],"names":[],"mappings":"AAAA,cAAc,iBAAiB,CAAA;AAC/B,cAAc,sBAAsB,CAAA;AACpC,cAAc,kCAAkC,CAAA;AAChD,cAAc,kCAAkC,CAAA;AAChD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA;AACjD,cAAc,mCAAmC,CAAA"}
@@ -3,6 +3,9 @@ export type ByteRange = {
3
3
  start: number;
4
4
  end: number;
5
5
  };
6
+ export type ByteSlice = ByteRange & {
7
+ content?: Buffer;
8
+ };
6
9
  export type FileSplitDelimiter = {
7
10
  value: string;
8
11
  keep?: {
@@ -20,9 +23,11 @@ export type ByteDelimiterScannerOptions = {
20
23
  trim?: ByteTrim;
21
24
  skipEmpty?: boolean;
22
25
  encoding?: BufferEncoding;
26
+ inlineMaxBytes?: number;
23
27
  };
28
+ export declare const DEFAULT_INLINE_MAX_BYTES: number;
24
29
  export declare class ByteDelimiterScanner extends Transform {
25
- readonly ranges: ByteRange[];
30
+ readonly slices: ByteSlice[];
26
31
  private readonly delimiter;
27
32
  private readonly keepPrevious;
28
33
  private readonly keepNext;
@@ -30,14 +35,16 @@ export declare class ByteDelimiterScanner extends Transform {
30
35
  private readonly trimStart;
31
36
  private readonly trimEnd;
32
37
  private readonly skipEmpty;
38
+ private readonly inlineMaxBytes;
33
39
  private pending;
34
40
  private pendingAt;
41
+ private searchAt;
35
42
  private sliceStart;
36
43
  private matches;
37
- constructor({ delimiter, every, trim, skipEmpty, encoding, }: ByteDelimiterScannerOptions);
44
+ constructor({ delimiter, every, trim, skipEmpty, encoding, inlineMaxBytes, }: ByteDelimiterScannerOptions);
38
45
  _transform(chunk: Buffer, _encoding: BufferEncoding, callback: TransformCallback): void;
39
46
  _flush(callback: TransformCallback): void;
40
47
  private scan;
41
48
  private cut;
42
49
  }
43
- //# sourceMappingURL=byte-delimiter-scanner.d.ts.map
50
+ //# sourceMappingURL=ByteDelimiterScanner.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ByteDelimiterScanner.d.ts","sourceRoot":"","sources":["../../src/helpers/ByteDelimiterScanner.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,KAAK,iBAAiB,EAAE,MAAM,aAAa,CAAA;AAE/D,MAAM,MAAM,SAAS,GAAG;IACtB,KAAK,EAAE,MAAM,CAAA;IACb,GAAG,EAAE,MAAM,CAAA;CACZ,CAAA;AAED,MAAM,MAAM,SAAS,GAAG,SAAS,GAAG;IAClC,OAAO,CAAC,EAAE,MAAM,CAAA;CACjB,CAAA;AAED,MAAM,MAAM,kBAAkB,GAAG;IAC/B,KAAK,EAAE,MAAM,CAAA;IACb,IAAI,CAAC,EAAE;QACL,QAAQ,CAAC,EAAE,MAAM,CAAA;QACjB,IAAI,CAAC,EAAE,MAAM,CAAA;KACd,CAAA;CACF,CAAA;AAED,MAAM,MAAM,QAAQ,GAAG;IACrB,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,GAAG,CAAC,EAAE,MAAM,CAAA;CACb,CAAA;AAED,MAAM,MAAM,2BAA2B,GAAG;IACxC,SAAS,EAAE,kBAAkB,CAAA;IAC7B,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,IAAI,CAAC,EAAE,QAAQ,CAAA;IACf,SAAS,CAAC,EAAE,OAAO,CAAA;IACnB,QAAQ,CAAC,EAAE,cAAc,CAAA;IACzB,cAAc,CAAC,EAAE,MAAM,CAAA;CACxB,CAAA;AAED,eAAO,MAAM,wBAAwB,QAAY,CAAA;AAEjD,qBAAa,oBAAqB,SAAQ,SAAS;IACjD,SAAgB,MAAM,EAAE,SAAS,EAAE,CAAK;IAExC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAQ;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAQ;IACjC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAQ;IACjC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAQ;IAChC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAQ;IAEvC,OAAO,CAAC,OAAO,CAA0B;IACzC,OAAO,CAAC,SAAS,CAAI;IACrB,OAAO,CAAC,QAAQ,CAAI;IACpB,OAAO,CAAC,UAAU,CAAI;IACtB,OAAO,CAAC,OAAO,CAAI;gBAEP,EACV,SAAS,EACT,KAAS,EACT,IAAI,EACJ,SAAgB,EAChB,QAAiB,EACjB,cAAyC,GAC1C,EAAE,2BAA2B;IAuCvB,UAAU,CACf,KAAK,EAAE,MAAM,EACb,SAAS,EAAE,cAAc,EACzB,QAAQ,EAAE,iBAAiB;IAStB,MAAM,CAAC,QAAQ,EAAE,iBAAiB;IAyBzC,OAAO,CAAC,IAAI;IAyCZ,OAAO,CAAC,GAAG;CAkBZ"}
@@ -1,6 +1,7 @@
1
1
  import { Transform } from 'node:stream';
2
+ export const DEFAULT_INLINE_MAX_BYTES = 16 * 1024;
2
3
  export class ByteDelimiterScanner extends Transform {
3
- ranges = [];
4
+ slices = [];
4
5
  delimiter;
5
6
  keepPrevious;
6
7
  keepNext;
@@ -8,11 +9,13 @@ export class ByteDelimiterScanner extends Transform {
8
9
  trimStart;
9
10
  trimEnd;
10
11
  skipEmpty;
12
+ inlineMaxBytes;
11
13
  pending = Buffer.alloc(0);
12
14
  pendingAt = 0;
15
+ searchAt = 0;
13
16
  sliceStart = 0;
14
17
  matches = 0;
15
- constructor({ delimiter, every = 1, trim, skipEmpty = true, encoding = 'utf8', }) {
18
+ constructor({ delimiter, every = 1, trim, skipEmpty = true, encoding = 'utf8', inlineMaxBytes = DEFAULT_INLINE_MAX_BYTES, }) {
16
19
  super();
17
20
  this.delimiter = Buffer.from(delimiter.value, encoding);
18
21
  this.keepPrevious = delimiter.keep?.previous ?? 0;
@@ -21,6 +24,7 @@ export class ByteDelimiterScanner extends Transform {
21
24
  this.trimStart = trim?.start ?? 0;
22
25
  this.trimEnd = trim?.end ?? 0;
23
26
  this.skipEmpty = skipEmpty;
27
+ this.inlineMaxBytes = inlineMaxBytes;
24
28
  this.sliceStart = this.trimStart;
25
29
  if (this.delimiter.length === 0)
26
30
  throw new TypeError('Split delimiter must not be empty');
@@ -35,6 +39,8 @@ export class ByteDelimiterScanner extends Transform {
35
39
  this.trimStart < 0 ||
36
40
  this.trimEnd < 0)
37
41
  throw new TypeError('Split trim sizes must be non-negative integers');
42
+ if (!Number.isInteger(inlineMaxBytes) || inlineMaxBytes < 0)
43
+ throw new TypeError('Split inlineMaxBytes must be a non-negative integer');
38
44
  }
39
45
  _transform(chunk, _encoding, callback) {
40
46
  this.scan(this.pending.length > 0 ? Buffer.concat([this.pending, chunk]) : chunk);
@@ -43,21 +49,23 @@ export class ByteDelimiterScanner extends Transform {
43
49
  _flush(callback) {
44
50
  const limit = this.pendingAt + this.pending.length - this.trimEnd;
45
51
  while (true) {
46
- const last = this.ranges.at(-1);
52
+ const last = this.slices.at(-1);
47
53
  if (!last || last.end <= limit)
48
54
  break;
49
55
  if (last.start >= limit) {
50
- this.ranges.pop();
56
+ this.slices.pop();
51
57
  continue;
52
58
  }
53
59
  last.end = limit;
60
+ if (last.content)
61
+ last.content = last.content.subarray(0, limit - last.start);
54
62
  }
55
63
  if (this.sliceStart <= limit)
56
- this.cut(this.sliceStart, limit);
64
+ this.cut(this.sliceStart, limit, this.pending);
57
65
  callback();
58
66
  }
59
67
  scan(buffer) {
60
- let searchFrom = 0;
68
+ let searchFrom = Math.max(0, this.searchAt - this.pendingAt);
61
69
  while (true) {
62
70
  const index = buffer.indexOf(this.delimiter, searchFrom);
63
71
  if (index === -1)
@@ -69,17 +77,27 @@ export class ByteDelimiterScanner extends Transform {
69
77
  this.matches += 1;
70
78
  if (this.matches % this.cutEvery !== 0)
71
79
  continue;
72
- this.cut(this.sliceStart, at + this.keepPrevious);
80
+ this.cut(this.sliceStart, at + this.keepPrevious, buffer);
73
81
  this.sliceStart = at + this.delimiter.length - this.keepNext;
74
82
  }
75
83
  const tail = Math.max(searchFrom, buffer.length - (this.delimiter.length - 1));
76
- this.pending = buffer.subarray(tail);
77
- this.pendingAt += tail;
84
+ this.searchAt = this.pendingAt + tail;
85
+ const openAt = this.sliceStart - this.pendingAt;
86
+ const isOpenSliceInline = openAt >= 0 && buffer.length - openAt <= this.inlineMaxBytes;
87
+ const keepFrom = isOpenSliceInline ? Math.min(tail, openAt) : tail;
88
+ this.pending = buffer.subarray(keepFrom);
89
+ this.pendingAt += keepFrom;
78
90
  }
79
- cut(start, end) {
91
+ cut(start, end, buffer) {
80
92
  if (this.skipEmpty && end <= start)
81
93
  return;
82
- this.ranges.push({ start, end });
94
+ const isInline = start >= this.pendingAt && end - start <= this.inlineMaxBytes;
95
+ if (!isInline) {
96
+ this.slices.push({ start, end });
97
+ return;
98
+ }
99
+ const content = Buffer.from(buffer.subarray(start - this.pendingAt, end - this.pendingAt));
100
+ this.slices.push({ start, end, content });
83
101
  }
84
102
  }
85
- //# sourceMappingURL=byte-delimiter-scanner.js.map
103
+ //# sourceMappingURL=ByteDelimiterScanner.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ByteDelimiterScanner.js","sourceRoot":"","sources":["../../src/helpers/ByteDelimiterScanner.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAA0B,MAAM,aAAa,CAAA;AAiC/D,MAAM,CAAC,MAAM,wBAAwB,GAAG,EAAE,GAAG,IAAI,CAAA;AAEjD,MAAM,OAAO,oBAAqB,SAAQ,SAAS;IACjC,MAAM,GAAgB,EAAE,CAAA;IAEvB,SAAS,CAAQ;IACjB,YAAY,CAAQ;IACpB,QAAQ,CAAQ;IAChB,QAAQ,CAAQ;IAChB,SAAS,CAAQ;IACjB,OAAO,CAAQ;IACf,SAAS,CAAS;IAClB,cAAc,CAAQ;IAE/B,OAAO,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;IACjC,SAAS,GAAG,CAAC,CAAA;IACb,QAAQ,GAAG,CAAC,CAAA;IACZ,UAAU,GAAG,CAAC,CAAA;IACd,OAAO,GAAG,CAAC,CAAA;IAEnB,YAAY,EACV,SAAS,EACT,KAAK,GAAG,CAAC,EACT,IAAI,EACJ,SAAS,GAAG,IAAI,EAChB,QAAQ,GAAG,MAAM,EACjB,cAAc,GAAG,wBAAwB,GACb;QAC5B,KAAK,EAAE,CAAA;QAEP,IAAI,CAAC,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAA;QACvD,IAAI,CAAC,YAAY,GAAG,SAAS,CAAC,IAAI,EAAE,QAAQ,IAAI,CAAC,CAAA;QACjD,IAAI,CAAC,QAAQ,GAAG,SAAS,CAAC,IAAI,EAAE,IAAI,IAAI,CAAC,CAAA;QACzC,IAAI,CAAC,QAAQ,GAAG,KAAK,CAAA;QACrB,IAAI,CAAC,SAAS,GAAG,IAAI,EAAE,KAAK,IAAI,CAAC,CAAA;QACjC,IAAI,CAAC,OAAO,GAAG,IAAI,EAAE,GAAG,IAAI,CAAC,CAAA;QAC7B,IAAI,CAAC,SAAS,GAAG,SAAS,CAAA;QAC1B,IAAI,CAAC,cAAc,GAAG,cAAc,CAAA;QACpC,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,SAAS,CAAA;QAEhC,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC;YAC7B,MAAM,IAAI,SAAS,CAAC,mCAAmC,CAAC,CAAA;QAE1D,IAAI,IAAI,CAAC,YAAY,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,GAAG,CAAC;YAC5C,MAAM,IAAI,SAAS,CAAC,iDAAiD,CAAC,CAAA;QAExE,IAAI,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM;YAC3D,MAAM,IAAI,SAAS,CACjB,iEAAiE,CAClE,CAAA;QAEH,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC;YACvC,MAAM,IAAI,SAAS,CAAC,wCAAwC,CAAC,CAAA;QAE/D,IACE,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC;YACjC,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO,CAAC;YAC/B,IAAI,CAAC,SAAS,GAAG,CAAC;YAClB,IAAI,CAAC,OAAO,GAAG,CAAC;YAEhB,MAAM,IAAI,SAAS,CAAC,gDAAgD,CAAC,CAAA;QAEvE,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,cAAc,CAAC,IAAI,cAAc,GAAG,CAAC;YACzD,MAAM,IAAI,SAAS,CAAC,qDAAqD,CAAC,CAAA;IAC9E,CAAC;IAEM,UAAU,CACf,KAAa,EACb,SAAyB,EACzB,QAA2B;QAE3B,IAAI,CAAC,IAAI,CACP,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CACvE,CAAA;QAED,QAAQ,CAAC,IAAI,EAAE,KAAK,CAAC,CAAA;IACvB,CAAC;IAEM,MAAM,CAAC,QAA2B;QACvC,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,OAAO,CAAA;QAEjE,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAA;YAE/B,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,GAAG,IAAI,KAAK;gBAAE,MAAK;YAErC,IAAI,IAAI,CAAC,KAAK,IAAI,KAAK,EAAE,CAAC;gBACxB,IAAI,CAAC,MAAM,CAAC,GAAG,EAAE,CAAA;gBAEjB,SAAQ;YACV,CAAC;YAED,IAAI,CAAC,GAAG,GAAG,KAAK,CAAA;YAEhB,IAAI,IAAI,CAAC,OAAO;gBACd,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,EAAE,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,CAAA;QAC/D,CAAC;QAED,IAAI,IAAI,CAAC,UAAU,IAAI,KAAK;YAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,UAAU,EAAE,KAAK,EAAE,IAAI,CAAC,OAAO,CAAC,CAAA;QAE5E,QAAQ,EAAE,CAAA;IACZ,CAAC;IAEO,IAAI,CAAC,MAAc;QACzB,IAAI,UAAU,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,CAAA;QAE5D,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,KAAK,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,EAAE,UAAU,CAAC,CAAA;YAExD,IAAI,KAAK,KAAK,CAAC,CAAC;gBAAE,MAAK;YAEvB,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;YAEjC,UAAU,GAAG,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAA;YAE1C,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS;gBAAE,SAAQ;YAEjC,IAAI,CAAC,OAAO,IAAI,CAAC,CAAA;YAEjB,IAAI,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,QAAQ,KAAK,CAAC;gBAAE,SAAQ;YAEhD,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,UAAU,EAAE,EAAE,GAAG,IAAI,CAAC,YAAY,EAAE,MAAM,CAAC,CAAA;YAEzD,IAAI,CAAC,UAAU,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAA;QAC9D,CAAC;QAED,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CACnB,UAAU,EACV,MAAM,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAC5C,CAAA;QAED,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,GAAG,IAAI,CAAA;QAErC,MAAM,MAAM,GAAG,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,SAAS,CAAA;QAE/C,MAAM,iBAAiB,GACrB,MAAM,IAAI,CAAC,IAAI,MAAM,CAAC,MAAM,GAAG,MAAM,IAAI,IAAI,CAAC,cAAc,CAAA;QAE9D,MAAM,QAAQ,GAAG,iBAAiB,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,CAAA;QAElE,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAA;QACxC,IAAI,CAAC,SAAS,IAAI,QAAQ,CAAA;IAC5B,CAAC;IAEO,GAAG,CAAC,KAAa,EAAE,GAAW,EAAE,MAAc;QACpD,IAAI,IAAI,CAAC,SAAS,IAAI,GAAG,IAAI,KAAK;YAAE,OAAM;QAE1C,MAAM,QAAQ,GACZ,KAAK,IAAI,IAAI,CAAC,SAAS,IAAI,GAAG,GAAG,KAAK,IAAI,IAAI,CAAC,cAAc,CAAA;QAE/D,IAAI,CAAC,QAAQ,EAAE,CAAC;YACd,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,CAAC,CAAA;YAEhC,OAAM;QACR,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,CAAC,IAAI,CACzB,MAAM,CAAC,QAAQ,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,EAAE,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,CAC9D,CAAA;QAED,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC,CAAA;IAC3C,CAAC;CACF"}
@@ -0,0 +1,32 @@
1
+ import { Transform, type TransformCallback } from 'node:stream';
2
+ import type { ByteTrim, FileSplitDelimiter } from './ByteDelimiterScanner.js';
3
+ export type ByteRecordFilterOptions = {
4
+ delimiter: FileSplitDelimiter;
5
+ accept: (record: Buffer) => boolean;
6
+ trim?: ByteTrim;
7
+ encoding?: BufferEncoding;
8
+ maxRecordBytes?: number;
9
+ };
10
+ export declare const DEFAULT_FILTER_MAX_RECORD_BYTES: number;
11
+ export declare class ByteRecordFilter extends Transform {
12
+ accepted: number;
13
+ private readonly delimiter;
14
+ private readonly joint;
15
+ private readonly keepPrevious;
16
+ private readonly keepNext;
17
+ private readonly trimStart;
18
+ private readonly trimEnd;
19
+ private readonly maxRecordBytes;
20
+ private readonly accept;
21
+ private pending;
22
+ private pendingAt;
23
+ private received;
24
+ private searchAt;
25
+ private recordStart;
26
+ constructor({ delimiter, accept, trim, encoding, maxRecordBytes, }: ByteRecordFilterOptions);
27
+ _transform(chunk: Buffer, _encoding: BufferEncoding, callback: TransformCallback): void;
28
+ _flush(callback: TransformCallback): void;
29
+ private scan;
30
+ private settle;
31
+ }
32
+ //# sourceMappingURL=ByteRecordFilter.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ByteRecordFilter.d.ts","sourceRoot":"","sources":["../../src/helpers/ByteRecordFilter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,KAAK,iBAAiB,EAAE,MAAM,aAAa,CAAA;AAE/D,OAAO,KAAK,EAAE,QAAQ,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAA;AAE7E,MAAM,MAAM,uBAAuB,GAAG;IACpC,SAAS,EAAE,kBAAkB,CAAA;IAC7B,MAAM,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,OAAO,CAAA;IACnC,IAAI,CAAC,EAAE,QAAQ,CAAA;IACf,QAAQ,CAAC,EAAE,cAAc,CAAA;IACzB,cAAc,CAAC,EAAE,MAAM,CAAA;CACxB,CAAA;AAED,eAAO,MAAM,+BAA+B,QAAc,CAAA;AAE1D,qBAAa,gBAAiB,SAAQ,SAAS;IACtC,QAAQ,SAAI;IAEnB,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAQ;IAC9B,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAQ;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAQ;IACjC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAQ;IAChC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAQ;IACvC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAA6B;IAEpD,OAAO,CAAC,OAAO,CAA0B;IACzC,OAAO,CAAC,SAAS,CAAI;IACrB,OAAO,CAAC,QAAQ,CAAI;IACpB,OAAO,CAAC,QAAQ,CAAQ;IACxB,OAAO,CAAC,WAAW,CAAQ;gBAEf,EACV,SAAS,EACT,MAAM,EACN,IAAI,EACJ,QAAiB,EACjB,cAAgD,GACjD,EAAE,uBAAuB;IAqBnB,UAAU,CACf,KAAK,EAAE,MAAM,EACb,SAAS,EAAE,cAAc,EACzB,QAAQ,EAAE,iBAAiB;IAiBtB,MAAM,CAAC,QAAQ,EAAE,iBAAiB;IAYzC,OAAO,CAAC,IAAI;IA8CZ,OAAO,CAAC,MAAM;CAcf"}
@@ -0,0 +1,89 @@
1
+ import { Transform } from 'node:stream';
2
+ export const DEFAULT_FILTER_MAX_RECORD_BYTES = 1024 * 1024;
3
+ export class ByteRecordFilter extends Transform {
4
+ accepted = 0;
5
+ delimiter;
6
+ joint;
7
+ keepPrevious;
8
+ keepNext;
9
+ trimStart;
10
+ trimEnd;
11
+ maxRecordBytes;
12
+ accept;
13
+ pending = Buffer.alloc(0);
14
+ pendingAt = 0;
15
+ received = 0;
16
+ searchAt;
17
+ recordStart;
18
+ constructor({ delimiter, accept, trim, encoding = 'utf8', maxRecordBytes = DEFAULT_FILTER_MAX_RECORD_BYTES, }) {
19
+ super();
20
+ this.delimiter = Buffer.from(delimiter.value, encoding);
21
+ this.keepPrevious = delimiter.keep?.previous ?? 0;
22
+ this.keepNext = delimiter.keep?.next ?? 0;
23
+ this.joint = this.delimiter.subarray(this.keepPrevious, this.delimiter.length - this.keepNext);
24
+ this.trimStart = trim?.start ?? 0;
25
+ this.trimEnd = trim?.end ?? 0;
26
+ this.maxRecordBytes = maxRecordBytes;
27
+ this.accept = accept;
28
+ this.searchAt = this.trimStart;
29
+ this.recordStart = this.trimStart;
30
+ if (this.delimiter.length === 0)
31
+ throw new TypeError('Split delimiter must not be empty');
32
+ }
33
+ _transform(chunk, _encoding, callback) {
34
+ this.pending =
35
+ this.pending.length > 0 ? Buffer.concat([this.pending, chunk]) : chunk;
36
+ this.received += chunk.length;
37
+ try {
38
+ const output = this.scan(this.received - this.trimEnd);
39
+ if (output.length > 0)
40
+ this.push(output);
41
+ callback();
42
+ }
43
+ catch (error) {
44
+ callback(error);
45
+ }
46
+ }
47
+ _flush(callback) {
48
+ const limit = this.received - this.trimEnd;
49
+ const output = [];
50
+ if (this.recordStart <= limit)
51
+ this.settle(this.recordStart, limit, output);
52
+ if (output.length > 0)
53
+ this.push(Buffer.concat(output));
54
+ callback();
55
+ }
56
+ scan(limit) {
57
+ const output = [];
58
+ while (true) {
59
+ const index = this.pending.indexOf(this.delimiter, this.searchAt - this.pendingAt);
60
+ if (index === -1) {
61
+ this.searchAt = Math.max(this.searchAt, this.pendingAt + this.pending.length - (this.delimiter.length - 1));
62
+ break;
63
+ }
64
+ const at = this.pendingAt + index;
65
+ if (at + this.delimiter.length > limit)
66
+ break;
67
+ this.settle(this.recordStart, at + this.keepPrevious, output);
68
+ this.recordStart = at + this.delimiter.length - this.keepNext;
69
+ this.searchAt = at + this.delimiter.length;
70
+ }
71
+ const end = this.pendingAt + this.pending.length;
72
+ const keepFrom = Math.min(Math.max(Math.min(this.recordStart, this.searchAt), this.pendingAt), end);
73
+ this.pending = this.pending.subarray(keepFrom - this.pendingAt);
74
+ this.pendingAt = keepFrom;
75
+ if (end - this.recordStart > this.maxRecordBytes)
76
+ throw new RangeError(`Split where can only test records up to ${this.maxRecordBytes} bytes, found a record starting at byte ${this.recordStart} without a delimiter`);
77
+ return Buffer.concat(output);
78
+ }
79
+ settle(start, end, output) {
80
+ const record = this.pending.subarray(start - this.pendingAt, end - this.pendingAt);
81
+ if (!this.accept(record))
82
+ return;
83
+ if (this.accepted > 0)
84
+ output.push(this.joint);
85
+ output.push(record);
86
+ this.accepted += 1;
87
+ }
88
+ }
89
+ //# sourceMappingURL=ByteRecordFilter.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"ByteRecordFilter.js","sourceRoot":"","sources":["../../src/helpers/ByteRecordFilter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAA0B,MAAM,aAAa,CAAA;AAY/D,MAAM,CAAC,MAAM,+BAA+B,GAAG,IAAI,GAAG,IAAI,CAAA;AAE1D,MAAM,OAAO,gBAAiB,SAAQ,SAAS;IACtC,QAAQ,GAAG,CAAC,CAAA;IAEF,SAAS,CAAQ;IACjB,KAAK,CAAQ;IACb,YAAY,CAAQ;IACpB,QAAQ,CAAQ;IAChB,SAAS,CAAQ;IACjB,OAAO,CAAQ;IACf,cAAc,CAAQ;IACtB,MAAM,CAA6B;IAE5C,OAAO,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;IACjC,SAAS,GAAG,CAAC,CAAA;IACb,QAAQ,GAAG,CAAC,CAAA;IACZ,QAAQ,CAAQ;IAChB,WAAW,CAAQ;IAE3B,YAAY,EACV,SAAS,EACT,MAAM,EACN,IAAI,EACJ,QAAQ,GAAG,MAAM,EACjB,cAAc,GAAG,+BAA+B,GACxB;QACxB,KAAK,EAAE,CAAA;QAEP,IAAI,CAAC,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAA;QACvD,IAAI,CAAC,YAAY,GAAG,SAAS,CAAC,IAAI,EAAE,QAAQ,IAAI,CAAC,CAAA;QACjD,IAAI,CAAC,QAAQ,GAAG,SAAS,CAAC,IAAI,EAAE,IAAI,IAAI,CAAC,CAAA;QACzC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,QAAQ,CAClC,IAAI,CAAC,YAAY,EACjB,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CACtC,CAAA;QACD,IAAI,CAAC,SAAS,GAAG,IAAI,EAAE,KAAK,IAAI,CAAC,CAAA;QACjC,IAAI,CAAC,OAAO,GAAG,IAAI,EAAE,GAAG,IAAI,CAAC,CAAA;QAC7B,IAAI,CAAC,cAAc,GAAG,cAAc,CAAA;QACpC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAA;QACpB,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAA;QAC9B,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,SAAS,CAAA;QAEjC,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC;YAC7B,MAAM,IAAI,SAAS,CAAC,mCAAmC,CAAC,CAAA;IAC5D,CAAC;IAEM,UAAU,CACf,KAAa,EACb,SAAyB,EACzB,QAA2B;QAE3B,IAAI,CAAC,OAAO;YACV,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAA;QACxE,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,MAAM,CAAA;QAE7B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAC,CAAA;YAEtD,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC;gBAAE,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAA;YAExC,QAAQ,EAAE,CAAA;QACZ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,QAAQ,CAAC,KAAc,CAAC,CAAA;QAC1B,CAAC;IACH,CAAC;IAEM,MAAM,CAAC,QAA2B;QACvC,MAAM,KAAK,GAAG,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAA;QAE1C,MAAM,MAAM,GAAa,EAAE,CAAA;QAE3B,IAAI,IAAI,CAAC,WAAW,IAAI,KAAK;YAAE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,WAAW,EAAE,KAAK,EAAE,MAAM,CAAC,CAAA;QAE3E,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC;YAAE,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAA;QAEvD,QAAQ,EAAE,CAAA;IACZ,CAAC;IAEO,IAAI,CAAC,KAAa;QACxB,MAAM,MAAM,GAAa,EAAE,CAAA;QAE3B,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAChC,IAAI,CAAC,SAAS,EACd,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAC/B,CAAA;YAED,IAAI,KAAK,KAAK,CAAC,CAAC,EAAE,CAAC;gBACjB,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,GAAG,CACtB,IAAI,CAAC,QAAQ,EACb,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CACnE,CAAA;gBAED,MAAK;YACP,CAAC;YAED,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;YAEjC,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,KAAK;gBAAE,MAAK;YAE7C,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,WAAW,EAAE,EAAE,GAAG,IAAI,CAAC,YAAY,EAAE,MAAM,CAAC,CAAA;YAE7D,IAAI,CAAC,WAAW,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAA;YAC7D,IAAI,CAAC,QAAQ,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAA;QAC5C,CAAC;QAED,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,CAAA;QAEhD,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CACvB,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,IAAI,CAAC,SAAS,CAAC,EACnE,GAAG,CACJ,CAAA;QAED,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,CAAA;QAC/D,IAAI,CAAC,SAAS,GAAG,QAAQ,CAAA;QAEzB,IAAI,GAAG,GAAG,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,cAAc;YAC9C,MAAM,IAAI,UAAU,CAClB,2CAA2C,IAAI,CAAC,cAAc,2CAA2C,IAAI,CAAC,WAAW,sBAAsB,CAChJ,CAAA;QAEH,OAAO,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAA;IAC9B,CAAC;IAEO,MAAM,CAAC,KAAa,EAAE,GAAW,EAAE,MAAgB;QACzD,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAClC,KAAK,GAAG,IAAI,CAAC,SAAS,EACtB,GAAG,GAAG,IAAI,CAAC,SAAS,CACrB,CAAA;QAED,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC;YAAE,OAAM;QAEhC,IAAI,IAAI,CAAC,QAAQ,GAAG,CAAC;YAAE,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAA;QAE9C,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAA;QAEnB,IAAI,CAAC,QAAQ,IAAI,CAAC,CAAA;IACpB,CAAC;CACF"}
@@ -1,2 +1,2 @@
1
- export declare function assertWhereSerializable(node: unknown, path: string): void;
1
+ export declare function assertWhereSerializable(node: unknown, path: string, label?: string, operations?: Record<string, unknown>): void;
2
2
  //# sourceMappingURL=assert-where-serializable.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"assert-where-serializable.d.ts","sourceRoot":"","sources":["../../src/tools/assert-where-serializable.ts"],"names":[],"mappings":"AAEA,wBAAgB,uBAAuB,CAAC,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,GAAG,IAAI,CA6BzE"}
1
+ {"version":3,"file":"assert-where-serializable.d.ts","sourceRoot":"","sources":["../../src/tools/assert-where-serializable.ts"],"names":[],"mappings":"AAEA,wBAAgB,uBAAuB,CACrC,IAAI,EAAE,OAAO,EACb,IAAI,EAAE,MAAM,EACZ,KAAK,SAAe,EACpB,UAAU,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAc,GAC/C,IAAI,CA+BN"}
@@ -1,19 +1,19 @@
1
1
  import { OPERATIONS } from '../constants/operations.js';
2
- export function assertWhereSerializable(node, path) {
2
+ export function assertWhereSerializable(node, path, label = 'Join where', operations = OPERATIONS) {
3
3
  if (typeof node === 'function' || node instanceof RegExp)
4
- throw new TypeError(`Join where must serialize into the workflow history, found ${typeof node === 'function' ? 'a function' : 'a RegExp'} at ${path}`);
4
+ throw new TypeError(`${label} must serialize into the workflow history, found ${typeof node === 'function' ? 'a function' : 'a RegExp'} at ${path}`);
5
5
  if (typeof node === 'undefined')
6
- throw new TypeError(`Join where must serialize into the workflow history, found undefined at ${path}`);
6
+ throw new TypeError(`${label} must serialize into the workflow history, found undefined at ${path}`);
7
7
  if (node === null || typeof node !== 'object')
8
8
  return;
9
9
  if (Array.isArray(node)) {
10
- node.forEach((item, index) => assertWhereSerializable(item, `${path}[${index}]`));
10
+ node.forEach((item, index) => assertWhereSerializable(item, `${path}[${index}]`, label, operations));
11
11
  return;
12
12
  }
13
13
  for (const [key, value] of Object.entries(node)) {
14
- if (key.startsWith('$') && !(key in OPERATIONS))
15
- throw new TypeError(`Unsupported join where operator ${key} at ${path}`);
16
- assertWhereSerializable(value, `${path}.${key}`);
14
+ if (key.startsWith('$') && !(key in operations))
15
+ throw new TypeError(`Unsupported ${label.toLowerCase()} operator ${key} at ${path}`);
16
+ assertWhereSerializable(value, `${path}.${key}`, label, operations);
17
17
  }
18
18
  }
19
19
  //# sourceMappingURL=assert-where-serializable.js.map
@@ -1 +1 @@
1
- {"version":3,"file":"assert-where-serializable.js","sourceRoot":"","sources":["../../src/tools/assert-where-serializable.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAA;AAEvD,MAAM,UAAU,uBAAuB,CAAC,IAAa,EAAE,IAAY;IACjE,IAAI,OAAO,IAAI,KAAK,UAAU,IAAI,IAAI,YAAY,MAAM;QACtD,MAAM,IAAI,SAAS,CACjB,8DACE,OAAO,IAAI,KAAK,UAAU,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,UAC9C,OAAO,IAAI,EAAE,CACd,CAAA;IAEH,IAAI,OAAO,IAAI,KAAK,WAAW;QAC7B,MAAM,IAAI,SAAS,CACjB,2EAA2E,IAAI,EAAE,CAClF,CAAA;IAEH,IAAI,IAAI,KAAK,IAAI,IAAI,OAAO,IAAI,KAAK,QAAQ;QAAE,OAAM;IAErD,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,IAAI,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAC3B,uBAAuB,CAAC,IAAI,EAAE,GAAG,IAAI,IAAI,KAAK,GAAG,CAAC,CACnD,CAAA;QAED,OAAM;IACR,CAAC;IAED,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QAChD,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,IAAI,UAAU,CAAC;YAC7C,MAAM,IAAI,SAAS,CAAC,mCAAmC,GAAG,OAAO,IAAI,EAAE,CAAC,CAAA;QAE1E,uBAAuB,CAAC,KAAK,EAAE,GAAG,IAAI,IAAI,GAAG,EAAE,CAAC,CAAA;IAClD,CAAC;AACH,CAAC"}
1
+ {"version":3,"file":"assert-where-serializable.js","sourceRoot":"","sources":["../../src/tools/assert-where-serializable.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAA;AAEvD,MAAM,UAAU,uBAAuB,CACrC,IAAa,EACb,IAAY,EACZ,KAAK,GAAG,YAAY,EACpB,aAAsC,UAAU;IAEhD,IAAI,OAAO,IAAI,KAAK,UAAU,IAAI,IAAI,YAAY,MAAM;QACtD,MAAM,IAAI,SAAS,CACjB,GAAG,KAAK,oDACN,OAAO,IAAI,KAAK,UAAU,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,UAC9C,OAAO,IAAI,EAAE,CACd,CAAA;IAEH,IAAI,OAAO,IAAI,KAAK,WAAW;QAC7B,MAAM,IAAI,SAAS,CACjB,GAAG,KAAK,iEAAiE,IAAI,EAAE,CAChF,CAAA;IAEH,IAAI,IAAI,KAAK,IAAI,IAAI,OAAO,IAAI,KAAK,QAAQ;QAAE,OAAM;IAErD,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,IAAI,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAC3B,uBAAuB,CAAC,IAAI,EAAE,GAAG,IAAI,IAAI,KAAK,GAAG,EAAE,KAAK,EAAE,UAAU,CAAC,CACtE,CAAA;QAED,OAAM;IACR,CAAC;IAED,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QAChD,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,IAAI,UAAU,CAAC;YAC7C,MAAM,IAAI,SAAS,CACjB,eAAe,KAAK,CAAC,WAAW,EAAE,aAAa,GAAG,OAAO,IAAI,EAAE,CAChE,CAAA;QAEH,uBAAuB,CAAC,KAAK,EAAE,GAAG,IAAI,IAAI,GAAG,EAAE,EAAE,KAAK,EAAE,UAAU,CAAC,CAAA;IACrE,CAAC;AACH,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@t4h.framework/fs",
3
- "version": "0.9.5",
3
+ "version": "0.11.0",
4
4
  "description": "File system module for the T4H Framework",
5
5
  "homepage": "https://github.com/tech4humans-brasil/framework/tree/main/packages/fs",
6
6
  "bugs": "https://github.com/tech4humans-brasil/framework/issues",
@@ -32,12 +32,12 @@
32
32
  "test:watch": "vitest"
33
33
  },
34
34
  "devDependencies": {
35
- "@t4h.framework/core": "^0.14.0",
35
+ "@t4h.framework/core": "^0.16.0",
36
36
  "typescript": "^5.9.3",
37
37
  "vitest": "^4.0.18"
38
38
  },
39
39
  "peerDependencies": {
40
- "@t4h.framework/core": "^0.14.0"
40
+ "@t4h.framework/core": ">=0.15.0 <1.0.0"
41
41
  },
42
42
  "packageManager": "yarn@4.12.0",
43
43
  "engines": {
@@ -1 +0,0 @@
1
- {"version":3,"file":"byte-delimiter-scanner.d.ts","sourceRoot":"","sources":["../../src/helpers/byte-delimiter-scanner.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,KAAK,iBAAiB,EAAE,MAAM,aAAa,CAAA;AAE/D,MAAM,MAAM,SAAS,GAAG;IACtB,KAAK,EAAE,MAAM,CAAA;IACb,GAAG,EAAE,MAAM,CAAA;CACZ,CAAA;AAED,MAAM,MAAM,kBAAkB,GAAG;IAC/B,KAAK,EAAE,MAAM,CAAA;IACb,IAAI,CAAC,EAAE;QACL,QAAQ,CAAC,EAAE,MAAM,CAAA;QACjB,IAAI,CAAC,EAAE,MAAM,CAAA;KACd,CAAA;CACF,CAAA;AAED,MAAM,MAAM,QAAQ,GAAG;IACrB,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,GAAG,CAAC,EAAE,MAAM,CAAA;CACb,CAAA;AAED,MAAM,MAAM,2BAA2B,GAAG;IACxC,SAAS,EAAE,kBAAkB,CAAA;IAC7B,KAAK,CAAC,EAAE,MAAM,CAAA;IACd,IAAI,CAAC,EAAE,QAAQ,CAAA;IACf,SAAS,CAAC,EAAE,OAAO,CAAA;IACnB,QAAQ,CAAC,EAAE,cAAc,CAAA;CAC1B,CAAA;AAED,qBAAa,oBAAqB,SAAQ,SAAS;IACjD,SAAgB,MAAM,EAAE,SAAS,EAAE,CAAK;IAExC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAQ;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAQ;IACjC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAQ;IACjC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAQ;IAClC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAQ;IAChC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IAEnC,OAAO,CAAC,OAAO,CAA0B;IACzC,OAAO,CAAC,SAAS,CAAI;IACrB,OAAO,CAAC,UAAU,CAAI;IACtB,OAAO,CAAC,OAAO,CAAI;gBAEP,EACV,SAAS,EACT,KAAS,EACT,IAAI,EACJ,SAAgB,EAChB,QAAiB,GAClB,EAAE,2BAA2B;IAmCvB,UAAU,CACf,KAAK,EAAE,MAAM,EACb,SAAS,EAAE,cAAc,EACzB,QAAQ,EAAE,iBAAiB;IAStB,MAAM,CAAC,QAAQ,EAAE,iBAAiB;IAsBzC,OAAO,CAAC,IAAI;IAgCZ,OAAO,CAAC,GAAG;CAKZ"}
@@ -1 +0,0 @@
1
- {"version":3,"file":"byte-delimiter-scanner.js","sourceRoot":"","sources":["../../src/helpers/byte-delimiter-scanner.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAA0B,MAAM,aAAa,CAAA;AA4B/D,MAAM,OAAO,oBAAqB,SAAQ,SAAS;IACjC,MAAM,GAAgB,EAAE,CAAA;IAEvB,SAAS,CAAQ;IACjB,YAAY,CAAQ;IACpB,QAAQ,CAAQ;IAChB,QAAQ,CAAQ;IAChB,SAAS,CAAQ;IACjB,OAAO,CAAQ;IACf,SAAS,CAAS;IAE3B,OAAO,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;IACjC,SAAS,GAAG,CAAC,CAAA;IACb,UAAU,GAAG,CAAC,CAAA;IACd,OAAO,GAAG,CAAC,CAAA;IAEnB,YAAY,EACV,SAAS,EACT,KAAK,GAAG,CAAC,EACT,IAAI,EACJ,SAAS,GAAG,IAAI,EAChB,QAAQ,GAAG,MAAM,GACW;QAC5B,KAAK,EAAE,CAAA;QAEP,IAAI,CAAC,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAA;QACvD,IAAI,CAAC,YAAY,GAAG,SAAS,CAAC,IAAI,EAAE,QAAQ,IAAI,CAAC,CAAA;QACjD,IAAI,CAAC,QAAQ,GAAG,SAAS,CAAC,IAAI,EAAE,IAAI,IAAI,CAAC,CAAA;QACzC,IAAI,CAAC,QAAQ,GAAG,KAAK,CAAA;QACrB,IAAI,CAAC,SAAS,GAAG,IAAI,EAAE,KAAK,IAAI,CAAC,CAAA;QACjC,IAAI,CAAC,OAAO,GAAG,IAAI,EAAE,GAAG,IAAI,CAAC,CAAA;QAC7B,IAAI,CAAC,SAAS,GAAG,SAAS,CAAA;QAC1B,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,SAAS,CAAA;QAEhC,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC;YAC7B,MAAM,IAAI,SAAS,CAAC,mCAAmC,CAAC,CAAA;QAE1D,IAAI,IAAI,CAAC,YAAY,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,GAAG,CAAC;YAC5C,MAAM,IAAI,SAAS,CAAC,iDAAiD,CAAC,CAAA;QAExE,IAAI,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM;YAC3D,MAAM,IAAI,SAAS,CACjB,iEAAiE,CAClE,CAAA;QAEH,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC;YACvC,MAAM,IAAI,SAAS,CAAC,wCAAwC,CAAC,CAAA;QAE/D,IACE,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC;YACjC,CAAC,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO,CAAC;YAC/B,IAAI,CAAC,SAAS,GAAG,CAAC;YAClB,IAAI,CAAC,OAAO,GAAG,CAAC;YAEhB,MAAM,IAAI,SAAS,CAAC,gDAAgD,CAAC,CAAA;IACzE,CAAC;IAEM,UAAU,CACf,KAAa,EACb,SAAyB,EACzB,QAA2B;QAE3B,IAAI,CAAC,IAAI,CACP,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CACvE,CAAA;QAED,QAAQ,CAAC,IAAI,EAAE,KAAK,CAAC,CAAA;IACvB,CAAC;IAEM,MAAM,CAAC,QAA2B;QACvC,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,OAAO,CAAA;QAEjE,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAA;YAE/B,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,GAAG,IAAI,KAAK;gBAAE,MAAK;YAErC,IAAI,IAAI,CAAC,KAAK,IAAI,KAAK,EAAE,CAAC;gBACxB,IAAI,CAAC,MAAM,CAAC,GAAG,EAAE,CAAA;gBAEjB,SAAQ;YACV,CAAC;YAED,IAAI,CAAC,GAAG,GAAG,KAAK,CAAA;QAClB,CAAC;QAED,IAAI,IAAI,CAAC,UAAU,IAAI,KAAK;YAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,UAAU,EAAE,KAAK,CAAC,CAAA;QAE9D,QAAQ,EAAE,CAAA;IACZ,CAAC;IAEO,IAAI,CAAC,MAAc;QACzB,IAAI,UAAU,GAAG,CAAC,CAAA;QAElB,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,KAAK,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,EAAE,UAAU,CAAC,CAAA;YAExD,IAAI,KAAK,KAAK,CAAC,CAAC;gBAAE,MAAK;YAEvB,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;YAEjC,UAAU,GAAG,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAA;YAE1C,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS;gBAAE,SAAQ;YAEjC,IAAI,CAAC,OAAO,IAAI,CAAC,CAAA;YAEjB,IAAI,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,QAAQ,KAAK,CAAC;gBAAE,SAAQ;YAEhD,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,UAAU,EAAE,EAAE,GAAG,IAAI,CAAC,YAAY,CAAC,CAAA;YAEjD,IAAI,CAAC,UAAU,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAA;QAC9D,CAAC;QAED,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CACnB,UAAU,EACV,MAAM,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAC5C,CAAA;QAED,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAA;QACpC,IAAI,CAAC,SAAS,IAAI,IAAI,CAAA;IACxB,CAAC;IAEO,GAAG,CAAC,KAAa,EAAE,GAAW;QACpC,IAAI,IAAI,CAAC,SAAS,IAAI,GAAG,IAAI,KAAK;YAAE,OAAM;QAE1C,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,CAAC,CAAA;IAClC,CAAC;CACF"}