@nxtedition/slice 1.0.3 → 1.0.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +126 -0
- package/package.json +14 -6
- package/lib/index.test.d.ts +0 -1
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) nxtedition
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# @nxtedition/slice
|
|
2
|
+
|
|
3
|
+
A high-performance buffer slice and pool allocator for Node.js.
|
|
4
|
+
|
|
5
|
+
## Why
|
|
6
|
+
|
|
7
|
+
Node.js `Buffer.subarray()` is slow. Every call creates a new `Buffer` object — a typed array wrapper with prototype chain setup, internal slot initialization, and bounds validation. This overhead is negligible for occasional use, but becomes a bottleneck in hot paths — protocol parsers, binary codecs, streaming pipelines — where thousands of sub-views are created per second.
|
|
8
|
+
|
|
9
|
+
`Buffer.allocUnsafe()` is worse. Allocations above the pool size (`Buffer.poolSize`) threshold go through `allocBuffer` which crosses into C++ to create a new `ArrayBuffer` backing store. The pooled fast path still involves bookkeeping and pool management overhead, and every allocation produces a new `Buffer` object that the GC must eventually collect.
|
|
10
|
+
|
|
11
|
+
`Slice` avoids this entirely. It is a plain JavaScript object with `buffer`, `byteOffset`, and `byteLength` fields. Creating a slice is just setting three properties — no typed array wrapper creation, no GC pressure from short-lived `Buffer` objects. Operations like `toString`, `copy`, and `compare` delegate directly to the underlying buffer with the correct offsets.
|
|
12
|
+
|
|
13
|
+
`PoolAllocator` takes this further. Like Node's internal pool, it has management overhead — but it rarely (if ever) allocates new backing stores, and because `Slice` is a plain object rather than a typed array, resizing or freeing a slice doesn't produce garbage for V8 to collect. It pre-allocates a large contiguous buffer and hands out regions using power-of-2 bucketing. When a slice is freed, its slot is recycled. When a slice is resized within the same bucket, no data moves at all — just a field update. This gives you `malloc`/`realloc`/`free` semantics with near-zero overhead per operation. The trade-off is upfront memory allocation and internal fragmentation from power-of-2 rounding — a 10-byte allocation uses a 16-byte slot. Buckets are also independent: a freed 16-byte slot cannot satisfy a 32-byte request, so the pool can become fragmented if allocation sizes are uneven. Use `stats` to monitor pool utilization and tune the pool size for your workload.
|
|
14
|
+
|
|
15
|
+
## Install
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
npm install @nxtedition/slice
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Usage
|
|
22
|
+
|
|
23
|
+
```js
|
|
24
|
+
import { Slice, PoolAllocator } from '@nxtedition/slice'
|
|
25
|
+
|
|
26
|
+
// Create a slice from an existing buffer
|
|
27
|
+
const buf = Buffer.from('hello world')
|
|
28
|
+
const slice = new Slice(buf, 6, 5)
|
|
29
|
+
slice.toString() // 'world'
|
|
30
|
+
|
|
31
|
+
// Use a pool allocator for high-throughput allocation
|
|
32
|
+
const pool = new PoolAllocator()
|
|
33
|
+
const s = new Slice()
|
|
34
|
+
|
|
35
|
+
pool.realloc(s, 64) // allocate 64 bytes from pool
|
|
36
|
+
s.write('hello')
|
|
37
|
+
pool.realloc(s, 128) // grow — may reuse same slot
|
|
38
|
+
pool.realloc(s, 0) // free — slot is recycled
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Benchmarks
|
|
42
|
+
|
|
43
|
+
Measured on Apple M3 Pro, Node.js v25.3.0:
|
|
44
|
+
|
|
45
|
+
### Allocation
|
|
46
|
+
|
|
47
|
+
| Operation | `Buffer.allocUnsafe` | `Buffer.allocUnsafeSlow` | `PoolAllocator` | Speedup |
|
|
48
|
+
| ---------------- | -------------------- | ------------------------ | --------------- | ------- |
|
|
49
|
+
| alloc 64 bytes | 28.53 ns | 31.33 ns | **3.87 ns** | 7.4x |
|
|
50
|
+
| alloc 256 bytes | 36.98 ns | 170.56 ns | **4.34 ns** | 8.5x |
|
|
51
|
+
| alloc 1024 bytes | 65.48 ns | 238.49 ns | **4.36 ns** | 15.0x |
|
|
52
|
+
| alloc 4096 bytes | 301.19 ns | 292.50 ns | **4.29 ns** | 70.2x |
|
|
53
|
+
|
|
54
|
+
### Slice creation vs `Buffer.subarray`
|
|
55
|
+
|
|
56
|
+
| Operation | `Buffer.subarray` | `Slice` | Speedup |
|
|
57
|
+
| ------------------- | ----------------- | ----------- | ------- |
|
|
58
|
+
| subarray 64 bytes | 27.32 ns | **9.68 ns** | 2.8x |
|
|
59
|
+
| subarray 1024 bytes | 26.75 ns | **9.52 ns** | 2.8x |
|
|
60
|
+
|
|
61
|
+
### Combined operations
|
|
62
|
+
|
|
63
|
+
| Operation | `Buffer.subarray` | `Slice` | Speedup |
|
|
64
|
+
| ------------------------------------- | ----------------- | ------------- | ------- |
|
|
65
|
+
| subarray + toString (64 bytes) | 78.89 ns | **60.12 ns** | 1.3x |
|
|
66
|
+
| alloc/free 64 bytes | 25.24 ns | **22.82 ns** | 1.1x |
|
|
67
|
+
| alloc/free 256 bytes | 34.07 ns | **22.63 ns** | 1.5x |
|
|
68
|
+
| realloc churn (64 → 128 → 64) | 70.20 ns | **20.80 ns** | 3.4x |
|
|
69
|
+
| realloc in-place (grow within bucket) | 45.41 ns | **8.28 ns** | 5.5x |
|
|
70
|
+
| 10 concurrent allocs then free | 312.96 ns | **259.34 ns** | 1.2x |
|
|
71
|
+
|
|
72
|
+
## API
|
|
73
|
+
|
|
74
|
+
### `Slice`
|
|
75
|
+
|
|
76
|
+
A lightweight view over a `Buffer` with explicit offset and length tracking.
|
|
77
|
+
|
|
78
|
+
#### `new Slice(buffer?: Buffer, byteOffset?: number, byteLength?: number, maxByteLength?: number)`
|
|
79
|
+
|
|
80
|
+
Creates a new slice. All parameters are optional — defaults to an empty slice.
|
|
81
|
+
|
|
82
|
+
#### Properties
|
|
83
|
+
|
|
84
|
+
- `buffer: Buffer` — The underlying `Buffer`
|
|
85
|
+
- `byteOffset: number` — Start offset into the buffer
|
|
86
|
+
- `byteLength: number` — Current length in bytes
|
|
87
|
+
- `maxByteLength: number` — Maximum capacity in bytes
|
|
88
|
+
- `length: number` — Alias for `byteLength`
|
|
89
|
+
|
|
90
|
+
#### Methods
|
|
91
|
+
|
|
92
|
+
- `reset(): void` — Clear the slice back to empty state. **Note:** this does not return the slot to the `PoolAllocator` — you must call `realloc(slice, 0)` to free pool memory.
|
|
93
|
+
- `copy(target: Buffer | Slice, targetStart?: number, sourceStart?: number, sourceEnd?: number): number` — Copy data to a `Buffer` or `Slice`. Returns bytes copied.
|
|
94
|
+
- `compare(target: Buffer | Slice, targetStart?: number, targetEnd?: number, sourceStart?: number, sourceEnd?: number): -1 | 0 | 1` — Compare with a `Buffer` or `Slice`
|
|
95
|
+
- `write(string: string, offset?: number, length?: number, encoding?: BufferEncoding): number` — Write a string into the slice. Returns bytes written.
|
|
96
|
+
- `set(source: Buffer | Slice | null | undefined, offset?: number): void` — Copy from a `Buffer` or `Slice` into this slice
|
|
97
|
+
- `at(index: number): number` — Read byte at index (supports negative indexing)
|
|
98
|
+
- `test(expr: { test(buffer: Buffer, byteOffset: number, byteLength: number): boolean }): boolean` — Test the slice against an expression object
|
|
99
|
+
- `toString(encoding?: BufferEncoding, start?: number, end?: number): string` — Convert to string
|
|
100
|
+
- `toBuffer(start?: number, end?: number): Buffer` — Return a `Buffer` view
|
|
101
|
+
|
|
102
|
+
#### Static
|
|
103
|
+
|
|
104
|
+
- `Slice.EMPTY_BUF: Buffer` — Shared empty buffer singleton
|
|
105
|
+
|
|
106
|
+
### `PoolAllocator`
|
|
107
|
+
|
|
108
|
+
Pre-allocates a contiguous memory pool and manages slices using power-of-2 bucketing.
|
|
109
|
+
|
|
110
|
+
#### `new PoolAllocator(poolTotal?: number)`
|
|
111
|
+
|
|
112
|
+
Creates a pool allocator. Default pool size is 128 MB.
|
|
113
|
+
|
|
114
|
+
#### Methods
|
|
115
|
+
|
|
116
|
+
- `realloc(slice: Slice, byteLength: number): Slice` — Allocate, resize, or free a slice. Pass `0` to free.
|
|
117
|
+
- `isFromPool(slice: Slice | null | undefined): boolean` — Check if a slice was allocated from this pool
|
|
118
|
+
|
|
119
|
+
#### Properties
|
|
120
|
+
|
|
121
|
+
- `size: number` — Total size of all active allocations
|
|
122
|
+
- `stats: { size: number, padding: number, ratio: number, poolTotal: number, poolUsed: number, poolSize: number, poolCount: number }` — Detailed allocation statistics
|
|
123
|
+
|
|
124
|
+
## License
|
|
125
|
+
|
|
126
|
+
MIT
|
package/package.json
CHANGED
|
@@ -1,24 +1,32 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nxtedition/slice",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.7",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"main": "lib/index.js",
|
|
6
6
|
"types": "lib/index.d.ts",
|
|
7
7
|
"files": [
|
|
8
|
-
"lib"
|
|
8
|
+
"lib",
|
|
9
|
+
"README.md",
|
|
10
|
+
"LICENSE"
|
|
9
11
|
],
|
|
10
|
-
"license": "
|
|
12
|
+
"license": "MIT",
|
|
13
|
+
"publishConfig": {
|
|
14
|
+
"access": "public"
|
|
15
|
+
},
|
|
11
16
|
"scripts": {
|
|
12
17
|
"build": "rimraf lib && tsc && amaroc ./src/index.ts && mv src/index.js lib/",
|
|
13
18
|
"prepublishOnly": "yarn build",
|
|
14
19
|
"typecheck": "tsc --noEmit",
|
|
15
|
-
"test": "node --test",
|
|
16
|
-
"test:ci": "node --test"
|
|
20
|
+
"test": "yarn build && node --test",
|
|
21
|
+
"test:ci": "yarn build && node --test",
|
|
22
|
+
"test:coverage": "node --test --experimental-test-coverage"
|
|
17
23
|
},
|
|
18
24
|
"devDependencies": {
|
|
25
|
+
"@types/node": "^25.2.3",
|
|
19
26
|
"amaroc": "^1.0.1",
|
|
27
|
+
"oxlint-tsgolint": "^0.12.2",
|
|
20
28
|
"rimraf": "^6.1.2",
|
|
21
29
|
"typescript": "^5.9.3"
|
|
22
30
|
},
|
|
23
|
-
"gitHead": "
|
|
31
|
+
"gitHead": "8dbd8386c6d4c511ffa81a904c58b6f648811a52"
|
|
24
32
|
}
|
package/lib/index.test.d.ts
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
export {};
|