bumparena 0.0.6 → 0.0.8

package/README.md

@@ -1,74 +1,90 @@
-## High-Speed: Linear Bulk Import
+# BumpArena
 
-Use `reserve()` for high-volume, linear data ingestion. To guarantee maximum throughput and deterministic behavior, `reserve()` **always appends** memory at the end of the arena buffer. No compaction, no relocation, no surprises.
+**4x faster than standard arrays and uses only ~40% of the RAM!** 🚀💾
 
-This makes it ideal for tight loops, streaming parsers, and bulk loaders where predictable memory behavior is critical.
+BumpArena is a high-performance memory arena for JavaScript and TypeScript. It provides contiguous memory allocation, fast pointer-based access, and minimal garbage collection overhead, making it ideal for handling large datasets efficiently.
 
 ---
 
-### Allocation Strategies
+## Features
 
-#### Performance & GC Efficiency
+- Contiguous memory buffer for fast allocations
+- Pointer-based access with generation tracking
+- Recycled buckets for efficient memory reuse
+- Compact memory footprint
+- Compatible with TypeScript and JavaScript
 
-This Arena is explicitly designed to bypass the typical performance pitfalls of the V8 runtime:
+---
+
+## Installation
 
-**GC Invisibility**  
-The entire arena is backed by a single `ArrayBuffer`. Since the Garbage Collector does not traverse raw buffer contents, this completely avoids costly mark-and-sweep scans over arena memory.
+```bash
+npm install bumparena
+```
 
-**Zero Object Churn**  
-Data insertion does not allocate JavaScript objects. All writes are performed directly against the buffer. The engine reuses the same execution paths without producing heap garbage.
+or using Yarn:
 
-**Deferred Management Overhead**  
-Logical pointers (`ArenaLocation`, implemented as `BigInt`) are created *only* when `label()` is invoked. This keeps pointer bookkeeping out of performance-critical hot paths.
+```bash
+yarn add bumparena
+```
 
 ---
 
-### Technical Architecture
+## Quick Start
+
+```ts
+import { Arena } from "bumparena";
 
-#### Memory Layout (16-Byte Header)
+// Create a new arena with default settings
+const arena = new Arena();
 
-Each allocation block is prefixed with a fixed-size metadata header:
+// Allocate some data
+const data = new Uint8Array([1, 2, 3, 4, 5]);
+const ptr = arena.alloc(data);
 
-- Size  
-- Generation ID  
-- Flags / reserved space  
+// Read the data back
+const readData = arena.read(ptr);
+console.log(readData); // Uint8Array [1,2,3,4,5]
 
-`reserve()` eagerly initializes these headers to ensure full compatibility with `read()` and `label()` without requiring additional bookkeeping passes.
+// Free the allocation
+arena.free(ptr);
 
-This guarantees **O(1)** access and validation at read time.
+// Reserve a block for manual writes
+const reserved = arena.reserve(10);
+reserved.set(new Uint8Array([10, 20, 30]));
+console.log(reserved);
+```
 
 ---
 
-#### Pointer Security & Validation
+## Advanced Usage
 
-An `ArenaLocation` is encoded as a **64-bit BigInt**:
+- Direct allocation from existing buffers with `directAlloc()`
+- Custom headers for allocations (`header0`, `header1`, `header2`)
+- Iterate all allocations using `label()`
+- Estimate memory usage with `estimate(size, amount)`
 
-- **High 32 bits** → Physical byte offset inside the arena  
-- **Low 32 bits** → Generation ID  
+---
 
-When `read()` is called, the arena validates the pointer by comparing its generation ID with the one stored in the block header:
+## Benchmarks
 
-- ✅ Match → valid, data is returned  
-- ❌ Mismatch → block was freed or recycled → `null` is returned  
+| Implementation | Time | Heap Used | Notes |
+|-----------------------|------------|-----------|------------|
+| BumpArena (Optimized) | 183,098 ms | 4.82 GB | 50M items |
+| Standard Array | 765,961 ms | 11.22 GB | 50M items |
 
-This provides **use-after-free protection** without reference tracking, weak maps, or GC involvement.
+> ~4x faster and uses ~40% of the RAM compared to standard JavaScript arrays
 
 ---
 
-### 📖 Documentation & Tests
-
-The integrated test suite doubles as a **reference implementation**.  
-See the `// --- SHOWCASE & TESTS ---` section at the end of the source file for:
-
-- Stress tests for dynamic resizing  
-- Validation of pointer expiration semantics  
-- Advanced bucket reuse and fragmentation behavior  
-- Edge cases around generation rollover  
+## Compatibility
 
-To run the showcase directly, execute the file with Node.js. No external dependencies required.
+- Node.js ✅
+- Bun ✅
+- Browser (via compiled JS) ✅
 
 ---
 
-### ⚖️ License
+## License
 
-MIT License — use freely in commercial and non-commercial projects.
+MIT © eugen252009

package/dist/arena.d.ts

@@ -0,0 +1,64 @@
+export type ArenaLocation = bigint & {
+    readonly __data_pointer: unique symbol;
+};
+export interface ArenaOptions {
+    initalSize?: number;
+    littleEndian?: boolean;
+    allignment?: 8 | 16 | 32 | 64;
+    bucketOffsets?: number[];
+    bucketCapacities?: number[];
+}
+export interface ArenaCustomHeaders {
+    header0: number;
+    header1: number;
+    header2: number;
+}
+export interface ArenaHeaders {
+    totalLength: number;
+    payloadlength: number;
+    deleted: boolean;
+    header0: number;
+    header1: number;
+    header2: number;
+}
+export declare class Arena {
+    private HEADER_SIZE_BYTES;
+    private _buffer;
+    private _view8;
+    private _view32;
+    private _offset;
+    private _emptySpots;
+    private _allignMask;
+    private _allignShift;
+    private _bucketOffsets;
+    private _bucketCapacities;
+    private _bucketcount;
+    constructor(options?: ArenaOptions);
+    private _u;
+    private _idx32;
+    private _makePtr;
+    private _getOffset;
+    private _getBucketCount;
+    private _setBucketCount;
+    private _getBucketOffset;
+    private _setBucketOffset;
+    private _initBlock;
+    alloc(data: Uint8Array, headers?: ArenaCustomHeaders): ArenaLocation;
+    read(location: ArenaLocation): Uint8Array | null;
+    free(location: ArenaLocation): ArenaLocation;
+    private _checkForSpace;
+    private _resize;
+    size(): number;
+    getBuffer(): Uint8Array;
+    reserve(size: number): Uint8Array;
+    translate(ptr: ArenaLocation): {
+        start: number;
+        generation: bigint;
+    };
+    readWithHeaders(ptr: ArenaLocation): Uint8Array | null;
+    label(): Array<ArenaLocation>;
+    getHeaders(ptr: ArenaLocation): ArenaHeaders;
+    estimate(size: number, amnt: number): number;
+    directAlloc(source: Uint8Array, startn: number, endn: number): ArenaLocation;
+    clear(): void;
+}

package/dist/arena.js

@@ -0,0 +1,223 @@
+var HEADERS;
+(function (HEADERS) {
+    HEADERS[HEADERS["TOTAL_LENGTH_0_32"] = 0] = "TOTAL_LENGTH_0_32";
+    HEADERS[HEADERS["PAYLOAD_LENGTH_0_32"] = 1] = "PAYLOAD_LENGTH_0_32";
+    HEADERS[HEADERS["GENERATION_BYTE_0_32"] = 2] = "GENERATION_BYTE_0_32";
+    HEADERS[HEADERS["DELETED_8"] = 12] = "DELETED_8";
+    HEADERS[HEADERS["USER_STATUS_0_8"] = 13] = "USER_STATUS_0_8";
+    HEADERS[HEADERS["USER_STATUS_1_8"] = 14] = "USER_STATUS_1_8";
+    HEADERS[HEADERS["USER_STATUS_2_8"] = 15] = "USER_STATUS_2_8";
+})(HEADERS || (HEADERS = {}));
+export class Arena {
+    constructor(options) {
+        this.HEADER_SIZE_BYTES = 16;
+        this._buffer = new ArrayBuffer(options?.initalSize || 64 * 1024);
+        this._view8 = new Uint8Array(this._buffer);
+        this._view32 = new Uint32Array(this._buffer);
+        this._offset = 0;
+        //@ts-ignore
+        this._allignMask = ((options?.allignment || 8) - 1);
+        this._allignShift = Math.log2(this._allignMask + 1);
+        this._bucketCapacities = options?.bucketCapacities || [1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024];
+        this._bucketOffsets = [];
+        this._bucketcount = this._bucketCapacities.length;
+        let currentOffset = 0;
+        for (const cap of this._bucketCapacities) {
+            this._bucketOffsets.push(currentOffset);
+            currentOffset += (cap + 1);
+        }
+        this._emptySpots = new Uint32Array(currentOffset);
+    }
+    _u(n) {
+        return n >>> 0;
+    }
+    _idx32(byteOffset) {
+        return (byteOffset >>> 0) >> 2;
+    }
+    _makePtr(offset, gen) {
+        const bOffset = BigInt(offset >>> 0);
+        const bGen = BigInt(gen ?? 0);
+        return ((bOffset << 32n) | (bGen & 0xffffffffn));
+    }
+    _getOffset(ptr) {
+        return Number(BigInt(ptr) >> 32n) >>> 0;
+    }
+    _getBucketCount(bucketIdx) {
+        return this._emptySpots[this._bucketOffsets[bucketIdx]];
+    }
+    _setBucketCount(bucketIdx, count) {
+        this._emptySpots[this._bucketOffsets[bucketIdx]] = count;
+    }
+    _getBucketOffset(bucketIdx, slotIdx) {
+        return this._emptySpots[this._bucketOffsets[bucketIdx] + slotIdx + 1];
+    }
+    _setBucketOffset(bucketIdx, slotIdx, offset) {
+        this._emptySpots[this._bucketOffsets[bucketIdx] + slotIdx + 1] = offset;
+    }
+    _initBlock(start, dataLength, headers) {
+        const idx = this._idx32(start);
+        const { header0: h0, header1: h1, header2: h2 } = headers;
+        this._view32[idx + HEADERS.TOTAL_LENGTH_0_32] = this._u((dataLength + this.HEADER_SIZE_BYTES + this._allignMask) & ~this._allignMask);
+        this._view32[idx + HEADERS.PAYLOAD_LENGTH_0_32] = dataLength;
+        this._view32[idx + (HEADERS.DELETED_8 >> 2)] = ((h2 || 0) << 24) | ((h1 || 0) << 16) | ((h0 || 0) << 8);
+        this._view8[start + HEADERS.DELETED_8] = 0;
+    }
+    alloc(data, headers) {
+        headers || (headers = { header0: 0, header1: 0, header2: 0 });
+        const needed = (data.byteLength + this.HEADER_SIZE_BYTES + this._allignMask) & ~this._allignMask;
+        const bucketIdx = (needed >> this._allignShift) - 1;
+        if (bucketIdx >= 0 && bucketIdx < this._bucketcount) {
+            const count = this._getBucketCount(bucketIdx);
+            if (count > 0) {
+                const recycledOffset = this._u(this._getBucketOffset(bucketIdx, count - 1));
+                this._setBucketCount(bucketIdx, count - 1);
+                this._initBlock(recycledOffset, data.byteLength, headers);
+                const gen = this._view32[this._idx32(recycledOffset) + HEADERS.GENERATION_BYTE_0_32];
+                return this._makePtr(recycledOffset, gen);
+            }
+        }
+        const start = this._u(this._offset);
+        if (this._checkForSpace(needed))
+            this._resize();
+        this._initBlock(start, data.byteLength, headers);
+        this._view8.set(data, start + this.HEADER_SIZE_BYTES);
+        this._offset = this._u((start + needed) & ~this._allignMask);
+        const gen = this._view32[this._idx32(start) + HEADERS.GENERATION_BYTE_0_32];
+        return this._makePtr(start, gen);
+    }
+    read(location) {
+        const { start, generation } = this.translate(location);
+        const idx = this._idx32(start);
+        const currgen = BigInt(this._view32[idx + HEADERS.GENERATION_BYTE_0_32]);
+        if (generation !== currgen)
+            return null;
+        if (this._view8[start + HEADERS.DELETED_8] === 1)
+            return null;
+        const dataLength = this._view32[idx + HEADERS.PAYLOAD_LENGTH_0_32];
+        return this._view8.subarray(start + this.HEADER_SIZE_BYTES, start + this.HEADER_SIZE_BYTES + dataLength);
+    }
+    free(location) {
+        const { start, generation: _ } = this.translate(location);
+        const idx = this._idx32(start);
+        const currgen = this._view32[idx + HEADERS.GENERATION_BYTE_0_32];
+        this._view32[idx + HEADERS.GENERATION_BYTE_0_32] = currgen + 1;
+        this._view8[this._u(start) + HEADERS.DELETED_8] = 1;
+        const totalBlockSize = this._view32[this._idx32(start) + HEADERS.TOTAL_LENGTH_0_32];
+        const bucketIdx = (totalBlockSize >> this._allignShift) - 1;
+        if (bucketIdx >= 0 && bucketIdx < this._bucketcount) {
+            const count = this._getBucketCount(bucketIdx);
+            const capacity = this._bucketCapacities[bucketIdx];
+            if (count < capacity) {
+                this._setBucketOffset(bucketIdx, count, start);
+                this._setBucketCount(bucketIdx, count + 1);
+            }
+        }
+        return this._makePtr(0, 0);
+    }
+    _checkForSpace(size) {
+        if (this._buffer.byteLength >= (this._offset + size))
+            return false;
+        return true;
+    }
+    _resize() {
+        //@ts-ignore
+        if (this._buffer.transfer)
+            this._buffer = this._buffer.transfer(this._buffer.byteLength * 2);
+        else {
+            const newBuffer = new ArrayBuffer(this._buffer.byteLength * 2);
+            new Uint8Array(newBuffer).set(this._view8);
+            this._buffer = newBuffer;
+        }
+        this._view8 = new Uint8Array(this._buffer);
+        this._view32 = new Uint32Array(this._buffer);
+    }
+    size() {
+        return this._buffer.byteLength;
+    }
+    getBuffer() {
+        return this._view8.subarray(0, this._u(this._offset));
+    }
+    reserve(size) {
+        const start = this._u(this._offset);
+        this._checkForSpace((this.HEADER_SIZE_BYTES + size + this._allignMask) & ~this._allignMask) && this._resize();
+        this._initBlock(start, size, { header0: 0, header1: 0, header2: 0 });
+        this._offset = ((this._offset + size + this.HEADER_SIZE_BYTES + this._allignMask) & ~this._allignMask) >>> 0;
+        return this._view8.subarray(start + this.HEADER_SIZE_BYTES, start + this.HEADER_SIZE_BYTES + size);
+    }
+    translate(ptr) {
+        const start = this._getOffset(ptr);
+        const generation = ptr & 0xffffffffn;
+        return { start, generation };
+    }
+    readWithHeaders(ptr) {
+        const { start, generation } = this.translate(ptr);
+        let idx32 = this._idx32(start);
+        let length = this._view32[idx32 + HEADERS.PAYLOAD_LENGTH_0_32];
+        if (BigInt(this._view32[idx32 + HEADERS.GENERATION_BYTE_0_32]) !== generation)
+            return null;
+        return this._view8.subarray(start + 12, start + this.HEADER_SIZE_BYTES + Number(length));
+    }
+    label() {
+        const ptrArray = new Array();
+        const limit = this._offset;
+        let pos = 0;
+        while (this._u(pos) < this._u(limit)) {
+            const totalLength = this._view32[this._idx32(pos)];
+            if (totalLength === 0) {
+                pos = this._idx32((pos + 16) & ~15);
+                continue;
+            }
+            const isDeleted = this._view8[this._u(pos) + HEADERS.DELETED_8] === 1;
+            if (!isDeleted) {
+                ptrArray.push(this._makePtr(pos, this._view32[this._idx32(pos) + HEADERS.GENERATION_BYTE_0_32]));
+            }
+            pos += totalLength;
+            if (totalLength === 0)
+                break;
+        }
+        return ptrArray;
+    }
+    getHeaders(ptr) {
+        const { start, generation: _ } = this.translate(ptr);
+        const idx = this._u(start);
+        return {
+            totalLength: Number(this._view32[this._idx32(start) + HEADERS.TOTAL_LENGTH_0_32].toString()),
+            payloadlength: Number(this._view32[this._idx32(start) + HEADERS.PAYLOAD_LENGTH_0_32].toString()),
+            deleted: this._view8[idx + HEADERS.DELETED_8] === 1,
+            header0: Number(this._view8[idx + HEADERS.USER_STATUS_0_8].toString()),
+            header1: Number(this._view8[idx + HEADERS.USER_STATUS_1_8].toString()),
+            header2: Number(this._view8[idx + HEADERS.USER_STATUS_2_8].toString()),
+        };
+    }
+    estimate(size, amnt) {
+        return (((size + this._allignMask) & ~this._allignMask) + this.HEADER_SIZE_BYTES) * amnt;
+    }
+    directAlloc(source, startn, endn) {
+        const start = this._u(this._offset);
+        const len = endn - startn;
+        const needed = (len + this.HEADER_SIZE_BYTES + this._allignMask) & ~this._allignMask;
+        const bucketIdx = (needed >> this._allignShift) - 1;
+        if (bucketIdx >= 0 && bucketIdx < this._bucketcount) {
+            const count = this._getBucketCount(bucketIdx);
+            if (count > 0) {
+                const recycledOffset = this._u(this._getBucketOffset(bucketIdx, count - 1));
+                this._setBucketCount(bucketIdx, count - 1);
+                this._initBlock(recycledOffset, len, { header0: 0, header1: 0, header2: 0 });
+                const gen = this._view32[this._idx32(recycledOffset) + HEADERS.GENERATION_BYTE_0_32];
+                this._view8.set(source.subarray(startn, endn), recycledOffset + this.HEADER_SIZE_BYTES);
+                return this._makePtr(recycledOffset, gen);
+            }
+        }
+        if (this._checkForSpace(needed))
+            this._resize();
+        this._initBlock(start, len, { header0: 0, header1: 0, header2: 0 });
+        this._offset = this._u((start + needed) & ~this._allignMask);
+        this._view8.set(source.subarray(startn, endn), start + this.HEADER_SIZE_BYTES);
+        const gen = this._view32[this._idx32(start) + HEADERS.GENERATION_BYTE_0_32];
+        return this._makePtr(start, gen);
+    }
+    clear() {
+        this._offset = 0;
+        this._emptySpots.fill(0);
+    }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bumparena",
-  "version":"0.0.6",
+  "version":"0.0.8",
   "private": false,
   "type": "module",
   "main": "dist/arena.js",
@@ -10,7 +10,7 @@
   "scripts": {
     "build": "tsc",
     "test": "node ./dist/test/test.test.js",
-    "pub": "tsc; npm publish"
+    "push": "bun test && tsc && npm pub"
   },
  "repository": {
     "type": "git",

package/bench/bench.md

@@ -1,38 +0,0 @@
-# Benchmark 1
-
-## bun arena.ts;
-183098.55 ms
-{
-  rss: 11240919040,
-  heapTotal: 8595075072,
-  heapUsed: 4817152633,
-  external: 2157456969,
-  arrayBuffers: 2156686216,
-}
-arenasize: 2097152 KB
-
-
-
-## bun array.ts
-765961.27 ms
-Count: 50000000
-{
-  rss: 28036014080,
-  heapTotal: 9476152320,
-  heapUsed: 11222438346,
-  external: 7170077276,
-  arrayBuffers: 2921230601,
-}
-
---- 
-
-## Direct Comparison
-
-| Metric | Arena Implementation (Optimized) | Standard Array Implementation | Difference / Factor |
-| :--- | :--- | :--- | :--- |
-| **Total Time** | **183,098 ms** (3.05 min) | **765,961 ms** (12.76 min) | **~4.2x Faster** |
-| **Time per 50M Lines** | **~18.3 sec** | **~76.6 sec** | **- 58.3 sec / Round** |
-| **RAM Usage (RSS)** | **11.24 GB** | **28.04 GB** | **16.8 GB Saved** |
-| **Heap Used (JS Objects)** | **4.82 GB** | **11.22 GB** | **2.3x More Efficient** |
-| **External (Buffer)** | **2.16 GB** | **7.17 GB** | **Compact Memory Footprint** |
-