bumparena 0.0.7 → 0.9.0

package/README.md

@@ -1,8 +1,29 @@
-# BumpArena
+# 🚀 BumpArena: High-Performance Memory Management for Bun/TS
 
-**4x faster than standard arrays and uses only ~40% of the RAM!** 🚀💾
+[![npm version](https://img.shields.io/npm/v/bumparena.svg?style=flat-square)](https://www.npmjs.com/package/bumparena)
+[![license](https://img.shields.io/npm/l/bumparena.svg?style=flat-square)](https://github.com/dein-nutzername/bumparena/blob/main/LICENSE)
 
-BumpArena is a high-performance memory arena for JavaScript and TypeScript. It provides contiguous memory allocation, fast pointer-based access, and minimal garbage collection overhead, making it ideal for handling large datasets efficiently.
+**4.2x faster than standard arrays while using ~60% less RAM!** 💾⚡
+
+BumpArena is a high-performance memory arena designed for JavaScript and TypeScript (optimized for Bun). It provides contiguous memory allocation, fast pointer-based access, and minimal Garbage Collection (GC) overhead, making it the ideal choice for handling industrial-scale datasets and real-time telemetry.
+
+---
+
+## 📊 Performance Benchmark (50M Objects)
+
+I compared the **BumpArena** implementation against a standard **Array-based** approach by processing a dataset of 50 million records.
+
+| Metric | Standard Array | **BumpArena** | Improvement |
+| :--- | :--- | :--- | :--- |
+| **Total Time** | 765.9s (12.7 min) | **183.1s (3.05 min)** | **4.2x Faster** 🚀 |
+| **Throughput** | ~652k lines/s | **~2.73M lines/s** | **+318% Speed** |
+| **RAM Usage (RSS)** | 28.04 GB | **11.24 GB** | **16.8 GB Saved** 📉 |
+| **Heap Efficiency** | 11.22 GB | **4.82 GB** | **2.3x Lower Usage** |
+
+### 🛠 Why it's faster:
+* **Zero GC Pressure:** By using a pre-allocated buffer, we bypass the costly JavaScript Garbage Collector.
+* **Cache Locality:** Contiguous memory layout ensures the CPU stays fast and avoids cache misses.
+* **Industrial Scale:** Designed to handle 50M+ data points without breaking the heap.
 
 ---
 
@@ -34,25 +55,30 @@ yarn add bumparena
 
 ```ts
 import { Arena } from "bumparena";
+import fs from "node:fs";
+
+// 1. Initialize (e.g., 1GB Arena)
+const arena = new Arena({ initalSize: 1024 * 1024 * 1024 });
 
-// Create a new arena with default settings
-const arena = new Arena();
+// 2. Map data instantly (Zero-Copy)
+const data = new Uint8Array([10, 20, 30, 40]);
+const ptr = arena.alloc(data); 
 
-// Allocate some data
-const data = new Uint8Array([1, 2, 3, 4, 5]);
-const ptr = arena.alloc(data);
+// 3. Retrieve (O(1) access)
+const view = arena.read(ptr);
 
-// Read the data back
-const readData = arena.read(ptr);
-console.log(readData); // Uint8Array [1,2,3,4,5]
+// 4. Persistence "From Nothing"
+// Save the entire memory state 1:1 as a binary image
+fs.writeFileSync("database.bin", arena.getBuffer());
 
-// Free the allocation
-arena.free(ptr);
+// 5. Reload (Zero Parsing Time)
+// Simply load the bytes back into a new Arena buffer
+const savedData = fs.readFileSync("database.bin");
+const restoredArena = new Arena({ initalSize: savedData.byteLength });
+restoredArena.putBytes(savedData); // Structure is restored instantly
 
-// Reserve a block for manual writes
-const reserved = arena.reserve(10);
-reserved.set(new Uint8Array([10, 20, 30]));
-console.log(reserved);
+//Clear your Arena, if you want a restart
+arena.clear()
 ```
 
 ---
@@ -66,6 +92,99 @@ console.log(reserved);
 
 ---
 
+## Binary Header Specification
+
+BumpArena uses a fixed 16-byte packed header. Alignment is applied to the entire block, ensuring that every new Header starts at a memory address divisible by your chosen alignment (e.g., 8, 16, 32 bytes).
+
+### Memory Layout & Alignment
+
+The total_length field does not just represent the sum of the header and data; it includes the padding required to align the next block in the Arena.
+
+| Offset (Byte) | Field | Type | Description |
+| :--- | :--- | :--- | :--- |
+| `0x00` | `total_length` | `uint32` | Header + Payload + Alignment Padding |
+| `0x04` | `payload_length` | `uint32` | Exact size of the user data |
+| `0x08` | `generation` | `uint32` | Validation counter (prevents ABA/stale pointer issues) |
+| `0x0C` | `deleted` | `uint8` | Status flag (`0x01` = deleted, `0x00` = active) |
+| `0x0D` | `user_header0` | `uint8` | Custom metadata slot 1 |
+| `0x0E` | `user_header1` | `uint8` | Custom metadata slot 2 |
+| `0x0F` | `user_header2` | `uint8` | Custom metadata slot 3 |
+| `0x10` | **Payload** | `u8[]` | User data starts here (Fixed Offset) |
+| `...` | **Padding** | `u8[N]` | Internal padding to align the next Block |
+
+### Implementation Examples
+
+Typescript
+
+```ts
+// Example: How BumpArena maps the header internally
+const view = new DataView(arena.getBuffer().buffer);
+const ptr = 0x1234; // Current allocation pointer
+
+const header = {
+    totalLength:   view.getUint32(ptr + 0,  true), // Little-endian
+    payloadLength: view.getUint32(ptr + 4,  true),
+    generation:    view.getUint32(ptr + 8,  true),
+    deleted:       view.getUint8(ptr + 12),
+    userHeader:    new Uint8Array(arena.getBuffer().buffer, ptr + 13, 3)
+};
+
+// The actual payload starts at ptr + 16
+const payload = new Uint8Array(arena.getBuffer().buffer, ptr + 16, header.payloadLength);
+```
+
+C/++
+
+```c
+typedef struct __attribute__((packed)) {
+  uint32_t total_length; // Jump to next header: current_ptr + total_length
+  uint32_t payload_length;
+  uint32_t generation;
+  uint8_t deleted;
+  uint8_t user_header0;
+  uint8_t user_header1;
+  uint8_t user_header2;uint8_t  payload[];      // Data starts at offset 16 (0x10)
+} ArenaData;
+```
+
+Rust
+
+```rust
+#[repr(C, packed)]
+pub struct ArenaData {
+    pub total_length: u32, // Jump to next header: current_ptr + total_length
+    pub payload_length: u32,
+    pub generation: u32,
+    pub deleted: u8,
+    pub user_header0: u8,
+    pub user_header1: u8,
+    pub user_header2: u8,
+    // Payload follows immediately at offset 16
+}
+```
+
+go
+
+```go
+type ArenaData struct {
+	TotalLength   uint32  // Jump to next header: current_ptr + total_length
+	PayloadLength uint32  // the length of the Payload itself
+	Generation    uint32  // The Generation bits
+	Deleted       uint8   // 0x1=true,0x0=false
+	UserHeader   [3]uint8 // User Metadata
+  // Payload follows immediately at offset 16
+}
+```
+
+Key Advantages
+Zero-Copy: Directly cast your binary data to these structures in any language.
+
+Stale Pointer Protection: The generation field allows you to verify if a pointer still refers to the original data or if the memory has been reused.
+
+Alignment: 16-byte boundaries are a "sweet spot" for modern hardware architectures,.
+
+---
+
 ## Benchmarks
 
 | Implementation | Time | Heap Used | Notes |

package/dist/arena.d.ts

@@ -0,0 +1,64 @@
+export type ArenaLocation = bigint & {
+    readonly __data_pointer: unique symbol;
+};
+export interface ArenaOptions {
+    initalSize?: number;
+    littleEndian?: boolean;
+    allignment?: 8 | 16 | 32 | 64;
+    bucketOffsets?: number[];
+    bucketCapacities?: number[];
+}
+export interface ArenaCustomHeaders {
+    header0: number;
+    header1: number;
+    header2: number;
+}
+export interface ArenaHeaders {
+    totalLength: number;
+    payloadlength: number;
+    deleted: boolean;
+    header0: number;
+    header1: number;
+    header2: number;
+}
+export declare class Arena {
+    private HEADER_SIZE_BYTES;
+    private _buffer;
+    private _view8;
+    private _view32;
+    private _offset;
+    private _emptySpots;
+    private _allignMask;
+    private _allignShift;
+    private _bucketOffsets;
+    private _bucketCapacities;
+    private _bucketcount;
+    constructor(options?: ArenaOptions);
+    private _u;
+    private _idx32;
+    private _makePtr;
+    private _getOffset;
+    private _getBucketCount;
+    private _setBucketCount;
+    private _getBucketOffset;
+    private _setBucketOffset;
+    private _initBlock;
+    alloc(data: Uint8Array, headers?: ArenaCustomHeaders): ArenaLocation;
+    read(location: ArenaLocation): Uint8Array | null;
+    free(location: ArenaLocation): ArenaLocation;
+    private _checkForSpace;
+    private _resize;
+    size(): number;
+    getBuffer(): Uint8Array;
+    reserve(size: number): Uint8Array;
+    translate(ptr: ArenaLocation): {
+        start: number;
+        generation: bigint;
+    };
+    readWithHeaders(ptr: ArenaLocation): Uint8Array | null;
+    label(): Array<ArenaLocation>;
+    getHeaders(ptr: ArenaLocation): ArenaHeaders;
+    estimate(size: number, amnt: number): number;
+    directAlloc(source: Uint8Array, startn: number, endn: number): ArenaLocation;
+    clear(): void;
+}

package/interface.d.ts

@@ -0,0 +1,48 @@
+export type ArenaLocation = bigint & { readonly __data_pointer: unique symbol };
+export interface ArenaOptions {
+	initalSize?: number
+	littleEndian?: boolean
+	allignment?: 8 | 16 | 32 | 64
+	bucketOffsets?: number[];
+	bucketCapacities?: number[];
+}
+export interface ArenaCustomHeaders {
+	header0: number,
+	header1: number,
+	header2: number
+}
+export interface InspectStruct {
+	offset: number;
+	generation_ptr: number;
+	generation: number;
+	isSafe: boolean;
+	totalLength: number;
+	payloadLength: number;
+	isDeleted: boolean;
+	UserMetaData0: number;
+	UserMetaData1: number;
+	UserMetaData2: number;
+	payload?: Uint8Array;
+}
+export interface ArenaHeaders {
+	totalLength: number
+	payloadLength: number
+	deleted: boolean
+	header0: number
+	header1: number
+	header2: number
+}
+export interface IStorageStrategy {
+	alloc(data: Uint8Array, headers?: ArenaCustomHeaders): ArenaLocation
+	read(location: ArenaLocation): Uint8Array | null
+	free(location: ArenaLocation): ArenaLocation
+	estimate(size: number, amnt: number): number
+	reset(): void
+	clear(): void
+
+	/** Eager Collection */
+	collectActiveRecords(): ArenaLocation[]
+
+	/** Lazy Iterator */
+	records(): Generator<[Uint8Array, ArenaLocation]>
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bumparena",
-  "version":"0.0.7",
+  "version":"0.9.0",
   "private": false,
   "type": "module",
   "main": "dist/arena.js",

package/bench/bench.md

@@ -1,38 +0,0 @@
-# Benchmark 1
-
-## bun arena.ts;
-183098.55 ms
-{
-  rss: 11240919040,
-  heapTotal: 8595075072,
-  heapUsed: 4817152633,
-  external: 2157456969,
-  arrayBuffers: 2156686216,
-}
-arenasize: 2097152 KB
-
-
-
-## bun array.ts
-765961.27 ms
-Count: 50000000
-{
-  rss: 28036014080,
-  heapTotal: 9476152320,
-  heapUsed: 11222438346,
-  external: 7170077276,
-  arrayBuffers: 2921230601,
-}
-
---- 
-
-## Direct Comparison
-
-| Metric | Arena Implementation (Optimized) | Standard Array Implementation | Difference / Factor |
-| :--- | :--- | :--- | :--- |
-| **Total Time** | **183,098 ms** (3.05 min) | **765,961 ms** (12.76 min) | **~4.2x Faster** |
-| **Time per 50M Lines** | **~18.3 sec** | **~76.6 sec** | **- 58.3 sec / Round** |
-| **RAM Usage (RSS)** | **11.24 GB** | **28.04 GB** | **16.8 GB Saved** |
-| **Heap Used (JS Objects)** | **4.82 GB** | **11.22 GB** | **2.3x More Efficient** |
-| **External (Buffer)** | **2.16 GB** | **7.17 GB** | **Compact Memory Footprint** |
-