bun-shared-memory 0.1.8 → 0.2.1

package/CHANGELOG.md

@@ -5,6 +5,104 @@ All notable changes to this project are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.1] - 2026-11-04
+
+### Performance
+
+- Inline the read-cache fast path in `readProp`/`readValue` instead of calling
+  `readCachedValue`, which returned a polymorphic `{hit,value}` union and
+  allocated an object on every cache hit. Keeping the read path monomorphic
+  lets the JIT optimize it: number reads dropped from ~12× to ~6× slower than
+  native, boolean reads from ~8.4× to ~5.5×, and all array index reads
+  improved (array-read summary ~9.7× → ~8.7×).
+- Add `updateBigIntSlot`/`updateArrayBigIntSlot` fast paths that write int64
+  bigints directly into the slot, avoiding the `ValueSlot` object allocation
+  of the generic encode path. `bigint[]` index writes dropped from ~24.8× to
+  ~18× slower than native.
+- Extract the array-proxy set handler body into a named `writeArrayProp`
+  function and inline the `writeLock`, so no closure is allocated per write.
+- Cache the key's version-cell index in the array proxy so writes bump the
+  version without a per-write Map lookup. Object array-prop index writes
+  dropped from ~33.9× to ~29× slower than native.
+- Real-world benchmark: the shared-memory "get user" endpoint improved from
+  ~4.18M to ~5.79M req/s (+39%), widening the gap over the comparison
+  backends (sqlite 6.2×→8.5×, redis 218×→302×, mysql 233×→322× slower than
+  shared memory).
+
+## [0.2.0] - 2026-11-04
+
+### Added
+
+- Slab allocator for string values and arrays. Overwriting a value no longer
+  leaves the old block as dead space: freed blocks are pushed onto a shared
+  free list (one per table, in a `SharedArrayBuffer`) and reused by later
+  allocations, so the tables grow far less over time and defragmentation runs
+  less often. The free list is shared across workers via
+  `exportBuffers()`/`fromBuffers()`.
+- Inline short strings. Strings up to 11 UTF-8 bytes are packed directly into
+  the 16-byte value slot (type + length in the top 4 bits of `c`, bytes in
+  `a`/`b`/`c`), avoiding a heap allocation and the string-table write on every
+  write. Longer strings still use the slab-allocated ArrayTable with in-place
+  reuse.
+- A complete multi-worker todo-list example (`examples/todo-list/`) that
+  serves both the frontend and the API on a single port via Elysia workers.
+
+### Performance
+
+- Cache each object property getter's stable version-cell index (resolved
+  lazily on first read) and load the version directly via
+  `Atomics.load(mem.versionView, cellIndex)`, replacing the per-read
+  `index === null` check and the `getVersion` → `versionCellIndex` multiply +
+  divide. Object property reads improved a further 3–17%.
+- Cache the top-level array ref and version-cell index on the `SharedRef`
+  target so index writes skip the per-write DataTable header reads, Map
+  lookups, and the generic encode path. `moveArray` now bumps the layout
+  version so the cached ref is invalidated on array growth. Closes the
+  remaining array-write gap: number[]/boolean[]/string[] index writes are now
+  at or above native speed.
+- Cache the array proxy for an object's array property on the accessor getter
+  so it isn't recreated on every write (a write bumps the version and
+  invalidates the field cache), and cache the property slot offset + array ref
+  in the array proxy so index writes skip the per-write `getObjectLayout` Map
+  lookup, `properties.get`, and DataTable header reads. Object array-prop
+  index writes dropped from ~254× to ~33× slower than native.
+
+### Changed
+
+- Raise the defragmentation trigger: a table now triggers a defrag pass when
+  its used length exceeds 80% of capacity (was: when free space fell below
+  10%), and only when at least 64 blocks have been freed. Because writes are
+  lock-free bump allocations, the used length grows monotonically even when
+  the live data set stays the same size; the new threshold defragments only
+  when the buffer is genuinely fragmented. During a defrag pass, lock-free
+  writes fall back to the mutex and in-flight writes are awaited before
+  compacting.
+
+### Fixed
+
+- Fix slab-allocator block corruption in `free()`. `alloc()` padded blocks to
+  `MIN_BLOCK_SIZE` internally but callers advanced the cursor by the unpadded
+  size, so blocks overlapped and `free()`'s metadata overwrote the neighbour
+  block. `alloc()` now returns the actual reserved (padded) size and all
+  callers advance by it. Free-list metadata was also reduced from 12 to 8
+  bytes (next pointer + size) and the magic marker dropped (double-frees are
+  now detected by walking the free list), lowering `MIN_BLOCK_SIZE` from 12 to
+  8 so small string blocks no longer inflate the ArrayTable during
+  defragmentation.
+
+## [0.1.8] - 2026-11-02
+
+### Fixed
+
+- Share each `GrowableBuffer`'s append cursor across workers. The cursor was a
+  private per-instance field, so when a worker reconstructed `SharedMemory`
+  via `fromBuffers()` its cursor stayed `0` and the first write in that worker
+  wrote at offset `0`, overwriting existing data (e.g. a key string) and
+  corrupting the shared value. The cursor now lives in a `SharedArrayBuffer`
+  (as a `BigInt64Array`) read/written with `Atomics`, so every worker appends
+  at the correct offset. Reported while building the multi-worker todo-list
+  example.
+
 ## [0.1.7] - 2026-11-02
 
 ### Added
@@ -172,6 +270,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - O(1) in-place `push`/`pop` with capacity-based array growth.
 - Benchmarks for all data types vs native.
 
+[0.2.0]: https://github.com/Kapsonfire/bun-shared-memory/compare/v0.1.8...v0.2.0
+[0.1.8]: https://github.com/Kapsonfire/bun-shared-memory/compare/v0.1.7...v0.1.8
 [0.1.7]: https://github.com/Kapsonfire/bun-shared-memory/compare/v0.1.6...v0.1.7
 [0.1.6]: https://github.com/Kapsonfire/bun-shared-memory/compare/v0.1.5...v0.1.6
 [0.1.5]: https://github.com/Kapsonfire/bun-shared-memory/compare/v0.1.4...v0.1.5

package/README.md

@@ -40,14 +40,14 @@ Measured on Linux (Bun 1.4.4, 100,000 requests per methodology, sorted by
 speed; MySQL/Redis benchmarked over both TCP and Unix socket; each methodology
 is the minimum over 3 runs × 5 samples to filter CPU-contention noise):
 
-| Methodology | Transport | Requests | Elapsed (ms) | req/s      | vs. shared |
-|-------------|-----------|----------|--------------|------------|------------|
-| `shared`    | —         | 100,000  | 21           | 4,876,492  | —          |
-| `sqlite`    | —         | 100,000  | 188          | 531,116    | 9.2× slower |
-| `redis`     | socket    | 100,000  | 1,262        | 79,229     | 61.5× slower |
-| `mysql`     | socket    | 100,000  | 1,605        | 62,322     | 78.2× slower |
-| `redis`     | tcp       | 100,000  | 2,817        | 35,497     | 137.4× slower |
-| `mysql`     | tcp       | 100,000  | 2,976        | 33,602     | 145.1× slower |
+| Methodology | Transport | Requests | Elapsed (ms) | req/s     | vs. shared    |
+| ----------- | --------- | -------- | ------------ | --------- | ------------- |
+| `shared`    | —         | 100,000  | 21           | 4,876,492 | —             |
+| `sqlite`    | —         | 100,000  | 188          | 531,116   | 9.2× slower   |
+| `redis`     | socket    | 100,000  | 1,262        | 79,229    | 61.5× slower  |
+| `mysql`     | socket    | 100,000  | 1,605        | 62,322    | 78.2× slower  |
+| `redis`     | tcp       | 100,000  | 2,817        | 35,497    | 137.4× slower |
+| `mysql`     | tcp       | 100,000  | 2,976        | 33,602    | 145.1× slower |
 
 Shared memory is the fastest by a wide margin: ~9.2× faster than a live SQLite
 read and ~61–145× faster than live MySQL/Redis reads. Unix sockets are
@@ -74,16 +74,16 @@ See [SPEC.md](./SPEC.md) for the full specification.
 
 ## 📊 Stats
 
-| | |
-|---|---|
-| **Version** | [![npm version](https://img.shields.io/npm/v/bun-shared-memory?color=cb3837)](https://www.npmjs.com/package/bun-shared-memory) |
-| **Downloads / month** | [![npm downloads](https://img.shields.io/npm/dm/bun-shared-memory?color=cb3837)](https://www.npmjs.com/package/bun-shared-memory) |
-| **Stars** | [![GitHub stars](https://img.shields.io/github/stars/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory) |
-| **Forks** | [![GitHub forks](https://img.shields.io/github/forks/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory) |
-| **Open issues** | [![GitHub issues](https://img.shields.io/github/issues/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory/issues) |
-| **Open PRs** | [![GitHub pull requests](https://img.shields.io/github/issues-pr/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory/pulls) |
-| **License** | [![License](https://img.shields.io/github/license/Kapsonfire/bun-shared-memory?color=green)](https://github.com/Kapsonfire/bun-shared-memory/blob/main/LICENSE) |
-| **Language** | [![TypeScript](https://img.shields.io/badge/TypeScript-7-blue?color=3178c6)](https://www.typescriptlang.org) |
+|                       |                                                                                                                                                                     |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Version**           | [![npm version](https://img.shields.io/npm/v/bun-shared-memory?color=cb3837)](https://www.npmjs.com/package/bun-shared-memory)                                      |
+| **Downloads / month** | [![npm downloads](https://img.shields.io/npm/dm/bun-shared-memory?color=cb3837)](https://www.npmjs.com/package/bun-shared-memory)                                   |
+| **Stars**             | [![GitHub stars](https://img.shields.io/github/stars/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory)                   |
+| **Forks**             | [![GitHub forks](https://img.shields.io/github/forks/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory)                   |
+| **Open issues**       | [![GitHub issues](https://img.shields.io/github/issues/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory/issues)          |
+| **Open PRs**          | [![GitHub pull requests](https://img.shields.io/github/issues-pr/Kapsonfire/bun-shared-memory?color=181717)](https://github.com/Kapsonfire/bun-shared-memory/pulls) |
+| **License**           | [![License](https://img.shields.io/github/license/Kapsonfire/bun-shared-memory?color=green)](https://github.com/Kapsonfire/bun-shared-memory/blob/main/LICENSE)     |
+| **Language**          | [![TypeScript](https://img.shields.io/badge/TypeScript-7-blue?color=3178c6)](https://www.typescriptlang.org)                                                        |
 
 ## Data Types
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bun-shared-memory",
-  "version": "0.1.8",
+  "version": "0.2.1",
   "description": "Share data between Bun workers/threads using SharedArrayBuffer (zero-copy, 4-buffer architecture, Atomics-based synchronization).",
   "type": "module",
   "sideEffects": false,
@@ -47,6 +47,7 @@
   },
   "devDependencies": {
     "@types/bun": "latest",
+    "cofusion": "^0.0.2",
     "elysia": "^1.4.30"
   }
 }

package/src/growable-buffer.ts

@@ -93,6 +93,16 @@ export function largestPowerOfTwo(n: number): number {
   return 2 ** Math.floor(Math.log2(n));
 }
 
+/**
+ * Minimum size of a slab-allocated block. The free-list metadata written by
+ * `free` (next pointer + size = 8 bytes) must fit inside the block, otherwise
+ * it would overwrite the following block. All blocks are padded up to at
+ * least this size.
+ */
+const MIN_BLOCK_SIZE = 8;
+
+export { MIN_BLOCK_SIZE };
+
 export class GrowableBuffer {
   private chunks: Chunk[] = [];
   private readonly maxByteLength: number;
@@ -109,6 +119,17 @@ export class GrowableBuffer {
    * existing data at offset 0.
    */
   private readonly cursor: BigInt64Array;
+  /**
+   * Shared head of the free list. Stored in a SharedArrayBuffer so all workers
+   * that reconstruct this buffer from `fromBuffers()` see the same free list
+   * and reuse freed blocks instead of appending at the end.
+   *
+   * The free list is a singly-linked list of freed blocks. Each freed block
+   * stores, in its first 4 bytes, the byte offset of the next free block (or
+   * -1 for the end of the list). The head is the offset of the first free
+   * block, or -1 when the list is empty.
+   */
+  private readonly freeListHead: Int32Array;
   /** Smallest chunk index that is not yet active (see `activate`). */
   private nextChunkToActivate = 0;
 
@@ -120,12 +141,20 @@ export class GrowableBuffer {
     singleBuffer = false,
     chargeInitial = true,
     cursorBuffer?: SharedArrayBuffer,
+    freeListBuffer?: SharedArrayBuffer,
   ) {
     this.singleBuffer = singleBuffer;
     this.budget = budget;
     this.cursor = new BigInt64Array(
       cursorBuffer ?? new SharedArrayBuffer(BigInt64Array.BYTES_PER_ELEMENT),
     );
+    this.freeListHead = new Int32Array(
+      freeListBuffer ?? new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT),
+    );
+    // Initialize the free list head to -1 (empty) on first construction.
+    if (freeListBuffer === undefined) {
+      Atomics.store(this.freeListHead, 0, -1);
+    }
     if (singleBuffer && typeof initialSizeOrBuffer !== "number") {
       const buffer = Array.isArray(initialSizeOrBuffer)
         ? initialSizeOrBuffer[0]
@@ -285,6 +314,11 @@ export class GrowableBuffer {
     return this.cursor.buffer as SharedArrayBuffer;
   }
 
+  /** The shared free-list head buffer (for export to workers). */
+  get freeListBuffer(): SharedArrayBuffer {
+    return this.freeListHead.buffer as SharedArrayBuffer;
+  }
+
   /** Total logical capacity across all chunks (active or not). */
   get capacity(): number {
     return this.singleBuffer
@@ -391,6 +425,148 @@ export class GrowableBuffer {
     Atomics.add(this.cursor, 0, BigInt(n));
   }
 
+  /**
+   * Atomically reserve `size` bytes at the end of the buffer and return the
+   * start offset. This is the lock-free bump-allocation primitive: the shared
+   * cursor is incremented with a single `Atomics.add`, so concurrent workers
+   * each get a distinct, non-overlapping range without a mutex. The caller
+   * writes its data into `[offset, offset + size)`.
+   *
+   * Chunk growth is preserved: `ensureCapacity` activates any chunks needed to
+   * cover the reserved range. Because `SharedArrayBuffer.grow()` is visible
+   * across workers, activation needs no broadcast.
+   *
+   * Unlike `alloc`, this never consults the free list, so freed blocks are not
+   * reused on this path (dead space is reclaimed by defragmentation).
+   */
+  reserve(size: number): number {
+    // Lock-free bump allocation with chunk-boundary alignment. The shared
+    // cursor is advanced with a compare-exchange loop so concurrent workers
+    // each get a distinct, non-overlapping range without a mutex. If the
+    // block would cross a chunk boundary, the offset is aligned up to the next
+    // chunk so the caller's typed writes (setInt32 etc.) never straddle two
+    // chunks (those accessors only write within a single chunk).
+    while (true) {
+      const current = Number(Atomics.load(this.cursor, 0));
+      let offset = current;
+      if (!this.singleBuffer) {
+        const within = current & (this.chunkSize - 1);
+        if (within + size > this.chunkSize) {
+          offset = current + (this.chunkSize - within);
+        }
+      }
+      const next = offset + size;
+      if (
+        Atomics.compareExchange(
+          this.cursor,
+          0,
+          BigInt(current),
+          BigInt(next),
+        ) === BigInt(current)
+      ) {
+        this.ensureCapacity(next);
+        return offset;
+      }
+    }
+  }
+
+  /**
+   * Allocate `size` bytes, reusing a freed block from the free list when one
+   * of at least `size` bytes is available (first-fit), otherwise appending at
+   * the end of the buffer.
+   *
+   * Returns the byte offset of the allocated block and the actual reserved
+   * (padded) size. The caller must write the block's data and then call
+   * `advance(reserved)` only when the block was appended (not reused), because
+   * a reused block is already counted as used.
+   *
+   * @returns the offset, the reserved size, and whether the block was freshly
+   *          appended (true) or reused from the free list (false).
+   */
+  alloc(size: number): { offset: number; appended: boolean; size: number } {
+    // Pad small blocks up to MIN_BLOCK_SIZE so the free-list metadata written
+    // by `free` always fits inside the block and never overwrites a neighbour.
+    size = Math.max(size, MIN_BLOCK_SIZE);
+    // First-fit search over the free list for a block of at least `size`.
+    let prev = -1;
+    let cur = Atomics.load(this.freeListHead, 0);
+    while (cur !== -1) {
+      const next = this.getInt32(cur);
+      if (this.blockSize(cur) >= size) {
+        // Unlink `cur` from the free list.
+        if (prev === -1) {
+          Atomics.store(this.freeListHead, 0, next);
+        } else {
+          this.setInt32(prev, next);
+        }
+        return { offset: cur, appended: false, size };
+      }
+      prev = cur;
+      cur = next;
+    }
+    // No reusable block; append at the end.
+    const offset = this.byteLength;
+    this.ensureCapacity(offset + size);
+    return { offset, appended: true, size };
+  }
+
+  /**
+   * Free a block previously allocated with `alloc`, pushing it onto the free
+   * list so it can be reused. Detects double frees: freeing a block that is
+   * already on the free list throws an Error.
+   *
+   * The block's first 8 bytes are overwritten with the free-list metadata
+   * (next pointer + size), so the caller must not read the block's data after
+   * freeing it.
+   */
+  free(offset: number, size: number): void {
+    // Pad to MIN_BLOCK_SIZE to match the size `alloc` actually reserved, so
+    // `blockSize` reports the real block size for reuse decisions.
+    size = Math.max(size, MIN_BLOCK_SIZE);
+    // Double-free detection: a block already on the free list must not be
+    // freed again. Walk the list (free is not a hot path, so the O(n) scan is
+    // acceptable).
+    let cur = Atomics.load(this.freeListHead, 0);
+    while (cur !== -1) {
+      if (cur === offset) {
+        throw new Error(
+          `Double free detected at offset ${offset} (size ${size})`,
+        );
+      }
+      cur = this.getInt32(cur);
+    }
+    const head = Atomics.load(this.freeListHead, 0);
+    this.setInt32(offset, head);
+    this.setInt32(offset + 4, size);
+    Atomics.store(this.freeListHead, 0, offset);
+  }
+
+  /**
+   * Read the size of a freed block, stored in bytes [4..7] when it was
+   * freed. Used by `alloc` to check whether a freed block is large enough.
+   */
+  private blockSize(offset: number): number {
+    return this.getInt32(offset + 4);
+  }
+
+  /**
+   * Walk the free list and return the number of freed blocks and the total
+   * bytes they occupy. Used to measure fragmentation: many small freed blocks
+   * scattered through the buffer indicate high fragmentation, which a
+   * defragmentation pass can compact.
+   */
+  freeListStats(): { count: number; bytes: number } {
+    let count = 0;
+    let bytes = 0;
+    let cur = Atomics.load(this.freeListHead, 0);
+    while (cur !== -1) {
+      count++;
+      bytes += this.blockSize(cur);
+      cur = this.getInt32(cur);
+    }
+    return { count, bytes };
+  }
+
   /**
    * Reset the used length to `n` bytes. Used by defragmentation after
    * compacting live data to the front of the buffer. Chunks beyond the
@@ -399,6 +575,9 @@ export class GrowableBuffer {
    */
   reset(n: number): void {
     Atomics.store(this.cursor, 0, BigInt(n));
+    // After compaction the free list's offsets are invalid (blocks moved), so
+    // clear it. Freed blocks are re-added lazily on the next `free`.
+    Atomics.store(this.freeListHead, 0, -1);
     if (this.singleBuffer) return;
     const neededChunks = Math.ceil(n / this.chunkSize);
     for (let i = 0; i < this.chunks.length; i++) {