@torrent-tv/proxy 2.9.77 → 2.9.78

package/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 2.9.78
+
+- **New**: Reads cross the thread boundary as **positions instead of bytes**. The pieces already live in a `SharedArrayBuffer`, so the torrent thread now sends an offset and a length and the main thread reads those bytes where they lie. What this removes is the copy that used to sit on the critical path — 18.84 ms per 10 MB segment on the field host, spent in the same thread that runs the torrent, at the moment a viewer is waiting for that segment. A piece is **pinned** for as long as a fragment of it is outstanding, and unpinned only once the main thread confirms it has finished reading, so eviction cannot take the memory out from under a reader; one fragment is in flight at a time, because the store guarantees only two resident pieces at its smallest budget and holding two pins while asking for a third would deadlock it. Verified against a partially downloaded 5.5 GB torrent: a range read whole matches the same range read in parts, a read spanning a piece boundary matches its two halves, and ffmpeg parses the file through this path (`matroska h264/ac3 5939 s`). The arithmetic is covered by tests, including a file that does not start on a piece boundary — the case where treating file offsets as torrent offsets returns the right number of wrong bytes.
+- **Chore**: `SharedPieceStore` gained `reside`, which brings a piece into memory and reports where it sits without the copy `get` has to make (WebTorrent keeps what `get` returns), and `findSharedStore`, which walks WebTorrent's store wrappers to reach ours rather than assuming their number or order.
+- **Known**: the copy is not gone from the system, only from the torrent thread — the main thread still copies each fragment out of the pool before handing it on, because nothing tells us when the socket has finished with those bytes, and releasing the piece earlier would risk serving whatever landed in the slot next. Removing that last copy needs the body write to report completion, which is a change to the stream route rather than to this transport.
+
 ## 2.9.77
 
 - **Fix**: Anything naming a source while that source was still being added got `Unknown source` — which is false, because the source exists and is merely not ready. Adding a magnet takes as long as its metadata does, seconds to tens of seconds, and the browser polls stats and asks for a playback plan throughout that window. The worker registered the torrent only once the add had **finished**; it now registers the pending add itself, so callers wait for it. Reproduced with a magnet nobody seeds: stats, the file listing and a read all failed instantly while the add was in flight, and all three now wait. A source that was never added is still an error, and a failed add is forgotten rather than replayed to every later caller.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.77",
+  "version": "2.9.78",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/piece-store/shared-piece-store.js

@@ -51,6 +51,31 @@ export function collectStoreStats() {
   return [...liveStores].map((store) => store.stats());
 }
 
+/**
+ * The shared store behind a torrent, or `null` if it is not one of ours.
+ *
+ * WebTorrent wraps whatever store it is given — today in `ImmediateChunkStore`,
+ * and historically in a piece cache as well — and offers no way to ask for the
+ * innermost one. Walking the `store` chain finds it regardless of how many
+ * wrappers there are or what order they sit in, which is sturdier than reaching
+ * for a fixed `torrent.store.store`.
+ *
+ * @param {{ store?: object } | null | undefined} torrent
+ * @returns {SharedPieceStore | null}
+ */
+export function findSharedStore(torrent) {
+  let candidate = torrent?.store;
+  // Bounded rather than `while (candidate)`: a store that referenced itself
+  // would otherwise hang the thread instead of failing.
+  for (let depth = 0; candidate && depth < 8; depth += 1) {
+    if (candidate instanceof SharedPieceStore) {
+      return candidate;
+    }
+    candidate = candidate.store;
+  }
+  return null;
+}
+
 /**
  * Ceiling for the automatic budget, and the share of free memory it will take.
  *
@@ -389,6 +414,53 @@ export class SharedPieceStore {
     fetch().then((bytes) => done(null, bytes), (error) => done(error));
   }
 
+  /**
+   * Ensure a piece is in memory and say where it sits — without copying it.
+   *
+   * This is {@link get} minus its final copy, and it exists for exactly one
+   * caller: the reader that hands pieces to the other thread. That thread maps
+   * the same {@link sharedBuffer}, so an offset and a length are all it needs,
+   * and the bytes never move. `get` cannot serve that purpose because
+   * WebTorrent keeps what `get` returns while the slot underneath may be
+   * reused.
+   *
+   * The caller MUST hold a pin across the whole read — the returned offset
+   * stays valid only while the piece is pinned.
+   *
+   * @param {number} index
+   * @returns {Promise<{ offset: number, length: number } | null>} `null` when
+   *   the store holds no such piece, in memory or on disk.
+   */
+  async reside(index) {
+    if (this.#closed) {
+      throw new Error("Piece store is closed.");
+    }
+
+    const slot = this.#slotOf.get(index);
+    if (slot !== undefined) {
+      this.#lru.touch(index);
+      this.#counters.fromMemory += 1;
+      return this.locate(index);
+    }
+
+    if (!this.#disk.has(index)) {
+      return null;
+    }
+
+    const pieceLength = this.#lengthOf(index);
+    const revived = await this.#claimSlot();
+    const target = this.#pool.subarray(
+      revived * this.#chunkLength,
+      revived * this.#chunkLength + pieceLength
+    );
+    await this.#disk.read(index, target);
+    this.#slotOf.set(index, revived);
+    this.#lru.touch(index);
+    this.#counters.fromDisk += 1;
+    this.#counters.revivals += 1;
+    return this.locate(index);
+  }
+
   /**
    * Close the store, keeping the spill file.
    *

package/services/torrent-worker/client.js

@@ -37,6 +37,10 @@ export class TorrentWorkerClient {
   #caller;
   /** Receive-side handles for in-flight reads, keyed by request id. */
   #reads = new Map();
+  /** Each torrent's piece pool, so a fragment can be read where it lies. */
+  #poolBySource = new Map();
+  /** Which pool an in-flight read belongs to, keyed by request id. */
+  #poolByRead = new Map();
 
   /**
    * @param {{ maxDiskBytes?: number, memoryBytes?: number }} [options]
@@ -64,6 +68,24 @@ export class TorrentWorkerClient {
         return;
       }
       switch (message?.type) {
+        case Event.FRAGMENT: {
+          // The bytes are already here — this thread maps the same pool. Read
+          // them where they lie, then say so, which is what lets the worker
+          // unpin the piece and move on. The copy exists only because the
+          // consumer keeps what it is given while the slot may be reused; it is
+          // one copy on this thread rather than one on the torrent's.
+          const pool = this.#poolByRead.get(message.id);
+          const bytes = pool
+            ? Uint8Array.prototype.slice.call(
+                new Uint8Array(pool, message.offset, message.length)
+              )
+            : null;
+          if (bytes) {
+            this.#reads.get(message.id)?.push(bytes);
+          }
+          this.#worker.postMessage({ type: Event.FRAGMENT_DONE, id: message.id });
+          break;
+        }
         case Event.CHUNK: {
           const bytes = message.bytes;
           this.#reads.get(message.id)?.push(
@@ -74,6 +96,7 @@ export class TorrentWorkerClient {
         case Event.READ_END:
           this.#reads.get(message.id)?.close();
           this.#reads.delete(message.id);
+          this.#poolByRead.delete(message.id);
           break;
         case Event.LOG:
           logger.info(`torrent-worker: ${message.message}`);
@@ -190,9 +213,16 @@ export class TorrentWorkerClient {
       onCancel: () => {
         void this.#caller.call(Command.CANCEL_READ, { readId }).catch(() => undefined);
         this.#reads.delete(readId);
+        this.#poolByRead.delete(readId);
       }
     });
     this.#reads.set(readId, receive);
+    // Which pool this read's fragments will point into. Recorded before the
+    // command is sent, because the first fragment can arrive immediately.
+    const pool = this.#poolBySource.get(sourceKey);
+    if (pool) {
+      this.#poolByRead.set(readId, pool);
+    }
 
     // The worker replies to READ_RANGE only once the body is fully sent; a
     // failure before that must surface on the stream, not vanish.
@@ -224,6 +254,11 @@ export class TorrentWorkerClient {
    */
   async getTorrent({ sourceKey, sourceType, source }) {
     const info = await this.addSource({ sourceKey, sourceType, source });
+    // The torrent's piece pool. Both threads now hold the same memory, so a
+    // read can be answered with an offset instead of with bytes.
+    if (info.sharedBuffer) {
+      this.#poolBySource.set(sourceKey, info.sharedBuffer);
+    }
     const client = this;
     return {
       infoHash: info.infoHash,

package/services/torrent-worker/piece-reader.js

@@ -0,0 +1,175 @@
+/**
+ * @file Reading a byte range as positions in shared memory, not as bytes.
+ *
+ * The pieces already live in a `SharedArrayBuffer` the main thread can map. So
+ * the torrent thread does not need to hand over any bytes at all: it can say
+ * *where* a piece sits and let the other side read it there. What crosses the
+ * boundary is two numbers per piece.
+ *
+ * That is the whole point of the exercise. The alternative — copying each piece
+ * into memory we own and transferring it — costs 18.84 ms per 10 MB segment on
+ * the field host, and costs it **on the critical path**, in the thread that is
+ * also running the torrent, at the moment a viewer is waiting for that segment.
+ * Here the copy is gone entirely rather than moved.
+ *
+ * Two obligations come with it, and both are enforced rather than assumed:
+ *
+ *  - a piece being read is **pinned**, so eviction cannot take the memory out
+ *    from under the reader mid-read;
+ *  - the pin is released only once the other thread reports it has finished
+ *    with those bytes — not when they were sent, because nothing was sent.
+ */
+
+import { findSharedStore } from "../piece-store/shared-piece-store.js";
+
+/**
+ * Wait until a piece has been downloaded and verified.
+ *
+ * WebTorrent announces this as `verified`. The bitfield is re-checked after the
+ * listener is attached because the piece can complete in between, and a missed
+ * event here would wait forever.
+ *
+ * @param {import("webtorrent").Torrent} torrent
+ * @param {number} index
+ * @param {{ isCancelled: () => boolean }} cancellation
+ * @returns {Promise<void>}
+ */
+function whenPieceReady(torrent, index, cancellation) {
+  if (torrent.bitfield?.get(index)) {
+    return Promise.resolve();
+  }
+
+  return new Promise((resolve, reject) => {
+    /** @param {number} verifiedIndex */
+    const onVerified = (verifiedIndex) => {
+      if (verifiedIndex === index) {
+        cleanup();
+        resolve();
+      }
+    };
+    const onDestroyed = () => {
+      cleanup();
+      reject(new Error(`Torrent went away while waiting for piece ${index}.`));
+    };
+    // Cancellation is polled rather than pushed: a superseded seek destroys the
+    // read, and without this the wait would outlive it and hold a pin.
+    const poll = setInterval(() => {
+      if (cancellation.isCancelled()) {
+        cleanup();
+        reject(new Error(`Read cancelled while waiting for piece ${index}.`));
+      }
+    }, 250);
+
+    function cleanup() {
+      clearInterval(poll);
+      torrent.removeListener("verified", onVerified);
+      torrent.removeListener("close", onDestroyed);
+    }
+
+    torrent.on("verified", onVerified);
+    torrent.once("close", onDestroyed);
+
+    // The piece may have arrived between the check above and this listener.
+    if (torrent.bitfield?.get(index)) {
+      cleanup();
+      resolve();
+    }
+  });
+}
+
+/**
+ * A fragment of a read: where to find it, and how to let it go.
+ *
+ * @typedef {object} PieceFragment
+ * @property {number} pieceIndex
+ * @property {number} offset - Byte offset into the shared pool.
+ * @property {number} length
+ * @property {() => void} release - Drops this fragment's pin. Call exactly once.
+ */
+
+/**
+ * Walk a byte range of a file, yielding each piece's position in shared memory.
+ *
+ * Yields at most one fragment per piece; the first and last are usually partial.
+ * The caller must `release()` every fragment it receives, including on failure —
+ * an unreleased pin permanently costs a slot.
+ *
+ * @param {object} params
+ * @param {import("webtorrent").Torrent} params.torrent
+ * @param {number} params.fileIndex
+ * @param {number} params.start - Inclusive, relative to the file.
+ * @param {number} params.end - Inclusive, relative to the file.
+ * @param {{ isCancelled: () => boolean }} params.cancellation
+ * @returns {AsyncGenerator<PieceFragment>}
+ */
+export async function* readFragments({ torrent, fileIndex, start, end, cancellation }) {
+  const store = findSharedStore(torrent);
+  if (!store) {
+    throw new Error("This torrent is not backed by a shared piece store.");
+  }
+
+  const file = torrent.files?.[fileIndex];
+  if (!file) {
+    throw new Error(`File ${fileIndex} not found.`);
+  }
+
+  const pieceLength = torrent.pieceLength;
+  // Piece numbers are torrent-wide, so a file's own offsets have to be lifted
+  // into the torrent's address space first.
+  const absoluteStart = file.offset + start;
+  const absoluteEnd = file.offset + end;
+  const firstPiece = Math.floor(absoluteStart / pieceLength);
+  const lastPiece = Math.floor(absoluteEnd / pieceLength);
+
+  // Ask for these pieces first. `select` puts them in the download set at all;
+  // `critical` marks them as wanted now, which is what allows a piece to be
+  // fetched out of sequential order for a reader that is waiting on it.
+  if (typeof torrent.select === "function") {
+    torrent.select(firstPiece, lastPiece, 1);
+  }
+  if (typeof torrent.critical === "function") {
+    torrent.critical(firstPiece, lastPiece);
+  }
+
+  for (let pieceIndex = firstPiece; pieceIndex <= lastPiece; pieceIndex += 1) {
+    if (cancellation.isCancelled()) {
+      return;
+    }
+
+    const pieceStart = pieceIndex * pieceLength;
+    const fromWithinPiece = Math.max(absoluteStart, pieceStart) - pieceStart;
+    const toWithinPiece = Math.min(absoluteEnd, pieceStart + pieceLength - 1) - pieceStart;
+
+    await whenPieceReady(torrent, pieceIndex, cancellation);
+
+    // Pinned BEFORE it is located, and before any await that could let an
+    // eviction run: the offset is only meaningful while the piece is held.
+    store.pin(pieceIndex);
+    let located = null;
+    try {
+      located = await store.reside(pieceIndex);
+    } catch (error) {
+      store.unpin(pieceIndex);
+      throw error;
+    }
+
+    if (!located) {
+      store.unpin(pieceIndex);
+      throw new Error(`Piece ${pieceIndex} is verified but absent from the store.`);
+    }
+
+    let releasedThisPiece = false;
+    yield {
+      pieceIndex,
+      offset: located.offset + fromWithinPiece,
+      length: toWithinPiece - fromWithinPiece + 1,
+      release() {
+        if (releasedThisPiece) {
+          return;
+        }
+        releasedThisPiece = true;
+        store.unpin(pieceIndex);
+      }
+    };
+  }
+}

package/services/torrent-worker/protocol.js

@@ -80,6 +80,18 @@ export const Event = {
   ERROR: "error",
   /** One piece of a READ_RANGE body; its bytes are transferred, never copied. */
   CHUNK: "chunk",
+  /**
+   * Where a piece of the body sits in the torrent's shared pool — an offset and
+   * a length, no bytes at all. The main thread maps the same memory and reads it
+   * in place; see `piece-reader.js` for why this replaces sending the bytes.
+   */
+  FRAGMENT: "fragment",
+  /**
+   * The main thread has finished with a FRAGMENT and its pin may be dropped.
+   * Distinct from {@link CHUNK_ACK}, which only reports queue capacity: this one
+   * is a promise that nothing is reading those bytes any more.
+   */
+  FRAGMENT_DONE: "fragment-done",
   /** A READ_RANGE ended; no further CHUNKs bear that request id. */
   READ_END: "read-end",
   /** The main thread consumed a chunk — see {@link STREAM_HIGH_WATER_CHUNKS}. */

package/services/torrent-worker/worker.js

@@ -25,7 +25,8 @@ import "./install-webrtc-shim.js";
 import { parentPort, workerData } from "node:worker_threads";
 import { createSendStream } from "./channel.js";
 import { createFileClaims } from "./file-claims.js";
-import { Command, Event, STREAM_CHUNK_BYTES } from "./protocol.js";
+import { readFragments } from "./piece-reader.js";
+import { Command, Event } from "./protocol.js";
 
 // Imported dynamically, and that is load-bearing: static imports are RESOLVED
 // during linking, before any module body runs, so a statically imported pool
@@ -33,7 +34,7 @@ import { Command, Event, STREAM_CHUNK_BYTES } from "./protocol.js";
 // the hook above had a chance to register. Verified the hard way: with a static
 // import the process still aborted, and the stack named the genuine polyfill.
 const { TorrentPool } = await import("../torrent-pool.js");
-const { collectStoreStats } = await import("../piece-store/shared-piece-store.js");
+const { collectStoreStats, findSharedStore } = await import("../piece-store/shared-piece-store.js");
 
 const pool = new TorrentPool({
   maxDiskBytes: workerData?.maxDiskBytes,
@@ -84,6 +85,57 @@ async function requireTorrent(sourceKey) {
   return pending;
 }
 
+/**
+ * Fragments waiting for the main thread to say it has finished reading them,
+ * keyed by request id. One per read, because only one fragment is in flight.
+ *
+ * @type {Map<number, () => void>}
+ */
+const fragmentWaiters = new Map();
+
+/**
+ * Wake a read that is waiting for a fragment to be confirmed.
+ *
+ * Used both by the confirmation itself and by cancellation — a cancelled read
+ * will never be confirmed, and without this it would wait forever holding a pin.
+ *
+ * @param {number} id
+ * @returns {void}
+ */
+function settleFragment(id) {
+  const done = fragmentWaiters.get(id);
+  if (done) {
+    fragmentWaiters.delete(id);
+    done();
+  }
+}
+
+/**
+ * Send one fragment's position and wait until the main thread is done with it.
+ *
+ * The pin is dropped only after the confirmation, because until then the other
+ * thread may still be reading those exact bytes.
+ *
+ * @param {number} id
+ * @param {import("./piece-reader.js").PieceFragment} fragment
+ * @returns {Promise<void>}
+ */
+function sendFragment(id, fragment) {
+  return new Promise((resolve) => {
+    fragmentWaiters.set(id, () => {
+      fragment.release();
+      resolve();
+    });
+    parentPort.postMessage({
+      type: Event.FRAGMENT,
+      id,
+      pieceIndex: fragment.pieceIndex,
+      offset: fragment.offset,
+      length: fragment.length
+    });
+  });
+}
+
 /**
  * Stream a byte range back as CHUNK messages.
  *
@@ -120,39 +172,31 @@ async function streamRange({ id, sourceKey, fileIndex, start, end }) {
   // an empty input.
   const releaseRead = pool.acquireFile(torrent, fileIndex);
 
-  const options = start === null || start === undefined ? {} : { start, end };
-  const source = file.createReadStream(options);
-
-  // Coalesce WebTorrent's own chunking (piece-sized, often much smaller) up to
-  // our chunk size: a round trip costs ~100 µs, so sending its native pieces
-  // straight through would multiply the crossings for no benefit.
-  let pendingParts = [];
-  let pendingBytes = 0;
-
-  const flush = async () => {
-    if (pendingBytes === 0) {
-      return;
-    }
-    const merged = pendingParts.length === 1 ? pendingParts[0] : Buffer.concat(pendingParts, pendingBytes);
-    pendingParts = [];
-    pendingBytes = 0;
-    await sender.send(merged);
-  };
+  const rangeStart = start ?? 0;
+  const rangeEnd = end ?? file.length - 1;
 
   let failed = false;
   try {
-    for await (const part of source) {
+    // Positions in shared memory, not bytes: the main thread maps the same pool
+    // and reads each fragment in place, so nothing is copied and nothing is
+    // transferred. See `piece-reader.js`.
+    for await (const fragment of readFragments({
+      torrent,
+      fileIndex,
+      start: rangeStart,
+      end: rangeEnd,
+      cancellation: sender
+    })) {
       if (sender.isCancelled()) {
+        fragment.release();
         break;
       }
-      pendingParts.push(part);
-      pendingBytes += part.length;
-      if (pendingBytes >= STREAM_CHUNK_BYTES) {
-        await flush();
-      }
-    }
-    if (!sender.isCancelled()) {
-      await flush();
+      // One fragment in flight at a time. Each one holds a piece pinned, and
+      // the store guarantees only two resident pieces at its smallest budget —
+      // holding two pins while asking for a third would deadlock it against
+      // itself. The round trip costs ~100 µs against a piece worth megabytes,
+      // so there is nothing to win by overlapping them.
+      await sendFragment(id, fragment);
     }
   } catch (error) {
     // The end-of-read marker means "the body is complete". Sending it after a
@@ -164,15 +208,16 @@ async function streamRange({ id, sourceKey, fileIndex, start, end }) {
     throw error;
   } finally {
     readsById.delete(id);
+    // Any fragment still awaiting confirmation will never get one now; settling
+    // it here releases its pin rather than leaking a held slot.
+    settleFragment(id);
     releaseRead();
     if (!failed) {
       sender.end();
     }
-    // A cancelled read must stop the underlying torrent stream too, or the
-    // pieces keep being fetched for a viewer who has gone.
-    if (typeof source.destroy === "function") {
-      source.destroy();
-    }
+    // Nothing else to tear down: the reader owns no stream of its own, and a
+    // cancelled read stops at its next fragment boundary because it polls the
+    // same `sender` for cancellation.
   }
 }
 
@@ -208,6 +253,10 @@ async function runCommand(command, params, id) {
       return {
         infoHash: torrent.infoHash,
         name: torrent.name,
+        // The piece pool itself. A `SharedArrayBuffer` crosses as a reference to
+        // the same memory rather than a copy, which is what lets the main thread
+        // read a piece where it already lies instead of being sent its bytes.
+        sharedBuffer: findSharedStore(torrent)?.sharedBuffer ?? null,
         // Files cross as plain data; the objects stay here.
         files: (torrent.files ?? []).map((file, index) => ({
           index,
@@ -282,6 +331,9 @@ async function runCommand(command, params, id) {
 
     case Command.CANCEL_READ: {
       readsById.get(params.readId)?.cancel();
+      // A cancelled read will never have its outstanding fragment confirmed, so
+      // wake it here — otherwise it waits forever with a piece pinned.
+      settleFragment(params.readId);
       return true;
     }
 
@@ -305,6 +357,13 @@ parentPort.on("message", async (message) => {
     return;
   }
 
+  // The main thread has finished reading a fragment out of shared memory, so
+  // its piece may be unpinned and the read may continue.
+  if (message?.type === Event.FRAGMENT_DONE) {
+    settleFragment(message.id);
+    return;
+  }
+
   const { command, id, params } = message ?? {};
   try {
     const result = await runCommand(command, params ?? {}, id);

package/test/piece-reader.test.js

@@ -0,0 +1,162 @@
+/**
+ * @file Turning a byte range into positions in shared memory.
+ *
+ * This arithmetic fails silently when it is wrong: the response comes back the
+ * right length and full of the wrong bytes. Piece numbers are torrent-wide
+ * while a read is expressed in file coordinates, and the first and last pieces
+ * of a range are almost always partial — so every case here is a boundary.
+ */
+
+import test from "node:test";
+import assert from "node:assert/strict";
+import { EventEmitter } from "node:events";
+import { readFragments } from "../services/torrent-worker/piece-reader.js";
+import { SharedPieceStore } from "../services/piece-store/shared-piece-store.js";
+import os from "node:os";
+import path from "node:path";
+import fs from "node:fs/promises";
+
+const PIECE = 1024;
+
+/**
+ * A torrent whose pieces are all present, backed by a real store so that
+ * `locate`/`reside`/`pin` behave as they do in production.
+ *
+ * @param {{ fileOffset: number, fileLength: number, totalLength: number }} shape
+ */
+async function fakeTorrent({ fileOffset, fileLength, totalLength }) {
+  const directory = await fs.mkdtemp(path.join(os.tmpdir(), "piece-reader-test-"));
+  const store = new SharedPieceStore(PIECE, {
+    length: totalLength,
+    memoryBytes: 64 * PIECE,
+    path: directory,
+    name: "test"
+  });
+
+  const pieceCount = Math.ceil(totalLength / PIECE);
+  for (let index = 0; index < pieceCount; index += 1) {
+    const length = index === pieceCount - 1 ? totalLength - index * PIECE : PIECE;
+    const piece = Buffer.alloc(length);
+    // Each byte encodes its own absolute position, so a misplaced offset is
+    // visible in the value itself rather than only in the length.
+    for (let at = 0; at < length; at += 1) {
+      piece[at] = (index * PIECE + at) % 251;
+    }
+    await new Promise((resolve, reject) => {
+      store.put(index, piece, (error) => (error ? reject(error) : resolve()));
+    });
+  }
+
+  const torrent = Object.assign(new EventEmitter(), {
+    pieceLength: PIECE,
+    store,
+    bitfield: { get: () => true },
+    files: [{ offset: fileOffset, length: fileLength, name: "file.bin" }],
+    select() {},
+    critical() {}
+  });
+
+  return { torrent, store, directory };
+}
+
+/** Collect a range through the reader, as the worker does. */
+async function readRange(torrent, start, end) {
+  const collected = [];
+  const positions = [];
+  const pool = Buffer.from(torrent.store.sharedBuffer);
+  for await (const fragment of readFragments({
+    torrent,
+    fileIndex: 0,
+    start,
+    end,
+    cancellation: { isCancelled: () => false }
+  })) {
+    collected.push(Buffer.from(pool.subarray(fragment.offset, fragment.offset + fragment.length)));
+    positions.push({ piece: fragment.pieceIndex, length: fragment.length });
+    fragment.release();
+  }
+  return { bytes: Buffer.concat(collected), positions };
+}
+
+/** What the bytes at an absolute torrent offset should be. */
+function expectedBytes(absoluteStart, length) {
+  const expected = Buffer.alloc(length);
+  for (let at = 0; at < length; at += 1) {
+    expected[at] = (absoluteStart + at) % 251;
+  }
+  return expected;
+}
+
+test("a range inside one piece is read from that piece only", async () => {
+  const { torrent, store, directory } = await fakeTorrent({
+    fileOffset: 0,
+    fileLength: 4 * PIECE,
+    totalLength: 4 * PIECE
+  });
+  try {
+    const { bytes, positions } = await readRange(torrent, 100, 199);
+    assert.equal(positions.length, 1);
+    assert.deepEqual(bytes, expectedBytes(100, 100));
+  } finally {
+    store.destroy(() => undefined);
+    await fs.rm(directory, { recursive: true, force: true });
+  }
+});
+
+test("a range spanning pieces reassembles in order", async () => {
+  const { torrent, store, directory } = await fakeTorrent({
+    fileOffset: 0,
+    fileLength: 4 * PIECE,
+    totalLength: 4 * PIECE
+  });
+  try {
+    const start = PIECE - 10;
+    const end = 2 * PIECE + 9;
+    const { bytes, positions } = await readRange(torrent, start, end);
+    assert.deepEqual(positions.map((entry) => entry.piece), [0, 1, 2]);
+    assert.deepEqual(bytes, expectedBytes(start, end - start + 1));
+  } finally {
+    store.destroy(() => undefined);
+    await fs.rm(directory, { recursive: true, force: true });
+  }
+});
+
+test("a file that does not start at a piece boundary is still read correctly", async () => {
+  // The usual case in a multi-file torrent, and the one where using file
+  // offsets as if they were torrent offsets returns the wrong bytes at the
+  // right length.
+  const fileOffset = PIECE + 300;
+  const { torrent, store, directory } = await fakeTorrent({
+    fileOffset,
+    fileLength: 2 * PIECE,
+    totalLength: 5 * PIECE
+  });
+  try {
+    const { bytes } = await readRange(torrent, 0, 1499);
+    assert.deepEqual(bytes, expectedBytes(fileOffset, 1500));
+  } finally {
+    store.destroy(() => undefined);
+    await fs.rm(directory, { recursive: true, force: true });
+  }
+});
+
+test("every fragment releases its pin, so nothing stays held", async () => {
+  const { torrent, store, directory } = await fakeTorrent({
+    fileOffset: 0,
+    fileLength: 4 * PIECE,
+    totalLength: 4 * PIECE
+  });
+  try {
+    await readRange(torrent, 0, 4 * PIECE - 1);
+    // With every piece unpinned the store can still make room; if a pin leaked
+    // it would eventually refuse.
+    const before = store.stats().blockedByPins;
+    for (let index = 0; index < 200; index += 1) {
+      await store.reside(index % 4);
+    }
+    assert.equal(store.stats().blockedByPins, before, "a pin was left behind");
+  } finally {
+    store.destroy(() => undefined);
+    await fs.rm(directory, { recursive: true, force: true });
+  }
+});