@torrent-tv/proxy 2.9.2 → 2.9.4

package/CHANGELOG.md

@@ -1,3 +1,12 @@
+## 2.9.4
+
+- **Chore**: Temporary `[net-debug]` instrumentation in `data-channel-handler.js` now splits transfer timing into `fetchMs` (waiting for the local route, incl. ffmpeg segment finalization), `ttfbMs` (time to first body chunk), `sendMs` (channel send duration) and `chunks`, to locate where early-segment latency is spent (transport vs segment production).
+
+## 2.9.3
+
+- **New**: WebRTC data-channel response bodies are now sent as **binary** frames (`sendMessageBinary`) instead of base64-encoded JSON `response-chunk` messages, removing the ~33% base64 overhead and the JSON encode cost. Frame layout: `[flags(1)][idLen(1)][requestId(ASCII)][payload]`. Control messages (`response-start`, `response-error`, `pong`) remain JSON strings. Requires the matching browser client (server ≥ 0.8.0); **deploy the server before the proxy**.
+- **New**: Backpressure on the send loop — `data-channel-handler.js` pauses queuing body chunks once the channel's `bufferedAmount()` exceeds 8 MB and resumes when it drains below 1 MB (`setBufferedAmountLowThreshold` + `onBufferedAmountLow`), with a 5 s timeout fallback. Prevents the SCTP send buffer from ballooning and stalling throughput.
+
 ## 2.6.3
 
 - **Fix**: Data channel handler now logs **all** requests regardless of body presence — `GET /transcode/…`, `GET /api/…/progress`, `GET /api/…/stats` etc. were previously invisible in logs. Non-2xx response statuses and fetch errors are also logged, enabling diagnosis of HLS manifest load failures.

package/CLAUDE.md

@@ -0,0 +1,62 @@
+# proxy — @torrent-tv/proxy (WebTorrent + ffmpeg)
+
+Downloads a torrent and streams the chosen file to the browser, transcoding to
+HLS only when needed. See the parent `../CLAUDE.md` for the overall architecture
+and release process.
+
+## Deployment-agnostic — important
+
+The HA addon is only ONE way to run this; bare npm and Docker are planned. Keep
+all code free of Home-Assistant assumptions. Anything host-specific (GPU
+devices, ffmpeg build, CLI flags) belongs in the `ha-addon` layer. Hardware
+detection must probe at runtime and fall back gracefully; do not assume a
+Linux-only host (e.g. POSIX-only signals must degrade elsewhere).
+
+## Layout
+
+- `bin/cli.js` — CLI entry; resolves `ffmpegBin` (uses `--ffmpeg-bin` if given,
+  else bundled ffmpeg-static, else PATH `ffmpeg`).
+- `server.js` — Fastify setup; detects the video encoder at startup
+  (`detectVideoEncoder`) and passes it to `HlsSessionManager`.
+- `routes/<path>/<method>.js` — same convention as the server repo.
+  - `routes/stream/get.js` — byte-range torrent file streaming (HTTP 206).
+  - `routes/api/playback-plan/post.js` — codec/container/duration probe result.
+  - `routes/transcode/session-file/get.js` — serves HLS playlist/segments;
+    long-polls while a segment is being produced, returns retryable 503 (never
+    202 — hls.js can't consume it).
+- `services/`:
+  - `playback-planner.js` — single ffmpeg probe returns audioCodec, videoCodec,
+    container, durationSeconds. `mode` is advisory; the browser decides.
+  - `hls-session-manager.js` — one ffmpeg per (source, file, settings). Serves a
+    synthetic full-duration VOD playlist; produces segments on demand; restarts
+    ffmpeg at the requested segment for server-side seeking. Short idle TTL.
+    Uses the detected `videoEncoder` for video re-encode (copy otherwise).
+  - `hwaccel.js` — detect best H.264 encoder (NVENC/QSV/VAAPI/V4L2M2M) with a
+    STRICT startup test: encode `testsrc2` through the real HLS pipeline, then
+    verify each segment decodes independently (catches non-IDR/corrupted hw
+    output). Falls back to software libx264. Runtime fallback to software if a
+    hw encode later fails. v4l2m2m is gated by this test (fails on HA Yellow).
+  - `data-channel-handler.js` — forwards WebRTC data-channel requests to the
+    local HTTP server (loopback), so the same routes serve both transports.
+
+## Gotchas
+
+- Do NOT use `-hls_playlist_type event` — it breaks duration/seek. VOD only.
+- A transitive dep (`ip-set`, via webtorrent) ships a hostile
+  `preinstall: npx only-allow pnpm` that breaks `npm install`. The addon works
+  around it with `--ignore-scripts` + a targeted rebuild of `node-datachannel`;
+  if you ever change install flow, keep that in mind.
+
+## Changelog
+
+Every behavioural change must be recorded in `CHANGELOG.md` — add an entry under
+a new `## <version>` heading at the top (the next patch version that
+`npm run patch` will publish), following the existing
+`- **New**/**Fix**/**Chore**:` format. See the parent `../CLAUDE.md`.
+
+## Release
+
+`npm run patch` (publishes to npm + pushes tags). The HA addon then needs its
+own version bump to pull the new package. Publish proxy BEFORE bumping the addon.
+
+**Any proxy change requires bumping the ha-addon version** (`ha-addon/torrent_tv_proxy/config.yaml`). The addon installs the proxy from npm at build time and the build is cached; without a version bump the plugin will NOT update and keeps running the old proxy. So after `npm run patch`, always bump the addon `config.yaml` version, push, and update the addon.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@torrent-tv/proxy",
-  "version": "2.9.2",
+  "version": "2.9.4",
   "description": "Torrent proxy client that exposes webseed-like HTTP stream endpoint.",
   "license": "GPL-3.0-or-later",
   "publishConfig": {

package/services/data-channel-handler.js

@@ -16,12 +16,23 @@
  *
  * Proxy → Browser
  * ```
- * { type: "response-start", requestId, status, headers }
- * { type: "response-chunk", requestId, data: string (base64), done: boolean }
- * { type: "response-error", requestId, error: string }
- * { type: "pong",           id }
+ * { type: "response-start", requestId, status, headers }   (JSON string)
+ * { type: "response-error", requestId, error: string }     (JSON string)
+ * { type: "pong",           id }                            (JSON string)
  * ```
  *
+ * Response bodies are sent as BINARY data-channel messages (not JSON), to
+ * avoid the ~33% base64 overhead and the JSON encode/decode cost. Each binary
+ * frame is laid out as:
+ * ```
+ * byte 0          flags     (bit 0: done)
+ * byte 1          idLen     (length of the requestId in bytes)
+ * bytes 2..2+N    requestId (ASCII)
+ * bytes 2+N..     payload   (raw body bytes; empty on the final done frame)
+ * ```
+ * Control messages stay JSON strings so the browser can distinguish them from
+ * body frames by message type (string vs ArrayBuffer).
+ *
  * The protocol mirrors the tunnel relay protocol so both transports share
  * the same mental model and the same browser-side `WebRtcProxy` implementation.
  */
@@ -151,6 +162,10 @@ export function createDataChannelHandler({ proxyPort, onLog }) {
     const requestHeaders = { ...(forwardedHeaders ?? {}), host: `127.0.0.1:${proxyPort}` };
 
     let response;
+    // [net-debug] TEMPORARY: time spent in the local fetch (waiting for the
+    // route to return a response — e.g. long-polling until an HLS segment is
+    // finalized by ffmpeg) vs. the body transfer over the data channel.
+    const fetchStartedAt = Date.now();
     try {
       response = await fetch(targetUrl, {
         method,
@@ -177,30 +192,116 @@ export function createDataChannelHandler({ proxyPort, onLog }) {
     send(channel, { type: "response-start", requestId, status: response.status, headers: responseHeaders });
 
     if (!response.body) {
-      send(channel, { type: "response-chunk", requestId, data: "", done: true });
+      sendChunk(channel, requestId, null, true);
       return;
     }
 
     try {
       const reader = response.body.getReader();
+      // [net-debug] TEMPORARY: measure transfer size/time and channel buffering.
+      // fetchMs = time waiting for the route (incl. ffmpeg segment finalization).
+      // ttfbMs  = time from body-read start to the first chunk with data (loopback).
+      // sendMs  = total body read+send duration over the data channel.
+      const fetchMs = Date.now() - fetchStartedAt;
+      const sendStartedAt = Date.now();
+      let firstByteMs = -1;
+      let chunks = 0;
+      let totalBytes = 0;
+      let maxBuffered = 0;
       while (true) {
         const { done, value } = await reader.read();
         if (done) {
-          send(channel, { type: "response-chunk", requestId, data: "", done: true });
+          sendChunk(channel, requestId, null, true);
+          const elapsedMs = Date.now() - sendStartedAt;
+          let bufferedNow = 0;
+          try { bufferedNow = typeof channel.bufferedAmount === "function" ? channel.bufferedAmount() : 0; } catch { /* ignore */ }
+          log(
+            `[net-debug] sent ${path}${queryInfo} bytes=${totalBytes} fetchMs=${fetchMs} ` +
+              `ttfbMs=${firstByteMs} sendMs=${elapsedMs} chunks=${chunks} ` +
+              `maxBuffered=${maxBuffered} bufferedAtEnd=${bufferedNow}`
+          );
           break;
         }
-        send(channel, {
-          type: "response-chunk",
-          requestId,
-          data: Buffer.from(value).toString("base64"),
-          done: false
-        });
+        if (firstByteMs < 0) firstByteMs = Date.now() - sendStartedAt;
+        chunks += 1;
+        totalBytes += value.length;
+        try {
+          const b = typeof channel.bufferedAmount === "function" ? channel.bufferedAmount() : 0;
+          if (b > maxBuffered) maxBuffered = b;
+        } catch { /* ignore */ }
+        sendChunk(channel, requestId, value, false);
+        // Backpressure: do not keep queuing chunks once the channel's outgoing
+        // buffer is large — wait for it to drain. Prevents the SCTP send buffer
+        // from ballooning, which stalls throughput.
+        await waitForBufferDrain(channel);
       }
     } catch {
-      send(channel, { type: "response-chunk", requestId, data: "", done: true });
+      sendChunk(channel, requestId, null, true);
+    }
+  }
+
+  /**
+   * Send a response body frame as a BINARY data-channel message.
+   * Layout: [flags(1)][idLen(1)][requestId(ASCII)][payload].
+   *
+   * @param {DataChannel}            channel
+   * @param {string}                 requestId
+   * @param {Uint8Array | null}      bytes - Body bytes, or null/empty for the done frame.
+   * @param {boolean}                done
+   * @returns {void}
+   */
+  function sendChunk(channel, requestId, bytes, done) {
+    try {
+      const idBuf = Buffer.from(requestId, "ascii");
+      const header = Buffer.allocUnsafe(2 + idBuf.length);
+      header[0] = done ? 1 : 0;
+      header[1] = idBuf.length;
+      idBuf.copy(header, 2);
+      const frame =
+        bytes && bytes.length > 0 ? Buffer.concat([header, Buffer.from(bytes)]) : header;
+      channel.sendMessageBinary(frame);
+    } catch {
+      // Channel closed between check and send — safe to ignore.
     }
   }
 
+  /**
+   * Resolve once the channel's outgoing buffer has drained below the low-water
+   * mark. No-op (resolves immediately) when the buffer is already small or the
+   * channel does not expose buffer APIs. A timeout fallback guards against a
+   * missed low-water event so the send loop can never deadlock.
+   *
+   * @param {DataChannel} channel
+   * @returns {Promise<void>}
+   */
+  function waitForBufferDrain(channel) {
+    return new Promise((resolve) => {
+      try {
+        if (typeof channel.bufferedAmount !== "function" || channel.bufferedAmount() <= DC_BUFFER_HIGH_WATER) {
+          resolve();
+          return;
+        }
+        let settled = false;
+        const done = () => {
+          if (settled) return;
+          settled = true;
+          resolve();
+        };
+        channel.setBufferedAmountLowThreshold(DC_BUFFER_LOW_WATER);
+        channel.onBufferedAmountLow(done);
+        // Guard against a race where the buffer drained between the check above
+        // and registering the callback (the low-water event would never fire).
+        if (channel.bufferedAmount() <= DC_BUFFER_LOW_WATER) {
+          done();
+          return;
+        }
+        setTimeout(done, DC_BUFFER_DRAIN_TIMEOUT_MS);
+      } catch {
+        resolve();
+      }
+    });
+  }
+
   /**
    * Serialise `message` to JSON and send it over the data channel.
    * Errors are silently swallowed — the channel may have closed between
@@ -226,3 +327,10 @@ export function createDataChannelHandler({ proxyPort, onLog }) {
  * Only the known proxy API and streaming routes are accepted.
  */
 const PATH_ALLOWLIST_RE = /^(?:\/api\/|\/stream(?:$|\?)|\/?transcode\/|\/health(?:z)?(?:$|\?))/;
+
+/** Pause sending body chunks once the channel buffer exceeds this many bytes. */
+const DC_BUFFER_HIGH_WATER = 8 * 1024 * 1024;
+/** Resume sending once the channel buffer drains to this many bytes. */
+const DC_BUFFER_LOW_WATER = 1 * 1024 * 1024;
+/** Safety fallback so the send loop cannot deadlock on a missed drain event. */
+const DC_BUFFER_DRAIN_TIMEOUT_MS = 5000;