@leejpsd/nextjs-cache-handler 0.1.1 → 0.3.0

package/CHANGELOG.md

@@ -1,11 +1,128 @@
 # Changelog
 
+## 0.3.0
+
+### Minor Changes
+
+- [`dc1ac0a`](https://github.com/leejpsd/nextjs-cache-handler/commit/dc1ac0a0b29e73485f4af587f8ec0f4b68d1d54f) Thanks [@leejpsd](https://github.com/leejpsd)! - Reliability fixes (reconnect backoff instead of a permanent connect-failure
+  latch, bounded memory fallback, namespace-scoped tag propagation without
+  truncation, strict `fallback: "never"` honored by the ISR handler, working
+  ESM peer loading) plus new features: Next.js 15 ISR support, request-scoped
+  GET deduplication, transparent gzip/brotli value compression, Redis Sentinel
+  support, a built-in OpenTelemetry adapter at `/otel`, and `memoryMaxEntries`.
+
 All notable changes to this project are documented here. The format is
 based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
 this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+### Added
+
+- **Next.js 15 support** for the singular `cacheHandler` (ISR): both ctx
+  generations accepted (`ctx.revalidate`/`kindHint` and
+  `ctx.cacheControl`/`kind`); `peerDependencies.next` widened to
+  `>=15.0.0 <17`. Verified end to end against Next 15.5 (build, ISR
+  HIT/MISS, tag/path revalidation, fetch-cache TTL, cross-process reads).
+- **Request-scoped GET deduplication** (singular handler):
+  `resetRequestCache()` is now a real implementation — duplicate reads of a
+  cache key within one request share a single Redis GET. New metric event
+  `cache.get.deduped`.
+- **Transparent value compression** (`compression: "gzip" | "brotli"`,
+  default off) via node:zlib — marker-prefixed storage with auto-detecting
+  reads, safe to enable on a live cache with a mixed fleet. Payloads < 1 KiB
+  stay plain.
+- **Redis Sentinel support** (`{ type: "sentinel", sentinels, name }`) via
+  ioredis master discovery — verified locally including a live failover.
+- **Built-in OpenTelemetry adapter** at
+  `@leejpsd/nextjs-cache-handler/otel` (`createOtelMetricEmitter`);
+  `@opentelemetry/api` is an optional peer resolved at runtime.
+- **`memoryMaxEntries` option** (default 1000) bounding the in-memory
+  fallback stores with LRU eviction.
+
+### Fixed
+
+- **Connect failures no longer latch the process into memory-only mode.**
+  Reconnects retry with exponential backoff (1s→30s cap) and dropped
+  connections are replaced (dead clients disposed).
+- **`refreshTags` scans only the current build namespace** and processes
+  markers chunk-by-chunk — the previous 500-key cap silently left tags
+  unpropagated; the local mirror is now rebuilt per complete scan (bounded
+  by live markers) and `memTagExp` is capped.
+- **`fallback: "never"` is honored by the singular handler** (it previously
+  read/wrote the memory fallback unconditionally, silently degrading strict
+  mode to per-process caching).
+- **ESM build can load optional peers**: bare `require()` calls became
+  esbuild's throwing `__require` fallback in ESM; now routed through
+  `createRequire` with a CWD fallback for symlinked installs (npm file
+  installs, pnpm).
+
+### Changed
+
+- Multi-tag `updateTags` fans out concurrently (one round trip via
+  auto-pipelining) instead of one await per tag; ioredis clients enable
+  `enableAutoPipelining`.
+- Removed the declared-but-never-emitted `cache.stale.refresh.skipped`
+  metric event type.
+
+## [0.2.0] - 2026-05-10
+
+First minor bump. Three differentiators land in this release.
+
+### Added
+
+- **Single-flight refresh lock** (`singleFlight: true`,
+  `singleFlightLockTtlSec: 10`). Opt-in stampede protection at the SWR
+  boundary. Uses the bundled `refresh-tag-lock.lua` script over a Redis
+  SETNX-style lock; only the leader's stale read triggers the
+  background refresh, all others fall through to the follower path.
+  Two new `MetricEvent` types (`cache.stale.refresh.leader`,
+  `cache.stale.refresh.follower`) appear in `onMetric` so operators can
+  verify leadership balance. Lock-acquisition failures are best-effort
+  and always degrade to the safe follower path; the stale entry is
+  served regardless.
+- **OpenTelemetry reference adapter** at
+  [`examples/opentelemetry/`](./examples/opentelemetry/). The library
+  itself stays dependency-free; the example shows the smallest viable
+  wiring of `onMetric` to OTel counters and histograms with bounded
+  cardinality (no cache keys or tag names emitted as attributes).
+- **Integration tests against real Redis 7** (21 scenarios) covering
+  both `redis@5` and `ioredis` adapters. Each adapter runs the same
+  test grid so a regression in either client library is pinpointed
+  immediately. New scripts: `npm run test:integration`,
+  `npm run test:integration:up` / `:down` for the docker-compose
+  fixture.
+- **CI integration job** runs the integration suite against an
+  ephemeral Redis 7 service container on every PR.
+- **GitHub Actions OIDC publish path** in `.github/workflows/release.yml`
+  with `id-token: write` and `NPM_CONFIG_PROVENANCE=true`. Currently
+  scoped to `workflow_dispatch` trigger so the first GHA-driven publish
+  can be observed; flip to `push: main` once the changesets cadence
+  stabilizes. Tarballs published from the workflow will land on npm
+  with verified provenance.
+
+### Changed
+
+- `MetricEventType` union grew three new entries:
+  `cache.stale.refresh.leader`, `cache.stale.refresh.follower`,
+  `cache.stale.refresh.skipped`. The legacy `cache.stale` event still
+  fires when `singleFlight` is off (default).
+- `HandlerState` now carries an `instanceId` (read from `HOSTNAME` /
+  `ECS_TASK_ID` / `pid-<pid>`) and embeds it in every refresh-lock
+  acquisition for operator-side observability.
+
+### Compatibility
+
+- No breaking changes. All existing 0.1.x configurations continue to
+  work; `singleFlight` defaults to `false`.
+
+### Verified
+
+- 72 unit tests (8 files) + 21 integration tests (1 file × 21 scenarios)
+  all green.
+- arethetypeswrong: 28/28 cells green.
+- publint: clean.
+
 ## [0.1.1] - 2026-05-10
 
 Documentation-only patch released alongside the first round of external
@@ -59,8 +176,6 @@ runtime cache key shapes.
 - Provenance attestation is disabled for the local-publish flow; v0.2
   will switch to GitHub Actions OIDC publish
 
-
-
 ### Added — `cacheHandlers` (plural) for `'use cache'`
 
 - `createCacheComponentsHandler(opts)` factory implementing the full

package/README.md

@@ -3,12 +3,20 @@
 [![npm](https://img.shields.io/npm/v/@leejpsd/nextjs-cache-handler.svg)](https://www.npmjs.com/package/@leejpsd/nextjs-cache-handler)
 [![license](https://img.shields.io/npm/l/@leejpsd/nextjs-cache-handler.svg)](./LICENSE)
 
-> **Stable `v0.1.0`** — install with
+> **`v0.3.0`** — install with
 > `npm install @leejpsd/nextjs-cache-handler`. Production-validated against
-> AWS ECS Fargate with multi-instance Redis (24h live-traffic soak: 0
-> errors, 0 leaks, 2ms Redis ping, namespace isolation working).
-
-The Redis cache handler for **Next.js 16** that ships **both** `cacheHandler`
+> AWS ECS Fargate with multi-instance Redis (v0.1: 24h live-traffic soak;
+> v0.3: fresh multi-instance verification incl. a live Redis-reboot drill
+> with zero 5xx — see
+> [`docs/staging-verification-2026-08-01.md`](./docs/staging-verification-2026-08-01.md)).
+>
+> v0.3 adds: **Next.js 15 support** (ISR handler), **reconnect with
+> exponential backoff** (no more permanent memory-only latch), **request-scoped
+> GET deduplication**, transparent **gzip/brotli compression**, **Redis
+> Sentinel** support, a built-in **OpenTelemetry emitter** at `/otel`, and an
+> LRU-bounded memory fallback.
+
+The Redis cache handler for **Next.js 15/16** that ships **both** `cacheHandler`
 (ISR / Pages Router) **and** `cacheHandlers` (`'use cache'` directive,
 `cacheComponents: true`) — the area where
 [`@fortedigital/nextjs-cache-handler`](https://github.com/fortedigital/nextjs-cache-handler)
@@ -49,18 +57,19 @@ As of 2026-05, the leading OSS Redis handler `@fortedigital/nextjs-cache-handler
 declares `peerDependencies.next: ">=16.1.5"` but the README marks the new
 plural interface as ❌ **"Not yet supported - Help needed"**:
 
-> 📅 **Compatibility matrix verified 2026-05-10.** The OSS Next.js cache
+> 📅 **Compatibility matrix re-verified 2026-07-31** (from each project's
+> published README/registry metadata). The OSS Next.js cache
 > handler ecosystem moves quickly — please verify
 > [`@fortedigital`](https://github.com/fortedigital/nextjs-cache-handler#compatibility)
 > and
 > [`nextjs-turbo-redis-cache`](https://github.com/trieb-work/nextjs-turbo-redis-cache#features)
 > directly before relying on this comparison.
 
-| Next 16 feature | this | @fortedigital 3.2.0 | nextjs-turbo-redis-cache 1.13 |
+| Feature | this (0.3.0) | @fortedigital 3.2.1 | nextjs-turbo-redis-cache 1.15 |
 |---|---|---|---|
 | `cacheHandlers` config (plural) | ✅ | ❌ Help needed | ✅ since 1.11 |
 | `'use cache'` directive | ✅ | ❌ Help needed | ✅ since 1.11 |
-| `'use cache: remote'` | ✅ | ❌ Help needed | partial |
+| `'use cache: remote'` | ✅ default handler (dedicated multi-tier: roadmap) | ❌ Help needed | partial |
 | `'use cache: private'` | n/a (uncustomizable) | n/a | n/a |
 | `cacheComponents: true` | ✅ | ❌ Help needed | ✅ |
 | Build-phase skip (`PHASE_PRODUCTION_BUILD`) | ✅ | ✅ (singular only) | ✅ |
@@ -70,13 +79,17 @@ plural interface as ❌ **"Not yet supported - Help needed"**:
 | Redis Cluster | ✅ (cluster adapter, see Production checklist) | ✅ | ✅ |
 | ioredis support | ✅ | ✅ | ✅ |
 | In-memory fallback (TTL-aware) | ✅ | partial | ✅ L1 + Redis L2 |
-| OpenTelemetry hook | ✅ `onMetric` | ❌ | ❌ |
+| Next 15 support (ISR handler) | ✅ `>=15.0.0` | ✅ (legacy 2.x line) | ✅ `>=15.0.3` |
+| Request-scoped GET dedup | ✅ | ❌ | ✅ |
+| Built-in value compression | ✅ gzip/brotli option | example only | example only |
+| Redis Sentinel | ✅ (local failover drill) | ❌ | ❌ |
+| OpenTelemetry | ✅ built-in `/otel` emitter + `onMetric` hook | ❌ | ❌ |
+| Reconnect strategy | ✅ exponential backoff | client-level | error-threshold restart |
 | Live-traffic dogfood (24h+) | ✅ AWS ECS Fargate | not published | not published |
 
 PR [#207](https://github.com/fortedigital/nextjs-cache-handler/pull/207) on
-`@fortedigital` has been stalled for 3+ months on the same issue: the
-upstream review insisted on `PHASE_PRODUCTION_BUILD` handling, which this
-package has from the start.
+`@fortedigital` (their `cacheHandlers` attempt) was held up in review over
+`PHASE_PRODUCTION_BUILD` handling — which this package has from the start.
 
 ---
 
@@ -171,13 +184,17 @@ export default async function Page() {
 ```ts
 interface CacheHandlerOptions {
   client: RedisClientFactory | RedisClientConfig;
-  keyPrefix?: string;            // default: "next-cache:" / "next-incremental:"
+  keyPrefix?: string;             // default: "next-cache:" / "next-incremental:"
   buildNamespace?: string | (() => string);  // default: env DEPLOYMENT_VERSION || GIT_HASH || "unversioned"
-  abortTimeoutMs?: number;       // default: 1500
+  abortTimeoutMs?: number;        // default: 1500
   fallback?: "auto" | "always" | "never";    // default: "auto"
   staleWhileRevalidate?: boolean; // default: true (cache-components only)
-  isBuildPhase?: () => boolean;  // override PHASE_PRODUCTION_BUILD detection
-  hashTag?: boolean;             // default: false (set true on Redis Cluster)
+  singleFlight?: boolean;         // default: false — see "Single-flight refresh lock" below
+  singleFlightLockTtlSec?: number; // default: 10
+  isBuildPhase?: () => boolean;   // override PHASE_PRODUCTION_BUILD detection
+  hashTag?: boolean;              // default: false (set true on Redis Cluster)
+  memoryMaxEntries?: number;      // default: 1000 — LRU cap for the in-memory fallback
+  compression?: "gzip" | "brotli"; // default: off — transparent value compression (node:zlib)
   onMetric?: (event: MetricEvent) => void;
   logger?: Logger;
 }
@@ -185,10 +202,12 @@ interface CacheHandlerOptions {
 type RedisClientConfig =
   | { type: "redis";    url: string; password?: string; tls?: boolean; connectTimeout?: number }
   | { type: "ioredis";  url: string; password?: string; tls?: boolean; connectTimeout?: number }
-  | { type: "cluster";  nodes: { host: string; port: number }[]; password?: string; tls?: boolean };
+  | { type: "cluster";  nodes: { host: string; port: number }[]; password?: string; tls?: boolean }
+  | { type: "sentinel"; sentinels: { host: string; port: number }[]; name: string;
+      password?: string; sentinelPassword?: string; tls?: boolean; connectTimeout?: number };
 ```
 
-Full reference: [`docs/api.md`](./docs/api.md) (auto-generated).
+Full reference: [`docs/api.md`](./docs/api.md).
 
 ---
 
@@ -227,12 +246,82 @@ Full reference: [`docs/api.md`](./docs/api.md) (auto-generated).
 | **Self-hosted Redis 7+** | `{ type: "redis", url }` or `{ type: "ioredis", url }` | ✅ AWS ElastiCache 24h soak |
 | **Redis Cluster** | `{ type: "cluster", nodes }` + `hashTag: true` | unit-tested, not yet load-tested at scale |
 | **Upstash Redis** | `{ type: "redis", url: "rediss://..." }` (TLS auto-detected) | not yet validated, expected to work via the standard Redis protocol |
-| **AWS ElastiCache (replication group)** | `{ type: "redis", url: "rediss://..." }` | ✅ reference deployment |
-| **Vercel KV** | not yet supported — `@vercel/kv` adapter ships in v0.3 | — |
+| **AWS ElastiCache (replication group)** | `{ type: "redis", url: "rediss://..." }` | ✅ reference deployment (re-verified 2026-08-01, Seoul) |
+| **Redis Sentinel** | `{ type: "sentinel", sentinels, name }` | ✅ local master/replica failover drill |
+| **Vercel KV** | not yet supported — dedicated adapter on the roadmap | — |
 | **DragonflyDB / KeyDB** | Redis-protocol compatible — `{ type: "redis", url }` should work | not validated |
 
 ---
 
+## Single-flight refresh lock (v0.2)
+
+The `cacheHandlers` (plural) interface returns stale entries inside the
+SWR window so users get an instant response while the background
+refresh completes. With many instances, the moment an entry crosses the
+`revalidate` boundary, every instance independently triggers its own
+refresh — N parallel re-renders for the same key, each hitting your
+origin once.
+
+`singleFlight: true` adds an opt-in Redis lock (`refresh-tag-lock.lua`,
+default TTL 10s) at the SWR boundary. The first instance to acquire it
+becomes the **leader** and runs the refresh; the rest become
+**followers**, keep serving the same stale entry, and wait for the
+leader's write to land. The lock is observability-only at the handler
+layer — Next.js still drives the actual refresh; we just suppress the
+stampede.
+
+```js
+createCacheComponentsHandler({
+  client: { type: "redis", url: process.env.REDIS_URL },
+  singleFlight: true,         // default false
+  singleFlightLockTtlSec: 10, // default 10
+});
+```
+
+Two new `MetricEvent` types appear on `onMetric`:
+
+| event type | meaning |
+|---|---|
+| `cache.stale.refresh.leader` | this instance just acquired the lock and is the designated refresher |
+| `cache.stale.refresh.follower` | another instance holds the lock; we serve stale and skip the refresh |
+
+If lock acquisition fails (Redis hiccup, TTL race), the handler
+defaults to the follower path — the stale entry is always served, never
+dropped. This is intentional: the lock is an optimization, not a
+correctness-critical primitive.
+
+When **not** to enable single-flight: small fleets (1–2 instances) where
+Next's per-process serialization already covers the stampede risk.
+Adding a Redis round-trip per stale read isn't free.
+
+See [`docs/architecture.md`](./docs/architecture.md#single-flight-refresh-lock-v02-)
+for the full state machine and a reference to the Lua script body.
+
+---
+
+## OpenTelemetry instrumentation (v0.2)
+
+The handler doesn't bundle `@opentelemetry/api` (zero runtime
+dependencies stays a goal). Instead, the `onMetric(event)` hook gives
+strictly-typed events you can pipe into whatever observability stack
+you already run.
+
+[`examples/opentelemetry/`](./examples/opentelemetry/) is a copy-paste
+reference wrapper that:
+
+- exposes a `nextjs_cache.events_total` counter dimensioned on
+  `type` / `freshness` / `backend` / `reason` / `op`
+- exposes a `nextjs_cache.op_latency_ms` histogram for events that
+  carry an `ms` field
+- keeps cardinality bounded — cache keys and tag names are never
+  emitted as attributes
+
+See [`examples/opentelemetry/README.md`](./examples/opentelemetry/README.md)
+for setup and three suggested dashboards (hit rate, single-flight
+leadership distribution, op latency tails).
+
+---
+
 ## How it differs from `@fortedigital/nextjs-cache-handler`
 
 Three deliberate departures, all rooted in lessons from production
@@ -261,10 +350,16 @@ eventual implementation should converge on the same behavior.
 
 ## Compatibility
 
-- **Next.js**: `>=16.1.5 <17`
+- **Next.js**: `>=15.0.0 <17`
+  - `cacheHandler` (singular, ISR): Next **15 and 16** — the handler accepts
+    both ctx shapes (Next 15's `ctx.revalidate` / `kindHint`, Next 16's
+    `ctx.cacheControl` / `kind`)
+  - `cacheHandlers` (plural, `'use cache'`): Next **>=16.1.5** only — the
+    interface does not exist before 16
 - **Node.js**: `>=20`
 - **redis**: `>=5.0.0` (peer, optional)
-- **ioredis**: `>=5.0.0` (peer, optional)
+- **ioredis**: `>=5.0.0` (peer, optional) — also powers `type: "cluster"` and
+  `type: "sentinel"` (Sentinel master discovery with automatic failover)
 
 ESM and CJS dual-published, full TypeScript types, validated via
 [`arethetypeswrong`](https://arethetypeswrong.github.io) and
@@ -274,14 +369,20 @@ ESM and CJS dual-published, full TypeScript types, validated via
 
 ## Roadmap
 
-- **v0.1.0** *(target: 2026-05)* — Both handlers, SWR, Lua atomicity,
-  build-phase skip, `redis@5` + `ioredis` adapters, AbortSignal,
-  in-memory fallback with TTL, soft-tag freshness check
-- **v0.2.0** — Single-flight refresh lock (Lua SETNX), OpenTelemetry
-  forwarder example
-- **v0.3.0** — Vercel KV / Upstash Redis adapter, `'use cache: remote'`
-  multi-tier setup
-- **v0.4.0** — Cache stampede protection, request-scoped memoization
+- **v0.1.0** *(2026-05)* — Both handlers, SWR, Lua atomicity, build-phase
+  skip, `redis@5` + `ioredis` adapters, AbortSignal, in-memory fallback
+  with TTL, soft-tag freshness check ✅
+- **v0.2.0** *(2026-05)* — Single-flight refresh lock with leader/follower
+  metrics, OpenTelemetry reference adapter, integration tests against
+  real Redis 7 (21 scenarios over `redis@5` + `ioredis`), GitHub Actions
+  OIDC publish path with provenance attestation ✅
+- **v0.3.0** *(2026-08)* — Next.js 15 ISR support, reconnect backoff (no
+  permanent connect-failure latch), request-scoped GET deduplication,
+  gzip/brotli compression, Redis Sentinel, built-in OpenTelemetry emitter
+  (`/otel`), LRU-bounded memory fallback, ESM peer-loading fixes ✅
+- **v0.4.0** — Build-phase cache seeding (prepopulate prebuilt pages),
+  pub/sub-based tag propagation, Redis Cluster e2e validation, Vercel KV /
+  Upstash adapter, `'use cache: remote'` multi-tier setup
 
 ---