@durable-streams/client 0.2.0 → 0.2.2

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@durable-streams/client",
   "description": "TypeScript client for the Durable Streams protocol",
-  "version": "0.2.0",
+  "version": "0.2.2",
   "author": "Durable Stream contributors",
   "license": "Apache-2.0",
   "repository": {
@@ -36,18 +36,25 @@
     "./package.json": "./package.json"
   },
   "sideEffects": false,
+  "bin": {
+    "intent": "./bin/intent.js"
+  },
   "files": [
     "dist",
-    "src"
+    "src",
+    "skills",
+    "bin",
+    "!skills/_artifacts"
   ],
   "dependencies": {
     "@microsoft/fetch-event-source": "^2.0.1",
     "fastq": "^1.19.1"
   },
   "devDependencies": {
+    "@tanstack/intent": "latest",
     "fast-check": "^4.4.0",
     "tsdown": "^0.9.0",
-    "@durable-streams/server": "0.1.7"
+    "@durable-streams/server": "0.2.2"
   },
   "engines": {
     "node": ">=18.0.0"

package/skills/getting-started/SKILL.md

@@ -0,0 +1,223 @@
+---
+name: getting-started
+description: >
+  First-time setup for Durable Streams. Install @durable-streams/client,
+  create a stream with DurableStream.create(), read with stream(), subscribe
+  to live updates, resume from saved offsets. Covers offset semantics ("-1",
+  "now", opaque tokens), LiveMode (false, true, "long-poll", "sse"), and
+  StreamResponse consumption (.json(), .text(), .subscribeJson()).
+type: lifecycle
+library: durable-streams
+library_version: "0.2.1"
+sources:
+  - "durable-streams/durable-streams:README.md"
+  - "durable-streams/durable-streams:packages/client/src/stream-api.ts"
+  - "durable-streams/durable-streams:packages/client/src/stream.ts"
+---
+
+# Durable Streams — Getting Started
+
+Durable Streams is an HTTP-based protocol for persistent, resumable, append-only
+event streams. Use `stream()` for reading and `DurableStream` when you also need
+to create or write to streams.
+
+## Setup
+
+```typescript
+import { stream, DurableStream } from "@durable-streams/client"
+
+// Create a JSON stream (use DurableStream for write operations)
+const handle = await DurableStream.create({
+  url: "https://your-server.com/v1/stream/my-stream",
+  contentType: "application/json",
+})
+
+// Write some data
+await handle.append(JSON.stringify({ event: "user.created", userId: "123" }))
+await handle.append(JSON.stringify({ event: "user.updated", userId: "123" }))
+
+// Read all data (use stream() for read-only access)
+const res = await stream({
+  url: "https://your-server.com/v1/stream/my-stream",
+  offset: "-1",
+  live: false,
+})
+const items = await res.json()
+// [{ event: "user.created", userId: "123" }, { event: "user.updated", userId: "123" }]
+```
+
+## Core Patterns
+
+### Read all existing data (catch-up)
+
+```typescript
+import { stream } from "@durable-streams/client"
+
+const res = await stream({
+  url: "https://your-server.com/v1/stream/my-stream",
+  offset: "-1", // Start from beginning
+  live: false, // Stop after catching up
+})
+const data = await res.json()
+const savedOffset = res.offset // Save for resumption
+```
+
+### Subscribe to live updates
+
+```typescript
+import { stream } from "@durable-streams/client"
+
+const res = await stream({
+  url: "https://your-server.com/v1/stream/my-stream",
+  offset: "-1", // Catch up first, then continue live
+  live: true, // Auto-selects best transport (SSE for JSON, long-poll for binary)
+})
+
+res.subscribeJson(async (batch) => {
+  for (const item of batch.items) {
+    console.log("Received:", item)
+  }
+  saveCheckpoint(batch.offset) // Persist for resumption
+})
+```
+
+### Resume from a saved offset
+
+```typescript
+import { stream } from "@durable-streams/client"
+
+const savedOffset = loadCheckpoint() // Load previously saved offset
+
+const res = await stream({
+  url: "https://your-server.com/v1/stream/my-stream",
+  offset: savedOffset, // Resume from where we left off
+  live: true,
+})
+
+res.subscribeJson(async (batch) => {
+  for (const item of batch.items) {
+    processItem(item)
+  }
+  saveCheckpoint(batch.offset)
+})
+```
+
+### Create and write to a stream
+
+```typescript
+import { DurableStream, IdempotentProducer } from "@durable-streams/client"
+
+const handle = await DurableStream.create({
+  url: "https://your-server.com/v1/stream/my-stream",
+  contentType: "application/json",
+})
+
+// For simple one-off writes, use append() directly
+await handle.append(JSON.stringify({ event: "hello" }))
+
+// For sustained writes, use IdempotentProducer (faster, exactly-once)
+const producer = new IdempotentProducer(handle, "my-service", {
+  autoClaim: true,
+  onError: (err) => console.error("Write failed:", err),
+})
+
+producer.append(JSON.stringify({ event: "world" })) // Fire-and-forget
+await producer.flush() // Ensure delivery before shutdown
+await producer.close()
+```
+
+## Common Mistakes
+
+### CRITICAL Parsing or constructing offsets manually
+
+Wrong:
+
+```typescript
+const nextOffset = `${parseInt(offset.split("_")[0]) + 1}_0`
+```
+
+Correct:
+
+```typescript
+const nextOffset = response.offset // Always use server-returned offset
+```
+
+Offsets are opaque tokens. The internal format is an implementation detail that may change between server versions.
+
+Source: PROTOCOL.md section 6 (Offsets)
+
+### CRITICAL Using offset 0 instead of "-1" for stream start
+
+Wrong:
+
+```typescript
+const res = await stream({ url, offset: "0" })
+```
+
+Correct:
+
+```typescript
+const res = await stream({ url, offset: "-1" })
+```
+
+The special start-of-stream offset is the string `"-1"`, not `"0"`. Using `"0"` may miss data or return 400.
+
+Source: README.md offset semantics section
+
+### HIGH Calling multiple consumption methods on same response
+
+Wrong:
+
+```typescript
+const res = await stream({ url, offset: "-1" })
+const data = await res.json()
+res.subscribeJson((batch) => {
+  /* ... */
+}) // throws ALREADY_CONSUMED!
+```
+
+Correct:
+
+```typescript
+const res = await stream({ url, offset: "-1", live: true })
+res.subscribeJson((batch) => {
+  for (const item of batch.items) {
+    /* process */
+  }
+})
+```
+
+StreamResponse enforces single consumption. Choose one consumption method per response.
+
+Source: packages/client/src/response.ts
+
+### HIGH Setting live mode for one-shot reads
+
+Wrong:
+
+```typescript
+const res = await stream({ url, offset: "-1", live: true })
+const data = await res.json() // hangs until stream closes
+```
+
+Correct:
+
+```typescript
+const res = await stream({ url, offset: "-1", live: false })
+const data = await res.json() // returns immediately with existing data
+```
+
+Use `live: false` for catch-up reads. `live: true` keeps the connection open waiting for new data.
+
+Source: packages/client/src/types.ts LiveMode type
+
+## See also
+
+- [writing-data](../writing-data/SKILL.md) — IdempotentProducer for production-grade writes
+- [server-deployment](../server-deployment/SKILL.md) — Setting up a server to develop against
+
+Note: Streams must be created with `DurableStream.create()` before they can be read. See the writing-data skill for stream creation.
+
+## Version
+
+Targets @durable-streams/client v0.2.1.

package/skills/go-to-production/SKILL.md

@@ -0,0 +1,243 @@
+---
+name: go-to-production
+description: >
+  Production readiness checklist for durable streams. Switch from dev server
+  to Caddy binary, configure CDN caching with offset-based URLs,
+  Cache-Control and ETag headers, Stream-Cursor for cache collision prevention,
+  TTL and Stream-Expires-At for stream lifecycle, HTTPS requirement, request
+  collapsing for fan-out, CORS configuration. Load before deploying durable
+  streams to production.
+type: lifecycle
+library: durable-streams
+library_version: "0.2.1"
+requires:
+  - server-deployment
+sources:
+  - "durable-streams/durable-streams:PROTOCOL.md"
+  - "durable-streams/durable-streams:packages/caddy-plugin/README.md"
+---
+
+This skill builds on durable-streams/server-deployment. Read it first for server setup basics.
+
+# Durable Streams — Go to Production Checklist
+
+Run through each section before deploying to production.
+
+## Server Checks
+
+### Check: Using Caddy production server (not dev server)
+
+Expected:
+
+```bash
+./durable-streams-server run --config Caddyfile
+```
+
+Fail condition: Importing `DurableStreamTestServer` from `@durable-streams/server` in production code.
+
+Fix: Download the Caddy binary from GitHub releases and configure with a Caddyfile.
+
+### Check: File-backed persistence configured
+
+Expected:
+
+```
+durable_streams {
+  data_dir ./data
+  max_file_handles 200
+}
+```
+
+Fail condition: No `data_dir` in Caddyfile — server uses in-memory storage and loses data on restart.
+
+Fix: Add `data_dir` pointing to a persistent directory.
+
+## Transport Checks
+
+### Check: HTTPS enabled
+
+Expected:
+
+```typescript
+const res = await stream({
+  url: "https://your-server.com/v1/stream/my-stream",
+})
+```
+
+Fail condition: Using `http://` URLs in production. Pre-signed URLs and auth tokens are bearer credentials — HTTP exposes them in transit. HTTP/1.1 also limits browsers to ~6 concurrent connections per origin.
+
+Fix: Configure TLS on the Caddy server (Caddy provides automatic HTTPS by default).
+
+## Stream Lifecycle Checks
+
+### Check: TTL or expiration set on streams
+
+Expected:
+
+```typescript
+const handle = await DurableStream.create({
+  url: "https://server.com/v1/stream/my-stream",
+  contentType: "application/json",
+  headers: { "Stream-TTL": "86400" }, // 24 hours
+})
+```
+
+Fail condition: Streams created without TTL persist forever, causing unbounded storage growth.
+
+Fix: Set `Stream-TTL` (seconds) or `Stream-Expires-At` (ISO timestamp) on stream creation. Use exactly one, not both.
+
+### Check: Not specifying both TTL and Expires-At
+
+Expected:
+
+```typescript
+headers: { "Stream-TTL": "86400" }
+// OR
+headers: { "Stream-Expires-At": "2026-04-01T00:00:00Z" }
+```
+
+Fail condition: Providing both `Stream-TTL` and `Stream-Expires-At` returns 400 Bad Request.
+
+Fix: Use one or the other. TTL is relative (seconds from creation), Expires-At is absolute.
+
+## CDN and Caching Checks
+
+### Check: CDN-friendly URL structure
+
+Expected:
+
+Reads use offset-based URLs that are naturally cacheable:
+
+```
+GET /v1/stream/my-stream?offset=abc123
+```
+
+The server returns `Cache-Control` and `ETag` headers automatically for historical reads. CDNs can cache and collapse requests — 10,000 viewers at the same offset become one upstream request.
+
+Fail condition: Overriding or stripping `Cache-Control` headers at the CDN/proxy layer.
+
+Fix: Allow the server's `Cache-Control` and `ETag` headers to pass through to the CDN.
+
+### Check: Stream-Cursor header preserved
+
+`Stream-Cursor` prevents CDN cache collisions when the same offset returns different data (e.g., after stream truncation). Ensure your CDN does not strip this header.
+
+Fail condition: CDN strips `Stream-Cursor` from responses.
+
+Fix: Configure CDN to pass through `Stream-Cursor` response header.
+
+## Error Handling Checks
+
+### Check: onError handler configured for live streams
+
+Expected:
+
+```typescript
+const res = await stream({
+  url,
+  offset: "-1",
+  live: true,
+  onError: (error) => {
+    if (error.status === 401) return // Stop retrying
+    return {} // Retry transient errors
+  },
+})
+```
+
+Fail condition: No `onError` handler — permanent errors (401, 403) silently retry forever.
+
+Fix: Add `onError` handler that stops retrying for non-transient errors.
+
+## Common Production Mistakes
+
+### CRITICAL Using HTTP in production with browser clients
+
+Wrong:
+
+```typescript
+const res = await stream({ url: "http://api.example.com/v1/stream/my-stream" })
+```
+
+Correct:
+
+```typescript
+const res = await stream({ url: "https://api.example.com/v1/stream/my-stream" })
+```
+
+Pre-signed URLs and auth tokens are bearer credentials. HTTP exposes these in transit. Also, HTTP/1.1 limits browsers to ~6 concurrent connections per origin.
+
+Source: packages/client/src/utils.ts warnIfUsingHttpInBrowser
+
+### HIGH Not setting TTL or expiration on streams
+
+Wrong:
+
+```typescript
+const handle = await DurableStream.create({
+  url: "https://server.com/v1/stream/my-stream",
+  contentType: "application/json",
+})
+```
+
+Correct:
+
+```typescript
+const handle = await DurableStream.create({
+  url: "https://server.com/v1/stream/my-stream",
+  contentType: "application/json",
+  headers: { "Stream-TTL": "86400" },
+})
+```
+
+Without TTL, streams persist forever causing unbounded storage growth.
+
+Source: PROTOCOL.md TTL and Expiry section
+
+### MEDIUM Specifying both TTL and Expires-At
+
+Wrong:
+
+```typescript
+headers: {
+  "Stream-TTL": "86400",
+  "Stream-Expires-At": "2026-04-01T00:00:00Z",
+}
+```
+
+Correct:
+
+```typescript
+headers: {
+  "Stream-TTL": "86400",  // OR Expires-At, not both
+}
+```
+
+The protocol requires exactly one. Providing both returns 400 Bad Request.
+
+Source: PROTOCOL.md TTL and Expiry section
+
+### HIGH Tension: Ephemeral producers vs. persistent coordination
+
+This skill's patterns conflict with writing-data. `autoClaim: true` is convenient for serverless/ephemeral workers but sacrifices cross-restart coordination. Persistent long-running workers may benefit from explicit epoch management for proper multi-worker coordination.
+
+See also: durable-streams/writing-data/SKILL.md § Common Mistakes
+
+## Pre-Deploy Summary
+
+- [ ] Using Caddy production server (not dev server)
+- [ ] `data_dir` configured for persistence
+- [ ] HTTPS enabled
+- [ ] TTL or Expires-At set on all streams
+- [ ] Not using both TTL and Expires-At together
+- [ ] CDN passes through Cache-Control, ETag, and Stream-Cursor headers
+- [ ] `onError` handler configured for live streams
+- [ ] `max_file_handles` tuned for expected stream count
+
+## See also
+
+- [server-deployment](../server-deployment/SKILL.md) — Initial server setup
+- [writing-data](../writing-data/SKILL.md) — Producer configuration for production
+
+## Version
+
+Targets durable-streams v0.2.1.