@agent-foundry/replay-server 1.0.0

package/.cursor/project.mdc

@@ -0,0 +1,694 @@
+---
+alwaysApply: true
+---
+# Replay Server - Project Documentation
+
+## Project Overview
+
+Replay Server is a **video rendering service** for AgentFoundry mini-game replays. It uses Puppeteer to load games in a headless browser, injects replay manifests, and captures video frames that are streamed directly to FFmpeg for encoding.
+
+**Purpose**: Convert game replay manifests into MP4 video files for sharing and playback.
+
+**Tech Stack**:
+- **Node.js 18+** - Runtime
+- **TypeScript 5.3.3** - Type safety
+- **Express 4.18.2** - HTTP API server
+- **Puppeteer 22.0.0** - Headless browser automation
+- **FFmpeg** - Video encoding (via fluent-ffmpeg)
+- **pnpm** - Package manager (monorepo workspace)
+
+**Key Features**:
+- Headless browser rendering (zero code duplication with game)
+- Static bundle hosting (no external game server needed)
+- Streaming video encoding (no temporary files)
+- HTTP API for async job processing
+- CLI tool for offline rendering
+
+---
+
+## Architecture
+
+### High-Level Architecture
+
+```mermaid
+graph TB
+    subgraph Client["Client"]
+        API["HTTP API<br/>POST /render"]
+        CLI["CLI Tool<br/>render.ts"]
+    end
+    
+    subgraph Server["Replay Server"]
+        Express["Express Server<br/>Port 3001"]
+        JobManager["Job Manager<br/>In-Memory Store"]
+        Renderer["PuppeteerRenderer"]
+    end
+    
+    subgraph Browser["Headless Browser"]
+        Puppeteer["Puppeteer<br/>Chromium"]
+        GamePage["Game Page<br/>?mode=replay"]
+        Manifest["Manifest Injection<br/>window.__REPLAY_MANIFEST__"]
+    end
+    
+    subgraph Encoding["Video Encoding"]
+        FFmpeg["FFmpeg Process<br/>libx264/h264_nvenc"]
+        VideoFile["Output MP4<br/>H.264"]
+    end
+    
+    subgraph Bundles["Bundle Hosting"]
+        StaticServer["Static Server<br/>/bundles/*"]
+        GameBundle["Game Bundle<br/>bundles/game-life-restart/"]
+    end
+    
+    API --> Express
+    CLI --> Renderer
+    Express --> JobManager
+    Express --> Renderer
+    Renderer --> Puppeteer
+    Puppeteer --> GamePage
+    GamePage --> Manifest
+    Renderer --> FFmpeg
+    FFmpeg --> VideoFile
+    Puppeteer --> StaticServer
+    StaticServer --> GameBundle
+```
+
+### Rendering Flow
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Server
+    participant Renderer
+    participant Browser
+    participant Game
+    participant FFmpeg
+    
+    Client->>Server: POST /render {manifest, config}
+    Server->>Server: Create job (pending)
+    Server->>Client: {jobId, status: "pending"}
+    
+    Server->>Renderer: render(manifest, config)
+    Renderer->>Browser: Launch Chromium
+    Browser->>Game: Navigate to gameUrl?mode=replay
+    Game->>Game: Wait for React init
+    Renderer->>Game: Inject manifest (window.__REPLAY_MANIFEST__)
+    Renderer->>Game: Dispatch 'replay-manifest-loaded' event
+    Game->>Game: Enter replay mode
+    Game->>Renderer: Set data-replay-ready="true"
+    
+    Renderer->>FFmpeg: Launch FFmpeg process
+    loop For each age in timeline
+        Renderer->>Game: Dispatch 'replay-next-age' event
+        Game->>Game: Advance to next age
+        loop For each frame (fps * secondsPerAge)
+            Renderer->>Browser: Capture screenshot (JPEG)
+            Renderer->>FFmpeg: Write frame to stdin
+        end
+    end
+    
+    Renderer->>FFmpeg: Close stdin
+    FFmpeg->>FFmpeg: Encode video
+    FFmpeg->>Server: Write MP4 file
+    Renderer->>Server: Update job (completed)
+    Client->>Server: GET /status/:jobId
+    Server->>Client: {status: "completed", outputPath}
+    Client->>Server: GET /download/:jobId
+    Server->>Client: Video file
+```
+
+### Key Design Decisions
+
+#### 1. Puppeteer-Based Rendering
+
+**Why**: Zero code duplication. The actual game runs in a headless browser, ensuring the video looks exactly like gameplay.
+
+**How**: 
+- Game is loaded from URL (local dev) or bundle (production)
+- Replay manifest is injected via `window.__REPLAY_MANIFEST__`
+- Game enters "replay mode" and plays back events
+- Screenshots captured at each frame
+
+#### 2. Bundle Hosting
+
+**Why**: Eliminates need for external game server. Bundles are self-contained static files.
+
+**How**:
+- Game bundles built with `VITE_BASE_PATH=/bundles/<bundleId>/`
+- Served from Express static middleware at `/bundles/<bundleId>/`
+- Manifest includes `bundleId` field
+- Server resolves game URL: `http://localhost:PORT/bundles/<bundleId>/`
+
+#### 3. Streaming FFmpeg Encoding
+
+**Why**: No temporary files, lower disk I/O, faster encoding.
+
+**How**:
+- Screenshots captured as JPEG buffers
+- Frames written directly to FFmpeg stdin
+- FFmpeg configured with `-f image2pipe -vcodec mjpeg -i -`
+- Process handles stdin drain events for backpressure
+
+#### 4. Game URL Resolution Priority
+
+1. `config.gameUrl` (explicit URL in render request)
+2. `manifest.bundleId` → `/bundles/<bundleId>/` (bundle hosting)
+3. `GAME_URL` environment variable (fallback)
+
+---
+
+## Core Components
+
+### 1. HTTP Server (`src/server/index.ts`)
+
+Express-based API server handling render jobs, job status, and bundle management.
+
+**Key Responsibilities**:
+- Accept render requests with manifest
+- Manage job lifecycle (pending → processing → completed/failed)
+- Serve static game bundles
+- Provide job status and download endpoints
+
+**Job Storage**: In-memory Map (for now, could be Redis/DB in production)
+
+**Routes**:
+- `POST /render` - Start render job
+- `GET /jobs` - List all jobs
+- `GET /status/:jobId` - Get job status
+- `GET /download/:jobId` - Download completed video
+- `GET /bundles` - List available bundles
+- `GET /health` - Health check
+- `GET /debug/chrome` - Debug Chromium installation
+
+### 2. Puppeteer Renderer (`src/renderer/PuppeteerRenderer.ts`)
+
+Core rendering engine that orchestrates browser, game loading, and frame capture.
+
+**Key Responsibilities**:
+- Launch and manage headless Chromium instance
+- Navigate to game and inject manifest
+- Wait for game to enter replay mode
+- Capture frames and stream to FFmpeg
+- Handle errors and cleanup
+
+**Key Methods**:
+- `render(manifest, config)` - Main rendering entry point
+- `initBrowser(config)` - Launch Chromium with optimized flags
+- `loadGame(gameUrl, manifest)` - Navigate and inject manifest
+- `captureAndEncodeFrames(manifest, config)` - Frame capture loop
+- `launchFFmpeg(encoder, config)` - Start FFmpeg process
+
+**Browser Configuration**:
+- Single-process mode (container-friendly)
+- Disabled GPU, sandbox, extensions
+- Optimized for headless rendering
+- Chinese fonts support (Noto CJK, WQY Zenhei)
+
+### 3. CLI Tool (`src/cli/render.ts`)
+
+Command-line interface for offline video rendering.
+
+**Usage**:
+```bash
+pnpm --filter replay-server render -- \
+  --manifest ./replay.json \
+  --output ./output.mp4 \
+  --game-url http://localhost:5173 \
+  --width 1080 \
+  --height 1920 \
+  --fps 30
+```
+
+**Features**:
+- Load manifest from file or URL
+- Progress bar display
+- Configurable video settings
+- Hardware acceleration support
+
+---
+
+## API Endpoints
+
+### POST /render
+
+Start a new render job.
+
+**Request Body**:
+```json
+{
+  "manifest": {
+    "schema": "lifeRestart.replay.v1",
+    "bundleId": "game-life-restart",
+    "gameId": "test-game",
+    "timeline": [...],
+    "highlights": []
+  },
+  "config": {
+    "width": 1080,
+    "height": 1920,
+    "fps": 30,
+    "secondsPerAge": 2,
+    "gameUrl": "http://localhost:5173"  // Optional, overrides bundleId
+  }
+}
+```
+
+**Alternative** (with manifestUrl):
+```json
+{
+  "manifestUrl": "https://example.com/replay.json",
+  "config": {...}
+}
+```
+
+**Response**:
+```json
+{
+  "jobId": "5eaee18f-48a7-4193-b594-86756ac76ce2",
+  "status": "pending",
+  "message": "Render job started"
+}
+```
+
+**Status Codes**:
+- `200` - Job created successfully
+- `400` - Invalid manifest or missing required fields
+- `500` - Internal server error
+
+### GET /jobs
+
+List all render jobs.
+
+**Response**:
+```json
+{
+  "jobs": [
+    {
+      "jobId": "5eaee18f-48a7-4193-b594-86756ac76ce2",
+      "status": "completed",
+      "progress": {
+        "stage": "complete",
+        "current": 1,
+        "total": 1,
+        "message": "Completed in 45.2s"
+      },
+      "error": null,
+      "createdAt": "2024-01-01T00:00:00.000Z",
+      "completedAt": "2024-01-01T00:05:00.000Z"
+    }
+  ],
+  "count": 1
+}
+```
+
+### GET /status/:jobId
+
+Get status of a specific job.
+
+**Response**:
+```json
+{
+  "jobId": "5eaee18f-48a7-4193-b594-86756ac76ce2",
+  "status": "processing",
+  "progress": {
+    "stage": "capture",
+    "current": 150,
+    "total": 300,
+    "message": "Capturing & encoding frame 150/300 (libx264)"
+  },
+  "error": null,
+  "createdAt": "2024-01-01T00:00:00.000Z",
+  "completedAt": null
+}
+```
+
+**Status Values**:
+- `pending` - Job created, not started
+- `processing` - Currently rendering
+- `completed` - Video ready for download
+- `failed` - Error occurred (check `error` field)
+
+### GET /download/:jobId
+
+Download completed video file.
+
+**Response**: Binary MP4 file
+
+**Status Codes**:
+- `200` - Video file
+- `400` - Job not completed
+- `404` - Job or file not found
+
+### GET /bundles
+
+List available game bundles.
+
+**Response**:
+```json
+{
+  "bundles": [
+    {
+      "bundleId": "game-life-restart",
+      "path": "/app/packages/replay-server/bundles/game-life-restart",
+      "url": "http://localhost:3001/bundles/game-life-restart/",
+      "ready": true
+    }
+  ],
+  "count": 1,
+  "bundlesDir": "/app/packages/replay-server/bundles"
+}
+```
+
+### GET /health
+
+Health check endpoint.
+
+**Response**:
+```json
+{
+  "status": "ok",
+  "gameUrl": "http://localhost:5173",
+  "bundlesDir": "/app/packages/replay-server/bundles",
+  "bundleCount": 1
+}
+```
+
+### GET /debug/chrome
+
+Debug Chromium installation (useful for containerized deployments).
+
+**Response**:
+```json
+{
+  "executablePath": "/usr/bin/chromium",
+  "tests": {
+    "version": {
+      "ok": true,
+      "output": "Chromium 120.0.6099.224"
+    },
+    "dependencies": {
+      "ok": true,
+      "summary": "All libraries found"
+    },
+    "chineseFonts": {
+      "ok": true,
+      "count": 15,
+      "sample": ["Noto Sans CJK SC", "WQY Zenhei"]
+    }
+  }
+}
+```
+
+---
+
+## Game Integration Requirements
+
+For a game to work with the replay renderer, it must support replay mode:
+
+### 1. Replay Mode Detection
+
+The game should check for `?mode=replay` query parameter:
+
+```typescript
+useEffect(() => {
+  const urlParams = new URLSearchParams(window.location.search);
+  if (urlParams.get('mode') === 'replay') {
+    // Enter replay mode
+  }
+}, []);
+```
+
+### 2. Manifest Injection
+
+The renderer injects the manifest in two ways:
+
+**Method 1**: Window property
+```typescript
+if ((window as any).__REPLAY_MANIFEST__) {
+  const manifest = (window as any).__REPLAY_MANIFEST__;
+  enterReplayMode(manifest);
+}
+```
+
+**Method 2**: Custom event
+```typescript
+window.addEventListener('replay-manifest-loaded', (e: CustomEvent) => {
+  const manifest = e.detail;
+  enterReplayMode(manifest);
+});
+```
+
+### 3. Replay Control
+
+The renderer dispatches `replay-next-age` events to advance through the timeline:
+
+```typescript
+window.addEventListener('replay-next-age', () => {
+  // Advance to next age in replay
+  replayNextAge();
+});
+```
+
+### 4. Ready Indicator
+
+The game must set `data-replay-ready="true"` when ready for capture:
+
+```tsx
+<div data-replay-ready="true">
+  {/* game content */}
+</div>
+```
+
+The renderer waits for this attribute before starting frame capture.
+
+### 5. React Initialization
+
+The renderer waits for React to initialize by checking for:
+- `ion-app` element exists
+- `ion-spinner` element does not exist (loading complete)
+
+---
+
+## Deployment
+
+### Local Development
+
+```bash
+# From project root
+pnpm dev:replay-server
+
+# Or from package directory
+cd packages/replay-server
+pnpm run dev
+```
+
+**Requirements**:
+- Node.js 18+
+- FFmpeg in PATH
+- Game running at `http://localhost:5173` OR bundle in `bundles/` directory
+
+### Docker (Recommended)
+
+Multi-stage build that:
+1. Builds game bundle (`game-life-restart`)
+2. Builds replay-server
+3. Creates runtime image with both embedded
+
+**Build**:
+```bash
+# From project root
+docker build -f packages/replay-server/Dockerfile -t replay-server:latest .
+```
+
+**Run**:
+```bash
+docker run -d \
+  --name replay-server \
+  -p 3001:3001 \
+  --shm-size=2g \
+  replay-server:latest
+```
+
+**Docker Compose**:
+```bash
+cd packages/replay-server
+docker-compose -f docker-compose.local.yml up --build
+```
+
+### Aliyun Function Compute
+
+**Build for FC**:
+```bash
+docker buildx build \
+  --platform linux/amd64 \
+  --provenance=false \
+  --sbom=false \
+  -t ${ACR_REGISTRY}/${ACR_NAMESPACE}/replay-server:latest \
+  -f packages/replay-server/Dockerfile \
+  --push \
+  .
+```
+
+**Function Configuration**:
+- **Memory**: Minimum 2048 MB (2GB), recommended 4GB
+- **Timeout**: Minimum 300 seconds (5 minutes), recommended 600 seconds
+- **Instance Concurrency**: 1 (required - prevents resource contention)
+- **CPU**: Auto-scaled with memory
+
+**Environment Variables**:
+- `PORT=9000` (FC sets this automatically)
+- `BUNDLES_DIR=/app/packages/replay-server/bundles`
+- `OUTPUT_DIR=/tmp/output` (use /tmp for writable directory)
+
+**Deploy Script**:
+```bash
+cd packages/replay-server
+ACR_NAMESPACE=your-namespace ./scripts/deploy-aliyun.sh
+```
+
+---
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PORT` | `3001` | Server port |
+| `GAME_URL` | `http://localhost:5173` | Fallback game URL when no bundleId |
+| `BUNDLES_DIR` | `./bundles` | Directory containing game bundles |
+| `OUTPUT_DIR` | `./output` | Directory for rendered videos |
+| `PUPPETEER_EXECUTABLE_PATH` | `/usr/bin/chromium` | Chromium executable path |
+
+### Render Configuration
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `width` | `720` | Video width (pixels) |
+| `height` | `1280` | Video height (pixels) |
+| `fps` | `16` | Frames per second |
+| `secondsPerAge` | `1.2` | Seconds to display each age |
+| `useHardwareAcceleration` | `false` | Use h264_nvenc (GPU) instead of libx264 (CPU) |
+
+### Video Output
+
+- **Format**: MP4 (H.264)
+- **Resolution**: 720x1280 (portrait, for mobile/Feed)
+- **FPS**: 16 (configurable)
+- **Duration**: ~1.2s per age (configurable)
+- **Encoder**: libx264 (CPU) by default, h264_nvenc (GPU) optional
+
+---
+
+## Building Game Bundles
+
+Before deploying, build game bundles that will be hosted by replay-server.
+
+### Using Build Script
+
+```bash
+cd packages/replay-server
+./scripts/build-bundle.sh game-life-restart
+```
+
+This will:
+1. Build the game from `repo/game-life-restart`
+2. Copy `dist/` to `bundles/game-life-restart/`
+
+### Manual Build
+
+```bash
+# Build the game
+cd repo/game-life-restart
+pnpm install
+VITE_BASE_PATH=/bundles/game-life-restart/ pnpm build
+
+# Copy to bundles
+mkdir -p packages/replay-server/bundles/game-life-restart
+cp -r dist/* packages/replay-server/bundles/game-life-restart/
+```
+
+### Docker (Automatic)
+
+The Dockerfile includes a multi-stage build that automatically builds and embeds the game bundle. No manual steps required.
+
+---
+
+## File Structure
+
+```
+packages/replay-server/
+├── .cursor/
+│   ├── project.mdc      # This file
+│   └── dev.mdc          # Development rules
+├── bundles/              # Game bundles (gitignored)
+│   └── game-life-restart/
+├── samples/             # Sample replay manifests
+│   └── life-replay-*.json
+├── scripts/
+│   ├── build-bundle.sh  # Build game bundle
+│   └── deploy-aliyun.sh # Deploy to ACR
+├── src/
+│   ├── cli/
+│   │   └── render.ts    # CLI tool
+│   ├── renderer/
+│   │   ├── index.ts
+│   │   └── PuppeteerRenderer.ts  # Core renderer
+│   ├── server/
+│   │   └── index.ts     # Express HTTP server
+│   └── index.ts         # Package exports
+├── Dockerfile           # Multi-stage Docker build
+├── docker-compose.local.yml
+├── package.json
+├── tsconfig.json
+└── README.md
+```
+
+---
+
+## Key Files Reference
+
+### Server
+- `src/server/index.ts` - Express HTTP API server (407 lines)
+- Handles render jobs, bundle serving, job status
+
+### Renderer
+- `src/renderer/PuppeteerRenderer.ts` - Core rendering engine (513 lines)
+- Browser management, game loading, frame capture, FFmpeg integration
+
+### CLI
+- `src/cli/render.ts` - Command-line tool (244 lines)
+- Manifest loading, progress display, renderer invocation
+
+### Configuration
+- `package.json` - Dependencies and scripts
+- `tsconfig.json` - TypeScript configuration
+- `Dockerfile` - Multi-stage build for production
+
+---
+
+## Dependencies
+
+### Core Dependencies
+- `express` - HTTP server framework
+- `puppeteer` - Headless browser automation
+- `fluent-ffmpeg` - FFmpeg wrapper (note: actual FFmpeg binary required)
+- `cors` - CORS middleware
+- `uuid` - Job ID generation
+
+### Type Dependencies
+- `@types/express`
+- `@types/fluent-ffmpeg`
+- `@types/cors`
+- `@types/uuid`
+- `@types/node`
+
+### Shared Package
+- `@agent-foundry/replay` - Replay manifest type definitions
+
+---
+
+## Related Documentation
+
+- [README.md](../README.md) - Main documentation with usage examples
+- [../replay-shared/README.md](../replay-shared/README.md) - Replay manifest schema
+- [../../repo/game-life-restart/doc/replay/](../../repo/game-life-restart/doc/replay/) - Game replay integration guide

package/.dockerignore

@@ -0,0 +1,11 @@
+node_modules
+dist
+output
+temp
+*.log
+.git
+.gitignore
+README.md
+samples
+.DS_Store
+*.md