mes-engine 0.0.2 → 1.0.0

package/docs/README.md

@@ -1,170 +1,173 @@
-# Video Processing Framework Documentation
-
-Comprehensive documentation for the mes-engine video processing framework.
-
-## Table of Contents
-
-1. [Getting Started](#getting-started)
-2. [Core Concepts](#core-concepts)
-3. [Components](#components)
-4. [Advanced Usage](#advanced-usage)
-5. [API Reference](#api-reference)
-
-## Getting Started
-
-### Installation
-```bash
-npm install mes-engine
-```
-
-### Engine Requirements
-
-#### FFmpeg Engine
-```bash
-npm install ffmpeg-static
-```
-
-#### GStreamer Engine
-Requires GStreamer to be installed on your system:
-- Ubuntu/Debian: `apt-get install gstreamer1.0-tools`
-- MacOS: `brew install gstreamer`
-- Windows: Download from GStreamer website
-
-### Basic Configuration
-
-```typescript
-import { VideoProcessor, FFmpegEngine, FileSystemStorage } from 'mes-engine';
-
-const config = {
-  chunkSize: 10,          // seconds
-  cacheDir: './cache',
-  maxCacheSize: 1000,     // MB
-  defaultQualities: [
-    { height: 1080, bitrate: '4000k' },
-    { height: 720, bitrate: '2500k' },
-    { height: 480, bitrate: '1000k' }
-  ]
-};
-
-const processor = new VideoProcessor({
-  engine: new FFmpegEngine(),
-  storage: new FileSystemStorage(),
-  config
-});
-```
-
-## Core Concepts
-
-### Video Processing Flow
-
-1. **Input**: Raw video file
-2. **Chunking**: Video split into segments
-3. **Transcoding**: Multiple quality versions
-4. **Storage**: Chunks saved to storage provider
-5. **Streaming**: Adaptive delivery
-
-### Events System
-
-```typescript
-processor.on(VideoEvent.CHUNK_PROCESSED, (data) => {
-  console.log(`Chunk ${data.chunkNumber} processed at ${data.quality}p`);
-});
-
-processor.on(VideoEvent.PROCESSING_COMPLETE, (manifest) => {
-  console.log('Processing complete:', manifest);
-});
-```
-
-## Components
-
-### Processing Engines
-
-- **FFmpegEngine**: Standard video processing
-- **GStreamerEngine**: High-performance alternative
-- **Custom Engines**: Extend `VideoEngine` class
-
-```typescript
-class CustomEngine extends VideoEngine {
-  async processChunk(
-    inputPath: string,
-    outputPath: string,
-    startTime: number,
-    quality: QualityLevel
-  ): Promise<void> {
-    // Implementation
-  }
-
-  async getDuration(inputPath: string): Promise<number> {
-    // Implementation
-  }
-}
-```
-
-### Storage Providers
-
-- **FileSystemStorage**: Local file system storage
-- **Custom Storage**: Implement `StorageProvider`
-
-```typescript
-class CustomStorage extends StorageProvider {
-  async saveChunk(chunkPath: string, data: Buffer): Promise<void> {
-    // Implementation
-  }
-
-  async getChunk(chunkPath: string): Promise<Buffer> {
-    // Implementation
-  }
-
-  async deleteChunk(chunkPath: string): Promise<void> {
-    // Implementation
-  }
-}
-```
-
-### Caching Strategies
-
-- **InternalCache**: Memory-based LRU cache
-- **ExternalCache**: Remote cache service
-- **Custom Cache**: Extend `CacheStrategy`
-
-## Advanced Usage
-
-### Custom Quality Levels
-
-```typescript
-const customQualities = [
-  { height: 2160, bitrate: '8000k' }, // 4K
-  { height: 1440, bitrate: '6000k' }, // 2K
-  { height: 1080, bitrate: '4000k' }, // Full HD
-];
-
-const manifest = await processor.processVideo('input.mp4', {
-  qualities: customQualities
-});
-```
-
-### Bandwidth-Aware Streaming
-
-```typescript
-import { BandwidthDetector } from 'mes-engine';
-
-const detector = new BandwidthDetector();
-detector.addSample(bytesTransferred, durationMs);
-const estimatedBandwidth = detector.getEstimatedBandwidth();
-```
-
-## API Reference
-
-See [API.md](./API.md) for detailed API documentation.
-
-## Testing
-
-```bash
-npm run test        # Run all tests
-npm run test:watch  # Watch mode
-npm run test:coverage # Coverage report
-```
-
-## Contributing
-
+# Video Processing Framework Documentation
+
+Comprehensive documentation for the mes-engine video processing framework.
+
+## Table of Contents
+
+1. [Getting Started](#getting-started)
+2. [Core Concepts](#core-concepts)
+3. [Components](#components)
+4. [HLS & Adaptive Streaming](./HLS.md)
+5. [Storage Providers](./storage.md)
+6. [Caching Strategies](./caching.md)
+7. [Advanced Usage](#advanced-usage)
+8. [API Reference](./API.md)
+
+## Getting Started
+
+### Installation
+```bash
+npm install mes-engine
+```
+
+### Engine Requirements
+
+#### FFmpeg Engine
+```bash
+npm install ffmpeg-static
+```
+
+#### GStreamer Engine
+Requires GStreamer to be installed on your system:
+- Ubuntu/Debian: `apt-get install gstreamer1.0-tools`
+- MacOS: `brew install gstreamer`
+- Windows: Download from GStreamer website
+
+### Basic Configuration
+
+```typescript
+import { VideoProcessor, FFmpegEngine, FileSystemStorage } from 'mes-engine';
+
+const config = {
+  chunkSize: 10,          // seconds
+  cacheDir: './cache',
+  maxCacheSize: 1024 * 1024 * 1024, // 1GB
+  defaultQualities: [
+    { height: 1080, bitrate: '4000k' },
+    { height: 720, bitrate: '2500k' },
+    { height: 480, bitrate: '1000k' }
+  ]
+};
+
+const processor = new VideoProcessor(
+  new FFmpegEngine(),
+  new FileSystemStorage(),
+  config
+);
+```
+
+## Core Concepts
+
+### Video Processing Flow
+
+1. **Input**: Raw video file
+2. **Chunking**: Video split into segments
+3. **Transcoding**: Multiple quality versions
+4. **Storage**: Chunks saved to storage provider
+5. **Streaming**: Adaptive delivery
+
+### Events System
+
+```typescript
+processor.on(VideoEvent.CHUNK_PROCESSED, (data) => {
+  console.log(`Chunk ${data.chunkNumber} processed at ${data.quality}p`);
+});
+
+processor.on(VideoEvent.PROCESSING_COMPLETE, (manifest) => {
+  console.log('Processing complete:', manifest);
+});
+```
+
+## Components
+
+### Processing Engines
+
+- **FFmpegEngine**: Standard video processing
+- **GStreamerEngine**: High-performance alternative
+- **Custom Engines**: Extend `VideoEngine` class
+
+```typescript
+class CustomEngine extends VideoEngine {
+  async processChunk(
+    inputPath: string,
+    outputPath: string,
+    startTime: number,
+    quality: QualityLevel
+  ): Promise<void> {
+    // Implementation
+  }
+
+  async getDuration(inputPath: string): Promise<number> {
+    // Implementation
+  }
+}
+```
+
+### Storage Providers
+
+- **FileSystemStorage**: Local file system storage
+- **Custom Storage**: Implement `StorageProvider`
+
+```typescript
+class CustomStorage extends StorageProvider {
+  async saveChunk(chunkPath: string, data: Buffer): Promise<void> {
+    // Implementation
+  }
+
+  async getChunk(chunkPath: string): Promise<Buffer> {
+    // Implementation
+  }
+
+  async deleteChunk(chunkPath: string): Promise<void> {
+    // Implementation
+  }
+}
+```
+
+### Caching Strategies
+
+- **InternalCache**: Memory-based LRU cache
+- **ExternalCache**: Remote cache service
+- **Custom Cache**: Extend `CacheStrategy`
+
+## Advanced Usage
+
+### Custom Quality Levels
+
+```typescript
+const customQualities = [
+  { height: 2160, bitrate: '8000k' }, // 4K
+  { height: 1440, bitrate: '6000k' }, // 2K
+  { height: 1080, bitrate: '4000k' }, // Full HD
+];
+
+const manifest = await processor.processVideo('input.mp4', {
+  qualities: customQualities
+});
+```
+
+### Bandwidth-Aware Streaming
+
+```typescript
+import { BandwidthDetector } from 'mes-engine';
+
+const detector = new BandwidthDetector();
+detector.addSample(bytesTransferred, durationMs);
+const estimatedBandwidth = detector.getEstimatedBandwidth();
+```
+
+## API Reference
+
+See [API.md](./API.md) for detailed API documentation.
+
+## Testing
+
+```bash
+npm run test        # Run all tests
+npm run test:watch  # Watch mode
+npm run test:coverage # Coverage report
+```
+
+## Contributing
+
 See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.

package/docs/caching.md

@@ -0,0 +1,62 @@
+# Caching Strategies
+
+Intelligent caching is built into `mes-engine` to reduce storage latency and improve streaming performance.
+
+## Cache Options
+
+The `CacheOptions` interface defines how the cache behaves:
+
+```typescript
+interface CacheOptions {
+    maxSize: number;           // Maximum size of the cache in bytes
+    ttl: number;               // Time To Live in seconds
+    preloadNextChunk: boolean; // Whether to automatically fetch the next chunk
+}
+```
+
+## Supported Strategies
+
+### `InternalCache`
+A memory-based cache using a Least Recently Used (LRU) eviction policy. It is highly effective for hot content.
+
+```typescript
+import { InternalCache, FileSystemStorage } from 'mes-engine';
+
+const storage = new FileSystemStorage();
+const cache = new InternalCache({
+    maxSize: 1024 * 1024 * 500, // 500MB
+    ttl: 3600,
+    preloadNextChunk: true
+}, storage);
+```
+
+## Key Features
+
+### Next-Chunk Preloading
+When `preloadNextChunk` is enabled, the `InternalCache` will automatically attempt to fetch the next sequential chunk (e.g., `chunk_1` if `chunk_0` was requested) and store it in memory. This significantly reduces buffer times for sequential playback.
+
+## Custom Caching
+
+You can implement your own strategy (e.g., using Redis) by extending the `CacheStrategy` class:
+
+```typescript
+import { CacheStrategy } from 'mes-engine';
+
+class RedisCache extends CacheStrategy {
+    async set(key: string, value: Buffer): Promise<void> {
+        // Redis SETEX
+    }
+
+    async get(key: string): Promise<Buffer | null> {
+        // Redis GET
+    }
+
+    async preload(key: string): Promise<void> {
+        // Custom preloading logic
+    }
+
+    async clear(): Promise<void> {
+        // Redis FLUSHDB
+    }
+}
+```

package/docs/engines.md

@@ -1,58 +1,62 @@
-# Engines Support
-
-## FFmpegEngine
-The `FFmpegEngine` uses `ffmpeg` for video processing. You will need to install an FFmpeg binary to use this engine.
-
-### Installation of FFmpeg
-To use the `FFmpegEngine`, you must install FFmpeg or use `ffmpeg-static` to include the FFmpeg binary. You can install `ffmpeg-static` by running:
-
-```bash
-npm install ffmpeg-static
-```
-
-Alternatively, you can use any other FFmpeg binary that is compatible with your system.
-
-### Example Usage:
-
-```typescript
-import { FFmpegEngine } from 'mes-engine';
-
-// Create an FFmpeg engine instance
-const engine = new FFmpegEngine();
-
-// Use it to process video chunks
-engine.processChunk('input.mp4', 'output.mp4', 0, { height: 720, bitrate: '2000k' })
-  .then(() => {
-    console.log('Chunk processed successfully');
-  })
-  .catch(error => {
-    console.error('Error processing chunk:', error);
-  });
-```
-
-## Custom Engines
-You can implement your own video processing engine by extending the `VideoEngine` class. Here’s an example of how to create a custom engine:
-
-### Example Custom Engine:
-
-```typescript
-import { VideoEngine } from 'mes-engine';
-
-class CustomVideoEngine extends VideoEngine {
-  async processChunk(inputPath: string, outputPath: string, startTime: number, quality: any): Promise<void> {
-    // Implement custom video processing logic here
-  }
-
-  async getDuration(inputPath: string): Promise<number> {
-    // Implement custom duration calculation logic here
-    return 120; // Example
-  }
-}
-```
-
-For more details on implementing custom engines, see the [VideoEngine class](../src/core/VideoEngine.ts) in the codebase.
-
-## Other Supported Engines
-You can extend this framework with any other video processing engine that implements the `VideoEngine` interface.
-
-For more details, check the [source code](https://github.com/Bum-Ho12/mes-engine) and adapt it as needed for your requirements.
+# Engines Support
+
+## FFmpegEngine
+The `FFmpegEngine` uses `ffmpeg` for video processing. You will need to install an FFmpeg binary to use this engine.
+
+### Installation of FFmpeg
+To use the `FFmpegEngine`, you must install FFmpeg or use `ffmpeg-static` to include the FFmpeg binary. You can install `ffmpeg-static` by running:
+
+```bash
+npm install ffmpeg-static
+```
+
+Alternatively, you can use any other FFmpeg binary that is compatible with your system.
+
+### Example Usage:
+
+```typescript
+import { FFmpegEngine } from 'mes-engine';
+
+// Create an FFmpeg engine instance
+const engine = new FFmpegEngine();
+
+// Use it to process video chunks
+engine.processChunk('input.mp4', 'output.mp4', 0, { height: 720, bitrate: '2000k' })
+  .then(() => {
+    console.log('Chunk processed successfully');
+  })
+  .catch(error => {
+    console.error('Error processing chunk:', error);
+  });
+```
+
+## Custom Engines
+You can implement your own video processing engine by extending the `VideoEngine` class. Here’s an example of how to create a custom engine:
+
+### Example Custom Engine:
+
+```typescript
+import { VideoEngine } from 'mes-engine';
+
+class CustomVideoEngine extends VideoEngine {
+  async processChunk(inputPath: string, outputPath: string, startTime: number, quality: any): Promise<void> {
+    // Implement custom video processing logic here
+  }
+
+  async getDuration(inputPath: string): Promise<number> {
+    // Implement custom duration calculation logic here
+    return 120; // Example
+  }
+
+  async extractScreenshot(inputPath: string, outputPath: string, time: number): Promise<void> {
+    // Implement custom screenshot extraction logic here
+  }
+}
+```
+
+For more details on implementing custom engines, see the [VideoEngine class](../src/core/VideoEngine.ts) in the codebase.
+
+## Other Supported Engines
+You can extend this framework with any other video processing engine that implements the `VideoEngine` interface.
+
+For more details, check the [source code](https://github.com/Bum-Ho12/mes-engine) and adapt it as needed for your requirements.

package/docs/storage.md

@@ -0,0 +1,57 @@
+# Storage Providers
+
+`mes-engine` uses an abstract storage layer to remain independent of where your video chunks are actually stored.
+
+## Base Class: `StorageProvider`
+
+All storage providers must extend the `StorageProvider` abstract class and implement its three core methods:
+
+```typescript
+export abstract class StorageProvider {
+    abstract saveChunk(chunkPath: string, data: Buffer): Promise<void>;
+    abstract getChunk(chunkPath: string): Promise<Buffer>;
+    abstract deleteChunk(chunkPath: string): Promise<void>;
+}
+```
+
+## Built-in Providers
+
+### `FileSystemStorage`
+The default provider for local development and on-premise deployments. It saves chunks directly to the server's disk.
+
+```typescript
+import { FileSystemStorage } from 'mes-engine';
+
+const storage = new FileSystemStorage();
+```
+
+## Creating a Custom Provider
+
+You can easily create a provider for S3, Azure Blob Storage, or any other service by extending `StorageProvider`.
+
+### Example: S3 Storage Provider
+
+```typescript
+import { StorageProvider } from 'mes-engine';
+import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
+
+class S3Storage extends StorageProvider {
+    private client = new S3Client({});
+
+    async saveChunk(chunkPath: string, data: Buffer): Promise<void> {
+        await this.client.send(new PutObjectCommand({
+            Bucket: 'my-videos',
+            Key: chunkPath,
+            Body: data
+        }));
+    }
+
+    async getChunk(chunkPath: string): Promise<Buffer> {
+        // Implement S3 GetObject and return Buffer
+    }
+
+    async deleteChunk(chunkPath: string): Promise<void> {
+        // Implement S3 DeleteObject
+    }
+}
+```

package/examples/full-demo/backend/.env

@@ -0,0 +1,6 @@
+PORT=3001
+UPLOAD_DIR=./uploads
+PROCESSED_DIR=./processed
+MAX_FILE_SIZE=524288000
+CACHE_SIZE=104857600
+CACHE_TTL=3600