@kadi.build/file-sharing 1.0.0

package/README.md

@@ -0,0 +1,477 @@
+# @kadi.build/file-sharing
+
+File sharing service with tunneling, authentication, and a local S3-compatible interface.  
+Integrates [`@kadi.build/file-manager`](https://www.npmjs.com/package/@kadi.build/file-manager) and [`@kadi.build/tunnel-services`](https://www.npmjs.com/package/@kadi.build/tunnel-services) into one turnkey solution.
+
+## Features
+
+- **HTTP File Server** — Static file serving with directory listing, range requests, CORS, and multi-scheme authentication
+- **Local S3-Compatible API** — Emulates AWS S3 endpoints locally so you can use `@aws-sdk/client-s3` against your local filesystem
+- **Tunnel Integration** — Expose your local server publicly via KĀDI (default), ngrok, serveo, localtunnel, or pinggy
+- **Authentication** — Basic Auth, Bearer Token, and API Key (header / query param) for HTTP; AWS SigV4 / SigV2 / Bearer for S3
+- **Secrets Management** — Automatic `.env` loading (walks up parent directories for monorepo support) + environment variables + explicit config
+- **Download Monitoring** — Track active downloads, progress, speed, and completion statistics
+- **Graceful Shutdown** — Priority-ordered shutdown with timeout and force-kill
+- **Webhook Notifications** — Event-driven notifications to external endpoints
+- **Monitoring Dashboard** — Console-based real-time dashboard
+
+## Installation
+
+```bash
+npm install @kadi.build/file-sharing
+```
+
+## Quick Start
+
+### Basic File Sharing
+
+```javascript
+import { FileSharingServer } from '@kadi.build/file-sharing';
+
+const server = new FileSharingServer({
+  staticDir: '/path/to/files',
+  port: 3000
+});
+
+await server.start();
+console.log(`Serving files at ${server.getInfo().localUrl}`);
+await server.stop();
+```
+
+### With Public Tunnel (KĀDI)
+
+```javascript
+const server = new FileSharingServer({
+  staticDir: '/path/to/files',
+  port: 3000,
+  tunnel: {
+    enabled: true
+    // KĀDI is the default — reads token from KADI_TUNNEL_TOKEN env var or .env
+  }
+});
+
+await server.start();
+console.log(`Public URL: ${server.tunnelUrl}`);
+```
+
+### With Authentication
+
+```javascript
+const server = new FileSharingServer({
+  staticDir: '/path/to/files',
+  port: 3000,
+  auth: { apiKey: 'my-secret-key' },  // or { username: 'admin', password: 'secret' }
+  tunnel: { enabled: true }
+});
+
+await server.start();
+// Clients must send X-API-Key header, Bearer token, or ?apiKey= query param
+```
+
+### With Local S3 API
+
+```javascript
+import { FileSharingServer } from '@kadi.build/file-sharing';
+import { S3Client, ListObjectsV2Command, PutObjectCommand } from '@aws-sdk/client-s3';
+
+const server = new FileSharingServer({
+  staticDir: '/path/to/files',
+  port: 3000,
+  enableS3: true,
+  s3Port: 9000
+});
+
+await server.start();
+
+// Use the standard AWS SDK against your LOCAL filesystem
+const s3 = new S3Client({
+  endpoint: 'http://localhost:9000',
+  region: 'us-east-1',
+  credentials: { accessKeyId: 'minioadmin', secretAccessKey: 'minioadmin' },
+  forcePathStyle: true
+});
+
+const response = await s3.send(new ListObjectsV2Command({ Bucket: 'local' }));
+console.log(response.Contents);
+
+await s3.send(new PutObjectCommand({
+  Bucket: 'local',
+  Key: 'uploaded.txt',
+  Body: 'Hello from S3!'
+}));
+```
+
+### Quick Share (One-Liner)
+
+```javascript
+import { createQuickShare } from '@kadi.build/file-sharing';
+
+const { server, localUrl, publicUrl } = await createQuickShare('./my-files', {
+  tunnel: true,
+  auth: { apiKey: 'share-key-123' }
+});
+
+console.log(`Share this link: ${publicUrl}`);
+```
+
+---
+
+## Secrets & Authentication
+
+### How Secrets Are Loaded
+
+Secrets are resolved in **priority order** (first match wins):
+
+1. **Explicit constructor config** — values you pass directly
+2. **Environment variables** — `process.env.*`
+3. **`.env` file** — automatically found by walking up parent directories from `process.cwd()`
+
+The `.env` loader supports monorepo layouts: if your `.env` is at the workspace root and the package runs from `packages/file-sharing/`, it will find it automatically.
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| **Tunnel — KĀDI** | | |
+| `KADI_TUNNEL_TOKEN` | KĀDI authentication token | *(required for KĀDI)* |
+| `KADI_TUNNEL_SERVER` | KĀDI broker address | `broker.kadi.build` |
+| `KADI_TUNNEL_DOMAIN` | Tunnel domain | `tunnel.kadi.build` |
+| `KADI_TUNNEL_PORT` | KĀDI frps server port | `7000` |
+| `KADI_TUNNEL_SSH_PORT` | KĀDI SSH gateway port | `2200` |
+| `KADI_TUNNEL_MODE` | Connection mode: `ssh`, `frpc`, or `auto` | `auto` |
+| `KADI_AGENT_ID` | Agent identifier for proxy naming | `kadi` |
+| **Tunnel — Ngrok** | | |
+| `NGROK_AUTHTOKEN` | Ngrok auth token (also accepts `NGROK_AUTH_TOKEN`) | — |
+| **HTTP Authentication** | | |
+| `KADI_AUTH_API_KEY` | API key for Bearer / X-API-Key auth | — |
+| `KADI_AUTH_USERNAME` | HTTP Basic auth username | — |
+| `KADI_AUTH_PASSWORD` | HTTP Basic auth password | — |
+| **S3 Authentication** | | |
+| `KADI_S3_ACCESS_KEY` | S3 access key ID | `minioadmin` |
+| `KADI_S3_SECRET_KEY` | S3 secret access key | `minioadmin` |
+
+### `.env` File Example
+
+```env
+# Tunnel credentials
+KADI_TUNNEL_TOKEN=your-kadi-token-here
+KADI_TUNNEL_SERVER=broker.kadi.build
+KADI_TUNNEL_DOMAIN=tunnel.kadi.build
+KADI_TUNNEL_PORT=7000
+KADI_TUNNEL_SSH_PORT=2200
+KADI_TUNNEL_MODE=ssh
+KADI_AGENT_ID=my-agent
+
+# Optional: Ngrok (fallback)
+NGROK_AUTHTOKEN=your-ngrok-token
+
+# HTTP auth (optional — protects file downloads)
+KADI_AUTH_API_KEY=my-secret-api-key
+
+# S3 credentials (optional — default minioadmin/minioadmin)
+KADI_S3_ACCESS_KEY=my-access-key
+KADI_S3_SECRET_KEY=my-secret-key
+```
+
+### HTTP Authentication Schemes
+
+When `auth` is configured (via config, env vars, or `.env`), the HTTP server supports three authentication schemes:
+
+| Scheme | How to Authenticate |
+|--------|---------------------|
+| **Basic Auth** | `Authorization: Basic <base64(user:pass)>` |
+| **Bearer Token** | `Authorization: Bearer <apiKey>` |
+| **API Key** | `X-API-Key: <apiKey>` header or `?apiKey=<key>` query param |
+
+If `auth.apiKey` is set, all three token-based methods are accepted.  
+If `auth.username` + `auth.password` are set, only Basic Auth is accepted.
+
+### S3 Authentication
+
+When custom S3 credentials are set (anything other than the default `minioadmin`/`minioadmin`), the S3 server validates requests:
+
+| Method | How It Works |
+|--------|-------------|
+| **AWS SigV4** | Standard `Authorization: AWS4-HMAC-SHA256 Credential=<accessKeyId>/...` |
+| **AWS SigV2** | Legacy `Authorization: AWS <accessKeyId>:signature` |
+| **Bearer** | Convenience `Authorization: Bearer <accessKeyId>` |
+| **Pre-signed URL** | `?X-Amz-Credential=<accessKeyId>/...` query param |
+
+> With default credentials (`minioadmin`/`minioadmin`), auth is skipped for backward compatibility unless you set `enforceAuth: true` in the S3 config.
+
+---
+
+## API Reference
+
+### FileSharingServer
+
+Main orchestrating class.
+
+```javascript
+const server = new FileSharingServer({
+  // File serving
+  staticDir: process.cwd(),        // Directory to serve
+  port: 3000,                      // HTTP port
+  host: '0.0.0.0',                 // Bind address
+  enableDirectoryListing: true,    // Show directory listing
+  cors: true,                      // Enable CORS
+
+  // Authentication (or use env vars / .env)
+  auth: null,                      // { apiKey: 'key' } or { username: 'u', password: 'p' }
+
+  // S3 API
+  enableS3: false,                 // Enable S3-compatible API
+  s3Port: 9000,                    // S3 API port
+  s3Config: {                      // Passed to S3Server
+    accessKeyId: 'minioadmin',
+    secretAccessKey: 'minioadmin',
+    bucketName: 'local',
+    region: 'us-east-1',
+    enforceAuth: false             // Force auth even with default creds
+  },
+
+  // Tunnel
+  tunnel: {
+    enabled: false,                // Start tunnel on server.start()
+    service: 'kadi',               // 'kadi' | 'ngrok' | 'serveo' | 'localtunnel' | 'pinggy'
+    autoFallback: true,            // Fall back to other services on failure
+    autoReconnect: true,           // Auto-reconnect on tunnel drop
+    reconnectDelay: 5000,          // ms between reconnect attempts
+    // KĀDI-specific (or use env vars / .env)
+    kadiToken: undefined,
+    kadiServer: undefined,
+    kadiDomain: undefined,
+    kadiPort: undefined,           // frps server port (NOT the local port)
+    kadiSshPort: undefined,
+    kadiMode: undefined,           // 'ssh' | 'frpc' | 'auto'
+    kadiAgentId: undefined,
+    // Ngrok-specific
+    ngrokAuthToken: undefined,
+    // Advanced
+    managerOptions: {}             // Extra options passed to TunnelManager
+  },
+
+  // Shutdown
+  shutdown: {
+    gracefulTimeout: 30000,
+    finishActiveDownloads: true,
+    forceKillTimeout: 60000
+  },
+
+  // Monitoring
+  monitoring: {
+    enabled: true,
+    dashboard: false,              // Enable console dashboard
+    webhooks: []                   // ['https://hooks.example.com/events']
+  }
+});
+```
+
+#### Methods
+
+| Method | Returns | Description |
+|--------|---------|-------------|
+| `start()` | `Promise<ServerInfo>` | Start HTTP server (+ S3 and tunnel if enabled) |
+| `stop()` | `Promise<void>` | Gracefully stop all servers and tunnels |
+| `enableTunnel(options?)` | `Promise<TunnelInfo>` | Enable public tunnel after start |
+| `disableTunnel()` | `Promise<void>` | Close active tunnel |
+| `enableS3(options?)` | `Promise<{endpoint, port}>` | Enable S3 API dynamically |
+| `disableS3()` | `Promise<void>` | Disable S3 API |
+| `getInfo()` | `ServerInfo` | Get server info, URLs, and tunnel status |
+| `getStats()` | `DownloadStats` | Get download statistics |
+| `listFiles(subPath?)` | `Promise<FileEntry[]>` | List files in served directory |
+| `addWebhook(url, events?)` | `void` | Register a webhook endpoint |
+| `removeWebhook(url)` | `void` | Remove a webhook |
+
+#### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `isRunning` | `boolean` | Whether the server is running |
+| `port` | `number` | Current HTTP port |
+| `tunnelUrl` | `string \| null` | Current public tunnel URL |
+| `staticDir` | `string` | Directory being served |
+| `tunnel` | `object \| null` | Active tunnel info |
+| `tunnelManager` | `TunnelManager` | Underlying tunnel manager |
+
+#### Events
+
+| Event | Payload | When |
+|-------|---------|------|
+| `started` | `ServerInfo` | Server fully started |
+| `stopping` | — | Shutdown initiated |
+| `stopped` | — | Server fully stopped |
+| `download:start` | `{id, file, ...}` | File download begins |
+| `download:complete` | `{id, file, duration, ...}` | Download finished |
+| `download:error` | `{id, error}` | Download failed |
+| `upload:complete` | `{file, size}` | S3 upload finished |
+| `s3:started` | `{port, endpoint}` | S3 server started |
+| `s3:get` | `{bucket, key}` | S3 GetObject |
+| `s3:put` | `{bucket, key}` | S3 PutObject |
+| `s3:delete` | `{bucket, key}` | S3 DeleteObject |
+| `tunnel:created` | `TunnelInfo` | Tunnel established |
+| `tunnel:closed` | — | Tunnel shut down |
+| `tunnel:error` | `Error` | Tunnel failed (non-fatal) |
+| `http:started` | `{port, host}` | HTTP server listening |
+| `http:error` | `Error` | HTTP error |
+
+### HttpServerProvider
+
+Low-level HTTP file server with authentication.
+
+```javascript
+import { HttpServerProvider } from '@kadi.build/file-sharing/http';
+
+const httpServer = new HttpServerProvider({
+  port: 3000,
+  staticDir: './files',
+  enableDirectoryListing: true,
+  cors: true,
+  auth: { apiKey: 'my-key' }       // or { username: 'u', password: 'p' }
+});
+
+await httpServer.start();
+```
+
+### S3Server
+
+Local S3-compatible API server. Files are stored **on disk**, not on AWS.
+
+```javascript
+import { S3Server } from '@kadi.build/file-sharing/s3';
+
+const s3 = new S3Server({
+  port: 9000,
+  rootDir: './data',
+  bucketName: 'local',
+  region: 'us-east-1',
+  accessKeyId: 'my-key',           // default: 'minioadmin'
+  secretAccessKey: 'my-secret',    // default: 'minioadmin'
+  enforceAuth: true                // validate even with default creds
+});
+
+await s3.start();
+```
+
+**Supported S3 Operations:**
+
+- `ListBuckets`, `CreateBucket`, `DeleteBucket`, `HeadBucket`
+- `ListObjects`, `ListObjectsV2`
+- `GetObject` (with range requests)
+- `PutObject`
+- `DeleteObject`
+- `HeadObject`
+- `CreateMultipartUpload`, `UploadPart`, `CompleteMultipartUpload`
+
+### createQuickShare
+
+One-liner to share a directory with optional tunnel and auth.
+
+```javascript
+import { createQuickShare } from '@kadi.build/file-sharing';
+
+const { server, localUrl, publicUrl } = await createQuickShare('./my-files', {
+  port: 3000,                       // HTTP port (default: 3000)
+  tunnel: true,                     // Enable tunnel (default: false)
+  tunnelService: 'kadi',            // Tunnel service (default: 'kadi')
+  kadiToken: 'token',               // Or set KADI_TUNNEL_TOKEN env
+  ngrokAuthToken: 'token',          // Or set NGROK_AUTHTOKEN env
+  auth: { apiKey: 'share-key' },    // Protect downloads
+  tunnelOptions: {}                 // Extra TunnelManager options
+});
+
+console.log(`Local:  ${localUrl}`);
+console.log(`Public: ${publicUrl}`);
+```
+
+### DownloadMonitor
+
+Track download progress and statistics.
+
+```javascript
+import { DownloadMonitor } from '@kadi.build/file-sharing';
+
+const monitor = new DownloadMonitor();
+
+monitor.on('download:start', (dl) => console.log(`Started: ${dl.file}`));
+monitor.on('download:progress', (dl) => console.log(`${dl.progress.toFixed(1)}%`));
+monitor.on('download:complete', (dl) => console.log(`Done in ${dl.duration}ms`));
+
+monitor.startDownload('id', { file: 'test.txt', totalSize: 1000 });
+monitor.updateProgress('id', 500);
+monitor.completeDownload('id');
+
+console.log(monitor.getStats());
+// { activeCount, completedCount, totalBytes, peakConcurrent, ... }
+```
+
+### ShutdownManager
+
+Graceful shutdown with priority-ordered callbacks.
+
+```javascript
+import { ShutdownManager } from '@kadi.build/file-sharing';
+
+const sm = new ShutdownManager({ gracefulTimeout: 10000 });
+
+sm.register(async () => console.log('Close DB'), 1);     // priority 1 = first
+sm.register(async () => console.log('Close HTTP'), 10);   // priority 10 = later
+
+await sm.shutdown();
+```
+
+---
+
+## Sub-path Exports
+
+```javascript
+// Main export — all classes and utilities
+import { FileSharingServer, createQuickShare } from '@kadi.build/file-sharing';
+
+// S3 server only
+import { S3Server } from '@kadi.build/file-sharing/s3';
+
+// HTTP server only
+import { HttpServerProvider } from '@kadi.build/file-sharing/http';
+
+// Re-exports from dependencies (convenience)
+import { FileManager, createFileManager, TunnelManager } from '@kadi.build/file-sharing';
+```
+
+---
+
+## Tunnel Services
+
+KĀDI is the default and recommended tunnel service. It supports two connection modes:
+
+| Mode | How It Works | Requirements |
+|------|-------------|--------------|
+| **SSH** (default) | `ssh -R` through KĀDI's SSH gateway — zero-dependency | SSH client in `$PATH` |
+| **frpc** | Uses the frp client binary for higher reliability | `frpc` binary in `$PATH` |
+| **auto** | Prefers `frpc` if available, falls back to SSH | — |
+
+Set `KADI_TUNNEL_MODE=ssh` (or `frpc`) in your `.env` to force a specific mode.
+
+If KĀDI credentials are not provided, the tunnel will automatically fall back to free services (serveo → localtunnel → pinggy → localhost.run) unless `autoFallback: false` is set.
+
+---
+
+## Dependencies
+
+| Package | Purpose |
+|---------|---------|
+| `@kadi.build/file-manager` | File management utilities |
+| `@kadi.build/tunnel-services` | Multi-provider tunnel management |
+| `express` | HTTP framework |
+| `cors` | CORS middleware |
+| `mime-types` | MIME type detection |
+| `xml2js` | XML generation for S3 responses |
+| `chalk` | Console styling |
+
+## Requirements
+
+- Node.js ≥ 18.0.0
+
+## License
+
+MIT

package/package.json

@@ -0,0 +1,60 @@
+{
+  "name": "@kadi.build/file-sharing",
+  "version": "1.0.0",
+  "description": "File sharing service with tunneling and local S3-compatible interface",
+  "type": "module",
+  "main": "src/index.js",
+  "exports": {
+    ".": {
+      "import": "./src/index.js",
+      "types": "./types/index.d.ts"
+    },
+    "./s3": {
+      "import": "./src/S3Server.js"
+    },
+    "./http": {
+      "import": "./src/HttpServerProvider.js"
+    }
+  },
+  "files": [
+    "src/",
+    "types/",
+    "README.md"
+  ],
+  "scripts": {
+    "test": "node tests/run-all-tests.js",
+    "test:smoke": "node tests/test-dependency-smoke.js",
+    "test:http": "node tests/test-http-server.js",
+    "test:s3": "node tests/test-s3-server.js",
+    "test:integration": "node tests/test-file-sharing.js",
+    "test:monitor": "node tests/test-download-monitor.js",
+    "test:shutdown": "node tests/test-shutdown-manager.js",
+    "test:streaming": "node tests/test-streaming.js"
+  },
+  "keywords": [
+    "file-sharing",
+    "http-server",
+    "s3-compatible",
+    "tunnel",
+    "kadi",
+    "file-server",
+    "download-monitor"
+  ],
+  "dependencies": {
+    "@kadi.build/file-manager": "^1.0.0",
+    "@kadi.build/tunnel-services": "^1.0.2",
+    "chalk": "^5.3.0",
+    "cors": "^2.8.5",
+    "express": "^4.18.0",
+    "mime-types": "^2.1.35",
+    "xml2js": "^0.5.0"
+  },
+  "devDependencies": {
+    "@aws-sdk/client-s3": "^3.0.0",
+    "supertest": "^6.0.0"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "license": "MIT"
+}