@kadi.build/file-sharing 1.2.1 → 1.3.0

package/README.md

@@ -122,15 +122,59 @@ console.log(`Share this link: ${publicUrl}`);
 
 ### How Secrets Are Loaded
 
-Secrets are resolved in **priority order** (first match wins):
+As of v1.3.0, secrets are resolved using a **3-tier vault pattern** with fallback:
 
-1. **Explicit constructor config** — values you pass directly
-2. **Environment variables** — `process.env.*`
-3. **`.env` file** — automatically found by walking up parent directories from `process.cwd()`
+| Priority | Source | What it provides |
+|----------|--------|------------------|
+| 1 (highest) | `process.env` | Direct overrides at startup |
+| 2 | **Constructor config** | Values passed to `FileSharingServer` |
+| 3 | **Vault** (`secrets.toml`) | Encrypted tokens via `secret-ability` |
+| 4 | **`config.yml`** | Non-secret settings (walks up from CWD) |
+| 5 (fallback) | **`.env` file** | Legacy fallback (walks up from CWD) |
 
-The `.env` loader supports monorepo layouts: if your `.env` is at the workspace root and the package runs from `packages/file-sharing/`, it will find it automatically.
+The caller (typically `deploy-ability` or `kadi-deploy`) must have `secret-ability` installed via `kadi install` for vault access. If no vault is found, the system gracefully falls back to `.env` files.
 
-### Environment Variables
+### Vault Setup
+
+Store tunnel tokens in the `tunnel` vault and file-sharing secrets in the `file-sharing` vault:
+
+```bash
+# Tunnel tokens (shared with tunnel-services)
+kadi secret set -v tunnel KADI_TUNNEL_TOKEN <your-token>
+kadi secret set -v tunnel NGROK_AUTH_TOKEN <your-token>
+
+# File-sharing secrets (optional)
+kadi secret set -v file-sharing KADI_AUTH_API_KEY <your-api-key>
+kadi secret set -v file-sharing KADI_S3_ACCESS_KEY <your-key>
+kadi secret set -v file-sharing KADI_S3_SECRET_KEY <your-secret>
+```
+
+### config.yml
+
+Place a `config.yml` in your project root (or any parent directory):
+
+```yaml
+# Tunnel config (shared across tunnel-services consumers)
+tunnel:
+  server_addr: broker.kadi.build
+  tunnel_domain: tunnel.kadi.build
+  server_port: 7000
+  ssh_port: 2200
+  mode: frpc
+  transport: wss
+  wss_control_host: tunnel-control.kadi.build
+  agent_id: kadi
+
+# File-sharing specific config
+file-sharing:
+  port: 3000
+  host: 0.0.0.0
+  enable_directory_listing: true
+  cors: true
+  enable_s3: false
+```
+
+### Environment Variables (overrides)
 
 | Variable | Description | Default |
 |----------|-------------|---------|
@@ -141,8 +185,8 @@ The `.env` loader supports monorepo layouts: if your `.env` is at the workspace
 | `KADI_TUNNEL_PORT` | KĀDI frps server port | `7000` |
 | `KADI_TUNNEL_SSH_PORT` | KĀDI SSH gateway port | `2200` |
 | `KADI_TUNNEL_MODE` | Connection mode: `ssh`, `frpc`, or `auto` | `auto` |
-| `KADI_TUNNEL_TRANSPORT` | Transport protocol: `wss` (via gateway on :443) or `tcp` (direct) | `wss` |
-| `KADI_TUNNEL_WSS_HOST` | WSS gateway hostname (e.g., `tunnel-control.kadi.build`) | — |
+| `KADI_TUNNEL_TRANSPORT` | Transport protocol: `wss` or `tcp` | `wss` |
+| `KADI_TUNNEL_WSS_HOST` | WSS gateway hostname | — |
 | `KADI_AGENT_ID` | Agent identifier for proxy naming | `kadi` |
 | **Tunnel — Ngrok** | | |
 | `NGROK_AUTHTOKEN` | Ngrok auth token (also accepts `NGROK_AUTH_TOKEN`) | — |
@@ -154,31 +198,6 @@ The `.env` loader supports monorepo layouts: if your `.env` is at the workspace
 | `KADI_S3_ACCESS_KEY` | S3 access key ID | `minioadmin` |
 | `KADI_S3_SECRET_KEY` | S3 secret access key | `minioadmin` |
 
-### `.env` File Example
-
-```env
-# Tunnel credentials
-KADI_TUNNEL_TOKEN=your-kadi-token-here
-KADI_TUNNEL_SERVER=broker.kadi.build
-KADI_TUNNEL_DOMAIN=tunnel.kadi.build
-KADI_TUNNEL_PORT=7000
-KADI_TUNNEL_SSH_PORT=2200
-KADI_TUNNEL_MODE=ssh
-KADI_TUNNEL_TRANSPORT=wss
-KADI_TUNNEL_WSS_HOST=tunnel-control.kadi.build
-KADI_AGENT_ID=my-agent
-
-# Optional: Ngrok (fallback)
-NGROK_AUTHTOKEN=your-ngrok-token
-
-# HTTP auth (optional — protects file downloads)
-KADI_AUTH_API_KEY=my-secret-api-key
-
-# S3 credentials (optional — default minioadmin/minioadmin)
-KADI_S3_ACCESS_KEY=my-access-key
-KADI_S3_SECRET_KEY=my-secret-key
-```
-
 ### HTTP Authentication Schemes
 
 When `auth` is configured (via config, env vars, or `.env`), the HTTP server supports three authentication schemes:
@@ -584,6 +603,56 @@ Set `KADI_TUNNEL_TRANSPORT=wss` and `KADI_TUNNEL_WSS_HOST=tunnel-control.kadi.bu
 
 If KĀDI credentials are not provided, the tunnel will automatically fall back to free services (serveo → localtunnel → pinggy → localhost.run) unless `autoFallback: false` is set.
 
+### Container Usage — Installing `frpc`
+
+When running inside a Docker container (e.g. via `kadi deploy`), the KĀDI tunnel requires the **frpc** binary to be available at build time. Alpine-based images (like `node:22-alpine`) do not include it by default.
+
+Add the following lines to the `run` array in your `agent.json` build config:
+
+```json
+"run": [
+  "apk add --no-cache curl openssh-client",
+  "FRPC_VERSION=0.61.1 && wget -qO /tmp/frpc.tar.gz https://github.com/fatedier/frp/releases/download/v${FRPC_VERSION}/frp_${FRPC_VERSION}_linux_amd64.tar.gz && tar -xzf /tmp/frpc.tar.gz -C /tmp && mv /tmp/frp_${FRPC_VERSION}_linux_amd64/frpc /usr/local/bin/frpc && chmod +x /usr/local/bin/frpc && rm -rf /tmp/frpc.tar.gz /tmp/frp_${FRPC_VERSION}_linux_amd64"
+]
+```
+
+**What these lines do:**
+
+1. **`apk add --no-cache curl openssh-client`** — Installs `curl` and the SSH client (needed for SSH-mode fallback).
+2. **`FRPC_VERSION=0.61.1 && wget …`** — Downloads the frpc binary (v0.61.1) from the official [fatedier/frp](https://github.com/fatedier/frp) releases, extracts it to `/usr/local/bin/frpc`, and cleans up temp files.
+
+> **Note:** For non-Alpine images (Debian/Ubuntu-based), replace `apk add` with `apt-get update && apt-get install -y curl openssh-client`.
+
+See [`arcadedb-ability/agent.json`](../arcadedb-ability/agent.json) and [`backup-ability/agent.json`](../backup-ability/agent.json) for real-world examples.
+
+### Troubleshooting Tunnels
+
+If your deployed container fails to establish a tunnel, the most common cause is a **missing `frpc` binary**. Symptoms include:
+
+- Tunnel creation silently fails or times out
+- Logs show `frpc: not found` or the service falls back to SSH mode unexpectedly
+- The agent connects to the broker but is not reachable via its tunnel URL
+
+**Fix:** Ensure the frpc installation lines above are in your `agent.json` `build.*.run` array, then rebuild:
+
+```bash
+kadi build
+```
+
+Verify frpc is installed inside the container:
+
+```bash
+frpc --version
+# Expected: frpc version 0.61.1
+```
+
+| Symptom | Likely Cause | Fix |
+|---------|-------------|-----|
+| `frpc: not found` | frpc not installed in the container image | Add the `run` lines above and rebuild |
+| Tunnel times out | Network/firewall blocking port 7000 | Set `KADI_TUNNEL_TRANSPORT=wss` to use port 443 |
+| SSH fallback instead of frpc | frpc binary missing from `$PATH` | Install frpc (see above) |
+| `Permission denied` on frpc | Binary not executable | Ensure `chmod +x /usr/local/bin/frpc` is in your build |
+
 ---
 
 ## Dependencies

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kadi.build/file-sharing",
-  "version": "1.2.1",
+  "version": "1.3.0",
   "description": "File sharing service with tunneling and local S3-compatible interface",
   "type": "module",
   "main": "src/index.js",
@@ -47,6 +47,7 @@
   "dependencies": {
     "@kadi.build/file-manager": "^1.0.0",
     "@kadi.build/tunnel-services": "^1.0.5",
+    "js-yaml": "^4.1.0",
     "chalk": "^5.3.0",
     "cors": "^2.8.5",
     "express": "^4.18.0",

package/src/FileSharingServer.js

@@ -6,13 +6,20 @@
  * file sharing solution.
  *
  * KĀDI is the default tunnel service.
+ *
+ * Configuration resolution:
+ *   1. config.yml walk-up → "tunnel" section (non-secret settings)
+ *      + "file-sharing" section (server-specific settings)
+ *   2. secrets.toml vault → "tunnel" vault for tokens (via secret-ability)
+ *   3. .env walk-up → fallback when no vault found
+ *   4. process.env → always wins (direct environment overrides)
  */
 
 import { EventEmitter } from 'events';
 import fs from 'fs';
 import path from 'path';
 import { createFileManager } from '@kadi.build/file-manager';
-import { TunnelManager } from '@kadi.build/tunnel-services';
+import { TunnelManager, resolveTunnelConfig, buildTunnelManagerConfig } from '@kadi.build/tunnel-services';
 import { HttpServerProvider } from './HttpServerProvider.js';
 import { S3Server } from './S3Server.js';
 import { DownloadMonitor } from './DownloadMonitor.js';
@@ -20,12 +27,41 @@ import { ShutdownManager } from './ShutdownManager.js';
 import { MonitoringDashboard } from './MonitoringDashboard.js';
 import { EventNotifier } from './EventNotifier.js';
 
+let yaml;
+try {
+  const m = await import('js-yaml');
+  yaml = m.default || m;
+} catch {
+  yaml = null;
+}
+
+// ─── Walk-up helpers ──────────────────────────────────────────────────
+
 /**
- * Parse a .env file's content and inject into process.env.
- * Existing env vars take priority (are NOT overwritten).
+ * Walk up from startDir looking for a filename.
+ * @param {string} filename
+ * @param {string} [startDir]
+ * @returns {string|null}
+ */
+function _walkUpFind(filename, startDir) {
+  let dir = startDir || process.cwd();
+  while (true) {
+    const candidate = path.join(dir, filename);
+    if (fs.existsSync(candidate)) return candidate;
+    const parent = path.dirname(dir);
+    if (parent === dir) break;
+    dir = parent;
+  }
+  return null;
+}
+
+/**
+ * Parse a .env file's content into an object (does NOT inject into process.env).
  * @param {string} content  raw file content
+ * @returns {Record<string, string>}
  */
 function _parseEnvFile(content) {
+  const result = {};
   for (const line of content.split('\n')) {
     const trimmed = line.trim();
     if (!trimmed || trimmed.startsWith('#')) continue;
@@ -38,82 +74,137 @@ function _parseEnvFile(content) {
         (val.startsWith("'") && val.endsWith("'"))) {
       val = val.slice(1, -1);
     }
-    // Real env vars take priority over .env file values
-    if (process.env[key] === undefined) {
-      process.env[key] = val;
-    }
+    result[key] = val;
   }
+  return result;
 }
 
 /**
- * Walk up from cwd looking for a .env file (supports monorepo layouts
- * where .env lives at the workspace root, not inside packages/*).
- * Stops at the filesystem root.
+ * Load file-sharing specific config from config.yml "file-sharing" section.
+ * @returns {Record<string, any>}
  */
-function _loadEnvFile() {
-  let dir = process.cwd();
-  const root = path.parse(dir).root;
+function _loadFileSharingConfig() {
+  if (!yaml) return {};
+  const configPath = _walkUpFind('config.yml');
+  if (!configPath) return {};
+  try {
+    const parsed = yaml.load(fs.readFileSync(configPath, 'utf8'));
+    return (parsed && parsed['file-sharing']) || {};
+  } catch {
+    return {};
+  }
+}
 
-  while (true) {
+/**
+ * Load file-sharing secrets from vault or .env fallback.
+ *
+ * Secret keys for file-sharing (S3 credentials, auth):
+ *   vault "file-sharing" (or configured vault):
+ *     KADI_S3_ACCESS_KEY, KADI_S3_SECRET_KEY
+ *     KADI_AUTH_USERNAME, KADI_AUTH_PASSWORD, KADI_AUTH_API_KEY
+ *
+ * Tunnel secrets are handled by resolveTunnelConfig() separately.
+ *
+ * @param {object} [kadiClient]
+ * @returns {Promise<Record<string, string>>}
+ */
+async function _loadFileSharingSecrets(kadiClient) {
+  const { execSync } = await import('child_process');
+  const VAULT_NAME = 'file-sharing';
+  const KEYS = [
+    'KADI_S3_ACCESS_KEY', 'KADI_S3_SECRET_KEY',
+    'KADI_AUTH_USERNAME', 'KADI_AUTH_PASSWORD', 'KADI_AUTH_API_KEY',
+  ];
+  const secrets = {};
+
+  // Tier 1 — native secret-ability
+  if (kadiClient) {
     try {
-      const envPath = path.join(dir, '.env');
-      const content = fs.readFileSync(envPath, 'utf8');
-      _parseEnvFile(content);
-      return; // found and loaded — done
-    } catch {
-      // not found in this directory — keep climbing
+      const sa = await kadiClient.loadNative('secret-ability');
+      try {
+        for (const key of KEYS) {
+          try {
+            const r = await sa.invoke('get', { vault: VAULT_NAME, key });
+            if (r && r.value) secrets[key] = r.value;
+          } catch { /* skip */ }
+        }
+      } finally {
+        try { await sa.disconnect(); } catch { /* ignore */ }
+      }
+      if (Object.keys(secrets).length > 0) return secrets;
+    } catch { /* fall through */ }
+  }
+
+  // Tier 2 — kadi CLI
+  let cliAvailable;
+  try {
+    execSync('kadi --version', { encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'] });
+    cliAvailable = true;
+  } catch {
+    cliAvailable = false;
+  }
+  if (cliAvailable) {
+    for (const key of KEYS) {
+      try {
+        const val = execSync(`kadi secret get -v ${VAULT_NAME} ${key}`, {
+          encoding: 'utf8', stdio: ['pipe', 'pipe', 'pipe'],
+        }).trim();
+        if (val) secrets[key] = val;
+      } catch { /* skip */ }
     }
-    const parent = path.dirname(dir);
-    if (parent === dir || dir === root) break; // reached fs root
-    dir = parent;
+    if (Object.keys(secrets).length > 0) return secrets;
   }
+
+  // Tier 3 — .env fallback
+  const envPath = _walkUpFind('.env');
+  if (envPath) {
+    const parsed = _parseEnvFile(fs.readFileSync(envPath, 'utf8'));
+    for (const key of KEYS) {
+      if (parsed[key]) secrets[key] = parsed[key];
+    }
+  }
+
+  return secrets;
 }
 
 /**
- * Load secrets from environment variables and optional .env file.
- * Supports KADI_ prefixed vars and common tunnel/auth env vars.
- *
- * Env vars (all optional — constructor config takes priority):
- *   KADI_TUNNEL_TOKEN   — KĀDI tunnel auth token
- *   KADI_TUNNEL_SERVER  — KĀDI broker address (default: broker.kadi.build)
- *   KADI_TUNNEL_DOMAIN  — KĀDI tunnel domain (default: tunnel.kadi.build)
- *   KADI_TUNNEL_PORT    — KĀDI frpc port (default: 7000)
- *   KADI_TUNNEL_SSH_PORT— KĀDI SSH gateway port (default: 2200)
- *   KADI_TUNNEL_MODE    — ssh | frpc | auto (default: auto)
- *   KADI_TUNNEL_TRANSPORT— wss | tcp (default: wss)
- *   KADI_TUNNEL_WSS_HOST — WSS gateway hostname (e.g., tunnel-control.kadi.build)
- *   KADI_AGENT_ID       — Agent identifier for proxy naming
- *   NGROK_AUTHTOKEN     — Ngrok auth token (also: NGROK_AUTH_TOKEN)
- *   KADI_S3_ACCESS_KEY  — S3 access key (default: minioadmin)
- *   KADI_S3_SECRET_KEY  — S3 secret key (default: minioadmin)
- *   KADI_AUTH_USERNAME   — HTTP Basic auth username
- *   KADI_AUTH_PASSWORD   — HTTP Basic auth password
- *   KADI_AUTH_API_KEY    — API key for Bearer/X-API-Key auth
+ * Load ALL secrets needed by FileSharingServer.
+ * Merges tunnel secrets (from tunnel config resolver) + file-sharing specific secrets.
+ * process.env overrides always win.
  *
- * @returns {object} Secrets object (values are strings or undefined)
+ * @param {object} [kadiClient]
+ * @returns {Promise<object>}
  */
-function loadSecrets() {
-  // Find and load .env file — walk up parent directories (monorepo support).
-  // e.g. packages/file-sharing/ → packages/ → workspace root (where .env lives)
-  _loadEnvFile();
+async function loadSecrets(kadiClient) {
+  // Resolve tunnel config + secrets via tunnel-services configResolver
+  const tunnelResolved = await resolveTunnelConfig({ kadiClient });
+  const tunnelSecrets = tunnelResolved.secrets;
 
+  // Load file-sharing specific secrets (S3, auth)
+  const fsSecrets = await _loadFileSharingSecrets(kadiClient);
 
+  // process.env overrides (always win)
+  const env = process.env;
   return {
-    kadiToken:      process.env.KADI_TUNNEL_TOKEN,
-    kadiServer:     process.env.KADI_TUNNEL_SERVER,
-    kadiDomain:     process.env.KADI_TUNNEL_DOMAIN,
-    kadiPort:       process.env.KADI_TUNNEL_PORT ? Number(process.env.KADI_TUNNEL_PORT) : undefined,
-    kadiSshPort:    process.env.KADI_TUNNEL_SSH_PORT ? Number(process.env.KADI_TUNNEL_SSH_PORT) : undefined,
-    kadiMode:       process.env.KADI_TUNNEL_MODE,
-    kadiAgentId:    process.env.KADI_AGENT_ID,
-    kadiTransport:      process.env.KADI_TUNNEL_TRANSPORT,
-    kadiWssControlHost: process.env.KADI_TUNNEL_WSS_HOST,
-    ngrokAuthToken: process.env.NGROK_AUTHTOKEN || process.env.NGROK_AUTH_TOKEN,
-    s3AccessKey:    process.env.KADI_S3_ACCESS_KEY,
-    s3SecretKey:    process.env.KADI_S3_SECRET_KEY,
-    authUsername:   process.env.KADI_AUTH_USERNAME,
-    authPassword:   process.env.KADI_AUTH_PASSWORD,
-    authApiKey:     process.env.KADI_AUTH_API_KEY
+    kadiToken:      env.KADI_TUNNEL_TOKEN      || tunnelSecrets.kadi_token,
+    kadiServer:     env.KADI_TUNNEL_SERVER      || tunnelResolved.config.server_addr,
+    kadiDomain:     env.KADI_TUNNEL_DOMAIN      || tunnelResolved.config.tunnel_domain,
+    kadiPort:       env.KADI_TUNNEL_PORT        ? Number(env.KADI_TUNNEL_PORT) :
+                    tunnelResolved.config.server_port,
+    kadiSshPort:    env.KADI_TUNNEL_SSH_PORT    ? Number(env.KADI_TUNNEL_SSH_PORT) :
+                    tunnelResolved.config.ssh_port,
+    kadiMode:       env.KADI_TUNNEL_MODE        || tunnelResolved.config.mode,
+    kadiAgentId:    env.KADI_AGENT_ID           || tunnelResolved.config.agent_id,
+    kadiTransport:      env.KADI_TUNNEL_TRANSPORT  || tunnelResolved.config.transport,
+    kadiWssControlHost: env.KADI_TUNNEL_WSS_HOST   || tunnelResolved.config.wss_control_host,
+    ngrokAuthToken: env.NGROK_AUTHTOKEN || env.NGROK_AUTH_TOKEN || tunnelSecrets.ngrok_token,
+    s3AccessKey:    env.KADI_S3_ACCESS_KEY  || fsSecrets.KADI_S3_ACCESS_KEY,
+    s3SecretKey:    env.KADI_S3_SECRET_KEY  || fsSecrets.KADI_S3_SECRET_KEY,
+    authUsername:   env.KADI_AUTH_USERNAME   || fsSecrets.KADI_AUTH_USERNAME,
+    authPassword:   env.KADI_AUTH_PASSWORD   || fsSecrets.KADI_AUTH_PASSWORD,
+    authApiKey:     env.KADI_AUTH_API_KEY    || fsSecrets.KADI_AUTH_API_KEY,
+    // Pass through for TunnelManager config building
+    _tunnelResolved: tunnelResolved,
   };
 }
 
@@ -138,9 +229,15 @@ function _clientHost(host) {
 }
 
 export class FileSharingServer extends EventEmitter {
+  /**
+   * @param {object} [config]
+   * @param {object} [config.kadiClient]  KadiClient for native vault access
+   */
   constructor(config = {}) {
     super();
 
+    this._kadiClient = config.kadiClient || null;
+
     this.config = {
       staticDir: process.cwd(),
       port: 3000,
@@ -201,12 +298,35 @@ export class FileSharingServer extends EventEmitter {
       this.config.s3Port = 0;
     }
 
-    // ----------------------------------------------------------------
-    // Load secrets from env vars / .env (constructor config takes priority)
-    // ----------------------------------------------------------------
-    const secrets = loadSecrets();
+    // Initialize components (secrets resolved lazily in start())
+    this.fileManager = createFileManager();
+    this.httpServer = null; // Initialized in _initSecrets()
+    this.s3Server = null;
+    this.tunnelManager = null;
+
+    this.downloadMonitor = new DownloadMonitor();
+    this.shutdownManager = new ShutdownManager(this.config.shutdown);
+    this.monitoringDashboard = new MonitoringDashboard();
+    this.eventNotifier = new EventNotifier();
+
+    // State
+    this.isRunning = false;
+    this._secretsResolved = false;
+    this.tunnel = null;
+    this._startTime = null;
+  }
+
+  /**
+   * Resolve secrets (vault → .env → process.env) and initialize components
+   * that depend on secret values. Called automatically from start().
+   */
+  async _initSecrets() {
+    if (this._secretsResolved) return;
+
+    // ── Load secrets (config.yml + vault + .env + process.env) ──
+    const secrets = await loadSecrets(this._kadiClient);
 
-    // Build auth config: explicit config > env vars > null
+    // Build auth config: explicit config > vault/env > null
     if (!this.config.auth) {
       if (secrets.authApiKey) {
         this.config.auth = { apiKey: secrets.authApiKey };
@@ -215,9 +335,7 @@ export class FileSharingServer extends EventEmitter {
       }
     }
 
-    // Initialize components
-    this.fileManager = createFileManager();
-
+    // HTTP server
     this.httpServer = new HttpServerProvider({
       port: this.config.port,
       host: this.config.host,
@@ -228,7 +346,7 @@ export class FileSharingServer extends EventEmitter {
       ...(this.config.httpConfig || {})
     });
 
-    this.s3Server = null;
+    // S3 server
     if (this.config.enableS3) {
       this.s3Server = new S3Server({
         port: this.config.s3Port,
@@ -240,44 +358,54 @@ export class FileSharingServer extends EventEmitter {
       });
     }
 
-    // Build TunnelManager config from env vars + explicit tunnel config.
-    // Use || to let explicit config override env, and filter out undefined
-    // values so TunnelManager's own defaults are preserved.
+    // Build TunnelManager config from tunnel resolver + explicit tunnel config.
     const tunnelCfg = this.config.tunnel || {};
-    const tunnelManagerConfig = _filterDefined({
-      primaryService: tunnelCfg.service || 'kadi',
-      autoFallback: tunnelCfg.autoFallback,
-      // KĀDI auth (env vars as fallback)
-      kadiToken:    tunnelCfg.kadiToken    || secrets.kadiToken,
-      kadiServer:   tunnelCfg.kadiServer   || secrets.kadiServer,
-      kadiDomain:   tunnelCfg.kadiDomain   || secrets.kadiDomain,
-      kadiPort:     tunnelCfg.kadiPort     || secrets.kadiPort,
-      kadiSshPort:  tunnelCfg.kadiSshPort  || secrets.kadiSshPort,
-      kadiMode:     tunnelCfg.kadiMode     || secrets.kadiMode,
-      kadiAgentId:  tunnelCfg.kadiAgentId  || secrets.kadiAgentId,
-      // KĀDI transport (WSS gateway)
-      kadiTransport:      tunnelCfg.kadiTransport      || secrets.kadiTransport,
-      kadiWssControlHost: tunnelCfg.kadiWssControlHost  || secrets.kadiWssControlHost,
-      // Ngrok auth (env vars as fallback)
-      ngrokAuthToken: tunnelCfg.ngrokAuthToken || secrets.ngrokAuthToken,
-      // Pass through any extra service-specific options
-      ...(tunnelCfg.managerOptions || {})
-    });
-    this.tunnelManager = new TunnelManager(tunnelManagerConfig);
-    this.downloadMonitor = new DownloadMonitor();
-    this.shutdownManager = new ShutdownManager(this.config.shutdown);
-    this.monitoringDashboard = new MonitoringDashboard();
-    this.eventNotifier = new EventNotifier();
+    let tunnelManagerConfig;
+
+    if (secrets._tunnelResolved) {
+      // Use the tunnel config resolver output, with explicit overrides
+      tunnelManagerConfig = buildTunnelManagerConfig(secrets._tunnelResolved, _filterDefined({
+        primaryService: tunnelCfg.service,
+        autoFallback: tunnelCfg.autoFallback,
+        kadiToken:    tunnelCfg.kadiToken,
+        kadiServer:   tunnelCfg.kadiServer,
+        kadiDomain:   tunnelCfg.kadiDomain,
+        kadiPort:     tunnelCfg.kadiPort,
+        kadiSshPort:  tunnelCfg.kadiSshPort,
+        kadiMode:     tunnelCfg.kadiMode,
+        kadiAgentId:  tunnelCfg.kadiAgentId,
+        kadiTransport:      tunnelCfg.kadiTransport,
+        kadiWssControlHost: tunnelCfg.kadiWssControlHost,
+        ngrokAuthToken: tunnelCfg.ngrokAuthToken,
+        ...(tunnelCfg.managerOptions || {})
+      }));
+    } else {
+      // Fallback: build from secrets directly (backward compat)
+      tunnelManagerConfig = _filterDefined({
+        primaryService: tunnelCfg.service || 'kadi',
+        autoFallback: tunnelCfg.autoFallback,
+        kadiToken:    tunnelCfg.kadiToken    || secrets.kadiToken,
+        kadiServer:   tunnelCfg.kadiServer   || secrets.kadiServer,
+        kadiDomain:   tunnelCfg.kadiDomain   || secrets.kadiDomain,
+        kadiPort:     tunnelCfg.kadiPort     || secrets.kadiPort,
+        kadiSshPort:  tunnelCfg.kadiSshPort  || secrets.kadiSshPort,
+        kadiMode:     tunnelCfg.kadiMode     || secrets.kadiMode,
+        kadiAgentId:  tunnelCfg.kadiAgentId  || secrets.kadiAgentId,
+        kadiTransport:      tunnelCfg.kadiTransport      || secrets.kadiTransport,
+        kadiWssControlHost: tunnelCfg.kadiWssControlHost  || secrets.kadiWssControlHost,
+        ngrokAuthToken: tunnelCfg.ngrokAuthToken || secrets.ngrokAuthToken,
+        ...(tunnelCfg.managerOptions || {})
+      });
+    }
 
-    // State
-    this.isRunning = false;
-    this.tunnel = null;
-    this._startTime = null;
+    this.tunnelManager = new TunnelManager(tunnelManagerConfig);
 
     // Wire up events
     this._setupEventForwarding();
     this._setupShutdownHandlers();
     this._setupWebhooks();
+
+    this._secretsResolved = true;
   }
 
   /**
@@ -289,10 +417,15 @@ export class FileSharingServer extends EventEmitter {
       return this.getInfo();
     }
 
+    // Resolve secrets from vault/config.yml/.env (lazy init)
+    await this._initSecrets();
+
     this._startTime = Date.now();
 
     // Start HTTP server
     const httpResult = await this.httpServer.start();
+    // Sync the actual bound port (important when port=0, i.e. OS-assigned)
+    this.config.port = httpResult.port;
     this.emit('http:started', httpResult);
 
     // Start S3 server if enabled