openship 0.1.0 → 0.1.1

package/packages/adapters/docs/EXECUTOR.md

@@ -0,0 +1,174 @@
+# Command Executor — Local & SSH Abstraction
+
+> The single abstraction that makes the entire platform work on both local and remote servers.
+
+## The Problem
+
+Openship needs to:
+- Run shell commands (git, npm, kill, systemctl)
+- Read/write files (PID files, Traefik YAML configs, log files)
+- Check if files exist (`.git` dirs, ACME storage)
+
+Sometimes on the **same machine** (self-hosted, single-server). Sometimes on a **remote server** via SSH (multi-server, managed hosting).
+
+Without an abstraction, every function would need two code paths.
+
+## The Solution
+
+```typescript
+interface CommandExecutor {
+  exec(command: string, opts?: { timeout?: number }): Promise<string>;
+  streamExec(command: string, onLog: (log: LogEntry) => void): Promise<{ code: number; output: string }>;
+  writeFile(path: string, content: string): Promise<void>;
+  readFile(path: string): Promise<string>;
+  exists(path: string): Promise<boolean>;
+  mkdir(path: string): Promise<void>;
+  rm(path: string): Promise<void>;
+  dispose(): Promise<void>;
+}
+```
+
+Two implementations, same interface:
+
+| Method | `LocalExecutor` | `SshExecutor` |
+|---|---|---|
+| `exec()` | `child_process.exec()` | `ssh2.Client.exec()` |
+| `streamExec()` | `child_process.spawn()` detached (process group leader) | `ssh2.Client.exec()` with stream events |
+| `writeFile()` | `fs.writeFile()` | SFTP `writeFile()` |
+| `readFile()` | `fs.readFile()` | SFTP `readFile()` |
+| `exists()` | `fs.access()` | SFTP `stat()` |
+| `mkdir()` | `fs.mkdir({ recursive })` | `ssh exec("mkdir -p")` |
+| `rm()` | `fs.rm()` | SFTP `unlink()` |
+| `dispose()` | no-op | `client.end()` |
+
+## Who Uses It
+
+```
+CommandExecutor (single instance in selfhosted mode)
+    │
+    ├── BareRuntime
+    │     build()    → git clone, npm install, npm build (streamExec)
+    │     deploy()   → nohup npm start & echo $! (exec)
+    │     stop()     → kill PID (exec)
+    │     isAlive()  → kill -0 PID (exec)
+    │     logs       → tail -n (exec) or readFile
+    │     PID mgmt   → readFile / writeFile / rm
+    │
+    ├── TraefikProvider
+    │     registerRoute() → writeFile (YAML config)
+    │     removeRoute()   → rm (config file)
+    │     provisionCert() → readFile (ACME storage JSON)
+    │
+    ├── SystemManager (via checks + installers)
+    │     checks    → exec("docker --version"), exec("node -v"), etc.
+    │     installs  → streamExec("curl ... | sh"), streamExec("apt install")
+    │
+    └── FileStateStore
+          get()  → readFile (state JSON)
+          set()  → writeFile (state JSON)
+```
+
+## Security
+
+### SSH: Key-Based Auth Only
+```typescript
+// Password auth is deliberately not supported
+if (!config.privateKey && !config.sshAgent) {
+  throw new Error("SSH requires either privateKey or sshAgent");
+}
+```
+
+### No User Input in Shell Strings
+- `InstallerConfig` values (email, domain) are validated before use
+- Shell commands use hardcoded arguments
+- `DEBIAN_FRONTEND=noninteractive` is always set
+
+### Connection Lifecycle
+- SSH connection is established **lazily** on first use
+- Reused for all subsequent operations (no reconnect overhead)
+- `dispose()` must be called on shutdown to close the connection
+
+## Real-Time Log Streaming
+
+Both executors support `streamExec()` — the key to showing live install/build progress:
+
+```typescript
+// stdout lines → onLog({ level: "info", message: line })
+// stderr lines → onLog({ level: "warn", message: line })
+// returns { code, output } when command exits
+```
+
+This powers:
+- Build progress in the dashboard (git output, npm install progress)
+- System setup wizard (live installation output)
+- Any command that needs to show output as it happens
+
+## Factory
+
+```typescript
+import { createExecutor } from "@repo/adapters";
+
+// Local — commands run on this machine
+const local = createExecutor();
+
+// Remote — commands run on the SSH server
+const remote = createExecutor({
+  host: "192.168.1.100",
+  username: "deploy",
+  privateKey: decryptedKey,
+});
+```
+
+The platform does this automatically — you don't call `createExecutor()` directly in service code. It's wired through `PlatformConfig.ssh`.
+
+## Important: Docker is Different
+
+`DockerRuntime` does **NOT** use `CommandExecutor`. It uses dockerode's built-in SSH transport:
+
+```typescript
+// DockerRuntime connects to Docker Engine directly
+new Dockerode({ protocol: "ssh", host, username, sshOptions: { privateKey } })
+```
+
+This is correct because:
+- Docker Engine API is a separate protocol (HTTP over socket/SSH/TCP)
+- dockerode handles multiplexing, streaming, and image pulls natively
+- Wrapping Docker commands in `executor.exec("docker ...")` would lose all that
+
+The shared executor handles everything **except** Docker container management:
+- Traefik config files (YAML writes)
+- System checks and installations
+- Bare runtime process management (when not using Docker)
+
+## Process Groups & Cleanup
+
+### LocalExecutor — detached spawn
+
+`streamExec()` uses `spawn("sh", ["-c", command], { detached: true })`. The `detached: true` flag makes the child process a **process group leader**. This means:
+
+- The child and all its descendants share the same PGID
+- `kill -- -${pid}` kills the entire tree in one shot
+- No orphan processes when a build is cancelled
+
+### BareRuntime — setsid for deployed processes
+
+Deploy uses `setsid nohup npm start &` — `setsid` creates a new session and process group. The stored PID equals the PGID, so:
+
+```
+kill -- -${pid}        ← SIGTERM entire group (npm + node + children)
+kill -9 -- -${pid}     ← SIGKILL entire group (fallback after 10s)
+```
+
+Fallback to plain `kill ${pid}` if group kill isn't supported (some minimal environments).
+
+### Why not just `kill ${pid}`?
+
+```
+npm start
+  └─ node server.js
+       └─ worker.js
+
+kill ${pid}       ← only kills npm, node+worker become orphans
+kill -- -${pid}   ← kills npm, node, AND worker — clean shutdown
+```
+

package/packages/adapters/docs/LOG-STREAMING.md

@@ -0,0 +1,175 @@
+# Log Streaming — From Adapter to Terminal
+
+> How log data flows from command execution to the user's browser terminal.
+
+## Two Streaming Paths
+
+| Path | When | Encoding |
+|---|---|---|
+| **Live SSE** | Build/deploy in progress | base64 over SSE |
+| **Status API** | Historical / page load | Plain text in JSON response |
+
+## Live Streaming Flow
+
+```
+Adapter exec output
+    │
+    │  Cloud:  Oblien returns native base64 chunks
+    │  Bare:   executor.streamExec returns plain text lines
+    │  Docker: container.logs() returns plain text lines
+    │
+    ▼
+LogEntry { message: string, rawData?: string }
+    │
+    │  Cloud entries:  rawData = native base64 from Oblien
+    │                  message = decoded text (for DB / display)
+    │  Bare/Docker:    message = plain text
+    │                  rawData = undefined
+    │
+    ▼
+logCallback(entry)  [build.service.ts]
+    │
+    ├─ logs.push(entry)              ← accumulate for DB persistence
+    │
+    └─ sessionManager.appendLog(id, entry)
+         │
+         ├─ session.logs.push(entry) ← in-memory for replay
+         │
+         └─ broadcast to SSE subscribers
+              │
+              ▼
+         formatLogPayload(entry, eventId)
+              │
+              │  Single handover point for encoding:
+              │  base64Data = entry.rawData ?? Buffer.from(entry.message).toString("base64")
+              │
+              │  Cloud:     rawData exists → pass through (zero encoding work)
+              │  Bare/Docker: rawData undefined → encode plain text to base64
+              │
+              ▼
+         SSE event: { type: "log", data: base64, eventId, step?, stepStatus?, level }
+              │
+              ▼
+         Browser: EventSource receives SSE
+              │
+              ▼
+         useSSEStream.ts processLogData()
+              │
+              │  atob(data) → Uint8Array → TextDecoder → text
+              │  Write bytes to xterm.js terminal
+              │
+              ▼
+         Terminal renders output
+```
+
+## Historical Replay Flow
+
+```
+Frontend loads deployment page
+    │
+    ▼
+GET /api/deployments/:id/build-status
+    │
+    ▼
+build.service.ts getBuildSessionStatus()
+    │
+    ├─ Check DB: findBuildSessionByDeploymentId()
+    ├─ Check memory: sessionManager.getSession()
+    │
+    │  logEntries = dbRow.logs ?? memSession.logs ?? []
+    │  logsText = logEntries.map(l => l.message).join("\n")
+    │
+    ▼
+Response: { logs: logsText, status, ... }
+    │
+    ▼
+Frontend: useDeploymentBuild.ts loadBuildSession()
+    │
+    │  data.logs.split("\n")
+    │    .filter(line => line.trim())
+    │    .map(line => ({ type: "info", text: line }))
+    │    .forEach(log => writeToTerminal(encode(log.text + "\r\n")))
+    │
+    ▼
+Terminal renders historical output
+```
+
+## Why Base64 for SSE?
+
+Build output contains binary-unfriendly characters (ANSI color codes, progress bars, carriage returns). Base64 ensures safe transport over SSE text frames without escaping issues.
+
+## Step Events
+
+Step events use the same `LogEntry` transport but with extra fields:
+
+```typescript
+{
+  timestamp: "2026-03-11T...",
+  message: "Installing dependencies (npm)",   // human-readable
+  level: "info",
+  step: "install",                             // which pipeline step
+  stepStatus: "running"                        // running | completed | failed | skipped
+}
+```
+
+The frontend stepper UI reads `step` + `stepStatus` from SSE messages to update the progress indicator. The `message` text is logged to the terminal simultaneously.
+
+## DB Persistence
+
+When a build finishes (success or failure):
+
+```
+finishBuildSession(sessionId, status, durationMs, logs: LogEntry[])
+    │
+    └─ UPDATE build_sessions SET
+         status = 'ready' | 'failed',
+         duration_ms = ...,
+         logs = JSON(LogEntry[])    ← full array serialized as JSON
+```
+
+The `logs` column stores the complete `LogEntry[]` array. On historical load, `message` fields are joined into plain text. `rawData` fields are not persisted — they're only used during live SSE transport.
+
+## Adapter-Specific Details
+
+### Cloud (Oblien)
+
+```
+Oblien exec.stream(cmd) → AsyncGenerator<ExecStreamEvent>
+    │
+    │  event.event = "stdout" → event.data is base64
+    │  event.event = "stderr" → event.data is base64
+    │  event.event = "exit"   → event.exit_code
+    │  event.event = "output" → event.stdout/stderr are base64
+    │
+    ▼
+execAndStream() emit():
+    message = Buffer.from(b64, "base64").toString("utf-8").trimEnd()
+    rawData = b64   ← native, no re-encoding needed
+```
+
+### Bare (Executor)
+
+```
+executor.streamExec("npm install", onLog)
+    │
+    │  spawn("sh", ["-c", command], { detached: true })
+    │  stdout.on("data") → split by \n → logEntry(line)
+    │  stderr.on("data") → split by \n → logEntry(line, "warn")
+    │
+    ▼
+logEntry():
+    { timestamp, message: plainText, level }
+    // No rawData — session-manager encodes at handover point
+```
+
+### Docker
+
+```
+container.logs({ follow: true }) → ReadableStream
+    │
+    │  Buffer chunks → split by \n → parse Docker timestamp prefix
+    │  { timestamp: dockerTimestamp, message: rest, level: parseLevel(rest) }
+    │
+    ▼
+Same as bare — plain text, no rawData
+```

package/packages/adapters/docs/SYSTEM.md

@@ -0,0 +1,191 @@
+# System Layer — Server Setup & Provisioning
+
+> Self-hosted only. Handles component detection, installation, and state caching.
+
+## When Does This Run?
+
+- **Cloud mode**: Never. `system` is `null`. Oblien manages everything.
+- **Desktop mode**: Never. `system` is `null`. Local dev doesn't need server setup.
+- **Self-hosted**: Active. `SystemManager` validates and provisions the server.
+
+## Components by Runtime Mode
+
+| Component | Docker mode | Bare mode |
+|---|---|---|
+| Docker Engine | Required | — |
+| Git | Required | Required |
+| Node.js | — | Required |
+| Bun | — | Optional |
+| Traefik | Required | Required |
+
+## Setup Flow
+
+```
+Dashboard Setup Wizard
+    │
+    │  1. Collects InstallerConfig from user:
+    │     • ACME email (for Let's Encrypt)
+    │     • Domain (primary domain)
+    │     • Traefik mode (file / docker)
+    │
+    ▼
+SystemManager.setup(onLog, config)
+    │
+    │  Phase 1: CHECK — run health checks
+    │     checkDocker(executor)  → "docker --version" + "docker info"
+    │     checkGit(executor)     → "git --version"
+    │     checkNode(executor)    → "node -v"
+    │     checkTraefik(executor) → "traefik version" + systemctl status
+    │
+    │  Phase 2: INSTALL — install missing components
+    │     installDocker(executor, onLog)  → curl get.docker.com | sh
+    │     installTraefik(executor, onLog) → apt install + config files
+    │     installNode(executor, onLog)    → curl nvm | sh
+    │     (all stream real-time output via onLog callback)
+    │
+    │  Phase 3: VALIDATE — re-run checks
+    │     Confirm everything is healthy after installation
+    │
+    │  Phase 4: CACHE — persist state
+    │     SetupStateStore.set(state)
+    │
+    ▼
+Server is ready → setupComplete: true
+```
+
+## State Caching
+
+The system layer does **NOT** re-run checks on every request. State is cached.
+
+### Hot Path (every request)
+
+```typescript
+// In service code before deploying:
+await system.requireFeature("deploy");
+// → reads in-memory cache → returns immediately (< 1ms)
+```
+
+### Cache Layers
+
+```
+┌──────────────────┐
+│  In-memory cache  │ ← fastest, lives in SystemManager instance
+│  (cachedState)    │
+├──────────────────┤
+│  SetupStateStore  │ ← file on disk or row in database
+│  (persistent)     │    survives server restarts
+└──────────────────┘
+```
+
+### Cache Invalidation
+
+| Trigger | Behavior |
+|---|---|
+| First boot (no state) | `isReady()` returns false, forces setup |
+| After `setup()` completes | State written with `setupComplete: true` |
+| After 24 hours | Background re-verification (non-blocking) |
+| Manual `invalidate()` | Clears both caches, next check runs fresh |
+| Component failure | Service layer calls `invalidate()` |
+
+### State Shape
+
+```typescript
+interface SetupState {
+  setupComplete: boolean;
+  mode: "docker" | "bare";
+  components: Record<string, {
+    installed: boolean;
+    version?: string;
+    installedAt?: string;
+  }>;
+  lastVerifiedAt?: string;
+  updatedAt: string;
+}
+```
+
+## Feature Gating
+
+Features map to prerequisite components:
+
+```typescript
+// Docker mode rules
+{ feature: "build",   requires: ["git", "docker"],  message: "Build requires Git and Docker" }
+{ feature: "deploy",  requires: ["docker"],          message: "Deploy requires Docker" }
+{ feature: "routing", requires: ["traefik"],          message: "Routing requires Traefik" }
+{ feature: "ssl",     requires: ["traefik"],          message: "SSL requires Traefik with ACME" }
+
+// Bare mode rules
+{ feature: "build",   requires: ["git", "node"],     message: "Build requires Git and Node.js" }
+{ feature: "deploy",  requires: ["node"],             message: "Deploy requires Node.js" }
+{ feature: "routing", requires: ["traefik"],          message: "Routing requires Traefik" }
+{ feature: "ssl",     requires: ["traefik"],          message: "SSL requires Traefik with ACME" }
+```
+
+Usage in service code:
+
+```typescript
+const { system } = platform();
+
+// Guard a deployment
+if (system) await system.requireFeature("deploy");
+// → throws if Docker/Node not installed
+
+// Check without throwing
+if (system) {
+  const { ready, missing } = await system.checkFeature("ssl");
+  if (!ready) console.warn("SSL not available:", missing);
+}
+```
+
+## InstallerConfig
+
+Pre-collected from the user **before** starting setup. No interactive prompts during installation.
+
+```typescript
+interface InstallerConfig {
+  /** Email for Let's Encrypt ACME registration */
+  acmeEmail?: string;
+  /** Primary domain for Traefik */
+  domain?: string;
+  /** Traefik provider mode */
+  traefikMode?: "file" | "docker";
+}
+```
+
+Validation:
+- `acmeEmail` → basic email regex check
+- `domain` → alphanumeric + dots, no shell-unsafe characters
+- Values are **never** interpolated into shell strings without validation
+
+## SetupStateStore
+
+Pluggable storage backend:
+
+```typescript
+interface SetupStateStore {
+  get(): Promise<SetupState | null>;
+  set(state: SetupState): Promise<void>;
+  clear(): Promise<void>;
+}
+```
+
+| Implementation | Storage | Use Case |
+|---|---|---|
+| `FileStateStore` | JSON file via executor | Default, works everywhere |
+| Custom DB store | Database row | API layer can provide one |
+
+The `FileStateStore` uses the executor, so state files work on both local and remote servers.
+
+## Important Design Decisions
+
+1. **No system layer in cloud mode** — Oblien manages infrastructure. Zero RCE from the API.
+
+2. **Checks run once, not per-request** — State is cached in memory + persistent store. Feature checks read cache.
+
+3. **24h auto-reverification** — Stale cache triggers background re-check without blocking the current request.
+
+4. **Non-interactive installers** — Config collected upfront. All install scripts run with `DEBIAN_FRONTEND=noninteractive`.
+
+5. **All through executor** — Every shell command, every file read/write. Works identically local or over SSH.
+
+6. **Log streaming** — Setup wizard shows real-time output via `onLog` callback → SSE to dashboard.

package/packages/adapters/package.json

@@ -1,13 +1,13 @@
 {
-  "name": "openship-adapters",
+  "name": "@repo/adapters",
   "version": "0.1.0",
   "type": "module",
-  "main": "dist/index.js",
-  "types": "dist/index.d.ts",
+  "main": "src/index.ts",
+  "types": "src/index.ts",
   "exports": {
     ".": {
-      "import": "./dist/index.js",
-      "types": "./dist/index.d.ts"
+      "types": "./src/index.ts",
+      "import": "./src/index.ts"
     }
   },
   "scripts": {
@@ -17,11 +17,16 @@
     "clean": "rm -rf dist .turbo node_modules"
   },
   "dependencies": {
-    "openship-core": "workspace:*"
+    "@repo/core": "workspace:*",
+    "dockerode": "^4.0.9",
+    "oblien": "^2.1.8",
+    "ssh2": "^1.17.0"
   },
   "devDependencies": {
-    "openship-tsconfig": "workspace:*",
-    "tsup": "^8.1.0",
-    "typescript": "^5.4.0"
+    "@types/dockerode": "^4.0.1",
+    "@types/node": "^22.13.0",
+    "@types/ssh2": "^1.15.5",
+    "tsup": "^8.5.1",
+    "typescript": "^5.9.3"
   }
 }