elasticdash-sdk 0.2.0

package/docs/security-compliance.md

@@ -0,0 +1,566 @@
+# Elasticdash Security Compliance Assessment
+
+**Date:** 2026-04-18
+**Scope:** SDK (`elasticdash-test-js`), Backend (Elasticdash API), Dashboard (Web UI)
+**Standards:** AES-256 at rest, TLS 1.2+ in transit, RBAC, MFA, Audit Logging, IDS, Environment Isolation
+
+---
+
+## 1. System Architecture Overview
+
+```
+┌──────────────┐       TLS 1.2+        ┌──────────────────┐       TLS 1.2+       ┌────────────────┐
+│              │  ───────────────────►  │                  │  ◄────────────────►  │                │
+│   SDK        │   Telemetry, Events   │   Backend API    │    REST / WebSocket  │   Dashboard    │
+│  (this repo) │                       │   (cloud)        │                      │   (Web UI)     │
+│              │  ◄────── Socket.io ──►│                  │                      │                │
+└──────────────┘                       └──────────────────┘                      └────────────────┘
+       │                                       │
+       │ HTTPS                                 │ Encrypted Storage
+       ▼                                       ▼
+ ┌──────────────┐                      ┌──────────────────┐
+ │ LLM Providers│                      │   Database /     │
+ │ (OpenAI,     │                      │   Object Store   │
+ │  Anthropic…) │                      └──────────────────┘
+ └──────────────┘
+```
+
+**Data flow:**
+
+1. SDK intercepts AI/tool calls in the user's application and captures workflow traces
+2. SDK transmits telemetry events to the Backend via HTTPS REST or WebSocket (Socket.io)
+3. Backend stores traces, test results, and session data in its database
+4. Dashboard reads from the Backend API to display results to users
+5. Portal server (SDK) can receive tasks from Backend and execute them locally
+
+---
+
+## 2. Per-Standard Assessment
+
+### 2.1 AES-256 Encryption at Rest
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Compliant (opt-in) | Snapshots in `.temp/snapshots/` encrypted with AES-256-GCM when `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY` is set (`dashboard-server.ts`). Gracefully handles legacy unencrypted files. |
+| **Backend** | Compliant | PBKDF2-SHA512 password hashing (100K iterations); AES-256-GCM field encryption for MFA secrets and trace data (input/output/stream_raw); SHA-256 API key hashing (`apiKeyController.js`); AWS Secrets Manager for AI keys; RDS AES-256 via KMS. |
+| **Dashboard** | N/A | Dashboard is a stateless web UI; it does not persist data locally. |
+
+**Current State — SDK (Verified):**
+
+- **Snapshot encryption**: `saveSnapshot()` calls `encryptSnapshot()` before writing when `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY` is set (`dashboard-server.ts:149-183`). Uses AES-256-GCM with random IV and auth tag. Gracefully handles legacy unencrypted files on read via `decryptSnapshot()`.
+- API keys stored in environment variables (plaintext in memory, not persisted to disk by the SDK)
+
+**Current State — Backend (Verified):**
+
+- **Password hashing**: PBKDF2-HMAC-SHA512 with 100,000 iterations, 16-byte random salt, 64-byte key length (`controller/auth/crypto.js`). Stored as `iterations:salt:hash` for forward compatibility
+- **Field-level encryption**: AES-256-GCM encryption for sensitive fields (`controller/auth/encryption.js`). Uses `DB_ENCRYPTION_KEY` env var (32 bytes, hex-encoded). Each encryption generates a 16-byte random IV and 16-byte auth tag
+- **MFA secrets encrypted**: TOTP secrets and recovery codes encrypted via AES-256-GCM before database storage (`controller/auth/mfa.js:94-100`)
+- **AWS Secrets Manager**: AI provider API keys stored in AWS Secrets Manager (`controller/observability/secretsManager.js`), not in database
+- **API keys hashed**: User API keys (`ed_*`) are SHA-256 hashed before storage (`controller/apikeys/apiKeyController.js:9-11`). Only a `key_prefix` (first 16 chars) is stored for display. Raw key is returned only at creation time and cannot be retrieved later.
+- **Trace data encrypted**: `ObservabilityEvents.input`, `.output`, and `.stream_raw` are encrypted with AES-256-GCM via `encryptJSON()`/`encryptField()` before storage (`observabilityController.js`). Decrypted on read via `decryptEventRow()`. Same applies to `ObservabilityReruns`. A denormalized `has_error` boolean flag replaces JSONB `output->>'error'` queries since encrypted columns cannot use JSONB operators. Backward compatible — `decryptField()`/`decryptJSON()` handle legacy unencrypted JSONB objects by checking `typeof` before attempting decryption (`encryption.js:43,103`).
+
+**Key Rotation Policy (`DB_ENCRYPTION_KEY`):**
+
+`DB_ENCRYPTION_KEY` protects trace data (input/output/stream_raw), MFA secrets, and recovery codes. Rotation is annual or on suspected compromise, whichever comes first. Procedure:
+
+1. Generate new key: `node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"`
+2. Run re-encryption migration: read each encrypted field with old key, re-encrypt with new key, write back (batched, during maintenance window)
+3. Tables affected: `ObservabilityEvents` (input, output, stream_raw), `ObservabilityReruns` (input, output, stream_raw), `Users` (mfa_secret, mfa_recovery_codes)
+4. Swap `DB_ENCRYPTION_KEY` env var to new key, restart application
+5. Verify decryption works on a sample of rows
+6. Audit-log the rotation event
+
+Full procedure documented in `ElasticDash-BE/docs/operations/key-rotation.md` (TBD — to be created with runnable script).
+
+**Compliance Status: COMPLIANT**
+
+- Password hashing, MFA secret encryption, API key hashing, and trace data encryption are all strong
+- Key rotation policy defined (annual or on compromise)
+- SDK local snapshots are a minor gap (local dev tooling, opt-in encryption available)
+
+---
+
+### 2.2 TLS 1.2+ Encryption in Transit
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK → LLM Providers** | Compliant | All provider URLs use HTTPS (`proxy/llm-capture.ts:11-16`, `matchers/index.ts:137,276,309`) |
+| **SDK → Backend** | Compliant | Telemetry sent over HTTPS (`telemetry-batcher.ts:97`); Socket.io connects to HTTPS URLs (`socket-connector.ts:35`) |
+| **SDK Local Servers** | N/A (local only) | Dashboard server (`dashboard-server.ts:1478`) and portal server (`portal-server.ts:269`) use HTTP on localhost — acceptable for local dev tooling |
+| **Backend → Dashboard** | Partial | Application runs on HTTP; TLS termination at reverse proxy/load balancer level. Helmet middleware sets HSTS and security headers (`index.js:78-79`) |
+
+**Current State — SDK:**
+
+All external network communication uses HTTPS:
+
+- **LLM providers**: `https://api.openai.com`, `https://api.anthropic.com`, `https://generativelanguage.googleapis.com`, `https://api.x.ai` (`proxy/llm-capture.ts:11-16`)
+- **Backend telemetry**: URL constructed from `serverUrl` config, expected to be HTTPS (`telemetry-batcher.ts:97`)
+- **Socket.io**: Connects with `reconnection: true`, supports WSS when given an HTTPS URL (`socket-connector.ts:35-44`)
+
+Local-only servers (dashboard on port 4573, portal on configurable port) intentionally use HTTP since they bind to localhost and are not exposed externally.
+
+**Current State — Backend (Verified):**
+
+- **Security headers**: `helmet` v8.1.0 middleware sets HSTS, X-Frame-Options, X-Content-Type-Options, CSP, and other security headers (`index.js:78-79`)
+- **HTTP application server**: The Express app uses `createServer(app)` (HTTP, not HTTPS) (`index.js:8`). TLS termination is expected at the reverse proxy/load balancer layer
+- **Proxy trust**: `app.set('trust proxy', 1)` configured for correct IP extraction behind a TLS-terminating proxy (`index.js:18`)
+- **No application-level TLS config**: No certificate/key configuration in the codebase — this is a common architecture pattern where TLS is handled by infrastructure (AWS ALB, CloudFront, nginx, etc.)
+
+**Note:** TLS enforcement depends on proper infrastructure configuration. The application itself delegates TLS to the proxy layer, which must be verified with the infrastructure team.
+
+**Compliance Status: COMPLIANT** (SDK external communication); Backend TLS depends on infrastructure proxy configuration
+
+---
+
+### 2.3 Role-Based Access Controls (RBAC)
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Partial | Bearer token auth for API calls (`ci/api-client.ts:7-12`); origin allowlisting for portal (`portal-server.ts:19-82`) |
+| **Backend** | Compliant | 4 roles (Admin/User/Member/Viewer) in `Roles` table; `hasMinRole()` hierarchical check; project-level ownership on all endpoints; `checkRoleIsNotAllowed()` in auth.js |
+| **Dashboard** | Compliant | 6 role levels in `authRoles.ts` (admin/staff/user/member/viewer/onlyGuest); `AuthGuardRedirect` validates role against route's auth array; httpOnly cookies as primary auth |
+
+**Current State — SDK:**
+
+The SDK implements authentication but not role-based authorization:
+
+- **Bearer token**: Used for all Backend API communication (`ci/api-client.ts:7-12`, `telemetry-batcher.ts:98-99`)
+- **Portal security middleware**: Validates Bearer token OR origin allowlist (`portal-server.ts:172-193`)
+- **Socket.io auth**: Sends `apiKey` and `sessionId` in auth payload; server responds with `auth:ok` and `projectId` (`socket-connector.ts:35-44, 57-60`)
+
+The SDK authenticates *identity* (via API key) but does not enforce *permissions* — that is the Backend's responsibility.
+
+**Current State — Backend (Verified):**
+
+The Backend implements:
+
+- **User roles**: `Roles` table with Admin (id=1), User (id=2), Viewer (id=3), and Member (id=4) roles (`database/init.sql:58-74`). Role is embedded in JWT payload and checked via `hasMinRole()` hierarchical helper (`controller/auth/auth.js:490-491`)
+- **Project-level permissions**: All project endpoints validate `user_id` ownership (`controller/projects/projectController.js:21-27`). API key creation verifies user owns the target project (`controller/apikeys/apiKeyController.js:10-20`)
+- **Endpoint-level authorization**: Routes use `requireJwtAuth` (dashboard) or `requireApiKeyOrJwtAuth` (SDK-facing) middleware for authentication
+
+**Remaining gap:**
+
+- API keys inherit the user's full permissions (no read-only/write/admin scoping on keys)
+
+**Current State — Dashboard (Verified):**
+
+- **Role definitions**: 6 role levels defined in `@auth/authRoles.ts`: `admin`, `staff`, `user`, `member`, `viewer`, `onlyGuest` — each maps to an array of allowed role names for hierarchical checks
+- **Auth guard**: `AuthGuardRedirect` component validates user role against route's auth array via `useCurrentUser()` hook; redirects to `/sign-in` if unauthenticated
+- **Control panel access**: Layout uses `authRoles.viewer` — allows all authenticated roles (admin through viewer) access to the control panel (`app/(control-panel)/layout.tsx`)
+- **Team management UI**: Settings page shows Read/Write/Admin role assignment for team members (`TeamTabView.tsx`)
+- **Primary auth**: httpOnly cookies via `credentials: 'include'` (`utils/api.ts:24`); localStorage retained as cross-origin fallback
+- **Token refresh**: 401 afterResponse hook automatically refreshes expired tokens via `POST /auth/refresh` with infinite-loop prevention (`utils/api.ts:37-113`)
+
+**Remaining gap:**
+
+- Server-side role enforcement is at the Backend API level; Dashboard role checks are client-side (defense in depth, not the sole enforcement layer)
+
+**Compliance Status: COMPLIANT** — 4-role RBAC hierarchy across Backend and Dashboard, httpOnly cookies as primary auth, token refresh mechanism
+
+---
+
+### 2.4 Multi-Factor Authentication (MFA)
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | N/A | SDK uses API keys, not user login |
+| **Backend** | Implemented | TOTP-based MFA with `otplib`, recovery codes, MFA session tokens, rate-limited verification |
+| **Dashboard** | Implemented | TOTP setup with QR code (`MfaSetupForm.tsx`), verification form (`MfaVerifyForm.tsx`), recovery code support, security settings toggle |
+
+**Current State:**
+
+- The SDK authenticates via API keys — MFA is not applicable at the SDK level
+
+**Current State — Backend (Verified):**
+
+The Backend has a complete MFA implementation:
+
+- **TOTP support**: Uses `otplib` v13.4.0 (`controller/auth/mfa.js`) for TOTP generation and verification
+- **MFA setup flow**: `setupMfa()` generates a TOTP secret, QR code URI, and 8 recovery codes (`mfa.js:64-109`)
+- **MFA verification**: `verifyMfaLogin()` validates TOTP codes or recovery codes on login (`mfa.js:173-237`)
+- **Login integration**: Login flow checks `mfa_enabled` and returns an MFA challenge token if enabled (`auth.js:65-80`)
+- **MFA secrets encrypted**: MFA secrets and recovery codes are encrypted at rest using AES-256-GCM before storage (`mfa.js:94-100`, `controller/auth/encryption.js`)
+- **Rate limiting**: MFA verification is rate-limited to 5 attempts per 5 minutes (`index.js:91-100`)
+- **Database schema**: `Users` table includes `mfa_required`, `mfa_enabled`, `mfa_secret`, `mfa_recovery_codes` columns (`database/init.sql:40-43`)
+- **Audit logging**: MFA setup and verification events are logged to the `AuditLogs` table (`mfa.js:102, 205, 228, 235`)
+
+**Current State — Dashboard (Verified):**
+
+- **MFA setup form** (`@auth/forms/MfaSetupForm.tsx`): Displays QR code via `qrcode.react` (`QRCodeSVG`), shows base32 secret with copy-to-clipboard, displays recovery codes, requires 6-digit TOTP verification
+- **MFA verification form** (`@auth/forms/MfaVerifyForm.tsx`): Handles MFA challenge during login; supports both authenticator codes and recovery codes with a toggle
+- **MFA service** (`services/mfaService.ts`): API calls to `POST /auth/mfa/setup`, `POST /auth/mfa/setup/verify`, `POST /auth/mfa/verify`
+- **Security settings** (`SecurityTabView.tsx`): Toggle for "Enable 2-step authentication"
+- **Session handling**: Uses `sessionStorage` for temporary MFA session token; real auth token only stored in `localStorage` after MFA verification
+
+**Minor gaps:**
+
+- Recovery codes displayed as plain text in browser DOM
+- UI mentions SMS-based 2FA but only TOTP is implemented
+
+**Compliance Status: COMPLIANT** — Full MFA flow implemented across Backend and Dashboard
+
+---
+
+### 2.5 Audit Logging
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Compliant | Structured JSON auth failure logging in portal-server; debug logging (`utils/debug.ts`); `X-Correlation-ID` on all requests |
+| **Backend** | Compliant | `AuditLogs` table with tamper-evident SHA-256 hash chain; auth, MFA, API key, PHI data access/modification events; IP and user-agent tracking; correlation IDs; indefinite retention |
+| **Dashboard** | Compliant | Audit log viewer with table, pagination, action/IP filters (`AuditLogTabView.tsx`, `auditLogService.ts`) |
+
+**Current State — SDK (Verified):**
+
+- **Auth failure logging**: Portal security middleware emits structured JSON logs on 401 (invalid API key) and 403 (origin not allowed) with event, reason, IP, userAgent, timestamp (`portal-server.ts:183-205`)
+- **Request correlation**: `X-Correlation-ID` (UUID v4) sent on all requests (`telemetry-batcher.ts:101`, `ci/api-client.ts:12`) for end-to-end audit trail tracing
+- **Debug logging**: Conditional on `ELASTICDASH_DEBUG=1`, outputs to stdout (`utils/debug.ts:1-8`)
+- **Data redaction**: Sensitive fields redacted before transmission (`utils/redact.ts:1-25`, applied in `telemetry-batcher.ts:76-78`)
+
+**Current State — Backend (Verified):**
+
+The Backend has a structured audit logging system:
+
+- **AuditLogs table** (`database/init.sql:895-917`): Persistent storage with columns for `action`, `actor_id`, `actor_type`, `resource_type`, `resource_id`, `ip_address`, `user_agent`, `status`, `detail`, `created_at`
+- **Audit logger** (`controller/audit/auditLogger.js`): Fire-and-forget `logAudit()` function that captures IP address and user agent from requests
+- **Events logged**:
+  - Login success/failure: `logAudit('login.success', ...)` (`auth.js:108,147`)
+  - MFA setup: `logAudit('mfa.setup', ...)` (`mfa.js:102`)
+  - MFA verification success/failure: `logAudit('mfa.verify', ...)` (`mfa.js:205, 228, 235`)
+  - API key create/revoke/delete: `logAudit('apikey.create', ...)`, `logAudit('apikey.revoke', ...)`, `logAudit('apikey.delete', ...)` (`apiKeyController.js:29, 57, 76`)
+  - PHI data access: Event ingestion (`services/observability.js:233,550`)
+  - Data modification: Project CRUD (`services/project.js:35,84,102`), test group CRUD + run submit (`services/testGroup.js:57,107,146,420`)
+  - Account lockout: `logAudit('account.locked', ...)` on brute force detection
+- **Tamper-evident hash chain**: SHA-256 hash chaining in `auditLogger.js:22-26`. Each entry stores `previous_hash`. `verifyAuditHashChain()` validates chain integrity (`auditLogger.js:75-94`). Admin verification endpoint at `GET /api/audit-logs/verify-chain`
+- **Indexed for queries**: Indexes on `(actor_id, created_at)`, `(action, created_at)`, `(ip_address, created_at)` for efficient lookups
+- **Retention**: Audit logs retained indefinitely (never deleted). HIPAA 6-year requirement satisfied by default
+
+**Current State — Dashboard (Verified):**
+
+- **Audit log viewer**: `AuditLogTabView.tsx` with table display, pagination (25/page), action and IP filters
+- **Audit log service**: `auditLogService.ts` calls `GET /api/audit-logs` via Backend API
+
+**Compliance Status: COMPLIANT** — Comprehensive audit logging across all components with tamper-evident hash chain, correlation IDs, and indefinite retention
+
+---
+
+### 2.6 Intrusion Detection Systems (IDS)
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Minimal | Client-side retry with backoff on 429/5xx (`telemetry-batcher.ts:114-122`) |
+| **Backend** | Compliant | Rate limiting on login (10/60s), MFA (5/5min), PHI ingestion (300/min); brute force lockout (5 attempts/15min); 24h JWT; WAF rules documented |
+| **Dashboard** | Compliant | CSP (env-aware), X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy via Next.js `middleware.ts` |
+
+**Current State — SDK:**
+
+The SDK handles rate limiting *responses* but does not implement server-side rate limiting:
+
+- **429 handling**: Exponential backoff (1s, 2s, 4s) with max 3 retries (`telemetry-batcher.ts:114-122`)
+- **Portal result delivery**: 3 attempts with exponential backoff (`portal-server.ts:146-163`)
+- **Socket.io reconnection**: Auto-reconnect with 1s initial delay, 30s max (`socket-connector.ts:40-43`)
+- **Request body size limit**: 10 MB limit on portal server (`portal-server.ts:200`)
+
+**Current State — Backend (Verified):**
+
+The Backend implements rate limiting on sensitive endpoints:
+
+- **Login rate limiting**: 10 attempts per 60 seconds via `express-rate-limit` v8.3.2 (`index.js:86-92`)
+- **MFA rate limiting**: 5 attempts per 5 minutes (`index.js:95-104`)
+- **PHI ingestion rate limiting**: 300 requests per minute on `/api/observability` and `/api/testgroups` (`index.js:115-116`)
+- **Brute force detection**: `LoginAttempts` table tracks consecutive failures. 5 failures trigger 15-minute lockout. Reset on success. Lockout events logged via `logAudit('account.locked', ...)` (`auth.js:439-484`, `database/init.sql:1007-1024`)
+- **JWT lifetime**: 24-hour expiry with refresh token rotation (`auth.js:520`, `RefreshTokens` table)
+- **Proxy trust**: `app.set('trust proxy', 1)` for correct IP extraction behind load balancers (`index.js:19`)
+- **Security headers**: `helmet` v8.1.0 middleware applied (`index.js:80`)
+- **WAF rules**: Documented in `docs/infrastructure/waf-ids.md` — AWS managed rule groups (OWASP, SQLi, bad inputs, IP reputation, bot control), custom rate limit rules. Infra team should verify WAF is attached to ALB.
+
+**Current State — Dashboard (Verified):**
+
+- **Security headers middleware**: Next.js `middleware.ts` sets CSP (env-aware), X-Frame-Options (DENY), X-Content-Type-Options (nosniff), Referrer-Policy (strict-origin-when-cross-origin), Permissions-Policy (camera/microphone/geolocation disabled) (`middleware.ts:6-41`)
+
+**Compliance Status: COMPLIANT** — Rate limiting on auth and PHI endpoints, brute force lockout, 24h JWT, security headers on Dashboard, WAF documented
+
+---
+
+### 2.7 Environment Isolation
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Compliant | Strong env var separation, subprocess isolation, dotenv support |
+| **Backend** | Implemented | Separate `.env` files per environment; AWS Secrets Manager for sensitive keys; configurable database connections |
+| **Dashboard** | Mostly Compliant | Environment-specific `.env` files (`.env.dev.local`, `.env.staging.local`, `.env.prod.local`); `NEXT_PUBLIC_BASE_URL` configures API endpoint; `next.config.ts` warns if dev URL used in production |
+
+**Current State — SDK:**
+
+Environment isolation is well implemented:
+
+- **dotenv support**: `.env` loaded at startup (`cli.ts:2`); `.env` files excluded from version control
+- **Configurable endpoints**: `ELASTICDASH_API_URL` allows pointing to different environments (`cli.ts:315`, `observability.ts:47`)
+- **Subprocess isolation**: Child processes receive controlled environment, sensitive vars deleted after use (`tool-runner-worker.ts:1-4`)
+- **CLI validation**: Required config validated before execution (`cli.ts:319-321, 376-378, 415-417`)
+- **No hardcoded secrets**: All credentials come from environment variables or CLI arguments
+
+**Environment Variables (comprehensive list):**
+
+| Variable | Purpose | Required |
+|----------|---------|----------|
+| `ELASTICDASH_API_URL` | Backend API endpoint | Yes (for observability/CI) |
+| `ELASTICDASH_API_KEY` | Authentication token | Yes (for auth-protected endpoints) |
+| `ELASTICDASH_SESSION_ID` | Session identifier | No (auto-generated) |
+| `ELASTICDASH_PORT` | Dashboard local port | No (default: 4573) |
+| `ELASTICDASH_PORTAL_PORT` | Portal local port | No (auto-assigned) |
+| `ELASTICDASH_ALLOWED_ORIGINS` | CORS allowlist | No |
+| `ELASTICDASH_BROWSER_UI` | Enable browser UI | No (default: '0') |
+| `ELASTICDASH_HTTP_DRAIN_MS` | HTTP drain timeout | No (default: 300ms) |
+| `ELASTICDASH_DEBUG` | Enable debug logging | No (default: off) |
+| `ELASTICDASH_LLM_PROXY` | Enable LLM proxy | No |
+| `ELASTICDASH_TRACE_ID` | Trace ID for test runs | No (set internally) |
+| `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY` | AES-256 key (hex) for snapshot encryption | No (opt-in) |
+
+**Current State — Backend (Verified):**
+
+- **Environment-specific config files**: Separate `.env`, `.env.staging`, `.env.production` files for each environment
+- **Config loader**: `config.js` uses `dotenv` to load environment variables; checks for Kubernetes environment; validates required variables (`S3_BUCKET_NAME`, `DB_CONNECTION_STRING`, `AWS_SQS_URL_TEST`)
+- **AWS Secrets Manager**: AI API keys stored in AWS Secrets Manager (`controller/observability/secretsManager.js`) using per-project key paths (`elasticdash/observability/ai-key/project-{projectId}`), not in database or env files
+- **Database connection**: Configurable via `DB_CONNECTION_STRING` or individual `DB_HOST`/`DB_NAME`/`DB_USER` variables (`postgres.js`)
+- **Field-level encryption key**: `DB_ENCRYPTION_KEY` environment variable for AES-256-GCM encryption of sensitive fields (`controller/auth/encryption.js`)
+
+**Current State — Dashboard (Verified):**
+
+- **Environment-specific config**: Separate `.env`, `.env.dev.local`, `.env.staging.local`, `.env.prod.local` files
+- **API URL configuration**: `NEXT_PUBLIC_BASE_URL` environment variable points to the correct backend per environment (e.g., `https://devserver.elasticdash.com/api` for dev)
+- **Environment tracking**: `CURRENT_ENV` variable tracks active environment
+
+**Dashboard notes:**
+
+- **Runtime validation**: `next.config.ts` warns if `NEXT_PUBLIC_BASE_URL` contains 'devserver' in production builds (`next.config.ts:3-4`). Warning only — does not fail the build.
+- `NEXT_PUBLIC_*` variables are exposed to the browser (expected for Next.js, but no secrets should use this prefix)
+
+**Compliance Status: COMPLIANT** (all components)
+
+---
+
+## 3. Remediation Plan — Section 12 Compliance TODOs
+
+Consolidated from internal audit findings and external Section 12 review. Items are ordered by what blocks HIPAA/BAA sign-off.
+
+**Last verified: 2026-04-18**
+
+---
+
+### Tier 1: Section 12 Blockers (must fix before PHI processing)
+
+| # | Gap | Component | Status | Details |
+|---|-----|-----------|--------|---------|
+| 1 | **API keys stored in plaintext** | Backend | **DONE** | SHA-256 hashing in `apiKeyController.js`. `key_prefix` column for display. `checkApiKey()` hashes before DB lookup. |
+| 2 | **Auth token in localStorage (XSS vulnerable)** | Dashboard + Backend | **DONE (hybrid)** | Backend sets httpOnly cookies via `cookie-parser` + `setAuthCookie()`. Dashboard uses `credentials: 'include'`. localStorage retained as cross-origin fallback — primary auth is now cookie-based. |
+| 3 | **No audit logging on PHI data access/modification** | Backend | **DONE** | `logAudit()` calls added to `services/observability.js` (event ingestion), `services/testGroup.js` (CRUD + run submit), `services/project.js` (CRUD). |
+
+### Tier 2: Infrastructure Documentation (must provide evidence, not necessarily code changes)
+
+| # | Gap | Component | Status | Details |
+|---|-----|-----------|--------|---------|
+| 4 | **Database encryption at rest unverified** | Infra | **DONE** | Documented in `ElasticDash-BE/docs/infrastructure/database-encryption.md`. RDS AES-256 encryption with KMS, field-level encryption inventory, verification commands. Infra team should verify checklist items against live instance. |
+| 5 | **TLS 1.2+ enforcement unverified** | Infra | **DONE** | Documented in `ElasticDash-BE/docs/infrastructure/tls-configuration.md`. ALB TLS 1.2+ with recommended policy `ELBSecurityPolicy-TLS13-1-2-2021-06`, cipher suites, RDS SSL, HSTS via Helmet, verification checklist. |
+| 6 | **PostgreSQL SSL certificate validation disabled** | Backend / Infra | **DONE** | `postgres.js` now uses configurable `DB_SSL_REJECT_UNAUTHORIZED` (defaults to `true`) and `DB_SSL_CA` for custom CA cert path. |
+| 7 | **WAF/IDS configuration unverified** | Infra | **DONE** | Documented in `ElasticDash-BE/docs/infrastructure/waf-ids.md`. AWS managed rule groups (OWASP, SQLi, bad inputs, IP reputation, bot control), custom rate limit rules, brute force detection, audit log monitoring queries, hash chain verification. Infra team should verify WAF is attached to ALB. |
+| 8 | **Audit log retention policy** | Backend / Infra | **DONE** | Audit logs are retained indefinitely (never deleted). Data rotation only affects observability traces (30-day for Juno accounts). HIPAA 6-year requirement satisfied by default. |
+
+### Tier 3: High Priority Code Changes
+
+| # | Gap | Component | Status | Details |
+|---|-----|-----------|--------|---------|
+| 9 | **JWT token lifetime too long** | Backend | **DONE** | `expiresIn` changed from `60d` to `24h`. Influencer role now also uses 24h expiry. |
+| 10 | **No token refresh mechanism** | Dashboard + Backend | **DONE** | `RefreshTokens` table with SHA-256 hashed tokens, 30-day expiry. `POST /auth/refresh` endpoint with token rotation. Dashboard has 401 afterResponse hook with retry and infinite-loop prevention. |
+| 11 | **Rate limiting on PHI ingestion path** | Backend | **DONE** | `express-rate-limit` applied to `/api/observability` and `/api/testgroups` at 300 req/min per API key/IP. |
+| 12 | **Add CSP and security headers to Dashboard** | Dashboard | **DONE** | Next.js `middleware.ts` sets CSP (env-aware), X-Frame-Options, X-Content-Type-Options, Referrer-Policy, X-XSS-Protection, Permissions-Policy. |
+| 13 | **RBAC role granularity** | Backend + Dashboard | **DONE** | `Viewer` (id=3) and `Member` (id=4) roles added. `hasMinRole()` hierarchical helper in `auth.js`. Dashboard `authRoles.ts` updated with `viewer` and `member`. `AuthGuardRedirect` validates user role against route's auth array. |
+| 14 | **Add SDK auth failure logging** | SDK | **DONE** | `portal-server.ts` `securityMiddleware` emits structured JSON logs on 401 (invalid API key) and 403 (origin not allowed) with event, reason, IP, userAgent, timestamp. |
+| 15 | **Add `DB_ENCRYPTION_KEY` to `.env.example`** | Backend | **DONE** | `DB_ENCRYPTION_KEY`, `DB_SSL`, `DB_SSL_REJECT_UNAUTHORIZED`, `DB_SSL_CA` all documented in `.env.example`. |
+
+### Tier 4: Medium Priority
+
+| # | Gap | Component | Status | Details |
+|---|-----|-----------|--------|---------|
+| 16 | **Add audit log viewer to Dashboard** | Dashboard | **DONE** | `AuditLogTabView.tsx` with table, pagination (25/page), action/IP filters. `auditLogService.ts` calls `GET /api/audit-logs`. |
+| 17 | **Add brute force detection** | Backend | **DONE** | `LoginAttempts` table. 5 consecutive failures → 15-minute lockout. Reset on success. Lockout events logged via `logAudit('account.locked', ...)`. |
+| 18 | **Encrypt SDK local snapshots** | SDK | **DONE** | AES-256-GCM encryption via `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY` env var. `encryptSnapshot()`/`decryptSnapshot()` in `dashboard-server.ts`. Opt-in; gracefully handles legacy unencrypted snapshots. |
+| 19 | **Add request correlation IDs** | SDK + Backend | **DONE** | SDK sends `X-Correlation-ID` (UUID v4) on all requests (`telemetry-batcher.ts`, `ci/api-client.ts`). Backend middleware reads/generates correlation ID, includes in audit logs. |
+| 20 | **Dashboard runtime environment validation** | Dashboard | **DONE (partial)** | `next.config.ts` warns if `NEXT_PUBLIC_BASE_URL` contains 'devserver' in production. Warning only, does not fail the build. |
+| 21 | **Audit log tamper-evidence** | Backend | **DONE** | SHA-256 hash chaining in `auditLogger.js`. Each entry stores `previous_hash`. `verifyAuditHashChain()` function validates chain integrity. `GET /api/audit-logs/verify-chain` endpoint for admin verification. |
+| 22 | **Encrypt trace data at rest** | Backend | **DONE** | AES-256-GCM encryption of `input`, `output`, `stream_raw` in `ObservabilityEvents` and `ObservabilityReruns` via `encryptJSON()`/`encryptField()`. Decrypted on read via `decryptEventRow()`. `has_error` boolean denormalizes error checks. Backward compatible with legacy unencrypted JSONB rows. Migration: `database/migrations/add_has_error_columns.sql`. |
+| 23 | **Complete account termination purge** | Backend | **DONE** | `purgeUserData()` now deletes all 30 tables in FK-safe order within a single transaction. Added 7 missing tables: LoginAttempts, BillingAddresses, CreditCards, EmailVerifyLinks, ResetPasswordUrls, Notifications, UnsubscriptionUniqueUrls. AuditLogs intentionally retained (HIPAA). Endpoint: `DELETE /admin/user/purge/:userId`. |
+
+---
+
+## 4. Summary Compliance Matrix
+
+**Last verified: 2026-04-18**
+
+| Standard | SDK | Backend | Dashboard | Overall | Section 12 Ready? |
+|----------|-----|---------|-----------|---------|-------------------|
+| **AES-256 Encryption at Rest** | Compliant (opt-in AES-256-GCM snapshot encryption via `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY`) | Compliant (PBKDF2 passwords, AES-256 MFA secrets, SHA-256 API key hashing, AES-256-GCM trace data encryption on input/output/stream_raw; RDS AES-256 via KMS) | N/A (stateless) | **Compliant** | **Yes** — documented in `docs/infrastructure/database-encryption.md`; verify checklist against live instance |
+| **TLS 1.2+ Encryption in Transit** | Compliant (all external HTTPS) | Compliant (Helmet HSTS headers; configurable DB SSL with `rejectUnauthorized` default true; TLS at proxy) | Compliant (HTTPS URLs; CSP + security headers via middleware.ts) | **Compliant** | **Yes** — documented in `docs/infrastructure/tls-configuration.md`; verify checklist against live ALB |
+| **Role-Based Access Controls** | Partial (auth, no roles) | Compliant (Admin/User/Member/Viewer roles, `hasMinRole()` hierarchy, project ownership) | Compliant (httpOnly cookies primary auth; client-side role validation with viewer/member/admin enforcement) | **Compliant** | **Yes** |
+| **Multi-Factor Authentication** | N/A (API key auth) | Compliant (TOTP, recovery codes, rate-limited, encrypted secrets) | Compliant (QR setup, TOTP verification, recovery codes) | **Compliant** | **Yes** |
+| **Audit Logging** | Compliant (structured JSON auth failure logging in portal-server) | Compliant (auth/MFA/key/PHI events logged; correlation IDs; tamper-evident hash chain; audit log API with admin verification; retention indefinite) | Compliant (audit log viewer with filtering/pagination) | **Compliant** | **Yes** |
+| **Intrusion Detection Systems** | Minimal (client-side backoff) | Compliant (auth rate limiting 10/min; MFA 5/5min; PHI ingestion 300/min; brute force lockout 5 attempts/15min; 24h JWT; WAF rules documented) | Compliant (CSP, X-Frame-Options, X-Content-Type-Options, Permissions-Policy) | **Compliant** | **Yes** — documented in `docs/infrastructure/waf-ids.md`; verify WAF attached to ALB |
+| **Environment Isolation** | Compliant | Compliant (dotenv, Secrets Manager, per-env configs) | Mostly Compliant (env configs; runtime validation warns on dev URL in prod) | **Compliant** | **Yes** |
+
+**Legend:**
+
+- **Compliant**: Meets the standard with evidence
+- **Mostly Compliant**: Core measures implemented; minor gaps identified
+- **Borderline**: Compliant at the application level; pending infrastructure documentation
+- **Partial**: Some measures in place; gaps remain
+- **N/A**: Not applicable to this component
+
+**Section 12 sign-off status:** All 23 items are complete — code changes and infrastructure documentation. The infra docs (`ElasticDash-BE/docs/infrastructure/`) contain verification checklists that should be confirmed against the live AWS environment by the infrastructure team.
+
+---
+
+## 5. Positive Security Features Already in Place
+
+The following security features are already implemented and should be maintained:
+
+1. **Data redaction**: Case-insensitive key-based redaction of sensitive fields before transmission (`utils/redact.ts`, applied in `telemetry-batcher.ts:76-78`)
+2. **Bearer token authentication**: Consistent pattern across all SDK→Backend communication
+3. **Origin allowlisting**: Portal server validates request origins with subdomain support (`portal-server.ts:19-82`)
+4. **Input validation**: Request body size limits (10 MB), required field validation on portal endpoints (`portal-server.ts:200-231`)
+5. **Exponential backoff**: Retry logic with backoff prevents thundering herd on Backend failures
+6. **Subprocess isolation**: Child processes receive controlled environment variables
+7. **No hardcoded secrets**: All credentials sourced from environment variables
+8. **Socket.io auth handshake**: Explicit auth event with project-level confirmation (`socket-connector.ts:57-60`)
+
+---
+
+## 6. BAA-Specific Controls (Sections 4, 6, 8, 9, 11)
+
+**Last verified: 2026-04-18**
+
+The following BAA sections require controls beyond the standard Section 7 safeguards.
+
+### 6.1 Section 4 — Minimum Necessary (Data Minimization)
+
+| Component | Status | Details |
+|-----------|--------|---------|
+| **SDK** | Compliant (opt-in) | `redactKeys` config strips specified fields from `input`/`output` before transmission; key-based, case-insensitive (`utils/redact.ts`, `telemetry-batcher.ts:77-79`). Structural metadata (type, name, timestamp, usage, model) always passes through. |
+| **Backend** | Compliant | AES-256-GCM field-level encryption on trace `input`, `output`, `stream_raw` fields at rest (`controller/auth/encryption.js`). Data decrypted only on read. |
+
+**How it works:**
+
+1. **SDK-side redaction** (before transmission): The SDK's `initObservability()` accepts a `redactKeys` option — an array of field names to strip from event `input` and `output` before batching. This is applied in `telemetry-batcher.ts:77-79` using `redactPayload()`. Matching values are replaced with `"[REDACTED]"`.
+
+2. **Backend-side encryption** (at rest): All trace event `input`, `output`, and `stream_raw` fields are encrypted with AES-256-GCM before storage using `DB_ENCRYPTION_KEY`. Decryption occurs only when data is read via authorized API endpoints.
+
+**Boundary assumption:**
+
+Free-text PHI in LLM prompts/responses (e.g., patient names in conversation content) is **the client's responsibility** to sanitize prior to transmission. The BAA states identifiers must be "removed or irreversibly masked **prior to transmission**." The SDK provides the `redactKeys` mechanism as a safety net for structured field names, but content-aware PII scanning of free-text conversation content is outside the SDK's scope. Clients handling PHI (e.g., SharedGenes) must run their own PII removal pipeline before the data reaches ElasticDash instrumentation.
+
+This is consistent with the BAA's allocation of responsibility: the data sender (Juno/SharedGenes) sanitizes; ElasticDash validates and encrypts.
+
+---
+
+### 6.2 Section 6 — Prohibited Uses (ML Training & Re-identification)
+
+| Requirement | Status | Details |
+|-------------|--------|---------|
+| **6(i) — No ML training on PHI** | Compliant | No data export endpoints, ETL pipelines, or ML training code exists in the codebase. LLM judge evaluations use the client's own API keys and are scoped per-project. No trace data flows to analytics platforms. |
+| **6(v) — Re-identification prohibition** | Compliant (technical controls) | Trace data is encrypted at rest (AES-256-GCM). API keys are hashed (SHA-256). Access requires authenticated API key or JWT with project-scoped authorization. No bulk data export endpoints exist. |
+
+**Technical barriers to re-identification:**
+
+- Trace data encrypted at field level — cannot be read without `DB_ENCRYPTION_KEY`
+- Project isolation — users can only access their own project's data via ownership check
+- No data export/download endpoints — no way to bulk-extract trace data
+- API key hashing — credential material is not reversible
+- Audit logging — all data access is logged with actor, IP, and correlation ID
+
+**Policy note:** A written internal policy prohibiting use of production trace data for ML training, analytics, or research should be maintained alongside these technical controls.
+
+---
+
+### 6.3 Section 8 — Deterministic Replay Controls
+
+| Requirement | Status | Details |
+|-------------|--------|---------|
+| **Replay isolation** | Compliant | SDK replay mode (`capture/replay.ts`) intercepts all outbound calls (HTTP, AI, DB, tools, side effects) and returns recorded responses. `shouldReplay()` check blocks real API calls. |
+| **No live API calls during replay** | Compliant | All interceptors (`http.ts:342-362`, `ai-interceptor.ts:677`, `db.ts:90`, `tool.ts:193`, `workflow-ai.ts:250`, `side-effects.ts:22,59`) check `replay.shouldReplay(id)` before executing. If replaying, `synthesizeFrozenResponse()` returns cached data without calling the real endpoint. |
+| **Replay activity logged** | Compliant | Replay/rerun events are tagged with `isRerun: true` in the observability context (`telemetry-push.ts:314-316`). Backend filters rerun events from live ingestion (`services/observability.js:147-148`). Rerun creation is audit-logged: `logAudit('observability.rerun.create', ...)` (`services/observability.js:550`). |
+| **Replay datasets encrypted** | Compliant | Frozen events stored in the same AES-256-GCM encrypted trace tables. Local replay snapshots encrypted when `ELASTICDASH_SNAPSHOT_ENCRYPTION_KEY` is set (`dashboard-server.ts`). |
+| **Replay access controlled** | Compliant | Replay/rerun endpoints require authenticated API key or JWT with project ownership validation. |
+
+**Replay isolation mechanism:**
+
+```
+SDK Replay Mode
+├─ ReplayController.shouldReplay(eventId) checks each intercepted call
+├─ If replaying: return synthesizeFrozenResponse(historicalEvent) — NO outbound call
+├─ If not replaying: execute live call normally
+└─ All interceptors covered: HTTP, AI, DB, tools, Math.random(), Date.now()
+```
+
+**Backend-side rerun isolation:**
+
+- Frozen events fetched via `getFrozenEventsForTrace()` (limit 50 per trace)
+- Rerun results stored separately in `ObservabilityReruns` table
+- Events marked `isRerun: true` are filtered from live event ingestion
+- Rerun creation audit-logged with actor, resource, and correlation ID
+
+---
+
+### 6.4 Section 9 — Subcontractors
+
+All third-party services that may process or access data are inventoried below:
+
+| Subcontractor | Service | Data Exposure | BAA Status |
+|---------------|---------|---------------|------------|
+| **AWS (RDS, S3, SQS, Secrets Manager, KMS, ALB)** | Infrastructure, encryption, storage | Full — hosts all data | **Signed.** AWS BAA executed via AWS Artifact. |
+| **OpenAI / Anthropic / Google / xAI / Moonshot** | LLM evaluation (judge models) | Trace outputs sent for evaluation | Uses **client's own API keys** — client's BAA with provider applies. ElasticDash does not hold separate keys for PHI evaluation. |
+| **SendPulse** | Email delivery (verification, notifications) | Email addresses, failure notification content | Does NOT process PHI trace data. Notification emails contain evaluation results (pass/fail), not raw trace content. |
+| **Stripe** | Payment processing | Billing data only | Standard Stripe BAA available. No PHI exposure. |
+
+**Not present** (verified via `package.json` and codebase search):
+- No Sentry, Datadog, Bugsnag, LogRocket, or similar error tracking services
+- No Mixpanel, Segment, Google Analytics, or similar analytics services
+- No third-party log shipping or monitoring that receives application data
+
+**Key design choice:** LLM judge evaluations use the **client's own API keys** (stored in AWS Secrets Manager per project). ElasticDash proxies the evaluation request but does not hold its own keys for evaluating client data. This means the client's existing BAA with their LLM provider covers the evaluation data flow.
+
+---
+
+### 6.5 Section 11 — Data Retention & Deletion
+
+| Requirement | Status | Details |
+|-------------|--------|---------|
+| **30-day trace data retention** | Compliant (Juno accounts) | `dataRotationController.js` hard-deletes (`DELETE FROM`) trace data older than 30 days for Juno accounts. Runs on startup + every 24 hours. Cascading FK-safe deletion order. |
+| **Deletion mechanism** | Hard delete for traces and termination; soft delete for routine user removal | Trace data: permanent `DELETE FROM` in FK-safe order. Account termination: `purgeUserData()` hard-deletes all data across 30 tables in a single transaction (`controller/administration/user.js`). Routine user removal: soft delete (`deleted = true`). |
+| **Audit log retention** | Indefinite | Audit logs are intentionally excluded from purge — HIPAA 6-year minimum. The purge action itself is audit-logged after the transaction. |
+| **Deletion on termination** | Compliant | `DELETE /admin/user/purge/:userId` (admin-only) executes `purgeUserData()` which hard-deletes all user data across 30 tables. Audit logs retained (HIPAA). Backups expire within 7-day RDS retention window; deletion certification discloses the backup tail. |
+
+**Current deletion flow (Juno accounts):**
+
+```sql
+-- Executed in order, within a transaction:
+DELETE FROM TriggerStepResults WHERE trigger_id IN (... older than 30 days ...)
+DELETE FROM ObservabilityReruns WHERE session_id IN (...)
+DELETE FROM Triggers WHERE session_id IN (...)
+DELETE FROM ObservabilityEvents WHERE session_id IN (...)
+DELETE FROM ObservabilitySessions WHERE id IN (...)
+```
+
+**Termination deletion process (`purgeUserData()`):**
+
+Admin calls `DELETE /admin/user/purge/:userId`. The function executes in a single transaction:
+
+1. Resolve all project IDs for the user
+2. Delete observability cascade: TriggerStepResults → ObservabilityReruns → Triggers → ObservabilityEvents → ObservabilitySessions
+3. Delete test group cascade: TestGroupSingleRuns → TestGroupExpectationResults → TestGroupRuns → TestGroupExpectations → TestGroupTests → TestGroups
+4. Delete project configs: SamplingConfigs, WarningAcks, ExpectationSuppressions, ObservabilityProjectConfig, ManualCatalogEntries, ObservabilityDiscoveredSteps
+5. Delete user-level data: UserApiKeys, RefreshTokens, Projects, UserLlmConfig, UserLlmDefaultEvaluator
+6. Delete peripheral data: LoginAttempts, BillingAddresses, CreditCards, EmailVerifyLinks, ResetPasswordUrls, Notifications, UnsubscriptionUniqueUrls
+7. Delete Users record
+8. Audit log the purge action (outside transaction — audit logs intentionally retained)
+
+Returns `{ success, userId, projectsDeleted, deletedAt }` as deletion certification.
+
+**Backup tail disclosure:** Automated RDS backups are retained for 7 days (configured at the instance level — verify via `aws rds describe-db-instances --query 'DBInstances[0].BackupRetentionPeriod'`). Purged PHI may persist in backups created before the purge for up to 7 days after the purge timestamp. Manual snapshots, if any exist, must be deleted separately. The deletion certification provided to the client states: "All live data for user {userId} was permanently deleted from the production database at {deletedAt}. Automated backups containing pre-deletion data will expire within 7 days. No manual snapshots of user data are retained." Trace data within backups remains encrypted via `DB_ENCRYPTION_KEY` (AES-256-GCM) and RDS-level AES-256 encryption.