@celilo/cli 0.1.0

package/README.md

@@ -0,0 +1,1566 @@
+# Celilo Backend
+
+Phase 0 implementation of Celilo's backend services.
+
+## Phase 0 Scope
+
+CLI-only interface for:
+- Module import and validation
+- Manifest validation
+- Variable resolution
+- Template generation
+- Secret management
+- **Zero-configuration modules** (Phase 0 Part 4):
+  - Automatic hostname/zone assignment from well-known capabilities
+  - Automatic VMID/IP allocation (IPAM)
+  - Automatic network config derivation from zones
+  - Automatic VM resource defaults from manifests
+
+## Project Structure
+
+```
+celilo/backend/
+├── src/
+│   ├── db/
+│   │   ├── schema.ts          # Drizzle ORM schema
+│   │   ├── schema.test.ts     # Schema tests
+│   │   ├── client.ts          # Database connection
+│   │   └── migrate.ts         # Migration runner
+│   ├── manifest/
+│   │   ├── schema.ts          # Zod schemas for manifest validation
+│   │   ├── validate.ts        # Manifest validation logic
+│   │   └── validate.test.ts   # Validation tests
+│   ├── module/
+│   │   ├── import.ts          # Module import logic
+│   │   └── import.test.ts     # Import tests
+│   ├── variables/
+│   │   ├── types.ts           # Variable resolution types
+│   │   ├── parser.ts          # Variable parsing logic
+│   │   ├── parser.test.ts     # Parser tests
+│   │   ├── resolver.ts        # Variable resolution logic
+│   │   ├── resolver.test.ts   # Resolver tests
+│   │   ├── context.ts         # Resolution context builder
+│   │   └── context.test.ts    # Context tests
+│   ├── secrets/
+│   │   ├── master-key.ts      # Master key generation and management
+│   │   ├── master-key.test.ts # Master key tests
+│   │   ├── encryption.ts      # AES-256-GCM encryption/decryption
+│   │   └── encryption.test.ts # Encryption tests
+│   ├── templates/
+│   │   ├── types.ts           # Template generation types
+│   │   ├── generator.ts       # Template generation logic
+│   │   └── generator.test.ts  # Generation tests
+│   └── index.ts               # Entry point
+├── drizzle/                   # Generated migrations
+├── package.json
+├── tsconfig.json
+├── biome.json                 # Linting/formatting config
+└── drizzle.config.ts          # Drizzle Kit config
+```
+
+## Database Schema
+
+### Tables
+
+**`modules`** - Module metadata and manifest data
+- `id` (TEXT, PK) - Module identifier
+- `name` (TEXT) - Display name
+- `version` (TEXT) - Semantic version
+- `description` (TEXT) - Optional description
+- `state` (TEXT) - Lifecycle state (IMPORTED, VALIDATED, CONFIGURED, etc.)
+- `manifest_data` (JSON) - Full manifest content
+- `source_path` (TEXT) - Original import path
+- `imported_at` (TIMESTAMP) - Import timestamp
+- `updated_at` (TIMESTAMP) - Last update timestamp
+- `error_message` (TEXT) - Error details if state is ERROR
+
+**`module_configs`** - User configuration key-value pairs
+- `id` (INTEGER, PK, AUTO)
+- `module_id` (TEXT, FK → modules.id) - Module reference
+- `key` (TEXT) - Configuration key
+- `value` (TEXT) - Configuration value
+- `created_at` (TIMESTAMP)
+- `updated_at` (TIMESTAMP)
+
+**`capabilities`** - Registered capabilities from modules
+- `id` (INTEGER, PK, AUTO)
+- `module_id` (TEXT, FK → modules.id) - Provider module
+- `capability_name` (TEXT) - Capability identifier (e.g., dns_external)
+- `version` (TEXT) - Capability version
+- `data` (JSON) - Capability data (nameserver, zone, etc.)
+- `registered_at` (TIMESTAMP)
+
+**`secrets`** - Encrypted secrets per module
+- `id` (INTEGER, PK, AUTO)
+- `module_id` (TEXT, FK → modules.id) - Module reference
+- `name` (TEXT) - Secret name
+- `encrypted_value` (TEXT) - AES-256-GCM encrypted value
+- `iv` (TEXT) - Initialization vector
+- `auth_tag` (TEXT) - Authentication tag
+- `created_at` (TIMESTAMP)
+- `updated_at` (TIMESTAMP)
+
+All tables use CASCADE DELETE when module is removed.
+
+## Dependencies
+
+### Required
+
+- **Bun** (v1.0+) - JavaScript runtime
+- **Ansible** (v2.9+) - Required for Ansible Vault secret encryption
+- **Terraform** (v1.0+) - Required for validating generated Terraform code
+
+See [../SETUP.md](../SETUP.md) for installation instructions.
+
+## Setup
+
+```bash
+# Install dependencies
+bun install
+
+# Generate migrations (if schema changed)
+bunx drizzle-kit generate
+
+# Run migrations
+bun run db:migrate
+
+# Run tests
+bun test                    # Unit tests (216 tests)
+bun run test:integration    # Integration tests (56 tests)
+
+# Lint and format
+bun run lint:fix
+bun run format
+```
+
+## Development
+
+### Common Development Workflows
+
+**Full development cycle**:
+```bash
+# 1. Make code changes
+vim src/module/import.ts
+
+# 2. Run unit tests (< 1s)
+bun test src/module/import.test.ts
+
+# 3. Run all unit tests (includes zero-config integration tests)
+bun test
+
+# 4. Run CLI-based integration tests (10-20s)
+bun run test:integration
+
+# 5. Before commit: Run all tests
+bun test && bun run test:integration
+
+# 6. Before major release: Run slow tests (3-5min)
+bun run test:integration-slow
+```
+
+**Working with the CLI**:
+```bash
+# Use the wrapper script (works from any directory)
+../../celilo module list
+
+# Or run directly from backend
+bun run src/cli/index.ts module list
+
+# Or use bun link for system-wide command
+bun link
+celilo module list  # From anywhere
+```
+
+**Testing module changes**:
+```bash
+# 1. Import test module
+bun run src/cli/index.ts module import ../../modules/test-module
+
+# 2. Configure
+bun run src/cli/index.ts module config set test-module hostname test-app
+bun run src/cli/index.ts module config set test-module container_ip 10.0.20.100
+
+# 3. Generate
+bun run src/cli/index.ts module generate test-module
+
+# 4. Inspect generated files
+ls -la /tmp/celilo/modules/test-module/generated/
+
+# 5. Clean up
+bun run src/cli/index.ts module remove test-module
+```
+
+**Working with secrets**:
+```bash
+# Set module secret
+bun run src/cli/index.ts secret set homebridge api_key test-key-123
+
+# Get Ansible Vault password (for inspecting secrets.yml)
+bun run src/cli/index.ts system vault-password
+
+# View encrypted Ansible secrets
+ansible-vault view /tmp/celilo/modules/homebridge/generated/ansible/inventory/secrets.yml \
+  --vault-password-file=<(bun run src/cli/index.ts system vault-password)
+```
+
+**Zero-Configuration Workflow** (Phase 0 Part 4):
+```bash
+# One-time system setup: Configure network zones
+celilo system config set network.dmz.subnet "10.0.10.0/24"
+celilo system config set network.dmz.gateway "10.0.10.1"
+celilo system config set network.dmz.vlan "10"
+celilo system config set network.dmz.bridge "vmbr0"
+
+# Import module with well-known capability (e.g., Caddy with public_web)
+celilo module import ./modules/caddy
+
+# Configure ONLY app-specific settings (infrastructure auto-configured!)
+celilo module config set caddy domain "example.com"
+
+# Generate - everything else is automatic!
+celilo module generate caddy
+
+# Debug: Show all configuration (user + auto-derived)
+celilo module show-config caddy
+
+# Debug: Show zone and network settings
+celilo module show-zone caddy
+
+# IPAM Management: Reserve VMIDs/IPs for existing infrastructure
+celilo ipam vmid reserve 2100 --reason "Existing Proxmox VM"
+celilo ipam ip reserve 10.0.10.1-10.0.10.9 --zone dmz --reason "Infrastructure"
+
+# IPAM Management: List allocations and reservations
+celilo ipam list-allocations
+celilo ipam vmid list-reservations
+celilo ipam ip list-reservations
+```
+
+**What Gets Auto-Configured**:
+- ✅ **Hostname** - From well-known capabilities (e.g., `public_web` → `www`)
+- ✅ **Zone** - From well-known capabilities (e.g., `public_web` → `dmz`)
+- ✅ **VMID** - Auto-allocated from 2100+ (IPAM)
+- ✅ **Container IP** - Auto-allocated from zone subnet (IPAM)
+- ✅ **Gateway** - From `network.{zone}.gateway` system config
+- ✅ **VLAN** - From `network.{zone}.vlan` system config
+- ✅ **Subnet** - From `network.{zone}.subnet` system config
+- ✅ **Bridge** - From `network.{zone}.bridge` system config
+- ✅ **VM Resources** - cores, memory, disk, storage from manifest defaults
+
+See [../../design/MODULE_DEVELOPMENT_GUIDE.md](../../design/MODULE_DEVELOPMENT_GUIDE.md) for complete zero-config documentation.
+
+### Database Location
+
+**Default locations**:
+- **macOS**: `~/Library/Application Support/celilo/celilo.db`
+- **Linux**: `/var/lib/celilo/celilo.db`
+- **Development** (`ENVIRONMENT=dev`): `./celilo-data/celilo.db`
+
+**Override**: Set `CELILO_DB_PATH` environment variable to use a custom location.
+
+**Override base directory**: Set `CELILO_DATA_DIR` to change where all Celilo data (including database) is stored.
+
+### Testing Strategy
+
+**Three test tiers** (see [TESTING_STRATEGY.md](../../design/TESTING_STRATEGY.md) for details):
+
+**Tier 1: Unit Tests** (< 1 second)
+```bash
+bun test                    # Run all unit tests (494 tests)
+bun test schema.test.ts     # Run specific test file
+bun test --watch            # Watch mode for active development
+```
+
+**Tier 2: Fast Integration Tests** (10-20 seconds)
+```bash
+bun run test:integration    # All integration tests (run via bun test)
+
+# Coverage:
+# - Module import/validation via CLI
+# - Configuration management
+# - Template generation
+# - Ansible/Terraform generation
+# - Well-known capability auto-assignment
+# - IPAM auto-allocation
+# - Zone-based networking
+# - VM resource defaults
+
+# What gets skipped:
+# - Nix builds
+# - Docker builds
+# - Slow module builds (Caddy)
+```
+
+**Tier 3: Slow Integration Tests** (3-5 minutes)
+```bash
+bun run test:integration-slow  # Full builds + end-to-end
+
+# Includes everything from fast tests PLUS:
+# - Full Nix builds (Caddy with modules)
+# - Docker builds
+# - Complete module packaging
+```
+
+**Before committing** (MANDATORY per Rule 7.4):
+```bash
+bun test && bun run test:integration
+```
+
+**Before releasing**:
+```bash
+bun test && bun run test:integration && bun run test:integration-slow
+```
+
+### Database Studio
+
+View database contents with Drizzle Studio:
+
+```bash
+bun run db:studio
+```
+
+Opens web interface at http://localhost:4983
+
+### Debugging
+
+**Enable verbose output**:
+```bash
+# Set log level
+export CONDUCTOR_LOG_LEVEL=debug
+bun run src/cli/index.ts module import ./path
+
+# Or use Bun's debugger
+bun --inspect run src/cli/index.ts module import ./path
+```
+
+**Inspect database**:
+```bash
+# Use Drizzle Studio
+bun run db:studio
+
+# Or sqlite3 directly
+sqlite3 celilo.db
+sqlite> .tables
+sqlite> SELECT * FROM modules;
+sqlite> SELECT * FROM module_configs WHERE module_id = 'homebridge';
+```
+
+**Check generated files**:
+```bash
+# Generated files location
+ls -la /tmp/celilo/modules/<module-id>/generated/
+
+# View Terraform
+cat /tmp/celilo/modules/<module-id>/generated/terraform/main.tf
+
+# View Ansible
+cat /tmp/celilo/modules/<module-id>/generated/ansible/playbook.yml
+
+# Check encrypted secrets
+ansible-vault view /tmp/celilo/modules/<module-id>/generated/ansible/inventory/secrets.yml \
+  --vault-password-file=<(bun run src/cli/index.ts system vault-password)
+```
+
+## Architecture Decisions
+
+### Why Drizzle ORM?
+
+- Type-safe schema-first approach
+- Excellent TypeScript inference
+- Lightweight (no heavy runtime)
+- SQL-like API (easier for developers familiar with SQL)
+
+### Why SQLite?
+
+- Single-file database (easy backup/restore)
+- No separate server process needed
+- Sufficient for home lab scale
+- ACID compliant with WAL mode
+
+### Foreign Key Cascades
+
+All child tables (configs, capabilities, secrets) cascade delete when parent module is removed. This ensures no orphaned data.
+
+### JSON Columns
+
+`manifest_data` and capability `data` use JSON columns for flexibility. These are validated with Zod schemas in application code.
+
+## Manifest Validation
+
+### Schema (`src/manifest/schema.ts`)
+
+Zod schemas for validating module manifest structure:
+
+- **ModuleManifestSchema** - Complete manifest validation
+- **VariableDeclareSchema** - Variable declarations
+- **VariableUseSchema** - Variable usage from capabilities
+- **CapabilityRequirementSchema** - Required capabilities
+- **CapabilityProviderSchema** - Provided capabilities
+- **LifecycleHookSchema** - Lifecycle hooks (on_install, health_check, etc.)
+
+### Validation Functions (`src/manifest/validate.ts`)
+
+**`validateManifest(yamlContent: string): ValidationResult`**
+- Parses YAML and validates against schema
+- Returns success with typed manifest or errors with path/message
+- Policy function - no side effects
+
+**`validateCapabilityRequirements(manifest, availableCapabilities): ValidationError | null`**
+- Checks that all required capabilities exist in system
+- Caller provides list of available capabilities (from database)
+
+**`validateVariableSources(manifest): ValidationError | null`**
+- Validates that capability references in variables match required capabilities
+- Ensures consistency between `requires.capabilities` and `variables.uses`
+
+### Example Usage
+
+```typescript
+import { validateManifest } from './manifest/validate';
+
+const yamlContent = await readFile('./modules/homebridge/manifest.yml', 'utf-8');
+const result = validateManifest(yamlContent);
+
+if (!result.success) {
+  console.error('Validation errors:', result.errors);
+  return;
+}
+
+const manifest = result.data;
+console.log(`Module: ${manifest.name} v${manifest.version}`);
+```
+
+## Module Import
+
+### Import Functions (`src/module/import.ts`)
+
+Following Rule 10.1, functions are separated by responsibility:
+
+**Policy Functions** (validation, no side effects):
+- **`validateModuleDirectory(sourcePath)`** - Checks directory exists and has manifest.yml
+- **`readModuleManifest(sourcePath)`** - Reads and validates manifest from YAML
+
+**Execution Functions** (perform I/O):
+- **`copyModuleFiles(sourcePath, targetPath)`** - Recursively copies all module files
+- **`insertModuleToDb(manifest, targetPath, db?)`** - Inserts module record to database
+- **`moduleExists(moduleId, db?)`** - Checks if module already exists
+
+**Orchestration Function**:
+- **`importModule(options)`** - Main entry point, coordinates all steps
+
+### Import Process
+
+1. **Validate** directory structure and manifest
+2. **Check** module doesn't already exist
+3. **Copy** files to target location (`/tmp/celilo/modules/{module-id}/`)
+4. **Insert** module record to database with state `IMPORTED`
+
+### Example Usage
+
+```typescript
+import { importModule } from './module/import';
+
+const result = await importModule({
+  sourcePath: './modules/homebridge',
+  targetBasePath: '/data/celilo/modules',
+});
+
+if (!result.success) {
+  console.error('Import failed:', result.error);
+  return;
+}
+
+console.log(`Imported module: ${result.moduleId}`);
+console.log(`Files copied to: ${result.targetPath}`);
+```
+
+### Dependency Injection
+
+Functions accept optional `db` parameter for testing (Rule 2.3):
+
+```typescript
+// Production - uses global singleton
+const exists = moduleExists('homebridge');
+
+// Testing - inject test database
+const testDb = createDbClient({ path: './test.db' });
+const exists = moduleExists('homebridge', testDb);
+```
+
+## Variable Resolution
+
+### System Overview (`src/variables/`)
+
+The variable resolution system follows Rule 10.1 with clear separation of concerns:
+
+**Parser** (`parser.ts`) - Policy function:
+- Extracts variable references from template strings
+- Pattern: `$type:path.to.value`
+- Validates variable format
+
+**Resolver** (`resolver.ts`) - Policy + Orchestration:
+- `resolveVariable()` - Resolves single variable from context
+- `resolveTemplate()` - Resolves all variables in template
+
+**Context** (`context.ts`) - Execution function:
+- `buildResolutionContext()` - Fetches data from database
+- `buildContextFromData()` - Creates context from explicit data (testing)
+
+### Variable Types
+
+**`$self:path`** - Module's own configuration
+- Example: `$self:container_ip` → `192.168.0.50`
+- Source: `module_configs` table
+
+**`$system:path`** - System-wide configuration
+- Example: `$system:management.ip` → `192.168.0.10`
+- Source: Hardcoded defaults (Phase 0), database (Phase 1+)
+
+**`$secret:name`** - Encrypted secrets
+- Example: `$secret:api_key` → `decrypted_value`
+- Source: `secrets` table (Phase 0: plaintext, Phase 0.5: encrypted)
+
+**`$capability:name.path`** - Capability provider data
+- Example: `$capability:dns_external.nameserver` → `ns1.example.com`
+- Source: `capabilities` table
+- Format: `capability_name.data.path`
+
+### Example Usage
+
+```typescript
+import { buildResolutionContext, resolveTemplate } from './variables';
+
+// Build context from database
+const context = await buildResolutionContext('homebridge');
+
+// Resolve template
+const template = `
+hostname: $self:hostname
+ip: $self:container_ip
+gateway: $system:management.ip
+dns: $capability:dns_external.nameserver
+api_key: $secret:api_key
+`;
+
+const result = resolveTemplate(template, context);
+
+if (result.success) {
+  console.log(result.content);
+  // hostname: homebridge
+  // ip: 192.168.0.50
+  // gateway: 192.168.0.10
+  // dns: ns1.example.com
+  // api_key: secret123
+} else {
+  console.error('Resolution errors:', result.errors);
+}
+```
+
+### Resolution Process
+
+1. **Parse** - Extract all variable references from template
+2. **Resolve** - For each variable:
+   - Determine type (`self`, `system`, `secret`, `capability`)
+   - Lookup value from appropriate data source in context
+   - Handle nested paths for capabilities
+3. **Replace** - Substitute all variables with resolved values
+4. **Return** - Success with content or errors with details
+
+### Testing
+
+All functions are testable without database:
+
+```typescript
+import { buildContextFromData, resolveTemplate } from './variables';
+
+const context = buildContextFromData('test-module', {
+  selfConfig: { ip: '192.168.0.50' },
+  secrets: { key: 'secret' },
+  capabilities: { dns: { server: 'ns1' } },
+});
+
+const result = resolveTemplate('ip: $self:ip', context);
+// result.success === true
+// result.content === 'ip: 192.168.0.50'
+```
+
+## Secret Encryption
+
+### System Overview (`src/secrets/`)
+
+AES-256-GCM encryption for module secrets following Rule 10.1 separation:
+
+**Master Key** (`master-key.ts`):
+- `generateMasterKey()` - Policy: Generates 32-byte key
+- `isValidMasterKey()` - Policy: Validates key format
+- `writeMasterKey()` - Execution: Writes to file with 0600 permissions
+- `readMasterKey()` - Execution: Reads from file
+- `getOrCreateMasterKey()` - Orchestration: Ensures key exists
+- `masterKeyExists()` - Check if key file exists
+
+**Encryption** (`encryption.ts`):
+- `encryptSecret()` - Policy: AES-256-GCM encryption with random IV
+- `decryptSecret()` - Policy: Decrypts with key verification
+- `isValidEncryptedSecret()` - Policy: Validates encrypted data format
+
+### Encryption Details
+
+**Algorithm**: AES-256-GCM (Authenticated Encryption with Associated Data)
+- **Key**: 256 bits (32 bytes)
+- **IV**: 128 bits (16 bytes) - randomly generated per encryption
+- **Auth Tag**: 128 bits (16 bytes) - ensures integrity
+
+**Storage Format**:
+```typescript
+{
+  encryptedValue: string,  // Hex-encoded ciphertext
+  iv: string,              // Hex-encoded initialization vector
+  authTag: string          // Hex-encoded authentication tag
+}
+```
+
+### Master Key Management
+
+**Location**:
+- Development: `/tmp/celilo/master.key` (configurable via `CELILO_MASTER_KEY_PATH`)
+- Production: `/etc/celilo/master.key`
+
+**Generation**:
+```typescript
+import { getOrCreateMasterKey } from './secrets/master-key';
+
+// Automatically generates if missing
+const masterKey = await getOrCreateMasterKey();
+```
+
+**File Permissions**: 0600 (owner read/write only)
+
+### Example Usage
+
+```typescript
+import { getOrCreateMasterKey } from './secrets/master-key';
+import { encryptSecret, decryptSecret } from './secrets/encryption';
+
+// Get master key (generates if missing)
+const masterKey = await getOrCreateMasterKey();
+
+// Encrypt a secret
+const encrypted = encryptSecret('my-api-key-12345', masterKey);
+console.log(encrypted);
+// {
+//   encryptedValue: '4a3b2c1d...',
+//   iv: 'f1e2d3c4...',
+//   authTag: 'a9b8c7d6...'
+// }
+
+// Decrypt the secret
+const plaintext = decryptSecret(encrypted, masterKey);
+console.log(plaintext); // 'my-api-key-12345'
+```
+
+### Security Features
+
+1. **Authenticated Encryption**: GCM mode provides both confidentiality and integrity
+2. **Random IVs**: Each encryption uses unique IV, same plaintext → different ciphertext
+3. **Key Validation**: Master key length enforced (32 bytes)
+4. **Tamper Detection**: Auth tag verification prevents corrupted/modified ciphertext
+5. **Fail-Fast**: Invalid keys or corrupted data throw clear errors
+
+### Error Handling
+
+```typescript
+try {
+  const decrypted = decryptSecret(encrypted, masterKey);
+} catch (error) {
+  // Possible errors:
+  // - "Master key must be 32 bytes"
+  // - "Invalid encrypted secret: missing required fields"
+  // - "Invalid IV length: expected 16, got N"
+  // - "Failed to decrypt secret: [reason]" (wrong key, corrupted data)
+}
+```
+
+### Testing
+
+All functions are pure (no global state):
+
+```typescript
+import { generateMasterKey, encryptSecret, decryptSecret } from './secrets';
+
+const masterKey = generateMasterKey();
+const encrypted = encryptSecret('test-secret', masterKey);
+const decrypted = decryptSecret(encrypted, masterKey);
+// decrypted === 'test-secret'
+```
+
+## Ansible Vault Integration
+
+### System Overview (`src/ansible/`, `src/secrets/vault.ts`)
+
+Celilo uses Ansible Vault to encrypt secrets in generated Ansible configurations, ensuring no plaintext secrets are written to disk.
+
+**Key Components**:
+- **`vault.ts`** - Derives vault password from master key (deterministic)
+- **`ansible/secrets.ts`** - Generates and encrypts secrets.yml file
+- **`ansible-resolver.ts`** - Converts `$secret:` variables to Jinja2 `{{ }}` syntax
+
+### Encryption Flow
+
+1. **Decrypt secrets** from database (using master key)
+2. **Format as YAML** with all module secrets
+3. **Derive vault password** from master key (SHA-256)
+4. **Encrypt with ansible-vault** (AES-256)
+5. **Write encrypted file** to `ansible/inventory/secrets.yml`
+
+### Vault Password Derivation
+
+```typescript
+import { deriveVaultPassword, getVaultPassword } from './secrets/vault';
+
+// Deterministic derivation from master key
+const masterKey = await getOrCreateMasterKey();
+const vaultPassword = deriveVaultPassword(masterKey);
+// Same master key always produces same vault password
+
+// Or use orchestration function
+const vaultPassword = await getVaultPassword();
+```
+
+**Algorithm**: `SHA-256(concat("ansible-vault:", masterKey))`
+
+**Properties**:
+- Deterministic (same master key → same vault password)
+- Domain-separated (prevents key reuse attacks)
+- Standard hash output (64 hex characters)
+
+### Generated Ansible Structure
+
+**Playbook** (`playbook.yml`):
+```yaml
+---
+- name: Deploy Module
+  hosts: module-host
+  become: true
+
+  vars_files:
+    - inventory/secrets.yml  # Encrypted with ansible-vault
+
+  roles:
+    - module-role
+```
+
+**Secrets File** (`inventory/secrets.yml`):
+```
+$ANSIBLE_VAULT;1.1;AES256
+35646166616130633832363334383234306139626264373935623630393937313639623334356138
+6561383035303565323364373934306632336461626562630a386237663365633039396631623639
+...
+```
+
+**Templates** (e.g., `templates/config.json`):
+```json
+{
+  "api_key": "{{ api_key }}",
+  "password": "{{ db_password }}"
+}
+```
+
+### Variable Resolution
+
+Ansible templates use different resolution from Terraform:
+
+**Terraform** (resolved at generation time):
+- `$self:hostname` → `"myhost"`
+- `$system:dns.primary` → `"192.168.0.1"`
+- `$secret:api_key` → `"actual_secret_value"` (**plaintext in file!**)
+
+**Ansible** (resolved at runtime):
+- `$self:hostname` → `"myhost"` (resolved at generation)
+- `$system:dns.primary` → `"192.168.0.1"` (resolved at generation)
+- `$secret:api_key` → `{{ api_key }}` (Jinja2 variable, resolved from encrypted secrets.yml)
+
+### CLI Usage
+
+**Get vault password**:
+```bash
+# Display vault password
+celilo system vault-password
+
+# Use with ansible-vault
+ansible-vault view inventory/secrets.yml \
+  --vault-password-file=<(celilo system vault-password)
+
+# Edit encrypted secrets
+ansible-vault edit inventory/secrets.yml \
+  --vault-password-file=<(celilo system vault-password)
+```
+
+**Run Ansible playbook**:
+```bash
+cd /tmp/celilo/modules/homebridge/generated/ansible
+
+# Option 1: Process substitution (recommended)
+ansible-playbook playbook.yml \
+  --vault-password-file=<(celilo system vault-password)
+
+# Option 2: Environment variable
+export ANSIBLE_VAULT_PASSWORD=$(celilo system vault-password)
+echo "$ANSIBLE_VAULT_PASSWORD" | ansible-playbook playbook.yml --vault-password-file=/dev/stdin
+```
+
+### Security Considerations
+
+**Benefits**:
+- ✅ No plaintext secrets on disk
+- ✅ Industry-standard encryption (Ansible Vault AES-256)
+- ✅ Deterministic (same master key → same vault password)
+- ✅ Works with standard Ansible tooling
+- ✅ Secrets encrypted independently from celilo database
+
+**Requirements**:
+- ⚠️ **Ansible must be installed** - Generation fails without ansible-vault
+- ⚠️ Master key must be protected (file permissions 0600)
+- ⚠️ Vault password derivation is deterministic (anyone with master key can decrypt)
+
+**Threat Model**:
+- **Protects against**: Accidental secret exposure (logs, commits, backups)
+- **Does NOT protect against**: Attacker with master key access
+- **Assumption**: Master key is stored securely with restricted file permissions
+
+### Implementation Details
+
+**Files**:
+- `src/secrets/vault.ts` - Vault password derivation
+- `src/ansible/secrets.ts` - Secrets file generation and encryption
+- `src/variables/ansible-resolver.ts` - Jinja2 variable conversion
+- `src/templates/generator.ts` - Integration with template generation
+
+**Functions**:
+- `deriveVaultPassword(masterKey)` - Deterministic password derivation
+- `generateAnsibleSecrets(moduleId, outputPath, db)` - Full generation pipeline
+- `encryptWithAnsibleVault(yamlContent, password)` - Ansible Vault wrapper
+- `convertSecretsToJinja(content, context)` - Variable conversion for Ansible
+
+### Testing
+
+**Unit Tests** (13 tests):
+```bash
+bun test src/secrets/vault.test.ts
+bun test src/variables/ansible-resolver.test.ts
+```
+
+**Integration Tests**:
+```bash
+bun run test:integration
+# Validates:
+# - Secrets encrypted with ansible-vault
+# - Ansible templates use Jinja2 variables
+# - Vault password command works
+# - Generated secrets can be decrypted
+```
+
+**Manual Verification**:
+```bash
+# 1. Generate module
+celilo module generate homebridge
+
+# 2. Check secrets file is encrypted
+head /tmp/celilo/modules/homebridge/generated/ansible/inventory/secrets.yml
+# Should start with: $ANSIBLE_VAULT;1.1;AES256
+
+# 3. Decrypt and verify
+ansible-vault view /tmp/celilo/modules/homebridge/generated/ansible/inventory/secrets.yml \
+  --vault-password-file=<(celilo system vault-password)
+# Should show YAML with secret values
+
+# 4. Check templates use Jinja2
+cat /tmp/celilo/modules/homebridge/generated/ansible/roles/homebridge/templates/config.json
+# Should contain {{ variable_name }}, not plaintext secrets
+```
+
+## Template Generation
+
+### System Overview (`src/templates/`)
+
+Template generation following Rule 10.1 separation:
+
+**Policy Functions**:
+- `isTemplateFile(filename)` - Checks if file has template extension (`.tpl`, `.j2`)
+- `getOutputFilename(template)` - Removes template extension from filename
+
+**Execution Functions**:
+- `discoverTemplateFiles(baseDir)` - Recursively finds template files
+- `readTemplateFiles(modulePath, paths)` - Reads template content from disk
+- `writeGeneratedFiles(outputPath, files)` - Writes generated files to disk
+
+**Orchestration Function**:
+- `generateTemplates(options)` - Main entry point, coordinates entire generation process
+
+### Template Directories
+
+Celilo looks for templates in:
+- `terraform/` - Terraform configuration templates
+- `ansible/` - Ansible playbook templates
+
+Both directories are searched recursively.
+
+### Template Extensions
+
+- **`.tpl`** - Generic template files (Terraform style)
+- **`.j2`** - Jinja2 template files (Ansible style)
+
+Output files have extension removed:
+- `main.tf.tpl` → `main.tf`
+- `playbook.yml.j2` → `playbook.yml`
+
+### Generation Process
+
+1. **Discover** - Find all template files in `terraform/` and `ansible/` directories
+2. **Build Context** - Load module config, secrets, and capabilities from database
+3. **Read** - Load template content from files
+4. **Resolve** - Replace variables using variable resolution system
+5. **Write** - Save generated files to output directory
+6. **Return** - Success with file list or detailed error messages
+
+### Example Usage
+
+```typescript
+import { generateTemplates } from './templates/generator';
+
+const result = await generateTemplates({
+  moduleId: 'homebridge',
+  modulePath: '/data/modules/homebridge',
+  outputPath: '/data/modules/homebridge/generated',
+});
+
+if (result.success) {
+  console.log(`Generated ${result.files.length} files:`);
+  for (const file of result.files) {
+    console.log(`  - ${file.path}`);
+  }
+} else {
+  console.error('Generation failed:', result.error);
+}
+```
+
+### Module Structure
+
+```
+modules/homebridge/
+├── manifest.yml
+├── terraform/
+│   ├── main.tf.tpl          # → generated/terraform/main.tf
+│   ├── variables.tf.tpl     # → generated/terraform/variables.tf
+│   └── outputs.tf.tpl       # → generated/terraform/outputs.tf
+└── ansible/
+    ├── playbook.yml.tpl     # → generated/ansible/playbook.yml
+    └── roles/
+        └── homebridge/
+            └── tasks/
+                └── main.yml.tpl  # → generated/ansible/roles/homebridge/tasks/main.yml
+```
+
+### Template Example
+
+**Input** (`terraform/main.tf.tpl`):
+```hcl
+resource "proxmox_lxc" "container" {
+  hostname = "$self:hostname"
+  cores    = $self:cores
+  memory   = $self:memory
+
+  network {
+    name   = "eth0"
+    bridge = "vmbr0"
+    ip     = "$self:container_ip/24"
+    gw     = "$system:management.ip"
+  }
+
+  provisioner "ansible" {
+    playbook = "./ansible/playbook.yml"
+  }
+}
+```
+
+**Output** (`generated/terraform/main.tf`):
+```hcl
+resource "proxmox_lxc" "container" {
+  hostname = "homebridge"
+  cores    = 2
+  memory   = 2048
+
+  network {
+    name   = "eth0"
+    bridge = "vmbr0"
+    ip     = "192.168.0.50/24"
+    gw     = "192.168.0.10"
+  }
+
+  provisioner "ansible" {
+    playbook = "./ansible/playbook.yml"
+  }
+}
+```
+
+### Error Handling
+
+Clear error messages for common failures:
+
+```typescript
+// Module path doesn't exist
+{ success: false, error: "Module path does not exist: /path" }
+
+// No templates found
+{ success: false, error: "No template files found in module" }
+
+// Variable resolution failed
+{
+  success: false,
+  error: "Failed to resolve variables in templates:\n" +
+         "terraform/main.tf:\n" +
+         "  $self:missing_var: Self variable 'missing_var' not found in module configuration"
+}
+
+// File write failed
+{ success: false, error: "Failed to write generated files", details: Error }
+```
+
+### Integration
+
+Template generation integrates with:
+- **Variable Resolution** - Resolves all `$self:`, `$system:`, `$secret:`, `$capability:` variables
+- **Database** - Loads module configuration and capability data
+- **File System** - Reads templates and writes generated code
+
+## Next Steps
+
+Phase 0 Part 1.7:
+1. ✅ Database layer
+2. ✅ Manifest validation
+3. ✅ Module import logic
+4. ✅ Variable resolution system
+5. ✅ Secret encryption (AES-256-GCM)
+6. ✅ Template generation (File I/O + variable resolution)
+7. CLI interface (Commands + user interaction)
+
+## CLI Interface
+
+### System Overview (`src/cli/`)
+
+The CLI interface provides command-line tools for all Phase 0 operations following Rule 10.1 separation:
+
+**Parser** (`parser.ts`) - Policy functions:
+- `parseArguments()` - Parses command-line arguments into structured format
+- `validateRequiredArgs()` - Validates argument count
+- `getArg()`, `getFlag()`, `hasFlag()` - Type-safe accessors
+
+**Commands** (`commands/`) - Orchestration functions:
+- `module-import.ts` - Import modules from directory
+- `module-list.ts` - List installed modules
+- `module-config.ts` - Get/set module configuration
+- `module-generate.ts` - Generate templates for modules
+- `secret-set.ts` - Set encrypted secrets
+
+**Entry Point** (`index.ts`) - Main orchestrator:
+- `runCli()` - Routes commands to handlers
+- `main()` - Entry point with error handling and exit codes
+
+### Commands
+
+**Module Import**
+```bash
+celilo module import <path> [--target <path>]
+
+# Example
+celilo module import ./modules/homebridge
+celilo module import ./modules/homebridge --target /data/celilo/modules
+```
+
+**Module List**
+```bash
+celilo module list
+
+# Output
+Installed modules:
+
+homebridge (v1.0.0) - IMPORTED
+  Bridge for HomeKit accessories
+
+pihole (v2.1.0) - CONFIGURED
+```
+
+**Module Configuration**
+```bash
+# Set configuration value
+celilo module config set <module-id> <key> <value>
+
+# Get specific value
+celilo module config get <module-id> [key]
+
+# Examples
+celilo module config set homebridge hostname mybridge
+celilo module config set homebridge container_ip 192.168.0.50
+celilo module config get homebridge hostname
+celilo module config get homebridge  # Get all config
+```
+
+**Module Generate**
+```bash
+celilo module generate <module-id> [--output <path>]
+
+# Examples
+celilo module generate homebridge
+celilo module generate homebridge --output ./generated
+```
+
+**Secret Management**
+```bash
+celilo secret set <module-id> <name> <value>
+
+# Example
+celilo secret set homebridge api_key mykey123
+```
+
+**System Configuration**
+```bash
+# Set system-wide config
+celilo system config set <key> <value>
+celilo system config get [key]
+
+# Get Ansible Vault password
+celilo system vault-password
+
+# Examples
+celilo system config set dns.primary 192.168.0.1
+celilo system config get dns.primary
+ansible-vault view secrets.yml --vault-password-file=<(celilo system vault-password)
+```
+
+**Help**
+```bash
+celilo help
+celilo --help
+celilo -h
+celilo module --help  # Command-specific help
+```
+
+### Argument Parsing
+
+The CLI uses a simple argument parser without external dependencies:
+
+**Format**: `celilo <command> [subcommand] [args...] [--flags]`
+
+**Parsing Rules**:
+- First argument is the command (`module`, `secret`, `help`)
+- Second argument is the subcommand if it doesn't start with `--`
+- Remaining arguments are positional args or flags
+- Flags start with `--` and can be boolean (`--verbose`) or string (`--target /path`)
+
+**Examples**:
+```bash
+# Command only
+celilo help
+
+# Command + subcommand
+celilo module list
+
+# Command + subcommand + args
+celilo module import ./path
+
+# Command + subcommand + args + flags
+celilo module generate homebridge --output ./out
+
+# Nested subcommand
+celilo module config set homebridge hostname myhost
+```
+
+### Error Handling
+
+All commands return structured results:
+```typescript
+interface CommandSuccess {
+  success: true;
+  message: string;
+  data?: unknown;
+}
+
+interface CommandError {
+  success: false;
+  error: string;
+  details?: unknown;
+}
+```
+
+**Exit Codes**:
+- `0` - Success
+- `1` - Error (validation, not found, etc.)
+
+**Error Messages** include usage hints:
+```
+Error: Missing required arguments. Expected 1, got 0
+
+Usage: celilo module import <path> [--target <path>]
+```
+
+### Testing
+
+All CLI components have comprehensive tests:
+- **`parser.test.ts`** - Argument parsing (28 tests)
+- **`cli.test.ts`** - Integration tests (23 tests)
+
+Tests use environment variables to configure database and master key paths for isolation.
+
+### Entry Point
+
+The main entry point is `src/cli/index.ts`, which exports `main()`:
+```typescript
+export async function main(): Promise<void> {
+  const result = await runCli(process.argv);
+  // Handle output and exit codes
+}
+```
+
+**Recommended:** Use the top-level wrapper script (from anywhere):
+```bash
+/path/to/celilo/celilo module list
+/path/to/celilo/celilo system config get
+```
+
+Or run directly from backend directory:
+```bash
+bun run src/cli/index.ts module list
+bun run src/cli/index.ts module import ./path
+```
+
+Or build a binary:
+```bash
+bun build src/cli/index.ts --compile --outfile celilo
+./celilo module list
+```
+
+### Architecture
+
+**CLI Orchestration**:
+```
+User Input
+   ↓
+parseArguments()   [Policy - parse and validate]
+   ↓
+runCli()          [Orchestration - route to handler]
+   ↓
+handleModuleXxx()  [Orchestration - coordinate operations]
+   ↓
+importModule()     [Execution - database, filesystem]
+generateTemplates()
+etc.
+```
+
+**Separation of Concerns**:
+- **Parser** - Pure policy functions, no side effects
+- **Handlers** - Orchestrate multiple operations
+- **Core modules** - Execute actual work (import, generate, etc.)
+- **Entry point** - Handle errors and exit codes
+
+All commands integrate with existing modules:
+- **Database** - Module metadata, configuration, secrets
+- **Manifest validation** - Parse and validate manifests
+- **Module import** - Copy files and insert to database
+- **Variable resolution** - Resolve template variables
+- **Secret encryption** - Encrypt/decrypt with master key
+- **Template generation** - Generate Terraform/Ansible code
+
+## Troubleshooting
+
+### Common Errors and Solutions
+
+#### "Module ID must use kebab-case"
+
+**Error**:
+```
+Error: Manifest validation failed
+Module ID must use kebab-case (lowercase letters, numbers, hyphens between segments)
+```
+
+**Cause**: Module ID uses underscores, uppercase, or invalid characters.
+
+**Solution**: Use kebab-case for all module IDs:
+```yaml
+# ❌ WRONG
+metadata:
+  name: dns_external   # Underscores
+
+# ✅ CORRECT
+metadata:
+  name: dns-external   # Kebab-case
+```
+
+**Pattern**: `/^[a-z0-9]+(-[a-z0-9]+)*$/`
+
+---
+
+#### "Variable resolution failed: Self variable not found"
+
+**Error**:
+```
+Failed to resolve variables in templates:
+terraform/main.tf:
+  $self:container_ip: Self variable 'container_ip' not found in module configuration
+```
+
+**Cause**: Template references variable that isn't configured.
+
+**Solution**: Set the missing variable:
+```bash
+celilo module config set <module-id> container_ip 10.0.20.100
+```
+
+Or check for typos in template:
+```hcl
+# Check spelling
+ip = "$self:container_ip"  # Not 'containerIP', 'container-ip', etc.
+```
+
+---
+
+#### "ENOENT: no such file or directory" (path with spaces)
+
+**Error**:
+```
+Error: ENOENT: no such file or directory, open '/Users/user/Application Support/celilo/...'
+```
+
+**Cause**: Unquoted path in shell command with spaces.
+
+**Solution**: Always quote paths in shell commands:
+```typescript
+// ❌ WRONG
+execSync(`cd ${modulePath} && make build`);
+
+// ✅ CORRECT
+execSync(`cd "${modulePath}" && make build`);
+
+// ✅ BEST
+import { shellEscape } from '@/utils/shell';
+execSync(`cd ${shellEscape(modulePath)} && make build`);
+```
+
+---
+
+#### "Drizzle where clause returns all rows"
+
+**Error**: Query returns all modules instead of filtering by ID.
+
+**Cause**: Using JavaScript operators instead of Drizzle operators.
+
+**Solution**: Use Drizzle operator functions:
+```typescript
+import { eq } from 'drizzle-orm';
+
+// ❌ WRONG - returns ALL rows!
+db.select().from(modules).where(modules.id === 'homebridge');
+
+// ✅ CORRECT - filters correctly
+db.select().from(modules).where(eq(modules.id, 'homebridge'));
+```
+
+**Always use**: `eq()`, `ne()`, `gt()`, `lt()`, `and()`, `or()`, etc.
+
+---
+
+#### "ansible-vault: command not found"
+
+**Error**:
+```
+Error: Failed to encrypt secrets with ansible-vault
+ansible-vault: command not found
+```
+
+**Cause**: Ansible is not installed or not in PATH.
+
+**Solution**: Install Ansible:
+```bash
+# macOS
+brew install ansible
+
+# Ubuntu/Debian
+sudo apt install ansible
+
+# Verify
+ansible-vault --version
+```
+
+---
+
+#### "Test passes alone, fails in suite"
+
+**Error**: Test passes when run individually, fails when run with others.
+
+**Cause**: State leakage between tests (shared database, temp files, environment).
+
+**Solution**: Add proper setup/teardown:
+```typescript
+import { describe, test, beforeEach, afterEach } from 'bun:test';
+
+describe('Feature Tests', () => {
+  let testDb: Database;
+  let tempDir: string;
+
+  beforeEach(async () => {
+    testDb = await setupTestDatabase();      // Fresh database
+    tempDir = await createTempDirectory();   // Fresh directory
+  });
+
+  afterEach(async () => {
+    await cleanupTestDatabase(testDb);       // MUST await!
+    await removeTempDirectory(tempDir);      // MUST await!
+  });
+
+  test('isolated test', async () => {
+    // Test runs with clean state
+  });
+});
+```
+
+---
+
+#### "Cannot find module '@/something'"
+
+**Error**:
+```
+Error: Cannot find module '@/module/import'
+```
+
+**Cause**: TypeScript path aliases not resolved.
+
+**Solution**: Check `tsconfig.json` has correct paths:
+```json
+{
+  "compilerOptions": {
+    "baseUrl": ".",
+    "paths": {
+      "@/*": ["src/*"]
+    }
+  }
+}
+```
+
+And run with Bun (which respects tsconfig paths):
+```bash
+bun run src/cli/index.ts  # ✅ Works
+node src/cli/index.ts     # ❌ Doesn't resolve @/ aliases
+```
+
+---
+
+#### "Master key permission denied"
+
+**Error**:
+```
+Error: EACCES: permission denied, open '/etc/celilo/master.key'
+```
+
+**Cause**: Master key file has wrong permissions or wrong owner.
+
+**Solution**: Fix permissions:
+```bash
+# Check current permissions
+ls -la /tmp/celilo/master.key
+
+# Fix permissions (owner read/write only)
+chmod 600 /tmp/celilo/master.key
+
+# Or regenerate (WARNING: invalidates all encrypted secrets!)
+rm /tmp/celilo/master.key
+celilo system vault-password  # Generates new key
+```
+
+---
+
+#### "Migration already exists"
+
+**Error**:
+```
+Error: Migration 0005_add_capability_secrets.sql already exists
+```
+
+**Cause**: Schema changed but migration not generated or conflicting migration number.
+
+**Solution**: Check for duplicates:
+```bash
+# List migrations
+ls drizzle/
+
+# Remove duplicate or conflicting migration
+rm drizzle/0005_duplicate.sql
+
+# Regenerate
+bunx drizzle-kit generate
+```
+
+---
+
+### Debug Checklist
+
+When debugging issues, check:
+
+1. **Test isolation**:
+   - [ ] `beforeEach`/`afterEach` properly clean up
+   - [ ] Tests can run in any order
+   - [ ] Tests use isolated database/filesystem
+
+2. **Path handling**:
+   - [ ] All shell commands quote paths
+   - [ ] Tested with paths containing spaces
+   - [ ] No hardcoded absolute paths
+
+3. **Drizzle queries**:
+   - [ ] Using operator functions (`eq()`, not `===`)
+   - [ ] Importing operators from `'drizzle-orm'`
+   - [ ] Not using callback form in `where()`
+
+4. **Identifier naming**:
+   - [ ] Module IDs use kebab-case
+   - [ ] No underscores or uppercase
+   - [ ] Pattern: `/^[a-z0-9]+(-[a-z0-9]+)*$/`
+
+5. **Variable resolution**:
+   - [ ] Using colon syntax (`$self:var`, not `$self.var`)
+   - [ ] Simple syntax for standalone values
+   - [ ] Braced syntax for concatenation (`${self:disk}G`)
+
+6. **Secrets**:
+   - [ ] Master key exists and has 0600 permissions
+   - [ ] Ansible installed for vault encryption
+   - [ ] Secrets encrypted in database
+
+---
+
+### Getting Help
+
+**Documentation**:
+- [CLAUDE.md](../../CLAUDE.md) - Engineering standards
+- [TESTING_STRATEGY.md](../../design/TESTING_STRATEGY.md) - Testing approach
+- [IDENTIFIER_NAMING_CONVENTIONS.md](../../design/IDENTIFIER_NAMING_CONVENTIONS.md) - Naming rules
+- [TEMPLATE_VARIABLE_SYNTAX.md](../../design/TEMPLATE_VARIABLE_SYNTAX.md) - Variable syntax
+
+**Debugging Tools**:
+- Drizzle Studio: `bun run db:studio`
+- Test watch mode: `bun test --watch`
+- Verbose logs: `export CONDUCTOR_LOG_LEVEL=debug`
+
+---
+
+## Next Steps
+
+Phase 0 Part 1.7:
+1. ✅ Database layer
+2. ✅ Manifest validation
+3. ✅ Module import logic
+4. ✅ Variable resolution system
+5. ✅ Secret encryption (AES-256-GCM)
+6. ✅ Template generation (File I/O + variable resolution)
+7. ✅ CLI interface (Commands + user interaction)
+
+**Phase 0 Part 1 is complete!**
+
+Next: Phase 0 Part 2 - Web UI (React + tRPC)