@celilo/cli 0.8.1 → 0.8.2

package/AGENTS.md

@@ -0,0 +1,94 @@
+# AGENTS.md — `@celilo/cli`
+
+Celilo is a **home-lab orchestration system**: you describe infrastructure as
+**modules**, and celilo deploys them onto a fleet (Proxmox LXC/VM) and wires them
+together through versioned **capabilities**. This file orients an AI agent to
+build and ship apps on celilo. **Read the local docs below before searching the
+web** — they are the source of truth; the hosted copies at
+`https://celilo.computer/docs` lag them.
+
+This file ships **inside the npm package**: the `./`-prefixed docs in the doc map
+are installed alongside it (grep them offline). The repo-relative entries point
+at the source repository / hosted docs, which the tarball does not carry.
+
+## Mental model (learn these five things)
+
+- **Module** — a directory with a `manifest.yml` and `celilo/scripts/` hooks,
+  packaged into a `.netapp` artifact and published to **celilo-registry**.
+  Everything celilo deploys is a module.
+- **Capability** — a module `provides:` capabilities (e.g. `public_web`, `idp`,
+  `source_forge`) at a **versioned contract**, and consumers `requires:` them.
+  Consumers bind to the **capability** version (`provides.capabilities[].version`)
+  — *not* the module's own `version`. Bump the capability version on a contract
+  change; that's the load-bearing semver.
+- **Hook** — a lifecycle script (`on_install`, `health_check`, `on_backup`, …) in
+  `celilo/scripts/`, written with `defineHook(...)` from `@celilo/capabilities`.
+  Hooks receive typed clients for the capabilities they `requires`/`optional`.
+- **Build bus** — app repo CI publishes a `.netapp` to celilo-registry →
+  celilo-mgr **polls** → `module upgrade` → deploy → verify. No manual hop.
+- **Versioning** — `manifest.yml#version` is the **payload** version; `+N` is the
+  recipe revision; `version_source` declares how the version is set. See below.
+
+## Building an app (the happy path)
+
+An **app** is a module that provides no capability (it serves content), so set
+`version_source: { kind: changeset }`.
+
+1. **`manifest.yml`** — `id` (kebab-case), `celilo_contract: "1.0"`, `version`,
+   the capabilities you `requires:` (e.g. `public_web`, `idp`), `variables`,
+   `hooks`, and `version_source`. Keep the schema directive on line 1 so your
+   editor validates:
+   `# yaml-language-server: $schema=../../schemas/module-manifest.schema.json`
+2. **Hooks** in `celilo/scripts/` — at least `on_install` (provision) and
+   `health_check`. Use `defineHook`; required capabilities are typed non-null.
+3. **CI/CD** — copy the reference workflows + add the `register-forge` deploy
+   hook so your repo gets a scoped publish token (see `APP_CI_REFERENCE.md`).
+4. **Version with changesets** — `celilo module changeset --bump <major|minor|
+   patch>` in each behaviour-changing PR; `celilo module version` stamps
+   `manifest.yml#version` + CHANGELOG at release. Pick the bump by deploy **blast
+   radius** (major ⇒ safe deploy w/ backup; patch ⇒ fast).
+5. **Ship** — `celilo module check .` then `celilo module publish .`. On the
+   fleet: `module update` (registry sweep) → `module deploy <id>` → `module
+   health`.
+
+## CLI you'll use
+
+```
+celilo module import <name|path>        # pull a module
+celilo module check [dir]               # validate: schema, capabilities, git-hygiene
+celilo module changeset --bump <b> -m … # author a version changeset (module-id keyed)
+celilo module version [dir]             # stamp version+CHANGELOG from changesets
+celilo module publish [dir]             # build .netapp + publish to celilo-registry
+celilo module deploy <id> / update / health   # deploy + verify on the fleet
+```
+
+## Conventions / rules
+
+- Module `id` is **kebab-case**; pin `celilo_contract: "1.0"`.
+- **`version_source`** picks how `version` is sourced: `changeset` (apps/content —
+  authored via changesets, ordered by `+N`), `pin` (wraps upstream software —
+  `version` = the installed upstream version, resolver-checked), `recipe`
+  (config-only modules — no payload version; `+N` orders). Default `recipe`.
+- The capability **contract** semver lives in `provides.capabilities[].version`,
+  independent of the module `version`. Don't conflate them.
+- **Never hand-edit `schemas/module-manifest.schema.json`** — it's generated from
+  `src/manifest/schema.ts` via `bun run export:schema`.
+- Validate before publishing; `celilo module check` mirrors the publish gates.
+
+## Doc map (read in this order)
+
+Shipped **inside this package** (offline, grep-friendly):
+
+- `./CLI_USAGE.md` — full CLI reference + common workflows.
+- `./schemas/system_config.json` — the system-config contract.
+- `./src/manifest/schema.ts` — Zod source of the manifest schema (the published
+  `module-manifest.schema.json` is generated from it via `bun run export:schema`).
+
+In the **source repo / hosted docs** (not carried in the tarball):
+
+- `design/MODULE_DEVELOPMENT_GUIDE.md` — author a module (start here).
+- `v2/BUILD_BUS.md` — how a change reaches the fleet.
+- `v2/APP_CI_REFERENCE.md` — wire an app's CI/CD (the copy-me recipe).
+- `v2/MODULE_VERSIONING.md` — what `version` means; `version_source`.
+- `schemas/module-manifest.schema.json` — the manifest contract.
+- `https://celilo.computer/docs` — hosted docs (LAN; the repo docs lead).

package/CLI_USAGE.md

@@ -0,0 +1,433 @@
+# Celilo CLI Usage Guide
+
+Common workflows and command patterns for the Celilo CLI.
+
+## Quick Reference
+
+```bash
+# Import and setup
+celilo module import <path>
+celilo system config set <key> <value>
+celilo module config set <module> <key> <value>
+celilo secret set <module> <name> <value>
+celilo module generate <module>
+
+# Query and inspect
+celilo module list
+celilo module config get <module>
+celilo system config get [key]
+celilo system vault-password
+
+# Manage
+celilo module remove <module>
+celilo help
+```
+
+## Common Workflows
+
+### 1. Setting Up a New Module
+
+```bash
+# Step 1: Import the module
+./celilo module import modules/homebridge
+# Output: Successfully imported module: homebridge
+#         Files copied to: /tmp/celilo/modules/homebridge
+
+# Step 2: Configure system-wide settings (if not already set)
+./celilo system config set dns.primary 192.168.0.1
+./celilo system config set dns.fallback "8.8.8.8 1.1.1.1"
+./celilo system config set routing.internal_gateway 192.168.0.254
+
+# Step 3: Configure module-specific settings
+./celilo module config set homebridge vmid 2110
+./celilo module config set homebridge hostname iot
+./celilo module config set homebridge container_ip "192.168.0.110/24"
+./celilo module config set homebridge gateway 192.168.0.254
+./celilo module config set homebridge vlan 192
+./celilo module config set homebridge cores 2
+./celilo module config set homebridge memory 2048
+./celilo module config set homebridge storage datacenter
+./celilo module config set homebridge rootfs_size 20G
+
+# Step 4: Set secrets
+./celilo secret set homebridge api_key "your_api_key_here"
+./celilo secret set homebridge db_password "your_password_here"
+
+# Step 5: Generate infrastructure code
+./celilo module generate homebridge
+# Output: Successfully generated 8 files:
+#           - terraform/main.tf
+#           - terraform/variables.tf
+#           - ...
+
+# Step 6: Review generated files
+ls -la /tmp/celilo/modules/homebridge/generated/
+cd /tmp/celilo/modules/homebridge/generated/
+```
+
+### 2. Inspecting Configuration
+
+```bash
+# List all modules
+./celilo module list
+
+# Get all config for a module
+./celilo module config get homebridge
+
+# Get specific config value
+./celilo module config get homebridge hostname
+
+# Get all system config
+./celilo system config get
+
+# Get specific system config
+./celilo system config get dns.primary
+```
+
+### 3. Working with Secrets
+
+```bash
+# Set a secret
+./celilo secret set homebridge api_key "secret_value"
+
+# View encrypted secrets file
+MODULE=homebridge
+SECRETS_FILE=/tmp/celilo/modules/$MODULE/generated/ansible/inventory/secrets.yml
+
+# Check it's encrypted
+head -1 $SECRETS_FILE
+# Should show: $ANSIBLE_VAULT;1.1;AES256
+
+# Decrypt and view
+ansible-vault view $SECRETS_FILE \
+  --vault-password-file=<(./celilo system vault-password)
+
+# Edit encrypted secrets directly (if needed)
+ansible-vault edit $SECRETS_FILE \
+  --vault-password-file=<(./celilo system vault-password)
+```
+
+### 4. Regenerating After Changes
+
+```bash
+# Change configuration
+./celilo module config set homebridge cores 4
+./celilo module config set homebridge memory 4096
+
+# Regenerate
+./celilo module generate homebridge
+
+# Generated files will reflect new configuration
+```
+
+### 5. Removing a Module
+
+```bash
+# Remove module (cascade deletes all config and secrets)
+./celilo module remove homebridge
+
+# Verify removal
+./celilo module list
+# Should no longer show homebridge
+
+# Manually clean up generated files if needed
+rm -rf /tmp/celilo/modules/homebridge/
+```
+
+### 6. Using Generated Infrastructure
+
+```bash
+# Terraform workflow
+cd /tmp/celilo/modules/homebridge/generated/terraform
+terraform init
+terraform plan
+terraform apply
+
+# Ansible workflow
+cd /tmp/celilo/modules/homebridge/generated/ansible
+ansible-playbook playbook.yml \
+  -i inventory/ \
+  --vault-password-file=<(celilo system vault-password)
+```
+
+## Advanced Usage
+
+### Running from Different Directories
+
+```bash
+# From anywhere (using wrapper script)
+/path/to/celilo/celilo module list
+
+# From backend directory (using bun)
+cd /path/to/celilo/backend
+bun run src/cli/index.ts module list
+
+# From backend directory (using npm script)
+cd /path/to/celilo/backend
+bun run dev -- module list
+```
+
+### Custom Database Location
+
+```bash
+# Set environment variable
+export CELILO_DB_PATH=/custom/path/celilo.db
+
+# Run migrations
+bun run db:migrate
+
+# Use CLI (will use custom database)
+./celilo module list
+```
+
+### Custom Output Directory
+
+```bash
+# Generate to specific location
+./celilo module generate homebridge --output /custom/output/path
+```
+
+### Scripting with Celilo
+
+```bash
+#!/bin/bash
+# Example: setup-homebridge.sh
+
+set -e  # Exit on error
+
+MODULE=homebridge
+
+# Import module
+./celilo module import "modules/$MODULE"
+
+# Configure system
+./celilo system config set dns.primary 192.168.0.1
+./celilo system config set routing.internal_gateway 192.168.0.254
+
+# Configure module (read from config file or environment)
+./celilo module config set $MODULE vmid "${VMID}"
+./celilo module config set $MODULE hostname "${HOSTNAME}"
+./celilo module config set $MODULE container_ip "${CONTAINER_IP}"
+
+# Set secrets (from secure source)
+./celilo secret set $MODULE api_key "${API_KEY}"
+
+# Generate
+./celilo module generate $MODULE
+
+echo "Setup complete! Generated files at:"
+echo "  /tmp/celilo/modules/$MODULE/generated/"
+```
+
+### Debugging
+
+```bash
+# Check if module was imported correctly
+./celilo module list
+
+# Verify configuration
+./celilo module config get homebridge
+
+# Check database directly
+sqlite3 celilo.db "SELECT * FROM modules;"
+sqlite3 celilo.db "SELECT * FROM module_configs WHERE module_id='homebridge';"
+
+# View encrypted secrets (verify they're not plaintext)
+sqlite3 celilo.db "SELECT name, substr(encrypted_value, 1, 20) || '...' AS encrypted FROM secrets WHERE module_id='homebridge';"
+
+# Check generated files exist
+ls -la /tmp/celilo/modules/homebridge/generated/
+
+# Validate generated Terraform
+cd /tmp/celilo/modules/homebridge/generated/terraform
+terraform fmt -check
+terraform validate
+```
+
+## Common Patterns
+
+### Bulk Configuration
+
+```bash
+# Set multiple values in sequence
+for key in vmid hostname container_ip gateway vlan cores memory storage; do
+  ./celilo module config set homebridge $key "${!key}"
+done
+```
+
+### Configuration from File
+
+```bash
+# Read config from YAML/JSON and apply
+# Example config.yml:
+#   homebridge:
+#     vmid: 2110
+#     hostname: iot
+#     ...
+
+# Using yq or jq
+while IFS='=' read -r key value; do
+  ./celilo module config set homebridge "$key" "$value"
+done < <(yq eval '.homebridge | to_entries | .[] | .key + "=" + (.value | tostring)' config.yml)
+```
+
+### Export Configuration
+
+```bash
+# Export all config for a module
+./celilo module config get homebridge > homebridge-config.txt
+
+# Export system config
+./celilo system config get > system-config.txt
+```
+
+### Validate Before Generate
+
+```bash
+# Check all required config is set
+REQUIRED_KEYS="vmid hostname container_ip gateway vlan"
+
+for key in $REQUIRED_KEYS; do
+  if ! ./celilo module config get homebridge $key &>/dev/null; then
+    echo "ERROR: Missing required config: $key"
+    exit 1
+  fi
+done
+
+# All required config present, generate
+./celilo module generate homebridge
+```
+
+## Tips & Tricks
+
+### 1. Use Absolute Paths for Module Import
+
+```bash
+# Relative path (preferred - works from celilo root)
+./celilo module import ../modules/homebridge
+
+# Absolute path (use your actual path)
+./celilo module import /path/to/celilo/modules/homebridge
+```
+
+The wrapper script handles relative path conversion automatically for `module import`.
+
+### 2. Quote Values with Spaces or Special Characters
+
+```bash
+# Correct
+./celilo module config set homebridge bridge_name "Home Bridge"
+./celilo system config set dns.fallback "8.8.8.8 1.1.1.1"
+
+# Wrong (will only set first word)
+./celilo module config set homebridge bridge_name Home Bridge
+```
+
+### 3. Store Vault Password Securely
+
+```bash
+# DO NOT store vault password in plain files
+# BAD: echo "password" > vault-pass.txt
+
+# GOOD: Use celilo CLI with process substitution
+ansible-vault view secrets.yml \
+  --vault-password-file=<(./celilo system vault-password)
+
+# GOOD: Export to environment (for scripts)
+export ANSIBLE_VAULT_PASSWORD=$(./celilo system vault-password)
+```
+
+### 4. Check Help for Any Command
+
+```bash
+./celilo help                    # General help
+./celilo module --help           # Module command help
+./celilo module config --help    # Config subcommand help (shows same general help)
+```
+
+### 5. Validate Manifests Before Import
+
+```bash
+# Module imports will validate automatically, but you can check manually
+cd backend
+bun run src/cli/index.ts module import ../modules/homebridge
+# Look for validation error messages
+```
+
+## Troubleshooting
+
+### "Module not found" After Import
+
+**Check import succeeded**:
+```bash
+./celilo module list
+```
+
+**Verify files copied**:
+```bash
+ls -la /tmp/celilo/modules/homebridge/
+```
+
+**Check database**:
+```bash
+sqlite3 celilo.db "SELECT id, name, state FROM modules;"
+```
+
+### "Variable not found" During Generate
+
+**Check configuration is set**:
+```bash
+./celilo module config get homebridge
+```
+
+**Check system config**:
+```bash
+./celilo system config get
+```
+
+**Identify missing variables**:
+```bash
+# Error message will show which variables are missing:
+# Error: Failed to resolve variables in templates:
+# terraform/main.tf:
+#   $self:cores: Self variable 'cores' not found in module configuration
+```
+
+Set the missing config and regenerate.
+
+### "Failed to generate Ansible secrets"
+
+**Check Ansible is installed**:
+```bash
+ansible-vault --version
+```
+
+**If Ansible is missing**:
+```bash
+# macOS
+brew install ansible
+
+# Ubuntu/Debian
+sudo apt-get install ansible
+```
+
+### Generated Files Not Where Expected
+
+**Default location**: `/tmp/celilo/modules/<module-id>/generated/`
+
+**Custom location**:
+```bash
+./celilo module generate homebridge --output /custom/path
+```
+
+**Check generation output**:
+```bash
+./celilo module generate homebridge
+# Output shows: "Output: /tmp/celilo/modules/homebridge/generated"
+```
+
+## Next Steps
+
+- Read [INTEGRATION_TESTS.md](backend/INTEGRATION_TESTS.md) for testing workflows
+- Read [backend/README.md](backend/README.md) for architecture details
+- Check [SETUP.md](SETUP.md) for installation and setup guides