popeye-cli 1.7.0 → 1.8.0

package/README.md

@@ -709,6 +709,70 @@ The `qaEnabled` field in `ProjectState` controls whether the 7-phase workflow is
 | `consensus.threshold` | 95 | Code plan consensus threshold |
 | `consensus.testPlanThreshold` | 90 | Test plan consensus threshold (lower to avoid over-engineering test plans) |
 
+### Deferred Database Integration
+
+Fullstack and ALL projects include a 3-phase deferred database integration system. Instead of requiring a running database at project creation time, Popeye generates all the necessary database scaffolding and provides tools to configure and apply it later.
+
+#### Phase 1: Types and Templates
+
+The database layer generates production-ready templates for:
+
+| Template Set | Contents |
+|-------------|----------|
+| **SQLAlchemy** | Connection module, startup hook, health route, conftest, Alembic config, initial migration |
+| **Alembic** | `alembic.ini`, `env.py`, initial migration script |
+| **pgvector** | Vector extension setup for AI/ML workloads |
+| **Docker** | `docker-compose.yml` with PostgreSQL service, health checks, volumes |
+| **Environment** | `.env.example` with `DATABASE_URL`, `ADMIN_SETUP_TOKEN` |
+
+Database configuration is tracked in `ProjectState.dbConfig` with the following status lifecycle:
+
+```
+none -> configured -> migrating -> ready
+                  \-> error
+```
+
+#### Phase 2: State Machine and CLI Commands
+
+The database lifecycle is managed by a state machine (`db-state-machine.ts`) that enforces valid transitions between statuses. Two CLI commands provide management:
+
+**`/db [action]`** -- Database management:
+- `/db status` -- Show current database configuration and status
+- `/db configure` -- Interactively set DATABASE_URL and write to `.env`
+- `/db apply` -- Run the setup pipeline (create tables, run migrations)
+
+**`/doctor`** -- Readiness checks (8 checks):
+- `.env` file exists with `DATABASE_URL`
+- Backend directory structure is valid
+- `requirements.txt` includes database dependencies
+- Alembic configuration is present
+- Docker Compose includes postgres service
+- Database connection is reachable
+- Migrations are up to date
+- Health endpoint returns OK
+
+#### Phase 3: Admin Wizard UI
+
+The admin wizard generates a complete database setup experience with both backend and frontend components:
+
+**Backend (FastAPI):**
+- `admin_auth.py` middleware -- Validates `X-Admin-Token` header against `ADMIN_SETUP_TOKEN` env var
+- `admin_db.py` routes -- RESTful endpoints under `/api/admin/db`:
+  - `GET /status` -- Current DB status, migration count, connection info
+  - `POST /configure` -- Save database URL to `.env`
+  - `POST /test` -- Test database connectivity
+  - `POST /apply` -- Run Alembic migrations
+  - `POST /rollback` -- Roll back the last migration
+
+**Frontend (React):**
+- `useAdminApi` hook -- Authenticated API calls with `X-Admin-Token` header
+- `DbStatusBanner` -- Displays current database status with color-coded indicators
+- `ConnectionForm` -- Database URL input with connection test button
+- `MigrationProgress` -- Real-time migration progress display
+- `DbSetupStepper` -- Step-by-step wizard guiding the full setup flow
+
+Both the `fullstack.ts` and `all.ts` generators wire the admin wizard layer automatically. The admin wizard files are included in `getFullstackProjectFiles()` and `getAllProjectFiles()` for validation.
+
 ### Observability Features
 
 - **Workflow Logging**: Detailed logs written to `docs/WORKFLOW_LOG.md`
@@ -855,6 +919,9 @@ popeye
 /model [provider] [model]  Show/set AI model (openai/gemini/grok)
 /model <provider> list     Show known models for a provider
 /upgrade [target]          Upgrade project type (e.g., fullstack -> all)
+/overview [fix]            Project review with analysis; fix to auto-discover docs
+/db [action]               Database management (status/configure/apply)
+/doctor                    Run database and project readiness checks
 /info                      Show system info (Claude CLI status, API keys, etc.)
 /clear                     Clear screen
 /exit                      Exit interactive mode
@@ -1150,6 +1217,13 @@ my-project/
 │   │   ├── src/
 │   │   │   ├── components/
 │   │   │   │   └── ui/        # shadcn/ui components
+│   │   │   ├── admin/              # Admin wizard components
+│   │   │   │   ├── useAdminApi.ts      # Authenticated API hook
+│   │   │   │   ├── DbStatusBanner.tsx  # DB status indicator
+│   │   │   │   ├── ConnectionForm.tsx  # DB URL input + test
+│   │   │   │   ├── MigrationProgress.tsx # Migration progress
+│   │   │   │   ├── DbSetupStepper.tsx  # Setup wizard stepper
+│   │   │   │   └── index.ts           # Admin component exports
 │   │   │   ├── pages/
 │   │   │   ├── hooks/
 │   │   │   ├── lib/
@@ -1165,11 +1239,22 @@ my-project/
 │   │
 │   └── backend/               # FastAPI backend
 │       ├── src/
-│       │   ├── api/
-│       │   │   └── routes/
+│       │   ├── {package}/
+│       │   │   ├── routes/
+│       │   │   │   ├── health_db.py    # DB health endpoint
+│       │   │   │   └── admin_db.py     # Admin wizard routes
+│       │   │   ├── middleware/
+│       │   │   │   ├── __init__.py
+│       │   │   │   └── admin_auth.py   # X-Admin-Token validation
+│       │   │   ├── db.py               # SQLAlchemy connection module
+│       │   │   └── main.py
 │       │   ├── models/
 │       │   ├── services/
 │       │   └── main.py
+│       ├── alembic/                    # Database migrations
+│       │   ├── env.py
+│       │   └── versions/
+│       ├── alembic.ini
 │       ├── tests/
 │       │   └── conftest.py
 │       ├── pyproject.toml
@@ -1215,7 +1300,7 @@ my-project/
 ├── README.md
 ├── .gitignore
 ├── .env.example
-├── docker-compose.yml         # Full stack orchestration
+├── docker-compose.yml         # Full stack orchestration (includes PostgreSQL service)
 ├── popeye.md                  # Project configuration
 └── .popeye/
     ├── state.json
@@ -1434,8 +1519,10 @@ src/
 ├── cli/                  # CLI interface
 │   ├── index.ts          # Command setup
 │   ├── output.ts         # Output formatting
-│   ├── interactive.ts    # REPL mode (with /model, /upgrade, /overview commands)
+│   ├── interactive.ts    # REPL mode (with /model, /upgrade, /overview, /db, /doctor commands)
 │   └── commands/         # Individual commands
+│       ├── db.ts         # Database management CLI (status/configure/apply)
+│       └── doctor.ts     # Readiness checks (8 checks for DB and project health)
 ├── adapters/             # AI service adapters
 │   ├── claude.ts         # Claude Agent SDK (with rate limiting)
 │   ├── openai.ts         # OpenAI API (default reviewer, marketing persona for websites)
@@ -1456,6 +1543,8 @@ src/
 │   ├── website-context.ts # Doc discovery, brand assets, content context, dual-mode validation
 │   ├── website-content-scanner.ts # Post-generation placeholder fingerprint scanner
 │   ├── doc-parser.ts     # Product doc parsing (name, tagline, features, pricing, color)
+│   ├── database.ts       # DB layer orchestration (generatePythonDatabaseLayer, getDatabaseFiles)
+│   ├── admin-wizard.ts   # Admin wizard orchestration (generateAdminWizardLayer, getAdminWizardFiles)
 │   ├── all.ts            # ALL project scaffolding (exports 5 generator functions)
 │   └── templates/        # File templates
 │       ├── python.ts
@@ -1466,6 +1555,10 @@ src/
 │       ├── website-components.ts # Header, Footer, Navigation components
 │       ├── website-seo.ts      # JSON-LD, sitemap, robots, 404, 500, manifest, meta
 │       ├── website-conversion.ts # Lead capture route, contact form, env examples
+│       ├── database-python.ts  # SQLAlchemy + Alembic + pgvector templates (12 functions)
+│       ├── database-docker.ts  # Docker Compose + postgres service templates
+│       ├── admin-wizard-python.ts  # FastAPI admin auth middleware + admin DB routes
+│       ├── admin-wizard-react.ts   # React admin wizard components (status, forms, stepper)
 │       └── index.ts            # Template module exports
 ├── state/                # State management
 │   ├── persistence.ts    # File operations
@@ -1482,6 +1575,8 @@ src/
 │   ├── milestone-workflow.ts
 │   ├── task-workflow.ts  # 7-phase task workflow with QA gate
 │   ├── tester.ts         # QA skill: test planning, review, fix plans (provider-agnostic)
+│   ├── db-state-machine.ts # DB lifecycle state transitions (canTransition, transitionDbStatus)
+│   ├── db-setup-runner.ts  # DB setup pipeline runner (env parsing, migration prereqs)
 │   ├── test-runner.ts    # Test execution
 │   ├── workflow-logger.ts # Persistent logging (test-planning, test-review stages)
 │   ├── plan-storage.ts   # Consensus docs storage (per-app feedback)
@@ -1498,9 +1593,11 @@ src/
 │   └── auto-fix.ts       # Automatic error fixing (enhanced ENOENT tracking)
 └── types/                # TypeScript types
     ├── project.ts        # OutputLanguage, isWorkspace(), flexible OpenAIModelSchema
-    ├── workflow.ts       # ProjectStateSchema (qaEnabled, qa* task fields)
+    ├── workflow.ts       # ProjectStateSchema (qaEnabled, dbConfig, qa* task fields)
     ├── consensus.ts      # GeminiModelSchema, GrokModelSchema, reviewerPersona, testPlanThreshold
     ├── tester.ts         # TestVerdict, TestPlanOutput, TestRunReview, TestFixPlan
+    ├── database.ts       # DbStatus, DbMode, DbConfig, DbSetupStep Zod schemas
+    ├── database-runtime.ts # SetupStepResult, ReadinessCheck runtime schemas
     └── website-strategy.ts # WebsiteStrategyDocument, BrandAssetsContract, DesignTokens
 ```
 

package/cheatsheet.md

@@ -0,0 +1,407 @@
+# Popeye CLI Cheatsheet
+
+Quick reference for all Popeye CLI commands, interactive mode slash commands, and configuration options.
+
+---
+
+## CLI Commands
+
+### `popeye-cli create <idea>`
+
+Create a new project from a natural language description.
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-n, --name <name>` | Project name | Auto-generated |
+| `-l, --language <lang>` | Output language (`python`, `typescript`, `fullstack`, `website`, `all`) | `python` |
+| `-m, --model <model>` | OpenAI model for consensus | `gpt-4o` |
+| `-o, --output <dir>` | Output directory | Current directory |
+| `--threshold <percent>` | Consensus threshold percentage | `95` |
+| `--max-iterations <n>` | Maximum consensus iterations | `5` |
+| `--skip-scaffold` | Skip initial project scaffolding | `false` |
+
+```bash
+popeye-cli create "todo app with user authentication" -l fullstack -n my-todo
+```
+
+---
+
+### `popeye-cli interactive` (alias: `i`)
+
+Start interactive mode for guided project creation and management. This is the default when running `popeye-cli` with no arguments.
+
+```bash
+popeye-cli interactive
+popeye-cli i
+```
+
+---
+
+### `popeye-cli status [directory]`
+
+Show current project status and progress.
+
+| Option | Description |
+|--------|-------------|
+| `-v, --verbose` | Show detailed status |
+| `--json` | Output as JSON |
+
+```bash
+popeye-cli status ./my-project --verbose
+```
+
+---
+
+### `popeye-cli validate [directory]`
+
+Validate that a project structure is complete and ready for execution.
+
+```bash
+popeye-cli validate ./my-project
+```
+
+---
+
+### `popeye-cli summary [directory]`
+
+Show a detailed project summary including plan, milestones, and current progress.
+
+```bash
+popeye-cli summary ./my-project
+```
+
+---
+
+### `popeye-cli resume [directory]`
+
+Resume an interrupted workflow from where it left off.
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `--threshold <percent>` | Consensus threshold | `95` |
+| `--max-iterations <n>` | Max consensus iterations | `5` |
+| `--max-retries <n>` | Max task retries | `3` |
+
+```bash
+popeye-cli resume ./my-project
+```
+
+---
+
+### `popeye-cli reset [directory]`
+
+Reset a project to a specific phase, discarding progress beyond that point.
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-p, --phase <phase>` | Phase to reset to (`plan`, `execution`) | `plan` |
+| `-f, --force` | Skip confirmation prompt | `false` |
+
+```bash
+popeye-cli reset ./my-project --phase plan --force
+```
+
+---
+
+### `popeye-cli cancel [directory]`
+
+Cancel and delete a project entirely.
+
+| Option | Description |
+|--------|-------------|
+| `-f, --force` | Skip confirmation prompt |
+
+```bash
+popeye-cli cancel ./my-project --force
+```
+
+---
+
+### `popeye-cli auth <subcommand>`
+
+Manage authentication for AI provider services.
+
+| Subcommand | Description |
+|------------|-------------|
+| `status` | Show authentication status for all services |
+| `login [service]` | Authenticate with a service (`claude`, `openai`, `gemini`, `grok`, `all`) |
+| `logout [service]` | Remove stored credentials for a service |
+| `claude` | Authenticate with Claude CLI |
+| `openai` | Authenticate with OpenAI API |
+| `gemini` | Authenticate with Gemini API |
+| `grok` | Authenticate with xAI Grok API |
+
+The `login`, `openai`, `gemini`, and `grok` subcommands accept `--api-key <key>` to provide a key directly.
+
+```bash
+popeye-cli auth status
+popeye-cli auth login openai --api-key sk-...
+popeye-cli auth logout all
+```
+
+---
+
+### `popeye-cli config <subcommand>`
+
+Manage CLI configuration settings.
+
+| Subcommand | Description |
+|------------|-------------|
+| `show` | Show current configuration (`--json` for JSON output) |
+| `defaults` | Show default configuration values (`--json` for JSON output) |
+| `get <key>` | Get a specific config value (e.g., `consensus.threshold`) |
+| `path` | Show the configuration file path |
+| `init` | Create a configuration file (`-f, --format <json\|yaml>`) |
+
+```bash
+popeye-cli config show --json
+popeye-cli config get consensus.threshold
+popeye-cli config init --format yaml
+```
+
+---
+
+### `popeye-cli db <subcommand>`
+
+Manage database configuration and setup for fullstack/all projects.
+
+| Subcommand | Description |
+|------------|-------------|
+| `status [directory]` | Show database configuration and lifecycle status |
+| `configure [directory]` | Configure database mode (local Docker or managed) and connection URL |
+| `apply [directory]` | Run the full setup pipeline: connectivity check, extensions, migrations, seed, readiness |
+
+The `apply` subcommand accepts `--skip-seed` to skip the seed step.
+
+```bash
+popeye-cli db status ./my-project
+popeye-cli db configure ./my-project
+popeye-cli db apply ./my-project --skip-seed
+```
+
+**Database lifecycle**: `unconfigured` -> `configured` -> `applying` -> `ready` (or `error`)
+
+---
+
+### `popeye-cli doctor [directory]`
+
+Run comprehensive project and database readiness checks.
+
+Checks performed:
+1. **Project State** -- Verifies `.popeye/` state directory exists
+2. **DB Layer** -- Confirms database layer was generated
+3. **Docker Compose** -- Checks PostgreSQL service is defined
+4. **DATABASE_URL** -- Validates the env var is configured
+5. **DB Reachability** -- Tests actual database connectivity
+6. **pgvector Extension** -- Checks if the vector extension is available
+7. **Migrations Applied** -- Queries `alembic_version` for migration status
+8. **Health Endpoint** -- Pings the backend `/health/db` endpoint
+
+```bash
+popeye-cli doctor ./my-project
+```
+
+---
+
+## Interactive Mode Slash Commands
+
+Enter these commands during an interactive session (started via `popeye-cli interactive`).
+
+### Help and Info
+
+| Command | Description |
+|---------|-------------|
+| `/help`, `/h`, `/?` | Show all available commands |
+| `/info`, `/check` | Show system info: Claude CLI status, API auth status, environment |
+
+### Project Creation and Management
+
+| Command | Description |
+|---------|-------------|
+| `/new <idea>` | Start a new project (skips existing project check) |
+| `/resume` | Resume an interrupted project with project discovery |
+| `/status` | Show current project status and progress |
+| `/overview [fix]` | Show full project plan and milestone review. Add `fix` to re-discover docs and auto-fix issues |
+
+```
+/new todo app with drag and drop
+/resume
+/overview fix
+```
+
+### Authentication
+
+| Command | Description |
+|---------|-------------|
+| `/auth` | Re-run the authentication flow for all AI services |
+
+### Configuration
+
+| Command | Description |
+|---------|-------------|
+| `/config` | Show full configuration summary |
+| `/config reviewer <provider>` | Set reviewer model (`openai`, `gemini`, `grok`) |
+| `/config arbitrator <provider\|off>` | Set arbitrator model or disable it |
+| `/config language <lang>` | Set project output language |
+| `/config model <provider> [model]` | Show or set AI model for a provider |
+
+### Language and Model Selection
+
+| Command | Description |
+|---------|-------------|
+| `/language <lang>`, `/lang`, `/l` | Set output language: `be`, `fe`, `fs`, `web`, `all` |
+| `/model` | Show current models for all providers |
+| `/model <provider>` | Show available models for a provider |
+| `/model <provider> <model>` | Set a specific model for a provider |
+
+```
+/lang fullstack
+/model openai gpt-4o-mini
+/model gemini gemini-2.0-flash
+/model grok grok-3
+```
+
+### Project Upgrade
+
+| Command | Description |
+|---------|-------------|
+| `/upgrade` | Show interactive upgrade menu |
+| `/upgrade <target>` | Upgrade project to a different type (`fullstack`, `website`, `all`, etc.) |
+
+Available upgrade paths depend on the current project type:
+- `python` -> `fullstack`, `all`
+- `typescript` -> `fullstack`, `all`
+- `fullstack` -> `all`
+- `website` -> `all`
+
+```
+/upgrade fullstack
+/upgrade all
+```
+
+### Database and Health
+
+| Command | Description |
+|---------|-------------|
+| `/db status` | Show database lifecycle status |
+| `/db configure` | Configure database (redirects to CLI) |
+| `/db apply` | Apply database setup (redirects to CLI) |
+| `/doctor` | Run all readiness checks inline |
+
+### Session Control
+
+| Command | Description |
+|---------|-------------|
+| `/clear`, `/cls` | Clear screen and redraw the UI |
+| `/exit`, `/quit`, `/q` | Exit Popeye CLI |
+
+### Default Behavior
+
+Typing anything without a `/` prefix treats the input as a project idea for creation or refinement.
+
+---
+
+## Language Aliases
+
+| Language | Aliases | What it generates |
+|----------|---------|-------------------|
+| `python` | `be`, `backend`, `py` | FastAPI backend API |
+| `typescript` | `fe`, `frontend`, `ts` | React + Vite frontend |
+| `fullstack` | `fs` | Monorepo: React frontend + FastAPI backend + PostgreSQL |
+| `website` | `web` | Next.js marketing/landing website |
+| `all` | -- | Complete stack: frontend + backend + website + shared packages |
+
+---
+
+## Available AI Models
+
+### OpenAI
+
+`gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `o1-preview`, `o1-mini` (custom models also accepted)
+
+### Gemini
+
+`gemini-2.0-flash`, `gemini-2.0-pro`, `gemini-1.5-flash`, `gemini-1.5-pro` (custom models also accepted)
+
+### Grok (xAI)
+
+`grok-3`, `grok-3-mini`, `grok-2` (custom models also accepted)
+
+---
+
+## Configuration Files
+
+Popeye looks for configuration in these files (in order):
+
+- `.popeyerc`
+- `.popeyerc.json`
+- `.popeyerc.yaml`
+- `popeye.config.js`
+
+### Config Sections
+
+| Section | Keys | Description |
+|---------|------|-------------|
+| `consensus` | `threshold`, `maxIterations`, `temperature`, `maxTokens` | Consensus engine settings |
+| `apis` | `openai.model`, `openai.timeout` | API provider settings |
+| `project` | `defaultLanguage`, `defaultName` | Project defaults |
+| `directories` | `output`, `state` | Directory paths |
+| `output` | `verbose`, `colors`, `progress` | Display settings |
+
+---
+
+## Database Lifecycle States
+
+```
+unconfigured ──> configured ──> applying ──> ready
+                     ^                        │
+                     │            error <──────┘
+                     │              │
+                     └──────────────┘
+```
+
+| State | Meaning |
+|-------|---------|
+| `unconfigured` | No `DATABASE_URL` set, DB layer not configured |
+| `configured` | URL set and DB reachable, migrations not yet applied |
+| `applying` | Setup pipeline is running (migrations, extensions, seed) |
+| `ready` | All checks passed, database is operational |
+| `error` | Setup failed, can retry from `configured` |
+
+---
+
+## Setup Pipeline Steps
+
+When you run `popeye-cli db apply`, the pipeline executes these steps in order:
+
+1. **check_connection** -- Verify database is reachable
+2. **ensure_extensions** -- Create required PostgreSQL extensions (pgvector)
+3. **apply_migrations** -- Run `alembic upgrade head`
+4. **seed_minimal** -- Execute seed script if present
+5. **readiness_tests** -- Verify database is fully operational
+6. **mark_ready** -- Transition status to `ready`
+
+---
+
+## Quick Examples
+
+```bash
+# Create a fullstack project
+popeye-cli create "task management app" -l fullstack -n taskmaster
+
+# Start interactive mode
+popeye-cli
+
+# Check project health
+popeye-cli doctor ./taskmaster
+
+# Set up the database
+popeye-cli db configure ./taskmaster
+popeye-cli db apply ./taskmaster
+
+# Resume after interruption
+popeye-cli resume ./taskmaster
+
+# Reset and re-plan
+popeye-cli reset ./taskmaster --phase plan
+```

package/dist/cli/commands/db.d.ts

@@ -0,0 +1,10 @@
+/**
+ * Database CLI command
+ * Provides subcommands: status, configure, apply
+ */
+import { Command } from 'commander';
+/**
+ * Create the db command with subcommands
+ */
+export declare function createDbCommand(): Command;
+//# sourceMappingURL=db.d.ts.map

package/dist/cli/commands/db.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"db.d.ts","sourceRoot":"","sources":["../../../src/cli/commands/db.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AA2PpC;;GAEG;AACH,wBAAgB,eAAe,IAAI,OAAO,CAqBzC"}