theslopmachine 0.6.2 → 0.7.1

package/assets/slopmachine/scaffold-playbooks/database-module-matrix.md

@@ -0,0 +1,80 @@
+# Database Module Matrix
+
+Use this after selecting the backend family. This matrix defines the standard database wiring modules we should compose into backend and fullstack scaffolds.
+
+## Shared database wiring contract
+
+- if the scaffold has persistence, it gets a real `./init_db.sh`
+- app runtime reads one generated config path such as `APP_CONFIG_PATH=/runtime/config/app-config.json`
+- no checked-in `.env`
+- no committed secrets or baked database credentials
+- database services stay internal-only in Compose unless the prompt explicitly requires host access
+- healthchecks are mandatory for every database service container
+- `./run_tests.sh` must prove both app readiness and real database access
+
+At the config level, normalize on a small shared shape even when each framework maps it into native config objects:
+
+```json
+{
+  "database": {
+    "driver": "postgresql|mysql|sqlite|mongodb",
+    "url": "...",
+    "host": "db",
+    "port": 5432,
+    "name": "app",
+    "user": "app",
+    "password_file": "/run/secrets/db_password"
+  }
+}
+```
+
+## Database selection defaults
+
+| Prompt / selector state | Default database module | Why |
+| --- | --- | --- |
+| `database=postgresql` | PostgreSQL | neutral service-container SQL default across most backend families |
+| `database=mysql` | MySQL | explicit MySQL ask, existing MySQL estate, or conventional Laravel/MySQL pairing |
+| `database=sqlite` | SQLite | smallest honest local persistence path with no credentials |
+| `database=mongodb` | MongoDB | explicit document-store or flexible-schema requirement |
+| `database=not_specified` | SQLite for the smallest honest baseline | baseline should still prove persistence without forcing a network DB |
+
+When the prompt or repo clearly requires a **service-container relational DB** but leaves the engine open, prefer **PostgreSQL** except for conventional PHP/Laravel defaults where MySQL is still reasonable.
+
+## Database module matrix
+
+| Module | Safe baseline direction | Use it when | Secret / config handling model | Migration / init expectations | Docker composition expectations | State location default |
+| --- | --- | --- | --- | --- | --- | --- |
+| **PostgreSQL** | PostgreSQL `17.x`, internal service name `db`, healthcheck via `pg_isready` | prompt wants a real service DB, relational integrity, concurrent writers, or production-shaped SQL infrastructure | app reads one generated DSN or config object; local disposable credentials may live in Compose only when the DB is internal-only and clearly documented as throwaway; otherwise mount secret files | `./init_db.sh` waits for healthy DB then runs family-native migrations (Alembic, Django migrations, Flyway/Liquibase, Prisma/TypeORM/Drizzle, Eloquent, goose, etc.); idempotent seed/bootstrap is allowed | `db` service on the internal bridge network, `expose: 5432`, named volume for data, readiness-gated `depends_on`, no host port by default | **service container state** with a named Docker volume |
+| **MySQL** | MySQL `8.4 LTS`, internal service name `db`, healthcheck via `mysqladmin ping` | prompt explicitly wants MySQL/MariaDB-family behavior, existing repo already uses it, or Laravel/PHP conventions dominate | same config-file contract as PostgreSQL; never commit credentials; local-only disposable credentials are acceptable only for isolated internal Compose use | `./init_db.sh` runs framework-native migrations after healthcheck; schema/bootstrap stays idempotent; avoid raw hand-edited SQL dumps as the primary scaffold contract | `db` service on internal network, `expose: 3306`, named volume, readiness-gated startup, no host port unless explicitly requested | **service container state** with a named Docker volume |
+| **SQLite** | SQLite `3` via language/runtime driver, DB file under a mounted runtime path such as `/runtime/data/app.sqlite` | prompt leaves DB open, the scaffold only needs the smallest honest persistence layer, or the backend family is still in category-open baseline mode | no DB credentials; keep the file path in generated runtime config; never commit live `.sqlite` state into the repo | `./init_db.sh` creates the file path, runs schema creation or migrations against that file, and can safely recreate it for tests; once schema evolution becomes real, switch to the family's migration tool instead of ad hoc SQL | **no separate DB service**; the app service mounts a named volume for DB state; app health should still prove it can open/query the file | **local app-mounted state** in a named Docker volume by default; use a gitignored host bind mount only if the prompt explicitly needs inspectable local files |
+| **MongoDB** | MongoDB `8.x`, internal service name `db`, healthcheck via `mongosh --eval 'db.adminCommand({ ping: 1 })'` | prompt explicitly wants a document store, event/log style documents, flexible schema, or an existing Mongo repo | same generated config-file contract; use URI plus optional secret-file paths; local disposable users are acceptable only for internal-only Compose bootstrap | `./init_db.sh` should create users/databases as needed, apply indexes/validators, and seed any baseline documents idempotently; think “indexes + validators + bootstrap” rather than SQL migrations | `db` service on internal network, `expose: 27017`, named volume, healthcheck, readiness-gated startup; no host port by default | **service container state** with a named Docker volume |
+
+## Host-volume local state vs service-container state
+
+| Database module | Default state model | When to use host-volume local state instead |
+| --- | --- | --- |
+| PostgreSQL | service-container named volume | almost never at scaffold time; only when the prompt explicitly needs inspectable raw database files on the host |
+| MySQL | service-container named volume | almost never at scaffold time; same rule as PostgreSQL |
+| SQLite | app-mounted named volume | when the prompt explicitly wants a visible local file for manual inspection or offline tooling; keep the bind mount gitignored |
+| MongoDB | service-container named volume | almost never at scaffold time |
+
+## Family-to-database pairing defaults
+
+| Backend family | Smallest honest default when DB is open | Default network/service DB | Notes |
+| --- | --- | --- | --- |
+| Nest + Fastify | SQLite | PostgreSQL | MongoDB only when the prompt explicitly wants a document model |
+| Express + TypeScript | SQLite | PostgreSQL | MySQL is fine when the repo already uses it; not the neutral default |
+| FastAPI | SQLite | PostgreSQL | this is the current category-open backend default pairing |
+| Flask | SQLite | PostgreSQL | stay lightweight unless the prompt proves a larger service DB is needed |
+| Django | SQLite for tiny local scaffolds only | PostgreSQL | keep Django migrations first-class; use DRF only when the API surface warrants it |
+| Spring Boot | SQLite only for rare tiny local baselines | PostgreSQL | MySQL is acceptable for explicit compatibility; PostgreSQL remains the neutral default |
+| Go + Chi / Gin | SQLite | PostgreSQL | use MongoDB only for explicit document-store prompts |
+| Laravel | SQLite for tiny local/test baselines | MySQL | PostgreSQL is still acceptable when the repo or prompt asks for it |
+
+## Placeholder handling
+
+- `database=not_specified` means **choose the smallest honest DB module for the selected backend family**, not “skip the database.”
+- if both backend family and database are open, default to **FastAPI + SQLite** per the current backend baseline direction.
+- if the prompt explicitly implies a multi-user or production-shaped service DB but does not name an engine, default to **PostgreSQL**.
+- if the prompt says PHP only and leaves the DB open, default to **Laravel + MySQL** unless the repo already standardized on PostgreSQL.
+- `not_specified` never means “commit secrets later”; keep the config-file + secret-file contract from day one.

package/assets/slopmachine/scaffold-playbooks/django-default.md

@@ -0,0 +1,166 @@
+# Django Default Scaffold Playbook
+
+Use this playbook when the request explicitly wants Django or when a backend baseline should prefer Django's official project/app bootstrap path over lighter Python microframeworks.
+
+Verified lab: `/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/django-baseline`
+
+## Goal
+
+Create a Django baseline that:
+
+- uses Django's official bootstrap path
+- keeps defaults pinned and conservative
+- wires the backend to a safe default database pattern without `.env`
+- ships a real `docker compose up --build` path
+- ships a real containerized `./run_tests.sh`
+- includes minimal real Django tests
+- keeps the README honest about included versus excluded scope
+- stops at baseline infrastructure instead of product-specific features
+
+## Runtime contract
+
+- required runtime command: `docker compose up --build`
+- required broad test command: `./run_tests.sh`
+- required database bootstrap command: `./init_db.sh`
+- all three commands must be real and working
+
+## Official/bootstrap path
+
+Per Django's documented admin commands, the baseline should start from:
+
+```bash
+django-admin startproject config .
+python manage.py startapp core
+```
+
+Those commands preserve the standard Django project shape while keeping the app split explicit.
+
+## Chosen default stack and version pins
+
+- Python `3.12`
+- Django `5.2.6`
+- Gunicorn `23.0.0`
+- SQLite `3` via Django's built-in SQLite backend
+
+Why this default won:
+
+- Django provides the official project generator, ORM, migrations, and test runner in one maintained stack
+- SQLite keeps the baseline fast, reproducible, and secret-free while still proving real persistence
+- Gunicorn provides a safer container runtime than `runserver` while keeping the app bootstrap simple
+
+## Safe database/config pattern
+
+- always route runtime configuration through one explicit config file path such as `APP_CONFIG_PATH=/runtime/config/app-config.json`
+- generate that config file at startup with a small bootstrap script instead of committing `.env`
+- generate `SECRET_KEY` into a Docker-managed runtime file under `/runtime/state`
+- default the database to a SQLite file under `/runtime/db/django-baseline.sqlite3` in a named volume
+- provide `./init_db.sh` as the single database bootstrap entrypoint and make it run `python manage.py migrate --noinput`
+
+## Docker / Compose pattern
+
+- one `app` service
+- one disposable `test` service behind a `test` profile
+- host exposure loopback-bound only via `127.0.0.1::8000`
+- named volumes for runtime config, generated secret state, and SQLite data
+- app healthcheck must hit `/health` inside the container and the endpoint must perform a real DB query
+
+## Minimal baseline behavior
+
+At scaffold time, include at least:
+
+- `GET /health` proves app boot + DB connectivity
+- `POST /items` and `GET /items` prove ORM-backed persistence
+- one migration for the minimal model
+- `python manage.py test` wired into `./run_tests.sh`
+
+Minimum real Django tests:
+
+- one test for `GET /health`
+- one test for `POST /items` then `GET /items`
+
+## README requirements
+
+`README.md` must already state:
+
+- scaffold status versus implemented scope
+- chosen stack and pinned versions
+- required runtime command: `docker compose up --build`
+- required test command: `./run_tests.sh`
+- database bootstrap command: `./init_db.sh`
+- how runtime config and `SECRET_KEY` are generated safely without `.env`
+- what is intentionally excluded from the baseline
+
+## Verification commands actually run for this prototype
+
+Commands run in `/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/django-baseline`:
+
+```bash
+docker compose build
+docker compose up --build -d
+docker compose ps
+docker compose port app 8000
+curl -fsS "http://127.0.0.1:32781/health"
+curl -fsS -X POST "http://127.0.0.1:32781/items" -H "content-type: application/json" -d '{"name":"manual-smoke-item"}'
+curl -fsS "http://127.0.0.1:32781/items"
+docker compose down --volumes --remove-orphans
+./run_tests.sh
+python3 - <<'PY'
+import signal
+import subprocess
+import time
+from urllib.request import urlopen
+
+cwd = "/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/django-baseline"
+proc = subprocess.Popen(["docker", "compose", "up", "--build"], cwd=cwd)
+error = None
+try:
+    deadline = time.time() + 180
+    while time.time() < deadline:
+        try:
+            port_output = subprocess.check_output(["docker", "compose", "port", "app", "8000"], cwd=cwd, text=True).strip()
+            if port_output:
+                host = port_output.split(":")[-1]
+                with urlopen(f"http://127.0.0.1:{host}/health", timeout=2) as response:
+                    body = response.read().decode("utf-8")
+                    if response.status == 200 and '"status": "ok"' in body and '"database": "ok"' in body:
+                        print(f"docker compose up --build reached healthy /health on http://127.0.0.1:{host}/health")
+                        break
+        except Exception as exc:
+            error = exc
+        time.sleep(2)
+    else:
+        raise RuntimeError(f"django baseline never became ready: {error}")
+finally:
+    proc.send_signal(signal.SIGINT)
+    try:
+        proc.wait(timeout=30)
+    except subprocess.TimeoutExpired:
+        proc.kill()
+        proc.wait(timeout=30)
+    subprocess.run(["docker", "compose", "down", "-v", "--remove-orphans"], cwd=cwd, check=True)
+PY
+```
+
+Observed results:
+
+- `docker compose ps` reported the app `Up ... (healthy)` and `docker compose port app 8000` returned `127.0.0.1:32781`
+- `GET /health` returned `{"status": "ok", "database": "ok", "app_name": "django-baseline-lab", "environment": "development"}`
+- `POST /items` returned `{"id": 1, "name": "manual-smoke-item", "created_at": "2026-04-14T21:27:33.090Z"}`
+- `GET /items` returned `[{"id": 1, "name": "manual-smoke-item", "created_at": "2026-04-14T21:27:33.090Z"}]`
+- `./run_tests.sh` reported `Ran 2 tests in 0.005s`, `OK`, and `Django smoke check passed at http://127.0.0.1:32794`
+- the interactive `docker compose up --build` path reached a healthy `/health` endpoint and printed `docker compose up --build reached healthy /health on http://127.0.0.1:32796/health`
+
+## Known caveats
+
+- this baseline uses direct Django migrations for the initial schema; introduce more advanced migration workflows only when the prompt grows beyond a baseline lab
+- SQLite is the safe default for local baseline work, not a production recommendation for concurrent high-write deployments
+- admin/auth are present only as framework defaults; they are not configured as finished product features
+
+## What to reuse later
+
+- Django official bootstrap shape
+- config-file-plus-secret-file runtime pattern instead of `.env`
+- Docker-first runtime contract
+- `./run_tests.sh` wrapper pattern
+- `./init_db.sh` as the single DB bootstrap entrypoint
+- health endpoint + one persistence-backed route as the minimum real backend proof

package/assets/slopmachine/scaffold-playbooks/docker-baseline.md

@@ -0,0 +1,189 @@
+# Docker Baseline Scaffold Playbook
+
+Use this playbook when a scaffold needs a reusable Docker-first baseline rather than product-specific feature work.
+
+## Goal
+
+Ship a Docker/Compose baseline that is honest, reusable, and already verified in a minimal frontend + backend + database lab.
+
+## Runtime contract
+
+- required runtime command: `docker compose up --build`
+- preferred readiness check in automation: `docker compose up --build --wait`
+- required broad test command: `./run_tests.sh`
+- no `.env` dependency for the baseline itself
+
+Verified lab: `/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/docker-baseline`
+
+## Compose defaults
+
+- use `compose.yaml`
+- set an explicit project name with `name:` or require a clean `COMPOSE_PROJECT_NAME`
+- expose only one app-facing edge service to the host by default
+- keep backend, worker, and database services on the internal bridge network only
+- put test runners behind a `test` profile or run them with `docker compose run --rm ...`
+- use `depends_on: condition: service_healthy` for readiness gating, not just start order
+- keep runtime defaults in Compose only when they are clearly local-only and non-sensitive
+
+## Known good service shape
+
+- `frontend`: edge/static or reverse-proxy container; only host-published service
+- `backend`: app container; internal only; healthchecked over localhost inside container
+- `db`: stateful service with named volume and native healthcheck
+- `tester`: containerized verification runner, excluded from default runtime path
+
+## Dockerfile patterns
+
+- use slim official base images
+- run as non-root where practical
+- copy dependency manifests before source to preserve build cache hits
+- use BuildKit cache mounts for package managers when supported
+- keep entrypoints single-purpose and deterministic
+- prefer one process per container
+
+Lab patterns that worked:
+
+- backend: `python:3.12-slim`, non-root `appuser`, `pip` cache mount, `CMD ["python", "-m", "app.server"]`
+- frontend: `nginxinc/nginx-unprivileged:1.27-alpine`
+
+## Secret / bootstrap rules
+
+- baseline must not require a checked-in `.env`
+- never bake production secrets into images or Compose
+- for local-only scaffold bootstrap, generate any disposable credentials or runtime values into ignored files or Docker-managed runtime state instead of checking placeholder literals into tracked files
+- production or shared-environment secrets must come from external overrides, secret stores, CI injection, or generated ignored files
+- document every required secret/input explicitly instead of relying on hidden env state
+
+## Port / network rules
+
+- bind host ports to loopback by default: `127.0.0.1:HOST_PORT:CONTAINER_PORT`
+- prefer high, low-collision local ports
+- publish only the edge service to host
+- use `expose` for internal services instead of `ports`
+- use one explicit bridge network unless the scaffold has a real reason for more
+
+Known good lab default:
+
+- frontend published on `127.0.0.1:18080:8080`
+- backend exposed internally on `8000`
+- Postgres exposed internally on `5432`
+
+## Healthcheck / readiness rules
+
+- every long-running runtime service gets a real healthcheck
+- database healthcheck should use its native readiness command (`pg_isready`, `mysqladmin ping`, etc.)
+- backend healthcheck should hit an in-container localhost readiness endpoint that checks critical dependencies
+- edge service healthcheck should hit its own local health endpoint
+- downstream services should wait on upstream health, not arbitrary sleeps
+- `docker compose up --build --wait` is the honest CI/readiness path
+
+Known good lab behavior:
+
+- db healthy first
+- backend starts only after db healthy and its `/healthz` performs a real Postgres query
+- frontend starts only after backend healthy
+
+## Cache / volume guidance
+
+- use named volumes only for mutable service state that should survive container replacement
+- use build cache mounts for package manager caches
+- avoid bind-mounting package caches as a default scaffold behavior
+- keep test runners stateless and disposable
+- default cleanup path should support `docker compose down -v --remove-orphans`
+
+Known good lab defaults:
+
+- named volume: Postgres data only
+- build cache mount: pip cache in backend image build
+
+## `./run_tests.sh` containerization rules
+
+- `./run_tests.sh` owns the portable broad-path verification
+- it should start the stack with `docker compose up -d --build --wait`
+- it should run a dedicated test container with `docker compose run --rm tester`
+- it should clean up containers, networks, and named volumes on exit
+- tests should verify edge reachability, backend readiness, and database connectivity from containers, not just static lint output
+
+Known good lab behavior:
+
+- `tester` uses the backend image plus a mounted verification script
+- runtime stack stays out of the test profile
+- cleanup removes one-off tester containers before `down`
+
+## Verification commands actually run
+
+All commands below were executed against the lab directory above.
+
+```bash
+docker compose -f "/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/docker-baseline/compose.yaml" config
+
+docker compose -f "/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/docker-baseline/compose.yaml" up -d --build --wait
+
+curl -fsS "http://127.0.0.1:18080/api/message"
+
+./run_tests.sh
+
+python3 - <<'PY'
+import signal
+import subprocess
+import time
+from urllib.request import urlopen
+
+cwd = "/Users/yohannesakd/code/eaglepoint/demonstration/scaffold-lab/docker-baseline"
+proc = subprocess.Popen(["docker", "compose", "up", "--build"], cwd=cwd)
+error = None
+try:
+    deadline = time.time() + 180
+    while time.time() < deadline:
+        try:
+            with urlopen("http://127.0.0.1:18080/healthz", timeout=2) as response:
+                if response.status == 200:
+                    print("frontend healthz reached while docker compose up --build was active")
+                    break
+        except Exception as exc:
+            error = exc
+        time.sleep(2)
+    else:
+        raise RuntimeError(f"frontend healthz never became ready: {error}")
+finally:
+    proc.send_signal(signal.SIGINT)
+    try:
+        proc.wait(timeout=30)
+    except subprocess.TimeoutExpired:
+        proc.kill()
+        proc.wait(timeout=30)
+    subprocess.run(["docker", "compose", "down", "-v", "--remove-orphans"], cwd=cwd, check=True)
+PY
+```
+
+Observed results:
+
+- `docker compose config` resolved cleanly
+- `docker compose up -d --build --wait` reached healthy frontend/backend/db state
+- `curl` returned the proxied backend payload through the frontend edge container
+- `./run_tests.sh` passed and printed `Verified frontend, backend, and database baseline.`
+- the interactive `docker compose up --build` path reached frontend readiness before shutdown
+
+## Recommended defaults to reuse later
+
+- Docker-first runtime contract with `docker compose up --build`
+- `compose.yaml` plus explicit project naming
+- one host-published edge service, loopback-bound, high port
+- internal-only backend and db with `expose`
+- healthchecks on every runtime service
+- readiness-gated `depends_on`
+- non-root containers where practical
+- build cache mounts for dependencies; named volumes only for persistent state
+- containerized `./run_tests.sh` with disposable tester service
+- no `.env` dependency in the baseline itself
+
+## Acceptance checklist
+
+Baseline is acceptable when:
+
+- `docker compose up --build` works
+- `./run_tests.sh` works
+- the sample proves frontend/backend/db composition
+- host port exposure is minimal and loopback-bound
+- healthchecks drive readiness cleanly
+- the baseline does not rely on `.env`