dojo.md 0.2.0 → 0.2.2

package/courses/docker-container-debugging/scenarios/level-2/docker-compose-debugging.yaml

@@ -0,0 +1,66 @@
+meta:
+  id: docker-compose-debugging
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug Docker Compose issues — diagnose service dependency failures, health check misconfigurations, and environment variable problems"
+  tags: [Docker, Docker-Compose, dependencies, health-checks, services, intermediate]
+
+state: {}
+
+trigger: |
+  Your Docker Compose application has 4 services but only 2 start
+  correctly:
+
+  $ docker compose up -d
+  $ docker compose ps
+  NAME          SERVICE    STATUS              PORTS
+  api           api        Up (unhealthy)      0.0.0.0:8080->8080/tcp
+  db            db         Up (healthy)        5432/tcp
+  redis         redis      exited (1)
+  worker        worker     waiting
+
+  Problems:
+  1. redis exited with code 1:
+  $ docker compose logs redis
+  Fatal: Could not read config file '/etc/redis/custom.conf': No such file
+
+  The compose file mounts a config file that doesn't exist:
+  volumes:
+    - ./redis.conf:/etc/redis/custom.conf
+
+  2. worker is waiting because depends_on has condition: service_healthy
+  for redis, which exited (never became healthy).
+
+  3. api shows "unhealthy" — the health check fails:
+  healthcheck:
+    test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
+  But curl isn't installed in the API image. The health endpoint works
+  fine — the health CHECK command fails.
+
+  4. Hidden issue: the API reads REDIS_URL from environment but the
+  compose file sets REDIS_HOST instead:
+  environment:
+    - REDIS_HOST=redis
+    - REDIS_PORT=6379
+
+  Task: Explain Docker Compose debugging. Write: how depends_on with
+  health checks works (condition: service_healthy vs service_started),
+  how to debug health check failures, environment variable patterns in
+  Compose (environment, env_file, variable substitution), common
+  Compose configuration mistakes, and the docker compose debugging
+  commands (logs, ps, config, exec).
+
+assertions:
+  - type: llm_judge
+    criteria: "Service dependencies are explained — depends_on with condition: service_started only waits for container to start (not ready), condition: service_healthy waits for health check to pass. If a dependency exits or stays unhealthy, dependent services won't start. Health checks need: test command, interval, timeout, retries, start_period. The test command must work inside the container (tools must be installed)"
+    weight: 0.35
+    description: "Service dependencies"
+  - type: llm_judge
+    criteria: "Debugging workflow is practical — docker compose config validates and shows the resolved YAML (catches syntax errors, shows interpolated variables), docker compose logs <service> shows service logs, docker compose ps shows health status, docker compose exec <service> sh for interactive debugging, docker compose up --build to rebuild images, docker compose down -v for clean restart"
+    weight: 0.35
+    description: "Debugging workflow"
+  - type: llm_judge
+    criteria: "Environment and common mistakes are covered — Compose environment sources: environment key (inline), env_file (.env file reference), shell variables with ${VAR} substitution, .env file auto-loaded for Compose variables. Common mistakes: wrong env var name, missing env file, volume mount source doesn't exist (Docker creates directory), health check command not available in image, circular dependencies"
+    weight: 0.30
+    description: "Environment and mistakes"

package/courses/docker-container-debugging/scenarios/level-2/docker-exec-debugging.yaml

@@ -0,0 +1,71 @@
+meta:
+  id: docker-exec-debugging
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug running containers with docker exec — diagnose runtime issues using interactive shells, process inspection, and filesystem exploration inside containers"
+  tags: [Docker, exec, debugging, interactive, process-inspection, intermediate]
+
+state: {}
+
+trigger: |
+  Your API container is responding slowly but docker logs shows nothing
+  unusual. You need to investigate the running container from the inside:
+
+  $ docker exec -it api-server sh
+  OCI runtime exec failed: exec: "sh": executable file not found
+
+  The image is based on distroless — no shell at all! You can't exec
+  into it. But there are alternatives:
+
+  Solution 1 — Use debug image variant:
+  $ docker run --rm -it --pid=container:api-server --net=container:api-server \
+    busybox sh
+  # Shares PID and network namespace — can see processes and connections
+
+  Solution 2 — Docker debug (Docker Desktop 4.27+):
+  $ docker debug api-server
+  # Attaches a debug shell even to distroless containers
+
+  Once inside a container with a shell, the investigation:
+
+  $ docker exec api-server ps aux
+  PID   USER     TIME  COMMAND
+  1     node     2:30  node /app/server.js
+  45    node     0:05  /app/node_modules/.bin/sharp (zombie)
+  46    node     0:05  /app/node_modules/.bin/sharp (zombie)
+
+  Zombie processes! The Node.js app spawns child processes for image
+  processing but doesn't properly wait() for them. PID 1 (node) isn't
+  a proper init and doesn't reap orphaned children.
+
+  $ docker exec api-server cat /proc/1/status | grep Threads
+  Threads: 45
+
+  45 threads — the thread pool is exhausted because zombie processes
+  still hold thread pool slots.
+
+  $ docker exec api-server cat /proc/meminfo
+  $ docker exec api-server df -h
+  $ docker exec api-server netstat -tlnp
+
+  Task: Explain docker exec debugging techniques. Write: how to exec
+  into containers (interactive shell, single commands), debugging
+  distroless/minimal images (namespace sharing, debug containers),
+  process inspection (ps, /proc filesystem, zombie detection), resource
+  inspection (memory, disk, network from inside), and the PID 1 zombie
+  reaping problem.
+
+assertions:
+  - type: llm_judge
+    criteria: "Docker exec techniques are explained — docker exec -it <container> sh/bash for interactive shell, docker exec <container> <command> for single command. Flags: -it (interactive + tty), -u root (run as root even if container runs as non-root), -e KEY=VALUE (set env vars). For distroless images: share namespaces with debug container (--pid=container:X --net=container:X), use docker debug (Docker Desktop), or docker cp to copy tools in. Never install debug tools in production images"
+    weight: 0.35
+    description: "Exec techniques"
+  - type: llm_judge
+    criteria: "Process and resource inspection are covered — ps aux shows all processes. /proc filesystem: /proc/1/status (PID 1 details), /proc/meminfo (memory), /proc/net/tcp (connections). Zombie processes: Z state in ps, caused by parent not calling wait(). PID 1 responsibility: in containers, PID 1 must reap adopted orphan processes. Node.js/Python aren't proper init systems. Fix: use tini (--init flag) or dumb-init as PID 1. Resource inspection: df -h (disk), free -m (memory), ss -tlnp (listening ports)"
+    weight: 0.35
+    description: "Process inspection"
+  - type: llm_judge
+    criteria: "Practical debugging workflow is provided — systematic approach: (1) docker logs first (no exec needed), (2) docker stats for resource overview, (3) docker top for process list, (4) docker exec for detailed investigation. Inside container: check processes, check disk space, check network connections, check file modifications. Compare with expected state. Common findings: zombie processes, thread pool exhaustion, file descriptor leaks (ls /proc/1/fd | wc -l), temp files filling disk, unexpected network connections"
+    weight: 0.30
+    description: "Debugging workflow"

package/courses/docker-container-debugging/scenarios/level-2/image-layer-optimization.yaml

@@ -0,0 +1,81 @@
+meta:
+  id: image-layer-optimization
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug Docker image size issues — analyze layers, optimize caching, reduce image size, and fix build cache invalidation"
+  tags: [Docker, image-optimization, layers, caching, size-reduction, intermediate]
+
+state: {}
+
+trigger: |
+  Your Docker image is 1.2GB for a simple Node.js application. The
+  application code is only 5MB. Where did the rest come from?
+
+  $ docker images myapp
+  REPOSITORY   TAG     SIZE
+  myapp        latest  1.2GB
+
+  $ docker history myapp
+  IMAGE        CREATED BY                                      SIZE
+  abc123       CMD ["node", "server.js"]                       0B
+  def456       COPY . /app                                     850MB
+  ghi789       RUN npm install                                 280MB
+  jkl012       COPY package.json /app/                         2KB
+  mno345       WORKDIR /app                                    0B
+  pqr678       FROM node:20                                    1.1GB (base)
+
+  Wait, the base image is 1.1GB, but the total is 1.2GB? Docker layers
+  are additive but share the base. The real issues:
+
+  1. COPY . /app is 850MB — it copied node_modules, .git directory,
+     build artifacts, and test data! No .dockerignore file exists.
+
+  2. npm install adds 280MB but many are devDependencies not needed
+     in production.
+
+  3. The base image node:20 (Debian-based) is huge. node:20-alpine
+     would be ~180MB.
+
+  After creating .dockerignore:
+  node_modules
+  .git
+  *.log
+  test/
+  .env
+
+  And optimizing the Dockerfile:
+  FROM node:20-alpine
+  WORKDIR /app
+  COPY package*.json ./
+  RUN npm ci --only=production
+  COPY . .
+  CMD ["node", "server.js"]
+
+  New size: 180MB (from 1.2GB — 85% reduction!)
+
+  But builds are slow because changing ANY source file invalidates the
+  npm install cache. The COPY package*.json first + RUN npm install
+  pattern preserves the cache when only source code changes.
+
+  Task: Explain Docker image optimization. Write: how layers work
+  (each instruction creates a layer, layers are cached and shared),
+  how build cache invalidation works (any change to a layer invalidates
+  all subsequent layers), .dockerignore best practices, the dependency-
+  first pattern for caching, base image selection (full vs slim vs
+  alpine vs distroless), and how to analyze image size with docker
+  history and dive.
+
+assertions:
+  - type: llm_judge
+    criteria: "Layer mechanics are explained — each RUN, COPY, ADD creates a new layer. Layers are cached by content hash. If a layer's content changes, that layer and ALL subsequent layers are rebuilt. This is why instruction order matters: put stable instructions first (dependencies) and frequently changing ones last (source code). Layers are shared between images with common base"
+    weight: 0.35
+    description: "Layer mechanics"
+  - type: llm_judge
+    criteria: "Cache optimization is covered — dependency-first pattern: COPY package.json first, RUN npm install, then COPY source code. This way npm install is only re-run when dependencies change, not when source code changes. .dockerignore excludes files from build context (like .gitignore), preventing unnecessary cache busting and reducing context size. Use npm ci instead of npm install for reproducible builds"
+    weight: 0.35
+    description: "Cache optimization"
+  - type: llm_judge
+    criteria: "Size reduction techniques are practical — use slim/alpine base images (1.1GB → 180MB), multi-stage builds for compiled languages, .dockerignore to exclude unnecessary files, clean package manager cache in same RUN (apt-get clean && rm -rf /var/lib/apt/lists/*), use --only=production for Node.js, analyze with docker history or dive tool. Combine RUN commands to reduce layers"
+    weight: 0.30
+    description: "Size reduction"

package/courses/docker-container-debugging/scenarios/level-2/intermediate-debugging-shift.yaml

@@ -0,0 +1,73 @@
+meta:
+  id: intermediate-debugging-shift
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Combined intermediate debugging shift — diagnose multi-service Docker Compose application with build, networking, health check, and logging issues"
+  tags: [Docker, troubleshooting, combined, shift-simulation, intermediate]
+
+state: {}
+
+trigger: |
+  You're debugging a Docker Compose application that a colleague set up.
+  It has 5 services and "nothing works":
+
+  $ docker compose up -d
+  $ docker compose ps
+  NAME         SERVICE      STATUS                  PORTS
+  frontend     frontend     Up (unhealthy)          0.0.0.0:3000->3000/tcp
+  api          api          exited (1)
+  worker       worker       waiting
+  postgres     postgres     Up (healthy)            5432/tcp
+  redis        redis        Up 2 seconds            6379/tcp
+
+  Investigation reveals 5 interconnected issues:
+
+  1. api — Build failed silently, running old cached image:
+  $ docker compose logs api
+  Error: Cannot find module '/app/dist/server.js'
+  The Dockerfile has a multi-stage build but COPY --from uses wrong
+  stage name. The old image (without the build fix) was cached.
+  Fix: docker compose build --no-cache api
+
+  2. worker — depends_on api with condition: service_healthy, but api
+  exited so worker waits forever.
+
+  3. frontend — Marked unhealthy. Health check uses curl but the
+  alpine-based image doesn't have curl.
+
+  4. redis — Running but no persistence configured. Every restart
+  loses session data. Also, no log rotation — redis logs growing fast.
+
+  5. Hidden issue — postgres password is set in docker-compose.yaml
+  as an environment variable visible in docker inspect. Should use
+  Docker secrets or env_file with restricted permissions.
+
+  $ docker system df
+  TYPE         TOTAL    ACTIVE   SIZE     RECLAIMABLE
+  Images       45       5        12.5GB   10.2GB (81%)
+  Containers   12       5        500MB    200MB
+  Volumes      23       3        8.5GB    6.2GB (72%)
+
+  Additionally, 81% of images and 72% of volumes are reclaimable
+  (unused). Disk space is filling up.
+
+  Task: Walk through diagnosing and fixing all issues. Write: the
+  triage approach for multi-service Compose applications, the fix for
+  each issue, how to clean up unused Docker resources (docker system
+  prune), security best practices for secrets in Compose, and the
+  maintenance checklist for Docker environments.
+
+assertions:
+  - type: llm_judge
+    criteria: "All five issues are diagnosed and fixed — (1) stale cached image: docker compose build --no-cache, fix COPY --from stage name, (2) worker waiting: fix api first, worker will start automatically, (3) frontend health check: use wget instead of curl or install curl in Dockerfile, (4) redis: add volume for persistence, configure log rotation, (5) secrets: use env_file with chmod 600 or Docker secrets, never plain environment variables for passwords"
+    weight: 0.35
+    description: "All issues fixed"
+  - type: llm_judge
+    criteria: "Cleanup and maintenance are covered — docker system df shows disk usage breakdown, docker system prune removes unused containers/images/networks, docker volume prune removes unused volumes (careful!), docker builder prune clears build cache. Maintenance checklist: regular prune, log rotation configured globally, base image updates, security scanning, volume backup strategy"
+    weight: 0.35
+    description: "Cleanup and maintenance"
+  - type: llm_judge
+    criteria: "Triage methodology is systematic — check docker compose ps first (status overview), check logs for exited services, check health for unhealthy services, fix dependencies bottom-up (database → api → worker → frontend), verify networking between services, check for resource issues (disk, memory). Security audit: check for secrets in environment, image vulnerabilities, privileged containers"
+    weight: 0.30
+    description: "Triage methodology"

package/courses/docker-container-debugging/scenarios/level-2/logging-and-log-rotation.yaml

@@ -0,0 +1,76 @@
+meta:
+  id: logging-and-log-rotation
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug Docker logging issues — diagnose disk exhaustion from logs, configure log rotation, and set up logging drivers"
+  tags: [Docker, logging, log-rotation, json-file, disk-space, intermediate]
+
+state: {}
+
+trigger: |
+  Your Docker host is running out of disk space. Investigation reveals
+  container logs are the culprit:
+
+  $ df -h /var/lib/docker
+  Filesystem      Size  Used  Avail  Use%
+  /dev/sda1       100G  95G   5G     95%
+
+  $ du -sh /var/lib/docker/containers/*/
+  45G  /var/lib/docker/containers/a1b2c3d4.../
+  30G  /var/lib/docker/containers/b2c3d4e5.../
+  15G  /var/lib/docker/containers/c3d4e5f6.../
+
+  90GB of logs! The containers have been running for months with no
+  log rotation configured.
+
+  $ ls -lh /var/lib/docker/containers/a1b2c3d4.../*-json.log
+  -rw-r----- 1 root root 45G Dec 01 10:00 a1b2c3d4...-json.log
+
+  A single 45GB log file! The default json-file logging driver has
+  NO rotation by default.
+
+  Emergency fix:
+  $ truncate -s 0 /var/lib/docker/containers/a1b2c3d4.../*-json.log
+
+  Proper fix — configure log rotation per container:
+  docker run --log-opt max-size=10m --log-opt max-file=3 myapp
+
+  Or globally in /etc/docker/daemon.json:
+  {
+    "log-driver": "json-file",
+    "log-opts": {
+      "max-size": "10m",
+      "max-file": "3"
+    }
+  }
+
+  But there's a caveat: changing daemon.json only affects NEW containers.
+  Existing containers keep their original logging configuration.
+
+  Additional issue: one service uses the fluentd logging driver:
+  $ docker logs fluentd-service
+  Error response from daemon: configured logging driver does not
+  support reading
+
+  Some logging drivers don't support docker logs command!
+
+  Task: Explain Docker logging and log management. Write: the default
+  json-file driver behavior (no rotation!), how to configure log
+  rotation (max-size, max-file), logging drivers (json-file, syslog,
+  fluentd, journald), which drivers support docker logs, daemon.json
+  global configuration, and production logging strategies.
+
+assertions:
+  - type: llm_judge
+    criteria: "Default logging behavior is explained — json-file driver stores logs as JSON in /var/lib/docker/containers/<id>/<id>-json.log. NO rotation by default — logs grow indefinitely and can fill disk. This is the single most common Docker disk space issue. Logs capture stdout and stderr from the container's PID 1 process. Applications logging to files inside the container are NOT captured by docker logs"
+    weight: 0.35
+    description: "Default behavior"
+  - type: llm_judge
+    criteria: "Log rotation configuration is covered — per-container: --log-opt max-size=10m --log-opt max-file=3 (keeps 3 files of 10MB = 30MB max). Global via daemon.json (only affects new containers). Logging drivers: json-file (local, supports docker logs), syslog (remote syslog server), fluentd (log aggregation), journald (systemd). Not all drivers support docker logs — fluentd and syslog don't by default"
+    weight: 0.35
+    description: "Log rotation"
+  - type: llm_judge
+    criteria: "Production strategy is practical — development: json-file with rotation. Production: centralized logging (fluentd → Elasticsearch, or CloudWatch/Datadog). Use dual logging (Docker 20.10+) to keep docker logs working while forwarding to centralized system. Monitor /var/lib/docker disk usage. Set global log rotation in daemon.json as a safety net. Use docker system df to check disk usage breakdown"
+    weight: 0.30
+    description: "Production strategy"

package/courses/docker-container-debugging/scenarios/level-2/multi-stage-build-debugging.yaml

@@ -0,0 +1,76 @@
+meta:
+  id: multi-stage-build-debugging
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug multi-stage Docker builds — diagnose missing artifacts, stage naming issues, and layer optimization problems"
+  tags: [Docker, multi-stage, build, optimization, layers, intermediate]
+
+state: {}
+
+trigger: |
+  Your multi-stage build produces an image but the application binary
+  is missing at runtime:
+
+  Dockerfile:
+  FROM golang:1.22 AS builder
+  WORKDIR /src
+  COPY go.mod go.sum ./
+  RUN go mod download
+  COPY . .
+  RUN CGO_ENABLED=0 go build -o /app/server ./cmd/server
+
+  FROM alpine:3.19
+  WORKDIR /app
+  COPY --from=build /app/server .
+  CMD ["./server"]
+
+  $ docker build -t myapp .
+  Step 8/9: COPY --from=build /app/server .
+  ERROR: invalid from flag value build: not found
+
+  The COPY uses --from=build but the stage is named "builder"!
+  After fixing to --from=builder:
+
+  $ docker build -t myapp .
+  Successfully built abc123def456
+
+  $ docker run myapp
+  sh: ./server: not found
+
+  The binary exists but won't run! On alpine, the binary was compiled
+  with CGO_ENABLED=0 (static linking), which should work. But wait —
+  the binary was built for linux/amd64 and you're on an Apple Silicon
+  Mac (linux/arm64).
+
+  Fix: Add GOARCH=arm64 or use --platform:
+  RUN GOOS=linux GOARCH=$(go env GOARCH) CGO_ENABLED=0 go build ...
+
+  After fixing, the image works but is 150MB. The Go binary is only
+  8MB. Why is the image so big?
+
+  $ docker history myapp
+  Alpine base: 7MB, binary: 8MB, go module cache: 135MB
+
+  The go module cache was accidentally copied! The COPY . . in the
+  builder stage included the vendor directory.
+
+  Task: Explain multi-stage build debugging. Write: how multi-stage
+  builds work (multiple FROM, COPY --from), common stage naming issues,
+  how to inspect intermediate stages (--target), artifact path debugging,
+  cross-compilation issues, image size optimization, and when to use
+  distroless vs alpine vs scratch.
+
+assertions:
+  - type: llm_judge
+    criteria: "Multi-stage mechanics are explained — each FROM starts a new stage, stages can be named with AS keyword, COPY --from=<stage> copies files from a previous stage. Only the final stage becomes the image. Intermediate stages are discarded. Use --target to build up to a specific stage for debugging. Stage names are case-sensitive and must match exactly"
+    weight: 0.35
+    description: "Multi-stage mechanics"
+  - type: llm_judge
+    criteria: "Common issues are diagnosed — stage name mismatch in COPY --from (most common error), binary not found at runtime (wrong platform, dynamic linking on static-only base), accidentally copying build artifacts (caches, source code) into final stage, missing runtime dependencies (libc on scratch/distroless). Use docker history to see layer sizes and find bloat"
+    weight: 0.35
+    description: "Common issues"
+  - type: llm_judge
+    criteria: "Base image selection is compared — scratch: 0MB, no shell, no libc (only for fully static binaries), alpine: ~7MB, has shell and musl libc (good for debugging), distroless: 10-50MB, no shell (secure, minimal attack surface). For Go: scratch or distroless. For Node/Python: language-specific slim/distroless. Trade-off: smaller = more secure but harder to debug"
+    weight: 0.30
+    description: "Base image selection"

package/courses/docker-container-debugging/scenarios/level-2/network-debugging-tools.yaml

@@ -0,0 +1,67 @@
+meta:
+  id: network-debugging-tools
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug Docker networking with tools — use tcpdump, curl, netcat, and nslookup inside containers to diagnose connectivity issues"
+  tags: [Docker, networking, tcpdump, debugging-tools, connectivity, intermediate]
+
+state: {}
+
+trigger: |
+  Your containerized API intermittently fails to connect to an external
+  payment gateway. The error is "connection timed out" but only for
+  some requests. You need to capture network traffic to understand why.
+
+  Problem: your container uses a minimal image with no debugging tools:
+
+  $ docker exec api-server curl http://payment-api:8443/health
+  OCI runtime exec failed: exec: "curl": executable file not found
+
+  $ docker exec api-server ping payment-api
+  OCI runtime exec failed: exec: "ping": executable file not found
+
+  Solution 1 — Use a sidecar debug container on the same network:
+  $ docker run --rm -it --network container:api-server nicolaka/netshoot
+
+  Now you have all networking tools sharing the api-server's network
+  namespace:
+
+  $ nslookup payment-api
+  Name:    payment-api
+  Address: 172.18.0.5
+
+  $ curl -v http://payment-api:8443/health
+  * Connected to payment-api (172.18.0.5) port 8443
+
+  $ tcpdump -i eth0 host 172.18.0.5 -w /tmp/capture.pcap
+  (capture packets for analysis)
+
+  $ nc -zv payment-api 8443
+  Connection to payment-api 8443 port [tcp/https-alt] succeeded!
+
+  The tcpdump capture reveals: some DNS responses take 5+ seconds
+  because the container's /etc/resolv.conf has ndots:5, causing
+  multiple DNS lookups for external hostnames before trying the
+  actual hostname.
+
+  Task: Explain Docker network debugging with tools. Write: how to
+  use debugging containers (netshoot) with --network container: mode,
+  key tools (curl, tcpdump, nslookup, dig, netcat, ss, traceroute),
+  how to capture and analyze packets, common DNS issues in containers
+  (ndots, search domains), and debugging external connectivity from
+  containers.
+
+assertions:
+  - type: llm_judge
+    criteria: "Debug container approach is explained — use --network container:<target> to share the target container's network namespace. The debug container sees the same IPs, routes, DNS config. nicolaka/netshoot is the standard toolbox image. Alternative: install tools temporarily with docker exec (if package manager available). Never install debug tools in production images"
+    weight: 0.35
+    description: "Debug container approach"
+  - type: llm_judge
+    criteria: "Key tools and usage are covered — nslookup/dig for DNS resolution testing, curl -v for HTTP debugging (shows connection, TLS, headers), tcpdump for packet capture (filter by host, port, protocol), netcat (nc -zv) for port connectivity testing, ss -tlnp for listing listening ports, traceroute for routing path. Each tool answers a different question in the debugging process"
+    weight: 0.35
+    description: "Tools and usage"
+  - type: llm_judge
+    criteria: "DNS issues are explained — ndots:5 in container resolv.conf means names with < 5 dots get search domains appended first (app.payment.com → app.payment.com.default.svc.cluster.local first). This causes slow DNS for external hostnames. Fix: use FQDN with trailing dot (payment-api.external.com.) or adjust ndots. Also: DNS caching, TTL, and how Docker's embedded DNS (127.0.0.11) works on custom networks"
+    weight: 0.30
+    description: "DNS issues"

package/courses/docker-container-debugging/scenarios/level-2/pid1-signal-handling.yaml

@@ -0,0 +1,71 @@
+meta:
+  id: pid1-signal-handling
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug PID 1 and signal handling issues — diagnose containers that won't stop gracefully, zombie processes, and init system problems"
+  tags: [Docker, PID-1, signals, tini, graceful-shutdown, zombie-processes, intermediate]
+
+state: {}
+
+trigger: |
+  Your container takes exactly 10 seconds to stop every time:
+
+  $ time docker stop api-server
+  api-server
+  real    0m10.032s
+
+  Docker stop sends SIGTERM, waits 10 seconds (default grace period),
+  then sends SIGKILL. Your container is ignoring SIGTERM!
+
+  $ docker exec api-server ps aux
+  PID   USER   COMMAND
+  1     root   /bin/sh -c node server.js
+  7     root   node server.js
+
+  PID 1 is /bin/sh, not your application! The shell is PID 1, and
+  shells as PID 1 don't forward signals to child processes. Node.js
+  (PID 7) never receives SIGTERM.
+
+  This happens because the Dockerfile uses shell form:
+  CMD node server.js
+  (interpreted as: /bin/sh -c "node server.js")
+
+  vs exec form:
+  CMD ["node", "server.js"]
+  (node becomes PID 1 directly)
+
+  But even with exec form, there's another problem — PID 1 processes
+  don't have default signal handlers in Linux. Unless your app
+  explicitly handles SIGTERM, it gets ignored.
+
+  Additionally, after running for a while:
+  $ docker exec api-server ps aux | grep defunct
+  15    root   [worker] <defunct>
+  23    root   [worker] <defunct>
+
+  Zombie processes! PID 1 is supposed to reap orphaned child processes,
+  but your app doesn't do that.
+
+  Solution: Use an init system (tini or --init flag):
+  $ docker run --init api-server
+
+  Task: Explain the PID 1 problem in Docker. Write: why PID 1 is
+  special (signal handling, zombie reaping), shell form vs exec form
+  CMD/ENTRYPOINT, what happens during docker stop (SIGTERM → wait →
+  SIGKILL), how to use tini/dumb-init as an init process, the --init
+  flag, and how to implement graceful shutdown in applications.
+
+assertions:
+  - type: llm_judge
+    criteria: "PID 1 problem is explained — PID 1 in Linux has special behavior: it doesn't have default signal handlers (SIGTERM is ignored unless explicitly handled), and it's responsible for reaping zombie child processes (wait() on orphaned children). If your app is PID 1 and doesn't handle these, it won't shut down gracefully and zombies accumulate"
+    weight: 0.35
+    description: "PID 1 problem"
+  - type: llm_judge
+    criteria: "Shell vs exec form is explained — shell form (CMD node server.js) wraps in /bin/sh -c, making shell PID 1 and app a child process. Shell doesn't forward signals. Exec form (CMD ['node', 'server.js']) makes app PID 1 directly, receiving signals. ENTRYPOINT has the same distinction. docker stop sends SIGTERM to PID 1, waits --stop-timeout (default 10s), then SIGKILL"
+    weight: 0.35
+    description: "Shell vs exec form"
+  - type: llm_judge
+    criteria: "Solutions are practical — tini: lightweight init process (10KB), handles signal forwarding and zombie reaping. Install in Dockerfile: RUN apk add tini, ENTRYPOINT ['tini', '--'], CMD ['node', 'server.js']. Or use --init flag at runtime: docker run --init myapp. For application-level handling: implement SIGTERM handler (close connections, flush logs, exit cleanly). dumb-init is an alternative to tini"
+    weight: 0.30
+    description: "Solutions"

package/courses/docker-container-debugging/scenarios/level-2/security-scanning-basics.yaml

@@ -0,0 +1,67 @@
+meta:
+  id: security-scanning-basics
+  level: 2
+  course: docker-container-debugging
+  type: output
+  description: "Debug container security issues — scan images for vulnerabilities, understand CVE severity, and fix common security problems"
+  tags: [Docker, security, scanning, Trivy, CVE, vulnerability, intermediate]
+
+state: {}
+
+trigger: |
+  Your CI/CD pipeline started failing after adding a security scan step:
+
+  $ trivy image myapp:latest
+  Total: 147 (CRITICAL: 3, HIGH: 12, MEDIUM: 45, LOW: 87)
+
+  CRITICAL vulnerabilities:
+  ┌─────────────────┬──────────────┬──────────┬─────────────────────────┐
+  │ Library         │ Vulnerability│ Severity │ Fixed Version           │
+  ├─────────────────┼──────────────┼──────────┼─────────────────────────┤
+  │ openssl         │ CVE-2024-XXX │ CRITICAL │ 3.0.13                  │
+  │ curl            │ CVE-2024-YYY │ CRITICAL │ 8.6.0                   │
+  │ node            │ CVE-2024-ZZZ │ CRITICAL │ 20.11.1                 │
+  └─────────────────┴──────────────┴──────────┴─────────────────────────┘
+
+  Questions:
+  1. All 3 critical CVEs are in the base image (node:20.10.0), not in
+     your application code. Updating the base image to node:20.11.1
+     fixes all three.
+
+  2. Of the 12 HIGH vulnerabilities, 8 are in packages your app doesn't
+     even use — they're just in the base image.
+
+  3. After updating the base image, re-scanning still shows 45 MEDIUM
+     and 87 LOW vulnerabilities. Should you block deployment?
+
+  4. Your company's policy: block on CRITICAL and HIGH, allow MEDIUM
+     and below. But one HIGH CVE has no fix available yet.
+
+  $ trivy image --severity CRITICAL,HIGH myapp:latest
+  Total: 1 (HIGH: 1)  # After base image update
+
+  The unfixable HIGH CVE is in a transitive dependency. Options:
+  - Accept the risk and add to ignore list
+  - Switch to a different library
+  - Wait for upstream fix
+
+  Task: Explain container security scanning. Write: how vulnerability
+  scanners work (match packages against CVE databases), scanner tools
+  (Trivy, Docker Scout, Snyk), severity levels and what they mean,
+  how to reduce vulnerabilities (base image selection, minimal images),
+  handling unfixable vulnerabilities, and integrating scanning into
+  CI/CD pipelines.
+
+assertions:
+  - type: llm_judge
+    criteria: "Scanning mechanics are explained — scanners extract the package list from image layers (OS packages + language dependencies), then match against CVE databases (NVD, vendor advisories). Each CVE has a severity (CRITICAL/HIGH/MEDIUM/LOW) based on CVSS score. Most vulnerabilities come from the base image, not application code. Updating the base image is the most impactful fix"
+    weight: 0.35
+    description: "Scanning mechanics"
+  - type: llm_judge
+    criteria: "Reduction strategies are covered — use minimal base images (alpine, distroless, slim) to reduce attack surface, regularly update base images, use multi-stage builds to exclude build tools from final image, remove unnecessary packages. Scanner comparison: Trivy (free, fast, broad), Docker Scout (native Docker integration), Snyk (developer-focused, fix suggestions)"
+    weight: 0.35
+    description: "Reduction strategies"
+  - type: llm_judge
+    criteria: "CI/CD integration and policy are practical — fail builds on CRITICAL: trivy image --exit-code 1 --severity CRITICAL myapp. Allow ignore list for accepted risks (.trivyignore file). Use --ignore-unfixed to skip CVEs without available patches. Policy: define severity threshold per environment (block CRITICAL in prod, warn in dev). Scan on every build, schedule weekly full scans for drift detection"
+    weight: 0.30
+    description: "CI/CD integration"