dojo.md 0.1.0 → 0.2.1

package/courses/github-actions-cicd/scenarios/level-3/security-hardening.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: security-hardening
+  level: 3
+  course: github-actions-cicd
+  type: output
+  description: "Harden GitHub Actions security — prevent supply chain attacks, pin actions by SHA, implement OIDC, and secure workflows"
+  tags: [github, actions, security, supply-chain, OIDC, hardening, advanced]
+
+state: {}
+
+trigger: |
+  Your security team conducted an audit of your GitHub Actions setup
+  and found critical vulnerabilities. You need to remediate all findings
+  and implement a security hardening plan.
+
+  Audit findings:
+
+  Finding 1 — Supply chain risk (Critical):
+  All 45 third-party actions are referenced by tag (e.g., @v4) not by
+  SHA. A compromised action tag could execute malicious code in your
+  CI. Example: `uses: some-popular-action/upload@v3` — if that
+  maintainer's account is compromised, v3 could be overwritten.
+
+  Finding 2 — Overly permissive GITHUB_TOKEN (High):
+  All workflows run with default GITHUB_TOKEN permissions (read+write
+  on all scopes). Most workflows only need read access to contents
+  and write access to pull-requests.
+
+  Finding 3 — Fork PR vulnerability (High):
+  Workflows triggered by pull_request_target have access to secrets
+  and run in the context of the base branch. A malicious fork could
+  submit a PR that exfiltrates secrets.
+
+  Finding 4 — No OIDC authentication (Medium):
+  AWS deployments use long-lived access keys stored as GitHub Secrets.
+  These keys never rotate and would be exposed if a workflow is
+  compromised. Should use OIDC for keyless authentication.
+
+  Finding 5 — Script injection (Medium):
+  Several workflows use github.event.pull_request.title directly in
+  run: commands. A PR title containing backticks or $() could execute
+  arbitrary commands.
+
+  Task: Remediate all 5 findings. For each, write: the vulnerability
+  explanation (how it could be exploited), the remediation (specific
+  YAML changes), the verification steps (how to confirm the fix works),
+  and the ongoing prevention strategy. Include: the complete OIDC setup
+  for AWS (IAM role, trust policy, workflow changes), the GITHUB_TOKEN
+  permission lockdown, and a security checklist for all future workflows.
+
+assertions:
+  - type: llm_judge
+    criteria: "All 5 vulnerabilities are remediated correctly — actions pinned by full SHA, GITHUB_TOKEN permissions restricted with permissions: key at workflow and job level, pull_request_target secured against fork attacks, OIDC configured for AWS, and script injection prevented with environment variables"
+    weight: 0.35
+    description: "Correct vulnerability remediation"
+  - type: llm_judge
+    criteria: "OIDC setup is complete — includes the AWS IAM OIDC identity provider, the IAM role trust policy with correct GitHub conditions (repo, branch, environment), the workflow changes to use aws-actions/configure-aws-credentials with OIDC, and explains why this is more secure than static keys"
+    weight: 0.35
+    description: "Complete OIDC setup"
+  - type: llm_judge
+    criteria: "Security checklist is actionable — covers at least 10 security practices for GitHub Actions, is formatted as a PR review checklist, and includes automated enforcement options (e.g., CodeQL for workflow scanning, required workflow for org-wide security policy)"
+    weight: 0.30
+    description: "Actionable security checklist"

package/courses/github-actions-cicd/scenarios/level-3/self-hosted-runners.yaml

@@ -0,0 +1,60 @@
+meta:
+  id: self-hosted-runners
+  level: 3
+  course: github-actions-cicd
+  type: output
+  description: "Manage self-hosted runners — set up, secure, scale, and maintain self-hosted runner infrastructure for GitHub Actions"
+  tags: [github, actions, self-hosted-runners, infrastructure, scaling, advanced]
+
+state: {}
+
+trigger: |
+  Your company is migrating from GitHub-hosted to self-hosted runners
+  for 3 reasons: cost reduction (spending $15K/month on GitHub runners),
+  specialized hardware needs (GPU for ML model testing), and compliance
+  requirements (code must not leave your network).
+
+  Infrastructure requirements:
+  - 80 engineers, 300 workflow runs per day
+  - Peak: 50 concurrent jobs during 10AM-2PM
+  - Job types: CI (80%), deployment (15%), ML training (5%)
+  - Runner OS: Linux (90%), macOS (10% for iOS builds)
+  - Security: runners must be ephemeral (clean state per job)
+  - Network: runners need access to internal services but not
+    unrestricted internet access
+
+  Architecture options to evaluate:
+  1. Bare metal servers in your data center
+  2. VMs on existing VMware infrastructure
+  3. Kubernetes with actions-runner-controller (ARC)
+  4. Auto-scaled cloud VMs (AWS EC2 with runner auto-scaler)
+
+  Security concerns:
+  - How to prevent a malicious workflow from compromising the runner
+  - How to handle secrets on self-hosted runners
+  - How to ensure runners are patched and updated
+  - How to isolate runners between teams/repos
+  - How to prevent crypto mining on organization runners
+
+  Task: Design the self-hosted runner infrastructure. Write: the
+  architecture recommendation (which option and why), the runner
+  configuration (labels, groups, scaling policies), the security
+  hardening guide (ephemeral runners, network isolation, secret
+  handling), the monitoring and alerting setup (runner health, queue
+  depth, job wait times), and the cost comparison (self-hosted vs
+  GitHub-hosted for your usage). Include the runner registration
+  automation and the disaster recovery plan.
+
+assertions:
+  - type: llm_judge
+    criteria: "Architecture recommendation is well-reasoned — evaluates all 4 options with pros/cons for the specific requirements, likely recommends ARC or cloud auto-scaling for ephemeral runners, and addresses GPU needs for ML jobs separately from general CI"
+    weight: 0.35
+    description: "Well-reasoned architecture"
+  - type: llm_judge
+    criteria: "Security hardening is comprehensive — covers ephemeral runners (clean state per job), network isolation (firewall rules, no internet for sensitive repos), runner groups for team isolation, automatic patching, and protection against malicious workflows"
+    weight: 0.35
+    description: "Comprehensive security hardening"
+  - type: llm_judge
+    criteria: "Cost comparison is data-driven — calculates the total cost of self-hosted (hardware, maintenance, networking, engineer time) vs GitHub-hosted ($15K/month), identifies the break-even point, and the monitoring setup tracks runner utilization to optimize costs"
+    weight: 0.30
+    description: "Data-driven cost comparison"

package/courses/github-actions-cicd/scenarios/level-3/workflow-optimization.yaml

@@ -0,0 +1,59 @@
+meta:
+  id: workflow-optimization
+  level: 3
+  course: github-actions-cicd
+  type: output
+  description: "Optimize workflow performance — reduce CI time from 30 to 10 minutes using parallelism, caching, and smart architecture"
+  tags: [github, actions, optimization, parallelism, performance, advanced]
+
+state: {}
+
+trigger: |
+  Your team's main CI workflow takes 30 minutes to complete. This is
+  the #1 developer experience complaint. The CTO wants it under 10
+  minutes. Here's the current workflow and timing breakdown:
+
+  Current workflow (sequential):
+  ```
+  checkout (30s) → install (3min) → lint (2min) → typecheck (3min) →
+  unit-tests (8min) → integration-tests (6min) → e2e-tests (5min) →
+  build (2min) → docker-build (1.5min) = 31 minutes total
+  ```
+
+  Additional data:
+  - Unit tests: 1,200 tests in 45 test files
+  - Integration tests: 80 tests requiring PostgreSQL + Redis
+  - E2E tests: 30 Cypress tests requiring a running dev server
+  - Build output: 12MB frontend bundle + Docker image
+  - Cache hit rate: 60% (should be 95%+)
+  - Runner: ubuntu-latest (2 CPU, 7GB RAM)
+
+  Optimization budget:
+  - Can split into multiple parallel jobs
+  - Can use larger runners (4 CPU, 16GB RAM) — 2x cost but 2x faster
+  - Can use test sharding (split tests across parallel workers)
+  - Can use remote caching (Turborepo remote cache or similar)
+  - Can restructure tests (some unit tests may actually be integration)
+
+  Task: Redesign the workflow for sub-10-minute completion. Write: the
+  optimized workflow YAML with parallel jobs, the test sharding
+  configuration (split unit tests across N workers), the caching
+  strategy (to achieve 95%+ hit rate), the runner sizing analysis
+  (standard vs larger runners — cost vs speed trade-off), and the
+  before/after comparison (timeline diagram showing parallel execution
+  vs sequential). Explain each optimization technique and its expected
+  impact.
+
+assertions:
+  - type: llm_judge
+    criteria: "Workflow is redesigned for parallelism — lint, typecheck, and unit tests run in parallel (not sequential), e2e tests use sharding across multiple runners, and the critical path is identified and optimized to under 10 minutes"
+    weight: 0.35
+    description: "Parallel workflow redesign"
+  - type: llm_judge
+    criteria: "Test sharding is correctly implemented — unit tests split across N parallel workers using matrix strategy with test file distribution, integration tests use proper service containers, and e2e tests are sharded with Cypress's parallelization"
+    weight: 0.35
+    description: "Correct test sharding"
+  - type: llm_judge
+    criteria: "Before/after comparison is clear — shows the timeline diagram (text-based) of sequential vs parallel execution, quantifies each optimization's contribution, and the cost analysis compares standard vs larger runners at the new parallel configuration"
+    weight: 0.30
+    description: "Clear before/after comparison"

package/courses/github-actions-cicd/scenarios/level-4/cicd-data-architecture.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: cicd-data-architecture
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Design CI/CD data architecture — build analytics pipelines for workflow metrics, cost tracking, and predictive insights"
+  tags: [github, actions, data-architecture, analytics, metrics, DORA, expert]
+
+state: {}
+
+trigger: |
+  You're the Head of Data Engineering building the data infrastructure
+  for CI/CD analytics. The platform team needs data to track DORA
+  metrics, optimize costs, and predict failures. Currently there's no
+  centralized CI/CD data — metrics are scattered across GitHub API
+  responses, runner logs, and deployment systems.
+
+  Data sources:
+  - GitHub Actions API: Workflow runs, job details, step timings, costs
+  - GitHub webhooks: Real-time workflow events (50,000+ events/day)
+  - Self-hosted runner metrics: CPU, memory, queue depth (Prometheus)
+  - Cloud provider APIs: Deployment status, resource costs
+  - Incident management: PagerDuty alerts, incident timelines
+  - Git data: Commit frequency, PR merge times, branch patterns
+
+  Analytics requirements:
+  1. DORA metrics dashboard (deployment frequency, lead time, MTTR,
+     change failure rate) — real-time, per team, trending
+  2. Cost analytics (per-team, per-repo, per-workflow cost allocation
+     with budget alerts)
+  3. Failure prediction (ML model to predict which PRs will have CI
+     failures based on files changed, time of day, author history)
+  4. Performance trending (workflow duration trends, bottleneck
+     detection, runner utilization optimization)
+  5. Compliance reporting (automated evidence for SOC 2 audits)
+
+  Constraints:
+  - Budget: $100K/year for data infrastructure
+  - Must handle 50K+ events/day without data loss
+  - DORA dashboard must update within 5 minutes of events
+  - Cost data must be accurate to within $1 per team per month
+  - 2-year data retention for compliance
+
+  Task: Design the CI/CD data architecture. Write: the data model
+  (entities, relationships, derived metrics), the ingestion pipeline
+  (webhook processing, API polling, stream processing), the storage
+  strategy (time-series for metrics, OLAP for analytics, feature store
+  for ML), the DORA metrics computation logic, and the ML pipeline for
+  failure prediction. Include the cost breakdown for the $100K budget.
+
+assertions:
+  - type: llm_judge
+    criteria: "DORA metrics computation is correct — deployment frequency counts production deploys per day, lead time measures commit-to-deploy, MTTR measures incident-open-to-resolved, change failure rate is failed-deploys/total-deploys, and all are computed per-team with correct time windows"
+    weight: 0.35
+    description: "Correct DORA metrics computation"
+  - type: llm_judge
+    criteria: "Architecture fits the $100K budget — uses cost-effective technologies (not enterprise tools), handles 50K+ events/day reliably, and the cost breakdown accounts for compute, storage, and data transfer across all components"
+    weight: 0.35
+    description: "Budget-fitting architecture"
+  - type: llm_judge
+    criteria: "ML failure prediction is practical — identifies useful features (files changed, test history, time of day, PR size, author failure rate), proposes a reasonable model (not over-engineered), and the feature store design supports both batch training and real-time inference"
+    weight: 0.30
+    description: "Practical ML failure prediction"

package/courses/github-actions-cicd/scenarios/level-4/cicd-economics-roi.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: cicd-economics-roi
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Analyze CI/CD economics — calculate the true cost, model ROI, and present the business case for CI/CD investment"
+  tags: [github, actions, economics, ROI, cost-analysis, business-case, expert]
+
+state: {}
+
+trigger: |
+  You're the Director of Engineering presenting a business case to the
+  CFO for a $2M CI/CD platform investment. The CFO asks: "We spend
+  $1.2M/year on GitHub Actions already. Why invest more?"
+
+  Current CI/CD costs:
+  - GitHub Actions: $1.2M/year (minutes + storage)
+  - Self-hosted runners: $300K/year (infrastructure + maintenance)
+  - Engineering time on CI/CD: 3 FTEs dedicated ($600K/year)
+  - Time lost to CI failures: estimated 2 hours/week per developer
+  - Total: ~$2.1M/year visible costs
+
+  Hidden costs (to calculate):
+  - Developer waiting time: 800 engineers × avg 20 min/day waiting
+    for CI = ? hours/year at $85/hour fully loaded
+  - Failed deployments: 12/quarter, avg 4 hours to fix, 3 engineers
+  - Rollbacks: 8/quarter, avg 2 hours, 2 engineers
+  - Context switching: developers context-switch while waiting for
+    30-minute CI runs
+
+  Proposed investment ($2M over 2 years):
+  1. CI/CD platform team: 5 FTEs ($1M/year)
+  2. Self-hosted runner infrastructure upgrade: $400K
+  3. Tooling and automation: $200K
+  4. Training program: $100K
+
+  Expected outcomes:
+  - Reduce average CI time from 25 min to 8 min
+  - Reduce deployment failures from 12/quarter to 2/quarter
+  - Eliminate 80% of developer CI wait time
+  - Standardize deployment across all 200 repos
+
+  Task: Build the complete economic model. Write: the true cost of
+  the current state (including all hidden costs), the investment
+  analysis for the proposed $2M (expected returns, payback period,
+  NPV), the risk-adjusted projection (what if improvements are only
+  50% of target), the DORA metrics improvement forecast (deployment
+  frequency, lead time, MTTR, change failure rate), and the 2-page
+  executive summary for the CFO. Include a sensitivity analysis.
+
+assertions:
+  - type: llm_judge
+    criteria: "Hidden costs are calculated — developer wait time is quantified (800 engineers × 20 min/day × 250 days × $85/hour), deployment failure costs are summed, and the total true cost is significantly higher than the visible $2.1M"
+    weight: 0.35
+    description: "Calculated hidden costs"
+  - type: llm_judge
+    criteria: "Investment analysis is rigorous — includes NPV with appropriate discount rate, payback period calculation, sensitivity analysis (best/expected/worst case), and the risk-adjusted projection shows the investment is worthwhile even at 50% of expected improvements"
+    weight: 0.35
+    description: "Rigorous investment analysis"
+  - type: llm_judge
+    criteria: "Executive summary is CFO-ready — leads with business impact (not technical details), presents the 'do nothing' cost, shows clear ROI, includes DORA metrics improvement as industry-standard benchmarks, and fits in 2 pages"
+    weight: 0.30
+    description: "CFO-ready executive summary"

package/courses/github-actions-cicd/scenarios/level-4/cicd-executive-communication.yaml

@@ -0,0 +1,58 @@
+meta:
+  id: cicd-executive-communication
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Executive communication about CI/CD — present platform health, investment needs, and strategic roadmap to C-suite audiences"
+  tags: [github, actions, executive, communication, strategy, DORA-metrics, expert]
+
+state: {}
+
+trigger: |
+  You're the CTO preparing 3 executive communications about CI/CD
+  platform strategy. Each audience has different concerns.
+
+  Communication 1 — Board of Directors quarterly update:
+  The board wants to understand "engineering velocity" as a competitive
+  metric. Your DORA metrics data:
+  - Deployment frequency: 45/day (up from 8/day last year)
+  - Lead time for changes: 2.3 hours (down from 5 days)
+  - Mean time to recovery: 18 minutes (down from 4 hours)
+  - Change failure rate: 2.1% (down from 12%)
+  All metrics are now in the "Elite" DORA category. The board asks:
+  "How does this compare to competitors and how does it impact revenue?"
+
+  Communication 2 — CFO budget review:
+  CI/CD costs increased 300% year-over-year ($400K → $1.6M) due to
+  growth from 100 to 500 engineers. The CFO wants to understand: is
+  this scaling linearly (bad) or sublinearly (good)? What's the cost
+  per deployment? How does the investment in the platform team ($3M)
+  pay back? And should the company consider moving from GitHub Actions
+  to a cheaper alternative?
+
+  Communication 3 — All-engineering town hall:
+  You're presenting the CI/CD platform roadmap for the next year. The
+  audience includes engineers frustrated with current CI speed ("builds
+  take 30 minutes"), teams that want more deployment autonomy ("why
+  do we need the platform team's approval to change our pipeline"),
+  and security engineers who want stricter controls.
+
+  Task: Write all 3 communications. For each: adapt the message to
+  the audience, use data effectively, include specific asks or next
+  steps, and anticipate tough questions. The board update should be
+  1 page, the CFO review should include financial models, and the
+  town hall should be an engaging presentation outline.
+
+assertions:
+  - type: llm_judge
+    criteria: "Board communication connects DORA metrics to business value — translates elite DORA metrics into competitive advantage (faster time-to-market, lower outage costs, higher engineering productivity), with industry benchmarks for context"
+    weight: 0.35
+    description: "Business-value board communication"
+  - type: llm_judge
+    criteria: "CFO review has solid financial analysis — shows cost-per-deployment trend (should be declining despite total cost increase), demonstrates sublinear cost scaling per engineer, and makes a data-driven case against switching platforms (migration cost vs savings)"
+    weight: 0.35
+    description: "Financial CFO review"
+  - type: llm_judge
+    criteria: "Town hall addresses all audience segments — acknowledges the 30-minute build frustration with a specific improvement plan, explains the balance between autonomy and governance, and gives security engineers confidence in the compliance roadmap"
+    weight: 0.30
+    description: "Multi-audience town hall"

package/courses/github-actions-cicd/scenarios/level-4/cicd-incident-response.yaml

@@ -0,0 +1,60 @@
+meta:
+  id: cicd-incident-response
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Respond to CI/CD incidents — handle platform outages, security breaches, and deployment failures at the organizational level"
+  tags: [github, actions, incident-response, outage, security, postmortem, expert]
+
+state: {}
+
+trigger: |
+  You're the Head of Platform Engineering handling 3 major CI/CD
+  incidents simultaneously. Each requires a different response strategy.
+
+  Incident 1 — GitHub Actions outage (Severity: P1):
+  GitHub Actions has been experiencing degraded performance for 2 hours.
+  Workflow runs are queuing but not starting. GitHub's status page says
+  "Investigating." Your entire engineering org (500 developers) is
+  blocked. 15 production deployments are queued, including a critical
+  security patch. The CEO asks: "When will engineering be unblocked?"
+  You need to: determine the blast radius, activate the business
+  continuity plan, get the security patch deployed manually, and
+  communicate to the organization.
+
+  Incident 2 — Credential leak via CI logs (Severity: P1):
+  A workflow accidentally printed a database connection string (with
+  password) to the build logs. The logs are public (open-source repo).
+  The credentials provide read/write access to the production database
+  containing 2M user records. The log has been visible for 6 hours.
+  You need to: assess data exposure, rotate credentials, scrub the
+  logs, notify the security team, and determine if data was accessed.
+
+  Incident 3 — Deployment pipeline corruption (Severity: P2):
+  A misconfigured reusable workflow update caused all deployments to
+  deploy to the wrong environment. For 4 hours, staging deployments
+  went to production and production deployments went to staging. 3
+  untested features are now in production, and production data is being
+  written by staging code. No data loss yet, but data consistency is
+  at risk.
+
+  Task: Handle all 3 incidents. For each, write: the incident
+  commander's playbook (first 15 minutes, first hour, resolution),
+  the communication plan (who gets told what and when), the technical
+  resolution steps, and the post-incident review (root cause, corrective
+  actions, process improvements). Include: the unified incident report
+  for the CTO and the long-term platform resilience improvements.
+
+assertions:
+  - type: llm_judge
+    criteria: "Incident response follows proper IC protocol — establishes incident commander, defines clear communication channels, provides regular status updates, and each incident has a clear escalation path and resolution criteria"
+    weight: 0.35
+    description: "Proper IC protocol"
+  - type: llm_judge
+    criteria: "Technical resolutions are correct — GitHub outage has a BCP (fallback to self-hosted or manual deploy for security patch), credential leak has immediate rotation + log scrubbing + access audit, and environment swap is fixed with immediate rollback and data integrity verification"
+    weight: 0.35
+    description: "Correct technical resolutions"
+  - type: llm_judge
+    criteria: "Post-incident improvements are systemic — identifies that all 3 incidents reveal platform resilience gaps (no GitHub fallback, secrets in logs possible, environment config as single point of failure) and proposes permanent fixes"
+    weight: 0.30
+    description: "Systemic post-incident improvements"

package/courses/github-actions-cicd/scenarios/level-4/cicd-org-design.yaml

@@ -0,0 +1,59 @@
+meta:
+  id: cicd-org-design
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Design CI/CD organization — build teams, define roles, and create career paths for CI/CD platform engineering"
+  tags: [github, actions, org-design, platform-engineering, teams, career-paths, expert]
+
+state: {}
+
+trigger: |
+  You're the VP of Engineering creating a dedicated Platform Engineering
+  team that owns CI/CD. The company has 600 engineers and no centralized
+  CI/CD ownership — each team manages their own workflows, leading to
+  inconsistency, duplication, and poor developer experience.
+
+  Current pain points:
+  - 40% of developer time in surveys is "fighting CI/CD issues"
+  - No standard deployment process across 200 repos
+  - 3 different cloud authentication methods (static keys, OIDC, SSO)
+  - Self-hosted runners managed by 3 different teams with no coordination
+  - Security team manually audits CI/CD configurations quarterly
+  - New team onboarding takes 2 weeks just for CI/CD setup
+
+  Organizational mandate:
+  - Create a Platform Engineering team focused on CI/CD
+  - Headcount: 15 FTEs (mix of engineers, SREs, and program managers)
+  - Budget: $3M/year (headcount + infrastructure)
+  - Goal: Reduce "fighting CI/CD" from 40% to 10% within 1 year
+
+  Design questions:
+  1. Team structure: How to organize 15 people across different functions
+  2. Interaction model: How does the platform team serve 600 engineers
+  3. Service catalog: What services does the team provide
+  4. Career ladder: How do platform engineers grow
+  5. Success metrics: How to measure the team's impact
+  6. Funding model: Chargeback to teams vs central funding
+
+  Task: Design the complete organization. Write: the team structure
+  (roles, responsibilities, reporting), the service catalog (what the
+  team provides and what it doesn't), the interaction model with product
+  teams (self-service platform, office hours, embedded support), the
+  career ladder (from junior platform engineer to staff/principal), the
+  success metrics and OKRs for the first year, and the funding model
+  with cost allocation.
+
+assertions:
+  - type: llm_judge
+    criteria: "Team structure is balanced — includes platform engineers (build tooling), SREs (reliability/observability), and a program manager (governance/adoption), with clear roles and the right ratio for a 15-person team"
+    weight: 0.35
+    description: "Balanced team structure"
+  - type: llm_judge
+    criteria: "Service catalog is clear about boundaries — defines what the team owns (shared workflows, runner fleet, deployment pipelines) vs what product teams own (app-specific tests, feature flags), with a self-service model that scales beyond the 15-person team"
+    weight: 0.35
+    description: "Clear service catalog boundaries"
+  - type: llm_judge
+    criteria: "Success metrics are measurable — includes both output metrics (platform uptime, deployment frequency) and outcome metrics (developer satisfaction, time-to-first-deploy for new teams), and the OKRs target reducing 'fighting CI/CD' from 40% to 10%"
+    weight: 0.30
+    description: "Measurable success metrics"

package/courses/github-actions-cicd/scenarios/level-4/cicd-platform-architecture.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: cicd-platform-architecture
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Architect a CI/CD platform — design the internal platform that manages runners, workflows, and deployments at enterprise scale"
+  tags: [github, actions, platform, architecture, runners, scaling, expert]
+
+state: {}
+
+trigger: |
+  You're the Staff Engineer designing the internal CI/CD platform for
+  a 800-engineer organization. The platform sits between GitHub Actions
+  and the cloud infrastructure, providing a managed experience.
+
+  Platform requirements:
+  1. Runner management: Auto-scaled runner pools for different workloads
+     (CI, deployment, ML training, iOS builds)
+  2. Workflow orchestration: Standard deployment pipelines that teams
+     adopt via reusable workflows
+  3. Secret management: Centralized secret store (HashiCorp Vault)
+     integrated with GitHub Actions via OIDC
+  4. Artifact management: Central artifact store with retention policies
+  5. Observability: Metrics, logs, and traces for all workflow runs
+  6. Cost management: Per-team cost allocation and budget enforcement
+
+  Scale requirements:
+  - 5,000 workflow runs per day
+  - 200 concurrent jobs during peak
+  - 4 runner pools: Linux CI (150 runners), Linux GPU (10), macOS (20),
+    Windows (20)
+  - 99.9% uptime (CI platform down = all teams blocked)
+  - < 30 second job startup time (cold start)
+
+  Technical constraints:
+  - Runs on existing Kubernetes cluster (AWS EKS)
+  - Must use actions-runner-controller (ARC) for runner management
+  - Must integrate with existing Vault, Datadog, and PagerDuty
+  - Must support both GitHub Enterprise Cloud and self-hosted runners
+  - Budget: $500K/year for infrastructure
+
+  Task: Design the CI/CD platform architecture. Write: the system
+  architecture (components, data flow, integrations), the runner
+  auto-scaling design (scale-to-zero, warm pools, GPU scheduling),
+  the observability stack (metrics to collect, dashboards, alerts),
+  the cost management system (per-team tracking, budget alerts),
+  and the disaster recovery plan (what happens when the platform goes
+  down). Include architecture diagrams (text-based) and capacity
+  planning calculations.
+
+assertions:
+  - type: llm_judge
+    criteria: "Architecture handles the scale — ARC configuration supports 200 concurrent jobs with proper auto-scaling policies, runner pool isolation prevents noisy-neighbor issues, cold start is addressed with warm pools, and the $500K budget is allocated across compute, storage, and networking"
+    weight: 0.35
+    description: "Scale-handling architecture"
+  - type: llm_judge
+    criteria: "Observability is comprehensive — tracks runner utilization, job queue depth, wait time, success rates, and cost per team. Integrates with Datadog for metrics and PagerDuty for alerts. Dashboards are designed for different audiences (engineers, managers, finance)"
+    weight: 0.35
+    description: "Comprehensive observability"
+  - type: llm_judge
+    criteria: "Disaster recovery is enterprise-grade — defines RTO and RPO for the CI platform, has a fallback plan when the platform is down (GitHub-hosted runners as failover), and the 99.9% uptime target is addressed with redundancy and monitoring"
+    weight: 0.30
+    description: "Enterprise-grade disaster recovery"

package/courses/github-actions-cicd/scenarios/level-4/cicd-training-program.yaml

@@ -0,0 +1,65 @@
+meta:
+  id: cicd-training-program
+  level: 4
+  course: github-actions-cicd
+  type: output
+  description: "Design CI/CD training program — create a comprehensive curriculum for building GitHub Actions expertise across an organization"
+  tags: [github, actions, training, education, certification, onboarding, expert]
+
+state: {}
+
+trigger: |
+  You're designing a company-wide GitHub Actions training program for
+  your 500-engineer organization. Currently, CI/CD knowledge is
+  concentrated in 10 people, and the rest of the org depends on them
+  for any workflow changes.
+
+  Current problems the training must address:
+  - Only 10 engineers can modify CI/CD workflows confidently
+  - New hires take 4 weeks before they can run their first deployment
+  - 40% of CI/CD support tickets are "how do I do X in GitHub Actions"
+  - Teams copy-paste workflows without understanding them
+  - Security misconfigurations are found quarterly in workflow audits
+  - 3 incidents in the last year caused by poorly written workflows
+
+  Target audience tiers:
+  1. All engineers: Basic workflow literacy (read and understand CI/CD)
+  2. Team leads: Workflow authoring (create and maintain team workflows)
+  3. Platform contributors: Advanced automation (build reusable
+     workflows, custom actions, platform integrations)
+  4. Security engineers: Security review of workflows
+
+  Available formats:
+  - Self-paced modules (budget: $40K to develop)
+  - Monthly live workshops (budget: $20K/year)
+  - Sandbox environment (dedicated GitHub org for practice)
+  - Certification program (gamification)
+  - Weekly office hours with the platform team
+
+  Constraints:
+  - Maximum 4 hours of training per engineer per quarter
+  - Must be available asynchronously (remote-first company)
+  - Must show measurable impact within 6 months
+  - Must reduce CI/CD support tickets by 50%
+
+  Task: Design the complete training program. Write: the curriculum
+  for each tier (learning objectives, modules, hands-on exercises),
+  the sandbox environment design (pre-configured repos for practice),
+  the certification system (levels, requirements, rewards), the
+  measurement framework (how to prove training reduces tickets and
+  incidents), and the rollout plan. Include sample exercises for each
+  tier.
+
+assertions:
+  - type: llm_judge
+    criteria: "Curriculum is tier-appropriate — all-engineers learn to read workflows and trigger deployments, team leads learn to author workflows and manage secrets, platform contributors learn reusable workflows and custom actions, and security engineers learn vulnerability assessment"
+    weight: 0.35
+    description: "Tier-appropriate curriculum"
+  - type: llm_judge
+    criteria: "Sandbox environment is practical — provides pre-configured repos with progressive exercises, simulates real scenarios (failing tests, flaky CI, deployment issues), and allows safe experimentation without affecting production"
+    weight: 0.35
+    description: "Practical sandbox environment"
+  - type: llm_judge
+    criteria: "Measurement framework proves ROI — tracks leading indicators (training completion, sandbox exercise pass rates) and lagging indicators (support ticket volume, incident count, time-to-first-deployment for new hires), with a clear before/after comparison"
+    weight: 0.30
+    description: "ROI-proving measurement framework"