dojo.md 0.1.0 → 0.2.1

package/courses/github-pr-review/scenarios/level-3/review-automation-platform.yaml

@@ -0,0 +1,61 @@
+meta:
+  id: review-automation-platform
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Build a review automation platform — design GitHub Apps and bots that automate review assignment, checks, and notifications"
+  tags: [github, pr-review, automation, GitHub-Apps, bots, platform, advanced]
+
+state: {}
+
+trigger: |
+  You're building an internal review automation platform (GitHub App)
+  for your 200-engineer organization. The platform should automate the
+  repetitive parts of the review process while keeping humans in the
+  loop for judgment calls.
+
+  Requirements:
+  1. Smart reviewer assignment: Based on code expertise (git blame),
+     current load, timezone overlap, and review history
+  2. PR categorization: Automatically label PRs by type (feature, bug,
+     refactor, dependency), size (S/M/L/XL), and risk (low/medium/high)
+  3. Review reminders: Escalating notifications when SLAs are missed
+  4. Stale review detection: Alert when reviews are approved but code
+     changed after approval
+  5. Merge readiness: Check all requirements met before allowing merge
+  6. Review analytics: Weekly digests for team leads
+
+  Technical constraints:
+  - Must use GitHub App (not personal access tokens)
+  - Must handle 500+ PRs per week across 30 repos
+  - Must be resilient to GitHub API rate limits (5,000 requests/hour)
+  - Must handle webhook delivery failures gracefully
+  - Must not expose any internal code or review data externally
+
+  The platform needs to integrate with:
+  - GitHub (webhooks + API)
+  - Slack (notifications)
+  - Jira (issue linking)
+  - PagerDuty (on-call reviewer for urgent PRs)
+
+  Task: Design the review automation platform. Write: the system
+  architecture (webhook handlers, job queues, data store, integrations),
+  the reviewer assignment algorithm (inputs, weighting, edge cases),
+  the PR categorization rules (heuristics for type, size, and risk),
+  the notification and escalation logic, and the API rate limit strategy.
+  Include the GitHub App manifest, webhook event list, and a sample
+  webhook handler for the PR opened event.
+
+assertions:
+  - type: llm_judge
+    criteria: "Architecture handles scale — uses webhook-driven processing with a job queue (not synchronous), handles 500+ PRs/week within GitHub API rate limits, has retry logic for webhook failures, and separates concerns (assignment, categorization, notifications)"
+    weight: 0.35
+    description: "Scalable architecture"
+  - type: llm_judge
+    criteria: "Reviewer assignment algorithm is sophisticated — considers code expertise (git blame/log), current review load, timezone, and rotation, with fallbacks when primary reviewers are unavailable. Edge cases like vacation, new hires, and single-expert code areas are handled"
+    weight: 0.35
+    description: "Sophisticated assignment algorithm"
+  - type: llm_judge
+    criteria: "PR categorization is practical — risk assessment considers files changed (config, auth, financial code = high risk), size buckets are meaningful (not just line count — considers file count and complexity), and categorization drives review requirements (high-risk PRs need more reviewers)"
+    weight: 0.30
+    description: "Practical PR categorization"

package/courses/github-pr-review/scenarios/level-3/review-culture-design.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: review-culture-design
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Design review culture — build a healthy code review culture that balances thoroughness, speed, psychological safety, and learning"
+  tags: [github, pr-review, culture, psychological-safety, learning, advanced]
+
+state: {}
+
+trigger: |
+  You're the VP of Engineering inheriting a team with a toxic review
+  culture. An anonymous survey revealed:
+
+  Survey results (60 respondents):
+  - 72% feel anxious when submitting PRs
+  - 58% have avoided submitting code changes due to fear of harsh reviews
+  - 45% say certain reviewers are "gatekeepers" who block PRs for
+    personal preferences
+  - 38% feel reviews focus on criticism without acknowledging good work
+  - 65% of junior engineers say reviews make them feel incompetent
+  - Only 22% say they learn from code reviews
+
+  Specific complaints:
+  - "Reviewer X rewrites my entire PR in comments — why didn't they just
+    write it themselves?"
+  - "I got 47 comments on a 50-line PR. Most were 'I would do this
+    differently.' Not bugs, just preferences."
+  - "When I push back on review feedback, I get labeled as 'difficult.'"
+  - "Senior engineers' PRs get rubber-stamped. Junior PRs get nitpicked."
+  - "I spent 2 weeks on a feature. The review said 'wrong approach,
+    start over.' No one told me before I started coding."
+
+  Good signs in the data:
+  - Teams that do pair programming have 85% satisfaction with reviews
+  - The mobile team (led by a specific manager) has 90% positive review
+    experience — what are they doing differently?
+  - Engineers who've been through reviewer training rate reviews 3x
+    more positively
+
+  Task: Design a culture transformation plan. Write: the diagnosis (root
+  causes of the toxic patterns), the review culture principles (code of
+  conduct for reviewers and authors), the specific interventions (what
+  to change immediately, in 30 days, and in 90 days), the reviewer
+  training program (curriculum, practice exercises, assessment), the
+  accountability mechanisms (how to address bad review behavior without
+  creating fear), and the success metrics. Study the mobile team's
+  positive patterns and scale them.
+
+assertions:
+  - type: llm_judge
+    criteria: "Diagnosis identifies root causes — power dynamics (senior vs junior treatment), missing early feedback (architecture reviews before coding), preference-vs-requirement confusion, and lack of reviewer accountability. Analyzes the mobile team's success factors"
+    weight: 0.35
+    description: "Root cause diagnosis"
+  - type: llm_judge
+    criteria: "Interventions are concrete and phased — immediate actions (review code of conduct, stop specific harmful patterns), 30-day actions (reviewer training, pair review program), 90-day actions (culture metrics, accountability processes), each with measurable outcomes"
+    weight: 0.35
+    description: "Concrete phased interventions"
+  - type: llm_judge
+    criteria: "Plan balances accountability with safety — addresses the 'gatekeeper' behavior without creating fear, includes mechanisms for authors to give feedback on reviews, and doesn't just create more rules but changes underlying dynamics"
+    weight: 0.30
+    description: "Accountability-safety balance"

package/courses/github-pr-review/scenarios/level-3/review-data-pipeline.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: review-data-pipeline
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Build a review analytics pipeline — collect, analyze, and act on PR review data to continuously improve review effectiveness"
+  tags: [github, pr-review, analytics, data-pipeline, metrics, advanced]
+
+state: {}
+
+trigger: |
+  You're building a review analytics system for your 100-engineer org.
+  Currently, review data lives only in GitHub and there's no way to
+  track trends, identify problems, or measure improvements. You need
+  to design a data pipeline that collects review data and produces
+  actionable insights.
+
+  Data sources available:
+  - GitHub API: PR data, reviews, comments, timelines, check runs
+  - GitHub webhooks: real-time events for PR state changes
+  - Slack: review request messages and response times
+  - Jira: linked issues, sprint assignments, priority levels
+  - Deployment system: deploy timestamps, rollback frequency
+
+  Key questions to answer with data:
+  1. Which teams have the fastest/slowest review cycles?
+  2. What percentage of review comments lead to code changes vs are
+     ignored or resolved without changes?
+  3. Do certain types of PRs (size, category, author seniority) have
+     more review rounds?
+  4. What's the correlation between review thoroughness and post-deploy
+     bug rate?
+  5. Are specific reviewers consistently over- or under-loaded?
+  6. How effective are automated checks at reducing human review load?
+
+  Constraints:
+  - Budget: $500/month for infrastructure
+  - Must not slow down GitHub workflows
+  - Must respect developer privacy (no individual shaming dashboards)
+  - Data retention: 2 years for compliance
+
+  Task: Design the review data pipeline. Write: the data model (what
+  entities to track, relationships, derived metrics), the collection
+  architecture (webhooks vs polling, real-time vs batch, storage), the
+  analytics layer (dashboards for eng managers, team leads, and ICs),
+  the privacy and ethics guidelines (what data is shown at what level),
+  and 3 specific analyses you would run first with expected findings
+  and recommended actions. Include the schema design and sample queries.
+
+assertions:
+  - type: llm_judge
+    criteria: "Data model is comprehensive — tracks PRs, reviews, comments, review rounds, time-to-first-review, time-to-merge, reviewer load, comment types (blocking vs suggestion), and correlates with deployment outcomes"
+    weight: 0.35
+    description: "Comprehensive data model"
+  - type: llm_judge
+    criteria: "Architecture is practical within constraints — fits $500/month budget (likely uses GitHub webhooks + a lightweight database like Postgres, not enterprise analytics tools), processes events without slowing GitHub, and handles 2-year retention"
+    weight: 0.35
+    description: "Budget-practical architecture"
+  - type: llm_judge
+    criteria: "Privacy is respected — dashboards show team-level metrics (not individual ranking), individual data is available only to the person themselves and their direct manager, and the ethics guidelines prevent using review metrics for performance reviews without context"
+    weight: 0.30
+    description: "Privacy-respecting analytics"

package/courses/github-pr-review/scenarios/level-4/enterprise-review-operations.yaml

@@ -0,0 +1,61 @@
+meta:
+  id: enterprise-review-operations
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Design enterprise review operations — build review infrastructure for 500+ engineer organizations with multiple business units"
+  tags: [github, pr-review, enterprise, operations, scaling, expert]
+
+state: {}
+
+trigger: |
+  You're the VP of Developer Experience at a 600-engineer company with
+  4 business units, 50 teams, and 200+ repositories. The CEO mandated
+  a company-wide code review standardization initiative after an audit
+  found that review practices vary wildly across business units.
+
+  Current state by business unit:
+  - Consumer (200 engineers, 80 repos): Fast-shipping culture, average
+    review time 4 hours, but 3 production incidents last quarter traced
+    to inadequate review
+  - Enterprise (150 engineers, 60 repos): Thorough reviews, average 3
+    days to merge, but shipping velocity is 40% below target
+  - Platform (120 engineers, 40 repos): Strong review culture, but
+    bottlenecked on 5 senior engineers who review everything
+  - Data/ML (130 engineers, 30 repos): Notebooks and experiments mixed
+    with production code, no distinction in review rigor
+
+  Cross-cutting challenges:
+  - 3 different GitHub Enterprise organizations (legacy from acquisitions)
+  - Inconsistent branch protection across all orgs
+  - No unified metrics — each BU reports differently
+  - Some teams use GitLab, most use GitHub
+  - Compliance requirements differ by BU (Consumer: GDPR, Enterprise:
+    SOX/SOC 2, Platform: PCI DSS, Data: CCPA + model governance)
+
+  Budget: $2M/year for tooling and headcount
+  Timeline: 12 months to reach "standardized" state
+
+  Task: Design the enterprise review operations strategy. Write: the
+  unified review standard (minimum bar for all BUs, with BU-specific
+  extensions), the organizational model (centralized review platform
+  team vs federated ownership), the tooling strategy (GitHub Enterprise
+  configuration, custom automation, analytics platform), the migration
+  plan for each BU (addressing their specific challenges), and the ROI
+  model (how the $2M investment pays back in reduced incidents, faster
+  shipping, and compliance readiness). Include the quarterly milestones
+  and success metrics.
+
+assertions:
+  - type: llm_judge
+    criteria: "Standards balance uniformity with BU autonomy — the minimum bar applies everywhere (e.g., no self-merge, required reviews), but BU-specific extensions handle different compliance needs and shipping cultures without forcing one-size-fits-all"
+    weight: 0.35
+    description: "Balanced standards"
+  - type: llm_judge
+    criteria: "Migration plan addresses each BU's specific challenge — Consumer needs quality without slowing down, Enterprise needs speed without losing rigor, Platform needs to distribute knowledge, and Data/ML needs production code separation from experiments"
+    weight: 0.35
+    description: "BU-specific migration plans"
+  - type: llm_judge
+    criteria: "ROI model is quantified — connects review improvements to business outcomes (reduced incidents = $ saved, faster shipping = revenue impact, compliance readiness = audit cost avoidance), and the $2M budget is allocated across tooling, headcount, and training"
+    weight: 0.30
+    description: "Quantified ROI model"

package/courses/github-pr-review/scenarios/level-4/expert-review-shift.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: expert-review-shift
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Expert review shift — navigate organizational crises, vendor decisions, executive escalations, and team morale challenges simultaneously"
+  tags: [github, pr-review, shift-simulation, crisis, organizational, expert]
+
+state: {}
+
+trigger: |
+  You're the VP of Developer Experience during a critical week where
+  multiple review-related crises converge.
+
+  Crisis 1 — Vendor lock-in emergency:
+  Your review automation vendor (ReviewBot Pro) announced they're being
+  acquired by a competitor. They're sunsetting their product in 90 days.
+  Your entire review assignment, SLA tracking, and analytics run on
+  their platform. 600 engineers depend on it daily. You need a migration
+  plan by Friday for the executive team meeting.
+
+  Crisis 2 — Compliance audit failure:
+  The SOC 2 auditor flagged 3 critical findings related to code review:
+  (a) 8% of PRs in the payment codebase were self-merged, (b) review
+  audit trail has gaps where comments were deleted, and (c) emergency
+  changes have no post-deployment review documentation. The remediation
+  deadline is 30 days. Failure means losing a $15M enterprise customer
+  who requires SOC 2 compliance.
+
+  Crisis 3 — Engineering morale crisis:
+  An anonymous post on the company's internal forum went viral: "Our
+  code review process is killing innovation." 200 engineers upvoted it.
+  The post includes specific complaints: mandatory 2-reviewer requirement
+  is "bureaucratic," the review SLA creates "artificial urgency," and
+  "reviewers block PRs over personal preferences." The CEO forwarded it
+  to you asking "is this true?"
+
+  Crisis 4 — Team capacity:
+  Your Review Platform team of 12 engineers has 3 people on medical
+  leave, 2 threatening to quit (burned out from the always-on review
+  platform), and the remaining 7 are already working on the next
+  quarterly OKR deliverables. You have no slack capacity.
+
+  Task: Handle all 4 crises. For each, write: the immediate triage
+  (what to do today), the 1-week action plan, the 30-day resolution
+  plan, and the communication to each stakeholder group. Then write the
+  integrated strategy memo showing how these crises are interconnected
+  and the unified approach to resolving them while protecting the team.
+
+assertions:
+  - type: llm_judge
+    criteria: "Vendor migration is practical — prioritizes critical functionality (assignment, SLA tracking), evaluates build-vs-buy in 90 days, identifies what can be replaced with GitHub native features immediately, and doesn't propose rebuilding everything from scratch"
+    weight: 0.35
+    description: "Practical vendor migration"
+  - type: llm_judge
+    criteria: "Compliance remediation is achievable in 30 days — addresses each finding with specific technical fixes (branch protection for self-merge, immutable audit trail, emergency review documentation process), and the plan is realistic given the team capacity crisis"
+    weight: 0.35
+    description: "Achievable compliance remediation"
+  - type: llm_judge
+    criteria: "Morale and capacity crises are handled with empathy — the CEO response acknowledges legitimate complaints while providing data context, the team capacity plan doesn't just add more work, and the unified strategy memo shows how solving one crisis helps the others"
+    weight: 0.30
+    description: "Empathetic crisis handling"

package/courses/github-pr-review/scenarios/level-4/review-data-architecture.yaml

@@ -0,0 +1,69 @@
+meta:
+  id: review-data-architecture
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Design review data architecture — build the data infrastructure for review analytics, ML-powered insights, and compliance reporting"
+  tags: [github, pr-review, data-architecture, analytics, ML, expert]
+
+state: {}
+
+trigger: |
+  You're the Head of Data Engineering building the data infrastructure
+  for the company's review analytics platform. The system needs to
+  support real-time dashboards, ML-powered insights, and compliance
+  audit reports.
+
+  Data sources:
+  - GitHub Events API: PR events, review events, comment events,
+    check run events (estimated 50,000 events/day)
+  - GitHub REST API: PR details, file diffs, user profiles
+  - Slack: Review request/response timestamps
+  - Jira: Issue metadata, sprint data, priority levels
+  - Deployment system: Deploy events, rollback events
+  - Incident management: Incident data, root cause analysis
+
+  Analytics requirements:
+  1. Real-time dashboard: Current review queue, SLA status, team load
+     (refreshed every 5 minutes)
+  2. Historical analytics: Trends over 2 years, team comparisons,
+     seasonal patterns
+  3. ML features: Reviewer expertise scoring, PR risk prediction,
+     review time estimation, anomaly detection (rubber stamp detection)
+  4. Compliance reports: Audit trails, SOC 2 evidence, monthly
+     compliance scorecards
+  5. Developer experience: Individual review statistics (private to
+     the engineer), career growth tracking
+
+  Technical requirements:
+  - Event-driven ingestion (no polling GitHub API)
+  - Schema evolution support (GitHub API changes frequently)
+  - 2-year data retention with automated archival
+  - Sub-second query response for dashboards
+  - ML feature store for model training and serving
+  - GDPR compliance (engineer data deletion on request)
+
+  Budget: $150K/year infrastructure, 4 engineers for 6 months to build
+
+  Task: Design the data architecture. Write: the data model (entities,
+  relationships, derived metrics, slowly changing dimensions), the
+  ingestion pipeline (event processing, transformation, loading), the
+  storage strategy (which database for which use case — OLTP vs OLAP
+  vs time-series), the ML feature store design (feature computation,
+  serving, versioning), and the privacy/compliance layer (data masking,
+  retention, deletion). Include the schema for key tables and sample
+  queries for each analytics requirement.
+
+assertions:
+  - type: llm_judge
+    criteria: "Data model captures review complexity — models PRs, reviews, comments, review rounds, reviewer load, time-between-events, and derived metrics like 'review depth score' and 'rubber stamp probability'. Handles the GitHub event schema correctly"
+    weight: 0.35
+    description: "Complexity-capturing data model"
+  - type: llm_judge
+    criteria: "Storage strategy is appropriate — uses different storage for different patterns (event stream processing for real-time, columnar/OLAP for historical analytics, time-series for metrics, and a feature store for ML), within the $150K/year budget"
+    weight: 0.35
+    description: "Appropriate storage strategy"
+  - type: llm_judge
+    criteria: "ML feature store is practical — features include reviewer expertise score, PR complexity score, and review time prediction with clear computation logic, versioning for model reproducibility, and serving layer for real-time inference"
+    weight: 0.30
+    description: "Practical ML feature store"

package/courses/github-pr-review/scenarios/level-4/review-economics-roi.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: review-economics-roi
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Analyze review economics — calculate the true cost of code review, model ROI of review improvements, and present to executive leadership"
+  tags: [github, pr-review, economics, ROI, cost-analysis, expert]
+
+state: {}
+
+trigger: |
+  You're the Director of Engineering presenting a business case to the
+  CFO for investing in code review infrastructure. The CFO's question:
+  "Code review costs us millions in engineer time. How do we know it's
+  worth it? And if we invest more, what's the return?"
+
+  Data you've collected:
+  - 600 engineers, average fully-loaded cost: $200K/year ($100/hour)
+  - Average engineer spends 6 hours/week on code review
+  - That's 600 × 6 × 52 = 187,200 review hours/year = $18.7M/year
+  - Average PR takes 25 minutes to review
+  - 78,000 PRs merged per year
+
+  Cost of review failures (incidents traced to review gaps):
+  - 12 production incidents/year attributed to review failures
+  - Average incident cost: $150K (engineering time, customer impact,
+    reputation)
+  - 3 compliance findings in last audit: $500K in remediation + fines
+  - 2 security vulnerabilities that passed review: $800K total impact
+  - Total cost of review failures: $3.5M/year
+
+  Proposed investments:
+  1. Review automation platform: $800K (build) + $200K/year (maintain)
+     Expected: Reduce review time by 30%, catch 60% of style issues
+  2. Reviewer training program: $150K/year
+     Expected: Reduce review rounds by 25%, improve bug detection by 15%
+  3. AI-assisted review: $300K/year (tooling)
+     Expected: Reduce first-review time by 40%, flag 50% of common issues
+  4. Review analytics dashboard: $200K (build) + $50K/year (maintain)
+     Expected: Identify bottlenecks, reduce merge time by 20%
+
+  Task: Build the complete economic model. Write: the true cost of
+  code review (including hidden costs like context switching, blocked
+  PRs, opportunity cost), the value model (what code review prevents,
+  quantified), the ROI analysis for each investment (with sensitivity
+  analysis), the 3-year financial projection, and the executive
+  presentation (2-page summary the CFO can present to the board).
+  Include the assumptions, risks, and what metrics to track to validate
+  the projections.
+
+assertions:
+  - type: llm_judge
+    criteria: "Cost model includes hidden costs — beyond the $18.7M direct cost, includes context-switching costs (review interrupting deep work), blocked PR costs (engineers waiting for review), and opportunity cost (what engineers could build instead of reviewing)"
+    weight: 0.35
+    description: "Complete cost model with hidden costs"
+  - type: llm_judge
+    criteria: "ROI analysis is rigorous — each investment has NPV calculation, payback period, sensitivity analysis (what if improvements are 50% less than expected?), and the analysis accounts for interaction effects (automation + training together may have compounding benefits)"
+    weight: 0.35
+    description: "Rigorous ROI analysis"
+  - type: llm_judge
+    criteria: "Executive presentation is CFO-ready — leads with the business impact (not technical details), quantifies the 'do nothing' cost, presents investment options with clear ROI comparison, and includes metrics to track for accountability"
+    weight: 0.30
+    description: "CFO-ready executive presentation"

package/courses/github-pr-review/scenarios/level-4/review-executive-communication.yaml

@@ -0,0 +1,61 @@
+meta:
+  id: review-executive-communication
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Executive communication about review — present review process health, investments, and strategy to C-suite and board audiences"
+  tags: [github, pr-review, executive, communication, strategy, expert]
+
+state: {}
+
+trigger: |
+  You're the CTO preparing 3 different communications about code review
+  for executive audiences with different concerns.
+
+  Communication 1 — Board of Directors quarterly update:
+  The board wants to understand "engineering quality" as a competitive
+  advantage. They've heard competitors are shipping faster. Your data:
+  - Code review catches 65% of bugs before production
+  - Average merge time: 1.8 days (industry median: 2.5 days)
+  - Review-related incidents: 4/quarter (down from 12 last year)
+  - Review investment: $3.2M/year (5.3% of engineering budget)
+  - Developer satisfaction with review: 78% (up from 45% last year)
+  The board asks: "Are we over-investing in review at the expense of
+  shipping speed?"
+
+  Communication 2 — CEO 1:1 escalation:
+  The Head of Product is complaining that code review is the #1
+  bottleneck to shipping features. They want to reduce required reviews
+  from 2 to 0 for "low-risk" PRs and allow self-merge for senior
+  engineers. Your security team says this violates SOC 2 controls. The
+  CEO wants your recommendation by Friday.
+
+  Communication 3 — All-hands engineering presentation:
+  The annual engineering all-hands is next week. You want to present
+  the "State of Code Review" — celebrating improvements, acknowledging
+  remaining challenges, and building excitement for the review platform
+  investment. Your audience is 600 engineers ranging from interns to
+  distinguished engineers.
+
+  Task: Write all 3 communications. For each: adapt the message to the
+  audience (board speaks business/risk, CEO wants a decision framework,
+  engineers want to understand the "why"), use data effectively (board
+  gets trends and benchmarks, CEO gets risk analysis, engineers get
+  impact stories), and include clear asks or next steps. The board
+  update should be 1 page, the CEO memo should be 2 pages with a
+  recommendation, and the all-hands should be an outline with speaker
+  notes.
+
+assertions:
+  - type: llm_judge
+    criteria: "Board communication speaks business language — frames review as risk management and competitive advantage (not engineering process), uses industry benchmarks to show strong performance, and answers the 'over-investing' question with data showing declining incidents and competitive merge times"
+    weight: 0.35
+    description: "Business-language board communication"
+  - type: llm_judge
+    criteria: "CEO memo provides a clear recommendation — doesn't just present options, takes a position on the self-merge request with risk analysis, proposes a middle ground that addresses product's velocity concerns while maintaining compliance, and includes a decision framework for future review policy changes"
+    weight: 0.35
+    description: "Decision-ready CEO memo"
+  - type: llm_judge
+    criteria: "All-hands presentation engages engineers — celebrates specific team and individual contributions, uses concrete impact stories (not just metrics), acknowledges pain points honestly, and builds excitement for the review platform investment with a clear vision"
+    weight: 0.30
+    description: "Engineer-engaging all-hands"

package/courses/github-pr-review/scenarios/level-4/review-incident-postmortem.yaml

@@ -0,0 +1,69 @@
+meta:
+  id: review-incident-postmortem
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Conduct review process postmortems — lead blameless investigations into systemic review failures and implement organizational improvements"
+  tags: [github, pr-review, postmortem, incident-response, process-improvement, expert]
+
+state: {}
+
+trigger: |
+  You're the Head of Engineering conducting a postmortem after the most
+  serious review-related incident in company history. A payment
+  processing bug caused $2.3M in duplicate charges over 72 hours before
+  detection.
+
+  Incident timeline:
+  - Day 0: PR #4521 "Optimize payment batch processing" submitted by a
+    senior engineer (800 lines, refactoring payment job scheduler)
+  - Day 1: First review by a mid-level engineer — "LGTM, nice refactor"
+    (spent 8 minutes on 800 lines based on GitHub activity timestamps)
+  - Day 1: Second review by another mid-level — requested minor naming
+    change, then approved after the change was made
+  - Day 2: PR merged after CI passed (all 247 tests green)
+  - Day 3: Deployed to production via automated pipeline
+  - Day 5: Customer reports duplicate charges. Investigation begins
+  - Day 6: Root cause identified — race condition in the batch scheduler
+    that causes payment jobs to be processed twice under high load
+  - Day 8: Hotfix deployed. Customer refund process begins
+
+  Review failures identified:
+  1. Neither reviewer had payment domain expertise (CODEOWNERS didn't
+     include the batch scheduler directory)
+  2. The 8-minute review of 800 lines was clearly insufficient
+  3. The race condition was subtle but identifiable with careful review
+     of the concurrency logic
+  4. Test coverage was high (85%) but no tests simulated concurrent
+     execution or high-load conditions
+  5. The PR was marked "refactoring" which reduced perceived risk
+  6. The senior author's reputation created unconscious trust bias
+
+  Additional context:
+  - The payment team lost their tech lead 2 months ago (not replaced)
+  - CODEOWNERS for payment code points to a team that was reorganized
+  - The review SLA pressure (merge within 1 day) may have rushed reviews
+  - 3 similar but smaller incidents occurred in the past year
+
+  Task: Lead the postmortem. Write: the blameless incident analysis
+  (contributing factors at the individual, team, process, and
+  organizational levels), the immediate fixes (what changes this week),
+  the systemic improvements (what changes in 30/60/90 days), the review
+  process changes specifically for payment/financial code, and the
+  executive summary for the board (including financial impact and
+  remediation plan). Address the pattern of 3 similar incidents and why
+  previous fixes didn't prevent this one.
+
+assertions:
+  - type: llm_judge
+    criteria: "Analysis is truly blameless — doesn't blame the 8-minute reviewer or the senior author, instead identifies systemic factors: outdated CODEOWNERS, missing domain expertise after tech lead departure, SLA pressure encouraging superficial reviews, and trust bias toward senior engineers"
+    weight: 0.35
+    description: "Truly blameless analysis"
+  - type: llm_judge
+    criteria: "Improvements address all levels — individual (reviewer training on concurrency), team (payment domain expertise gap), process (CODEOWNERS update, risk-based review requirements), and organizational (SLA policy that doesn't incentivize rubber-stamping)"
+    weight: 0.35
+    description: "Multi-level improvements"
+  - type: llm_judge
+    criteria: "Pattern analysis explains why previous fixes failed — the 3 prior incidents likely led to narrow fixes (fix the specific bug) rather than systemic changes, and this postmortem addresses the meta-problem of why postmortem action items aren't preventing recurrence"
+    weight: 0.30
+    description: "Pattern-breaking analysis"

package/courses/github-pr-review/scenarios/level-4/review-org-design.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: review-org-design
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Design review organization — build teams and roles dedicated to code review excellence, training, and tooling"
+  tags: [github, pr-review, org-design, teams, roles, career-paths, expert]
+
+state: {}
+
+trigger: |
+  You're the SVP of Engineering creating a dedicated "Developer
+  Productivity" organization that includes a Code Review Excellence
+  team. The company has 600 engineers and the CEO has approved headcount
+  for this new function.
+
+  The mandate:
+  - Reduce average merge time from 3 days to 1 day
+  - Reduce review-related production incidents by 80%
+  - Achieve 90%+ developer satisfaction with the review process (from
+    current 45%)
+  - Build review tooling and automation as a platform
+  - Create a reviewer training and certification program
+  - Establish review standards that scale with the company
+
+  Available headcount: 12 FTEs
+  Budget: $500K/year for tooling (separate from headcount)
+
+  Questions to answer:
+  1. What roles do you need? (Platform engineers, program managers,
+     trainers, data analysts?)
+  2. How does this team interact with the 50 product teams?
+  3. Where does this team sit in the org chart?
+  4. What does the career ladder look like for "review excellence"?
+  5. How do you avoid becoming a bottleneck or bureaucratic overhead?
+  6. What does success look like at 6 months, 1 year, and 2 years?
+
+  Constraint: The engineering culture values autonomy. Any centralized
+  function risks being seen as "the review police." You need buy-in from
+  team leads.
+
+  Task: Design the organization. Write: the team structure (roles,
+  responsibilities, reporting lines), the interaction model with product
+  teams (embedded vs centralized vs hybrid), the career ladder for review
+  excellence roles, the buy-in strategy (how to get team lead support
+  without being "review police"), the phased roadmap (what to deliver at
+  6 months, 1 year, 2 years), and the success metrics with accountability
+  framework.
+
+assertions:
+  - type: llm_judge
+    criteria: "Team structure is well-designed — includes platform engineers (build tooling), program managers (process), trainers (education), and data analysts (metrics), with clear roles and not too top-heavy for a 12-person team"
+    weight: 0.35
+    description: "Well-designed team structure"
+  - type: llm_judge
+    criteria: "Interaction model preserves autonomy — uses an enabling/platform model (not command-and-control), product teams opt in to services, the team provides tools and training rather than mandates, and the 'review police' perception is actively countered"
+    weight: 0.35
+    description: "Autonomy-preserving interaction model"
+  - type: llm_judge
+    criteria: "Career ladder is viable — creates career progression for review excellence roles (not a dead-end), connects to broader developer productivity career paths, and provides incentives for engineers to specialize in this area"
+    weight: 0.30
+    description: "Viable career ladder"