dojo.md 0.1.0 → 0.2.0

package/courses/github-pr-review/scenarios/level-3/cross-functional-review.yaml

@@ -0,0 +1,60 @@
+meta:
+  id: cross-functional-review
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Cross-functional PR review — coordinate reviews involving design, security, legal, and product stakeholders beyond engineering"
+  tags: [github, pr-review, cross-functional, design-review, security-review, advanced]
+
+state: {}
+
+trigger: |
+  You're leading the review process for a major feature PR that requires
+  input from multiple non-engineering stakeholders. The PR implements
+  GDPR consent management and data deletion for EU users.
+
+  The PR ("Implement GDPR consent and right-to-erasure"):
+  - 600 lines across 15 files
+  - Changes: consent UI, data collection APIs, deletion pipeline, audit
+    logging, cookie management, privacy policy integration
+
+  Required reviewers and their concerns:
+  1. Engineering (2 reviewers): Code quality, architecture, test coverage
+  2. Security team: Data handling, encryption at rest, access controls,
+     deletion completeness
+  3. Legal/Privacy: Consent language accuracy, deletion scope compliance
+     with Article 17, data retention exceptions
+  4. Design: Consent banner UX, accessibility, user flow clarity
+  5. Product: Feature completeness against GDPR requirements spec,
+     analytics impact (what tracking stops)
+
+  Challenges:
+  - Legal doesn't use GitHub — they review via email/Confluence
+  - Design reviews in Figma, not in code
+  - Security wants to run their own penetration test before approval
+  - Product wants to verify in a staging environment
+  - Each stakeholder has different availability (legal is 2 weeks out)
+  - The PR is too large for a single review session
+
+  Task: Design the cross-functional review process. Write: the review
+  coordination plan (who reviews what, in what order, with timeline),
+  the PR splitting strategy (if the 600-line PR should be broken up for
+  different reviewers), the tooling integration (how to bring GitHub,
+  Figma, Confluence, and email reviews into a single audit trail), the
+  sign-off process (how each stakeholder formally approves), and the
+  template for cross-functional PRs. Address the legal reviewer's 2-week
+  availability constraint.
+
+assertions:
+  - type: llm_judge
+    criteria: "Review coordination is realistic — sequences reviews logically (legal/design first since they have longest lead time, security pen test in parallel with code review, product verification last in staging), and the timeline accounts for the legal 2-week constraint"
+    weight: 0.35
+    description: "Realistic review coordination"
+  - type: llm_judge
+    criteria: "Tooling integration creates a single audit trail — brings email/Confluence legal feedback, Figma design comments, and GitHub code review into a traceable record, handles stakeholders who don't use GitHub"
+    weight: 0.35
+    description: "Unified audit trail"
+  - type: llm_judge
+    criteria: "The process is repeatable — the template and sign-off process could be reused for future cross-functional features, balances thoroughness with velocity, and doesn't create a 6-week review bottleneck"
+    weight: 0.30
+    description: "Repeatable cross-functional process"

package/courses/github-pr-review/scenarios/level-3/incident-driven-review.yaml

@@ -0,0 +1,63 @@
+meta:
+  id: incident-driven-review
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Incident-driven review improvement — analyze post-incident review failures and redesign processes to catch similar issues"
+  tags: [github, pr-review, incidents, post-mortem, process-improvement, advanced]
+
+state: {}
+
+trigger: |
+  Three production incidents in the last quarter were traced back to
+  code review failures — issues that should have been caught in review
+  but weren't. You've been tasked with analyzing the failures and
+  improving the review process.
+
+  Incident 1 — Data loss (Severity: Critical):
+  A migration PR dropped a column that was still used by a background
+  job. The PR had 2 approvals and CI was green. Neither reviewer checked
+  the background job code. The column name appeared in 3 files that
+  weren't part of the PR diff.
+  Impact: 48 hours of data loss for 12,000 users
+  Review gap: Reviewers only looked at files in the diff, not at callers
+  of the changed code
+
+  Incident 2 — Security breach (Severity: Critical):
+  A PR added a debug endpoint that returned user data without
+  authentication. It was merged as part of a 1,200-line "feature
+  complete" PR. The endpoint was on line 847 of a 900-line file.
+  Impact: 500 user records exposed for 3 days
+  Review gap: The PR was too large to review effectively, the debug
+  endpoint was buried in a massive diff, and the reviewer admitted
+  to "skimming" the last 400 lines
+
+  Incident 3 — Financial miscalculation (Severity: High):
+  A PR changed a currency conversion formula. The unit test used the
+  same incorrect formula as the implementation (copy-pasted), so tests
+  passed. The reviewer approved based on "tests pass."
+  Impact: $47,000 in incorrect charges over 2 weeks
+  Review gap: Tests validated consistency, not correctness. No
+  independent verification of the business logic
+
+  Task: For each incident, write: the root cause analysis of the review
+  failure (why the review process didn't catch it), the specific process
+  change that would prevent recurrence, the detection mechanism (how to
+  know if the process change is working), and the review checklist
+  addition. Then write a synthesis: the systemic patterns across all 3
+  incidents, the comprehensive review process improvements, and the
+  training program to address the skill gaps revealed by these incidents.
+
+assertions:
+  - type: llm_judge
+    criteria: "Root causes go beyond blaming reviewers — identifies systemic issues: #1 reveals that reviews don't check callers/dependencies, #2 reveals that large PRs can't be reviewed effectively, #3 reveals that test-pass ≠ correctness and independent verification is needed"
+    weight: 0.35
+    description: "Systemic root cause analysis"
+  - type: llm_judge
+    criteria: "Process changes are specific and preventive — #1 adds dependency/caller analysis to reviews (or automated tooling), #2 enforces PR size limits and adds security-specific review for auth-related code, #3 requires independent test verification for business-critical calculations"
+    weight: 0.35
+    description: "Specific preventive process changes"
+  - type: llm_judge
+    criteria: "Synthesis identifies patterns — connects all 3 incidents to broader themes (over-reliance on CI, review depth vs PR size, domain-specific review expertise), and the training program addresses the revealed skill gaps without creating blame"
+    weight: 0.30
+    description: "Pattern-connecting synthesis"

package/courses/github-pr-review/scenarios/level-3/large-scale-review-operations.yaml

@@ -0,0 +1,55 @@
+meta:
+  id: large-scale-review-operations
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Design large-scale review operations — create review processes for 100+ developer organizations with multiple teams and repositories"
+  tags: [github, pr-review, operations, scaling, organization, advanced]
+
+state: {}
+
+trigger: |
+  You're the Director of Engineering at a company scaling from 30 to
+  150 engineers over the next year. The current review process (informal,
+  team-based) is already breaking down at 30 engineers. You need to
+  design a review system that scales.
+
+  Current problems:
+  - 15 repositories with no consistent review process
+  - Some repos require 1 approval, others require 3
+  - No cross-repo review standards (code style varies wildly)
+  - Average merge time: 3 days (target: 1 day)
+  - Knowledge silos: only 2 people can review the payment code
+  - New hires take 3 months before they can review code confidently
+  - No review quality metrics — quantity is tracked but not quality
+  - Occasional "LGTM" rubber-stamp approvals on critical code
+
+  Planned growth:
+  - 5 new teams being formed (total: 10 teams)
+  - 3 new repos being created (total: 18 repos)
+  - Acquiring a company with 20 engineers and their own repo/processes
+  - Moving to a monorepo for shared libraries (under discussion)
+
+  Task: Design the review operations framework. Write: the review
+  standards document (minimum requirements for all repos, with team-
+  specific extensions), the reviewer assignment system (CODEOWNERS
+  strategy, rotation, knowledge distribution), the quality assurance
+  mechanisms (preventing rubber stamps, ensuring review depth), the
+  onboarding program for new reviewers (shadow reviewing, graduated
+  autonomy), and the metrics framework (what to measure, targets,
+  dashboards). Include the migration plan for existing repos and the
+  acquired company's integration.
+
+assertions:
+  - type: llm_judge
+    criteria: "Standards are scalable — creates a base standard that applies to all 18 repos with team-specific extensions, handles the monorepo question, and includes the acquired company integration with a gradual alignment plan rather than forcing immediate compliance"
+    weight: 0.35
+    description: "Scalable review standards"
+  - type: llm_judge
+    criteria: "Knowledge distribution is addressed — CODEOWNERS strategy prevents knowledge silos (rotation, mandatory cross-training), reviewer onboarding program reduces the 3-month ramp-up time, and the system handles the 2-person payment code bottleneck"
+    weight: 0.35
+    description: "Knowledge distribution strategy"
+  - type: llm_judge
+    criteria: "Quality assurance prevents gaming — mechanisms to detect rubber-stamp reviews (review depth metrics, comment quality), graduated approval requirements (critical code vs routine changes), and the metrics framework measures quality not just speed"
+    weight: 0.30
+    description: "Anti-gaming quality assurance"

package/courses/github-pr-review/scenarios/level-3/monorepo-review-process.yaml

@@ -0,0 +1,68 @@
+meta:
+  id: monorepo-review-process
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Design monorepo review process — handle review routing, ownership, and CI for PRs that touch multiple packages in a monorepo"
+  tags: [github, pr-review, monorepo, CODEOWNERS, CI, routing, advanced]
+
+state: {}
+
+trigger: |
+  Your company just migrated 12 microservice repos into a single
+  monorepo using Turborepo. The review process that worked for individual
+  repos is now broken. Key problems:
+
+  Monorepo structure:
+  ```
+  /
+  ├── apps/
+  │   ├── web/           (Frontend team, 8 devs)
+  │   ├── api/           (Backend team, 10 devs)
+  │   ├── mobile/        (Mobile team, 6 devs)
+  │   └── admin/         (Platform team, 4 devs)
+  ├── packages/
+  │   ├── ui/            (Design system — shared)
+  │   ├── auth/          (Auth library — shared)
+  │   ├── database/      (DB client — backend + platform)
+  │   ├── config/        (Shared config — everyone)
+  │   └── types/         (Shared types — everyone)
+  ├── infra/             (DevOps, 3 devs)
+  └── tools/             (DX team, 2 devs)
+  ```
+
+  Current problems:
+  - A PR touching `packages/types` requests review from ALL 33 engineers
+  - CI runs all tests for every PR (45 minutes), even single-line changes
+  - CODEOWNERS from individual repos were concatenated — conflicts and
+    overlaps everywhere
+  - Cross-package PRs (e.g., API change + type update + web update) have
+    no clear review strategy
+  - Shared packages have no clear ownership — everyone and no one
+
+  A recent incident: A change to `packages/auth` broke mobile login but
+  the mobile team wasn't requested as reviewers. It took 2 days to
+  discover the break.
+
+  Task: Design the monorepo review process. Write: the CODEOWNERS
+  strategy (ownership model for shared packages, cross-team routing),
+  the CI optimization (affected-package detection, selective test runs),
+  the cross-package PR guidelines (when to split PRs, when one PR is
+  fine, who reviews what), the shared package governance model (who
+  owns shared code, how changes are reviewed), and the incident
+  prevention strategy (ensuring breaking changes are caught by affected
+  teams). Include the CODEOWNERS file and a sample CI workflow.
+
+assertions:
+  - type: llm_judge
+    criteria: "CODEOWNERS handles the monorepo correctly — shared packages have designated owners plus affected-team routing (not all 33 engineers), apps have team ownership, and cross-package PRs are handled with both source and consumer team reviews"
+    weight: 0.35
+    description: "Correct monorepo CODEOWNERS"
+  - type: llm_judge
+    criteria: "CI optimization is practical — uses affected-package detection (Turborepo's --filter or similar), runs only relevant tests, but still runs shared-package tests when shared code changes. Reduces 45-minute CI to relevant-only runs"
+    weight: 0.35
+    description: "Practical CI optimization"
+  - type: llm_judge
+    criteria: "Incident prevention is addressed — the auth-breaking-mobile scenario is prevented through consumer-team notification on shared package changes, integration test requirements for cross-package changes, and a shared package change RFC process"
+    weight: 0.30
+    description: "Cross-package incident prevention"

package/courses/github-pr-review/scenarios/level-3/review-automation-platform.yaml

@@ -0,0 +1,61 @@
+meta:
+  id: review-automation-platform
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Build a review automation platform — design GitHub Apps and bots that automate review assignment, checks, and notifications"
+  tags: [github, pr-review, automation, GitHub-Apps, bots, platform, advanced]
+
+state: {}
+
+trigger: |
+  You're building an internal review automation platform (GitHub App)
+  for your 200-engineer organization. The platform should automate the
+  repetitive parts of the review process while keeping humans in the
+  loop for judgment calls.
+
+  Requirements:
+  1. Smart reviewer assignment: Based on code expertise (git blame),
+     current load, timezone overlap, and review history
+  2. PR categorization: Automatically label PRs by type (feature, bug,
+     refactor, dependency), size (S/M/L/XL), and risk (low/medium/high)
+  3. Review reminders: Escalating notifications when SLAs are missed
+  4. Stale review detection: Alert when reviews are approved but code
+     changed after approval
+  5. Merge readiness: Check all requirements met before allowing merge
+  6. Review analytics: Weekly digests for team leads
+
+  Technical constraints:
+  - Must use GitHub App (not personal access tokens)
+  - Must handle 500+ PRs per week across 30 repos
+  - Must be resilient to GitHub API rate limits (5,000 requests/hour)
+  - Must handle webhook delivery failures gracefully
+  - Must not expose any internal code or review data externally
+
+  The platform needs to integrate with:
+  - GitHub (webhooks + API)
+  - Slack (notifications)
+  - Jira (issue linking)
+  - PagerDuty (on-call reviewer for urgent PRs)
+
+  Task: Design the review automation platform. Write: the system
+  architecture (webhook handlers, job queues, data store, integrations),
+  the reviewer assignment algorithm (inputs, weighting, edge cases),
+  the PR categorization rules (heuristics for type, size, and risk),
+  the notification and escalation logic, and the API rate limit strategy.
+  Include the GitHub App manifest, webhook event list, and a sample
+  webhook handler for the PR opened event.
+
+assertions:
+  - type: llm_judge
+    criteria: "Architecture handles scale — uses webhook-driven processing with a job queue (not synchronous), handles 500+ PRs/week within GitHub API rate limits, has retry logic for webhook failures, and separates concerns (assignment, categorization, notifications)"
+    weight: 0.35
+    description: "Scalable architecture"
+  - type: llm_judge
+    criteria: "Reviewer assignment algorithm is sophisticated — considers code expertise (git blame/log), current review load, timezone, and rotation, with fallbacks when primary reviewers are unavailable. Edge cases like vacation, new hires, and single-expert code areas are handled"
+    weight: 0.35
+    description: "Sophisticated assignment algorithm"
+  - type: llm_judge
+    criteria: "PR categorization is practical — risk assessment considers files changed (config, auth, financial code = high risk), size buckets are meaningful (not just line count — considers file count and complexity), and categorization drives review requirements (high-risk PRs need more reviewers)"
+    weight: 0.30
+    description: "Practical PR categorization"

package/courses/github-pr-review/scenarios/level-3/review-culture-design.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: review-culture-design
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Design review culture — build a healthy code review culture that balances thoroughness, speed, psychological safety, and learning"
+  tags: [github, pr-review, culture, psychological-safety, learning, advanced]
+
+state: {}
+
+trigger: |
+  You're the VP of Engineering inheriting a team with a toxic review
+  culture. An anonymous survey revealed:
+
+  Survey results (60 respondents):
+  - 72% feel anxious when submitting PRs
+  - 58% have avoided submitting code changes due to fear of harsh reviews
+  - 45% say certain reviewers are "gatekeepers" who block PRs for
+    personal preferences
+  - 38% feel reviews focus on criticism without acknowledging good work
+  - 65% of junior engineers say reviews make them feel incompetent
+  - Only 22% say they learn from code reviews
+
+  Specific complaints:
+  - "Reviewer X rewrites my entire PR in comments — why didn't they just
+    write it themselves?"
+  - "I got 47 comments on a 50-line PR. Most were 'I would do this
+    differently.' Not bugs, just preferences."
+  - "When I push back on review feedback, I get labeled as 'difficult.'"
+  - "Senior engineers' PRs get rubber-stamped. Junior PRs get nitpicked."
+  - "I spent 2 weeks on a feature. The review said 'wrong approach,
+    start over.' No one told me before I started coding."
+
+  Good signs in the data:
+  - Teams that do pair programming have 85% satisfaction with reviews
+  - The mobile team (led by a specific manager) has 90% positive review
+    experience — what are they doing differently?
+  - Engineers who've been through reviewer training rate reviews 3x
+    more positively
+
+  Task: Design a culture transformation plan. Write: the diagnosis (root
+  causes of the toxic patterns), the review culture principles (code of
+  conduct for reviewers and authors), the specific interventions (what
+  to change immediately, in 30 days, and in 90 days), the reviewer
+  training program (curriculum, practice exercises, assessment), the
+  accountability mechanisms (how to address bad review behavior without
+  creating fear), and the success metrics. Study the mobile team's
+  positive patterns and scale them.
+
+assertions:
+  - type: llm_judge
+    criteria: "Diagnosis identifies root causes — power dynamics (senior vs junior treatment), missing early feedback (architecture reviews before coding), preference-vs-requirement confusion, and lack of reviewer accountability. Analyzes the mobile team's success factors"
+    weight: 0.35
+    description: "Root cause diagnosis"
+  - type: llm_judge
+    criteria: "Interventions are concrete and phased — immediate actions (review code of conduct, stop specific harmful patterns), 30-day actions (reviewer training, pair review program), 90-day actions (culture metrics, accountability processes), each with measurable outcomes"
+    weight: 0.35
+    description: "Concrete phased interventions"
+  - type: llm_judge
+    criteria: "Plan balances accountability with safety — addresses the 'gatekeeper' behavior without creating fear, includes mechanisms for authors to give feedback on reviews, and doesn't just create more rules but changes underlying dynamics"
+    weight: 0.30
+    description: "Accountability-safety balance"

package/courses/github-pr-review/scenarios/level-3/review-data-pipeline.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: review-data-pipeline
+  level: 3
+  course: github-pr-review
+  type: output
+  description: "Build a review analytics pipeline — collect, analyze, and act on PR review data to continuously improve review effectiveness"
+  tags: [github, pr-review, analytics, data-pipeline, metrics, advanced]
+
+state: {}
+
+trigger: |
+  You're building a review analytics system for your 100-engineer org.
+  Currently, review data lives only in GitHub and there's no way to
+  track trends, identify problems, or measure improvements. You need
+  to design a data pipeline that collects review data and produces
+  actionable insights.
+
+  Data sources available:
+  - GitHub API: PR data, reviews, comments, timelines, check runs
+  - GitHub webhooks: real-time events for PR state changes
+  - Slack: review request messages and response times
+  - Jira: linked issues, sprint assignments, priority levels
+  - Deployment system: deploy timestamps, rollback frequency
+
+  Key questions to answer with data:
+  1. Which teams have the fastest/slowest review cycles?
+  2. What percentage of review comments lead to code changes vs are
+     ignored or resolved without changes?
+  3. Do certain types of PRs (size, category, author seniority) have
+     more review rounds?
+  4. What's the correlation between review thoroughness and post-deploy
+     bug rate?
+  5. Are specific reviewers consistently over- or under-loaded?
+  6. How effective are automated checks at reducing human review load?
+
+  Constraints:
+  - Budget: $500/month for infrastructure
+  - Must not slow down GitHub workflows
+  - Must respect developer privacy (no individual shaming dashboards)
+  - Data retention: 2 years for compliance
+
+  Task: Design the review data pipeline. Write: the data model (what
+  entities to track, relationships, derived metrics), the collection
+  architecture (webhooks vs polling, real-time vs batch, storage), the
+  analytics layer (dashboards for eng managers, team leads, and ICs),
+  the privacy and ethics guidelines (what data is shown at what level),
+  and 3 specific analyses you would run first with expected findings
+  and recommended actions. Include the schema design and sample queries.
+
+assertions:
+  - type: llm_judge
+    criteria: "Data model is comprehensive — tracks PRs, reviews, comments, review rounds, time-to-first-review, time-to-merge, reviewer load, comment types (blocking vs suggestion), and correlates with deployment outcomes"
+    weight: 0.35
+    description: "Comprehensive data model"
+  - type: llm_judge
+    criteria: "Architecture is practical within constraints — fits $500/month budget (likely uses GitHub webhooks + a lightweight database like Postgres, not enterprise analytics tools), processes events without slowing GitHub, and handles 2-year retention"
+    weight: 0.35
+    description: "Budget-practical architecture"
+  - type: llm_judge
+    criteria: "Privacy is respected — dashboards show team-level metrics (not individual ranking), individual data is available only to the person themselves and their direct manager, and the ethics guidelines prevent using review metrics for performance reviews without context"
+    weight: 0.30
+    description: "Privacy-respecting analytics"

package/courses/github-pr-review/scenarios/level-4/enterprise-review-operations.yaml

@@ -0,0 +1,61 @@
+meta:
+  id: enterprise-review-operations
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Design enterprise review operations — build review infrastructure for 500+ engineer organizations with multiple business units"
+  tags: [github, pr-review, enterprise, operations, scaling, expert]
+
+state: {}
+
+trigger: |
+  You're the VP of Developer Experience at a 600-engineer company with
+  4 business units, 50 teams, and 200+ repositories. The CEO mandated
+  a company-wide code review standardization initiative after an audit
+  found that review practices vary wildly across business units.
+
+  Current state by business unit:
+  - Consumer (200 engineers, 80 repos): Fast-shipping culture, average
+    review time 4 hours, but 3 production incidents last quarter traced
+    to inadequate review
+  - Enterprise (150 engineers, 60 repos): Thorough reviews, average 3
+    days to merge, but shipping velocity is 40% below target
+  - Platform (120 engineers, 40 repos): Strong review culture, but
+    bottlenecked on 5 senior engineers who review everything
+  - Data/ML (130 engineers, 30 repos): Notebooks and experiments mixed
+    with production code, no distinction in review rigor
+
+  Cross-cutting challenges:
+  - 3 different GitHub Enterprise organizations (legacy from acquisitions)
+  - Inconsistent branch protection across all orgs
+  - No unified metrics — each BU reports differently
+  - Some teams use GitLab, most use GitHub
+  - Compliance requirements differ by BU (Consumer: GDPR, Enterprise:
+    SOX/SOC 2, Platform: PCI DSS, Data: CCPA + model governance)
+
+  Budget: $2M/year for tooling and headcount
+  Timeline: 12 months to reach "standardized" state
+
+  Task: Design the enterprise review operations strategy. Write: the
+  unified review standard (minimum bar for all BUs, with BU-specific
+  extensions), the organizational model (centralized review platform
+  team vs federated ownership), the tooling strategy (GitHub Enterprise
+  configuration, custom automation, analytics platform), the migration
+  plan for each BU (addressing their specific challenges), and the ROI
+  model (how the $2M investment pays back in reduced incidents, faster
+  shipping, and compliance readiness). Include the quarterly milestones
+  and success metrics.
+
+assertions:
+  - type: llm_judge
+    criteria: "Standards balance uniformity with BU autonomy — the minimum bar applies everywhere (e.g., no self-merge, required reviews), but BU-specific extensions handle different compliance needs and shipping cultures without forcing one-size-fits-all"
+    weight: 0.35
+    description: "Balanced standards"
+  - type: llm_judge
+    criteria: "Migration plan addresses each BU's specific challenge — Consumer needs quality without slowing down, Enterprise needs speed without losing rigor, Platform needs to distribute knowledge, and Data/ML needs production code separation from experiments"
+    weight: 0.35
+    description: "BU-specific migration plans"
+  - type: llm_judge
+    criteria: "ROI model is quantified — connects review improvements to business outcomes (reduced incidents = $ saved, faster shipping = revenue impact, compliance readiness = audit cost avoidance), and the $2M budget is allocated across tooling, headcount, and training"
+    weight: 0.30
+    description: "Quantified ROI model"

package/courses/github-pr-review/scenarios/level-4/expert-review-shift.yaml

@@ -0,0 +1,62 @@
+meta:
+  id: expert-review-shift
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Expert review shift — navigate organizational crises, vendor decisions, executive escalations, and team morale challenges simultaneously"
+  tags: [github, pr-review, shift-simulation, crisis, organizational, expert]
+
+state: {}
+
+trigger: |
+  You're the VP of Developer Experience during a critical week where
+  multiple review-related crises converge.
+
+  Crisis 1 — Vendor lock-in emergency:
+  Your review automation vendor (ReviewBot Pro) announced they're being
+  acquired by a competitor. They're sunsetting their product in 90 days.
+  Your entire review assignment, SLA tracking, and analytics run on
+  their platform. 600 engineers depend on it daily. You need a migration
+  plan by Friday for the executive team meeting.
+
+  Crisis 2 — Compliance audit failure:
+  The SOC 2 auditor flagged 3 critical findings related to code review:
+  (a) 8% of PRs in the payment codebase were self-merged, (b) review
+  audit trail has gaps where comments were deleted, and (c) emergency
+  changes have no post-deployment review documentation. The remediation
+  deadline is 30 days. Failure means losing a $15M enterprise customer
+  who requires SOC 2 compliance.
+
+  Crisis 3 — Engineering morale crisis:
+  An anonymous post on the company's internal forum went viral: "Our
+  code review process is killing innovation." 200 engineers upvoted it.
+  The post includes specific complaints: mandatory 2-reviewer requirement
+  is "bureaucratic," the review SLA creates "artificial urgency," and
+  "reviewers block PRs over personal preferences." The CEO forwarded it
+  to you asking "is this true?"
+
+  Crisis 4 — Team capacity:
+  Your Review Platform team of 12 engineers has 3 people on medical
+  leave, 2 threatening to quit (burned out from the always-on review
+  platform), and the remaining 7 are already working on the next
+  quarterly OKR deliverables. You have no slack capacity.
+
+  Task: Handle all 4 crises. For each, write: the immediate triage
+  (what to do today), the 1-week action plan, the 30-day resolution
+  plan, and the communication to each stakeholder group. Then write the
+  integrated strategy memo showing how these crises are interconnected
+  and the unified approach to resolving them while protecting the team.
+
+assertions:
+  - type: llm_judge
+    criteria: "Vendor migration is practical — prioritizes critical functionality (assignment, SLA tracking), evaluates build-vs-buy in 90 days, identifies what can be replaced with GitHub native features immediately, and doesn't propose rebuilding everything from scratch"
+    weight: 0.35
+    description: "Practical vendor migration"
+  - type: llm_judge
+    criteria: "Compliance remediation is achievable in 30 days — addresses each finding with specific technical fixes (branch protection for self-merge, immutable audit trail, emergency review documentation process), and the plan is realistic given the team capacity crisis"
+    weight: 0.35
+    description: "Achievable compliance remediation"
+  - type: llm_judge
+    criteria: "Morale and capacity crises are handled with empathy — the CEO response acknowledges legitimate complaints while providing data context, the team capacity plan doesn't just add more work, and the unified strategy memo shows how solving one crisis helps the others"
+    weight: 0.30
+    description: "Empathetic crisis handling"

package/courses/github-pr-review/scenarios/level-4/review-data-architecture.yaml

@@ -0,0 +1,69 @@
+meta:
+  id: review-data-architecture
+  level: 4
+  course: github-pr-review
+  type: output
+  description: "Design review data architecture — build the data infrastructure for review analytics, ML-powered insights, and compliance reporting"
+  tags: [github, pr-review, data-architecture, analytics, ML, expert]
+
+state: {}
+
+trigger: |
+  You're the Head of Data Engineering building the data infrastructure
+  for the company's review analytics platform. The system needs to
+  support real-time dashboards, ML-powered insights, and compliance
+  audit reports.
+
+  Data sources:
+  - GitHub Events API: PR events, review events, comment events,
+    check run events (estimated 50,000 events/day)
+  - GitHub REST API: PR details, file diffs, user profiles
+  - Slack: Review request/response timestamps
+  - Jira: Issue metadata, sprint data, priority levels
+  - Deployment system: Deploy events, rollback events
+  - Incident management: Incident data, root cause analysis
+
+  Analytics requirements:
+  1. Real-time dashboard: Current review queue, SLA status, team load
+     (refreshed every 5 minutes)
+  2. Historical analytics: Trends over 2 years, team comparisons,
+     seasonal patterns
+  3. ML features: Reviewer expertise scoring, PR risk prediction,
+     review time estimation, anomaly detection (rubber stamp detection)
+  4. Compliance reports: Audit trails, SOC 2 evidence, monthly
+     compliance scorecards
+  5. Developer experience: Individual review statistics (private to
+     the engineer), career growth tracking
+
+  Technical requirements:
+  - Event-driven ingestion (no polling GitHub API)
+  - Schema evolution support (GitHub API changes frequently)
+  - 2-year data retention with automated archival
+  - Sub-second query response for dashboards
+  - ML feature store for model training and serving
+  - GDPR compliance (engineer data deletion on request)
+
+  Budget: $150K/year infrastructure, 4 engineers for 6 months to build
+
+  Task: Design the data architecture. Write: the data model (entities,
+  relationships, derived metrics, slowly changing dimensions), the
+  ingestion pipeline (event processing, transformation, loading), the
+  storage strategy (which database for which use case — OLTP vs OLAP
+  vs time-series), the ML feature store design (feature computation,
+  serving, versioning), and the privacy/compliance layer (data masking,
+  retention, deletion). Include the schema for key tables and sample
+  queries for each analytics requirement.
+
+assertions:
+  - type: llm_judge
+    criteria: "Data model captures review complexity — models PRs, reviews, comments, review rounds, reviewer load, time-between-events, and derived metrics like 'review depth score' and 'rubber stamp probability'. Handles the GitHub event schema correctly"
+    weight: 0.35
+    description: "Complexity-capturing data model"
+  - type: llm_judge
+    criteria: "Storage strategy is appropriate — uses different storage for different patterns (event stream processing for real-time, columnar/OLAP for historical analytics, time-series for metrics, and a feature store for ML), within the $150K/year budget"
+    weight: 0.35
+    description: "Appropriate storage strategy"
+  - type: llm_judge
+    criteria: "ML feature store is practical — features include reviewer expertise score, PR complexity score, and review time prediction with clear computation logic, versioning for model reproducibility, and serving layer for real-time inference"
+    weight: 0.30
+    description: "Practical ML feature store"