@cubis/foundry 0.3.82 → 0.3.84

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/api-design/evals/assertions.md

@@ -1,53 +0,0 @@
-# API Design Eval Assertions
-
-## Eval 1: REST API Contract Design
-
-This eval tests the ability to design a RESTful API with proper resource modeling, HTTP semantics, pagination, error handling, and OpenAPI documentation for a task management system.
-
-### Assertions
-
-| # | Type | What it checks | Why it matters |
-|---|------|----------------|----------------|
-| 1 | contains | `GET` — Proper HTTP method usage | REST APIs must use HTTP methods with their standardized semantics. GET for safe retrieval enables caching and is idempotent. Using POST for reads breaks HTTP semantics and cache layers. |
-| 2 | contains | `cursor` — Cursor-based pagination | Offset pagination breaks when data changes between requests (duplicates, skipped items). Cursor-based pagination provides stable pages regardless of concurrent writes. |
-| 3 | contains | `application/problem+json` — RFC 9457 error format | Ad-hoc error formats require custom parsing per API. Problem Details provides a standardized structure that clients can handle generically while still including API-specific detail fields. |
-| 4 | contains | `openapi` — OpenAPI specification | A spec-first approach ensures documentation is always synchronized with implementation. OpenAPI enables code generation, testing, and SDK creation from a single source of truth. |
-| 5 | contains | `filter` — Collection filtering support | Clients must be able to retrieve subsets of collections without downloading everything. Query parameter filters reduce payload size, database load, and client-side processing. |
-
-### What a passing response looks like
-
-- Resource hierarchy: `/v1/workspaces/{id}/projects/{id}/tasks/{id}/comments/{id}`.
-- GET (list with filters), GET (by ID), POST (create), PATCH (partial update), DELETE for each resource.
-- Cursor-based pagination: `?cursor=eyJpZCI6MTIzfQ&limit=25` with `next_cursor` in response.
-- Filtering: `?status=in_progress&priority=high&assignee=user_123&due_before=2025-12-31`.
-- Error responses with `application/problem+json` content type, including `type` URI, `title`, `status`, `detail`, and `errors` array for validation failures.
-- OpenAPI 3.1 spec with `info`, `paths`, `components/schemas`, and `examples`.
-- Standard response envelope or direct resource representation with consistent field naming (camelCase).
-- Rate limit headers: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`.
-
----
-
-## Eval 2: GraphQL Schema Design
-
-This eval tests the ability to design a GraphQL schema with Relay-style pagination, input types, union-based error handling, and DataLoader for N+1 prevention in a social content platform.
-
-### Assertions
-
-| # | Type | What it checks | Why it matters |
-|---|------|----------------|----------------|
-| 1 | contains | `Connection` — Relay-style pagination types | Relay connections provide a standardized cursor-based pagination contract that works with any GraphQL client. Custom pagination reinvents a solved problem and breaks client library expectations. |
-| 2 | contains | `input` — Dedicated mutation input types | Input types encapsulate mutation arguments into a single typed object, enabling validation, documentation, and future field additions without breaking existing mutations. |
-| 3 | contains | `DataLoader` — N+1 query prevention | Without DataLoader, resolving a list of posts with their authors generates N+1 database queries. DataLoader batches and deduplicates loader calls within a single request. |
-| 4 | contains | `union` — Union types for error modeling | Returning errors in the data layer (union of success and error types) gives clients type-safe error handling. Throwing exceptions forces clients to parse untyped error extensions. |
-| 5 | contains | `visibility` — Authorization in resolvers | Post visibility (public, followers, private) must be enforced in resolvers, not just in the client. Missing server-side authorization means private content is accessible via direct GraphQL queries. |
-
-### What a passing response looks like
-
-- Types: `User`, `Post`, `Comment`, `Reaction`, `MediaAttachment`, `Tag`.
-- Connection types: `PostConnection { edges: [PostEdge], pageInfo: PageInfo }` with `PostEdge { cursor, node }`.
-- Input types: `CreatePostInput { content, mediaIds, tagIds, visibility }`, `UpdatePostInput { content?, visibility? }`.
-- Union result types: `CreatePostResult = Post | ValidationError | AuthorizationError`.
-- Visibility enum: `enum PostVisibility { PUBLIC, FOLLOWERS, PRIVATE }`.
-- Feed query with DataLoader for batch-loading authors, reaction counts, and comment counts.
-- Resolver architecture: per-type DataLoaders instantiated per-request to avoid cross-request caching.
-- Subscription type for real-time updates: `onNewComment(postId: ID!): Comment`.

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/api-design/evals/evals.json

@@ -1,68 +0,0 @@
-{
-  "evals": [
-    {
-      "name": "rest-api-contract-design",
-      "description": "Validate that the skill produces a REST API contract with proper resource modeling, HTTP method semantics, pagination, error responses, and OpenAPI documentation for a task management system.",
-      "input": "Design a REST API for a task management system with workspaces, projects, tasks, and comments. Tasks can be assigned to users, have due dates, priorities (low, medium, high, urgent), and statuses (todo, in_progress, review, done). Include endpoints for CRUD operations, task filtering and search, pagination, bulk operations, and proper error responses. Provide an OpenAPI 3.1 specification for the core endpoints.",
-      "assertions": [
-        {
-          "type": "contains",
-          "value": "GET",
-          "description": "The API must use proper HTTP methods with GET for retrieval operations, demonstrating understanding of safe and idempotent HTTP semantics."
-        },
-        {
-          "type": "contains",
-          "value": "cursor",
-          "description": "The API must implement cursor-based pagination for collection endpoints to maintain page stability when data changes between requests."
-        },
-        {
-          "type": "contains",
-          "value": "application/problem+json",
-          "description": "Error responses must follow RFC 9457 Problem Details format with structured fields (type, title, status, detail) rather than ad-hoc error structures."
-        },
-        {
-          "type": "contains",
-          "value": "openapi",
-          "description": "The specification must be a valid OpenAPI 3.x document with paths, components, and schema references demonstrating a spec-first API design approach."
-        },
-        {
-          "type": "contains",
-          "value": "filter",
-          "description": "The API must support filtering on collection endpoints (by status, priority, assignee, due date) using query parameters to enable efficient client-side data retrieval."
-        }
-      ]
-    },
-    {
-      "name": "graphql-schema-design",
-      "description": "Validate that the skill produces a GraphQL schema with proper type definitions, connection-based pagination, input types, error handling, and resolver architecture for a social content platform.",
-      "input": "Design a GraphQL schema for a social content platform with users, posts, comments, and reactions. Users can follow other users. Posts have media attachments, tags, and visibility settings (public, followers, private). Include the schema with type definitions, queries, mutations, connection-based pagination (Relay-style), input validation types, and union types for error handling. Show the resolver architecture for the feed query with DataLoader for N+1 prevention.",
-      "assertions": [
-        {
-          "type": "contains",
-          "value": "Connection",
-          "description": "The schema must implement Relay-style connection types with edges and pageInfo for cursor-based pagination of collections (PostConnection, CommentConnection)."
-        },
-        {
-          "type": "contains",
-          "value": "input",
-          "description": "Mutations must use dedicated input types (CreatePostInput, UpdatePostInput) to encapsulate mutation arguments rather than passing individual scalar arguments."
-        },
-        {
-          "type": "contains",
-          "value": "DataLoader",
-          "description": "The resolver architecture must use DataLoader or an equivalent batching mechanism to prevent N+1 query problems when resolving nested relationships."
-        },
-        {
-          "type": "contains",
-          "value": "union",
-          "description": "The schema must use union types or result types (e.g., CreatePostResult = Post | ValidationError) to model operation outcomes that include both success and typed error cases."
-        },
-        {
-          "type": "contains",
-          "value": "visibility",
-          "description": "The schema must model the visibility enum (PUBLIC, FOLLOWERS, PRIVATE) and the resolver must enforce authorization based on the viewer's relationship to the post author."
-        }
-      ]
-    }
-  ]
-}

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/architecture-doc/evals/assertions.md

@@ -1,49 +0,0 @@
-# Architecture Documentation Eval Assertions
-
-## Eval 1: C4 Diagram Generation
-
-This eval tests the ability to produce correct C4 model diagrams at System Context and Container levels with proper notation, relationships, and technology labels.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                       |
-|---|----------|-------------------------------------------------|------------------------------------------------------------------------------------------------------|
-| 1 | contains | `Person` — External actor identification        | C4 requires human actors to be modeled as Person elements. Mixing users into system boxes obscures who interacts with the system. |
-| 2 | contains | `System_Boundary` — Boundary delineation        | Container diagrams must show what is inside vs outside the system. Without boundaries, readers cannot tell owned from external. |
-| 3 | contains | `Container` — Deployable unit modeling          | Each independently deployable unit must be a Container with a technology label. Lumping multiple services into one box hides deployment complexity. |
-| 4 | contains | `Rel` — Explicit relationship definitions       | Connections between elements need labeled relationships describing protocol and purpose. Lines without labels are meaningless. |
-| 5 | contains | `System_Ext` — External system identification   | Third-party integrations (Stripe, SendGrid) must be System_Ext to make integration boundaries and vendor dependencies visible. |
-
-### What a passing response looks like
-
-- A System Context diagram showing three Person types (end-user, admin, API consumer) and the analytics platform as a central System element.
-- External systems (Stripe, SendGrid) shown as System_Ext with labeled relationships.
-- A Container diagram within a System_Boundary showing: React SPA, Node.js/Express API, PostgreSQL database, Redis cache, Bull job processor.
-- Each Container labeled with its technology stack (e.g., "Container(api, 'API Server', 'Node.js / Express')").
-- Relationships between containers showing protocols (HTTPS, TCP, Redis protocol) and data flow direction.
-- A diagram key or legend explaining the notation.
-
----
-
-## Eval 2: ADR Writing
-
-This eval tests the ability to produce a well-structured Architecture Decision Record with context, alternatives analysis, and consequence documentation.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                       |
-|---|----------|-------------------------------------------------|------------------------------------------------------------------------------------------------------|
-| 1 | contains | `## Status` — Decision lifecycle tracking       | ADRs without status markers cannot be distinguished as active, superseded, or deprecated. Teams act on stale decisions. |
-| 2 | contains | `## Context` — Decision drivers documented      | The context section captures the forces and constraints that drove the decision. Without it, future readers cannot evaluate if the decision still applies. |
-| 3 | contains | `## Decision` — Clear statement of choice       | The decision must be stated explicitly. ADRs that discuss trade-offs without concluding leave readers uncertain about what was actually chosen. |
-| 4 | contains | `## Consequences` — Impact documentation        | Both positive and negative consequences must be recorded so teams can monitor for predicted risks and verify expected benefits. |
-| 5 | contains | `MongoDB` — Alternative analysis                | The ADR must discuss rejected alternatives with specific reasoning. One-sided ADRs that only describe the winner lack the trade-off analysis that gives ADRs their value. |
-
-### What a passing response looks like
-
-- A numbered ADR title (e.g., "ADR-001: Use PostgreSQL as Primary Database").
-- Status set to "Accepted" (or "Proposed" if presented as a draft).
-- Context section explaining: multi-tenant SaaS, strong relational data, flexible form submissions, team's PostgreSQL experience.
-- Discussion of both PostgreSQL (with JSONB for flexible schemas) and MongoDB (native document model) with specific trade-offs.
-- Decision section clearly stating the chosen option with brief rationale.
-- Consequences section listing: benefits (team familiarity, ACID transactions, JSONB flexibility), risks (JSONB query complexity, potential schema rigidity), and mitigations.

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/architecture-doc/evals/evals.json

@@ -1,66 +0,0 @@
-[
-  {
-    "name": "c4-diagram-generation",
-    "description": "Validate that the skill produces correct C4 model diagrams at Context and Container levels with proper notation, relationships, and technology labels.",
-    "input": "Create C4 model diagrams (System Context and Container level) for a SaaS analytics platform. The system has: a React single-page application, a REST API built with Node.js/Express, a PostgreSQL database, a Redis cache, a background job processor (Bull/Redis), and integrations with Stripe for billing and SendGrid for email. Users include end-users (dashboard viewers), admin users, and an external data ingestion API consumer. Provide the diagrams in Mermaid syntax.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "Person",
-        "description": "The System Context diagram must identify external actors as Person elements to distinguish human users from system integrations."
-      },
-      {
-        "type": "contains",
-        "value": "System_Boundary",
-        "description": "The Container diagram must use System_Boundary to clearly delineate which containers are inside the system vs external dependencies."
-      },
-      {
-        "type": "contains",
-        "value": "Container",
-        "description": "Each deployable unit (SPA, API, database, cache, job processor) must be modeled as a Container element with technology labels."
-      },
-      {
-        "type": "contains",
-        "value": "Rel",
-        "description": "Relationships between elements must be explicitly defined with Rel statements that describe the purpose and protocol of each connection."
-      },
-      {
-        "type": "contains",
-        "value": "System_Ext",
-        "description": "External systems (Stripe, SendGrid) must be modeled as System_Ext to distinguish them from internal containers and make integration boundaries visible."
-      }
-    ]
-  },
-  {
-    "name": "adr-writing",
-    "description": "Validate that the skill produces a well-structured Architecture Decision Record with proper sections, alternatives analysis, and consequence documentation.",
-    "input": "Write an Architecture Decision Record for the following decision: The team is choosing between PostgreSQL and MongoDB as the primary database for a multi-tenant SaaS application. The application has strong relational data (users, organizations, billing) but also needs to store flexible form submissions with varying schemas. The team has more PostgreSQL experience. Write the ADR in the standard format.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "## Status",
-        "description": "The ADR must include a Status section (Proposed, Accepted, Deprecated, or Superseded) to indicate the decision lifecycle stage."
-      },
-      {
-        "type": "contains",
-        "value": "## Context",
-        "description": "The ADR must include a Context section explaining the forces, constraints, and business requirements that drive the decision."
-      },
-      {
-        "type": "contains",
-        "value": "## Decision",
-        "description": "The ADR must include a Decision section that clearly states what was decided, not just what was considered."
-      },
-      {
-        "type": "contains",
-        "value": "## Consequences",
-        "description": "The ADR must include a Consequences section documenting both positive and negative outcomes of the decision."
-      },
-      {
-        "type": "contains",
-        "value": "MongoDB",
-        "description": "The ADR must discuss MongoDB as a considered alternative with specific trade-offs evaluated, not just mention the chosen option."
-      }
-    ]
-  }
-]

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/ci-cd-pipeline/evals/assertions.md

@@ -1,50 +0,0 @@
-# CI/CD Pipeline Eval Assertions
-
-## Eval 1: GitHub Actions Workflow Design
-
-This eval tests the core CI/CD skill: designing a GitHub Actions workflow with proper stage structure, matrix builds, caching, and security practices for a Node.js monorepo.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                       |
-|---|----------|-------------------------------------------------|------------------------------------------------------------------------------------------------------|
-| 1 | contains | `actions/checkout@` — Pinned action usage       | Every workflow starts with checkout. Pinning to a SHA digest prevents supply chain attacks via tag hijacking. |
-| 2 | contains | `matrix` — Matrix build strategy                | Testing across Node 18 and Node 20 catches version-specific bugs before they reach production. A single version gives false confidence. |
-| 3 | contains | `cache` — Dependency caching                    | pnpm store caching eliminates redundant package downloads. Without caching, dependency installation dominates pipeline duration. |
-| 4 | contains | `needs:` — Job dependency declarations          | Explicit job dependencies enforce stage ordering: install -> lint/typecheck/test -> build. Without `needs`, jobs run in arbitrary order. |
-| 5 | contains | `pull_request` — Event trigger configuration    | The workflow must trigger on pull_request events and conditionally gate deploy-preview steps to PRs targeting main. |
-
-### What a passing response looks like
-
-- A `.github/workflows/ci.yml` file with `on: pull_request` and `on: push` triggers.
-- A matrix strategy specifying `node-version: [18, 20]` with `pnpm` setup.
-- Dependency caching using `actions/cache` or pnpm's built-in store path with a lockfile-based key.
-- Parallel lint, typecheck, and test jobs that declare `needs: install`.
-- Build and deploy-preview jobs gated with `if: github.event_name == 'pull_request' && github.base_ref == 'main'`.
-- All third-party actions pinned to full SHA digests rather than floating version tags.
-
----
-
-## Eval 2: Deployment Gate Configuration
-
-This eval tests the deployment skill: designing a promotion pipeline with environment protection, approval gates, progressive rollout, and automated rollback.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                       |
-|---|----------|-------------------------------------------------|------------------------------------------------------------------------------------------------------|
-| 1 | contains | `environment:` — GitHub environment usage       | GitHub environments provide built-in protection rules, deployment logs, and secret scoping. Ad-hoc conditionals lose these features. |
-| 2 | contains | `health` — Health check verification            | Deploying without verifying health creates silent failures. Health checks confirm the new version serves traffic correctly. |
-| 3 | contains | `rollback` — Rollback mechanism                 | Automated rollback on canary health failure limits blast radius. Without rollback, a bad canary degrades 10% of traffic indefinitely. |
-| 4 | contains | `canary` — Canary deployment stage              | Canary routing sends a small traffic percentage to the new version first. Skipping canary means 100% of users hit a bad deploy simultaneously. |
-| 5 | contains | `staging` — Staging environment definition      | Staging as the first promotion step catches integration issues before any production traffic is affected. Auto-deploy on main push provides fast feedback. |
-
-### What a passing response looks like
-
-- Three GitHub environment definitions: `staging`, `production-canary`, `production`.
-- Staging job triggered by `push` to `main` with no required reviewers.
-- Production-canary job with `environment: production-canary` and one required reviewer.
-- Production job with `environment: production` and two required reviewers.
-- Health check step after each deployment that polls an endpoint and fails the job on unhealthy response.
-- Rollback step in the canary job that triggers when the health check fails, reverting to the previous container image tag.
-- Clear separation between container image build (done once) and deployment (done per environment).

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/ci-cd-pipeline/evals/evals.json

@@ -1,66 +0,0 @@
-[
-  {
-    "name": "github-actions-workflow-design",
-    "description": "Validate that the skill produces a GitHub Actions CI workflow with proper stage structure, caching, matrix builds, and security hardening for a Node.js monorepo.",
-    "input": "Design a GitHub Actions CI workflow for a Node.js monorepo (pnpm workspaces) with three packages: api, web, and shared. The workflow should run lint, typecheck, and unit tests in parallel after dependency installation. Use matrix builds to test on Node 18 and Node 20. Implement pnpm store caching. Only run the build and deploy-preview steps for pull requests targeting main. Pin all action versions to SHA digests.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "actions/checkout@",
-        "description": "The workflow must use actions/checkout with a pinned version to clone the repository, forming the foundation of every CI pipeline step."
-      },
-      {
-        "type": "contains",
-        "value": "matrix",
-        "description": "The workflow must use a matrix strategy to test across multiple Node.js versions, ensuring compatibility is verified before merge."
-      },
-      {
-        "type": "contains",
-        "value": "cache",
-        "description": "The workflow must implement dependency caching (pnpm store or actions/cache) to avoid re-downloading packages on every run."
-      },
-      {
-        "type": "contains",
-        "value": "needs:",
-        "description": "Jobs must declare dependencies using the 'needs' keyword so that lint/typecheck/test stages only run after successful dependency installation."
-      },
-      {
-        "type": "contains",
-        "value": "pull_request",
-        "description": "The workflow must trigger on pull_request events and conditionally run deploy-preview steps only for PRs targeting main."
-      }
-    ]
-  },
-  {
-    "name": "deployment-gate-configuration",
-    "description": "Validate that the skill produces a deployment pipeline with environment protection rules, approval gates, progressive rollout, and rollback capabilities.",
-    "input": "Design a GitHub Actions deployment pipeline that promotes a Docker container image through three environments: staging (auto-deploy on main push), production-canary (requires one approval, deploys to 10% of traffic), and production (requires two approvals, full rollout). Include health check verification after each deployment stage. If the canary health check fails, automatically roll back. Use GitHub environments with protection rules.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "environment:",
-        "description": "The workflow must use GitHub environment declarations to leverage built-in protection rules, deployment history, and secret scoping."
-      },
-      {
-        "type": "contains",
-        "value": "health",
-        "description": "The pipeline must include health check verification after each deployment to confirm the new version is serving traffic correctly."
-      },
-      {
-        "type": "contains",
-        "value": "rollback",
-        "description": "The pipeline must include an automatic or triggered rollback mechanism for when canary health checks fail."
-      },
-      {
-        "type": "contains",
-        "value": "canary",
-        "description": "The pipeline must implement a canary deployment stage that routes a small percentage of traffic to the new version before full rollout."
-      },
-      {
-        "type": "contains",
-        "value": "staging",
-        "description": "The pipeline must define a staging environment that auto-deploys on main branch pushes as the first promotion step."
-      }
-    ]
-  }
-]

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/code-review/evals/assertions.md

@@ -1,50 +0,0 @@
-# Code Review Eval Assertions
-
-## Eval 1: Review Feedback Quality
-
-This eval tests whether the skill identifies critical security and correctness issues in code and provides structured, actionable feedback.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                          |
-|---|----------|-------------------------------------------------|---------------------------------------------------------------------------------------------------------|
-| 1 | contains | `SQL injection` — Injection vulnerability        | String interpolation in SQL is the textbook injection vector. Missing this is a disqualifying review.   |
-| 2 | contains | `password` — Plaintext password storage          | Storing passwords without hashing (bcrypt/argon2) violates basic security. Must be caught in review.    |
-| 3 | contains | `validation` — Missing input validation          | Accepting raw user input without validation enables injection, type confusion, and data corruption.     |
-| 4 | contains | `role` — Privilege escalation via mass assignment| Users should not be able to set their own role. This is a privilege escalation vector.                  |
-| 5 | contains | `error` — Missing error handling                 | No try/catch means unhandled promise rejections crash the server or leak stack traces to clients.       |
-
-### What a passing response looks like
-
-- Identifies SQL injection as the highest-severity blocking issue.
-- Recommends parameterized queries with specific code example.
-- Flags plaintext password storage and recommends bcrypt or argon2.
-- Points out that `role` should not come from user input (default to a safe role, require admin action to elevate).
-- Notes missing input validation and recommends a validation library (zod, joi, express-validator).
-- Flags missing error handling and recommends try/catch with appropriate status codes.
-- Organizes feedback by severity: blocking issues first, then suggestions, then nits.
-
----
-
-## Eval 2: Review Prioritization
-
-This eval tests whether the skill can triage multiple PRs by risk and recommend appropriate review depth for each.
-
-### Assertions
-
-| # | Type     | What it checks                                  | Why it matters                                                                                          |
-|---|----------|-------------------------------------------------|---------------------------------------------------------------------------------------------------------|
-| 1 | contains | `payment` — High priority for financial code     | Payment processing errors have direct financial impact. Must be reviewed thoroughly.                    |
-| 2 | contains | `delete` — High risk for destructive operations  | Account deletion + auth changes create security and data loss risk. Requires deep review.              |
-| 3 | contains | `README` — Low priority for documentation typo   | A typo fix requires only a quick glance. Spending review time here is waste.                           |
-| 4 | contains | `lock file` — Context-aware diff assessment      | Lock file changes inflate the diff but carry low risk. The real review is the package.json change.     |
-| 5 | contains | `order` — Explicit prioritization                | Must provide a concrete review order, not just describe each PR in isolation.                          |
-
-### What a passing response looks like
-
-- Ranks PR #203 (account deletion) and PR #201 (payment webhook) as top priorities.
-- Recommends deep review for #203 (auth + destructive operation) and #201 (financial transactions).
-- Recommends a focused review for #205 (verify no behavior change despite "refactor" claim).
-- Recommends a light review for #204 (check package.json diff only, skip lock file).
-- Recommends instant approval for #202 (README typo, no code risk).
-- Provides a concrete review order: #203 -> #201 -> #205 -> #204 -> #202.

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/code-review/evals/evals.json

@@ -1,66 +0,0 @@
-[
-  {
-    "name": "review-feedback-quality",
-    "description": "Validate that the skill produces specific, actionable code review feedback organized by severity with the WHAT-WHY-HOW pattern.",
-    "input": "Review this Express.js endpoint:\n\n```javascript\napp.post('/api/users', async (req, res) => {\n  const { email, password, role } = req.body;\n  const user = await db.query(`INSERT INTO users (email, password, role) VALUES ('${email}', '${password}', '${role}') RETURNING *`);\n  res.json(user.rows[0]);\n});\n```\n\nThe PR description says: 'Add user registration endpoint. Closes #45.'",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "SQL injection",
-        "description": "Must identify the SQL injection vulnerability from string interpolation in the query, as this is a critical security issue."
-      },
-      {
-        "type": "contains",
-        "value": "password",
-        "description": "Must flag that the password is stored in plaintext without hashing, which is a critical security violation."
-      },
-      {
-        "type": "contains",
-        "value": "validation",
-        "description": "Must note the absence of input validation on email, password, and role fields before database insertion."
-      },
-      {
-        "type": "contains",
-        "value": "role",
-        "description": "Must identify the mass assignment / privilege escalation risk of accepting role directly from user input."
-      },
-      {
-        "type": "contains",
-        "value": "error",
-        "description": "Must flag the missing error handling — no try/catch means database errors will crash the server or leak stack traces."
-      }
-    ]
-  },
-  {
-    "name": "review-prioritization",
-    "description": "Validate that the skill correctly prioritizes review effort based on change risk and provides appropriately calibrated depth.",
-    "input": "I have 5 PRs to review today and limited time. Help me prioritize and decide review depth for each:\n\n1. PR #201: Updates the payment processing webhook handler to support a new payment provider. 150 lines changed in the billing service.\n2. PR #202: Fixes a typo in the README. 1 line changed.\n3. PR #203: Adds a new admin-only API endpoint that deletes user accounts. 80 lines, touches auth middleware.\n4. PR #204: Upgrades React from 18.2 to 18.3. 500 lines changed (mostly lock file). No code changes.\n5. PR #205: Refactors the logging module to use structured JSON output. 200 lines changed, no behavior change per the description.",
-    "assertions": [
-      {
-        "type": "contains",
-        "value": "payment",
-        "description": "Must identify the payment webhook PR as high priority since it handles financial transactions and has high business impact."
-      },
-      {
-        "type": "contains",
-        "value": "delete",
-        "description": "Must flag the account deletion endpoint as high risk due to destructive operations and auth surface area."
-      },
-      {
-        "type": "contains",
-        "value": "README",
-        "description": "Must identify the README typo fix as lowest priority requiring minimal review depth."
-      },
-      {
-        "type": "contains",
-        "value": "lock file",
-        "description": "Must recognize that most of the React upgrade diff is lock file changes and calibrate depth accordingly."
-      },
-      {
-        "type": "contains",
-        "value": "order",
-        "description": "Must provide a recommended review order that puts high-risk PRs first and low-risk PRs last."
-      }
-    ]
-  }
-]

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/csharp-best-practices/evals/assertions.md

@@ -1,35 +0,0 @@
-# C# Best Practices — Eval Assertions
-
-## Eval 1: Minimal API with Records
-
-Tests whether the agent produces idiomatic .NET 9 minimal API design with modern C# features.
-
-### Assertions
-
-1. **record-types-for-dtos** — Request and response models use `record` (e.g., `public record CreateOrderRequest(string CustomerName, List<OrderLineItem> Items)`) or `record struct` for small value types. Properties should use `required` where initialization is mandatory. Reject plain `class` DTOs with `{ get; set; }` — these lack value equality and encourage mutation.
-
-2. **typed-results-on-endpoints** — Endpoint methods return `Results<T1, T2, ...>` with specific `TypedResults` variants (e.g., `TypedResults.Ok(response)`, `TypedResults.NotFound()`, `TypedResults.ValidationProblem(errors)`). This provides OpenAPI schema generation without manual attributes and compile-time verification of all return paths. Reject endpoints returning `IResult` or `object`.
-
-3. **di-lifetime-correctness** — The service registration code uses `builder.Services.AddScoped<IOrderRepository, OrderRepository>()` for per-request database access, `AddTransient` for stateless operations, and `AddSingleton` for shared, thread-safe caches. Reject code that registers a scoped service and injects it into a singleton, which would cause a captive dependency (stale DbContext, connection leaks).
-
-4. **nullable-annotations** — The project enables `<Nullable>enable</Nullable>`. Optional response fields use `string?`, required fields use `string` (non-nullable). Pattern matching with null checks (`if (order is null)`) or `[NotNullWhen]` attributes on try-methods preserves compiler flow analysis. Reject code that uses `!` (null-forgiving) liberally without justification.
-
-5. **cancellation-token-propagation** — Every async endpoint handler has `CancellationToken cancellationToken` as a parameter (minimal API binds it automatically from the request). The token is passed through `await _repository.GetByIdAsync(id, cancellationToken)` and all downstream calls. Reject code that uses `default` or omits the token from inner calls.
-
----
-
-## Eval 2: Async Stream Processing
-
-Tests whether the agent builds a correct IAsyncEnumerable pipeline with cancellation and error handling.
-
-### Assertions
-
-1. **iasyncenumerable-pipeline** — Each pipeline stage is a method or extension method with signature like `IAsyncEnumerable<TOut> Transform(this IAsyncEnumerable<TIn> source, ...)`. Stages compose by chaining: `source.Filter(...).Enrich(...).Batch(100).WriteTo(sink)`. Reject implementations that buffer everything into a `List<T>` before processing — that defeats the streaming benefit.
-
-2. **cancellation-with-enumerator-token** — Methods use `[EnumeratorCancellation] CancellationToken ct = default` on the generator parameter and pass it to all awaited operations inside the `await foreach` or `yield` loop. When the consumer stops enumerating (e.g., due to cancellation), the producer's finally block runs for cleanup. Reject code that ignores cancellation in yield loops.
-
-3. **batching-with-backpressure** — The batching stage accumulates items into a `List<T>` or array up to a configured batch size, then yields the batch as `IReadOnlyList<T>`. The upstream producer naturally pauses when the downstream consumer has not yet requested the next batch (cooperative pull-based backpressure). Accept time-based flushing as a bonus. Reject fire-and-forget patterns with unbounded queues.
-
-4. **structured-error-handling** — Individual item failures are caught within the processing loop and handled (logged, counted, or sent to a dead-letter queue) without stopping the entire pipeline. Pipeline-level errors (connection loss, cancellation) propagate to the caller. Reject bare `catch (Exception)` that swallows all errors silently.
-
-5. **testable-stage-isolation** — Tests create in-memory async enumerables using `async IAsyncEnumerable<T>` generator methods or helper factories. Each stage is tested independently: filter stage is tested with known items and expected output count, batching stage is tested with exact batch boundary conditions (e.g., 10 items with batch size 3 yields batches of [3, 3, 3, 1]). Tests include cancellation scenarios.

package/workflows/workflows/agent-environment-setup/platforms/codex/skills/csharp-best-practices/evals/evals.json

@@ -1,64 +0,0 @@
-[
-  {
-    "id": "csharp-minimal-api-with-records",
-    "prompt": "Design a .NET 9 minimal API for an order management service. The API should use records for request/response DTOs, TypedResults for compile-time response contracts, dependency injection with proper lifetimes, and nullable reference types. Include endpoints for creating an order, getting an order by ID, and listing orders with pagination.",
-    "assertions": [
-      {
-        "id": "record-types-for-dtos",
-        "description": "Request and response types are declared as record or record struct (e.g., record CreateOrderRequest, record OrderResponse) with required properties and value equality semantics — not plain classes with mutable properties.",
-        "weight": 1.0
-      },
-      {
-        "id": "typed-results-on-endpoints",
-        "description": "Endpoint handler return types use TypedResults (e.g., Results<Ok<OrderResponse>, NotFound, ValidationProblem>) providing compile-time verification that all return paths match the declared result types.",
-        "weight": 1.0
-      },
-      {
-        "id": "di-lifetime-correctness",
-        "description": "Services are registered with appropriate lifetimes: repositories as Scoped (per-request), stateless validators as Transient, and shared caches as Singleton. No Scoped services injected into Singleton registrations.",
-        "weight": 1.0
-      },
-      {
-        "id": "nullable-annotations",
-        "description": "Nullable reference types are enabled. Optional fields use nullable types (string?) while required fields are non-nullable. Try-pattern methods use [NotNullWhen(true)] or similar attributes for flow analysis.",
-        "weight": 1.0
-      },
-      {
-        "id": "cancellation-token-propagation",
-        "description": "All async endpoint handlers accept CancellationToken as a parameter, and it is passed through to every async service and repository call in the chain.",
-        "weight": 1.0
-      }
-    ]
-  },
-  {
-    "id": "csharp-async-stream-processing",
-    "prompt": "Implement an async stream processing pipeline in C# 13 that reads events from a database cursor, transforms them through multiple stages (filtering, enrichment, batching), and writes results to an output sink. The pipeline should support cancellation, backpressure, and graceful error handling. Include unit tests for the pipeline stages.",
-    "assertions": [
-      {
-        "id": "iasyncenumerable-pipeline",
-        "description": "The pipeline uses IAsyncEnumerable<T> as the core abstraction, with each stage accepting and returning IAsyncEnumerable<T>. Stages are composed via extension methods or LINQ-style chaining, enabling lazy evaluation.",
-        "weight": 1.0
-      },
-      {
-        "id": "cancellation-with-enumerator-token",
-        "description": "IAsyncEnumerable methods accept [EnumeratorCancellation] CancellationToken parameters, and cancellation is checked or propagated within yield loops using Task.Delay, database reads, or explicit Task.checkCancellation checks.",
-        "weight": 1.0
-      },
-      {
-        "id": "batching-with-backpressure",
-        "description": "A batching stage collects items into fixed-size batches before yielding them downstream, implementing natural backpressure — the producer does not advance until the consumer processes the current batch.",
-        "weight": 1.0
-      },
-      {
-        "id": "structured-error-handling",
-        "description": "Error handling uses try-catch at stage boundaries with specific exception types or Result<T> patterns. Failed items are logged or routed to a dead-letter mechanism rather than crashing the entire pipeline.",
-        "weight": 1.0
-      },
-      {
-        "id": "testable-stage-isolation",
-        "description": "Unit tests verify individual pipeline stages in isolation by passing in-memory IAsyncEnumerable sources (e.g., from async generator methods) and asserting on collected output. Tests cover normal flow, empty input, cancellation, and error cases.",
-        "weight": 1.0
-      }
-    ]
-  }
-]