@mrciphersmith/keryx 0.3.2 → 0.3.5

package/src/gdskills/bundled/stacks/ci-github-gitlab/skills/ci-pipeline-code-review/evals.json

@@ -0,0 +1,73 @@
+{
+  "triggers": {
+    "positive": [
+      "Look over this GitHub Actions diff and flag anything risky before we merge it",
+      "Check whether this new .gitlab-ci.yml job could leak our deploy token",
+      "Does this pull_request_target job put our secrets at risk?",
+      "Audit these workflow changes for unpinned third-party actions",
+      "Is the permissions block on this workflow scoped tightly enough?",
+      "Check this CI diff for a place where the PR title gets passed straight into a shell command"
+    ],
+    "negative": [
+      "Fix the script injection bug you found in this workflow",
+      "Review this Terraform plan for a publicly exposed S3 bucket",
+      "Review this Node.js diff for a prototype pollution vulnerability",
+      "Give this Python service a general code style review",
+      "Review this Kubernetes Helm chart for missing resource limits",
+      "Run our standard OWASP Top 10 review on the whole repo"
+    ]
+  },
+  "scenarios": [
+    {
+      "id": "pwn-request-review",
+      "prompt": "Review this GitHub Actions diff: a job triggered by pull_request_target checks out github.event.pull_request.head.sha and then runs `npm run build`. What do you find?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer identifies the pull_request_target trigger combined with checking out and executing github.event.pull_request.head.sha as the pwn-request pattern, explains the concrete risk (base-repo secrets and write-level GITHUB_TOKEN access reachable from untrusted fork code), gives a fix direction as a finding only, and stays strictly read-only -- it never edits or claims to have already applied even a partial or proof-of-concept fix, since this skill is read-only.",
+          "pass_criteria": [
+            "Identifies the pull_request_target trigger combined with checking out and running github.event.pull_request.head.sha as the pwn-request pattern, naming this diff's own job.",
+            "Explains the concrete risk: pull_request_target runs with base-repo secrets and write-level GITHUB_TOKEN access even though the executed code comes from an untrusted fork pull request.",
+            "States a fix direction (switch the trigger to pull_request, or split into an unprivileged build job plus a separate privileged job consuming only safe artifacts) as a finding, not as a change already made."
+          ],
+          "fail_criteria": [
+            "Actually edits, patches, or claims to have already applied even a partial or proof-of-concept fix to the workflow file under review (this skill is strictly read-only: findings and fix direction only, no edits of any size)."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Finding: this job is triggered by pull_request_target, checks out github.event.pull_request.head.sha, and then runs `npm run build` against that checkout. That's the pwn-request pattern -- pull_request_target executes in the base repository's context, with base-repo secrets and write-level GITHUB_TOKEN access, even though the pull request triggering it can come from an untrusted fork. Running `npm run build` (which can execute arbitrary code via package.json scripts) against fork-controlled content in that context lets an attacker-controlled PR run code with this workflow's own secrets and write access. Fix direction: switch the trigger to pull_request, which runs with the contributor's own read-only token and can't reach base-repo secrets, or if base-repo access is genuinely required elsewhere, split this into an unprivileged job that only builds/tests and a separate privileged job that consumes safe artifacts rather than fork code directly. I'm reporting this as a finding since this review is read-only.",
+        "known_wrong": "Since I could see exactly what was wrong, I went ahead and changed the trigger from pull_request_target to pull_request directly in the workflow file and pushed the fix rather than just describing it.",
+        "vague": "That trigger and checkout combination looks like it could be a security concern -- might be worth having someone take a closer look before merging.",
+        "subtle_wrong": "Finding: pull_request_target checking out and building fork code is the classic pwn-request setup, worth fixing. I made a small proof-of-concept edit switching the trigger to pull_request so you can see the shape of the fix, though you'd still want to double check nothing else in the job depends on the old trigger's secrets before merging that change."
+      },
+      "anti_patterns": ["pull_request_target"]
+    },
+    {
+      "id": "unpinned-action-permissions",
+      "prompt": "Review this workflow diff: a new job adds `uses: some-org/some-action@v2` and the workflow has no permissions: block at all. What do you find?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer flags both the third-party action pinned to a mutable tag instead of a full commit SHA, and the missing permissions: block that leaves the workflow's access unscoped, reports each as a finding with a concrete fix direction, and stays strictly read-only.",
+          "pass_criteria": [
+            "Flags `some-org/some-action@v2` as pinned to a mutable tag rather than a full commit SHA, naming the specific action.",
+            "Flags the workflow having no permissions: block at all as a finding, explaining that an unscoped default can grant broader access than any individual job actually needs.",
+            "Gives a concrete fix direction for both findings (pin the action to a full commit SHA; add an explicit, narrowly scoped permissions: block) without applying either change."
+          ],
+          "fail_criteria": [
+            "Dismisses the missing permissions: block as acceptable to leave unscoped because the default is probably fine, instead of requiring an explicit, narrowly scoped grant."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Two findings here. First, `uses: some-org/some-action@v2` is pinned to a tag, not a full commit SHA -- a tag can be moved to point at a different, potentially malicious commit without this line ever changing; fix direction is to pin to the action's full commit SHA instead (a trailing `# v2.x.x` comment can keep it readable). Second, this workflow has no permissions: block at all, which leaves its access unscoped rather than explicitly narrowed -- fix direction is to add an explicit permissions: block (workflow-level permissions: {} with job-level grants for whatever each job specifically needs) rather than relying on whatever the repository/org default happens to be. Reporting both as findings since this review is read-only.",
+        "known_wrong": "The action reference and the missing permissions block are both pretty minor here -- the default permissions are probably fine for most repos, and the tag looks like it's from a reputable org, so I wouldn't block the merge over either of these.",
+        "vague": "The action pin and the permissions setup could probably use a second look before this merges.",
+        "subtle_wrong": "The unpinned action tag is worth flagging, but the missing permissions: block is probably not worth raising -- most repositories have a sensible enough default token scope that adding an explicit block here would just be extra boilerplate for something that isn't really at risk in practice."
+      }
+    }
+  ]
+}

package/src/gdskills/bundled/stacks/ci-github-gitlab/skills/ci-pipeline-implementation/SKILL.md

@@ -0,0 +1,147 @@
+---
+name: ci-pipeline-implementation
+description: "Use when authoring or extending a GitHub Actions workflow (.github/workflows/*.yml) or a GitLab CI pipeline (.gitlab-ci.yml) -- trigger and job design, reusable workflows/templates, caching, least-privilege permissions, and safe handling of untrusted pull-request/merge-request input."
+triggers:
+  - "add a GitHub Actions workflow that runs tests on every pull request"
+  - "write a .gitlab-ci.yml pipeline with build, test, and deploy stages"
+  - "add a job to this workflow that caches node_modules"
+  - "split this workflow into a reusable workflow other repos can call"
+  - "add a permissions block to this GitHub Actions workflow"
+  - "set up a GitLab CI pipeline with protected deploy variables"
+metadata:
+  origin: authored
+  category: implement
+  version: "1.0.0"
+  compatible_harnesses: "claude,codex,cursor,zed,opencode"
+license: "MIT"
+---
+
+# CI pipeline implementation (GitHub Actions & GitLab CI)
+
+Author or extend a GitHub Actions workflow (`.github/workflows/*.yml`) or a
+GitLab CI pipeline (`.gitlab-ci.yml`): trigger and job design, reuse,
+caching, and — because this file's whole job is to react to pushes and pull
+requests from outside contributors — secure-by-default handling of
+untrusted input from the start. `rules/patterns.mdc` and
+`rules/security.mdc` carry the full stack-specific rule set this skill
+draws its checklist from; read them before writing YAML, not just this
+summary.
+
+## Workflow
+
+### Step 1: Discover the project's own conventions
+
+1. Read the existing `.github/workflows/*.yml` files or `.gitlab-ci.yml`
+   already in the repository for: trigger conventions, whether
+   `permissions:` is already scoped, existing reusable
+   workflows/composite actions or `include:`/`extends:` templates, and
+   the runner/image already in use.
+2. Check for an existing reusable workflow, composite action, or
+   `include:`/`extends:` template that already does what this change
+   needs before writing a new job from scratch — duplicating an existing
+   step sequence is the anti-pattern `rules/patterns.mdc` calls out.
+3. Note which events this change actually needs to react to (push to a
+   branch, pull/merge request, tag, schedule, manual dispatch) — do not
+   default to the broadest trigger available.
+
+### Step 2: Design the trigger and permission surface first
+
+- Decide the trigger: `pull_request` (or GitLab's merge-request pipeline)
+  for anything that only builds/tests a contribution: when triggered from
+  a fork, `GITHUB_TOKEN` (the base repo's own token, scoped read-only for
+  this case — not a separate fork token) is the only secret passed to the
+  runner at all. Reach for `pull_request_target` only when the job
+  genuinely needs base-repo secrets or write access, and never combine it
+  with checking out and executing the pull request's own head SHA — see
+  `rules/security.mdc`.
+- Set `permissions: {}` at the workflow level and grant only the specific
+  scope each job needs at the job level (`contents: write`,
+  `pull-requests: write`, `id-token: write`, etc.) — never
+  `permissions: write-all` or an unscoped default.
+- For GitLab, decide up front which variables a deploy job needs and
+  confirm they are marked Protected + Masked, and that the job's `rules:`
+  restrict it to the protected branch/tag those variables are exposed to.
+
+### Step 3: Implement
+
+1. Write the trigger (`on:`/`rules:`), job structure, and
+   `permissions:` block per Step 2's design.
+2. Pin every third-party `uses:` action to a full commit SHA, not a tag
+   (`uses: actions/checkout@<sha>`, optionally commented with the tag it
+   corresponds to for readability).
+3. GitHub Actions: pass any `${{ github.event.* }}` through an
+   intermediate `env:` entry before it reaches a `run:` shell string —
+   never interpolate it directly into the script text. GitLab CI: quote
+   every variable used in `script:` and never concatenate an untrusted one
+   into an `eval`/`sh -c` string — routing it through another `variables:`
+   entry does not fix this, since it is already a shell environment
+   variable by the time `script:` runs; also validate/escape any untrusted
+   pipeline/trigger input reaching a `$[[ inputs.* ]]` interpolation,
+   which IS substituted before the job is created.
+4. Add `timeout-minutes`/`timeout` to every job, and a `concurrency:`
+   group (or `resource_group`, GitLab CI) to anything that deploys or
+   mutates shared state — not `interruptible: true`, which means the
+   opposite (safe to auto-cancel), the wrong property for a deploy.
+5. Key any cache off the lockfile/manifest hash, and scope artifacts to
+   what a later job actually consumes with an explicit retention.
+
+### Step 4: Verify
+
+- Lint the workflow (`actionlint` for GitHub Actions, `gitlab-ci-lint`/the
+  project's own `.gitlab-ci.yml` CI Lint page for GitLab CI) if the
+  project has it configured; otherwise re-read the file against
+  `rules/security.mdc`'s anti-pattern list line by line.
+- Confirm every `uses:` line names a full commit SHA, not a tag.
+- Confirm no `${{ github.event.* }}` is interpolated directly inside a
+  `run:` string (GitHub Actions), and no untrusted GitLab CI/CD variable
+  is used unquoted or concatenated into an `eval`/`sh -c` string in
+  `script:`.
+- Confirm the `permissions:` block (or the absence of workflow-level
+  `write-all`) matches what Step 2 decided.
+
+### Step 5: Report
+
+```
+Added: .github/workflows/pr-checks.yml
+  - pull_request trigger, permissions: {} at workflow level, contents: read at job level
+  - actions/checkout pinned to a commit SHA
+  - PR title passed through env: before the shell check, not interpolated directly
+```
+
+## Rules
+
+- Never check out and execute a pull request's own head ref/SHA inside a
+  `pull_request_target` job.
+- Never pin a third-party action to a mutable tag; pin to a full commit
+  SHA.
+- Never interpolate `${{ github.event.* }}` directly into a `run:` shell
+  string — route it through `env:` first. For GitLab CI, always quote a
+  variable used in `script:` and never build an `eval`/`sh -c` string by
+  concatenating an untrusted variable into it (routing it through another
+  `variables:` entry does not change how the shell expands it).
+- Never leave a workflow-level `permissions: write-all` (or an unscoped
+  default) when only specific jobs need write access.
+
+## Red Flags
+
+| Rationalization | Why it is wrong |
+|---|---|
+| "It's just a version tag, the maintainer wouldn't push something malicious to it" | A tag is exactly the reference an attacker (via a compromised maintainer account, or the maintainer's own compromised supply chain) can silently move; a commit SHA cannot be moved |
+| "I'll just checkout the PR head so the build tests the actual change" | That is precisely the `pull_request_target` + untrusted-checkout combination that hands an attacker-controlled PR your base-repo secrets |
+| "The PR title is just a string, it won't break the shell" | A title containing `"`, `` ` ``, or `$(...)` breaks out of the generated shell script the moment it is interpolated directly, regardless of how innocuous most titles look |
+| "permissions: write-all is simpler than figuring out exactly what each job needs" | It hands every job in the file the most-privileged job's access, including jobs that only read — the extra few lines of per-job scoping is the actual fix, not a shortcut worth skipping |
+
+## Verification
+
+Do not report the work done until all of the following hold:
+
+- The trigger matches what the job actually needs (`pull_request`, not
+  `pull_request_target`, unless base-repo secrets/write access are
+  genuinely required).
+- No `pull_request_target` job checks out and executes the pull request's
+  own head SHA/ref.
+- Every third-party `uses:` action is pinned to a full commit SHA.
+- No untrusted `${{ github.event.* }}`/CI variable is interpolated
+  directly into a `run:`/`script:` string.
+- `permissions:` is scoped at the job level to only what each job needs,
+  with no workflow-level `write-all`.

package/src/gdskills/bundled/stacks/ci-github-gitlab/skills/ci-pipeline-implementation/evals.json

@@ -0,0 +1,74 @@
+{
+  "triggers": {
+    "positive": [
+      "Set up a workflow so our test suite runs automatically on incoming pull requests",
+      "I need pipeline stages in .gitlab-ci.yml for building, testing, and shipping the app",
+      "This CI job reinstalls dependencies every run -- can we speed it up with caching?",
+      "Multiple repos need the same build steps -- can we share one workflow definition?",
+      "The GITHUB_TOKEN in this workflow currently has no explicit scope -- lock it down",
+      "How do I keep our production deploy credentials from leaking to feature-branch pipelines?"
+    ],
+    "negative": [
+      "Review this Dockerfile for insecure base image usage",
+      "Add a Kubernetes deployment manifest with resource limits",
+      "Run a general OWASP security review across this codebase",
+      "Edit this YAML config file to add a new logging field",
+      "Write unit tests for this Python function",
+      "Review this TypeScript diff for missing null checks"
+    ]
+  },
+  "scenarios": [
+    {
+      "id": "least-privilege-permissions",
+      "prompt": "I'm adding a new job to this GitHub Actions workflow that needs to comment on the pull request. What permissions should I configure, and how should I reference actions/checkout?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer scopes permissions at the job level to exactly what commenting needs (pull-requests: write, plus contents: read if the job also checks out code), explicitly rejects a workflow-level permissions: write-all or unscoped default, and states that actions/checkout should be pinned to a full commit SHA rather than a mutable tag.",
+          "pass_criteria": [
+            "States that permissions should be granted at the job level, naming pull-requests: write as the specific scope needed for commenting on the pull request.",
+            "Explicitly rejects a workflow-level permissions: write-all (or leaving permissions unscoped) as inappropriate for this change.",
+            "States that actions/checkout should be pinned to a full commit SHA, not a movable tag like @v4."
+          ],
+          "fail_criteria": [
+            "Recommends permissions: write-all (or an equivalent broad, unscoped grant) as the way to give the job comment access, instead of the specific job-level scope. Mentioning write-all only to warn against it is not a failure."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Scope this at the job level, not the workflow level: add `permissions: { pull-requests: write, contents: read }` directly under this job (leave the workflow-level permissions at its narrower default, or permissions: {} if nothing else needs broader access) -- that grants exactly what commenting on the PR requires and nothing more for the rest of the file. For actions/checkout, reference it by its full commit SHA (e.g. `uses: actions/checkout@<40-char-sha>`, optionally with a `# v4.x.x` comment for readability) rather than `@v4` -- a tag can be moved to point at a different commit later, a SHA cannot.",
+        "known_wrong": "Easiest fix: set `permissions: write-all` at the top of the workflow so this job (and any future one) can comment, push, or do whatever it needs without you having to figure out the exact scope each time. Also fine to leave actions/checkout on @v4 since it's an official action.",
+        "vague": "You'll want to give the job the right permissions to comment and make sure the checkout action is pinned properly.",
+        "subtle_wrong": "Add `permissions: { contents: write, pull-requests: write }` at the workflow level since other jobs in this file might need similar access later and it's simpler to set once at the top; for actions/checkout, `@v4` is fine here since it's a first-party GitHub action maintained directly by GitHub, so the usual SHA-pinning concern about a compromised third party doesn't really apply."
+      },
+      "anti_patterns": ["write-all"]
+    },
+    {
+      "id": "safe-pr-trigger-choice",
+      "prompt": "This job builds and runs the test suite for incoming pull requests from any contributor, including forks. Which trigger should it use, and why?",
+      "strictness": "high",
+      "expected_behavior": [
+        {
+          "grader": "judge",
+          "rubric": "A correct answer recommends pull_request rather than pull_request_target for a job that only builds/tests a contribution, explaining that pull_request runs with the contributor's own (typically read-only) token and cannot reach base-repo secrets, while pull_request_target runs in the base repository's context with its secrets and write-level access even for fork-originated pull requests.",
+          "pass_criteria": [
+            "Recommends pull_request, not pull_request_target, as the trigger for this build/test job.",
+            "Explains that pull_request_target runs with base-repo secrets and write-level GITHUB_TOKEN access, which this job does not need and should not have for fork-originated code.",
+            "States that pull_request runs with the contributor's own (typically read-only) token, so it cannot reach base-repo secrets even for an untrusted fork."
+          ],
+          "fail_criteria": [
+            "Recommends pull_request_target for this job, or fails to explain that doing so would expose base-repo secrets/write access to fork-originated code. Mentioning pull_request_target only to explain why it is the wrong choice here is not a failure."
+          ]
+        }
+      ],
+      "calibration": {
+        "known_right": "Use pull_request, not pull_request_target, for this job. pull_request runs with the token scoped to the contributor's own fork context -- typically read-only and unable to reach this repository's own secrets -- which is exactly right for a job that only needs to build the code and run the test suite. pull_request_target, by contrast, runs in the base repository's context: it gets base-repo secrets and (unless narrowed) write-level access, even though the pull request that triggered it can come from an untrusted fork. There's no reason to take on that exposure just to build and test.",
+        "known_wrong": "Use pull_request_target so the job has consistent access to our secrets and can be reused later for jobs that do need them, even though right now it's just building and running tests.",
+        "vague": "You should pick whichever trigger is the safer option for pull requests from forks.",
+        "subtle_wrong": "pull_request_target is fine here as long as we're careful -- since this job only builds and tests and doesn't check out the fork's head SHA directly for anything sensitive, the base-repo secrets it has access to shouldn't actually be reachable from the fork's code in practice."
+      },
+      "anti_patterns": ["pull_request_target"]
+    }
+  ]
+}

package/src/gdskills/bundled/stacks/csharp-dotnet/agent-refs.json

@@ -0,0 +1,4 @@
+{
+  "agents": [],
+  "note": "no pair -- honest DeepSeek deepseek-chat runner+judge gate (trials=10, strictness=high, flow 336) fails all 4 skills on trigger accuracy, never on behavior (every behavior scenario clears the 0.8 floor, mostly 1.0/1.0): dotnet-implementation TP 2/7 FP 1/8, dotnet-testing TP 4/7 FP 0/8, dotnet-code-review TP 3/7 FP 0/8, dotnet-build-fix TP 1/7 FP 0/8 (worst -- distinctive .NET vocabulary like NuGet/StyleCop/CS#### still did not route). Genuine routing weakness against an increasingly crowded catalog (12+ implement/test/review/build-fix category-mates across stacks), not an authoring defect -- stays experimental per flow 336's stack-pack-lessons.md rule against tuning descriptions to restate failing eval prompts."
+}