diff-sentry 0.7.0__tar.gz → 0.7.1__tar.gz

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/.claude/rules/detect-setup.md

@@ -11,9 +11,10 @@ paths:
 # Model roles, auth, and run failures
 
 - **Failures name themselves.** With `max_retries=1` (set in `detect.setup`), a non-retryable LM error (auth / billing / configuration / unsupported model) escapes after one attempt as the raw `dspy.LMError` subclass. An `RLMTaskError` means the single attempt ended in an adapter parse failure, an invalid result, or a retryable endpoint fault (timeout / 5xx / transport). `cli.run` turns both into a `status=failed` response. Check the endpoint before suspecting the schema.
-- **An empty answer at the cap is a planner that thought too long, not one that cannot follow the format.** A reasoning planner whose call ends `finish_reason=length` with empty content fails the run with `AdapterParseError` (the "LM Response" quoted is its promoted reasoning, or `{}`), which reads like a format failure; the raw call's `finish_reason` tells them apart. The fix is a thinking budget below `DS_PLANNER_MAX_TOKENS` through `DS_ROOT_LM_KWARGS`, never a whole-run retry or a different adapter.
+- **An empty answer at the cap is a planner that thought too long, not one that cannot follow the format.** A reasoning planner whose call ends `finish_reason=length` with empty content fails the run with `AdapterParseError`. Its message starts "This reply carried no content" when the reply quoted is the promoted reasoning channel, and quotes any other reply as the model wrote it; the raw call's `finish_reason` tells a cut at the cap from a reply that ignored the format. The fix is a thinking budget below `DS_PLANNER_MAX_TOKENS` through `DS_ROOT_LM_KWARGS`, never a whole-run retry or a different adapter.
 - **`DS_ROOT_LM_KWARGS` / `DS_SUB_LM_KWARGS` are aliases of rlm-harness's `RLM_MAIN_LM_KWARGS` / `RLM_SUB_LM_KWARGS`.** `setup` builds the kit's config itself and never calls the kit's `from_env`, the only reader of those two; `DetectConfig.from_env` reads both names and refuses them set to different values. It also refuses the keys the kit owns (model, endpoint, timeout), so a bad value stops the process rather than failing every run. The analyst's parameters never reach the classifier, which builds its own request.
-- **`DS_CLASSIFIER_LM_KWARGS` is the classifier's own**, over its `temperature=0.1`. `from_env` refuses the keys its settings own (model, messages, stream, n, timeout, max_tokens, max_completion_tokens) and, when the `classify` extra is installed, any key the OpenAI client does not take: that call would fail every time as an endpoint error, which the circuit breaker never counts.
+- **`DS_CLASSIFIER_LM_KWARGS` is the classifier's own**, over its `temperature=0.1`. `from_env` refuses the keys its settings own (model, messages, stream, n, timeout, max_tokens, max_completion_tokens) and, when the `classify` extra is installed, any key the OpenAI client does not take, so a bad value stops the process rather than failing every call.
 - **A classifier reply cut at `max_tokens` is an unusable reply that names the cut** (`deep_classify._CutReply`, read by the validator), not an endpoint error. The same request is cut the same way, so it is not re-sent, and a classifier that keeps thinking through its budget trips the circuit breaker. A reply cut after a whole verdict still counts. The tool_call records `cut: true`. The fix is a thinking budget in `DS_CLASSIFIER_LM_KWARGS` (raising `DS_CLASSIFIER_MAX_TOKENS` alone can turn a slow model's cut into a timeout).
+- **An error status that repeats is not re-sent** (`deep_classify.is_fast_fail_http`, passed to `make_model_tool` as `is_fast_fail`): a 400, 401, 403, 404 or 422 from the classifier's endpoint is an endpoint error after one call, and it counts toward the circuit breaker, so a wrong model name, key or request stops being called. A timeout, a transport error, a 408, 409, 429 or 5xx keeps the transient retry and never counts. The eval judge holds its own copy without 400 and 422: its breaker spans a whole batch, and a status one oversized change earns there would leave the later runs unscored.
 - **Subscription auth is a lazy, opt-in branch.** A role whose model is `claude-agent-sdk/<id>` runs on rlm-harness's `ClaudeAgentLM`, imported only inside `detect._maybe_subscription_lm` and never from `__init__.py`, so `import diff_sentry` stays dspy-free and a proxy-only install never touches the SDK.
 - **The classifier never runs on the subscription.** It is a separate OpenAI-compatible client (`deep_classify._selfclassify_chat`), so `config.from_env` rejects a `claude-agent-sdk/` classifier model, whether set explicitly or inherited from `DS_SUB_LM`. Mixed auth across roles is by design.

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/.claude/rules/indicators.md

@@ -44,7 +44,7 @@ paths:
 
 ## `curl-pipe-shell`
 
-A download piped into a shell or an interpreter is `critical`. It has three forms that are judged separately: the pipe, the command substitution and PowerShell's. In prose (a `title`, `body` or `comment`, or a documentation file: `.md .mdx .markdown .rst .adoc .ipynb .html .htm`) a documented installer is `medium`. Code files (`.sh`, `.go`, `.yml`, a Dockerfile, `requirements.txt`) and branches are never softened. The segment kind decides, never a label. The planner's `scan_indicators` applies the same softening to the region it is given.
+A download piped into a shell or an interpreter is `critical`. It has three forms that are judged separately: the pipe, the command substitution and PowerShell's. In prose (a `title`, `body` or `comment`, or a documentation file: `.md .mdx .markdown .rst .adoc .ipynb .html .htm`) a documented installer is `medium`. Code files (`.sh`, `.go`, `.yml`, a Dockerfile, `requirements.txt`) and branches are never softened. The segment kind decides, never a label. The planner's `scan_indicators` applies the same softening to the region it is given. A region with added lines and a `- ` line but no hunk or file header could be a patch slice or a list, so it is not softened, unless every line of it is a whole line, diff marker and all, of a documentation file's patch (`_Change.patch_of`): it is then read as the baseline reads that file.
 
 ### What it matches
 

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/.claude/rules/rubric-and-eval.md

@@ -14,6 +14,7 @@ paths:
 
 - `rl_export.rubric_signal` attaches the ATLAS 4-category (TF/TA/TG/PA) decomposition as LABELS: a fixed, no-LLM skeleton in `run_start` meta (`rubric_to_meta(default_rubric())`, one criterion per category, since the task is constant; there is no `generate_rubric` path) plus deterministic per-criterion `criteria_facts`.
 - The facts are a re-lens over `run_labels` / `run_metrics`, not a second derivation: `rubric.trace_facts` reuses them, so a criterion's `observed` cannot drift from what a trainer reads. `_CATEGORY_LENS` maps TF↔(verdict / signal / hit_iteration_cap), TA↔(scan / deep_classify / analyst / fetch / skill counts + circuit-breaks + analyst failures), TG↔(indicator_count / max_indicator_severity / signal / cited_unknown), PA↔(verdict / cited_unknown).
+- `deep_classify_circuit_breaks` counts the tool_calls the breaker short-circuited. From rlm-harness 1.22.0 a repeating error status (`is_fast_fail_http`) counts toward the breaker as an unusable reply does, so a break in a newer trace can mean a misconfigured classifier endpoint as well as a thrashing planner; the status in the run's endpoint-error tool_calls (`Error code: 401`) tells them apart, since a 429, a 5xx or a timeout records one too and does not count.
 - `CriterionFact` has no score/met field; the trainer scores. Never add a reward/score/met field to any rubric type.
 - `rubric.py` is a dspy-free leaf: it imports only `.schema` at top, and its `rl_export` reuse is a function-level import (that path is dspy-free too), so `response.py` can import it at top.
 - `response._rubric` surfaces it as `DetectionResponse.rubric` (`RubricReport`); the studio shows it as a card tagged `labels — not a score` that adds no judgement. Keep the card legacy-guarded: a response without a `rubric` renders nothing.

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/.github/workflows/ci.yml

@@ -1,6 +1,6 @@
 name: CI
 
-# Offline suite only: no live LLM, no network calls in tests, no Deno sandbox — dspy-bearing paths use
+# Offline suite only: no live LLM, no network calls in tests, no Deno sandbox: dspy-bearing paths use
 # DummyLM / rlm-harness's ScriptedInterpreter, transports are injected fakes, and the corpus pins the
 # indicator suite's hit/miss behavior (see AGENTS.md "Verify"). `uv sync` resolves rlm-harness from PyPI
 # at the exact version pinned in pyproject.toml / uv.lock.
@@ -9,7 +9,7 @@ on:
     branches: [main]
   pull_request:
 
-# Least privilege: this workflow only checks out code and runs tests/lint — it never writes. Specifying
+# Least privilege: this workflow only checks out code and runs tests/lint; it never writes. Specifying
 # `permissions:` also drops every unlisted scope to `none`, so a compromised dep/action on an untrusted
 # fork PR can't use the token to push, tag, or open issues.
 permissions:
@@ -62,7 +62,7 @@ jobs:
           for f in studio/tests/*.test.js; do echo "▸ $f"; node "$f"; done
 
   # Dogfood: this repo's own CI runs the ACTION this repo publishes (`uses: ./`), not a copy of it.
-  # So every PR here is also an integration test of the thing users install — a drifted action.yml
+  # So every PR here is also an integration test of the thing users install: a drifted action.yml
   # fails our own build before it reaches anyone's Marketplace install.
   #
   # DELIBERATELY the deterministic scan and not the full `classify` pipeline. `classify` needs DS_*

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/.github/workflows/release.yml

@@ -1,6 +1,6 @@
 name: Release
 
-# Publish to PyPI via Trusted Publishing (OIDC) — NO API token. INERT until a GitHub Release is
+# Publish to PyPI via Trusted Publishing (OIDC), with NO API token. INERT until a GitHub Release is
 # published, and needs one-time setup before the first release:
 #   1. On PyPI, add a Trusted Publisher for project `diff-sentry`:
 #        owner = qazbnm456, repo = diff-sentry, workflow = release.yml, environment = pypi
@@ -11,7 +11,7 @@ name: Release
 # consumers pin in `uses: qazbnm456/diff-sentry@vX`. So cutting a release ships the package AND the
 # Action; keep the README's quickstart ref in step with the tag you publish.
 #
-# Supply-chain: the two third-party actions are SHA-pinned — setup-uv, and (highest blast radius, it
+# Supply-chain: the two third-party actions are SHA-pinned: setup-uv, and (highest blast radius, it
 # uploads to PyPI) pypi-publish. Bump a SHA and its trailing version comment together. GitHub-owned
 # checkout/*-artifact stay on major tags.
 
@@ -34,7 +34,7 @@ jobs:
       - name: Build sdist + wheel
         run: uv build
 
-      # Verify the ARTIFACT, not the checkout — a wheel can build cleanly and still be unusable. Without
+      # Verify the ARTIFACT, not the checkout: a wheel can build cleanly and still be unusable. Without
       # a `[project.scripts]` entry point, for example, `uvx diff-sentry` (the exact command the GitHub
       # Action runs) cannot work, and nothing in the source tree says so. Installing the built wheel into
       # a clean environment and running it is what catches that.
@@ -75,7 +75,7 @@ jobs:
             exit 1
           fi
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@v7
         with:
           name: dist
           path: dist/
@@ -87,14 +87,14 @@ jobs:
     # would fire this job and ask PyPI to publish `diff-sentry` over OIDC. The Trusted Publisher binding
     # (owner+repo+workflow+environment) would reject it, but that is a rejected publish attempt against
     # OUR project, plus a probe surface and a confusing red run in someone else's repo. `build` above is
-    # deliberately NOT guarded — it only builds and tests an artifact, and a fork verifying its own
+    # deliberately NOT guarded: it only builds and tests an artifact, and a fork verifying its own
     # build is fine.
     if: github.repository == 'qazbnm456/diff-sentry'
     environment: pypi
     permissions:
       id-token: write          # OIDC token for PyPI Trusted Publishing
     steps:
-      - uses: actions/download-artifact@v4
+      - uses: actions/download-artifact@v8
         with:
           name: dist
           path: dist/

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/AGENTS.md

@@ -1,6 +1,6 @@
 # diff-sentry: agent guide
 
-A BewAIre-style malicious-change detector built on `rlm-harness`. It classifies ONE GitHub change (PR / issue / push) for malicious intent: the change is UNTRUSTED DATA held in a sandboxed REPL, the planner SUBMITs a judgement-only verdict, and the deterministic indicator EVIDENCE is unioned on read into a SIEM signal. `README.md` has the pipeline table and the honest caveats; rlm-harness's "Building a consumer" is the extension contract this project lives within.
+A BewAIre-style malicious-change detector built on `rlm-harness`. It classifies ONE GitHub change (PR / issue / push) for malicious intent: the change is UNTRUSTED DATA held in a sandboxed REPL, the planner SUBMITs a judgement-only verdict, and the deterministic indicator EVIDENCE is unioned on read into a SIEM signal. `README.md` is the user-facing overview and `docs/` holds the pipeline and the honest limits; rlm-harness's "Building a consumer" is the extension contract this project lives within.
 
 rlm-harness is an exact PyPI pin (`rlm-harness==X.Y.Z` in `pyproject.toml`, locked in `uv.lock`). Overlay `uv pip install -e ../rlm-harness` only while co-developing the kit, and bump the pin once the fix ships.
 

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to diff-sentry, a BewAIre-style detector that classifies one GitHub change (PR, issue or push) for malicious intent on [`rlm-harness`](https://github.com/qazbnm456/rlm-harness).
 
+## 0.7.1
+
+Upgrade from 0.7.0: `rlm-harness==1.22.0`. The GitHub Action's scan is unchanged; the model pipeline (`pr`, `issue`, `classify`, `watch`) raises fewer false signals on documentation changes and stops calling a misconfigured classifier sooner. Nothing needs configuring.
+
+### Changed
+- **A classifier endpoint that answers 400, 401, 403, 404 or 422 is called once**, not retried, and counts toward the circuit breaker, so a wrong model name, key or request stops being called. Timeouts, 429 and 5xx are retried as before. The eval judge does the same for 401, 403 and 404.
+- **A planner run that fails on an unparseable reply quotes the reply as the model wrote it**, and says so when the reply carried only reasoning.
+- **`rlm-harness==1.22.0`.**
+- **The README covers using diff-sentry**; how it works, the watcher, the known limits and development moved to `docs/`.
+
+### Fixed
+- **A planner re-scan of a slice of a documentation patch is softened as the scan of that file is**, so a README change that adds a documented installer and removes an indented line no longer signals as `critical`.
+
 ## 0.7.0
 
 Upgrade from 0.6.0: `rlm-harness==1.20.0` and `dspy` 3.4.1. The signal now reads only the evidence the planner grounded in the change, and several rules read lower or higher than before, so a SIEM that alerts on severity sees fewer false signals and some new ones (see **Changed**). Nothing needs configuring; the new `DS_*` settings are optional.

diff_sentry-0.7.1/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.5
+Name: diff-sentry
+Version: 0.7.1
+Summary: Catch malicious pull requests before they merge: scan a diff for supply-chain attack patterns with deterministic evidence a model cannot suppress.
+Author: Boik Su
+License-Expression: MIT
+License-File: LICENSE
+Keywords: dspy,malicious-code-detection,prompt-injection,rlm,security,siem,supply-chain-security
+Requires-Python: >=3.11
+Requires-Dist: httpx>=0.27
+Requires-Dist: pydantic>=2.11.0
+Requires-Dist: rlm-harness==1.22.0
+Provides-Extra: classify
+Requires-Dist: openai>=1.0; extra == 'classify'
+Provides-Extra: observe
+Requires-Dist: langfuse>=4.6.1; extra == 'observe'
+Requires-Dist: openinference-instrumentation-dspy>=0.1.36; extra == 'observe'
+Provides-Extra: subscription
+Requires-Dist: claude-agent-sdk>=0.1.60; extra == 'subscription'
+Description-Content-Type: text/markdown
+
+# diff-sentry
+
+**Catch a malicious pull request before you merge it.**
+
+diff-sentry reads a change the way an attacker hopes you won't: it treats the diff as untrusted data, looks for the shapes that real supply-chain attacks use, and reports evidence rather than an opinion.
+
+The fastest way to use it is as a GitHub Action. It needs no API key, no model, and no network access, so it runs on fork pull requests under the read-only token they already get.
+
+```yaml
+# .github/workflows/diff-sentry.yml
+name: diff-sentry
+on: pull_request
+
+permissions:
+  contents: read
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0          # the scan diffs against the PR base
+      - uses: qazbnm456/diff-sentry@v0.7.1
+```
+
+That is the whole setup. The job fails when an indicator at or above `high` fires.
+
+## What it catches
+
+| Family | Examples |
+|---|---|
+| Workflow configuration | `pull_request_target` that checks out the PR head (the pwn-request that broke AsyncAPI) by `ref:`, by a hand-rolled `git fetch`, or by opting back in with `allow-unsafe-pr-checkout: true`; plus `permissions: write-all` and CODEOWNERS reassignment |
+| Obfuscation | base64 that decodes to a shell payload, `_0x…` machine-mangled JavaScript, high-entropy blobs |
+| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers |
+| Exfiltration | secrets reaching a network sink, cross-process `/proc/<pid>/mem` reads, OAST callback services, IPFS and permaweb gateways |
+| Diff-presentation evasion | payloads shoved past the viewport by a long whitespace run, bidi-override and zero-width characters |
+| Provenance | forged `[bot]` identities, bot-mimicking new accounts, accounts created days before the event, profiles that describe an autonomous agent, unsigned or identity-mismatched commits, a fork deleted after the PR |
+| Author-controlled channels | a PR branch name carrying shell syntax, slash-command comments that fire `issue_comment` workflows |
+| Campaign memory (`watch` only) | an account or payload host already flagged in another repository, an edit that removes a flagged payload, a burst of changes from a new account |
+| Prompt injection | instructions aimed at an LLM reviewer or triage workflow |
+
+A plain workflow edit, or a `pull_request_target` on its own, is `medium`, below the failure threshold, so a benign workflow PR does not break your build. Deleted lines are not scanned, so removing a payload is not flagged. [How it works](docs/how-it-works.md#severities-are-tuned-against-noise) explains the tuning.
+
+## Inputs
+
+| Input | Default | Purpose |
+|---|---|---|
+| `fail-on` | `high` | Severity that fails the build (`info`/`low`/`medium`/`high`/`critical`). |
+| `report-only-paths` | none | Newline-separated globs scanned but never gated. For files whose job is to carry attack patterns: rule bodies, security fixtures, prompt templates, docs that tabulate payloads. |
+| `base-sha` | derived | Override the commit to diff against. |
+| `version` | matches the action tag | The diff-sentry release to install from PyPI. `local` installs from the current checkout instead, which is how this repo scans its own PRs with the rules in that PR. |
+
+Outputs: `failed`, `hit-count`, `max-severity`, and `report` (a JSON path).
+
+## Do not reach for `pull_request_target`
+
+Sooner or later you will want the scan to post its results as a PR comment, and a `pull_request` job on a fork PR has a read-only token. The tempting fix is `pull_request_target`, which is the exact misconfiguration that opened the AsyncAPI "Miasma" compromise, and which this action reports as `critical`.
+
+Do it the safe way instead: keep the scan on `pull_request` with no secrets, upload the `report` as an artifact, and post it from a separate `workflow_run` job that runs the base branch's code and never checks out the PR head. diff-sentry's own rules make the same distinction, so the safe shape does not trip them.
+
+## Evidence a model cannot suppress
+
+An LLM reviewer can be argued out of its conclusion by the diff it reads. The Action runs no model at all. In the full pipeline, the rules record their hits before any model takes a turn, and the alert is derived from those hits, so a prompt injection can skew a verdict but cannot remove the evidence ([details](docs/how-it-works.md#evidence-a-model-cannot-suppress)).
+
+## Beyond the Action
+
+https://github.com/user-attachments/assets/d200aee8-263b-483e-ad55-f90ae69f3ab0
+
+_The studio console: paste a real payload, watch the detection stream live, then read the evidence-framed verdict._
+
+If you operate the infrastructure, or maintain a project where a wrong call is expensive, the rest of the system runs locally with a model endpoint you choose:
+
+- **Deep classification**: a reasoning model reads a PR, issue or push in a sandbox and returns a verdict next to the evidence. See [How it works](docs/how-it-works.md).
+- **The studio**: a local console that streams a run as it happens. See [`studio/README.md`](studio/README.md).
+- **Watch**: monitors a list of repositories, alerts the moment the rules fire, and remembers the accounts and payload hosts it has seen. See [Watching a watchlist](docs/watch.md).
+- **Trajectories for training**: every run is recorded, and `export` turns the traces into a reward-free dataset for fine-tuning a smaller planner. See [The full pipeline](docs/how-it-works.md#the-full-pipeline).
+
+None of this is needed for the Action, and none of it sends your code anywhere you did not configure yourself.
+
+## Limits
+
+A strong prompt injection can skew a model's verdict, though never the evidence. Some pipe-to-shell shapes, and a secret that reaches a network call across lines, are missed. An external watcher cannot stop an attack that fires the moment a PR opens. [Honest limits](docs/limits.md) lists the known gaps.
+
+## Documentation
+
+- [How it works](docs/how-it-works.md): severity tuning, the evidence union, the full pipeline, running it locally, and the incidents the tests reconstruct.
+- [Watching a watchlist](docs/watch.md): the monitor, its alerts, its archive, and what the account dossier holds.
+- [Honest limits](docs/limits.md): what the rules and the pipeline do not see.
+- [Development](docs/development.md): the test suites and CI.
+- [CHANGELOG](CHANGELOG.md)
+
+## License
+
+MIT. Built on [`rlm-harness`](https://github.com/qazbnm456/rlm-harness).

diff_sentry-0.7.1/README.md

@@ -0,0 +1,95 @@
+# diff-sentry
+
+**Catch a malicious pull request before you merge it.**
+
+diff-sentry reads a change the way an attacker hopes you won't: it treats the diff as untrusted data, looks for the shapes that real supply-chain attacks use, and reports evidence rather than an opinion.
+
+The fastest way to use it is as a GitHub Action. It needs no API key, no model, and no network access, so it runs on fork pull requests under the read-only token they already get.
+
+```yaml
+# .github/workflows/diff-sentry.yml
+name: diff-sentry
+on: pull_request
+
+permissions:
+  contents: read
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0          # the scan diffs against the PR base
+      - uses: qazbnm456/diff-sentry@v0.7.1
+```
+
+That is the whole setup. The job fails when an indicator at or above `high` fires.
+
+## What it catches
+
+| Family | Examples |
+|---|---|
+| Workflow configuration | `pull_request_target` that checks out the PR head (the pwn-request that broke AsyncAPI) by `ref:`, by a hand-rolled `git fetch`, or by opting back in with `allow-unsafe-pr-checkout: true`; plus `permissions: write-all` and CODEOWNERS reassignment |
+| Obfuscation | base64 that decodes to a shell payload, `_0x…` machine-mangled JavaScript, high-entropy blobs |
+| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers |
+| Exfiltration | secrets reaching a network sink, cross-process `/proc/<pid>/mem` reads, OAST callback services, IPFS and permaweb gateways |
+| Diff-presentation evasion | payloads shoved past the viewport by a long whitespace run, bidi-override and zero-width characters |
+| Provenance | forged `[bot]` identities, bot-mimicking new accounts, accounts created days before the event, profiles that describe an autonomous agent, unsigned or identity-mismatched commits, a fork deleted after the PR |
+| Author-controlled channels | a PR branch name carrying shell syntax, slash-command comments that fire `issue_comment` workflows |
+| Campaign memory (`watch` only) | an account or payload host already flagged in another repository, an edit that removes a flagged payload, a burst of changes from a new account |
+| Prompt injection | instructions aimed at an LLM reviewer or triage workflow |
+
+A plain workflow edit, or a `pull_request_target` on its own, is `medium`, below the failure threshold, so a benign workflow PR does not break your build. Deleted lines are not scanned, so removing a payload is not flagged. [How it works](docs/how-it-works.md#severities-are-tuned-against-noise) explains the tuning.
+
+## Inputs
+
+| Input | Default | Purpose |
+|---|---|---|
+| `fail-on` | `high` | Severity that fails the build (`info`/`low`/`medium`/`high`/`critical`). |
+| `report-only-paths` | none | Newline-separated globs scanned but never gated. For files whose job is to carry attack patterns: rule bodies, security fixtures, prompt templates, docs that tabulate payloads. |
+| `base-sha` | derived | Override the commit to diff against. |
+| `version` | matches the action tag | The diff-sentry release to install from PyPI. `local` installs from the current checkout instead, which is how this repo scans its own PRs with the rules in that PR. |
+
+Outputs: `failed`, `hit-count`, `max-severity`, and `report` (a JSON path).
+
+## Do not reach for `pull_request_target`
+
+Sooner or later you will want the scan to post its results as a PR comment, and a `pull_request` job on a fork PR has a read-only token. The tempting fix is `pull_request_target`, which is the exact misconfiguration that opened the AsyncAPI "Miasma" compromise, and which this action reports as `critical`.
+
+Do it the safe way instead: keep the scan on `pull_request` with no secrets, upload the `report` as an artifact, and post it from a separate `workflow_run` job that runs the base branch's code and never checks out the PR head. diff-sentry's own rules make the same distinction, so the safe shape does not trip them.
+
+## Evidence a model cannot suppress
+
+An LLM reviewer can be argued out of its conclusion by the diff it reads. The Action runs no model at all. In the full pipeline, the rules record their hits before any model takes a turn, and the alert is derived from those hits, so a prompt injection can skew a verdict but cannot remove the evidence ([details](docs/how-it-works.md#evidence-a-model-cannot-suppress)).
+
+## Beyond the Action
+
+https://github.com/user-attachments/assets/d200aee8-263b-483e-ad55-f90ae69f3ab0
+
+_The studio console: paste a real payload, watch the detection stream live, then read the evidence-framed verdict._
+
+If you operate the infrastructure, or maintain a project where a wrong call is expensive, the rest of the system runs locally with a model endpoint you choose:
+
+- **Deep classification**: a reasoning model reads a PR, issue or push in a sandbox and returns a verdict next to the evidence. See [How it works](docs/how-it-works.md).
+- **The studio**: a local console that streams a run as it happens. See [`studio/README.md`](studio/README.md).
+- **Watch**: monitors a list of repositories, alerts the moment the rules fire, and remembers the accounts and payload hosts it has seen. See [Watching a watchlist](docs/watch.md).
+- **Trajectories for training**: every run is recorded, and `export` turns the traces into a reward-free dataset for fine-tuning a smaller planner. See [The full pipeline](docs/how-it-works.md#the-full-pipeline).
+
+None of this is needed for the Action, and none of it sends your code anywhere you did not configure yourself.
+
+## Limits
+
+A strong prompt injection can skew a model's verdict, though never the evidence. Some pipe-to-shell shapes, and a secret that reaches a network call across lines, are missed. An external watcher cannot stop an attack that fires the moment a PR opens. [Honest limits](docs/limits.md) lists the known gaps.
+
+## Documentation
+
+- [How it works](docs/how-it-works.md): severity tuning, the evidence union, the full pipeline, running it locally, and the incidents the tests reconstruct.
+- [Watching a watchlist](docs/watch.md): the monitor, its alerts, its archive, and what the account dossier holds.
+- [Honest limits](docs/limits.md): what the rules and the pipeline do not see.
+- [Development](docs/development.md): the test suites and CI.
+- [CHANGELOG](CHANGELOG.md)
+
+## License
+
+MIT. Built on [`rlm-harness`](https://github.com/qazbnm456/rlm-harness).

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/action.yml

@@ -44,7 +44,7 @@ inputs:
     # KEEP IN STEP WITH [project].version. release.yml fails the build if this drifts
     # from the tag being released, because a stale default here would silently run an
     # old scanner under a new action tag.
-    default: '0.7.0'
+    default: '0.7.1'
 
 outputs:
   failed:

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/diff_sentry/__init__.py

@@ -1,4 +1,4 @@
-"""diff-sentry — classify a GitHub change for malicious intent, as a traced RLM harness.
+"""diff-sentry: classify a GitHub change for malicious intent, as a traced RLM harness.
 
 A downstream *consumer* of rlm-harness (an exact PyPI pin) reproducing Datadog's BewAIre defense shape:
 ingest a GitHub PR/issue/push → analyze the diff as UNTRUSTED data in a sandboxed REPL → produce a
@@ -80,10 +80,10 @@ __all__ = [
     "screen",
 ]
 
-__version__ = "0.7.0"
+__version__ = "0.7.1"
 
 
-def __getattr__(name: str):  # PEP 562 — defer the dspy import to first use
+def __getattr__(name: str):  # PEP 562: defer the dspy import to first use
     if name in ("ClassifyChange", "setup"):
         from . import detect
 

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/diff_sentry/actors.py

@@ -1,4 +1,4 @@
-"""Host-side actor store — who is behind a change, and what else they did.
+"""Host-side actor store: who is behind a change, and what else they did.
 
 A per-change classifier sees one PR. The campaigns worth catching span many: hackerbot-claw hit seven
 repositories in ten days from one account, with one payload host. The defender of each repository saw one

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/diff_sentry/assemble.py

@@ -1,17 +1,17 @@
-"""Assemble the canonical verdict from the trace — the read-time step that makes the EVIDENCE a fact.
+"""Assemble the canonical verdict from the trace: the read-time step that makes the EVIDENCE a fact.
 
 `assemble_verdict(verdict, events)` attaches the deterministic fields the SYSTEM owns to the planner's
 JUDGEMENT (`ChangeVerdict`, no hits): the UNION of ALL indicator hits from the trace (baseline ∪ every
-`scan_indicators` tool call — MF3), the derived max severity, whether to SIGNAL, and any indicator ids
+`scan_indicators` tool call; MF3), the derived max severity, whether to SIGNAL, and any indicator ids
 the planner cited that match no recorded hit (a fabrication tell).
 
 The `signal` decision is DETERMINISTIC and partly independent of the planner: it fires if the planner's
 verdict is in `emit_on` OR a high/critical deterministic indicator fired. So a false-benign self-report
-cannot suppress hard evidence from reaching the SIEM — the planner can skew the verdict, never hide the
+cannot suppress hard evidence from reaching the SIEM: the planner can skew the verdict, never hide the
 indicators. (This is deterministic reduction over facts, NOT a model judgement smuggled into the read
-path — the verdict tier stays the planner's; only the evidence-driven floor is ours.)
+path: the verdict tier stays the planner's; only the evidence-driven floor is ours.)
 
-This runs everywhere the result is consumed — the live path (cli), re-render, and rl_export — so labels
+This runs everywhere the result is consumed: the live path (cli), re-render, and rl_export, so labels
 read facts too. A trace missing a newer field still reads: pydantic ignores unknown keys, and hits
 re-source from the trace.
 

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/diff_sentry/cli.py

@@ -11,11 +11,11 @@
     python -m diff_sentry archive verify                  # re-hash the snapshot archive
 
 `run()` is the programmatic entry. It records `traces/{run_id}.jsonl`, writes `responses/{run_id}.json`,
-and — host-side, AFTER the run — emits a SIEM signal when the assembled verdict warrants it. A host-side
+and (host-side, AFTER the run) emits a SIEM signal when the assembled verdict warrants it. A host-side
 BASELINE indicator scan runs at ingest and rides in the run_start meta, so the deterministic evidence is
 in the trace before the planner takes a turn (MF3). `scan`/`render`/`export` work offline;
 `pr`/`issue`/`classify` need model creds (DS_* env) + a Deno sandbox (`classify` only *ingests* offline).
-`scan` exposes that deterministic baseline on its own — the half we can run in our own CI on an untrusted
+`scan` exposes that deterministic baseline on its own: the half we can run in our own CI on an untrusted
 fork PR, where no creds exist and none may be handed over (see `_cmd_scan`).
 """
 
@@ -56,7 +56,7 @@ async def detect_from_event(
     setup(config)
     task = ClassifyChange(config=config, extra_tools=extra_tools, event=event)
     event_str = normalize_event(event)
-    # Host-side BASELINE scan — deterministic evidence in the trace before the planner runs (MF3).
+    # Host-side BASELINE scan: deterministic evidence in the trace before the planner runs (MF3).
     baseline = _baseline_indicators(event)
 
     async def _run():
@@ -67,23 +67,23 @@ async def detect_from_event(
             # The run's INITIAL STATE (the untrusted event is a REPL variable, not a chat turn) + prompt.
             "event": event_str,
             "instructions": task.instructions,
-            # The change under review — echoed into the response envelope + the SIEM signal.
+            # The change under review: echoed into the response envelope + the SIEM signal.
             "source": _source_echo(event),
             # The host-derived facts about the author and the event (account age as of the event, profile,
             # timestamps, fork, commit offsets), so an offline reader sees what the ingest saw.
             **({"provenance": event["provenance"]} if event.get("provenance") else {}),
-            # The deterministic baseline — assemble unions this with every scan_indicators tool call.
+            # The deterministic baseline: assemble unions this with every scan_indicators tool call.
             "baseline_indicators": baseline,
             # The emit threshold THIS run used, so an offline re-render/export re-derives the SAME signal.
             "emit_on": list(config.emit_on),
-            # The models actually used this run (roles → concrete names) — self-describing trace.
+            # The models actually used this run (roles → concrete names): self-describing trace.
             "planner": config.main_model,
             "analyst": config.sub_model,
             "classifier": config.classifier_model,
             # The budget THIS run ran under, so an offline reader computes hit_iteration_cap correctly.
             "max_iterations": config.max_iterations,
             "max_llm_calls": config.max_llm_calls,
-            # The ATLAS rubric decomposition carried as LABELS (structure only — the trainer scores dᵢ). A
+            # The ATLAS rubric decomposition carried as LABELS (structure only; the trainer scores dᵢ). A
             # FIXED skeleton (default_rubric): diff-sentry's task is constant, so only the per-criterion
             # FACTS vary per run (re-sourced from the trace on read; see rubric.criteria_facts).
             "rubric": rubric_to_meta(default_rubric()),
@@ -166,7 +166,7 @@ def run(
     emit_failed: bool = True,
 ) -> RunArtifacts:
     """THE programmatic entry: classify one change, write the response, and (host-side, after the run)
-    emit a SIEM signal when warranted. Never raises on a run failure — a failed run still writes an
+    emit a SIEM signal when warranted. Never raises on a run failure: a failed run still writes an
     informative response and returns result-less RunArtifacts."""
     from rlm_harness.trace import load_events
 
@@ -181,7 +181,7 @@ def run(
     try:
         verdict = asyncio.run(detect_from_event(event, config, trace_path=trace_path, run_id=run_id,
                                                 on_event=on_event, extra_tools=extra_tools))
-    except Exception as exc:  # noqa: BLE001 — a failed run is still self-contained + navigable
+    except Exception as exc:  # noqa: BLE001 (a failed run is still self-contained + navigable)
         events = load_events(trace_path, run_id) if os.path.exists(trace_path) else []
         resp = build_failed_response(run_id, events, f"{type(exc).__name__}: {exc}")
         response_path = _write((os.path.join(outdir, "responses"), f"{run_id}.json"),
@@ -299,11 +299,11 @@ def drop_deleted_lines(text: str) -> str:
     """Strip a unified diff's `-` deletion lines, keeping additions and context.
 
     A diff carries the code being REMOVED as well as the code being added, and the detectors read plain
-    text — so scanning a raw diff flags a change for a payload it is DELETING. That turns every
+    text, so scanning a raw diff flags a change for a payload it is DELETING. That turns every
     remediation commit red, which is the worst possible failure mode for a security gate: it penalises
     exactly the change you wanted. `---` (the file header) is kept; only true `-` deletion lines go.
 
-    Non-diff input is returned untouched — the check is per line, so a plain source file has no `-`-
+    Non-diff input is returned untouched: the check is per line, so a plain source file has no `-`-
     prefixed lines to lose. `scan --include-deletions` opts back in when you want to audit what a change
     removed (a payload can also be hidden by deleting a check)."""
     if not text:
@@ -314,10 +314,10 @@ def drop_deleted_lines(text: str) -> str:
 
 def _cmd_scan(args) -> int:
     """The deterministic HALF of the pipeline, standalone: run the indicator suite over text and exit
-    non-zero on a hit at/above the floor. No LLM, no network, no creds, no Deno — just `indicators`.
+    non-zero on a hit at/above the floor. No LLM, no network, no creds, no Deno, just `indicators`.
 
     This is what makes the detector safe to dogfood in our OWN CI. The full `pr`/`classify` path needs
-    `DS_*` creds, and the only way to hand creds to a fork PR's run is `pull_request_target` — the exact
+    `DS_*` creds, and the only way to hand creds to a fork PR's run is `pull_request_target`, the exact
     misconfiguration that opened the AsyncAPI "Miasma" supply-chain compromise. `scan` needs nothing, so
     it runs under the same read-only `contents: read` token an untrusted fork PR already gets."""
     from .indicators import scan_diff

{diff_sentry-0.7.0 → diff_sentry-0.7.1}/diff_sentry/config.py

@@ -1,4 +1,4 @@
-"""Configuration for diff-sentry — model ROLES, never hardcoded model names.
+"""Configuration for diff-sentry: model ROLES, never hardcoded model names.
 
 Three ROLES (the rlm-harness-consumer convention): the RLM PLANNER (root LM) drives the triage loop and
 holds the diff in the REPL; the ANALYST (sub LM, reached via `llm_query`) is an expensive brain for a
@@ -103,7 +103,7 @@ def _classifier_lm_kwargs() -> dict | None:
 
 @dataclass(frozen=True)
 class DetectConfig:
-    # Planner (main) + analyst (sub), sharing one OpenAI-compatible proxy. No model name is assumed —
+    # Planner (main) + analyst (sub), sharing one OpenAI-compatible proxy. No model name is assumed:
     # `from_env` requires them. Empty defaults are only for direct construction in tests.
     main_model: str = ""   # planner: a cheap, injection-resistant orchestrator driving the REPL loop
     sub_model: str = ""    # analyst: an expensive brain for a subtle case (reached via llm_query)
@@ -121,11 +121,11 @@ class DetectConfig:
     classifier_transient_retries: int = 1
     # Extra request parameters of the classifier's call, over its `temperature=0.1` (`_classifier_lm_kwargs`).
     classifier_lm_kwargs: dict | None = None
-    # After this many CONSECUTIVE invalid classifications the tool short-circuits (rlm-harness make_model_tool).
+    # After this many CONSECUTIVE unusable replies or repeating error statuses the tool short-circuits (make_model_tool).
     classifier_circuit_break: int = 4
 
     # ── RLM runtime knobs ────────────────────────────────────────────────────────────────────────
-    interpreter: str = "pyodide"    # read-not-execute: the classifier NEVER runs the change (see README)
+    interpreter: str = "pyodide"    # read-not-execute: the classifier NEVER runs the change (see docs/how-it-works.md)
     observe: bool = False
     adapter: str = "json"
     # Generous, NOT None: a reasoning planner truncated before its answer returns empty content.
@@ -135,14 +135,14 @@ class DetectConfig:
     # reasoning planner from thinking through the whole `planner_max_tokens` and answering nothing.
     planner_lm_kwargs: dict | None = None
     analyst_lm_kwargs: dict | None = None
-    # HARD ceiling on the single RLM episode. No outer multi-run loop (max_retries=1) — one change =
+    # HARD ceiling on the single RLM episode. No outer multi-run loop (max_retries=1): one change =
     # one trajectory, so the trace stays valid training data. A failed run is INFRA, not a schema bug:
     # a non-retryable LM error escapes as the raw dspy.LMError, the rest as RLMTaskError.
     max_iterations: int = 25
     max_llm_calls: int = 8          # caps ONLY analyst (llm_query) escalations
     max_output_chars: int = 10_000  # head+tail char cap dspy.RLM applies to each REPL output
 
-    # ── Enrichment fetch (opt-in, GitHub-allowlisted — MF2) ──────────────────────────────────────
+    # ── Enrichment fetch (opt-in, GitHub-allowlisted, MF2) ──────────────────────────────────────
     enable_fetch: bool = False      # OFF by default: fetch on attacker input is an exfil surface
     fetch_timeout: float = 20.0
     fetch_max_bytes: int = 400_000
@@ -157,7 +157,7 @@ class DetectConfig:
     siem_webhook_url: str = ""
     siem_token: str = ""
     # Verdicts (or a deterministic high/critical indicator) that emit a signal. A benign self-report
-    # cannot suppress a hard indicator — see assemble.assemble_verdict.
+    # cannot suppress a hard indicator; see assemble.assemble_verdict.
     emit_on: tuple[str, ...] = ("suspicious", "malicious")
 
     @property
@@ -170,7 +170,7 @@ class DetectConfig:
 
     @classmethod
     def screen_from_env(cls) -> DetectConfig:
-        """A config with NO model roles — only the SIEM emitter and `emit_on`. For the deterministic screen
+        """A config with NO model roles, only the SIEM emitter and `emit_on`. For the deterministic screen
         (`watch --screen-only`), which never runs the model and so must not require `DS_ROOT_LM`."""
         return cls(
             siem_webhook_url=os.getenv("DS_SIEM_WEBHOOK_URL", ""),