diff-sentry 0.7.1__tar.gz → 0.8.0__tar.gz

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/.claude/rules/indicators.md

@@ -121,7 +121,7 @@ All in `indicators._soften_prose`, beside `invisible-unicode` in right-to-left t
 
 ## The file-structure rules (`scan_files`)
 
-A dev machine infected by a loader of the Contagious Interview / PolinRider family pushes its payload with ordinary work: obfuscated JavaScript appended to a config that runs on lint or build, a "font" that is a script, and a way to run it nobody types (a VS Code task that runs on folder open, an npm script now starting with `node <new file> &&`). `scan_files` reads what only the file list knows; `scan_change` runs it beside `scan_content` and `scan_metadata`.
+A dev machine infected by a loader of the Contagious Interview / PolinRider family pushes its payload with ordinary work: obfuscated JavaScript appended to a config that runs on lint or build, a "font" that is a script, and a way to run it nobody types (a VS Code task that runs on folder open, an npm script now starting with `node <new file> &&`). `scan_files` reads what only the file list knows; `scan_change` runs it beside `scan_content` and `scan_metadata`, and the Action's `scan` (`scan_diff(files=True)`) runs the same rules on each file of a diff, reading whether it is added or renamed from its header (`new file mode`, `--- /dev/null`, `rename to`) and its patch from its hunks. A path git quotes (`"b/p\303\251/x.pth"`, a byte outside printable ASCII) is unquoted. The planner's scan does not run them: a diff quoted in a comment is no file of the change, and the baseline already ran them on the change's own files.
 
 - **`package.json` and the VS Code files are read as JSONC tokens, never line by line** (`_jsonc_tokens` over the new side of the patch, each token marked written or context). A comment between a key and its value, a value on the next line, a `\u` escape, a one-line `"scripts": {...}`, a task's `args` array and `allowAutomaticTasks` nested under `"task"` are ordinary spellings, and a line regex misses each of them. A hit needs a token the change wrote.
 - **It pairs a run entry with the change's file LIST, not with another file's text.** `run-entry-new-file` asks whether a `package.json` script, or a VS Code task `command` with its `args`, runs a file whose status is `added`. That is a structural fact, so it does not break "a paired rule only pairs within one file". A target resolves against the `package.json` directory, and against the parent of the `.vscode` folder for a VS Code file.
@@ -131,11 +131,21 @@ A dev machine infected by a loader of the Contagious Interview / PolinRider fami
 - **`vscode-autorun` is `high`**: a written `"runOn": "folderOpen"` in a `.vscode/` file or a `.code-workspace`, and a written `task.allowAutomaticTasks` set to `true` or `"on"`, which turns automatic tasks on for the workspace (VS Code planned to make `off` the default). A maintainer adding a folder-open watcher task is an accepted over-alert: the setting runs code on open, the campaign's own trigger.
 - **`binary-name-holds-text` is `high`**: a text patch under a font, image, archive or native-binary name, or added lines with no patch (a text diff too large to show). A real file of those formats has a NUL in its first bytes, so GitHub sends no patch and counts no lines. A Git LFS pointer is exempt only when its whole new side is pointer keys (`version` first, then `oid sha256:<64 hex>`, `size <n>`), so an update that rewrites only `oid` and `size` stays exempt and a script with a pointer line inside does not. `.svg` is text by nature and `.pdf` is often stored as text, so neither is on the list.
 - **`obfuscated-dev-config` is `critical`**: `obfuscated-string-array` in the ADDED lines of a config a developer's tools execute (`eslint`, `vite`, `vitest`, `webpack`, `rollup`, `next`, `nuxt`, `tailwind`, `postcss`, `babel`, `jest`, `playwright`, `prettier`, `hardhat`, `commitlint`, `stylelint` configs and their `rc` files, `truffle-config`, `gatsby-*`, `.mocharc`, gulpfile, Gruntfile, `karma.conf`). Removing it is cleanup.
+- **`python-startup-hook` is `high`**: a written `.pth` line that starts with `import` and a blank (the only lines `site` executes; the rest of a `.pth` is a list of paths), after an optional BOM, with each line of the patch split as `site` splits it (`str.splitlines`, so a lone CR or a form feed starts a line of its own); and any written line, or a rename, of a `sitecustomize` or `usercustomize` module or package. A `.pth` renamed without a change to its content shows no line and is a known gap. Python runs both at every start of an interpreter whose site-packages holds them, the litellm 1.107.0 wheels' trigger. The shims of packaging and instrumentation tools (virtualenv, setuptools' `distutils-precedence.pth`, coverage, OpenTelemetry's `sitecustomize.py`) are an accepted over-alert: they run code at every start too.
 - **The token walks are bounded**: a comment never spans two hunks (the lines between them are unseen), and a `command`'s object is searched at most `_OBJECT_REACH` tokens each way, for at most `_MAX_RUN_ENTRIES` entries a file.
 - **A structural hit's id carries its path** (`form=path`): the same setting in `tasks.json` and `settings.json` is two findings, not one deduped into the other.
 - **`obfuscated-string-array` (`high`) runs everywhere**: javascript-obfuscator's rotation loop `while(!![])` with the push/shift rotation or 20 hex-indexed decoder calls. It needs no `_0x` names, which a mangled-name setting removes (`obfuscated-identifiers` still reads those). Hand-rewritten output (`while(!0)`) is a known gap.
 - **Measured:** over 857 live changes (6,858 files from four large repositories) the rules fired only on the one change they were written for, and the string-array signature appeared in none of 3,553 JavaScript files over 30 KB (bundles and minified libraries among them). Terser and esbuild write `!0` and `for(;;)`, never `!![]`.
 
+## Python loaders (`_scan_py_exec`)
+
+Text rules, so they read every channel and a value the planner decodes. Over 39,589 distinct Python files of PyPI packages (725 MB), the three fired on two files: an `eval` beside `urlopen` in a test helper and mypy's daemon.
+
+- **`packed-code-exec` is `high`**: `exec` / `eval` (the builtins, also as `builtins.exec` or `getattr(__builtins__, 'exec')`; not a method `.exec(`, not `literal_eval`) whose argument is a decoding call (the base64 family, `decodebytes`, hex, `decompress`, `marshal.loads`; not `json.loads`), through `compile(`, a module alias, a bare imported name or `__import__('zlib')`; a name assigned a decoding call and run after a `;` on the same line; and JavaScript's `eval` / `new Function` over `atob(` or `Buffer.from(`. A decoded value run on a later line is the model stage's to read.
+- **`remote-code-exec` is `high`**: a Python HTTP client (`requests`, `urlopen`, `urlretrieve`, `httpx`, `aiohttp`, `urllib3`) and `exec` / `eval` in the same file.
+- **`detached-process-spawn` also covers Python**: `os.fork` and `os.setsid` in the same file. `subprocess.Popen(start_new_session=True)` is a known gap.
+- **`obfuscated-payload` inflates first**: a base64 blob that decodes to a zlib stream (any window size) or a gzip member is decompressed before it is read for a payload, each blob to at most 16 times its size and the blobs of one text to 256 KiB in all, because every inflated byte is read by the rules again. A raw deflate stream, and a blob past the budget, are read as they decode: decoy blobs in front can spend the budget, and the `exec` that runs the payload still fires `packed-code-exec` on its own.
+
 ## What the kind of a segment changes
 
 `_scope_to_file` runs on every segment of an event but the branch name (`scan_content`) and on every file of a diff (`scan_diff`), after the detectors and the prose softening. A branch name keeps every hit, and so does the planner's `scan_indicators`, but for a maintainer's `inline-code-exec` it copies (`_written_by_maintainer`, above).

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/.claude/skills/release/SKILL.md

@@ -33,5 +33,7 @@ check it by hand.
    …`, notes = the CHANGELOG section). The user publishes it in the web UI with "Publish this Action to
    the GitHub Marketplace" ticked (the CLI cannot tick it). Publishing is what builds, verifies, and
    uploads to PyPI via Trusted Publishing, and the upload is irreversible.
-6. After `release.yml` succeeds, re-run the failed consumer-install job
+6. After `release.yml` succeeds and PyPI's simple index lists the version (`curl -s -H "Accept:
+   application/vnd.pypi.simple.v1+json" https://pypi.org/simple/diff-sentry/`; the index lags the upload by
+   minutes, and a rerun before it lists the version fails to resolve), re-run the failed consumer-install job
    (`gh run rerun <ci run id> --failed`); green proves the published package installs and scans.

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/CHANGELOG.md

@@ -2,6 +2,22 @@
 
 All notable changes to diff-sentry, a BewAIre-style detector that classifies one GitHub change (PR, issue or push) for malicious intent on [`rlm-harness`](https://github.com/qazbnm456/rlm-harness).
 
+## 0.8.0
+
+Upgrade from 0.7.1: the GitHub Action now runs the file-structure rules and catches Python loaders, so a pull request it passed before can fail it now (see **Added** and **Fixed**). Nothing needs configuring, and `rlm-harness==1.22.0` is unchanged.
+
+### Added
+- **Rules for the litellm 1.107.0 loader shape** ([BerriAI/litellm#47826](https://github.com/BerriAI/litellm/issues/47826)), all `high`:
+  - `python-startup-hook`: a `.pth` line that Python runs at every start, and a `sitecustomize` or `usercustomize` module or package;
+  - `packed-code-exec`: `exec` or `eval` over a decoding call such as `zlib.decompress(base64.b64decode(...))`, and JavaScript's `eval` or `new Function` over `atob` or `Buffer.from`;
+  - `remote-code-exec`: a Python HTTP client and `exec` or `eval` in the same file;
+  - `detached-process-spawn` also covers Python's `os.fork` with `os.setsid`.
+- **A packed payload is inflated before it is read**: a base64 blob holding a zlib or gzip stream is decompressed, within a size budget, and scanned for a payload.
+
+### Fixed
+- **The Action's scan runs the file-structure rules**: a VS Code task that runs on folder open, an npm lifecycle hook or task that runs a file the change adds, a script under an image or font name, obfuscated code in a developer tool's config, and the new startup hooks. Before, only the model pipeline and `watch` ran them.
+- **A path git quotes in a diff header** (a byte outside printable ASCII) is read as its file instead of folding into the file before it.
+
 ## 0.7.1
 
 Upgrade from 0.7.0: `rlm-harness==1.22.0`. The GitHub Action's scan is unchanged; the model pipeline (`pr`, `issue`, `classify`, `watch`) raises fewer false signals on documentation changes and stops calling a misconfigured classifier sooner. Nothing needs configuring.

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: diff-sentry
-Version: 0.7.1
+Version: 0.8.0
 Summary: Catch malicious pull requests before they merge: scan a diff for supply-chain attack patterns with deterministic evidence a model cannot suppress.
 Author: Boik Su
 License-Expression: MIT
@@ -42,7 +42,7 @@ jobs:
       - uses: actions/checkout@v7
         with:
           fetch-depth: 0          # the scan diffs against the PR base
-      - uses: qazbnm456/diff-sentry@v0.7.1
+      - uses: qazbnm456/diff-sentry@v0.8.0
 ```
 
 That is the whole setup. The job fails when an indicator at or above `high` fires.
@@ -53,7 +53,7 @@ That is the whole setup. The job fails when an indicator at or above `high` fire
 |---|---|
 | Workflow configuration | `pull_request_target` that checks out the PR head (the pwn-request that broke AsyncAPI) by `ref:`, by a hand-rolled `git fetch`, or by opting back in with `allow-unsafe-pr-checkout: true`; plus `permissions: write-all` and CODEOWNERS reassignment |
 | Obfuscation | base64 that decodes to a shell payload, `_0x…` machine-mangled JavaScript, high-entropy blobs |
-| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers |
+| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers, Python startup hooks (`.pth`, `sitecustomize.py`), packed or downloaded code run with `exec` |
 | Exfiltration | secrets reaching a network sink, cross-process `/proc/<pid>/mem` reads, OAST callback services, IPFS and permaweb gateways |
 | Diff-presentation evasion | payloads shoved past the viewport by a long whitespace run, bidi-override and zero-width characters |
 | Provenance | forged `[bot]` identities, bot-mimicking new accounts, accounts created days before the event, profiles that describe an autonomous agent, unsigned or identity-mismatched commits, a fork deleted after the PR |

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/README.md

@@ -21,7 +21,7 @@ jobs:
       - uses: actions/checkout@v7
         with:
           fetch-depth: 0          # the scan diffs against the PR base
-      - uses: qazbnm456/diff-sentry@v0.7.1
+      - uses: qazbnm456/diff-sentry@v0.8.0
 ```
 
 That is the whole setup. The job fails when an indicator at or above `high` fires.
@@ -32,7 +32,7 @@ That is the whole setup. The job fails when an indicator at or above `high` fire
 |---|---|
 | Workflow configuration | `pull_request_target` that checks out the PR head (the pwn-request that broke AsyncAPI) by `ref:`, by a hand-rolled `git fetch`, or by opting back in with `allow-unsafe-pr-checkout: true`; plus `permissions: write-all` and CODEOWNERS reassignment |
 | Obfuscation | base64 that decodes to a shell payload, `_0x…` machine-mangled JavaScript, high-entropy blobs |
-| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers |
+| Execution | pipe-to-shell installs, `${IFS}` space-evasion, detached child processes, inline `node -e`, fetch-to-disk droppers, Python startup hooks (`.pth`, `sitecustomize.py`), packed or downloaded code run with `exec` |
 | Exfiltration | secrets reaching a network sink, cross-process `/proc/<pid>/mem` reads, OAST callback services, IPFS and permaweb gateways |
 | Diff-presentation evasion | payloads shoved past the viewport by a long whitespace run, bidi-override and zero-width characters |
 | Provenance | forged `[bot]` identities, bot-mimicking new accounts, accounts created days before the event, profiles that describe an autonomous agent, unsigned or identity-mismatched commits, a fork deleted after the PR |

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/action.yml

@@ -44,7 +44,7 @@ inputs:
     # KEEP IN STEP WITH [project].version. release.yml fails the build if this drifts
     # from the tag being released, because a stale default here would silently run an
     # old scanner under a new action tag.
-    default: '0.7.1'
+    default: '0.8.0'
 
 outputs:
   failed:

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/diff_sentry/__init__.py

@@ -80,7 +80,7 @@ __all__ = [
     "screen",
 ]
 
-__version__ = "0.7.1"
+__version__ = "0.8.0"
 
 
 def __getattr__(name: str):  # PEP 562: defer the dspy import to first use

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/diff_sentry/cli.py

@@ -336,7 +336,7 @@ def _cmd_scan(args) -> int:
             label = path
         if not args.include_deletions:
             text = drop_deleted_lines(text)
-        hits.extend(scan_diff(text, location=label))
+        hits.extend(scan_diff(text, location=label, files=True))
 
     worst = max((severity_rank(h.severity) for h in hits), default=-1)
     failed = worst >= threshold

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/diff_sentry/indicators.py

@@ -30,6 +30,7 @@ import posixpath
 import re
 import unicodedata
 import warnings
+import zlib
 from collections import Counter, deque
 from collections.abc import Callable, Iterable, Iterator
 from functools import partial
@@ -401,6 +402,31 @@ _JS_NET_FETCH_RE = re.compile(
     re.IGNORECASE)
 _FILE_WRITE_RE = re.compile(r"writeFileSync|createWriteStream|\bfs\.write", re.IGNORECASE)
 
+# Python's side of a loader (the litellm 1.107.0 wheels): code run straight out of a decoding call, code fetched and run,
+# and a daemon that leaves its session. `exec(` / `eval(` are the builtins: a method (`.exec(`) or a longer name
+# (`literal_eval(`) is not one. The decoding call may sit behind a module alias (`import zlib as _a; exec(_a.decompress(`).
+# The decoding calls: the base64 family, hex, compression, and `marshal.loads` (compiled code). A bare name imported with
+# `from base64 import b64decode` and a module loaded with `__import__('zlib')` count as well as `zlib.` and an alias.
+_DECODE_CALL = (r"(?:(?:[\w.]{0,80}\.|__import__\s{0,20}\(\s{0,20}['\"]\w{1,20}['\"]\s{0,20}\)\s{0,20}\.)?(?:urlsafe_|standard_)?"
+                r"(?:decompress|b64decode|b32decode|b16decode|b85decode|a85decode|decodebytes|unhexlify|fromhex)"
+                r"|marshal\s{0,20}\.\s{0,20}loads)\s{0,20}\(")
+# The builtins `exec` / `eval`, also reached through `builtins.` or `getattr(__builtins__, 'exec')`; not a method (`.exec(`) or
+# a longer name (`literal_eval(`).
+_PY_EXEC = (r"(?:(?<![\w.$])(?:exec|eval)|(?<![\w.])(?:__builtins__|builtins)\s{0,20}\.\s{0,20}(?:exec|eval)"
+            r"|\bgetattr\s{0,20}\(\s{0,20}(?:__builtins__|builtins)\s{0,20},\s{0,20}['\"](?:exec|eval)['\"]\s{0,20}\))\s{0,20}\(")
+_PACKED_EXEC_RE = re.compile(
+    _PY_EXEC + r"\s{0,20}(?:compile\s{0,20}\(\s{0,20})?" + _DECODE_CALL
+    # decoded into a name on the same line, then that name run: `c = zlib.decompress(base64.b64decode(s)); exec(c)`
+    + r"|(?<![\w.])(?P<name>\w{1,40})\s{0,20}=\s{0,20}" + _DECODE_CALL + r"[^\n]{0,400}?;[ \t]{0,20}" + _PY_EXEC
+    + r"\s{0,20}(?:compile\s{0,20}\(\s{0,20})?(?P=name)\b"
+    + r"|(?<![\w.$])(?:eval|new[ \t]+Function)\s{0,20}\(\s{0,20}(?:window\.)?(?:atob|Buffer\s{0,20}\.\s{0,20}from)\s{0,20}\(")
+_PY_EXEC_RE = re.compile(_PY_EXEC)
+_PY_NET_FETCH_RE = re.compile(
+    r"\brequests\s{0,20}\.\s{0,20}(?:get|post|request|Session)\b|\burlopen\s{0,20}\(|\burlretrieve\s{0,20}\("
+    r"|\bhttpx\s{0,20}\.\s{0,20}(?:get|post|request|Client|AsyncClient)\b|\baiohttp\b|\burllib3\b")
+_PY_FORK_RE = re.compile(r"\bos\s{0,20}\.\s{0,20}fork\s{0,20}\(")
+_PY_SETSID_RE = re.compile(r"\bos\s{0,20}\.\s{0,20}setsid\s{0,20}\(")
+
 # Tier 3: content-addressed / permaweb gateways. Miasma staged its second-stage binary on IPFS. Same
 # FP-safe reasoning as the dev tunnels: these have real mainstream use (web3 apps, NFT metadata), so a
 # reference alone is a sub-floor `medium`. When the change actually FETCHES from one and writes the
@@ -555,26 +581,54 @@ def _mostly_text(decoded: str) -> bool:
     return odd <= max(1, len(decoded) // 10)
 
 
+# What the compressed blobs of one text may inflate to, in all, and one blob at most per byte of it: a small blob can
+# decompress huge, and every inflated byte is read by the rules again. A packer's source compresses about threefold.
+_INFLATE_BUDGET = 256 << 10
+_INFLATE_RATIO = 16
+
+
+def _compressed(raw: bytes) -> bool:
+    """A gzip member, or a zlib stream of any window size (its two header bytes are a multiple of 31, deflate method)."""
+    return raw[:2] == b"\x1f\x8b" or (len(raw) >= 2 and raw[0] & 0x0F == 8 and raw[0] >> 4 <= 7
+                                       and ((raw[0] << 8) | raw[1]) % 31 == 0)
+
+
+def _inflated(raw: bytes, limit: int) -> bytes:
+    """`raw` decompressed when it is a zlib or gzip stream (the packer's `zlib.compress` before `b64encode`), at most
+    `limit` bytes of it; any other bytes as they are."""
+    if limit <= 0 or not _compressed(raw):
+        return raw
+    try:
+        return zlib.decompressobj(wbits=47).decompress(raw, limit)
+    except zlib.error:
+        return raw
+
+
 def _scan_obfuscation(text: str) -> list[IndicatorHit]:
     """Decode base64 blobs and RE-SCAN the decoded bytes for shell/exfil payloads (de-obfuscation)."""
     hits: list[IndicatorHit] = []
     seen: set[str] = set()
+    budget = _INFLATE_BUDGET
     for m in _B64_RE.finditer(text):
         blob = m.group(0)
         if blob in seen:
             continue
         seen.add(blob)
         try:
-            decoded = base64.b64decode(blob + "=" * (-len(blob) % 4), validate=True).decode("utf-8", "replace")
+            raw = base64.b64decode(blob + "=" * (-len(blob) % 4), validate=True)
         except (binascii.Error, ValueError):
             continue
+        inflated = _inflated(raw, min(budget, _INFLATE_RATIO * len(raw)))
+        if inflated is not raw:
+            budget -= len(inflated)
+        decoded = inflated.decode("utf-8", "replace")
         if not _mostly_text(decoded):
             continue
         # Only flag base64 that DECODES to something suspicious; a plain data blob isn't an indicator.
         # (Rescan shell/exfil/URL/callback-domain payloads; CI-skip and permission markers aren't usefully
         # base64-wrapped, so they're not in the inner set.)
         inner = (_scan_shell(decoded, False) + _scan_urls(decoded) + _scan_exfil(decoded)
-                 + _scan_exfil_domains(decoded) + _scan_js_exec(decoded))
+                 + _scan_exfil_domains(decoded) + _scan_js_exec(decoded) + _scan_py_exec(decoded))
         if inner:
             worst = max((h.severity for h in inner), key=lambda s: _sev_rank(s))
             hits.append(_hit("obfuscated-payload", worst,
@@ -757,6 +811,27 @@ def _scan_js_exec(text: str, patch: bool = False) -> list[IndicatorHit]:
     return hits
 
 
+def _scan_py_exec(text: str) -> list[IndicatorHit]:
+    """Running code that arrives packed or over the network, and Python's daemon shape. Each has no ordinary reading:
+    a program that ships its own source runs it, it does not decode it first; a client that fetches data parses it."""
+    hits: list[IndicatorHit] = []
+    packed = _PACKED_EXEC_RE.search(text)
+    if packed:
+        hits.append(_hit("packed-code-exec", "high", "runs code straight out of a decoding call (packed payload)",
+                         _snip(text, packed.start(), packed.end())))
+    fetch = _PY_NET_FETCH_RE.search(text)
+    run = _PY_EXEC_RE.search(text) if fetch else None
+    if run:
+        hits.append(_hit("remote-code-exec", "high", "fetches over the network AND runs code with `exec` / `eval` (loader shape)",
+                         _snip(text, fetch.start(), fetch.end())))
+    fork = _PY_FORK_RE.search(text)
+    if fork and _PY_SETSID_RE.search(text):
+        hits.append(_hit("detached-process-spawn", "high",
+                         "forks a process that leaves its session (`os.fork` + `os.setsid`, a daemon outliving its parent)",
+                         _snip(text, fork.start(), fork.end())))
+    return hits
+
+
 def _scan_hex_identifiers(text: str) -> list[IndicatorHit]:
     """javascript-obfuscator's `_0x…` mangling: obfuscation that base64 detection cannot see."""
     found = _HEX_IDENT_RE.findall(text)
@@ -842,7 +917,32 @@ def _sev_rank(sev: str) -> int:
 # The trade is explicit: a payload deliberately split across two files does not pair. That is the
 # right way round: a detector that invents a critical out of two unrelated files trains people to
 # disable it, and the single-signal rules (most of them) are unaffected.
-_DIFF_FILE_RE = re.compile(r"^diff --git a/(?P<a>.+?) b/(?P<b>.+?)$", re.MULTILINE)
+# git quotes a path with a byte outside printable ASCII (`core.quotePath`, on by default), a quote, a backslash or a control
+# character: `diff --git "a/p\\303\\251/x.pth" "b/p\\303\\251/x.pth"`, each side on its own.
+_DIFF_FILE_RE = re.compile(r'^diff --git (?:"a/(?P<qa>(?:[^"\\\n]|\\.)+)"|a/(?P<a>.+?)) (?:"b/(?P<qb>(?:[^"\\\n]|\\.)+)"|b/(?P<b>.+?))$',
+                           re.MULTILINE)
+_GIT_ESCAPES = {"a": 7, "b": 8, "t": 9, "n": 10, "v": 11, "f": 12, "r": 13}
+_GIT_OCTAL_RE = re.compile(r"\\([0-3][0-7]{2})|\\(.)", re.DOTALL)
+
+
+def _git_unquote(path: str) -> str:
+    """A path as git's quoting spells it, with its C escapes (`\\303\\251`, `\\t`, `\\"`) turned back into its bytes."""
+    out = bytearray()
+    last = 0
+    for m in _GIT_OCTAL_RE.finditer(path):
+        out += path[last:m.start()].encode("utf-8", "surrogatepass")
+        out.append(int(m.group(1), 8) if m.group(1) else _GIT_ESCAPES.get(m.group(2), ord(m.group(2)) if m.group(2) < "\x80" else 63))
+        last = m.end()
+    out += path[last:].encode("utf-8", "surrogatepass")
+    return out.decode("utf-8", "replace")
+
+
+def _diff_path(m: re.Match) -> str:
+    if m.group("qb") is not None:
+        return _git_unquote(m.group("qb"))
+    if m.group("b") is not None:
+        return m.group("b")
+    return _git_unquote(m.group("qa")) if m.group("qa") is not None else m.group("a")
 
 
 def split_diff_by_file(text: str) -> list[tuple[str, str]]:
@@ -860,7 +960,7 @@ def split_diff_by_file(text: str) -> list[tuple[str, str]]:
         out.append(("", head))
     for i, m in enumerate(marks):
         end = marks[i + 1].start() if i + 1 < len(marks) else len(text)
-        out.append((m.group("b") or m.group("a"), text[m.start():end]))
+        out.append((_diff_path(m), text[m.start():end]))
     return out
 
 
@@ -3200,12 +3300,29 @@ def scan_content(event: dict) -> list[IndicatorHit]:
     return list(deduped.values())
 
 
-def scan_diff(text: str, *, location: str = "") -> list[IndicatorHit]:
+_DIFF_ADDED_RE = re.compile(r"^(?:new file mode |--- /dev/null\r?$)", re.MULTILINE)
+_DIFF_RENAMED_RE = re.compile(r"^rename to ", re.MULTILINE)
+_DIFF_HUNK_RE = re.compile(r"^@@", re.MULTILINE)
+
+
+def _diff_file(path: str, segment: str) -> dict:
+    """One file of a `diff --git` diff in the shape of an event's file entry: added when its header says so, its patch the
+    hunks (a binary file's `Binary files ... differ` has none), its additions counted."""
+    hunk = _DIFF_HUNK_RE.search(segment)
+    header, patch = (segment[:hunk.start()], segment[hunk.start():]) if hunk else (segment, "")
+    status = "added" if _DIFF_ADDED_RE.search(header) else "renamed" if _DIFF_RENAMED_RE.search(header) else "modified"
+    return {"filename": path, "status": status,
+            "additions": sum(1 for line in patch.split("\n") if line.startswith("+")), "deletions": 0, "patch": patch}
+
+
+def scan_diff(text: str, *, location: str = "", files: bool = False) -> list[IndicatorHit]:
     """`scan_indicators` with per-file scoping: a paired rule can only fire when ONE file carries both
     halves. Non-diff input (a region the planner pulled out, a plain file) falls through to a single
     whole-text scan, so this is safe as the entry point for either shape. Both are read as a patch (a leading `+` is a diff marker):
     the host-side baseline reads every channel with its own flag, and the union keeps its stricter hit. Only the files of a
-    `diff --git` diff are a file's patch to `inline-code-exec` (`patch`); the fallthrough is read as it is."""
+    `diff --git` diff are a file's patch to `inline-code-exec` (`patch`); the fallthrough is read as it is. `files` also runs the
+    file-structure rules on each file of the diff, as `scan_files` does on an event's file list: the Action's `scan` reads a
+    diff of the change, while a diff the planner finds quoted in a comment is no file of the change."""
     raw = text or ""
     text = _lf(raw)
     segments = split_diff_by_file(text)
@@ -3213,10 +3330,16 @@ def scan_diff(text: str, *, location: str = "") -> list[IndicatorHit]:
         return scan_indicators(raw, location=location)
     lone_cr = _LONE_CR_RE.search(raw) or "\0" in raw     # the files were cut out of the text `_lf` made
     deduped: dict[str, IndicatorHit] = {}
+    # cut with only CRLF made LF: a lone CR, which `_lf` makes a line break, is a character of a `.pth` line to the patch
+    entries = [_diff_file(path, segment) for path, segment in split_diff_by_file(raw.replace("\r\n", "\n")) if path] if files else []
+    added = frozenset(posixpath.normpath(f["filename"]) for f in entries if f["status"] == "added")
     for path, segment in segments:
         found = scan_indicators(segment, location=path or location, patch=bool(path) and not lone_cr)
         for h in _or_closed(partial(_scope_to_file, found, path, segment), found, "_scope_to_file") if path else found:
             keep_worse(deduped, h)
+    for f in entries:     # the file-structure rules an event's file list gets (`scan_files`), from the diff's own headers
+        for h in _file_hits(f, added):
+            keep_worse(deduped, h)
     if _LONE_CR_RE.search(raw):
         # `scan_indicators` reads a lone CR as a character of a word for a text it is handed as it is; the files of a diff were cut out of the text with the CR made a newline
         letters = _lf(_LONE_CR_RE.sub("x", raw))
@@ -3233,7 +3356,7 @@ def scan_diff(text: str, *, location: str = "") -> list[IndicatorHit]:
 _DETECTORS = (
     _scan_shell, _scan_obfuscation, _scan_exfil, _scan_urls, _scan_exfil_domains, _scan_ci_paths, _scan_ci_bypass,
     _scan_workflow_perms, _scan_injection, _scan_entropy, _scan_workflow_config, _scan_diff_evasion, _scan_js_exec,
-    _scan_hex_identifiers, _scan_string_array, _scan_run_asset,
+    _scan_hex_identifiers, _scan_string_array, _scan_run_asset, _scan_py_exec,
 )
 
 
@@ -3696,6 +3819,15 @@ def _file_structure_hits(f: dict, added: frozenset[str]) -> list[IndicatorHit]:
             found, autorun = _autorun_hits(path, tokens)
             hits += found
         hits += _run_entry_hits(f, added, tokens, autorun)
+    base = posixpath.basename(low)
+    if base.endswith(".pth"):
+        line = next((part for part in _pth_written(patch) if _PTH_IMPORT_RE.match(part)), None)
+        if line is not None:
+            hits.append(_hit("python-startup-hook", "high", "a `.pth` file that runs code at every Python start",
+                             line[:_MAX_EVIDENCE], location=path, form=path))
+    elif _CUSTOMIZE_RE.search(low) and (_added_lines(patch).strip() or f["status"] == "renamed"):
+        hits.append(_hit("python-startup-hook", "high", f"a `{_CUSTOMIZE_RE.search(low).group(1)}` module (Python runs it at every start)",
+                         _snip(_added_lines(patch), 0, 0, pad=160) or f"renamed to {path[:160]}", location=path, form=path))
     if _DEV_CONFIG_RE.search(path):
         new = _added_lines(patch)
         if _scan_string_array(new):
@@ -3705,6 +3837,21 @@ def _file_structure_hits(f: dict, added: frozenset[str]) -> list[IndicatorHit]:
     return hits
 
 
+# The lines `site` executes in a `.pth` file of site-packages: those that start with `import` and a blank, at every start
+# of the interpreter (the rest of the file is a list of paths). It reads the file as `utf-8-sig` and splits it with
+# `str.splitlines` (3.13; earlier, text mode, which also ends a line at a lone CR), so a line of the patch can hold several.
+_PTH_IMPORT_RE = re.compile(r"\ufeff?import[ \t]")
+# `sitecustomize` and `usercustomize`, as a module or a package, which `site` imports at every start.
+_CUSTOMIZE_RE = re.compile(r"(?:^|/)((?:site|user)customize)(?:\.py|/__init__\.py)$")
+
+
+def _pth_written(patch: str) -> list[str]:
+    """The lines of a `.pth` file that the patch writes, split as `site` splits them: a lone CR, a form feed and the other
+    `str.splitlines` breaks end a line there, though no line of the patch ends at them."""
+    return [part for line in patch.split("\n") if line.startswith("+") and not line.startswith("+++")
+            for part in line[1:].splitlines()]
+
+
 def _added_lines(patch: str) -> str:
     return "\n".join(text for text, written in _new_side(patch) if written)
 
@@ -3718,16 +3865,20 @@ def scan_files(event: dict) -> list[IndicatorHit]:
     added = frozenset(posixpath.normpath(f["filename"]) for f in files if f["status"] == "added")
     deduped: dict[str, IndicatorHit] = {}
     for f in files:
-        try:
-            found = _file_structure_hits(f, added)
-        except Exception as exc:  # noqa: BLE001 (fail CLOSED, as `scan_indicators` does)
-            found = [_hit("scanner-error", "high", "a file-structure rule raised on this file, so it was not fully scanned",
-                          f"_file_structure_hits: {type(exc).__name__}", location=f["filename"])]
-        for h in found:
+        for h in _file_hits(f, added):
             keep_worse(deduped, h)
     return list(deduped.values())
 
 
+def _file_hits(f: dict, added: frozenset[str]) -> list[IndicatorHit]:
+    """`_file_structure_hits` of one file, or a `scanner-error` for it when a rule raises."""
+    try:
+        return _file_structure_hits(f, added)
+    except Exception as exc:  # noqa: BLE001 (fail CLOSED, as `scan_indicators` does)
+        return [_hit("scanner-error", "high", "a file-structure rule raised on this file, so it was not fully scanned",
+                     f"_file_structure_hits: {type(exc).__name__}", location=f["filename"])]
+
+
 def scan_change(event: dict) -> list[IndicatorHit]:
     """Everything an event's own content and structure show: the text rules over each file, the title, the
     body, the head branch and the author's/outsiders' comments (`scan_content`), plus the structural rules

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/docs/how-it-works.md

@@ -49,8 +49,10 @@ None of this is required to use the Action, and none of it sends your code anywh
 
 ## Grounded in real incidents
 
-Two incidents are reconstructed offline and pinned by tests, so a rule change that breaks coverage fails the build.
+Three incidents are reconstructed offline and pinned by tests, so a rule change that breaks coverage fails the build.
 
 **`hackerbot-claw`** (Datadog's [BewAIre writeup](https://www.datadoghq.com/blog/engineering/stopping-hackerbot-claw-with-bewaire/)). The original artifacts are gone and the attacker account was deleted, so `tests/corpus/hackerbot_claw_incident.json` reconstructs all three events: a `${IFS}`/base64 payload hidden in a *filename* under `documentation/rules/` that de-obfuscates to a published pipe-to-shell IOC, a prompt injection telling a triage LLM to bulk-label and exfiltrate, and a prompt injection appending a fake owner to CODEOWNERS. Each is asserted under a neutral benign verdict, so the alert is evidence-driven: a false-benign self-report could not have suppressed any of them.
 
 **AsyncAPI "Miasma"**. The attack was Node end to end, and `tests/test_indicators.py` reconstructs its stages: the `pull_request_target` workflow that checks out the PR head, the payload pushed off the diff viewport by a whitespace run, the IPFS fetch-to-disk dropper, and the `_0x` obfuscation injected into a source file. Each rule ships with its negative case, so the tuning is pinned as tightly as the detection.
+
+**litellm 1.107.0** ([BerriAI/litellm#47826](https://github.com/BerriAI/litellm/issues/47826)). The wheels carried a `.pth` file that imports a zlib+base64-packed credential stealer at every Python start, and a `sitecustomize.py` that forks a daemon to fetch code and run it. The commits came from a lookalike fork straight into the release build, so no pull request held them. `tests/test_python_loader.py` holds that the same files, sent as a pull request, fail the Action's scan.

{diff_sentry-0.7.1 → diff_sentry-0.8.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "diff-sentry"
-version = "0.7.1"
+version = "0.8.0"
 description = "Catch malicious pull requests before they merge: scan a diff for supply-chain attack patterns with deterministic evidence a model cannot suppress."
 readme = "README.md"
 requires-python = ">=3.11"