diff-sentry 0.4.3__tar.gz → 0.6.0__tar.gz

diff_sentry-0.6.0/.claude/rules/detect-setup.md

@@ -0,0 +1,23 @@
+---
+paths:
+  - "diff_sentry/detect.py"
+  - "diff_sentry/config.py"
+  - "diff_sentry/cli.py"
+  - "diff_sentry/deep_classify.py"
+  - "diff_sentry/__init__.py"
+  - ".env.example"
+---
+
+# Model roles, auth, and run failures
+
+- **Failures name themselves.** With `max_retries=1` (set in `detect.setup`), a non-retryable LM error
+  (auth / billing / configuration / unsupported model) escapes after one attempt as the raw
+  `dspy.LMError` subclass. An `RLMTaskError` means the single attempt ended in an adapter parse failure,
+  an invalid result, or a retryable endpoint fault (timeout / 5xx / transport). `cli.run` turns both into
+  a `status=failed` response. Check the endpoint before suspecting the schema.
+- **Subscription auth is a lazy, opt-in branch.** A role whose model is `claude-agent-sdk/<id>` runs on
+  rlm-harness's `ClaudeAgentLM`, imported only inside `detect._maybe_subscription_lm` and never from
+  `__init__.py`, so `import diff_sentry` stays dspy-free and a proxy-only install never touches the SDK.
+- **The classifier never runs on the subscription.** It is a separate OpenAI-compatible client
+  (`deep_classify._selfclassify_chat`), so `config.from_env` rejects a `claude-agent-sdk/` classifier
+  model, whether set explicitly or inherited from `DS_SUB_LM`. Mixed auth across roles is by design.

diff_sentry-0.6.0/.claude/rules/handoff.md

@@ -0,0 +1,45 @@
+# Context preservation (read before auto-compacting)
+
+Durable knowledge belongs in tracked files; a handoff summary carries only the in-flight session state
+they do not hold yet.
+
+- **Stable invariants** → `AGENTS.md` **Hard invariants**, or the topic rule under `.claude/rules/`.
+- **Shipped changes** → the commit message, and a `CHANGELOG.md` entry at release.
+- **Open / proposed work** → the issue tracker, or `AGENTS.md` **Layout** (next increments).
+
+A handoff summary covers, in order:
+
+1. **Decisions agreed this session** that are not yet in those files, each with its reason. Promote
+   durable ones into `AGENTS.md` or a topic rule before they fade.
+2. **Files / symbols changed**, as `path:symbol — final shape` one-liners (e.g.
+   `assemble.py:assemble_verdict — signal = verdict∈emit_on OR severity≥floor`). No diffs, no
+   intermediate revisions.
+3. **Current status**: what passes (`uv run pytest`, the eval suite, `uvx ruff check .`) with counts,
+   what is broken, the last command and its result. One paragraph.
+4. **Open items** not yet tracked, each marked `proposed`, `accepted-not-done`, or `rejected`.
+5. **Seams touched**: `classify_backend` (`"self"` or a dedicated backend in progress), the host-side
+   pre-filter / live GitHub+SIEM wiring, and the rlm-harness pin. A resumed session must not re-open a
+   seam that moved.
+6. **The user's in-flight intent and acceptance criteria.**
+
+Do not preserve anything already in `AGENTS.md`, `README.md`, `.env.example`, or `pyproject.toml`; tool
+output, file listings, or file contents readable from disk; or exploration narration that led to no
+decision.
+
+Format (keep it under ~40 lines):
+
+```
+## Session state
+- Goal: <one sentence>
+- Status: <what passes, what doesn't, last command + result>
+- Seams: <only the ones touched, one line each>
+
+## Decisions
+- <decision> — <why>   (→ AGENTS.md / topic rule / commit message)
+
+## Changed
+- <path:symbol> — <what & why>
+
+## Open
+- [proposed|accepted-not-done|rejected] <item>
+```

diff_sentry-0.6.0/.claude/rules/indicators.md

@@ -0,0 +1,281 @@
+---
+paths:
+  - "diff_sentry/indicators.py"
+  - "diff_sentry/normalize.py"
+  - "tests/corpus/**"
+  - "tests/test_indicators.py"
+  - "diff_sentry/hosts.py"
+  - "tests/oracle/**"
+  - "tests/test_oracle.py"
+  - "tests/test_oracle_harness.py"
+  - "diff_sentry/shelllex.py"
+  - "diff_sentry/linewide.py"
+  - "tests/test_shelllex.py"
+  - "tests/test_enclosure.py"
+  - "tests/test_linewide.py"
+  - "tests/test_gate_edges.py"
+  - "tests/test_metadata_rules.py"
+  - "tests/test_adversarial_input.py"
+  - "tests/test_detection_quality.py"
+  - "tests/test_incident_*.py"
+---
+
+# Indicator rules
+
+## Rules that hold for every indicator
+
+- **Deterministic, pure Python, no subprocess.** A subprocess spawned inside the live dspy.RLM/asyncio process
+  hangs; a heavier scanner belongs host-side, after the run. `mint_id` is a sha1 over rule and evidence (no time,
+  no randomness), so a baseline hit and an in-loop re-scan of the same content dedupe to one union member.
+  Evidence is a bounded snippet (`_MAX_EVIDENCE`), never the whole diff.
+- **Severities are tuned, not paranoid.** A plain workflow-file edit is `medium`, below the signal floor on
+  purpose: a benign workflow PR must not force a SIEM signal, and a real payload inside the workflow fires the
+  high/critical shell and obfuscation rules itself. A CODEOWNERS reassignment is `high`. A bare
+  `pull_request_target` (the ordinary label-bot shape) is `medium`; the same trigger checking out the PR head is
+  `pwn-request` / `critical`. `dev-tunnel-endpoint`, `content-addressed-host` (IPFS) and `dynamic-code-eval` are
+  sub-floor corroborators; pairs with no innocent reading (`child_process` + `detached`, network-fetch +
+  file-write) are `high`. Requiring both halves is how a common primitive becomes evidence without becoming noise.
+- **Every rule ships with its negative case**, and `tests/corpus/` pins the suite's hit/miss behavior.
+- **`pwn-request` has three checkout forms, each requiring the privileged trigger:** the head expression on a
+  `ref:` (directly or one binding hop away), a hand-rolled `git fetch` / `gh pr checkout`, and
+  `allow-unsafe-pr-checkout: true` (the actions/checkout v5.1+ opt-in; the line states the intent, so no dataflow
+  is needed). Each form has its own title. A privileged trigger is recognised in the common YAML spellings (map
+  key, `on: name`, flow list or map, block-list item, quotes, anchor, trailing comment). `issue_comment` counts
+  beside `pull_request_target` and `workflow_run` (it runs with the base repository's secrets), but that trigger
+  alone is not a finding; `pull_request_review` and `pull_request_review_comment` are not privileged.
+- **A paired rule only pairs within one file.** `scan_diff` splits a unified diff on its file headers and
+  `scan_content` does the same for an event. `normalize.content_segments` is the single definition both
+  `raw_content` and the baseline build on. Per-file scoping stops two halves from different files composing into a
+  `critical` no file contains, and keeps a hit id byte-stable between a whole-diff and a single-file scan, which is
+  what lets the baseline and an in-loop scan dedupe. Non-diff input falls through to a whole-text scan.
+- **The in-loop tool registers as exactly `scan_indicators`.** dspy registers a tool under its `__name__` and the
+  prompt calls `scan_indicators(region)`; the inner callable is renamed on purpose (`make_indicator_tool`).
+- **Hits are capped at 25 per (rule, place, title)** (`_MAX_HITS_PER_RULE`), most severe first, so decoys cannot
+  evict the critical that follows them. The title is part of the key because `curl-pipe-shell` has three forms. The
+  SIEM payload lists the 100 most severe and says how many there were (`indicator_count`).
+- **A detector that raises is a finding, not a silence.** `scan_indicators` runs each detector in `_DETECTORS`
+  under a guard; one that raises records a `high` `scanner-error` hit and the rest still run.
+  `tests/test_metadata_rules.py` feeds thousands of random hostile strings through the scanners and requires that
+  none raises. The prose softening and the rendered view run under the same kind of guard (`_or_closed`): a crash
+  keeps the installer as strict as the detector found it and adds a `scanner-error` hit.
+- **Every regex is linear on hostile input.** The watcher runs these rules on any user's text (up to 65,536
+  characters in a body) on one poller thread, and a regex that backtracks does not return. Habits: `[ \t]*` after a
+  line anchor, never `\s*`; no two adjacent quantifiers that can take the same character; alternatives disjoint
+  on their first character; every unbounded `[^\n]*` in front of another pattern gets a `{0,N}` window; host
+  patterns anchored and label-bounded. `tests/test_adversarial_input.py` times named vectors and, in a subprocess,
+  runs 65,536-character inputs built from the literals of every module-level regex in the package. A regex written
+  inside a function is not picked up: compile it at module level.
+- **The change text is scanned once per change in the poller.** `cli._baseline_indicators(event, content_hits=...)`
+  and `cli.screen(..., baseline=...)` take the result of the one `scan_change`.
+- **Command words are bounded by ASCII alphanumerics, never `\b` or `\w`**, which know Unicode: `__curl ... | sh__`
+  (Markdown emphasis) and `執行curl ...` (CJK prose has no spaces) must still be found.
+- **`invisible-unicode`** covers bidi overrides, embeddings and isolates; the word joiner and invisible math
+  operators; the Hangul fillers; a ZWSP anywhere but right after `@` (Dependabot renders `@​user`); a BOM
+  anywhere but the start of a line; Unicode tag characters (except the three subdivision flags); the
+  variation-selector supplement; and three or more plain variation selectors in a row.
+- **`branch-name-injection` is `high` unless the author is an OWNER, MEMBER or COLLABORATOR** (then `medium`): a
+  maintainer can name a branch `fix/handle_${HOME}_expansion`.
+
+## `curl-pipe-shell`
+
+A download piped into a shell or an interpreter is `critical`. It has three forms that are judged separately: the
+pipe, the command substitution and PowerShell's. In prose (a `title`, `body` or `comment`, or a documentation file:
+`.md .mdx .markdown .rst .adoc .ipynb .html .htm`) a documented installer is `medium`. Code files (`.sh`, `.go`,
+`.yml`, a Dockerfile, `requirements.txt`) and branches are never softened. The segment kind decides, never a label.
+The planner's `scan_indicators` applies the same softening to the region it is given.
+
+### What it matches
+
+- **The pipe form.** The shell is `sh bash zsh dash ksh ash fish csh tcsh` by name or path (quotes or backslashes
+  threaded through the letters are fine), `$SHELL`, or one of them behind `env`, `busybox`, `command`, `exec`,
+  `nohup`, `nice`, `timeout`, `stdbuf`, `sudo`, `doas`, `pkexec` or `su -c`. An interpreter that reads its program from the pipe
+  (`python3 -`, `perl`, `ruby`, `node`, `php`) counts too, but not `python -m`, a script file or `-c` / `-e`. The first
+  pipe into a shell is the sink; up to three stages in between; the download word may itself be quoted or split by a
+  backslash-newline; up to 300 characters of arguments sit between it and the pipe, after the padding that would
+  push a real command out of that window has been folded. The window reads quoted strings (which may run over a line),
+  `$'...'` as an ANSI-C string, and a backslash outside a string as bash does (`\'`, `\|` and `\$` are characters of a word,
+  `curl -H 'a<newline>b' URL \' | sh`). A second window reads `$` as a plain character (after `\$` or `$$`, and in dash,
+  `$'` starts no ANSI-C string), and a third ignores quotes altogether (it finds a Markdown table cell's `\|`). A match of
+  any of the three counts.
+- **The command substitution form:** `bash -c "$(curl ...)"`, `bash <(curl ...)`, `source <(curl ...)`,
+  `eval "$(curl ...)"`, `python3 <(curl ...)`, here-strings, with the usual prefixes.
+- **The PowerShell form:** `iwr ... | iex` (`curl` and `wget` are aliases of `iwr` there), `iex (irm ...)`,
+  `iex ((New-Object Net.WebClient).DownloadString(...))`, `[scriptblock]::Create(...)`.
+- A backslash-continued command and a pipe at the end of a line are joined before matching (`_join_lines`), in the
+  scan and in the softening alike. A line that looks like a table row but is not one keeps its lines, and its
+  pipeline is appended at the end of the text; the appended lines are matched as a text of their own (`_matches`). Prose and HTML files are also scanned as rendered (`_rendered`, below), and the
+  hits only that view has are added.
+
+### When prose makes it `medium`
+
+Every download of the form in the segment must meet all of these; one failure keeps the whole form `critical`.
+The form is part of the hit id, and two hits that share an id keep the more severe one (`keep_worse`).
+
+1. **Known installers only** (`hosts.KNOWN_INSTALLERS`: exact hosts, and exact path patterns for shared hosts such
+   as `raw.githubusercontent.com/nvm-sh/nvm/...`). `hosts.installer_only` is strict: every flag is on an allowlist
+   (`-sSfL`, `-o-`, `--proto '=https'`, `--retry N`; `--proxy`, `--config`, `--next` and the like fail), every
+   argument is a clean URL, and a stand-in (`...`, `<url>`) names no download. `powershell_installer_only` applies the same
+   test to PowerShell. Never add a whole platform (github.com, raw.githubusercontent.com) to the list.
+2. **A plain shell side** (pipe form). A bare shell name or one under `/bin`, `/usr/bin`, `/usr/local/bin`, plain
+   flags, no `-c`, no script operand (`sh ./evil.sh`), no `=` or URL; only decode or copy filters between the
+   download and the shell (`base64 gunzip gzip zcat xz tee cat`); an interpreter only for a known installer and only
+   with the switches it documents.
+3. **It stands on its own logical line.** Nothing on the line feeds it (`|`, `=`, `&`, `;`, `$`, `xargs`, `-exec`;
+   only `NONINTERACTIVE=`, `CI=` and `DEBIAN_FRONTEND=` may precede it), and no program runs it elsewhere or in its
+   place (`_wrapped` reads every word in front of the command, after markup, labels, list markers and redirects are
+   dropped). The command starts within `_MAX_PROSE_LEAD` characters of its line or inside an inline code span, and
+   the rest of the line is prose.
+4. **Nothing earlier in the segment changes what runs** (`_HIJACK_RE`): a curl config, an alias or function named
+   `curl` or a shell, an assignment to `PATH`, `HOME`, `LD_*`, `BASH_ENV`, `SSL_CERT_*`, a URL assigned to any
+   variable, a proxy variable, a write to `/etc/hosts`, PowerShell's `Set-Alias`. This is a brake on false positives,
+   not a proof: what defeats it is still recorded at `medium` and handed to the planner as evidence.
+5. **The installer is not enclosed** by anything a wrapper opened above it, and the text has no shape the
+   softening does not read (both below).
+
+### Is the installer enclosed?
+
+`ssh evil.example 'NEWLINE curl ... | sh NEWLINE '` runs the installer inside ssh's argument, so a documented installer
+stays `critical` when something above it opens a wrapper's quote, here-document or substitution. `enclosed` in
+`_soften_prose` asks two readings of the folded join; **either one** keeps the installer `critical`.
+
+- **The line-wide floor (`linewide.py`)** is the line-wide rule that preceded the lexer, kept unchanged. A line
+  opens something when its last 300 characters (minus quote, list and prompt markers) hold an odd number of
+  apostrophes (one between two letters does not count), an odd number of double quotes or a `<<`, and a word in it
+  could run what follows (`linewide.wrapped`: a wrapper, an interpreter, a path, a name glued to the command). The
+  60 lines after an opening line are enclosed. The floor also asks `wrapped` of the text in front of the command on
+  its own line. It counts the quotes the text spells, so it sees the ones a YAML scalar, a string literal, `printf`, a
+  variable or Markdown rendering turns into the quote a shell is handed, which the lexer cannot. It knows no fence,
+  removed line, closed string or syntax error. A text with a NUL is read with and without it.
+  - **Guarantee:** what the line-wide rule rates `critical` through its opening-line rule or lead check is never
+    softened. Exceptions: the removed lines, hunk headers (not the section heading after the second `@@`, which is a line of the
+    new file) and `\ No newline` notes of a patch (the rendered view is of the new file, so a line break it makes
+    carries the marker of its line), and a NUL anywhere in the text (it is read without the NUL, as the shells read it).
+  - **Do not narrow the floor or relax the lexer's answer on top of it.** Test the lexer's relaxations through
+    quote-free openers (`ssh evil.example $(`).
+- **The lexer (`shelllex.py`)** is a one-pass quoting lexer with a stack of open constructs (quotes, `$'...'`,
+  backticks, `$(...)`, `<(...)`, `(...)`, `$((...))`, `${...}`) and a queue of pending here-documents, held to
+  `bash -n` and `dash -n` by the oracle. What the two shells read differently, or what needs a parser, is `unknown`
+  (the docstring lists them) and counts as enclosed on the lines after the first one on which a command may begin like
+  a wrapper's. A construct counts only when the command that holds it is a wrapper and does not read as prose
+  (`_begins_with_wrapper`, `_wrapped`): `ssh evil.example '`, `docker run x sh -c "`, `ssh evil.example <<'EOF'`. A
+  capitalised first word, an apostrophe, a URL, or a word such as `the a an is it to of use run` makes prose, and a
+  plain-word host (`ssh host '`) reads as prose.
+  - The diff's `+` and a blockquote's `>` are removed, and the text is read in each stretch between code fences. A
+    fence is a line of three or more backticks or tildes, at most three blanks in, with nothing in front of it. A
+    fence behind a quote or list marker cuts nothing. The lines of an HTML block are text. A string opened in one
+    block is not open in the next. CommonMark's container rules are not modelled further.
+  - Every reading is also tried from each wrapper word as a fresh start, because prose or a table cell can sit in
+    front of a command (`Run:ssh`, `| docker |`). The 60 lines after it are read as a script of its own.
+  - `eval` runs the script its first string holds, and a string piped into a shell or an interpreter is a script
+    too; both are read as such to a depth of three.
+  - A wrapper-led line whose comment holds a quote is read a second time as a zsh in a terminal reads it, where `#` is no comment.
+  - The enclosure reads are held to a budget of 12 units per character plus 2,000, and a command of more than 600
+    characters is judged from its head and tail. The budget fails closed.
+
+### Shapes the softening does not read, and the rendered view
+
+- **The gate (`_unread_shape`)** is fail-closed: a text with a shape that the softening does not read is not softened
+  at all, and the extras of its rendered view are not either. The shapes are a lone CR, a diff block in a
+  document (a line of the new file that starts with `+` and a non-blank), a notebook array, the strings of a list that
+  hold a download and a program, a multi-line array assignment, a YAML single-quoted scalar that goes on, and a
+  template placeholder that spells a quote. A crash of the check is a shape.
+- **The rendered view (`_rendered`)** reads a text as a reader copies it: entities decoded, comments and tags
+  gone, a block tag a line break, links reduced to their text, emphasis markers removed, a word in backticks reduced to the
+  word, a code span of a quote read as that quote. Its stricter hit stands (stricter wins). `_shield_html_fences` blanks
+  fence markers inside HTML blocks first. A second view without those last two readings is read as well, because a
+  backtick they erase still counts in front of a command (`ssh host <code> ' </code> curl ... | sh`). `.rst`, `.adoc`,
+  `.html`, `.htm` and `.ipynb` are not Markdown, so no fence or HTML block cuts there.
+
+### Accepted over-alerts and known gaps
+
+The first four bullets are over-alerts, not misses: for the examples they give, the installer stays `critical`.
+
+- A letter-apostrophe-letter (`change's`) after a path-like first word makes the rest of that stretch of prose
+  enclosed (`/format the change's notes`, `bin/evil doesn't work.`). Markup hides a relative path from the raw
+  reading, but an absolute path (`/usr/bin/x`, `./x`, `~/x`) or a typographic quote hides nothing, and the rendered
+  view reads a code span or emphasis away, so `` `src/main.py` doesn't read the config. `` before an installer is
+  `critical`. A shell would read the apostrophe as an unterminated string.
+- A line led by a wrapper that runs what follows whose comment holds a quote (`uv run pytest  # run the project's tests`;
+  not `npm test` or a plain-word host), and prose with an unmatched quote or backtick next to a bare path or a wrapper head.
+- A table row with a quote, a closer or a stray `$(` or backtick opener followed by a paragraph joins that paragraph; a cell
+  after a command written as plain text or `<code>` is a stage of its pipeline; a `+` row inside a fenced `diff` block reads
+  as shell text.
+- A tilde fence (the rendered view erases `~~~`), and any text with a gate shape.
+- Known gaps: a hunk or text that starts inside a construct opened above what is visible; indented code read as
+  prose; commands that are not known wrappers (`awk`, `parallel`, `aws ssm`, `csh`, `lua -e`);
+  a patch whose removed lines or hunk headers sit between `curl U |` and `sh`; single-line sink forms such as
+  `| >/dev/null sh`, `| { sh; }`, `| setsid sh`, `| xargs -0 sh -c`; a comment word followed by a backslash; a
+  here-document on a multi-line compound command; a redirect after a here-document delimiter; more than about 300
+  characters of distinct words between `curl` and the pipe; a quoted-space argument (`' '`) that the padding fold turns into
+  a lone quote (`" "` and a string that ends in a blank are folded the same way), together with a quoted argument that runs
+  over a line; an argument that is a multi-line `$(...)`, backtick span, `${...}` or `<(...)` in front of the pipe; a quoted
+  `|#` among the arguments (the rest of the line reads as a comment); a `$'` that is an ANSI-C string in one place and `$$` followed by a quote in another. The quote-aware window reads a
+  backslash as a shell does and must keep doing so: reading `\'` as a plain character would catch some of these by
+  accident, but it closes strings a shell never opens, and the window built that way backtracked exponentially on
+  `$'a'` chains and cost hundreds of microseconds per character on `\"` chains.
+- A table cell spells its pipe `\|`, and that is read as a pipe only in a table row that starts with a pipe; elsewhere `\|`
+  is an argument of the command (`curl \| URL | sh` is `critical`). So an installer written with `\|` in a table without
+  leading pipes, in a paragraph or in an HTML cell is `critical` too, although Markdown renders the `\|` as a pipe there. A
+  line that starts with a pipe inside a multi-line quoted argument is read as a table row, and its `\|` as a cell's pipe.
+- A lone CR is a character of a word to a shell. The shell rules read the text once more with each lone CR as a letter
+  (`curl URL<CR>| sh` runs; `\<CR>` is an escaped CR), also for the planner tool and `diff_sentry scan`, which read a
+  file or stdin with the CR kept.
+
+### Measured
+
+- **Over-alert rate.** The uv installer inserted as its own paragraph into 2,400 real Markdown documents (7,200
+  insertions): about 2.5% read `critical` where the line-wide rule alone said `medium`, and none read `medium` where it
+  said `critical`. About a quarter of those come from a gate shape (under 1% of documents hold one). Without the
+  floor, about 3% of the insertions would have been softened that the line-wide rule kept `critical`. An installer
+  whose own line carries a quote (the rustup one) over-alerts more, roughly one insertion in eight on a similar corpus,
+  because a plain apostrophe in the prose above pairs with that quote.
+- **Cost is linear.** The worst shape (`curl(` repeated) costs about 10 µs per character per channel (0.7 s for a 65 KB body,
+  about 20 s for a 2 MB patch); the hostile enclosure and gate shapes cost 2 to 17 µs per character; plain prose is
+  about 2 µs and a README 8 to 14. Tests hold the cost per character flat from 16 KiB to 128 KiB.
+- **The lexer agrees with real shells.** The oracle (`DS_ORACLE=1`) has found no case of the lexer calling a construct closed
+  that `bash -n` or `dash -n` reports open since the last fix.
+
+### How it is verified
+
+- **The oracle** (`tests/test_oracle.py`, opt-in, Docker): seeded candidates run under the container's real `bash`
+  and `dash` with stub `curl`/`wget` and stub programs that only log, no network, no capabilities, a read-only root,
+  and a canary download before and after each batch. Four properties: recall in code files, recall in prose
+  (whatever a shell really runs stays `critical`), the lexer against `bash -n` / `dash -n`, and soundness of the
+  softening (whatever is reduced to `medium` fetches nothing but a known installer and calls nothing else). A new
+  spelling goes into `tests/oracle/grammar.py` first. The oracle sees under-detection and unsafe softening, never extra
+  alerts, and does not cover PowerShell, zsh, or effects only the real installer would show; guards that only add
+  alerts are pinned by unit tests instead.
+- **Unit pins.** `tests/test_linewide.py` pins the floor against its source rule; `tests/test_gate_edges.py` pins the
+  gate, rendered view and shield patterns by the shortest texts on which a mutant of each differs;
+  `tests/test_enclosure.py`, `tests/test_shelllex.py` and `tests/test_metadata_rules.py` pin the rest end to end.
+  A mutant that no text tells from the code is an equivalent one (an alternative another covers, a possessive
+  quantifier, a class member under `re.IGNORECASE`, a cost bound).
+
+## Other prose softening
+
+All in `indicators._soften_prose`; a code file and a branch are never softened.
+
+- **`prompt-injection`** whose only trigger is the bare phrase "system prompt" is `medium`: ordinary in a thread
+  about an LLM product. It is `high` when:
+  - a real override phrase is present (`ignore previous instructions`, `your system prompt`, `print the system
+    prompt`, `new instructions:`, and the previous / prior / earlier / the above family), which must name what is
+    dropped (instructions, rules, guidance, context) or ask for an action (approve, merge, lgtm, reply, print,
+    reveal, send, exfiltrate, grant, delete, execute). "Ignore all previous commits" and a bare "ignore the above"
+    are ordinary sentences;
+  - an imperative starts a sentence aimed at the phrase (`Ignore the system prompt ...`, `New system prompt: ...`):
+    the verb follows the start of the text or a line, punctuation, a quote, a markdown or diff marker, a table pipe,
+    a bullet, a vocative, or `could you` / `can you` / `you need to` / `you will`; never a hyphen, an apostrophe or a
+    word;
+  - it is addressed to a reviewer or a model: `please`, `you must/should`, or a name (`agents`, `Claude`, `the AI`,
+    `reviewers`) with a directive modal, then the verb. A name without a modal describes them. Not through a negation.
+
+  It stays `medium` when the verb starts a library title with a subordinate clause (`Ignore system prompt when
+  counting tokens`), when the phrase starts a longer name (`system prompt caching`) or is a whole table cell, and
+  when negated, except that a clause about the review itself (`when reviewing this PR`, `and merge`) makes it an
+  instruction. A variable (`SYSTEM_PROMPT`), a flag or a path is not the phrase.
+- **`pwn-request`** is `medium`: a workflow shown in a body, a comment or a documentation file is not one that runs.
+  The same text in a workflow file stays `critical`.
+- **`diff-viewport-evasion`** is `medium` in a comment only (a padded markdown table). In a file, including a
+  markdown file, it stays `high`.
+- A planner re-scan of text that is not byte-identical to the baseline's window mints another id (a title is scanned
+  with the author appended by the baseline), and the stricter baseline hit for a code file always stays in the union.

diff_sentry-0.6.0/.claude/rules/monitoring.md

@@ -0,0 +1,106 @@
+---
+paths:
+  - "diff_sentry/watch.py"
+  - "diff_sentry/actors.py"
+  - "diff_sentry/snapshot.py"
+  - "diff_sentry/ingest.py"
+  - "diff_sentry/hosts.py"
+  - "diff_sentry/dbutil.py"
+  - "tests/test_watch.py"
+  - "tests/test_actors.py"
+  - "tests/test_snapshot.py"
+  - "tests/test_hosts.py"
+  - "tests/test_ingest_metadata.py"
+  - "tests/test_metadata_rules.py"
+  - "tests/test_campaign_replay.py"
+  - "tests/test_cli_watch.py"
+  - "tests/corpus/hackerbot_claw_campaign.json"
+---
+
+# Monitoring: the watcher, the actor store, the archive
+
+- **All of it is host-side.** `watch`, `actors`, `snapshot` and the ingest enrichment run outside the
+  trajectory. The planner never queries the store, the queue, or GitHub. The one thing it receives is the
+  `actor_history` summary (repositories, outcomes, rule names, hours, and which profile FIELDS changed, never
+  the profile text), rendered as an untrusted section because the logins and refs in it came from users.
+- **Cross-event facts are provenance, not a second signal.** `ActorStore.correlate` is read-only and runs
+  BEFORE `record`. Its output lands in `event["provenance"]` under plain keys (`actor_*`, `ioc_reuse`,
+  `content_revisions`, `evidence_scrubbed`) and `indicators.scan_provenance` turns them into baseline hits.
+  Never add a hits or signal path from the store. A change scanned on its own carries none of those keys
+  and fires none of those rules.
+- **A deleted account is not an actor.** GitHub's `ghost` is one identity for every deleted user, so `actor_key`
+  keys a `ghost` change on the change itself: two unrelated deleted authors never share a history.
+- **Examine oldest first.** Correlation reads what came before a change, so the listing (newest first) is
+  sorted ascending before it is examined. A burst examined newest-first would show its last member nothing.
+- **Attribution follows the author of the text, decided structurally.** `IndicatorHit.via` names the commenter
+  and is set from the segment KIND in `normalize.content_segments_ex`; it is never parsed out of `location`,
+  which is a free-form file path an author can name `comment:alice`. A hit whose `via` is another account is
+  recorded against that commenter (`channel='comment'`, only when it reached the floor or fired a slash
+  command), and `correlate` reads only the author's own hits, so a drive-by's payload host does not link the
+  maintainer's next change and a moderator deleting the comment is not the author scrubbing evidence. A verdict
+  on a change whose floor-level evidence is only a third party's goes to that party (`finalize`).
+- **A verdict clears an inference, never evidence.** (`actor-known-bad` is the one model-derived fact that
+  enters the baseline: it needs a stored `malicious` verdict, expires after 30 days, and does not apply to
+  `Bot` accounts.) A `benign` verdict clears the account-level flag
+  (`actor-prior-flag`, an inference about a person), except over a prompt injection, where the verdict may be
+  the injection working (`ActorStore._stands`, the only implementation). It never clears `ioc-reuse`, the
+  payload-host link: a local model has a real false-benign rate, and letting its verdict erase evidence would
+  let one missed payload launder a campaign. `actor-known-bad` needs a `malicious` verdict. The account-level
+  rules do not apply to `Bot` accounts; the change-level ones do.
+- **The first copy is kept.** `SnapshotStore.put` runs before anything else touches a change; objects are never
+  rewritten and the index is append-only. An edit is a new revision with its own run id (`-r1`), so its trace never
+  overwrites the original's, and `prior_indicators` carry the earlier hits forward so a removed payload cannot
+  remove the evidence. `evidence-scrubbed` fires when a floor-level hit of an earlier revision is gone (matched by
+  rule and location, not id; a payload rewritten to read as prose counts as gone) and names the vanished hit ids,
+  so a later edit that still lacks them is the same finding and does not page again. Snapshot the event as
+  ingested, before correlation attaches facts, so the archive holds what GitHub said and not what we derived.
+- **Time is the cost, not money.** With a local model every change reaches the model by default; the screen
+  sets the order (`_priority`), not eligibility. The caps (`deep_min_severity`, `max_queue`,
+  `max_deep_per_hour`) default off. A shed change keeps its `screened/` record and a `shed` state.
+- **The screen is not a trajectory.** `cli.screen` writes `screened/{run_id}.json`, never a trace, so the
+  rollout corpus stays trajectories. Its early alert is `stage=screen`; the run's own is `stage=final`.
+- **One model worker.** dspy's global configuration is bound to the thread that first set it up. The poller
+  is a second thread and touches no model. Do not add a worker pool without proving `setup` is safe there.
+- **A failed change must not block the ones behind it, and must be retried.** A failure holds the watermark below it
+  and withholds the page-1 ETag (otherwise the next listing answers 304 and it is never retried), backs off
+  exponentially, and after `_MAX_FAILS` is parked as `error` so it stops holding the listing. The others are skipped
+  by their stored `updated_at`. A listing deeper than `_MAX_PAGES` is reported as truncated, never advanced past
+  silently. A rate-limit refusal (`429`, or `403` saying so) is not a failure of the change: wait for the reset, and
+  `_ingest` subtracts the calls it made from the budget the list responses report.
+- **A read that failed is not a change.** `_comments` returns None on failure and the event carries
+  `comments_incomplete`; the watcher then keeps what it has instead of judging the empty thread as an edit.
+  Anything that feeds the digest must distinguish "none" from "could not read".
+- **Run ids are unambiguous.** `owner__name-pr-7` (`-r1` per revision). Owners cannot contain `_`, so two
+  repositories never collide; the one-off `pr`/`issue` commands keep their older `owner-name-pr-7` scheme, which can.
+- **A failed model run is retried, and pages nothing.** It backs off up to `_MAX_ATTEMPTS` (about four hours, so a
+  local model restarting does not fail the queue) with `cli.run` told `emit_failed=False`, then is buried
+  (`claimed=2`, event kept) for `--requeue-failed`. Claiming counts as an attempt, so an input that kills the process
+  cannot loop across restarts. The queue row stays claimed until the run completes, and `recover()` (called by
+  `serve` and `--once`, never by a read-only look) requeues it under a per-directory `flock`. Neither thread may die
+  on an unexpected exception.
+- **Undelivered alerts are retried a few at a time.** `_retry_alerts` re-sends at most `_ALERT_RETRIES_PER_POLL`
+  (5) per poll, so a SIEM that hangs (a POST is bounded by 15 s) cannot stall polling for a whole backlog.
+- **An alert counts once delivered.** `changes.alerted` gains ids only after the SIEM answered 2xx
+  (`EmitResult.ok`); a refusal or outage leaves `alert_pending`, re-sent from `screened/{run_id}.json` on later
+  polls. `emitted` alone is also True for a 5xx.
+- **State files are versioned.** `dbutil.open_versioned` stamps `PRAGMA user_version` and refuses a file with
+  tables but another stamp; there is no migration, and the error says to move the directory aside.
+- **Account age is measured as of the event.** `_age_days(created_at, as_of)` takes the event's own time;
+  the wall clock is only the fallback for a payload with none. A replay of a historical incident must
+  carry `created_at`, or every old attacker looks established (`test_campaign_replay` pins this).
+- **Attacker-authored strings stay out of the MF1 sandwich.** Profile fields, branch names, and comments
+  are bounded at ingest and rendered in the untrusted section only. Comments are text-scanned only when
+  the change's author or an outsider wrote them and the commenter is not a `Bot`: maintainers quote
+  install commands and release notes constantly.
+- **Data minimization.** The `email` fields of a profile and of commits are reduced to a domain in the event
+  and masked in the archive (`snapshot.scrub_emails`). Free text is archived as captured: it is evidence.
+  Nothing prunes the archive, the store, or queued events.
+- **Adding a fact:** capture it in `ingest.py` (best-effort: a failed call omits the key, unless the value feeds
+  the digest, in which case mark the event incomplete instead of reporting "none"), put it in
+  `provenance` (or a bounded event field), read it in `scan_provenance` / `scan_metadata` with a negative
+  case, and keep it sub-floor unless it is standalone evidence. Replay the incident corpus.
+- **Read-only means read-only.** `--stats` and `actor` open the state `mode=ro` (`open_versioned(read_only=True)`)
+  and neither create, stamp, nor requeue anything. Only `serve` and `--once` call `recover()`, under the lock.
+- **Terminal output is sanitized.** `format_dossier` replaces control characters; `_json_out` escapes the C1 range
+  `json.dumps` leaves raw; URL and token parsing cut at the first control character. A string an attacker wrote
+  must not be able to drive the operator's terminal.

diff_sentry-0.6.0/.claude/rules/rubric-and-eval.md

@@ -0,0 +1,42 @@
+---
+paths:
+  - "diff_sentry/rubric.py"
+  - "diff_sentry/rl_export.py"
+  - "diff_sentry/response.py"
+  - "diff_sentry/assemble.py"
+  - "eval/**"
+  - "studio/**"
+---
+
+# ATLAS rubric labels and the eval member
+
+## The rubric is a reward-free label surface
+
+- `rl_export.rubric_signal` attaches the ATLAS 4-category (TF/TA/TG/PA) decomposition as LABELS: a fixed,
+  no-LLM skeleton in `run_start` meta (`rubric_to_meta(default_rubric())`, one criterion per category,
+  since the task is constant; there is no `generate_rubric` path) plus deterministic per-criterion
+  `criteria_facts`.
+- The facts are a re-lens over `run_labels` / `run_metrics`, not a second derivation: `rubric.trace_facts`
+  reuses them, so a criterion's `observed` cannot drift from what a trainer reads. `_CATEGORY_LENS` maps
+  TF↔(verdict / signal / hit_iteration_cap), TA↔(scan / deep_classify / analyst / fetch / skill counts +
+  circuit-breaks + analyst failures), TG↔(indicator_count / max_indicator_severity / signal /
+  cited_unknown), PA↔(verdict / cited_unknown).
+- `CriterionFact` has no score/met field; the trainer scores. Never add a reward/score/met field to any
+  rubric type.
+- `rubric.py` is a dspy-free leaf: it imports only `.schema` at top, and its `rl_export` reuse is a
+  function-level import (that path is dspy-free too), so `response.py` can import it at top.
+- `response._rubric` surfaces it as `DetectionResponse.rubric` (`RubricReport`); the studio shows it as a
+  "labels — not a score" card that adds no judgement. Keep the card legacy-guarded: a response without a
+  `rubric` renders nothing.
+
+## The eval member measures; it never rewards
+
+- `eval/` (`diff-sentry-eval`) scores a recorded run's assembled verdict with a fixed external ATLAS
+  LLM-as-judge: TF/TA/TG/PA on 0–10, TF primary, per-category means only, no composite, no threshold.
+- It is a one-way reader: it reaches diff-sentry only through `verdict_from_events` / `run_labels` /
+  `run_metrics` / `AssembledVerdict`, and `diff_sentry` never imports `diff_sentry_eval`
+  (`eval/tests/test_boundary.py` enforces it).
+- The judge is rubric-free (it never reads `rubric_signal`, which would bias the measure) and holds
+  read-never-execute: it assesses the classification statically and treats the change as untrusted.
+- A run it cannot score (never finalized, no usable verdict, judge failed) is `unscored`, never a fake 0.
+  See `eval/README.md`.

diff_sentry-0.6.0/.claude/skills/release/SKILL.md

@@ -0,0 +1,37 @@
+---
+name: release
+description: Cut a diff-sentry release — bump the five version sites together, write the CHANGELOG entry, and publish the GitHub Release that ships PyPI + the Marketplace Action. Use when asked to release, bump the version, or tag vX.Y.Z.
+---
+
+# Cut a diff-sentry release
+
+A release moves FIVE version sites together, plus a `CHANGELOG.md` entry:
+
+1. `pyproject.toml` `[project].version`
+2. `diff_sentry/__init__.py` `__version__`
+3. `action.yml` `inputs.version.default` (decides which PyPI release a consumer's `uses: …@vX.Y.Z` installs)
+4. `README.md` quickstart `uses: qazbnm456/diff-sentry@vX.Y.Z`
+5. `uv.lock`, via `uv lock`
+
+`release.yml` fails the build unless the tag, the built wheel's version, the installed `__version__`, and
+the `action.yml` default all agree, and it checks that the installed console script flags a known-malicious
+diff. Nothing gates the README `uses:` line: a stale one sends new adopters to the previous release, so
+check it by hand.
+
+## Steps
+
+1. Freeze the boundary: `git log v<last>..HEAD`. Every `feat:` / `fix:` / `deps:` commit inside it is a
+   CHANGELOG candidate.
+2. Write the `CHANGELOG.md` entry in the file's existing style: one sentence per user-visible outcome,
+   breaking changes and required actions explicit, no debugging story.
+3. Bump sites 1–4, run `uv lock`, then verify: `uv run pytest`, the eval suite
+   (`uv run --package diff-sentry-eval --extra dev python -m pytest eval/tests`), `uvx ruff check .`.
+4. Commit (`chore: release X.Y.Z`) and push to `main` once the user approves. CI's `consumer install
+   path (PyPI)` job then goes RED on this commit, by design: it installs `action.yml`'s default version
+   from PyPI, which does not exist until step 5. Every other job must be green before you continue.
+5. Create a DRAFT release (`gh release create vX.Y.Z --draft --target <release sha> --title … --notes-file
+   …`, notes = the CHANGELOG section). The user publishes it in the web UI with "Publish this Action to
+   the GitHub Marketplace" ticked (the CLI cannot tick it). Publishing is what builds, verifies, and
+   uploads to PyPI via Trusted Publishing, and the upload is irreversible.
+6. After `release.yml` succeeds, re-run the failed consumer-install job
+   (`gh run rerun <ci run id> --failed`); green proves the published package installs and scans.

diff_sentry-0.6.0/.env.example

@@ -0,0 +1,67 @@
+# diff-sentry: model roles and runtime knobs. Copy to .env; a live run needs these plus a Deno sandbox.
+
+# --- Model roles (required for a live `pr`/`issue`/`classify` run) ---
+# planner: a cheap, injection-resistant orchestrator that holds the diff in the REPL and triages.
+DS_ROOT_LM=your-planner-model
+# analyst: an expensive model reached via llm_query for a subtle source->sink (short questions only).
+DS_SUB_LM=your-analyst-model
+# One OpenAI-compatible proxy serves all roles. When DS_BASE_URL is set the model names are the plain
+# ids your endpoint serves (e.g. qwen/qwen3-next), not litellm-prefixed.
+DS_BASE_URL=https://your-openai-compatible-endpoint/v1
+DS_API_KEY=sk-...
+
+# --- OR: run planner + analyst on a Claude Pro/Max subscription (no API key) ---
+# A role with the `claude-agent-sdk/<id>` prefix runs on your Claude subscription through the Claude
+# Agent SDK (`rlm_harness.ClaudeAgentLM`, the `subscription` extra). Each call is a pure completion:
+# no tools, no filesystem, no settings leakage. Prerequisites:
+#   - `uv sync --extra subscription`
+#   - the Claude Code CLI installed and logged in (`claude` then /login, or `claude setup-token` and
+#     export CLAUDE_CODE_OAUTH_TOKEN)
+#   - `unset ANTHROPIC_API_KEY`: the CLI prefers it over the subscription and would bill API credit,
+#     so the adapter refuses to start while it is set
+# Prefer full model ids over aliases, which drift.
+#   DS_ROOT_LM=claude-agent-sdk/claude-sonnet-5     # planner on Sonnet 5
+#   DS_SUB_LM=claude-agent-sdk/claude-fable-5       # analyst on Fable 5
+# DS_BASE_URL/DS_API_KEY then serve only the classifier, which always needs an OpenAI-compatible
+# endpoint (a subscription-only run is not supported). Set DS_CLASSIFIER_LM to a real model id (plus
+# DS_CLASSIFIER_BASE_URL/DS_CLASSIFIER_API_KEY if it is a separate box). Left unset, it inherits the
+# `claude-agent-sdk/…` DS_SUB_LM and config loading fails with an actionable error.
+# Budgets use the same env vars as proxy mode. Each call adds ~2-5s of CLI-spawn overhead, so keep
+# these runs interactive, not batch. DS_PLANNER_MAX_TOKENS has no effect here: the Agent SDK exposes
+# no token or sampling controls.
+
+# --- Second-stage classifier (the swappable seam) ---
+# Defaults to the analyst model on DS_BASE_URL/DS_API_KEY. Required when the analyst runs on the
+# subscription (see above).
+# DS_CLASSIFIER_LM=your-strong-classifier
+# DS_CLASSIFIER_BASE_URL=https://your-classifier-endpoint/v1
+# DS_CLASSIFIER_API_KEY=sk-...
+# DS_CLASSIFIER_CIRCUIT_BREAK=4
+
+# --- RLM runtime ---
+# DS_INTERPRETER=pyodide       # read-not-execute: the sandbox never runs the change. `deno` is an alias.
+# DS_MAX_ITERATIONS=25
+# DS_MAX_LLM_CALLS=8           # caps only analyst (llm_query) escalations
+# DS_ADAPTER=json              # portable structured output (OpenAI + vLLM/NIM)
+# DS_PLANNER_MAX_TOKENS=16384  # generous, so a reasoning planner isn't truncated to empty content
+
+# --- Enrichment fetch (off by default: an exfil surface on attacker-authored input; GitHub-allowlisted) ---
+# DS_ENABLE_FETCH=false
+# DS_FETCH_ALLOW_CIDRS=198.18.0.0/16   # only if behind a fake-IP proxy / split-DNS VPN
+
+# --- Host-side SIEM emitter (opt-in; deterministic POST after the run) ---
+# DS_SIEM_WEBHOOK_URL=https://your-siem/intake
+# DS_SIEM_TOKEN=...
+# DS_EMIT_ON=suspicious,malicious
+
+# --- Watching a watchlist (`diff-sentry watch`) ---
+# Time is the only cost with a local model, so the caps are OFF unless set.
+# DS_WATCH_REPOS=owner/name,other/name     # or --repos / --repos-file
+# DS_WATCH_STATE_DIR=./output/state        # watch.db, actors.db, snapshots/
+# DS_WATCH_INTERVAL=60                     # seconds between polls (raised if GitHub sends X-Poll-Interval)
+# DS_WATCH_BACKFILL_HOURS=0                # on first sight of a repo, also examine changes this recent
+# DS_WATCH_RECHECK_HOURS=24                # shorten the re-check schedule (15 min, 1 h, 6 h, 24 h); it cannot extend it
+# DS_WATCH_DEEP_MIN_SEVERITY=info          # info = every change goes to the model; raise it to gate on the screen
+# DS_WATCH_MAX_QUEUE=0                     # 0 = unbounded; otherwise the lowest priority is shed and recorded
+# DS_WATCH_MAX_DEEP_PER_HOUR=0             # 0 = unlimited model runs per hour
+# DS_WATCH_RATE_RESERVE=200                # pause polling when the API's remaining budget falls below this