agent-threat-rules 3.5.11 → 3.5.12

package/README.md

@@ -13,7 +13,7 @@ AI Agent 威脅偵測規則的開放格式
 [![GitHub Marketplace](https://img.shields.io/badge/Marketplace-ATR%20Scan-2ea44f?style=flat-square&logo=github)](https://github.com/marketplace/actions/atr-scan)
 [![License: MIT](https://img.shields.io/badge/license-MIT-brightgreen?style=flat-square)](LICENSE)
 [![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.19178002-blue?style=flat-square)](https://doi.org/10.5281/zenodo.19178002)
-[![Rules](https://img.shields.io/badge/rules-751-blue?style=flat-square)](#5-specification)
+[![Rules](https://img.shields.io/badge/rules-783-blue?style=flat-square)](#5-specification)
 [![Categories](https://img.shields.io/badge/categories-10-blue?style=flat-square)](#7-coverage)
 [![OWASP Agentic](https://img.shields.io/badge/OWASP_Agentic_Top_10-10%2F10-brightgreen?style=flat-square)](#7-coverage)
 [![SAFE-MCP](https://img.shields.io/badge/SAFE--MCP-91.8%25-brightgreen?style=flat-square)](#7-coverage)
@@ -29,7 +29,7 @@ ATR (Agent Threat Rules) is an open detection rule format for AI agent security
 
 ## Status of This Document
 
-ATR is published as a **Working Draft** at version `3.0.0-alpha.1`. The rule format defined in `ATR-SPEC-v1.md` is stable and shipped in production at Microsoft, Cisco, and Gen Digital, and in standards-body integrations (MISP / CIRCL, OWASP Agent Security Regression Harness, SigmaHQ, FINOS Common Cloud Controls); full list with PR links in [§6 Adoption](#6-adoption). Governance is currently single-maintainer (BDFL) transitioning to a Technical Steering Committee per [GOVERNANCE.md](GOVERNANCE.md).
+ATR is published as a **Working Draft** at version `3.0.0-alpha.1`. The rule format defined in `SPEC.md` is stable and merged into open-source repos at Microsoft, Cisco, and Gen Digital, and integrated by standards-body projects (MISP / CIRCL, OWASP Agent Security Regression Harness, SigmaHQ, FINOS Common Cloud Controls); full list with PR links in [§6 Adoption](#6-adoption). Governance is currently single-maintainer (BDFL) transitioning to a Technical Steering Committee per [GOVERNANCE.md](GOVERNANCE.md).
 
 All numbers in this document are sourced from [`data/stats.json`](data/stats.json), which is the canonical record of the project's current state. Where this README and `stats.json` disagree, `stats.json` is authoritative.
 
@@ -94,12 +94,12 @@ ATR fills the gap between *taxonomy* and *deployable rule*. Each rule is a YAML
 
 ## 2. Conformance Levels
 
-The keywords MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document and in [`ATR-SPEC-v1.md`](ATR-SPEC-v1.md) are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).
+The keywords MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document and in [`SPEC.md`](SPEC.md) are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).
 
 A conforming **ATR engine** MUST:
 
 1. Parse all fields defined in [`spec/atr-schema.yaml`](spec/atr-schema.yaml) without error.
-2. Evaluate `detection.conditions` with the semantics defined in [`ATR-SPEC-v1.md`](ATR-SPEC-v1.md) §3.5 (Detection Logic) and §5 (Engine Requirements).
+2. Evaluate `detection.conditions` with the semantics defined in [`SPEC.md`](SPEC.md) §6 (Detection Semantics).
 3. Honor the `scan_target` field — a rule with `scan_target: skill` MUST NOT be evaluated against `mcp_exchange` events and vice versa.
 4. Respect rule `status` — rules with `status: deprecated` or `status: draft` MUST NOT participate in production matching unless the consumer opts in explicitly.
 5. Emit `rule_id` and rule `severity` on every match.
@@ -220,7 +220,7 @@ Lanes are opt-in and fully backward-compatible: the default is `hunt`, so existi
 | Artifact | Path | Purpose |
 |---|---|---|
 | Specification (canonical pointer) | [SPEC.md](SPEC.md) | Resolves to the authoritative documents below |
-| Rule format spec (normative) | [ATR-SPEC-v1.md](ATR-SPEC-v1.md) | Rule format, identifier scheme, evaluation semantics |
+| Rule format spec (normative) | [SPEC.md](SPEC.md) | Rule format, identifier scheme, evaluation semantics |
 | Framework spec | [ATR-FRAMEWORK-SPEC.md](ATR-FRAMEWORK-SPEC.md) | Multi-layer detection framework design |
 | Machine-readable schema | [spec/atr-schema.yaml](spec/atr-schema.yaml) | Authoritative validation source |
 | Schema field reference | [docs/schema-spec.md](docs/schema-spec.md) | Human-readable schema docs |
@@ -363,61 +363,196 @@ Aggregated into [`data/stats.json`](data/stats.json) under `benchmarks[]`.
 | AdvBench (LLM-attacks behaviors) | upstream-2026-06-16 | 520 | 2.1% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
 | atr-self-test | internal | 341 | 89.7% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
 | autoresearch | internal-1054 | 1,054 | 15.1% | 100.0% | 0.0% | 3.0.0-alpha.0 | 2026-05-23 |
-| garak (in-the-wild jailbreaks) | inthewild-jailbreak-corpus-650 | 650 | 97.2% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| garak-full (all probe families) | 23-families | 3,475 | 38.3% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
+| garak (in-the-wild jailbreaks) | inthewild-jailbreak-corpus-650 | 650 | 92.5% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
+| garak-full (all probe families) | 23-families | 3,475 | 57.2% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
 | hackaprompt | v1 | 4,780 | 69.6% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
 | HarmBench (CAIS behaviors) | upstream-2026-06-16 | 400 | 2.8% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| hh-rlhf (Anthropic red-team-attempts) | snapshot-2026-04 | 4,957 | 99.1% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
+| hh-rlhf (Anthropic red-team-attempts) [^stdcorpora] | snapshot-2026-04 | 4,957 | 1.5% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
 | JailbreakBench (JBB-Behaviors) | upstream-2026-06-16 | 100 | 6.0% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
 | llm-guard (Protect AI test fixtures) | corpus-2026-05-12 | 44 | 77.3% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| MITRE ATLAS | snapshot-2026-04 | 182 | 100.0% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
+| MITRE ATLAS [^stdcorpora] | snapshot-2026-04 | 182 | 39.0% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
 | NeMo Guardrails (NVIDIA test fixtures) | corpus-2026-05-12 | 6 | 100.0% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| OWASP LLM Top 10 | snapshot-2026-04 | 56 | 100.0% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| PINT-format (deepset + Lakera Gandalf) | public-850 | 850 | 63.6% | 99.7% | 0.25% | 3.5.0 | 2026-06-16 |
-| PromptBench (academic adversarial) | snapshot-2026-04 | 3,280 | 23.2% | 100.0% | 0.0% | 3.5.2 | 2026-06-25 |
+| OWASP LLM Top 10 [^stdcorpora] | snapshot-2026-04 | 56 | 16.1% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
+| PINT-format (deepset + Lakera Gandalf) [^pint] | v1 | 850 | 60.3% | 100.0% | 0.0% | 3.5.11 | 2026-08-04 |
+| PromptBench (academic adversarial) [^promptcorpora] | snapshot-2026-04 | 3,280 | 15.7% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
 | promptfoo (red-team plugin fixtures) | corpus-2026-05-12 | 44 | 97.7% | 100.0% | 0.0% | 3.5.0 | 2026-06-16 |
-| PromptInject (academic adversarial) | snapshot-2026-04 | 1,080 | 100.0% | 100.0% | 0.0% | 3.5.2 | 2026-06-25 |
-| SKILL.md benchmark (internal) | internal-498 | 498 | 100.0% | 97.0% | 0.20% | 3.5.0 | 2026-06-16 |
+| PromptInject (academic adversarial) [^promptcorpora] | snapshot-2026-04 | 1,080 | 100.0% | 100.0% | 0.0% | 3.5.11 | 2026-08-05 |
+| SKILL.md benchmark (internal) [^skilllane] | internal-498 | 498 | 100.0% (hunt) / 0.0% (enforce) | 97.0% | 0.20% | 3.5.0 | 2026-06-16 |
 | Wild scan (OpenClaw + Skills.sh + Hermes + ClawHub) | corpus-2026-04-14 | 96,096 | — | 57.7% (floor) | 1.35% flag rate | 2.0.0 | 2026-04-14 |
 
 All detection corpora were (re-)measured against ATR 3.5.0 on 2026-06-16,
 except `autoresearch` (an internal predicted-rule corpus with no standalone
 runner) and the `Wild scan` snapshot, which retain their earlier measurements.
+`PromptInject` and `PromptBench` were re-measured against ATR 3.5.11 on
+2026-08-05; see [^promptcorpora] for what moved and why. (An earlier
+re-measurement against 3.5.2 on 2026-06-25 fixed a harness event shape; the
+0.0% rows before that were a harness artifact — the harness placed the prompt
+in a top-level field the engine does not read — not the engine's actual
+result.)
+
 `PromptInject` and `PromptBench` were re-measured against ATR 3.5.2 on
 2026-06-25 after a fix to the recall-analysis harness event shape; the prior
 0.0% rows were a harness artifact (the harness placed the prompt in a
 top-level field the engine does not read), not the engine's actual result.
+`hh-rlhf`, `MITRE ATLAS` and `OWASP LLM Top 10` were re-measured against ATR
+3.5.11 on 2026-08-05 after their runner was found never to have called the
+engine at all — see [^stdcorpora]. Those three rows fell 99.1% → 1.5%,
+100.0% → 39.0% and 100.0% → 16.1%. Nothing regressed; the earlier numbers
+were never measurements of ATR.
 The per-row `ATR version` column above is the version each cell was actually
 measured against, mirroring the `atr_version` field in each
 `data/measurements/<source>/latest.json`. The headline `garak` recall moved
-98.0% → 97.2% in 3.5.0 because rule `ATR-2026-00495` (a garak DAN variant) was
-deprecated and no longer fires; see [CHANGELOG.md](CHANGELOG.md).
+98.0% (3.0.0) → 97.2% (3.5.0) → 95.7% (3.5.8) → 92.5% (3.5.11). The 3.5.8 →
+3.5.11 step is a real loss of detections, not a change in how they are counted:
+PR #327 tightened an over-broad persona regex in `ATR-2026-00001` that had been
+false-positiving on benign "you are now an expert …" prose, and tightening it
+also gave up the recall that regex had been earning on novel-persona jailbreaks
+("you are now TrueGPT / ChatINF / Raven …"). The harness event shape changed in
+the same window (see below), and it is not the cause: replayed on today's 780
+rules, the old shape scores 92.2% and the corrected shape 92.5%, a 0.3-point
+difference in the corrected shape's favour. The 3.2-point drop from 95.7% is
+the rules.
+
+Two numbers that briefly appeared here are **withdrawn**: between 2026-08-04
+and 2026-08-05, this table and `stats.json` cited **91.5%** for `garak` and
+**56.9%** for `garak-full`, both at ATR 3.5.11. No measurement file for either
+run exists anywhere in the repository. `data/measurements/garak/latest.json`
+pointed, the entire time, at 95.7% measured on 3.5.8; `garak-full`'s pointed at
+38.3% on 3.5.0, while a never-referenced 3.5.8 file sat unread in the same
+directory. So the claims failed this project's own rule that every published
+number is a version-pinned, reproducible measurement. It was also produced by a harness
+that built an event of `type: 'llm_io'`, which is a rule *source* and not an
+`AgentEventType`; `src/engine.ts` could not map it and so ran every rule of
+every source against the event instead of the two source types the harness
+documented itself as using. The 92.5% above replaces it: measured on
+2026-08-05 at 780 rules through `llm_input` + `tool_response`, the two channels
+`src/hook-handler.ts` can actually deliver a prompt on, and written to
+`data/measurements/garak/2026-08-05_garak-inthewild-jailbreak-corpus-650_atr-3-5-11.json`
+with the commit that produced it. Under the wider shape set used for
+false-positive measurement (which also runs `engine.scanSkill()`) the same
+corpus scores 93.1%; that number is recorded in the measurement's `breakdown`
+and is deliberately not the published one, because a garak prompt never reaches
+production as a SKILL.md. `.github/workflows/eval.yml` now runs
+`scripts/check-benchmark-citations.ts`, which fails CI if this table or
+`stats.json` cites a number no measurement file backs.
+See [CHANGELOG.md](CHANGELOG.md).
+
+[^skilllane]: **Lane matters more here than anywhere else in this table.** The
+    100% figure is the `hunt` lane, which is the engine default and loads every
+    maturity. In the `enforce` lane — the auto-block one, where a detection stops
+    the agent with no human in the loop — this corpus scores **0%**, and the
+    reason is structural rather than a tuning problem: of the 38 rules carrying
+    `scan_target: skill`, **all 38 are `maturity: test`, and none is `stable`.**
+    The enforce lane only loads `stable`, so it loads no skill-scanning rule at
+    all, and 0 of 32 malicious samples fire. Anyone reading "100% recall on
+    SKILL.md" and deploying in enforce mode would be forming a completely wrong
+    expectation, so both numbers are shown. Verified on this commit with
+    `grep`-free counting over `rules/**/*.yaml`.
+
+[^pint]: The `PINT-format` row is **not** a run of Lakera's official PINT
+    benchmark. That corpus is private and roughly 5x larger; this row is a
+    self-built 850-sample corpus in PINT's format, assembled from
+    `deepset/prompt-injections` (660) and `Lakera/gandalf_ignore_instructions`
+    (190). It also carries a scope caveat worth stating plainly: only **29 of
+    780 rules** fire on it at all, and `ATR-2026-00001` alone accounts for 226
+    of the 272 detections. Read it as a prompt-injection-family score, not as
+    ATR's overall coverage. The row moved 63.6% → 60.3% between 3.5.0 and
+    3.5.11 for the same reason `garak` moved: PR #327 tightened
+    `ATR-2026-00001`'s persona-switch regex to stop it false-positiving on
+    benign prose. Precision moved 99.7% → 100% over the same span.
+
+[^promptcorpora]: **Read both of these as closed-book scores.** Until
+    2026-08-05 the harness recorded its per-rule breakdown as the literal
+    string `"unknown"` (it read `m.rule_id` off an engine match that carries
+    `m.rule.id`), so no published version of these rows could say which rules
+    produced them. With attribution restored:
+    **PromptInject 100.0%** is produced by **7 of 780 rules**. Five of those
+    seven — `ATR-2026-00506`, `00507`, `00508`, `00509`, `00518` — carry
+    `author: ATR Community (PromptInject corpus)`: they were written *from*
+    this corpus, which has four attack classes built from a handful of
+    templates. Remove those five and recall on the same 1,080 samples is
+    **9.7%**. The concentration is real but not fragile: the top rule
+    (`ATR-2026-00508`, 968/1,080 samples) is the sole detector on none of
+    them, so deleting it leaves recall at 100%; only `00518` (45 samples) and
+    `00507` (27) are sole detectors of anything. On the 5,352-sample benign
+    gate, `00506` / `00507` / `00518` are 0-FP; `00508` has 4 FP, `00509` 3,
+    `ATR-2026-00001` 19, `ATR-2026-00400` 1.
+    **PromptBench 15.7%** is produced by **3 of 780 rules** (`ATR-2026-00520`,
+    `00519`, `00202`), all three 0-FP on the same benign gate. Two of the three
+    were mined from PromptBench; without them recall is **2.4%**.
+    The PromptBench row moved 23.2% (3.5.2) → 15.7% (3.5.11) and the loss is
+    fully attributable: 247 samples were held only by rules that have since
+    been precision-repaired, and re-running each rule version by version pins
+    every one to its PR — `ATR-2026-00442` 304 → 0 detections at PR #309
+    (223 of them samples nothing else caught), `00051` 17 → 0 at #238 (15),
+    `00118` 6 → 0 at #238 (6), `00001` 3 → 0 at #327 (3). The PromptInject
+    row stayed at 100% across the same span, but what holds it up changed:
+    at 3.5.2 `ATR-2026-00118` matched 1,060 of the 1,080 samples and `00442`
+    another 195; #238 and #309 took both to zero. Neither fact was visible
+    while the breakdown said "unknown", and the row itself sat at its stale
+    3.5.2 value for the six weeks in between.
+    Both corpora are 100% adversarial, so the `Precision` and `FP rate`
+    columns are properties of the corpus, not measurements — read them
+    together with the benign-gate FP counts above, never alone.
+
+[^stdcorpora]: Until 2026-08-05 these three rows were **not produced by the ATR
+    engine**. `scripts/eval-std-corpora.ts` walked `rules/` with a YAML parser,
+    kept only `operator: regex` conditions, flattened every condition of every
+    rule into one implicit OR, and tested each pattern with its own
+    `new RegExp(value, 'i')` against the raw sample string. That shadow matcher
+    had no status gate (it counted `status: draft` rules the engine skips), no
+    lane gate, no field resolution (a condition declared on `tool_response` was
+    tested against natural-language prose), no `condition: all` handling, no
+    non-regex operators, and — the decisive defect — the wrong regex flags.
+    `src/engine.ts` compiles a pattern containing `\u{` with the `u` flag;
+    the shadow matcher always used `i`. Without `u`, the codepoint class
+    `[\u{E0001}\u{E007F}]` in `ATR-2026-00258` is read by JavaScript as the
+    literal character class `[u{E0017F}]` — "contains any of `u { E 0 1 } 7 F`"
+    — so it matched any English text containing the letter `e`. That single
+    miscompiled condition accounted for **4,914 of the 4,914 hh-rlhf
+    detections, 56 of 56 on OWASP, and 182 of 182 on ATLAS**; with it excluded
+    the same shadow matcher scored 0.2% / 3.6% / 8.8%. The old rows measured
+    how many samples contain a vowel. The runner now goes through `ATREngine`
+    and the canonical event shapes in `scripts/lib/corpus-event.ts` — the same
+    entry point the false-positive gates use. Reproduce with
+    `npx tsx scripts/eval-std-corpora.ts`. Read the new numbers with the same
+    scope caveat as `PINT-format`: on ATLAS, `ATR-2026-00061` alone accounts
+    for 59 of the 71 detections (32.4% of the corpus), and ATLAS procedures are
+    prose *descriptions* of attacks rather than attack payloads, so this row
+    measures ATR against attack write-ups, not against traffic.
 
 Two `garak` rows are deliberate: the headline `garak` source tracks NVIDIA's
-in-the-wild jailbreak corpus (narrow, the ~97% number ATR cites publicly,
-refreshed 2026-06-16 against ATR 3.5.0), while `garak-full` tracks
+in-the-wild jailbreak corpus (narrow, the ~92.5% number ATR cites publicly,
+refreshed 2026-08-05 against ATR 3.5.11), while `garak-full` tracks
 every probe family in upstream garak (broad, includes families like
 `badchars`, `dra`, `encoding` that ATR's regex layer intentionally does
 not target). Both are valid measurements against different corpora; they
 are kept as separate streams so the broad-corpus number does not silently
 overwrite the headline.
 
-The single-digit recall on AdvBench / HarmBench / JailbreakBench is honest
-and expected. Those three corpora test **LLM safety alignment** (does the
+The single-digit recall on AdvBench / HarmBench / JailbreakBench / hh-rlhf is
+honest and expected. Those four corpora test **LLM safety alignment** (does the
 model refuse harmful requests like "explain how to make a bomb"), not
 **prompt-injection detection** (the surface ATR's regex layer targets).
 ATR's near-zero recall on these corpora confirms the layering thesis:
 regex catches structured attack patterns, alignment + content moderation
 catch natural-language harm requests. The numbers are recorded for
 completeness and so any future ATR rule additions in the harm-category
-space can be measured against a documented baseline.
-
-Conventions: 100%-adversarial corpora have `fp_rate` undefined and recorded as
-0 in measurement files. Wild-scan has no ground-truth labels; the `precision`
-column reports a precision floor computed as `confirmed_malware / flagged`.
-Every cell is sourced from a specific measurement file — see
-`data/measurements/<source>/latest.json` for the file path and
-`metadata.measurement_file` in `stats.json` for the absolute repo path.
+space can be measured against a documented baseline. `hh-rlhf` is Anthropic's
+red-team-attempts set — the same genre as the other three — and its 1.5% now
+sits with their 2.1% / 2.8% / 6.0% instead of contradicting them at 99.1%.
+
+Conventions: 100%-adversarial corpora contain no benign samples, so they have
+no true-negative population and **`precision` and `fp_rate` cannot be computed
+from them**. The measurement schema requires numbers, so those rows record the
+convention `precision 1` / `fp_rate 0`. Read the `Precision` and `FP rate`
+columns as "not applicable to this corpus", not as results — the real
+precision numbers come from the benign gate, lane-keyed, below. Wild-scan has
+no ground-truth labels either; its `precision` column reports a precision floor
+computed as `confirmed_malware / flagged`. Every cell is sourced from a
+specific measurement file — see `data/measurements/<source>/latest.json` for
+the file path and `metadata.measurement_file` in `stats.json` for the absolute
+repo path.
 
 False-positive rate is lane-keyed as of v3.5.0, not a single overall figure.
 ATR ships detection lanes (`enforce` / `alert` / `hunt`); on a 65K-sample
@@ -533,7 +668,7 @@ The 96,096-skill ecosystem scan was made possible by the maintainers of OpenClaw
 ### Normative
 
 - [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119) — Key words for use in RFCs to Indicate Requirement Levels.
-- [ATR-SPEC-v1.md](ATR-SPEC-v1.md) — ATR rule format specification, v1.0 Draft.
+- [SPEC.md](SPEC.md) — ATR rule format specification, v1.0 Draft.
 - [spec/atr-schema.yaml](spec/atr-schema.yaml) — Authoritative machine-readable schema.
 
 ### Informative

package/dist/action-executor.d.ts

@@ -7,7 +7,19 @@
  *
  * @module agent-threat-rules/action-executor
  */
-import type { ActionResult, ExecutionContext, PlatformAdapter } from "./types.js";
+import type { ATRAction, ActionResult, ExecutionContext, PlatformAdapter } from "./types.js";
+/**
+ * Map action names to PlatformAdapter method names.
+ *
+ * Exported because it is the authoritative list of actions this engine can
+ * actually dispatch. Rule files also carry SPEC Appendix A vocabulary
+ * (`revoke_credential`, `quarantine_artifact`, ...) that has no entry here and
+ * therefore never reaches an adapter — see executeOne's "Unknown action" branch.
+ * src/quality/action-eligibility.ts ranks exactly these keys by blast radius, and
+ * tests/action-eligibility.test.ts pins the two sets equal, so implementing a new
+ * adapter method forces an explicit decision about how destructive it is.
+ */
+export declare const ACTION_METHOD_MAP: Readonly<Record<ATRAction, keyof PlatformAdapter>>;
 export interface ActionExecutorConfig {
     readonly adapter: PlatformAdapter;
     readonly dryRun?: boolean;

package/dist/action-executor.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"action-executor.d.ts","sourceRoot":"","sources":["../src/action-executor.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,KAAK,EAEV,YAAY,EACZ,gBAAgB,EAChB,eAAe,EAChB,MAAM,YAAY,CAAC;AAgCpB,MAAM,WAAW,oBAAoB;IACnC,QAAQ,CAAC,OAAO,EAAE,eAAe,CAAC;IAClC,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC;IAC1B,QAAQ,CAAC,gBAAgB,CAAC,EAAE,CAAC,MAAM,EAAE,YAAY,KAAK,IAAI,CAAC;CAC5D;AAED,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAkB;IAC1C,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAiC;gBAEvD,MAAM,EAAE,oBAAoB;IAMxC;;;;;;;;OAQG;IACG,OAAO,CAAC,OAAO,EAAE,gBAAgB,GAAG,OAAO,CAAC,SAAS,YAAY,EAAE,CAAC;IAgB1E;;OAEG;IACH,OAAO,CAAC,kBAAkB;IAW1B;;OAEG;YACW,UAAU;IAmDxB,2CAA2C;IAC3C,cAAc,IAAI,MAAM;IAIxB,uCAAuC;IACvC,QAAQ,IAAI,OAAO;CAGpB"}
+{"version":3,"file":"action-executor.d.ts","sourceRoot":"","sources":["../src/action-executor.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,KAAK,EACV,SAAS,EACT,YAAY,EACZ,gBAAgB,EAChB,eAAe,EAChB,MAAM,YAAY,CAAC;AAiBpB;;;;;;;;;;GAUG;AACH,eAAO,MAAM,iBAAiB,EAAE,QAAQ,CAAC,MAAM,CAAC,SAAS,EAAE,MAAM,eAAe,CAAC,CAYhF,CAAC;AAEF,MAAM,WAAW,oBAAoB;IACnC,QAAQ,CAAC,OAAO,EAAE,eAAe,CAAC;IAClC,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC;IAC1B,QAAQ,CAAC,gBAAgB,CAAC,EAAE,CAAC,MAAM,EAAE,YAAY,KAAK,IAAI,CAAC;CAC5D;AAED,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAkB;IAC1C,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAU;IACjC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAiC;gBAEvD,MAAM,EAAE,oBAAoB;IAMxC;;;;;;;;OAQG;IACG,OAAO,CAAC,OAAO,EAAE,gBAAgB,GAAG,OAAO,CAAC,SAAS,YAAY,EAAE,CAAC;IAgB1E;;OAEG;IACH,OAAO,CAAC,kBAAkB;IAW1B;;OAEG;YACW,UAAU;IAmDxB,2CAA2C;IAC3C,cAAc,IAAI,MAAM;IAIxB,uCAAuC;IACvC,QAAQ,IAAI,OAAO;CAGpB"}

package/dist/action-executor.js

@@ -21,8 +21,18 @@ const ACTION_PRIORITY = {
     snapshot: 9,
     shadow: 10,
 };
-/** Map action names to PlatformAdapter method names */
-const ACTION_METHOD_MAP = {
+/**
+ * Map action names to PlatformAdapter method names.
+ *
+ * Exported because it is the authoritative list of actions this engine can
+ * actually dispatch. Rule files also carry SPEC Appendix A vocabulary
+ * (`revoke_credential`, `quarantine_artifact`, ...) that has no entry here and
+ * therefore never reaches an adapter — see executeOne's "Unknown action" branch.
+ * src/quality/action-eligibility.ts ranks exactly these keys by blast radius, and
+ * tests/action-eligibility.test.ts pins the two sets equal, so implementing a new
+ * adapter method forces an explicit decision about how destructive it is.
+ */
+export const ACTION_METHOD_MAP = {
     block_input: "blockInput",
     block_output: "blockOutput",
     block_tool: "blockTool",

package/dist/action-executor.js.map

@@ -1 +1 @@
-{"version":3,"file":"action-executor.js","sourceRoot":"","sources":["../src/action-executor.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AASH,sEAAsE;AACtE,MAAM,eAAe,GAAwC;IAC3D,UAAU,EAAE,CAAC;IACb,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC;IACf,UAAU,EAAE,CAAC;IACb,kBAAkB,EAAE,CAAC;IACrB,kBAAkB,EAAE,CAAC;IACrB,aAAa,EAAE,CAAC;IAChB,KAAK,EAAE,CAAC;IACR,QAAQ,EAAE,CAAC;IACX,QAAQ,EAAE,CAAC;IACX,MAAM,EAAE,EAAE;CACX,CAAC;AAEF,uDAAuD;AACvD,MAAM,iBAAiB,GAAuD;IAC5E,WAAW,EAAE,YAAY;IACzB,YAAY,EAAE,aAAa;IAC3B,UAAU,EAAE,WAAW;IACvB,kBAAkB,EAAE,mBAAmB;IACvC,aAAa,EAAE,cAAc;IAC7B,KAAK,EAAE,OAAO;IACd,MAAM,EAAE,QAAQ;IAChB,QAAQ,EAAE,UAAU;IACpB,QAAQ,EAAE,UAAU;IACpB,kBAAkB,EAAE,mBAAmB;IACvC,UAAU,EAAE,WAAW;CACxB,CAAC;AAQF,MAAM,OAAO,cAAc;IACR,OAAO,CAAkB;IACzB,MAAM,CAAU;IAChB,gBAAgB,CAAkC;IAEnE,YAAY,MAA4B;QACtC,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC;QAC9B,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,KAAK,CAAC;QACrC,IAAI,CAAC,gBAAgB,GAAG,MAAM,CAAC,gBAAgB,CAAC;IAClD,CAAC;IAED;;;;;;;;OAQG;IACH,KAAK,CAAC,OAAO,CAAC,OAAyB;QACrC,MAAM,OAAO,GAAG,IAAI,CAAC,kBAAkB,CAAC,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QACjE,MAAM,OAAO,GAAmB,EAAE,CAAC;QAEnC,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;YAC7B,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;YACtD,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YAErB,IAAI,IAAI,CAAC,gBAAgB,EAAE,CAAC;gBAC1B,IAAI,CAAC,gBAAgB,CAAC,MAAM,CAAC,CAAC;YAChC,CAAC;QACH,CAAC;QAED,OAAO,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAChC,CAAC;IAED;;OAEG;IACK,kBAAkB,CACxB,OAA6B;QAE7B,MAAM,MAAM,GAAG,CAAC,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC;QACrC,OAAO,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YAC1B,MAAM,EAAE,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,MAAM,EAAE,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,OAAO,EAAE,GAAG,EAAE,CAAC;QACjB,CAAC,CAAC,CAAC;IACL,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,UAAU,CACtB,MAAiB,EACjB,OAAyB;QAEzB,MAAM,SAAS,GAAG,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;QAE3C,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YAChB,OAAO,MAAM,CAAC,MAAM,CAAC;gBACnB,MAAM;gBACN,OAAO,EAAE,IAAI;gBACb,OAAO,EAAE,4BAA4B,MAAM,EAAE;gBAC7C,SAAS;aACV,CAAC,CAAC;QACL,CAAC;QAED,IAAI,CAAC;YACH,MAAM,UAAU,GAAG,iBAAiB,CAAC,MAAM,CAAC,CAAC;YAC7C,IAAI,CAAC,UAAU,EAAE,CAAC;gBAChB,OAAO,MAAM,CAAC,MAAM,CAAC;oBACnB,MAAM;oBACN,OAAO,EAAE,KAAK;oBACd,OAAO,EAAE,mBAAmB,MAAM,EAAE;oBACpC,SAAS;iBACV,CAAC,CAAC;YACL,CAAC;YAED,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,UAAU,CAEzB,CAAC;YAEd,IAAI,OAAO,MAAM,KAAK,UAAU,EAAE,CAAC;gBACjC,OAAO,MAAM,CAAC,MAAM,CAAC;oBACnB,MAAM;oBACN,OAAO,EAAE,KAAK;oBACd,OAAO,EAAE,YAAY,IAAI,CAAC,OAAO,CAAC,IAAI,yBAAyB,UAAU,EAAE;oBAC3E,SAAS;iBACV,CAAC,CAAC;YACL,CAAC;YAED,OAAO,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;QAClD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,OAAO,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;YACjE,OAAO,MAAM,CAAC,MAAM,CAAC;gBACnB,MAAM;gBACN,OAAO,EAAE,KAAK;gBACd,OAAO,EAAE,WAAW,MAAM,aAAa,OAAO,EAAE;gBAChD,SAAS;aACV,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,2CAA2C;IAC3C,cAAc;QACZ,OAAO,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC;IAC3B,CAAC;IAED,uCAAuC;IACvC,QAAQ;QACN,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;CACF"}
+{"version":3,"file":"action-executor.js","sourceRoot":"","sources":["../src/action-executor.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AASH,sEAAsE;AACtE,MAAM,eAAe,GAAwC;IAC3D,UAAU,EAAE,CAAC;IACb,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC;IACf,UAAU,EAAE,CAAC;IACb,kBAAkB,EAAE,CAAC;IACrB,kBAAkB,EAAE,CAAC;IACrB,aAAa,EAAE,CAAC;IAChB,KAAK,EAAE,CAAC;IACR,QAAQ,EAAE,CAAC;IACX,QAAQ,EAAE,CAAC;IACX,MAAM,EAAE,EAAE;CACX,CAAC;AAEF;;;;;;;;;;GAUG;AACH,MAAM,CAAC,MAAM,iBAAiB,GAAuD;IACnF,WAAW,EAAE,YAAY;IACzB,YAAY,EAAE,aAAa;IAC3B,UAAU,EAAE,WAAW;IACvB,kBAAkB,EAAE,mBAAmB;IACvC,aAAa,EAAE,cAAc;IAC7B,KAAK,EAAE,OAAO;IACd,MAAM,EAAE,QAAQ;IAChB,QAAQ,EAAE,UAAU;IACpB,QAAQ,EAAE,UAAU;IACpB,kBAAkB,EAAE,mBAAmB;IACvC,UAAU,EAAE,WAAW;CACxB,CAAC;AAQF,MAAM,OAAO,cAAc;IACR,OAAO,CAAkB;IACzB,MAAM,CAAU;IAChB,gBAAgB,CAAkC;IAEnE,YAAY,MAA4B;QACtC,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC;QAC9B,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,KAAK,CAAC;QACrC,IAAI,CAAC,gBAAgB,GAAG,MAAM,CAAC,gBAAgB,CAAC;IAClD,CAAC;IAED;;;;;;;;OAQG;IACH,KAAK,CAAC,OAAO,CAAC,OAAyB;QACrC,MAAM,OAAO,GAAG,IAAI,CAAC,kBAAkB,CAAC,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QACjE,MAAM,OAAO,GAAmB,EAAE,CAAC;QAEnC,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;YAC7B,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;YACtD,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YAErB,IAAI,IAAI,CAAC,gBAAgB,EAAE,CAAC;gBAC1B,IAAI,CAAC,gBAAgB,CAAC,MAAM,CAAC,CAAC;YAChC,CAAC;QACH,CAAC;QAED,OAAO,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAChC,CAAC;IAED;;OAEG;IACK,kBAAkB,CACxB,OAA6B;QAE7B,MAAM,MAAM,GAAG,CAAC,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC;QACrC,OAAO,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YAC1B,MAAM,EAAE,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,MAAM,EAAE,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,OAAO,EAAE,GAAG,EAAE,CAAC;QACjB,CAAC,CAAC,CAAC;IACL,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,UAAU,CACtB,MAAiB,EACjB,OAAyB;QAEzB,MAAM,SAAS,GAAG,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;QAE3C,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YAChB,OAAO,MAAM,CAAC,MAAM,CAAC;gBACnB,MAAM;gBACN,OAAO,EAAE,IAAI;gBACb,OAAO,EAAE,4BAA4B,MAAM,EAAE;gBAC7C,SAAS;aACV,CAAC,CAAC;QACL,CAAC;QAED,IAAI,CAAC;YACH,MAAM,UAAU,GAAG,iBAAiB,CAAC,MAAM,CAAC,CAAC;YAC7C,IAAI,CAAC,UAAU,EAAE,CAAC;gBAChB,OAAO,MAAM,CAAC,MAAM,CAAC;oBACnB,MAAM;oBACN,OAAO,EAAE,KAAK;oBACd,OAAO,EAAE,mBAAmB,MAAM,EAAE;oBACpC,SAAS;iBACV,CAAC,CAAC;YACL,CAAC;YAED,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,UAAU,CAEzB,CAAC;YAEd,IAAI,OAAO,MAAM,KAAK,UAAU,EAAE,CAAC;gBACjC,OAAO,MAAM,CAAC,MAAM,CAAC;oBACnB,MAAM;oBACN,OAAO,EAAE,KAAK;oBACd,OAAO,EAAE,YAAY,IAAI,CAAC,OAAO,CAAC,IAAI,yBAAyB,UAAU,EAAE;oBAC3E,SAAS;iBACV,CAAC,CAAC;YACL,CAAC;YAED,OAAO,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;QAClD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,OAAO,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;YACjE,OAAO,MAAM,CAAC,MAAM,CAAC;gBACnB,MAAM;gBACN,OAAO,EAAE,KAAK;gBACd,OAAO,EAAE,WAAW,MAAM,aAAa,OAAO,EAAE;gBAChD,SAAS;aACV,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,2CAA2C;IAC3C,cAAc;QACZ,OAAO,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC;IAC3B,CAAC;IAED,uCAAuC;IACvC,QAAQ;QACN,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;CACF"}

package/dist/cli.js

@@ -31,6 +31,7 @@ const GREEN = '\x1b[32m';
 const RED = '\x1b[31m';
 const DIM = '\x1b[2m';
 const BOLD = '\x1b[1m';
+const YELLOW = '\x1b[33m';
 function printUsage() {
     console.log(`
 ${BOLD}ATR - Agent Threat Rules${RESET}
@@ -227,7 +228,9 @@ async function cmdTest(target, options) {
     let totalTests = 0;
     let passed = 0;
     let failed = 0;
+    let skipped = 0;
     const failures = [];
+    const skippedRules = [];
     // Map extended agent_source types to basic event-compatible source types
     // so the engine's source type filter doesn't skip rules during testing.
     // The engine only recognizes: llm_io, tool_call, mcp_exchange, agent_behavior, multi_agent_comm
@@ -241,6 +244,22 @@ async function cmdTest(target, options) {
     for (const rule of rules) {
         if (!rule.test_cases)
             continue;
+        // method=behavioral rules cannot be evaluated here, and that is by design:
+        // engine.ts returns null for them because windowed metric evaluation needs
+        // state across many events, which the synchronous evaluate() API does not
+        // carry. Running their cases anyway reports "Expected: triggered, Got:
+        // not_triggered" for every true_positive, which reads as a detection
+        // failure when the truth is that this engine never looked. Count them as
+        // unevaluable and say so, rather than failing them or passing them
+        // silently. (method=trace is different: it has a real evaluation path
+        // below, so its cases still run.)
+        if (rule.detection?.method === 'behavioral') {
+            const cases = (rule.test_cases.true_positives?.length ?? 0) + (rule.test_cases.true_negatives?.length ?? 0);
+            totalTests += cases;
+            skipped += cases;
+            skippedRules.push({ ruleId: rule.id, cases, reason: 'method=behavioral requires a streaming evaluator' });
+            continue;
+        }
         // For testing, normalize extended source types so the engine doesn't filter them out
         const originalSourceType = rule.agent_source?.type;
         const baseSourceType = EXTENDED_SOURCE_TO_BASE[originalSourceType ?? ''];
@@ -306,7 +325,7 @@ async function cmdTest(target, options) {
         }
     }
     if (jsonOutput) {
-        console.log(JSON.stringify({ totalRules: rules.length, totalTests, passed, failed, failures }, null, 2));
+        console.log(JSON.stringify({ totalRules: rules.length, totalTests, passed, failed, skipped, failures, skippedRules }, null, 2));
         return;
     }
     console.log(`\n${BOLD}ATR Test Runner${RESET}`);
@@ -317,6 +336,12 @@ async function cmdTest(target, options) {
     if (failed > 0) {
         console.log(`  ${RED}Failed:${RESET}          ${failed}`);
     }
+    if (skipped > 0) {
+        console.log(`  ${YELLOW}Unevaluable:${RESET}     ${skipped}`);
+        for (const s of skippedRules) {
+            console.log(`    ${DIM}${s.ruleId}: ${s.cases} case(s) — ${s.reason}${RESET}`);
+        }
+    }
     console.log(`${DIM}${'─'.repeat(60)}${RESET}`);
     if (failures.length > 0) {
         console.log(`\n${RED}Failures:${RESET}\n`);
@@ -328,7 +353,11 @@ async function cmdTest(target, options) {
         process.exit(1);
     }
     else {
-        console.log(`\n${GREEN}All tests passed.${RESET}\n`);
+        // The literal "All tests passed" is what rule-quality.yml greps for, so it
+        // has to survive; the suffix keeps it from overstating what actually ran.
+        console.log(skipped > 0
+            ? `\n${GREEN}All tests passed.${RESET} ${DIM}(${skipped} case(s) unevaluable — see above)${RESET}\n`
+            : `\n${GREEN}All tests passed.${RESET}\n`);
     }
 }
 function buildEventFromTestCase(tc, rule) {
@@ -344,7 +373,12 @@ function buildEventFromTestCase(tc, rule) {
     // Object-style: input: { tool_name: "...", tool_args: "...", response: "..." }
     // Flat-style: input: "...", tool_response: "...", tool_name: "..."
     // Tool-call-style: tool_call: { name: "...", args: "..." }
-    const rawInput = tc['input'];
+    // Accept `user_input:` as a top-level alias for `input:` — several tool_call
+    // rules write their natural-language TPs as `- user_input: "..."`, which the
+    // harness previously ignored (it only read `input`), building an EMPTY event so
+    // the rule's own documented attack tested as not_triggered (a false failure that
+    // left the rule looking broken while it detects fine in production).
+    const rawInput = tc['input'] ?? tc['user_input'];
     let input = '';
     let toolName = str(tc['tool_name']);
     let toolArgs = str(tc['tool_args']);
@@ -400,13 +434,16 @@ function buildEventFromTestCase(tc, rule) {
         skill_chain: 'tool_call',
     };
     let type = SOURCE_TO_EVENT[sourceType] ?? 'llm_input';
-    // If rule expects tool_call but test case has only plain text input
-    // (no tool_name, tool_args, or tool_response), use llm_input event type
-    // since the content is natural language, not a tool invocation.
-    // This prevents the engine's tool_name fallback from treating text as a tool name.
-    if (type === 'tool_call' && !toolName && !toolArgs && !toolResponse && !toolDescription) {
-        type = 'llm_input';
-    }
+    // NOTE: previously, a tool_call rule with a plain-string TP was downgraded to
+    // an llm_input event. That mis-tested tool_call-native rules: a shell command or
+    // SQL string IS a tool invocation, and evaluating it as llm_input made the engine
+    // apply its cross-context confidence penalty, dropping the match below threshold
+    // so the rule's own documented attack tested as not_triggered (false failure that
+    // silently left broken rules in the enforce lane). We now keep the native
+    // tool_call type — the string flows in via `content` (below), and tool_name is
+    // pinned to "" (below) so the engine's tool_name fallback never treats the
+    // content as a tool name. Plain-string TPs of tool_call rules therefore test in
+    // their real (native) context.
     // Determine the primary content based on what the rule conditions check
     // Problem 2 & 3: For rules that check tool_response, the tool_response
     // should be the primary content when the event type resolves tool_response from content
@@ -431,8 +468,27 @@ function buildEventFromTestCase(tc, rule) {
     // Always set tool_name (even empty) to prevent engine fallback
     // from using event.content as tool_name for tool_call events
     fields['tool_name'] = toolName;
-    if (toolArgs)
-        fields['tool_args'] = toolArgs;
+    // Production ALWAYS populates tool_args: src/hook-handler.ts builds
+    // `tool_args: JSON.stringify(toolInput)` for BOTH the PreToolUse (tool_call)
+    // and the PostToolUse (tool_response) event. The engine resolves this field as
+    // `event.fields.tool_args ?? (event.type === 'tool_call' ? event.content :
+    // undefined)` (src/engine.ts resolveField), so a test case that supplies only
+    // a plain `input:` string leaves tool_args unresolvable on every non-tool_call
+    // event and EVERY `field: tool_args` condition silently evaluates to false.
+    // The rule then reports not_triggered on its own documented attack while
+    // detecting it correctly in production — a harness defect that reads exactly
+    // like a broken rule and invites weakening a rule that is right. 40 rules
+    // declare `field: tool_args`.
+    // This is the test-case-runner twin of the measurement-shape defect that
+    // scripts/lib/corpus-event.ts fixed for the FP gate: a true positive must be
+    // exercised on the same shape the benign corpus is charged against.
+    // Deliberately narrower than that harness's wide shape: only the INPUT string
+    // is offered as tool_args. A `tool_response:` test case asserts what a tool
+    // RETURNED, which is not an argument, so pouring it into tool_args would let a
+    // tool_args layer pass a TP for the wrong reason.
+    const toolArgsValue = toolArgs || input;
+    if (toolArgsValue)
+        fields['tool_args'] = toolArgsValue;
     // For content-based rules, set content and agent_message fields
     fields['content'] = content;
     fields['agent_message'] = content;