stackone-defender 0.8.2__tar.gz → 0.9.1__tar.gz

stackone_defender-0.9.1/.release-please-manifest.json

@@ -0,0 +1 @@
+{".":"0.9.1"}

{stackone_defender-0.8.2 → stackone_defender-0.9.1}/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [0.9.1](https://github.com/StackOneHQ/defender-py/compare/stackone-defender-v0.9.0...stackone-defender-v0.9.1) (2026-10-20)
+
+
+### Features
+
+* **ENG-3500:** bring-your-own-LLM Tier 3 and policy options ([#38](https://github.com/StackOneHQ/defender-py/issues/38)) ([9923491](https://github.com/StackOneHQ/defender-py/commit/9923491c413bafa09d8aee0c47aabd43ed049d09))
+
+## [0.9.0](https://github.com/StackOneHQ/defender-py/compare/stackone-defender-v0.8.2...stackone-defender-v0.9.0) (2026-10-20)
+
+
+### Features
+
+* **ENG-3500:** record-oriented, chunked Tier-3 review in tier3_only ([#35](https://github.com/StackOneHQ/defender-py/issues/35)) ([072d7ed](https://github.com/StackOneHQ/defender-py/commit/072d7ed7f0474afa5ffe87a66920d279ad7fafdd))
+
+
+### Miscellaneous Chores
+
+* **ENG-3500:** release 0.9.0 ([#37](https://github.com/StackOneHQ/defender-py/issues/37)) ([aa4a4e3](https://github.com/StackOneHQ/defender-py/commit/aa4a4e39fcb7a7adc6d43fbab8d7b84817613dbc))
+
 ## [0.8.2](https://github.com/StackOneHQ/defender-py/compare/stackone-defender-v0.7.4...stackone-defender-v0.8.2) (2026-08-19)
 
 

{stackone_defender-0.8.2 → stackone_defender-0.9.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: stackone-defender
-Version: 0.8.2
+Version: 0.9.1
 Summary: Indirect prompt injection defense for AI agents using tool calls
 Project-URL: Homepage, https://github.com/StackOneHQ/stackone-defender
 Project-URL: Repository, https://github.com/StackOneHQ/stackone-defender
@@ -16,6 +16,8 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Security
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.11
+Provides-Extra: llm
+Requires-Dist: httpx>=0.27; extra == 'llm'
 Provides-Extra: onnx
 Requires-Dist: numpy>=1.24.0; extra == 'onnx'
 Requires-Dist: onnxruntime>=1.16.0; extra == 'onnx'
@@ -83,6 +85,15 @@ pip install stackone-defender[sfe]
 # or: uv add "stackone-defender[sfe]"
 ```
 
+**Tier 3 over an OpenAI-compatible endpoint (optional)** — add extras:
+
+```bash
+pip install stackone-defender[llm]
+# or: uv add "stackone-defender[llm]"
+```
+
+The `[llm]` extra installs [`httpx`](https://www.python-httpx.org/) for `create_openai_compatible_tier3_provider`. `create_llm_tier3_provider` needs no extra.
+
 The `[sfe]` extra installs [`fasttext-ng`](https://pypi.org/project/fasttext-ng/) (provides the `fasttext` module). It requires **NumPy 2.3+**. PyPI may ship a wheel only for some platforms; otherwise pip/uv builds from source (needs a C++ toolchain).
 
 ## Quick start
@@ -148,15 +159,58 @@ Packed-chunk MiniLM classifier (int8 ONNX ~22 MB, bundled):
 | jayavibhav (adversarial) | 0.9717 | ~1k |
 | **Average** | **0.9079** | ~25k |
 
-### Optional Tier 3 — LLM adjudication (consumer-supplied)
+### Optional Tier 3 — LLM adjudication (bring your own LLM)
 
-Authoritative LLM-based classification for the cases Tier 2 finds ambiguous. The package ships **only** the orchestration and the `Tier3Provider` interface — the actual model endpoint (a hosted LLM, OpenAI, an internal inference service) lives in your code, keeping proprietary models and credentials out of the package.
+Authoritative LLM-based classification for the cases Tier 2 finds ambiguous. You bring the model. Defender gives you three ways to plug it in:
+
+1. **`create_openai_compatible_tier3_provider`**: any OpenAI-compatible `/chat/completions` endpoint (OpenAI, vLLM, Ollama, OpenRouter, LiteLLM). Needs the `[llm]` extra.
+2. **`create_llm_tier3_provider`**: any other LLM. You write one `complete` function that calls the model; defender owns the prompt, the untrusted-text markers, strict reply parsing and refusal handling.
+3. **A hand-written provider**: an object with `classify(text, *, ctx=None)` that owns everything itself.
 
 Two modes, selected via `defender_mode`:
 - **`"cascade"`** (default): Tier 1 → Tier 2 → Tier 3, with Tier 3 invoked only when the Tier 2 effective score falls in the gray band (default `[0.3, 0.85)`). The Tier 3 verdict overrides Tier 2 on the escalated chunk — a `block` forces a block, an `allow` rescues it. Outside the band defender skips the round trip.
 - **`"tier3_only"`**: skip Tier 2; the block/allow decision is the Tier 3 verdict alone. Tier 1 still runs detection; the returned `sanitized` payload is the original content.
 
-Register a provider once at startup, then opt in per instance:
+```python
+import os
+
+from stackone_defender import create_openai_compatible_tier3_provider, create_prompt_defense
+
+judge = create_openai_compatible_tier3_provider(
+    base_url="https://api.openai.com/v1",
+    api_key=os.environ["OPENAI_API_KEY"],
+    model="gpt-4.1-mini",
+)
+defense = create_prompt_defense(
+    block_high_risk=True,
+    enable_tier3=True,
+    defender_mode="tier3_only",
+    tier3={"provider": judge, "timeout_ms": 10_000, "on_error": "block"},
+)
+result = await defense.defend_tool_result_async(
+    tool_output, "gmail_get_message", context={"user_goal": "Summarize my unread mail"}
+)
+```
+
+Around any other LLM, `complete` gets a request with `messages` (system + user), `deadline` and `json_schema`, and returns the reply text (sync or async):
+
+```python
+import time
+
+from stackone_defender import create_llm_tier3_provider
+
+async def complete(req):
+    # req.deadline is absolute (time.time() seconds), or None when tier3.timeout_ms isn't set.
+    timeout = req.deadline - time.time() if req.deadline else 30
+    reply = await my_llm.chat(req.messages, timeout=timeout)
+    return reply.text  # or {"text": ..., "refused": True} / {"text": ..., "score": p_block}
+
+judge = create_llm_tier3_provider(complete=complete)
+```
+
+The default prompt (`DEFAULT_TIER3_PROMPT`, version `TIER3_PROMPT_VERSION`) wraps the tool output, user goal and tool description in random-id markers and asks for `{"decision": "block" | "allow", "reason": "..."}`. Pass `prompt=Tier3PromptTemplate(system=..., user=...)` (the `user` part must contain `{{content}}` once; also `{{toolName}}`, `{{userGoal}}`, `{{toolDescription}}`) or a builder function that gets a `PromptInput`. A refusal, a content-filter hit, or a reply that isn't exactly one JSON verdict **blocks** (`result.tier3.refused`), because attacker text can cause those on purpose. `result.tier3.reason` is model output that can quote the attacker: log it, but never feed it back to an agent.
+
+A hand-written provider, registered once at startup:
 
 ```python
 from typing import Any
@@ -166,8 +220,9 @@ from stackone_defender.types import Tier3Verdict
 
 class MyProvider:
     def classify(self, text: str, *, ctx: dict[str, Any] | None = None) -> Tier3Verdict:
-        # Call your LLM endpoint here (sync or awaitable). ctx["toolName"] is available.
-        verdict = call_my_llm(text, tool_name=(ctx or {}).get("toolName"))
+        # Call your LLM endpoint here (sync or awaitable). ctx has tool_name, mode, chunk,
+        # user_goal / tool_description (untrusted), and deadline when timeout_ms is set.
+        verdict = call_my_llm(text, tool_name=(ctx or {}).get("tool_name"))
         return Tier3Verdict(decision="block" if verdict.block else "allow", score=verdict.p_block)
 
 set_default_tier3_provider(MyProvider())
@@ -179,15 +234,47 @@ defense = create_prompt_defense(
     tier3={
         "provider": MyProvider(),                          # optional: overrides the registry for this instance
         "escalation_band": {"lower": 0.3, "upper": 0.85},  # cascade gray band; [lower, upper), defaults shown
-        "max_text_length": 10000,                          # caps text passed to the provider
+        "max_text_length": 10000,                          # max chars per provider call
+        "max_chunks": 5,                                   # tier3_only: chunks reviewed per tool result
+        "on_oversize": "skip",                             # tier3_only: "skip" | "block" | "scan_anyway"
         "block_threshold": 0.622,                          # optional; decide on score, not the model's word
     },
 )
 ```
 
+**What the reviewer sees (`tier3_only`).** The tool result is serialized as `field: value` lines, one block per record, so a bare value keeps its field name (`status: create` rather than `create`, which reads like an instruction). Nested keys are dotted (`user.name: …`), list items are indexed (`items[0]: …`), and each line of a multi-line value keeps its prefix. A large list of only numbers collapses to `key: [N numbers]`. Keys are reviewed even when the value is a number, boolean, `None` or empty, because keys can carry injected text. Python-only types are reviewed too: mappings and dataclasses as objects, sets and other sequences as lists, UTF-8 `bytes` as text, and anything else as `str(value)`.
+
+The serialized text is split into chunks of at most `max_text_length` characters (lengths count UTF-16 code units, as in the TypeScript package). Neighbouring chunks overlap by up to 500 characters, so an injection cut at a chunk boundary still appears whole. All chunks are reviewed in parallel, and any blocking chunk blocks the result. `result.tier3_chunk_summary` reports `chunks`, `blocked` and `blocking_chunk`.
+
+**Oversize (`on_oversize`).** Up to about `max_chunks` chunks are reviewed per tool result. Content beyond that is oversize, and `coverage_degraded` is `True`. Blocking below requires `block_high_risk=True`.
+- `"skip"` (default): review the content that fits and allow the rest. A fitting injection still blocks. This favors availability, so set `"block"` or `"scan_anyway"` to fail closed.
+- `"block"`: block without calling the provider. `tier3_chunk_summary` is `None` and `result.tier3.skip_reason` says why.
+- `"scan_anyway"`: review what fits, then block on the unreviewed rest unless a chunk already blocked.
+
+In `"skip"` and `"scan_anyway"`, `tier3_chunk_summary.oversize` is `True`.
+
+A failed chunk is skipped (fail open), and `result.tier3.errored` and `coverage_degraded` are `True`. With a verdict, `result.tier3.note` says how many chunks failed; it also gives the reason for an `on_oversize` block when the provider allowed. When every chunk fails there's no verdict, and `result.tier3.skip_reason` gives the first failure.
+
 **Choosing the operating point (`block_threshold`).** By default the model's `decision` word is authoritative — but that word is its argmax, an implicit 0.5 cut that moves on its own whenever the model is retrained. Set `tier3.block_threshold` to decide on `verdict.score` (P(block)) instead: the cut becomes an explicit config value — raise it to trade recall for fewer false positives, lower it for the reverse; `0.5` reproduces argmax. It requires a provider that reports `score` as P(block) (not "confidence in whichever decision I made" — those invert on allows). If `score` is missing or outside `[0, 1]` the verdict's `decision` is used instead and defender warns once, so a provider that cannot report a score degrades to the default rather than failing.
 
-**Fail-open semantics.** A provider error or timeout records a `skip_reason` on `result.tier3`; in cascade defender falls back to the Tier 2 decision, in `tier3_only` it allows the request. `enable_tier3=True` with no registered provider falls back to the standard Tier 1 + Tier 2 cascade and logs one warning per instance — Tier 3 misconfiguration never silently disables defense. `result.tier3` carries the verdict (a `Tier3Verdict`, or a `Tier3Skip` when the provider ran but returned nothing usable) when Tier 3 runs, and is `None` when it doesn't.
+**Policy options** (all in `tier3={...}`; camelCase keys work too):
+
+| Option | Default | What it does |
+|---|---|---|
+| `allow_override` | `True` | Cascade: whether a Tier-3 `allow` can undo a Tier-2 block. `False`: Tier 3 only adds blocks. `{"max_tier2_score": x}`: only when Tier 2's undamped score is below `x`. A general-purpose LLM can be talked into `allow`, so prefer `False` or a cap (defender warns once otherwise). |
+| `on_error` | cascade `"tier2"`, tier3_only `"allow"` | What a failure (error, timeout, invalid verdict) does. `"tier2"` keeps Tier 2's verdict. `"allow"` treats it as an allow. `"block"` blocks: in cascade when Tier 2 rated the payload medium or higher; in tier3_only when any chunk failed (and `on_oversize` then defaults to `"scan_anyway"`). A missing provider fails closed too. |
+| `decide` | none | `decide(Tier3DecisionInput) -> "block" \| "allow" \| "default"`, called once per call after Tier 3 ran or failed. Replaces `allow_override` and `on_error` for that call. Gets metadata only (never content). A raise or bad return means `"default"`. |
+| `timeout_ms` | none | Per provider call, from when it gets a concurrency slot. An async provider is cancelled at the deadline (`ctx["deadline"]`); a sync one can't be interrupted. |
+| `max_concurrency` | unbounded | Max in-flight provider calls per instance (per event loop). Set `timeout_ms` with it. |
+| `stop_on_first_block` | `False` | tier3_only: cancel the remaining chunk calls once one blocks. |
+| `on_event` | none | Called once per provider call with a `Tier3Event` (kind, mode, chunk, latency, usage, redacted error code). No content. Exceptions are ignored. |
+| `include_raw` | `True` | Keep `raw` on `result.tier3`. `raw` can quote tool content. |
+
+**Result fields.** `result.tier3` is a `Tier3Verdict` (or a `Tier3Skip` when no verdict was obtained) and `None` when Tier 3 didn't run. Besides `decision`, `score`, `reason`, `refused` and `prompt_version`, defender sets: `errored` (a real failure), `note` (why the outcome isn't the verdict's alone), `override_applied` / `override_suppressed` (cascade), and `decided_by` (`"policy"` or `"options"` when `decide` is set). Provider error text in `skip_reason`, `note` and events is redacted (keys, tokens, URL paths) and capped.
+
+**Cancellation.** Cancelling the task that awaits `defend_tool_result_async` cancels in-flight provider calls and raises `CancelledError`.
+
+**Fail-open by default.** Without `on_error="block"`, a provider error or timeout records a `skip_reason`; in cascade defender falls back to the Tier 2 decision, in `tier3_only` it allows. `enable_tier3=True` with no provider (and no `on_error="block"`) falls back to Tier 1 + Tier 2 and logs one warning per instance, so Tier 3 misconfiguration never silently disables defense.
 
 ### `allowed` vs `risk_level`
 
@@ -224,7 +311,7 @@ defense = create_prompt_defense(
 )
 ```
 
-### `defense.defend_tool_result(value, tool_name)`
+### `defense.defend_tool_result(value, tool_name, *, context=None)`
 
 Runs Tier 1 sanitization on risky fields of the **original** payload, then Tier 2 on strings from the SFE-filtered view when SFE is on (otherwise the full value). Optional `tier2_fields` restricts Tier 2 extraction to specific keys; omit it to classify **all** strings (matches `@stackone/defender` 0.6.3). **Synchronous** — no `await`.
 
@@ -257,7 +344,8 @@ class DefenseResult:
     cold_load: bool | None = None
     # Operational signals
     tier2_available: bool | None = None       # False when Tier 2 enabled but failed to load
-    coverage_degraded: bool | None = None      # True when Tier 1 detection coverage was capped
+    coverage_degraded: bool | None = None      # True when Tier 1 detection or the Tier 3 review didn't cover everything
+    tier3_chunk_summary: Tier3ChunkSummary | None = None  # tier3_only: chunks, blocked, blocking_chunk, oversize
 ```
 
 ### `defense.defend_tool_results(items)`
@@ -337,6 +425,25 @@ uv sync --group dev
 uv run pytest
 ```
 
+### Parity with the TypeScript package
+
+This package mirrors [`@stackone/defender`](https://github.com/StackOneHQ/defender). `tests/conformance/` is a pinned copy of its shared conformance fixtures: input → expected-output cases for Tier-3 serialization, prompt rendering, verdict parsing, error redaction and policy decisions. `tests/test_conformance.py` runs them against this package.
+
+- `scripts/sync_conformance.sh <ref>` copies the fixtures at a TS tag or commit and records it in `tests/conformance/VERSION`. CI checks the copy matches (`--check`).
+- Every case passes. A new TS case this package can't match yet goes in `tests/conformance_known_failures.txt`, where it runs as a strict xfail; remove its line once a port makes it pass.
+- Port contract for the runner: a rejected prompt template and an unusable verdict reply raise `ValueError`; optional result fields are `None` when unset (not `False`).
+
+Traps when porting from TypeScript (the fixtures pin TS behavior):
+
+- Lengths and slicing are in UTF-16 code units in TS, but `len()` counts code points.
+- TS `Object.entries` puts integer-like keys first. `json.loads` keeps insertion order.
+- Number rendering differs: `String(1.0)` is `"1"` but `str(1.0)` is `"1.0"`. Likewise `-0`, `1e-7` and `1e16` render differently in the two languages.
+- `core/tier3_input.py` handles the three traps above for Tier-3 input. It works on a UTF-16 view of each string, orders keys like `Object.entries`, and renders numbers like `String(n)`. One deliberate difference: Python ints below 1e21 print exactly, where JS would round past 2**53.
+- Python `re` has no `\p{Cf}`, and `unicodedata` versions differ across Python releases.
+- `str.splitlines()` also splits on `\x0b`, `\x0c` and `\x1c`, but TS splits only on `\r`, `\n`, `\u2028`, `\u2029` and `\u0085`.
+- JS `trim()` removes a BOM, but `str.strip()` doesn't.
+- `json.loads` accepts `NaN`, but JS `JSON.parse` rejects it. Use `parse_constant` to reject it.
+
 ## License
 
 Apache-2.0 — see [LICENSE](./LICENSE).

{stackone_defender-0.8.2 → stackone_defender-0.9.1}/README.md

@@ -57,6 +57,15 @@ pip install stackone-defender[sfe]
 # or: uv add "stackone-defender[sfe]"
 ```
 
+**Tier 3 over an OpenAI-compatible endpoint (optional)** — add extras:
+
+```bash
+pip install stackone-defender[llm]
+# or: uv add "stackone-defender[llm]"
+```
+
+The `[llm]` extra installs [`httpx`](https://www.python-httpx.org/) for `create_openai_compatible_tier3_provider`. `create_llm_tier3_provider` needs no extra.
+
 The `[sfe]` extra installs [`fasttext-ng`](https://pypi.org/project/fasttext-ng/) (provides the `fasttext` module). It requires **NumPy 2.3+**. PyPI may ship a wheel only for some platforms; otherwise pip/uv builds from source (needs a C++ toolchain).
 
 ## Quick start
@@ -122,15 +131,58 @@ Packed-chunk MiniLM classifier (int8 ONNX ~22 MB, bundled):
 | jayavibhav (adversarial) | 0.9717 | ~1k |
 | **Average** | **0.9079** | ~25k |
 
-### Optional Tier 3 — LLM adjudication (consumer-supplied)
+### Optional Tier 3 — LLM adjudication (bring your own LLM)
 
-Authoritative LLM-based classification for the cases Tier 2 finds ambiguous. The package ships **only** the orchestration and the `Tier3Provider` interface — the actual model endpoint (a hosted LLM, OpenAI, an internal inference service) lives in your code, keeping proprietary models and credentials out of the package.
+Authoritative LLM-based classification for the cases Tier 2 finds ambiguous. You bring the model. Defender gives you three ways to plug it in:
+
+1. **`create_openai_compatible_tier3_provider`**: any OpenAI-compatible `/chat/completions` endpoint (OpenAI, vLLM, Ollama, OpenRouter, LiteLLM). Needs the `[llm]` extra.
+2. **`create_llm_tier3_provider`**: any other LLM. You write one `complete` function that calls the model; defender owns the prompt, the untrusted-text markers, strict reply parsing and refusal handling.
+3. **A hand-written provider**: an object with `classify(text, *, ctx=None)` that owns everything itself.
 
 Two modes, selected via `defender_mode`:
 - **`"cascade"`** (default): Tier 1 → Tier 2 → Tier 3, with Tier 3 invoked only when the Tier 2 effective score falls in the gray band (default `[0.3, 0.85)`). The Tier 3 verdict overrides Tier 2 on the escalated chunk — a `block` forces a block, an `allow` rescues it. Outside the band defender skips the round trip.
 - **`"tier3_only"`**: skip Tier 2; the block/allow decision is the Tier 3 verdict alone. Tier 1 still runs detection; the returned `sanitized` payload is the original content.
 
-Register a provider once at startup, then opt in per instance:
+```python
+import os
+
+from stackone_defender import create_openai_compatible_tier3_provider, create_prompt_defense
+
+judge = create_openai_compatible_tier3_provider(
+    base_url="https://api.openai.com/v1",
+    api_key=os.environ["OPENAI_API_KEY"],
+    model="gpt-4.1-mini",
+)
+defense = create_prompt_defense(
+    block_high_risk=True,
+    enable_tier3=True,
+    defender_mode="tier3_only",
+    tier3={"provider": judge, "timeout_ms": 10_000, "on_error": "block"},
+)
+result = await defense.defend_tool_result_async(
+    tool_output, "gmail_get_message", context={"user_goal": "Summarize my unread mail"}
+)
+```
+
+Around any other LLM, `complete` gets a request with `messages` (system + user), `deadline` and `json_schema`, and returns the reply text (sync or async):
+
+```python
+import time
+
+from stackone_defender import create_llm_tier3_provider
+
+async def complete(req):
+    # req.deadline is absolute (time.time() seconds), or None when tier3.timeout_ms isn't set.
+    timeout = req.deadline - time.time() if req.deadline else 30
+    reply = await my_llm.chat(req.messages, timeout=timeout)
+    return reply.text  # or {"text": ..., "refused": True} / {"text": ..., "score": p_block}
+
+judge = create_llm_tier3_provider(complete=complete)
+```
+
+The default prompt (`DEFAULT_TIER3_PROMPT`, version `TIER3_PROMPT_VERSION`) wraps the tool output, user goal and tool description in random-id markers and asks for `{"decision": "block" | "allow", "reason": "..."}`. Pass `prompt=Tier3PromptTemplate(system=..., user=...)` (the `user` part must contain `{{content}}` once; also `{{toolName}}`, `{{userGoal}}`, `{{toolDescription}}`) or a builder function that gets a `PromptInput`. A refusal, a content-filter hit, or a reply that isn't exactly one JSON verdict **blocks** (`result.tier3.refused`), because attacker text can cause those on purpose. `result.tier3.reason` is model output that can quote the attacker: log it, but never feed it back to an agent.
+
+A hand-written provider, registered once at startup:
 
 ```python
 from typing import Any
@@ -140,8 +192,9 @@ from stackone_defender.types import Tier3Verdict
 
 class MyProvider:
     def classify(self, text: str, *, ctx: dict[str, Any] | None = None) -> Tier3Verdict:
-        # Call your LLM endpoint here (sync or awaitable). ctx["toolName"] is available.
-        verdict = call_my_llm(text, tool_name=(ctx or {}).get("toolName"))
+        # Call your LLM endpoint here (sync or awaitable). ctx has tool_name, mode, chunk,
+        # user_goal / tool_description (untrusted), and deadline when timeout_ms is set.
+        verdict = call_my_llm(text, tool_name=(ctx or {}).get("tool_name"))
         return Tier3Verdict(decision="block" if verdict.block else "allow", score=verdict.p_block)
 
 set_default_tier3_provider(MyProvider())
@@ -153,15 +206,47 @@ defense = create_prompt_defense(
     tier3={
         "provider": MyProvider(),                          # optional: overrides the registry for this instance
         "escalation_band": {"lower": 0.3, "upper": 0.85},  # cascade gray band; [lower, upper), defaults shown
-        "max_text_length": 10000,                          # caps text passed to the provider
+        "max_text_length": 10000,                          # max chars per provider call
+        "max_chunks": 5,                                   # tier3_only: chunks reviewed per tool result
+        "on_oversize": "skip",                             # tier3_only: "skip" | "block" | "scan_anyway"
         "block_threshold": 0.622,                          # optional; decide on score, not the model's word
     },
 )
 ```
 
+**What the reviewer sees (`tier3_only`).** The tool result is serialized as `field: value` lines, one block per record, so a bare value keeps its field name (`status: create` rather than `create`, which reads like an instruction). Nested keys are dotted (`user.name: …`), list items are indexed (`items[0]: …`), and each line of a multi-line value keeps its prefix. A large list of only numbers collapses to `key: [N numbers]`. Keys are reviewed even when the value is a number, boolean, `None` or empty, because keys can carry injected text. Python-only types are reviewed too: mappings and dataclasses as objects, sets and other sequences as lists, UTF-8 `bytes` as text, and anything else as `str(value)`.
+
+The serialized text is split into chunks of at most `max_text_length` characters (lengths count UTF-16 code units, as in the TypeScript package). Neighbouring chunks overlap by up to 500 characters, so an injection cut at a chunk boundary still appears whole. All chunks are reviewed in parallel, and any blocking chunk blocks the result. `result.tier3_chunk_summary` reports `chunks`, `blocked` and `blocking_chunk`.
+
+**Oversize (`on_oversize`).** Up to about `max_chunks` chunks are reviewed per tool result. Content beyond that is oversize, and `coverage_degraded` is `True`. Blocking below requires `block_high_risk=True`.
+- `"skip"` (default): review the content that fits and allow the rest. A fitting injection still blocks. This favors availability, so set `"block"` or `"scan_anyway"` to fail closed.
+- `"block"`: block without calling the provider. `tier3_chunk_summary` is `None` and `result.tier3.skip_reason` says why.
+- `"scan_anyway"`: review what fits, then block on the unreviewed rest unless a chunk already blocked.
+
+In `"skip"` and `"scan_anyway"`, `tier3_chunk_summary.oversize` is `True`.
+
+A failed chunk is skipped (fail open), and `result.tier3.errored` and `coverage_degraded` are `True`. With a verdict, `result.tier3.note` says how many chunks failed; it also gives the reason for an `on_oversize` block when the provider allowed. When every chunk fails there's no verdict, and `result.tier3.skip_reason` gives the first failure.
+
 **Choosing the operating point (`block_threshold`).** By default the model's `decision` word is authoritative — but that word is its argmax, an implicit 0.5 cut that moves on its own whenever the model is retrained. Set `tier3.block_threshold` to decide on `verdict.score` (P(block)) instead: the cut becomes an explicit config value — raise it to trade recall for fewer false positives, lower it for the reverse; `0.5` reproduces argmax. It requires a provider that reports `score` as P(block) (not "confidence in whichever decision I made" — those invert on allows). If `score` is missing or outside `[0, 1]` the verdict's `decision` is used instead and defender warns once, so a provider that cannot report a score degrades to the default rather than failing.
 
-**Fail-open semantics.** A provider error or timeout records a `skip_reason` on `result.tier3`; in cascade defender falls back to the Tier 2 decision, in `tier3_only` it allows the request. `enable_tier3=True` with no registered provider falls back to the standard Tier 1 + Tier 2 cascade and logs one warning per instance — Tier 3 misconfiguration never silently disables defense. `result.tier3` carries the verdict (a `Tier3Verdict`, or a `Tier3Skip` when the provider ran but returned nothing usable) when Tier 3 runs, and is `None` when it doesn't.
+**Policy options** (all in `tier3={...}`; camelCase keys work too):
+
+| Option | Default | What it does |
+|---|---|---|
+| `allow_override` | `True` | Cascade: whether a Tier-3 `allow` can undo a Tier-2 block. `False`: Tier 3 only adds blocks. `{"max_tier2_score": x}`: only when Tier 2's undamped score is below `x`. A general-purpose LLM can be talked into `allow`, so prefer `False` or a cap (defender warns once otherwise). |
+| `on_error` | cascade `"tier2"`, tier3_only `"allow"` | What a failure (error, timeout, invalid verdict) does. `"tier2"` keeps Tier 2's verdict. `"allow"` treats it as an allow. `"block"` blocks: in cascade when Tier 2 rated the payload medium or higher; in tier3_only when any chunk failed (and `on_oversize` then defaults to `"scan_anyway"`). A missing provider fails closed too. |
+| `decide` | none | `decide(Tier3DecisionInput) -> "block" \| "allow" \| "default"`, called once per call after Tier 3 ran or failed. Replaces `allow_override` and `on_error` for that call. Gets metadata only (never content). A raise or bad return means `"default"`. |
+| `timeout_ms` | none | Per provider call, from when it gets a concurrency slot. An async provider is cancelled at the deadline (`ctx["deadline"]`); a sync one can't be interrupted. |
+| `max_concurrency` | unbounded | Max in-flight provider calls per instance (per event loop). Set `timeout_ms` with it. |
+| `stop_on_first_block` | `False` | tier3_only: cancel the remaining chunk calls once one blocks. |
+| `on_event` | none | Called once per provider call with a `Tier3Event` (kind, mode, chunk, latency, usage, redacted error code). No content. Exceptions are ignored. |
+| `include_raw` | `True` | Keep `raw` on `result.tier3`. `raw` can quote tool content. |
+
+**Result fields.** `result.tier3` is a `Tier3Verdict` (or a `Tier3Skip` when no verdict was obtained) and `None` when Tier 3 didn't run. Besides `decision`, `score`, `reason`, `refused` and `prompt_version`, defender sets: `errored` (a real failure), `note` (why the outcome isn't the verdict's alone), `override_applied` / `override_suppressed` (cascade), and `decided_by` (`"policy"` or `"options"` when `decide` is set). Provider error text in `skip_reason`, `note` and events is redacted (keys, tokens, URL paths) and capped.
+
+**Cancellation.** Cancelling the task that awaits `defend_tool_result_async` cancels in-flight provider calls and raises `CancelledError`.
+
+**Fail-open by default.** Without `on_error="block"`, a provider error or timeout records a `skip_reason`; in cascade defender falls back to the Tier 2 decision, in `tier3_only` it allows. `enable_tier3=True` with no provider (and no `on_error="block"`) falls back to Tier 1 + Tier 2 and logs one warning per instance, so Tier 3 misconfiguration never silently disables defense.
 
 ### `allowed` vs `risk_level`
 
@@ -198,7 +283,7 @@ defense = create_prompt_defense(
 )
 ```
 
-### `defense.defend_tool_result(value, tool_name)`
+### `defense.defend_tool_result(value, tool_name, *, context=None)`
 
 Runs Tier 1 sanitization on risky fields of the **original** payload, then Tier 2 on strings from the SFE-filtered view when SFE is on (otherwise the full value). Optional `tier2_fields` restricts Tier 2 extraction to specific keys; omit it to classify **all** strings (matches `@stackone/defender` 0.6.3). **Synchronous** — no `await`.
 
@@ -231,7 +316,8 @@ class DefenseResult:
     cold_load: bool | None = None
     # Operational signals
     tier2_available: bool | None = None       # False when Tier 2 enabled but failed to load
-    coverage_degraded: bool | None = None      # True when Tier 1 detection coverage was capped
+    coverage_degraded: bool | None = None      # True when Tier 1 detection or the Tier 3 review didn't cover everything
+    tier3_chunk_summary: Tier3ChunkSummary | None = None  # tier3_only: chunks, blocked, blocking_chunk, oversize
 ```
 
 ### `defense.defend_tool_results(items)`
@@ -311,6 +397,25 @@ uv sync --group dev
 uv run pytest
 ```
 
+### Parity with the TypeScript package
+
+This package mirrors [`@stackone/defender`](https://github.com/StackOneHQ/defender). `tests/conformance/` is a pinned copy of its shared conformance fixtures: input → expected-output cases for Tier-3 serialization, prompt rendering, verdict parsing, error redaction and policy decisions. `tests/test_conformance.py` runs them against this package.
+
+- `scripts/sync_conformance.sh <ref>` copies the fixtures at a TS tag or commit and records it in `tests/conformance/VERSION`. CI checks the copy matches (`--check`).
+- Every case passes. A new TS case this package can't match yet goes in `tests/conformance_known_failures.txt`, where it runs as a strict xfail; remove its line once a port makes it pass.
+- Port contract for the runner: a rejected prompt template and an unusable verdict reply raise `ValueError`; optional result fields are `None` when unset (not `False`).
+
+Traps when porting from TypeScript (the fixtures pin TS behavior):
+
+- Lengths and slicing are in UTF-16 code units in TS, but `len()` counts code points.
+- TS `Object.entries` puts integer-like keys first. `json.loads` keeps insertion order.
+- Number rendering differs: `String(1.0)` is `"1"` but `str(1.0)` is `"1.0"`. Likewise `-0`, `1e-7` and `1e16` render differently in the two languages.
+- `core/tier3_input.py` handles the three traps above for Tier-3 input. It works on a UTF-16 view of each string, orders keys like `Object.entries`, and renders numbers like `String(n)`. One deliberate difference: Python ints below 1e21 print exactly, where JS would round past 2**53.
+- Python `re` has no `\p{Cf}`, and `unicodedata` versions differ across Python releases.
+- `str.splitlines()` also splits on `\x0b`, `\x0c` and `\x1c`, but TS splits only on `\r`, `\n`, `\u2028`, `\u2029` and `\u0085`.
+- JS `trim()` removes a BOM, but `str.strip()` doesn't.
+- `json.loads` accepts `NaN`, but JS `JSON.parse` rejects it. Use `parse_constant` to reject it.
+
 ## License
 
 Apache-2.0 — see [LICENSE](./LICENSE).

{stackone_defender-0.8.2 → stackone_defender-0.9.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "stackone-defender"
-version = "0.8.2"
+version = "0.9.1"
 description = "Indirect prompt injection defense for AI agents using tool calls"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -28,6 +28,8 @@ onnx = ["onnxruntime>=1.16.0", "tokenizers>=0.15.0", "numpy>=1.24.0"]
 # fasttext-ng provides the `fasttext` module (maintained bindings; supports 3.13).
 # Pulls numpy>=2.3; SFE still fail-opens when import/load fails.
 sfe = ["fasttext-ng>=0.9.3"]
+# create_openai_compatible_tier3_provider (Tier 3 over any OpenAI-compatible endpoint).
+llm = ["httpx>=0.27"]
 
 [dependency-groups]
 dev = [
@@ -38,6 +40,7 @@ dev = [
   "tokenizers>=0.15.0",
   "numpy>=1.24.0",
   "fasttext-ng>=0.9.3",
+  "httpx>=0.27",
 ]
 
 [build-system]

stackone_defender-0.9.1/scripts/sync_conformance.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# Sync the shared conformance fixtures from the TypeScript package (StackOneHQ/defender).
+#
+#   scripts/sync_conformance.sh <ref>   copy conformance/ at <ref> (a tag like defender-v0.9.2, or a commit SHA)
+#                                       into tests/conformance/ and record <ref> in tests/conformance/VERSION
+#   scripts/sync_conformance.sh --check verify tests/conformance/ matches the ref in VERSION (used in CI)
+set -euo pipefail
+
+REPO="StackOneHQ/defender"
+DEST="tests/conformance"
+
+# A full commit SHA or a release tag only: a branch would float, and anything else may not be a TS ref.
+valid_ref() {
+	[[ "$1" =~ ^[0-9a-f]{40}$ || "$1" =~ ^defender-v[0-9]+\.[0-9]+\.[0-9]+$ ]]
+}
+
+fetch() {
+	local ref="$1" out="$2"
+	valid_ref "$ref" || { echo "invalid ref '${ref}': use a full 40-char commit SHA or a defender-vX.Y.Z tag" >&2; exit 2; }
+	# Whole archive (small: LFS models come through as pointers); portable across GNU and BSD tar.
+	curl -fsSL --retry 3 --retry-delay 2 --max-time 60 "https://codeload.github.com/${REPO}/tar.gz/${ref}" | tar -xz -C "$out" --strip-components=1
+	[ -d "$out/conformance" ] || { echo "no conformance/ folder at ${REPO}@${ref}" >&2; exit 1; }
+}
+
+cd "$(dirname "$0")/.."
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+if [ "${1:-}" = "--check" ]; then
+	ref="$(tr -d '[:space:]' < "$DEST/VERSION")"
+	fetch "$ref" "$tmp"
+	if diff -r --exclude VERSION "$tmp/conformance" "$DEST"; then
+		echo "tests/conformance matches ${REPO}@${ref}"
+	else
+		echo "tests/conformance differs from ${REPO}@${ref}: re-run scripts/sync_conformance.sh ${ref}" >&2
+		exit 1
+	fi
+	exit 0
+fi
+
+ref="${1:?usage: $0 <ref> | --check}"
+fetch "$ref" "$tmp"
+rm -rf "$DEST"
+mkdir -p "$DEST"
+cp -R "$tmp/conformance/." "$DEST/"
+echo "$ref" > "$DEST/VERSION"
+echo "synced ${REPO}@${ref} → ${DEST}"

{stackone_defender-0.8.2 → stackone_defender-0.9.1}/src/stackone_defender/__init__.py

@@ -12,6 +12,18 @@ Usage:
 """
 
 from .classifiers.onnx_classifier import get_default_model_path
+from .classifiers.tier3_llm_provider import (
+    DEFAULT_TIER3_PROMPT,
+    TIER3_PROMPT_VERSION,
+    VERDICT_JSON_SCHEMA,
+    LlmCompletionRequest,
+    PromptInput,
+    Tier3PromptTemplate,
+    build_tier3_messages,
+    create_llm_tier3_provider,
+    parse_verdict_json,
+)
+from .classifiers.tier3_openai_compatible import create_openai_compatible_tier3_provider
 from .classifiers.tier3_orchestrator import get_default_tier3_provider, set_default_tier3_provider
 from .core.prompt_defense import PromptDefense, create_prompt_defense
 from .sfe.preprocess import (
@@ -28,6 +40,7 @@ from .types import (
     MultiheadConfig,
     RiskLevel,
     Tier1Result,
+    Tier3ChunkSummary,
     Tier3Provider,
     Tier3TokenUsage,
     Tier3Verdict,
@@ -35,25 +48,36 @@ from .types import (
 from .utils.boundary import contains_boundary_patterns, generate_boundary_instructions
 
 __all__ = [
+    "DEFAULT_TIER3_PROMPT",
+    "TIER3_PROMPT_VERSION",
+    "VERDICT_JSON_SCHEMA",
     "DefenderMode",
     "DefenseResult",
     "DropDecision",
+    "LlmCompletionRequest",
     "MultiheadConfig",
     "PromptDefense",
+    "PromptInput",
     "RiskLevel",
     "SfePredictor",
     "SfePreprocessResult",
     "Tier1Result",
+    "Tier3ChunkSummary",
+    "Tier3PromptTemplate",
     "Tier3Provider",
     "Tier3TokenUsage",
     "Tier3Verdict",
+    "build_tier3_messages",
     "contains_boundary_patterns",
+    "create_llm_tier3_provider",
+    "create_openai_compatible_tier3_provider",
     "create_prompt_defense",
     "generate_boundary_instructions",
     "get_default_model_path",
     "get_default_predictor",
     "get_default_sfe_model_path",
     "get_default_tier3_provider",
+    "parse_verdict_json",
     "set_default_tier3_provider",
     "sfe_preprocess",
 ]