weir-scan 0.1.0__tar.gz

weir_scan-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.uv_cache/
+dist/
+build/
+*.egg-info/

weir_scan-0.1.0/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.5
+Name: weir-scan
+Version: 0.1.0
+Summary: weir - deterministic, post-hoc verification engine for AI-agent sessions
+Project-URL: Homepage, https://github.com/IdoGol24/weir
+Project-URL: Repository, https://github.com/IdoGol24/weir
+Project-URL: Issues, https://github.com/IdoGol24/weir/issues
+Project-URL: Changelog, https://github.com/IdoGol24/weir/blob/main/weir/CHANGELOG.md
+Project-URL: Security, https://github.com/IdoGol24/weir/blob/main/weir/SECURITY.md
+License-Expression: Apache-2.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Requires-Dist: click>=8.1
+Requires-Dist: jinja2>=3.1
+Requires-Dist: msgspec>=0.18
+Description-Content-Type: text/markdown
+
+# Weir - unit tests for your agents
+
+[![CI](https://github.com/IdoGol24/weir/actions/workflows/ci.yml/badge.svg)](https://github.com/IdoGol24/weir/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/weir-scan)](https://pypi.org/project/weir-scan/) [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+
+<p align="center">
+  <img src="docs/assets/weir.jpg" alt="A weir: a low dam across a river, with water flowing evenly over its crest" width="720">
+  <br>
+  <em>A weir is a low dam built across a river to regulate and measure its
+  flow - the water keeps moving; the measurement happens anyway.</em>
+  <br>
+  <sub>Damhead Weir, Water of Leith. Photo by 501ghost, Wikimedia Commons, CC0.</sub>
+</p>
+
+You cannot unit-test an agent by string-matching its output, and an LLM
+judge drifts with its model. But what you actually need to assert is
+structural: did untrusted tool output reach an outbound sink, did the
+payment tool fire without its guard, did the secret leave the session.
+Those facts live in the traces your agent already emits.
+
+Weir is that assertion. Point it at an OpenTelemetry GenAI export; it
+exits `1` if a forbidden flow happened - with a witness path you can walk
+node by node - and `0` if not. Deterministic and byte-identical on every
+run, no LLM in the loop, no network access, never runs your agent (all
+tested guarantees). Like its namesake: the session keeps flowing; the
+measurement happens anyway.
+
+## Try it in two minutes
+
+```
+pip install weir-scan
+weir gauge your-export.jsonl   # or: weir gauge --sample
+```
+
+<!-- verify: gauge --sample -->
+```
+evidentiary coverage: 0%
+argument capture: 0%
+degraded: 100%
+tool arguments not captured - this scope is emitted by Traceloop/OpenLLMetry's LangChain instrumentation, which captures content to span attributes by default; check TRACELOOP_TRACE_CONTENT (false disables capture) in the traced service's environment
+  linkage: explicit (gen_ai.tool.call.id present)
+  payloads: absent - content capture is off
+at your current telemetry: coverage reporting YES - taint/scan NO
+content capture is off; for OTel GenAI instrumentations built on the util-genai layer, set OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental and OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY to capture gen_ai.input.messages / gen_ai.output.messages / gen_ai.tool.call.arguments and unlock cross-step analysis
+```
+
+The gauge answers the question every other tool skips: can your telemetry
+support the assertion you want to write? Most real exports cannot yet
+(content capture is off by default across the ecosystem), so it names the
+exact switch - derived from the instrumentation recorded in the trace
+itself. Flip it, re-run, and `weir scan` becomes your unit test:
+
+<!-- verify: scan fixtures/injection-exfil.json -->
+```
+1 verdict-grade finding(s)
+finding: injection-exfil-to-outbound-sink
+  source: financial_account_identifier at node 2 (tool_result)
+  sink: send_email at node 6
+  witness path: n2 -> n3 -> n4 -> n5 -> n6
+  join tiers crossed: explicit
+  verdict grade: yes
+  matched value: 22 chars
+```
+
+Exit `1` - fail the build. The finding carries its evidence, with the
+matched secret redacted. Structural facts do not flap: rewording and
+step-count variance cannot move them, and when evidence genuinely weakens
+a finding demotes with its reason stated instead of silently flipping.
+
+## Why you can trust it
+
+- **It reads your traces, not ours.** The suite includes a frozen JSONL
+  capture produced by `opentelemetry-sdk` and Google's protojson encoder
+  that no weir code touched ([provenance](fixtures/foreign/PROVENANCE.md)).
+  Ingestion is reject-narrow: only not-telemetry is refused; every real
+  malformation degrades under one of 18 named contract rows with a
+  remediation, generated from code and drift-tested:
+  [docs/contract.md](docs/contract.md).
+- **Attacker content cannot rewire it.** Joins follow evidence tiers
+  (explicit id, then span nesting, then content-mined); content-mined
+  evidence fills absences only, ambiguity is reported rather than
+  resolved, and a finding crossing a content-mined join is never
+  verdict-grade. If you can make attacker content do more than add
+  visible low-confidence noise, that is a security bug:
+  [SECURITY.md](SECURITY.md).
+- **The gauge is calibrated against known ground truth.** A paired
+  generator emits each scenario as native traces and OTLP-JSON from one
+  plan; the adapter's acceptance test is byte-for-byte equivalence, and
+  gauge numbers are pinned against corpora with known degradation,
+  mutation-proven.
+- **Every external claim is sourced.** Remediation strings that describe
+  someone else's software carry a recorded source and check date:
+  [REMEDIATION_SOURCES.md](REMEDIATION_SOURCES.md).
+
+## What ships, what does not
+
+Shipped: the OTel GenAI adapter (pinned `otel-genai/1.42.0`), session
+graph, verbatim taint and evaluation, the gauge and its capability ladder,
+HTML reports, the paired generator and corpora, one teaching rule
+(injection-to-exfiltration). Roadmap: the `weir diff` baseline gate, the
+rule-contribution gate and a broader rule set, signed bundles, more
+dialect rows.
+
+Open core, in writing: the engine - including the future diff gate - is
+Apache-2.0 permanently; authenticity, never entitlement. Full signed
+boundary: [docs/open-core.md](docs/open-core.md).
+
+Install name `weir-scan` (PyPI `weir` was taken); import `weir`; command
+`weir`. Apache-2.0.

weir_scan-0.1.0/README.md

@@ -0,0 +1,108 @@
+# Weir - unit tests for your agents
+
+[![CI](https://github.com/IdoGol24/weir/actions/workflows/ci.yml/badge.svg)](https://github.com/IdoGol24/weir/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/weir-scan)](https://pypi.org/project/weir-scan/) [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+
+<p align="center">
+  <img src="docs/assets/weir.jpg" alt="A weir: a low dam across a river, with water flowing evenly over its crest" width="720">
+  <br>
+  <em>A weir is a low dam built across a river to regulate and measure its
+  flow - the water keeps moving; the measurement happens anyway.</em>
+  <br>
+  <sub>Damhead Weir, Water of Leith. Photo by 501ghost, Wikimedia Commons, CC0.</sub>
+</p>
+
+You cannot unit-test an agent by string-matching its output, and an LLM
+judge drifts with its model. But what you actually need to assert is
+structural: did untrusted tool output reach an outbound sink, did the
+payment tool fire without its guard, did the secret leave the session.
+Those facts live in the traces your agent already emits.
+
+Weir is that assertion. Point it at an OpenTelemetry GenAI export; it
+exits `1` if a forbidden flow happened - with a witness path you can walk
+node by node - and `0` if not. Deterministic and byte-identical on every
+run, no LLM in the loop, no network access, never runs your agent (all
+tested guarantees). Like its namesake: the session keeps flowing; the
+measurement happens anyway.
+
+## Try it in two minutes
+
+```
+pip install weir-scan
+weir gauge your-export.jsonl   # or: weir gauge --sample
+```
+
+<!-- verify: gauge --sample -->
+```
+evidentiary coverage: 0%
+argument capture: 0%
+degraded: 100%
+tool arguments not captured - this scope is emitted by Traceloop/OpenLLMetry's LangChain instrumentation, which captures content to span attributes by default; check TRACELOOP_TRACE_CONTENT (false disables capture) in the traced service's environment
+  linkage: explicit (gen_ai.tool.call.id present)
+  payloads: absent - content capture is off
+at your current telemetry: coverage reporting YES - taint/scan NO
+content capture is off; for OTel GenAI instrumentations built on the util-genai layer, set OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental and OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY to capture gen_ai.input.messages / gen_ai.output.messages / gen_ai.tool.call.arguments and unlock cross-step analysis
+```
+
+The gauge answers the question every other tool skips: can your telemetry
+support the assertion you want to write? Most real exports cannot yet
+(content capture is off by default across the ecosystem), so it names the
+exact switch - derived from the instrumentation recorded in the trace
+itself. Flip it, re-run, and `weir scan` becomes your unit test:
+
+<!-- verify: scan fixtures/injection-exfil.json -->
+```
+1 verdict-grade finding(s)
+finding: injection-exfil-to-outbound-sink
+  source: financial_account_identifier at node 2 (tool_result)
+  sink: send_email at node 6
+  witness path: n2 -> n3 -> n4 -> n5 -> n6
+  join tiers crossed: explicit
+  verdict grade: yes
+  matched value: 22 chars
+```
+
+Exit `1` - fail the build. The finding carries its evidence, with the
+matched secret redacted. Structural facts do not flap: rewording and
+step-count variance cannot move them, and when evidence genuinely weakens
+a finding demotes with its reason stated instead of silently flipping.
+
+## Why you can trust it
+
+- **It reads your traces, not ours.** The suite includes a frozen JSONL
+  capture produced by `opentelemetry-sdk` and Google's protojson encoder
+  that no weir code touched ([provenance](fixtures/foreign/PROVENANCE.md)).
+  Ingestion is reject-narrow: only not-telemetry is refused; every real
+  malformation degrades under one of 18 named contract rows with a
+  remediation, generated from code and drift-tested:
+  [docs/contract.md](docs/contract.md).
+- **Attacker content cannot rewire it.** Joins follow evidence tiers
+  (explicit id, then span nesting, then content-mined); content-mined
+  evidence fills absences only, ambiguity is reported rather than
+  resolved, and a finding crossing a content-mined join is never
+  verdict-grade. If you can make attacker content do more than add
+  visible low-confidence noise, that is a security bug:
+  [SECURITY.md](SECURITY.md).
+- **The gauge is calibrated against known ground truth.** A paired
+  generator emits each scenario as native traces and OTLP-JSON from one
+  plan; the adapter's acceptance test is byte-for-byte equivalence, and
+  gauge numbers are pinned against corpora with known degradation,
+  mutation-proven.
+- **Every external claim is sourced.** Remediation strings that describe
+  someone else's software carry a recorded source and check date:
+  [REMEDIATION_SOURCES.md](REMEDIATION_SOURCES.md).
+
+## What ships, what does not
+
+Shipped: the OTel GenAI adapter (pinned `otel-genai/1.42.0`), session
+graph, verbatim taint and evaluation, the gauge and its capability ladder,
+HTML reports, the paired generator and corpora, one teaching rule
+(injection-to-exfiltration). Roadmap: the `weir diff` baseline gate, the
+rule-contribution gate and a broader rule set, signed bundles, more
+dialect rows.
+
+Open core, in writing: the engine - including the future diff gate - is
+Apache-2.0 permanently; authenticity, never entitlement. Full signed
+boundary: [docs/open-core.md](docs/open-core.md).
+
+Install name `weir-scan` (PyPI `weir` was taken); import `weir`; command
+`weir`. Apache-2.0.

weir_scan-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "weir-scan"
+version = "0.1.0"
+description = "weir - deterministic, post-hoc verification engine for AI-agent sessions"
+license = "Apache-2.0"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "msgspec>=0.18",
+    "click>=8.1",
+    "jinja2>=3.1",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3.12",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Quality Assurance",
+    "Topic :: Security",
+    "Typing :: Typed",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/weir"]
+
+[project.scripts]
+weir = "weir.cli.main:main"
+
+[project.urls]
+Homepage = "https://github.com/IdoGol24/weir"
+Repository = "https://github.com/IdoGol24/weir"
+Issues = "https://github.com/IdoGol24/weir/issues"
+Changelog = "https://github.com/IdoGol24/weir/blob/main/weir/CHANGELOG.md"
+Security = "https://github.com/IdoGol24/weir/blob/main/weir/SECURITY.md"

weir_scan-0.1.0/schema_artifacts/canonical_trace.v1.schema.json

@@ -0,0 +1,289 @@
+{
+  "$defs": {
+    "CanonicalTrace": {
+      "additionalProperties": false,
+      "properties": {
+        "joins": {
+          "items": {
+            "$ref": "#/$defs/JoinRecord"
+          },
+          "type": "array"
+        },
+        "metadata": {
+          "$ref": "#/$defs/TraceMetadata"
+        },
+        "nodes": {
+          "items": {
+            "$ref": "#/$defs/TraceNode"
+          },
+          "type": "array"
+        },
+        "schema_version": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "schema_version",
+        "nodes",
+        "joins",
+        "metadata"
+      ],
+      "title": "CanonicalTrace",
+      "type": "object"
+    },
+    "JoinConfidence": {
+      "description": "Join precedence per R1.4, amended by the M4 design (section 3):\n    explicit id > parent/child nesting > content-mined > heuristic.\n    CONTENT_MINED never competes with the envelope tiers - it fills absences\n    only, and any finding whose witness path traverses it is never\n    verdict-grade. HEURISTIC is named by R1.4 but nothing assigns it in M4.",
+      "enum": [
+        "content_mined",
+        "explicit",
+        "heuristic",
+        "nested"
+      ],
+      "title": "JoinConfidence"
+    },
+    "JoinRecord": {
+      "additionalProperties": false,
+      "description": "Ties a tool_call node to its tool_result node (R1.4). `join_confidence`\n    is carried permanently - never discarded after joining - because it feeds\n    the \u00a77 verdict-grade decision downstream. `join_source` is the MECHANISM\n    detail (which attribute / which mined pattern, per dialect); it is\n    adapter-determined provenance and sits in the M4 equivalence\n    normalization set, so the enum never grows for mechanism reasons.",
+      "properties": {
+        "join_confidence": {
+          "$ref": "#/$defs/JoinConfidence"
+        },
+        "join_source": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null
+        },
+        "tool_call_source_ref": {
+          "type": "string"
+        },
+        "tool_result_source_ref": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "tool_call_source_ref",
+        "tool_result_source_ref",
+        "join_confidence"
+      ],
+      "title": "JoinRecord",
+      "type": "object"
+    },
+    "LlmCallPayload": {
+      "additionalProperties": false,
+      "properties": {
+        "content": {
+          "type": "string"
+        },
+        "type": {
+          "enum": [
+            "llm_call"
+          ]
+        }
+      },
+      "required": [
+        "type",
+        "content"
+      ],
+      "title": "LlmCallPayload",
+      "type": "object"
+    },
+    "NodeKind": {
+      "enum": [
+        "llm_call",
+        "tool_call",
+        "tool_result",
+        "user_input"
+      ],
+      "title": "NodeKind"
+    },
+    "ToolCallPayload": {
+      "additionalProperties": false,
+      "properties": {
+        "args": {
+          "additionalProperties": {
+            "type": [
+              "null",
+              "boolean",
+              "integer",
+              "number",
+              "string",
+              "array",
+              "object"
+            ]
+          },
+          "type": "object"
+        },
+        "tool_name": {
+          "type": "string"
+        },
+        "type": {
+          "enum": [
+            "tool_call"
+          ]
+        }
+      },
+      "required": [
+        "type",
+        "tool_name",
+        "args"
+      ],
+      "title": "ToolCallPayload",
+      "type": "object"
+    },
+    "ToolResultPayload": {
+      "additionalProperties": false,
+      "properties": {
+        "content": {
+          "type": "string"
+        },
+        "type": {
+          "enum": [
+            "tool_result"
+          ]
+        }
+      },
+      "required": [
+        "type",
+        "content"
+      ],
+      "title": "ToolResultPayload",
+      "type": "object"
+    },
+    "TraceMetadata": {
+      "additionalProperties": false,
+      "description": "R1.6: adapter + detected framework versions, so the Gauge can flag drift.",
+      "properties": {
+        "adapter_name": {
+          "type": "string"
+        },
+        "adapter_version": {
+          "type": "string"
+        },
+        "framework_name": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null
+        },
+        "framework_version": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null
+        },
+        "instrumentation_scope": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null
+        }
+      },
+      "required": [
+        "adapter_name",
+        "adapter_version"
+      ],
+      "title": "TraceMetadata",
+      "type": "object"
+    },
+    "TraceNode": {
+      "additionalProperties": false,
+      "properties": {
+        "actor": {
+          "type": "string"
+        },
+        "degraded": {
+          "default": false,
+          "type": "boolean"
+        },
+        "id": {
+          "type": "string"
+        },
+        "kind": {
+          "$ref": "#/$defs/NodeKind"
+        },
+        "payload": {
+          "anyOf": [
+            {
+              "$ref": "#/$defs/ToolCallPayload"
+            },
+            {
+              "$ref": "#/$defs/ToolResultPayload"
+            },
+            {
+              "$ref": "#/$defs/LlmCallPayload"
+            },
+            {
+              "$ref": "#/$defs/UserInputPayload"
+            }
+          ],
+          "discriminator": {
+            "mapping": {
+              "llm_call": "#/$defs/LlmCallPayload",
+              "tool_call": "#/$defs/ToolCallPayload",
+              "tool_result": "#/$defs/ToolResultPayload",
+              "user_input": "#/$defs/UserInputPayload"
+            },
+            "propertyName": "type"
+          }
+        },
+        "source_ref": {
+          "type": "string"
+        },
+        "timestamp": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "id",
+        "kind",
+        "timestamp",
+        "actor",
+        "source_ref",
+        "payload"
+      ],
+      "title": "TraceNode",
+      "type": "object"
+    },
+    "UserInputPayload": {
+      "additionalProperties": false,
+      "properties": {
+        "content": {
+          "type": "string"
+        },
+        "type": {
+          "enum": [
+            "user_input"
+          ]
+        }
+      },
+      "required": [
+        "type",
+        "content"
+      ],
+      "title": "UserInputPayload",
+      "type": "object"
+    }
+  },
+  "$ref": "#/$defs/CanonicalTrace"
+}

weir_scan-0.1.0/src/weir/adapters/__init__.py

@@ -0,0 +1,5 @@
+"""Input adapters: foreign telemetry -> CanonicalTrace.
+
+Outside the pure analysis core (the purity contract forbids core modules
+from importing this package) and constrained by its own contract to reach
+only the weir.schema seam."""

weir_scan-0.1.0/src/weir/adapters/otel/__init__.py

@@ -0,0 +1,17 @@
+"""OTel GenAI OTLP-JSON adapter (M4): bytes -> CanonicalTrace + ledger.
+
+Reject-narrow: OtlpRejectError (exit-2 material) fires only for
+not-JSON / not-OTLP-shaped input. Everything else degrades, named."""
+
+from weir.adapters.otel._contract import REMEDIATION, Degradation, DegradationReason
+from weir.adapters.otel._map import ADAPTER_NAME, ADAPTER_VERSION, AdapterResult, map_wire
+from weir.adapters.otel._wire import OtlpRejectError, decode_input
+
+__all__ = [
+    "ADAPTER_NAME", "ADAPTER_VERSION", "AdapterResult", "Degradation",
+    "DegradationReason", "OtlpRejectError", "REMEDIATION", "adapt_otlp",
+]
+
+
+def adapt_otlp(data: bytes) -> AdapterResult:
+    return map_wire(decode_input(data))