agentlahon 0.0.1__tar.gz

agentlahon-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 agentcheck contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agentlahon-0.0.1/PKG-INFO

@@ -0,0 +1,187 @@
+Metadata-Version: 2.4
+Name: agentlahon
+Version: 0.0.1
+Summary: Unit tests for AI agents — catch when your agent does the wrong thing, not just when it says the wrong thing.
+Author-email: Anurag Lahon <anuraglahondp@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/anuraglahon16/agentcheck
+Project-URL: Repository, https://github.com/anuraglahon16/agentcheck
+Project-URL: Issues, https://github.com/anuraglahon16/agentcheck/issues
+Keywords: ai,agents,evaluation,evals,llm,testing,governance
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Provides-Extra: judge
+Requires-Dist: openai>=1; extra == "judge"
+Provides-Extra: webapp
+Requires-Dist: flask>=3; extra == "webapp"
+Dynamic: license-file
+
+# agentlahon
+
+### Unit tests for AI agents — catch when your agent *does* the wrong thing, not just when it *says* the wrong thing.
+
+![tests](https://img.shields.io/badge/tests-passing-2fbf71) ![python](https://img.shields.io/badge/python-3.9%2B-blue) ![license](https://img.shields.io/badge/license-MIT-black) ![deps](https://img.shields.io/badge/core%20deps-zero-black)
+
+> **Status:** v0.0.1, experimental — APIs may change. Core is zero-dependency. Feedback and first users very welcome.
+
+Text-only evals grade what an agent *says*. But agents take *actions* — call tools, write to databases, move money. The dangerous failure is when the reply looks perfect while the actions are wrong. **agentlahon grades both**, and maps every check to a governance control so a green suite doubles as an audit-ready **AI assurance report**.
+
+```
+FAIL  decline-only-noncarried
+    ✓ reply mentions 'unable'          found                          ← SAID the right thing
+    ✓ never records a sale             no such action ✓
+    ✗ never restocks (money moves)     took 1×: reorder Invitation-cards ×9000   ← DID the wrong thing
+```
+
+> That failure is invisible to every text-based eval. Only checking the agent's **actions** catches it. This is the beachhead: single-turn LLM eval is crowded — **agent** eval (multi-step, tool-calling, side-effecting) is wide open.
+
+```python
+Scenario("decline-only", "5000 flyers, 10000 tickets", checks=[
+    expect.output_contains("unable"),      # say-level
+    expect.no_action("reorder_item",       # do-level — the part text evals miss
+        where=lambda a: a.result.get("ordered")),   # assert on the *effect*, not just the call
+])
+```
+
+## Install
+
+```bash
+pip install agentlahon     # PyPI distribution name
+pip install -e .            # or from source
+```
+
+Core is dependency-free. Python ≥ 3.9.
+
+## Quickstart
+
+```python
+from agentlahon import Scenario, expect, evaluate, print_terminal, write_html
+
+def agent(text):              # wrap YOUR agent -> AgentRun(output, actions)
+    ...
+
+scenarios = [
+    Scenario("decline-noncarried", "5000 flyers, 10000 tickets", checks=[
+        expect.output_contains("unable"),
+        expect.no_action("reorder_item"),   # the do-level check text evals miss
+        expect.no_pii(),
+    ]),
+]
+report = evaluate(agent, scenarios)
+print_terminal(report)
+write_html(report)            # shareable assurance_report.html
+```
+
+## The evals flywheel — look at data → tag → generate assertions
+
+The hard part of evals isn't running assertions, it's *knowing what to assert*. agentlahon logs real runs, lets you review and tag failures (open coding), rolls them into a failure taxonomy, and turns a tagged-bad trace into the assertion that would have caught it — the Hamel Husain / Shreya Shankar error-analysis loop, for agent actions.
+
+```bash
+agentlahon run suite.py --log traces.jsonl   # 1. log real runs
+agentlahon review traces.jsonl               # 2. page through, tag failures
+agentlahon analyze traces.jsonl              # 3. failure taxonomy (what dominates)
+agentlahon suggest traces.jsonl t0003        # 4. trace -> ready-to-paste assertion
+```
+
+```
+[t0003] input: 5000 flyers, 2000 posters, 10000 tickets
+      trace:  1. reorder_item(Invitation cards, 9000) → ordered=True
+      suggested assertions:
+        expect.no_action("reorder_item", where=lambda a: (a.result or {}).get("ordered"))
+            # reorder_item took effect — guard against it when it should not fire
+```
+
+## LLM-as-judge — for subjective checks, aligned before you trust it
+
+Code assertions can't judge "is this reply faithful / on-policy / correct for our domain?" — that needs an LLM judge. agentlahon does it the rigorous way: **binary** verdicts, an **optional domain reference** to grade against, and an **alignment** step that scores the judge against *your* human labels. An unaligned judge is worse than none.
+
+```python
+from agentlahon import expect, openai_complete, llm_judge, align, print_alignment
+
+complete = openai_complete(model="gpt-4o-mini")          # pluggable; pip install "agentlahon[judge]"
+
+check = expect.judge("Does the reply stay within our refund policy?",
+                     complete=complete,
+                     reference=open("refund_policy.md").read())   # optional domain doc
+
+# Don't trust the judge until it agrees with you:
+labeled = [("we'll refund within 30 days", True), ("sure, full refund anytime", False), ...]
+print_alignment(align(llm_judge("within policy?", complete), labeled))
+#  accuracy 92%  TPR 95%  TNR 88%  κ 0.83  -> TRUSTWORTHY
+```
+
+A judge that rubber-stamps everything scores **TNR 0% → NEEDS WORK** — caught before it hides real failures.
+
+## Traces — *what* went wrong, not just *that* it did
+
+When a check fails, agentlahon prints the agent's action trace and points at the exact step:
+
+```
+✗ never restocks (money moves)   took 1×: [Invitation cards ×9000]
+trace (what the agent did):
+  · 1. tool_reorder_item(Flyers, 5050)          → ordered=False   (refused, harmless)
+  · 2. tool_reorder_item(Poster paper, 2050)    → ordered=False   (refused, harmless)
+  ✗ 3. tool_reorder_item(Invitation cards, 9000)→ ordered=True    ← never restocks (money moves)
+```
+
+The failing check is linked to the offending step (`ScenarioResult.blame()`), so you go from red to root cause instantly. Same trace renders in the HTML report.
+
+## CLI (drop into CI)
+
+A suite file defines `scenarios` and `agent`; the CLI exits non-zero on findings:
+
+```bash
+agentlahon run examples/suite.py --html report.html
+```
+
+## Run the examples (no API key)
+
+```bash
+python examples/run_demo.py       # synthetic agent with a planted bug
+python examples/run_beavers.py    # points at a REAL pydantic-ai agent, catches a real regression
+pytest                            # 6 core tests
+```
+
+## Layout
+
+```
+src/agentlahon/   core · checks · adapters · report · cli
+examples/         run_demo · run_beavers · suite
+tests/            test_core
+```
+
+## Checks
+
+| Family | Checks | Control (NIST AI RMF) |
+|---|---|---|
+| say-level | `output_contains`, `output_absent` | MEASURE-2.3 Task performance |
+| **do-level** | `no_action`, `action_taken`, `max_actions`, `actions_only_on` | MANAGE-2.1 Action safety |
+| privacy | `no_pii` | MEASURE-2.10 Privacy |
+| transparency | `no_internal_leak` | MEASURE-2.9 Transparency |
+| faithfulness | `faithful(judge)` — plug in LLM-as-judge | MEASURE-2.5 Validity |
+
+## Roadmap (v0 → product)
+
+- [x] Function-capture adapter (auto-records real tool calls **and effects**)
+- [x] CI integration (`agentlahon run` exits non-zero on findings)
+- [ ] Native adapters for OpenAI/Anthropic tool-calls & LangGraph traces
+- [ ] LLM-as-judge faithfulness + bias checks
+- [ ] Regression mode: diff a run against a saved baseline on model/prompt change
+- [ ] Hosted dashboard + shareable report links (the paid layer)
+
+Status: **v0.0.1** — installable package, CLI, effect-level checks, terminal + HTML
+report, tests, and a working run against a real pydantic-ai agent.

agentlahon-0.0.1/README.md

@@ -0,0 +1,155 @@
+# agentlahon
+
+### Unit tests for AI agents — catch when your agent *does* the wrong thing, not just when it *says* the wrong thing.
+
+![tests](https://img.shields.io/badge/tests-passing-2fbf71) ![python](https://img.shields.io/badge/python-3.9%2B-blue) ![license](https://img.shields.io/badge/license-MIT-black) ![deps](https://img.shields.io/badge/core%20deps-zero-black)
+
+> **Status:** v0.0.1, experimental — APIs may change. Core is zero-dependency. Feedback and first users very welcome.
+
+Text-only evals grade what an agent *says*. But agents take *actions* — call tools, write to databases, move money. The dangerous failure is when the reply looks perfect while the actions are wrong. **agentlahon grades both**, and maps every check to a governance control so a green suite doubles as an audit-ready **AI assurance report**.
+
+```
+FAIL  decline-only-noncarried
+    ✓ reply mentions 'unable'          found                          ← SAID the right thing
+    ✓ never records a sale             no such action ✓
+    ✗ never restocks (money moves)     took 1×: reorder Invitation-cards ×9000   ← DID the wrong thing
+```
+
+> That failure is invisible to every text-based eval. Only checking the agent's **actions** catches it. This is the beachhead: single-turn LLM eval is crowded — **agent** eval (multi-step, tool-calling, side-effecting) is wide open.
+
+```python
+Scenario("decline-only", "5000 flyers, 10000 tickets", checks=[
+    expect.output_contains("unable"),      # say-level
+    expect.no_action("reorder_item",       # do-level — the part text evals miss
+        where=lambda a: a.result.get("ordered")),   # assert on the *effect*, not just the call
+])
+```
+
+## Install
+
+```bash
+pip install agentlahon     # PyPI distribution name
+pip install -e .            # or from source
+```
+
+Core is dependency-free. Python ≥ 3.9.
+
+## Quickstart
+
+```python
+from agentlahon import Scenario, expect, evaluate, print_terminal, write_html
+
+def agent(text):              # wrap YOUR agent -> AgentRun(output, actions)
+    ...
+
+scenarios = [
+    Scenario("decline-noncarried", "5000 flyers, 10000 tickets", checks=[
+        expect.output_contains("unable"),
+        expect.no_action("reorder_item"),   # the do-level check text evals miss
+        expect.no_pii(),
+    ]),
+]
+report = evaluate(agent, scenarios)
+print_terminal(report)
+write_html(report)            # shareable assurance_report.html
+```
+
+## The evals flywheel — look at data → tag → generate assertions
+
+The hard part of evals isn't running assertions, it's *knowing what to assert*. agentlahon logs real runs, lets you review and tag failures (open coding), rolls them into a failure taxonomy, and turns a tagged-bad trace into the assertion that would have caught it — the Hamel Husain / Shreya Shankar error-analysis loop, for agent actions.
+
+```bash
+agentlahon run suite.py --log traces.jsonl   # 1. log real runs
+agentlahon review traces.jsonl               # 2. page through, tag failures
+agentlahon analyze traces.jsonl              # 3. failure taxonomy (what dominates)
+agentlahon suggest traces.jsonl t0003        # 4. trace -> ready-to-paste assertion
+```
+
+```
+[t0003] input: 5000 flyers, 2000 posters, 10000 tickets
+      trace:  1. reorder_item(Invitation cards, 9000) → ordered=True
+      suggested assertions:
+        expect.no_action("reorder_item", where=lambda a: (a.result or {}).get("ordered"))
+            # reorder_item took effect — guard against it when it should not fire
+```
+
+## LLM-as-judge — for subjective checks, aligned before you trust it
+
+Code assertions can't judge "is this reply faithful / on-policy / correct for our domain?" — that needs an LLM judge. agentlahon does it the rigorous way: **binary** verdicts, an **optional domain reference** to grade against, and an **alignment** step that scores the judge against *your* human labels. An unaligned judge is worse than none.
+
+```python
+from agentlahon import expect, openai_complete, llm_judge, align, print_alignment
+
+complete = openai_complete(model="gpt-4o-mini")          # pluggable; pip install "agentlahon[judge]"
+
+check = expect.judge("Does the reply stay within our refund policy?",
+                     complete=complete,
+                     reference=open("refund_policy.md").read())   # optional domain doc
+
+# Don't trust the judge until it agrees with you:
+labeled = [("we'll refund within 30 days", True), ("sure, full refund anytime", False), ...]
+print_alignment(align(llm_judge("within policy?", complete), labeled))
+#  accuracy 92%  TPR 95%  TNR 88%  κ 0.83  -> TRUSTWORTHY
+```
+
+A judge that rubber-stamps everything scores **TNR 0% → NEEDS WORK** — caught before it hides real failures.
+
+## Traces — *what* went wrong, not just *that* it did
+
+When a check fails, agentlahon prints the agent's action trace and points at the exact step:
+
+```
+✗ never restocks (money moves)   took 1×: [Invitation cards ×9000]
+trace (what the agent did):
+  · 1. tool_reorder_item(Flyers, 5050)          → ordered=False   (refused, harmless)
+  · 2. tool_reorder_item(Poster paper, 2050)    → ordered=False   (refused, harmless)
+  ✗ 3. tool_reorder_item(Invitation cards, 9000)→ ordered=True    ← never restocks (money moves)
+```
+
+The failing check is linked to the offending step (`ScenarioResult.blame()`), so you go from red to root cause instantly. Same trace renders in the HTML report.
+
+## CLI (drop into CI)
+
+A suite file defines `scenarios` and `agent`; the CLI exits non-zero on findings:
+
+```bash
+agentlahon run examples/suite.py --html report.html
+```
+
+## Run the examples (no API key)
+
+```bash
+python examples/run_demo.py       # synthetic agent with a planted bug
+python examples/run_beavers.py    # points at a REAL pydantic-ai agent, catches a real regression
+pytest                            # 6 core tests
+```
+
+## Layout
+
+```
+src/agentlahon/   core · checks · adapters · report · cli
+examples/         run_demo · run_beavers · suite
+tests/            test_core
+```
+
+## Checks
+
+| Family | Checks | Control (NIST AI RMF) |
+|---|---|---|
+| say-level | `output_contains`, `output_absent` | MEASURE-2.3 Task performance |
+| **do-level** | `no_action`, `action_taken`, `max_actions`, `actions_only_on` | MANAGE-2.1 Action safety |
+| privacy | `no_pii` | MEASURE-2.10 Privacy |
+| transparency | `no_internal_leak` | MEASURE-2.9 Transparency |
+| faithfulness | `faithful(judge)` — plug in LLM-as-judge | MEASURE-2.5 Validity |
+
+## Roadmap (v0 → product)
+
+- [x] Function-capture adapter (auto-records real tool calls **and effects**)
+- [x] CI integration (`agentlahon run` exits non-zero on findings)
+- [ ] Native adapters for OpenAI/Anthropic tool-calls & LangGraph traces
+- [ ] LLM-as-judge faithfulness + bias checks
+- [ ] Regression mode: diff a run against a saved baseline on model/prompt change
+- [ ] Hosted dashboard + shareable report links (the paid layer)
+
+Status: **v0.0.1** — installable package, CLI, effect-level checks, terminal + HTML
+report, tests, and a working run against a real pydantic-ai agent.

agentlahon-0.0.1/pyproject.toml

@@ -0,0 +1,47 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "agentlahon"
+version = "0.0.1"
+description = "Unit tests for AI agents — catch when your agent does the wrong thing, not just when it says the wrong thing."
+readme = "README.md"
+requires-python = ">=3.9"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Anurag Lahon", email = "anuraglahondp@gmail.com" }]
+keywords = ["ai", "agents", "evaluation", "evals", "llm", "testing", "governance"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Testing",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Typing :: Typed",
+]
+dependencies = []          # core is intentionally dependency-free
+
+[project.optional-dependencies]
+dev = ["pytest>=7"]
+judge = ["openai>=1"]      # only needed for a live LLM-as-judge
+webapp = ["flask>=3"]      # the dashboard product surface
+
+[project.scripts]
+agentlahon = "agentlahon.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/anuraglahon16/agentcheck"
+Repository = "https://github.com/anuraglahon16/agentcheck"
+Issues = "https://github.com/anuraglahon16/agentcheck/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+agentlahon = ["py.typed"]

agentlahon-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

agentlahon-0.0.1/src/agentlahon/__init__.py

@@ -0,0 +1,31 @@
+"""agentlahon — unit tests for AI agents.
+
+Catch when your agent does the wrong thing, not just when it says the wrong
+thing. Score say-level (reply) and do-level (actions/side-effects) behavior,
+mapped to governance controls.
+
+    from agentlahon import Scenario, expect, evaluate, print_terminal
+
+    scenarios = [
+        Scenario("decline-noncarried", "5000 flyers, 10000 tickets", checks=[
+            expect.output_contains("unable"),
+            expect.no_action("reorder_item"),   # <- the do-level check text evals miss
+            expect.no_pii(),
+        ]),
+    ]
+    report = evaluate(my_agent_adapter, scenarios)
+    print_terminal(report)
+"""
+from .core import Action, AgentRun, Scenario, Check, Report, evaluate
+from .checks import expect, CONTROLS
+from .report import print_terminal, write_html
+from .adapters import capture
+from .store import TraceStore, TraceRecord
+from .analysis import taxonomy, suggest_checks
+from .judge import llm_judge, openai_complete, align, print_alignment
+
+__all__ = ["Action", "AgentRun", "Scenario", "Check", "Report", "evaluate",
+           "expect", "CONTROLS", "print_terminal", "write_html", "capture",
+           "TraceStore", "TraceRecord", "taxonomy", "suggest_checks",
+           "llm_judge", "openai_complete", "align", "print_alignment"]
+__version__ = "0.0.1"

agentlahon-0.0.1/src/agentlahon/adapters.py

@@ -0,0 +1,72 @@
+"""Adapters that auto-capture an agent's actions so you don't wire them by hand.
+
+`capture()` wraps a set of functions on a module (typically the tool functions
+an agent calls) and records every invocation as an `Action`. Point it at your
+real code, run the agent, and you get the `AgentRun.actions` list for free —
+this is how agentlahon sees *what the agent did*, not just what it said.
+
+Works with any framework whose tools bottom out in Python callables:
+pydantic-ai, LangChain, or a hand-rolled loop. For pydantic-ai specifically,
+the tool wrappers call module-level `tool_*` functions, so capturing those
+records the real side-effects the model triggered.
+"""
+from __future__ import annotations
+
+import json
+from contextlib import contextmanager
+from typing import Dict, Iterable, List
+
+from .core import Action
+
+
+def _parse_result(value):
+    """Parse a tool's return into a dict when possible, so checks can inspect it."""
+    if isinstance(value, dict):
+        return value
+    if isinstance(value, str):
+        try:
+            return json.loads(value)
+        except (ValueError, TypeError):
+            return value
+    return value
+
+
+def _summarize(tool: str, args: tuple, kwargs: dict) -> Dict:
+    """Best-effort, readable arg snapshot — keeps the report legible."""
+    snap = dict(kwargs)
+    for i, a in enumerate(args):
+        # skip long/opaque positional blobs (JSON strings, dates); keep short ones
+        if isinstance(a, (int, float)):
+            snap[f"arg{i}"] = a
+        elif isinstance(a, str) and len(a) <= 60:
+            snap[f"arg{i}"] = a
+    return snap
+
+
+@contextmanager
+def capture(module, names: Iterable[str]):
+    """Temporarily wrap ``module.<name>`` funcs to log calls; restore on exit.
+
+    Yields a live list of `Action`s. Callers that reference the wrapped names as
+    module globals (the usual case) will hit the wrapper, so real tool calls are
+    recorded without touching the agent's code.
+    """
+    log: List[Action] = []
+    originals = {}
+    for name in names:
+        originals[name] = getattr(module, name)
+
+        def make(n, orig):
+            def wrapper(*args, **kwargs):
+                out = orig(*args, **kwargs)
+                log.append(Action(tool=n, args=_summarize(n, args, kwargs),
+                                  result=_parse_result(out)))
+                return out
+            return wrapper
+
+        setattr(module, name, make(name, originals[name]))
+    try:
+        yield log
+    finally:
+        for name, orig in originals.items():
+            setattr(module, name, orig)

agentlahon-0.0.1/src/agentlahon/analysis.py

@@ -0,0 +1,77 @@
+"""Error analysis + trace→assertion suggestions.
+
+Two jobs from the evals playbook:
+  * `taxonomy()` — cluster tagged failures by their open-coded label, so you
+    see which failure modes dominate (the "axial coding" rollup).
+  * `suggest_checks()` — turn a tagged-bad trace into concrete `expect.*`
+    assertions that would have caught it. This closes the loop: look at data →
+    label failure → generate the eval.
+"""
+from __future__ import annotations
+
+from typing import Dict, List, Tuple
+
+from .core import Action
+from .store import TraceRecord, TraceStore
+
+_EFFECT_KEYS = ("ordered", "recorded", "sold", "success", "created", "sent")
+_ITEM_ARGS = ("item", "name", "product", "arg0")
+
+
+def taxonomy(store: TraceStore) -> Dict[str, int]:
+    """{failure-label: count} over tagged failures — most common first."""
+    counts: Dict[str, int] = {}
+    for r in store.failures():
+        label = r.annotation.label or "(unlabeled)"
+        counts[label] = counts.get(label, 0) + 1
+    return dict(sorted(counts.items(), key=lambda kv: -kv[1]))
+
+
+def _effect_key(a: Action) -> str:
+    if isinstance(a.result, dict):
+        for k in _EFFECT_KEYS:
+            if a.result.get(k):
+                return k
+    return ""
+
+
+def suggest_checks(rec: TraceRecord) -> List[Tuple[str, str]]:
+    """Propose (assertion_code, rationale) for a bad trace.
+
+    Heuristics over what the agent actually DID, so the suggestions are grounded
+    in this trace — not a generic checklist.
+    """
+    out: List[Tuple[str, str]] = []
+    seen = set()
+
+    for a in rec.actions:
+        key = _effect_key(a)
+        if key and a.tool not in seen:
+            seen.add(a.tool)
+            out.append((
+                f'expect.no_action("{a.tool}", '
+                f'where=lambda a: (a.result or {{}}).get("{key}"))',
+                f'{a.tool} took effect ({key}=True) — guard against it when it should not fire',
+            ))
+
+    # scope guard if actions carry an item-like arg
+    for a in rec.actions:
+        arg = next((k for k in _ITEM_ARGS if k in a.args), None)
+        if arg and (a.tool, "scope") not in seen:
+            seen.add((a.tool, "scope"))
+            out.append((
+                f'expect.actions_only_on("{a.tool}", "{arg}", ALLOWED)',
+                f'{a.tool} acted on {a.args.get(arg)!r} — restrict it to an allow-set',
+            ))
+            break
+
+    # privacy guard if the reply looks like it leaked something
+    low = rec.output.lower()
+    if "@" in rec.output or any(t in low for t in ("margin", "cost", "internal")):
+        out.append(('expect.no_pii()  # and expect.no_internal_leak()',
+                    "reply may leak PII or internal info"))
+
+    if not out:
+        out.append(('expect.output_absent("<bad phrase>")',
+                    "no action effects found — assert on the reply text instead"))
+    return out