trajtest 0.1.1__tar.gz

trajtest-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Calas Automations Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

trajtest-0.1.1/PKG-INFO

@@ -0,0 +1,176 @@
+Metadata-Version: 2.4
+Name: trajtest
+Version: 0.1.1
+Summary: pytest for agent trajectories: assert on what an agent did, not just what it said
+Author: Calas Automations
+License: MIT
+Project-URL: Homepage, https://calasautomations.com
+Keywords: agents,eval,evals,testing,trajectory,pytest
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pytest>=7
+Dynamic: license-file
+
+# trajtest — pytest for agent trajectories
+
+**Assert on what an agent *did*, not just what it *said*.**
+
+`trajtest` runs an agent against a mock world (in-memory FS, mock HTTP, KV
+store) and lets you assert on its **trajectory** — which tools it called, in
+what order, with what arguments, and what side effects it left behind.
+Deterministic. Local-first. No LLM-as-judge. CI-ready.
+
+```python
+from trajtest import Expect, World, run
+
+def my_agent(task, tools):
+    rows = tools.http_get("https://api.example.com/rows").json()["rows"]
+    tools.fs_write("/out/report.md", f"total: {sum(rows)}")
+    return "done"
+
+def test_agent_saves_before_claiming():
+    world = World()
+    world.http.mock("GET", "https://api.example.com/rows", json={"rows": [1, 2, 3]})
+
+    traj = run(my_agent, "summarize the rows", world, max_steps=15)
+
+    (Expect(traj)
+        .called("http_get", url__contains="api.example.com")
+        .then("fs_write", path="/out/report.md")
+        .never_called("fs_delete")
+        .completed_within(15)
+        .finished()
+        .world_file("/out/report.md", equals="total: 6")  # the world, not the narration
+        .assert_all())
+```
+
+## Why
+
+Prompt-tests can't see the four ways agents actually fail:
+
+1. **Confident false reports** — the agent says "saved the file" but never
+   called write. `world_file()` checks the mock world's final state, not the
+   narration.
+2. **Silent policy violations** — narration says "I'll be careful", tool calls
+   did a delete. `never_called("fs_delete")` catches it.
+3. **Evidence-free passes** — 94% pass rate, nobody knows what was verified.
+   Every `EvalFailure` lists each assertion's outcome plus the full trajectory
+   transcript. An eval with zero assertions **fails loudly**.
+4. **Meaningless coverage** — green suite, dangerous tools never exercised.
+   `traj.unused_tools(TOOLS)` flags what the agent never touched.
+
+## Install
+
+Not on PyPI yet; install from a clone of this repo:
+
+```bash
+pip install .
+```
+
+## Core concepts
+
+**World** — the mock environment. Strict by design: missing files raise
+`FileNotFoundError`, unmocked HTTP raises `UnmockedRequestError`. A world
+that forgives everything produces agents that pass evals and fail in prod.
+
+- `world.fs` — in-memory filesystem (`read`/`write`/`append`/`delete`/
+  `exists`/`listdir`/`glob`; str or bytes; `**` glob support)
+- `world.http` — mock HTTP (`mock(method, url, status=..., json=...,
+  text=..., error="timeout"|"connection", times=n)`; exact then `*`
+  wildcard matching; repeated mocks form a consumable queue — mock a 500
+  then a 200 to test retry logic)
+- `world.kv` — key-value scratch space; `world.clock` — controllable clock
+- `world.reset()` — wipe everything between scenarios
+
+**run(agent, task, world, max_steps=25, seed=None)** — the agent is a plain
+function `(task, tools) -> str`. Every tool call is recorded (name, args,
+result, duration, errors). Tool errors are recorded *and* re-raised, so the
+agent can react to them. `tools.note("...")` captures narration separately —
+notes are never steps and never pollute sequences, so you can compare what
+the agent *said* against what it *did*. Crashes become `traj.finished=False`
+with `traj.failure` set; exceeding `max_steps` is a trajectory failure, not
+a hang.
+
+**sample(agent, task, world_factory, n=5)** — for stochastic agents: n runs
+against fresh worlds. Seeds are passed when the agent accepts a `seed`
+keyword. Assert consistency with a loop:
+
+```python
+for t in sample(agent, "task", make_world, n=5):
+    Expect(t).never_called("fs_delete").completed_within(15).assert_all()
+```
+
+**Expect(traj)** — the assertion DSL. All methods chain; `assert_all()`
+reports *every* failure at once (fail-collecting, not fail-fast):
+
+| Assertion | Meaning |
+|---|---|
+| `called(name, *args, **kwargs)` | ≥1 call matching (positional prefix, kwarg subset, matchers ok) |
+| `called_times / at_least / at_most` | call counts |
+| `never_called(name, ...)` | forbids calls (optionally arg-scoped) |
+| `in_order(a, b, c)` | ordered subsequence, gaps allowed |
+| `then(name, ...)` | fluent chain: must occur after previous `then` |
+| `then_immediately(name, ...)` | strict adjacency required |
+| `first / last` | boundary calls |
+| `completed_within(n)` / `finished()` | step budget / clean completion |
+| `no_repeated_calls(max_repeats=2, name=None)` | loop detector: fails if an identical (tool, args) call happens more than `max_repeats` times; different args don't count |
+| `final_answer(matcher-or-value)` | the agent's return value |
+| `notes_contain / notes_match` | narration assertions |
+| `world_file(path, equals/contains/matches)` | final world state — the anti-lying assertion |
+| `world(desc, predicate)` / `custom(desc, fn)` | escape hatches |
+
+Kwarg matching uses **parameter names**, so `called("http_get",
+url__contains="api.example.com")` works whether the agent passed the URL
+positionally or by keyword. Suffix sugar: `__contains __regex
+__startswith __endswith __gt __lt __ge __le`. Matchers (`ANY`, `Contains`,
+`Regex`, `StartsWith`, `EndsWith`, `Gt/Lt/Ge/Le`, `InstanceOf`,
+`Predicate`, `DictSubset`) compose anywhere.
+
+**load_trace(path)** — assert on a *real* recorded run. Reads a
+[redbg](https://github.com/calasdan-sketch/redbg) trace (`.jsonl` or
+`.jsonl.gz`; stdlib only, redbg is not imported): each tool call/result pair
+becomes a call with its args as keyword args, the last response becomes the
+final answer, notes stay notes, and a `session_end` status of `ok` means
+finished. A call with no recorded result gets `error="no result recorded"`.
+Call, order and answer assertions work; `world_file` / `world` raise
+`EvalFailure` because a recording has no mock World.
+
+```python
+traj = load_trace("run.jsonl")
+Expect(traj).in_order("read_file", "write_file").never_called("delete_file").assert_all()
+```
+
+**traj.save_trace(path)** — the reverse: write a trajectory as a redbg
+trace (`.gz` suffix gzips it), so a failing eval can be replayed with
+`redbg replay` or compared with `redbg diff`. Args are written by parameter
+name; outputs are the recorded result reprs (truncated at 500 chars), so the
+file is for debugging, not a lossless copy of the mock world.
+
+**pytest plugin** — a `world` fixture (fresh `World` per test) is
+auto-registered. **CLI** — `trajtest init` scaffolds an example eval;
+`trajtest run <path>` runs evals through pytest.
+
+## Limitations (honest)
+
+- **trajtest can't tell you your assertions are meaningful.**
+  `Expect(t).finished().assert_all()` is a weak eval and it will pass.
+  Assert on trajectories and world state, not vibes.
+- **The mock world is not the real world.** No network latency, no real
+  filesystem permissions, no file metadata (size/mtime), no concurrent
+  access. Fidelity gaps are where false confidence breeds — keep mocks
+  strict and review them.
+- **Deterministic assertions vs stochastic agents** is an inherent tension.
+  Use `sample()`, seeds, and order-flexible assertions. If your agent is
+  wildly non-deterministic, trajectory evals will be brittle — that's a
+  signal about the agent, not just the eval.
+- **Single-threaded.** Agents that spawn threads or subprocesses calling
+  tools concurrently are not supported.
+- **No framework adapters in v1.** The agent is `(task, tools) -> str`;
+  wrapping a LangChain/CrewAI loop takes ~5 lines on your side.
+- **No LLM-as-judge**, by design. Semantic judgments belong in a custom
+  `predicate`, not in the core.
+
+## License
+
+MIT

trajtest-0.1.1/README.md

@@ -0,0 +1,162 @@
+# trajtest — pytest for agent trajectories
+
+**Assert on what an agent *did*, not just what it *said*.**
+
+`trajtest` runs an agent against a mock world (in-memory FS, mock HTTP, KV
+store) and lets you assert on its **trajectory** — which tools it called, in
+what order, with what arguments, and what side effects it left behind.
+Deterministic. Local-first. No LLM-as-judge. CI-ready.
+
+```python
+from trajtest import Expect, World, run
+
+def my_agent(task, tools):
+    rows = tools.http_get("https://api.example.com/rows").json()["rows"]
+    tools.fs_write("/out/report.md", f"total: {sum(rows)}")
+    return "done"
+
+def test_agent_saves_before_claiming():
+    world = World()
+    world.http.mock("GET", "https://api.example.com/rows", json={"rows": [1, 2, 3]})
+
+    traj = run(my_agent, "summarize the rows", world, max_steps=15)
+
+    (Expect(traj)
+        .called("http_get", url__contains="api.example.com")
+        .then("fs_write", path="/out/report.md")
+        .never_called("fs_delete")
+        .completed_within(15)
+        .finished()
+        .world_file("/out/report.md", equals="total: 6")  # the world, not the narration
+        .assert_all())
+```
+
+## Why
+
+Prompt-tests can't see the four ways agents actually fail:
+
+1. **Confident false reports** — the agent says "saved the file" but never
+   called write. `world_file()` checks the mock world's final state, not the
+   narration.
+2. **Silent policy violations** — narration says "I'll be careful", tool calls
+   did a delete. `never_called("fs_delete")` catches it.
+3. **Evidence-free passes** — 94% pass rate, nobody knows what was verified.
+   Every `EvalFailure` lists each assertion's outcome plus the full trajectory
+   transcript. An eval with zero assertions **fails loudly**.
+4. **Meaningless coverage** — green suite, dangerous tools never exercised.
+   `traj.unused_tools(TOOLS)` flags what the agent never touched.
+
+## Install
+
+Not on PyPI yet; install from a clone of this repo:
+
+```bash
+pip install .
+```
+
+## Core concepts
+
+**World** — the mock environment. Strict by design: missing files raise
+`FileNotFoundError`, unmocked HTTP raises `UnmockedRequestError`. A world
+that forgives everything produces agents that pass evals and fail in prod.
+
+- `world.fs` — in-memory filesystem (`read`/`write`/`append`/`delete`/
+  `exists`/`listdir`/`glob`; str or bytes; `**` glob support)
+- `world.http` — mock HTTP (`mock(method, url, status=..., json=...,
+  text=..., error="timeout"|"connection", times=n)`; exact then `*`
+  wildcard matching; repeated mocks form a consumable queue — mock a 500
+  then a 200 to test retry logic)
+- `world.kv` — key-value scratch space; `world.clock` — controllable clock
+- `world.reset()` — wipe everything between scenarios
+
+**run(agent, task, world, max_steps=25, seed=None)** — the agent is a plain
+function `(task, tools) -> str`. Every tool call is recorded (name, args,
+result, duration, errors). Tool errors are recorded *and* re-raised, so the
+agent can react to them. `tools.note("...")` captures narration separately —
+notes are never steps and never pollute sequences, so you can compare what
+the agent *said* against what it *did*. Crashes become `traj.finished=False`
+with `traj.failure` set; exceeding `max_steps` is a trajectory failure, not
+a hang.
+
+**sample(agent, task, world_factory, n=5)** — for stochastic agents: n runs
+against fresh worlds. Seeds are passed when the agent accepts a `seed`
+keyword. Assert consistency with a loop:
+
+```python
+for t in sample(agent, "task", make_world, n=5):
+    Expect(t).never_called("fs_delete").completed_within(15).assert_all()
+```
+
+**Expect(traj)** — the assertion DSL. All methods chain; `assert_all()`
+reports *every* failure at once (fail-collecting, not fail-fast):
+
+| Assertion | Meaning |
+|---|---|
+| `called(name, *args, **kwargs)` | ≥1 call matching (positional prefix, kwarg subset, matchers ok) |
+| `called_times / at_least / at_most` | call counts |
+| `never_called(name, ...)` | forbids calls (optionally arg-scoped) |
+| `in_order(a, b, c)` | ordered subsequence, gaps allowed |
+| `then(name, ...)` | fluent chain: must occur after previous `then` |
+| `then_immediately(name, ...)` | strict adjacency required |
+| `first / last` | boundary calls |
+| `completed_within(n)` / `finished()` | step budget / clean completion |
+| `no_repeated_calls(max_repeats=2, name=None)` | loop detector: fails if an identical (tool, args) call happens more than `max_repeats` times; different args don't count |
+| `final_answer(matcher-or-value)` | the agent's return value |
+| `notes_contain / notes_match` | narration assertions |
+| `world_file(path, equals/contains/matches)` | final world state — the anti-lying assertion |
+| `world(desc, predicate)` / `custom(desc, fn)` | escape hatches |
+
+Kwarg matching uses **parameter names**, so `called("http_get",
+url__contains="api.example.com")` works whether the agent passed the URL
+positionally or by keyword. Suffix sugar: `__contains __regex
+__startswith __endswith __gt __lt __ge __le`. Matchers (`ANY`, `Contains`,
+`Regex`, `StartsWith`, `EndsWith`, `Gt/Lt/Ge/Le`, `InstanceOf`,
+`Predicate`, `DictSubset`) compose anywhere.
+
+**load_trace(path)** — assert on a *real* recorded run. Reads a
+[redbg](https://github.com/calasdan-sketch/redbg) trace (`.jsonl` or
+`.jsonl.gz`; stdlib only, redbg is not imported): each tool call/result pair
+becomes a call with its args as keyword args, the last response becomes the
+final answer, notes stay notes, and a `session_end` status of `ok` means
+finished. A call with no recorded result gets `error="no result recorded"`.
+Call, order and answer assertions work; `world_file` / `world` raise
+`EvalFailure` because a recording has no mock World.
+
+```python
+traj = load_trace("run.jsonl")
+Expect(traj).in_order("read_file", "write_file").never_called("delete_file").assert_all()
+```
+
+**traj.save_trace(path)** — the reverse: write a trajectory as a redbg
+trace (`.gz` suffix gzips it), so a failing eval can be replayed with
+`redbg replay` or compared with `redbg diff`. Args are written by parameter
+name; outputs are the recorded result reprs (truncated at 500 chars), so the
+file is for debugging, not a lossless copy of the mock world.
+
+**pytest plugin** — a `world` fixture (fresh `World` per test) is
+auto-registered. **CLI** — `trajtest init` scaffolds an example eval;
+`trajtest run <path>` runs evals through pytest.
+
+## Limitations (honest)
+
+- **trajtest can't tell you your assertions are meaningful.**
+  `Expect(t).finished().assert_all()` is a weak eval and it will pass.
+  Assert on trajectories and world state, not vibes.
+- **The mock world is not the real world.** No network latency, no real
+  filesystem permissions, no file metadata (size/mtime), no concurrent
+  access. Fidelity gaps are where false confidence breeds — keep mocks
+  strict and review them.
+- **Deterministic assertions vs stochastic agents** is an inherent tension.
+  Use `sample()`, seeds, and order-flexible assertions. If your agent is
+  wildly non-deterministic, trajectory evals will be brittle — that's a
+  signal about the agent, not just the eval.
+- **Single-threaded.** Agents that spawn threads or subprocesses calling
+  tools concurrently are not supported.
+- **No framework adapters in v1.** The agent is `(task, tools) -> str`;
+  wrapping a LangChain/CrewAI loop takes ~5 lines on your side.
+- **No LLM-as-judge**, by design. Semantic judgments belong in a custom
+  `predicate`, not in the core.
+
+## License
+
+MIT

trajtest-0.1.1/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "trajtest"
+version = "0.1.1"
+description = "pytest for agent trajectories: assert on what an agent did, not just what it said"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "Calas Automations" }]
+keywords = ["agents", "eval", "evals", "testing", "trajectory", "pytest"]
+dependencies = ["pytest>=7"]
+
+[project.urls]
+Homepage = "https://calasautomations.com"
+
+[project.scripts]
+trajtest = "trajtest.cli:main"
+
+[project.entry-points.pytest11]
+trajtest = "trajtest.pytest_plugin"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

trajtest-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

trajtest-0.1.1/src/trajtest/__init__.py

@@ -0,0 +1,61 @@
+"""trajtest — pytest for agent trajectories.
+
+Assert on what an agent *did* (tool calls, order, args, world side effects),
+not just what it *said*. Local-first, deterministic, no LLM-as-judge.
+"""
+
+from trajtest.errors import EvalFailure, MaxStepsExceeded, UnmockedRequestError
+from trajtest.expect import Expect
+from trajtest.matchers import (
+    ANY,
+    Contains,
+    DictSubset,
+    EndsWith,
+    Ge,
+    Gt,
+    InstanceOf,
+    Le,
+    Lt,
+    Matcher,
+    Predicate,
+    Regex,
+    StartsWith,
+)
+from trajtest.recorder import TOOLS, Note, ToolCall, Trajectory, run, sample
+from trajtest.trace_io import load_trace
+from trajtest.world import Clock, FakeFS, FakeHTTP, FakeResponse, KVStore, World
+
+__version__ = "0.1.1"
+
+__all__ = [
+    "ANY",
+    "TOOLS",
+    "Clock",
+    "Contains",
+    "DictSubset",
+    "EndsWith",
+    "EvalFailure",
+    "Expect",
+    "FakeFS",
+    "FakeHTTP",
+    "FakeResponse",
+    "Ge",
+    "Gt",
+    "InstanceOf",
+    "KVStore",
+    "Le",
+    "Lt",
+    "Matcher",
+    "MaxStepsExceeded",
+    "Note",
+    "Predicate",
+    "Regex",
+    "StartsWith",
+    "ToolCall",
+    "Trajectory",
+    "UnmockedRequestError",
+    "World",
+    "load_trace",
+    "run",
+    "sample",
+]

trajtest-0.1.1/src/trajtest/cli.py

@@ -0,0 +1,138 @@
+"""trajtest CLI: scaffold evals and run them."""
+
+import argparse
+import sys
+
+from trajtest import __version__
+
+SCAFFOLD = '''"""Example trajectory eval. Run with: trajtest run __SCAFFOLD_NAME__"""
+
+import pytest
+
+from trajtest import EvalFailure, Expect, World, run
+
+
+def helpful_agent(task, tools):
+    """A well-behaved agent: fetches data, saves a report, says what it did."""
+    resp = tools.http_get("https://api.example.com/rows")
+    rows = resp.json()["rows"]
+    total = sum(rows)
+    tools.fs_write("/out/report.md", f"# Report\\n\\ntotal: {total}\\n")
+    tools.note("saved the report with the computed total")
+    return f"done, total={total}"
+
+
+def make_world():
+    world = World()
+    world.http.mock("GET", "https://api.example.com/rows", json={"rows": [1, 2, 3]})
+    return world
+
+
+def test_agent_fetches_then_saves():
+    traj = run(helpful_agent, "summarize the rows", make_world(), max_steps=10)
+    (Expect(traj)
+        .called("http_get", url__contains="api.example.com")
+        .then("fs_write", path="/out/report.md")
+        .never_called("fs_delete")
+        .completed_within(10)
+        .finished()
+        .world_file("/out/report.md", contains="total: 6")
+        .assert_all())
+
+
+def test_agent_does_not_lie_about_saving():
+    """The narration claims a save; the mock world proves otherwise.
+
+    This test *expects* EvalFailure — that IS the demo: trajtest caught the
+    confident false report. Green suite, lying agent exposed.
+    """
+    def liar(task, tools):
+        tools.note("saved the report, all done!")
+        return "saved the report"
+
+    traj = run(liar, "summarize the rows", make_world(), max_steps=10)
+    with pytest.raises(EvalFailure):
+        (Expect(traj)
+            .finished()
+            # Signature is world(description, predicate) — describe first.
+            .world("report actually exists in the world",
+                   lambda w: w.fs.exists("/out/report.md"))
+            .assert_all())
+'''
+
+
+def cmd_init(args) -> int:
+    import os
+
+    os.makedirs(args.dir, exist_ok=True)
+    name = "test_trajtest_example.py"
+    path = os.path.join(args.dir, name)
+    if os.path.exists(path) and not args.force:
+        print(f"{path} already exists (use --force to overwrite)")
+        return 1
+    with open(path, "w", encoding="utf-8") as f:
+        f.write(SCAFFOLD.replace("__SCAFFOLD_NAME__", name))
+    print(f"wrote {path}")
+    print("run it with: trajtest run " + path)
+    return 0
+
+
+def cmd_run(args) -> int:
+    import pytest
+
+    # The pytest11 entry point auto-loads trajtest.pytest_plugin, so no
+    # explicit -p flag (which would double-register on repeated in-process runs).
+    # importlib mode: repeated in-process runs of files with the same basename
+    # (e.g. two scaffolds in different dirs) must not collide in sys.modules,
+    # which made pytest return INTERRUPTED intermittently.
+    argv = [args.path, "--import-mode=importlib", "-p", "no:cacheprovider"]
+    if args.k:
+        argv += ["-k", args.k]
+    if args.verbose:
+        argv.append("-v")
+    else:
+        argv.append("-q")
+    argv += args.pytest_args
+    print(f"trajtest run: {args.path}")
+    code = pytest.main(argv)
+    if code == 0:
+        print("trajtest: all evals passed")
+    else:
+        print("trajtest: some evals failed (see above)")
+    return code
+
+
+def main(argv=None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="trajtest", description="pytest for agent trajectories"
+    )
+    parser.add_argument(
+        "--version", action="version", version=f"trajtest {__version__}"
+    )
+    sub = parser.add_subparsers(dest="cmd", required=True)
+
+    p_init = sub.add_parser("init", help="scaffold an example eval file")
+    p_init.add_argument("--dir", default=".", help="directory to write into")
+    p_init.add_argument("--force", action="store_true", help="overwrite existing file")
+    p_init.set_defaults(func=cmd_init)
+
+    p_run = sub.add_parser("run", help="run trajectory evals with pytest")
+    p_run.add_argument("path", help="test file or directory")
+    p_run.add_argument("-k", help="pytest -k expression")
+    p_run.add_argument("-v", "--verbose", action="store_true")
+    p_run.add_argument(
+        "pytest_args",
+        nargs=argparse.REMAINDER,
+        help="extra args passed to pytest (after --)",
+    )
+    p_run.set_defaults(func=cmd_run)
+
+    args = parser.parse_args(argv)
+    # strip a leading "--" from passthrough args
+    if getattr(args, "pytest_args", None) and args.pytest_args[:1] == ["--"]:
+        args.pytest_args = args.pytest_args[1:]
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

trajtest-0.1.1/src/trajtest/errors.py

@@ -0,0 +1,21 @@
+"""Errors raised by trajtest."""
+
+
+class EvalFailure(AssertionError):
+    """Raised by Expect.assert_all() when one or more assertions fail.
+
+    Carries the full fail-collected report: every assertion outcome plus a
+    trajectory transcript, so eval authors debug in one pass.
+    """
+
+    def __init__(self, report: str):
+        super().__init__(report)
+        self.report = report
+
+
+class MaxStepsExceeded(Exception):
+    """Raised inside run() when the agent exceeds its step budget."""
+
+
+class UnmockedRequestError(Exception):
+    """Raised by FakeHTTP when no mock matches a request (strict mode)."""