@testdriverai/agent 7.11.83-test → 7.11.84-test
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testdriver:agent
|
|
3
|
+
description: How the TestDriver agent behaves on GitHub issues, pull requests, and @mentions
|
|
4
|
+
---
|
|
5
|
+
<!-- Generated from agent.mdx. DO NOT EDIT. -->
|
|
6
|
+
|
|
7
|
+
The TestDriver agent is an AI teammate that lives in your GitHub repository. It's
|
|
8
|
+
**code review that runs your app** — instead of only reading a diff and guessing
|
|
9
|
+
what might break, it starts a real sandbox, drives the real app, and reports what
|
|
10
|
+
actually happens.
|
|
11
|
+
|
|
12
|
+
## Scope: it only builds and maintains tests
|
|
13
|
+
|
|
14
|
+
The agent is focused. The one thing it does is **write, debug, fix, and maintain
|
|
15
|
+
automated tests** with TestDriver, and review pull requests by writing and running
|
|
16
|
+
a test against the change. It will politely decline off-scope work — it won't edit
|
|
17
|
+
your application code, fix product bugs, answer unrelated programming questions, or
|
|
18
|
+
act as a general coding assistant.
|
|
19
|
+
|
|
20
|
+
## How it behaves depends on the surface
|
|
21
|
+
|
|
22
|
+
The agent reacts to three different GitHub surfaces, and its behavior is
|
|
23
|
+
noticeably different on each.
|
|
24
|
+
|
|
25
|
+
| Surface | Trigger | What it does |
|
|
26
|
+
|---------|---------|--------------|
|
|
27
|
+
| **Pull request opened** | Automatic (no mention) | Silent, evidence-based **code review** |
|
|
28
|
+
| **Issue opened** | Automatic (no mention) | **Onboarding / test-building** |
|
|
29
|
+
| **@mention** (issue or PR) | You mention the bot | Interactive, chatty conversation |
|
|
30
|
+
|
|
31
|
+
## Pull requests — automatic code review
|
|
32
|
+
|
|
33
|
+
When a pull request is **opened**, the agent reviews it automatically — nobody has
|
|
34
|
+
to ask.
|
|
35
|
+
|
|
36
|
+
- **The review is the deliverable, not chatter.** An automatic review is
|
|
37
|
+
unsolicited, so the agent stays quiet while it works and posts a **single review
|
|
38
|
+
comment** at the end rather than scattering inline threads across the diff or
|
|
39
|
+
narrating each step.
|
|
40
|
+
- **It runs the change when it can.** Its default first move is to start a session,
|
|
41
|
+
work out what the PR changes from a user's point of view, and write and run a
|
|
42
|
+
TestDriver test that exercises exactly that. A bug it *watched happen* is far
|
|
43
|
+
more valuable than one it inferred.
|
|
44
|
+
- **Findings are labeled by how they were found.** Each finding is marked
|
|
45
|
+
`observed` (proven by actually running the change) or `suspected` (reasoned out
|
|
46
|
+
from reading the diff). If the installation isn't signed in to TestDriver, the
|
|
47
|
+
agent can't run anything, so it reviews by reading and marks every finding
|
|
48
|
+
`suspected`.
|
|
49
|
+
- **It focuses on real bugs.** Logic errors, regressions, broken edge cases,
|
|
50
|
+
mishandled errors — not style, naming, or "consider extracting this."
|
|
51
|
+
- **Zero findings is a good review.** If the change looks correct, it says so in a
|
|
52
|
+
sentence. If there's nothing worth reviewing (only lockfiles, generated files, or
|
|
53
|
+
formatting), it posts nothing at all.
|
|
54
|
+
- **It comments, it never blocks.** The review is always a `COMMENT` — it never
|
|
55
|
+
approves and never requests changes, so it can't gate a merge.
|
|
56
|
+
|
|
57
|
+
It also fires a native GitHub **Check Run** for the PR's TestDriver suite, and
|
|
58
|
+
re-runs that check on every push (not a full re-review — push more commits and
|
|
59
|
+
@mention the agent if you want it to review again).
|
|
60
|
+
|
|
61
|
+
<Note>
|
|
62
|
+
The agent never reviews its own pull requests. PRs opened by the bot are skipped,
|
|
63
|
+
so it won't wake itself up to review the tests it just committed.
|
|
64
|
+
</Note>
|
|
65
|
+
|
|
66
|
+
## Issues — onboarding and test building
|
|
67
|
+
|
|
68
|
+
When an issue is **opened**, the agent treats it as a cue to help you build a test.
|
|
69
|
+
|
|
70
|
+
- **If the issue describes a task**, it acts on the request — writing or debugging
|
|
71
|
+
the test you asked for.
|
|
72
|
+
- **If the issue is empty**, it introduces itself, explains that it writes and
|
|
73
|
+
debugs computer-use tests, and walks you through signing in to TestDriver and
|
|
74
|
+
creating your first test.
|
|
75
|
+
- **It finishes by opening a pull request** with the test it wrote.
|
|
76
|
+
|
|
77
|
+
## @mentions — interactive conversation
|
|
78
|
+
|
|
79
|
+
Mentioning the bot (on either an issue or a PR) starts a **solicited**, interactive
|
|
80
|
+
turn. Because you asked directly, the agent is chatty: it posts progress updates,
|
|
81
|
+
shows you screenshots of what's happening in the sandbox, and answers follow-ups in
|
|
82
|
+
the thread.
|
|
83
|
+
|
|
84
|
+
On a PR, an @mention overrides the silent auto-review behavior — you'll see the full
|
|
85
|
+
play-by-play instead of just the final review comment.
|
|
86
|
+
|
|
87
|
+
You can also manage the agent from a comment:
|
|
88
|
+
|
|
89
|
+
- **`@bot logout`** (or "sign out") revokes the installation's shared TestDriver
|
|
90
|
+
sign-in and tears down the live sandbox, so the next mention will prompt whoever
|
|
91
|
+
responds to sign in again.
|
|
92
|
+
|
|
93
|
+
## Signing in
|
|
94
|
+
|
|
95
|
+
Reading a diff needs nothing, but **running** the app needs a TestDriver session,
|
|
96
|
+
which requires the installation to be signed in. Sign-in state doesn't decide
|
|
97
|
+
*whether* the agent reviews — it decides *how good* the review gets. A signed-out
|
|
98
|
+
install still gets its bugs caught from reading the code; signing in upgrades
|
|
99
|
+
findings from `suspected` to `observed`.
|