@testdriverai/agent 7.11.240-test → 7.11.241-canary

package/ai/skills/testdriver-agent/SKILL.md

@@ -1,99 +1,182 @@
 ---
 name: testdriver:agent
-description: How the TestDriver agent behaves on GitHub issues, pull requests, and @mentions
+description: How the TestDriver agent reviews pull requests, builds tests, and answers @mentions on GitHub.
 ---
 <!-- Generated from agent.mdx. DO NOT EDIT. -->
 
-The TestDriver agent is an AI teammate in your GitHub repository. It is
-**code review that runs your app**. It does not only read a diff and guess
-what can break. It starts a real sandbox, controls the real app, and reports what
-really happens.
-
-## Scope: it only builds and maintains tests
-
-The agent has one purpose. It does one thing: it **writes, debugs, fixes, and keeps
-automated tests** with TestDriver. It reviews pull requests. It writes and runs
-a test against the change. It does not do work that is not in scope. It does not edit
-your application code, fix product bugs, answer other programming questions, or
-act as a general coding assistant.
-
-## How it behaves depends on the surface
-
-The agent reacts to three different GitHub surfaces. Its behavior is
-different on each one.
-
-| Surface | Trigger | What it does |
-|---------|---------|--------------|
-| **Pull request opened** | Automatic (no mention) | Silent, evidence-based **code review** |
-| **Issue opened** | Automatic (no mention) | **Onboarding / test-building** |
-| **@mention** (issue or PR) | You mention the bot | Interactive, chatty conversation |
-
-## Pull requests — automatic code review
-
-When a pull request is **opened**, the agent reviews it automatically. Nobody has
-to ask.
-
-- **The review is the deliverable, not chatter.** No person asks for an automatic review.
-  Thus the agent stays quiet while it works. It posts a **single review
-  comment** at the end. It does not put many inline threads across the diff. It does not
-  tell each step.
-- **It runs the change when it can.** Its default first move is to start a session,
-  work out what the PR changes from a user's point of view, and write and run a
-  TestDriver test that exercises exactly that. A bug it *watched happen* is far
-  more valuable than one it inferred.
-- **Findings are labeled by how they were found.** Each finding is marked
-  `observed` (proven by actually running the change) or `suspected` (reasoned out
-  from reading the diff). If the installation isn't signed in to TestDriver, the
-  agent can't run anything, so it reviews by reading and marks every finding
-  `suspected`.
-- **It focuses on real bugs.** Logic errors, regressions, broken edge cases,
-  mishandled errors — not style, naming, or "consider extracting this."
-- **Zero findings is a good review.** If the change looks correct, it says so in a
-  sentence. If there's nothing worth reviewing (only lockfiles, generated files, or
-  formatting), it posts nothing at all.
-- **It comments, it never blocks.** The review is always a `COMMENT` — it never
-  approves and never requests changes, so it can't gate a merge.
-
-It also fires a native GitHub **Check Run** for the PR's TestDriver suite, and
-re-runs that check on every push (not a full re-review — push more commits and
-@mention the agent if you want it to review again).
-
-<Note>
-  The agent never reviews its own pull requests. PRs opened by the bot are skipped,
-  so it won't wake itself up to review the tests it just committed.
-</Note>
-
-## Issues — onboarding and test building
-
-When an issue is **opened**, the agent treats it as a cue to help you build a test.
-
-- **If the issue describes a task**, it acts on the request — writing or debugging
-  the test you asked for.
-- **If the issue is empty**, it introduces itself, explains that it writes and
-  debugs computer-use tests, and walks you through signing in to TestDriver and
-  creating your first test.
-- **It finishes by opening a pull request** with the test it wrote.
-
-## @mentions — interactive conversation
-
-Mentioning the bot (on either an issue or a PR) starts a **solicited**, interactive
-turn. Because you asked directly, the agent is chatty: it posts progress updates,
-shows you screenshots of what's happening in the sandbox, and answers follow-ups in
-the thread.
-
-On a PR, an @mention overrides the silent auto-review behavior — you'll see the full
-play-by-play instead of just the final review comment.
-
-You can also manage the agent from a comment:
-
-- **`@bot logout`** (or "sign out") revokes the installation's shared TestDriver
-  sign-in and tears down the live sandbox, so the next mention will prompt whoever
-  responds to sign in again.
+The TestDriver agent is an AI teammate that lives in your GitHub repository.
+
+- It reviews every pull request's code for bugs.
+- For user-visible changes, it drives your app in a real sandbox and reports what happened.
+- It writes, runs, and maintains your end-to-end tests.
+
+To install it, see [Add TestDriver to GitHub](/quickstart-github).
+
+## What the agent does
+
+The agent does two jobs:
+
+- **Code review.** It reviews pull requests like a senior engineer. It reads the diff for logic errors, security problems, and regressions. When it can, it runs the change in a sandbox to prove what it finds.
+- **Test building.** It writes new TestDriver tests, debugs and fixes failing ones, and keeps them up to date.
+
+It declines work outside that scope:
+
+- It doesn't edit your application code.
+- It doesn't fix product bugs.
+- It doesn't act as a general coding assistant.
+
+When it finds a bug in a pull request, it reports the bug instead of pushing a fix.
+
+## How it responds on each surface
+
+The agent behaves differently depending on what woke it up:
+
+| Event | Trigger | What the agent does |
+| --- | --- | --- |
+| Pull request opened, or new commits pushed | Automatic | Reviews the change and posts one review comment |
+| Issue opened | Automatic | Acts on the request in the issue, or onboards a new user |
+| @mention on an issue or pull request | You mention the agent | Works with you interactively in the thread |
+
+Automatic turns are unsolicited, so the agent posts only when it has something useful to say.
+
+When you @mention it, someone is waiting. It always replies and narrates its progress.
+
+## Pull request reviews
+
+The agent reviews a pull request without being asked when it's opened or gets new commits.
+
+<Steps>
+  <Step title="Triage the change">
+
+    The agent reads the changed files and decides whether there's anything to review or run.
+
+    A pull request that only touches lockfiles, generated output, or formatting gets an 👀 reaction and no comment.
+
+  </Step>
+  <Step title="Read the code">
+
+    It reviews the diff like a senior engineer. It opens surrounding files only when the diff isn't enough on its own.
+
+    It looks for:
+
+    - Logic errors
+    - Unhandled error paths
+    - Security problems
+    - Race conditions
+    - Broken contracts for callers
+    - Risky logic with no tests
+
+    It skips style issues that a linter would catch.
+
+  </Step>
+  <Step title="Run the change">
+
+    This step runs if the change has a user-visible surface and your installation is signed in to TestDriver. The agent:
+
+    - Starts a sandbox.
+    - Drives the exact flow the diff touches.
+    - Runs any existing tests that cover it.
+    - Writes and runs a new test for the change.
+
+    While it works, a single comment on the pull request shows the latest screenshot. The comment finishes with the run recording.
+
+  </Step>
+  <Step title="Post the review">
+
+    It posts one review comment with:
+
+    - A summary and verdict
+    - A short walkthrough of each meaningful file
+    - Its findings
+
+    On a re-review after a push, it reassesses its earlier findings against the new code. It drops the fixed ones and checks the new commits for regressions.
+
+  </Step>
+</Steps>
+
+### What's in a finding
+
+Each finding has:
+
+- A file and line
+- A severity: `critical`, `major`, `minor`, or `nit`
+- A category
+- A suggested replacement, when the fix is small and certain
+
+Findings are also labelled by how the agent found them:
+
+- **`observed`**: the agent saw the bug happen during a run in the sandbox.
+- **`suspected`**: the agent inferred the bug from reading the code.
+
+If a run confirms a suspected finding, the agent upgrades it to `observed` instead of listing it twice.
+
+If the change looks correct, the review says so in a sentence and has no findings.
+
+### The review never blocks a merge
+
+Reviews are posted as GitHub `COMMENT` reviews. The agent never approves a pull request or requests changes, so it can't gate a merge.
+
+### Tests from a review
+
+During a review, the agent may write a test worth keeping: one that passes, or one that cleanly reproduces a bug.
+
+It opens a separate pull request for that test and links it from the review. The pull request uses a `testdriver/session/…` branch and targets your default branch.
+
+It never pushes to the branch it's reviewing.
+
+### Pull requests the agent skips
+
+- **Draft pull requests** get no review until they're marked ready for review.
+- **Pull requests opened by bots** are never reviewed. This includes the agent's own test pull requests, so it doesn't review its own work.
+- **Teams without a subscription** don't get automatic reviews, because automatic review is a paid feature. If your team has used its allowance, the pull request gets one short notice explaining why there's no review.
+
+## Issues
+
+When an issue is opened, the agent reads its title and description:
+
+- **If the issue describes a test task**, the agent does it. It asks you to sign in only if a step needs it.
+- **If the issue is empty** and you look like a new user, the agent introduces itself. It walks you through signing in to TestDriver and creating your first test.
+- **If the agent can't help with the issue**, such as a product bug report or a feature request, it stays silent instead of posting an off-topic reply.
+
+Whenever the agent creates or changes a test file, it finishes by opening a pull request with the test.
+
+## @mentions
+
+Mention the agent on an issue or pull request to start an interactive conversation. Because you asked directly, the agent:
+
+- Posts progress updates and sandbox screenshots as it works.
+- Answers follow-up questions in the thread.
+- Opens a pull request with any test it writes.
+
+On a pull request, an @mention gets you this play-by-play instead of the quiet automatic review.
+
+### Sign out
+
+To revoke your installation's TestDriver sign-in, mention the agent with `logout`:
+
+```text
+@testdriverai logout
+```
+
+`sign out` and `disconnect` work too.
+
+The agent revokes the installation's shared sign-in and shuts down its live sandbox. The next person to mention it is asked to sign in again.
 
 ## Signing in
 
-Reading a diff needs nothing, but **running** the app needs a TestDriver session,
-which requires the installation to be signed in. Sign-in state doesn't decide
-*whether* the agent reviews — it decides *how good* the review gets. A signed-out
-install still gets its bugs caught from reading the code; signing in upgrades
-findings from `suspected` to `observed`.
+Reading a diff doesn't need a TestDriver account, but running your app does.
+
+When your installation isn't signed in, the agent still reviews pull requests by reading the code. It marks every finding `suspected`.
+
+Signing in lets it run each change, so findings it proves become `observed`.
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Add TestDriver to GitHub" icon="github" href="/quickstart-github">
+    Install the GitHub app and get your first review.
+  </Card>
+  <Card title="Run tests in CI/CD" icon="code-branch" href="/ci-cd">
+    Run your TestDriver suite in GitHub Actions and other CI systems.
+  </Card>
+</CardGroup>