@openhands/extensions 0.16.0 → 0.18.0

package/skills/github-pr-reviewer/README.md

@@ -12,25 +12,31 @@ This skill is activated by:
 ## Features
 
 - Reviews PRs on demand by watching for a GitHub label event
+- Watches several repositories from a single automation, each with its own state
 - Processes each label application exactly once, with persistent state
 - Re-review support by removing and re-applying the label
 - Suppresses stale reviews when the PR head commit changes mid-review
-- Uses a real cloned checkout and full PR context instead of only a truncated diff
-- Posts acknowledgement and final review comments with AI disclosure
+- Hands the agent the reviewed commit already checked out, and removes that
+  checkout when the review ends, so nothing accumulates between runs
+- Publishes a real pull request review, with inline comments where a finding
+  maps to a changed line, and verifies on GitHub that it landed
+- Posts acknowledgement comments with AI disclosure
 - Configurable review tone and polling schedule
 
 ## Prerequisites
 
 Set `GITHUB_PERSONAL_ACCESS_TOKEN` in OpenHands Settings -> Secrets. The token
-must be able to read the repository, read pull requests, read issue events, and
-write issue comments.
+must be able to read the repositories and their contents, read issue events,
+write issue comments, and **write pull request reviews** — the review is
+published through the pull request reviews API, so read-only pull request access
+is not enough.
 
 ## Quick Start
 
 Ask OpenHands:
 
-> "Set up a PR review automation for my `myorg/backend` repo using the
-> `openhands-review` label and concise reviews."
+> "Set up a PR review automation for my `myorg/backend` and `myorg/frontend`
+> repos using the `openhands-review` label and concise reviews."
 
 After setup, apply the configured label to a pull request to queue a review. To
 request another review later, remove and re-apply the label.

package/skills/github-pr-reviewer/SKILL.md

@@ -2,23 +2,30 @@
 name: github-pr-reviewer
 description: >
   Create an automation that reviews GitHub pull requests when a configurable
-  trigger label is applied. Polls GitHub deterministically, starts one
-  OpenHands review conversation per label event, inspects full repository and
-  PR context, and posts the final review comment back to GitHub.
+  trigger label is applied. Polls one or more repositories deterministically,
+  starts one OpenHands review conversation per label event with the pull
+  request's head commit already checked out, and publishes the review to GitHub.
 triggers:
   - /pr-reviewer:setup
 ---
 
 # GitHub PR Reviewer Automation
 
-Create a cron automation that watches a GitHub repository for pull requests
-with a review trigger label, starts an OpenHands review conversation once per
-label event, and posts the AI review as a GitHub comment.
+Create a cron automation that watches one or more GitHub repositories for pull
+requests with a review trigger label, starts an OpenHands review conversation
+once per label event, and publishes the AI review to GitHub.
 Windows PowerShell equivalents for the setup, packaging, upload, and API-check shell snippets are in `references/windows.md`.
 
 The automation script is deterministic: PR discovery, label-event tracking,
-state persistence, stale-result suppression, and GitHub comment posting are
-handled in Python. The LLM is invoked only for the review itself.
+state persistence, stale-result suppression, the repository checkout, and its
+removal are all handled in Python. The LLM is invoked only for the review
+itself.
+
+The script prepares each review's workspace before the agent starts: the pull
+request's head commit is downloaded as a tarball and extracted to a directory of
+its own, which becomes the conversation's working directory. The agent is told
+not to clone, fetch, check out, or delete anything, and the script removes the
+checkout once the conversation has stopped. Nothing accumulates between runs.
 
 ---
 
@@ -31,7 +38,13 @@ Verify that the following secret is set in **OpenHands Settings -> Secrets**:
 | Secret name | Token type | Minimum permissions |
 |---|---|---|
 | `GITHUB_PERSONAL_ACCESS_TOKEN` | Classic PAT | `repo` for private repos or `public_repo` for public repos |
-| `GITHUB_PERSONAL_ACCESS_TOKEN` | Fine-grained PAT | Contents: Read, Metadata: Read, Pull requests: Read, Issues: Read and Write |
+| `GITHUB_PERSONAL_ACCESS_TOKEN` | Fine-grained PAT | Contents: Read, Metadata: Read, Pull requests: **Read and Write**, Issues: Read and Write |
+
+Pull-request **write** access is required because the agent publishes a pull
+request review, not just an issue comment. A token with only Pull requests: Read
+will poll happily and then fail at the point of publishing.
+
+When several repositories are monitored, the token must cover all of them.
 
 Check with:
 ```bash
@@ -57,12 +70,13 @@ Run the `curl` check above.
 - If the API returns `{"message": "Bad credentials"}`: tell the user the
   token is invalid and ask them to update it. Stop.
 
-### Step 2 - Collect repository
+### Step 2 - Collect repositories
 
-Ask: *"Which GitHub repository should be monitored?
-(Format: `owner/repo`, e.g. `myorg/backend`)"*
+Ask: *"Which GitHub repositories should be monitored?
+(Format: `owner/repo`, e.g. `myorg/backend`. List several separated by commas to
+review them all from one automation.)"*
 
-Validate access:
+Validate access to **each** repository:
 ```bash
 curl -s "https://api.github.com/repos/{owner}/{repo}" \
   -H "Authorization: Bearer $GITHUB_PERSONAL_ACCESS_TOKEN" \
@@ -76,7 +90,13 @@ else:
 "
 ```
 
-Record `REPO = "{owner}/{repo}"`.
+Record every accepted repository into `REPOS = ["{owner}/{repo}", ...]`. If one
+repository fails the check, say which and ask whether to continue without it.
+
+Each repository is polled independently and keeps its own state, so pull-request
+numbers never collide between them. The trigger label, tone, and schedule are
+shared by all of them; a repository needing different settings wants its own
+automation.
 
 ### Step 3 - Collect trigger label
 
@@ -122,9 +142,15 @@ Record as `CRON_SCHEDULE`.
 Read `scripts/main.py` from this skill's directory. Apply exactly five constant
 substitutions near the top of the file:
 
+> The script also reads a `config.json` shipped beside it, if there is one, over
+> these constants. That is how the catalog entry
+> (`automations/catalog/github-pr-reviewer/`) configures an unmodified copy,
+> since a declarative host cannot rewrite Python. This setup path substitutes the
+> constants and ships no `config.json`, so the two never collide.
+
 | Placeholder | Replace with |
 |---|---|
-| `REPO = "owner/repo"` | `REPO = "{owner_repo}"` |
+| `REPOS = ["owner/repo"]` | `REPOS = ["{owner_repo}", ...]` - one entry per repository collected in Step 2 |
 | `TRIGGER_LABEL = "openhands-review"` | `TRIGGER_LABEL = "{trigger_label}"` |
 | `REVIEW_TONE = "thorough"` | `REVIEW_TONE = "{review_tone}"` |
 | `REVIEW_STYLE_INSTRUCTIONS = ""` | `REVIEW_STYLE_INSTRUCTIONS = "{style_instructions}"` |
@@ -132,6 +158,7 @@ substitutions near the top of the file:
 
 Use a safe string writer such as `json.dumps(value)` when inserting user-provided
 repository names, labels, or style instructions into Python string literals.
+`json.dumps(list_of_repos)` produces the whole `REPOS` list safely in one step.
 
 Write the customized script to a temporary build directory:
 ```bash
@@ -173,14 +200,19 @@ curl -s -X POST "${OPENHANDS_HOST}/api/automation/v1" \
   -H "X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY" \
   -H "Content-Type: application/json" \
   -d "{
-    \"name\": \"GitHub PR Reviewer: {owner}/{repo} label {trigger_label}\",
+    \"name\": \"GitHub PR Reviewer: {repo_summary} label {trigger_label}\",
     \"trigger\": {\"type\": \"cron\", \"schedule\": \"{cron_schedule}\"},
     \"tarball_path\": \"$TARBALL_PATH\",
     \"entrypoint\": \"python3 main.py\",
-    \"timeout\": 300
+    \"timeout\": 600
   }" | python3 -m json.tool
 ```
 
+Use the single repository as `{repo_summary}` when there is one, and something
+like `3 repos` when there are several. A poll now downloads a tarball per queued
+review, so the timeout allows for that; a run never waits for a review to
+finish, only for it to be started.
+
 Record the returned `id`.
 
 ### Step 9 - Confirm
@@ -190,44 +222,67 @@ Tell the user:
 > ✅ **GitHub PR Reviewer** is running!
 >
 > - Automation ID: `{id}`
-> - Repository: `{owner}/{repo}`
+> - Repositories: `{owner}/{repo}`, ... (one line each)
 > - Trigger label: `{trigger_label}`
 > - Review tone: `{tone}`
 > - Polling schedule: `{cron_schedule}`
-> - State file: `~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_{id}.json`
+> - State file per repository:
+>   `~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_{id}_{owner}__{repo}.json`
 >
 > Apply the `{trigger_label}` label to a pull request to queue a review. Each
 > label event is processed once. To request another review, remove and re-apply
 > the label.
+>
+> The review is published as a pull request review on the head commit, with
+> inline comments where a finding maps to a changed line.
 
 ---
 
 ## Runtime Behaviour (per poll)
 
-Each cron run executes `main.py`, which:
+Each cron run executes `main.py`, which resolves and validates
+`GITHUB_PERSONAL_ACCESS_TOKEN` once, then processes every repository in `REPOS`
+independently. One repository failing does not stop the others; the run fails
+only if every repository fails.
+
+For each repository:
 
-1. Loads state from the JSON file (see `references/state-schema.md`).
-2. Resolves and validates `GITHUB_PERSONAL_ACCESS_TOKEN` and repository access.
+1. Loads that repository's state (see `references/state-schema.md`).
+2. Verifies repository access.
 3. Lists open PRs, newest-updated first.
 4. For each open PR carrying `TRIGGER_LABEL`:
    - Refetches current PR metadata to avoid acting on stale list data.
    - Finds the latest matching GitHub `labeled` issue event.
    - Skips the event if it has already been tracked.
-   - Starts an OpenHands conversation with a review prompt that includes PR
-     metadata, the exact head SHA, label event details, and instructions to
-     clone the repo, inspect PR discussion, review comments, changed files,
-     diff, and surrounding code.
+   - Downloads the PR's head commit as a tarball and extracts it to
+     `{WORKSPACE_BASE}/repositories/{owner}__{repo}/pr-{number}-{sha12}`. The
+     archive is checked as it is unpacked: a single root, no absolute or `..`
+     paths, and symlinks skipped rather than materialised.
+   - Starts an OpenHands conversation **whose working directory is that
+     checkout**, with a review prompt carrying PR metadata, the exact head SHA,
+     and label event details.
    - Posts an acknowledgement comment with the label event, head SHA, and
      conversation link.
-   - Records the label-event review in state with `status: "active"`.
+   - Records the review in state with `status: "active"` and the checkout path.
+   - If the checkout or the conversation cannot be created, the checkout is
+     removed and nothing is recorded, so the next poll retries the label event.
 5. For each active review conversation:
    - Marks it closed without posting if the PR has closed or merged.
    - Suppresses stale results if the PR head SHA changed after the review was
      queued.
    - When the conversation reaches `idle`, `finished`, `error`, or `stuck`,
-     posts the agent's final response as a GitHub comment and marks the review
-     closed.
-6. Saves state atomically and fires the completion callback.
+     asks GitHub whether a review by the token's own user exists for that head
+     SHA. If it does, the review is complete. If it does not, the agent's final
+     response is posted as a comment so the work is not lost.
+   - Abandons a conversation that has not reached a terminal status within two
+     hours, so its checkout can be reclaimed.
+6. Removes the checkout of every finished review, but only after confirming the
+   conversation has stopped - deleting it under a running agent would remove its
+   working directory. When that cannot be confirmed the directory is left alone
+   and the next poll tries again.
+7. Saves that repository's state atomically.
+
+The completion callback fires once for the whole run.
 
 ---
 
@@ -237,6 +292,9 @@ Each cron run executes `main.py`, which:
   review lifecycle diagram.
 - **`scripts/main.py`** - The complete automation script. Customize the five
   constants at the top before packaging.
+- **`tests/test_main.py`** - Unit tests for the checkout, its removal, and state
+  handling. Run them from the skill root with `python -m pytest tests/` after
+  editing the script.
 
 ---
 
@@ -246,7 +304,11 @@ Each cron run executes `main.py`, which:
 |---|---|---|
 | Bot never queues reviews | Trigger label not present or no matching `labeled` event | Apply the configured label to the PR |
 | "Bad credentials" in run logs | Token expired | Rotate and update `GITHUB_PERSONAL_ACCESS_TOKEN` |
-| 404 on repo access | Repo name wrong or no access | Re-check `owner/repo` and token permissions |
+| 404 on repo access | Repo name wrong or no access | Re-check the entry in `REPOS` and the token's permissions |
+| One repository is skipped, others work | That repository failed its access check | Read the `=== owner/repo ===` block in the run log |
 | Same PR not reviewed after new commits | Label event was already processed | Remove and re-apply the trigger label |
 | Review result never posts | Conversation still running or stuck | Open the conversation link from the acknowledgement comment |
 | Stale review suppressed | PR head SHA changed while the agent was reviewing | Re-apply the trigger label after the latest commit |
+| Review arrives as a plain comment, not a review | Publishing failed, so the script posted the text as a fallback | Check that the token has Pull requests: Read and Write |
+| Agent reports it cannot clone the repo | Prompt asked it not to; the workspace is already the checkout | No action - the code is at the head SHA in its working directory |
+| Checkouts remain under `repositories/` | Their conversations had not stopped yet | They are removed by a later poll once the conversation is terminal |

package/skills/github-pr-reviewer/commands/pr-reviewer-setup.md

@@ -1,6 +1,6 @@
 ---
 # auto-generated by sync_extensions.py
-description: Create an automation that reviews GitHub pull requests when a configurable trigger label is applied. Polls GitHub deterministically, starts one OpenHands review conversation per label event, inspects full repository and PR context, and posts the final review comment back to GitHub.
+description: Create an automation that reviews GitHub pull requests when a configurable trigger label is applied. Polls one or more repositories deterministically, starts one OpenHands review conversation per label event with the pull request's head commit already checked out, and publishes the review to GitHub.
 ---
 
 Read and follow the complete instructions in the SKILL.md file located in this skill's directory.

package/skills/github-pr-reviewer/references/state-schema.md

@@ -1,22 +1,28 @@
 # State Schema
 
-The automation maintains a JSON state document that persists across polling runs.
-It is the source of truth for which trigger-label events have queued reviews
-and which conversations are still active.
+The automation maintains a JSON state document **per repository**, persisted
+across polling runs. It is the source of truth for which trigger-label events
+have queued reviews, which conversations are still active, and which repository
+checkouts are still on disk.
+
+Each repository in `REPOS` gets its own document, so pull-request numbers from
+different repositories never share a bucket.
 
 ---
 
 ## Storage
 
 **Primary (cloud):** The state is stored in the automation service's built-in KV
-store under the key `"state"`. The KV store is available when `AUTOMATION_KV_TOKEN`
-is injected into the run environment. Each automation has its own isolated namespace.
+store under the key `state:{owner}__{repo}` — for example
+`state:OpenHands__extensions`. The KV store is available when
+`AUTOMATION_KV_TOKEN` is injected into the run environment. Each automation has
+its own isolated namespace.
 
-**Fallback (local/dev):** When the KV store is not available, the state is written
-to a local JSON file at:
+**Fallback (local/dev):** When the KV store is not available, the state is
+written to a local JSON file at:
 
 ```
-{WORKSPACE_BASE_ROOT}/automation-state/github_pr_reviewer_label_event_{automation_id}.json
+{WORKSPACE_BASE_ROOT}/automation-state/github_pr_reviewer_label_event_{automation_id}_{owner}__{repo}.json
 ```
 
 `WORKSPACE_BASE_ROOT` is derived by going two levels up from the `WORKSPACE_BASE`
@@ -25,19 +31,28 @@ environment variable, stripping `automation-runs/{run_id}`.
 Example on a local install:
 
 ```
-~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_abc12345-....json
+~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_abc12345-..._myorg__backend.json
 ```
 
 The `automation_id` is read from the `AUTOMATION_EVENT_PAYLOAD` environment
 variable, field `automation_id`.
 
+### Upgrading from a single-repository automation
+
+Earlier versions stored one document under the bare key `state` (or a filename
+without the repository suffix). On the first poll after an upgrade, that document
+is adopted for the repository named in its own `repo` field, and written back
+under the new per-repository key. Reviews already handled are therefore not
+re-run. A repository that does not match the legacy document simply starts with
+empty state.
+
 ---
 
 ## Top-Level Schema
 
 ```jsonc
 {
-  "version": 2,
+  "version": 3,
   "repo": "owner/repo",
   "trigger_label": "openhands-review",
   "updated_at": 1717200000.0,
@@ -65,6 +80,7 @@ Value: **ReviewRecord**
   "html_url": "https://github.com/owner/repo/pull/42",
   "status": "active",
   "conversation_id": "550e8400-e29b-41d4-a716-446655440000",
+  "workspace_dir": "/workspace/repositories/owner__repo/pr-42-0123456789ab",
   "last_activity": 1717200000.0
 }
 ```
@@ -73,12 +89,28 @@ Value: **ReviewRecord**
 
 | Status | Meaning |
 |---|---|
+| `starting` | The label event is claimed and the checkout/conversation is being set up. Written and saved before that work begins, so a poll overlapping the claiming one skips this event instead of reviewing the same commit twice. Becomes `active` once the conversation exists, or the record is deleted if setup fails, so the next poll retries. A `starting` record older than `STALLED_CLAIM_SECONDS` (15 min) belongs to a poll that died mid-setup and is released. |
 | `active` | Review conversation is running or waiting to be collected |
-| `closed` | Final result was posted, or the PR closed before collection |
+| `closed` | Final result was handled, or the PR closed before collection |
 | `stale` | PR head SHA changed before the review completed, so the result was suppressed |
+| `expired` | Conversation never reached a terminal status within `MAX_ACTIVE_AGE` (2 h) and was abandoned |
 
 When a review becomes stale, `stale_reason` records the old and new head SHAs.
 When a review closes after posting, `completed_at` records the completion time.
+When a review expires, `expired_after` records how many seconds it had been
+waiting.
+
+### `workspace_dir`
+
+The directory holding the reviewed commit, created by the script before the
+conversation starts and used as that conversation's working directory. It is
+removed once the conversation is confirmed stopped, and the key is deleted from
+the record at the same time.
+
+The key therefore doubles as the "still on disk" marker: a record carrying a
+`workspace_dir` after it has left `active` is retried on every later poll until
+the removal succeeds. A checkout is never removed while its conversation is
+still running, and never outside `{WORKSPACE_BASE}/repositories/`.
 
 ---
 
@@ -108,14 +140,29 @@ because the trigger label was absent.
 Trigger label applied on GitHub
         |
         v
-[active]  - conversation created, acknowledgement comment posted
+[starting] - label event claimed and saved, before any slow work
+        |
+        +-- setup fails ----------------------------> record deleted, retried next poll
+        |
+        +-- claiming poll dies mid-setup -----------> released after 15 min
+        |
+        v
+  checkout prepared at head SHA, conversation created
+        |
+        v
+[active]  - acknowledgement comment posted
         |
-        +-- PR closes/merges before collection --> [closed] without posting
+        +-- PR closes/merges before collection ------> [closed]  without posting
         |
-        +-- PR head SHA changes before collection --> [stale] without posting
+        +-- PR head SHA changes before collection ---> [stale]   without posting
+        |
+        +-- no terminal status within 2 h -----------> [expired] without posting
+        |
+        v
+[closed]  - review confirmed on GitHub, or the agent's text posted as a comment
         |
         v
-[closed]  - final review comment posted
+  checkout removed once the conversation has stopped
 ```
 
 ---
@@ -123,18 +170,21 @@ Trigger label applied on GitHub
 ## Resetting State
 
 To force the automation to reconsider previous label events, delete the state
-from the KV store (cloud) or the fallback file (local).
+for that repository from the KV store (cloud) or the fallback file (local).
 
 **Cloud (KV store):**
 ```bash
-curl -X DELETE "${OPENHANDS_HOST}/api/automation/v1/kv/state" \
+curl -X DELETE "${OPENHANDS_HOST}/api/automation/v1/kv/state:owner__repo" \
   -H "Authorization: Bearer ${AUTOMATION_KV_TOKEN}"
 ```
 
 **Local (file fallback):**
 ```bash
-rm ~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_<id>.json
+rm ~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_<id>_owner__repo.json
 ```
 
+Resetting state also forgets which checkouts are outstanding, so remove any
+leftover directories under `{WORKSPACE_BASE}/repositories/owner__repo/` yourself.
+
 Usually, prefer removing and re-applying the trigger label. That preserves
 history while creating a new review request.