@introspection-ai/operator 0.0.0-stage → 0.1.1

package/skills/build-review/SKILL.md

@@ -0,0 +1,51 @@
+---
+name: build-review
+description: Review a drafted Introspection agent (Recipe) with fresh-context reviewers, fix what they find, and package it as a pull request ready for the owner's review. Use after build-draft, when the cases have run locally. The last step of build; deploy follows.
+---
+
+# Build: review
+
+Check the draft as someone who wasn't in the conversation would, fix what they find, and hand the owner a pull request they can review without having followed the work.
+
+## Before you start
+
+- Check the stage first: no approved cases in `evals/cases/` means build-brainstorm; approved cases but no case results recorded means build-draft.
+
+- You need the `introspection` CLI (`npm i -g @introspection-ai/cli`). No login unless the repository is the Introspection hosted one.
+- The `pr/<name>` branch has the cases (`evals/cases/`), the design (`docs/DESIGN.md`, Parts A and B) and the Recipe, and build-draft's last commit records the case results. If any of these is missing, go back to the step that makes it.
+
+## Steps
+
+1. **Run the checks again from a clean state:** `introspection check`, then `introspection eval run`. Use these results, not build-draft's, in the review.
+2. **Review with fresh context.** Give each reviewer only the branch, never the conversation, and one question each. Use separate subagents where they're available; otherwise run each question as its own confined, read-only Pi call, started in the agent's repository:
+
+   ```bash
+   cd <repository> && pi --print --no-session --no-context-files --no-skills \
+     --no-extensions --no-mcp --tools read,grep,find,ls \
+     "<the question>. Review only the files in this folder."
+   ```
+
+   A reviewer needs no shell, no edits and nothing outside the repository; an unconfined call can wander into the owner's other folders. "Reviewers unavailable" is never a reason to skip review.
+   [The review questions](references/review-questions.md) cover the design against the cases, the Recipe against the design, the checks against the cases, and what a stranger would misread.
+3. **Fix what blocks,** and run the checks again. Record what wasn't fixed and why. A finding that needs the owner's decision (a case that seems wrong, a default they should see) goes in the pull request's summary as a question. Don't change an approved case yourself.
+4. **Check nothing private is committed:** no client documents (case inputs are referenced by hash), no credentials, no tokens. Remove anything found from the branch history, not just the last commit.
+5. **Package the pull request** with [the description template](references/pr-description.md), then push `pr/<name>`:
+   - **GitHub, whenever the repository is there:** open or update a pull request for the branch (`gh pr create`, or `gh api` with the access token when running as the Operator).
+   - **The Introspection hosted repository otherwise:** push the branch; its description is the review package, which you show the owner in chat.
+6. **Hand over to deploy.** The pull request is ready for review: deploy creates its runtime version, connects the agent's data and tests a real conversation before it's ready to land.
+
+## Guardrails
+
+- **Bounded:** at most the four review questions, each pass with a time limit (about five minutes), and at most two rounds of fix-then-re-review. Anything still open after that goes into the pull request description as an open item, not another loop.
+- **A fix re-runs only what it touches, once.** After a fix, re-run `introspection check` and the cases the fix affects, one run each. Review isn't a second draft: don't tune the agent's instructions against one output and run it again, since that fits the agent to a single run. If a case still fails, record it in Part B and the pull request as an open item.
+- **Reviewers only report.** Findings come back as text; only this step edits the repository, so one reviewer can't undo another's work.
+- **No real data, credentials or pushes during review.** Reviewers read the repository; they don't run the agent against real connections or push anything.
+- **Never widen access to finish a review.** If a pass can't reach something (a permission prompt, a missing tool), record that and continue; never ask the owner for broader access or use paths outside the repository.
+- **The repository holds only what the Recipe needs.** Working files a run left behind (scratch scripts, test outputs, copies of inputs) are a blocking finding: remove them, and keep run outputs under a gitignored folder, since anything in the package ships with the agent.
+
+## Done when
+
+- No working files are left in the repository outside what the Recipe needs.
+- `introspection check` passes, the Recipe boots (`introspection local -p "hello"`), and every case passes or has a written reason in Part B.
+- No reviewer finding is left blocking.
+- The pull request exists (or the hosted branch is pushed) with a description a business owner can read.

package/skills/build-review/references/pr-description.md

@@ -0,0 +1,27 @@
+# Pull request description
+
+Written for the owner first: plain language, no tool or file names in the first two sections.
+
+```markdown
+## What this agent does
+<Part A's job and who uses it, in two or three sentences>
+
+## How it did on your cases
+| Case | Situation | Result |
+| --- | --- | --- |
+| C1 | ... | passes |
+| C4 | ... | fails: needs your email connected (deploy will do this) |
+
+## What we assumed
+<each default from "Assumed defaults", and how to change it>
+
+## Questions for you
+<anything the review found that needs your decision; "None" if nothing>
+
+## For reviewers
+- Design: `docs/DESIGN.md` (Part B lists each capability and the cases that need it)
+- Cases: `evals/cases/`; checks: `evals/`
+- Last run: <command and result summary>
+- Review: <the fresh-context findings, and what was fixed or left with a reason>
+- Tested against fakes only so far; deploy connects real data and tests a real conversation.
+```

package/skills/build-review/references/review-questions.md

@@ -0,0 +1,10 @@
+# Review questions
+
+One reviewer per question, each with fresh context: the branch only, never the conversation. A finding names the file and line, says what is wrong, and says whether it blocks.
+
+1. **Design against cases.** Does every behaviour and capability in Part B cite a case that needs it? Is anything built that no case needs? Is every "It will never" and "It leaves to you" item covered by a case?
+2. **Recipe against design.** Does the Recipe do what Part B says, and nothing it doesn't? Are tools and permissions no wider than the cases need? Does anything fetch instructions or packages while the agent runs? Does the Recipe boot, and does every declared tool exist in the installed runtime? Does it produce a usable result on the owner's real sample when run from a folder outside the repository? Does every exact value the owner gave (author, side, model) appear in the Recipe? Is anything specific to a test input in the instructions? Is anything in the repository that the Recipe doesn't need, such as scripts or outputs a run left behind?
+3. **Checks against cases.** Does each case marked `code` have a check that would fail on the case's bad result? Are any checks testing the implementation instead of the owner's outcome? Does any check pass trivially?
+4. **The stranger's read.** Reading only Part A and the pull request description, what would a business owner misunderstand about what the agent does, what it needs from them, or what it won't do?
+
+A finding blocks when the agent would do something the owner didn't approve, a case could pass while the owner's bad result happens, or private data is committed. Everything else is advice.