@cursor/july 0.1.111 → 0.1.113

package/dist/docs/guides/jev.md

@@ -0,0 +1,348 @@
+# Use Jev in review tools
+
+Use Jev for narrow decisions where your software factory needs a typed
+answer, not another paragraph. A review agent can filter speculative
+findings, classify pull request risk, choose a reviewer, or decide
+whether a merge needs a documentation follow-up. Jev returns a choice,
+score, or probability; your TypeScript decides what happens next.
+
+Ask one question per judgment. If an approval depends on risk, test
+coverage, and the size of the change, ask three questions in one call
+and combine the answers in code.
+
+Set `TYPESAFE_API_KEY`. The default model is `jev-latest`.
+
+```ts
+// agent/extensions/jev.ts
+import jev from "@cursor/july/extensions/jev";
+
+export default jev();
+```
+
+## Start only the review turns you need
+
+You can call Jev from a channel hook before a model turn starts. When a
+pull request opens, this hook reads its title, labels, and filenames,
+then asks whether the change needs security review. A high probability
+starts the review. Otherwise the hook returns `null`, so the agent does
+not run and nothing is posted to GitHub.
+
+```ts
+// agent/channels/github.ts
+import {
+  defaultGitHubAuth,
+  githubChannel,
+} from "@cursor/july/channels/github";
+import { above, decide } from "@cursor/july/extensions/jev";
+
+export default githubChannel({
+  botName: "security-reviewer",
+  cursorAccount: { repos: ["acme/checkout"] },
+  onPullRequest: async (ctx, pr) => {
+    if (pr.action !== "opened" && pr.action !== "ready_for_review") {
+      return null;
+    }
+
+    const octokit = await ctx.github.getOctokit();
+    const [{ data }, files] = await Promise.all([
+      octokit.rest.pulls.get({
+        owner: ctx.repository.owner,
+        repo: ctx.repository.name,
+        pull_number: pr.number,
+      }),
+      octokit.paginate(octokit.rest.pulls.listFiles, {
+        owner: ctx.repository.owner,
+        repo: ctx.repository.name,
+        pull_number: pr.number,
+      }),
+    ]);
+    const answers = await decide({
+      state: {
+        title: data.title,
+        body: data.body,
+        labels: data.labels.map(label => label.name),
+        files: files.map(file => file.filename),
+      },
+      questions: {
+        review: {
+          type: "boolean",
+          instructions:
+            "Does this change need security review? Answer yes for auth, permissions, secrets, request parsing, or external inputs.",
+        },
+      },
+    });
+
+    if (!above(answers.review, 0.8)) {
+      return null;
+    }
+    // `auth` starts a model turn running as the pull request sender.
+    return { auth: defaultGitHubAuth(ctx) };
+  },
+});
+```
+
+Only pull request metadata goes to Jev here. The review turn still reads
+the diff itself.
+
+## Filter findings before you post them
+
+Let the chat model draft a finding, then ask Jev whether the finding is
+a real bug in the new code. Below your threshold, the tool returns and
+the author never sees the draft. Above it, the finding becomes a review
+comment.
+
+```ts
+// agent/tools/post_finding.ts
+import { parseGitHubPrContinuationKey } from "@cursor/july/channels/github";
+import { above, decide } from "@cursor/july/extensions/jev";
+import { defineTool } from "@cursor/july/tools";
+import { z } from "zod";
+
+export default defineTool({
+  description:
+    "Post one security finding on this session's pull request. Call once. Hold when it is not a real bug.",
+  inputSchema: z.object({
+    title: z.string(),
+    summary: z.string().describe("What the pull request changes."),
+    draft: z.string().describe("The finding to post, one or two sentences."),
+  }),
+  async execute({ title, summary, draft }, ctx) {
+    if (ctx.session.purpose === "eval") {
+      return { posted: false, reason: "eval" };
+    }
+
+    const ref = parseGitHubPrContinuationKey(ctx.session.continuationKey ?? "");
+    if (ref === undefined) {
+      throw new Error("post_finding requires a GitHub pull request session");
+    }
+
+    const answers = await decide({
+      state: { title, summary, draft },
+      questions: {
+        real: {
+          type: "boolean",
+          instructions:
+            "Is the draft an exploitable bug in the new code, not a style note or a hypothetical?",
+        },
+      },
+    });
+
+    if (!above(answers.real, 0.85)) {
+      return { posted: false, reason: "clean" };
+    }
+
+    const octokit = await ctx.host.github.getOctokit();
+    await octokit.rest.pulls.createReview({
+      owner: ref.owner,
+      repo: ref.repo,
+      pull_number: ref.number,
+      event: "COMMENT",
+      body: draft,
+    });
+    return {
+      posted: true,
+      pr: `${ref.owner}/${ref.repo}#${ref.number}`,
+    };
+  },
+});
+```
+
+## Approve changes by risk tier
+
+You can use the same pattern for Agentic Owners. Ask Jev to put the
+pull request in a closed set of risk tiers. Approve only a confident
+`very-low` or `low`; send everything else to a person.
+
+```ts
+import { decide, needsHuman } from "@cursor/july/extensions/jev";
+
+const answers = await decide({
+  state: { title, summary },
+  questions: {
+    risk: {
+      type: "choice",
+      instructions: "What risk tier is this pull request?",
+      criteria: {
+        "very-low": "docs, formatting, or a mechanical rename",
+        low: "a local change with tests and no new trust boundary",
+        medium: "auth, billing, or a behavior change callers depend on",
+        high: "a likely exploit, data loss, or a broken public contract",
+      },
+    },
+  },
+});
+
+const tier = answers.risk.choice;
+if (needsHuman(answers.risk) || tier === "medium" || tier === "high") {
+  return { verdict: "hold", tier };
+}
+return { verdict: "approve", tier };
+```
+
+Your GitHub tool resolves the pull request from `ctx.session` and posts
+that verdict. The model does not choose the repository, pull request, or
+approval event.
+
+## Open documentation follow-ups selectively
+
+After a pull request merges, a code-wiki agent can ask whether the
+change introduced a durable fact that belongs in the docs. A low
+probability means no follow-up. A high probability opens a documentation
+pull request instead of turning every merge into churn.
+
+```ts
+import { above, decide } from "@cursor/july/extensions/jev";
+import { openDocsPullRequest } from "../lib/wiki";
+
+const answers = await decide({
+  state: { title, summary },
+  questions: {
+    updateDocs: {
+      type: "boolean",
+      instructions:
+        "Does this merge change a durable contract that the project docs should explain?",
+    },
+  },
+});
+
+if (!above(answers.updateDocs, 0.8)) {
+  return { action: "skip" };
+}
+return openDocsPullRequest({ title, summary });
+```
+
+## Write tools with Jev
+
+`decide` is a regular async host function. Call it from a channel hook,
+server tool, or router. It returns the answer map directly. `evaluate`
+makes the same request and returns `{ answers }`.
+
+Use a boolean for a yes-or-no gate, a choice for a closed set such as
+risk tiers or owners, and a score for an ordered rubric. `above` checks
+a boolean probability or score. `needsHuman` checks whether a boolean
+or the selected choice clears your confidence bar.
+
+Keep the questions atomic and combine them in TypeScript. For example,
+ask separately whether a finding is real, whether its impact is
+user-visible, and whether the changed line is new. Your code owns the
+rule that decides whether all three are enough to post.
+
+## Let the agent ask Jev
+
+Mounting the extension adds a read-only harness tool named
+`<namespace>__evaluate`. With the default `jev` filename, the model sees
+`jev__evaluate`.
+
+The tool accepts one state and a list of boolean, choice, or score
+questions. It returns `{ answers }` and never posts, approves, or opens
+a pull request. Use it when the agent needs the result during the turn.
+Use `decide` inside a project tool when the answer and the write belong
+in one operation.
+
+## Skills included with the extension
+
+The extension adds two skills by default:
+
+- `jev__questions` teaches the model how to structure atomic questions,
+  choose a question type, and read the answers.
+
+- `jev__gated-write` teaches the model to put `decide` and the write in
+  one server tool, hold on low confidence, and skip writes during evals.
+
+The model sees each skill's description and loads the full procedure
+when it applies.
+
+## Choose what to mount
+
+Both contribution groups are on by default. Turn off the harness tool
+when Jev should only run inside tools you wrote. Turn off the skills
+when your agent already has its own Jev instructions.
+
+```ts
+// agent/extensions/jev.ts
+import jev from "@cursor/july/extensions/jev";
+
+export default jev({
+  harnessTools: false,
+  skills: true,
+});
+```
+
+`harnessTools: false` removes `jev__evaluate` from discovery.
+`skills: false` removes both Jev skills. These switches do not remove
+the exported helpers, so project tools can still import `decide`,
+`above`, and `needsHuman`.
+
+## Jev and trajectory evals
+
+An eval checks a whole turn: the agent called the read, and it stayed
+off approve. `decide` is the one call inside the tool, before GitHub.
+Use the [Evals guide](/docs/evals.md) when you want the turn to keep
+behaving. Use `decide` when the tool itself needs a tier or a yes.
+
+## Test without the live API
+
+Pass `fetch` and `apiKey` and the same `decide` path runs in a test.
+
+```ts
+import { decide } from "@cursor/july/extensions/jev";
+
+const answers = await decide({
+  state: {
+    title: "Skip the refund auth check",
+    summary: "acme/checkout#42 drops the session check on POST /refunds.",
+  },
+  questions: {
+    risk: {
+      type: "choice",
+      instructions: "What risk tier is this pull request?",
+      criteria: {
+        "very-low": "docs, formatting, or a mechanical rename",
+        low: "a local change with tests and no new trust boundary",
+        medium: "auth, billing, or a behavior change callers depend on",
+        high: "a likely exploit, data loss, or a broken public contract",
+      },
+    },
+  },
+  apiKey: "sk-test",
+  fetch: async () =>
+    new Response(
+      JSON.stringify({
+        answers: {
+          risk: {
+            type: "choice",
+            choice: "high",
+            probabilities: {
+              "very-low": 0.02,
+              low: 0.05,
+              medium: 0.18,
+              high: 0.75,
+            },
+          },
+        },
+      }),
+      { status: 200 }
+    ),
+});
+```
+
+## Practices
+
+- Pass the pull request title, a short summary, and the draft finding.
+  Don't send the checkout.
+
+- Calibrate `above` and `needsHuman` on pull requests you have already
+  labeled. `needsHuman` defaults to `0.7`.
+
+## Related
+
+- [Agentic owners](/docs/templates/agentic-owners.md): a risk tier, then
+  the host approves or asks for reviewers
+- [Security reviewer](/docs/templates/security-reviewer.md): a finding, or
+  no comment
+- [Thermo review](/docs/templates/thermo-review.md): bugs and breakage,
+  posted the same way
+- [Code wiki](/docs/reference/cli.md#init): documentation
+  follow-ups after a merge
+- [Evals](/docs/evals.md): checks on the full turn
+- [GitHub agents](/docs/guides/github.md): how the review gets onto the pull request