@computesdk/pi-morph 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +27 -5
- package/index.ts +25 -12
- package/package.json +12 -4
- package/tools/api.ts +93 -0
- package/tools/apply.ts +71 -0
- package/tools/compact.ts +54 -0
- package/tools/index.ts +16 -0
- package/tools/search.ts +487 -0
package/README.md
CHANGED
|
@@ -1,12 +1,16 @@
|
|
|
1
1
|
# @computesdk/pi-morph
|
|
2
2
|
|
|
3
|
-
[Morph](https://www.morphllm.com)
|
|
3
|
+
[Morph](https://www.morphllm.com) for Pi: the open-weight coding models as a provider, and Morph's single-purpose models as tools.
|
|
4
4
|
|
|
5
5
|
```sh
|
|
6
6
|
pi install npm:@computesdk/pi-morph # or: pi -e /path/to/pi-extensions/packages/pi-morph
|
|
7
7
|
```
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Run `/login morph` and paste your key, or set `MORPH_API_KEY` in your environment; one credential covers both halves (the models and the tools), and the environment wins when both are set.
|
|
10
|
+
|
|
11
|
+
## Models
|
|
12
|
+
|
|
13
|
+
Set the key and the models show up in `/model` as `morph/morph-kimik3` and friends.
|
|
10
14
|
|
|
11
15
|
| Model | Input | Context | $/1M in | $/1M out |
|
|
12
16
|
| --- | --- | --- | --- | --- |
|
|
@@ -18,13 +22,31 @@ Set `MORPH_API_KEY` in your environment and the models show up in `/model` as `m
|
|
|
18
22
|
| `morph-dsv41flash-1017` — DeepSeek V4.1 Flash 1017 | text, image | 1M | 0.12 | 0.80 |
|
|
19
23
|
| `morph-dsv4flash` — DeepSeek V4 Flash 0731 | text | 1M | 0.14 | 0.40 |
|
|
20
24
|
|
|
21
|
-
Endpoint `https://api.morphllm.com/v1`, bearer auth, `api: "openai-completions"` — Pi's built-in implementation does the request, stream and tool-call plumbing, so the
|
|
25
|
+
Endpoint `https://api.morphllm.com/v1`, bearer auth, `api: "openai-completions"` — Pi's built-in implementation does the request, stream and tool-call plumbing, so the provider is one `pi.registerProvider` call with the catalog above. Prices are per 1M tokens from [morphllm.com/pricing](https://www.morphllm.com/pricing); `cacheWrite` is not billed. `service_tier: "standby"` halves the price on supported models — Pi can send it through `samplingParams`.
|
|
22
26
|
|
|
23
27
|
Pi's thinking levels map onto Morph's `reasoning_effort` ladder (`none | low | high | xhigh | max`). DeepSeek documents `none`, so `off` maps to it; for the other models `off` is marked unsupported (`null`) rather than sent.
|
|
24
28
|
|
|
29
|
+
## Tools
|
|
30
|
+
|
|
31
|
+
The same package adds three tools. Morph's single-purpose models take request shapes of their own rather than chat completions, so they are tools here, not entries in the catalog above.
|
|
32
|
+
|
|
33
|
+
| Tool | What it does | Endpoint |
|
|
34
|
+
| --- | --- | --- |
|
|
35
|
+
| `morph_apply` | Applies an edit to a file with Fast Apply (~10.5k tok/s, ~98% accuracy) | `POST /v1/code/apply` |
|
|
36
|
+
| `morph_compact` | Compresses long text, keeping the parts a follow-up question needs | `POST /v1/compact` |
|
|
37
|
+
| `morph_search` | Answers a code question with the WarpGrep search subagent (~6s) | `POST /v1/chat/completions` with `morph-warp-grep-v2.1` |
|
|
38
|
+
|
|
39
|
+
`morph_apply` rewrites the file it is given: you pass the lines that change plus `// ... existing code ...` markers for the unchanged spans, and Morph merges the edit in. It is marked `destructiveHint`, so a permission extension can gate it. The other two only read.
|
|
40
|
+
|
|
41
|
+
`morph_search` is a subagent, not a grep wrapper. It explores the repository in its own context window for up to six turns and returns only the file/line spans it settled on, with their code, so the dead ends never reach your context. Its search tools (`grep_search`, `read`, `list_directory`, `glob`, `finish`) are built into Morph's model; this package executes them locally and feeds the results back.
|
|
42
|
+
|
|
43
|
+
That local implementation never runs a shell. Morph asks for directory listings as shell commands (`ls -la src`), which are parsed here rather than executed, and every read is confined to the directory the search was given. `node_modules`, `.git`, and build or virtualenv trees are skipped.
|
|
44
|
+
|
|
45
|
+
Usage from these endpoints is reported to Pi without a cost, because Morph does not publish per-token prices for them.
|
|
46
|
+
|
|
25
47
|
## Not included
|
|
26
48
|
|
|
27
|
-
|
|
49
|
+
The Model Router (`/v1/router/classify`, `/v1/router/multimodel`) labels how complex a prompt is so a caller can choose a model. It is left out because Pi already picks the model for a session, so there is nothing for a tool to decide.
|
|
28
50
|
|
|
29
51
|
## Development
|
|
30
52
|
|
|
@@ -32,6 +54,6 @@ Morph's single-purpose models — Fast Apply (`morph-v3-fast`, `morph-v3-large`,
|
|
|
32
54
|
pnpm test # tsc --noEmit && node test.ts
|
|
33
55
|
```
|
|
34
56
|
|
|
35
|
-
`test.ts` loads
|
|
57
|
+
`test.ts` loads both extensions with a fake Pi. It checks every registered model — id, name, modalities, context and output limits, costs, uniqueness, and that each Pi thinking level is either mapped to a provider value or explicitly unsupported — then exercises the tools' local search implementation against a scratch directory: the repo structure WarpGrep is handed, argument coercion for `grep`/`read`/`list_directory`/`glob`, span parsing, path confinement, and usage mapping. Nothing in the suite touches the network.
|
|
36
58
|
|
|
37
59
|
MIT licensed.
|
package/index.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// Morph — open-weight coding models on an OpenAI-compatible API, plus specialized apply/grep/compact models.
|
|
2
|
-
// Base URL: https://api.morphllm.com/v1 · Auth:
|
|
2
|
+
// Base URL: https://api.morphllm.com/v1 · Auth: `/login morph` (stored in auth.json) or MORPH_API_KEY in the environment.
|
|
3
3
|
// Docs: https://docs.morphllm.com · Models: https://docs.morphllm.com/sdk/components/fast-models
|
|
4
4
|
//
|
|
5
5
|
// This is a Pi provider extension: `pi.registerProvider` adds the endpoint and its model catalog, and Pi's
|
|
@@ -14,11 +14,15 @@
|
|
|
14
14
|
// Prices are USD per 1M tokens from https://www.morphllm.com/pricing (live at /api/models/json). Update them
|
|
15
15
|
// there when they change.
|
|
16
16
|
|
|
17
|
-
import type { ExtensionAPI
|
|
17
|
+
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
18
|
+
import { createProvider, envApiKeyAuth, type Model, type ThinkingLevelMap } from "@earendil-works/pi-ai";
|
|
19
|
+
import { openAICompletionsApi } from "@earendil-works/pi-ai/api/openai-completions.lazy";
|
|
18
20
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
21
|
+
const ID = "morph";
|
|
22
|
+
const BASE_URL = "https://api.morphllm.com/v1";
|
|
23
|
+
|
|
24
|
+
/** Pi's chat-model shape for the one API Morph speaks. */
|
|
25
|
+
type ChatModel = Model<"openai-completions">;
|
|
22
26
|
// Pi thinking levels (off/minimal/low/medium/high/xhigh/max) → the provider's `reasoning_effort` value.
|
|
23
27
|
// Morph documents the ladder for DeepSeek: none | low | high | xhigh | max (`medium`→high, `minimal`→low).
|
|
24
28
|
// For the other models `off` is not documented, so it is marked unsupported (`null`) rather than sent.
|
|
@@ -35,6 +39,9 @@ const model = (id: string, name: string, { input = ["text"], cost, effort = EFFO
|
|
|
35
39
|
type: "chat",
|
|
36
40
|
id,
|
|
37
41
|
name,
|
|
42
|
+
provider: ID,
|
|
43
|
+
api: "openai-completions",
|
|
44
|
+
baseUrl: BASE_URL,
|
|
38
45
|
reasoning: true,
|
|
39
46
|
thinkingLevelMap: effort,
|
|
40
47
|
input,
|
|
@@ -54,11 +61,17 @@ export const MODELS: ChatModel[] = [
|
|
|
54
61
|
];
|
|
55
62
|
|
|
56
63
|
export default function morphProvider(pi: ExtensionAPI) {
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
+
// A complete provider rather than the name+config form, because that is what carries native
|
|
65
|
+
// auth: envApiKeyAuth gives `/login morph` a key prompt, stores the credential in auth.json,
|
|
66
|
+
// and falls back to MORPH_API_KEY in the environment when nothing is stored.
|
|
67
|
+
pi.registerProvider(
|
|
68
|
+
createProvider({
|
|
69
|
+
id: ID,
|
|
70
|
+
name: "Morph",
|
|
71
|
+
baseUrl: BASE_URL,
|
|
72
|
+
auth: { apiKey: envApiKeyAuth("Morph API key", ["MORPH_API_KEY"]) },
|
|
73
|
+
models: MODELS,
|
|
74
|
+
api: openAICompletionsApi(),
|
|
75
|
+
}),
|
|
76
|
+
);
|
|
64
77
|
}
|
package/package.json
CHANGED
|
@@ -1,14 +1,16 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@computesdk/pi-morph",
|
|
3
|
-
"version": "1.0.
|
|
4
|
-
"description": "Morph
|
|
3
|
+
"version": "1.0.3",
|
|
4
|
+
"description": "Morph for Pi: open-weight coding models (Kimi K3, GLM 5.3, DeepSeek V4) on Morph's OpenAI-compatible API, plus Fast Apply, Compact, and WarpGrep as tools.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
7
7
|
"pi",
|
|
8
8
|
"pi-extension",
|
|
9
9
|
"provider",
|
|
10
|
+
"tools",
|
|
10
11
|
"morph",
|
|
11
12
|
"morphllm",
|
|
13
|
+
"warpgrep",
|
|
12
14
|
"openai-compatible"
|
|
13
15
|
],
|
|
14
16
|
"license": "MIT",
|
|
@@ -30,11 +32,13 @@
|
|
|
30
32
|
},
|
|
31
33
|
"pi": {
|
|
32
34
|
"extensions": [
|
|
33
|
-
"./index.ts"
|
|
35
|
+
"./index.ts",
|
|
36
|
+
"./tools/index.ts"
|
|
34
37
|
]
|
|
35
38
|
},
|
|
36
39
|
"files": [
|
|
37
40
|
"index.ts",
|
|
41
|
+
"tools",
|
|
38
42
|
"README.md",
|
|
39
43
|
"LICENSE"
|
|
40
44
|
],
|
|
@@ -43,11 +47,15 @@
|
|
|
43
47
|
"test": "tsc --noEmit && node test.ts"
|
|
44
48
|
},
|
|
45
49
|
"peerDependencies": {
|
|
46
|
-
"@earendil-works/pi-
|
|
50
|
+
"@earendil-works/pi-ai": "*",
|
|
51
|
+
"@earendil-works/pi-coding-agent": "*",
|
|
52
|
+
"typebox": "*"
|
|
47
53
|
},
|
|
48
54
|
"devDependencies": {
|
|
55
|
+
"@earendil-works/pi-ai": "catalog:",
|
|
49
56
|
"@earendil-works/pi-coding-agent": "catalog:",
|
|
50
57
|
"@types/node": "catalog:",
|
|
58
|
+
"typebox": "catalog:",
|
|
51
59
|
"typescript": "catalog:"
|
|
52
60
|
}
|
|
53
61
|
}
|
package/tools/api.ts
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Shared plumbing for Morph's single-purpose endpoints.
|
|
3
|
+
*
|
|
4
|
+
* The chat models in `../index.ts` go through Pi's OpenAI-compatible provider implementation.
|
|
5
|
+
* Fast Apply, Compact, and WarpGrep have bespoke request shapes, so the tools call them directly.
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
import fs from "node:fs";
|
|
9
|
+
import os from "node:os";
|
|
10
|
+
import path from "node:path";
|
|
11
|
+
|
|
12
|
+
export const BASE_URL = "https://api.morphllm.com/v1";
|
|
13
|
+
|
|
14
|
+
/** Morph reports OpenAI-style token counts. */
|
|
15
|
+
export interface MorphUsage {
|
|
16
|
+
prompt_tokens?: number;
|
|
17
|
+
completion_tokens?: number;
|
|
18
|
+
total_tokens?: number;
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
/** Structurally the `Usage` that `AgentToolResult` carries. */
|
|
22
|
+
export interface Usage {
|
|
23
|
+
input: number;
|
|
24
|
+
output: number;
|
|
25
|
+
cacheRead: number;
|
|
26
|
+
cacheWrite: number;
|
|
27
|
+
totalTokens: number;
|
|
28
|
+
cost: { input: number; output: number; cacheRead: number; cacheWrite: number; total: number };
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/**
|
|
32
|
+
* Morph does not publish per-token prices for the specialized endpoints, so their usage is
|
|
33
|
+
* reported with no cost rather than a guessed one.
|
|
34
|
+
*/
|
|
35
|
+
export function usageFrom(usage: MorphUsage | undefined): Usage {
|
|
36
|
+
const input = usage?.prompt_tokens ?? 0;
|
|
37
|
+
const output = usage?.completion_tokens ?? 0;
|
|
38
|
+
return {
|
|
39
|
+
input,
|
|
40
|
+
output,
|
|
41
|
+
cacheRead: 0,
|
|
42
|
+
cacheWrite: 0,
|
|
43
|
+
totalTokens: usage?.total_tokens ?? input + output,
|
|
44
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
|
45
|
+
};
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* One credential covers models and tools: `MORPH_API_KEY` in the environment, or the key
|
|
50
|
+
* `/login morph` stored in the agent directory's auth.json (the provider resolves the same two,
|
|
51
|
+
* stored first; here the env var wins so the shell can override a stored key).
|
|
52
|
+
*/
|
|
53
|
+
export function apiKey(): string {
|
|
54
|
+
const key = process.env.MORPH_API_KEY?.trim() || storedKey();
|
|
55
|
+
if (!key) throw new Error("MORPH_API_KEY is not set and no Morph credential is stored; set it or run /login morph");
|
|
56
|
+
return key;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
/** Read the key `/login morph` wrote. Only literal keys: a "!command" credential is Pi-side resolution. */
|
|
60
|
+
function storedKey(): string | undefined {
|
|
61
|
+
try {
|
|
62
|
+
const dir = process.env.PI_CODING_AGENT_DIR || path.join(os.homedir(), ".pi", "agent");
|
|
63
|
+
const key = JSON.parse(fs.readFileSync(path.join(dir, "auth.json"), "utf8"))?.morph?.key;
|
|
64
|
+
return typeof key === "string" && !key.startsWith("!") ? key : undefined;
|
|
65
|
+
} catch {
|
|
66
|
+
return undefined;
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
/** POST JSON to Morph. A non-2xx response becomes a tool error. */
|
|
71
|
+
export async function post<T>(path: string, body: unknown, signal?: AbortSignal): Promise<T> {
|
|
72
|
+
const res = await fetch(`${BASE_URL}${path}`, {
|
|
73
|
+
method: "POST",
|
|
74
|
+
headers: { authorization: `Bearer ${apiKey()}`, "content-type": "application/json" },
|
|
75
|
+
body: JSON.stringify(body),
|
|
76
|
+
signal,
|
|
77
|
+
});
|
|
78
|
+
const text = await res.text();
|
|
79
|
+
if (!res.ok) {
|
|
80
|
+
throw new Error(`Morph ${path} failed: ${res.status} ${res.statusText}${text ? ` — ${text.slice(0, 400)}` : ""}`);
|
|
81
|
+
}
|
|
82
|
+
try {
|
|
83
|
+
return JSON.parse(text) as T;
|
|
84
|
+
} catch {
|
|
85
|
+
throw new Error(`Morph ${path} returned a non-JSON body: ${text.slice(0, 200)}`);
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
/** Cap a model-facing result and say what was dropped. */
|
|
90
|
+
export function cap(lines: string[], max: number, hint: string): string {
|
|
91
|
+
if (lines.length <= max) return lines.join("\n");
|
|
92
|
+
return `${lines.slice(0, max).join("\n")}\n... ${lines.length - max} more lines dropped; ${hint}`;
|
|
93
|
+
}
|
package/tools/apply.ts
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
import fs from "node:fs/promises";
|
|
2
|
+
import path from "node:path";
|
|
3
|
+
import { defineTool, withFileMutationQueue } from "@earendil-works/pi-coding-agent";
|
|
4
|
+
import { Type } from "typebox";
|
|
5
|
+
import { post, usageFrom, type MorphUsage } from "./api.ts";
|
|
6
|
+
|
|
7
|
+
/** Fast Apply merges an edit snippet into an existing file. `auto` lets Morph pick the model. */
|
|
8
|
+
export const APPLY_MODELS = ["auto", "morph-v3-fast", "morph-v3-large"] as const;
|
|
9
|
+
|
|
10
|
+
export const ApplyParams = Type.Object({
|
|
11
|
+
path: Type.String({ description: "File to edit, relative to the working directory" }),
|
|
12
|
+
instructions: Type.String({
|
|
13
|
+
description: 'One first-person sentence describing the change, e.g. "I am adding error handling to divide()". Used to disambiguate the edit.',
|
|
14
|
+
}),
|
|
15
|
+
edit: Type.String({
|
|
16
|
+
description: "Only the lines that change, with `// ... existing code ...` marking the unchanged spans in between",
|
|
17
|
+
}),
|
|
18
|
+
model: Type.Optional(
|
|
19
|
+
Type.Union(
|
|
20
|
+
APPLY_MODELS.map((model) => Type.Literal(model)),
|
|
21
|
+
{ description: "Fast Apply model. Default: auto." },
|
|
22
|
+
),
|
|
23
|
+
),
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
interface ApplyResponse {
|
|
27
|
+
mergedCode?: string;
|
|
28
|
+
merged_code?: string;
|
|
29
|
+
usage?: MorphUsage;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
export const applyTool = defineTool({
|
|
33
|
+
name: "morph_apply",
|
|
34
|
+
label: "Fast Apply",
|
|
35
|
+
description:
|
|
36
|
+
"Edit a file with Morph's Fast Apply model (10,500+ tok/s, ~98% accuracy). Send the lines that change plus `// ... existing code ...` markers for the unchanged spans between them, and the file is rewritten with the edit merged in. Use it when you know what to write but the edit is spread across a large file; use `edit` for exact, mechanical replacements.",
|
|
37
|
+
promptSnippet: "morph_apply: edit a file through Morph's Fast Apply model",
|
|
38
|
+
parameters: ApplyParams,
|
|
39
|
+
annotations: { destructiveHint: true, openWorldHint: true },
|
|
40
|
+
async execute(_toolCallId, params, signal, onUpdate, ctx) {
|
|
41
|
+
const file = path.resolve(ctx.cwd, params.path);
|
|
42
|
+
const model = params.model ?? "auto";
|
|
43
|
+
return withFileMutationQueue(file, async () => {
|
|
44
|
+
let original: string;
|
|
45
|
+
try {
|
|
46
|
+
original = await fs.readFile(file, "utf8");
|
|
47
|
+
} catch {
|
|
48
|
+
throw new Error(`cannot read ${params.path}; Fast Apply merges into an existing file`);
|
|
49
|
+
}
|
|
50
|
+
onUpdate?.({
|
|
51
|
+
content: [{ type: "text", text: `Fast Apply (${model}) → ${params.path}` }],
|
|
52
|
+
details: { path: params.path, model },
|
|
53
|
+
});
|
|
54
|
+
const res = await post<ApplyResponse>(
|
|
55
|
+
"/code/apply",
|
|
56
|
+
{ initial_code: original, edit_snippet: params.edit, instructions: params.instructions, model },
|
|
57
|
+
signal,
|
|
58
|
+
);
|
|
59
|
+
const merged = res.mergedCode ?? res.merged_code;
|
|
60
|
+
if (typeof merged !== "string") throw new Error("Morph returned no merged code");
|
|
61
|
+
await fs.writeFile(file, merged, "utf8");
|
|
62
|
+
const before = original.split("\n").length;
|
|
63
|
+
const after = merged.split("\n").length;
|
|
64
|
+
return {
|
|
65
|
+
content: [{ type: "text", text: `Applied ${params.path} with ${model}: ${before} → ${after} lines` }],
|
|
66
|
+
details: { path: params.path, model, lines: { before, after } },
|
|
67
|
+
usage: usageFrom(res.usage),
|
|
68
|
+
};
|
|
69
|
+
});
|
|
70
|
+
},
|
|
71
|
+
});
|
package/tools/compact.ts
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
import { defineTool } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { Type } from "typebox";
|
|
3
|
+
import { post, usageFrom, type MorphUsage } from "./api.ts";
|
|
4
|
+
|
|
5
|
+
export const CompactParams = Type.Object({
|
|
6
|
+
input: Type.String({ description: "Text to compress: source code, a log, a chat transcript, or a mix" }),
|
|
7
|
+
query: Type.Optional(
|
|
8
|
+
Type.String({ description: "What the follow-up needs; without it Morph infers the intent from the input" }),
|
|
9
|
+
),
|
|
10
|
+
ratio: Type.Optional(
|
|
11
|
+
Type.Number({ minimum: 0, maximum: 1, description: "Target compression ratio, e.g. 0.5 keeps about half" }),
|
|
12
|
+
),
|
|
13
|
+
preserve_recent: Type.Optional(Type.Number({ minimum: 0, description: "Trailing messages to keep verbatim" })),
|
|
14
|
+
});
|
|
15
|
+
|
|
16
|
+
interface CompactResponse {
|
|
17
|
+
output?: string;
|
|
18
|
+
messages?: unknown;
|
|
19
|
+
usage?: MorphUsage;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
export const compactTool = defineTool({
|
|
23
|
+
name: "morph_compact",
|
|
24
|
+
label: "Compact",
|
|
25
|
+
description:
|
|
26
|
+
"Compress text with Morph's Compact model (~33k tok/s), keeping the parts a follow-up question needs. Use it on long code, logs, or transcripts before they go into context. Returns the compressed text; nothing else changes.",
|
|
27
|
+
promptSnippet: "morph_compact: compress long text through Morph's Compact model",
|
|
28
|
+
parameters: CompactParams,
|
|
29
|
+
annotations: { readOnlyHint: true, idempotentHint: true, openWorldHint: true },
|
|
30
|
+
async execute(_toolCallId, params, signal) {
|
|
31
|
+
const res = await post<CompactResponse>(
|
|
32
|
+
"/compact",
|
|
33
|
+
{
|
|
34
|
+
input: params.input,
|
|
35
|
+
query: params.query,
|
|
36
|
+
compression_ratio: params.ratio,
|
|
37
|
+
preserve_recent: params.preserve_recent,
|
|
38
|
+
},
|
|
39
|
+
signal,
|
|
40
|
+
);
|
|
41
|
+
const output = res.output;
|
|
42
|
+
if (typeof output !== "string") throw new Error("Morph returned no compacted output");
|
|
43
|
+
const before = params.input.length;
|
|
44
|
+
const after = output.length;
|
|
45
|
+
return {
|
|
46
|
+
content: [{ type: "text", text: output }],
|
|
47
|
+
details: {
|
|
48
|
+
chars: { before, after },
|
|
49
|
+
ratio: before > 0 ? Number((after / before).toFixed(3)) : 1,
|
|
50
|
+
},
|
|
51
|
+
usage: usageFrom(res.usage),
|
|
52
|
+
};
|
|
53
|
+
},
|
|
54
|
+
});
|
package/tools/index.ts
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { applyTool } from "./apply.ts";
|
|
3
|
+
import { compactTool } from "./compact.ts";
|
|
4
|
+
import { searchTool } from "./search.ts";
|
|
5
|
+
|
|
6
|
+
/**
|
|
7
|
+
* Morph's specialized models as Pi tools: Fast Apply, Compact, and WarpGrep.
|
|
8
|
+
*
|
|
9
|
+
* The provider in `../index.ts` serves the chat models. These three endpoints take bespoke
|
|
10
|
+
* request shapes, so they are tools rather than entries in the provider's model catalog.
|
|
11
|
+
*/
|
|
12
|
+
export default function morphTools(pi: ExtensionAPI) {
|
|
13
|
+
pi.registerTool?.(applyTool);
|
|
14
|
+
pi.registerTool?.(compactTool);
|
|
15
|
+
pi.registerTool?.(searchTool);
|
|
16
|
+
}
|
package/tools/search.ts
ADDED
|
@@ -0,0 +1,487 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* WarpGrep: a code-search subagent.
|
|
3
|
+
*
|
|
4
|
+
* `morph-warp-grep-v2.1` explores a repository in its own context window and returns only the
|
|
5
|
+
* file/line spans it settled on. The model has its search tools built in, so this module executes
|
|
6
|
+
* its calls locally and feeds the results back, turn by turn, until it calls `finish`.
|
|
7
|
+
*
|
|
8
|
+
* These tools never shell out. Morph's reference implementation pipes the model's own
|
|
9
|
+
* `list_directory` command through a shell; that command is model-written text influenced by
|
|
10
|
+
* whatever is in the repository, so here a bare `ls <path>` is parsed and every read is confined
|
|
11
|
+
* to the search root.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
import fs from "node:fs/promises";
|
|
15
|
+
import type { Dirent } from "node:fs";
|
|
16
|
+
import path from "node:path";
|
|
17
|
+
import { defineTool } from "@earendil-works/pi-coding-agent";
|
|
18
|
+
import { Type } from "typebox";
|
|
19
|
+
import { BASE_URL, cap, post, usageFrom, type MorphUsage, type Usage } from "./api.ts";
|
|
20
|
+
|
|
21
|
+
export const WARP_GREP_MODEL = "morph-warp-grep-v2.1";
|
|
22
|
+
|
|
23
|
+
const MAX_TURNS = 6;
|
|
24
|
+
const MAX_STRUCTURE_LINES = 2000;
|
|
25
|
+
const MAX_WALKED_FILES = 20_000;
|
|
26
|
+
const MAX_FILE_BYTES = 1_000_000;
|
|
27
|
+
const LIMITS = { grep: 200, read: 800, list: 500, glob: 100 };
|
|
28
|
+
const SKIP = new Set([".git", "node_modules", ".pi", ".venv", "__pycache__", ".next", "dist", "build"]);
|
|
29
|
+
|
|
30
|
+
export const SearchParams = Type.Object({
|
|
31
|
+
query: Type.String({ description: "What to find, in plain language, e.g. \"where are rate limits enforced?\"" }),
|
|
32
|
+
path: Type.Optional(Type.String({ description: "Directory to search. Default: the working directory." })),
|
|
33
|
+
turns: Type.Optional(
|
|
34
|
+
Type.Number({ minimum: 1, maximum: 10, description: `Exploration turns to allow. Default: ${MAX_TURNS}.` }),
|
|
35
|
+
),
|
|
36
|
+
});
|
|
37
|
+
|
|
38
|
+
export interface Span {
|
|
39
|
+
path: string;
|
|
40
|
+
ranges: { start: number; end: number }[];
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
interface ChatToolCall {
|
|
44
|
+
id: string;
|
|
45
|
+
type?: string;
|
|
46
|
+
function?: { name?: string; arguments?: string };
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
interface ChatMessage {
|
|
50
|
+
role: string;
|
|
51
|
+
content?: string | null;
|
|
52
|
+
tool_calls?: ChatToolCall[];
|
|
53
|
+
tool_call_id?: string;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
interface ChatResponse {
|
|
57
|
+
choices?: { message?: { content?: string | null; tool_calls?: ChatToolCall[] } }[];
|
|
58
|
+
usage?: MorphUsage;
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
interface SearchDetails {
|
|
62
|
+
query: string;
|
|
63
|
+
turns: number;
|
|
64
|
+
spans: Span[];
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
/* ------------------------------------------------------------------ arguments */
|
|
68
|
+
|
|
69
|
+
const asStr = (value: unknown): string | undefined =>
|
|
70
|
+
typeof value === "string" && value.trim() ? value.trim() : undefined;
|
|
71
|
+
|
|
72
|
+
/** Morph sends loose argument types: `limit` may be the string "50", `case_sensitive` the string "false". */
|
|
73
|
+
const asInt = (value: unknown): number | undefined => {
|
|
74
|
+
const n = typeof value === "number" ? value : typeof value === "string" ? Number.parseInt(value, 10) : Number.NaN;
|
|
75
|
+
return Number.isFinite(n) ? n : undefined;
|
|
76
|
+
};
|
|
77
|
+
|
|
78
|
+
const asBool = (value: unknown): boolean | undefined => {
|
|
79
|
+
if (typeof value === "boolean") return value;
|
|
80
|
+
if (typeof value === "string") {
|
|
81
|
+
if (/^(true|1|yes)$/i.test(value)) return true;
|
|
82
|
+
if (/^(false|0|no)$/i.test(value)) return false;
|
|
83
|
+
}
|
|
84
|
+
return undefined;
|
|
85
|
+
};
|
|
86
|
+
|
|
87
|
+
export function parseArgs(raw: unknown): Record<string, unknown> {
|
|
88
|
+
if (typeof raw !== "string" || !raw.trim()) return {};
|
|
89
|
+
try {
|
|
90
|
+
const parsed = JSON.parse(raw) as unknown;
|
|
91
|
+
return parsed && typeof parsed === "object" ? (parsed as Record<string, unknown>) : {};
|
|
92
|
+
} catch {
|
|
93
|
+
return {};
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/** Resolve a model-supplied path and keep it inside the search root. */
|
|
98
|
+
export function within(root: string, candidate: string | undefined): string {
|
|
99
|
+
const abs = path.resolve(root, candidate ?? ".");
|
|
100
|
+
const rel = path.relative(root, abs);
|
|
101
|
+
if (rel.startsWith("..") || path.isAbsolute(rel)) {
|
|
102
|
+
throw new Error(`refusing to leave the search root: ${candidate}`);
|
|
103
|
+
}
|
|
104
|
+
return abs;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/** `**`, `*`, and `?` against a relative path. */
|
|
108
|
+
export function globToRegExp(pattern: string): RegExp {
|
|
109
|
+
let out = "";
|
|
110
|
+
for (let i = 0; i < pattern.length; i++) {
|
|
111
|
+
const char = pattern[i] as string;
|
|
112
|
+
if (char === "*") {
|
|
113
|
+
if (pattern[i + 1] === "*") {
|
|
114
|
+
out += ".*";
|
|
115
|
+
i++;
|
|
116
|
+
if (pattern[i + 1] === "/") i++;
|
|
117
|
+
} else {
|
|
118
|
+
out += "[^/]*";
|
|
119
|
+
}
|
|
120
|
+
} else if (char === "?") {
|
|
121
|
+
out += "[^/]";
|
|
122
|
+
} else {
|
|
123
|
+
out += char.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
return new RegExp(`^${out}$`);
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
/* ---------------------------------------------------------------------- files */
|
|
130
|
+
|
|
131
|
+
async function walk(dir: string, out: string[]): Promise<void> {
|
|
132
|
+
if (out.length >= MAX_WALKED_FILES) return;
|
|
133
|
+
let entries: Dirent[];
|
|
134
|
+
try {
|
|
135
|
+
entries = await fs.readdir(dir, { withFileTypes: true });
|
|
136
|
+
} catch {
|
|
137
|
+
return;
|
|
138
|
+
}
|
|
139
|
+
for (const entry of entries) {
|
|
140
|
+
if (out.length >= MAX_WALKED_FILES) return;
|
|
141
|
+
const full = path.join(dir, entry.name);
|
|
142
|
+
if (entry.isDirectory()) {
|
|
143
|
+
if (SKIP.has(entry.name)) continue;
|
|
144
|
+
await walk(full, out);
|
|
145
|
+
} else if (entry.isFile()) {
|
|
146
|
+
out.push(full);
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
async function collect(dir: string): Promise<string[]> {
|
|
152
|
+
const out: string[] = [];
|
|
153
|
+
await walk(dir, out);
|
|
154
|
+
return out;
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
/** Text of a file, or undefined when it is binary, unreadable, or oversized. */
|
|
158
|
+
async function readText(file: string): Promise<string | undefined> {
|
|
159
|
+
try {
|
|
160
|
+
const stat = await fs.stat(file);
|
|
161
|
+
if (!stat.isFile() || stat.size > MAX_FILE_BYTES) return undefined;
|
|
162
|
+
const text = await fs.readFile(file, "utf8");
|
|
163
|
+
return text.includes("\0") ? undefined : text;
|
|
164
|
+
} catch {
|
|
165
|
+
return undefined;
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
/** Flat list of absolute paths: the root, then everything to `depth`, as WarpGrep expects. */
|
|
170
|
+
export async function repoStructure(root: string, depth = 2): Promise<string> {
|
|
171
|
+
const out = [root];
|
|
172
|
+
let level = [root];
|
|
173
|
+
for (let d = 1; d <= depth; d++) {
|
|
174
|
+
const next: string[] = [];
|
|
175
|
+
for (const dir of level) {
|
|
176
|
+
let entries: Dirent[];
|
|
177
|
+
try {
|
|
178
|
+
entries = await fs.readdir(dir, { withFileTypes: true });
|
|
179
|
+
} catch {
|
|
180
|
+
continue;
|
|
181
|
+
}
|
|
182
|
+
for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
|
|
183
|
+
if (entry.isSymbolicLink() || SKIP.has(entry.name)) continue;
|
|
184
|
+
if (!entry.isDirectory() && !entry.isFile()) continue;
|
|
185
|
+
const full = path.join(dir, entry.name);
|
|
186
|
+
out.push(full);
|
|
187
|
+
if (entry.isDirectory()) next.push(full);
|
|
188
|
+
if (out.length >= MAX_STRUCTURE_LINES) return out.join("\n");
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
level = next;
|
|
192
|
+
}
|
|
193
|
+
return out.join("\n");
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
/* ------------------------------------------------------------------ the tools */
|
|
197
|
+
|
|
198
|
+
export async function grepSearch(root: string, args: Record<string, unknown>): Promise<string> {
|
|
199
|
+
const pattern = asStr(args.pattern) ?? asStr(args.query) ?? asStr(args.search);
|
|
200
|
+
if (!pattern) return "grep_search needs a `pattern`.";
|
|
201
|
+
const dir = within(root, asStr(args.path));
|
|
202
|
+
const limit = asInt(args.limit ?? args.output_lines) ?? LIMITS.grep;
|
|
203
|
+
const context = asInt(args.output_context_lines ?? args.context_lines) ?? 0;
|
|
204
|
+
const caseSensitive = asBool(args.case_sensitive) ?? false;
|
|
205
|
+
let re: RegExp;
|
|
206
|
+
try {
|
|
207
|
+
re = new RegExp(pattern, caseSensitive ? "" : "i");
|
|
208
|
+
} catch (err) {
|
|
209
|
+
return `grep_search: invalid pattern ${pattern} — ${(err as Error).message}`;
|
|
210
|
+
}
|
|
211
|
+
const hits: string[] = [];
|
|
212
|
+
for (const file of await collect(dir)) {
|
|
213
|
+
const text = await readText(file);
|
|
214
|
+
if (text === undefined) continue;
|
|
215
|
+
const lines = text.split("\n");
|
|
216
|
+
for (let i = 0; i < lines.length; i++) {
|
|
217
|
+
if (!re.test(lines[i] as string)) continue;
|
|
218
|
+
const from = Math.max(0, i - context);
|
|
219
|
+
const to = Math.min(lines.length - 1, i + context);
|
|
220
|
+
for (let c = from; c <= to; c++) hits.push(`${path.relative(root, file)}:${c + 1}:${lines[c]}`);
|
|
221
|
+
if (hits.length >= limit) return cap(hits, limit, "narrow the pattern or lower `limit`");
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
return hits.length ? cap(hits, limit, "narrow the pattern") : "no matches";
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
export async function readLines(root: string, args: Record<string, unknown>): Promise<string> {
|
|
228
|
+
const target = asStr(args.path) ?? asStr(args.file) ?? asStr(args.file_path) ?? asStr(args.target_file);
|
|
229
|
+
if (!target) return "read needs a `path`.";
|
|
230
|
+
const text = await readText(within(root, target));
|
|
231
|
+
if (text === undefined) return `read: cannot read ${target} (missing, binary, or larger than ${MAX_FILE_BYTES} bytes)`;
|
|
232
|
+
const all = text.split("\n");
|
|
233
|
+
const spec = asStr(args.lines);
|
|
234
|
+
const selected: number[] = [];
|
|
235
|
+
if (spec) {
|
|
236
|
+
for (const part of spec.split(",")) {
|
|
237
|
+
const [start, end] = part.split("-").map((value) => Number.parseInt(value, 10));
|
|
238
|
+
if (!Number.isFinite(start)) continue;
|
|
239
|
+
const last = Number.isFinite(end as number) ? (end as number) : (start as number);
|
|
240
|
+
for (let i = start as number; i <= last; i++) if (i >= 1 && i <= all.length) selected.push(i);
|
|
241
|
+
}
|
|
242
|
+
} else {
|
|
243
|
+
const start = asInt(args.start) ?? asInt(args.start_line) ?? 1;
|
|
244
|
+
const end = asInt(args.end) ?? asInt(args.end_line) ?? Math.min(all.length, start + LIMITS.read - 1);
|
|
245
|
+
for (let i = start; i <= end && i <= all.length; i++) selected.push(i);
|
|
246
|
+
}
|
|
247
|
+
if (!selected.length) return `read: no lines in range for ${target}`;
|
|
248
|
+
return cap(
|
|
249
|
+
selected.slice(0, LIMITS.read).map((n) => `${n}|${all[n - 1]}`),
|
|
250
|
+
LIMITS.read,
|
|
251
|
+
"ask for a narrower line range",
|
|
252
|
+
);
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
/** Morph asks for listings as shell commands (`ls -la src`). Parse them; never run them. */
|
|
256
|
+
export async function listDirectory(root: string, args: Record<string, unknown>): Promise<string> {
|
|
257
|
+
const raw = asStr(args.command) ?? asStr(args.path) ?? ".";
|
|
258
|
+
const parsed = raw
|
|
259
|
+
.replace(/^\s*ls\b/, "")
|
|
260
|
+
.split(/\s+/)
|
|
261
|
+
.filter((token) => token && !token.startsWith("-"))
|
|
262
|
+
.join(" ");
|
|
263
|
+
const target = parsed || asStr(args.path) || ".";
|
|
264
|
+
const dir = within(root, target);
|
|
265
|
+
let entries: Dirent[];
|
|
266
|
+
try {
|
|
267
|
+
entries = await fs.readdir(dir, { withFileTypes: true });
|
|
268
|
+
} catch {
|
|
269
|
+
return `list_directory: cannot list ${target}`;
|
|
270
|
+
}
|
|
271
|
+
const lines = entries
|
|
272
|
+
.filter((entry) => !SKIP.has(entry.name))
|
|
273
|
+
.sort((a, b) => a.name.localeCompare(b.name))
|
|
274
|
+
.map((entry) => `${path.join(target, entry.name)}${entry.isDirectory() ? "/" : ""}`);
|
|
275
|
+
return lines.length ? cap(lines, LIMITS.list, "list a subdirectory") : `${target} is empty`;
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
export async function globFiles(root: string, args: Record<string, unknown>): Promise<string> {
|
|
279
|
+
const pattern = asStr(args.pattern) ?? asStr(args.glob) ?? "*";
|
|
280
|
+
const base = within(root, asStr(args.path));
|
|
281
|
+
const re = globToRegExp(pattern);
|
|
282
|
+
const matched: string[] = [];
|
|
283
|
+
for (const file of await collect(base)) {
|
|
284
|
+
const rel = path.relative(base, file);
|
|
285
|
+
if (re.test(rel) || re.test(path.basename(file))) matched.push(file);
|
|
286
|
+
}
|
|
287
|
+
const stamped = await Promise.all(
|
|
288
|
+
matched.map(async (file) => ({
|
|
289
|
+
file,
|
|
290
|
+
mtime: await fs.stat(file).then((stat) => stat.mtimeMs).catch(() => 0),
|
|
291
|
+
})),
|
|
292
|
+
);
|
|
293
|
+
stamped.sort((a, b) => b.mtime - a.mtime);
|
|
294
|
+
if (!stamped.length) return `no files match ${pattern}`;
|
|
295
|
+
return cap(
|
|
296
|
+
stamped.map((entry) => path.relative(root, entry.file)),
|
|
297
|
+
LIMITS.glob,
|
|
298
|
+
"narrow the pattern",
|
|
299
|
+
);
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
export async function runSearchTool(root: string, call: ChatToolCall): Promise<string> {
|
|
303
|
+
const name = (call.function?.name ?? "").toLowerCase();
|
|
304
|
+
const args = parseArgs(call.function?.arguments);
|
|
305
|
+
try {
|
|
306
|
+
switch (name) {
|
|
307
|
+
case "grep":
|
|
308
|
+
case "grep_search":
|
|
309
|
+
return await grepSearch(root, args);
|
|
310
|
+
case "read":
|
|
311
|
+
case "read_file":
|
|
312
|
+
return await readLines(root, args);
|
|
313
|
+
case "list_directory":
|
|
314
|
+
case "ls":
|
|
315
|
+
return await listDirectory(root, args);
|
|
316
|
+
case "glob":
|
|
317
|
+
case "find":
|
|
318
|
+
return await globFiles(root, args);
|
|
319
|
+
default:
|
|
320
|
+
return `unknown tool ${name}; available: grep_search, read, list_directory, glob, finish`;
|
|
321
|
+
}
|
|
322
|
+
} catch (err) {
|
|
323
|
+
return `${name}: ${(err as Error).message}`;
|
|
324
|
+
}
|
|
325
|
+
}
|
|
326
|
+
|
|
327
|
+
/* --------------------------------------------------------------------- finish */
|
|
328
|
+
|
|
329
|
+
/** `finish` reports spans as newline-separated `path:1-50,60-70` or `path:*` specs. */
|
|
330
|
+
export function parseSpans(files: unknown): Span[] {
|
|
331
|
+
const specs = Array.isArray(files)
|
|
332
|
+
? files.map((entry) => (typeof entry === "string" ? entry : String((entry as { path?: string })?.path ?? "")))
|
|
333
|
+
: typeof files === "string"
|
|
334
|
+
? files.split("\n")
|
|
335
|
+
: [];
|
|
336
|
+
const spans: Span[] = [];
|
|
337
|
+
for (const raw of specs) {
|
|
338
|
+
const spec = raw.trim();
|
|
339
|
+
if (!spec) continue;
|
|
340
|
+
const colon = spec.lastIndexOf(":");
|
|
341
|
+
const hasRange = colon > 0 && !spec.endsWith(":*");
|
|
342
|
+
const file = hasRange ? spec.slice(0, colon) : spec.replace(/:\*$/, "");
|
|
343
|
+
const range = hasRange ? spec.slice(colon + 1) : "";
|
|
344
|
+
const ranges = range
|
|
345
|
+
.split(",")
|
|
346
|
+
.map((part) => {
|
|
347
|
+
const [start, end] = part.split("-").map((value) => Number.parseInt(value, 10));
|
|
348
|
+
return Number.isFinite(start) ? { start: start as number, end: Number.isFinite(end as number) ? (end as number) : (start as number) } : undefined;
|
|
349
|
+
})
|
|
350
|
+
.filter((entry): entry is { start: number; end: number } => entry !== undefined);
|
|
351
|
+
spans.push({ path: file, ranges });
|
|
352
|
+
}
|
|
353
|
+
return spans;
|
|
354
|
+
}
|
|
355
|
+
|
|
356
|
+
/** The code the spans point at, which is all the parent agent sees of the search. */
|
|
357
|
+
export async function spanText(root: string, spans: Span[]): Promise<string> {
|
|
358
|
+
const parts: string[] = [];
|
|
359
|
+
for (const span of spans) {
|
|
360
|
+
let text: string | undefined;
|
|
361
|
+
try {
|
|
362
|
+
text = await readText(within(root, span.path));
|
|
363
|
+
} catch {
|
|
364
|
+
text = undefined;
|
|
365
|
+
}
|
|
366
|
+
if (text === undefined) {
|
|
367
|
+
parts.push(`${span.path} (unreadable)`);
|
|
368
|
+
continue;
|
|
369
|
+
}
|
|
370
|
+
const all = text.split("\n");
|
|
371
|
+
const ranges = span.ranges.length ? span.ranges : [{ start: 1, end: Math.min(all.length, LIMITS.read) }];
|
|
372
|
+
const body: string[] = [];
|
|
373
|
+
for (const range of ranges) {
|
|
374
|
+
for (let i = Math.max(1, range.start); i <= Math.min(all.length, range.end); i++) body.push(`${i}|${all[i - 1]}`);
|
|
375
|
+
}
|
|
376
|
+
parts.push(`${span.path}\n${cap(body, LIMITS.read, "read the file for the rest")}`);
|
|
377
|
+
}
|
|
378
|
+
return parts.join("\n\n");
|
|
379
|
+
}
|
|
380
|
+
|
|
381
|
+
/* ----------------------------------------------------------------------- loop */
|
|
382
|
+
|
|
383
|
+
export interface SearchResult {
|
|
384
|
+
answer: string;
|
|
385
|
+
spans: Span[];
|
|
386
|
+
turns: number;
|
|
387
|
+
usage: Usage;
|
|
388
|
+
}
|
|
389
|
+
|
|
390
|
+
function turnNote(turn: number, maxTurns: number): string {
|
|
391
|
+
const remaining = maxTurns - turn;
|
|
392
|
+
const note = `You have used ${turn} turn${turn === 1 ? "" : "s"} and have ${remaining} remaining.`;
|
|
393
|
+
return remaining <= 1 ? `${note} You have run out of turns to explore the code base and MUST call the finish tool now.` : note;
|
|
394
|
+
}
|
|
395
|
+
|
|
396
|
+
export interface SearchOptions {
|
|
397
|
+
signal?: AbortSignal;
|
|
398
|
+
maxTurns?: number;
|
|
399
|
+
onTurn?: (turn: number, tools: string[]) => void;
|
|
400
|
+
}
|
|
401
|
+
|
|
402
|
+
export async function warpgrep(root: string, query: string, options: SearchOptions = {}): Promise<SearchResult> {
|
|
403
|
+
const maxTurns = Math.min(Math.max(options.maxTurns ?? MAX_TURNS, 1), 10);
|
|
404
|
+
const structure = await repoStructure(root);
|
|
405
|
+
const messages: ChatMessage[] = [
|
|
406
|
+
{ role: "user", content: `<repo_structure>\n${structure}\n</repo_structure>\n\n<search_string>\n${query}\n</search_string>` },
|
|
407
|
+
];
|
|
408
|
+
const total: MorphUsage = { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 };
|
|
409
|
+
|
|
410
|
+
for (let turn = 1; turn <= maxTurns; turn++) {
|
|
411
|
+
const res = await post<ChatResponse>(
|
|
412
|
+
"/chat/completions",
|
|
413
|
+
{ model: WARP_GREP_MODEL, messages, temperature: 0, max_tokens: 2048 },
|
|
414
|
+
options.signal,
|
|
415
|
+
);
|
|
416
|
+
total.prompt_tokens = (total.prompt_tokens ?? 0) + (res.usage?.prompt_tokens ?? 0);
|
|
417
|
+
total.completion_tokens = (total.completion_tokens ?? 0) + (res.usage?.completion_tokens ?? 0);
|
|
418
|
+
total.total_tokens = (total.total_tokens ?? 0) + (res.usage?.total_tokens ?? 0);
|
|
419
|
+
|
|
420
|
+
const message = res.choices?.[0]?.message;
|
|
421
|
+
const calls = message?.tool_calls ?? [];
|
|
422
|
+
options.onTurn?.(turn, calls.map((call) => call.function?.name ?? "?"));
|
|
423
|
+
|
|
424
|
+
const finish = calls.find((call) => (call.function?.name ?? "").toLowerCase() === "finish");
|
|
425
|
+
if (finish) {
|
|
426
|
+
const spans = parseSpans(parseArgs(finish.function?.arguments).files);
|
|
427
|
+
return { answer: await spanText(root, spans), spans, turns: turn, usage: usageFrom(total) };
|
|
428
|
+
}
|
|
429
|
+
if (!calls.length) {
|
|
430
|
+
const prose = typeof message?.content === "string" ? message.content.trim() : "";
|
|
431
|
+
return { answer: prose, spans: [], turns: turn, usage: usageFrom(total) };
|
|
432
|
+
}
|
|
433
|
+
|
|
434
|
+
messages.push({ role: "assistant", content: message?.content ?? null, tool_calls: calls });
|
|
435
|
+
const results = await Promise.all(calls.map((call) => runSearchTool(root, call)));
|
|
436
|
+
for (let i = 0; i < calls.length; i++) {
|
|
437
|
+
messages.push({ role: "tool", tool_call_id: (calls[i] as ChatToolCall).id, content: results[i] as string });
|
|
438
|
+
}
|
|
439
|
+
messages.push({ role: "user", content: turnNote(turn, maxTurns) });
|
|
440
|
+
}
|
|
441
|
+
return { answer: "", spans: [], turns: maxTurns, usage: usageFrom(total) };
|
|
442
|
+
}
|
|
443
|
+
|
|
444
|
+
/* ----------------------------------------------------------------------- tool */
|
|
445
|
+
|
|
446
|
+
export const searchTool = defineTool({
|
|
447
|
+
name: "morph_search",
|
|
448
|
+
label: "WarpGrep",
|
|
449
|
+
description:
|
|
450
|
+
"Search a codebase with Morph's WarpGrep subagent (~6s). Give it a plain-language question such as \"where are rate limits enforced?\" and it explores the repository in its own context window for a few turns, then returns only the file/line spans it settled on and their code. Use it for where/what/why questions that span many files; use `grep` when you already know the exact string.",
|
|
451
|
+
promptSnippet: "morph_search: answer a code question with Morph's WarpGrep search subagent",
|
|
452
|
+
parameters: SearchParams,
|
|
453
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
454
|
+
async execute(_toolCallId, params, signal, onUpdate, ctx) {
|
|
455
|
+
const root = path.resolve(ctx.cwd, params.path ?? ".");
|
|
456
|
+
const result = await warpgrep(root, params.query, {
|
|
457
|
+
signal,
|
|
458
|
+
maxTurns: params.turns,
|
|
459
|
+
onTurn: (turn, tools) =>
|
|
460
|
+
onUpdate?.({
|
|
461
|
+
content: [{ type: "text", text: `WarpGrep turn ${turn}: ${tools.join(", ") || "thinking"}` }],
|
|
462
|
+
details: { query: params.query, turns: turn, spans: [] },
|
|
463
|
+
}),
|
|
464
|
+
});
|
|
465
|
+
const details: SearchDetails = { query: params.query, turns: result.turns, spans: result.spans };
|
|
466
|
+
if (!result.answer) {
|
|
467
|
+
return {
|
|
468
|
+
content: [
|
|
469
|
+
{
|
|
470
|
+
type: "text",
|
|
471
|
+
text: `WarpGrep did not converge within ${result.turns} turns on: ${params.query}. Try a narrower query, or grep for a string you can name.`,
|
|
472
|
+
},
|
|
473
|
+
],
|
|
474
|
+
details,
|
|
475
|
+
usage: result.usage,
|
|
476
|
+
isError: true,
|
|
477
|
+
};
|
|
478
|
+
}
|
|
479
|
+
return {
|
|
480
|
+
content: [{ type: "text", text: result.answer }],
|
|
481
|
+
details,
|
|
482
|
+
usage: result.usage,
|
|
483
|
+
};
|
|
484
|
+
},
|
|
485
|
+
});
|
|
486
|
+
|
|
487
|
+
export { BASE_URL };
|