@cursor/july 0.1.103 → 0.1.104

package/dist/docs/design/filesystem.md

@@ -0,0 +1,376 @@
+# One FileSystem for every Agent SDK agent
+
+> **Status:** first implementation, this PR. `@cursor/july` only.
+> No Cursor SDK change.
+> **Audience:** Agent SDK maintainers.
+
+Every local turn binds a **FileSystem** and synthesizes native file
+tools from it. There is no "is there an override?" branch on the
+tool path. Authors pick the tree with `root`. Omit `root` and the
+tree is the host disk, through internal `localFileSystem`.
+
+```ts
+// Host disk. Relative paths resolve against the harness cwd.
+export default defineAgent({
+  name: "reviewer",
+});
+
+// Override. The model sees only this tree. Paths are posix `/`.
+export default defineAgent({
+  name: "notes",
+  root: tempFs({ "/notes.md": "hello" }),
+});
+```
+
+---
+
+## Names
+
+| Name | Kind | Meaning |
+| --- | --- | --- |
+| `FileSystem` | Type on `@cursor/july` | Path verbs. Authors provide this. |
+| `root` | Field on `defineAgent` | The FileSystem the agent runs on. Omit it and the root is the host disk. |
+
+No `root()` method and no public `Workspace` type.
+`local.workspaceDir` stays the harness cwd. `workspaceFiles` stays
+per-send seed files. Those names are not the path namespace.
+
+Do not pass `local.root` into `Agent.create`. The published SDK
+does not have that field.
+
+---
+
+## FileSystem
+
+```ts
+interface FileSystem {
+  list(prefix?: string): AsyncIterable<DirEntry>;
+  stat(path: string): Promise<FileInfo>;
+  glob(pattern: string, dir?: string): AsyncIterable<DirEntry>;
+  read(path: string): Promise<string | Uint8Array>;
+  grep(query: GrepQuery): AsyncIterable<GrepResult>;
+  write(path: string, contents: string): Promise<void>;
+  delete(path: string): Promise<void>;
+  search(query: SearchQuery): AsyncIterable<SearchResult>;
+}
+```
+
+Every method is required. A tree without an index yields nothing
+from `search`. Methods do not fall back to the host tree.
+Diagnostics are not a FileSystem verb.
+
+Path verbs take a path (and contents for `write`). Bags with two
+or more independent options (`grep`, `search`) take a
+named query object.
+
+`read` returns text as a string, or image bytes as a
+`Uint8Array` (`jpeg` / `png` / `gif` / `webp`). Native Read
+applies `offset` / `limit` and numbers lines on text. Images
+become a base64 image content part, the same shape the Cursor
+harness sends the model.
+
+FileSystem methods throw; they do not return a status envelope.
+`@cursor/july` does not export `FileSystemResult`.
+
+### Paths
+
+Authored `root` (`tempFs`, `unionFs`, `overlayFs`, `hostFs`, a custom tree) is one posix
+namespace rooted at `/`. Query paths and yielded paths are
+absolute (`/src/a.ts`). No trailing slash except `/`. `""`, `"/"`,
+and an omitted optional scope mean the root. Prefix tests are
+path-segment: `/agent` matches `/agent/skills`, not
+`/agent-tools`. `normalizeFsPath` resolves `..`. `/tmp` is
+reserved for turn-local scratch. See `/tmp` scratch.
+
+Host-default `localFileSystem` follows Cursor SDK natives:
+relative paths resolve against the harness cwd; absolute paths
+are used as-is. Yielded paths are host paths, not rewritten to
+`/`. `search` is empty (no host index).
+
+### Iterators
+
+Async iterators are already sorted. `unionFs` k-way merges them
+and does not re-sort a source.
+
+- `list` / `glob`: lexicographic by `path`. Paths are unique.
+- `grep`: by `path`, then `lineNumber` ascending. Keep every hit.
+- `search`: by `score` descending, then `path` ascending.
+
+A file is not under itself, so listing a file path is empty.
+Empty `list` is not "missing"; `stat` answers that.
+
+`glob(pattern, dir?)` matches relative to `dir`, not
+basename-anywhere (`*.ts` under `/src` is `/src/a.ts`, not
+`/src/nested/c.ts`).
+
+`grep` has no `output_mode` or `offset`. A slash-free `glob`
+(`*.ts`) matches the basename at any depth under `path`.
+`grep` / `search` take optional `maxResults` as predicate
+pushdown; implementations should stop after that many hits.
+Composers push the same cap to children and apply it again
+after the merge.
+
+Native-tool pagination (`head_limit`, `offset`, `output_mode`,
+`numResults`) stays on synthesized tools. Native `LS` is built
+from `list` + `stat`; there is no `ls` method.
+
+### Errors
+
+FileSystem methods throw `PathError`. `path` is the absolute
+posix path the error is about. Messages stay these contract
+strings so synthesized tools can show them as-is.
+
+- `list` / `stat` / `glob` must not throw for a missing path.
+  Missing is an empty iterator or `{ kind: "missing" }`.
+- `read` / `delete` missing file: `FileNotFoundError`
+  (`not found: ${path}`).
+- `write` has no `not found`. I/O uses the OS message or
+  `WriteFailedError` (`write failed: ${path}`). A read-only
+  mount uses `ReadOnlyMountError` (`read-only mount: ${path}`).
+- Synthesized StrReplace is `read` + exact replace + `write`.
+  Uniqueness failures throw `EditError` (`oldText is empty`,
+  `oldText and newText are exactly the same`, `The string to
+  replace was not found in the file`, and `The string to replace
+  was found multiple times in the file. Use replace_all to replace
+  all occurrences, or include more context to only edit one
+  occurrence.`). Write-back failures are `WriteFailedError`.
+- `grep` throws `InvalidGrepPatternError`
+  (`invalid grep pattern: ${pattern}`). A missing host search
+  root throws. A missing virtual path yields nothing.
+- `search` without an index is empty success.
+- `unionFs` with no mount throws `NoMountError`
+  (`no mount for path: ${path}`) on `read` / `write` /
+  `delete`. Walk verbs stay empty or missing.
+- `unionFs` catches `PathError` from a child and rethrows
+  `error.withPath(mountPrefix + childPath)`. Untyped `Error` still
+  gets a best-effort message rewrite.
+
+Invoke wraps FileSystem throws into the executor success / error
+envelope.
+
+### Write and StrReplace
+
+`write(path, contents)` creates or overwrites a file. Success is
+empty. Native Write maps `fileText` onto `contents` and prints
+`Wrote contents to ${path}`.
+
+StrReplace is not a FileSystem verb. The synthesized tool reads
+the file, applies an exact non-overlapping replace, and writes
+the result. It fails when `oldText` is empty, equals `newText`,
+is missing, or occurs more than once without `replaceAll`.
+Success is `{ occurrencesModified }`. Splice is literal; do not
+use `String.prototype.replace`. Native aliases (`old_string`,
+`new_string`, `replace_all`) map onto those fields at invoke
+time.
+
+---
+
+## Helpers
+
+| Helper | Role |
+| --- | --- |
+| `emptyFs()` | No files. Overlay leaf or missing catalog. Reads miss; writes are read-only. |
+| `tempFs(files)` | In-memory snapshot. Tests and fixtures. |
+| `unionFs({ "/repo": a, "/plans": b })` | Longest-prefix compose, path-segment match. Children are FileSystems rooted at `/`; `unionFs` strips the mount prefix on the way in and re-prefixes yields. List and glob use `unique` so an earlier mount shadows a later one. Grep and search keep every hit. Grep and search at `/` fan out to every child. |
+| `overlayFs([upper, lower])` | Same-root overlay. Earlier layers win. Read-only. A downed layer costs its own files, not the overlay — `stat` / `grep` / `search` only fail when no layer served. |
+| `hostFs(files, scope)` | `HostFilesApi` as a FileSystem rooted at `/`. Mount with `unionFs({ "/host": hostFs(files, scope) })`. `search` is empty (no index). Write without a cached etag is unconditional; after a successful read or write, write is CAS on that etag. |
+
+`localFileSystem({ cwd })` is the host-default handle. It is not
+exported from `@cursor/july`. Name helpers and `mergeSorted`
+stay private. `emptyFs`, `tempFs`, `unionFs`, `overlayFs`, `hostFs`, and
+the `PathError` classes (`FileNotFoundError`,
+`ReadOnlyMountError`, `NoMountError`, `InvalidGrepPatternError`,
+`WriteFailedError`, `EditError`,
+`DeleteNotSupportedError`) are the public helpers.
+
+---
+
+## Tool synthesis
+
+Local turns synthesize these natives from the bound FileSystem:
+`Read`, `LS`, `Grep`, `Glob`, `Write`, `StrReplace`, `Delete`,
+`SemanticSearch`. `list` + `stat` synthesize `LS`. `search`
+synthesizes `SemanticSearch`. `StrReplace` is `read` + exact
+replace + `write`. Other execute bodies call the matching
+FileSystem method.
+
+Authored `tools` can filter that set. `tools: ["read", "shell"]`
+on a host-default agent synthesizes `Read` only.
+`tools: ["edit"]` synthesizes `Write` and `StrReplace`.
+`tools: ["semSearch"]` synthesizes `SemanticSearch`.
+`tools: []` synthesizes nothing.
+
+Host-default does not synthesize `SemanticSearch`. Native
+`semSearch` stays on so the host index can run;
+`localFileSystem.search` is empty. Authored `root` synthesizes
+`SemanticSearch` from `FileSystem.search` and does not advertise
+native `semSearch`. `search` is not an allowlist alias, so an
+authored `search` tool is not treated as SemanticSearch.
+`ReadLints` is not a FileSystem verb. Authored
+`tools: ["readLints"]` fails closed so the host language service
+cannot run.
+
+Do not also ship `agent/tools/Read.ts` (and the other synthesized
+names). Discovery fails closed on that clash for host-default and
+authored `root`.
+
+A factory root re-resolves the handle before a turn. The session
+cache is the handle, not a tree snapshot. Unscoped `callTool`
+uses the invocation id and does not cache, so factories cannot
+share one tree across callers. A static `root` is the authored
+object on every session — Write / StrReplace / Delete /
+`workspaceFiles` are shared. Return a new handle from
+`(ctx) => …` when a session should have its own tree. A factory
+that returns the same object still shares.
+
+Per-send `workspaceFiles` write through the FileSystem.
+`memory/` is reserved (same as the host-default workspace
+symlink into durable agent memory) and is refused after
+normalize, including `./memory/x` and `foo/../memory/x`.
+Host-default `list` / `glob` / `grep` do not follow directory
+symlinks, including `memory/` — same as native LS/Glob/Grep.
+Exact-path `read` / `stat` still follow.
+
+Authored `root` skips session-directory materialize, git init,
+and host-cwd checks. First turns prepend `buildAgentsMdContent`
+and print `Workspace Path: /`. Host-default agents keep today's
+materialize path.
+
+---
+
+## Allowlist
+
+| `root` | `tools` | Model-facing harness |
+| --- | --- | --- |
+| omitted | omitted | The seven host-disk file tools (no synthesized `SemanticSearch`). Host shell / webSearch / sandbox / native `semSearch` stay. Host file natives are hidden with `disallowedTools: ["read","ls","grep","glob","edit","delete"]`. |
+| omitted | `["read", "shell"]` | Synthesized `Read` + SDK `shell` + `mcp`. |
+| omitted | `[]` | No synthesized file tools. |
+| set | omitted | All eight synthesized file tools (including `SemanticSearch`) + webSearch + webFetch + updateTodos + readTodos + `mcp`. Bind `/tmp` scratch. Not the host-default full catalog (no shell / sandbox / task / native semSearch). |
+| set | `["updateTodos"]` | All eight synthesized file tools + `updateTodos` + `mcp`. |
+| set | `["semSearch"]` | Synthesized `SemanticSearch` + `mcp`. Native host index stays off. |
+| set | `[]` | No synthesized file tools. Legal and empty. |
+| set | `["shell"]` / `["task"]` / `["readLints"]` | Fail closed. |
+| set | `["webSearch"]` / `["webFetch"]` | All eight synthesized file tools + the listed web tool + `mcp`. Bind `/tmp` scratch. |
+
+`mcp` is granted when any tools ride that path. File-family
+allowlist names are omitted from the SDK allowlist; the
+synthesized tools replace them.
+
+`local.sandbox` is Cursor's process sandbox around child
+processes on the host-default path. It is not a FileSystem.
+
+---
+
+## Fail closed
+
+| Combination | Result |
+| --- | --- |
+| `root` + `local.cwd` / `workspaceDir` / `sandbox` | Fail closed |
+| `root` + `runtime: "cloud"` or `"grokbot"` | Fail closed |
+| `root` + `tools` containing `"shell"` | Fail closed |
+| `root` + `tools` containing `"task"` | Fail closed. A child keeps the host toolset. Sharing `root` needs a Cursor SDK inherit, not an overridden Task. |
+| `root` + `tools` containing `"readLints"` | Fail closed. Diagnostics are not a FileSystem verb. |
+| `agent/tools/Read.ts` (etc.) | Fail closed on host-default and authored `root`. |
+
+---
+
+## `/tmp` scratch for webSearch and webFetch
+
+Native `WebSearch` and `WebFetch` stay native. They still scratch
+large output to `{harnessCwd}/agent-tools/{uuid}.txt` on the host
+disk. Under `root`, the turn composes
+`unionFs({ "/": authored, "/tmp": tempFs() })`, copies only the file
+the native tool wrote this turn, and drops that composed root at
+end of turn.
+
+### Why `/tmp`
+
+The FileSystem is the model's disk. Host `{cwd}/agent-tools/` is a
+harness path. Under `root`, the model never sees it.
+
+`/tmp` is the reserved scratch prefix. An authored `/tmp` is
+shadowed. Do not put durable files there. `{cwd}/agent-tools/` and
+`/tmp` are different namespaces.
+
+`/agent-tools` is not reserved. Prefix `/agent` does not cover
+`/agent-tools`. Those notes are path-prefix tests, not a mount.
+
+Process or VM death also loses `/tmp`. Same as POSIX `/tmp`. That
+is acceptable. Do not persist scratch across turns or process
+restarts.
+
+### Overlay
+
+| Piece | Rule |
+| --- | --- |
+| Native tool | Still searches and fetches. |
+| Bind | Per turn: `unionFs({ "/": authored, "/tmp": tempFs() })`. `/tmp` is a mount, not a FileSystem type. |
+| Session cache | Authored handle may stay cached. The composed root does not. |
+| Drop | End of turn. Next turn gets a new empty `/tmp`. |
+| Small results | Stay inline. No disk, no rewrite. |
+| Copy | Only a file the native tool wrote this turn under `{cwd}/agent-tools/`. |
+| Dest | `/tmp/{uuid}.txt`. Same uuid as the host file. |
+| Host file | Delete after copy. |
+| Other paths | Ignore. Do not copy from the result text at large. |
+
+Do not put that composed root on the session handle. A later turn
+must not read another turn's `/tmp`.
+
+### Result rewrite
+
+Keep the native tool descriptions. They do not mention spill.
+
+This package does not change the Cursor SDK. Native WebSearch /
+WebFetch results land in the AgentService conversation inside
+the local executor. That transcript is not rewritten here.
+
+What this package rewrites:
+
+| Surface | Overlay (`root` set) |
+| --- | --- |
+| `webSearch` / `webFetch` result | Host path and follow-up, in strings or structured fields (`output_location`). Other tools are untouched. |
+| Synthesized Read / Grep input | `{cwd}/agent-tools/{uuid}.txt` → `/tmp/{uuid}.txt` |
+
+| Field | Host path (`root` unset) | Overlay (`root` set) |
+| --- | --- | --- |
+| Path | `{cwd}/agent-tools/{uuid}.txt` | `/tmp/{uuid}.txt` |
+| Follow-up | Native text (today: shell / grep / `read_file`) | Read / Grep |
+
+Two modes, two paths. Do not invent a third.
+
+WebFetch proto has `output_location`. WebSearch only embeds the
+path in the chunk string. Rewrite whichever form the native tool
+used.
+
+The model follow path is Read / Grep on `/tmp/{uuid}.txt`,
+including when the model still holds the host path.
+
+### Out of scope here
+
+MCP `spillLargeMcpTextOutput` uses the same host `agent-tools`
+directory. The same overlay can cover it later. This section is
+webSearch and webFetch only.
+
+---
+
+## Live `root`
+
+When the tree depends on the conversation, pass a factory. The
+factory returns a handle. Agent SDK may reuse the object for the
+session. Reuse is not a snapshot of mount contents.
+
+```ts
+export default defineAgent({
+  name: "notes",
+  root: (ctx) => notesForSession(ctx.sessionId),
+});
+```
+
+```ts
+interface RootContext {
+  sessionId: string;
+  agentName?: string;
+  continuationKey?: string;
+  host?: { files: HostFilesApi; reminders?: ReminderHostApi };
+}
+```