@cursor/july 0.1.92 → 0.1.94

package/dist/docs/design/agsh.md

@@ -0,0 +1,406 @@
+# agsh: a shell for deployed agents
+
+## What this is
+
+`agsh` (agent shell) is a standalone CLI that connects to one agent-sdk
+deployment and turns the agent's live tool surface into commands. Every tool
+the deployment can execute (authored server tools and tools provided by the
+agent's MCP connections) becomes a subcommand with a synopsis derived from its
+input schema, a man-page style `--help`, and a place in an interactive shell.
+
+It is a separate binary and a separate package from `agent-sdk`. The
+`agent-sdk` CLI stays what it is today: the developer workflow tool for
+authoring, validating, deploying, and debugging agent projects. `agsh` is the
+operator's tool for working *inside* one deployed agent. The split also keeps
+heavy presentation dependencies (markdown rendering, syntax highlighting, the
+shell interpreter) out of `@cursor/july`, which ships to every agent project.
+
+## The experience
+
+```
+$ agsh help                        # list of commands, man-page style
+$ agsh read --help                 # man-page style: NAME, SYNOPSIS, DESCRIPTION, OPTIONS
+$ agsh read /repo/README.md
+$ agsh datadog_list_monitors --query "service:api"
+$ agsh                             # bare: interactive shell on a TTY, script from stdin otherwise
+❯ ls /repo | grep -i readme
+❯ read /repo/config.json | jq .version
+```
+
+Every invocation binds to the deployment's latest session by default, with
+`--session` and `--continuation-token` overrides, and prints the session
+identifier as a final stderr line.
+
+## Configuration
+
+`agsh` is a client only; it never boots an agent. Every invocation needs a
+target deployment, given by flags or by environment variables. Flags always
+win over the environment.
+
+Global command line options, accepted on every command and on the bare shell
+launch:
+
+| Option | Environment default | Meaning |
+| --- | --- | --- |
+| `--target <url \| name>` | `AGENT_SHELL_TARGET` | The deployment to talk to: a URL is a local deployment (`http://127.0.0.1:39400/executor`), a name a production one (`change-monitor-executor`). |
+| `--team <team>` | `AGENT_SHELL_TEAM` | Team override for production resolution, when the login spans several. |
+| `--bearer-token <token>` | `AGENT_SHELL_BEARER_TOKEN` | Explicit bearer auth for a deployment that is not behind the Cursor login. |
+| `--session <id>` | | Bind to a specific session instead of the latest. |
+| `--continuation-token <token>` | | Bind by continuation token instead of session id. |
+| `--output <text\|json>` | | Result rendering: human-friendly views (default) or raw JSON. |
+| `-h`, `--help` | | Per-command help. |
+
+One parameter carries the whole target selection, and the value's shape
+encodes the mode: a URL (`http://` or `https://`) targets a local
+deployment, anything else names a production one. Two options with a
+precedence rule would invite exactly the confusion a target selector must
+not have; with one parameter the only rule is that the flag beats the
+environment. A URL is self-contained down to the agent because one local
+agent-sdk serve process hosts every agent of the project (change-monitor's
+dev stack mounts `/executor` and `/planner` from a single port); a
+production deployment is a single agent, so its name is the complete
+address (`--team` narrows resolution when the login spans several).
+Authentication defaults to the stored Cursor login (the same engine-access
+credential agent-sdk uses); `--bearer-token` is the escape hatch for direct
+deployments. Session flags are per invocation and have no environment
+default: a session is state, not configuration. Color output follows the
+`NO_COLOR` convention and TTY detection; there is no agsh-specific color
+setting. No configuration file: one environment variable pins a working
+target for a terminal session
+(`AGENT_SHELL_TARGET=change-monitor-executor`, or a URL for a local stack),
+which is the whole persistent-configuration need.
+
+With no target from flags or environment, every command fails with a message
+naming both ways to provide one.
+
+## Architecture
+
+### A new package
+
+A new workspace package (working name `packages/agsh`, bin `agsh`) that
+depends on `@cursor/july` for target resolution, stored Cursor login, and the
+HTTP client plumbing. It owns the presentation stack: `marked` for terminal
+markdown (moved out of `@cursor/july`), with syntax highlighting (`shiki`)
+arriving in the phase that renders code; the shell interpreter is
+purpose-built (see Rationale).
+No new abstraction seam between the two packages; `agsh` imports what it
+needs until a second consumer justifies extracting a thin client.
+
+### The tool catalog
+
+At startup `agsh` fetches one live catalog of everything invocable on the
+deployment. This is the piece the current `/v1/info` cannot provide: `/v1/info`
+projects the authored manifest, and connection tools only exist at runtime,
+resolved per session under the connection's auth. A new endpoint provides the
+live view (see Backend changes).
+
+Catalog entries carry exactly one identifier each: the tool name exactly as
+the agent sees it. Authored server tools keep their authored name (`read`).
+Connection tools appear under their model-facing advertised name (the
+sanitized passthrough name from `advertise-tools.ts`, e.g.
+`datadog_list_monitors`). The CLI never invents a different naming format:
+a tool name copied from a session transcript is a valid `agsh` command, and
+vice versa. Where a tool came from — the upstream connector name when the
+tool declares one, the connection name otherwise — is a field on the
+catalog entry, not part of the identifier.
+
+### Two command tiers
+
+Each catalog entry becomes a command, through one of two shapes:
+
+**Curated commands for builtin tools.** The well-known tool names (`ls`,
+`read`, `grep`, `glob`, `diff`, ...) get hand-designed, POSIX-flavored
+command shapes, hardcoded in `agsh` next to their titles. These tools are
+what an operator types all day; their shapes should feel like the unix
+commands they mirror, not like generated bindings. A curated shape decides
+which schema fields are positional operands and which are flags, and every
+input has exactly one spelling: an operand is only an operand, never also a
+flag.
+
+```
+$ agsh read /repo/package.json --limit 2
+{
+  "name": "change-monitor",
+→ ses_a99d1b69c329eb75a2ec8603
+
+$ agsh grep -i -A 2 toolEffect /repo/src
+src/tool-policy.ts:12:export type ToolEffect = "read" | "write";
+...
+→ ses_a99d1b69c329eb75a2ec8603
+
+$ agsh ls /repo --ignore-globs '*.test.ts' --ignore-globs 'node_modules/**'
+```
+
+`read` takes its path as an operand mapped to the schema's `path` field, with
+`--offset` and `--limit` as integer flags. `grep` follows POSIX grep:
+`grep [options] <pattern> [path]`, with the rg-style options (`-i`, `-A`,
+`-B`, `-C`, `--output-mode`, `--head-limit`) mapping onto the schema fields
+of the same names (kebab-cased). `ls` shows array input: an array field's flag repeats once
+per element. A curated shape binds to the deployment's live schema at
+startup; when a deployment's tool lacks the expected field, the command
+degrades to the generic shape below rather than guessing.
+
+A curated shape may also reformat the tool's text result toward the unix
+command's own output conventions: the VFS ls tool returns the model-facing
+tree (`  - name/` rows under a header), and `agsh ls` prints it as standard
+ls does, one name per line with the trailing slash kept on directories. The
+tool's result string itself stays what the model sees; when a result does
+not match the expected shape it prints verbatim.
+
+**Generated commands for MCP tools.** Connection tools are dynamically
+discovered, so no special treatment is possible; they get a uniform
+schema-derived mapping:
+
+- Every schema property is accepted as one flag, spelled as the
+  kebab-cased property name (`org_slug` → `--org-slug`) — the unix
+  convention; kebab collisions gain a numeric suffix. Properties already
+  shaped like flags (grep's `-i`) stay literal. No positionals, no other
+  aliases.
+- Object-typed properties flatten recursively into one flag per leaf,
+  dash-joined (`--telemetry-context` for `telemetry.context`), so every
+  option reads as a plain value; a free-form object with no declared
+  properties stays one JSON-valued flag. A leaf is required only when its
+  whole ancestor chain is.
+- Values are coerced by schema type: booleans are valueless flags, numbers
+  and integers are parsed, arrays accept the flag repeated once per element,
+  enums are validated before the call.
+
+```
+$ agsh datadog_list_monitors --query "service:api" --limit 10
+```
+
+In both tiers `-h`/`--help` and the global target and session flags are
+reserved and injected, a flag that names no schema property fails before any
+request (listing the tool's actual properties), and the bound session prints
+as a final stderr line.
+
+### Result rendering
+
+Raw JSON on a terminal is not an experience for people, so `--output=text`
+(the default) renders structured results through a small set of views,
+selected automatically by the shape of the value each call actually returned;
+tool metadata plays no part, since most tools advertise no output schema, and
+many return structured data as JSON text. A string result that parses as a
+JSON object or array counts as structured. An array of objects renders as a
+table (columns are the union of keys, missing cells stay blank, the table
+clamps to the terminal width); a single object renders as a property view
+(aligned keys, scalar lists as bullets, nested structures indented); an
+object that is nothing but an error wrapper renders as an `Error:` line;
+plain text prints verbatim. `--output=json` renders the structured value as
+raw JSON. The rendering never depends on the TTY: piped and interactive
+output carry the same content, only color follows TTY detection.
+
+### Help rendering
+
+`--help` on a tool renders a man-page layout: NAME (the tool name, with the
+tool's `title` beside it when the catalog carries one; titles are curated
+data, never derived from the description), SYNOPSIS (operands from the
+curated shape; options never enumerate — they summarize as `[options...]`,
+man-page style, so the line stays bounded), DESCRIPTION (the tool
+description rendered as terminal markdown), OPERANDS (positional arguments,
+curated commands only), and OPTIONS. Descriptions of operands and options
+come from the schema's property descriptions. Effect and approval metadata
+render as notes when declared. Everything except the curated shape derives
+from `GET /v1/tools/:name`; nothing else is hand-written per tool.
+
+```
+$ agsh read --help
+NAME
+    read - Read a file
+
+SYNOPSIS
+    read [options...] <path>
+
+DESCRIPTION
+    Reads a file from the local filesystem. This tool can also read image
+    files when called with the appropriate path. Formats supported:
+    jpeg/jpg, png, gif, webp.
+
+OPERANDS
+    <path>
+        The absolute path of the file to read.
+
+OPTIONS
+    --offset <integer>
+        The line number to start reading from. Positive values are 1-indexed
+        from the start of the file. Negative values count backwards from the
+        end. Only provide if the file is too large to read at once.
+
+    --limit <integer>
+        The number of lines to read. Only provide if the file is too large
+        to read at once.
+
+NOTES
+    Effect: read (performs no writes).
+```
+
+`agsh help` lists the available command names grouped by source, authored
+tools first, then one group per upstream connector (its name is the group
+header — one aggregating connection can host tools from several connectors,
+and the connector name is what an operator recognizes). Each row is the
+name, with the title beside it when the tool declares one; everything else
+lives behind the command's `--help`:
+
+The agent's description renders as a DESCRIPTION section when the deployment
+declares one (`/v1/info` carries both name and description).
+
+```
+$ agsh help
+NAME
+    change-monitor-executor
+
+DESCRIPTION
+    Executes monitoring plans against changed code.
+
+COMMANDS
+    diff                     Show workspace changes
+    glob                     Find files by pattern
+    grep                     Search file contents
+    ls                       List a directory
+    read                     Read a file
+    report_change_issue
+    report_change_succeeded
+
+DATADOG
+    datadog_list_monitors    List monitors
+    ...
+
+Run any command with --help for its synopsis and options.
+```
+
+### Shell mode
+
+Invoked bare, `agsh` starts a shell. On a TTY this is a REPL; on a pipe it
+reads a script from stdin, so `echo 'ls /' | agsh` and here-docs work.
+
+The interpreter is purpose-built and minimal: tokenizing (quotes, escapes),
+pipelines, and `;` / `&&` / `||`. The command namespace is exactly the
+deployment's tool catalog plus a small curated set of local pipe filters
+(`head`, `tail`, `wc`, stdin-filtering `grep`), so a tool name can never be
+shadowed. There is no local filesystem, no variables, no control flow: agsh
+has nothing local to operate on, and every command is a single traced
+`POST /v1/tools/:name` call.
+
+The shell binds one session identity at launch (latest by default) and keeps
+it for the whole run, so a sequence of tool calls observes one consistent
+session context.
+
+## Backend changes on the agent-sdk runtime
+
+Two read endpoints, mirroring the invocation path:
+
+**`GET /v1/tools`: the live tool listing.** Returns the session's tool
+namespace exactly as a turn would assemble it: authored server tools plus the
+advertised passthrough tools synthesized from connections, under their
+model-facing names. Entries are light (name, source, and `title` when one is
+known); everything else lives behind the detail endpoint. Titles have two
+sources and no new authoring surface: connection tools inherit the upstream
+server's MCP title, which the host already propagates length-capped off
+listings; tools that do not come from MCP get theirs from a hardcoded
+name-to-title table in the runtime's endpoint implementation, covering the
+well-known tool names. A tool in neither place has no title. Accepts the same
+optional session binding as invocation (`session` or `continuationToken`)
+because advertised inventories can be tenant-scoped and resolved per session.
+Implementation reuses the existing plumbing: the discovered manifest for
+authored tools and the advertise-tools synthesis (`McpHost.listTools`, or the
+`oneOff` path when per-session auth substitution applies) for connection
+tools. This is not a duplicate of `/v1/info`: the info document stays the
+static authored manifest; the listing is the runtime view that only the
+running deployment can answer.
+
+**`GET /v1/tools/:name`: one tool's full description.** Description, input
+schema, output schema when declared, effect when declared, approval
+requirement, and source connection. Same path as invocation
+(`POST /v1/tools/:name`), different method: GET describes what POST executes,
+for the same identifier.
+
+Invocation needs no new naming scheme. Advertised connection tools are
+synthesized as ordinary server tools in the session's namespace, so
+`POST /v1/tools/:name` addresses them by their model-facing name like any
+authored tool, with the same session binding, policy checks, and per-call
+tracing. (The direct-call path did need the synthesis step added: it now
+resolves the advertised listing for the call's session identity when the
+authored lookup misses.)
+
+Phase 1 ships the minimal runtime surface agsh calls: the `effect`
+projection in `/v1/info` (rendered in per-tool help), the scratch-workspace
+fallback on direct calls, and `continuationToken` binding on
+`POST /v1/tools/:toolName`. The detail endpoint in phase 2 also closes the
+output-schema gap; `/v1/info` stays as it is.
+
+## Local development loop
+
+`factory/change-monitor` is the test bed. Its `pnpm start` already serves the
+planner and executor locally through the agent-sdk dev runtime
+(`agent-sdk serve --dir . --dev`). The loop:
+
+1. `cd factory/change-monitor && pnpm start` (local stack, both agents).
+2. `agsh --target http://127.0.0.1:<port>/<agent>` against it, via a dev shim
+   analogous to `agent-sdk-dev` so the CLI runs from the worktree.
+3. Iterate end to end: VFS verbs (`ls`, `read`, `grep`, `glob`, `diff`) for the
+   authored-tool path, and the planner's tenant connectors for the
+   connection-tool path once `GET /v1/tools` exists.
+
+## Removing the inspector surface from agent-sdk
+
+The inspector CLI is still on development branches, so nothing migrates: the
+CLI-side code is removed from `agent-sdk` and `agsh` is built in its place.
+
+- The verb commands (`ls`, `read`, `grep`, `glob`, `diff`) become the
+  curated tier: their hand-designed shapes, schema-binding logic (including
+  the candidate-field fallback), and session binding carry over. The
+  schema-to-argv flag mapping seeds the generated tier for MCP tools.
+- The `tools` and `skills` commands disappear entirely. `agsh help` and
+  per-tool `--help` are the discovery surface.
+- `marked` and `shiki` leave `@cursor/july`; agsh's help rendering takes
+  `marked`, and `shiki` returns when agsh ships syntax highlighting.
+  `agent-sdk` keeps its developer workflow commands unchanged.
+
+## Plan
+
+1. **Package and core invocation.** Create the package, port target
+   resolution, the schema-to-argv mapping, and help rendering from the
+   inspector code. Authored tools only, against the existing endpoints.
+   Verified end to end on the local change-monitor stack.
+2. **Live catalog.** Add `GET /v1/tools` and `GET /v1/tools/:name` to
+   the agent-sdk runtime, with the hardcoded title table for non-MCP tools, and verify
+   direct invocation resolves advertised connection tools by their
+   model-facing names. Connection tools appear as commands. Verified against
+   the planner's connectors.
+3. **Shell mode.** The purpose-built mini-shell: REPL on TTY, script on
+   stdin, tools as the command namespace, one session per shell run.
+4. **Cleanup.** Remove the inspector CLI surface and presentation
+   dependencies from `@cursor/july`.
+
+## Rationale and rejected alternatives
+
+**Why not extend `agent-sdk`.** The audiences differ: `agent-sdk` is for the
+person building and deploying an agent; this tool is for the person operating
+inside one. Bundling also forces every agent project to carry markdown
+rendering, syntax highlighting, and a bash interpreter it never uses.
+
+**Name.** `agsh` reads as "agent shell", is four characters, collides with
+nothing common, and works as a shell prompt name. Considered: `august`
+(pairs with `july` but says nothing about purpose), `toolsh` (awkward to
+pronounce), `cursor-shell` (too broad; this is scoped to one agent).
+
+**Why a REST catalog instead of the MCP endpoint.** The deployment already
+speaks MCP at `/v1/mcp/tools`, including a per-connection bridge, but the
+bridge is bound to an active turn and speaks JSON-RPC. The CLI wants a plain
+authenticated GET with session binding that returns the assembled tool
+namespace under the names the model sees. Wrapping that in MCP framing buys
+nothing for a first-party client.
+
+**Why a purpose-built interpreter instead of just-bash.** just-bash was the
+original plan (a full bash emulation with a custom-command extension point),
+and a prototype disproved it: custom commands replace its coreutils but can
+never shadow its shell builtins, and `read`, `test`, `type`, and `help` are
+builtins — so the flagship `read` tool is unreachable, and the precedence is
+not ours to control (vercel-labs owns the package). No other embeddable JS
+shell interpreter has a workable custom-command story (mvdan-sh's JS build
+does not expose one; bash-parser is a parser only). agsh also needs almost
+none of bash: no local filesystem, no variables, no control flow — just
+tokenizing, pipelines, and a command namespace it fully owns. A TypeScript
+REPL with tools as async functions (the shape of change-monitor's `script`
+tool) was considered and kept as a possible later addition; it trades away
+the unix muscle memory the curated commands exist for.