@pmndrs/pine 0.0.0-stage → 0.1.0

package/docs/references/agents.md

@@ -0,0 +1,368 @@
+# Agents
+
+An agent is a folder under `agents/` in pine's home. The folder name is its id (lowercase
+letters, digits, `-`, `_`, starting with a letter or digit; `create_agent` wants a letter first
+and 2 to 31 characters).
+
+```
+agents/gonzo/
+  agent.json     settings (see configuration.md); {} is enough
+  workspace/     where it works, and the files it keeps itself:
+    AGENTS.md      its purpose and how it works
+    SOUL.md        who it is: character and tone
+    MEMORY.md      what should outlive a conversation
+  skills/        its own skills, one folder each; the agent adds them with add_skill
+  conversations/ what was set up for one conversation alone (its skills); made when needed
+  extensions/    optional: its own pi extensions
+  AGENTS.md      optional: rules of yours that the agent cannot change
+```
+
+## Creating an agent
+
+Ask the main agent ("create an agent that keeps my books"): it calls `create_agent`, you confirm
+what it proposes, and the folder above is made with `agent.json` and a workspace that holds its
+instructions (`AGENTS.md`) and stubs of `SOUL.md` and `MEMORY.md`. The agent exists at once,
+without a restart. Only the main agent has this tool; the
+others tell you to ask it. See [setup.md](setup.md).
+
+By hand: make the folder with an `agent.json` (`{}` is enough) and a `workspace/AGENTS.md`,
+run `pine check`, and restart.
+
+## The files an agent keeps itself
+
+An agent's instructions are files in its workspace, and the agent can write them. Together
+with one file about you they are read into its prompt with every conversation:
+
+| File | Holds | Who writes it |
+|---|---|---|
+| `workspace/AGENTS.md` | Its purpose and how it works. You shape it by telling the agent ("your purpose is ...", "from now on answer in tables") | The agent, and you |
+| `workspace/SOUL.md` | Who it is: character and tone | The agent, and you |
+| `workspace/MEMORY.md` | What matters beyond a conversation: decisions, facts, preferences, open ends | The agent |
+| `USER.md` in pine's home | What every agent knows about you. One file for all agents | The main agent, and you |
+| `agents/<id>/AGENTS.md` | Optional. Rules of yours that the agent reads and cannot change | You |
+
+Every agent is told what these files are for: to write into `AGENTS.md` what you say it is for
+or how you want things done, to put into `MEMORY.md` what it learns that matters later, at
+once and briefly, and to tidy its memory when it has grown long (merge, drop what no longer
+holds, move details into other files and leave a pointer). When `MEMORY.md` passes about 6 000
+characters, pine reminds the agent, because all of it is read with every message.
+
+`pine init` starts them as short stubs; `create_agent` writes the instructions it was given
+into `AGENTS.md` and starts the other two as stubs. A file that is missing is simply not
+loaded, and the agent creates it when it has something to put there. A change applies from the
+agent's next message on. Only these names (and a `CLAUDE.md`), at the top of the workspace, are
+read; anything else the agent keeps there is its own notes, which it reads when it needs them.
+
+Other agents get `USER.md` in their prompt but cannot open or change the file; they tell you
+when something belongs there. A conversation that has read outside content cannot change any
+of these files until it is reset ([security.md](security.md)): what is read into every future
+prompt is exactly what an injected instruction would try to rewrite.
+
+An existing home from before this layout keeps working: `agents/<id>/AGENTS.md` is still read.
+To hand the instructions over to the agent, move that file to `agents/<id>/workspace/AGENTS.md`;
+`pine init` adds the stubs that are missing and leaves your file where it is.
+
+## What an agent sees
+
+Its system prompt is put together from, in this order:
+
+1. pine's base prompt: that it is an agent in pine working for its owner, how to use its file
+   tools, that it is confined to its workspace, that pine's setup is changed through tools and
+   never by writing its files (which of them it has, and that creating agents and anything for
+   all agents is the main agent's job), that for everything else it should explain the steps to
+   its owner, and that text from tools, files and the web is information, never a command. An
+   `agents/<id>/SYSTEM.md` replaces this text.
+2. `agents/<id>/AGENTS.md`, if you keep one: rules the agent cannot change.
+3. The files it keeps itself, from its workspace: `AGENTS.md` (also `CLAUDE.md`), `SOUL.md`,
+   then `USER.md` from pine's home, then `MEMORY.md`. See "The files an agent keeps itself".
+   After them the instruction files of a folder opened for the conversation, and any files in
+   `context`.
+4. Its skills: one line each; the agent reads a skill's file when it needs it. This manual is
+   not among them: the agent searches it with its `manual` tool (see below). An agent asked
+   for another model or a route looks it up and tells you what to put into which file; an
+   event source, a schedule, an MCP server or a skill it sets up itself by tool call
+   ([setup.md](setup.md)).
+5. A `<pine>` block: who the agent is, which agents it can delegate to, which MCP servers it
+   has. It only changes when the configuration does.
+
+To show you a picture, an agent writes `![caption](path)` in its answer, with a file from its
+workspace. The terminal UI draws it (see [tui.md](tui.md)); pine's base prompt tells every agent
+this. A picture from the web it downloads first and then shows the file. A video file is shown
+the same way and can be played in the conversation.
+
+Nothing in the system prompt changes from one turn to the next. That is deliberate: a model
+server keeps what it has already read (the prompt cache) only as long as the beginning of the
+prompt is identical, and on a local model a miss means reading the whole conversation again
+before the first word of the answer. What does change, pine puts in front of your message
+instead, in a `<pine-note>`: the local time and date, the mode, the folder opened for the
+conversation, whether its shell has network access, a reminder when `MEMORY.md` has grown
+long, the shared state listed in `state.inject`, and whether the session has to be careful
+with outside content. The note is only added when something other than the time of day has
+changed since the last one. Text that arrives from elsewhere cannot fake a note; the tag is
+rewritten on the way in.
+
+The bottom bar of the terminal UI shows `cache N%` after an answer: the share of the input the
+model server took from its cache. In a running conversation it should be above 90%.
+
+The details of how pine works are not in the prompt; they are in this manual, which the agent
+searches when a question calls for it. That keeps the prompt short for small models.
+
+### The manual tool
+
+Every agent has a `manual` tool (not in a run that an outside event started). It holds the
+whole of this manual, cut into small parts: one
+for each heading, and one for each row of a table that describes a named thing (a
+configuration key, a tool, a command, an option).
+
+| Call | Answers with |
+|---|---|
+| `manual` `search: "<a few words>"` | The parts that fit best, each with its id and the sentence that matched. Several queries can be given at once, one thing each. |
+| `manual` `read: "<id of a part>"` | That part in full; several ids at once. A long part comes in pages (the id with `@2`, `@3`). |
+| `manual` with neither | The list of the manual's parts. |
+
+A search is by words, not by exact wording. Words an agent or you would use and the manual
+does not ("cron job", "reminder", "permission", "watcher") find the manual's own words through
+a list of synonyms. A source type that was registered in this installation is found as well,
+with its options. When nothing matches, the answer says so and suggests other words; it never
+says that pine cannot do something.
+
+`pine manual <words>` is the same search in a shell (without the source types registered in
+an installation), and `pine manual --read <id>` reads a part: a quick way to see what an agent
+would find. One call takes up to 6 queries and reads up to 8 parts.
+
+Nothing from your own home folder leaks in: pi would normally also pick up `~/.agents/skills`
+and instruction files in the folders above the workspace; pine filters those out. Edits to the
+instruction files take effect on the agent's next task.
+
+## Workspaces
+
+Point `workspace` at the folder the agent should work in, typically an existing repository:
+
+```json
+{ "description": "The house: contracts, utilities, solar", "workspace": "~/dev/house" }
+```
+
+The agent's shell and file tools can read and write there and nowhere else in your home. Each
+agent also has a private home folder (`sandbox/<id>/`), which is what `~` means inside its
+shell. Agents cannot read each other's workspaces; they talk through `delegate`.
+
+Two agents may share a workspace if you give them the same path. They can then overwrite each
+other's files.
+
+A single conversation can work somewhere else without any of this: `pine <folder>`, or the
+folder in the bottom bar of the terminal UI, opens a folder of yours for that one conversation.
+Its tools then start there and may change what is in it, and the folder's `AGENTS.md` (or
+`CLAUDE.md`) applies next to the agent's own. See [coding.md](coding.md).
+
+## Sessions
+
+A session is one conversation with history.
+
+| You pass | What happens |
+|---|---|
+| nothing | The agent's `main` session: one long conversation, compacted by pi when it fills up |
+| a name, for example `tg-123456` or `case-okapi` | A separate named conversation |
+| `new` | A fresh conversation for that one task; it is not continued |
+
+Routes pick the session with `"session"`, the CLI with `--session`. What you write on Telegram
+arrives in the conversation that subscribed to the Telegram source. Runs started by untrusted
+events use a twin of the session with `~u` appended,
+so they never see or extend your own conversation.
+
+Tasks for one session queue up. With `"whenBusy": "steer"` on a route, a new message is instead
+shown to the agent in the middle of what it is doing.
+
+`pine reset <agent> [session]` (or `/reset` in the terminal UI) starts a session
+over. The old history is kept on disk next to it with a `.reset-<time>` suffix for
+`limits.eventRetentionDays`. What the conversation was subscribed to on its own ends with the
+reset.
+
+A named conversation does not stay for ever. One that is not an agent's `main`, not pinned, not
+open in a tab of the terminal UI, not subscribed to an event source and without a schedule of
+its own is removed with its history after `limits.conversationDays` (7) without a message. That holds for every named
+conversation: one you opened in the terminal UI, a delegation thread's, one a route names. Pin the ones to keep (`/pin`, or the list of
+conversations in the terminal UI; see [tui.md](tui.md)), or set `limits.conversationDays` to
+`0` to keep them all. Everything that was set up for one conversation alone goes with it: its
+event sources and subscriptions, schedules, MCP servers and skills ([setup.md](setup.md)).
+
+## Handing work to another agent
+
+The `delegate` tool exists for an agent when its `delegates` list is not empty. The main agent
+(`"main"` in `pine.json`) may delegate to everyone by default.
+
+```json
+{ "delegates": ["gonzo", "scout"] }
+```
+
+What the model sees:
+
+```
+delegate(agent, task, wait?, thread?)
+```
+
+- The other agent gets only the task text, prefixed with who sent it. It runs in its own
+  workspace with its own tools and instructions.
+- By default the caller waits and gets the answer as the tool's result.
+- `wait: false` returns at once; the answer arrives later as a new message in the caller's
+  session.
+- Each delegation starts the other agent fresh. `thread: "okapi"` instead continues a named
+  conversation with it, so it remembers earlier tasks of that thread.
+- An agent already waiting further up the chain cannot be called again, and chains stop at
+  `limits.maxDelegationDepth`. The model gets an error it can act on.
+
+Write `description` for the delegating model: it is the only thing it knows about the other
+agent. "Contract law; knows the case files in its workspace" works;
+"Gonzo" does not.
+
+## Sub-agents
+
+`subagent(tasks: string[])` runs up to `subagents.maxParallel` copies of the same agent at once
+(4 unless the agent's `agent.json` says otherwise).
+Each copy has the agent's workspace, instructions and tools but an empty conversation, cannot
+delegate or spawn further, and returns only its final answer. Use it for searches and checks
+that would otherwise fill the context:
+
+> Find every clause about notice periods in `contracts/`. Use sub-agents, one per file.
+
+A sub-agent works under the permissions of the conversation that started it: access you granted
+there and your "always" answers apply to it, and what it is granted applies to the conversation.
+It has the agent's MCP servers (unless `subagents.mcp` is `false`) and all its tools, or the
+ones listed in `subagents.tools`.
+
+Sub-agents count against the model slots. With fewer slots than sub-agents they queue; nothing
+deadlocks, because the waiting parent lends out its own slot.
+
+A sub-agent is a task like any other, with its own number and its own session, and its work can
+be looked at while it runs and afterwards:
+
+- In the terminal UI, the conversation shows a line when one starts and when it answers. A click
+  on either opens a tab with what it was given, what it thought and wrote, and every tool call
+  with the start of its result. `/task <id>` opens the same tab; the agent list (click the agent
+  name in the bottom bar) lists the ones this conversation started.
+- `pine tasks` lists them, `pine task <id>` shows the task and its result, and
+  `GET /tasks/<id>/transcript` on the control API returns the steps.
+- The session file itself stays in `agents/<id>/sessions/_tasks/<task id>/`.
+
+The same works for a task that was delegated to another agent.
+
+## Shared state
+
+Small facts that several agents and routes need: who is home, the mode of the house, the state
+of the printer.
+
+- Routes write it: `"set": { "house/mode": "{{payload.value}}" }`.
+- Agents read it with `state_get` and write it with `state_set`, within `state.read` and
+  `state.write`.
+- `state.inject` shows keys to an agent in every turn, always current, without growing the
+  conversation:
+
+```json
+{ "state": { "inject": ["house/#"], "write": ["house/#"] } }
+```
+
+State values are shown to agents as facts, not as instructions, and are cut to 200 characters.
+Do not route untrusted text into a key that agents see.
+
+## What an agent's answer does
+
+- The final text of a task is its result. If the task has a reply target, it is sent there.
+- `NO_REPLY` (or `SILENT`) as the whole answer sends nothing.
+- A failed task (model error, timeout) sends "`<agent>` could not finish: ..." to the reply
+  target, so you are not left waiting. A cancelled task sends nothing.
+
+## pi extensions
+
+Any pi extension works in pine sessions. One that adds a line to every system prompt:
+
+```ts
+// ~/.pine/agents/gonzo/extensions/citations.ts
+export default function (pi) {
+	pi.on("before_agent_start", async (event) => ({ systemPrompt: `${event.systemPrompt}\n\nCite the paragraph for every legal claim.` }));
+}
+```
+
+Files in `agents/<id>/extensions/` are found automatically; list others under `extensions`.
+pi's extension guide is in `node_modules/@earendil-works/pi-coding-agent/docs/extensions.md`.
+Things that need a terminal UI (dialogs, widgets) have no effect in the daemon.
+
+`.pi/extensions/` in a workspace is deliberately not loaded; see `trustWorkspace` in
+[configuration.md](configuration.md).
+
+## Skills
+
+A skill is a folder with a `SKILL.md` that starts with a name and a description:
+
+```markdown
+---
+name: inverter
+description: Read and interpret the inverter's daily export. Use for questions about solar yield.
+---
+
+1. The exports are in `data/inverter/YYYY-MM.csv` ...
+```
+
+The agent sees the description in its system prompt and reads the file when the task calls for
+it. Files the instructions refer to (a script, a template) lie beside the `SKILL.md`.
+
+### Ask the agent
+
+> Remember how to do this: make a skill for filing invoices.
+
+The agent calls `add_skill` with a name, the description and the instructions, and optionally a
+folder of its workspace whose files go along. pine writes the skill; the agent has it from its
+next message on. Asking again under the same name replaces it, and `remove_skill` deletes it.
+
+Like an event source or an MCP server, a skill is for the conversation it was asked for in,
+for the agent, or for all agents ([setup.md](setup.md)):
+
+| Said | Who gets it | You are asked |
+|---|---|---|
+| "make a skill for this" in a side conversation | That conversation; it goes when the conversation is removed | No |
+| the same in the agent's main conversation, or "for you as an agent" | Every conversation of that agent | Only from a side conversation |
+| "for all agents" | Every agent. Only the main agent can do this | Yes |
+
+A name is used once among everything that would stand beside the skill, so no conversation
+ever has two skills of one name.
+
+A skill is part of what an agent is told, like its `AGENTS.md`. It is therefore written by
+pine, in pine's home where the agent reads but does not write, and never from a conversation
+that has read outside content: there `add_skill` refuses. What is a fact and not a way of
+doing something belongs into the agent's `MEMORY.md`.
+
+### Put it there yourself
+
+A folder with a `SKILL.md` in one of these places is a skill, with no entry anywhere:
+
+| Folder | For |
+|---|---|
+| `~/.pine/skills/<name>/` | all agents |
+| `~/.pine/agents/<id>/skills/<name>/` | that agent |
+
+What lies in these two folders an agent can also replace (`add_skill` under the same name) and
+remove (`remove_skill`), whoever put it there.
+
+A skill that lies somewhere else is named under `skills` in `pine.json` (for all) or in an
+`agent.json` (for that agent). Agents can use those and cannot replace or remove them.
+
+A skill's name is lowercase letters, digits and single hyphens, at most 64 characters; `pine`
+is taken. The description may have 1024 characters, the instructions 40 000. Files that go
+along: at most 200 and 4 MB together, real files only (a link is refused).
+
+`pine skills` lists every skill, whom it is for and where it lies.
+
+## Checking on agents
+
+```bash
+bin/pine status
+```
+
+```bash
+bin/pine tasks --agent gonzo
+```
+
+```bash
+bin/pine events --topic "agent/#" -f
+```
+
+`pine task <id>` shows one task in full: prompt, result, error, trust, and statistics (turns,
+tokens, which tools it called, how long it took).

package/docs/references/coding.md

@@ -0,0 +1,126 @@
+# Working on code
+
+Two things turn a conversation into a place to work on a project: a **folder** that is opened
+for it, and the **code pane** beside it that shows what git says changed there, the files, and
+what the agent's tool calls printed.
+
+```bash
+cd ~/dev/shop
+pine .
+```
+
+![The code pane beside a conversation, showing the diff of a file the agent changed](../images/code.png)
+
+## A folder for a conversation
+
+An agent works in its own workspace. A conversation can be given a folder of yours on top of
+that: a repository, a project. From its next message on
+
+- the agent's shell and file tools start in that folder (a relative path means a path there),
+- it may read and change what is in it, like its own workspace,
+- the folder's instruction files (`AGENTS.md`, `CLAUDE.md`) apply, next to the agent's own,
+- the agent is told which folder it works in and that its own workspace is still there.
+
+Everything else stays as it is: the rest of your home is closed, the network is closed unless
+you opened it, and the agent's mode decides what it asks before doing.
+
+The folder belongs to that one conversation. Another conversation of the same agent does not
+have it, and neither has another agent it delegates to; sub-agents it starts do. It stays until
+you take it away or remove the conversation, also across restarts. A renamed conversation
+keeps it.
+
+There are four ways to open one:
+
+| | |
+|---|---|
+| `pine .` or `pine <folder>` | Starts the terminal UI with the tabs of last time and one more on that folder: a conversation of the main agent, named after the folder. If a conversation already works there, that one is opened again |
+| The folder in the bottom bar | Click it (`no folder`, or the folder's path) for a list: no folder, this conversation's folder, the folder pine was started in (unless that is your home folder), folders other conversations of this agent use, and "Pick a folder …", which opens the system's folder chooser on macOS |
+| `/folder <path>`, `/folder none` | The same by command (`off` and `-` mean `none` too). `/folder` alone opens the list |
+| Drag a folder onto the terminal | It becomes the folder of the conversation in front. (A file dropped there is attached to your next message, as always) |
+
+![The folder menu over a conversation that works in a project folder](../images/folder.png)
+
+Only you can do this. There is no tool for it: an agent cannot open a folder for itself, and
+nothing an agent reads can.
+
+A folder is refused when it is, contains or lies in pine's home (your home folder contains it,
+if pine's home is `~/.pine`), pine's own program, or another agent's workspace. It cannot be
+changed while the conversation is working.
+
+For a project an agent should always work in, without opening anything, set `workspace` in its
+`agent.json` instead ([agents.md](agents.md)).
+
+## The code pane
+
+`▤ code` at the right end of the tab bar, `ctrl+g` or `/code` opens a pane beside the
+conversation; the same closes it. It shows the conversation's folder, or the agent's workspace
+when no folder is open. In a terminal of 100 columns or more it takes half the width (at most
+96 columns); in a narrower one it takes the place of the conversation until it is closed.
+
+```
+ pine  1 main ×  2 main·shop ×   +                                                  ▤ code
+───────────────────────────────────────────────────────────────────────────────────────────
+ make the total include shipping          │  Changes   Files   Output                 ↻  ×
+                                          │ main · 2 changed · ~/dev/shop
+ ✓ edit src/cart.ts 12 ms                 │─────────────────────────────────────────────────
+ ✓ bash npm test 2.1 s                    │ M src/cart.ts  +4 −1
+                                          │ + src/shipping.ts  +12
+ The total now includes shipping ...      │
+```
+
+| View | Shows | A click |
+|---|---|---|
+| **Changes** | What differs from the last commit: changed (`M`), added (`A`), new (`+`), deleted (`D`) and renamed (`R`) files with the lines added and removed, and the branch | on a file shows its diff, added lines green and removed ones red; `‹ back` returns to the list |
+| **Files** | The folder as a tree | on a folder opens or closes it; on a file shows its text with line numbers and colours, or a picture as a picture |
+| **Output** | Every tool call of the conversation's last 8 messages, newest first: the tool, what it was called with, how long it took | on a call shows everything it printed, not only the first lines the conversation shows |
+
+The pane follows the agent: when a tool call finishes, what is shown is read again, so a diff
+grows while the agent edits. `↻` reads it again by hand, `×` closes the pane. The wheel scrolls
+whichever side the pointer is over. Whether the pane was open is remembered for the next start.
+
+`/code changes`, `/code files` and `/code output` open it on that view; `/code off` closes it.
+
+The pane only shows. It changes no file, runs no command of the project and sends nothing to a
+model. Long lines wrap. A file is shown up to 256 KB, a diff up to 200 000 characters, a tool's
+output up to 200 000 characters and 4000 lines; binary files are named, not shown. Changes
+lists the first 300 files, and Files shows up to 500 entries of a folder.
+
+While the pane is open, pictures and videos in the conversation are drawn with coloured half
+blocks also on terminals that can show real pictures. A terminal draws a real picture only at
+the left edge of a full line, where it would lie across the pane.
+
+## What it is safe against
+
+- **Looking is not running.** A repository can name programs for git to start: a diff driver, a
+  text converter, hooks, a file system monitor. The pane runs git in a sandbox that can write
+  nothing and has no network, and with those features switched off, so a folder you only opened
+  to look at cannot make anything run.
+- **Nothing outside the folder.** Paths are resolved with their links and must stay inside the
+  folder; a link that leads out of it is neither listed nor opened.
+- **A folder is for your conversation, not for what arrives in it.** A run that an outside
+  event started, not you, does not get the folder: it works in the agent's own workspace. A
+  conversation of yours that has read outside content (a web page) keeps the folder, with the
+  two limits below. See [security.md](security.md).
+- **Instruction files.** `AGENTS.md` and `CLAUDE.md` of an opened folder are instructions to
+  the agent, like those of its workspace. Open folders you trust as much as you trust their
+  authors with your agent. Once a conversation has read outside content, it can no longer
+  change these files.
+- **Git hooks and configuration.** What lies in `.git/hooks` and `.git/config` runs with your
+  rights the next time you use git in that folder. A conversation that has read outside content
+  can still commit and branch there but cannot change those. One that has read none can, like
+  the rest of the folder: it acts on what you told it.
+
+## From a script
+
+The control API ([operations.md](operations.md)) has the same things:
+
+| Request | Purpose |
+|---|---|
+| `GET /agents/<id>/folder?session=` | The folder of a conversation, and the agent's workspace |
+| `GET /agents/<id>/folders` | Every conversation of the agent that has a folder |
+| `POST /agents/<id>/folder` `{ session, folder }` | Open a folder for a conversation; `folder: null` closes it |
+| `GET /agents/<id>/view/changes?session=` | What git says changed in the conversation's folder |
+| `GET /agents/<id>/view/diff?session=&path=` | The diff of one file (path from the top of the repository) |
+| `GET /agents/<id>/view/tree?session=&path=` | What is in one folder (path from the conversation's folder) |
+| `GET /agents/<id>/view/file?session=&path=` | A file's text |
+| `GET /tasks/<id>/transcript?full=1` | A task's steps with the whole output of each tool call |