@zihanw/pi-forge 0.1.0 → 0.2.0

package/README.md

@@ -1,80 +1,31 @@
 # pi-forge
 
-Pi extension package for prompt stack and agent profile management. This first milestone implements file-backed prompt stacks.
+[English](README.md) | [简体中文](README.zh-CN.md)
 
-## Package setup
+**pi-forge** lets you customize how Pi thinks and behaves. It gives you prompt stacks — JSON files that can replace, append to, or prepend Pi's default system prompt while controlling the AI's personality, visible tools, conversation history layout, and cross-turn state.
 
-This repository is a Pi package. Its package manifest exposes `./src/index.ts` as the extension entrypoint.
+Think of it as a character sheet for your AI agent.
 
-For local development in this checkout, `.pi/settings.json` points at the package root:
+## What you can do with it
 
-```json
-{
-  "packages": [".."]
-}
-```
+- **Give Pi a personality** — turn it into a creative writer, a roleplay partner, a strict code reviewer, or anything in between.
+- **Switch contexts instantly** — one command to swap between "coding mode", "writing mode", and "translation mode".
+- **Control what the AI sees** — choose which tools, skills, and project context appear in each prompt.
+- **Remember things across turns** — let the agent track progress, store notes, and recall user preferences throughout a session.
+- **Import SillyTavern presets** — bring your existing ST character presets into Pi with one command.
+- **Debug your prompts** — intercept and inspect exactly what gets sent to the model.
 
-After starting Pi in this directory, trust the project and use `/reload` if needed.
+## Quick start
 
-After publishing under your chosen npm package name, install it with:
+### Install
 
 ```bash
 pi install npm:@zihanw/pi-forge
 ```
 
-## Prompt stacks
-
-Prompt stacks live in:
-
-```txt
-.pi/prompt-stacks/*.json
-```
-
-Activation rules:
-
-1. A restored `/preset use <id>` selection is used first when that stack still exists and has no validation errors.
-2. A restored `/preset use none` selection disables prompt stack replacement.
-3. `.pi/prompt-stacks/default.json` auto-activates when present unless the stack sets `"autoActivate": false`.
-4. Otherwise, the first stack with `"autoActivate": true` is used.
-5. `/preset use <id>` switches stacks for the session and persists that choice.
-6. `/preset use none` disables prompt stack replacement and persists that choice.
-
-When a stack is active, pi-forge replaces Pi's default system prompt by default and rebuilds the first provider request for each user message around a movable `chat-history` slot. Tool-result follow-up turns use Pi's natural context so post-history instructions are not repeatedly re-appended after every tool call.
-
-## Commands
-
-```txt
-/preset list
-/preset use <id|none>
-/preset preview [id]
-/preset validate [id]
-/preset reload
-/preset vars [set <name> <value>|get <name>|clear [name]]
-/preset import-silly <path> [character_id]
-/intercept
-```
-
-## SillyTavern preset import
-
-Import a SillyTavern preset JSON into `.pi/prompt-stacks/<id>.json` and write a migration report to `.pi/forge/import-reports/<id>.md`:
-
-```txt
-/preset import-silly <path> [character_id]
-```
-
-If a preset contains multiple `prompt_order` entries, pass the desired `character_id`. Imported stacks are created with `autoActivate: false`; activate one with `/preset use <id>` after reviewing the report.
-
-## Agent tools
-
-### forge_set_var
-
-Sets a persistent session variable. Only variables starting with `agent.` can be written by the agent; other variables are read-only. Useful for cross-turn state tracking in roleplay (character mood, story progress) or coding (task checkpoints, discovered facts).
-
-When the `variables` slot is active in a prompt stack, the agent sees its variable state and can update it. Without the slot, variables still work for macro substitution but the agent won't have a structured view of the state.
-
-## Stack format
+### Your first prompt stack
 
-Minimal example:
+Create `.pi/prompt-stacks/default.json`:
 
 ```json
 {
@@ -86,134 +37,271 @@ Minimal example:
   "items": [
     {
       "kind": "block",
-      "id": "main-role",
+      "id": "role",
+      "name": "Main Role",
       "enabled": true,
       "role": "system",
-      "content": "You are a precise coding assistant."
+      "content": "You are a friendly and concise coding assistant. Prefer short answers with code examples."
     },
     {
       "kind": "slot",
       "id": "tools",
+      "name": "Available Tools",
       "enabled": true,
       "role": "system",
       "slot": "tools"
     },
-    {
-      "kind": "block",
-      "id": "history-open",
-      "enabled": true,
-      "role": "user",
-      "content": "<conversation_context>"
-    },
     {
       "kind": "slot",
       "id": "chat-history",
+      "name": "Chat History",
       "enabled": true,
       "slot": "chat-history"
-    },
-    {
-      "kind": "block",
-      "id": "history-close",
-      "enabled": true,
-      "role": "user",
-      "content": "</conversation_context>"
     }
   ]
 }
 ```
 
-## Items
+That's it. Restart Pi or run `/preset reload`. If no stack is already selected, `default.json` auto-activates. If you previously chose another stack or `/preset use none`, run `/preset use default`.
 
-### Blocks
+### Visual editor
 
-Static text:
+Prefer clicking over typing JSON? pi-forge has a built-in web editor:
 
-```json
-{
-  "kind": "block",
-  "id": "post-history-nudge",
-  "enabled": true,
-  "role": "user",
-  "content": "Reason carefully about the latest request before answering."
-}
 ```
+/preset ui
+```
+
+Drag, drop, edit, validate, preview, import, export, fork, and delete stacks — all in your browser.
 
-### Slots
+Import accepts native pi-forge stack JSON and SillyTavern preset JSON. SillyTavern presets are converted to prompt stacks automatically; if a preset contains multiple `character_id` configs, the editor asks which one to use.
 
-Live Pi runtime content:
+The editor runs on `127.0.0.1:41738` by default with a session token. Writes require a trusted project and stay inside `.pi/prompt-stacks`; successful save, import, fork, and delete actions reload into the current Pi session. Use `/preset ui restart` or `/preset ui stop` when needed.
+
+To use a different fixed port, create `.pi/forge/config.json`:
 
 ```json
 {
-  "kind": "slot",
-  "id": "skills",
-  "enabled": true,
-  "role": "system",
-  "slot": "skills"
+  "webEditor": {
+    "port": 41738
+  }
 }
 ```
 
-Supported slots:
-
-- `chat-history` — current Pi conversation context from the session
-- `tools` — active tool names and prompt snippets
-- `tool-guidelines` — active tool guidance
-- `skills` — loaded Pi skills
-- `project-context` — project instructions/context files
-- `append-system-prompt` — user-provided append system prompt text
-- `variables` — static/session/turn variables rendered as structured XML
-- `date`
-- `cwd`
-- `date-cwd`
-- `active-model`
-- `pi-docs`
-
-### Variables slot
-
-Renders variable state as structured XML:
-
-```xml
-<prompt_variables>
-  <static>
-    <char>Agent</char>
-  </static>
-  <session>
-    <agent.mood>happy</agent.mood>
-    <agent.progress>step 3</agent.progress>
-  </session>
-  <turn>
-    <recent>just happened</recent>
-  </turn>
-</prompt_variables>
+## Use cases
+
+### 🎭 Roleplay & creative writing
+
+Turn Pi into a character. Define their personality in the system prompt, inject writing style rules as user messages, and use `{{lastUserMessage}}` to re-insert the user's input after the conversation history.
+
+Useful pattern:
+- Put long-term character rules in a `system` block.
+- Keep Pi runtime context (tools, skills, project) in `user` slots.
+- Set the `chat-history` slot to skip the latest user message.
+- Add a final `user` block with `{{lastUserMessage}}`.
+
+This keeps the latest request clear and avoids duplicating it.
+
+For a copyable starter stack, see [examples/default-prompt-stack.json](examples/default-prompt-stack.json).
+
+### 🧑‍💻 Focused code review
+
+Create a `reviewer.json` stack with a strict review block: "prioritize correctness, regressions, security, and missing tests." Keep the `tools`, `project-context`, `variables`, and `chat-history` slots enabled so Pi can still inspect the repo and remember review state.
+
+Use `mode: "append"` if you want to keep Pi's normal coding behavior and only add the sharper review lens.
+
+### 🌐 Translation mode
+
+Create a small `translator.json` stack with one system block for tone and target language, then keep `chat-history` and `{{lastUserMessage}}` in the layout. This works well for switching between bilingual editing, literal translation, and localization review without changing your default assistant.
+
+### 🔀 Multi-mode switching
+
+Create separate stacks for different tasks:
+
 ```
+.pi/prompt-stacks/
+  coder.json       # strict coding assistant
+  writer.json      # creative writing partner
+  translator.json  # bilingual translator
+```
+
+Switch with `/preset use coder`, `/preset use writer`, etc.
 
-Options:
+### 🧠 Cross-turn memory
+
+Define state the agent can read and write:
 
 ```json
-{
-  "includeStatic": true,
-  "includeSession": true,
-  "includeTurn": true
+"state": {
+  "definitions": {
+    "agent.progress": {
+      "type": "string",
+      "scope": "session",
+      "description": "What we're working on",
+      "agentWritable": true
+    }
+  }
 }
 ```
 
-Use this slot to give the agent visibility into mutable state that it can also update via `forge_set_var`.
+The agent updates it with `forge_state_set`. You can also set state manually:
+
+```
+/state set user.preference "use TypeScript, not JavaScript"
+```
+
+### 📦 SillyTavern migration
+
+Bring your ST presets into Pi:
+
+```
+/preset import-silly ~/SillyTavern/presets/my-preset.json
+```
+
+pi-forge converts the preset to a prompt stack and generates a migration report showing what was handled and what needs manual tweaking.
+
+### 🔍 Prompt debugging
+
+See exactly what gets sent to the model:
 
-## Roles
+```
+/payload next save=.pi/forge/payloads/last.json
+```
 
-- `system` items are compiled into the replacement system prompt.
-- `user` items are inserted as ephemeral user messages.
-- `assistant` items are inserted as ephemeral assistant messages.
-- `custom` items are inserted as hidden Pi custom messages and converted to user context by Pi.
+Or preview your compiled prompt without sending anything:
+
+```
+/preset preview
+```
+
+## How it works
+
+A prompt stack is a JSON file with two kinds of items:
+
+| Kind | What it does |
+|------|-------------|
+| **Block** | Static text inserted at a specific position (system prompt, user message, assistant message) |
+| **Slot** | Dynamic content from Pi's runtime — tools, skills, chat history, date, project context, etc. |
+
+Items are arranged in order. When the stack is active, pi-forge:
+
+1. Builds a system prompt from your `system`-role blocks and slots, then applies it with the stack's `mode`.
+2. Inserts `user`/`assistant` blocks and slots around the conversation history.
+3. Expands `{{macros}}` like `{{lastUserMessage}}`, `{{date}}`, and custom variables.
+
+### Slots at a glance
+
+| Slot | What it inserts |
+|------|----------------|
+| `chat-history` | The current conversation |
+| `tools` | Available tools and their descriptions |
+| `tool-guidelines` | Tool usage instructions |
+| `skills` | Loaded Pi skills |
+| `project-context` | Project instructions and context files |
+| `variables` | Agent and user state (progress, preferences, notes) |
+| `date` / `cwd` / `date-cwd` | Current date and working directory |
+| `active-model` | Which model is being used |
+| `append-system-prompt` | User's appended system prompt text |
+| `pi-docs` | Pi documentation guidance |
+
+### Modes
+
+- **replace** (default) — your stack replaces Pi's system prompt entirely.
+- **append** — your stack is added after Pi's default system prompt.
+- **prepend** — your stack is added before Pi's default system prompt.
+
+## Common commands
+
+### Managing stacks
+
+| Command | What it does |
+|---------|-------------|
+| `/preset list` | Show all available stacks |
+| `/preset use <id>` | Activate a stack |
+| `/preset use none` | Disable prompt stacks for the session |
+| `/preset preview [id]` | See the compiled prompt |
+| `/preset validate [id]` | Check a stack for issues |
+| `/preset status` | Show the active stack and diagnostics summary |
+| `/preset diagnostics` | Show runtime diagnostics |
+| `/preset reload` | Reload stacks from disk |
+| `/preset ui [stop\|restart]` | Open, stop, or restart the web editor |
+
+### State management
+
+| Command | What it does |
+|---------|-------------|
+| `/state list` | Show all session state |
+| `/state status` | Show state definitions and current values |
+| `/state set <name> <value>` | Set a state variable |
+| `/state get <name>` | Read a state variable |
+| `/state clear [name]` | Clear state (all or by name) |
+| `/preset vars ...` | Legacy variable commands kept for older stacks |
+
+### Import & debug
+
+| Command | What it does |
+|---------|-------------|
+| `/preset import-silly <path>` | Import a SillyTavern preset |
+| `/intercept` | Show the next provider payload |
+| `/payload next [save=<path>]` | Show and optionally save the next payload |
+
+## Common macros
+
+Use these in block content to insert dynamic values:
+
+| Macro | Expands to |
+|-------|-----------|
+| `{{lastUserMessage}}` | The user's latest message |
+| `{{date}}` | Current date (YYYY-MM-DD) |
+| `{{time}}` | Current time (HH:MM:SS) |
+| `{{cwd}}` | Current working directory |
+| `{{tools}}` | Comma-separated tool names |
+| `{{selectedTools}}` | Alias for selected tool names |
+| `{{activeModel}}` | Current model (provider/id) |
+| `{{char}}` / `{{user}}` | Custom variables from your stack |
+
+### Variable macros
+
+```
+{{setvar::name::value}}       set a turn variable (cleared each message)
+{{setsessionvar::name::value}} set a session variable (persists)
+{{setvar::session::name::value}} also set a session variable
+{{getvar::name}}              read a variable (turn → session → static)
+{{getturnvar::name}}          read only a turn variable
+{{getsessionvar::name}}       read only a session variable
+{{clearvar::name}}            clear a variable
+{{clearturnvar::name}}        clear a turn variable
+{{clearsessionvar::name}}     clear a session variable
+```
+
+## Stack reference
+
+### Full item types
+
+**Block:**
+
+```json
+{
+  "kind": "block",
+  "id": "unique-id",
+  "name": "Readable label",
+  "enabled": true,
+  "role": "system",
+  "content": "Your text here. Use {{macros}} for dynamic content."
+}
+```
 
-`chat-history` expands to the live conversation at its position. By default, only the first enabled `chat-history` slot is expanded.
+Valid roles: `system`, `user`, `assistant`, `custom`.
 
-To omit the latest user message from a chat history slot, use:
+**Slot:**
 
 ```json
 {
   "kind": "slot",
-  "id": "chat-history",
+  "id": "unique-id",
+  "name": "Chat History",
   "enabled": true,
+  "role": "user",
   "slot": "chat-history",
   "options": {
     "includeLastUserMessage": false
@@ -221,66 +309,97 @@ To omit the latest user message from a chat history slot, use:
 }
 ```
 
-This is useful for SillyTavern-style stacks that re-insert `{{lastUserMessage}}` later as a post-history instruction.
+### Chat history options
+
+```json
+"options": {
+  "includeLastUserMessage": false
+}
+```
+
+Set to `false` when you use `{{lastUserMessage}}` after the history — prevents the user's message from appearing twice.
 
-To intentionally duplicate history:
+### Variables slot options
 
 ```json
 {
-  "context": {
-    "allowDuplicateChatHistory": true
+  "kind": "slot",
+  "id": "state",
+  "enabled": true,
+  "role": "user",
+  "slot": "variables",
+  "options": {
+    "includeScopes": ["session"],
+    "includeNamespaces": ["user.*", "agent.*"],
+    "includeMetadata": true,
+    "format": "xml",
+    "maxValueChars": 1200
   }
 }
 ```
 
-## Macros
+### State definitions
 
-Supported macros in block content:
+```json
+"state": {
+  "schemaVersion": 1,
+  "definitions": {
+    "agent.progress": {
+      "type": "string",
+      "scope": "session",
+      "description": "Current task progress",
+      "agentWritable": true
+    },
+    "user.preference": {
+      "type": "string",
+      "scope": "session",
+      "description": "User's preference for this session",
+      "userWritable": true
+    }
+  }
+}
+```
 
-- `{{cwd}}`
-- `{{date}}`
-- `{{lastUserMessage}}`
-- `{{selectedTools}}` / `{{tools}}`
-- `{{activeModel}}`
-- custom variables from the stack `variables` object, e.g. `{{char}}`
+Supported types: `string`, `number`, `boolean`, `null`, `object`, `array`, `string[]`, `number[]`, `boolean[]`, `unknown`, and unions like `string | null`.
 
-### Variables
+## Agent tools
 
-Static variables come from the stack file:
+pi-forge registers two tools that the AI agent can call:
 
-```json
-"variables": {
-  "char": "Assistant",
-  "user": "USER"
-}
-```
+### `forge_state_set`
+
+Batch-update persistent state. Only `agent.*` names are writable. Use for cross-turn tracking:
+
+- Task progress (`agent.progress`)
+- Open questions (`agent.openQuestions`)
+- Story state (`agent.storyState`)
+- User-requested notes (`agent.notes`)
 
-Mutable turn variables are cleared for each user message:
+### `forge_set_var`
 
-```txt
-{{setvar::name::value}}
-{{setturnvar::name::value}}
-{{getvar::name}}
-{{getturnvar::name}}
-{{var::name}}
-{{clearvar::name}}
+Legacy alias for setting a single string value. Prefer `forge_state_set`.
+
+## Package setup for development
+
+```bash
+git clone <repo>
+cd pi-forge
+# .pi/settings.json already points at the package root
+pi    # start Pi, trust the project, /reload if needed
 ```
 
-Mutable session variables persist in the Pi session as extension state:
+Run tests:
 
-```txt
-{{setsessionvar::name::value}}
-{{setvar::session::name::value}}
-{{getsessionvar::name}}
-{{getvar::name}}
-{{clearsessionvar::name}}
-{{clearvar::session::name}}
+```bash
+npm test
 ```
 
-Lookup order for `{{getvar::name}}`, `{{var::name}}`, and bare `{{name}}` is:
+Typecheck:
+
+```bash
+npm run typecheck
+```
 
-1. turn variables
-2. session variables
-3. static stack variables
+## License
 
-`setvar` macros output empty text. Unknown macros warn by default and are kept literally.
+MIT