@oyasmi/pipiclaw 0.3.4 → 0.4.0

package/docs/memory-rfc.md

@@ -0,0 +1,291 @@
+# Pipiclaw Memory Model RFC
+
+## Status
+
+Draft
+
+## Goals
+
+1. Define a minimal memory model for `pipiclaw` aligned with the `nanobot` style.
+2. Make memory flow explicit: record, read, consolidate, and reuse.
+3. Keep raw transport/session files cold and separate from memory.
+4. Limit what is loaded into session context by default.
+5. Preserve simple file-based operations and avoid introducing a dedicated memory database or memory-specific tools.
+
+## Non-Goals
+
+1. No OpenClaw-style vector retrieval, embedding index, or memory plugins.
+2. No `memory_search` or `memory_get` tool.
+3. No automatic runtime mutation of workspace-level `MEMORY.md`, `SOUL.md`, or `AGENTS.md`.
+4. No automatic loading or scanning of `log.jsonl` or `context.jsonl` as part of memory behavior.
+
+## Design Summary
+
+Pipiclaw memory is split into two channel-level files plus one workspace-level admin file:
+
+- `workspace/MEMORY.md`
+  - Stable, admin-managed, shared background memory.
+  - Not automatically updated by runtime consolidation.
+- `<channel>/MEMORY.md`
+  - Durable channel memory.
+  - Updated automatically by consolidation.
+  - May also be updated manually by the agent.
+- `<channel>/HISTORY.md`
+  - Channel history summaries.
+  - Updated automatically by consolidation only.
+  - Not intended for direct manual maintenance by the agent.
+
+Raw storage remains separate:
+
+- `<channel>/log.jsonl`
+  - Raw message log only.
+  - Cold storage.
+  - Not proactively loaded or scanned by runtime.
+- `<channel>/context.jsonl`
+  - Raw session persistence only.
+  - Cold storage.
+  - Not proactively loaded or scanned by runtime for memory purposes.
+
+## File Model
+
+### Workspace-Level Files
+
+- `SOUL.md`
+  - Loaded into session context at session start.
+  - Read-only from the agent's perspective unless a human explicitly changes it.
+- `AGENTS.md`
+  - Loaded into session context at session start.
+  - Read-only from the agent's perspective unless a human explicitly changes it.
+- `MEMORY.md`
+  - Not loaded by default.
+  - Listed in the system prompt with its role and path.
+  - Intended to be read on demand by the agent.
+  - Stable and admin-managed.
+
+### Channel-Level Files
+
+- `MEMORY.md`
+  - Not loaded by default.
+  - Listed in the system prompt with its role and path.
+  - Intended to be read on demand by the agent.
+  - Primary durable memory for the channel.
+- `HISTORY.md`
+  - Not loaded by default.
+  - Listed in the system prompt with its role and path.
+  - Intended to be read on demand by the agent.
+  - Append-oriented summary history for older context.
+- `log.jsonl`
+  - Raw archive only.
+  - Not memory.
+  - Runtime must not proactively load or scan it.
+- `context.jsonl`
+  - Raw session archive only.
+  - Not memory.
+  - Runtime must not proactively load or scan it.
+
+## Default Context Loading
+
+At session start, runtime loads the following into session context:
+
+1. Workspace-level `SOUL.md`
+2. Workspace-level `AGENTS.md`
+3. Built-in tool descriptions
+4. Summaries of both workspace-level and channel-level skills
+
+At session start, runtime does not load the following by default:
+
+1. Workspace-level `MEMORY.md`
+2. Channel-level `MEMORY.md`
+3. Channel-level `HISTORY.md`
+4. `<channel>/log.jsonl`
+5. `<channel>/context.jsonl`
+
+The system prompt must explicitly tell the agent:
+
+1. Where `workspace/MEMORY.md`, `<channel>/MEMORY.md`, and `<channel>/HISTORY.md` live.
+2. What each file is for.
+3. That these files are not preloaded.
+4. That the agent is encouraged to read them on demand when memory or history is relevant.
+
+Changes to workspace-level `SOUL.md` and `AGENTS.md` take effect on new sessions. They are not reloaded on every turn.
+
+## Agent Read/Write Rules
+
+### Reads
+
+The agent should prefer:
+
+1. `<channel>/MEMORY.md` for durable channel facts, decisions, preferences, and ongoing state.
+2. `<channel>/HISTORY.md` for older summarized context.
+3. `workspace/MEMORY.md` for stable shared background.
+
+The agent should not treat `log.jsonl` or `context.jsonl` as normal memory sources. Those files are for raw recovery, debugging, or explicit transcript-level investigation only.
+
+### Writes
+
+The agent may:
+
+1. Read `workspace/MEMORY.md`
+2. Read `<channel>/MEMORY.md`
+3. Read `<channel>/HISTORY.md`
+4. Manually update `<channel>/MEMORY.md` when necessary
+
+The agent should not manually maintain `<channel>/HISTORY.md` during normal operation. `HISTORY.md` is runtime-managed.
+
+The runtime must never automatically update:
+
+1. `workspace/SOUL.md`
+2. `workspace/AGENTS.md`
+3. `workspace/MEMORY.md`
+
+## Consolidation
+
+Consolidation is the primary mechanism that keeps memory moving.
+
+Consolidation takes recent session material and produces two outputs:
+
+1. Durable facts and live channel state in `<channel>/MEMORY.md`
+2. Older summarized history in `<channel>/HISTORY.md`
+
+Consolidation must not write to workspace-level `MEMORY.md`.
+
+### Trigger Model
+
+Runtime should run consolidation automatically only when a session is about to compact or trim context.
+
+There is no separate turn-count trigger, token-watermark trigger, or per-turn memory-dirty trigger in this RFC.
+
+### Consolidation Input
+
+Consolidation operates on recent session material already in the active session state. It does not scan `log.jsonl` or `context.jsonl`.
+
+### Consolidation Output Rules
+
+`<channel>/MEMORY.md` should contain:
+
+1. Durable facts about the channel, user, project, preferences, constraints, and long-lived open threads.
+2. Current state that matters across turns.
+3. Cleaned and deduplicated entries.
+4. Structured sections rather than free-form prose.
+
+`<channel>/HISTORY.md` should contain:
+
+1. Append-oriented summaries of older conversation chunks.
+2. Resolved decisions and notable milestones.
+3. Enough context for later recovery without replaying raw transcripts.
+
+### Consolidation Semantics
+
+Consolidation should:
+
+1. Extract durable facts from recent session material.
+2. Append new channel-memory entries into `<channel>/MEMORY.md` during normal operation.
+3. Periodically clean up `<channel>/MEMORY.md` with a larger sweep that removes outdated entries, merges duplicates, and tightens wording.
+4. Append or roll forward summarized older material into `<channel>/HISTORY.md`.
+5. Periodically fold older `HISTORY.md` blocks into coarser summaries while keeping newer blocks more detailed.
+6. Remove redundancy between the two files.
+7. Prefer stable facts in `MEMORY.md` and narrative progression in `HISTORY.md`.
+
+This RFC adopts an append-first strategy for `MEMORY.md`, with periodic cleanup passes rather than full rewrite on every consolidation.
+
+Consolidation should not:
+
+1. Dump raw message transcripts into `HISTORY.md`
+2. Copy large blocks from `context.jsonl` or `log.jsonl`
+3. Preserve outdated facts just because they were once true
+
+## Suggested File Semantics
+
+`<channel>/MEMORY.md` should be organized as stable sections such as:
+
+1. `## Identity / Participants`
+2. `## Preferences`
+3. `## Ongoing Work`
+4. `## Constraints`
+5. `## Decisions`
+6. `## Open Loops`
+
+`<channel>/HISTORY.md` should be organized as chronological summary blocks such as:
+
+1. Dated headings
+2. Short summaries of work periods
+3. Key decisions and outcomes
+
+Exact formatting can evolve, but the split between durable memory and summarized history should remain stable.
+
+## Consolidation Execution Model
+
+Consolidation uses a two-phase execution model:
+
+1. Inline phase
+   - Runs synchronously when a compaction or trim is about to happen.
+   - Responsible for producing the minimum safe memory updates needed before context is reduced.
+2. Background phase
+   - Runs later as a per-channel queued maintenance pass.
+   - Responsible for larger cleanup work, including `MEMORY.md` cleanup sweeps and `HISTORY.md` folding.
+
+The background phase must not replace the inline phase for compaction safety. It is an additional maintenance pass, not the primary durability guarantee.
+
+## Compaction And Trimming Contract
+
+Consolidation is hard-gated before compaction or trim.
+
+This means:
+
+1. If a compaction or trim would discard context, runtime must first run inline consolidation.
+2. If inline consolidation fails, compaction or trim must not proceed.
+3. After successful inline consolidation, the runtime may compact or trim.
+
+This RFC does not allow trim-first or compact-first behavior when memory durability depends on consolidation.
+
+## Runtime Responsibilities
+
+Runtime is responsible for:
+
+1. Loading only the default context described in this RFC.
+2. Exposing file locations and roles clearly in the system prompt.
+3. Running automatic consolidation.
+4. Updating `<channel>/MEMORY.md` and `<channel>/HISTORY.md` during consolidation.
+5. Keeping `log.jsonl` and `context.jsonl` cold.
+
+Runtime is not responsible for:
+
+1. Auto-loading memory files into every turn
+2. Indexing memory into a vector database
+3. Providing special memory-only tools
+
+## Session Model
+
+Session context is warm.
+
+Memory files are warm-on-demand.
+
+`log.jsonl` and `context.jsonl` are cold.
+
+The intended flow is:
+
+1. Session starts with `SOUL.md`, `AGENTS.md`, built-in tool descriptions, and skill summaries.
+2. Agent reads channel or workspace memory files if the task needs them.
+3. Conversation progresses in session state.
+4. Runtime consolidates session material into `<channel>/MEMORY.md` and `<channel>/HISTORY.md`.
+5. Older raw storage remains available but is not part of ordinary memory behavior.
+
+## Migration Impact
+
+This RFC implies the following behavior changes from the current implementation:
+
+1. `workspace/MEMORY.md` and `<channel>/MEMORY.md` are no longer injected into the system prompt by default.
+2. `SOUL.md` and `AGENTS.md` are no longer reloaded on every turn.
+3. `log.jsonl` is no longer scanned to rebuild context as part of normal turn processing.
+4. `context.jsonl` is no longer treated as a memory source.
+5. Consolidation becomes the main persistence path for channel memory.
+
+## Fixed Decisions
+
+This RFC fixes the following design choices:
+
+1. Consolidation triggers only before compaction or trim.
+2. `<channel>/MEMORY.md` uses append-first updates with periodic cleanup sweeps.
+3. `<channel>/HISTORY.md` uses append-first updates with periodic folding of older blocks.
+4. Consolidation uses a two-phase execution model: inline for safety, background for maintenance.
+5. Consolidation is hard-gated before compaction or trim.

package/docs/subagent/pi-subagent-analyse.txt

@@ -0,0 +1,190 @@
+多 Agent / Sub-Agent 模式分析
+
+  一、pi-coding-agent 当前支持到什么程度？
+
+  当前已有完整但处于 example 状态的 sub-agent 实现：
+
+  1. 进程级隔离的 SubAgent 扩展 (packages/coding-agent/examples/extensions/subagent/)
+
+  这是一个完整的 Extension，支持三种执行模式：
+
+  ┌──────────┬────────────────────────────────────────────────┬────────────────────────────────┐
+  │   模式   │                      语法                      │              描述              │
+  ├──────────┼────────────────────────────────────────────────┼────────────────────────────────┤
+  │ Single   │ { agent: "name", task: "..." }                 │ 单个子代理执行单个任务         │
+  ├──────────┼────────────────────────────────────────────────┼────────────────────────────────┤
+  │ Parallel │ { tasks: [{agent, task}, ...] }                │ 最多 8 个任务，并发度 4        │
+  ├──────────┼────────────────────────────────────────────────┼────────────────────────────────┤
+  │ Chain    │ { chain: [{agent, task: "...{previous}..."}] } │ 链式执行，前一个输出注入后一个 │
+  └──────────┴────────────────────────────────────────────────┴────────────────────────────────┘
+
+  实现方式是 spawn 独立 pi 子进程（--mode json --no-session），通过 NDJSON
+  事件流收集结果。每个子代理有完全独立的 context window。
+
+  2. Agent 发现系统 (agents.ts)
+
+  - 从 ~/.pi/agents/*.md（用户级）和 .pi/agents/*.md（项目级）加载 Agent 定义
+  - 每个 Agent 是一个带 YAML frontmatter 的 Markdown 文件，定义
+  name、description、tools、model、systemPrompt
+  - 项目级 Agent 覆盖同名的用户级 Agent
+
+  3. 轻量 Worker Agent 模式 (pipiclaw/src/memory-consolidation.ts:208-243)
+
+  pipiclaw 已经在用一种极简 sub-agent 模式：直接在进程内 new Agent() 创建一个无工具、纯文本生成的
+  worker，用于内存整理。这是最低开销的 sub-agent 形态。
+
+  4. 底层基础设施
+
+  - Agent 类完全支持独立实例化，构造参数灵活（tools、model、systemPrompt、thinkingLevel 等均可定制）
+  - AgentSession 的 Extension 系统提供 60+ 事件钩子，可以拦截/增强工具调用
+  - registerTool() API 可以在运行时注入新工具（包括"调用子代理"这个工具）
+
+  总结：核心能力已就绪，但 subagent 扩展还在 examples/ 目录，不是标准内置功能。
+
+  ---
+  二、Multi-Agent vs Sub-Agent 的区别
+
+  这是两种根本不同的协作范式：
+
+  Sub-Agent（主从模式）               Multi-Agent（对等协作模式）
+  ┌───────────┐                     ┌──────────┐    ┌──────────┐
+  │  主 Agent  │                     │ Agent A  │◄──►│ Agent B  │
+  │           │                     │  (编码)   │    │  (测试)   │
+  │  ┌──────┐ │                     └──────────┘    └──────────┘
+  │  │子Agent│ │                          ▲               ▲
+  │  │(工具) │ │                          │               │
+  │  └──────┘ │                     ┌──────────┐         │
+  └───────────┘                     │ 协调层    │◄────────┘
+                                    └──────────┘
+
+  ┌──────────┬──────────────────────────────────────────┬────────────────────────────────────────┐
+  │   维度   │                Sub-Agent                 │              Multi-Agent               │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 关系     │ 主从/调用方-被调用方                     │ 对等/协作                              │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 发起方   │ 主 Agent 通过 tool call 发起             │ 协调层编排，或 Agent 互相触发          │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 上下文   │ 子 Agent 只看到主 Agent 传入的 task 描述 │ 每个 Agent 有自己的完整上下文          │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 生命周期 │ 短暂——完成任务即销毁                     │ 长期运行，可持续对话                   │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 结果流向 │ 子 → 主（返回文本结果）                  │ 双向或多向（共享状态/消息传递）        │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 决策权   │ 主 Agent 决定何时调用、给什么任务        │ 各 Agent 可自主决策                    │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 复杂度   │ 低（现有 tool 框架直接支持）             │ 高（需要消息总线、共享状态、冲突解决） │
+  ├──────────┼──────────────────────────────────────────┼────────────────────────────────────────┤
+  │ 典型场景 │ "帮我搜索这个"、"审查这段代码"           │ "一个写代码、一个写测试、一个做Review" │
+  └──────────┴──────────────────────────────────────────┴────────────────────────────────────────┘
+
+  对 pipiclaw 的实际建议：先做 Sub-Agent，这是投入产出比最高的路径。 Multi-Agent
+  的协调成本和调试复杂度目前不适合 DingTalk bot 场景。
+
+  ---
+  三、在 pipiclaw 中如何设计和规划
+
+  Phase 1：内置 Sub-Agent 工具（短期，投入小）
+
+  将 examples/extensions/subagent/ 的核心能力集成到 pipiclaw 中，但做适配：
+
+  pipiclaw 适配要点：
+  1. 不能 spawn `pi` CLI 子进程（pipiclaw 无 CLI）
+     → 改为进程内 new Agent() + 注入精简 tool 集
+  2. Agent 发现：复用 .pi/agents/*.md 机制
+     → 或从 AGENTS.md 中解析 agent 定义
+   3. 输出回传：子 Agent 结果直接返回给主 Agent 的 tool result
+  4. 资源限制：子 Agent 的 token 消耗需要计入总量
+
+  核心实现路径：
+
+  // pipiclaw 的 sub-agent tool 骨架
+  function createSubAgentTool(parentConfig: ChannelConfig): AgentTool {
+    return {
+      name: "subagent",
+      execute: async (toolCallId, { agent, task }, signal) => {
+        const agentConfig = discoverAgents(workDir, "both")
+          .agents.find(a => a.name === agent);
+
+        // 进程内创建，比 spawn 更轻量
+        const worker = new Agent({
+          initialState: {
+            systemPrompt: agentConfig.systemPrompt,
+            model: agentConfig.model ?? parentModel,
+            tools: resolveTools(agentConfig.tools), // bash, read, write 等
+          },
+          getApiKey,
+          beforeToolCall, // 可复用主 Agent 的安全拦截
+        });
+
+        await worker.prompt(task);
+        await worker.waitForIdle();
+        return extractResult(worker);
+      }
+    };
+  }
+
+  Phase 2：Parallel Sub-Agent 支持（中期）
+
+  当 LLM 判断任务可分解时，允许并行派发多个 sub-agent：
+
+  - 实现 parallel 模式（复用 examples 中的 mapWithConcurrencyLimit）
+  - 添加 chain 模式用于流水线式任务
+
+  Phase 3：Agent 自主决策触发（长期，实验性）
+
+  让主 Agent 学会在适当场景自动使用 sub-agent，而不是用户手动指定：
+
+  - 在 SOUL.md 中定义策略（如"代码修改量超过 3 个文件时，自动启用 sub-agent 做代码审查"）
+  - 或者通过 Extension 监听 tool call 事件，在特定条件下自动触发
+
+  ---
+  四、目标效果
+
+  ┌──────────────┬─────────────────────────────────────────┬────────────────────────────────────┐
+  │     场景     │              无 Sub-Agent               │            有 Sub-Agent            │
+  ├──────────────┼─────────────────────────────────────────┼────────────────────────────────────┤
+  │ 复杂代码任务 │ 主 Agent 在单一 context                 │ 拆分子任务，每个子 Agent           │
+  │              │ 中处理所有文件，容易超长截断            │ 聚焦具体文件/模块                  │
+  ├──────────────┼─────────────────────────────────────────┼────────────────────────────────────┤
+  │ 代码审查     │ 改完即结束                              │ 主 Agent 改代码 → sub-agent 自动   │
+  │              │                                         │ review → 反馈改进                  │
+  ├──────────────┼─────────────────────────────────────────┼────────────────────────────────────┤
+  │ 信息搜集     │ 串行搜索，效率低                        │ 并行派 3 个 sub-agent              │
+  │              │                                         │ 同时搜不同方向                     │
+  ├──────────────┼─────────────────────────────────────────┼────────────────────────────────────┤
+  │ 测试         │ 手动让 Agent 跑测试                     │ 代码写完后 sub-agent               │
+  │              │                                         │ 自动运行测试并汇总                 │
+  ├──────────────┼─────────────────────────────────────────┼────────────────────────────────────┤
+  │ 长对话场景   │ context window 膨胀，质量下降           │ 重计算子任务卸载到独立 context 的  │
+  │              │                                         │ sub-agent                          │
+  └──────────────┴─────────────────────────────────────────┴────────────────────────────────────┘
+
+  核心价值：context window 分治。 DingTalk bot
+  场景中，用户可能在一个频道里提出复杂的多步骤需求，sub-agent 让主 Agent
+  可以将重计算子任务卸载，保持主 context 的精简和高质量。
+
+  ---
+  五、怎样决定是否启用 Sub-Agent
+
+  不应该让用户手动决定——由 Agent 自主判断，但需要给出决策框架：
+
+  适合启用 sub-agent 的条件（可写入 SOUL.md 作为 Agent 指令）：
+
+  1. 任务可分解性：当前任务可以拆分成独立子问题（如"搜索 A 并搜索 B"→ parallel）
+  2. context 压力：主 context 已接近上限，或需要处理大量代码文件
+  3. 专业化需求：任务需要不同"人格"（如编码 Agent vs 审查 Agent）
+  4. 耗时操作：长时间运行的 bash 命令或大范围文件搜索
+  5. 质量保证：写代码后希望有独立视角的审查
+
+  不适合使用的场景：
+
+  1. 简单问答：用户问一个简单问题，没有调用开销的必要
+  2. 上下文密集：子任务严重依赖当前对话上下文，传递成本高
+  3. 交互式任务：需要频繁和用户确认的任务（sub-agent 是批处理模式）
+  4. 成本敏感：每个 sub-agent 都有额外的 token 开销
+
+  实现建议：在 pipiclaw 中，初期可以不做自动决策，而是：
+  - 在 AGENTS.md 中定义可用的 Agent 列表和各自能力
+  - 在 SOUL.md 中告诉主 Agent "当遇到 X 场景时，可以使用 subagent 工具"
+  - 让 LLM 基于指令自主判断——这本身就是 LLM 擅长的决策
+