@oyasmi/pipiclaw 0.5.1 → 0.5.3

package/docs/improve-memory/spec.md

@@ -1,357 +0,0 @@
-# Pipiclaw Context Upgrade Spec
-
-## Status
-
-Draft
-
-## Purpose
-
-This spec defines the next-stage context architecture for `pipiclaw`.
-
-It extends the existing [memory RFC](/Users/oyasmi/projects/pipiclaw/docs/memory-rfc.md) rather than replacing it wholesale.
-
-The primary goal is not "more memory files". The goal is to materially improve:
-
-1. cross-turn continuity
-2. long-task stability
-3. compaction resilience
-4. sub-agent usefulness
-5. skill usefulness in a long-running DingTalk channel workspace
-
-The design remains file-based and runtime-local. It does not introduce embeddings, vector search, or a dedicated memory database.
-
-## Problems To Solve
-
-The current model is good enough to persist durable facts, but still leaves several gaps:
-
-1. `MEMORY.md` and `HISTORY.md` are available but not proactively surfaced, so the model often fails to use them at the right time.
-2. "current working state" is mixed into durable memory and is therefore either too weak, too stale, or too noisy.
-3. compaction happens with only indirect help from memory consolidation, which is weaker than having an explicit working-state artifact.
-4. sub-agents are context-isolated, but there is no standard way to give them the most relevant channel memory automatically.
-5. skills are loaded, but they do not yet participate in a richer lifecycle where memory and runtime hooks reinforce one another.
-
-## Non-Goals
-
-1. No vector retrieval, embedding index, semantic database, or memory plugin framework.
-2. No special standalone `memory_search` tool for the model.
-3. No automatic mutation of workspace-level `SOUL.md`, `AGENTS.md`, or workspace `MEMORY.md`.
-4. No proactive scanning of `log.jsonl` or `context.jsonl` as normal memory inputs.
-5. No attempt to clone the full `claude-code` agent OS, swarm runtime, or prompt-cache infrastructure in this phase.
-
-## Context Layers
-
-The upgraded context model has five layers.
-
-### 1. Identity Layer
-
-- `workspace/SOUL.md`
-- `workspace/AGENTS.md`
-
-Semantics:
-
-- loaded into session context at session start
-- human-managed
-- authoritative
-- not rewritten by runtime maintenance
-
-### 2. Shared Durable Memory
-
-- `workspace/MEMORY.md`
-
-Semantics:
-
-- stable shared background
-- read on demand
-- not auto-rewritten by runtime
-
-### 3. Channel Durable Memory
-
-- `<channel>/MEMORY.md`
-
-Semantics:
-
-- durable facts, decisions, preferences, constraints, medium-horizon open loops
-- append-first, then cleanup/fold
-- channel-scoped
-- runtime-managed
-
-Important change:
-
-`MEMORY.md` is no longer the primary owner of minute-by-minute "what am I doing right now" state. It may still contain open loops, but detailed active execution state belongs to `SESSION.md`.
-
-### 4. Channel Working Memory
-
-- `<channel>/SESSION.md`
-
-Semantics:
-
-- current task state
-- active files and commands
-- current hypotheses and next steps
-- recent important errors and corrections
-- channel-scoped handoff artifact across sessions and compactions
-- runtime-managed
-
-This is the major addition in this spec.
-
-### 5. Channel History
-
-- `<channel>/HISTORY.md`
-
-Semantics:
-
-- summarized chronological recovery material
-- older work periods, decisions, milestones
-- runtime-managed
-- not intended for normal manual maintenance
-
-## Cold Storage
-
-These files remain cold:
-
-- `<channel>/log.jsonl`
-- `<channel>/context.jsonl`
-
-They are not part of normal memory recall and are not proactively loaded into prompts.
-
-## Core Invariants
-
-The upgraded design must preserve these invariants:
-
-1. File-based operation remains the source of truth.
-2. The model should not need to remember to manually read the right memory file in common cases.
-3. `SESSION.md` is the working-state artifact.
-4. `MEMORY.md` is the durable-facts artifact.
-5. `HISTORY.md` is the chronological-summary artifact.
-6. Workspace memory remains human-managed.
-7. Raw transport/session archives remain cold.
-8. Failure of a background updater must not corrupt persisted memory files.
-9. The system must degrade gracefully when an updater fails.
-10. Memory improvements must help sub-agents and skills, not just the main agent.
-
-## File Semantics
-
-### `SESSION.md`
-
-`SESSION.md` is channel-scoped hot memory.
-
-It should answer:
-
-1. What is the user currently trying to achieve?
-2. What is the current state of work?
-3. Which files and commands matter right now?
-4. What constraints and recent failures should not be forgotten?
-5. What are the next likely steps?
-
-It should not become:
-
-1. a raw transcript
-2. a duplicate of all durable facts already in `MEMORY.md`
-3. an infinite worklog
-
-Read/write rule:
-
-1. runtime-managed by default
-2. eligible for automatic targeted recall
-3. not intended for normal manual maintenance by the main agent
-4. may be edited only when an explicit user/admin instruction makes that the task itself
-
-### `MEMORY.md`
-
-`MEMORY.md` remains the durable channel memory, but the threshold for what belongs here gets stricter.
-
-It should keep:
-
-1. stable preferences
-2. durable decisions
-3. medium-horizon constraints
-4. important open loops that must survive beyond the current execution burst
-
-It should avoid:
-
-1. step-by-step active worklog
-2. temporary local debugging observations unless they have lasting value
-3. detailed "current state" that will likely be obsolete after a few turns
-
-Read/write rule:
-
-1. readable on demand
-2. writable by runtime consolidation
-3. still manually writable when necessary
-4. cleanup is allowed to remove transient state that should now live in `SESSION.md`
-
-### `HISTORY.md`
-
-`HISTORY.md` remains the recovery narrative.
-
-It should keep:
-
-1. notable work periods
-2. milestones
-3. decision outcomes
-4. enough chronology for later recovery
-
-It should avoid:
-
-1. dense per-turn detail
-2. raw snippets copied from transcript
-3. facts better represented in `MEMORY.md`
-
-Read/write rule:
-
-1. readable on demand
-2. runtime-managed
-3. not intended for ordinary manual edits
-
-## Closed-Loop Lifecycle
-
-The upgraded model introduces an explicit closed loop:
-
-1. active session work happens in warm context
-2. recent work updates `SESSION.md`
-3. stable facts and medium-horizon open loops are promoted into `MEMORY.md`
-4. older narrative is summarized into `HISTORY.md`
-5. future prompts receive targeted recall from `SESSION.md`, `MEMORY.md`, and `HISTORY.md`
-
-This loop must work even when:
-
-1. the session compacts
-2. the user starts a new session in the same channel
-3. the process restarts
-4. the main agent delegates work to a sub-agent
-
-## Recall Model
-
-The system should stop relying purely on "the model may remember to read memory files".
-
-Instead, each turn may inject a small amount of relevant memory context chosen from:
-
-1. `SESSION.md`
-2. channel `MEMORY.md`
-3. workspace `MEMORY.md`
-4. channel `HISTORY.md`
-
-Selection rules:
-
-1. small budget
-2. high precision
-3. recency-aware
-4. section-aware
-5. prefer `SESSION.md` when current work state matters
-6. prefer `MEMORY.md` when durable constraints or preferences matter
-7. prefer `HISTORY.md` when recovery of older narrative matters
-
-## SESSION.md Lifecycle Contract
-
-`SESSION.md` must follow this lifecycle:
-
-1. created automatically with the channel memory files
-2. not loaded wholesale into every prompt
-3. eligible for targeted recall injection
-4. updated in the background during normal work
-5. synchronously refreshed before context-reduction boundaries when possible
-6. carried across `/new` session boundaries within the same channel
-7. cleaned and condensed periodically so it remains current
-8. recreated automatically if missing on an old channel directory
-
-Important semantic rule:
-
-`/new` starts a new model session, but does not imply "forget current channel work". `SESSION.md` is allowed to survive `/new` if the channel still has active work.
-
-## Relationship Between SESSION.md And Existing Memory Files
-
-The relationship is:
-
-1. `SESSION.md` owns high-churn working state
-2. `MEMORY.md` owns durable and semi-durable channel knowledge
-3. `HISTORY.md` owns older narrative recovery
-
-Promotion rules:
-
-1. stable facts discovered in `SESSION.md` may be promoted into `MEMORY.md`
-2. resolved work periods may be summarized into `HISTORY.md`
-3. stale transient content should be removed from `SESSION.md`
-4. cleanup may also remove overly transient content that had historically accumulated in `MEMORY.md`
-
-This lets the system gradually migrate away from using channel `MEMORY.md` as the sole carrier of both durable and active state.
-
-## Source Of Truth Precedence
-
-When the same topic appears in multiple files, runtime and prompts should treat them in this precedence order:
-
-1. `SESSION.md` for current active work state
-2. `MEMORY.md` for durable constraints and decisions
-3. `HISTORY.md` for older chronology
-
-Practical implication:
-
-1. a stale `MEMORY.md` note must not override a fresher `SESSION.md` current-state section
-2. a stale `HISTORY.md` block must not override a fresher durable decision in `MEMORY.md`
-
-## Compaction Contract
-
-Compaction should become `SESSION.md`-aware.
-
-The preferred order is:
-
-1. refresh `SESSION.md`
-2. run inline durable consolidation into `MEMORY.md` and `HISTORY.md`
-3. compact using the latest `SESSION.md` as part of the retained context strategy
-
-Graceful degradation rules:
-
-1. if `SESSION.md` refresh fails, use the last persisted `SESSION.md`
-2. if durable consolidation fails, compaction may still proceed if `SESSION.md` is available
-3. failures must be logged and retried in background maintenance
-
-This keeps compaction safe without turning memory maintenance into a hard availability dependency.
-
-## Migration Contract
-
-Existing channel directories already containing only `MEMORY.md` and `HISTORY.md` must migrate safely.
-
-Migration rules:
-
-1. missing `SESSION.md` is treated as normal and repaired by ensure-file bootstrap
-2. existing `MEMORY.md` and `HISTORY.md` content remains authoritative
-3. no one-time destructive rewrite of historical `MEMORY.md` content is required
-4. transient content historically living in `MEMORY.md` is cleaned gradually by normal maintenance
-
-This keeps rollout incremental and low-risk for live DingTalk channels.
-
-## Sub-Agent Contract
-
-Sub-agents remain isolated by default, but the runtime gains a standard way to provide them with the right memory context.
-
-Two modes should exist:
-
-1. isolated
-   - current behavior
-   - only runtime basics plus the explicitly provided task
-2. contextual
-   - runtime prepends relevant memory and working-state context automatically
-
-The contextual mode should pull from the same recall pipeline as the main agent, but with stricter budgets.
-
-## Skill Contract
-
-Skills should become lightweight participants in the upgraded context system.
-
-Expected improvements:
-
-1. richer frontmatter describing when a skill should be used
-2. optional scoped hooks for session lifecycle events
-3. optional declaration of memory relevance or path relevance
-
-The skill system is still intentionally lighter than the `claude-code` version. The goal is stronger reuse and better context shaping, not a full plugin platform.
-
-## Rollout Principle
-
-This spec is designed for staged implementation:
-
-1. first improve recall
-2. then add `SESSION.md`
-3. then bridge compaction
-4. then upgrade sub-agent and skill integration
-
-At each stage, existing `MEMORY.md` and `HISTORY.md` behavior must continue to work.

package/docs/memory-rfc.md

@@ -1,297 +0,0 @@
-# Pipiclaw Memory Model RFC
-
-## Status
-
-Superseded in part by [`docs/improve-memory/spec.md`](/Users/oyasmi/projects/pipiclaw/docs/improve-memory/spec.md) and [`docs/improve-memory/design.md`](/Users/oyasmi/projects/pipiclaw/docs/improve-memory/design.md).
-
-This RFC remains useful for the original file-based memory rationale, but it no longer fully describes the current design because the runtime now introduces:
-
-1. `SESSION.md` as a distinct channel working-memory layer
-2. proactive relevant-memory injection
-3. a revised lifecycle between `SESSION.md`, `MEMORY.md`, and `HISTORY.md`
-
-## Goals
-
-1. Define a minimal memory model for `pipiclaw` aligned with the `nanobot` style.
-2. Make memory flow explicit: record, read, consolidate, and reuse.
-3. Keep raw transport/session files cold and separate from memory.
-4. Limit what is loaded into session context by default.
-5. Preserve simple file-based operations and avoid introducing a dedicated memory database or memory-specific tools.
-
-## Non-Goals
-
-1. No OpenClaw-style vector retrieval, embedding index, or memory plugins.
-2. No `memory_search` or `memory_get` tool.
-3. No automatic runtime mutation of workspace-level `MEMORY.md`, `SOUL.md`, or `AGENTS.md`.
-4. No automatic loading or scanning of `log.jsonl` or `context.jsonl` as part of memory behavior.
-
-## Design Summary
-
-Pipiclaw memory is split into two channel-level files plus one workspace-level admin file:
-
-- `workspace/MEMORY.md`
-  - Stable, admin-managed, shared background memory.
-  - Not automatically updated by runtime consolidation.
-- `<channel>/MEMORY.md`
-  - Durable channel memory.
-  - Updated automatically by consolidation.
-  - May also be updated manually by the agent.
-- `<channel>/HISTORY.md`
-  - Channel history summaries.
-  - Updated automatically by consolidation only.
-  - Not intended for direct manual maintenance by the agent.
-
-Raw storage remains separate:
-
-- `<channel>/log.jsonl`
-  - Raw message log only.
-  - Cold storage.
-  - Not proactively loaded or scanned by runtime.
-- `<channel>/context.jsonl`
-  - Raw session persistence only.
-  - Cold storage.
-  - Not proactively loaded or scanned by runtime for memory purposes.
-
-## File Model
-
-### Workspace-Level Files
-
-- `SOUL.md`
-  - Loaded into session context at session start.
-  - Read-only from the agent's perspective unless a human explicitly changes it.
-- `AGENTS.md`
-  - Loaded into session context at session start.
-  - Read-only from the agent's perspective unless a human explicitly changes it.
-- `MEMORY.md`
-  - Not loaded by default.
-  - Listed in the system prompt with its role and path.
-  - Intended to be read on demand by the agent.
-  - Stable and admin-managed.
-
-### Channel-Level Files
-
-- `MEMORY.md`
-  - Not loaded by default.
-  - Listed in the system prompt with its role and path.
-  - Intended to be read on demand by the agent.
-  - Primary durable memory for the channel.
-- `HISTORY.md`
-  - Not loaded by default.
-  - Listed in the system prompt with its role and path.
-  - Intended to be read on demand by the agent.
-  - Append-oriented summary history for older context.
-- `log.jsonl`
-  - Raw archive only.
-  - Not memory.
-  - Runtime must not proactively load or scan it.
-- `context.jsonl`
-  - Raw session archive only.
-  - Not memory.
-  - Runtime must not proactively load or scan it.
-
-## Default Context Loading
-
-At session start, runtime loads the following into session context:
-
-1. Workspace-level `SOUL.md`
-2. Workspace-level `AGENTS.md`
-3. Built-in tool descriptions
-4. Summaries of both workspace-level and channel-level skills
-
-At session start, runtime does not load the following by default:
-
-1. Workspace-level `MEMORY.md`
-2. Channel-level `MEMORY.md`
-3. Channel-level `HISTORY.md`
-4. `<channel>/log.jsonl`
-5. `<channel>/context.jsonl`
-
-The system prompt must explicitly tell the agent:
-
-1. Where `workspace/MEMORY.md`, `<channel>/MEMORY.md`, and `<channel>/HISTORY.md` live.
-2. What each file is for.
-3. That these files are not preloaded.
-4. That the agent is encouraged to read them on demand when memory or history is relevant.
-
-Changes to workspace-level `SOUL.md` and `AGENTS.md` take effect on new sessions. They are not reloaded on every turn.
-
-## Agent Read/Write Rules
-
-### Reads
-
-The agent should prefer:
-
-1. `<channel>/MEMORY.md` for durable channel facts, decisions, preferences, and ongoing state.
-2. `<channel>/HISTORY.md` for older summarized context.
-3. `workspace/MEMORY.md` for stable shared background.
-
-The agent should not treat `log.jsonl` or `context.jsonl` as normal memory sources. Those files are for raw recovery, debugging, or explicit transcript-level investigation only.
-
-### Writes
-
-The agent may:
-
-1. Read `workspace/MEMORY.md`
-2. Read `<channel>/MEMORY.md`
-3. Read `<channel>/HISTORY.md`
-4. Manually update `<channel>/MEMORY.md` when necessary
-
-The agent should not manually maintain `<channel>/HISTORY.md` during normal operation. `HISTORY.md` is runtime-managed.
-
-The runtime must never automatically update:
-
-1. `workspace/SOUL.md`
-2. `workspace/AGENTS.md`
-3. `workspace/MEMORY.md`
-
-## Consolidation
-
-Consolidation is the primary mechanism that keeps memory moving.
-
-Consolidation takes recent session material and produces two outputs:
-
-1. Durable facts and live channel state in `<channel>/MEMORY.md`
-2. Older summarized history in `<channel>/HISTORY.md`
-
-Consolidation must not write to workspace-level `MEMORY.md`.
-
-### Trigger Model
-
-Runtime should run consolidation automatically only when a session is about to compact or trim context.
-
-There is no separate turn-count trigger, token-watermark trigger, or per-turn memory-dirty trigger in this RFC.
-
-### Consolidation Input
-
-Consolidation operates on recent session material already in the active session state. It does not scan `log.jsonl` or `context.jsonl`.
-
-### Consolidation Output Rules
-
-`<channel>/MEMORY.md` should contain:
-
-1. Durable facts about the channel, user, project, preferences, constraints, and long-lived open threads.
-2. Current state that matters across turns.
-3. Cleaned and deduplicated entries.
-4. Structured sections rather than free-form prose.
-
-`<channel>/HISTORY.md` should contain:
-
-1. Append-oriented summaries of older conversation chunks.
-2. Resolved decisions and notable milestones.
-3. Enough context for later recovery without replaying raw transcripts.
-
-### Consolidation Semantics
-
-Consolidation should:
-
-1. Extract durable facts from recent session material.
-2. Append new channel-memory entries into `<channel>/MEMORY.md` during normal operation.
-3. Periodically clean up `<channel>/MEMORY.md` with a larger sweep that removes outdated entries, merges duplicates, and tightens wording.
-4. Append or roll forward summarized older material into `<channel>/HISTORY.md`.
-5. Periodically fold older `HISTORY.md` blocks into coarser summaries while keeping newer blocks more detailed.
-6. Remove redundancy between the two files.
-7. Prefer stable facts in `MEMORY.md` and narrative progression in `HISTORY.md`.
-
-This RFC adopts an append-first strategy for `MEMORY.md`, with periodic cleanup passes rather than full rewrite on every consolidation.
-
-Consolidation should not:
-
-1. Dump raw message transcripts into `HISTORY.md`
-2. Copy large blocks from `context.jsonl` or `log.jsonl`
-3. Preserve outdated facts just because they were once true
-
-## Suggested File Semantics
-
-`<channel>/MEMORY.md` should be organized as stable sections such as:
-
-1. `## Identity / Participants`
-2. `## Preferences`
-3. `## Ongoing Work`
-4. `## Constraints`
-5. `## Decisions`
-6. `## Open Loops`
-
-`<channel>/HISTORY.md` should be organized as chronological summary blocks such as:
-
-1. Dated headings
-2. Short summaries of work periods
-3. Key decisions and outcomes
-
-Exact formatting can evolve, but the split between durable memory and summarized history should remain stable.
-
-## Consolidation Execution Model
-
-Consolidation uses a two-phase execution model:
-
-1. Inline phase
-   - Runs synchronously when a compaction or trim is about to happen.
-   - Responsible for producing the minimum safe memory updates needed before context is reduced.
-2. Background phase
-   - Runs later as a per-channel queued maintenance pass.
-   - Responsible for larger cleanup work, including `MEMORY.md` cleanup sweeps and `HISTORY.md` folding.
-
-The background phase must not replace the inline phase for compaction safety. It is an additional maintenance pass, not the primary durability guarantee.
-
-## Compaction And Trimming Contract
-
-Consolidation is hard-gated before compaction or trim.
-
-This means:
-
-1. If a compaction or trim would discard context, runtime must first run inline consolidation.
-2. If inline consolidation fails, compaction or trim must not proceed.
-3. After successful inline consolidation, the runtime may compact or trim.
-
-This RFC does not allow trim-first or compact-first behavior when memory durability depends on consolidation.
-
-## Runtime Responsibilities
-
-Runtime is responsible for:
-
-1. Loading only the default context described in this RFC.
-2. Exposing file locations and roles clearly in the system prompt.
-3. Running automatic consolidation.
-4. Updating `<channel>/MEMORY.md` and `<channel>/HISTORY.md` during consolidation.
-5. Keeping `log.jsonl` and `context.jsonl` cold.
-
-Runtime is not responsible for:
-
-1. Auto-loading memory files into every turn
-2. Indexing memory into a vector database
-3. Providing special memory-only tools
-
-## Session Model
-
-Session context is warm.
-
-Memory files are warm-on-demand.
-
-`log.jsonl` and `context.jsonl` are cold.
-
-The intended flow is:
-
-1. Session starts with `SOUL.md`, `AGENTS.md`, built-in tool descriptions, and skill summaries.
-2. Agent reads channel or workspace memory files if the task needs them.
-3. Conversation progresses in session state.
-4. Runtime consolidates session material into `<channel>/MEMORY.md` and `<channel>/HISTORY.md`.
-5. Older raw storage remains available but is not part of ordinary memory behavior.
-
-## Migration Impact
-
-This RFC implies the following behavior changes from the current implementation:
-
-1. `workspace/MEMORY.md` and `<channel>/MEMORY.md` are no longer injected into the system prompt by default.
-2. `SOUL.md` and `AGENTS.md` are no longer reloaded on every turn.
-3. `log.jsonl` is no longer scanned to rebuild context as part of normal turn processing.
-4. `context.jsonl` is no longer treated as a memory source.
-5. Consolidation becomes the main persistence path for channel memory.
-
-## Fixed Decisions
-
-This RFC fixes the following design choices:
-
-1. Consolidation triggers only before compaction or trim.
-2. `<channel>/MEMORY.md` uses append-first updates with periodic cleanup sweeps.
-3. `<channel>/HISTORY.md` uses append-first updates with periodic folding of older blocks.
-4. Consolidation uses a two-phase execution model: inline for safety, background for maintenance.
-5. Consolidation is hard-gated before compaction or trim.