omnius 1.0.591 → 1.0.592

package/docs/session-diary-llm-training-analysis.md

@@ -0,0 +1,218 @@
+# Session Diary: LLM Training Alignment Analysis
+
+## Question
+
+Is the session diary format what LLMs are trained on, such that it helps system handling?
+
+## Answer: Partially
+
+### What LLMs ARE Trained On (Matches)
+
+| Session Diary Feature | LLM Training Data Match |
+|-----------------------|-------------------------|
+| Markdown structure | ✓ Common in docs, READMEs, wikis |
+| Timestamp headers | ✓ Git commits, changelogs |
+| Task descriptions | ✓ GitHub issues, Stack Overflow |
+| File lists | ✓ PR descriptions, commit messages |
+| Summary fields | ✓ Documentation patterns |
+
+### What LLMs Are NOT Trained On (Gaps)
+
+| Session Diary Feature | Why It's Novel |
+|-----------------------|----------------|
+| `toolCalls: number` | LLMs don't see tool call counts in training |
+| `toolsUsed: string[]` | Tool names are model-specific, not universal |
+| `completed: boolean` | Binary status is unusual in natural text |
+| `source: "task_complete"` | Internal state tracking, not user-facing |
+| `_dedupeHash` | Implementation detail, not semantic |
+
+## Current Schema (from `omnius-directory.ts:537-564`)
+
+```typescript
+interface SessionContextEntry {
+  savedAt: string;        // ISO timestamp
+  task: string;           // What task was being done
+  summary: string;        // Agent's summary
+  filesModified: string[]; // Files touched
+  toolCalls: number;      // Tool calls completed
+  toolsUsed?: string[];   // Tool names used
+  provenance?: string;    // Path to provenance file
+  assistantResponse?: string; // Visible reply
+  source?: "task_complete" | "manual" | "api";
+  completed: boolean;     // Success status
+  model: string;          // Model used
+  sessionId?: string;     // Link related entries
+  _dedupeHash?: string;   // Fast deduplication
+}
+```
+
+## What Would Make It More LLM-Friendly
+
+### 1. Add Explicit Outcome Field
+
+Current: `completed: boolean` (binary)
+Better: `outcome: "success" | "partial" | "failed" | "blocked"`
+
+LLMs understand these natural language outcomes better than booleans.
+
+### 2. Add Decision Log
+
+```typescript
+decisions?: Array<{
+  decision: string;    // What was decided
+  reason: string;      // Why
+  alternatives?: string[]; // What was rejected
+}>
+```
+
+This captures reasoning that LLMs can learn from.
+
+### 3. Add Blockers Field
+
+```typescript
+blockers?: Array<{
+  type: "dependency" | "error" | "ambiguity" | "resource";
+  description: string;
+  resolution?: string;
+}>
+```
+
+LLMs benefit from seeing what blocked progress and how it was resolved.
+
+### 4. Compress Summaries
+
+Current: 280 char free-form text
+Better: Structured one-liner
+
+```typescript
+summary: {
+  outcome: string;      // One sentence result
+  keyChange: string;    // Most important change
+  nextStep?: string;    // What's left to do
+}
+```
+
+### 5. Add Token Efficiency Metrics
+
+```typescript
+metrics?: {
+  inputTokens: number;
+  outputTokens: number;
+  toolCallTokens: number;
+  contextWindowUsed: number; // percentage
+}
+```
+
+This helps LLMs understand their own efficiency patterns.
+
+## Proposed Enhanced Schema
+
+```typescript
+interface SessionContextEntryV2 {
+  // Core (unchanged)
+  savedAt: string;
+  task: string;
+  model: string;
+  
+  // Enhanced outcome
+  outcome: "success" | "partial" | "failed" | "blocked";
+  summary: {
+    result: string;      // ≤100 chars
+    keyChange: string;   // ≤80 chars
+    nextStep?: string;   // ≤80 chars
+  };
+  
+  // Files (unchanged)
+  filesModified: string[];
+  
+  // Tools (enhanced)
+  tools: {
+    names: string[];     // Tool names used
+    calls: number;       // Total calls
+    failed?: number;     // Failed calls
+    retries?: number;    // Retry count
+  };
+  
+  // NEW: Decision log
+  decisions?: Array<{
+    at: string;          // When decided
+    choice: string;      // What was chosen
+    reason: string;      // Why
+    rejected?: string[]; // Alternatives not taken
+  }>;
+  
+  // NEW: Blockers
+  blockers?: Array<{
+    type: "dependency" | "error" | "ambiguity" | "resource";
+    what: string;        // What blocked
+    resolved: boolean;   // Was it resolved?
+    how?: string;        // How (if resolved)
+  }>;
+  
+  // NEW: Efficiency metrics
+  metrics?: {
+    inputTokens: number;
+    outputTokens: number;
+    contextWindowPct: number;
+    durationMs: number;
+  };
+  
+  // Session linkage (unchanged)
+  sessionId?: string;
+  source?: "task_complete" | "manual" | "api";
+}
+```
+
+## Rendered Example (Current vs Proposed)
+
+### Current Format
+
+```markdown
+## 2026-04-22 07:44 ✓ what can this help us to do plantUML first, dont reinvent here
+
+- **Prompt:** what can this help us to do plantUML first, dont reinvent here
+- **Model:** zai-org/GLM-5-Turbo
+- **Tools:** 5 calls
+- **Summary:** PlantUML provides 40+ diagram types that can visualize CRL: (1) Math notation for logic symbols (⊤⊥⊢⊨∀∃), (2) Class/Object diagrams for concept nodes...
+```
+
+### Proposed Format (More LLM-parseable)
+
+```markdown
+## 2026-04-22 07:44 ✓ plantUML for CRL visualization
+
+- **Task:** what can this help us to do plantUML first, dont reinvent here
+- **Outcome:** success
+- **Result:** PlantUML can visualize CRL with 40+ diagram types
+- **KeyChange:** Identified math notation + class diagrams for concept nodes
+- **Tools:** web_search, web_fetch, task_complete (5 calls)
+- **Model:** zai-org/GLM-5-Turbo
+- **Duration:** 55s, 5 turns
+```
+
+## Implementation Path
+
+1. **Backward Compatible**: Add new fields as optional, keep old fields
+2. **Gradual Migration**: New sessions use V2 format, old sessions stay V1
+3. **Renderer Update**: `renderSessionDiaryV2()` with structured output
+4. **Context Injection**: Use V2 format in system prompt context
+
+## Research Basis
+
+- **Chain-of-Thought Prompting** (Wei et al., 2022): Structured reasoning improves LLM performance
+- **Self-Consistency** (Wang et al., 2022): Multiple reasoning paths help accuracy
+- **Constitutional AI** (Anthropic, 2022): Explicit principles guide behavior
+- **Tool Learning** (Qin et al., 2023): Tool-use patterns benefit from structured logging
+
+## Conclusion
+
+The session diary format is **partially aligned** with LLM training data. The markdown structure and task descriptions match what LLMs see. However, the internal state tracking (tool counts, dedupe hashes) is novel.
+
+**Key improvements for LLM-friendliness:**
+1. Replace `completed: boolean` with `outcome: "success" | "partial" | "failed" | "blocked"`
+2. Add structured `decisions` log
+3. Add `blockers` field
+4. Compress summaries to structured one-liners
+5. Add efficiency metrics
+
+These changes would make the session diary more parseable by LLMs and more useful for cross-session learning.

package/docs/telegram-dmn-curiosity-outreach-scaffold.md

@@ -0,0 +1,91 @@
+# Telegram DMN Curiosity Outreach Scaffold
+
+Date: 2026-05-15
+
+This document tracks the root design for idle Telegram channel "daydream" behavior:
+private meta-analysis while a public group is idle, curiosity-driven exploration, scoped
+tool awareness, same-group follow-up planning, private DM follow-up planning, and persona
+document steering. It is intentionally a scaffold: artifacts describe possible actions and
+tool affordances, while the live agent loop still decides whether to act in the current
+conversation state.
+
+## Research Anchors
+
+- Default mode network (DMN): idle or low-demand intervals are useful for memory-linked,
+  abstract, and socially grounded cognition. Smallwood et al. describe DMN regions as
+  increasing activity across complex cognition tied to memory and abstract thought, not
+  just "rest." Source: https://www.nature.com/articles/s41583-021-00474-4
+- Curiosity and memory: curiosity is not just novelty seeking; it biases learning and
+  memory through reward/hippocampal circuits. Gruber, Gelman, and Ranganath's Neuron work
+  anchors the curiosity-to-memory mapping used here. Source:
+  https://escholarship.org/uc/item/2zd605r7
+- Context engineering: modern agent reliability depends on context retrieval, generation,
+  processing, and management, not just prompt text. The 2025 survey frames memory,
+  tool-integrated reasoning, RAG, and multi-agent systems as integrated context systems.
+  Source: https://arxiv.org/abs/2507.13334
+- Agentic memory, 2026: AgeMem exposes memory operations as tool-based actions so the
+  agent can decide when to store, retrieve, update, summarize, or discard information.
+  This supports making "remember this person/face/voice/thread" an agentic operation
+  rather than regex routing. Source: https://arxiv.org/abs/2601.01885
+- Autonomous agent memory survey, 2026: memory should be a write-manage-read loop coupled
+  with perception and action, with attention to contradiction handling, privacy,
+  consolidation, latency, forgetting, and multimodal memory. Source:
+  https://arxiv.org/abs/2603.07670
+- Agentic context engineering, ICLR 2026: evolving playbooks should preserve detail via
+  structured, incremental reflection and curation to avoid context collapse. Source:
+  https://arxiv.org/abs/2510.04618
+- Telegram primitives: `sendMessage` supports 1-4096 visible characters plus
+  `reply_parameters`; `sendChatAction` is meant only for noticeable work in progress.
+  The scaffold should keep meta-analysis private and make reply intent explicit when
+  visible sends happen. Source: https://core.telegram.org/bots/api#sendmessage
+
+## Design Translation
+
+- DMN behavior becomes an artifact write during idle group periods, not a public message.
+- Human stimulation becomes structured input: direct address, reply chains, open
+  questions, multimodal content, repeated participants, tone pressure, and uncertainty.
+- Curiosity becomes a queue of thread candidates with source message anchors and
+  intensity. The queue is evidence for the model, not a hard trigger.
+- Outreach is represented as plans:
+  - `same_group`: a public follow-up candidate, gate `model_decision`.
+  - `private_dm`: a one-to-one candidate, currently gate `admin_review` because private
+    contact permissions and anti-spam policy need explicit configuration before delivery.
+- Persona steering is imported from the scoped personality document and included in the
+  artifact so the follow-up planner sees the room's behavioral history.
+- Tooling is scoped to the Telegram chat id and surfaced as a suite: memory search/write,
+  identity memory, recent Telegram media, vision/OCR/video, audio/transcription, and
+  public web research.
+
+## Code Anchors
+
+- `packages/cli/src/tui/telegram-channel-dmn.ts`
+  - `TelegramChannelDaydreamArtifact.version = 2`
+  - `metaAnalysisSignals`
+  - `humanStimulationSignals`
+  - `curiosityThreads`
+  - `scopedExplorationTools`
+  - `outreachPlans`
+  - `personaSteering`
+  - `formatTelegramChannelDaydreamContext`
+- `packages/cli/src/tui/telegram-bridge.ts`
+  - `buildTelegramChannelDaydreamInput` now injects scoped personality context into
+    idle artifacts.
+- `packages/cli/tests/telegram-bot-api-10.test.ts`
+  - regression coverage for artifact injection into live Telegram context.
+
+## Integration Checklist
+
+- [x] Add human stimulation fields to idle channel artifacts.
+- [x] Add curiosity thread extraction with source message anchors.
+- [x] Add scoped exploration tool suite to the artifact.
+- [x] Add same-group and private-DM outreach plan structures.
+- [x] Add explicit context-management/meta-analysis signals.
+- [x] Add scoped persona steering into the artifact.
+- [x] Keep actual public/DM delivery behind model/policy gates.
+- [ ] Add runtime configuration for enabling/disabling same-group proactive follow-up.
+- [ ] Add runtime configuration and consent checks for private DM proactive follow-up.
+- [ ] Add a model-evaluated DMN action pass that consumes the artifact and chooses:
+  stay silent, research privately, consolidate memory, reply in group, or request DM
+  permission.
+- [ ] Add delivery quotas and cooldowns per chat, per user, and per topic.
+- [ ] Add UI/TUI rendering for artifact summaries and source-message backreferences.