tachibot-mcp 2.0.2

package/docs/WORKFLOW_OUTPUT.md

@@ -0,0 +1,198 @@
+# Workflow Output Configuration
+
+Control how workflow step outputs are displayed.
+
+## Quick Reference
+
+### YAML Configuration
+```yaml
+output:
+  format: detailed
+  truncateSteps: false  # Show full output (no truncation)
+  maxStepTokens: 2500   # Only applies if truncateSteps=true
+```
+
+### Runtime Override
+```bash
+# Use workflow defaults
+workflow --name code-review --query "..."
+
+# Show full output (override workflow settings)
+workflow --name code-review --query "..." --truncateSteps false
+
+# Custom token limit
+workflow --name code-review --query "..." --maxStepTokens 5000
+```
+
+---
+
+## Configuration Options
+
+### `truncateSteps` (boolean)
+**Default:** `true`
+
+Controls whether step outputs are truncated.
+
+- `true` - Truncate at `maxStepTokens` limit (better readability)
+- `false` - Show complete output (no truncation, full context)
+
+**When to use false:**
+- Comprehensive analysis workflows (code review, UX research)
+- Debugging workflow outputs
+- When you need full context from each step
+
+**When to use true:**
+- Quick iterative development
+- When outputs are very large (>10k tokens)
+- To reduce visual clutter in terminal
+
+### `maxStepTokens` (number)
+**Default:** `2500` tokens (~10,000 characters)
+
+Maximum tokens per step output when `truncateSteps=true`.
+
+**Token Reference:**
+- 500 tokens ≈ 2,000 chars ≈ 1 paragraph
+- 2,500 tokens ≈ 10,000 chars ≈ 2-3 pages (default)
+- 5,000 tokens ≈ 20,000 chars ≈ 5-6 pages
+- 10,000 tokens ≈ 40,000 chars ≈ 10-12 pages
+
+**Recommended settings by workflow type:**
+
+| Workflow Type | truncateSteps | maxStepTokens | Reason |
+|--------------|---------------|---------------|---------|
+| Quick brainstorm | `true` | `1000` | Fast iteration |
+| Code review | `false` | - | Need full analysis |
+| Research | `false` | - | Comprehensive findings |
+| UX analysis | `false` | - | Full user journey |
+| Debug workflow | `true` | `5000` | Balance detail/readability |
+| Multi-step pipeline | `true` | `2500` | Standard default |
+
+---
+
+## Examples
+
+### Example 1: Code Review (Show Everything)
+```yaml
+name: comprehensive-code-review
+output:
+  format: detailed
+  truncateSteps: false  # Need full review details
+```
+
+**Result:** All step outputs shown in full - analysis, architecture review, security findings, etc.
+
+### Example 2: Research Pipeline (Custom Limit)
+```yaml
+name: research-pipeline
+output:
+  format: detailed
+  truncateSteps: true
+  maxStepTokens: 5000  # Larger limit for research
+```
+
+**Result:** Each step truncated at ~5000 tokens (~20k characters), good balance for research workflows.
+
+### Example 3: Quick Brainstorm (Default)
+```yaml
+name: quick-brainstorm
+output:
+  format: detailed
+  # Uses defaults: truncateSteps=true, maxStepTokens=2500
+```
+
+**Result:** Concise output, easier to scan quickly.
+
+### Example 4: Runtime Override
+```bash
+# Workflow has truncateSteps=true, but override to see full output
+workflow --name quick-brainstorm --query "..." --truncateSteps false
+
+# Use different token limit
+workflow --name research --query "..." --maxStepTokens 10000
+```
+
+---
+
+## Understanding Truncation Messages
+
+When output is truncated, you'll see:
+```
+...(output truncated: ~3420 tokens, limit: 2500 tokens. Set truncateSteps=false for full output)...
+```
+
+This tells you:
+- **Actual size:** ~3420 tokens
+- **Limit applied:** 2500 tokens
+- **How to see full output:** Set `truncateSteps=false`
+
+---
+
+## Migration from Old API
+
+### ❌ Old (deprecated)
+```yaml
+output:
+  maxStepOutputLength: -1  # Magic number, unclear
+  maxStepOutputLength: 10000  # Characters, hard to estimate
+```
+
+### ✅ New (recommended)
+```yaml
+output:
+  truncateSteps: false  # Clear intent
+  maxStepTokens: 2500   # Meaningful for AI context
+```
+
+---
+
+## Best Practices
+
+1. **Start with defaults** (`truncateSteps=true`, `maxStepTokens=2500`)
+2. **Set `truncateSteps=false` for:**
+   - Complex analysis workflows
+   - When step outputs feed into each other
+   - Production-critical workflows
+3. **Increase `maxStepTokens` for:**
+   - Research-heavy workflows
+   - Multi-perspective analysis
+   - Detailed technical reports
+4. **Use runtime overrides** when testing:
+   - `--truncateSteps false` to debug
+   - `--maxStepTokens 10000` for one-off deep dives
+
+---
+
+## Token vs Character Conversion
+
+Internally, the system converts tokens to approximate characters using:
+- **1 token ≈ 4 characters** (for English text)
+
+| Tokens | Characters | Typical Content |
+|--------|-----------|-----------------|
+| 500 | ~2,000 | Short summary |
+| 1,000 | ~4,000 | 1 page report |
+| 2,500 | ~10,000 | 2-3 pages (default) |
+| 5,000 | ~20,000 | 5-6 pages |
+| 10,000 | ~40,000 | 10-12 pages |
+
+For code, JSON, or special characters, the ratio may vary (typically 2-3 chars/token).
+
+---
+
+## FAQ
+
+### Why tokens instead of characters?
+Tokens are more meaningful for AI systems and give better estimates of context window usage.
+
+### Can I set different limits per step?
+Not yet, but it's on the roadmap. Current settings apply to all steps in a workflow.
+
+### Does this affect model output?
+No! This only controls **display** truncation. Model responses are controlled by `maxTokens` in the step configuration.
+
+### What if I want unlimited output?
+Set `truncateSteps: false`
+
+### Can runtime parameters override workflow settings?
+Yes! Runtime parameters take precedence over workflow YAML settings.

package/docs/WORKFLOW_PROGRESS_TRACKING.md

@@ -0,0 +1,305 @@
+# Workflow Progress Tracking - Live Session Logging
+
+## Problem Solved
+
+Long-running workflows (5-10 minutes with 7+ steps) previously showed no progress until completion. Users saw only "Tool ran without output or errors" during execution.
+
+## Solution: SessionLogger Integration
+
+We integrated the existing `SessionLogger` class into the workflow system to provide:
+- **Real-time progress files** - Monitor workflow execution live
+- **Persistent session logs** - Full markdown reports saved after completion
+- **Step-by-step tracking** - Each step logs immediately after completion
+- **Verbose console output** - Detailed progress in server logs
+
+## How It Works
+
+### 1. Automatic Session Logging
+
+When you run any workflow, the system now:
+
+1. **Creates a session** at workflow start:
+   ```
+   📝 Session log: .workflow-sessions/session-2025-10-18-02-15-30-workflow-creative-brainstorm-yaml.md
+      To monitor progress: tail -f .workflow-sessions/session-2025-10-18-02-15-30-workflow-creative-brainstorm-yaml.md
+   ```
+
+2. **Logs each step** immediately after completion:
+   - Step number and name
+   - Tool used
+   - Full input and output
+   - Execution time
+   - Metadata (workflow name, progress)
+
+3. **Saves final report** when workflow completes:
+   ```
+   ✅ Workflow complete! Full session saved to: .workflow-sessions/session-2025-10-18-02-15-30-workflow-creative-brainstorm-yaml.md
+   ```
+
+### 2. Session File Format
+
+Each session creates a markdown file with:
+
+```markdown
+# Focus MCP workflow-creative-brainstorm-yaml Session
+
+## Metadata
+- **Session ID**: session-2025-10-18-02-15-30-workflow-creative-brainstorm-yaml
+- **Date**: 2025-10-18T02:15:30.000Z
+- **Mode**: workflow-creative-brainstorm-yaml
+- **Models Used**: gemini_brainstorm (workflow), openai_brainstorm (workflow), ...
+- **Total Duration**: 285.3s
+- **Query**: "Analysis of personality/komaai-expressions.ts..."
+
+---
+
+## Step 1: Step 1/7
+
+**Model**: gemini_brainstorm
+**Provider**: workflow
+**Duration**: 12.5s
+
+### Prompt:
+```
+Generate creative ideas for: Analysis of personality/komaai-expressions.ts...
+```
+
+### Response:
+[Full step output here...]
+
+---
+
+## Step 2: Step 2/7
+...
+```
+
+### 3. Monitoring Live Progress
+
+**Option 1: Tail the session file** (recommended)
+```bash
+# Get the session file path from console output, then:
+tail -f .workflow-sessions/session-*.md
+```
+
+**Option 2: Watch the directory**
+```bash
+watch -n 2 'ls -lth .workflow-sessions | head -10'
+```
+
+**Option 3: Server console logs**
+The server stderr shows verbose progress:
+```
+🚀 STARTING STEP 1/7: initial-ideas (gemini_brainstorm)
+...
+✅ STEP 1/7 COMPLETE: initial-ideas
+[Step output displayed]
+
+📍 Step 1: STEP 1/7
+Model: gemini_brainstorm (workflow)
+Duration: 12500ms
+🤖 Response:
+[Output preview...]
+```
+
+## Usage Examples
+
+### Running a Workflow with Progress Tracking
+
+```typescript
+// Via MCP tool
+mcp__tachibot-mcp__workflow({
+  name: "creative-brainstorm-yaml",
+  query: "Your topic here",
+  truncateSteps: false,  // Get full output
+  maxStepTokens: 5000    // Control detail level
+})
+```
+
+While it runs, in another terminal:
+```bash
+# Monitor progress live
+tail -f .workflow-sessions/session-*.md
+
+# Or watch for updates
+watch -n 1 'tail -20 .workflow-sessions/session-*.md'
+```
+
+### Session File Location
+
+- **Directory**: `./.workflow-sessions/` (in project root)
+- **Format**: `session-YYYY-MM-DD-HH-MM-SS-workflow-{name}.md`
+- **Auto-save**: After every step completion
+
+## Configuration
+
+The SessionLogger is configured in `src/workflows/custom-workflows.ts`:
+
+```typescript
+this.sessionLogger = new SessionLogger({
+  saveSession: true,      // Save to file
+  verbose: true,          // Console output
+  autoSave: true,         // Save after each step
+  outputFormat: "markdown",
+  sessionDir: "./.workflow-sessions",
+});
+```
+
+### Customization Options
+
+You can modify settings in the constructor:
+
+- `saveSession: boolean` - Enable/disable file saving
+- `verbose: boolean` - Show detailed console logs
+- `autoSave: boolean` - Save after each step (vs. only at end)
+- `outputFormat: "markdown" | "json" | "html"` - Output format
+- `sessionDir: string` - Where to save sessions
+
+## File Management
+
+### Viewing Past Sessions
+
+```bash
+# List all sessions
+ls -lt .workflow-sessions/
+
+# Search by workflow name
+ls .workflow-sessions/*creative-brainstorm*
+
+# View a session
+cat .workflow-sessions/session-2025-10-18-02-15-30-workflow-creative-brainstorm-yaml.md
+```
+
+### Cleaning Old Sessions
+
+The SessionLogger includes a cleanup utility:
+
+```typescript
+// In code (if needed):
+await sessionLogger.clearOldSessions(30); // Keep last 30 days
+```
+
+Or manually:
+```bash
+# Remove sessions older than 7 days
+find .workflow-sessions -name "*.md" -mtime +7 -delete
+```
+
+## Benefits
+
+1. **No More Black Box** - See exactly what's happening at each step
+2. **Debugging** - Full logs for troubleshooting workflow issues
+3. **Monitoring** - Watch progress in real-time with `tail -f`
+4. **Persistence** - Complete session history for review
+5. **Transparency** - See input/output of every step
+
+## Architecture
+
+### Code Changes Made
+
+1. **`src/workflows/custom-workflows.ts`**:
+   - Added `SessionLogger` import
+   - Created `sessionLogger` instance in constructor
+   - Added `startSession()` at workflow start
+   - Added `logStep()` after each step completion
+   - Added `endSession()` on completion/error
+
+2. **Session Flow**:
+   ```
+   Workflow Start
+       ↓
+   sessionLogger.startSession() → Creates .md file
+       ↓
+   For each step:
+       Execute step
+       ↓
+       sessionLogger.logStep() → Appends to .md file
+       ↓
+   All steps complete
+       ↓
+   sessionLogger.endSession() → Saves final summary
+   ```
+
+## Next Steps After Restart
+
+1. **Restart Claude Code** - Pick up the new server with SessionLogger
+2. **Run a test workflow**:
+   ```
+   workflow test-brainstorm-mini "test topic"
+   ```
+3. **Open a new terminal** and run:
+   ```bash
+   tail -f .workflow-sessions/session-*.md
+   ```
+4. **Watch the magic** - See steps appear in real-time!
+
+## Advanced: Programmatic Access
+
+The SessionLogger can be used programmatically:
+
+```typescript
+import { SessionLogger } from "./session/session-logger.js";
+
+const logger = new SessionLogger({
+  saveSession: true,
+  verbose: true,
+  sessionDir: "./my-sessions"
+});
+
+// Start session
+const sessionId = await logger.startSession("my-mode", "my query");
+
+// Log steps
+await logger.logStep(
+  "gpt-5",
+  "openai",
+  "brainstorm",
+  "Generate ideas about...",
+  "Here are 5 ideas...",
+  { custom: "metadata" }
+);
+
+// End and save
+await logger.endSession(true);
+```
+
+## Troubleshooting
+
+### Session Directory Not Created
+
+If `.workflow-sessions/` doesn't exist:
+```bash
+# Check permissions
+ls -la . | grep workflow-sessions
+
+# Create manually if needed
+mkdir -p .workflow-sessions
+chmod 755 .workflow-sessions
+```
+
+### No Output During Workflow
+
+1. Check server logs: `tail -f /tmp/server-debug.log`
+2. Verify rebuild: `npm run build`
+3. Restart server: Kill old processes and reconnect MCP
+
+### Verbose Mode Too Noisy
+
+Set `verbose: false` in SessionLogger constructor to reduce console output while keeping file logging.
+
+## Summary
+
+**Before**: Workflows run for minutes with no feedback
+**After**: Live progress files + verbose logs + persistent session history
+
+**Key Command**:
+```bash
+tail -f .workflow-sessions/session-*.md  # Watch your workflow run!
+```
+
+---
+
+**Implementation Status**: ✅ Complete
+**Testing Status**: ⏳ Pending server restart
+**Documentation**: ✅ This file
+
+Run `npm run build` and restart Claude Code to enable this feature!