ace-handbook 0.19.0

data/handbook/guides/multi-agent-research.g.md

@@ -0,0 +1,251 @@
+---
+doc-type: guide
+title: Multi-Agent Research Guide
+purpose: Documentation for ace-handbook/handbook/guides/multi-agent-research.g.md
+ace-docs:
+  last-updated: 2026-02-22
+  last-checked: 2026-03-21
+---
+
+# Multi-Agent Research Guide
+
+This guide explains when and how to leverage multiple AI agents for research tasks, combining their diverse capabilities to produce higher-quality, more comprehensive results.
+
+## Purpose
+
+Multi-agent research uses parallel execution across different AI models followed by structured synthesis to:
+- Reduce individual model blind spots through cross-validation
+- Surface diverse perspectives and approaches
+- Increase confidence in findings through consensus
+- Extend effective context through task division
+
+## When to Use Multi-Agent Research
+
+### Good Use Cases
+
+| Scenario | Why Multi-Agent Helps |
+|----------|----------------------|
+| Complex research topics | Different models have different knowledge and reasoning styles |
+| High-stakes decisions | Cross-validation reduces hallucinations and errors |
+| Novel domains | Diverse perspectives surface more options |
+| Standard-setting | Consensus building across multiple viewpoints |
+| Exploratory research | Broader coverage of solution space |
+
+### When NOT to Use
+
+| Scenario | Why Single Agent is Better |
+|----------|---------------------------|
+| Simple, well-defined tasks | Overhead not justified |
+| Time-critical work | Parallel + synthesis adds latency |
+| Cost-sensitive projects | 3x+ token usage |
+| Clear single source of truth | No benefit from diversity |
+| Implementation tasks | Execution benefits from consistency |
+
+### Decision Criteria
+
+Ask yourself:
+1. **Is the task primarily research or implementation?** Multi-agent works best for research.
+2. **Would multiple perspectives add value?** If one answer is clearly correct, single agent suffices.
+3. **Is the cost justified?** Multi-agent typically costs 3-5x a single-agent approach.
+4. **Do you have time for synthesis?** Plan for 20-30% additional effort for combination.
+
+## Agent Selection Criteria
+
+### Quality Over Diversity
+
+Research shows that mixing high-quality agents produces better results than adding lower-quality agents for diversity. The key finding:
+
+> "Diversity in MoA proposers might have an adverse effect... MoA performance is sensitive to the quality of the models being mixed."
+> — Princeton Research on Mixture-of-Agents
+
+### Selection Principles
+
+1. **Choose capable models**: Each agent should be able to complete the task solo
+2. **Prefer complementary strengths**: Agents with different training data or specializations
+3. **Avoid weak links**: One poor agent can degrade overall synthesis quality
+4. **Consider context limits**: Match agent capabilities to task requirements
+
+### Recommended Configurations
+
+| Research Type | Suggested Agents | Rationale |
+|---------------|------------------|-----------|
+| Code analysis | Claude, Codex, Gemini | Strong code understanding |
+| Technical docs | Claude, Gemini | Strong reasoning and knowledge |
+| Architecture | Claude, Gemini, GPT-4 | Diverse design perspectives |
+| Security review | Multiple specialized models | Defense in depth |
+
+## Task Formulation for Parallel Work
+
+### Consistent Prompting
+
+All agents should receive the same core prompt to enable meaningful comparison:
+
+```markdown
+## Research Task
+
+**Topic**: [Clear description of what to research]
+**Scope**: [Boundaries of the investigation]
+**Expected Outputs**:
+- Main report with findings
+- Supplementary artifacts (guides, templates, etc.)
+- Recommendations with rationale
+
+## Context
+[Shared context all agents need]
+
+## Deliverables Format
+[Consistent structure for outputs]
+```
+
+### Avoid These Pitfalls
+
+- **Vague prompts**: Lead to incomparable outputs
+- **Different scopes**: Make synthesis difficult
+- **Inconsistent output formats**: Complicate comparison
+- **Missing context**: Results in agents making different assumptions
+
+## Cross-Review Protocol
+
+The cross-review phase is critical for surfacing blind spots and building toward consensus.
+
+### Process
+
+1. **Distribution**: Each agent receives all other agents' reports
+2. **Analysis**: Each agent identifies:
+   - Points of agreement (reinforces confidence)
+   - Points of disagreement (requires resolution)
+   - Gaps in other reports (opportunities for contribution)
+   - Improvements to incorporate
+3. **Self-Enhancement**: Each agent improves own report with peer insights
+4. **Attribution**: Credit sources when incorporating ideas
+
+### Cross-Review Prompt Template
+
+```markdown
+## Cross-Review Task
+
+You have completed initial research on [topic]. Below are reports from peer agents.
+
+### Peer Reports
+[Include all peer reports]
+
+### Your Task
+1. Identify agreements across reports
+2. Note disagreements and evaluate which position is stronger
+3. Find gaps in your report that peers covered
+4. Enhance your report by incorporating valuable insights (with attribution)
+5. Document your reasoning for key decisions
+```
+
+## Synthesis Process Overview
+
+Synthesis combines multiple agent outputs into a unified result. See [Synthesize Research Workflow](wfi://handbook/synthesize-research) for detailed execution steps.
+
+### Key Phases
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    PHASE 1: INVENTORY                       │
+│  - List all reports and supplementary artifacts             │
+│  - Create comparison matrix (artifacts × agents)            │
+│  - Rate quality and completeness                            │
+└─────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────┐
+│                    PHASE 2: COMPARE                         │
+│  - Side-by-side analysis by artifact type                   │
+│  - Note conflicts and disagreements                         │
+│  - Rate depth, accuracy, actionability                      │
+└─────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────┐
+│                    PHASE 3: RESOLVE                         │
+│  - Document conflicts with both positions                   │
+│  - Research/verify factual disagreements                    │
+│  - Make reasoned decisions with rationale                   │
+└─────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────┐
+│                    PHASE 4: SYNTHESIZE                      │
+│  - Select base (usually most comprehensive report)          │
+│  - Merge unique contributions from others                   │
+│  - Ensure consistent terminology                            │
+│  - Credit source reports                                    │
+└─────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────┐
+│                    PHASE 5: VALIDATE                        │
+│  - Completeness check (nothing valuable lost)               │
+│  - Consistency check (no contradictions)                    │
+│  - Gap identification (for future work)                     │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Aggregator Role
+
+The synthesis should be performed by a dedicated aggregator (human or agent) who:
+- Has access to all agent outputs
+- Can make objective comparisons
+- Documents all decisions with rationale
+- Produces unified, consistent output
+
+## Quality Validation Checklist
+
+### Before Parallel Research
+
+- [ ] Research question is clearly defined
+- [ ] All agents receive identical prompts
+- [ ] Output format is specified consistently
+- [ ] Shared context is complete and accurate
+- [ ] Success criteria are defined
+
+### After Cross-Review
+
+- [ ] Each agent produced enhanced report
+- [ ] Agreements are documented
+- [ ] Disagreements are identified
+- [ ] Sources are properly attributed
+
+### After Synthesis
+
+- [ ] Comparison matrix is complete
+- [ ] All conflicts have documented resolutions
+- [ ] Unified report covers all key findings
+- [ ] No contradictions in final output
+- [ ] Gaps are identified for future work
+- [ ] All sources are credited
+
+## Key Insights from Research
+
+### Industry Patterns
+
+| Pattern | Description | Benefit |
+|---------|-------------|---------|
+| Mixture-of-Agents (MoA) | Proposers generate, aggregator synthesizes | Outperforms single agents |
+| Iterative Consensus (ICE) | 2-3 rounds of critique until consensus | +7-15% accuracy |
+| LLM-BLENDER | Rank responses, blend best ones | Selects highest quality |
+
+### Best Practices
+
+1. **Quality > Quantity**: 3 quality agents outperform 5 mediocre ones
+2. **Document decisions**: Synthesis rationale is valuable for future reference
+3. **Human validation**: Final quality check remains important
+4. **Iterate if needed**: 2-3 review rounds typically sufficient
+
+## Related Resources
+
+- [Research Comparison Template](tmpl://research-comparison) - Structured comparison matrix
+- [Synthesize Research Workflow](wfi://handbook/synthesize-research) - Detailed synthesis process
+- [Parallel Research Workflow](wfi://handbook/parallel-research) - Setting up parallel execution
+
+## References
+
+### Academic Sources
+- [Mixture-of-Agents (arXiv 2406.04692)](https://arxiv.org/abs/2406.04692)
+- [Multi-Agent Collaboration Mechanisms Survey (arXiv 2501.06322)](https://arxiv.org/abs/2501.06322)
+- [Iterative Consensus Ensemble (ScienceDirect)](https://www.sciencedirect.com/science/article/abs/pii/S0010482525010820)
+
+### Industry Sources
+- [LLM Orchestration Best Practices (orq.ai)](https://orq.ai/blog/llm-orchestration)
+- [Multi-Agent LLMs in 2025 (SuperAnnotate)](https://www.superannotate.com/blog/multi-agent-llms)
+- [Awesome-LLM-Ensemble (GitHub)](https://github.com/junchenzhi/Awesome-LLM-Ensemble)

data/handbook/guides/performance/ruby.md

@@ -0,0 +1,25 @@
+# Ruby Performance Examples
+
+This file provides Ruby-specific examples related to the main [Performance Guide](../performance.g.md).
+
+* **Benchmarking:** `Benchmark` standard library module.
+* **Profiling:** `stackprof`, `ruby-prof` gems.
+* **Memory Analysis:** `memory_profiler` gem.
+
+```ruby
+require 'benchmark'
+
+iterations = 100_000
+
+Benchmark.bm(7) do |x|
+  x.report("String Interpolation:") { iterations.times do; "User ID: #{123}"; end }
+  x.report("String Concat:")      { iterations.times do; 'User ID: ' + 123.to_s; end }
+end
+
+# Example using stackprof (conceptual)
+# require 'stackprof'
+# StackProf.run(mode: :cpu, out: 'tmp/stackprof-cpu.dump') do
+#   # Code to profile
+#   1000.times { perform_complex_operation }
+# end
+```

data/handbook/guides/performance/rust.md

@@ -0,0 +1,48 @@
+# Rust Performance Examples
+
+This file provides Rust-specific examples related to the main [Performance Guide](../performance.g.md).
+
+* **Benchmarking:** `cargo bench` (requires nightly toolchain or stable with `criterion` crate), `std::time::Instant`.
+* **Profiling:** `perf` (Linux), Instruments (macOS), `cargo flamegraph`, `pprof` crate.
+* **Memory Analysis:** Valgrind (with caution), `dhat` crate, platform-specific tools.
+
+```rust
+use std::time::Instant;
+
+// Simple timing using std::time::Instant
+fn main() {
+    let start = Instant::now();
+
+    // Code to measure
+    let mut sum = 0;
+    for i in 0..1_000_000 {
+        sum += i;
+    }
+
+    let duration = start.elapsed();
+
+    println!("Time elapsed in expensive_function() is: {:?}", duration);
+    println!("Sum: {}", sum); // Prevent optimization from removing the loop
+}
+
+// For proper benchmarking, use `cargo bench` with `criterion` crate
+// Example (in benches/my_benchmark.rs):
+/*
+use criterion::{black_box, criterion_group, criterion_main, Criterion};
+
+fn fibonacci(n: u64) -> u64 {
+    match n {
+        0 => 1,
+        1 => 1,
+        n => fibonacci(n-1) + fibonacci(n-2),
+    }
+}
+
+fn criterion_benchmark(c: &mut Criterion) {
+    c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
+}
+
+criterion_group!(benches, criterion_benchmark);
+criterion_main!(benches);
+*/
+```

data/handbook/guides/performance/typescript.md

@@ -0,0 +1,32 @@
+# TypeScript Performance Examples
+
+This file provides TypeScript-specific examples related to the main [Performance Guide](../performance.g.md).
+
+* **Benchmarking:** `console.time`/`console.timeEnd`, `performance.now()` (Node.js/Browser), libraries like `benchmark.js`.
+* **Profiling:** Node.js inspector (`node --inspect`), Chrome DevTools profiler.
+* **Memory Analysis:** Node.js inspector, Chrome DevTools memory tab.
+
+```typescript
+// Simple benchmarking using console.time
+const iterations = 100000;
+
+console.time('Array Push');
+const arr1: number[] = [];
+for (let i = 0; i < iterations; i++) {
+  arr1.push(i);
+}
+console.timeEnd('Array Push');
+
+console.time('Array Pre-alloc');
+const arr2: number[] = new Array(iterations);
+for (let i = 0; i < iterations; i++) {
+  arr2[i] = i;
+}
+console.timeEnd('Array Pre-alloc');
+
+// Using performance.now() (more precise, available in Node and Browsers)
+const start = performance.now();
+// Code to measure
+const end = performance.now();
+console.log(`Execution time: ${end - start} ms`);
+```

data/handbook/guides/performance.g.md

@@ -0,0 +1,211 @@
+---
+doc-type: guide
+title: Performance Tuning Guidelines
+purpose: Documentation for ace-handbook/handbook/guides/performance.g.md
+ace-docs:
+  last-updated: 2026-01-08
+  last-checked: 2026-03-21
+---
+
+# Performance Tuning Guidelines
+
+## Goal
+
+This guide provides standard practices and techniques for identifying, measuring, and optimizing
+performance aspects (memory usage, execution speed, concurrency) within the project.
+
+## 1. Memory Management
+
+- **Object Pooling:** Use pooling for expensive or frequently created/destroyed objects (e.g.,
+  database connections, threads, large buffers) if your language/framework provides suitable libraries.
+
+```javascript
+  // Pseudo-code example: Using a generic connection pool
+  const connectionPool = createPool({
+    create: () => new DatabaseConnection(),
+    destroy: (conn) => conn.close(),
+    max: 10 // Max number of pooled connections
+  });
+
+  function performDatabaseQuery() {
+    const connection = await connectionPool.acquire();
+    try {
+      // Use connection
+    } finally {
+      connectionPool.release(connection);
+    }
+  }
+  ```
+
+- **Memory Leaks:** Be vigilant about memory leaks in long-running processes or applications. Use
+  language-specific tools (profilers, heap analyzers) to detect and fix leaks.
+  - Explicitly release resources when done.
+  - Break circular references if using languages with reference counting or specific GC patterns.
+  - Consider explicit garbage collection triggers *judiciously* if needed, but prefer designing code to be GC-friendly.
+
+  ```javascript
+  // Conceptual example of resource cleanup
+  function executeBatch(batch) {
+    // Optional: trigger GC before potentially large allocation/operation
+    triggerGarbageCollectionIfNeeded();
+
+    const results = batch.map(task => {
+      const agent = createAgentForTask(task);
+      try {
+        return agent.execute(task);
+      } finally {
+        agent.cleanupResources(); // Ensure resource cleanup
+      }
+    });
+
+    // Optional: trigger GC after potentially large operation
+    triggerGarbageCollectionIfNeeded();
+    return results;
+  }
+  ```
+
+## 2. Benchmarking
+
+Use appropriate benchmarking libraries for your language/stack to measure the performance of critical code paths.
+
+- **Micro-benchmarks:** Measure the speed of small functions or operations using dedicated benchmarking tools.
+- **Memory Profiling:** Measure memory allocation and identify potential leaks using memory analysis tools.
+
+```javascript
+// Pseudo-code example: Micro-benchmarking
+const suite = new Benchmark.Suite;
+
+suite
+  .add('Operation A', function() {
+    performOperationA();
+  })
+  .add('Operation B', function() {
+    performOperationB();
+  })
+  .on('cycle', function(event) {
+    console.log(String(event.target));
+  })
+  .on('complete', function() {
+    console.log('Fastest is ' + this.filter('fastest').map('name'));
+  })
+  .run({ 'async': true });
+
+// Pseudo-code example: Memory profiling concept
+startMemoryProfiling();
+```
+
+```javascript
+// Pseudo-code example: Memory profiling concept
+startMemoryProfiling();
+
+for (let i = 0; i < 100; i++) {
+  agent.execute(someTask);
+}
+
+const report = stopMemoryProfiling();
+printMemoryReport(report); // Analyze allocations, retained objects, etc.
+```
+
+## 3. Threading & Concurrency Optimization
+
+Optimize concurrent operations carefully.
+
+- **Thread Pools:** Use thread pools to manage a fixed number of threads, reducing the overhead of
+  thread creation/destruction and controlling resource usage using libraries appropriate for your
+  language.
+
+```javascript
+  // Pseudo-code example: Thread pool configuration
+  const threadPool = createThreadPool({
+    minThreads: 2,
+    maxThreads: Math.max(getCpuCoreCount() - 1, 2),
+    maxQueueSize: 100,
+    fallbackPolicy: 'callerRuns' // Policy if queue is full
+  });
+  ```
+
+- **Asynchronous Operations:** Prefer non-blocking I/O and asynchronous patterns (`async/await`,
+  Promises, Futures, callbacks, etc.) where possible, especially for I/O-bound tasks.
+- **Batch Processing:** Process tasks in batches using available concurrency mechanisms.
+
+```javascript
+  // Pseudo-code example: Batch processing with async/await and promises
+  async function processBatch(tasks) {
+    const promises = tasks.map(task => {
+      // Schedule task execution (e.g., using a thread pool or async function)
+      return scheduleAsyncTask(() => agent.execute(task));
+    });
+    // Wait for all tasks in the batch to complete
+    return await Promise.all(promises);
+  }
+  ```
+
+## 4. Monitoring Points
+
+Instrument your code to send performance metrics (timing, counts, gauges) to a monitoring system
+(e.g., StatsD, Prometheus, Datadog).
+
+- Wrap critical operations or external calls with timing measurements.
+- Track queue sizes, pool usage, error rates.
+
+```javascript
+// Pseudo-code example: Instrumentation
+function measureOperation(operationName, func) {
+  const start = highResolutionTimeNow();
+  try {
+    const result = func();
+    const duration = highResolutionTimeNow() - start;
+    // Send timing metric to monitoring system (e.g., StatsD)
+    statsd.timing(`agent.${operationName}.success`, duration);
+    return result;
+  } catch (error) {
+    const duration = highResolutionTimeNow() - start;
+    // Send error count and timing
+    statsd.increment(`agent.${operationName}.error`);
+    statsd.timing(`agent.${operationName}.error`, duration);
+    throw error;
+  }
+}
+```javascript
+
+// Usage
+function execute(task) {
+  return measureOperation('execution', () => {
+    // Original task execution logic
+  });
+}
+
+```
+
+### 5. Performance Testing
+
+- **Benchmarking:** Measure the execution time of critical code paths before and after optimization.
+- **Load Testing:** Simulate realistic user load to identify bottlenecks under stress.
+- **Profiling in CI:** Integrate basic performance checks into CI to catch regressions early.
+
+**Example Benchmarking (Conceptual):**
+
+```javascript
+// Pseudocode for benchmarking
+startTime = getCurrentTime();
+runCriticalFunction(testData);
+endTime = getCurrentTime();
+duration = endTime - startTime;
+logBenchmarkResult("criticalFunction", duration);
+
+// Compare against baseline or previous runs
+if (duration > baselineDuration * 1.1) {
+  reportPerformanceRegression("criticalFunction", duration, baselineDuration);
+}
+```
+
+## Language/Environment-Specific Examples
+
+For specific examples of profiling tools, benchmarking libraries, memory analysis techniques, or language-specific
+performance optimizations (e.g., Ruby's `benchmark` module, Python's `cProfile`, Go's `pprof`, Node.js performance
+hooks), please refer to the examples in the [./performance/](./performance/) sub-directory.
+
+## Related Documentation
+
+- [Testing Guidelines](guide://testing) (Benchmarking, Load Testing)
+- [Quality Assurance](./quality-assurance.g.md) (Monitoring)

data/handbook/guides/prompt-caching.g.md

@@ -0,0 +1,83 @@
+---
+doc-type: guide
+title: Prompt Caching Pattern
+purpose: Documentation for ace-handbook/handbook/guides/prompt-caching.g.md
+ace-docs:
+  last-updated: 2026-03-04
+  last-checked: 2026-03-21
+---
+
+# Prompt Caching Pattern
+
+Standardized pattern for gems that generate prompts for LLM interactions.
+
+## Overview
+
+Gems that generate prompts (ace-review, ace-docs, ace-prompt-prep) use the `PromptCacheManager` from ace-support-core to store prompts in a predictable location for debugging and inspection.
+
+## Directory Structure
+
+```
+.cache/
+└── {gem-name}/
+    └── sessions/
+        └── {operation}-{timestamp}/
+            ├── system.prompt.md    # System prompt
+            ├── user.prompt.md      # User prompt
+            └── metadata.yml        # Session metadata (optional)
+```
+
+## API Usage
+
+```ruby
+require 'ace/core/molecules/prompt_cache_manager'
+
+# Create session directory
+session_dir = Ace::Core::Molecules::PromptCacheManager.create_session(
+  "ace-my-gem",
+  "my-operation"
+)
+# Returns: .ace-local/my-gem/sessions/my-operation-20261116-143022/
+
+# Save prompts
+Ace::Core::Molecules::PromptCacheManager.save_system_prompt(
+  system_prompt_content,
+  session_dir
+)
+
+Ace::Core::Molecules::PromptCacheManager.save_user_prompt(
+  user_prompt_content,
+  session_dir
+)
+
+# Save metadata (optional)
+metadata = {
+  "timestamp" => Time.now.utc.iso8601,
+  "gem" => "ace-my-gem",
+  "operation" => "my-operation",
+  "model" => "google:gemini-2.5-flash",
+  "prompt_sizes" => { "system" => 1234, "user" => 5678 }
+}
+Ace::Core::Molecules::PromptCacheManager.save_metadata(metadata, session_dir)
+```
+
+## Benefits
+
+- **Consistent locations**: All prompt caches in predictable `.cache/{gem}/sessions/` structure
+- **Standard naming**: `system.prompt.md`, `user.prompt.md` across all gems
+- **Git worktree support**: Uses ProjectRootFinder internally
+- **Easy debugging**: Inspect exact prompts sent to LLMs
+- **Metadata tracking**: Optional standardized metadata format
+
+## Production Examples
+
+| Gem | Usage |
+|-----|-------|
+| ace-docs | analyze-consistency operation |
+| ace-prompt-prep | enhanced prompts with content-based deduplication |
+| ace-review | session caching (optional migration to shared utility) |
+
+## Related
+
+- [ace-gems.g.md](../../../docs/ace-gems.g.md) - Gem development overview
+- [ADR-022](../../../docs/decisions/ADR-022-configuration-default-and-override-pattern.md) - Configuration patterns

data/handbook/guides/quality-assurance/ruby.md

@@ -0,0 +1,33 @@
+# Ruby Quality Assurance Examples
+
+This file provides Ruby-specific examples related to the main [Quality Assurance Guide](../quality-assurance.g.md).
+
+* **Linters/Formatters:** `rubocop`, `standardrb`
+* **Static Analysis:** `brakeman` (Security), `reek` (Code Smells)
+* **Test Coverage:** `simplecov`
+* **CI Configuration:** Examples for GitHub Actions, GitLab CI, etc. using Ruby setup actions.
+
+**Example `.standard.yml` (StandardRB config):**
+
+```yaml
+fix: true               # default: false
+parallel: true          # default: false
+format: progress        # default: Standard::Formatter
+ignore:
+  - 'db/schema.rb'
+  - 'vendor/**/*'
+```
+
+**Example `simplecov` setup (in `spec/spec_helper.rb` or `test/test_helper.rb`):**
+
+```ruby
+require 'simplecov'
+SimpleCov.start do
+  add_filter '/spec/' # Ignore spec files themselves
+  add_filter '/vendor/'
+  # add_group 'Controllers', 'app/controllers'
+  # minimum_coverage 90
+end
+
+# Rest of spec_helper/test_helper
+```