@cakemail-org/cakemail-cli 1.5.0 → 2.0.0

package/docs/testing/KENOGAMI_TRUTH_RECONCILIATION_SYSTEM.md

@@ -0,0 +1,1369 @@
+# Kenogami: Multi-Source Truth Reconciliation System
+
+## Executive Summary
+
+The fundamental challenge: **Truth exists in multiple places**, each with different authority and purpose. The knowledge graph is not the ultimate source of truth—it's a **reconciliation layer** that harmonizes truths from multiple authoritative sources, detects conflicts, identifies gaps, and maintains coherence across the entire system.
+
+**Core Insight:** Don't force a single source of truth. Instead, recognize that truth is **distributed** and **context-dependent**, then build systems to **reconcile**, **validate**, and **synthesize** these distributed truths.
+
+---
+
+## The Reality: Distributed Sources of Truth
+
+### Source 1: Product Code (Implementation Truth)
+
+**What it represents:** What the system *actually does*
+
+```typescript
+// cakemail-cli/src/commands/campaigns.ts
+export function createCampaignsCommand() {
+  return new Command('campaigns')
+    .command('list')
+    .option('-s, --status <status>', 'Filter by status')
+    .option('-l, --limit <number>', 'Limit results')
+    .action(async (options) => {
+      // Implementation details
+    });
+}
+```
+
+**Authority Level:** 🔴 **HIGHEST** - This is objective reality
+**Characteristics:**
+- Definitive for "what is implemented"
+- Changes through code commits
+- Version controlled
+- Can be analyzed programmatically (AST parsing, type inference)
+
+**Limitations:**
+- Doesn't explain *why*
+- Doesn't document *intent*
+- Doesn't describe *workflows* or *best practices*
+- Doesn't capture *user perspective*
+
+### Source 2: API/OpenAPI Specification (Contract Truth)
+
+**What it represents:** What the system *promises to do*
+
+```yaml
+# Cakemail API OpenAPI spec
+/campaigns:
+  get:
+    parameters:
+      - name: status
+        in: query
+        schema:
+          type: string
+          enum: [draft, sent, scheduled]
+```
+
+**Authority Level:** 🟠 **HIGH** - This is the contract
+**Characteristics:**
+- Authoritative for API behavior
+- Version controlled
+- Machine-readable
+- Can drift from implementation (bug or spec issue)
+
+**Limitations:**
+- May lag behind implementation
+- Doesn't include CLI-specific features
+- Doesn't describe user journeys
+
+### Source 3: Original Documentation (Intent Truth)
+
+**What it represents:** What the product team *intended* to communicate
+
+```markdown
+# Creating Campaigns
+
+To create a campaign, use the `campaigns create` command.
+You'll need a list ID and sender ID...
+```
+
+**Authority Level:** 🟡 **MEDIUM** - This is the official narrative
+**Characteristics:**
+- Captures product team's intent
+- Includes context, rationale, workflows
+- Human-written, human-optimized
+- May drift from reality over time
+
+**Limitations:**
+- Can become outdated
+- May have inconsistencies
+- Subjective writing style
+
+### Source 4: User Behavior Data (Usage Truth)
+
+**What it represents:** How users *actually use* the product
+
+```json
+{
+  "command": "cakemail campaigns list --status sent",
+  "frequency": 1247,
+  "success_rate": 0.94,
+  "common_next_command": "cakemail campaigns get <id>"
+}
+```
+
+**Authority Level:** 🟡 **MEDIUM** - This is behavioral reality
+**Characteristics:**
+- Shows actual usage patterns
+- Reveals what users care about
+- Indicates what works / doesn't work
+- Data-driven insights
+
+**Limitations:**
+- Doesn't explain intent
+- May show workarounds, not ideal usage
+- Includes both correct and incorrect usage
+
+### Source 5: Customer Support Tickets (Problem Truth)
+
+**What it represents:** Where users *struggle*
+
+```
+Ticket #4521: "I tried 'campaigns list --status sending' but got an error.
+The docs say 'sending' is a valid status, but it doesn't work."
+
+Root Cause: Docs outdated. Status 'sending' was replaced with 'in_progress' in v1.5
+```
+
+**Authority Level:** 🟢 **LOW-MEDIUM** - This is the pain point signal
+**Characteristics:**
+- Identifies documentation gaps
+- Shows common misconceptions
+- Reveals UX problems
+- Indicates priority areas
+
+**Limitations:**
+- Biased toward problems (not successes)
+- May include user error
+- Needs human interpretation
+
+### Source 6: Chatbot Conversations (Learning Truth)
+
+**What it represents:** What users *ask about* and how they *learn*
+
+```
+User: "How do I create a campaign?"
+Bot: "Use 'cakemail campaigns create --name "My Campaign" --list-id 123'"
+User: "What's a list ID?"
+Bot: "A list ID is the unique identifier for your contact list..."
+```
+
+**Authority Level:** 🟢 **LOW** - This is the learning journey
+**Characteristics:**
+- Shows knowledge gaps
+- Reveals terminology confusion
+- Indicates documentation clarity issues
+- Captures natural language queries
+
+**Limitations:**
+- Conversational, not canonical
+- May include bot hallucinations
+- Context-dependent
+
+### Source 7: Test Results (Validation Truth)
+
+**What it represents:** What has been *verified* to work
+
+```json
+{
+  "test": "campaigns list with status filter",
+  "status": "PASS",
+  "actual_output": {...},
+  "matches_documentation": true,
+  "matches_implementation": true
+}
+```
+
+**Authority Level:** 🔵 **VALIDATION** - This is verification
+**Characteristics:**
+- Proves what works
+- Detects regressions
+- Validates consistency
+- Objective measurement
+
+**Limitations:**
+- Only tests what we think to test
+- Can have bugs in tests themselves
+- Doesn't capture intent
+
+---
+
+## The Problem: Truth Conflicts & Gaps
+
+### Conflict Types
+
+**Type 1: Code vs Documentation**
+```
+Code:      Parameter 'status' accepts: ['draft', 'sent', 'scheduled', 'failed']
+Docs:      Parameter 'status' accepts: ['draft', 'sent', 'sending', 'scheduled']
+Conflict:  'failed' missing from docs, 'sending' doesn't exist in code
+```
+
+**Type 2: API Spec vs Implementation**
+```
+Spec:      Max limit is 100
+CLI Code:  No max limit enforced (accepts any number)
+API Code:  Max limit is 50
+Conflict:  Three different truths!
+```
+
+**Type 3: Documentation Inconsistency**
+```
+Tutorial:    "Use --format json to get JSON output"
+Reference:   "JSON is the default format"
+Quick Start: "Use -f table for readable output"
+Conflict:    What's actually the default?
+```
+
+**Type 4: Usage vs Intent**
+```
+Intended:  Users should use 'campaigns create' with all parameters
+Actual:    80% of users use interactive mode (no parameters)
+Conflict:  Documentation focuses on non-interactive, users prefer interactive
+```
+
+**Type 5: Support vs Documentation**
+```
+Docs:       "Authentication is automatic"
+Tickets:    50 tickets about "How do I authenticate?"
+Conflict:   Documentation assumption doesn't match user understanding
+```
+
+### Gap Types
+
+**Type 1: Undocumented Features**
+```
+Code:       Supports '--batch' flag for non-interactive mode
+Docs:       No mention of batch mode
+Gap:        Feature exists but not documented
+```
+
+**Type 2: Documented but Not Implemented**
+```
+Docs:       "Use --template-id to create campaign from template"
+Code:       --template-id option doesn't exist yet
+Gap:        Documentation ahead of implementation (planned feature?)
+```
+
+**Type 3: Missing Workflows**
+```
+Commands:   campaigns create, campaigns schedule, campaigns send
+Docs:       Each command documented separately
+Gap:        Complete workflow not documented
+Usage Data: Users struggle to understand the sequence
+```
+
+**Type 4: Hidden Knowledge**
+```
+Support:    "First confirm your sender before creating campaigns"
+Docs:       No mention of sender confirmation requirement
+Code:       Checks for confirmed sender, returns cryptic error if not
+Gap:        Domain knowledge only in support team's heads
+```
+
+---
+
+## The Solution: Multi-Layer Truth Reconciliation
+
+### Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    TRUTH SOURCES (Multiple)                      │
+└─────────────────────────────────────────────────────────────────┘
+     │         │         │         │         │         │         │
+     │Product  │  API    │Original │ Usage   │Support │ Chatbot │
+     │ Code    │  Spec   │  Docs   │  Data   │Tickets │  Logs   │
+     │         │         │         │         │         │         │
+     └────┬────┴────┬────┴────┬────┴────┬────┴────┬────┴────┬────┘
+          │         │         │         │         │         │
+          └─────────┴─────────┴─────────┴─────────┴─────────┘
+                              │
+                              ▼
+         ┌────────────────────────────────────────────────┐
+         │         EXTRACTION & INGESTION LAYER           │
+         │  - Code AST Parser                             │
+         │  - OpenAPI Parser                              │
+         │  - Markdown Parser                             │
+         │  - Analytics Aggregator                        │
+         │  - Ticket Classifier                           │
+         │  - Conversation Analyzer                       │
+         └────────────────┬───────────────────────────────┘
+                          │
+                          ▼
+         ┌────────────────────────────────────────────────┐
+         │         RECONCILIATION ENGINE                  │
+         │  - Conflict Detection                          │
+         │  - Gap Analysis                                │
+         │  - Truth Scoring                               │
+         │  - Canonical Resolution                        │
+         └────────────────┬───────────────────────────────┘
+                          │
+                          ▼
+         ┌────────────────────────────────────────────────┐
+         │         KNOWLEDGE GRAPH (Reconciled)           │
+         │                                                │
+         │  ┌──────────┐    ┌──────────┐    ┌─────────┐ │
+         │  │  Facts   │    │Conflicts │    │  Gaps   │ │
+         │  │ (Nodes)  │    │ (Edges)  │    │ (Nodes) │ │
+         │  └──────────┘    └──────────┘    └─────────┘ │
+         │                                                │
+         │  Each fact has provenance:                    │
+         │  - source: [code, spec, docs, usage, ...]     │
+         │  - confidence: 0.0-1.0                        │
+         │  - last_verified: timestamp                   │
+         │  - conflicts_with: [other facts]              │
+         └────────────────┬───────────────────────────────┘
+                          │
+              ┌───────────┴──────────┐
+              │                      │
+              ▼                      ▼
+    ┌──────────────────┐   ┌──────────────────┐
+    │   OUTPUT LAYER   │   │  FEEDBACK LAYER  │
+    │  - Articles      │   │  - CI/CD Alerts  │
+    │  - Tests         │   │  - Dashboards    │
+    │  - API Docs      │   │  - GitHub Issues │
+    │  - Chatbot KB    │   │  - Slack Notifs  │
+    └──────────────────┘   └──────────────────┘
+```
+
+---
+
+## Truth Reconciliation Strategies
+
+### Strategy 1: Hierarchical Authority
+
+Different sources have different authority for different questions:
+
+```typescript
+interface TruthAuthority {
+  question: string;
+  primarySource: Source;
+  validationSources: Source[];
+  tieBreaker: Source;
+}
+
+const authorityRules: TruthAuthority[] = [
+  {
+    question: "What parameters does this command accept?",
+    primarySource: "product_code",  // Code is ground truth
+    validationSources: ["api_spec", "test_results"],
+    tieBreaker: "test_results"  // If code and spec conflict, tests prove reality
+  },
+  {
+    question: "What is the recommended workflow?",
+    primarySource: "documentation",  // Product team's intent
+    validationSources: ["usage_data"],
+    tieBreaker: "usage_data"  // But if users do something different, consider it
+  },
+  {
+    question: "What terminology should we use?",
+    primarySource: "documentation",  // Canonical terms
+    validationSources: ["chatbot_logs"],
+    tieBreaker: "chatbot_logs"  // Users' natural language reveals confusion
+  },
+  {
+    question: "Is this feature working correctly?",
+    primarySource: "test_results",  // Tests prove it works
+    validationSources: ["support_tickets"],
+    tieBreaker: "support_tickets"  // Tickets reveal edge cases tests missed
+  }
+];
+```
+
+### Strategy 2: Temporal Consensus
+
+Truth can change over time. Track when each source was last updated:
+
+```typescript
+interface TemporalFact {
+  fact: "campaigns list supports --format parameter";
+  sources: [
+    {
+      source: "code",
+      value: true,
+      timestamp: "2025-01-15T10:00:00Z",
+      version: "v1.5.0"
+    },
+    {
+      source: "documentation",
+      value: true,
+      timestamp: "2025-01-20T14:30:00Z",  // Updated 5 days later
+      version: "docs-rev-234"
+    },
+    {
+      source: "test_results",
+      value: true,
+      timestamp: "2025-01-15T10:05:00Z",  // Tested immediately
+      lastRun: "2025-10-26T08:00:00Z"
+    }
+  ];
+}
+
+// Reconciliation logic
+function reconcile(fact: TemporalFact): Resolution {
+  const newestSource = maxBy(fact.sources, s => s.timestamp);
+  const outdatedSources = fact.sources.filter(s =>
+    s.timestamp < newestSource.timestamp - STALENESS_THRESHOLD
+  );
+
+  if (outdatedSources.length > 0) {
+    return {
+      status: "NEEDS_UPDATE",
+      canonical: newestSource.value,
+      stale: outdatedSources,
+      action: `Update ${outdatedSources.map(s => s.source).join(', ')}`
+    };
+  }
+
+  return { status: "CONSISTENT" };
+}
+```
+
+### Strategy 3: Confidence Scoring
+
+Not all statements are equal. Score confidence based on multiple factors:
+
+```typescript
+interface ConfidenceScore {
+  fact: string;
+  confidence: number;  // 0.0 - 1.0
+  factors: {
+    sourceAuthority: number;     // How authoritative is the source?
+    sourceFreshness: number;      // How recent is the information?
+    sourceConsensus: number;      // How many sources agree?
+    validationStatus: number;     // Has it been tested/verified?
+    userConfirmation: number;     // Do users' actions confirm it?
+    supportEvidence: number;      // Do tickets confirm or contradict?
+  };
+}
+
+function calculateConfidence(fact: Fact): ConfidenceScore {
+  const scores = {
+    sourceAuthority: calculateAuthorityScore(fact.sources),
+    sourceFreshness: calculateFreshnessScore(fact.sources),
+    sourceConsensus: calculateConsensusScore(fact.sources),
+    validationStatus: fact.testResults?.passed ? 1.0 : 0.5,
+    userConfirmation: calculateUsageAlignment(fact, usageData),
+    supportEvidence: calculateTicketAlignment(fact, tickets)
+  };
+
+  // Weighted average
+  const weights = {
+    sourceAuthority: 0.3,
+    sourceFreshness: 0.2,
+    sourceConsensus: 0.2,
+    validationStatus: 0.15,
+    userConfirmation: 0.10,
+    supportEvidence: 0.05
+  };
+
+  const confidence = Object.entries(scores).reduce(
+    (sum, [key, score]) => sum + score * weights[key],
+    0
+  );
+
+  return { fact: fact.statement, confidence, factors: scores };
+}
+```
+
+### Strategy 4: Gap Detection Patterns
+
+Systematically identify gaps using cross-source analysis:
+
+```typescript
+class GapDetector {
+  async detectGaps(): Promise<Gap[]> {
+    const gaps: Gap[] = [];
+
+    // Gap Type 1: Code features not documented
+    const undocumentedFeatures = await this.findUndocumentedFeatures();
+    gaps.push(...undocumentedFeatures.map(f => ({
+      type: 'undocumented_feature',
+      severity: 'high',
+      description: `Feature '${f.name}' exists in code but not documented`,
+      sources: { has: ['code'], missing: ['documentation'] },
+      action: 'Document this feature'
+    })));
+
+    // Gap Type 2: Documentation references non-existent features
+    const ghostFeatures = await this.findGhostFeatures();
+    gaps.push(...ghostFeatures.map(f => ({
+      type: 'ghost_feature',
+      severity: 'critical',
+      description: `Documentation mentions '${f.name}' but it doesn't exist in code`,
+      sources: { has: ['documentation'], missing: ['code'] },
+      action: 'Remove from docs OR implement feature'
+    })));
+
+    // Gap Type 3: High usage features not well documented
+    const underdocumentedFeatures = await this.findUnderdocumented();
+    gaps.push(...underdocumentedFeatures.map(f => ({
+      type: 'underdocumented',
+      severity: 'medium',
+      description: `Feature '${f.name}' is heavily used (${f.usageCount} times) but docs are minimal`,
+      sources: { has: ['code', 'usage_data'], insufficient: ['documentation'] },
+      action: 'Expand documentation with examples and best practices'
+    })));
+
+    // Gap Type 4: Common support questions not in docs
+    const faqGaps = await this.findFAQGaps();
+    gaps.push(...faqGaps.map(q => ({
+      type: 'missing_faq',
+      severity: 'medium',
+      description: `Common question: "${q.question}" (${q.ticketCount} tickets)`,
+      sources: { has: ['support_tickets'], missing: ['documentation'] },
+      action: 'Add FAQ entry or improve relevant documentation'
+    })));
+
+    // Gap Type 5: Workflows implied but not documented
+    const workflowGaps = await this.findWorkflowGaps();
+    gaps.push(...workflowGaps.map(w => ({
+      type: 'missing_workflow',
+      severity: 'high',
+      description: `Common sequence: ${w.steps.join(' → ')} (${w.frequency} occurrences)`,
+      sources: { has: ['usage_data'], missing: ['documentation'] },
+      action: 'Document this workflow as a tutorial'
+    })));
+
+    return gaps;
+  }
+
+  async findUndocumentedFeatures(): Promise<Feature[]> {
+    // Find features in code
+    const codeFeatures = await this.extractFeaturesFromCode();
+
+    // Find features in docs
+    const docFeatures = await this.extractFeaturesFromDocs();
+
+    // Features in code but not in docs
+    return codeFeatures.filter(cf =>
+      !docFeatures.some(df => df.name === cf.name)
+    );
+  }
+
+  async findWorkflowGaps(): Promise<Workflow[]> {
+    // Analyze command sequences from usage data
+    const sequences = await this.analyzeCommandSequences(usageData);
+
+    // Find common patterns (e.g., "create → schedule → send")
+    const commonWorkflows = sequences.filter(s =>
+      s.frequency > WORKFLOW_THRESHOLD
+    );
+
+    // Check if documented
+    const documentedWorkflows = await this.getDocumentedWorkflows();
+
+    return commonWorkflows.filter(cw =>
+      !documentedWorkflows.some(dw => dw.matches(cw))
+    );
+  }
+}
+```
+
+### Strategy 5: Conflict Resolution Pipeline
+
+When sources conflict, use a systematic resolution process:
+
+```typescript
+class ConflictResolver {
+  async resolve(conflict: Conflict): Promise<Resolution> {
+    // Step 1: Identify conflict type
+    const conflictType = this.classifyConflict(conflict);
+
+    // Step 2: Apply appropriate resolution strategy
+    switch (conflictType) {
+      case 'code_vs_docs':
+        return this.resolveCodeVsDocs(conflict);
+
+      case 'spec_vs_implementation':
+        return this.resolveSpecVsImpl(conflict);
+
+      case 'intent_vs_usage':
+        return this.resolveIntentVsUsage(conflict);
+
+      case 'multiple_doc_versions':
+        return this.resolveDocInconsistency(conflict);
+
+      default:
+        return this.escalateToHuman(conflict);
+    }
+  }
+
+  async resolveCodeVsDocs(conflict: Conflict): Promise<Resolution> {
+    // Code is authoritative for "what is"
+    // But check if code might be the bug
+
+    // Run tests
+    const testResult = await this.runTests(conflict.codeVersion);
+
+    if (!testResult.passed) {
+      // Code is broken, docs might be right
+      return {
+        canonical: conflict.docsVersion,
+        action: 'FIX_CODE',
+        reason: 'Code fails tests, documentation describes correct behavior',
+        evidence: testResult
+      };
+    }
+
+    // Code works, check if it matches API spec
+    const specCheck = await this.checkAgainstSpec(conflict.codeVersion);
+
+    if (!specCheck.matches) {
+      // Code works but doesn't match spec - need decision
+      return {
+        canonical: null,
+        action: 'DECISION_REQUIRED',
+        reason: 'Code works but deviates from API spec',
+        options: [
+          { action: 'UPDATE_SPEC', rationale: 'Code is better' },
+          { action: 'UPDATE_CODE', rationale: 'Spec is the contract' }
+        ]
+      };
+    }
+
+    // Code is correct, update docs
+    return {
+      canonical: conflict.codeVersion,
+      action: 'UPDATE_DOCS',
+      reason: 'Code is tested and matches spec'
+    };
+  }
+
+  async resolveIntentVsUsage(conflict: Conflict): Promise<Resolution> {
+    // Product intent says one thing, users do another
+
+    const usageStrength = conflict.usageData.frequency;
+    const intentClarity = conflict.documentation.clarity_score;
+
+    if (usageStrength > HIGH_USAGE_THRESHOLD && intentClarity < LOW_CLARITY_THRESHOLD) {
+      // Users found a better way despite poor docs
+      return {
+        canonical: 'HYBRID',
+        action: 'DOCUMENT_BOTH',
+        reason: 'Users prefer alternative approach, both should be documented',
+        recommendation: {
+          primary: conflict.usageData.approach,
+          alternative: conflict.documentation.approach
+        }
+      };
+    }
+
+    if (conflict.supportTickets.count > TICKET_THRESHOLD) {
+      // Intent is confusing users
+      return {
+        canonical: conflict.usageData.approach,
+        action: 'UPDATE_INTENT',
+        reason: 'User behavior reveals better UX',
+        evidence: conflict.supportTickets
+      };
+    }
+
+    // Intent is clear, users just need guidance
+    return {
+      canonical: conflict.documentation.approach,
+      action: 'IMPROVE_DOCS',
+      reason: 'Add examples showing intended usage pattern'
+    };
+  }
+}
+```
+
+---
+
+## Knowledge Graph Schema: Multi-Source Model
+
+### Enhanced Node Schema
+
+```typescript
+interface FactNode {
+  id: string;
+  type: 'command' | 'parameter' | 'concept' | 'workflow' | 'error';
+
+  // The canonical fact (reconciled truth)
+  canonicalValue: any;
+  confidence: number;  // 0.0 - 1.0
+
+  // Provenance: where did this fact come from?
+  sources: SourceEvidence[];
+
+  // Temporal tracking
+  firstSeen: Date;
+  lastUpdated: Date;
+  lastVerified: Date;
+
+  // Conflict tracking
+  hasConflicts: boolean;
+  conflicts?: Conflict[];
+
+  // Gap tracking
+  hasGaps: boolean;
+  gaps?: Gap[];
+
+  // Metadata
+  tags: string[];
+  version: string;
+}
+
+interface SourceEvidence {
+  source: 'code' | 'spec' | 'docs' | 'usage' | 'support' | 'chatbot' | 'tests';
+  value: any;
+  confidence: number;
+  timestamp: Date;
+  version?: string;
+
+  // Provenance details
+  location?: string;  // File path, URL, ticket ID, etc.
+  extractedBy?: string;  // Parser, human, AI, etc.
+  validatedBy?: string[];  // What validated this?
+
+  // Metadata
+  metadata?: {
+    commit?: string;
+    author?: string;
+    reviewers?: string[];
+    [key: string]: any;
+  };
+}
+
+interface Conflict {
+  id: string;
+  type: 'value_mismatch' | 'presence_mismatch' | 'intent_mismatch';
+  severity: 'low' | 'medium' | 'high' | 'critical';
+
+  sources: [SourceEvidence, SourceEvidence];  // Conflicting sources
+  description: string;
+
+  // Resolution
+  status: 'detected' | 'analyzing' | 'resolved' | 'escalated';
+  resolution?: Resolution;
+  resolvedAt?: Date;
+  resolvedBy?: string;  // 'ai' | 'human' | 'test'
+}
+
+interface Gap {
+  id: string;
+  type: 'undocumented' | 'unimplemented' | 'undertested' | 'underused' | 'faq_missing';
+  severity: 'low' | 'medium' | 'high' | 'critical';
+
+  description: string;
+  evidence: SourceEvidence[];
+
+  // Impact
+  affectedUsers?: number;
+  supportTicketCount?: number;
+  usageFrequency?: number;
+
+  // Resolution
+  status: 'detected' | 'planned' | 'in_progress' | 'resolved';
+  assignedTo?: string;
+  dueDate?: Date;
+}
+```
+
+### Graph Relationships with Provenance
+
+```cypher
+// Command with multiple source evidence
+CREATE (c:Command {
+  id: 'cmd-campaigns-list',
+  canonicalValue: {
+    name: 'campaigns list',
+    parameters: ['status', 'limit', 'page']
+  },
+  confidence: 0.95
+})
+
+// Source evidence from code
+CREATE (e1:Evidence {
+  source: 'code',
+  location: 'src/commands/campaigns.ts:45',
+  extractedAt: datetime('2025-10-26T10:00:00Z'),
+  value: {
+    parameters: ['status', 'limit', 'page', 'sort']  // Note: has 'sort'!
+  }
+})
+
+// Source evidence from docs
+CREATE (e2:Evidence {
+  source: 'documentation',
+  location: 'docs/commands/campaigns.md',
+  extractedAt: datetime('2025-10-20T14:00:00Z'),
+  value: {
+    parameters: ['status', 'limit', 'page']  // Note: missing 'sort'!
+  }
+})
+
+// Source evidence from tests
+CREATE (e3:Evidence {
+  source: 'test_results',
+  location: 'tests/campaigns.test.ts',
+  lastRun: datetime('2025-10-26T08:00:00Z'),
+  value: {
+    parameters: ['status', 'limit', 'page', 'sort'],  // Confirms 'sort' exists
+    allPassed: true
+  }
+})
+
+// Relationships
+CREATE (c)-[:HAS_EVIDENCE {confidence: 0.9}]->(e1)
+CREATE (c)-[:HAS_EVIDENCE {confidence: 0.7}]->(e2)
+CREATE (c)-[:HAS_EVIDENCE {confidence: 1.0}]->(e3)
+
+// Conflict detected
+CREATE (conflict:Conflict {
+  type: 'value_mismatch',
+  description: 'Code and tests show sort parameter exists, docs missing it',
+  severity: 'medium'
+})
+CREATE (e1)-[:CONFLICTS_WITH]->(e2)
+CREATE (c)-[:HAS_CONFLICT]->(conflict)
+
+// Gap detected
+CREATE (gap:Gap {
+  type: 'undocumented',
+  description: 'sort parameter not documented',
+  severity: 'medium'
+})
+CREATE (c)-[:HAS_GAP]->(gap)
+```
+
+---
+
+## Operational Workflows
+
+### Workflow 1: Continuous Reconciliation (Automated)
+
+```typescript
+// Runs continuously in CI/CD pipeline
+class ContinuousReconciliation {
+  async reconcile() {
+    // 1. Extract truth from all sources
+    const codeExtract = await this.extractFromCode();
+    const specExtract = await this.extractFromSpec();
+    const docsExtract = await this.extractFromDocs();
+    const usageExtract = await this.extractFromUsage();
+    const supportExtract = await this.extractFromSupport();
+    const chatExtract = await this.extractFromChatbot();
+    const testExtract = await this.extractFromTests();
+
+    // 2. Ingest into graph
+    await this.ingestEvidence([
+      codeExtract,
+      specExtract,
+      docsExtract,
+      usageExtract,
+      supportExtract,
+      chatExtract,
+      testExtract
+    ]);
+
+    // 3. Detect conflicts
+    const conflicts = await this.detectConflicts();
+
+    // 4. Detect gaps
+    const gaps = await this.detectGaps();
+
+    // 5. Auto-resolve what we can
+    const autoResolved = await this.autoResolve(conflicts);
+
+    // 6. Escalate what we can't
+    const escalated = conflicts.filter(c => !autoResolved.includes(c));
+    await this.createGitHubIssues(escalated);
+
+    // 7. Generate reports
+    await this.generateReport({
+      conflicts: { total: conflicts.length, resolved: autoResolved.length, escalated: escalated.length },
+      gaps: { total: gaps.length, byType: groupBy(gaps, 'type') },
+      confidence: this.calculateOverallConfidence()
+    });
+
+    // 8. Update outputs (docs, tests)
+    if (autoResolved.length > 0) {
+      await this.regenerateAffectedOutputs(autoResolved);
+    }
+  }
+
+  async extractFromCode(): Promise<SourceEvidence[]> {
+    // Use AST parsing to extract command structure
+    const ast = await parseTypeScript('src/commands/**/*.ts');
+
+    const commands = ast.findAll('Command')
+      .map(node => ({
+        source: 'code',
+        type: 'command',
+        value: {
+          name: node.name,
+          parameters: node.options.map(o => ({
+            name: o.name,
+            type: inferType(o.type),
+            required: o.required,
+            default: o.default
+          }))
+        },
+        location: `${node.file}:${node.line}`,
+        timestamp: node.lastModified,
+        metadata: {
+          commit: node.commit,
+          author: node.author
+        }
+      }));
+
+    return commands;
+  }
+
+  async extractFromUsage(): Promise<SourceEvidence[]> {
+    // Query analytics database for command usage
+    const usageData = await analytics.query(`
+      SELECT
+        command,
+        COUNT(*) as frequency,
+        AVG(success::int) as success_rate,
+        MODE() WITHIN GROUP (ORDER BY next_command) as common_next_command
+      FROM command_logs
+      WHERE timestamp > NOW() - INTERVAL '30 days'
+      GROUP BY command
+      HAVING COUNT(*) > 10
+    `);
+
+    return usageData.map(row => ({
+      source: 'usage',
+      type: 'command_usage',
+      value: {
+        command: row.command,
+        frequency: row.frequency,
+        successRate: row.success_rate,
+        commonNextCommand: row.common_next_command
+      },
+      timestamp: new Date(),
+      confidence: row.frequency > 100 ? 0.9 : 0.7
+    }));
+  }
+
+  async extractFromSupport(): Promise<SourceEvidence[]> {
+    // Analyze support tickets with NLP
+    const tickets = await supportSystem.getRecentTickets(30);
+
+    const classified = await Promise.all(
+      tickets.map(async ticket => {
+        const analysis = await this.llm.analyze(`
+          Classify this support ticket:
+
+          Title: ${ticket.title}
+          Description: ${ticket.description}
+          Resolution: ${ticket.resolution}
+
+          Extract:
+          1. What feature/command is this about?
+          2. What was the user's confusion?
+          3. Is this a bug, doc issue, or UX issue?
+          4. What's the root cause?
+        `);
+
+        return {
+          source: 'support',
+          type: analysis.issueType,
+          value: {
+            feature: analysis.feature,
+            confusion: analysis.confusion,
+            rootCause: analysis.rootCause
+          },
+          location: ticket.id,
+          timestamp: ticket.created_at,
+          confidence: 0.6  // Human verification recommended
+        };
+      })
+    );
+
+    return classified;
+  }
+
+  async detectConflicts(): Promise<Conflict[]> {
+    // Query graph for facts with multiple conflicting sources
+    const query = `
+      MATCH (f:Fact)-[:HAS_EVIDENCE]->(e1:Evidence)
+      MATCH (f)-[:HAS_EVIDENCE]->(e2:Evidence)
+      WHERE e1.value <> e2.value
+        AND e1.source <> e2.source
+        AND NOT exists((e1)-[:CONFLICTS_WITH]->(e2))
+      RETURN f, e1, e2
+    `;
+
+    const results = await this.graph.query(query);
+
+    return results.map(({ f, e1, e2 }) => ({
+      id: generateId(),
+      type: this.classifyConflictType(e1, e2),
+      severity: this.assessSeverity(e1, e2, f),
+      sources: [e1, e2],
+      description: this.describeConflict(f, e1, e2),
+      status: 'detected'
+    }));
+  }
+}
+```
+
+### Workflow 2: Human-Driven Resolution
+
+```typescript
+// Interactive dashboard for resolving conflicts and gaps
+class ReconciliationDashboard {
+  async showDashboard() {
+    const conflicts = await this.graph.getUnresolvedConflicts();
+    const gaps = await this.graph.getUnresolvedGaps();
+
+    console.log(`
+┌───────────────────────────────────────────────────────────┐
+│          Kenogami Reconciliation Dashboard                 │
+└───────────────────────────────────────────────────────────┘
+
+📊 Status:
+  ├─ Overall Confidence: ${this.calculateConfidence()}%
+  ├─ Unresolved Conflicts: ${conflicts.length}
+  ├─ Unresolved Gaps: ${gaps.length}
+  └─ Last Sync: ${this.lastSyncTime}
+
+🚨 Critical Conflicts (${conflicts.filter(c => c.severity === 'critical').length}):
+${this.renderConflicts(conflicts.filter(c => c.severity === 'critical'))}
+
+📝 High-Priority Gaps (${gaps.filter(g => g.severity === 'high').length}):
+${this.renderGaps(gaps.filter(g => g.severity === 'high'))}
+
+🔍 Actions:
+  1. Review conflicts
+  2. Review gaps
+  3. Run reconciliation
+  4. Generate report
+  5. Update sources
+
+Choice:
+    `);
+  }
+
+  async reviewConflict(conflict: Conflict) {
+    console.log(`
+┌───────────────────────────────────────────────────────────┐
+│ Conflict: ${conflict.description}                          │
+│ Severity: ${conflict.severity}                            │
+└───────────────────────────────────────────────────────────┘
+
+Source 1: ${conflict.sources[0].source}
+Location: ${conflict.sources[0].location}
+Value: ${JSON.stringify(conflict.sources[0].value, null, 2)}
+Confidence: ${conflict.sources[0].confidence}
+Last Updated: ${conflict.sources[0].timestamp}
+
+Source 2: ${conflict.sources[1].source}
+Location: ${conflict.sources[1].location}
+Value: ${JSON.stringify(conflict.sources[1].value, null, 2)}
+Confidence: ${conflict.sources[1].confidence}
+Last Updated: ${conflict.sources[1].timestamp}
+
+🤖 AI Recommendation: ${await this.getAIRecommendation(conflict)}
+
+🔧 Resolution Options:
+  1. Use Source 1 (${conflict.sources[0].source})
+  2. Use Source 2 (${conflict.sources[1].source})
+  3. Merge both
+  4. Create new canonical value
+  5. Escalate to product team
+  6. Run tests to verify
+
+Choice:
+    `);
+
+    const choice = await this.prompt();
+    return this.applyResolution(conflict, choice);
+  }
+}
+```
+
+### Workflow 3: Feedback Loop from Outputs
+
+```typescript
+// Outputs (docs, tests) feed back findings to the graph
+class OutputFeedbackLoop {
+  async processTestResults(results: TestResult[]) {
+    for (const result of results) {
+      if (!result.passed) {
+        // Test failed - reality doesn't match expectation
+        await this.analyzeTestFailure(result);
+      } else if (result.actualOutput !== result.expectedOutput) {
+        // Test passed but output different - possible drift
+        await this.flagPotentialDrift(result);
+      }
+    }
+  }
+
+  async analyzeTestFailure(result: TestResult) {
+    // Use AI to determine what's wrong
+    const analysis = await this.llm.analyze(`
+      Test failed:
+
+      Test: ${result.testName}
+      Command: ${result.command}
+      Expected: ${result.expectedOutput}
+      Actual: ${result.actualOutput}
+      Error: ${result.error}
+
+      Analyze:
+      1. Is the test wrong (expected output incorrect)?
+      2. Is the code wrong (bug in implementation)?
+      3. Is the documentation wrong (describes non-existent behavior)?
+      4. Is the API spec wrong?
+
+      Provide recommendation.
+    `);
+
+    if (analysis.rootCause === 'documentation_wrong') {
+      // Update graph with correct information from test
+      await this.updateGraph({
+        source: 'test_results',
+        value: result.actualOutput,
+        confidence: 0.9,
+        evidence: result,
+        suggestedAction: 'UPDATE_DOCS'
+      });
+
+      // Create conflict between docs and reality
+      await this.createConflict({
+        type: 'docs_vs_reality',
+        sources: ['documentation', 'test_results'],
+        recommendation: 'Update documentation to match actual behavior'
+      });
+    } else if (analysis.rootCause === 'code_bug') {
+      // Create GitHub issue for bug
+      await this.createGitHubIssue({
+        title: `Bug: ${result.testName} fails`,
+        labels: ['bug', 'test-failure'],
+        body: `Test expects: ${result.expectedOutput}\nActual: ${result.actualOutput}\n\n${analysis.explanation}`
+      });
+    }
+  }
+
+  async processUserFeedback(feedback: UserFeedback) {
+    // User reports: "The docs say X but when I tried it, Y happened"
+    await this.createConflict({
+      type: 'user_reported_mismatch',
+      sources: ['documentation', 'user_feedback'],
+      severity: 'medium',
+      description: feedback.description,
+      evidence: feedback
+    });
+
+    // Track this for analysis
+    await this.graph.createNode('UserReport', {
+      source: 'user_feedback',
+      type: feedback.type,
+      description: feedback.description,
+      timestamp: new Date(),
+      user: feedback.userId
+    });
+  }
+}
+```
+
+---
+
+## Metrics & Monitoring
+
+### Health Metrics
+
+```typescript
+interface SystemHealth {
+  // Confidence metrics
+  overallConfidence: number;  // 0-100%
+  confidenceBySource: {
+    code: number;
+    spec: number;
+    documentation: number;
+    tests: number;
+    usage: number;
+  };
+
+  // Conflict metrics
+  totalConflicts: number;
+  conflictsBySeverity: {
+    critical: number;
+    high: number;
+    medium: number;
+    low: number;
+  };
+  conflictResolutionRate: number;  // % resolved
+  averageTimeToResolve: number;  // hours
+
+  // Gap metrics
+  totalGaps: number;
+  gapsByType: {
+    undocumented: number;
+    unimplemented: number;
+    undertested: number;
+    faq_missing: number;
+  };
+  gapClosureRate: number;  // % closed per week
+
+  // Freshness metrics
+  averageSourceAge: {
+    code: number;  // days since last update
+    documentation: number;
+    tests: number;
+  };
+  staleSourcesCount: number;  // sources older than threshold
+
+  // Coverage metrics
+  documentationCoverage: number;  // % of code features documented
+  testCoverage: number;  // % of documented features tested
+  usageAlignment: number;  // % of documented features actually used
+}
+
+// Alert thresholds
+const ALERTS = {
+  overallConfidence: { critical: 60, warning: 75 },
+  criticalConflicts: { critical: 5, warning: 2 },
+  documentationCoverage: { critical: 70, warning: 85 },
+  staleDocumentation: { critical: 90, warning: 60 }  // days
+};
+```
+
+### Dashboard Visualization
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│              Kenogami System Health Dashboard                │
+└─────────────────────────────────────────────────────────────┘
+
+📊 Overall Health: 87% ████████░░ GOOD
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+🎯 Confidence Scores:
+
+  Code:          95% █████████░  Excellent
+  Spec:          88% ████████░░  Good
+  Documentation: 72% ███████░░░  Needs Attention ⚠️
+  Tests:         91% █████████░  Good
+  Usage Data:    85% ████████░░  Good
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+🚨 Conflicts:
+
+  Total: 23
+    Critical:  2  🔴
+    High:      5  🟠
+    Medium:   12  🟡
+    Low:       4  🟢
+
+  Resolution Rate: 78% (18/23 resolved this week)
+  Avg Time to Resolve: 4.2 hours
+
+  Top Conflicts:
+    1. API spec vs implementation (authentication flow)
+    2. Documentation inconsistency (default output format)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+📝 Gaps:
+
+  Total: 47
+    Critical:   3  🔴  Fix immediately
+    High:      12  🟠  Plan this sprint
+    Medium:    21  🟡  Backlog
+    Low:       11  🟢  Nice to have
+
+  Top Gaps:
+    1. Workflow: Create → Schedule → Send (45 users, 0 docs)
+    2. Parameter --batch not documented (12 support tickets)
+    3. FAQ: "How do I authenticate?" (23 tickets)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+📈 Coverage:
+
+  Documentation: 82% ████████░░  (112/136 commands)
+  Testing:       94% █████████░  (128/136 commands)
+  Usage:         76% ███████░░░  (104/136 commands used)
+
+  Undocumented but used:  8 features  ⚠️
+  Documented but unused: 24 features  ℹ️
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+⏰ Freshness:
+
+  Code:          2 days ago  ✅
+  Documentation: 14 days ago ⚠️
+  Tests:         1 day ago   ✅
+
+  Stale sources: 3 (docs older than 30 days)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+🎬 Actions:
+
+  ⚡ Immediate:
+    • Resolve 2 critical conflicts
+    • Document 3 critical gaps
+
+  📅 This Week:
+    • Update stale documentation (3 articles)
+    • Close 5 high-priority gaps
+    • Review 12 medium conflicts
+
+  🔮 Trends:
+    • Confidence improving (+5% this week) ↗️
+    • Conflicts decreasing (-3 resolved) ↘️
+    • Gaps increasing (+7 detected) ⚠️
+```
+
+---
+
+## Summary: Philosophical Shift
+
+### Old Mental Model: Single Source of Truth
+
+```
+"The code is the truth" ❌
+"The docs are the truth" ❌
+"The spec is the truth" ❌
+```
+
+**Problem:** No single source captures the full truth. Each source has partial truth.
+
+### New Mental Model: Distributed Truth with Reconciliation
+
+```
+"Truth is distributed across multiple authoritative sources,
+each with different authority for different questions.
+The knowledge graph reconciles these truths,
+detects conflicts and gaps,
+and synthesizes a coherent understanding
+that serves all stakeholders."
+```
+
+**Reality:**
+- **Code** tells us what IS implemented
+- **Spec** tells us what SHOULD be implemented
+- **Documentation** tells us what we INTEND users to understand
+- **Usage** tells us what users ACTUALLY do
+- **Support** tells us where users STRUGGLE
+- **Tests** tell us what we've VERIFIED
+- **Chatbot** tells us how users LEARN
+
+**Kenogami's Role:**
+Not to pick one truth, but to:
+1. **Capture** all truths with provenance
+2. **Detect** when truths conflict
+3. **Analyze** why conflicts exist
+4. **Resolve** conflicts using appropriate strategies
+5. **Identify** gaps where truth is missing
+6. **Synthesize** coherent output for each audience
+7. **Evolve** as all sources evolve
+
+**Key Insight:**
+The knowledge graph is not the source of truth—it's the **reconciliation layer** that makes sense of distributed, evolving, sometimes-conflicting truths, maintaining coherence and surfacing problems for human decision-making.
+
+This is fundamentally a **distributed systems problem** applied to knowledge management.