@weavelogic/knowledge-graph-agent 0.2.0 → 0.4.0

package/README.md

@@ -1,20 +1,19 @@
 # @weavelogic/knowledge-graph-agent
 
-A powerful NPM library for creating and managing knowledge graphs for Claude Code. Integrates with Obsidian-style documentation and claude-flow for AI coordination.
+A powerful NPM library for creating and managing knowledge graphs for Claude Code. Integrates with Obsidian-style documentation, claude-flow for AI coordination, and provides enterprise-grade features for production deployments.
 
 [![npm version](https://img.shields.io/npm/v/@weavelogic/knowledge-graph-agent.svg)](https://www.npmjs.com/package/@weavelogic/knowledge-graph-agent)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-## Features
+## Features at a Glance
 
-- **Knowledge Graph Generation** - Automatically generate knowledge graphs from markdown documentation
-- **Documentation Analyzer** - Advanced analyzer that migrates existing docs to structured knowledge graphs
-- **Obsidian-Style Output** - Wikilinks, callouts, frontmatter, and tags following Obsidian conventions
-- **Automatic Categorization** - AI-powered content analysis for intelligent document categorization
-- **Docs Initialization** - Create structured documentation directories following weave-nn methodology
-- **CLAUDE.md Management** - Generate and update CLAUDE.md configuration files
-- **Claude-Flow Integration** - Sync knowledge graphs with claude-flow memory for cross-session persistence
-- **Full-Text Search** - SQLite-backed search across all documentation
+| Phase | Feature Area | Description |
+|-------|--------------|-------------|
+| 1 | Cultivation System | Codebase analysis, seed generation, deep analysis |
+| 2 | Core Features | Agent system, rules engine, workflows, MCP server |
+| 3 | Advanced Integration | Service management, config migrations, health monitoring |
+| 4 | Enterprise Features | Chunking, backup/recovery, advanced caching, diagnostics |
+| 5 | External Integrations | Workflow DevKit, RuVector semantic search, Exochain audit trail |
 
 ## Installation
 
@@ -30,426 +29,1096 @@ npx @weavelogic/knowledge-graph-agent init
 
 ## Quick Start
 
-### 1. Initialize Knowledge Graph
-
 ```bash
-# Initialize in current project
+# Initialize knowledge graph in project
 kg init
 
-# Initialize with custom paths
-kg init --path /path/to/project --docs documentation
+# Bootstrap primitives from codebase analysis
+kg init-primitives
+
+# Generate knowledge graph
+kg graph
+
+# Start MCP server for Claude integration
+kg mcp start
 ```
 
-### 2. Analyze & Migrate Existing Documentation (NEW in v0.2.0)
+---
 
-```bash
-# Analyze docs and create knowledge graph structure
-kg analyze
+## Phase 1: Cultivation System
+
+### SeedGenerator
+
+Analyzes codebases to bootstrap knowledge graphs with primitive nodes.
+
+**Supported Ecosystems:**
+- **Node.js**: `package.json` dependencies
+- **Python**: `requirements.txt`, `pyproject.toml`
+- **PHP**: `composer.json`
+- **Rust**: `Cargo.toml`
+- **Go**: `go.mod`
+- **Java**: `pom.xml`, `build.gradle`
+
+```typescript
+import { SeedGenerator, analyzeSeed, initPrimitives } from '@weavelogic/knowledge-graph-agent';
 
-# Preview what would be created (dry run)
-kg analyze --dry-run --verbose
+// Full analysis
+const analysis = await analyzeSeed('/project', 'docs');
+console.log(`Found ${analysis.frameworks.length} frameworks`);
+console.log(`Found ${analysis.dependencies.length} dependencies`);
+console.log(`Found ${analysis.services.length} services`);
 
+// Generate primitive nodes
+const result = await initPrimitives('/project', 'docs', { dryRun: false });
+console.log(`Generated ${result.documentsGenerated.length} documents`);
+```
+
+### DeepAnalyzer with claude-flow Integration
+
+Advanced codebase analysis using multi-agent orchestration.
+
+```bash
 # Deep analysis with claude-flow agents
 kg analyze deep
 
-# Generate analysis report only
-kg analyze report
+# Generate analysis report
 kg analyze report --json
 ```
 
-### 3. Initialize Documentation Directory
+### Shadow Cache for File Tracking
 
-```bash
-# Create docs with weave-nn structure
-kg docs init
+Tracks file changes efficiently using SHA-256 hashing.
+
+```typescript
+import { ShadowCache } from '@weavelogic/knowledge-graph-agent';
 
-# Create minimal docs
-kg docs init --no-examples
+const cache = new ShadowCache('/project/.kg');
+const changed = await cache.getChangedFiles('/project/docs');
+console.log(`${changed.length} files changed since last sync`);
 ```
 
-### 4. Generate Knowledge Graph
+### CLI Commands (Phase 1)
 
-```bash
-# Full generation
-kg graph
+| Command | Description |
+|---------|-------------|
+| `kg init-primitives` | Bootstrap primitives from codebase analysis |
+| `kg init-primitives --dry-run` | Preview without writing files |
+| `kg analyze-codebase` | Analyze dependencies and services |
+| `kg cultivate` | Run cultivation pipeline |
+| `kg cultivate --deep` | Deep analysis with agents |
 
-# Incremental update
-kg graph --update
-```
+---
 
-### 5. Convert Existing Docs
+## Phase 2: Core Features
 
-```bash
-# Convert docs/ to docs-nn/ with proper structure
-kg convert docs
+### Agent System
+
+A comprehensive agent framework with specialized agents for different tasks.
+
+**Agent Types:**
+- `ResearcherAgent` - Research and information gathering
+- `CoderAgent` - Code generation and modification
+- `TesterAgent` - Test creation and validation
+- `AnalystAgent` - Analysis and recommendations
+- `ArchitectAgent` - System design and architecture
+
+```typescript
+import {
+  BaseAgent,
+  getRegistry,
+  AgentType,
+  createTask
+} from '@weavelogic/knowledge-graph-agent';
+
+// Get the agent registry
+const registry = getRegistry();
 
-# Add frontmatter to existing files
-kg frontmatter add docs/
+// Spawn a researcher agent
+const agent = await registry.spawn(AgentType.RESEARCHER, {
+  name: 'docs-researcher',
+  claudeFlow: { enabled: true, namespace: 'kg' },
+});
+
+// Execute a task
+const task = createTask('Research authentication patterns', {
+  priority: TaskPriority.HIGH,
+  input: { topic: 'OAuth2' },
+});
 
-# Validate frontmatter
-kg frontmatter validate docs/
+const result = await agent.execute(task);
+console.log(result.data);
 ```
 
-### 6. Update CLAUDE.md
+**BaseAgent Features:**
+- Task execution with timeout and retry handling
+- Input validation
+- Output formatting
+- Error handling and logging
+- Claude-flow hooks integration
+- Message passing between agents
 
-```bash
-# Generate CLAUDE.md with knowledge graph integration
-kg claude update
+### RulesEngine for Automation
+
+Define and execute automation rules based on triggers and conditions.
+
+```typescript
+import { RulesEngine, RuleType, TriggerType } from '@weavelogic/knowledge-graph-agent';
+
+const engine = new RulesEngine();
 
-# Use full template with all sections
-kg claude update --template full
+engine.registerRule({
+  id: 'auto-tag',
+  name: 'Auto-tag documentation',
+  type: RuleType.TRANSFORM,
+  trigger: { type: TriggerType.FILE_CHANGE, pattern: 'docs/**/*.md' },
+  condition: (context) => context.file.endsWith('.md'),
+  action: async (context) => { /* auto-tag logic */ },
+});
+
+engine.start();
 ```
 
-### 7. Sync with Claude-Flow
+### Workflow Registry
 
-```bash
-# Sync to claude-flow memory
-kg sync
+Multi-step workflow orchestration with dependencies.
+
+```typescript
+import { WorkflowRegistry } from '@weavelogic/knowledge-graph-agent';
+
+const registry = new WorkflowRegistry();
+
+registry.register({
+  id: 'documentation-sync',
+  name: 'Documentation Sync',
+  steps: [
+    { id: 'analyze', action: 'analyze-docs' },
+    { id: 'generate', action: 'generate-graph', dependsOn: ['analyze'] },
+    { id: 'sync', action: 'sync-memory', dependsOn: ['generate'] },
+  ],
+});
 
-# Show MCP commands for manual sync
-kg sync --show-commands
+await registry.execute('documentation-sync', { projectRoot: '/project' });
 ```
 
-## CLI Commands
+### VaultMemorySync
 
-| Command | Description |
-|---------|-------------|
-| `kg init` | Initialize knowledge graph in project |
-| `kg analyze` | **NEW** Analyze & migrate docs to knowledge graph structure |
-| `kg analyze deep` | **NEW** Deep analysis with claude-flow agents |
-| `kg analyze report` | **NEW** Generate analysis report without creating files |
-| `kg graph` | Generate/update knowledge graph |
-| `kg docs init` | Initialize docs directory |
-| `kg docs status` | Show documentation status |
-| `kg convert docs` | Convert existing docs to weave-nn structure |
-| `kg frontmatter add` | Add frontmatter to markdown files |
-| `kg frontmatter validate` | Validate frontmatter in files |
-| `kg frontmatter update` | Update/regenerate frontmatter |
-| `kg claude update` | Update CLAUDE.md |
-| `kg claude preview` | Preview CLAUDE.md content |
-| `kg sync` | Sync with claude-flow memory |
-| `kg stats` | Display graph statistics |
-| `kg search <query>` | Search the knowledge graph |
+Synchronize knowledge graph with claude-flow memory.
+
+```typescript
+import { VaultMemorySync } from '@weavelogic/knowledge-graph-agent';
+
+const sync = new VaultMemorySync({
+  vaultPath: '/project/docs',
+  namespace: 'knowledge-graph',
+});
+
+await sync.syncAll();
+```
 
-## Documentation Analyzer (v0.2.0)
+### Git Integration
 
-The new analyzer automatically:
+Automatic commit message generation and staged changes analysis.
 
-1. **Categorizes documents** into appropriate directories based on content analysis
-2. **Extracts metadata** including title, type, tags, and key concepts
-3. **Generates frontmatter** with proper weave-nn fields
-4. **Creates wikilinks** between related documents
-5. **Builds MOC files** (Map of Content) for navigation
-6. **Identifies research areas** and TODOs for follow-up
-7. **Links to originals** for reference back to source documentation
+```bash
+# Show status with suggested commit message
+kg commit
+
+# Auto-commit with generated message
+kg commit -a
 
-### Analyzer Output Structure
+# Interactive commit mode
+kg commit -i
 
+# Show recent commits
+kg commit log
 ```
-docs-nn/
-├── MOC.md                    # Master index
-├── PRIMITIVES.md             # Core building blocks
-├── concepts/
-│   ├── _MOC.md              # Category index
-│   └── *.md                 # Concept documents
-├── components/
-│   ├── _MOC.md
-│   └── *.md
-├── services/
-├── features/
-├── integrations/
-├── standards/
-├── guides/
-├── references/
-├── _templates/
-├── _attachments/
-└── _archive/
+
+```typescript
+import { analyzeGitChanges, generateCommitMessage } from '@weavelogic/knowledge-graph-agent';
+
+const changes = await analyzeGitChanges('/project');
+const message = generateCommitMessage(changes);
+console.log(message);
 ```
 
-### Generated Document Format
+### MCP Server with 30+ Tools
 
-Each analyzed document includes:
+Model Context Protocol server for Claude integration.
+
+```bash
+# Start MCP server
+kg mcp start
+
+# Or via npx
+npx @weavelogic/knowledge-graph-agent mcp start
+```
+
+**Available MCP Tools:**
+
+| Category | Tools |
+|----------|-------|
+| Graph | `kg_query`, `kg_stats`, `kg_get_node`, `kg_list_tags` |
+| Cache | `kg_cache_stats` |
+| Agents | `kg_agent_spawn`, `kg_agent_list` |
+| System | `kg_health`, `kg_health_check` |
+| Search | `kg_search_nodes`, `kg_search_tags` |
+| Workflow | `kg_workflow_start`, `kg_workflow_status`, `kg_workflow_list` |
+| Vector | `kg_vector_search`, `kg_vector_upsert` |
+| Trajectory | `kg_trajectory_list` |
+| Audit | `kg_audit_query`, `kg_audit_checkpoint` |
+| Sync | `kg_sync_status` |
 
-```markdown
----
-title: "Document Title"
-type: concept
-status: active
-tags:
-  - tag1
-  - tag2
-category: concepts
-description: "Auto-generated summary"
-created: 2025-01-01
-updated: 2025-01-01
-original: "docs/original-file.md"
-related:
-  - "[[Related Document]]"
 ---
 
-# Document Title
+## Phase 3: Advanced Integration
 
-## Overview
+### ServiceManager
 
-Auto-generated summary...
+Centralized management for background services with health monitoring.
 
-> [!info] Original Documentation
-> See [[docs/original-file.md|original document]] for full details.
+```typescript
+import { createServiceManager, ServiceHandler } from '@weavelogic/knowledge-graph-agent';
+
+const manager = createServiceManager();
+
+// Register a service
+await manager.register(
+  {
+    id: 'file-watcher',
+    name: 'File Watcher',
+    type: 'watcher',
+    autoStart: true,
+    healthCheckInterval: 60000,
+    restartOnFailure: true,
+    maxRestarts: 3,
+  },
+  myWatcherService
+);
 
-## Key Concepts
+// Listen for events
+manager.on('started', (state) => console.log(`${state.name} started`));
+manager.on('healthCheck', ({ id, healthy }) => console.log(`${id}: ${healthy}`));
 
-- **Concept 1**
-- **Concept 2**
+// Graceful shutdown
+process.on('SIGTERM', () => manager.shutdown());
+```
 
-## Related
+### ConfigManager with Migrations
 
-- [[Related Document 1]]
-- [[Related Document 2]]
+Configuration management with version migrations and validation.
 
-## Research Needed
+```typescript
+import { createConfigManager } from '@weavelogic/knowledge-graph-agent';
 
-> [!warning] Areas Requiring Further Research
-> - Area needing investigation
+const config = createConfigManager('/project');
 
-## TODOs
+// Get configuration
+const dbConfig = config.get('database');
+console.log(dbConfig.path);
 
-- [ ] Task to complete
-- [ ] Another task
+// Update configuration
+config.update({
+  cache: { maxSize: 2000, evictionPolicy: 'lfu' },
+});
+
+// Register and run migrations
+config.registerMigration({
+  version: '1.1.0',
+  up: (cfg) => ({ ...cfg, newField: 'value' }),
+  down: (cfg) => { const { newField, ...rest } = cfg; return rest; },
+});
 
-## Tags
+config.migrate('1.1.0');
 
-#tag1 #tag2
+// Validate configuration
+const { valid, errors } = config.validate();
 ```
 
-## Programmatic Usage
+### HealthMonitor
+
+System health monitoring with alerts and history.
 
 ```typescript
-import {
-  createKnowledgeGraph,
-  createDatabase,
-  initDocs,
-  generateClaudeMd,
-  quickInit,
-  // New in v0.2.0
-  analyzeDocs,
-  convertDocs,
-  addFrontmatter,
-} from '@weavelogic/knowledge-graph-agent';
+import { createHealthMonitor, createMemoryCheck, createDatabaseCheck } from '@weavelogic/knowledge-graph-agent';
 
-// Quick initialization
-const result = await quickInit({
-  projectRoot: '/path/to/project',
-  docsPath: 'docs',
-  generateGraph: true,
-  updateClaudeMd: true,
-});
+const monitor = createHealthMonitor({ checkInterval: 30000 });
+
+// Register health checks
+monitor.registerCheck(createMemoryCheck(85)); // Alert at 85% heap usage
+monitor.registerCheck(createDatabaseCheck('/project/.kg/knowledge.db'));
 
-// Analyze and migrate documentation (NEW)
-const analyzeResult = await analyzeDocs({
-  sourceDir: 'docs',
-  targetDir: 'docs-nn',
-  projectRoot: '/path/to/project',
-  useClaudeFlow: false,
-  createMOC: true,
-  linkOriginal: true,
-  dryRun: false,
+// Listen for alerts
+monitor.on('alert', (alert) => {
+  console.error(`[${alert.severity}] ${alert.component}: ${alert.message}`);
 });
 
-console.log(`Analyzed: ${analyzeResult.filesAnalyzed}`);
-console.log(`Created: ${analyzeResult.filesCreated}`);
-console.log(`MOC files: ${analyzeResult.mocFilesCreated}`);
+// Start monitoring
+monitor.start();
+
+// Get current health
+const health = await monitor.runAllChecks();
+console.log(`System status: ${health.status}`);
+console.log(`Uptime: ${health.uptime}ms`);
+```
+
+### DecisionTracker for Explainability
 
-// Convert existing docs
-const convertResult = await convertDocs({
-  sourceDir: 'docs',
-  targetDir: 'docs-nn',
-  projectRoot: '/path/to/project',
-  autoCategory: true,
+Track and explain AI reasoning and decisions.
+
+```typescript
+import { createDecisionTracker } from '@weavelogic/knowledge-graph-agent';
+
+const tracker = createDecisionTracker();
+
+// Start a reasoning chain
+const chainId = tracker.startChain('Process user query');
+
+// Record decisions
+const decision = tracker.recordDecision({
+  action: 'Extract entities from query',
+  reasoning: [
+    'Query contains multiple noun phrases',
+    'NLP analysis identified 3 potential entities',
+    'Confidence scores above threshold',
+  ],
+  confidence: 'high',
+  context: {
+    trigger: 'user_query',
+    inputs: { query: 'Tell me about React and TypeScript' },
+    constraints: ['max 10 entities'],
+    alternatives: [],
+  },
+});
+
+// Record outcome
+tracker.recordOutcome(decision.id, {
+  success: true,
+  result: { entities: ['React', 'TypeScript'] },
+  duration: 150,
+  sideEffects: [],
 });
 
-// Add frontmatter to files
-const fmResult = await addFrontmatter({
-  target: 'docs',
-  projectRoot: '/path/to/project',
-  type: 'concept',
-  status: 'active',
+// Get explanation
+console.log(tracker.explainDecision(decision.id));
+
+// End chain
+tracker.endChain(chainId);
+
+// Get statistics
+const stats = tracker.getStats();
+console.log(`Success rate: ${stats.successRate * 100}%`);
+```
+
+---
+
+## Phase 4: Enterprise Features
+
+### Chunker for Large Documents
+
+Split large documents into manageable chunks with multiple strategies.
+
+**Strategies:**
+- `fixed` - Character-based chunking with overlap
+- `semantic` - Paragraph-based chunking
+- `markdown` - Respects markdown structure (headers, sections)
+- `code` - Preserves code blocks as atomic units
+- `adaptive` - Auto-selects best strategy based on content
+
+```typescript
+import { createChunker, chunkDocument } from '@weavelogic/knowledge-graph-agent';
+
+// Quick usage
+const result = chunkDocument(largeMarkdownContent, {
+  strategy: 'adaptive',
+  maxSize: 2000,
+  overlap: 50,
 });
 
-// Or use individual components
-const db = createDatabase('.kg/knowledge.db');
-const graph = createKnowledgeGraph('my-project', '/project/root');
+console.log(`Split into ${result.chunks.length} chunks`);
+console.log(`Processing time: ${result.processingTime}ms`);
 
-// Generate docs
-await initDocs({
-  projectRoot: '/project/root',
-  docsPath: 'docs',
-  includeExamples: true,
+// With configuration
+const chunker = createChunker({
+  defaultStrategy: 'markdown',
+  defaultMaxSize: 4000,
 });
 
-// Generate CLAUDE.md
-const claudeContent = generateClaudeMd({
-  projectRoot: '/project/root',
-  includeKnowledgeGraph: true,
-  includeClaudeFlow: true,
+const chunks = chunker.chunk(content);
+for (const chunk of chunks) {
+  console.log(`Chunk ${chunk.id}: ${chunk.metadata.wordCount} words`);
+}
+```
+
+### BackupManager with Compression
+
+Automatic backup with gzip compression and checksums.
+
+```typescript
+import { createBackupManager } from '@weavelogic/knowledge-graph-agent';
+
+const backup = createBackupManager('/project/.kg/knowledge.db', {
+  enabled: true,
+  interval: 86400000, // 24 hours
+  maxBackups: 5,
+  compress: true,
+  backupPath: '/project/.kg/backups',
 });
+
+// Create backup
+const info = await backup.createBackup();
+console.log(`Backup created: ${info.id}`);
+console.log(`Size: ${info.size} bytes`);
+console.log(`Checksum: ${info.checksum}`);
+
+// List backups
+const backups = backup.listBackups();
+for (const b of backups) {
+  console.log(`${b.id}: ${b.timestamp.toISOString()}`);
+}
+
+// Restore from backup
+await backup.restore(backups[0].id);
+
+// Start automatic backups
+backup.startAutoBackup();
 ```
 
-## Documentation Structure
+### IntegrityChecker for Database Validation
+
+Comprehensive database integrity checking and repair.
+
+```typescript
+import { createIntegrityChecker, checkDatabaseIntegrity } from '@weavelogic/knowledge-graph-agent';
+
+// Quick check
+const result = checkDatabaseIntegrity('/project/.kg/knowledge.db');
+console.log(`Valid: ${result.valid}`);
+console.log(`Errors: ${result.errors.length}`);
+console.log(`Warnings: ${result.warnings.length}`);
+
+// Detailed check with repair
+const checker = createIntegrityChecker('/project/.kg/knowledge.db');
+const check = checker.check();
+
+for (const table of check.tables) {
+  console.log(`${table.name}: ${table.rowCount} rows, valid: ${table.valid}`);
+  if (!table.valid) {
+    console.log(`  Issues: ${table.issues.join(', ')}`);
+  }
+}
 
-The generated docs directory follows the weave-nn methodology:
+// Repair database
+if (!check.valid) {
+  const repaired = checker.repair();
+  console.log(`Repair ${repaired ? 'successful' : 'failed'}`);
+}
 
+// Get database stats
+const stats = checker.getStats();
+console.log(`Pages: ${stats.pageCount}, Free: ${stats.freePages}`);
+console.log(`WAL mode: ${stats.walMode}`);
 ```
-docs/
-├── README.md           # Documentation home
-├── PRIMITIVES.md       # Technology primitives
-├── MOC.md              # Map of Content
-├── concepts/           # Abstract concepts
-│   ├── architecture/
-│   └── patterns/
-├── components/         # Reusable components
-│   ├── ui/
-│   └── utilities/
-├── services/           # Backend services
-│   ├── api/
-│   └── workers/
-├── features/           # Product features
-│   ├── core/
-│   └── advanced/
-├── integrations/       # External integrations
-│   ├── databases/
-│   ├── auth-providers/
-│   └── storage/
-├── standards/          # Coding standards
-│   ├── coding-standards/
-│   └── build-tools/
-├── guides/             # How-to guides
-│   ├── getting-started/
-│   └── testing/
-└── references/         # API references
-    ├── api/
-    └── cli/
+
+### AdvancedCache with Multiple Eviction Policies
+
+High-performance caching with LRU, LFU, FIFO, and TTL eviction.
+
+```typescript
+import { createAdvancedCache } from '@weavelogic/knowledge-graph-agent';
+
+const cache = createAdvancedCache<User>({
+  maxSize: 50 * 1024 * 1024, // 50MB
+  maxEntries: 1000,
+  defaultTTL: 300000, // 5 minutes
+  evictionPolicy: 'lru', // 'lru' | 'lfu' | 'fifo' | 'ttl'
+  onEvict: (key, value) => console.log(`Evicted: ${key}`),
+});
+
+// Basic operations
+cache.set('user:123', userData, { tags: ['user', 'active'] });
+const user = cache.get<User>('user:123');
+
+// Tag-based operations
+const activeUsers = cache.getByTag('active');
+cache.deleteByTag('user');
+
+// Get or set with factory
+const result = await cache.getOrSet('expensive:key', async () => {
+  return await expensiveComputation();
+}, { ttl: 60000 });
+
+// Statistics
+const stats = cache.getStats();
+console.log(`Hit rate: ${(stats.hitRate * 100).toFixed(1)}%`);
+console.log(`Evictions: ${stats.evictions}`);
+
+// Change policy at runtime
+cache.setEvictionPolicy('lfu');
 ```
 
-## Knowledge Graph Schema
+### Diagnostics CLI
+
+Comprehensive troubleshooting tools.
+
+```bash
+# Run full diagnostics
+kg diag run
+
+# Attempt to fix issues
+kg diag run --fix
+
+# Check system health
+kg diag health
+
+# Repair database issues
+kg diag repair
 
-### Node Types
+# Create database backup
+kg diag backup
 
-| Type | Description |
-|------|-------------|
-| `concept` | Abstract concepts and ideas |
-| `technical` | Technical components and implementations |
-| `feature` | Product features and capabilities |
-| `primitive` | Base technology primitives |
-| `service` | Backend services and APIs |
-| `guide` | How-to guides and tutorials |
-| `standard` | Coding standards and conventions |
-| `integration` | External integrations |
+# List existing backups
+kg diag backup --list
 
-### Node Frontmatter
+# Show system information
+kg diag info
+```
 
-```yaml
 ---
-title: My Document
-type: concept
-status: active
-tags: [concept, architecture]
-created: 2025-01-01
-updated: 2025-01-01
-aliases: [Alternative Name]
-related: [other-document]
-category: concepts
-description: Brief description
-original: docs/source-file.md
+
+## Phase 5: External Integrations
+
+### Workflow DevKit Integration (useworkflow.dev)
+
+Integration with the Workflow DevKit for durable workflow orchestration with Postgres persistence.
+
+**Key Concepts:**
+- **Postgres World**: Durable state persistence with transactional guarantees
+- **"use workflow" / "use step" directives**: Declarative workflow definition
+- **DurableAgent**: Agents that survive restarts and can resume from any step
+- **GOAP Planning**: Goal-Oriented Action Planning for intelligent task decomposition
+
+```typescript
+import { WorkflowManager } from '@weavelogic/knowledge-graph-agent';
+
+const workflow = new WorkflowManager({
+  postgres: process.env.DATABASE_URL,
+  durableAgents: true,
+});
+
+// Start a durable workflow
+const execution = await workflow.start('document-analysis', {
+  input: { path: '/docs' },
+  resumable: true,
+});
+
+// Check workflow status
+const status = await workflow.getStatus(execution.id);
+console.log(`Step ${status.currentStep}/${status.totalSteps}: ${status.state}`);
+
+// List active workflows
+const active = await workflow.list({ status: 'running' });
+```
+
+**CLI Commands:**
+
+| Command | Description |
+|---------|-------------|
+| `kg workflow start <name>` | Start a named workflow |
+| `kg workflow status <id>` | Get workflow execution status |
+| `kg workflow list` | List active workflow executions |
+| `kg workflow resume <id>` | Resume a paused workflow |
+| `kg workflow cancel <id>` | Cancel a running workflow |
+| `kg workflow history <name>` | Show workflow execution history |
+| `kg workflow steps <name>` | List steps in a workflow definition |
+
+**MCP Tools:**
+- `kg_workflow_start` - Start a workflow execution
+- `kg_workflow_status` - Get execution status and progress
+- `kg_workflow_list` - List workflows with filtering
+
 ---
+
+### RuVector Integration
+
+High-performance semantic vector search with HNSW indexing and trajectory tracking for self-learning agents.
+
+**Key Features:**
+- **HNSW Approximate Nearest Neighbor**: Sub-millisecond search on millions of vectors
+- **Hybrid Search**: Combines vector similarity with keyword matching
+- **Trajectory Tracking**: Records decision paths for experience replay
+- **Self-Learning**: Agents improve through trajectory analysis
+
+```typescript
+import { VectorStore, TrajectoryTracker } from '@weavelogic/knowledge-graph-agent';
+
+// Initialize vector store
+const vectors = new VectorStore({
+  dimensions: 1536,
+  metric: 'cosine',
+  hnswConfig: { m: 16, efConstruction: 200 },
+});
+
+// Upsert vectors with metadata
+await vectors.upsert([
+  { id: 'doc-1', values: embedding, metadata: { type: 'concept', tags: ['api'] } },
+]);
+
+// Semantic search
+const results = await vectors.search(queryEmbedding, {
+  topK: 10,
+  filter: { type: 'concept' },
+  includeMetadata: true,
+});
+
+// Hybrid search (vector + keyword)
+const hybridResults = await vectors.hybridSearch({
+  vector: queryEmbedding,
+  keywords: ['authentication', 'oauth'],
+  alpha: 0.7, // Weight: 70% vector, 30% keyword
+});
+
+// Trajectory tracking for self-learning
+const tracker = new TrajectoryTracker();
+tracker.recordStep({
+  state: { context: 'user-query' },
+  action: 'search-knowledge-graph',
+  reward: 0.8,
+  nextState: { context: 'results-found' },
+});
+
+const trajectories = tracker.list({ minReward: 0.7 });
 ```
 
-## Claude-Flow Integration
+**CLI Commands:**
 
-The knowledge graph integrates with claude-flow for:
+| Command | Description |
+|---------|-------------|
+| `kg vector search <query>` | Semantic search across knowledge graph |
+| `kg vector upsert <file>` | Add/update vectors from file |
+| `kg vector delete <ids>` | Delete vectors by ID |
+| `kg vector stats` | Show vector index statistics |
+| `kg trajectory list` | List recorded trajectories |
+| `kg trajectory analyze` | Analyze trajectory patterns |
+| `kg trajectory export` | Export trajectories for training |
+| `kg trajectory prune` | Remove low-value trajectories |
+
+**MCP Tools:**
+- `kg_vector_search` - Semantic search with filters
+- `kg_vector_upsert` - Add or update vectors
+- `kg_trajectory_list` - List agent trajectories
 
-- **Memory Persistence**: Store graph data in claude-flow memory
-- **Cross-Session Context**: Access knowledge across Claude sessions
-- **Agent Coordination**: Share knowledge between swarm agents
-- **Deep Analysis**: Use claude-flow agents for comprehensive documentation analysis
+---
 
-### MCP Commands
+### Exochain Integration
 
-```javascript
-// Store node in memory
-mcp__claude-flow__memory_usage {
-  action: "store",
-  key: "node/<id>",
-  namespace: "knowledge-graph",
-  value: { ... }
-}
+Cryptographically verifiable audit trail with DAG-BFT consensus for distributed knowledge graph operations.
 
-// Retrieve node
-mcp__claude-flow__memory_usage {
-  action: "retrieve",
-  key: "node/<id>",
-  namespace: "knowledge-graph"
-}
+**Key Features:**
+- **DAG-BFT Consensus**: Byzantine Fault Tolerant ordering without leader
+- **Hybrid Logical Clocks (HLC)**: Causally consistent timestamps
+- **BLAKE3 Hashing**: Fast cryptographic content addressing
+- **Ed25519 Signatures**: Agent-signed operations
+- **Syndication**: Replicate audit logs across nodes
 
-// Search nodes
-mcp__claude-flow__memory_search {
-  pattern: "node/*",
-  namespace: "knowledge-graph"
-}
+```typescript
+import { AuditTrail, SyncManager } from '@weavelogic/knowledge-graph-agent';
+
+// Initialize audit trail
+const audit = new AuditTrail({
+  signingKey: process.env.AGENT_PRIVATE_KEY,
+  hlcEnabled: true,
+  dagBftQuorum: 3,
+});
+
+// Record an audited operation
+const checkpoint = await audit.checkpoint({
+  operation: 'node_created',
+  nodeId: 'concept-123',
+  agent: 'coder-agent',
+  data: { title: 'Authentication Flow' },
+});
+
+console.log(`Checkpoint: ${checkpoint.hash}`);
+console.log(`HLC: ${checkpoint.hlc}`);
+console.log(`Signature: ${checkpoint.signature}`);
+
+// Query audit trail
+const history = await audit.query({
+  nodeId: 'concept-123',
+  since: '2024-01-01',
+  operations: ['node_created', 'node_updated'],
+});
+
+// Sync with remote nodes
+const sync = new SyncManager({
+  peers: ['node1.example.com', 'node2.example.com'],
+  syndication: true,
+});
+
+const status = await sync.getStatus();
+console.log(`Synced: ${status.lastSyncedHash}`);
+console.log(`Peers: ${status.connectedPeers}/${status.totalPeers}`);
 ```
 
+**CLI Commands:**
+
+| Command | Description |
+|---------|-------------|
+| `kg audit query` | Query audit trail with filters |
+| `kg audit checkpoint` | Create a manual checkpoint |
+| `kg audit verify <hash>` | Verify checkpoint integrity |
+| `kg audit export` | Export audit log |
+| `kg audit stats` | Show audit trail statistics |
+| `kg sync status` | Show synchronization status |
+| `kg sync peers` | List connected peers |
+| `kg sync force` | Force synchronization |
+
+**MCP Tools:**
+- `kg_audit_query` - Query audit trail
+- `kg_audit_checkpoint` - Create checkpoint
+- `kg_sync_status` - Get sync status
+
+---
+
+## CLI Commands Reference
+
+### Initialization
+
+| Command | Description |
+|---------|-------------|
+| `kg init` | Initialize knowledge graph in project |
+| `kg init-primitives` | Bootstrap primitives from codebase |
+| `kg init-primitives --dry-run` | Preview without writing files |
+| `kg analyze-codebase` | Analyze dependencies and services |
+
+### Knowledge Graph
+
+| Command | Description |
+|---------|-------------|
+| `kg graph` | Generate/update knowledge graph |
+| `kg graph --update` | Incremental update |
+| `kg stats` | Display graph statistics |
+| `kg search <query>` | Search the knowledge graph |
+
+### Documentation
+
+| Command | Description |
+|---------|-------------|
+| `kg docs init` | Initialize docs directory |
+| `kg docs status` | Show documentation status |
+| `kg analyze` | Analyze & migrate to knowledge graph |
+| `kg analyze deep` | Deep analysis with claude-flow |
+| `kg analyze report` | Generate analysis report |
+| `kg convert docs` | Convert docs/ to docs-nn/ |
+
+### Frontmatter
+
+| Command | Description |
+|---------|-------------|
+| `kg frontmatter add` | Add frontmatter to markdown files |
+| `kg frontmatter validate` | Validate frontmatter |
+| `kg frontmatter update` | Update/regenerate frontmatter |
+
+### SOP Compliance
+
+| Command | Description |
+|---------|-------------|
+| `kg sop init` | Initialize SOP standards layer |
+| `kg sop check` | Check compliance against SOPs |
+| `kg sop gaps` | Analyze compliance gaps |
+| `kg sop report` | Generate compliance report |
+| `kg sop list` | List available SOPs |
+
+### Claude Integration
+
+| Command | Description |
+|---------|-------------|
+| `kg claude update` | Update CLAUDE.md |
+| `kg claude preview` | Preview CLAUDE.md content |
+| `kg sync` | Sync with claude-flow memory |
+
+### Git Integration
+
+| Command | Description |
+|---------|-------------|
+| `kg commit` | Show status with suggested message |
+| `kg commit -a` | Auto-commit with generated message |
+| `kg commit -i` | Interactive commit mode |
+| `kg commit -m "msg"` | Commit with custom message |
+| `kg commit status` | Show git status with analysis |
+| `kg commit log` | Show recent commits |
+| `kg commit diff` | Show diff of changes |
+
+### Configuration
+
+| Command | Description |
+|---------|-------------|
+| `kg config show` | Show current configuration |
+| `kg config set <key> <value>` | Set configuration value |
+| `kg config reset` | Reset to defaults |
+
+### Diagnostics
+
+| Command | Description |
+|---------|-------------|
+| `kg diag run` | Run full diagnostics |
+| `kg diag run --fix` | Attempt to fix issues |
+| `kg diag health` | Check system health |
+| `kg diag repair` | Repair database issues |
+| `kg diag backup` | Create database backup |
+| `kg diag backup --list` | List existing backups |
+| `kg diag info` | Show system information |
+
+### Workflow (Phase 5)
+
+| Command | Description |
+|---------|-------------|
+| `kg workflow start <name>` | Start a named workflow |
+| `kg workflow status <id>` | Get workflow execution status |
+| `kg workflow list` | List active workflow executions |
+| `kg workflow resume <id>` | Resume a paused workflow |
+| `kg workflow cancel <id>` | Cancel a running workflow |
+| `kg workflow history <name>` | Show workflow execution history |
+| `kg workflow steps <name>` | List steps in a workflow definition |
+
+### Vector Search (Phase 5)
+
+| Command | Description |
+|---------|-------------|
+| `kg vector search <query>` | Semantic search across knowledge graph |
+| `kg vector upsert <file>` | Add/update vectors from file |
+| `kg vector delete <ids>` | Delete vectors by ID |
+| `kg vector stats` | Show vector index statistics |
+
+### Trajectory (Phase 5)
+
+| Command | Description |
+|---------|-------------|
+| `kg trajectory list` | List recorded trajectories |
+| `kg trajectory analyze` | Analyze trajectory patterns |
+| `kg trajectory export` | Export trajectories for training |
+| `kg trajectory prune` | Remove low-value trajectories |
+
+### Audit Trail (Phase 5)
+
+| Command | Description |
+|---------|-------------|
+| `kg audit query` | Query audit trail with filters |
+| `kg audit checkpoint` | Create a manual checkpoint |
+| `kg audit verify <hash>` | Verify checkpoint integrity |
+| `kg audit export` | Export audit log |
+| `kg audit stats` | Show audit trail statistics |
+
+### Synchronization (Phase 5)
+
+| Command | Description |
+|---------|-------------|
+| `kg sync status` | Show synchronization status |
+| `kg sync peers` | List connected peers |
+| `kg sync force` | Force synchronization |
+
+---
+
 ## Configuration
 
 Create `.kg/config.json` to customize behavior:
 
 ```json
 {
+  "version": "1.0.0",
   "projectRoot": ".",
-  "docsRoot": "docs",
-  "graph": {
-    "includePatterns": ["**/*.md"],
-    "excludePatterns": ["node_modules/**", "dist/**"]
-  },
+  "docsPath": "docs",
   "database": {
-    "path": ".kg/knowledge.db"
+    "path": ".kg/knowledge.db",
+    "autoBackup": true,
+    "backupInterval": 86400000,
+    "maxBackups": 5
   },
-  "claudeFlow": {
+  "cache": {
     "enabled": true,
-    "namespace": "knowledge-graph",
-    "syncOnChange": true
+    "maxSize": 1000,
+    "ttl": 3600000,
+    "evictionPolicy": "lru"
+  },
+  "agents": {
+    "maxConcurrent": 5,
+    "defaultTimeout": 30000,
+    "retryAttempts": 3,
+    "claudeFlowEnabled": true
+  },
+  "services": {
+    "watcherEnabled": true,
+    "schedulerEnabled": true,
+    "syncEnabled": true,
+    "healthCheckInterval": 60000
+  },
+  "logging": {
+    "level": "info",
+    "format": "text"
   },
   "analyzer": {
     "targetDir": "docs-nn",
     "createMOC": true,
     "linkOriginal": true
+  },
+  "claudeFlow": {
+    "enabled": true,
+    "namespace": "knowledge-graph",
+    "syncOnChange": true
   }
 }
 ```
 
-## Changelog
+---
+
+## Programmatic Usage
+
+```typescript
+import {
+  // Core
+  createKnowledgeGraph,
+  createDatabase,
+
+  // Initialization
+  quickInit,
+  initDocs,
+  generateClaudeMd,
 
-### v0.2.0 (2025-12-29)
+  // Analysis
+  analyzeDocs,
+  convertDocs,
+  addFrontmatter,
+  analyzeSeed,
+  initPrimitives,
+
+  // Agents
+  BaseAgent,
+  getRegistry,
+  AgentType,
+  createTask,
+
+  // Services
+  createServiceManager,
+  createConfigManager,
+  createHealthMonitor,
+
+  // Enterprise
+  createChunker,
+  createBackupManager,
+  createIntegrityChecker,
+  createAdvancedCache,
+
+  // Reasoning
+  createDecisionTracker,
+
+  // Phase 5 - Workflow DevKit
+  WorkflowManager,
+  DurableAgent,
+  GOAPPlanner,
+
+  // Phase 5 - RuVector
+  VectorStore,
+  TrajectoryTracker,
+
+  // Phase 5 - Exochain
+  AuditTrail,
+  SyncManager,
+
+  // Types
+  type KGConfiguration,
+  type AgentConfig,
+  type ChunkOptions,
+  type WorkflowExecution,
+  type VectorSearchResult,
+  type Trajectory,
+  type AuditCheckpoint,
+} from '@weavelogic/knowledge-graph-agent';
+```
 
-- **NEW**: `kg analyze` command for advanced documentation analysis and migration
-- **NEW**: `kg analyze deep` for claude-flow powered deep analysis
-- **NEW**: `kg analyze report` for generating analysis reports
-- **NEW**: `kg convert docs` for converting existing documentation
-- **NEW**: `kg frontmatter add/validate/update` commands
-- **NEW**: Automatic categorization based on content analysis
-- **NEW**: MOC (Map of Content) generation for navigation
-- **NEW**: Obsidian-style callouts (`> [!info]`, `> [!warning]`)
-- **NEW**: Research area detection and TODO extraction
-- **NEW**: Cross-reference building between related documents
-- **IMPROVED**: Enhanced frontmatter generation with more fields
+---
 
-### v0.1.0 (2025-12-28)
+## Changelog
 
-- Initial release
-- Knowledge graph generation
+### v0.5.0 (Phase 5 - External Integrations)
+
+- **NEW**: Workflow DevKit integration with MCP tools and CLI commands
+  - `WorkflowManager` with Postgres World persistence
+  - DurableAgent support for resumable workflows
+  - GOAP planning for intelligent task decomposition
+- **NEW**: RuVector semantic search with trajectory tracking
+  - `VectorStore` with HNSW approximate nearest neighbor search
+  - Hybrid search combining vector similarity and keyword matching
+  - `TrajectoryTracker` for self-learning agents
+- **NEW**: Exochain audit trail with DAG-BFT consensus
+  - `AuditTrail` with Hybrid Logical Clocks (HLC)
+  - BLAKE3 hashing and Ed25519 signatures
+  - `SyncManager` for peer syndication
+- **NEW**: 9 new MCP tools (`kg_workflow_start`, `kg_workflow_status`, `kg_workflow_list`, `kg_vector_search`, `kg_vector_upsert`, `kg_trajectory_list`, `kg_audit_query`, `kg_audit_checkpoint`, `kg_sync_status`)
+- **NEW**: 22 new CLI subcommands across `workflow`, `vector`, `trajectory`, `audit`, and `sync` command groups
+- **NEW**: 249 new integration tests for external integrations
+
+### v0.4.0 (Phase 4 - Enterprise Features)
+
+- **NEW**: `Chunker` with 5 chunking strategies (fixed, semantic, markdown, code, adaptive)
+- **NEW**: `BackupManager` with gzip compression and SHA-256 checksums
+- **NEW**: `IntegrityChecker` for database validation and repair
+- **NEW**: `AdvancedCache` with LRU/LFU/FIFO/TTL eviction policies
+- **NEW**: `kg diag` CLI commands for diagnostics and troubleshooting
+- **NEW**: Tag-based cache operations
+- **NEW**: Automatic backup scheduling
+
+### v0.3.0 (Phase 3 - Advanced Integration)
+
+- **NEW**: `ServiceManager` for background process management
+- **NEW**: `ConfigManager` with migration support and validation
+- **NEW**: `HealthMonitor` with component health checks and alerts
+- **NEW**: `DecisionTracker` for AI reasoning explainability
+- **NEW**: `kg config` CLI commands
+- **IMPROVED**: Health check history and performance metrics
+
+### v0.2.0 (Phase 2 - Core Features)
+
+- **NEW**: Agent system with BaseAgent and specialized agents
+- **NEW**: `RulesEngine` for automation
+- **NEW**: `WorkflowRegistry` for multi-step workflows
+- **NEW**: `VaultMemorySync` for memory synchronization
+- **NEW**: Git integration with auto-commit
+- **NEW**: MCP Server with 20+ tools
+- **NEW**: `kg commit` CLI commands
+
+### v0.1.0 (Phase 1 - Cultivation System)
+
+- **NEW**: `SeedGenerator` for codebase analysis
+- **NEW**: `DeepAnalyzer` with claude-flow integration
+- **NEW**: `init-primitives` and `cultivate` CLI commands
+- **NEW**: Shadow Cache for file change tracking
+- Initial knowledge graph generation
 - Docs initialization
 - CLAUDE.md management
-- Claude-flow integration
 - Full-text search
 
+---
+
+## Roadmap
+
+### Completed Features
+
+- [x] Real-time collaboration support (implemented via webhook handlers in v0.5.0)
+- [x] Webhook integrations (implemented in v0.5.0)
+
+### Planned Features
+
+- [ ] Plugin system for custom analyzers
+- [ ] GraphQL API for queries
+- [ ] Web dashboard for visualization
+- [ ] Distributed caching with Redis
+
+### Deprecated Features
+
+- None currently
+
+---
+
 ## API Reference
 
 See [docs/API.md](./docs/API.md) for detailed API documentation.