musubix 1.4.4 → 1.7.0

package/docs/USER-GUIDE.md

@@ -15,10 +15,16 @@
 7. [C4 Code Generation](#c4-code-generation)
 8. [Symbolic Reasoning](#symbolic-reasoning) *(v1.2.0)*
 9. [Consistency Validation](#consistency-validation) *(v1.4.1)*
-10. [MCP Server Integration](#mcp-server-integration)
-11. [YATA Integration](#yata-integration)
-12. [Best Practices](#best-practices)
-13. [Troubleshooting](#troubleshooting)
+10. [Advanced Inference](#advanced-inference) *(v1.4.5)*
+11. [Interactive REPL Mode](#interactive-repl-mode) *(v1.5.0)*
+12. [YATA Local](#yata-local) *(v1.6.3)*
+13. [YATA Global](#yata-global) *(v1.6.3)*
+14. [KGPR - Knowledge Graph Pull Request](#kgpr---knowledge-graph-pull-request) *(v1.6.4)*
+15. [YATA Platform Enhancements](#yata-platform-enhancements) *(v1.7.0)*
+16. [MCP Server Integration](#mcp-server-integration)
+17. [YATA Integration](#yata-integration)
+18. [Best Practices](#best-practices)
+19. [Troubleshooting](#troubleshooting)
 
 ---
 
@@ -513,6 +519,580 @@ export function createAuthService(): IAuthService {
 
 ---
 
+## Interactive REPL Mode
+
+*(New in v1.5.0, Enhanced in v1.6.0)*
+
+MUSUBIX provides an interactive REPL (Read-Eval-Print-Loop) shell for real-time command execution and exploration.
+
+### Starting REPL
+
+```bash
+# Start interactive REPL
+musubix repl
+
+# With custom history file
+musubix repl --history ~/.musubix-repl-history
+
+# Without colors
+musubix repl --no-color
+```
+
+### REPL Features
+
+| Feature | Description |
+|---------|-------------|
+| Command completion | TAB completion for commands, options |
+| History navigation | UP/DOWN arrows, history search |
+| Session variables | `$name=value` to set, `$name` to reference |
+| Output formatting | JSON, YAML, Table auto-formatting |
+| CLI integration | Execute any CLI command directly |
+
+### Basic Usage
+
+```bash
+musubix> help                          # Show all commands
+musubix> help requirements             # Show command details
+musubix> requirements analyze input.md # Execute CLI command
+musubix> $project=my-app               # Set session variable
+musubix> design generate $project      # Use variable in command
+musubix> history                       # Show command history
+musubix> exit                          # Exit REPL
+```
+
+### Session Variables
+
+```bash
+# Set variables
+musubix> $req=REQ-001
+musubix> $file=./docs/requirements.md
+
+# Use in commands
+musubix> requirements validate $file
+musubix> trace impact $req
+
+# Special variable: $_ holds last result
+musubix> requirements analyze input.md
+musubix> $_                           # Access last result
+```
+
+### Output Formats
+
+```bash
+# Auto-detect best format (default)
+musubix> learn status
+
+# Force JSON output
+musubix> set format json
+musubix> learn patterns
+
+# Force YAML output
+musubix> set format yaml
+
+# Force table output
+musubix> set format table
+```
+
+### History Management
+
+```bash
+# Show recent commands
+musubix> history
+
+# Search history (Ctrl+R style)
+musubix> history search requirements
+
+# Clear history
+musubix> history clear
+```
+
+### REPL Components
+
+| Component | Purpose |
+|-----------|---------|
+| `ReplEngine` | Main REPL controller |
+| `CommandCompleter` | TAB completion provider |
+| `HistoryManager` | Command history persistence |
+| `SessionState` | Variable storage |
+| `OutputFormatter` | JSON/YAML/Table output |
+| `PromptRenderer` | Dynamic prompt display |
+
+---
+
+## YATA Local
+
+*(New in v1.6.3)*
+
+YATA Local provides a high-performance, SQLite-based local knowledge graph with built-in inference capabilities. Designed for single-user, offline scenarios where data sovereignty and speed are critical.
+
+### Features
+
+| Feature | Description |
+|---------|-------------|
+| **SQLite Storage** | WAL mode for concurrent reads, single-writer |
+| **Full-Text Search** | FTS5-based triple search |
+| **Graph Traversal** | BFS/DFS with depth control |
+| **Inference Engine** | 4 OWL-lite rules (transitivity, symmetry, inverse, domain/range) |
+| **Constraints** | 4 validation rules (cardinality, disjoint, unique, required) |
+| **ACID Transactions** | Full transaction support |
+
+### Installation
+
+```bash
+npm install @nahisaho/yata-local
+```
+
+### Quick Start
+
+```typescript
+import { YataLocal } from '@nahisaho/yata-local';
+
+// Initialize with default config
+const yata = new YataLocal('./knowledge.db');
+await yata.initialize();
+
+// Add triples
+await yata.addTriple({
+  subject: 'Person:john',
+  predicate: 'hasParent',
+  object: 'Person:mary'
+});
+
+// Query triples
+const results = await yata.query({
+  subject: 'Person:john',
+  predicate: 'hasParent'
+});
+
+// Full-text search
+const searchResults = await yata.search('john parent');
+
+// Graph traversal (BFS)
+const ancestors = await yata.traverse('Person:john', 'hasParent', {
+  direction: 'outgoing',
+  maxDepth: 5,
+  algorithm: 'bfs'
+});
+
+// Clean up
+await yata.close();
+```
+
+### Inference Engine
+
+YATA Local supports four OWL-lite inference rules:
+
+| Rule | Description | Example |
+|------|-------------|---------|
+| **Transitivity** | If A→B and B→C then A→C | hasAncestor is transitive |
+| **Symmetry** | If A→B then B→A | friendOf is symmetric |
+| **Inverse** | If A→B via P then B→A via P⁻¹ | hasChild ↔ hasParent |
+| **Domain/Range** | Type inference from predicate | hasAge implies Person |
+
+```typescript
+// Run inference
+const inferred = await yata.infer();
+console.log(`Inferred ${inferred.length} new triples`);
+```
+
+### Constraints
+
+```typescript
+// Define constraints
+await yata.addConstraint({
+  type: 'cardinality',
+  predicate: 'hasSpouse',
+  max: 1
+});
+
+// Validate
+const violations = await yata.validate();
+if (violations.length > 0) {
+  console.error('Constraint violations:', violations);
+}
+```
+
+### Configuration Options
+
+```typescript
+const yata = new YataLocal('./knowledge.db', {
+  // WAL mode for better concurrency (default: true)
+  walMode: true,
+  
+  // Enable FTS5 search (default: true)
+  enableSearch: true,
+  
+  // Auto-run inference on writes (default: false)
+  autoInfer: false,
+  
+  // Journal mode (default: 'wal')
+  journalMode: 'wal'
+});
+```
+
+---
+
+## YATA Global
+
+*(New in v1.6.3)*
+
+YATA Global is a distributed knowledge graph platform for team collaboration. It provides REST API access to shared knowledge graphs with offline support and intelligent synchronization.
+
+### Features
+
+| Feature | Description |
+|---------|-------------|
+| **REST API** | Full CRUD operations via HTTP |
+| **Offline Cache** | SQLite-based local cache |
+| **Sync Engine** | Push/Pull with conflict resolution |
+| **Conflict Resolution** | Last-write-wins or custom strategies |
+| **Authentication** | API key-based auth |
+| **Batch Operations** | Bulk triple operations |
+
+### Installation
+
+```bash
+npm install @nahisaho/yata-global
+```
+
+### Quick Start
+
+```typescript
+import { YataGlobal } from '@nahisaho/yata-global';
+
+// Initialize client
+const yata = new YataGlobal({
+  endpoint: 'https://yata.example.com/api',
+  apiKey: 'your-api-key',
+  graphId: 'project-knowledge'
+});
+
+await yata.initialize();
+
+// Add triples (batched)
+await yata.addTriples([
+  { subject: 'Task:001', predicate: 'assignedTo', object: 'User:alice' },
+  { subject: 'Task:001', predicate: 'status', object: 'in-progress' }
+]);
+
+// Query with filters
+const tasks = await yata.query({
+  predicate: 'assignedTo',
+  object: 'User:alice'
+});
+
+// Clean up
+await yata.close();
+```
+
+### Offline Support
+
+YATA Global supports offline-first operation with automatic synchronization:
+
+```typescript
+const yata = new YataGlobal({
+  endpoint: 'https://yata.example.com/api',
+  apiKey: 'your-api-key',
+  graphId: 'project-knowledge',
+  
+  // Offline configuration
+  offlineMode: true,
+  cachePath: './yata-cache.db',
+  syncInterval: 60000  // Auto-sync every 60 seconds
+});
+
+// Works offline - cached locally
+await yata.addTriple({
+  subject: 'Note:001',
+  predicate: 'content',
+  object: 'Important meeting notes'
+});
+
+// Manual sync when online
+await yata.sync();
+```
+
+### Conflict Resolution
+
+```typescript
+const yata = new YataGlobal({
+  // ... other options
+  
+  conflictStrategy: 'last-write-wins',  // Default
+  // or: 'server-wins', 'client-wins', 'manual'
+  
+  onConflict: async (local, remote) => {
+    // Custom resolution logic
+    console.log('Conflict detected:', local, remote);
+    return remote;  // Prefer remote version
+  }
+});
+```
+
+### Sync Status
+
+```typescript
+// Check sync status
+const status = await yata.getSyncStatus();
+console.log(`Pending changes: ${status.pendingPush}`);
+console.log(`Last sync: ${status.lastSyncAt}`);
+
+// Force full sync
+await yata.sync({ force: true });
+```
+
+### When to Use YATA Local vs YATA Global
+
+| Use Case | Recommended |
+|----------|-------------|
+| Personal knowledge base | YATA Local |
+| Single-user application | YATA Local |
+| Privacy-sensitive data | YATA Local |
+| Team collaboration | YATA Global |
+| Cross-device access | YATA Global |
+| Shared project knowledge | YATA Global |
+| Offline-first with sync | YATA Global |
+
+---
+
+## KGPR - Knowledge Graph Pull Request
+
+*(v1.6.4)*
+
+KGPR (Knowledge Graph Pull Request) enables safe sharing of knowledge graphs from YATA Local to YATA Global using a GitHub PR-like workflow.
+
+### Workflow
+
+```
+┌─────────────┐     ┌──────────────┐     ┌───────────────┐
+│ YATA Local  │ ──► │ KGPR (Draft) │ ──► │ YATA Global   │
+│ (Local KG)  │     │ (Extract)    │     │ (Review/Merge)│
+└─────────────┘     └──────────────┘     └───────────────┘
+
+Status Flow:
+draft → open → reviewing → approved/changes_requested → merged/closed
+```
+
+### Privacy Levels
+
+| Level | Filtered Content |
+|-------|------------------|
+| `strict` | File paths, URLs, credentials, all metadata |
+| `moderate` | File paths, URLs, credentials |
+| `none` | No filtering |
+
+### CLI Commands
+
+```bash
+# Create a KGPR
+musubix kgpr create -t "Add authentication patterns"
+
+# Preview diff before creating
+musubix kgpr diff --namespace myproject --privacy moderate
+
+# List KGPRs
+musubix kgpr list
+
+# Submit KGPR for review
+musubix kgpr submit <id>
+
+# Show KGPR details
+musubix kgpr show <id>
+
+# Close without merging
+musubix kgpr close <id>
+```
+
+### MCP Tools
+
+| Tool | Description |
+|------|-------------|
+| `kgpr_create` | Create KGPR from local knowledge graph |
+| `kgpr_diff` | Preview diff before creating KGPR |
+| `kgpr_list` | List all KGPRs |
+| `kgpr_submit` | Submit KGPR for review |
+| `kgpr_review` | Review KGPR (approve/changes_requested/commented) |
+
+### Example Usage
+
+```bash
+# 1. Preview what will be shared
+musubix kgpr diff --privacy strict
+
+# 2. Create KGPR with description
+musubix kgpr create -t "Share React patterns" -d "Learned patterns from project-x"
+
+# 3. Review the KGPR
+musubix kgpr show KGPR-001
+
+# 4. Submit for review
+musubix kgpr submit KGPR-001
+```
+
+---
+
+## YATA Platform Enhancements
+
+*(v1.7.0)*
+
+Version 1.7.0 introduces significant enhancements to the YATA platform with 5 major features.
+
+### Phase 1: Index Optimization
+
+Optimized query performance for YATA Local with composite indexes.
+
+```typescript
+import { IndexOptimizer } from '@nahisaho/yata-local';
+
+const optimizer = new IndexOptimizer(database);
+
+// Analyze and create optimal indexes
+const analysis = await optimizer.analyzeQueryPatterns();
+const created = await optimizer.createOptimalIndexes();
+
+// Check index health
+const health = await optimizer.checkIndexHealth();
+```
+
+**Key Features:**
+- Composite index creation for common query patterns
+- Index health monitoring with fragmentation detection
+- Automatic optimization recommendations
+
+### Phase 2: Enhanced Export Pipeline
+
+Powerful export capabilities with incremental export and multiple formats.
+
+```typescript
+import { ExportPipeline } from '@nahisaho/yata-local';
+
+const pipeline = new ExportPipeline(database);
+
+// Full export
+const fullData = await pipeline.exportFull({ namespace: 'myproject' });
+
+// Incremental export (changes since last export)
+const changes = await pipeline.exportIncremental({
+  since: lastExportTimestamp,
+  format: 'json'
+});
+
+// Export with transformation
+const transformed = await pipeline.exportWithTransform({
+  format: 'rdf',
+  includeMetadata: true
+});
+```
+
+**Supported Formats:**
+- JSON (default)
+- RDF/Turtle
+- N-Triples
+- Custom transformers
+
+### Phase 3: Global Sync Integration
+
+Seamless synchronization between YATA Local and YATA Global.
+
+```typescript
+import { GlobalSyncClient, SyncEngine } from '@nahisaho/yata-global';
+
+const client = new GlobalSyncClient({
+  endpoint: 'https://yata-global.example.com',
+  offlineMode: true
+});
+
+// Initialize sync
+await client.initialize();
+
+// Push local changes
+const syncResult = await client.sync({
+  namespace: 'myproject',
+  direction: 'push'
+});
+
+// Pull updates from global
+await client.sync({
+  namespace: 'shared-patterns',
+  direction: 'pull'
+});
+```
+
+**Features:**
+- Offline-first with automatic sync
+- Conflict resolution strategies
+- Selective namespace sync
+- Framework pattern repository
+
+### Phase 4: Code Generator Enhancement
+
+Advanced code generation from design documents.
+
+```typescript
+import { CodeGenerator } from '@nahisaho/yata-local';
+
+const generator = new CodeGenerator({
+  language: 'typescript',
+  outputDir: './src/generated'
+});
+
+// Generate from C4 design
+const result = await generator.generateFromC4(designDocument);
+
+// Generate with custom templates
+await generator.generate({
+  template: 'repository-pattern',
+  context: { entityName: 'User' }
+});
+```
+
+**Supported Patterns:**
+- Repository Pattern
+- Service Layer
+- Factory Pattern
+- Domain Events
+- Value Objects
+
+### Phase 5: YATA UI (Web Visualization)
+
+Web-based visualization and management interface for knowledge graphs.
+
+```typescript
+import { YataUIServer, createYataUIServer } from '@nahisaho/yata-ui';
+
+// Create and start server
+const server = createYataUIServer({
+  port: 3000,
+  enableRealtime: true
+});
+
+// Set data provider
+server.setDataProvider(async () => ({
+  nodes: await getEntities(),
+  edges: await getRelationships()
+}));
+
+await server.start();
+console.log(`UI available at ${server.getUrl()}`);
+```
+
+**UI Features:**
+- Interactive graph visualization
+- Real-time updates via WebSocket
+- Namespace filtering
+- Entity/Relationship editing
+- Export/Import functionality
+
+### v1.7.0 Package Summary
+
+| Package | Description |
+|---------|-------------|
+| `@nahisaho/yata-local` | IndexOptimizer, ExportPipeline, CodeGenerator |
+| `@nahisaho/yata-global` | GlobalSyncClient, SyncEngine, CacheManager |
+| `@nahisaho/yata-ui` | YataUIServer, Graph visualization |
+
+---
+
 ## MCP Server Integration
 
 ### CLI Startup
@@ -800,6 +1380,137 @@ const semanticDuplicates = validator.findSemanticDuplicates(allTriples);
 
 ---
 
+## Advanced Inference
+
+*(v1.4.5 New Feature)*
+
+### Overview
+
+Advanced Inference provides OWL 2 RL reasoning and Datalog evaluation capabilities for the knowledge graph. It supports materialization of implicit facts, rule-based inference, and human-readable explanations.
+
+### Key Components
+
+| Component | Description |
+|-----------|-------------|
+| `OWL2RLReasoner` | OWL 2 RL profile reasoning with 20+ built-in rules |
+| `DatalogEngine` | Stratified Datalog evaluation engine |
+| `InferenceExplainer` | Natural language explanation generator |
+| `ProgressReporter` | Real-time inference progress tracking |
+
+### OWL 2 RL Reasoning
+
+```typescript
+import { OWL2RLReasoner } from '@nahisaho/musubix-ontology-mcp';
+
+const reasoner = new OWL2RLReasoner({
+  maxIterations: 100,
+  enablePropertyChains: true,
+  enableInverseProperties: true
+});
+
+// Run reasoning on store
+const result = await reasoner.reason(store, {
+  onProgress: (progress) => {
+    console.log(`Iteration ${progress.iteration}: ${progress.newTriples} new triples`);
+  }
+});
+
+console.log(`Inferred ${result.inferredCount} new facts`);
+console.log(`Rules applied: ${result.rulesApplied.join(', ')}`);
+```
+
+### OWL 2 RL Rules
+
+| Rule ID | Name | Description |
+|---------|------|-------------|
+| `prp-dom` | Property Domain | Infer type from property domain |
+| `prp-rng` | Property Range | Infer type from property range |
+| `prp-inv1/2` | Inverse Properties | Infer inverse relationships |
+| `prp-trp` | Transitive Properties | Chain transitive properties |
+| `prp-symp` | Symmetric Properties | Infer symmetric relationships |
+| `cax-sco` | SubClassOf | Propagate class membership |
+| `scm-spo` | SubPropertyOf | Property subsumption |
+| `eq-rep-s/p/o` | SameAs Replacement | Substitute equal individuals |
+
+### Datalog Evaluation
+
+```typescript
+import { DatalogEngine } from '@nahisaho/musubix-ontology-mcp';
+
+const engine = new DatalogEngine();
+
+// Define rules
+const rules = [
+  {
+    head: { predicate: 'ancestor', args: ['?x', '?y'] },
+    body: [
+      { predicate: 'parent', args: ['?x', '?y'] }
+    ]
+  },
+  {
+    head: { predicate: 'ancestor', args: ['?x', '?z'] },
+    body: [
+      { predicate: 'parent', args: ['?x', '?y'] },
+      { predicate: 'ancestor', args: ['?y', '?z'] }
+    ]
+  }
+];
+
+// Evaluate rules
+const result = await engine.evaluate(rules, facts, {
+  onProgress: (progress) => {
+    console.log(`Stratum ${progress.stratum}: evaluating ${progress.rule}`);
+  }
+});
+
+console.log(`Derived ${result.derivedFacts.length} new facts`);
+```
+
+### Inference Explanation
+
+```typescript
+import { InferenceExplainer, ExplanationFormat } from '@nahisaho/musubix-ontology-mcp';
+
+const explainer = new InferenceExplainer(reasoner.getProvenanceLog());
+
+// Get explanation for a specific triple
+const explanation = explainer.explain(
+  'http://example.org/Animal',
+  'rdf:type',
+  'owl:Class',
+  ExplanationFormat.TEXT
+);
+
+console.log(explanation);
+// Output: "Animal is a Class because it is declared as owl:Class via rule cax-sco"
+
+// Generate HTML explanation
+const htmlExplanation = explainer.explain(
+  subject, predicate, object,
+  ExplanationFormat.HTML
+);
+```
+
+### Progress Reporting
+
+```typescript
+import { createProgressReporter } from '@nahisaho/musubix-ontology-mcp';
+
+const reporter = createProgressReporter({
+  onProgress: (info) => {
+    console.log(`Phase: ${info.phase}`);
+    console.log(`Iteration: ${info.iteration}/${info.maxIterations}`);
+    console.log(`Triples: ${info.totalTriples}`);
+    console.log(`New inferences: ${info.newInferences}`);
+  },
+  throttleMs: 500  // Report every 500ms
+});
+
+await reasoner.reason(store, { progressReporter: reporter });
+```
+
+---
+
 ## YATA Integration
 
 ### What is YATA?
@@ -992,6 +1703,6 @@ const client = createYATAClient({
 
 ---
 
-**Version**: 1.3.0  
-**Last Updated**: 2026-01-05  
+**Version**: 1.6.0  
+**Last Updated**: 2026-01-06  
 **MUSUBIX Project**