musubix 1.7.0 → 1.8.5

package/docs/USER-GUIDE.md

@@ -21,10 +21,15 @@
 13. [YATA Global](#yata-global) *(v1.6.3)*
 14. [KGPR - Knowledge Graph Pull Request](#kgpr---knowledge-graph-pull-request) *(v1.6.4)*
 15. [YATA Platform Enhancements](#yata-platform-enhancements) *(v1.7.0)*
-16. [MCP Server Integration](#mcp-server-integration)
-17. [YATA Integration](#yata-integration)
-18. [Best Practices](#best-practices)
-19. [Troubleshooting](#troubleshooting)
+16. [Formal Verification](#formal-verification) *(v1.7.5)*
+17. [Security Analysis](#security-analysis) *(v1.8.0)*
+18. [DFG/CFG Extraction](#dfgcfg-extraction) *(v2.0.0-alpha.1)*
+19. [Lean 4 Integration](#lean-4-integration) *(v2.0.0-alpha.1)*
+20. [YATA Scale](#yata-scale) *(v2.0.0-alpha.1)*
+21. [MCP Server Integration](#mcp-server-integration)
+22. [YATA Integration](#yata-integration)
+23. [Best Practices](#best-practices)
+24. [Troubleshooting](#troubleshooting)
 
 ---
 
@@ -1093,6 +1098,596 @@ console.log(`UI available at ${server.getUrl()}`);
 
 ---
 
+## Formal Verification
+
+*(v1.7.5)*
+
+The `@nahisaho/musubix-formal-verify` package provides formal verification capabilities using the Z3 SMT solver.
+
+### Installation
+
+```bash
+npm install @nahisaho/musubix-formal-verify
+# Optional: Install z3-solver for WebAssembly support
+npm install z3-solver
+```
+
+### Z3 SMT Solver Integration
+
+```typescript
+import { Z3Adapter, PreconditionVerifier, PostconditionVerifier } from '@nahisaho/musubix-formal-verify';
+
+// Create Z3 adapter (auto-selects backend)
+const z3 = await Z3Adapter.create();
+
+// Precondition verification
+const preVerifier = new PreconditionVerifier(z3);
+const result = await preVerifier.verify({
+  condition: { expression: 'amount > 0 && balance >= amount', format: 'javascript' },
+  variables: [
+    { name: 'amount', type: 'Int' },
+    { name: 'balance', type: 'Int' },
+  ],
+});
+
+console.log(result.status); // 'valid' | 'invalid' | 'unknown' | 'error'
+```
+
+### Hoare Triple Verification
+
+```typescript
+// Verify {P} C {Q}
+const postVerifier = new PostconditionVerifier(z3);
+const hoareResult = await postVerifier.verify({
+  precondition: { expression: 'balance >= amount', format: 'javascript' },
+  postcondition: { expression: 'balance_new == balance - amount', format: 'javascript' },
+  preVariables: [{ name: 'balance', type: 'Int' }, { name: 'amount', type: 'Int' }],
+  postVariables: [{ name: 'balance_new', type: 'Int' }],
+  transition: 'balance_new == balance - amount',
+});
+```
+
+### EARS to SMT Conversion
+
+```typescript
+import { EarsToSmtConverter } from '@nahisaho/musubix-formal-verify';
+
+const converter = new EarsToSmtConverter();
+
+// Convert EARS requirements to SMT-LIB2
+const results = converter.convertMultiple([
+  'THE system SHALL validate inputs',           // ubiquitous
+  'WHEN error, THE system SHALL notify user',   // event-driven
+  'WHILE busy, THE system SHALL queue requests', // state-driven
+  'THE system SHALL NOT expose secrets',        // unwanted
+  'IF admin, THEN THE system SHALL allow edit', // optional
+]);
+
+results.forEach(r => {
+  console.log(`Pattern: ${r.formula?.metadata.earsPattern.type}`);
+  console.log(`SMT: ${r.formula?.smtLib2}`);
+});
+```
+
+### Traceability Database
+
+```typescript
+import { TraceabilityDB, ImpactAnalyzer } from '@nahisaho/musubix-formal-verify';
+
+// Create SQLite-based traceability database
+const db = new TraceabilityDB('./trace.db');
+
+// Add nodes
+await db.addNode({ id: 'REQ-001', type: 'requirement', title: 'User Auth' });
+await db.addNode({ id: 'DES-001', type: 'design', title: 'AuthService' });
+await db.addNode({ id: 'CODE-001', type: 'code', title: 'auth.ts' });
+
+// Add traceability links
+await db.addLink({ source: 'DES-001', target: 'REQ-001', type: 'satisfies' });
+await db.addLink({ source: 'CODE-001', target: 'DES-001', type: 'implements' });
+
+// Impact analysis
+const analyzer = new ImpactAnalyzer(db);
+const impact = await analyzer.analyze('REQ-001');
+console.log(`Impacted nodes: ${impact.totalImpacted}`);
+```
+
+### v1.7.5 Package Summary
+
+| Package | Description |
+|---------|-------------|
+| `@nahisaho/musubix-formal-verify` | Z3 integration, Hoare verification, EARS→SMT, TraceabilityDB |
+
+### Supported Variable Types
+
+| Type | Description |
+|------|-------------|
+| `Int` | Integer values |
+| `Real` | Real numbers |
+| `Bool` | Boolean values |
+| `String` | String values |
+| `Array` | Array types |
+| `BitVec` | Bit vectors |
+
+---
+
+## Security Analysis
+
+*(v1.8.0)*
+
+The `@nahisaho/musubix-security` package provides comprehensive security analysis capabilities for TypeScript/JavaScript projects.
+
+### Installation
+
+```bash
+npm install @nahisaho/musubix-security
+```
+
+### Vulnerability Scanning
+
+Detects OWASP Top 10 and CWE Top 25 vulnerabilities using AST analysis:
+
+```typescript
+import { VulnerabilityScanner, createSecurityService } from '@nahisaho/musubix-security';
+
+// Scan a single file
+const scanner = new VulnerabilityScanner();
+const vulnerabilities = scanner.scanFile('src/api.ts');
+
+// Scan a directory
+const result = await scanner.scanDirectory('./src');
+console.log(`Found ${result.vulnerabilities.length} vulnerabilities`);
+console.log(`Scanned ${result.scannedFiles} files`);
+```
+
+### Detected Vulnerabilities
+
+| Category | CWE | Severity |
+|----------|-----|----------|
+| SQL Injection | CWE-89 | Critical |
+| Command Injection | CWE-78 | Critical |
+| XSS | CWE-79 | High |
+| Path Traversal | CWE-22 | High |
+| Code Injection | CWE-94 | Critical |
+| NoSQL Injection | CWE-943 | High |
+
+### Secret Detection
+
+Detects hardcoded credentials and sensitive information:
+
+```typescript
+import { SecretDetector } from '@nahisaho/musubix-security';
+
+const detector = new SecretDetector();
+const secrets = detector.scanContent(content, 'config.ts');
+const result = await detector.scan('./src');
+
+console.log(`Found ${result.summary.total} secrets`);
+```
+
+### Detected Secret Types
+
+| Type | Pattern |
+|------|--------|
+| AWS Access Key | `AKIA...` |
+| AWS Secret Key | 40-char base64 |
+| GitHub Token | `ghp_*`, `gho_*`, `ghu_*` |
+| Private Key | PEM format |
+| Database URL | `postgres://`, `mongodb://` |
+| JWT Secret | JWT signing secrets |
+| Stripe Key | `sk_live_*`, `sk_test_*` |
+
+### Taint Analysis
+
+Tracks data flow from user input (sources) to dangerous functions (sinks):
+
+```typescript
+import { TaintAnalyzer } from '@nahisaho/musubix-security';
+
+const analyzer = new TaintAnalyzer();
+const result = analyzer.analyze('./src');
+
+console.log(`Sources: ${result.sources.length}`);
+console.log(`Sinks: ${result.sinks.length}`);
+console.log(`Taint paths: ${result.paths.length}`);
+```
+
+### Dependency Auditing
+
+Integrates with npm audit to detect vulnerable dependencies:
+
+```typescript
+import { DependencyAuditor } from '@nahisaho/musubix-security';
+
+const auditor = new DependencyAuditor();
+const result = await auditor.audit('./project');
+
+console.log(`Critical: ${result.summary.critical}`);
+console.log(`High: ${result.summary.high}`);
+```
+
+### Integrated Security Service
+
+Combines all security analysis features:
+
+```typescript
+import { createSecurityService } from '@nahisaho/musubix-security';
+
+const service = createSecurityService();
+
+// Full security scan
+const result = await service.scan({
+  target: './src',
+  vulnerabilities: true,
+  taint: true,
+  secrets: true,
+  dependencies: true,
+  generateFixes: true,
+});
+
+console.log(`Total vulnerabilities: ${result.summary.totalVulnerabilities}`);
+console.log(`Total secrets: ${result.summary.totalSecrets}`);
+console.log(`Fixes generated: ${result.summary.fixesGenerated}`);
+```
+
+### Report Generation
+
+Generate reports in multiple formats:
+
+```typescript
+// SARIF format (GitHub Code Scanning compatible)
+const sarifReport = await service.generateReport(result, 'sarif');
+
+// Markdown format
+const mdReport = await service.generateReport(result, 'markdown');
+
+// HTML format
+const htmlReport = await service.generateReport(result, 'html');
+```
+
+### CLI Usage
+
+```bash
+# Full security scan
+npx musubix-security scan ./src
+
+# Vulnerability scan only
+npx musubix-security scan ./src --vulnerabilities-only
+
+# Secret detection
+npx musubix-security secrets ./src
+
+# Taint analysis
+npx musubix-security taint ./src
+
+# Dependency audit
+npx musubix-security audit ./project
+
+# Generate SARIF report
+npx musubix-security scan ./src --format sarif --output report.sarif
+```
+
+### v1.8.0 Package Summary
+
+| Package | Description |
+|---------|-------------|
+| `@nahisaho/musubix-security` | Vulnerability scanning, secret detection, taint analysis, dependency auditing |
+
+---
+
+## DFG/CFG Extraction
+
+*(v2.0.0-alpha.1)*
+
+The `@nahisaho/musubix-dfg` package provides Data Flow Graph (DFG) and Control Flow Graph (CFG) extraction for TypeScript and JavaScript code analysis.
+
+### Installation
+
+```bash
+npm install @nahisaho/musubix-dfg
+```
+
+### DFG Extraction
+
+Extract data flow graphs from source code:
+
+```typescript
+import { extractDFG, DFGExtractor } from '@nahisaho/musubix-dfg';
+
+// Simple extraction
+const dfg = extractDFG(sourceCode, 'typescript');
+
+// With configuration
+const extractor = new DFGExtractor({
+  includeImplicitFlows: true,
+  trackConstants: true,
+});
+const result = extractor.extract(sourceCode);
+
+console.log(`Nodes: ${result.nodes.length}`);
+console.log(`Edges: ${result.edges.length}`);
+```
+
+### CFG Extraction
+
+Extract control flow graphs:
+
+```typescript
+import { extractCFG, CFGExtractor } from '@nahisaho/musubix-dfg';
+
+const cfg = extractCFG(sourceCode);
+
+// Traverse CFG
+for (const block of cfg.blocks) {
+  console.log(`Block ${block.id}: ${block.statements.length} statements`);
+  console.log(`Successors: ${block.successors.join(', ')}`);
+}
+```
+
+### Def-Use Chain Analysis
+
+Build definition-use chains for variable tracking:
+
+```typescript
+import { analyzeDefUse } from '@nahisaho/musubix-dfg';
+
+const chains = analyzeDefUse(dfg);
+
+for (const chain of chains) {
+  console.log(`Variable: ${chain.variable}`);
+  console.log(`Defined at: line ${chain.definition.line}`);
+  console.log(`Used at: ${chain.uses.map(u => u.line).join(', ')}`);
+}
+```
+
+### Data Dependency Analysis
+
+Analyze data dependencies between statements:
+
+```typescript
+import { analyzeDataDependencies } from '@nahisaho/musubix-dfg';
+
+const deps = analyzeDataDependencies(dfg);
+
+for (const dep of deps) {
+  console.log(`${dep.from} -> ${dep.to}: ${dep.type}`);
+}
+```
+
+---
+
+## Lean 4 Integration
+
+*(v2.0.0-alpha.1)*
+
+The `@nahisaho/musubix-lean` package provides integration with the Lean 4 theorem prover for formal verification of requirements.
+
+### Installation
+
+```bash
+npm install @nahisaho/musubix-lean
+```
+
+### EARS to Lean Conversion
+
+Convert EARS requirements to Lean 4 theorems:
+
+```typescript
+import { EarsToLeanConverter } from '@nahisaho/musubix-lean';
+
+const converter = new EarsToLeanConverter();
+
+// Convert EARS requirement
+const earsRequirement = {
+  id: 'REQ-001',
+  type: 'event-driven',
+  trigger: 'user clicks submit',
+  response: 'system saves form data',
+};
+
+const theorem = converter.convert(earsRequirement);
+console.log(theorem.leanCode);
+// theorem REQ_001 : ∀ (s : State), 
+//   user_clicks_submit s → saves_form_data (next s)
+```
+
+### Lean AST Parsing
+
+Parse and manipulate Lean 4 code:
+
+```typescript
+import { LeanParser, LeanAST } from '@nahisaho/musubix-lean';
+
+const parser = new LeanParser();
+const ast = parser.parse(leanCode);
+
+// Traverse AST
+for (const decl of ast.declarations) {
+  if (decl.type === 'theorem') {
+    console.log(`Theorem: ${decl.name}`);
+    console.log(`Statement: ${decl.statement}`);
+  }
+}
+```
+
+### Proof Engine
+
+Execute proofs using Lean 4:
+
+```typescript
+import { LeanProofEngine } from '@nahisaho/musubix-lean';
+
+const engine = new LeanProofEngine({
+  leanPath: '/usr/bin/lean',
+  timeout: 30000,
+});
+
+const result = await engine.prove(theorem);
+
+if (result.success) {
+  console.log('Proof verified!');
+  console.log(`Proof: ${result.proof}`);
+} else {
+  console.log(`Failed: ${result.error}`);
+}
+```
+
+### ReProver Integration
+
+Use ReProver for automated proof search:
+
+```typescript
+import { ReProverClient } from '@nahisaho/musubix-lean';
+
+const reprover = new ReProverClient({
+  endpoint: 'http://localhost:8080',
+  maxDepth: 10,
+});
+
+const proof = await reprover.searchProof(theorem);
+
+if (proof.found) {
+  console.log('Proof found!');
+  console.log(proof.tactics);
+}
+```
+
+---
+
+## YATA Scale
+
+*(v2.0.0-alpha.1)*
+
+The `@nahisaho/yata-scale` package provides distributed knowledge graph capabilities with sharding, caching, and synchronization.
+
+### Installation
+
+```bash
+npm install @nahisaho/yata-scale
+```
+
+### High-Level API
+
+Use `YataScaleManager` for simplified access:
+
+```typescript
+import { YataScaleManager } from '@nahisaho/yata-scale';
+
+const yata = new YataScaleManager({
+  shards: 16,
+  cacheConfig: {
+    l1MaxSize: 1000,
+    l2MaxSize: 10000,
+    l3Path: './cache',
+  },
+});
+
+// Store entity
+await yata.putEntity({
+  id: 'entity-1',
+  type: 'Document',
+  properties: { title: 'Example' },
+});
+
+// Query
+const results = await yata.query('SELECT ?s WHERE { ?s rdf:type Document }');
+```
+
+### Sharding
+
+Distribute data across shards using consistent hashing:
+
+```typescript
+import { ShardManager, HashPartitionStrategy } from '@nahisaho/yata-scale';
+
+const shardManager = new ShardManager({
+  shardCount: 16,
+  virtualNodes: 150,
+  strategy: new HashPartitionStrategy(),
+});
+
+// Get shard for entity
+const shard = shardManager.getShardForEntity('entity-id');
+console.log(`Entity assigned to shard ${shard.id}`);
+
+// Rebalance on node changes
+await shardManager.addNode('node-5');
+```
+
+### Multi-Tier Caching
+
+Three-tier caching for optimal performance:
+
+```typescript
+import { CacheManager, L1Cache, L2Cache, L3Cache } from '@nahisaho/yata-scale';
+
+const cache = new CacheManager({
+  l1: new L1Cache({ maxSize: 1000 }),    // LRU cache
+  l2: new L2Cache({ maxSize: 10000 }),   // LFU cache
+  l3: new L3Cache({ path: './cache' }),  // Disk cache
+});
+
+// Cache operations
+await cache.set('key', value, { ttl: 3600 });
+const result = await cache.get('key');
+
+// Invalidation
+await cache.invalidate('key');
+await cache.invalidatePattern('user:*');
+```
+
+### Index Management
+
+Multiple index types for efficient queries:
+
+```typescript
+import { IndexManager, BPlusTreeIndex, FullTextIndex, GraphIndex } from '@nahisaho/yata-scale';
+
+const indexManager = new IndexManager();
+
+// B+ Tree for range queries
+indexManager.addIndex('timestamp', new BPlusTreeIndex());
+
+// Full-text for search
+indexManager.addIndex('content', new FullTextIndex());
+
+// Graph for traversal
+indexManager.addIndex('relationships', new GraphIndex());
+```
+
+### Vector Clock Synchronization
+
+Distributed synchronization with conflict resolution:
+
+```typescript
+import { SyncController, VectorClock, ConflictResolver } from '@nahisaho/yata-scale';
+
+const sync = new SyncController({
+  nodeId: 'node-1',
+  conflictStrategy: 'last-write-wins',
+});
+
+// Synchronize with peers
+await sync.synchronize();
+
+// Manual conflict resolution
+sync.onConflict((conflict) => {
+  console.log(`Conflict on ${conflict.key}`);
+  return conflict.local; // or conflict.remote
+});
+```
+
+### v2.0.0-alpha.1 Package Summary
+
+| Package | Tests | Description |
+|---------|-------|-------------|
+| `@nahisaho/musubix-dfg` | 30 | DFG/CFG extraction, Def-Use analysis |
+| `@nahisaho/musubix-lean` | 151 | Lean 4 theorem proving, EARS conversion |
+| `@nahisaho/yata-scale` | 57 | Distributed sharding, caching, sync |
+| **Total** | **238** | **Phase 1: Deep Symbolic Integration** |
+
+---
+
 ## MCP Server Integration
 
 ### CLI Startup
@@ -1703,6 +2298,6 @@ const client = createYATAClient({
 
 ---
 
-**Version**: 1.6.0  
-**Last Updated**: 2026-01-06  
+**Version**: 1.8.5  
+**Last Updated**: 2026-01-08  
 **MUSUBIX Project**