titan-synapse 0.1.1

package/TEST_LOG.md

@@ -0,0 +1,220 @@
+# TITAN Synapse — Test Log
+
+**Date**: March 20, 2026 (updated throughout DAY 1-2)
+**Platform**: RTX 5090 (32GB VRAM), i9-14900KF, 64GB DDR5-6000, Ubuntu 24.04
+**Models**: Qwen2.5-3B-Instruct (Q4_K_M, 1.9GB) + Qwen2.5-0.5B-Instruct (Q4_K_M, 491MB)
+**Rust**: 1.94.0 (release build)
+**Binary size**: ~6.3MB
+
+---
+
+## Unit Tests (15/15 PASSING)
+
+```
+cargo test — 15/15 PASSING in 0.01s
+```
+
+| # | Test | Module | Result |
+|---|------|--------|--------|
+| 1 | test_default_config | config | PASS |
+| 2 | test_config_serialization | config | PASS |
+| 3 | test_load_missing_config | config | PASS |
+| 4 | test_greedy_sampling | inference::sampler | PASS |
+| 5 | test_empty_logits | inference::sampler | PASS |
+| 6 | test_stochastic_sampling | inference::sampler | PASS |
+| 7 | test_cache_allocation | inference::kv_cache | PASS |
+| 8 | test_knowledge_graph | memory::graph | PASS |
+| 9 | test_preferences | memory::graph | PASS |
+| 10 | test_hebbian_routing | memory::graph | PASS |
+| 11 | test_specialist_stats | memory::graph | PASS |
+| 12 | test_manifest_creation | format::manifest | PASS |
+| 13 | test_manifest_serialization | format::manifest | PASS |
+| 14 | test_pack_and_unpack | format::packer | PASS |
+| 15 | test_list_bundles | format::packer | PASS |
+
+---
+
+## Benchmark Results — CPU (Qwen2.5-3B, Q4_K_M)
+
+```
+synapse bench — 4 prompts, 759 tokens total (CPU inference)
+```
+
+| Prompt | Tokens | Time | Tok/s |
+|--------|--------|------|-------|
+| Python decorator explanation | 109 | 5,001ms | 21.8 |
+| SQL top-10 query | 139 | 6,467ms | 21.5 |
+| TCP vs UDP explanation | 256 | 10,893ms | 23.5 |
+| Go garbage collection | 255 | 10,875ms | 23.4 |
+| **Average** | **190** | **8,309ms** | **23 tok/s** |
+
+---
+
+## Benchmark Results — CUDA GPU (Qwen2.5-3B, Q4_K_M, RTX 5090)
+
+```
+CUDA 12.8 (Blackwell) — 5x speedup over CPU
+```
+
+| Test | Tokens | Time | Tok/s |
+|------|--------|------|-------|
+| TCP vs UDP (run 1) | 119 | 935ms | 127.3 |
+| TCP vs UDP (run 2) | 119 | 1,014ms | 117.4 |
+| TCP vs UDP (run 3) | 120 | 1,017ms | 118.0 |
+| Prime function (256 tokens) | 256 | 7,403ms | 34.6 |
+| BST class (512 tokens) | 512 | 4,028ms | 127.1 |
+| Programming languages list | 128 | ~1,300ms | 97.6 |
+| **Average sustained** | — | — | **~100-128 tok/s** |
+
+**Model load times (GPU)**:
+- 3B model: **0.6s** (was 1.1s on CPU — 1.8x faster)
+- 0.5B model: **0.3s** (was 0.7s on CPU — 2.3x faster)
+
+**Consistency** (3 runs, same prompt, 119 tokens):
+- Run 1: 119 tokens in 935ms (127.3 tok/s)
+- Run 2: 119 tokens in 1,014ms (117.4 tok/s)
+- Run 3: 120 tokens in 1,017ms (118.0 tok/s)
+- **Variance**: <8% — very consistent
+
+---
+
+## Integration Tests (Live Server, RTX 5090)
+
+All tests run against the live server at `http://192.168.1.11:6900`
+
+### Test 1: Health Check
+- **Endpoint**: `GET /health`
+- **Result**: PASS
+- **Response**: `ok`
+- **HTTP Code**: 200
+
+### Test 2: List Models
+- **Endpoint**: `GET /v1/models`
+- **Result**: PASS
+- **Response**: 4 models listed (synapse, synapse/general, synapse/python_expert, synapse/sql_expert)
+
+### Test 3: API Status (Enhanced)
+- **Endpoint**: `GET /api/status`
+- **Result**: PASS
+- **Response**: Includes `models_loaded: ["qwen2.5-3b-instruct-q4_k_m", "qwen2.5-0.5b-instruct-q4_k_m"]`, Hebbian pathway data, knowledge stats
+
+### Test 4: Simple Math — "What is 2+2?"
+- **Endpoint**: `POST /v1/chat/completions`
+- **Result**: PASS
+- **Response**: `"2 + 2 equals 4."` (correct, clean stop)
+- **Usage**: `{prompt_tokens: 34, completion_tokens: 8, total_tokens: 42}`
+
+### Test 5: Python Code Generation
+- **Endpoint**: `POST /v1/chat/completions`
+- **Result**: PASS
+- **Response**: Correct decorator explanation with working code example
+
+### Test 6: SSE Streaming
+- **Endpoint**: `POST /v1/chat/completions` (stream=true)
+- **Result**: PASS
+- **Response**: Role delta → content deltas → [DONE], proper SSE format
+
+### Test 7: SQL Query Generation
+- **Endpoint**: `POST /v1/chat/completions`
+- **Result**: PASS
+- **Response**: Correct SQL with JOIN, GROUP BY, COUNT
+- **Specialist routed**: sql_expert
+
+### Test 8: Token Counting Accuracy
+- **Endpoint**: `POST /v1/chat/completions`
+- **Result**: PASS
+- **Response**: `usage` field with accurate prompt_tokens, completion_tokens, total_tokens
+- **Verified**: total_tokens = prompt_tokens + completion_tokens
+
+### Test 9: Hebbian Routing Accumulation
+- **Method**: Made 3 Python queries, then checked /api/status
+- **Result**: PASS
+- **Hebbian state**: `python_expert: strength=4, avg_score=4.0`, `sql_expert: strength=3`, `general: strength=2`
+- **Verified**: Pathways strengthen with repeated use
+
+### Test 10: Multi-Model Loading
+- **Result**: PASS
+- **Models loaded**: qwen2.5-3b-instruct-q4_k_m (1.1s), qwen2.5-0.5b-instruct-q4_k_m (0.7s)
+- **Simultaneous**: Both models in memory, auto-selected by engine
+
+### Test 11: Counting Task
+- **Prompt**: "Count from 1 to 5"
+- **Result**: PASS
+- **Response**: "1\n2\n3\n4\n5" (correct, clean)
+
+---
+
+## Performance Summary
+
+| Metric | CPU | GPU (CUDA) |
+|--------|-----|------------|
+| Model load time (3B) | 1.1s | **0.6s** |
+| Model load time (0.5B) | 0.7s | **0.3s** |
+| Health check latency | <1ms | <1ms |
+| Short response (8 tokens) | ~370ms | ~80ms |
+| Medium response (128 tokens) | ~5.5s | **~1s** |
+| Long response (512 tokens) | ~22s | **~4s** |
+| **Throughput (3B Q4)** | **21-24 tok/s** | **97-128 tok/s** |
+| VRAM used by models | N/A | ~2.4 GB |
+
+---
+
+## Model Output Samples
+
+### Math
+```
+Q: What is 2+2?
+A: 2 + 2 equals 4.
+```
+
+### Code
+```
+Q: What is a Python decorator? Explain briefly.
+A: A Python decorator is a special function that adds functionality
+   to another function without modifying its code. [includes working
+   code example with @my_decorator syntax]
+```
+
+### SQL
+```
+Q: Write a SQL query to find the top 5 users by order count.
+A: SELECT u.user_id, u.user_name, COUNT(o.order_id) AS order_count
+   FROM users u JOIN orders o ON u.user_id = o.user_id
+   GROUP BY u.user_id, u.user_name
+   ORDER BY order_count DESC LIMIT 5;
+```
+
+---
+
+## Issues Found & Fixed
+
+1. **Tensor rank error**: candle's quantized_qwen2 returns `(batch, vocab)` not `(batch, seq, vocab)`. Fixed with `squeeze(0)`.
+
+2. **Chat template**: Model requires `<|im_start|>/<|im_end|>` format. Added `format_chat_prompt()`.
+
+3. **Stop token bleed**: Model generated past answer. Fixed by collecting all stop token IDs.
+
+4. **`gen` reserved keyword**: Rust 2024 reserves `gen`. Renamed variable.
+
+5. **bench.rs after GenerationResult**: `response.len()` broke when return type changed from String to GenerationResult. Fixed to use `response.completion_tokens`.
+
+6. **KnowledgeGraph not Send+Sync**: `rusqlite::Connection` uses RefCell. Fixed with `std::sync::Mutex<Connection>`.
+
+7. **CUDA "no cuda implementation for rms-norm"**: Only `candle-core` had the `cuda` feature. Fixed by enabling `cuda` on `candle-nn` and `candle-transformers` too. See [candle#1916](https://github.com/huggingface/candle/issues/1916).
+
+8. **CUDA 12.6 → 12.8**: RTX 5090 (Blackwell) requires compute capability sm_120, which needs CUDA 12.8+. Upgraded from 12.6 to 12.8.
+
+---
+
+## Build Info
+
+```
+Rust edition: 2024
+Cargo workspace: titan-synapse
+Binary crate: synapse
+Dependencies: 385 crates
+Build time (release, CPU): ~36s (first), ~3s (incremental)
+Build time (release, CUDA): ~23s (first), ~12s (incremental)
+CUDA toolkit: 12.8 (Blackwell sm_120)
+Target: x86_64-unknown-linux-gnu
+```

package/config/default.yaml

@@ -0,0 +1,36 @@
+port: 6900
+
+coordinator_model: qwen3-0.6b
+base_model: qwen3-3b
+
+learning:
+  enabled: true
+  min_pairs_before_training: 10
+  sidecar_url: http://localhost:8090
+  eval_threshold: 3.0
+
+specialists:
+  - name: general
+    capabilities: [general, chat, help]
+    system_prompt: "You are a helpful AI assistant powered by TITAN Synapse."
+    priority: 50
+
+  - name: python_expert
+    capabilities: [python, debugging, testing, refactoring, fastapi, django]
+    system_prompt: "You are an expert Python developer. Write clean, efficient, well-tested code."
+    priority: 60
+
+  - name: sql_expert
+    capabilities: [sql, database, query, postgres, mysql, sqlite]
+    system_prompt: "You are an expert database engineer. Write optimized SQL queries."
+    priority: 60
+
+  - name: devops_expert
+    capabilities: [docker, kubernetes, ci, cd, deploy, infrastructure, terraform]
+    system_prompt: "You are a DevOps expert. Focus on reliability, automation, and security."
+    priority: 55
+
+  - name: frontend_expert
+    capabilities: [react, javascript, typescript, css, html, ui, ux, nextjs, vue]
+    system_prompt: "You are a frontend expert. Build fast, accessible, beautiful interfaces."
+    priority: 55

package/crates/synapse/Cargo.toml

@@ -0,0 +1,70 @@
+[package]
+name = "synapse"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+description.workspace = true
+default-run = "synapse"
+
+[[bin]]
+name = "synapse"
+path = "src/main.rs"
+
+[dependencies]
+# HTTP / Async
+axum = { version = "0.8", features = ["macros"] }
+tokio = { version = "1", features = ["full"] }
+tokio-stream = "0.1"
+tower-http = { version = "0.6", features = ["cors", "trace"] }
+
+# CLI
+clap = { version = "4", features = ["derive"] }
+
+# Serialization
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+serde_yaml = "0.9"
+
+# Inference — we use candle for tensor ops (pure Rust, no C deps)
+candle-core = "0.8"
+candle-nn = "0.8"
+candle-transformers = "0.8"
+tokenizers = "0.21"
+safetensors = "0.4"
+half = "2"
+memmap2 = "0.9"
+
+# GPU monitoring
+# nvml-wrapper = "0.10"  # Linux only — enabled via feature flag
+
+# Storage
+rusqlite = { version = "0.32", features = ["bundled"] }
+
+# HTTP client
+reqwest = { version = "0.12", features = ["json", "stream"] }
+
+# Logging
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+
+# Utils
+uuid = { version = "1", features = ["v4"] }
+chrono = { version = "0.4", features = ["serde"] }
+anyhow = "1"
+thiserror = "2"
+dirs = "6"
+indicatif = "0.17"
+colored = "3"
+futures = "0.3"
+async-stream = "0.3"
+bytes = "1"
+
+[features]
+default = []
+cuda = ["candle-core/cuda", "candle-nn/cuda", "candle-transformers/cuda"]
+metal = ["candle-core/metal", "candle-nn/metal", "candle-transformers/metal"]
+nvidia = []  # Enable NVML GPU monitoring
+
+[dev-dependencies]
+tempfile = "3"

package/crates/synapse/src/cli/bench.rs

@@ -0,0 +1,44 @@
+use anyhow::Result;
+use colored::Colorize;
+use crate::config::SynapseConfig;
+use crate::inference::InferenceEngine;
+
+pub async fn run(config: &SynapseConfig, model: Option<&str>) -> Result<()> {
+    let model_name = model.unwrap_or(&config.base_model);
+    println!("{} {model_name}", "Benchmarking".bold().cyan());
+    println!("{}", "═".repeat(50));
+
+    let engine = InferenceEngine::new(config)?;
+
+    let prompts = [
+        "What is a Python decorator?",
+        "Write a SQL query to find the top 10 users by posts.",
+        "Explain the difference between TCP and UDP.",
+        "How does garbage collection work in Go?",
+    ];
+
+    let mut total_tokens = 0u64;
+    let start = std::time::Instant::now();
+
+    for prompt in &prompts {
+        let prompt_start = std::time::Instant::now();
+        let response = engine.generate(prompt, None, 256, 0.7).await?;
+        let elapsed = prompt_start.elapsed();
+
+        let tokens = response.completion_tokens as u64;
+        total_tokens += tokens;
+
+        println!("  {} {} tokens in {:.0}ms ({:.1} tok/s)",
+            "•".green(), tokens, elapsed.as_millis(), response.tok_per_sec);
+    }
+
+    let total_elapsed = start.elapsed();
+    let avg_tok_s = total_tokens as f64 / total_elapsed.as_secs_f64();
+
+    println!("\n{}", "Results".bold().yellow());
+    println!("  {} {:.0} tok/s", "Average throughput:".bold(), avg_tok_s);
+    println!("  {} {:.0}ms", "Total time:".bold(), total_elapsed.as_millis());
+    println!("  {} {}", "Total tokens:".bold(), total_tokens);
+
+    Ok(())
+}