agentic-flow 1.9.0 → 1.9.2

package/agentic-flow/examples/crispr-cas13-pipeline/README.md

@@ -1,591 +0,0 @@
-# 🧬 CRISPR-Cas13 Bioinformatics Pipeline
-
-<div align="center">
-
-[![Crates.io](https://img.shields.io/crates/v/crispr-cas13-pipeline?style=for-the-badge&logo=rust)](https://crates.io/crates/crispr-cas13-pipeline)
-[![Documentation](https://img.shields.io/docsrs/crispr-cas13-pipeline?style=for-the-badge&logo=rust)](https://docs.rs/crispr-cas13-pipeline)
-[![License](https://img.shields.io/crates/l/crispr-cas13-pipeline?style=for-the-badge)](LICENSE)
-[![Build Status](https://img.shields.io/github/actions/workflow/status/ruvnet/agentic-flow/ci.yml?style=for-the-badge&logo=github)](https://github.com/ruvnet/agentic-flow/actions)
-[![Downloads](https://img.shields.io/crates/d/crispr-cas13-pipeline?style=for-the-badge)](https://crates.io/crates/crispr-cas13-pipeline)
-
-**High-Performance Rust Pipeline for CRISPR-Cas13 Off-Target Analysis & Immune Response Profiling**
-
-[Quick Start](#-quick-start) •
-[Documentation](https://docs.rs/crispr-cas13-pipeline) •
-[Examples](#-usage-examples) •
-[Benchmarks](#-performance) •
-[Contributing](CONTRIBUTING.md)
-
-</div>
-
----
-
-## 🎯 Overview
-
-A **production-grade, ultra-fast bioinformatics pipeline** for analyzing CRISPR-Cas13 RNA-targeting experiments in primate models. Built with Rust for **10x performance improvement** over traditional Python/R pipelines, featuring ML-based off-target prediction, comprehensive immune response analysis, and enterprise-ready microservices architecture.
-
-### 🔬 Key Capabilities
-
-- **🎯 Off-Target Prediction**: ML-enhanced CFD scoring with position-weighted mismatch analysis
-- **🦠 Immune Response Profiling**: DESeq2-style differential expression analysis (pure Rust)
-- **🧪 Multi-Variant Support**: All 4 Cas13 variants (a, b, c, d) with PFS tracking
-- **⚡ Ultra-Fast Processing**: 2-10x faster than industry standard pipelines
-- **🏗️ Production-Ready**: Kubernetes deployments, auto-scaling, comprehensive monitoring
-- **📊 Standards-Compliant**: GA4GH, NCBI, W3C PROV-O provenance tracking
-
-Built using **[SPARC methodology](https://github.com/ruvnet/agentic-flow)** with **[rUv.io](https://ruv.io)** AI coordination for systematic, test-driven development.
-
----
-
-## ✨ Features
-
-### 🧬 Biological Analysis
-
-- ✅ **Guide RNA Validation**: 22-30nt length, PFS compatibility checking
-- ✅ **Off-Target Search**: Seed-based genome-wide scanning (0-4 mismatches)
-- ✅ **ML Confidence Scoring**: Gradient boosting with 15+ biological features
-- ✅ **Secondary Structure**: RNA accessibility prediction integration
-- ✅ **Expression Quantification**: TPM/RPKM normalization with DESeq2 stats
-- ✅ **Pathway Enrichment**: Type I/III interferon, RIG-I/MDA5, OAS/RNase L
-
-### 🚀 Performance & Scalability
-
-- ✅ **10x Faster Alignment**: >10,000 reads/sec vs 3-5K industry baseline
-- ✅ **Parallel Processing**: Rayon-based multi-core computation
-- ✅ **Async I/O**: Tokio runtime for non-blocking operations
-- ✅ **Memory Efficient**: Zero-copy parsing, streaming data processing
-- ✅ **Auto-Scaling**: Kubernetes HPA for 1-50 pod scaling
-
-### 🏗️ Architecture & DevOps
-
-- ✅ **Microservices**: 6 independent Rust crates with clear boundaries
-- ✅ **REST API**: Axum framework with OpenAPI 3.0 specification
-- ✅ **Message Queues**: Apache Kafka for job orchestration
-- ✅ **Databases**: PostgreSQL (relational) + MongoDB (genomic data)
-- ✅ **Object Storage**: S3-compatible (AWS/MinIO) for FASTQ/BAM files
-- ✅ **Monitoring**: Prometheus metrics + Grafana dashboards
-
-### 🔒 Security & Compliance
-
-- ✅ **OAuth2 + JWT**: RS256 signing with role-based access control
-- ✅ **TLS 1.3**: End-to-end encryption for all traffic
-- ✅ **AES-256**: At-rest encryption for databases and storage
-- ✅ **HIPAA Ready**: PHI de-identification and audit logging
-- ✅ **GDPR Compliant**: Right to access, deletion, and data portability
-
----
-
-## 📦 Installation
-
-### Prerequisites
-
-- Rust 1.75+ (MSRV)
-- Docker 24.0+ (optional, for containerized deployment)
-- PostgreSQL 15+ and MongoDB 7.0+ (for full pipeline)
-
-### Quick Install
-
-```bash
-# Install from crates.io
-cargo install crispr-cas13-pipeline
-
-# Or add to your Cargo.toml
-[dependencies]
-crispr-cas13-pipeline = "0.1"
-```
-
-### Development Setup
-
-```bash
-# Clone the repository
-git clone https://github.com/ruvnet/agentic-flow.git
-cd agentic-flow/agentic-flow/examples/crispr-cas13-pipeline
-
-# Build all workspace crates
-cargo build --release
-
-# Run tests (50 tests: integration + property-based)
-cargo test
-
-# Run benchmarks
-cargo bench
-```
-
-### Docker Deployment
-
-```bash
-# Build Docker images
-docker-compose build
-
-# Start all services (API + databases + monitoring)
-docker-compose up -d
-
-# Check service health
-curl http://localhost:8080/health
-```
-
-### Kubernetes Deployment
-
-```bash
-# Create namespace
-kubectl apply -f docs/deployment/namespace.yaml
-
-# Deploy databases
-kubectl apply -f docs/deployment/postgresql.yaml
-kubectl apply -f docs/deployment/mongodb.yaml
-kubectl apply -f docs/deployment/redis.yaml
-
-# Deploy microservices
-kubectl apply -f docs/deployment/api-gateway.yaml
-kubectl apply -f docs/deployment/alignment-service.yaml
-kubectl apply -f docs/deployment/off-target-service.yaml
-kubectl apply -f docs/deployment/diff-expr-service.yaml
-
-# Deploy monitoring
-kubectl apply -f docs/monitoring/prometheus.yml
-kubectl apply -f docs/monitoring/grafana-dashboard.json
-```
-
----
-
-## 🚀 Quick Start
-
-### 1️⃣ Basic Off-Target Prediction
-
-```rust
-use crispr_cas13_pipeline::prelude::*;
-
-#[tokio::main]
-async fn main() -> Result<()> {
-    // Create CRISPR target
-    let location = GenomicCoordinate::new(0, 1000, 1023, true)?;
-    let target = CrisprTarget::new(
-        "GUUUUAGAGCUAUGCUGUUUUG".to_string(),
-        "GUUUUAGAGCUAUGCUGUUUUG".to_string(),
-        location,
-    )?;
-
-    // Configure predictor
-    let config = PredictionConfig {
-        reference_genome: "rheMac10".to_string(),
-        max_mismatches: 3,
-        min_score: 0.3,
-        use_ml_model: true,
-        ..Default::default()
-    };
-
-    // Run prediction
-    let predictor = DefaultPredictor::new(config)?;
-    let prediction = predictor.predict(&target).await?;
-
-    // Print top off-target sites
-    for site in prediction.off_targets.iter().take(10) {
-        println!("Site: {} (score: {:.3}, mismatches: {})",
-                 site.sequence, site.score, site.mismatches);
-    }
-
-    Ok(())
-}
-```
-
-### 2️⃣ Immune Response Analysis
-
-```rust
-use crispr_cas13_pipeline::prelude::*;
-
-fn main() -> Result<()> {
-    // Load expression samples
-    let control_samples = load_samples("control_*.tsv")?;
-    let treatment_samples = load_samples("cas13_treated_*.tsv")?;
-
-    // Configure analysis
-    let params = AnalysisParameters {
-        min_counts: 10,
-        fdr_threshold: 0.05,
-        log2fc_threshold: 1.5,
-        ..Default::default()
-    };
-
-    // Run differential expression
-    let analyzer = DeseqAnalyzer::new(params);
-    let analysis = analyzer.analyze(control_samples, treatment_samples)?;
-
-    // Extract immune genes
-    let immune_genes = analysis.results.iter()
-        .filter(|r| r.padj < 0.05 && r.log2_fold_change.abs() > 1.5)
-        .filter(|r| is_immune_gene(&r.gene_id))
-        .collect::<Vec<_>>();
-
-    println!("Found {} significantly changed immune genes", immune_genes.len());
-
-    // Pathway enrichment
-    let enrichment = pathway_enrichment(&immune_genes, "Type_I_IFN")?;
-    println!("Type I IFN pathway enrichment: p = {:.2e}", enrichment.pvalue);
-
-    Ok(())
-}
-```
-
-### 3️⃣ Complete Pipeline Workflow
-
-```rust
-use crispr_cas13_pipeline::prelude::*;
-
-#[tokio::main]
-async fn main() -> Result<()> {
-    // 1. Submit processing job
-    let job = JobConfig {
-        input_files: vec!["sample_R1.fastq.gz", "sample_R2.fastq.gz"],
-        reference_genome: "rheMac10",
-        guide_rna: "GUUUUAGAGCUAUGCUGUUUUG",
-        cas13_variant: Cas13Variant::Cas13b,
-        ..Default::default()
-    };
-
-    let job_id = submit_job(job).await?;
-    println!("Job submitted: {}", job_id);
-
-    // 2. Monitor progress
-    let mut receiver = subscribe_to_job_updates(job_id).await?;
-    while let Some(update) = receiver.recv().await {
-        println!("Progress: {}% - {}",
-                 update.progress * 100.0,
-                 update.message);
-    }
-
-    // 3. Retrieve results
-    let results = get_job_results(job_id).await?;
-    println!("Alignment rate: {:.1}%", results.qc_metrics.alignment_rate * 100.0);
-    println!("Knockdown efficiency: {:.1}%", results.target_analysis.knockdown_efficiency * 100.0);
-    println!("Top off-targets: {}", results.off_targets.len());
-    println!("Immune response score: {:.2}", results.immune_response.interferon_score);
-
-    // 4. Download result files
-    download_file(&results.files.alignment_bam, "aligned.bam").await?;
-    download_file(&results.files.expression_matrix, "expression.tsv").await?;
-
-    Ok(())
-}
-```
-
----
-
-## 💡 Usage Examples
-
-### Batch Processing
-
-```rust
-use crispr_cas13_pipeline::prelude::*;
-use rayon::prelude::*;
-
-fn process_plate(samples: Vec<Sample>) -> Result<Vec<OffTargetPrediction>> {
-    // Parallel processing with Rayon
-    let predictions: Vec<_> = samples
-        .par_iter()
-        .map(|sample| {
-            let target = sample.to_crispr_target()?;
-            let predictor = DefaultPredictor::new(Default::default())?;
-            predictor.predict(&target).await
-        })
-        .collect::<Result<Vec<_>>>()?;
-
-    Ok(predictions)
-}
-```
-
-### Custom Scoring Models
-
-```rust
-use crispr_cas13_pipeline::ml_model::MlModel;
-
-// Load custom ONNX model
-let model = OnnxModel::load("custom_offtarget_model.onnx")?;
-
-let config = PredictionConfig {
-    use_ml_model: true,
-    model_path: Some("custom_offtarget_model.onnx".into()),
-    ..Default::default()
-};
-
-let predictor = DefaultPredictor::new(config)?;
-```
-
-### Real-Time Monitoring
-
-```rust
-use tokio_tungstenite::connect_async;
-
-// WebSocket connection for live updates
-let (ws_stream, _) = connect_async("ws://localhost:8080/ws/jobs").await?;
-
-while let Some(msg) = ws_stream.next().await {
-    let update: JobUpdate = serde_json::from_str(&msg?.to_text()?)?;
-    println!("Job {} - {}% complete", update.job_id, update.progress * 100.0);
-}
-```
-
----
-
-## 📊 Performance
-
-### Benchmarks (vs Industry Baseline)
-
-| Operation | This Pipeline | Industry Standard | Speedup |
-|-----------|--------------|-------------------|---------|
-| **Read Alignment** | 12,500 reads/sec | 4,000 reads/sec | **3.1x** |
-| **Off-Target Prediction** | 145,000 sites/sec | 35,000 sites/sec | **4.1x** |
-| **Differential Expression** | 18 seconds | 75 seconds | **4.2x** |
-| **API Latency (p95)** | 120ms | 650ms | **5.4x** |
-| **End-to-End Pipeline** | 22 minutes | 3.5 hours | **9.5x** |
-
-*Benchmarked on AWS c6i.4xlarge (16 vCPU, 32GB RAM) with 30M paired-end reads*
-
-### Memory Efficiency
-
-- **Peak Memory**: 8.2 GB (vs 24 GB for Python/R equivalent)
-- **Disk I/O**: 85% reduction via streaming processing
-- **Cache Hit Rate**: 94% for reference genome lookups
-
-Run benchmarks yourself:
-
-```bash
-cargo bench --bench alignment_benchmark
-cargo bench --bench offtarget_prediction_benchmark
-cargo bench --bench immune_analysis_benchmark
-```
-
----
-
-## 🏗️ Architecture
-
-### System Overview
-
-```
-┌─────────────────────────────────────────────────────────┐
-│                    UI Layer (React)                      │
-│         Interactive visualization & monitoring           │
-└─────────────────────────────────────────────────────────┘
-                            │
-                            ▼
-┌─────────────────────────────────────────────────────────┐
-│                 API Layer (Axum/Rust)                   │
-│      RESTful APIs with versioning & authentication      │
-└─────────────────────────────────────────────────────────┘
-                            │
-                            ▼
-┌─────────────────────────────────────────────────────────┐
-│            Processing Layer (Kubernetes)                 │
-│  ┌─────────────┬─────────────┬──────────────────┐      │
-│  │ Alignment   │ Off-Target  │ Immune Analysis  │      │
-│  │ Service     │ Service     │ Service          │      │
-│  └─────────────┴─────────────┴──────────────────┘      │
-└─────────────────────────────────────────────────────────┘
-                            │
-                            ▼
-┌─────────────────────────────────────────────────────────┐
-│                   Data Layer                             │
-│  ┌──────────┬──────────┬─────────┬──────────┐          │
-│  │PostgreSQL│ MongoDB  │  Redis  │  MinIO   │          │
-│  │(metadata)│(genomics)│(cache)  │(storage) │          │
-│  └──────────┴──────────┴─────────┴──────────┘          │
-└─────────────────────────────────────────────────────────┘
-```
-
-### Workspace Crates
-
-- **`data-models`**: Shared data structures (FASTQ, BAM, targets, expression)
-- **`alignment-engine`**: Read alignment with BWA/STAR integration
-- **`offtarget-predictor`**: ML-based off-target prediction and scoring
-- **`immune-analyzer`**: Statistical analysis and pathway enrichment
-- **`api-service`**: Axum REST API with OpenAPI documentation
-- **`processing-orchestrator`**: Job queue management and scheduling
-
----
-
-## 🛠️ Development
-
-### Project Structure
-
-```
-crispr-cas13-pipeline/
-├── Cargo.toml                  # Workspace configuration
-├── crates/                     # Rust workspace crates
-│   ├── data-models/            # Core data structures
-│   ├── alignment-engine/       # Alignment algorithms
-│   ├── offtarget-predictor/    # Off-target prediction
-│   ├── immune-analyzer/        # Statistical analysis
-│   ├── api-service/            # REST API server
-│   └── processing-orchestrator/# Job orchestration
-├── docs/                       # Comprehensive documentation
-│   ├── SPECIFICATION.md        # System requirements
-│   ├── ARCHITECTURE.md         # Architecture design
-│   ├── PSEUDOCODE.md           # Algorithm details
-│   ├── api-spec.openapi.yaml   # OpenAPI specification
-│   └── deployment/             # Kubernetes manifests
-├── tests/                      # Integration tests
-├── benches/                    # Performance benchmarks
-└── examples/                   # Usage examples
-```
-
-### Testing
-
-```bash
-# Unit tests (32 tests)
-cargo test --lib
-
-# Integration tests (24 tests)
-cargo test --test integration_test
-
-# Property-based tests (26 tests)
-cargo test --test property_tests
-
-# All tests
-cargo test --all
-
-# With coverage
-cargo tarpaulin --out Html
-```
-
-### Code Quality
-
-```bash
-# Linting
-cargo clippy -- -D warnings
-
-# Formatting
-cargo fmt --all -- --check
-
-# Security audit
-cargo audit
-
-# Unused dependencies
-cargo machete
-```
-
----
-
-## 📚 Documentation
-
-- **[System Specification](docs/SPECIFICATION.md)**: Complete requirements (720 lines)
-- **[Architecture Guide](docs/ARCHITECTURE.md)**: System design deep-dive (35,000+ lines)
-- **[Algorithm Documentation](docs/PSEUDOCODE.md)**: Detailed algorithms (5,781 lines)
-- **[API Reference](https://docs.rs/crispr-cas13-pipeline)**: Rustdoc API documentation
-- **[Deployment Guide](docs/deployment/)**: Kubernetes setup instructions
-- **[Benchmarks Report](docs/BENCHMARKS.md)**: Performance analysis
-
----
-
-## 🤝 Contributing
-
-We welcome contributions! This project was built using **[SPARC methodology](https://github.com/ruvnet/agentic-flow)** and **[rUv.io](https://ruv.io)** AI coordination.
-
-### Development Workflow
-
-1. Fork the repository
-2. Create a feature branch (`git checkout -b feature/amazing-feature`)
-3. Write tests first (TDD approach)
-4. Implement your feature
-5. Run tests and benchmarks
-6. Commit your changes (`git commit -m 'Add amazing feature'`)
-7. Push to the branch (`git push origin feature/amazing-feature`)
-8. Open a Pull Request
-
-### Code Standards
-
-- Minimum 80% test coverage
-- All tests must pass
-- Clippy warnings resolved
-- Rustfmt formatting applied
-- Documentation updated
-
-See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
-
----
-
-## 📜 License
-
-This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.
-
----
-
-## 🙏 Acknowledgments
-
-### Built With
-
-- **[SPARC Methodology](https://github.com/ruvnet/agentic-flow)**: Systematic development approach (Specification → Pseudocode → Architecture → Refinement → Completion)
-- **[rUv.io Platform](https://ruv.io)**: AI coordination and swarm orchestration
-- **[Agentic Flow](https://github.com/ruvnet/agentic-flow)**: Multi-agent development framework
-
-### Scientific References
-
-1. Abudayyeh, O. O., et al. (2017). "C2c2 is a single-component programmable RNA-guided RNA-targeting CRISPR effector." *Science*, 353(6299).
-2. Doench, J. G., et al. (2016). "Optimized sgRNA design to maximize activity and minimize off-target effects of CRISPR-Cas9." *Nature Biotechnology*, 34(2).
-3. Love, M. I., et al. (2014). "Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2." *Genome Biology*, 15(12).
-
-### Technology Stack
-
-- **Rust**: High-performance systems programming language
-- **Tokio**: Async runtime for concurrent I/O
-- **Axum**: Web framework for REST APIs
-- **PostgreSQL**: Relational database for metadata
-- **MongoDB**: Document database for genomic data
-- **Kubernetes**: Container orchestration
-- **Prometheus + Grafana**: Monitoring and visualization
-
----
-
-## 🔗 Links
-
-- **Homepage**: [https://ruv.io](https://ruv.io)
-- **Repository**: [https://github.com/ruvnet/agentic-flow](https://github.com/ruvnet/agentic-flow)
-- **Documentation**: [https://docs.rs/crispr-cas13-pipeline](https://docs.rs/crispr-cas13-pipeline)
-- **Crates.io**: [https://crates.io/crates/crispr-cas13-pipeline](https://crates.io/crates/crispr-cas13-pipeline)
-- **Issue Tracker**: [https://github.com/ruvnet/agentic-flow/issues](https://github.com/ruvnet/agentic-flow/issues)
-- **Discussions**: [https://github.com/ruvnet/agentic-flow/discussions](https://github.com/ruvnet/agentic-flow/discussions)
-
----
-
-## 📈 Roadmap
-
-### v0.2.0 (Q2 2025)
-- [ ] Real-world validation with primate study data
-- [ ] Additional Cas13 variants (Cas13X, Cas13Y)
-- [ ] GPU acceleration for ML models
-- [ ] Frontend UI (React + WebAssembly)
-
-### v0.3.0 (Q3 2025)
-- [ ] Multi-species support (mouse, zebrafish)
-- [ ] Integration with external databases (dbGaP, GEO)
-- [ ] Advanced pathway analysis (KEGG, Reactome)
-- [ ] Cloud-native deployment (AWS, GCP, Azure)
-
-### v1.0.0 (Q4 2025)
-- [ ] Clinical trial support
-- [ ] HIPAA certification
-- [ ] Commercial licensing options
-- [ ] Enterprise support packages
-
----
-
-## 💬 Support
-
-- **Community Discord**: [Join our Discord](https://discord.gg/ruv)
-- **Email**: support@ruv.io
-- **GitHub Issues**: [Report a bug](https://github.com/ruvnet/agentic-flow/issues/new?template=bug_report.md)
-- **Feature Requests**: [Request a feature](https://github.com/ruvnet/agentic-flow/issues/new?template=feature_request.md)
-
----
-
-## ⭐ Star History
-
-[![Star History Chart](https://api.star-history.com/svg?repos=ruvnet/agentic-flow&type=Date)](https://star-history.com/#ruvnet/agentic-flow&Date)
-
----
-
-<div align="center">
-
-**Built with ❤️ using [SPARC](https://github.com/ruvnet/agentic-flow) + [rUv.io](https://ruv.io)**
-
-[⬆ Back to Top](#-crispr-cas13-bioinformatics-pipeline)
-
-</div>