npm - agentic-flow - Versions diffs - 1.9.0 → 1.9.2 - Mend

agentic-flow 1.9.0 → 1.9.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (1223) hide show

package/agentic-flow/examples/crispr-cas13-pipeline/docs/ARCHITECTURAL_DECISIONS.md DELETED Viewed

@@ -1,429 +0,0 @@
-# Architectural Decision Records (ADRs)
-## Overview
-This document records significant architectural decisions made for the CRISPR-Cas13 bioinformatics pipeline. Each decision includes context, alternatives considered, and the rationale for the chosen solution.
----
-## ADR-001: Microservices Architecture
-**Status**: Accepted
-**Date**: 2025-10-12
-**Decision Makers**: Architecture Agent, Technical Team
-### Context
-The CRISPR-Cas13 pipeline must process large volumes of genomic data with varying computational requirements (CPU-intensive alignment, memory-intensive differential expression, I/O-intensive data storage).
-### Decision
-Adopt a **microservices architecture** with independent services for each processing stage (alignment, off-target prediction, differential expression, immune response analysis).
-### Alternatives Considered
-1. **Monolithic Application**
-   - **Pros**: Simpler deployment, lower operational overhead
-   - **Cons**: Cannot scale components independently, single point of failure
-2. **Serverless Functions (AWS Lambda)**
-   - **Pros**: Auto-scaling, pay-per-execution
-   - **Cons**: 15-minute execution limit, cold starts, vendor lock-in
-3. **Microservices** ✅
-   - **Pros**: Independent scaling, fault isolation, technology flexibility
-   - **Cons**: Operational complexity, network overhead
-### Rationale
-- **Scalability**: Alignment service can scale to 20 replicas during peak load, while API gateway runs 3 replicas
-- **Fault Isolation**: Failure in off-target predictor doesn't affect differential expression
-- **Technology Flexibility**: Use R for statistical analysis, Python for ML, Rust for high-performance API
-### Consequences
-- **Positive**: 10x throughput improvement via horizontal scaling
-- **Negative**: Increased monitoring complexity (distributed tracing required)
----
-## ADR-002: Rust for API Gateway
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-API gateway handles ~1000 requests/sec with strict latency requirements (P95 < 200ms).
-### Decision
-Implement API gateway in **Rust with Axum framework**.
-### Alternatives Considered
-1. **Node.js (Express/Fastify)**
-   - **Pros**: Large ecosystem, rapid development
-   - **Cons**: 5x slower than Rust, garbage collection pauses
-2. **Go (Gin/Echo)**
-   - **Pros**: Good performance, built-in concurrency
-   - **Cons**: Less type safety than Rust, no memory safety guarantees
-3. **Rust (Axum)** ✅
-   - **Pros**: ~10x faster than Node.js, memory safety, zero-cost abstractions
-   - **Cons**: Steeper learning curve, longer compile times
-### Rationale
-- **Performance**: Benchmarks show 5-10ms median latency vs. 50ms for Node.js
-- **Safety**: No null pointers, no data races, no buffer overflows
-- **Efficiency**: 100MB memory footprint vs. 500MB for Node.js
-### Consequences
-- **Positive**: Lower infrastructure costs (fewer pods needed)
-- **Negative**: Longer onboarding time for developers unfamiliar with Rust
----
-## ADR-003: Apache Kafka for Job Orchestration
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need reliable message queue to decouple API from processing workers, handle job retries, and provide audit trail.
-### Decision
-Use **Apache Kafka** for job orchestration.
-### Alternatives Considered
-1. **RabbitMQ**
-   - **Pros**: Simpler setup, richer routing
-   - **Cons**: Lower throughput, no message replay
-2. **AWS SQS**
-   - **Pros**: Fully managed, auto-scaling
-   - **Cons**: Vendor lock-in, no message ordering
-3. **Apache Kafka** ✅
-   - **Pros**: High throughput (1M+ messages/sec), message replay, log-based architecture
-   - **Cons**: More complex setup and operations
-### Rationale
-- **Replay Capability**: Can re-process failed jobs from any offset
-- **Audit Trail**: All job events stored for 30 days
-- **Throughput**: Handles 10,000 jobs/day with <1s latency
-### Consequences
-- **Positive**: Zero job loss during system failures
-- **Negative**: Requires dedicated Kafka cluster (3+ brokers)
----
-## ADR-004: PostgreSQL + MongoDB Dual Database Strategy
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Data has both structured (experiments, samples, results) and semi-structured (genomic annotations, gene ontology) components.
-### Decision
-Use **PostgreSQL for relational data** and **MongoDB for document-based genomic data**.
-### Alternatives Considered
-1. **PostgreSQL Only (with JSONB)**
-   - **Pros**: Single database, ACID guarantees
-   - **Cons**: Poor performance for nested document queries
-2. **MongoDB Only**
-   - **Pros**: Flexible schema, horizontal scaling
-   - **Cons**: Weaker consistency, no foreign keys
-3. **PostgreSQL + MongoDB** ✅
-   - **Pros**: Best tool for each data type, optimized queries
-   - **Cons**: Two databases to maintain
-### Rationale
-- **PostgreSQL**: Perfect for experiments, samples, users (relational integrity critical)
-- **MongoDB**: Ideal for gene annotations (variable schema, nested documents)
-- **Performance**: PostgreSQL query time <50ms, MongoDB aggregation <100ms
-### Consequences
-- **Positive**: 3x faster genomic annotation queries
-- **Negative**: Data consistency requires distributed transactions (Saga pattern)
----
-## ADR-005: Kubernetes for Container Orchestration
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need container orchestration platform with auto-scaling, self-healing, and multi-cloud support.
-### Decision
-Deploy on **Kubernetes (v1.28+)**.
-### Alternatives Considered
-1. **Docker Swarm**
-   - **Pros**: Simpler than Kubernetes
-   - **Cons**: Limited ecosystem, less robust auto-scaling
-2. **AWS ECS**
-   - **Pros**: Deep AWS integration, simpler than K8s
-   - **Cons**: Vendor lock-in, limited multi-cloud
-3. **Kubernetes** ✅
-   - **Pros**: Industry standard, rich ecosystem, multi-cloud
-   - **Cons**: Steep learning curve, operational complexity
-### Rationale
-- **Auto-Scaling**: HPA scales alignment service from 2 to 20 pods based on CPU and queue depth
-- **Self-Healing**: Crashed pods automatically restarted
-- **Multi-Cloud**: Can run on AWS, GCP, or on-premise with same manifests
-### Consequences
-- **Positive**: 99.9% uptime with rolling updates
-- **Negative**: Requires dedicated DevOps expertise
----
-## ADR-006: Redis for Caching & Rate Limiting
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need sub-millisecond cache lookups for sessions, job statuses, and rate limiting.
-### Decision
-Use **Redis Cluster (6 nodes: 3 masters + 3 replicas)**.
-### Alternatives Considered
-1. **Memcached**
-   - **Pros**: Simpler, slightly faster
-   - **Cons**: No persistence, no complex data structures
-2. **In-Memory (HashMap)**
-   - **Pros**: Fastest (no network)
-   - **Cons**: Lost on pod restart, no sharing between pods
-3. **Redis** ✅
-   - **Pros**: Persistence, complex data types (sorted sets, hashes), pub/sub
-   - **Cons**: Slightly slower than Memcached
-### Rationale
-- **Performance**: GET operation <1ms
-- **Persistence**: AOF + RDB snapshots prevent data loss
-- **Rate Limiting**: Sliding window algorithm using sorted sets
-### Consequences
-- **Positive**: 50% reduction in database load
-- **Negative**: Redis cluster adds operational overhead
----
-## ADR-007: MinIO for Object Storage
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Store large sequencing files (FASTQ, BAM) totaling 50-100TB with S3-compatible API.
-### Decision
-Deploy **MinIO in distributed mode** (4 nodes × 4 drives, EC:4 erasure coding).
-### Alternatives Considered
-1. **AWS S3**
-   - **Pros**: Fully managed, 99.999999999% durability
-   - **Cons**: High egress costs ($0.09/GB), vendor lock-in
-2. **Ceph**
-   - **Pros**: Open source, proven at scale
-   - **Cons**: Complex setup, requires dedicated team
-3. **MinIO** ✅
-   - **Pros**: S3-compatible, easy deployment, lower cost
-   - **Cons**: Requires self-management
-### Rationale
-- **Cost**: $0.02/GB vs. $0.023/GB for S3 (no egress fees)
-- **Performance**: 10 Gbps throughput, <10ms latency
-- **Portability**: S3-compatible API, easy migration if needed
-### Consequences
-- **Positive**: 80% cost savings on storage
-- **Negative**: Must manage backups and disaster recovery
----
-## ADR-008: Prometheus + Grafana for Monitoring
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need comprehensive monitoring of 50+ microservices, databases, and infrastructure.
-### Decision
-Use **Prometheus for metrics collection** and **Grafana for visualization**.
-### Alternatives Considered
-1. **Datadog**
-   - **Pros**: Fully managed, great UX
-   - **Cons**: Expensive ($15/host/month), vendor lock-in
-2. **ELK Stack (Elasticsearch + Kibana)**
-   - **Pros**: Powerful log analysis
-   - **Cons**: Resource-intensive, not optimized for metrics
-3. **Prometheus + Grafana** ✅
-   - **Pros**: Open source, Kubernetes-native, PromQL query language
-   - **Cons**: Requires self-hosting
-### Rationale
-- **Integration**: Native Kubernetes support, service discovery
-- **Query Language**: PromQL enables complex aggregations
-- **Cost**: Free (self-hosted) vs. $10,000/month for Datadog
-### Consequences
-- **Positive**: 100% cost savings vs. SaaS monitoring
-- **Negative**: Must manage Prometheus/Grafana infrastructure
----
-## ADR-009: JWT with RS256 for Authentication
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need stateless authentication for REST API with role-based access control.
-### Decision
-Use **JWT tokens signed with RS256** (RSA public/private key).
-### Alternatives Considered
-1. **Session Cookies (Server-Side)**
-   - **Pros**: Simpler revocation
-   - **Cons**: Requires sticky sessions or shared session store
-2. **JWT with HS256 (HMAC)**
-   - **Pros**: Simpler (symmetric key)
-   - **Cons**: All services need secret key (security risk)
-3. **JWT with RS256** ✅
-   - **Pros**: Public key distribution via JWKS, stateless validation
-   - **Cons**: Slightly slower than HS256
-### Rationale
-- **Stateless**: No database lookup on every request
-- **Security**: Private key only on auth service, public key distributed
-- **Scalability**: API gateway validates tokens without auth service
-### Consequences
-- **Positive**: <1ms token validation overhead
-- **Negative**: Token revocation requires blacklist (Redis)
----
-## ADR-010: React + TypeScript for Frontend
-**Status**: Accepted
-**Date**: 2025-10-12
-### Context
-Need modern, interactive UI for experiment management and result visualization.
-### Decision
-Build frontend with **React 18 + TypeScript + Vite**.
-### Alternatives Considered
-1. **Vue.js**
-   - **Pros**: Simpler learning curve, better docs
-   - **Cons**: Smaller ecosystem, fewer enterprise adoptions
-2. **Svelte**
-   - **Pros**: Smaller bundle size, faster runtime
-   - **Cons**: Smaller ecosystem, fewer third-party libraries
-3. **React + TypeScript** ✅
-   - **Pros**: Largest ecosystem, strong typing, component reusability
-   - **Cons**: Larger bundle size (mitigated by code splitting)
-### Rationale
-- **Ecosystem**: Best visualization libraries (D3.js, Plotly, IGV.js)
-- **Type Safety**: TypeScript prevents runtime errors
-- **Developer Experience**: Vite provides <1s HMR
-### Consequences
-- **Positive**: Faster development with component libraries (Shadcn/ui)
-- **Negative**: 500KB bundle size (gzipped: 150KB)
----
-## Summary Table
-| ADR | Decision | Status | Impact |
-|-----|----------|--------|--------|
-| ADR-001 | Microservices Architecture | Accepted | 10x throughput |
-| ADR-002 | Rust for API Gateway | Accepted | 5-10x faster |
-| ADR-003 | Apache Kafka | Accepted | Zero job loss |
-| ADR-004 | PostgreSQL + MongoDB | Accepted | 3x faster queries |
-| ADR-005 | Kubernetes | Accepted | 99.9% uptime |
-| ADR-006 | Redis Cluster | Accepted | 50% DB load reduction |
-| ADR-007 | MinIO | Accepted | 80% cost savings |
-| ADR-008 | Prometheus + Grafana | Accepted | 100% cost savings |
-| ADR-009 | JWT with RS256 | Accepted | <1ms validation |
-| ADR-010 | React + TypeScript | Accepted | Faster development |
----
-**Document Version**: 1.0
-**Last Updated**: 2025-10-12
-**See Also**: [Architecture Overview](./ARCHITECTURE.md)