@adminide-stack/yantra-help-browser 12.0.16-alpha.6 → 12.0.16

package/lib/templates/content/technical-questions/data-processing.md

@@ -0,0 +1,102 @@
+# Data Processing Pipeline
+
+How Yantra ingests, transforms, analyzes, and stores your data — from raw input to search-ready content.
+
+---
+
+## Data Ingestion
+
+Yantra supports multiple ingestion pathways to accommodate diverse data sources:
+
+| Source       | Method                  | Details                                                                 |
+| ------------ | ----------------------- | ----------------------------------------------------------------------- |
+| Web content  | Automated crawling      | Configurable crawl schedules, depth limits, and domain allowlists       |
+| APIs         | REST/GraphQL connectors | Pre-built connectors for 50+ services (Slack, Notion, Confluence, etc.) |
+| File uploads | Drag-and-drop or API    | Supports PDF, DOCX, PPTX, CSV, Markdown, HTML, and plain text           |
+| Databases    | Direct connectors       | PostgreSQL, MySQL, MongoDB read-only connectors with incremental sync   |
+
+### Ingestion guarantees
+
+- **Exactly-once processing** — Deduplication ensures no content is indexed twice.
+- **Incremental updates** — Only new or modified content is re-processed, minimizing compute costs.
+- **Schema validation** — Incoming data is validated against expected schemas before entering the pipeline.
+
+---
+
+## Processing Architecture
+
+### Stream processing
+
+Yantra uses an event-driven architecture built on **Apache Kafka** for real-time data flow:
+
+1. **Producers** publish raw content events to topic partitions.
+2. **Stream processors** (Kafka Streams) consume events, apply transformations, and emit enriched records.
+3. **Consumers** write processed data to the appropriate storage layer (PostgreSQL, Elasticsearch, Redis).
+
+### Batch processing
+
+For large-volume imports or periodic re-indexing, Yantra runs batch jobs using distributed task queues:
+
+- Jobs are broken into chunks of 1,000 records.
+- Each chunk is processed in parallel across worker nodes.
+- Progress is tracked in real-time and visible in the Admin Dashboard.
+
+---
+
+## Content Analysis
+
+Every piece of content passes through a multi-stage analysis pipeline:
+
+### Text processing
+
+- **Language detection** — Automatically identifies 50+ languages.
+- **Tokenization** — Text is split into meaningful tokens using language-specific tokenizers.
+- **Normalization** — Unicode normalization, lowercasing, and stop-word removal.
+
+### Semantic analysis
+
+- **Embedding generation** — Each document is converted to a high-dimensional vector using state-of-the-art embedding models.
+- **Topic modeling** — Latent topics are extracted using LDA and transformer-based approaches.
+- **Entity recognition** — People, organizations, dates, locations, and custom entities are identified and tagged.
+- **Sentiment analysis** — Content sentiment (positive, negative, neutral) is scored for analytics.
+
+### Quality scoring
+
+| Signal           | Weight | Description                                      |
+| ---------------- | ------ | ------------------------------------------------ |
+| Completeness     | 25%    | Does the content cover its topic thoroughly?     |
+| Freshness        | 20%    | How recently was the content created or updated? |
+| Source authority | 30%    | How trustworthy is the source domain?            |
+| Readability      | 15%    | Flesch-Kincaid score and structural quality      |
+| Uniqueness       | 10%    | Duplicate and near-duplicate detection           |
+
+---
+
+## Data Storage
+
+### Multi-tier storage architecture
+
+| Tier | Technology                   | Purpose                                    | Retention            |
+| ---- | ---------------------------- | ------------------------------------------ | -------------------- |
+| Hot  | PostgreSQL + Redis           | Active queries, user data, real-time cache | Indefinite           |
+| Warm | Elasticsearch                | Full-text search index, vector search      | Indefinite           |
+| Cold | S3-compatible object storage | Archived content, raw uploads, backups     | Per retention policy |
+
+### Data lifecycle
+
+1. **Ingest** — Raw data lands in the processing queue.
+2. **Process** — Content is analyzed, enriched, and indexed.
+3. **Serve** — Processed data is available for search and retrieval.
+4. **Archive** — Older content is compressed and moved to cold storage based on access patterns.
+5. **Delete** — Data past its retention period is securely purged (overwritten + cryptographic erasure).
+
+---
+
+## Performance & Scalability
+
+- **Throughput** — The pipeline processes 10,000+ documents per minute at steady state.
+- **Latency** — End-to-end ingestion-to-searchable time is under 30 seconds for real-time sources.
+- **Horizontal scaling** — Worker nodes auto-scale based on queue depth. During peak loads, the system scales from 4 to 32 workers automatically.
+- **Backpressure handling** — If downstream systems slow down, the pipeline applies backpressure to producers rather than dropping data.
+
+> **Enterprise customers** can configure custom processing rules, retention policies, and data routing via the Admin Dashboard.

package/lib/templates/content/technical-questions/database-architecture.md

@@ -0,0 +1,120 @@
+# Database Architecture
+
+How Yantra stores, indexes, and retrieves data across multiple database systems optimized for different access patterns.
+
+---
+
+## Database Design Philosophy
+
+Yantra follows a **polyglot persistence** strategy — each data type is stored in the database engine best suited for its access pattern:
+
+| Data type                      | Engine                | Why                                                     |
+| ------------------------------ | --------------------- | ------------------------------------------------------- |
+| User accounts, billing, config | PostgreSQL 16         | ACID transactions, relational integrity, mature tooling |
+| Full-text + vector search      | Elasticsearch 8.x     | Inverted indexes, BM25 ranking, kNN vector search       |
+| Sessions, cache, rate limits   | Redis 7               | Sub-millisecond reads, TTL support, pub/sub             |
+| File uploads, backups          | S3-compatible storage | Virtually unlimited capacity, 11 nines durability       |
+
+---
+
+## PostgreSQL — Primary Database
+
+### Schema design principles
+
+- **Normalized core tables** — Users, organizations, subscriptions, and permissions follow 3NF to avoid data anomalies.
+- **JSONB for flexibility** — Metadata, user preferences, and integration configs use JSONB columns, combining schema flexibility with indexing support.
+- **Timestamped everything** — Every table includes `created_at` and `updated_at` columns with timezone-aware timestamps.
+- **Soft deletes** — Records are marked as deleted rather than physically removed, enabling audit trails and data recovery.
+
+### Indexing strategy
+
+| Index type | Use case                   | Example                                            |
+| ---------- | -------------------------- | -------------------------------------------------- |
+| B-tree     | Equality and range queries | `WHERE created_at > '2026-01-01'`                  |
+| GIN        | JSONB containment queries  | `WHERE metadata @> '{"type": "pdf"}'`              |
+| Partial    | Hot data subsets           | `WHERE status = 'active'` (index only active rows) |
+| Covering   | Avoid table lookups        | Include all `SELECT` columns in the index          |
+
+### High availability
+
+- **Streaming replication** — One synchronous standby + two async replicas.
+- **Automatic failover** — Patroni manages leader election; failover completes in < 10 seconds.
+- **Point-in-time recovery** — WAL archiving enables recovery to any second within the retention window (30 days).
+
+---
+
+## Elasticsearch — Search Index
+
+### Index architecture
+
+Each content type has its own Elasticsearch index with optimized mappings:
+
+- **Text fields** use `text` type with custom analyzers (language-specific stemming, synonym expansion).
+- **Vector fields** use `dense_vector` type for kNN semantic search.
+- **Keyword fields** for exact-match filtering (tags, content type, source).
+- **Date fields** for time-range queries and recency boosting.
+
+### Cluster topology
+
+- **3 dedicated master nodes** for cluster coordination.
+- **6+ data nodes** with SSD storage for search performance.
+- **2 coordinating nodes** for query routing and result aggregation.
+
+---
+
+## Redis — Cache & Real-Time
+
+### Cache patterns
+
+| Pattern             | Use case                     | TTL                     |
+| ------------------- | ---------------------------- | ----------------------- |
+| Query result cache  | Identical search queries     | 5 minutes               |
+| Session store       | User authentication sessions | 1 hour                  |
+| Rate limit counters | API rate limiting            | Rolling 1-minute window |
+| Pub/Sub channels    | Real-time notifications      | N/A (ephemeral)         |
+
+### Memory management
+
+- **Maxmemory policy** set to `allkeys-lru` — least recently used keys are evicted when memory limits are reached.
+- **Key namespacing** — All keys are prefixed by service name to avoid collisions.
+- **Cluster mode** — Redis Cluster with 6 nodes (3 primary + 3 replica) for horizontal scaling.
+
+---
+
+## Data Management
+
+### Backup strategy
+
+| What          | Frequency                   | Retention  | Method                          |
+| ------------- | --------------------------- | ---------- | ------------------------------- |
+| PostgreSQL    | Continuous WAL + daily full | 30 days    | pg_basebackup + WAL archiving   |
+| Elasticsearch | Daily snapshots             | 14 days    | Snapshot to S3                  |
+| Redis         | RDB snapshots + AOF         | 7 days     | Automated via Redis persistence |
+| File storage  | Cross-region replication    | Indefinite | S3 cross-region replication     |
+
+### Data governance
+
+- **Encryption at rest** — AES-256 for all database storage volumes.
+- **Encryption in transit** — TLS 1.3 for all database connections.
+- **Access control** — Database credentials are rotated monthly via HashiCorp Vault.
+- **Audit logging** — All schema changes and administrative queries are logged.
+
+---
+
+## Scalability
+
+### Current capacity
+
+| Metric                            | Value  |
+| --------------------------------- | ------ |
+| Total indexed documents           | 500M+  |
+| Database size (PostgreSQL)        | 2.4 TB |
+| Search index size (Elasticsearch) | 8.7 TB |
+| Peak queries per second           | 12,000 |
+| Average query latency             | 45 ms  |
+
+### Scaling strategy
+
+- **Vertical** — Increase instance sizes for immediate capacity (database-level).
+- **Horizontal** — Add read replicas (PostgreSQL), data nodes (Elasticsearch), or shard nodes (Redis) for linear scaling.
+- **Partitioning** — Time-based partitioning for PostgreSQL tables with high write volume.

package/lib/templates/content/technical-questions/infrastructure.md

@@ -0,0 +1,132 @@
+# Infrastructure & Reliability
+
+Yantra's cloud infrastructure is designed for high availability, global performance, and security at every layer.
+
+---
+
+## Cloud Architecture
+
+### Multi-region deployment
+
+Yantra runs across **3 AWS regions** (US East, EU West, AP Southeast) with active-active configuration. User traffic is routed to the nearest region via latency-based DNS routing.
+
+### Container orchestration
+
+All services run as Docker containers orchestrated by **Kubernetes (EKS)**:
+
+- **Namespaces** isolate production, staging, and development environments.
+- **Resource quotas** prevent any single service from consuming excessive cluster resources.
+- **Rolling deployments** ensure zero-downtime updates with automatic rollback on health check failures.
+- **Horizontal Pod Autoscaler** adjusts replica counts based on CPU, memory, and custom metrics.
+
+### Service mesh
+
+An Istio-based service mesh provides:
+
+- **Mutual TLS** between all services (zero-trust networking).
+- **Traffic management** — Canary deployments, circuit breaking, retry policies.
+- **Observability** — Distributed tracing with Jaeger, metrics with Prometheus.
+
+---
+
+## High Availability
+
+### Availability targets
+
+| Component      | Target SLA | Actual (trailing 12 months) |
+| -------------- | ---------- | --------------------------- |
+| API Gateway    | 99.99%     | 99.995%                     |
+| Search Service | 99.95%     | 99.98%                      |
+| AI Service     | 99.9%      | 99.94%                      |
+| Data Pipeline  | 99.9%      | 99.92%                      |
+
+### Redundancy design
+
+- **No single point of failure** — Every component has at least 2 replicas across different availability zones.
+- **Database failover** — Automated failover with < 10-second recovery for PostgreSQL and Redis.
+- **Cross-region replication** — Critical data is replicated across regions for disaster recovery.
+- **Graceful degradation** — If the AI service is unavailable, search still returns results without AI-generated summaries.
+
+### Disaster recovery
+
+| Metric                         | Target    | Actual     |
+| ------------------------------ | --------- | ---------- |
+| Recovery Time Objective (RTO)  | < 4 hours | 2.1 hours  |
+| Recovery Point Objective (RPO) | < 1 hour  | 15 minutes |
+| DR test frequency              | Quarterly | Monthly    |
+
+---
+
+## Monitoring & Observability
+
+### The three pillars
+
+| Pillar  | Tools                            | Details                                          |
+| ------- | -------------------------------- | ------------------------------------------------ |
+| Metrics | Prometheus + Grafana             | 2,000+ custom metrics, 15-second scrape interval |
+| Logs    | Fluentd + Elasticsearch + Kibana | Structured JSON logs, 30-day retention           |
+| Traces  | Jaeger + OpenTelemetry           | End-to-end request tracing across all services   |
+
+### Alerting
+
+- **PagerDuty integration** for critical alerts (P1/P2) with automatic escalation.
+- **Slack notifications** for warnings and informational alerts.
+- **Anomaly detection** — ML-based alerting detects unusual patterns before they become incidents.
+- **Runbooks** — Every alert links to a runbook with diagnosis steps and remediation procedures.
+
+---
+
+## Security Infrastructure
+
+### Network security
+
+- **VPC isolation** — All services run in private subnets with no direct internet access.
+- **WAF (Web Application Firewall)** — Protects against OWASP Top 10 threats.
+- **DDoS protection** — AWS Shield Advanced with automatic traffic scrubbing.
+- **Egress filtering** — Outbound traffic is restricted to known-good destinations.
+
+### Secrets management
+
+- **HashiCorp Vault** for all secrets, API keys, and database credentials.
+- **Automatic rotation** — Secrets are rotated on configurable schedules (default: 30 days).
+- **Just-in-time access** — Engineers request temporary elevated access via an approval workflow.
+
+---
+
+## CI/CD Pipeline
+
+### Deployment flow
+
+1. **Code push** — Developer pushes to a feature branch on GitHub.
+2. **CI checks** — Automated linting, type checking, unit tests, and integration tests run in GitHub Actions.
+3. **Build** — Docker images are built, scanned for vulnerabilities, and pushed to ECR.
+4. **Staging deploy** — ArgoCD deploys to the staging environment automatically.
+5. **QA validation** — Automated end-to-end tests + manual spot checks.
+6. **Production deploy** — Canary deployment to 5% of traffic, then gradual rollout to 100%.
+7. **Post-deploy monitoring** — Automated health checks verify error rates and latency for 30 minutes.
+
+### Deployment frequency
+
+- **Production deploys** — 8-12 per day across all services.
+- **Rollback time** — Under 60 seconds to previous version.
+- **Feature flags** — LaunchDarkly for gradual feature rollouts and instant kill switches.
+
+---
+
+## Global Performance
+
+### Content delivery
+
+- **200+ edge locations** via CloudFront CDN.
+- **Edge caching** for static assets with 85%+ cache hit rate.
+- **Dynamic content acceleration** — Optimized TCP connections and HTTP/3 support.
+
+### Latency optimization
+
+| Region         | Average API latency |
+| -------------- | ------------------- |
+| US East        | 35 ms               |
+| US West        | 52 ms               |
+| EU West        | 41 ms               |
+| AP Southeast   | 68 ms               |
+| Global average | 48 ms               |

package/lib/templates/content/technical-questions/performance-optimization.md

@@ -0,0 +1,105 @@
+# Performance Optimization
+
+How Yantra achieves sub-second response times at scale — and how you can get the most out of the platform.
+
+---
+
+## System Performance Targets
+
+| Metric                    | Target         | Current        |
+| ------------------------- | -------------- | -------------- |
+| Search latency (P50)      | < 200 ms       | 145 ms         |
+| Search latency (P99)      | < 1 s          | 780 ms         |
+| AI response latency (P50) | < 800 ms       | 620 ms         |
+| API uptime                | 99.95%         | 99.98%         |
+| Throughput                | 10,000 req/min | 14,200 req/min |
+
+---
+
+## Caching Strategy
+
+Yantra uses a **multi-layer caching architecture** to minimize redundant computation:
+
+### Layer 1 — Edge cache (CDN)
+
+Static assets and frequently requested API responses are cached at 200+ edge locations worldwide. Cache hit rates typically exceed 85%.
+
+### Layer 2 — Application cache (Redis)
+
+- **Query result cache** — Identical queries return cached results within 5 ms.
+- **Session cache** — User sessions and preferences are stored in Redis for instant access.
+- **Embedding cache** — Pre-computed embeddings for popular documents avoid redundant inference.
+
+### Layer 3 — Database cache
+
+- **Connection pooling** — PgBouncer maintains a pool of warm database connections, eliminating connection overhead.
+- **Materialized views** — Complex aggregation queries use pre-computed materialized views refreshed every 5 minutes.
+- **Query plan caching** — PostgreSQL's prepared statements reuse optimized query plans.
+
+---
+
+## Database Optimization
+
+### Indexing strategy
+
+- **B-tree indexes** on primary keys and frequently filtered columns.
+- **GIN indexes** on JSONB fields for flexible metadata queries.
+- **Partial indexes** on hot data subsets (e.g., active users, recent content).
+- **Covering indexes** that include all queried columns to avoid table lookups.
+
+### Query optimization
+
+Every query is analyzed for:
+
+1. **Execution plan review** — EXPLAIN ANALYZE is run on all new queries during code review.
+2. **N+1 detection** — Automated detection of N+1 query patterns in development.
+3. **Slow query logging** — Queries exceeding 100 ms are logged and reviewed weekly.
+4. **Connection management** — Read replicas handle analytics queries to keep the primary database responsive.
+
+---
+
+## Application-Level Optimization
+
+### Frontend performance
+
+- **Code splitting** — Routes are lazy-loaded so users only download the JavaScript they need.
+- **Image optimization** — All images are served in WebP/AVIF format with responsive srcsets.
+- **Prefetching** — The most likely next navigation target is prefetched in the background.
+- **Bundle analysis** — We maintain strict bundle size budgets (< 200 KB initial JS, < 50 KB CSS).
+
+### Backend performance
+
+- **Async I/O** — All I/O-bound operations (database queries, API calls, file reads) are non-blocking.
+- **Worker pools** — CPU-intensive tasks (embedding generation, document parsing) run in dedicated worker processes.
+- **Streaming responses** — AI-generated responses stream token-by-token for perceived instant feedback.
+- **Graceful degradation** — If a dependency is slow, the system returns partial results rather than timing out entirely.
+
+---
+
+## Infrastructure Optimization
+
+### Auto-scaling
+
+| Component              | Scaling metric                | Scale range        |
+| ---------------------- | ----------------------------- | ------------------ |
+| API servers            | Request rate + latency        | 4 to 64 pods       |
+| AI workers             | GPU utilization + queue depth | 2 to 16 nodes      |
+| Kafka consumers        | Consumer lag                  | 3 to 24 partitions |
+| Database read replicas | Connection count              | 1 to 4 replicas    |
+
+### Cost optimization
+
+- **Spot instances** for batch processing workloads (60% cost savings).
+- **Reserved instances** for always-on services (40% cost savings).
+- **Right-sizing** — Monthly reviews of resource utilization with automated recommendations.
+- **Tiered storage** — Infrequently accessed data is automatically moved to cheaper storage tiers.
+
+---
+
+## Tips for Optimizing Your Experience
+
+1. **Use specific queries** — More precise queries return faster, more relevant results.
+2. **Apply filters** — Narrowing by date, source, or content type reduces the search space.
+3. **Enable caching** — For API integrations, respect cache headers to avoid unnecessary requests.
+4. **Use batch endpoints** — Bundle multiple operations into a single batch request.
+5. **Monitor your usage** — The Dashboard shows your API usage patterns and suggests optimizations.

package/lib/templates/content/technical-questions/search-algorithms.md

@@ -0,0 +1,105 @@
+# Search Algorithms & Ranking
+
+How Yantra finds the most relevant results in milliseconds — the algorithms, ranking signals, and optimizations behind the search experience.
+
+---
+
+## Search Architecture
+
+Yantra's search system combines **three complementary approaches** to deliver highly relevant results:
+
+### 1. Keyword search (BM25)
+
+Traditional lexical matching using the BM25 algorithm. This excels at exact-match queries where the user knows specific terms.
+
+- **How it works** — Documents are scored based on term frequency (TF), inverse document frequency (IDF), and document length normalization.
+- **Best for** — Error codes, product names, exact phrases, technical identifiers.
+
+### 2. Semantic search (vector similarity)
+
+Neural embedding-based search that understands meaning beyond exact words.
+
+- **How it works** — Both the query and documents are converted to high-dimensional vectors. Results are ranked by cosine similarity.
+- **Best for** — Natural language questions, conceptual queries, "how do I..." questions.
+
+### 3. Hybrid search (default)
+
+Combines keyword and semantic results using **Reciprocal Rank Fusion (RRF)**, producing results that are both lexically precise and semantically relevant.
+
+```
+RRF_score = sum(1 / (k + rank_i)) for each ranking system i
+```
+
+This is Yantra's default search mode and performs best for most queries.
+
+---
+
+## Ranking Signals
+
+After initial retrieval, results are re-ranked using a learned ranking model that weighs multiple signals:
+
+| Signal            | Weight | Description                                          |
+| ----------------- | ------ | ---------------------------------------------------- |
+| Text relevance    | 35%    | BM25 + semantic similarity combined score            |
+| Source authority  | 20%    | Domain reputation, citation count, editorial quality |
+| Content freshness | 15%    | Recency of creation or last update                   |
+| User engagement   | 15%    | Click-through rate, time-on-page, bookmarks          |
+| Personalization   | 10%    | User's search history, preferred sources, role       |
+| Content quality   | 5%     | Readability score, completeness, structure           |
+
+### Boosting and suppression
+
+- **Recency boost** — Content published in the last 30 days receives a configurable relevance boost.
+- **Source pinning** — Admins can pin trusted sources to always appear in the top results.
+- **Duplicate suppression** — Near-duplicate documents (> 85% similarity) are collapsed, showing only the highest-quality version.
+
+---
+
+## Indexing System
+
+### How content gets indexed
+
+1. **Document intake** — New or updated content enters the indexing queue via the data pipeline.
+2. **Analysis** — The document is processed through language-specific analyzers (tokenization, stemming, synonym expansion).
+3. **Embedding** — A vector representation is generated using the configured embedding model.
+4. **Storage** — The analyzed text, metadata, and vector are written to Elasticsearch shards.
+5. **Availability** — The document becomes searchable within 1-2 seconds (near real-time).
+
+### Index management
+
+- **Shard allocation** — Each index is distributed across multiple shards for parallel query execution.
+- **Replica configuration** — Each shard has at least one replica for fault tolerance and read throughput.
+- **Index lifecycle** — Older indexes are automatically merged, force-merged, or moved to warm storage based on access patterns.
+
+---
+
+## Query Processing
+
+### What happens when you search
+
+1. **Query parsing** — The raw input is tokenized and analyzed using the same pipeline as indexed documents.
+2. **Query expansion** — Synonyms and related terms are added automatically (e.g., "k8s" expands to "kubernetes").
+3. **Spell correction** — Common typos are detected and corrected using edit-distance algorithms.
+4. **Intent detection** — The system classifies whether the user wants a factual answer, a list, a comparison, or a how-to guide.
+5. **Execution** — The query runs against both keyword and vector indexes in parallel.
+6. **Fusion + re-ranking** — Results are merged via RRF, then re-ranked by the learned ranking model.
+7. **Response assembly** — Top results are formatted with highlighted snippets and source metadata.
+
+---
+
+## Performance
+
+| Metric                     | Value                         |
+| -------------------------- | ----------------------------- |
+| Average search latency     | 145 ms                        |
+| P99 search latency         | < 780 ms                      |
+| Index refresh interval     | 1 second                      |
+| Maximum concurrent queries | 12,000/sec                    |
+| Index size                 | 500M+ documents across 8.7 TB |
+
+### Optimization techniques
+
+- **Query caching** — Identical queries are served from Redis cache (5-minute TTL).
+- **Shard routing** — Queries are routed to the shard most likely to contain relevant results.
+- **Early termination** — If the top results are already confident, remaining shards stop processing.
+- **Parallel execution** — Keyword and vector searches run concurrently, not sequentially.

package/lib/templates/content/technical-questions/technical-overview.md

@@ -0,0 +1,107 @@
+# Technical Architecture Overview
+
+A high-level look at Yantra's system design, technology choices, and engineering principles.
+
+---
+
+## System Architecture
+
+Yantra is built as a collection of loosely coupled **microservices** communicating over gRPC and async message queues. This design enables independent scaling, deployment, and fault isolation.
+
+### Core components
+
+| Component            | Responsibility                                        | Technology                       |
+| -------------------- | ----------------------------------------------------- | -------------------------------- |
+| API Gateway          | Request routing, auth, rate limiting                  | Kong / custom middleware         |
+| Search Service       | Query processing, ranking, retrieval                  | Elasticsearch + custom vector DB |
+| AI Service           | Model inference, prompt assembly, response generation | Python (FastAPI) + GPU clusters  |
+| Data Pipeline        | Content ingestion, processing, indexing               | Apache Kafka + Kafka Streams     |
+| User Service         | Authentication, profiles, preferences                 | Node.js (Express) + PostgreSQL   |
+| Notification Service | Real-time alerts, webhooks, emails                    | Go (Gin) + Redis Pub/Sub         |
+
+### Communication patterns
+
+- **Synchronous** — REST/gRPC for user-facing requests where latency matters.
+- **Asynchronous** — Kafka events for background processing (indexing, analytics, notifications).
+- **Pub/Sub** — Redis for real-time features (live search updates, collaborative cursors).
+
+---
+
+## Technology Stack
+
+### Backend
+
+| Layer         | Technologies                              |
+| ------------- | ----------------------------------------- |
+| Languages     | Python 3.12, Node.js 20, Go 1.22          |
+| Frameworks    | FastAPI, Express.js, Gin                  |
+| Databases     | PostgreSQL 16, Redis 7, Elasticsearch 8.x |
+| Message Queue | Apache Kafka 3.7                          |
+| ML/AI         | PyTorch, vLLM, LangChain                  |
+
+### Frontend
+
+| Layer     | Technologies                 |
+| --------- | ---------------------------- |
+| Framework | React 18 + TypeScript 5      |
+| Routing   | Remix (SSR)                  |
+| Styling   | Tailwind CSS + shadcn/ui     |
+| State     | Zustand + React Query        |
+| Mobile    | React Native (iOS & Android) |
+
+### Infrastructure
+
+| Layer             | Technologies                        |
+| ----------------- | ----------------------------------- |
+| Container runtime | Docker + Kubernetes (EKS)           |
+| CI/CD             | GitHub Actions + ArgoCD             |
+| Monitoring        | Prometheus, Grafana, Sentry         |
+| CDN               | CloudFront + edge caching           |
+| DNS               | Route 53 with latency-based routing |
+
+---
+
+## AI Integration
+
+### Multi-model approach
+
+Rather than being locked into a single AI provider, Yantra maintains integrations with multiple model providers:
+
+- **OpenAI** (GPT-4o, text-embedding-3-large)
+- **Anthropic** (Claude 3.5 Sonnet, Claude 3 Haiku)
+- **Open-source** (Llama 3, Mistral) for on-premise deployments
+
+The **Intelligent Router** selects the best model for each request based on task type, latency requirements, and cost constraints.
+
+### RAG (Retrieval-Augmented Generation)
+
+Yantra's core AI workflow follows the RAG pattern:
+
+1. **Retrieve** relevant documents from the vector index based on the user's query.
+2. **Augment** the prompt with retrieved context, conversation history, and system instructions.
+3. **Generate** a grounded response that cites specific sources.
+
+This approach ensures responses are factually anchored to your data rather than relying solely on the model's training data.
+
+---
+
+## Security Architecture
+
+### Defense in depth
+
+| Layer       | Measures                                                   |
+| ----------- | ---------------------------------------------------------- |
+| Network     | VPC isolation, WAF, DDoS protection, private subnets       |
+| Application | Input validation, CSRF protection, Content Security Policy |
+| Data        | AES-256 encryption at rest, TLS 1.3 in transit             |
+| Access      | RBAC, MFA, API key rotation, session management            |
+| Audit       | Immutable audit logs, real-time anomaly detection          |
+
+### Compliance
+
+Yantra maintains the following certifications and compliance standards:
+
+- **SOC 2 Type II** — Independently audited annually
+- **GDPR** — Full compliance with EU data protection regulations
+- **ISO 27001** — Information security management certification
+- **HIPAA** — Available for healthcare customers on Enterprise plans